Tag Archives: penetration testing

How to win BIG and even more!

I recently had a presentation in the OWASP Birmingham (UK) chapter meeting. The crowd was very friendly, and it was a good experience overall with a lot of free food! I definitely recommend attending the next one if you are close by.

In my presentation, I showed a few examples how I managed to win a lot of money in gambling games, cheated when doing my online shopping, and got more free gifts than necessary! Obviously all of my actions were as part of defined security assessments, and therefore I legally had the necessary permissions to carry out my tests.

My presentation’s description was:

I am going to review a number of interesting flaws that I have seen within the payment systems and gambling games. This includes examples that allowed me to win big while I was gambling very responsibly as well as simple methods that brought me free goods such as expensive books that I really didn’t need, fake moustaches, or even caskets for my fake funeral!

Disclaimer: all issues were reported responsibly to the companies and no moustache or slot machine was harmed in this process! I am not going to name any companies during this presentation.

Its slides are available via the SlideShare website:

After this presentation, Ashley Cox and I performed a research for NCC Group about abusing voucher codes. As a result, we also made the following blog post: Online shoplifting – exploiting e-commerce basket and voucher faults for five-finger discount.

Perhaps this how some people find glitches to post in the hotukdeals website!

I have also updated the whitepaper I had created for testing financially-oriented web applications to cover more discovered test-cases. This freely accessible guideline has been created for penetration testers and bug bounty hunters to assess ecommerce and financial services applications: https://www.nccgroup.trust/uk/our-research/common-security-issues-in-financially-orientated-web-applications/

I would personally be grateful if you could give a reference to me or this whitepaper if you have found it useful or you have managed to identify a vulnerability using this.

Bug Bounty vs Penetration Testing (Simple Unbiased Comparison)

I have listed some notes, based on my personal experience so far, about working with some limited direct and indirect bug bounty programs and performing vulnerability assessment. I hope this is useful for the readers; please feel free to contact me on my Twitter (@irsdl) to discuss any of these if you wish.

In my humble opinion, direct bug bounty programs and vulnerability assessments are both necessary although may not match some business models where one can be ineffective in practice. For example, a bug bounty program may not work for sensitive platforms or targets that are only accessible from certain locations or IP addresses. On the other hand, a broad time limited vulnerability assessment may not work for large open source applications that have been around and tested for quite some time. As far as I am aware, bug bounty programs cannot offer any assurance to the customers based on the regulations at the moment, but this may change in future.

By the way, I am not discussing indirect bug bounty programs in which a third party vendor pays for vulnerabilities of other vendors’ products.

My notes about each of these solutions are as follows:

Direct Bug Bounty Programs


  • More testers to cover more targets and to find more vulnerabilities. However, normally in practice only attractive programs receive high-quality vulnerabilities.
  • More time or unlimited time to cover more targets and to find more vulnerabilities. Again, this happens when a program is attractive to highly skilled testers and treats them well.
  • They can change the scope dynamically at any time. That said, this can cause problems between the program and reporters who have already started looking at targets that are in the scope.
  • Only pays for valid/in-scope findings (not saying it cannot be expensive though).
  • Can prevent breaches by buying bugs for a good price.
  • It can help companies with a huge attack surface to identify hidden issues that were not identified throughout previous assessments.
  • Works well for open source or non-profitable applications when sponsors such as Internet Bug Bounty pays for it.
  • Customers may feel safer when a company security model is mature and is handling reported security issues gracefully.
  • Encourage responsible disclosure culture.
  • Testers might care about the patching process when their bounties depend on it so they might be eager to help with the retest. However, this might make program less attractive to those who want to be paid quickly.
  • It helps security researchers to legally test the targets and makes the responsible disclosure process simpler for anyone who might want to report a security issue.


  • It is a competitive market. If a few companies pay more for vulnerabilities, it will make other programs less attractive and will reduce their quality. Therefore, quality in the bug bounty world is relative and those programs that cannot afford paying as much as giant companies for their security issues will be on the losing end.
  • Unless a program is attractive enough and is restricted to top bug bounty reporters in the world, the majority of testers in bug bounty are not experienced penetration testers and have less skills and are only after the low hanging fruits. As a result, there are high amount of false positives, fake, or duplicates from less experienced testers or those who have not read the bug bounty policy properly. In practice, less than %5 of reported issues can be valid for a popular program that increases human mistakes in handling real issues and put a high pressure on the triage team (https://www.synack.com/2017/05/05/managed-bug-bounties-quality-is-in-the-secret-sauce/).
  • Highly skilled penetration testers who are new to the bug bounty platforms cannot join private programs that are normally more attractive as they do not have enough points on the system. As a result, when most attractive programs are private, they can never climb up the point-based ladder without spending their time on less attractive programs which is not ideal.
  • The main problem with the penetration testing assessments is their time constraints because of limited budget. A Bug bounty program with limited time and limited scope is not even close to a penetration testing assessment and might not be useful at all unless it can attract good and strong testers.
  • Free or low paid programs can give customers false sense of security as they are not attractive to strong testers.
  • It is impossible to stop all the testers at once when needed as the communication method is normally via email and can be slow.
  • Newly developed functionality or less visible ones normally remain hidden to the testers. Therefore, a new section may remain untested while other more visible sections are being tested over and over again by different testers.
  • Testers are not normally known and therefore it is more difficult to trace their activities if needed in compare with a penetration testing. Attackers may also claim that they were after responsible disclosure.
  • Hard to design the right policy with a correct scope (requires knowledge of the whole system). Badly designed scope can reduce the effectiveness of bug bounty as well as damaging the vendor’s reputation regarding its security.
  • It may give attackers the opportunity to legally assess an organisation while having no intention to report the most serious issues. However, it is possible in some platforms to limit a penetration testing to certain testers who are better known because of their longer activities (this may make a bug bounty program private).
  • It may cause denial of service if a system is not ready for a heavy load as testers may use automated scanners even if using it is forbidden. Although systems are constantly being scanned automatically on the Internet, defining a bug bounty program on them will certainly increase the amount of scanners being used on the targets.
  • A lot of dispute over prioritising and paying for a vulnerability. Reporters constantly compare their issues with each other.
  • Mistrust over duplicate issues due to the lack of transparency in most of programs.
  • Reputation damage over solving issues badly (e.g. by ignoring the reports, not paying the right amount, or keeping the reporter in the dark while patching an issue)
  • Sorting the tax money is harder outside of the US as they use different currencies. Options in the UK are currently limited to use accountants, commercial software, or paper-based forms by using the correct currency conversion chart (if you receive them in different currencies than British Pound).

Vulnerability Assessment (commonly known as penetration testing*)


  • An official report or certificate of security assessment can be produced by the penetration testing company. This can be presented to partners or other organisations when a proof of security assessment is needed.
  • A higher quality report in compare with bug bounty reports in general.
  • Full attention on the specific target as days and scope are pre-defined and will not be deviated from.
  • A responsible company that can be contacted and is accountable for their assessment and their testers.
  • Level of support after the assessment in order to rectify the issues.
  • No false-positives due to the consultants’ experience and the QA process. However, human errors can happen but at least it will be investigated and the report will be updated when it is incorrect.
  • Better communication between the vendor and the testers in order to resolve any possible issues.
  • Regardless of the number of issues identified by the penetration testers, all will be presented to the client for the same fixed cost.
  • Testing and progress visibility
  • Good assurance for the customers regarding the assessment coverage, quality of testing, and an accountable testing company.
  • Accepted by different regulations such as by PCI.


  • Limited time of testing may reduce the likelihood of discovering well-hidden vulnerabilities that require more research or different skillset.
  • Vendors should normally pay additional fees for a retest when required as the retest might not be part of the original assessment.
  • Assigning more time to an assessment can be very expensive
  • Human factor plays an important role as number of testers on each assessment are limited. For example, experience and knowledge of assigned testers on an assessment and how they feel during the test plays an important role in identifying security issues! However, in a bug bounty program anyone can look at the system at the same time or later in time.
  • Difficult to change the scope during an assessment or assign more testers upon a short notice as resources are normally limited and testers’ specialities and skills are different.

I do like Bug Bounties! For me, it is like catching a fish! 🎣

With all that, bug bounty is still young and has a great future ahead!

A number of the following suggestions have already been implemented in some of the well-known bug bounty platforms but not in all.

Recommendations to Improve Bug Bounty Platforms **:

  • Giving people access to previously reported issues when they have reported a duplicate issue to make it clearer (similar to what Mozilla does in their issue tracker). If the duplicate issues have been received from another source, perhaps an additional tracking number and name of the finder can help people understand that the vendor is not cheating on them by announcing an issue as a duplicate.
  • Registering testers’ mobile number to keep them up to date with the latest changes when they sign up for a certain bug bounty program. The scope can only be visible to those who have signed up and confirmed a text message. Testers can then opt out if they are not interested anymore in receiving text messages about scope changes or any issue with the testing.
  • A verification process (perhaps in different levels) in which the consultants can provide their identification documents, certificates, address, and CV to the bug bounty platform in order to make their account verified. This can be used if vendors only want to pass their bug bounty to trusted/known people based on their verification level or their certificates (not just their bug bounty reputation).
  • A VPN service for the verified testers so they can always use certain IP addresses while testing. This gives vendors the opportunity to design appropriate firewall rules and to make their testing system only accessible to certain IPs.
  • A survey for those who have reported vulnerabilities measuring their investment in time or other resources, their view on the specific bug bounty program’s performance alongside possible vendor’s view about the tester. This data can be publicly available upon approval from the tester and the vendor.
  • A transparent platform to show statistics of every bug bounty programs. This should include but not be limited to the following information selectable/searchable by date: number of reported/accepted/duplicate/invalid issues with their severity, number of top 100 testers (or if there is another measure to show strong testers) reported issues, average amount of days to fix the issues/pay the bounty, average amount of bounty based on issue severity, number of accepted but unfixed issues.
  • Defining a consultancy system for the vendors so they can refine their scope based on feedbacks from the testers.
  • Creating a centralised platform to include the bug bounty targets for research purposes. It is often the case the people go to a great length to obtain a list like this that can be used for research purposes. For instance, to check a specific vulnerability on targets that use a certain technology.
  • Working with the governments to make it easier for testers to pay their tax or to give the money to charities outside the US.

Improvements for Bug Bounty Program Owners **:

A bug bounty program will be successful when it can attract strong testers to spend some time assessing the targets. It is important to design and perform it in a way that it encourages testers to come back to spend more time and report more vulnerabilities.

The following list shows some of the features that a bug bounty program should have to be popular among testers:

  • Clear scope and exclusion list
  • Quick acknowledgement of receiving issues
  • Quick decision on issues validity for the bounty
  • Reasonably short patching time
  • Quick pay-out (e.g. on validity rather than resolution when patching will take time)
  • Generous rewards for the findings, good quality reports, and help in retesting
  • Surprise rewards on nice findings and reports even when they are out of the scope
  • Supporting multiple payment options
  • Polite and professional triage team that can handle difficult cases gracefully



* IMHO, they are different though but that discussion is not the topic!
** whoami to comment on this? Just an experienced security researcher like many others.

Using Firefox Profiles in Security Testing

I have started using browser profiles for normal web application assessment since working at NCC Group due to the number of projects I had since the 1st week! I thought it is a good idea to share my solution as I have seen other people facing the same issue when dealing with multiple web application testing projects.

Why should we use a new browser profile for testing?

There are a number of reasons to use separate browser profiles per web application assessments. I have listed some of these reasons from my point of view:

A clean browser environment can be useful to keep the history/data especially when re-testing the same target.

Browser that is used for testing is normally unsafe for day to day browsing. For instance, client-side protections such as XSS auditors or NoScript add-on in Firefox should be disabled while testing in order to detect client-side issues properly. Proxies’ or other self-signed root certificates could be imported in the browser during the test.

It is sometimes necessary to use the private browsing option within the same browser with all the add-ons in order to test session management or broken access control issues (or perhaps to test an issue very quickly using another session but with the same user-agent). It should be noted that private browsing/incognito is not recommended for the actual testing as cache control, sensitive data in URL, or autocomplete issues might be missed and there will be no web history in the browser (especially useful for those times that the proxy is off for some reason).

Why should we automate this?

Having a fresh browser per each test with predefined settings and pre-installed add-ons/extensions can be time consuming if not automated.

I use Mozilla Firefox to make everything easier if the target website is not browser specific or if different user-agents in Firefox do not break the website functionality.

An easy way to do this using Firefox

Step 1: Creating a new empty profile (template)

A Firefox shortcut can be created or edited with the “-p” argument to show the “Choose User Profile” panel. Alternatively, the “-profilemanager” argument can be used to open this panel:

After clicking the “Create Profile” button, choose an appropriate profile name such as “Clean For Security Testing” to create a new profile (you can save it in a preferred location). This profile will be added to the list:

Now click the “Start Firefox” button to customise it.

Step 2: Preparing the template – Firefox settings

As Firefox hides the top menu automatically, we need to use the “Alt” key to access it. It is therefore my preferred choice to make it permanently visible by selecting “Menu Bar” from Views>Toolbars after pressing the “Alt” key. Throughout this post, I have used the top menu to point at items’ locations but the “Open Menu” button in Firefox can be used as well.

In order to reduce the number of requests Firefox sends automatically during testing and to improve the privacy, I normally do this:

– Untick all the boxes in Tools>Options>Advanced>Data Choices

– Untick all the boxes in Tools>Options>Advanced>Update and select “Never check for updates” – It is important to note that check for updates for both Firefox and its add-ons must be performed manually when using profiles.

– From “about:config”, set the value of “browser.newtabpage.enabled” and “network.captive-portal-service.enabled” to “false”.

– From “about:config”, set an empty string for “browser.selfsupport.url”

Other automated requests can be disabled using the Mozilla guideline ifnecessary: https://support.mozilla.org/en-US/kb/how-stop-firefox-making-automatic-connections

Step 3: Preparing the template – Proxy root certificates

Proxies’ root certificates should also be imported into the browser’s root certificate at this point to make it easier every time we use a new profile.

Step 4: Preparing the template – Installing required extensions / plugins / themes

Add-ons that are installed at this stage are not compulsory and it is up to you to install your favourites. However, the “FEBE” add-on should be installed to automate the process of creating new profiles based on the template:



CLEO extension can also be useful to back up all the extensions in a single file when needed!

My favourite generic extensions for testing are as follows:

ColorfulTabs: this helps me to instantly realise when the testing domain changes – so I can detect out-scoped or different domains using the tab colours! I have chosen “Generate Colors By Domain Hostname” in the extension configuration. I have also set the “Tab Border Radius” to 1 to fix the UI issue it may cause.

Firebug: I still like this extension although Firefox has implemented some of it internally now!

Form History Control: useful to create a quick screenshot for the report when reporting autocomplete issues!

FoxyProxy: very useful to manage different proxies. For instance, I have a few ports configured in it for different purposes and different proxies. If you configure it in your template profile, it will be backed up into your testing profiles as well.

HackBar: limited but quite handy extension to send quick GET/POST requests or to decode/encode something. I use its other features too and sometimes even use it like a notepad!!! As I do not like its panel to be always visible on top, I have added its icon (“Toggle HackBar”) to the address bar menu.

ProfileSwitcher: to switch between profiles quickly when needed!

Random Agent Spoofer: to change the user agent when needed very quickly.

Regular Expressions Tester: to test a Regular Expression when I have no better tools.

Web Developer: An old habit just like Firebug. Makes it easier to manipulate items in a HTML page. I used to use it to see the HTML/JS errors (to detect a lame XSS perhaps!) but it seems Firefox is doing a good job on that itself too.

Perhaps an awesome dark Firefox theme for the security template profile also helps. Something that is appropriate for the report screenshots of course! It may also help you to know when a security profile is in use so you will not use it for normal browsing.

The template is now ready. Do not use it for testing before restoring it into a new separate profile though!

Step 5: FEBE Options

Using Tools>FEBE, select “FEBE Options” and ensure that “Full profile” has been chosen. Then on the “Where to backup” menu in “FEBE Options”, select the backup files location and press the Ok button.

Step 6: Updates

You can ignore this step if you have already updated Firefox and its add-ons.

Ensure that Firefox is updated by going to Help>About Firefox from the top menu. Then from the top menu, go to Tools>Add-ons, select “Extensions” from the left menu and click on the settings button and choose “Check for Updates”. Restart the browser if it is required after the update.

Step 7: Clean up

It is time to clean the history before backing up the template to have a fresh start every time! You can ignore this step if there is nothing in the browser history.

From History>Clear Recent History select “Everything” and tick all the boxes and press “Clear Now”. From Tools>Advanced>Network, clear any remaining cached web contents.

Step 8: Backing up the template

Using Tools>FEBE, select the “Perform backup now” option.  Run it again if it was not successful the first time!

Step 9: Creating new profiles based on the template

When you are using the “Clean For Security Testing” profile, using Tools>FEBE, select the “Restore Profile” option:

First click on the “Create new profile” button and enter a name such as “BugBounty_Facebook”. Close any FEBE messages to make the process smoother.

Now, click on the “Select local backup to restore” button and choose the template profile:

After this stage, click on “Start profile restore” (twice if it did not work).

Step 10: Using the new profile

Now close Firefox and all its messages and open it again. The new profile should appear in the list:

Select it and start Firefox.

Creating another profile

Follow steps 6 to 10 whenever you need to create a new profile for testing now.

Quick start sample

I have created a backup file using my Firefox that you can download from here (https://soroush.secproject.com/downloadable/images/firefox_profile/profileFx53.0(FEBE8.9.3.1){Clean For Security Testing}.zip).

You need to have the FEBE extension installed to use this and to restore the “Clean For Security Testing” profile. Follow steps 6 to 10 after restoring this profile.

Common Security Issues in Web-Based Payment Systems (& Gambling Apps)

A guideline for penetration testers to assess eCommerce and financial services applications.

This document summarises NCC Group’s experience of assessing eCommerce and financial services applications, providing a checklist of common security issues seen in financial services web applications.

This paper has been published via the NCC Group website:

The PDF version of this paper is also available here: https://www.nccgroup.trust/globalassets/our-research/uk/whitepapers/2015/06/common-security-issues-in-financially-orientated-web-applications-_v1.1.pdf 

A local copy can be found in: https://soroush.secproject.com/downloadable/common-security-issues-in-financially-orientated-web-applications-_v1.1.pdf