The Dark Side of Bug Bounties

The Dark Side of Bug Bounties

Bug Bounties

Bug Bounty companies (often called crowd sourced penetration tests) are all the hype.  The primary argument for using their services is that they provide access to a large crowd of testers, which purportedly means that customers will always have a fresh set of eyes looking for bugs.  They also argue that traditional penetration testing teams are finite and, as a result, tend to go stale in terms of creativity, depth, and coverage.  While these arguments seem to make sense at face value, are they accurate?

Penetration Testing Company

The first thing to understand is that the quality of any penetration test isn’t determined by the volume of potential testers, but instead by their experience, talent, and overall capabilities.  A large group of testers with average talent will never outperform a small group of highly talented testers in terms of depth and quality.  A great parallel example of this is when the world’s largest orchestra played ninth symphonies of Dvorák and Beethoven.  While that orchestra was made up of 7,500 members, the quality of their song was nothing compared to that which is produced by The Boston Symphony Orchestra (which is made up of 91 musicians).

Interestingly, it appears that bug hunters are incentivized to spend as little time as possible per bounty.  This is because bug hunters need to maintain a profitable hourly rate while working or their work won’t be worth their time.  For example, a bug hunter might spend 15 minutes to find a bug and collect a $4,000.00 bounty, which is an effective rate of $16,000.00 per hour!  In other cases, a bug hunter might spend 40 hours to find a bug and collect a $500.00 bounty which is a measly $12.50 per hour in comparison.  Even worse they might spend copious time finding a complex bug only to learn that it is a duplicate and collect no bounty (wasted time).

This argument is further supported when we appraise the quality of bugs disclosed by most bug bounty programs.  We find that most of the bugs are rudimentary in terms of ease of discovery, general complexity, and exploitability.  The bugs regularly include cross-site scripting vulnerabilities, SQL injection vulnerabilities, easily spotted configuration mistakes, and other common problems.  On average they appear to be somewhat more complex than what might be discovered using industry standard automated vulnerability scanners and less complex than what we’ve seen exploited in historical breaches.  To be clear, this doesn’t suggest that all bug hunters are low talent individuals, but, instead, that they are not incentivized to go deep.

In contrast to bug bounty programs, genuine penetration testing firms are incentivized to bolster their brand by delivering depth, quality, and maximal coverage to their customers.  Most operate under a fixed cost agreement and are not rewarded based on volume of findings, but instead by the repeat business that is earned through the delivery of high-quality services.  They also provide substantially more technical and legal safety to their customers than bug bounty programs do.

For example, we evaluated the terms and conditions for several bug bounty companies and what we learned was surprising.  Unlike traditional penetration testing companies, bug bounty companies do not accept any responsibility for the damages or losses that might result from the use of their services.  They explicitly state that the bug hunters are independent third parties and that any remedy with respect to loss or damages that a customer seeks to obtain is limited to a claim against that bug hunter.  What’s more is that the vetting process for bug hunters is lax at best.  In nearly all cases, background checks are not run and even when they are run the bug hunter could provide a false identity. The validation around who a bug hunter really is, is also lacking. To sign up to most programs you simply need to validate your email address. In simple terms, organizations that use bug bounty programs accept all risk and have no realistic legal recourse, even if a bug hunter acts in a malicious manner.

To put this into context, bug bounty programs effectively provide anyone on the internet with a legitimate excuse to attack your infrastructure.  Since these attacks are expected as a part of the bug bounty program, it may impact your ability to differentiate between an actual attack and an attack from a legitimate bug hunter.  This creates an ideal opportunity for bona fide malicious actors to hide behind bug bounty programs while working to steal your data. When you combine this, with the fact that it takes an average of ~200 days for most organizations to detect a breach, the risk becomes even more apparent.

There’s also the issue of GDPR. GDPR increases the value of personal data on the black market and to organizations alike.  Under GDPR, if personal data of a European citizen is breached, the organization that suffered the breach can face heavy fines, penalties, and more. In article 4 of the GDPR, a personal data breach is defined as “a breach of security leading to the accidental or unlawful destruction, loss, alteration, unauthorized disclosure of, or access to personal data transmitted, stored or otherwise processed”. While bug bounty programs target configurations, systems, and implementations, they do not incentivize bug hunters to go after personal data.  However, because of GDPR, a malicious bug hunter who exploits a vulnerability that discloses personal data (accidental or not), may be incentivized to ransom their finding for a higher dollar value. Likewise, organizations might be incentivized to pay the ransom and report it as a bounty to avoid having to notify the Data Protection Authorities (“DPA”) as is required by GDPR.

On a positive note, many of our customers use bug bounty programs in tandem with our Realistic Threat Penetration Testing services.  Customers who use bug bounty programs have far less vulnerabilities in terms of low-hanging-fruit than ones who don’t.  In fact, we are confident that bug bounty programs are pointedly more effective at finding bugs than automated vulnerability scanning could ever be. It’s also true that these programs are more effective than penetration testing vendors who deliver services based on the product of automated vulnerability scans.  When compared to a research driven penetration test, however, the bug bounty programs pale in comparison.

Blog Posts

Karen Huggins

Chief Financial, HR and Admin Officer
Karen joined the Netragard team in 2017 and oversees Netragard’s financial, human resources as well as administration functions. She also provides project management support to the operations and overall strategy of Netragard.
 
Prior to joining Netragard, she worked at RBC Investor Services Bank in Luxembourg in the role of Financial Advisor to the Global CIO of Investor Services, as well as several years managing the Financial Risk team to develop and implement new processes in line with regulatory requirements around their supplier services/cost and to minimize the residual risk to the organization.
 
With over 20 years of experience in finance with global organizations, she brings new perspective that will help the organization become more efficient as a team. She received her Bachelor of Finance from The Florida State University in the US and her Master of Business Administration at ESSEC Business School in Paris, France.

Philippe Caturegli

Chief Hacking Officer
Philippe has over 20 years of experience in information security. Prior to joining Netragard, Philippe was a Senior Manager within the Information & Technology Risk practice at Deloitte Luxembourg where he led a team in charge of Security & Privacy engagements.

Philippe has over 10 years of experience in the banking and financial sector that includes security assessment of large and complex infrastructures and penetration testing of data & voice networks, operating systems, middleware and web applications in Europe, US and Middle East.

Previously, Philippe held roles within the information system security department of a global pharmaceutical company in London. While working with a heterogeneous network of over 100,000 users across the world and strict regulatory requirements, Philippe gained hands-on experience with various security technologies (VPN, Network and Application Firewalls, IDS, IPS, Host Intrusion Prevention, etc.)

Philippe actively participates in the Information Security community. He has discovered and published several security vulnerabilities in leading products such as Cisco, Symantec and Hewlett-Packard.

He is a Certified Information Systems Security Professional (CISSP), Certified Ethical Hacker (CEH), PCI Qualified Security Assessors (PCI-QSA), OSSTMM Professional Security Analyst (OPSA), OSSTMM Professional Security Tester (OPST), Certified in Risk and Information Systems Control (CRISC)and Associate Member of the Business Continuity Institute (AMBCI).

Adriel Desautels

Chief Technology Officer
Adriel T. Desautels, has over 20 years of professional experience in information security. In 1998, Adriel founded Secure Network Operations, Inc. which was home to the SNOsoft Research Team. SNOsoft gained worldwide recognition for its vulnerability research work which played a pivotal role in helping to establish today’s best practices for responsible disclosure. While running SNOsoft, Adriel created the zeroday Exploit Acquisition Program (“EAP”), which was transferred to, and continued to operate under Netragard.
 
In 2006, Adriel founded Netragard on the premise of delivering high-quality Realistic Threat Penetration Testing services, known today as Red Teaming. Adriel continues to act as a primary architect behind Netragard’s services, created and manages Netragard’s 0-day Exploit Acquisition Program and continues to be an advocate for ethical 0-day research, use and sales.
 
Adriel is frequently interviewed as a subject matter expert by media outlets that include, Forbes, The Economist, Bloomberg, Ars Technica, Gizmodo, and The Register. Adriel is often an invited keynote or panelist at events such as Blackhat USA, InfoSec World, VICELAND Cyberwar, BSides, and NAW Billion Dollar CIO Roundtable.