What real hackers know about the penetration testing industry that you don’t.

What real hackers know about the penetration testing industry that you don’t.

Real Time Dynamic Testing

The information security industry has become politicized and almost entirely ineffective as is evidenced by the continually increasing number of compromises. The vast majority of security vendors don’t sell security; they sell political solutions designed to satisfy the political security needs of third parties. Those third parties often include regulatory bodies, financial partners, government agencies, etc.   People are more concerned with satisfying the political aspects of security than they are with actually protecting themselves, their assets, or their customers from risk and harm.

For example, the Payment Card Industry Data Security Standard (PCI-DSS) came into existence back on December 15th, 2004. When the standard was created it defined a set of requirements that businesses needed to satisfy in order to be compliant. One of those requirements is that merchants must undergo regular penetration testing. While that requirement sounds good it completely fails to define any realistic measure against which tests should be performed. As a result the requirement is easily satisfied by the most basic vetted vulnerability scan so long as the vendor calls it a penetration test (same is still largely true for PCI 3.0).

To put this into perspective the V0 and V50 ballistics testing standards establish clear requirements for the performance of armor. These requirements take into consideration the velocity of a projectile, size of a projectile, number of strikes, etc. If penetration is achieved when testing against the standards then the armor fails and is not deployable.   If PCI-DSS were used in place of the V0 and V50 standards then it would suffice to test a bulletproof vest with a squirt gun.   In such a case the vest would be considered ready for deployment despite its likely failure in a real world scenario.

This is in part what happened to Target and countless others. Target’s former CEO, Gregg Steinhafel was quoted saying “Target was certified as meeting the standard for the payment card industry (PCI) in September 2013. Nonetheless, we suffered a data breach.” What does that tell us about the protective effectiveness of PCI? What good is a security regulation if it fails to provide the benefit that it was designed to deliver? More importantly, what does that say about the penetration testing industry as a whole?

While regulations are ineffective it is the customers choice to be politically oriented or security focused. In 2014, 80% of Netragard’s customers opted to receive political security testing services (check in the box) rather than genuine security testing services even after having been educated about the differences between both. Most businesses consider the political aspect of receiving a check in the box to be a higher priority than good security (this is also true of the public sector).

This political agenda motivates decision makers to select penetration testing vendors (or other security solutions) based on cost rather than quality. Instead of asking intelligent questions about the technical capabilities of a penetration testing team they ask technically irrelevant questions about finances, the types of industries that vendor may have serviced, if a vendor is in Gartner’s magic quadrant, etc. While those questions might provide a vague measure (at best) of vendor health they completely fail to provide any insight into real technical capability.   The irony is that genuine penetration testing services maintain both lower average upfront costs and lower average long-term costs than political penetration testing services.

The lower average upfront cost of genuine penetration testing comes from the diagnostic pricing methodology (called Attack Surface Pricing or ASMap Pricing) that genuine penetration testing vendor’s depend on. ASMap pricing measures the exact workload requirement by diagnosing every in-scope IP address and Web Application (“Target”) during the quote generation process. Because each Target offers different services, each one also requires a different amount of testing time for real manual testing. ASMap pricing never results in an overcharge or undercharge and is a requirement for genuine manual penetration testing. In fact, diagnostic pricing is the de facto standard for all service based industries with the exclusion of political penetration testing (more on that later).

The lower long-term costs associated with genuine penetration testing stem from the protective nature of genuine penetration testing services. If the cost in damages of a single successful compromise far exceed the cost of good security then clearly good security is more cost effective. Compare the average cost in damages of any major compromise to the cost of good security. Good security costs less, period.

Political penetration testing (the industry norm) uses a Count Based Pricing (“CBP”) methodology that almost always results in an overcharge. CBP takes the number of IP addresses that a customer reports to have and multiplies it by a cost per IP. CBP does not diagnose the targets in scope and is a blind pricing methodology. What happens if a customer tells a vendor that they have 100 IP addresses that need testing but only 1 IP address offers any connectable services? If CBP is being used then the customer will be charged for testing all 100 IP addresses when they should only be charged for 1. Is that ethical pricing?

A good example of CBP overcharge happened to one of our customers last year. This customer approached Netragard and another well-known Boston based firm.   The other firm produced a proposal using CBP based on the customer having 64 IP addresses. We produced a proposal using the ASMap methodology. When we presented our proposal to the customer ours came in over $55,000.00 less than the other vendor.   When the customer asked us how that was possible we explained that of their 64 IP addresses only 11 were live. Of the 11 only 2 presented any real testable surface. Needless to say the other vendor didn’t win the engagement.

CBP cannot be used to price a manual penetration testing engagement because it also runs the risk of undercharging. Any engagement priced with the CBP methodology is dependent on vulnerability scanning. This is because CBP is a blind pricing methodology that does not diagnose workload. If a customer is quoted at $5,000 to test 10 IP addresses CBP assumes the workload for the 10 IP addresses.

What happens if each IP address requires 10 hours of manual labor? Engagements priced with CBP rely on automated scanners to compensate for these potential overages and to ensure that the vendor always makes a profit.   Unfortunately this dependence on automated scanning degrades the quality of the engagement significantly.  The political penetration testing industry falsely promises manual services when in fact the final deliverable is more often than not a vetted vulnerability scan. This promotes a false sense of security that all too often leads to compromise.

Customers can choose to be lazy and make naïve, politically oriented security decisions or they can self-educate, choose good security and save themselves considerable time and money.   While the political security path appears simple and easy at the onset the unforeseen complexities and potential damages that lie are all too often catastrophic. How much money is your business worth and what are you doing to truly protect it?

We’re offering a challenge to anyone willing to accept. If you think that your network is secure then let us test it with our unrestricted methodology. If we don’t compromise your network then the test is done free of charge. If we do compromise then you pay cost plus 15%.   During the test we expect you to respond the same way that you would a real threat. We don’t expect to be whitelisted and we don’t expect you to lower your defenses. Before you accept this challenge let it be known that we’ve never failed. To date our unrestricted methodology maintains a 100% success rate with an average time to compromise of less than 4 hours. Chances are that you won’t know we’re in until it’s too late.

Do you accept?

Blog Posts

Karen Huggins

Chief Financial, HR and Admin Officer
Karen joined the Netragard team in 2017 and oversees Netragard’s financial, human resources as well as administration functions. She also provides project management support to the operations and overall strategy of Netragard.
Prior to joining Netragard, she worked at RBC Investor Services Bank in Luxembourg in the role of Financial Advisor to the Global CIO of Investor Services, as well as several years managing the Financial Risk team to develop and implement new processes in line with regulatory requirements around their supplier services/cost and to minimize the residual risk to the organization.
With over 20 years of experience in finance with global organizations, she brings new perspective that will help the organization become more efficient as a team. She received her Bachelor of Finance from The Florida State University in the US and her Master of Business Administration at ESSEC Business School in Paris, France.

Philippe Caturegli

Chief Hacking Officer
Philippe has over 20 years of experience in information security. Prior to joining Netragard, Philippe was a Senior Manager within the Information & Technology Risk practice at Deloitte Luxembourg where he led a team in charge of Security & Privacy engagements.

Philippe has over 10 years of experience in the banking and financial sector that includes security assessment of large and complex infrastructures and penetration testing of data & voice networks, operating systems, middleware and web applications in Europe, US and Middle East.

Previously, Philippe held roles within the information system security department of a global pharmaceutical company in London. While working with a heterogeneous network of over 100,000 users across the world and strict regulatory requirements, Philippe gained hands-on experience with various security technologies (VPN, Network and Application Firewalls, IDS, IPS, Host Intrusion Prevention, etc.)

Philippe actively participates in the Information Security community. He has discovered and published several security vulnerabilities in leading products such as Cisco, Symantec and Hewlett-Packard.

He is a Certified Information Systems Security Professional (CISSP), Certified Ethical Hacker (CEH), PCI Qualified Security Assessors (PCI-QSA), OSSTMM Professional Security Analyst (OPSA), OSSTMM Professional Security Tester (OPST), Certified in Risk and Information Systems Control (CRISC)and Associate Member of the Business Continuity Institute (AMBCI).

Adriel Desautels

Chief Technology Officer
Adriel T. Desautels, has over 20 years of professional experience in information security. In 1998, Adriel founded Secure Network Operations, Inc. which was home to the SNOsoft Research Team. SNOsoft gained worldwide recognition for its vulnerability research work which played a pivotal role in helping to establish today’s best practices for responsible disclosure. While running SNOsoft, Adriel created the zeroday Exploit Acquisition Program (“EAP”), which was transferred to, and continued to operate under Netragard.
In 2006, Adriel founded Netragard on the premise of delivering high-quality Realistic Threat Penetration Testing services, known today as Red Teaming. Adriel continues to act as a primary architect behind Netragard’s services, created and manages Netragard’s 0-day Exploit Acquisition Program and continues to be an advocate for ethical 0-day research, use and sales.
Adriel is frequently interviewed as a subject matter expert by media outlets that include, Forbes, The Economist, Bloomberg, Ars Technica, Gizmodo, and The Register. Adriel is often an invited keynote or panelist at events such as Blackhat USA, InfoSec World, VICELAND Cyberwar, BSides, and NAW Billion Dollar CIO Roundtable.