The information security industry has become politicized and almost entirely ineffective as is evidenced by the continually increasing number of compromises. The vast majority of security vendors don’t sell security; they sell political solutions designed to satisfy the political security needs of third parties. Those third parties often include regulatory bodies, financial partners, government agencies, etc. People are more concerned with satisfying the political aspects of security than they are with actually protecting themselves, their assets, or their customers from risk and harm.
For example, the Payment Card Industry Data Security Standard (PCI-DSS) came into existence back on December 15th, 2004. When the standard was created it defined a set of requirements that businesses needed to satisfy in order to be compliant. One of those requirements is that merchants must undergo regular penetration testing. While that requirement sounds good it completely fails to define any realistic measure against which tests should be performed. As a result the requirement is easily satisfied by the most basic vetted vulnerability scan so long as the vendor calls it a penetration test (same is still largely true for PCI 3.0).
To put this into perspective the V0 and V50 ballistics testing standards establish clear requirements for the performance of armor. These requirements take into consideration the velocity of a projectile, size of a projectile, number of strikes, etc. If penetration is achieved when testing against the standards then the armor fails and is not deployable. If PCI-DSS were used in place of the V0 and V50 standards then it would suffice to test a bulletproof vest with a squirt gun. In such a case the vest would be considered ready for deployment despite its likely failure in a real world scenario.
This is in part what happened to Target and countless others. Target’s former CEO, Gregg Steinhafel was quoted saying “Target was certified as meeting the standard for the payment card industry (PCI) in September 2013. Nonetheless, we suffered a data breach.” What does that tell us about the protective effectiveness of PCI? What good is a security regulation if it fails to provide the benefit that it was designed to deliver? More importantly, what does that say about the penetration testing industry as a whole?
While regulations are ineffective it is the customers choice to be politically oriented or security focused. In 2014, 80% of Netragard’s customers opted to receive political security testing services (check in the box) rather than genuine security testing services even after having been educated about the differences between both. Most businesses consider the political aspect of receiving a check in the box to be a higher priority than good security (this is also true of the public sector).
This political agenda motivates decision makers to select penetration testing vendors (or other security solutions) based on cost rather than quality. Instead of asking intelligent questions about the technical capabilities of a penetration testing team they ask technically irrelevant questions about finances, the types of industries that vendor may have serviced, if a vendor is in Gartner’s magic quadrant, etc. While those questions might provide a vague measure (at best) of vendor health they completely fail to provide any insight into real technical capability. The irony is that genuine penetration testing services maintain both lower average upfront costs and lower average long-term costs than political penetration testing services.
The lower average upfront cost of genuine penetration testing comes from the diagnostic pricing methodology (called Attack Surface Pricing or ASMap Pricing) that genuine penetration testing vendor’s depend on. ASMap pricing measures the exact workload requirement by diagnosing every in-scope IP address and Web Application (“Target”) during the quote generation process. Because each Target offers different services, each one also requires a different amount of testing time for real manual testing. ASMap pricing never results in an overcharge or undercharge and is a requirement for genuine manual penetration testing. In fact, diagnostic pricing is the de facto standard for all service based industries with the exclusion of political penetration testing (more on that later).
The lower long-term costs associated with genuine penetration testing stem from the protective nature of genuine penetration testing services. If the cost in damages of a single successful compromise far exceed the cost of good security then clearly good security is more cost effective. Compare the average cost in damages of any major compromise to the cost of good security. Good security costs less, period.
Political penetration testing (the industry norm) uses a Count Based Pricing (“CBP”) methodology that almost always results in an overcharge. CBP takes the number of IP addresses that a customer reports to have and multiplies it by a cost per IP. CBP does not diagnose the targets in scope and is a blind pricing methodology. What happens if a customer tells a vendor that they have 100 IP addresses that need testing but only 1 IP address offers any connectable services? If CBP is being used then the customer will be charged for testing all 100 IP addresses when they should only be charged for 1. Is that ethical pricing?
A good example of CBP overcharge happened to one of our customers last year. This customer approached Netragard and another well-known Boston based firm. The other firm produced a proposal using CBP based on the customer having 64 IP addresses. We produced a proposal using the ASMap methodology. When we presented our proposal to the customer ours came in over $55,000.00 less than the other vendor. When the customer asked us how that was possible we explained that of their 64 IP addresses only 11 were live. Of the 11 only 2 presented any real testable surface. Needless to say the other vendor didn’t win the engagement.
CBP cannot be used to price a manual penetration testing engagement because it also runs the risk of undercharging. Any engagement priced with the CBP methodology is dependent on vulnerability scanning. This is because CBP is a blind pricing methodology that does not diagnose workload. If a customer is quoted at $5,000 to test 10 IP addresses CBP assumes the workload for the 10 IP addresses.
What happens if each IP address requires 10 hours of manual labor? Engagements priced with CBP rely on automated scanners to compensate for these potential overages and to ensure that the vendor always makes a profit. Unfortunately this dependence on automated scanning degrades the quality of the engagement significantly. The political penetration testing industry falsely promises manual services when in fact the final deliverable is more often than not a vetted vulnerability scan. This promotes a false sense of security that all too often leads to compromise.
Customers can choose to be lazy and make naïve, politically oriented security decisions or they can self-educate, choose good security and save themselves considerable time and money. While the political security path appears simple and easy at the onset the unforeseen complexities and potential damages that lie are all too often catastrophic. How much money is your business worth and what are you doing to truly protect it?
We’re offering a challenge to anyone willing to accept. If you think that your network is secure then let us test it with our unrestricted methodology. If we don’t compromise your network then the test is done free of charge. If we do compromise then you pay cost plus 15%. During the test we expect you to respond the same way that you would a real threat. We don’t expect to be whitelisted and we don’t expect you to lower your defenses. Before you accept this challenge let it be known that we’ve never failed. To date our unrestricted methodology maintains a 100% success rate with an average time to compromise of less than 4 hours. Chances are that you won’t know we’re in until it’s too late.
