Introduction
Penetration Testing as a Service (PTaaS) has become a popular topic of discussion for many organizations seeking to improve their overall security posture. It eliminates the usual gap between traditional penetration testing schedules, provides real-time results to organizations, automatically retests reported issues, and costs less than traditional tests. Conceptually it is a fantastic idea, but in practice how effective is it and what is the real protective benefit?
Real-World Threat Actors: The Comparative Baseline
Before we delve into the technicalities and capabilities of PTaaS it’s important to establish a comparative baseline. The objective of penetration testing is to proactively identify and mitigate vulnerabilities before they are exploited by malicious threat-actors, which is how protective benefit is derived.
Modern threat actors deploy a range of sophisticated techniques, tactics, and procedures (TTP’s) to breach networks, continuously refining their methods to stay ahead of cybersecurity defenses. They exploit both known and novel vulnerabilities, leveraging advanced tactics supported by complex, purpose-built infrastructures. They create novel malware variants and proprietary command and control (C2) frameworks to maintain undetected access and covertly metastasize within networks while continually evolving to counter new defenses. Additionally, they make use of refined social engineering tactics, manipulating victims into compromising their own security.
This dynamic landscape forces the cybersecurity industry to perpetually adapt, as every existing security solution is essentially a response to a threat initially devised by threat-actors. Threat actors only need to be successful once while defenders need to get it right every time.
The capabilities of modern threat-actors define the minimum baseline for effective penetration testing. Testing at lesser levels of capability will result in otherwise avoidable gaps in coverage that attackers can capitalize on to cause a damaging breach.
What Is PTaaS and How Does It Work?
With the baseline for effective penetration testing established, a high-level comparative review of PTaaS is possible. The first step is to understand what PTaaS is and how it works.
PTaaS is a cloud-based cybersecurity service that enables organizations to access on-demand penetration testing in an agile process. PTaaS platforms rely extensively on automated tools for scanning vulnerabilities, which are essential for detecting known security issues. The automation aspect is crucial for keeping costs down and enabling continuous monitoring. These automated scanners operate by comparing the scanned environment against a pre-existing database of vulnerabilities, which means they might not catch new, unknown vulnerabilities. A common issue with these scanners is the occurrence of false positives, where they incorrectly flag a vulnerability that doesn’t actually exist. However, PTaaS counters this by incorporating a manual validation step, which significantly enhances the quality of the service beyond what is typically achieved with standard automated scanning.
Beyond the conventional methods of mapping vulnerabilities and the attack surface, PTaaS providers are increasingly integrating Artificial Intelligence (AI) to augment their services. This integration bolsters the efficiency and precision of PTaaS solutions, which may employ behavioral analysis to detect anomalies and utilize machine learning to analyze historical data, thereby evolving attack patterns.
Typically, PTaaS customers are provided with access to a web-based portal. Through this portal, customers can schedule and manage penetration testing projects, view real-time progress updates, and access detailed reports of vulnerabilities. These portals often allow users to configure and customize testing parameters, request retests, and directly interact with security experts for guidance on remediation strategies. Some allow for the integration of security findings with other tools and workflows, such as Jira or Slack, enhancing the efficiency of the remediation process.
PTaaS delivers notable advantages for organizations aiming to effectively manage known vulnerabilities. It saves time by eliminating the need for IT staff to investigate false positives as well as allowing the teams to focus on advancing and hardening their existing security posture.
The Limitations of PTaaS
PTaaS, while beneficial, also has important limitations that may not be immediately apparent to end users. Its reliance on automated vulnerability scanning makes it susceptible to not only false positives but, more importantly, false negatives. False negatives are failures to detect existing known vulnerabilities and can also surface because automated scanners cannot identify new, previously unknown vulnerabilities.
Consequently, any penetration testing service that depends heavily on automated scanning will leave gaps in coverage that become opportunities for threat-actors to cause a breach. In the case of PTaaS, closing these gaps would require a prohibitive increase in cost as a direct result of the increased manual labor and capability requirements.
For small businesses or organizations lacking a specialized security team, the persistent concern over threats from PTaaS can overwhelm IT staff, potentially diminishing their alertness to genuine security breaches.
While PTaaS is praised for its continuous surveillance and assessment of potential security vulnerabilities, its effectiveness hinges on the accuracy of the IT asset inventory. As organizations and end-users increasingly adopt decentralized systems and diverse cloud services, maintaining an up-to-date and detailed asset list becomes significantly more complex. Without a clear understanding of the location and nature of your assets, PTaaS may fall short of the insights provided by a seasoned expert attuned to identifying subtle security risks. Additionally, most security solutions are equipped to identify and block activities from scanners and commercial penetration testing tools. Without appropriate measures such as whitelisting the IP addresses of PTaaS scanners, these defenses will block some or all the legitimate testing efforts. If this happens, the environment will appear more secure than it actually is. This oversight can provide real-world attackers, who typically do not use detectable commercial tools, with increased opportunities to cause a damaging breach.
Conclusion
PTaaS falls short of meeting Netragard’s requirements for effective penetration testing. Its dependence on automated vulnerability scanning, commercial tools, and standardized exploitation methods, coupled with a focus on cost-efficiency, are significant constraints. While theoretically possible, elevating PTaaS to match the capabilities of real-world threat actors would necessitate a substantial, and likely cost-prohibitive, increase in customer cost. PTaaS marks an advancement in vulnerability management, but it does not match the technical depth and rigor of genuine penetration testing.
Nevertheless, PTaaS does provide considerable value and should be considered. It offers a substantial improvement over traditional vulnerability scanning services by integrating expert analysis to eliminate false positives and provide ongoing support. It also enables continuous monitoring of IT environments and should be seen as a complimentary service to genuine penetration testing, not its competition.
