Key Takeaways:
- Compliance does not equal security. Passing an audit confirms that certain requirements have been met. It doesn’t tell you whether an attacker can find a way into your environment.
- Scale is not sophistication. AI can generate thousands of automated attack events quickly. That’s scale. It doesn’t necessarily mean the attack itself is advanced.
- Automated and manual penetration testing answer different questions. Scanners are useful for finding known vulnerabilities across a large number of systems. Manual adversarial security testing looks at what an attacker can do with those vulnerabilities once they’re found.
- Threat intelligence needs context. The useful question isn’t whether a threat exists. It’s whether that threat has a realistic path to the systems and data your organization needs to protect.
Security teams have no shortage of tools. There are vulnerability scanners, compliance platforms, endpoint products, threat intelligence feeds and, increasingly, AI-powered security products promising to help teams find and respond to threats faster.
Organizations can have all of those things and still get breached.
This isn’t a new problem. When Netragard was founded in 2006, CEO Adriel Desautels encountered an enterprise security system that looked solid on paper. Its controls had been certified and management believed the perimeter was secure. When the environment was approached the way an adversary would approach it, a critical vulnerability led to network compromise in four minutes.
The tools have changed considerably since then. The basic disconnect hasn’t.
An organization can satisfy a security framework without understanding how an attacker would move through its environment. AI makes that problem more urgent because attackers can now probe systems, test permutations and automate repetitive work much faster than they could before.
Desautels recently spoke with Business Insight about some of the security mistakes he continues to see. One of the biggest is the tendency to confuse compliance with security.
Compliance-Driven Security Theater
Compliance frameworks have a purpose. The problem starts when passing the audit becomes the security strategy.
If a security team is primarily working toward a checklist, anything that falls outside that checklist can receive less attention. That could be an unusual trust relationship between systems, a business process, a vendor integration, an overprivileged account or something unique to the way that particular company operates.
Attackers aren’t limited by the framework.
As Desautels told Business Insight:
“Every organization is unique. Every person in an organization is unique. The path to compromise is different. The only way you’re going to build effective defenses is if you first suffer a breach at the hands of real threat actors, and you can do that either by hiring a team like ours, or at the hands of a bad guy.”
This is what Desautels describes as compliance-driven security theater.
Standards such as SOC 2, ISO 27001 and PCI-DSS give organizations a useful baseline. They can require controls, documentation and processes that absolutely should exist.
But an attacker isn’t checking whether those controls exist. They’re looking for the place where they fail.
That could be the relationship between two systems that trust each other too much. It could be a credential with more access than it needs. It could be an application workflow that behaves differently than anyone expected.
Passing an audit doesn’t answer those questions.
Compliance vs. Security
The cybersecurity compliance gap becomes particularly obvious when automated vulnerability scanning is treated as a replacement for offensive security testing.
Scanners are useful. They can cover large environments quickly and identify known vulnerabilities, missing patches, exposed services and common configuration problems.
What they don’t do particularly well is think about what happens next.
A scanner might identify an outdated service on one machine, a permission problem on another system and an exposed credential somewhere else. Those may appear in the report as three separate findings.
An attacker wants to know whether they can be chained together.
That’s a major difference between automated and manual penetration testing. Automated scanning is designed to identify known problems across a defined set of targets. Manual adversarial testing is concerned with what those problems allow someone to do.
A skilled penetration tester doesn’t stop because a vulnerability has been found. The finding creates another set of questions: Where does this lead? What trusts this machine? What can this account access? Can this seemingly minor weakness be combined with something else?
That’s how individual vulnerabilities become a path to compromise.
A list of findings is useful. Knowing which of those findings can get an attacker to sensitive data is considerably more useful.
Contextualized Threat Intelligence
Security teams have access to more threat intelligence than they can reasonably consume. Feeds can deliver enormous numbers of indicators, vulnerabilities, alerts and reports, but volume alone doesn’t tell a company what it should be worried about.
The threat has to be considered in the context of the organization.
As Desautels explained to Business Insight:
“Specifically, how does a threat align with your people, culture, and opportunities to get at your specific data? You take that intelligence to build threat-informed defenses, save money by not wasting it on what doesn’t matter, and build something with a 12,000% ROI if it prevents a single breach.”
That’s contextualized threat intelligence.
Instead of trying to defend every asset equally against every possible threat, start with the things the business can’t afford to lose. Which systems or data would cause serious damage if they were stolen, encrypted, manipulated or made unavailable? Who and what can access them? Which vendors, applications, credentials, service accounts and other identities connect to them? And how could an outsider realistically get there?
Those are the paths worth pressure-testing.
Netragard’s penetration testing services are designed to determine whether vulnerabilities, permissions and trust relationships create viable paths through an organization’s infrastructure.
Red team services take a different approach. A red team works toward an objective, such as gaining access to sensitive information or taking control of a critical system, while testing whether the organization’s people, processes and defensive controls can detect and stop the activity.
Neither exercise is about generating the thickest possible report. The useful information is whether an attacker can reach something you care about and, if so, how.
AI Attacks and Hyper-Automation
AI changes the speed and scale of offensive activity, but some of the discussion around AI attacks gives the technology more credit than it deserves.
During an OpenAI security evaluation reported by the Financial Times, an automated model generated more than 17,000 attack events against Hugging Face infrastructure.
Desautels’s assessment was less dramatic:
“OpenAI’s model generated over 17,000 events to attack Hugging Face. That’s not a crafty attack. That’s not advanced. That’s not even script kiddie level: it’s hyper automation, and it’s super easy to defeat. AI did not do anything novel.”
Netragard took a deeper look at what happened in AI Didn’t Go Rogue: How OpenAI’s Harness Failed and Turned Hugging Face Into Collateral Damage.
The problem is that scale and sophistication are often treated as the same thing.
AI can make attackers faster by reducing manual effort, automating reconnaissance, testing more permutations and handling repetitive tasks. But generating 17,000 attack events doesn’t make the underlying attack sophisticated.
If an AI agent gets into a network through an unpatched vulnerability, an overprivileged service account or a broken authorization boundary, it’s taking advantage of a weakness that was already there.
The attacker got faster. The security problem was already waiting for them.
New Technology, Familiar Security Failures
The same thing is happening on the defensive side.
Companies are moving quickly to deploy AI assistants, copilots and agents that can interact with internal systems and data. That introduces new considerations, but it doesn’t erase basic security principles around identity, permissions and access control.
The Azure OpenAI incident discussed in Netragard’s Are We Forgetting Basic Security Practices With AI? is a good example.
An enterprise AI assistant exposed sensitive internal SharePoint files to users who weren’t supposed to have access to them. It’s easy to describe something like that as an AI security problem, but the underlying failure was much more familiar. The assistant used service credentials with excessive read permissions and lacked adequate query-time authorization controls.
AI was involved. So were overprivileged credentials and weak access controls.
Those kinds of problems have been creating paths to compromise for years.
This is one of the risks of treating AI security as an entirely separate discipline. Companies can spend a lot of time worrying about what a model might do while overlooking the permissions, identities, integrations and systems the model already has access to.
Test the Environment, Not the Checklist
AI and automation will continue to make attackers faster. Security teams should plan for that, but it doesn’t require abandoning the fundamentals.
It makes the fundamentals less forgiving.
An overprivileged service account is more dangerous when automated tools can discover and abuse it quickly. An exposed service can be tested at greater scale. A trust relationship that nobody has looked at in years is still a trust relationship an attacker can follow.
Adding another security product doesn’t necessarily close those paths, and neither does passing another audit.
If an organization has broad permissions, unmonitored lateral trust paths, weak authorization controls or defenses that have never been tested under pressure, those weaknesses are available to an attacker whether the attacker is using manual tradecraft, a script or an autonomous AI agent.
The useful question isn’t whether the organization passed its last audit.
It’s what happens when someone tries to break in.
Test the assumptions. Follow the trust relationships. Find the paths that shouldn’t exist and close them before somebody else finds them.
FAQ
Does compliance equal security?
No. Compliance confirms that an organization meets a predefined set of requirements or controls. It does not demonstrate that the environment can withstand a targeted attack. Compliance frameworks are standardized, while enterprise infrastructure, permissions, applications, trust relationships and business processes are specific to each organization
What are the limitations of automated penetration testing?
Automated vulnerability scanners are effective at identifying known vulnerabilities, missing patches, exposed services and common misconfigurations at scale. They are more limited when it comes to interpreting business context, testing custom workflows, understanding trust relationships or chaining multiple findings together to determine whether they create a viable path to compromise.
What is contextualized threat intelligence?
Contextualized threat intelligence evaluates threats against an organization’s specific systems, people, data, business processes and attack surface. It helps security teams focus defensive resources on realistic attack paths rather than treating every possible threat as equally relevant.
How does red teaming differ from penetration testing?
Penetration testing typically evaluates a defined technical scope to identify and exploit vulnerabilities and determine whether they create paths to compromise. Red teaming is objective-driven. A red team simulates an adversary attempting to accomplish a specific mission while testing how the organization’s people, detection tools, processes and defensive controls respond.
