What Is Penetration Testing? An Honest Answer

Definition

What is Penetration Testing?

Penetration testing is a authorized simulated cyberattack against a computer system, network, or web application to find security vulnerabilities. Think of it as hiring a professional to attempt breaking into your systems before actual criminals do.

The core purpose is simple: identify where and how attackers could gain unauthorized access to your data, systems, or physical locations. Once identified, these vulnerabilities can be fixed before they’re exploited maliciously.

Real Penetration Testing vs. Vulnerability Scanning

Here’s a critical distinction many don’t understand: most “penetration tests” sold today are actually just automated vulnerability scans with minimal human analysis.

Real Penetration Testing:

  • Human experts simulate actual attack techniques
  • Validates every finding through exploitation
  • Zero or minimal false positives
  • Discovers unknown vulnerabilities
  • Identifies complex attack paths
  • Takes 2-4 weeks
  • Costs $15,000-$100,000+

Vulnerability Scanning:

  • Automated tools check for known vulnerabilities
  • Produces reports with hundreds of findings
  • High false-positive rate
  • Cannot find business logic flaws
  • Misses complex attack chains
  • Takes 1-3 days
  • Costs $2,000-$10,000

How to Spot Scans Masquerading as Penetration Tests

Pricing Red Flags:

  • Quote based only on number of IP addresses, number of web apps, number of pages
  • No in-depth discussion of scope or hands on assessment of workload
  • Prices that seem too good to be true for the work being requested
  • Same price per target regardless of complexity

Process Red Flags:

  • Promise results in less than a week
  • No pre-engagement planning phase
  • Methodology relies on automation for vulnerability discovery.
  • No discussion of rules of engagement, legacy systems, or blackout periods. 
  •  

Deliverable Red Flags:

  • Reports contain canned output, false positives, or scan policy information
  • High volume of low or informational risk findings (low value)
  • Little to no proof of exploitation
  • Generic remediation advice
  • Basic executive summary

The analogy we use: Testing your security with just automated scans is like testing a bulletproof vest with a squirt gun instead of live ammunition. It might check a box, but it doesn’t reflect real-world threats.

Types of Penetration Tests

Network Penetration Testing

Tests internal and external network infrastructure including:

  • Firewalls and network segmentation
  • Routers, switches, and wireless networks
  • Operating systems and services
  • Remote access systems

Web Application Testing

Evaluates custom applications for vulnerabilities like:

  • SQL injection and command injection
  • Cross-site scripting (XSS)
  • Authentication and session flaws
  • Business logic vulnerabilities

Mobile Application Testing

Assesses iOS and Android apps including:

  • Client-side security controls
  • Data storage and encryption
  • API communication security
  • Platform-specific vulnerabilities

Cloud Security Assessment

Tests cloud infrastructure and services across platforms like AWS, Azure, and Google Cloud:

  • Misconfigured storage buckets and databases
  • Identity and Access Management (IAM) weaknesses
  • Insecure APIs and serverless functions
  • Container and Kubernetes security
  • Cross-account access vulnerabilities
  • Compliance with cloud security frameworks

Social Engineering Testing

Evaluates human vulnerabilities through:

  • Phishing email campaigns
  • Phone-based pretexting
  • Physical social engineering
  • USB drop attacks

Physical Security Testing

Tests real-world security controls:

  • Badge cloning and access control bypass
  • Lock picking and physical barriers
  • Social engineering and tailgating
  • Sensitive area access

The Real Manual Penetration Testing Process

1. Initial Scoping & Attack Surface Mapping (ASMap)

  • Diagnostic Discovery: We analyze your actual attack surface – not just count IP addresses like amateurs. Only live services with exploitable interfaces count toward scope.
  • Threat Modeling: Identify crown jewels, critical assets, and likely attack vectors based on your industry and infrastructure – because a bank faces different threats than a SaaS startup.
  • Custom Pricing: Diagnostic based quote that reflects real work required, not arbitrary per-IP pricing that charges you for dead hosts and closed ports.

2. Reconnaissance & Intelligence Gathering

  • Passive Reconnaissance: OSINT collection, subdomain enumeration, leaked credentials, and public exposure analysis without touching your systems – finding what attackers already know about you.
  • Active Reconnaissance: Service fingerprinting, technology stack identification, and attack surface validation using manual techniques that automated scanners miss.
  • Social Intelligence: LinkedIn harvesting, organizational structure mapping, and identifying high-value targets for potential social engineering vectors.

3. Vulnerability Discovery & Analysis

  • Manual Testing: Real vulnerability research using custom scripts, manual code review, and logic flaw identification – not just running Nessus and calling it “manual.”
  • Novel Vulnerability Research: Applying Real Time Dynamic Testing™ methodology to discover known and novel vulnerabilities and complex vulnerability chains that compliance scanners never find.
  • Business Logic Testing: Understanding how your systems actually work to find authorization bypasses, race conditions, and workflow manipulation opportunities.

4. Exploitation & Post-Exploitation

  • Proof of Concept Development: Creating working exploits (if none exist) to demonstrate real impact – not theoretical CVSS scores but actual “here’s how we’d steal your data.”
  • Lateral Movement Mapping: Documenting the Path to Compromise showing how initial access escalates to domain admin or data exfiltration.
  • Impact Demonstration: Safely proving what attackers could do without actually damaging your production environment or stealing customer data.

5. Reporting & Knowledge Transfer

  • Executive Summary: Clear business impact analysis for C-suite, not 200 pages of Qualys output with fancy formatting.
  • Technical Deep Dive: Detailed vulnerability descriptions, reproduction steps when possible, and evidence for your technical team to actually fix issues.
  • Direct Tester Access: Walk-through sessions with the actual penetration tester who did the work – not a sales engineer reading from notes.

6. Remediation Support & Validation

  • Fix Review: Reviewing your proposed fixes, if needed, before implementation to ensure they actually address root causes, not just symptoms.
  • Knowledge Transfer: Teaching your team how attackers think, not just handing over a PDF and disappearing like most vendors.
  • Alternative Solutions: Providing multiple remediation options when the “correct” fix would break your business processes or require major architectural changes.

7. Retesting & Certificate of Security

  • Complimentary Retesting: Validating all fixes within 60-day window – included in original price because we actually want you to be secure.
  • Regression Testing: Ensuring fixes didn’t introduce new vulnerabilities or break existing security controls – because sometimes the cure is worse than the disease.
  • Certificate Issuance: Earning our Certificate of Security by actually fixing issues, not just acknowledging them in a risk register like PCI allows.

 

Timeline: Real Testing Takes Time

  • Typical Duration: 2-6 weeks depending on actual attack surface complexity
  • Not Time-Boxed: Testing continues until thorough, not until arbitrary hours exhausted
  • Efficiency Focus: Diagnostics ensure time spent on real targets, not scanning dead space

 

Important note: The process above is just one example of how testing might be organized. A genuine penetration test should be customized to your specific environment, risks, and objectives. Professional testers will adapt their approach based on your industry, compliance requirements, threat landscape, and unique concerns. The timeline, techniques, and focus areas should all be tailored to deliver maximum value for your specific situation.

Cost Expectations: What's Reasonable?

Small Business

1-50 employees

  • Basic external penetration test: $10,000-$20,000
  • External + internal penetration testing: $15,000-$30,000
  • Single web application: $12,500-$35,000
  • Cloud security configuration review: $12,000-$30,000

Mid-Market

50-500 employees

  • Comprehensive infrastructure test: $25,000-$50,000
  • Multiple Web applications: $15,000-$150,000
  • Multi-cloud security configuration assessments: $35,000-$80,000

Enterprise

500+ employees

  • Full scope test: $50,000-$200,000
  • Complex cloud environment testing: $75,000-$250,000
  • Red team exercises: $100,000-$500,000

Why the wide ranges? Factors include: Number of systems and applications, Complexity of environment, Cloud platforms and services in use, Testing team experience level, Depth of testing required, Compliance requirements, Geographic considerations and more.

Regulations That Require Penetration Testing

PCI DSS (Payment Card Industry Data Security Standard)

  • Who: Any organization that processes, stores, or transmits credit card data
  • Requirement: Annual penetration testing (11.3)
  • Quarterly ASV scanning also mandated
  • Must test after significant changes
  • Segmentation validation required

FFIEC (Federal Financial Institutions Examination Council)

  • Who: US banks, credit unions, and financial institutions
  • Requirement: Regular penetration testing based on risk assessment
  • Must test internet-facing systems
  • Social engineering testing recommended

SWIFT CSP (Customer Security Programme)

  • Who: Financial institutions using SWIFT network
  • Requirement: Annual penetration testing of SWIFT infrastructure
  • Must cover all SWIFT-connected components

23 NYCRR 500 (New York Cybersecurity Requirements)

  • Who: Financial services companies operating in New York
  • Requirement: Annual penetration testing and bi-annual vulnerability assessments
  • Must be based on risk assessment

HIPAA/HITECH

  • Who: Healthcare providers, health plans, healthcare clearinghouses, and business associates
  • Current: Risk assessments required, penetration testing recommended
  • Proposed (Dec 2024): Annual penetration testing will be mandatory
  • Must test physical and technical controls

FISMA (Federal Information Security Management Act)

  • Who: US federal agencies and contractors
  • Requirement: Annual security testing including penetration testing
  • Must follow NIST SP 800-53 controls

ISO 27001

  • Who: Any organization seeking ISO certification
  • Requirement: Regular testing as part of risk assessment
  • Frequency based on risk analysis

SOC 2

  • Who: Service organizations storing customer data in the cloud
  • Requirement: Testing to validate security controls
  • Frequency determined by auditor based on risk

TISAX (Trusted Information Security Assessment Exchange)

  • Who: Automotive industry suppliers
  • Requirement: Penetration testing for high protection needs
  • Must follow VDA ISA guidelines

NIST Cybersecurity Framework

  • Who: Critical infrastructure sectors (voluntary but becoming mandatory)
  • Requirement: Penetration testing as part of “Detect” function
  • Being adopted by various sectors as mandatory

EU NIS2 Directive

  • Who: Essential and important entities in EU (energy, transport, banking, healthcare, digital infrastructure)
  • Requirement: Regular security testing including penetration testing
  • Enforcement begins October 2024

Nevada Gaming Control Board Regulation 5

  • Who: Gaming establishments and online gaming operators in Nevada
  • Requirement: Annual penetration testing of critical systems

Malta Gaming Authority

  • Who: Online gaming operators licensed in Malta
  • Requirement: Penetration testing before launch and periodically

NY DFS Cybersecurity Regulation

  • Who: Insurance companies operating in New York
  • Requirement: Annual penetration testing and bi-annual vulnerability assessments

FERPA (Family Educational Rights and Privacy Act)

  • Who: Educational institutions receiving federal funding
  • Requirement: Security testing recommended for systems containing student records
  • Many institutions require annual testing as best practice

GLBA (Gramm-Leach-Bliley Act)

  • Who: Financial institutions offering consumer financial products/services
  • Requirement: Regular testing as part of information security program

Massachusetts 201 CMR 17.00

  • Who: Any organization storing Massachusetts residents’ personal information
  • Requirement: Regular monitoring and testing of security systems

California IoT Security Law

  • Who: Manufacturers of IoT devices sold in California
  • Requirement: Security testing before product release