Verify Your Security Provider — The truth behind manual testing.

Verify Your Security Provider — The truth behind manual testing.

Security Configuration Assessment

Something that I’ve been preaching for a while is that automated vulnerability scanners do not produce quality results and as such shouldn’t be relied on for penetration tests or vulnerability assessments. I’ve been telling people that they should look for a security company that offers manual testing, not just automated scans. The price points for quality work will be significantly higher, but in the end the value is much greater. After all the cost in damages of a single successful compromise is far greater than the cost of the best possible security services.

I’ve noticed that there are a bunch of vendors who claim to be performing manual testing.But when I dig into their methodologies their manual testing isn’t real manual testing at all, its just vetting of automated scanner results or testing based on the results.In other words they test on what the automated scanner reports and don’t do any real manual discovery.I’m not saying that tools like nessus (an automated scanner) don’t have their place, I’m just saying that they aren’t going to protect you from the bad guys.If you want to be protected from the threat, you need to be tested at a level that is a few notches higher than the threat that you are likely to face in the real world.

This is akin to how the Department of Defense tests the armor on its tanks, and I’ve probably mentioned this before somewhere on the blog. But, we don’t test our tanks against fire from bb guns and .22 caliber pistols. If we did that they wouldn’t be very effective in war. We test the tanks against a threat that is a few levels higher in intensity than what they are likely to face in the real world. As a result, the tank can withstand most threats and is a very effective weapon. Doing anything less isn’t going to protect you when the threat tries to align with your risks; you’ll end up being an expensive casualty of war.

So why do some security companies test at this lesser level? Its simple really, they are in the business of making money and care more about that then they do about actually protecting their customer’s infrastructure. Additionally, there is a market for that sort of low quality testing. There are some businesses that don’t actually care about their security posture; they just care about passing the test so that they can put a check in their compliancy box. Then there are other businesses that unknowingly get taken advantage by of vendors because they don’t know the difference between high quality and low quality services.

So what is the difference between high quality and low quality?From a high level perspective it’s the difference between real manual research based security testing or not. Once hackers have access, they can do anything to your data from steal it, to install back door technology in your product’s source code.Its happened before, and its going to happen again.

When a company tells you that they perform manual testing hold their feet to the fire. You can do the following things to verify it:

  • Dig into their methodology and ask them specific questions about how they perform their testing. (See our white papers on how to do that).
  • Don’t swallow jargon and terms that sound cool and don’t mean anything, use Wikipedia to look up the terms and make sure that they make sense.
  • Ask them for the names of their security experts and then use tools like Google, LinkedIn, Facebook and PIPL to do research on those experts. If nothing comes up then chances are their experts aren’t experts at all.
  • Search vulnerability databases like milw0rm, securityfocus, sirtfr, secunia, packetstormsecurity, etc. for the vendor’s name to see if they have research capabilities. If you don’t get anything in return then chances are that they don’t have research capabilities.If that’s the case then how do you expect them to perform quality manual testing?Chances are that they won’t be able to.

Remember you’re putting the integrity of your business and its respective name into their hands.

Blog Posts

Karen Huggins

Chief Financial, HR and Admin Officer
Karen joined the Netragard team in 2017 and oversees Netragard’s financial, human resources as well as administration functions. She also provides project management support to the operations and overall strategy of Netragard.
Prior to joining Netragard, she worked at RBC Investor Services Bank in Luxembourg in the role of Financial Advisor to the Global CIO of Investor Services, as well as several years managing the Financial Risk team to develop and implement new processes in line with regulatory requirements around their supplier services/cost and to minimize the residual risk to the organization.
With over 20 years of experience in finance with global organizations, she brings new perspective that will help the organization become more efficient as a team. She received her Bachelor of Finance from The Florida State University in the US and her Master of Business Administration at ESSEC Business School in Paris, France.

Philippe Caturegli

Chief Hacking Officer
Philippe has over 20 years of experience in information security. Prior to joining Netragard, Philippe was a Senior Manager within the Information & Technology Risk practice at Deloitte Luxembourg where he led a team in charge of Security & Privacy engagements.

Philippe has over 10 years of experience in the banking and financial sector that includes security assessment of large and complex infrastructures and penetration testing of data & voice networks, operating systems, middleware and web applications in Europe, US and Middle East.

Previously, Philippe held roles within the information system security department of a global pharmaceutical company in London. While working with a heterogeneous network of over 100,000 users across the world and strict regulatory requirements, Philippe gained hands-on experience with various security technologies (VPN, Network and Application Firewalls, IDS, IPS, Host Intrusion Prevention, etc.)

Philippe actively participates in the Information Security community. He has discovered and published several security vulnerabilities in leading products such as Cisco, Symantec and Hewlett-Packard.

He is a Certified Information Systems Security Professional (CISSP), Certified Ethical Hacker (CEH), PCI Qualified Security Assessors (PCI-QSA), OSSTMM Professional Security Analyst (OPSA), OSSTMM Professional Security Tester (OPST), Certified in Risk and Information Systems Control (CRISC)and Associate Member of the Business Continuity Institute (AMBCI).

Adriel Desautels

Chief Technology Officer
Adriel T. Desautels, has over 20 years of professional experience in information security. In 1998, Adriel founded Secure Network Operations, Inc. which was home to the SNOsoft Research Team. SNOsoft gained worldwide recognition for its vulnerability research work which played a pivotal role in helping to establish today’s best practices for responsible disclosure. While running SNOsoft, Adriel created the zeroday Exploit Acquisition Program (“EAP”), which was transferred to, and continued to operate under Netragard.
In 2006, Adriel founded Netragard on the premise of delivering high-quality Realistic Threat Penetration Testing services, known today as Red Teaming. Adriel continues to act as a primary architect behind Netragard’s services, created and manages Netragard’s 0-day Exploit Acquisition Program and continues to be an advocate for ethical 0-day research, use and sales.
Adriel is frequently interviewed as a subject matter expert by media outlets that include, Forbes, The Economist, Bloomberg, Ars Technica, Gizmodo, and The Register. Adriel is often an invited keynote or panelist at events such as Blackhat USA, InfoSec World, VICELAND Cyberwar, BSides, and NAW Billion Dollar CIO Roundtable.