[Latest Case Study] tiket.com & Its Compliance Journey To ISO 27001 and PCI-DSS Certification
logo
Penetration Testing

Between The Shades: Black, White & Gray-Box Penetration Testing

Black-box, gray-box, and white-box penetration testing are popular terms to describe the three primary types of penetration testing. How did they come about and what is the significance of each?

Data is to hackers what gold is to pirates. To safeguard the riches hidden behind internet-connected systems, businesses often opt for penetration testing — whether through a third party or with their internal security team — to reduce the risk of compromise. Penetration testing (or pentesting) is a simulated cyber attack and cybersecurity assessment to determine security posture and uncover vulnerabilities in a target system, be it a web application or network server.

Anyone who is familiar with pentesting would have heard of the three primary forms of pentesting: (i) Black-box, (ii) Gray-box, and (iii) White-box pentesting. In this blog post, I will explain what they are and how each differs from the other. First used in software testing, each type of pentesting describes a different approach taken by a pentester when designing test cases:

  • Black-box pentesting: As a business, do you want the pentester to act as if an outsider (or black hat hacker) with little to no knowledge of your systems? 
  • White-box pentesting: Would you rather provide the pentester a complete picture of your target system?
  • Gray-box pentesting: Or do you want to reveal some information but not too much? 

Naturally, with each of the approaches, the effort differs. Black-box pentesting requires little support from the client other than basic system information such as URL and IP address. White-box, on the other hand, requires full client support to provide privileged access and detailed system information. 

Which approach is preferred and makes the most sense for your business?

Black-Box Pentesting

Typically the most basic of pentests, a big advantage of black-box pentesting is the minimal effort required from the client. If a tester were to test, for instance, a food delivery business’ web application (Web Application Penetration Test), the tester will usually only be able to test the login page and any other publicly accessible pages (eg. Contact Form) for vulnerabilities. Unless the pentester is able to breach the login page and gain access to the application, no further functions would be testable. This is akin to an outsider trying to break into a system with limited knowledge.

Certainly, enumerating functions or fuzzing are other ways to identify potential vulnerabilities, but these often yield ineffective results even after long hours. The primary disadvantage of black-box then is that it yields limited information about real vulnerabilities of a target system.

Typical Vulnerabilities Uncovered In Black-Box Pentesting

  • Injection vulnerabilities on forms
  • Web server misconfigurations (eg. directory access to sensitive files, TLS issues)
  • Web server vulnerabilities (eg. Heartbleed, default CGI access)
  • Password guessing and Login Bypass
  • Network vulnerabilities

White-Box Pentesting

Unlike the black-box approach, white-box pentesting is very labor-intensive on the client’s end. The client’s duty is to help the tester understand the target system as much as possible, granting them full admin access and detailed information including source code and documentation. With this, enumeration or fuzzing is less required, since vulnerabilities can be directly verified on the source code or through the server access.

On the tester’s end, white-box pentesting is also very labor-intensive. A lot more time needs to be invested in analyzing the information and access provided. The advantage, though, is that white-box pentests provide the most comprehensive report when it comes to understanding complex vulnerabilities such as logic flaws or second order injections.

A white-box approach can uncover a vulnerability located in a backend batch job function that requires several conditions to be fulfilled together in order to be successfully exploited — for instance, being able to locate the right parameter to inject on the correct form with the correct test payload and knowing exactly when the execution happened. In a black-box or gray-box approach, it is not easy to detect such a vulnerability.

Typical Vulnerabilities Uncovered In White-Box Pentesting

  • Hidden or hard-to-reach functions
  • Security control bypass
  • Second order injections
  • Complex logic flaws
  • Vulnerabilities found in both black and gray-box approaches

Gray-Box Pentesting

The final shade of gray-box pentesting falls between the black-box and white-box approach, to no one’s surprise. Why is this gray-box pentesting the most popular option?

  • Balanced effort on client’s end: The client needs to only provide partial information such as API documentation about the target system.
  • Balanced effort on tester’s end: The partial information provided minimizes the need for enumeration or fuzzing of the target system.
  • Flexibility: If more support is required at any point in time, such as code snippets, the tester can simply request the information from the client.
  • Privacy on client’s end: The client does not need to reveal too much proprietary or sensitive information such as source code.

For instance, if the pentester was planning to exploit a cross-site scripting vulnerability, they would simply request for the blacklist or whitelist filter from the client and find the right payload to exploit the vulnerability. This can potentially save the tester hours of guessing or extensive fuzzing to find that payload. 

Black-Box, White-Box, Gray-Box: Which To Choose?

The gray-box and white-box approaches will give your business a clearer picture of your target system’s security posture. Security through obscurity is not a best security practice and should not be relied upon. In cryptography, Kerckhoffs's Principle states that a system should be secure even if all information on the system, except for the secret key, is made publicly available. (But please refrain from uploading your source code to your public Git repository to prove a point!) The penetration tester is your ally and should be trusted. Therefore, you should provide good support to the tester and ensure that the tester gets the necessary information or access to test it effectively.

You might then be wondering: what is the point of black-box pentesting? The black-box approach still has its place. It all depends on the objective of the penetration test. The rationale behind black-box testing is typically to assess how easily an outside attacker can cause harm to the target system given a limited amount of time and resources. An important distinction to be made here is that, even though a black-box test may not have yielded any findings, a motivated attacker may still be able to compromise the target system with enough time and resources. They are not bound to the scope of work of pentesters and are able to perform other forms of attacks, such as social engineering or physical intrusion, to gain the initial access to the target system. From there, the attacker is able to find and exploit vulnerabilities on the target system that a gray-box or white-box pentest could have detected.

Summing It Up

Each approach has its own benefits, and each organization should evaluate and choose the approach best suited for their needs at that point in time. It is also important to understand that penetration testing is not a cure-all solution. Even if your front door is made with military-grade technology, a burglar just needs to find an unlocked window to enter your house.

Frequently Asked Questions (FAQs)

How do I know if I need white-box over a black-box or gray-box pentest?

As a business, you should spend on things that have a positive Return on Investment (ROI). The next question is: how do you know if a white-box pentest will yield a positive ROI? First calculate the risk of your assets getting compromised and the potential damage of that. If the cost of a single white-box test is lower than the risk-adjusted loss or repeated gray-box tests, it might make sense to do a white-box pentest. The above example is an oversimplified way to make the decision. I would recommend you to consult an expert to provide you better advice if you are unsure.

What is the cost for the different types of pentests?

Most of the consultancy firms that provide penetration testing charge using a per-day rate. Generally, the effort required for white-box pentests is the highest, followed by gray-box and then, black-box. This may be counter intuitive as I did say that there are effort savings when conducting white-box or gray-box pentests. 

Reason is that it is almost impossible to expect the same comprehensive white-box results using a black-box approach even if the tester is given double the time of a white-box test. There are a lot of hurdles that the tester needs to solve, such as getting privileged access and gaining internal knowledge of the target system. The tester would need time to analyze every new piece of information. Naturally, if you expect the level of detail from a white-box test, the effort increases significantly with a black-box or gray-box approach.

Loh Soon Bock
Loh Soon Bock

Soon Bock (OSCP, CRT, CISSP) is a Horangi CyberOps Consultant who specializes in penetration testing and secure code review.

Subscribe to the Horangi Newsletter.

Hear from our Horangi tech experts as we go deep into up-and-coming cyber threats, new solutions, and talk about the future of cybersecurity.