What is Black Box Testing
Black box testing involves testing a system with no prior knowledge of its internal workings. A tester provides an input, and observes the output generated by the system under test. This makes it possible to identify how the system responds to expected and unexpected user actions, its response time, usability issues and reliability issues.
Black box testing is a powerful testing technique because it exercises a system end-to-end. Just like end-users “don’t care” how a system is coded or architected, and expect to receive an appropriate response to their requests, a tester can simulate user activity and see if the system delivers on its promises. Along the way, a black box test evaluates all relevant subsystems, including UI/UX, web server or application server, database, dependencies, and integrated systems.
An example of a security technology that performs black box testing is Dynamic Application Security Testing (DAST), which tests products in staging or production and provides feedback on compliance and security issues.
Black Box Testing Pros and Cons
| Pros | Cons | |
| 1. | Testers do not require technical knowledge, programming or IT skills | Difficult to automate |
| 2. | Testers do not need to learn implementation details of the system | Requires prioritization, typically infeasible to test all user paths |
| 3. | Tests can be executed by crowdsourced or outsourced testers | Difficult to calculate test coverage |
| 4. | Low chance of false positives | If a test fails, it can be difficult to understand the root cause of the issue |
| 5. | Tests have lower complexity, since they simply model common user behavior | Tests may be conducted at low scale or on a non-production-like environment |
Black Box and White Box Testing
Many practitioners combine black box testing with white box testing. White box testing involves testing an application with detailed inside information of its source code, architecture and configuration. It can expose issues like security vulnerabilities, broken paths or data flow issues, which black box testing cannot test comprehensively or at all.
By combining black box and white box testing, testers can achieve a comprehensive “inside out” inspection of a software application and increase coverage of quality and security issues.
Grey Box Testing
While white box testing assumes the tester has complete knowledge, and black box testing relies on the user’s perspective with no code insight, grey box testing is a compromise. It tests applications and environments with partial knowledge of internal workings. Grey box testing is commonly used for penetration testing, end-to-end system testing, and integration testing.
You can perform grey box testing using Interactive Security Testing (IAST) tools. IAST tools combine DAST and Static Application Security Testing (SAST), which is used in white box testing to evaluate static code. IAST tools enable you to combine the work of testers and developers and increase test coverage efficiently. For example, you are able to perform more directed tests which focus on areas or user paths that are most likely to contain flaws.
By combining these two testing methods you can ensure that tests:
- Apply knowledge of application structure to identify vulnerabilities and bugs
- Evaluate the application objectively and uncover UI/UX issues, as a real user would
- Cover all aspects of an applications functionality
Types Of Black Box Testing
Black box testing can be applied to three main types of tests: functional, non-functional, and regression testing.
Functional Testing
Black box testing can test specific functions or features of the software under test. For example, checking that it is possible to log in using correct user credentials, and not possible to log in using wrong credentials.
Functional testing can focus on the most critical aspects of the software (smoke testing/sanity testing), on integration between key components (integration testing), or on the system as a whole (system testing).
Non-Functional Testing
Black box testing can check additional aspects of the software, beyond features and functionality. A non-functional test does not check “if” the software can perform a specific action but “how” it performs that action.
Black box tests can uncover if software is:
- Usable and easy to understand for its users
- Performant under expected or peak loads
- Compatible with relevant devices, screen sizes, browsers or operating systems
- Exposed to security vulnerabilities or common security threats
Regression Testing
Black box testing can be used to check if a new version of the software exhibits a regression, or degradation in capabilities, from one version to the next. Regression testing can be applied to functional aspects of the software (for example, a specific feature no longer works as expected in the new version), or non-functional aspects (for example, an operation that performed well is very slow in the new version).
Black Box Testing Techniques
Equivalence Partitioning
Testers can divide possible inputs into groups or “partitions”, and test only one example input from each group. For example, if a system requires a user’s birth date and provides the same response for all users under the age of 18, and a different response for users over 18, it is sufficient for testers to check one birth date in the “under 18” group and one date in the “over 18” group.
Boundary Value Analysis
Testers can identify that a system has a special response around a specific boundary value. For example, a specific field may accept only values between 0 and 99. Testers can focus on the boundary values (-1, 0, 99 and 100), to see if the system is accepting and rejecting inputs correctly.
Decision Table Testing
Many systems provide outputs based on a set of conditions. Testers can then identify “rules” which are a combination of conditions, identify the outcome of each rule, and design a test case for each rule.
For example, a health insurance company may provide different premium based on the age of the insured person (under 40 or over 40) and whether they are a smoker or not. This generates a decision table with four rules and up to four outcomes—below is an example with three possible outcomes.
| Conditions | Rule 1 | Rule 2 | Rule 3 | Rule 4 |
| Under 40 | False | False | True | True |
| Smoker | False | True | False | True |
| Outcomes | ||||
| 1: High premium |
 |
|||
| 2: Medium premium |
|
|
||
| 3: Low premium |
|
In this case four use cases (one for each rule) would be sufficient to fully test the system.
State Transition Testing
In some systems, significant responses are generated when the system transitions from one state to another. A common example is a login mechanism which allows users to authenticate, but after a specific number of login attempts, transition to a different state, locking the account.
If testers identify a state transition mechanism, they can design test cases that probe the system when it transitions states. For example, for a system that locks the account after five failed login attempts, a test case can check what happens at the sixth login attempt.
Error Guessing
This technique involves testing for common mistakes developers make when building similar systems. For example, testers can check if the developer handled null values in a field, text in a numeric field or numbers in a text-only field, and sanitization of inputs—whether it is possible to submit user inputs that contain executable code, which has security significance.
A specific type of error guessing is testing for known software vulnerabilities that can affect the system under test.
Imperva Runtime Application Self Protection
Imperva Runtime Application Self Protection (RASP) complements white box and black box testing by adding an extra layer of protection once the application is already in production or in a realistic staging environment.
RASP has the following benefits:
- It helps test applications in-depth during fast, agile development cycles.
- It tests for unanticipated inputs, inspects and controls the system’s response.
- It provides analysis and detailed information on weaknesses and vulnerabilities, helping you quickly respond to attacks.
Imperva RASP provides these benefits, keeping your applications protected and giving you essential feedback for eliminating any additional risks. It requires no changes to code and integrates easily with existing applications and DevOps processes, protecting you from both known and zero-day attacks.
In addition, Imperva provides multi-layered protection to make sure websites and applications are available, easily accessible and safe. The Imperva application security solution includes:
- DDoS Protection—maintain uptime in all situations. Prevent any type of DDoS attack, of any size, from preventing access to your website and network infrastructure.
- CDN—enhance website performance and reduce bandwidth costs with a CDN designed for developers. Cache static resources at the edge while accelerating APIs and dynamic websites.
- Cloud WAF—permit legitimate traffic and prevent bad traffic. Safeguard your applications at the edge with an enterprise‑class cloud WAF.
- Attack analytics—mitigate and respond to real security threats efficiently and accurately with actionable intelligence across all your layers of defense.
- Account takeover protection—uses an intent-based detection process to identify and defends against attempts to take over users’ accounts for malicious purposes.
- API security—protects APIs by ensuring only desired traffic can access your API endpoint, as well as detecting and blocking exploits of vulnerabilities.
- Bot protection—analyzes your bot traffic to pinpoint anomalies, identifies bad bot behavior and validates it via challenge mechanisms that do not impact user traffic.
