What is Fuzzing (Fuzz Testing)?
Fuzzing is a quality assurance technique used to detect coding errors and security vulnerabilities in software, operating systems, or networks. It works by attempting to crash a system or trigger errors by supplying a large volume of random inputs. If a vulnerability is found, a fuzz testing platform (also called a fuzzer) can help determine the root cause.
Fuzzing systems are very good at finding certain types of vulnerabilities, including buffer overflow, denial of service (DoS), cross-site scripting, and code injection. However, they are less effective at dealing with silent security threats that do result in crashes or visible errors—such as spyware, worms, trojans, and rootkits.
While fuzzing is a simple technique, it is cost-effective and easy to scale. It can often uncover serious flaws that go unnoticed when creating and debugging software programs. However, it does not provide a complete picture of security, quality, or effectiveness of a software product. Therefore, it is usually combined with other techniques like black box testing, beta testing, and unit testing.
Types of Fuzz Testing
Application Fuzzing
This fuzzing method tests UI features such as buttons, input fields in forms, or options in command-line programs. It can similarly be used to test API commands. It works by accessing features at an unusually high frequency, providing invalid content such as too much text in input fields, and trying various random inputs.
Protocol Fuzzing
Protocols such as Hypertext Transfer Protocol (HTTP) are used to exchange data over the web. Protocol fuzzing is used to test the behavior of a server when bad content is sent over a given protocol. The main purpose is to prevent protocol requests from being misinterpreted as commands and executed on the server.
File Format Fuzzing
File format fuzzing creates a corrupted file and presents it to the target software for processing. This is relevant both for installed software and web applications that accept files as input. Files are usually in standard formats, such .jpg, .docx or .xml. A fuzzer can test the application by providing files that do not match the expected format or contain unexpected content. An advanced version of file format fuzzing can test features for specific file formats, such as image or video compression methods.
Fuzzing Security Tools
Generative Fuzzers
A generative fuzzer can be anything from completely random data to slightly manipulated data. For example, when fuzz testing HTTP traffic, it is possible to fuzz the entire packet, which means it would likely not reach its destination. Alternatively, a generative fuzzer can break the packet into its individual components and try to fuzz each of them separately, keeping the structure of the packet intact. This allows testing invalid packet content while keeping the TCP/IP and HTTP headers in place.
Mutation Fuzzers
Mutation-based fuzzers take a set of valid inputs and perform mutations on them to trigger errors or crashes in the software under test. Here are a few examples of mutation techniques:
- Least significant bit (LSB) flipping—changing the bit at the end of every binary integer.
- Obfuscating HTTP requests—appending a random value to each HTTP header value. This can be very effective for discovering vulnerabilities, and achieves high code coverage, because the input is sufficiently similar to the original valid input.
- Templates—using a valid data structure or format to increase the chance that a fuzzed input will be accepted by the target system. This can reduce the time and resources of a fuzz test by ensuring that inputs are well-formed.
Evolution Fuzzers
Evolutionary fuzzing is based on the use of genetic programming, designed to converge toward an input that will result in an error. Genetic algorithms use the concepts of mutation, crossover, and selection to find solutions to complex problems.
In fuzz testing, genetic algorithms can be used to generate a continuous set of test cases. Test case generation is based on a fuzzing framework and the responses received from fuzzing targets. The first set of test cases is created using a generative or mutation approach, and subsequent test cases are created using genetic progression:
- Each test case is scored, using various metrics that indicate how successful it was at causing an error.
- Test cases with the lowest score are removed.
- Small changes are made to each of the remaining test cases.
- Finally, the algorithm combines high-scoring test cases to find test cases with higher scores. This process is also used to supplement discarded test cases and try to find more successful candidates.
Fuzz Testing Best Practices
Here are a few ways you can evaluate the effectiveness of a fuzzing solution.
Improve Testing Speed
A critical metric for fuzzing is now many test cases you can run per second. The more test cases you can run in a given timeframe, the more likely you are to find a crash or error. Faster fuzz testing also makes it possible to integrate fuzzing into automated testing processes.
There are many things you can do to speed up your test cases, such as making generative or mutation routines more efficient, parallelizing test cases, reducing timeouts, and running your program in headless mode (without a user interface). If you are hosting fuzz testing yourself, you can also run it on more powerful hardware.
Reduce Test Cases
Because fuzzing randomly changes the input, test cases often contain variations or mutations that do not actually trigger an error. Reducing test cases means narrowing test cases to the smallest set of changes that are likely to cause a bug or crash.
This reduction can be done manually, but it can also be done automatically by the fuzzing solution. When a crash occurs, the fuzzer can rerun the test case multiple times. Each time a bug is triggered, it can incrementally reduce changes to the input (compared to a base valid input), until it arrives at the minimal change needed to trigger the error. This simplifies analysis and helps understand exactly which part of the input is related to the error or crash.
Track Code Coverage
Code coverage is a measure of how much of your software code a fuzzer executed. The notion is that the wider the coverage, the more extensively the fuzzer tests the programs. There are many ways to measure code coverage including lines, code blocks, branches, and code paths.
Measuring code coverage for fuzzing can be difficult, and might require binary instrumentation to track what code is running during each fuzz request.
Code coverage is not a perfect measure for fuzz testing because much of the application code will not result in an error, even if executed. Still, some form of code coverage measurement can provide insight into what the fuzzer is actually triggering in your program, and enable fine tuning of the fuzzer’s activity.
Application Security with Imperva
Imperva complements fuzz testing solutions with comprehensive protection for applications, APIs, and microservices:
Web Application Firewall – Prevent attacks with world-class analysis of web traffic to your applications.
Runtime Application Self-Protection (RASP) – Real-time attack detection and prevention from your application runtime environment goes wherever your applications go. Stop external attacks and injections and reduce your vulnerability backlog.
API Security – Automated API protection ensures your API endpoints are protected as they are published, shielding your applications from exploitation.
Advanced Bot Protection – Prevent business logic attacks from all access points – websites, mobile apps and APIs. Gain seamless visibility and control over bot traffic to stop online fraud through account takeover or competitive price scraping.
DDoS Protection – Block attack traffic at the edge to ensure business continuity with guaranteed uptime and no performance impact. Secure your on premises or cloud-based assets – whether you’re hosted in AWS, Microsoft Azure, or Google Public Cloud.
Attack Analytics – Ensures complete visibility with machine learning and domain expertise across the application security stack to reveal patterns in the noise and detect application attacks, enabling you to isolate and prevent attack campaigns.
Client-Side Protection – Gain visibility and control over third-party JavaScript code to reduce the risk of supply chain fraud, prevent data breaches, and client-side attacks.
