Incapsula is a heavy-duty security platform built to handle peak traffic rates and extreme conditions—including massive DDoS barrages. Our service processes 11+ million requests and 370 thousand attack events every minute. It also needs to be able to scale far beyond those performance numbers without a hitch.
To ensure platform resilience, every new software and configuration update we introduce is subjected to rigorous stress tests before deployment.
Tests are completed using a traffic generator—a tool used to simulate a variety of DDoS attacks, abnormal traffic patterns and other extreme situations we regularly encounter in the wild.
In this post, I want to cover the differences between hardware and software based traffic generators, why we decided to update to the TRex generator, as well as its advantages and limitations.
Finding a traffic generator that met our needs
Open-source software has always been our preference due to its flexibility and cost effectiveness. Until very recently, however, open-source packet generators were practically nonexistent, so we relied on appliances for our testing. Specifically, we used Spirent Avalanche, a very effective tool for simulating large and diversified traffic flows.
But several limitations convinced us that it was time for something new. These included:
- Cost—Stateful traffic generators, including Avalanche, require the use of hardware appliances, making them quite costly.
- Flexibility—Appliances are unable to adapt to the constant changes on the Incapsula platform.
- Scale—Commercial traffic generators are unable to scale upward to simulate the larger attacks we mitigate on a regular basis.
Getting a hand from TRex
Following some careful research, we discovered Cisco’s TRex realistic traffic generator—it’s open source, easy-to-use, fully scalable, and has an innovative and extendable software implementation that addresses the challenges with which we were faced.
Using TRex, we’re able to create templates of almost any kind of traffic. We can then amplify them to generate a variety of large-scale DDoS attacks, including TCP-SYN, UDP and ICMP floods. The tool is also able to generate enormous traffic flows, letting us simulate assaults from different clients to many target servers.
Additionally, TRex is much more cost-effective than Avalanche or other appliances. It ramps up in a fraction of the time and has a significantly higher performance capability.
As was explained to us by Hanoch Haim, Cisco Systems Principal Engineer and one of the minds behind TRex, “We designed TRex as a fully interactive, easy-to-use and highly scalable packet generator that can be customized to generate and amplify both client and server side traffic.”
The below chart details how Avalanche and TRex stack up against each other:
| Avalanche | TRex | |
| Layers 4–7 testing | Yes | Yes |
| Performance capability | 2.5–5 mpps | 160 mpps |
| Average ramp-up time | 30–60 seconds | 2 seconds |
| Average price | $100K–$200K | $10K (server + 10G cards) |
| Statefulness | Holds connection until end of session |
Stateful but doesn’t maintain connections |
| Package | Appliance | Software |
Using a traffic generator to create a real-world scenario
Despite its many advantages, TRex lacks certain features, however—specifically its own TCP/IP stack. This is needed to simulate the scale and variety of both clean and assault traffic flowing through our production servers. As a result, it doesn’t maintain connections, nor can it be used when another network element (e.g., a proxy) alters a TCP connection.
To simulate clean traffic, we reverted to our tried and true traffic generator—Avalanche. Avalanche performs many layer 7 operations not available with TRex. This includes resolving DNS queries, executing different HTTP requests and the handling of different profiles.
With Avalanche we create a realistic scenario in which clean traffic moves via a proxy server, terminates a connection and then opens a new one at the origin server. At the same time, TRex generates attack traffic aimed at our scrubbing servers, ensuring that they’re able to properly filter malicious packets.
While being a fairly recent development, large scale, open-source traffic generator tools are finally here. TRex has worked well as a DDoS attack traffic generator—until it can support a TCP/IP stack, we plan to use it in this capacity.
Meanwhile, TRex’s limitations have helped us build a system with specialized elements. These include Avalanche, which is used to represent layer 7 traffic that moves via our proxy server, terminates its connection and opens a new one at the origin server.
For more information about how we work with TRex and how it might be used to test your site, contact us.
Try Imperva for Free
Protect your business for 30 days on Imperva.
