Bots – ThreatRadar Bot Protection: Under the Hood (Third in a series)

In this third blog of the bots blog series, we will discuss in detail how Imperva ThreatRadarBot Protection works.
ThreatRadar Bot Protection Services is an add-on subscription service for Imperva SecureSphere Web Application Firewall (WAF). The client classification engine analyzes and classifies all incoming traffic to the customer’s site, and distinguishes between human and bot traffic, identifies “good” and “bad” bots, classifies traffic by browser type, and more. The granular level of information is then used to drive WAF policy enforcement decisions, like handling bad and suspected bots.
ThreatRadar Bot Protection leverages bot related threat intelligence built by Imperva Incapsula. Here are the detailed steps involved in bot classification. The process is somewhat identical to how Imperva Incapsula classifies bots.

Step 1: Looking at Header Data
By inspecting HTTP headers, Imperva SecureSphere WAF gains valuable insight into visitors, including various clues into whether each client is a human or automated, and whether or not it is malicious. For every client connecting to the web application, Imperva SecureSphere WAF checks the HTTP header data and order. The HTTP header is matched against a known set of bad signatures to determine if the client is malicious.
Step 2: IP and ASN Verification
The IP and ASN verification process is next on our checklist. Here Imperva SecureSphere WAFlooks for a couple of items, including the identity of the IP and ASN owners and whether they match with the visitor. The results are used to identify malicious bots posing as legitimate ones.
For example, if a bot claims to be from a search engine like Google, but neither the IPs nor the ASN used to match with that company, it’s a telltale sign that it’s likely a dangerous impostor.
Step 3: Client Finger Printing Technology
Imperva Incapsula reported a DDoS attack initiated by browser-based bots using legitimate user-agents and correct header data. They even went so far as to mimic human-like behavior. To thwart such attacks—which are becoming more common—our algorithms are augmented with additional security features.
If the client’s identity remains unconfirmed in Steps 1 & 2, Imperva SecureSphere WAFintroduces a non-intrusive JavaScript & cookie challenge in the HTTP response that checks for client behavior. The challenge empowers Imperva SecureSphere WAF to dig deeper, looking at attributes such as a JavaScript footprint and cookie/protocol support. Imperva SecureSphere WAF looks at the response from the client to determine a positive match to a malicious bot or a browser. The main purpose of this step is to confirm that the client claiming to be a browser is not a malicious bot masquerading as a browser.
Step 4: Classification Completed
At this point, Imperva SecureSphere WAF knows the client type – human or bot. If determined to be a bot, we also know the reputation of client – good, bad or unknown bots.
Step 5: Checking for Suspicious Bot Activity
Unknown bots without any known reputation get tracked within Imperva SecureSphere WAFfor abnormal/suspicious activity. You can use existing controls on Imperva SecureSphere WAF such as Geo-location, IP reputation, Anonymous Proxies/TOR nodes, Velocity checks, etc., to determine the legitimacy of an Unknown bot. Unknown bots behaving abnormally on the website get blocked while all other bots are allowed.
Step 6: CAPTCHA
When suspicious/abnormal activity gets detected, Imperva SecureSphere WAF customers can turn on CAPTCHA instead of outright blocking the session/IP address.
Also, watch this webinar that highlights the value proposition of Imperva ThreatRadar threat intelligence services for Imperva SecureSphere WAF.

In the final blog post of this series, we will discuss the logical steps within Account Takeover Protection.
Check out these other blog entries for more reading on ThreatRadar:
ThreatRadar: Finding Order Within the Chaos
Tor block, or not Tor block?
U.S. Treasury Department Report Re-affirms Imperva Messaging about Tor
The Bots are Attacking. Save Yourself.