Dynamic Application Profiling: What It Is and Why You Want Your WAF to Have It

Because web applications are unique, they have distinct structures and dynamics, and – unfortunately – different vulnerabilities. A web application security device, therefore, must understand the structure and usage of the protected applications. Depending on the complexity of the protected application, this task can entail managing thousands or even hundreds of thousands of constantly changing variables including users, URLs, directories, parameters, cookies, and HTTP methods.
Dynamic application profiling automates management by learning application structure and usage with little-to-no manual tuning. It streamlines configuration, provides up-to-date and accurate security, and substantially reduces administrative overhead.
We refer to Imperva’s patented dynamic application profiling technology as Dynamic Profiling. In this post, we’ll explain our approach, what makes us different, and why we consider dynamic application profiling must-have functionality when evaluating web application firewall solutions.

What is Dynamic Profiling?

SecureSphere’s Dynamic Profiling automatically models an application’s structure and elements to learn legitimate user behavior such as acceptable form field values and protected cookies. Valid application changes are automatically detected and incorporated into the profile over time. Because web applications often change, SecureSphere’s automatic learning capability ensures that the application profile is always up-to-date. By comparing profiled elements to actual traffic, SecureSphere can detect unacceptable behavior and prevent malicious activity with pinpoint precision. In this way, Dynamic Profiling delivers completely automated security without requiring manual configuration or tuning.
While Dynamic Profiling automatically builds the profile of protected web applications and detects application changes over time, it also allows organizations to manually adjust the application profile (although we hear from customers that they rarely manually modify the dynamic profile due to its accuracy).
All aspects of SecureSphere’s application profile are customizable – meaning security managers can modify the application profiles to bridge any differences between actual usage and corporate security policies.

Why Do You Need Dynamic Profiling?

Any application security architecture that relies upon manual rule creation by a security administrator requires constant rule-based tuning to account for changes to the applications. For example, many web application firewalls require manually-created rules to define expected behaviors for client-side scripts. These manual rules specify detailed application variables such as allowed URLs, parameters, parameter types, and parameter constraints. Maintenance of these rules can be a major source of operational overhead as many sites rely on hundreds of scripts. Any script change requires a parallel rule change to avoid false positives.
Considering that many security managers are not kept abreast of all application changes, manually maintaining a white list security model is untenable.
Dynamic Profiling overcomes the biggest drawback of other web application firewall solutions—manual rule creation and maintenance by automatically recognizing and incorporating application changes into its profile over time.

Dynamic Profiling in Depth: Technical Specifications

SecureSphere uses machine learning to create a positive security model of the applications’ profile. While doing so it avoids false-positive by learning the behavior of the common way to interact with the application. The positive security model is also used to detect anomalies from the common interaction with the application. By learning and analyzing web requests and responses to the production web applications, SecureSphere dynamically models the application structure, elements and expected application usage (see Figure 1). It takes approximately 2-5 days of analyzing live traffic to build the application profile.
SecureSphere automatically profiles the following elements:

URLs
Directories
Cookies
Form fields and URL parameters
HTIP methods
Referrers
User authentication forms and fields for application user tracking
XML elements
SOAP actions

Figure 1: Dynamic Profiling automatically builds a profile of application elements, structure and usage, including URLs, form fields, parameters, cookies and expected user input.
In addition, SecureSphere determines expected user behavior by analyzing many different web users and their usage patterns. Expected usage attributes include:

Form field and parameter value length (approximate minimum and maximum length)
If a parameter is required or if is optional
If a parameter can be modified by the end user
The parameter value type (for example: numeric, Latin characters, foreign characters)
Allowed character groups (slash, white spaces, quotes, periods, commas, etc.)
If a cookie is protected or if it can be modified by the user
If a cookie must be set by the web server or if it can stored in the browser cache

Differentiating Between Legitimate and Illegitimate Activity

Because SecureSphere automatically learns application elements, structure and usage based on real web traffic, it must differentiate between acceptable user requests and application attacks. Otherwise, it might be possible for SecureSphere to add illegitimate requests to the application profile. SecureSphere uses the following techniques to differentiate between acceptable and malicious activity:

SecureSphere ignores known malicious behavior (HTTP protocol violations, known attack signatures like SQL injection, double encoding, etc.) when building the profile
SecureSphere analyzes server responses. If the web server replies with an error code such as “404: Not Found” or “500:Internal Server Error”, then SecureSphere will ignore the request
SecureSphere ignores web requests that have no referrer or have an external referrer unless the server generates a “200: OK” or a “304: Not Modified” response code. These types of requests may be generated by robots or scripts.
SecureSphere analyzes many different attributes when developing the web application profile. It builds the profile based on the number of occurrences, length of time, and uniformity of requests.
In addition, customers can restrict learning to trusted source IP addresses. Or customers can ignore non-trusted IP addresses.

SecureSphere automatically updates the profile over time.

The administrator can be automatically alerted every time the profile changes.
For profile updates, the behavior must be repeated by multiple sources. In addition, the behavior must be seen a certain amount of times per hour during a minimum number of hours. By default, most elements will be learned if the element is accessed by at least 50 different IP addresses or users and if the behavior is repeated at least 50 times for at least 12 different hours. All of these profile settings are configurable.

Maintain Security, Reduce IT Overhead, Keep Users Productive

Dynamic application profiling addresses the taxing manageability issues of traditional web application firewalls. With Imperva’s Dynamic Profiling organizations can protect their sensitive web applications and back-end data in an automated way, without introducing excessive IT overhead and without blocking legitimate web users.
Learn more about Imperva SecureSphere Web Application Firewall.