Much has changed since we first started providing protection against DDoS attacks. Attacks which were once considered huge are now mitigated on a daily basis, attackers are becoming more sophisticated by the day, and mitigation takes a matter of seconds, as opposed to minutes, to kick in. But one thing that still remains unchanged is the manual process required to configure a security policy for an IP range.
A security policy is a set of values that defines the traffic’s baseline, taking into consideration factors such as:
- The normal amount of traffic to an IP range
- The point at which we consider the range to be at a risk of DDoS
- The steps required to mitigate the attack
The security policy struggle
For years, SOC (Security Operations Center) engineers have been responsible for configuring security policies, familiarizing themselves with the system through trial and error to develop the most effective methodologies for exceptional DDoS protection.
As talented and hard-working as these engineers might be, the growing number of customers and ranges in play means we are slowly but surely reaching a point where manual intervention just doesn’t scale. On top of that, it’s not just new security policies that require attention. Network traffic is a living thing and it tends to change with time. It grows as more users register to a web service, and it can shift when a site goes global or offers new services. Since the COVID-19 outbreak we’ve seen traffic patterns change significantly as companies adopted a work-from-home policy and started relying heavily on video conference apps. When such changes are taken into consideration, even more manpower is required to provide optimal protection. Finally, with so many policies being created and maintained manually, there is always the risk of human error.
The art of security policy creation
Creating a security policy has become almost an art form, with many different factors to take into consideration, such as:
- Most frequently used protocols and ports
- Recent changes to traffic
- Number of attacks
While these factors and more are manually accounted for by security experts to create a security policy, it wouldn’t be so easy for a machine to understand and translate them into a set of rules. SOC engineers have spent many years perfecting the art of policy creation, and without a clear formula for the enormous decision-making process they go through, the creation of a security policy algorithm might seem impossible.
Automation is key
What if we could develop an algorithm that automatically creates the optimal security policy for each IP range without any human interaction?
An algorithm with the potential to solve scaling issues because it wouldn’t require manual work. One that could be programmed not only to easily and automatically create new policies, but also to run updates on all existing security policies on a regular basis. And what if this algorithm allowed us to eliminate the need for human interaction and the subsequent risk of human error?
Imperva is leveraging Artificial Intelligence (AI) and Machine Learning to automate the creation of security policies and enhance its DDoS protection solutions. By creating the best security policy for each range and by teaching a machine to think like a SOC engineer we’re able to optimize the creation of new policies and ensure existing policies are scaled accordingly without the risk of human error. The idea was based on the thinking that, while it may be difficult for a machine to understand the many factors involved in creating a security policy, it can analyze traffic trends over time, compare recent and historic traffic, and consider distribution shifts with ease, allowing it to predict the best security policy for that range.
To create a security policy for a particular range, SOC Engineers analyze the time series traffic of that range along with other statistics.
We collect time series traffic from numerous ranges and their corresponding security policy, as created by SOC engineers. We then process the time series traffic and extract the most important features, such as maximum, average, and median, over time, to develop the predictive model. This data is further enriched with additional external information such as IP distribution, range risk, and policy creation date.
We then feed that information to a Machine Learning algorithm which connects the points between processed time series traffic and a security policy. With each iteration, the model attempts to predict the policy using the processed time series traffic, self-adjusting whenever it gets it wrong until it reaches an acceptable level of accuracy.
Testing for accuracy
In order to double-check the model and ensure its accuracy, we created a temporary manual mode where the model suggests a security policy to SOC engineers who then need to approve or decline it. Policies the model isn’t sure about are flagged as ‘risky’ to help SOC prioritize. Additionally, there’s an option whereby SOC engineers are able to add their own input so the algorithm can be tweaked as required.
In running these tests, the model has consistently predicted the same values as SOC engineers in 98% of the cases. For the remaining 2%, it has detected security policies that it isn’t sure about and flagged them for manual inspection.
Conclusions
With the growth of AI in recent years, the automation of manual tasks might seem easily achievable. In reality, though, it almost never is. As sophisticated as Machine Learning sounds, the algorithm we used was actually pretty basic stuff. Most of the work was around the human interaction side of things, such as figuring out SOC’s methodology, singling out edge cases, and understanding the feedback we received. Going through this process highlighted just how difficult it is to create a truly reliable security policy, and why being able to automate this process is invaluable.
Try Imperva for Free
Protect your business for 30 days on Imperva.
