In this blog post, we’ll share the results of an internal research project we conducted on our CDN customers focused on websites that are getting non-Server Name Indication (SNI) traffic.
The goal of our research was to answer the following questions:
- How much non-SNI traffic is seen?
- What is the reason for such non-SNI traffic?
- What are the HTTP clients responsible for this traffic?
- Do our customers need that traffic or should it be blocked?
If you are a website owner considering enforcement of an SNI-only policy, this article will interest you.
What is SNI and What is it Good For?
SNI is an extension of the TLS protocol used by HTTPS protocol. When using SNI, the HTTPS client (i.e., browser) indicates to the web server the hostname (same as indicated in the HTTP host header) during the TLS handshake. This allows the server, if it acts as a hosting server (i.e., serving multiple domains) to know which domain exactly the client is interested in and it acts accordingly. For example, providing the correct TLS certificate to the client.
The use of a single web server to serve many different domains became popular not only due to hosting services but also due to adoption of cloud, Kubernetes, and CDN architectures which utilize a load balancer as a single entry point to many backend services. This simplifies the operation and enhances security by reducing the attack surface.
When the client does not support SNI, the server, at the TLS phase, cannot determine which domain is needed by the client and is forced to act with a default behavior. It will serve a single default certificate and use a default cipher template (choosing which ciphers are allowed). It can either enable or disable mTLS for all domains uniformly.
On the other hand, if SNI is supported by the client, the server can fit the TLS protocol to the site’s needs:
- Present a certificate with only this domain. This may simplify the renewal process of the certificate by reducing the dependencies between different domain owners.
- Use different TLS protocol configurations per domain. For example: Different domains may allow different ciphers in order to allow enhanced security when needed, or the domain may want to implement mTLS to enhance security.
This is why more and more servers started supporting “SNI-only” traffic by default.
SNI extension was added to the standard in June 2003, Since then, almost all modern browsers and HTTP client libraries support SNI by default.
To summarize, SNI is an essential tool in modern web applications. It allows hosting services and load balancers to be aware of the target domain and to treat it differently.
What Imperva Learned About Non-SNI Traffic
The Imperva Web Application Firewall (WAF) consists of thousands of proxies that see trillions of HTTP requests on a monthly basis. We decided to take a closer look at our data to understand the usage patterns of non-SNI traffic.
We discovered that out of the sites that have a considerable amount of traffic—1 RPS or more—only 1.2% of sites have more than 1% of non-SNI requests, and only 1.9% of sites have more than 1% of non-SNI sessions.
Simply put, this means that the overwhelming majority of sites have little to no non-SNI traffic. Even though the amount of non-SNI traffic is almost negligible, we decided to take a look and understand where these requests come from.
After some extensive data analysis, we’ve noticed the following patterns:
Almost 50% of the non-SNI traffic came from imposters (i.e., clients that use a bogus user-agent). This is common practice for bots trying to hide their malicious activity by pretending to be a browser or other seemingly innocent client. What’s even more interesting is that while almost 50% of the non-SNI traffic came from Python, virtually none of the Python clients tried to impose another client. In other words, there’s only a small percentage of clients that aren’t imposters and aren’t Python. Looking at the percentage of browsers reveals that the small percentage of clients that aren’t imposters and aren’t Python are probably the culprit.
Seeing how 90% of the traffic is coming from bots, this most likely means that the bots are probably old, legacy services that no one bothered updating because they “work”. One thing that strengthens this hypothesis is the user-agent used by some of these bots. For example, we can see that the Python version in use, 2.6, doesn’t have SNI support out of the box.
The only true mystery stemming from these statistics is the relatively high percentage of browsers that don’t support SNI. This is surprising because virtually all browsers have supported SNI for many years. Internet Explorer and Firefox, for example, have supported SNI since 2006. Chrome was a late bloomer, but it started supporting SNI in 2010.
One way to explain the phenomenon of 10% of non-SNI traffic coming from outdated browser versions, is that some users intentionally hold on to hardcore legacy versions of their favorite browser. Another, less fun option, is that this is traffic generated by some compatibility test. For example, the below screenshots were taken from a tool that is trying to run old browsers. Note: The probability for old browsers to support modern applications that run react / angular is low. On the left – explorer 7, on the right explorer 8.
The non-SNI traffic that comes from browsers is a neglectable portion of the traffic: Only 0.12%.
We also looked into the top 10 clients, responsible for 83% of the non-SNI traffic. We found that they’re split into two categories: Obscure/deprecated bots and tools—such as Echoping or old versions of Java and Python, that access only 1 or 2 sites—or browser impersonations.
Other than our Client Classification mechanism, another hint that shows these clients are impersonating browsers is that they declare the browser version in the User Agent, and the browsers they claim to be, do in fact support SNI. For example, we saw clients claim to be Chrome 81, Chrome 120, and Safari 537, while Chrome started supporting SNI from version 6 and Safari from version 3.1.
Should Website Owners Support non-SNI Traffic?
By now, we understand why SNI support is needed and that HTTPS clients that don’t support SNI are hard to find. They’re either malicious, in the worst case, or not interesting to the business wise, in the best case.
As a website owner you should ask yourself, “Do I want my site to support non-SNI traffic?”
The short answer is: If you have a good reason to do so, then yes. Otherwise NO!
Good reasons could include:
- Your website serves a known, very old HTTPS client that does not support SNI (e.g., probably a bot / service to service communication, not browsers). Look for a service-to-service interface where its client is implemented using Python. Check the version of the python.
- You’ve got specific requirements from your customers to support non-SNI.
- In all other cases we recommend that you move to SNI-only.
Due to the above reasoning, at Imperva, we now enforce “SNI-only” mode on all newly created sites. This default setting allow our customers to utilize more TLS related features like site level certificates, cipher selection and mTLS. For more information check out this blog.
Try Imperva for Free
Protect your business for 30 days on Imperva.
