Beware of Crooks Wearing a MSIE 6.0 T-Shirt

The browser vendors are at war to win the precious real estate on our PCs called ‘My Default Browser’. It started in the 90’s when Netscape and Microsoft went head to head for market domination and it is at an all-time high today with Google providing a browser based OS, Microsoft moving Office to the cloud and Apple blocking Flash from running on Safari.

One of the byproducts of this war is the Browser Usage Statistics sites. Sites like w3schools, NetMarketshare and StatCounter show usage statistics for the popular browsers. Usually, there is no magic in the way these statistics are collected; you can easily do it yourself by extracting the browser type and version from the User-Agent HTTP header. An exception is Google Analytics and other JavaScript based services that collects this information using their JavaScript API, however, as far as I know, Google does not publish global browser statistics and you can only get these specifically for your site.

At Incapsula we do not rely on the User-Agent header in our classification, mainly because it can be easily faked. Instead we developed mechanisms that identify other, more subtle, attributes that are harder to fake. We use our engine to detect all sorts of clients like Spam bots, vulnerability scanners, SQL Injection worms and also browsers; real browsers.

The truth about MSIE 6.0

Below is a comparison that shows that while Incapsula’s customer websites are similar to the general website population (using Naïve classification), the actual numbers are very different.

To see why these numbers are misleading let’s now look at the version break down for a specific site. Furthermore, let’s put the naïve method to the test against Incapsula’s classification engine.

The most noticeable difference is for MSIE 6.0 statistics. While the logs might indicate a significant amount of V6 usage, the actual number is much smaller. The reason is that bots love to be MSIE 6.0. Maybe it’s because nobody else likes too or maybe because web developers expect it to do weird things and let it get away with a bad bots behavior.