It’s a well-known fact that security solutions must quickly adapt to new attack methods. There are several ways to achieve this goal, regularly applying security patches and updates, relying on threat intelligence and more.

At Imperva, we use pattern anomaly detection as one of the tools to identify emerging threats and build new defenses. Our security researchers analyze the detected patterns from time to time, and this is how we learned about the existence of the Ash botnet.

The pattern that arose from our anomaly detection engine was http://xxx.xxx.xxx.xxx/a.sh. The pattern indicates a resource name, from multiple remote servers, that is being used as a payload in numerous attacks including the now infamous Drupalgeddon2 exploit
It’s not so often that we get to see the full life-cycle of a botnet, from creation to demise. When we do get the opportunity, however, it gives us a unique perspective on how botnets operate and how to stop them.

Detecting the botnet

By common payload

While investigating attempts to attack our customers — close to 1000 Imperva-protected sites were attacked but successfully guarded by the Imperva Incapsula WAF. It turned out that the payload was being delivered from a group of several hundred IPs belonging to a large group of sites protected by Incapsula, over a period spanning more than 30 days. The specificity and distinctiveness of this payload, as well as the fact that it was being delivered by the same hacking tool, strongly suggests that all these IPs were part of a single botnet.

Payload description

The initial a.sh file bash script is a simple downloader bash script. It downloads a compressed archive, extracts and runs one executable file named “i”. The archive contains 21 different files – text, bash script, executable, static libraries, and Python. “i” is a 64 bit ELF executable file which surprisingly isn’t detected by any of the virus detection engines in Virus Total.

The purpose of “i” is twofold: propagation of the malware and crypto-mining. The propagation is done through a central C&C server using both known vulnerabilities (Drupalgedon) and phishing emails. The crypto-mining part is achieved by running four (!) different types of crypto-mining software, xmrig, jce_cn_cpu_miner, xmr_stack, and luk-cpu.

Out of these miners, we were able to track two wallets that currently hold ~200 XMR which is the equivalent of 28K$:

• 48edfHu7V9Z84YzzMa6fUueoELZ9ZRXq9VetWzYGzKt52XU5xvqgzYnDK9URnRoJMk1j8nLwEVsaSWJ4fhdUyZijBGUicoD
• 45Pw3bWFJXQiR1hN97huM6BbNPpnYdPytWnTUbkEm6KS9MExU7Gtr3nBsVoZA746qhCEwqVsFuPdwfXVtZwXxHQ6LDyfBaL

By other common behavioral patterns

Identifying behavioral correlations over time between different attackers is a powerful way to single out botnets. A similar payload used by multiple IPs, as found here, is an easily detectable common behavior pattern; however, when the payload is not known or is not consistent or distinctive enough, more subtle behavior correlations can be used.

One such pattern is the sites visited by source IPs as a function of time. By defining a measure that reflects whether or not two different IPs visited similar sets of sites at the same time, one can cluster attacking IPs into groups representing potential botnets.

It turns out that such behavioral correlations are present here too, allowing us to calibrate the similarity measure used in our clustering method. The combination of both methods allowed us to successfully identify additional botnets in our data.

Botnet structure and life-cycle

Attack distribution

Interestingly, the attack was led by a single “master” IP, which on peak days was responsible for the majority of attacks on hundreds of target sites while the remaining multitude of “minion” IPs each performed a much smaller portion of the attack, typically attacking about a dozen sites per day. Another noteworthy feature was that the master IP changed four times during the analyzed 60-day period such that there was always exactly one IP taking this role at any given time, as shown in the figure below, where each colored curve represents a different IP.

The master IPs are likely to have belonged to a machine or machines owned by the attacker while the remaining IPs probably belonged to infected application servers that joined the attack in turn, making up the minion botnet.

Amusingly, the master IP active between May 8- 22 (red curve), suddenly became idle between the dates May 15-18,  the onset of the FIFA World Cup, which may imply that the attacker is a football fan, and in keeping with the hypothesis that the activity of the master IP is not initiated by a bot but by a human, being the actual attacker’s machine.

Creation

As you can see from the above graph, the peak of the attack, starting around the 20th of May, was preceded by a period in which only one IP was attacking – using the same payloads. The same sites that would later be attacked by the other IPs in the network. The hacking tool used by this IP was different from that used during the peak days of the attack.

The period during which only this IP was active in the network can be seen as one of reconnaissance activity by the attacker, who was probably looking for vulnerable points in the sites which the botnet later targeted.

Duration

Apart from the master IPs, each of which attacked multiple sites daily over long durations, most IPs dropped out of the botnet after a few days, with 998 of 2034 (close to 50%) being observed for up to two days:

There is more than one possible factor that may explain the brief appearance of most of the IPs:

• Fast detection and removal of the infection from many infected machines
• Sparing usage of the infected IP by the attacker to avoid their being detected

Volatility

The participation of IPs in the botnet’s activity had an interesting dynamic. The IP turnover rate in the botnet was relatively high: as shown in the graph below. On average, approximately 25% of IPs that were active (marked in orange) in the botnet on a given day were not active (marked in blue) on the next day, being replaced by a similar number of new IPs.

We can see that the overall activity of the Ash botnet dramatically declined between June 15-18.  We couldn’t find a technical explanation for this phenomenon. However, it is important to note that those were the opening days of the FIFA World Cup event. It is not unlikely that the botnet operators took a break from their nefarious activity to enjoy a few soccer matches 🙂   

Taken as a whole, the total activity of the botnet peaked between May 20- 28 and started declining thenceforth.

In the figure below we can see each attacked site marked in a different color and the number of IPs attacking it.

The victim’s perspective

If a site happened to be compromised, it would have started mining cryptocurrencies for the attacker. This can potentially lead to denial of service if the attacker utilizes all of the CPU power for mining. Additionally, a compromised server would continue to spread malware to more victims, potentially gaining a bad reputation for the server and the possibility of it being blacklisted by security vendors.

The many-to-many relation between attacked sites and attacking IPs of the Ash botnet leads to a commonly encountered situation where, from the attacked site’s perspective, it’s difficult to recognize that a high-volume attack is taking place, since each IP delivers a handful of requests per day to each site and then moves on to the next site.

The figure below shows the number of requests sent by a single attacking IP to a single application in a single day. As can be seen, in the vast majority of cases (84%), a single attacking IP performed a single request to a single site in the same day.

Thus, by simply decorrelating the attacking IPs from their targets, the attacker can avoid detection by the site based on volumetric measures per source IP.

Such an asymmetry between the complexity of the attacker’s and defender’s respective tasks is typical of the cybersecurity domain.

Mitigation

As mentioned in the previous section, unless the payload is blocked by existing security rules – which is not the case in zero-day attacks – the victim has a hard time detecting such a distributed attack campaign. A wider perspective encompassing multiple target sites – like that of a cloud-based security provider – is required.

From such a viewpoint, behavioral correlations between IPs across different target sites become evident, and it can serve to identify synchronized attacks such as those performed by the Ash botnet. Moreover, in subsequent attacks by the botnet, mitigation can be achieved by blocking requests from IPs belonging to the botnet whenever synchronic and correlated activity is detected among the botnet’s IPs. Additionally, we recommend that you:

1. Stay up to date with the latest security patches
2. Deploy a security solution that has a global view to detect and block such malicious activities that are not visible from a single application point of view

Note that up-front blocking of all requests from the IPs that participated in the attack would jeopardize legitimate activity originating from machines belonging to legitimate users which have been unknowingly infected. Hence, additional indications for blocking, such as detection of an automated tool, should be used to separate legitimate from malicious activity originating from the same IPs.