A recent Facebook vulnerability involving an account takeover hack made headlines when Anand Prakash was probing Facebook to see if he could find vulnerabilities. He discovered that the password reset mechanism on Facebook could be gamed.
According to his blog, Prakash found a disconnect between the production environment and testing (beta) environment of Facebook. First, he tried a “brute force attack on the Facebook 6-digit code on the live website and was blocked after 10 to 12 attempts.” However, when he went to one of the Facebook beta URLs, he discovered that the rate limit was missing. He managed to brute force his own account and reset his password. Additionally, he was able to “view messages, credit/debit cards stored under payment section, and personal photos.” Facebook awarded Prakash a bounty of $15,000 for the discovery and with that the story ends, or does it?
This story plays itself out again and again…
There’s the meltdown of source code hosting site Code Spaces, a company that was forced out of business by the actions of an attacker who managed to gain access to its Amazon EC2 control panel. The hack resulted in either the partial or complete deletion of Code Spaces data, backups, machine configurations, and offsite backups. Both the company’s and its customer’s data were almost completely wiped out and all Code Spaces could do was apologize to customers as it closed the business. This InfoWorld.com article informs us that Code Spaces was mostly built on Amazon Web Services (AWS), the dominant player in the infrastructure-as-a-service (IAAS) market.
Similarly, BrowserStack, a browser testing service also hosted on AWS, was hacked by an attacker who gained access to its customers’ email address database. These customers were targeted with a fake shutdown notice. The company was lucky enough to notice unusual database activity which alerted them to something suspicious, . This Imperva blogpostexplains that the hackers succeeded because BrowserStack failed to decommission old prototype servers. They also failed to update these servers with security patches, enabling the attacker to gain a remote shell using the Shellshock vulnerability.
The post goes on to note that BrowserStack made another mistake. They left an AWS access key on the file system of the old server. Command line tools use AWS access keys when making API calls to enable cloud automation, but storing them locally is a dangerous practice which Amazon strongly discourages.
As we can see with Code Spaces and BrowserStack, this latest Facebook incident is just another canary-in-the-coal mine of risks created by publicly exposed development and test environments.
Why does this keep happening?
It’s important to understand the risks when working in cloud environments and why mistakes like those mentioned above keep happening. The way features and products are rolled out to the cloud creates a large number of potential vulnerabilities. Typically, there are separate environments for development teams who code the product, QA (testing) teams who test the product, and the production environment itself.
To facilitate proper development and testing, these non-production environments need to accurately reflect production and so they often have the full range of supporting services, including the various components that make the application work such as databases. In the production environment these components are maintained by operational staff. While in R&D and QA, it is often developers and other non-operational staff who maintain them.
It is challenging for R&D and QA staff to put the same effort into maintaining and securing their systems as is done in production, because their priorities are different. Their primary goal is to produce useful features on time and on budget, not to act as server admins. Their systems aren’t necessarily tested end-to-end, and usually aren’t managed to the same standards as production servers. This can result in Devops environments missing critical security patches, using default or common passwords, lacking multi-factor authentication, and having lower security standards in general.
The risk is created when these environments are exposed to the Internet either accidentally or (more likely) deliberately in order to facilitate development and testing. This risk is prevalent in cloud scenarios such as AWS or Azure because development and testing is usually conducted in the cloud as well.
There’s no shortage of potential pitfalls to watch out for when working with cloud services including:
- A vulnerable admin interface exposed to the Internet, potentially giving a hacker a backdoor with a little effort and luck
- Unpatched, vulnerable server-side components
- Ridiculously weak admin passwords
- Hard coded AWS keys left on a local drive, storing them locally is a dangerous practice which Amazon strongly discourages
- Large amounts of highly sensitive source code, configuration files, SSL private keys, and customer data in S3 buckets with no compartmentalization
What’s the answer?
It’s pretty straightforward actually. If the risk is created because R&D environments have the same application components and data as production systems, then to mitigate the risk they also should have the same security controls and be maintained according to the same standards.
The biggest problem is that many traditional security controls lack the flexibility and agility to allow them to integrate into a Devops workflow. Here at Imperva we have been working with Devops customers for several years to enable our products to support, rather than suppress, an agile development life cycle with frequent, impromptu, repeatable release processes.
Both our Imperva Incapsula cloud-based application delivery service and Imperva SecureSphere for AWS appliances can be driven by REST API calls, have numerous input and output integration points, support rapid scale up/scale down, and are being used by customers to prevent the sort of attacks described above—even in highly dynamic Devops environments.
It may seem like a lot of effort, and we know most organizations prioritize their features and their sales over locking down their R&D environments, but we only need to look at Code Spaces to see the real price for not understanding the risk and taking the required precautions. Maintaining and working in cloud environments like AWS requires additional monitoring, security mechanisms, and maintenance.
Try Imperva for Free
Protect your business for 30 days on Imperva.
