Navigating the Sea, Exploiting DigitalOcean APIs

Cloud service providers are now fundamental elements of internet infrastructure, granting organizations and individuals the ability to scale and efficiently store, manage, and process data. DigitalOcean is one such provider, well-regarded for its simplicity and developer-friendly platform, and often catering to small to medium-sized businesses and individual developers. With increasing reliance on these cloud platforms, the likelihood of exploitation or abuse of the services also grows, highlighting the essential role of security research and vulnerability management.

The plethora of services that cloud providers offer are often paired with sophisticated access management systems. Unfortunately, the complexity of these systems can unintentionally create opportunities for unauthorized access because of the increased attack surface, posing significant security risks.

I’ve come to know the ins and outs of the DigitalOcean platform from personal use, which, incidentally, played a key role in helping me identify a vulnerability in the platform recently. It’s often the case that really getting your hands dirty with the system uncovers the quirks that aren’t visible on the surface.

Discovering this vulnerability was not about using the platform for my usual needs, but rather, it was a deep dive into the nitty-gritty of its role-based access control. I wanted to see if any of those small, seemingly innocuous details could, in fact, be a gateway for compromising security. And sure enough, the exploration revealed several vulnerabilities associated with team functionalities and the permissions attributed to various roles.

DigitalOcean Teams

DigitalOcean uses a security best practice known as Role-Based Access Control (RBAC), which says that access permissions are granted to users based on their assigned roles within an organization. This policy aims to ensure that only authorized individuals can access specific resources relevant to their job functions. To find a vulnerability in RBAC, one must first understand the assumptions users have when utilizing this system.

This feature caught our attention. Tailored for project and business owners, it allows for delegation of server infrastructure tasks to collaborators, such as developers and financial administrators. Owners can assign one of three roles, offering a streamlined approach to access management. The simplicity of DigitalOcean’s three-role system not only ensures ease of use for its customers but also streamlined our vulnerability testing efforts.

Below is an explanation of the roles available to DigitalOcean users:

To assess the RBAC provided by DigitalOcean Teams, I conducted a test using my personal account and another newly created account. I invited additional users to join the team and set different roles to each one.

Broken Function Level Authorization: When Good Functions Go Bad

What is Broken Function Level Authorization?

Broken Function Level Authorization (BFLA) is a security flaw where users access functions beyond their permissions due to weak server-side controls. This not only exposes sensitive data but also is a type of business logic vulnerability. Unauthorized users can bypass intended workflows by exploiting this. Significantly, BFLA is ranked as the fifth most critical vulnerability in the OWASP API Security Top 10, emphasizing its importance in API security.

The Vulnerability

According to the features listed by DigitalOcean, users assigned the “Biller” role are intended to only have access to the team’s billing information and should not be able to access shared resources or modify team settings. However, my investigation revealed a discrepancy. Using the dashboard API endpoints `https://cloud.digitalocean.com/api/v1/*`, I discovered that a user with the “Biller” role had read access to almost all shared resources, such as SSH keys, Droplets, and Functions.

Additionally, I found that users with the “Biller” role could create, modify, and delete the team’s OAuth applications. This poses a potential security risk, as it could lead to the compromise of active OAuth application secrets or allow unauthorized modifications to sensitive properties, like the redirect URL.

Ghost Team Member

I also discovered an authorization bypass bug that allows a user to retain access to the majority of a team’s resources, even after they’re removed from the team.

I encountered this bug while using my older DigitalOcean account. It’s crucial to clarify that this vulnerability can only be exploited through older DigitalOcean accounts due to the way the Teams feature was initially established. Nonetheless, newer accounts remain vulnerable and were affected by this.

In the past, DigitalOcean did not have a Teams feature. I believe that when this feature was added, a default team was created, using the user’s ID and UUID as the team ID and UUID, respectively. I made this observation while reviewing the different requests made by the DigitalOcean dashboard, particularly looking at the `/api/v1/fleets` endpoint, which lists all the current teams the account is a member of. Comparing the response from my old account to the new account, I noticed that in my old account, the user ID, and UUID are the same as the team ID and UUID. Finding this peculiar, I decided to explore deeper to see whether this could be exploited.

While testing user removal from teams, I noticed that on my old account, the `/api/v1/fleets` endpoint still showed I was a member of a team I had been removed from. I couldn’t recreate this on my new account, which led me to believe this was due to the team I was removed from having the exact UUID and ID as my user IDs. However, when using the dashboard, I didn’t see the team I was removed from, and didn’t have access to their resources or information.

While exploring the DigitalOcean support portal, which utilizes Salesforce, I observed that when a user first visits the portal, they are authenticated by redirecting the browser to an OAuth URL to generate an access token, which is then used to retrieve their information:

The browser redirects several times, and in one of the redirects, we can clearly see an API token which matches the format DigitalOcean uses. I copied it and, to my surprise, this API token did have read access to most of the shared resources of the team I was no longer part of.

I wanted to see what would happen if I altered the scope from read to write, and as suspected, it worked. A DigitalOcean API token was generated, granting read and write permissions to the majority of resources in the team I was removed from.

Security History

DigitalOcean offers a security audit log detailing activity on shared resources, team settings, and configurations. We investigated its effectiveness in detecting our “Ghost Team Member” attack.

Currently, the activity log displays the fields: Action, User (full name), IP address, and Time. The use of only a user’s full name creates an opportunity for a spoofing attack, as an attacker can modify their account name to match the team owner’s, complicating detection.

We suggest the following improvements to DigitalOcean:

1. Enrich user details in the log to include email and or a unique identifier, like a user ID.
2. Add a “YOU” label next to activities performed by the logged-in user.
   

We believe making these changes could greatly assist in the detection of unauthorized access and activities.

Persistence

Another security concern arises from the combination of restrictions implemented by DigitalOcean. These restrictions prevent users with the “Biller” role from accessing various pages on the DigitalOcean dashboard, such as the personal API token management page. However, they still allow access to the dashboard backend functionality that manages these tokens, as well as the support portal, which, as demonstrated, can also generate access tokens.

If a user with the “Biller” role were to be compromised, malicious actors could exploit this to maintain unauthorized access, even after the account has been secured by its rightful owner.

DigitalOcean’s Response

Upon the discovery of these vulnerabilities, we immediately reached out to DigitalOcean with our findings. DigitalOcean responded promptly to our report, and quickly validated our findings on the identified vulnerabilities.

Acknowledging the discrepancies noted, especially in the “Biller” role functionality, they committed to updating their product documentation to reflect the actual capabilities and limitations of different roles more accurately. They initiated immediate corrective actions to address the identified vulnerabilities, particularly focusing on refining the role-based access controls.

Key Takeaways

1. Documenting RBAC to Set Expectations: When developing APIs, comprehensive documentation of Role-Based Access Control (RBAC) becomes crucial in setting the right expectations and behavior. This isn’t merely for compliance or reference; this documentation will serve as a guideline for all developers and administrators involved in the API project. Ensure that your RBAC documentation clearly outlines the roles, permissions, and access levels associated with each API endpoint, as well as how to configure them.
   
   Always be alert for any discrepancies between what your documentation promises and what your actual API implementation delivers. Such inconsistencies can indicate potentially serious security vulnerabilities, including the notorious and hard-to-detect BFLA vulnerabilities.
2. Adopt the Principle of Least Privilege: When setting up RBAC for your API, ensure that you are adhering to the principle of least privilege. Only assign and enable permissions that are necessary for each role to function effectively. This reduces the risk of unauthorized access and potential security vulnerabilities.
3. Enhanced Logging and Real-Time Monitoring: APIs are often targeted for various types of attacks. Implement detailed logging and real-time monitoring to catch any unauthorized or suspicious activities. Make sure to log access and usage of sensitive endpoints especially.
4. Use an API Security Product: Employing a specialized API security product can provide an additional layer of protection. For instance, Imperva API Security offers capabilities such as monitoring API traffic and identifying suspicious activity. It can help detect vulnerabilities like Broken Function Level Authorization. Using Imperva can also assist in real-time monitoring, helping you understand the data flow and assess whether activities are within the normal boundaries.
5. Have an Incident Response Plan: Just like any other system, your API could be susceptible to vulnerabilities. Prepare an incident response plan tailored for your API infrastructure to ensure quick remediation and minimal damage in case of security incidents.

Conclusion

We appreciate DigitalOcean for their responsiveness and cooperation in resolving the reported security issue. We believe in the significance of identifying and addressing vulnerabilities, and we are committed to ongoing collaborations with providers like DigitalOcean to make the online world safer for everyone.