ELB and Beyond: Introduction to Load Balancing Across Clouds

In a 24/7 world of websites and applications, downtime is simply not an option; uninterrupted availability has become absolutely imperative. Amazon’s Elastic Compute Cloud (EC2) service, along with its Elastic Load Balancing (ELB) offering, addresses this need by offering Amazon Web Services (AWS) users on-demand scalability and availability—as need dictates, additional servers are added to a virtual “data center” and entered into a load balancer’s destination list. This makes Amazon EC2/ELB a great solution for organizations having their entire environments residing within AWS.

Most enterprise-scale organizations, however, do not situate their entire network infrastructures within AWS, Rackspace—or any single cloud environment, for that matter. So, while Amazon ELB does a rather good job of load balancing within its native environment, it doesn’t work outside AWS. This can be a severe limitation in many common-use cases. This post addresses use cases where ELB is not sufficient and explores load balancing solutions for such scenarios.

Cloud Bursting and Hybrid Cloud Deployments

For security and risk mitigation reasons, enterprises frequently operate their own in-house data centers, while using the cloud to add extra capacity on demand. This method of deployment, known as cloud bursting, is used for a number of reasons, all typically having to do with planned surges in traffic—such as during a holiday season.

In other cases, organizations use a particular cloud as an interim step to an eventual migration of an application or environment to AWS. This provides the benefits of elasticity where needed, while still making use of existing infrastructure investment.

For any type of hybrid strategy, where some portion of the resource pool exists outside of Amazon, AWS load balancing options will not work since ELB doesn’t function beyond its own cloud boundary.

Cloud Failover

Another common hybrid cloud implementation is cloud failover—this couples a backup disaster recovery (or failover plan) with the benefit of cloud elasticity. When the primary data center becomes unavailable, cloud instances spin up; traffic gets routed to them in order to maintain application availability. The advantage of cloud failover is that it requires the organization to pay for a disaster recovery site only when it’s actually in use.

Cloud load balancers or DNS-based load balancing can both be used to route traffic in this manner, should it be necessary. However, ELB users should keep in mind that DNS-based load balancing has its own set of limitations. For example, DNS caching by a client or ISP can render it ineffectual.

Multi-Cloud Strategies

A multi-cloud strategy (e.g., AWS and Rackspace) can be a desirable implementation for a variety of reasons. One of the most commonly-cited reasons for using multiple clouds is to minimize risk of data loss or downtime due to component failure in a single cloud environment. Product familiarity, pricing, specific features available within each cloud, vendor commitments, or even uptime requirements—any of these factors can lead to the decision to use multiple clouds.

Again, due to ELB’s inability to work outside the Amazon environment, load balancing in this type of deployment has traditionally been done using DNS routing. In recent years, however, cloud load balancing solutions have become the preferred solution. This is because they provide more diverse and advanced algorithms, along with the ability to make dynamic routing decisions based on a more comprehensive set of criteria.

Load Balancing Across Clouds

In general, the ELB service is a great fit for many, but it is not a one-size-fits-all solution. If any of the use cases above matches your organization, then cloud-based load balancing is most likely your preferred option—provided it meets the following criteria:

1. Works Together with ELB

It should leverage ELB’s ability to load balance within a scaling AWS cloud. For example, by dynamically pulling a list of IPs by region from ELB, Incapsula’s load balancer evenly distributes the load between all of your servers, saving you from having to manually add/remove servers from a load balancer list.

2. HTTP/HTTPS–Based Load Balancing

It should deploy advanced distribution techniques, relying on real-time load assessment, instead of basic randomization. It should also actively monitor health and availability of your webserver and applications, doubling as a failover mechanism in time of need.

As an example, Incapsula’s load balancer uses a least-pending request algorithm instead of round-robin, allowing it to identify the most readily-available server and make it the target for the next incoming request—repeating the process while always routing to the least-congested machine.

3. Multiple Load Distribution Models

It should provide the ability to choose between several distribution methods, based on factors such as server load and connection times.

For load balancing within a single virtual private cloud, it should support:

• Least pending requests – The next request is routed to the origin server having the fewest number of pending HTTP requests.
• Least open connections – The next request is routed to the origin server having the fewest number of open TCP connections.
• Source IP hash – A hashing function persistently maps visitor IP addresses to one of the origin servers.

For load balancing between multiple virtual private clouds, it should offer:

• Best connection time – The most effective route is chosen, based on periodic response time sampling of servers.
• Geo-targeting – Traffic is routed to a specific data center based on each visitor’s geolocation. The option to redirect to another data center exists in case of failover.

Conclusion

While AWS is the dominant infrastructure-as-a-Service provider for a variety of organizations, many companies also have non-AWS deployments. ELB’s inability to load balance outside of AWS highlights the need for a cloud-based load balancer that works with ELB, while also load balancing across non-AWS environments.

A cloud-based load balancer can fit this need by providing organizations the scalability they expect from AWS, while also offering the flexibility to balance across platforms.