The complexity of peering agreements and regional considerations can make it a challenge to know which peering provider to work with. In this post, we’ll take a deep dive into the world of peering and offer best-practice advice to network operations teams on what to look for when evaluating a peering provider. We’ll also share some innovative ways that Incapsula leverages peering to improve the level of service we provide to our enterprise customers as an example.
We’ve previously written about the importance of deploying our network points of presence (POPs) at strategic internet hubs, such as Frankfurt and London, to take advantage of peering agreements with Tier 1 providers, other internet service providers (ISPs), leading hosting providers and large major network entities.
This post goes deeper and gives you important nuts-and-bolts considerations for deciding when, where, and with whom to peer.
Let’s start with a quick primer on what peering is and how it differs from transit services.
Internet transit and the need for peering
The internet is a vast amalgamation of connected networks (a.k.a. autonomous systems or AS) belonging to Tier 1 carriers, ISPs, hosting providers, large corporations, universities and small businesses. For traffic to flow from a user on network A to a user on network B, it typically needs to traverse multiple ISPs before it reaches its destination.
These indirect, multiple-hop network connections, based on agreements between networks, are known as transit. The closer an ISP is to the internet backbone, the more direct network connections it has. Thus, smaller ISPs and content providers that serve consumers and end users typically purchase transit services from Tier 1 and other larger networks. The transit provider’s routers announce to other networks that they can carry traffic to the network that has bought transit.
Transit supports traffic from any origin server going to any destination on the internet. At the same time, since traffic may pass through any number of ISPs on the way to the destination, round-trip times for transit traffic can often result in relatively high latency (we’ll get back to this later).
A tale of two cities
Routing traffic across the internet can be thought of as a car traveling between two cities. The driver needs to negotiate many roads, traffic lights, junctions and highways before reaching her destination. But what if all of these obstacles could be removed and the driver could hop directly onto a six-lane toll road that directly connected the two cities?
This concept of cutting out the middleman is the driving force behind peering. In practice, there are two kinds of peering — public peering and private peering. Which one to choose depends on the amount of traffic involved, use case, and, of course, budget.
Public peering
Peering takes place on a large Ethernet switch (or set of switches) or route servers with hundreds of ports that provide Layer 2 (physical) connectivity between two networks. Unlike transit arrangements that offer global connectivity, peering is based on local connectivity within a specific region. The peering provider sets up shop at a regional internet exchange (IX) consisting of one or more data centers in a single facility or campus.
The peering provider lets its members share routing information with each other based on its respective peering policies. All traffic goes through the switch (usually a single hop), which results in minimal latency (several orders of magnitude faster than transit). Considering the fact that SSL handshakes require three or four round trips, this added latency is perceptible for end users.
To avoid this situation companies interested in peering can purchase a cross-connect to the peering provider in the relevant region. These connections are based on a fixed monthly price for a predefined level of traffic (for example, $900/month for 1 Gbps).
Many hosting providers, content delivery networks (CDNs) and Tier 1 providers, as well as other entities with huge amounts of traffic (Google, Facebook and Amazon) use peering to reduce round-trip times, improve bandwidth utilization and reduce transit costs.
Incapsula works with regional peering providers such as AMS-IX, DE-CIX, HKIX, and others to minimize latency. These providers sit on the network backbone and enable our PoPs to benefit from direct connections to other CDNs and Tier 1 carriers. Our customers, as a result, enjoy the highest levels of network performance and provide their end users with the best possible experience.
Private peering
It’s important to note that peering is a voluntary game. Each company has its own peering policy and business considerations, and cannot be “forced” to share all of its IP ranges with other members. Commonly, large ISPs with global networks are not interested in peering with smaller entities, since the larger ISP would end up using most of its bandwidth allocation to transport traffic for end users of smaller entities without getting compensated. By excluding smaller ISPs and CDNs, the whales can force the smaller fish to pay for transit rather than peering with them.
This is the rationale behind private, or direct, peering.
Unlike peering over a public internet exchange, private peering allows for a direct, private connection (over BGP) between two networks. Large companies (like AWS) will charge smaller companies to access more of their IP ranges over a private connection. SaaS, gaming and other companies are quite willing to pay extra for private peering with a larger network entity in order to improve the user experience for their customers in a particular region.
How to choose a peering provider
There are a number of considerations you need to take into account when choosing a peering provider:
- Who are the members and what is their peering policy?
You can use tools such as PeeringDB to view the policies of your prospective peering partners. Generally speaking, policies are defined as open, selective, or restrictive.
- Open means that members are likely to agree to your peering request without any preconditions.
- Entities with selective peering policies may ask you to meet a few prerequisites like minimum traffic volumes.
- Companies with restrictive policies are the least likely to peer due to the fact that they already have all the connectivity they need within the region.
- How many members does the provider have?
In the world of peering, more is always better. Beyond quantity, however, the most important thing to check is that existing members already connect with the networks and providers you need to connect to.
- How do you know when it’s time to peer?
Unless money is not a factor, this decision comes down to comparing the peering costs with those of buying transit services on your own for your traffic volumes in a given region.
To make an accurate comparison, you need a clear understanding of your traffic patterns. Gaining this visibility is far from being a trivial matter. Monitoring tools can help you understand which AS your traffic is coming from, from which ISPs, the bandwidth, number of requests, number of packets, type of traffic, and other attributes.
Given the complexity of traffic management and business significance of making the right decision, Incapsula has developed a decision support system specifically designed to support this type of network monitoring and analysis.
Our system analyzes traffic patterns and identifies where it is worthwhile to do peering, which ASes in a given region to peer with, and whether to go with public or private peering.
How Incapsula Infrastructure Protection leverages peering
Incapsula Infrastructure Protection safeguards critical network infrastructure from network level DDoS attacks, such as UDP, SMTP, or SYN floods, across a C-class subnet. In the event of an attack, traffic is rerouted through our scrubbing centers using BGP announcements. The solution uses a GRE tunnel to forward clean traffic to the origin server on the enterprise network.
Alternatively, depending on the peering provider’s policy, Incapsula can connect to the enterprise network via peering instead of using a GRE tunnel. In both always-on and on-demand deployments, this option offers customers the following advantages:
- Increases reliability and stability by reducing number of hops to origin server
- Avoids inherent GRE limitations with respect to packet size
- Minimizes latency via physical connection to peering provider
Conclusion
Our peering relationships with Tier 1 providers, ISPs and hosting providers reduce latency for our customers’ end users, and allow us to extend our coverage and improve our service levels in strategic regions.
As we expand our network, both geographically and in capacity, we’ll continue to look for new and creative ways to integrate peering within our product offerings.
We’d love to hear about your experiences with peering providers. Share your advice, tips, and lessons learned in the comments box below.
Try Imperva for Free
Protect your business for 30 days on Imperva.
