In July 2015, Akamai became the first Content Delivery Network (CDN) to pass traffic at 100 Internet Exchange (IX) locations; an accomplishment that took 17 years. As of writing, we’re connected to 116 IXes of all sizes, ranging from n x 100G connections down to a single gigabit.
Over the years, being in so many places has helped us learn more about what IXes have in common, why they’re valuable, and how to help them grow. Here’s some of what we’ve learned along the way.
Why build an IX?
One of the biggest things an IX does is promote growth in an area where companies would have lower costs with more cooperation. Typically, most small end-user networks don’t need to talk with each other; consumers of data want to talk with sources of data, not other consumers. However, this all changes when there’s a source of content that everyone wants to talk to.
Much like a very large store might ‘anchor’ a shopping centre, a ‘content anchor’ like a CDN can provide a nucleus around which a group of small end-user networks can then form an IX. By allowing cheaper, direct exchange of traffic, the networks can lower their costs, attract more content networks to the region, and allow regional networks to expand their own businesses. On top of all that, they get high-speed, low-latency access to content, which would normally go over expensive transit links.
Watch Kam-Sze Yeung present on Akamai’s path to 100+ IXes at APRICOT 2016
Who has the biggest IX?
This is a question I get asked a lot and in many ways is a question of significance rather than size. There are a few ways to measure it, but to my mind the most important one is how many networks the IX brings together. The amount of bandwidth exchanged over an IX can sound very impressive, but it’s the amount of networks the IX connects which is a more accurate measure of success. More networks = more opportunities to directly exchange traffic = lower costs for everyone.
Should I peer over an IX or PNI?
When you already have a thriving IX, there’s the question of “Should I peer over the IX with a network, or should I arrange for a Private Network Interconnect (PNI) to them?”
An IX allows for quick access to numerous networks, and turning up a BGP session on an existing link is relatively cheap and fast. On the other hand, while an IX fabric can be very high bandwidth, IX members’ links will be comparatively small. It’s not hard to end up in a situation where a large ISP sends all of their traffic over an IX and overwhelms the smaller members, causing them to do traffic engineering, or having to take the expensive step of receiving the large ISP’s traffic over transit just to keep their IX link open for other members.
A PNI allows for dedicated bandwidth, but router ports cost money, and so do cross-connects. In addition, a PNI typically only goes to a single router, and there will be significant capacity shifts in the event of maintenance. In general, there will be a quantity of bandwidth which represents the break-even point.
What’s in it for a content provider?
What does a CDN (such as Akamai) get out of being a member of an IX? Cost reduction is a very large part of the equation. Clearly, serving a large amount of networks without having to haul bits across transit to reach them yields significant cost savings. In addition, the IX members save on transit and get significantly better performance in terms of latency, loss, and overall throughput.
On top of all that, the CDN gets better geographical coverage and capacity in a very efficient way. Most of all, though, they help grow the provider ecosystem in the area. As traffic grows in an area, more network providers are interested in providing service, which increases competition, improving connectivity, service, and prices.
What are the pros and cons of using route servers?
Many IXes have route servers, and they definitely have pros and cons.
On the pro side, they’re a very quick way to jumpstart entry into an IX. With one or two BGP sessions, you can see everyone’s routes, which allows for easy participation, as well as a snapshot of everyone’s routes for future capacity planning.
The downside is that your entire participation on the IX now rests on a single piece of equipment which could go down for maintenance. There are ways to mitigate that problem, including multiple route servers with different hardware. Multiple route servers can also be created through virtualization, but that adds complexity.
One general trend we have observed with large content providers is a move away from route-servers unless absolutely necessary. Instead, they will prefer to directly establish bilateral BGP peering sessions with each participant, because it gives more control over how each individual IX member is served content.
Cooperating to become something greater
The Internet is proof of how people can act in their own best interest through cooperation. IXes are proof of how networks can cooperate at a local level to become something greater than the sum of their parts. Akamai is actively exchanging traffic at 116 IXes, and we look forward to working with the Internet community to show up at more of them.
We have an open peering policy: if you’re at an IX with us, we will peer with you. Email us at peering@akamai.com
Matt Ringel is Senior Network Architect at Akamai Technologies.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.
Excellent and valuable information .