Having helped establish and manage a lot of Internet Exchange Points (IXPs) in Europe, Asia, the Middle East and even North America, I thought I would share some observations I have made over the years for those wanting to start an one.
Establishing the IXP
The hard part with establishing an IXP is not really the technical part, but building a community and trust.
Also, an IXP needs to be sustainable beyond the establishment phase. This includes having enough funds to operate and to upgrade equipment in the future as technology develops and traffic grows. Most IXPs ensure this by setting up a mutual or membership association so that the users of the IXP have equal say in the operation of the IXP. This is by far the most successful model of IXPs today.
The members pay a fee based on their port size to the association, so one fee for a 1GE port and one for a 10GE port. This is not based on the actual traffic, but the port speed. The membership all have one vote each in the running of the organization and normally appoints a board for the day-to-day operation of the IXP.
Secondly, new IXPs tend to believe it is a good idea to offer free ports to large content networks in order to attract other members.
At first glance, this might seem like a good idea, but you will quickly find yourself squeezed between the content networks and the broadband operators, which then also want their ports for free.
Generally, it is better to charge everyone equally and treat everyone equally. It might make it slightly harder for the IXP to get off the ground but it certainly makes running the IXP easier in the long run. Most of the large content owners are also quite supportive of new IXPs being formed in locations or regions where no IXP exists.
Thirdly, what is very important is to ensure that the IXP does not, in any way, interfere with the peering arrangements across the IXP. If the members want to, for example, sell transit to each other across the IXP, that is fine. If someone only wants to peer with a subset of the members, that is also fine.
Selecting the location for your IXP
Besides creating a community to support your IXP, the other important factor for establishing a successful IXP is its location.
Successful IXPs needs to be accessible for everyone, so it’s important to have a neutral data centre where any fibre infrastructure can be built in. These locations can be hard to find but are very critical for success. The wider the choice of fibre providers (or wlan for that matter in some parts of the world), and choice of carriers that are available, the better.
Data centres owned by current or former incumbents are not good choices as they generally tend to be very restrictive in what choice of infrastructure providers they offer.
Besides easy infrastructure access, the data centre also needs to provide a stable power feed, or you need to ensure you have UPSes and perhaps even generators (depending on ambition, the stability of the power grid, and if the data centre provides generator power).
Also, make sure there is room to grow the IXP. You might only start with two racks, but if you are successful you might need more, and moving the IXP is always painful.
What type of switch do I need?
There is an excellent presentation by Remco van Mook that covers many of the same topics as this post, and that also points out that ~50% of IXPs carry less than 20Gbps of traffic. As a new start-up IXP, you will most certainly carry less than 20Gbps.
What you need is a switch that will support 1GE and 10GE ports, preferably via SFPs, not an expensive and large switch with a massive amount of features (although the vendors will try and sell you one).
Instead, make sure you get a switch that you can manage and that you can monitor. These are the most important components of an IXP switch.
Co-location of services
For the benefit of the membership of your IXP, as well as the local Internet community, you should co-locate some critical shared resources with the IXP. The most important ones are:
- The local ccTLD slave service (that is, .uk, .fr, .sg, .bd and so forth). If the ccTLD registry is supportive of the IXP, then contact them and ask if they are willing to locate one of their unicast slaves behind your IXP router. Note: never connect the DNS slaves directly to the IXP L2 domain.
- It is likely that the ccTLD registry also uses an anycast provider for their slave service. In that case, it might be worth contacting that DNS anycast provider and ask them to connect to the IXP. You should, in any case, contact the DNS anycast providers as they are more than happy to peer at many of the IXPs. Many of these also operate or provide service to one of the DNS root servers. These are a critical part of the Internet infrastructure and having one co-located with your IXP will provide a lot of value to your local Internet community. PCH, Verisign and Netnod are some of the DNS anycast providers that locate nodes around the world.
- Route servers are also important to offer to your members, especially as you start to grow. The most stable software for route servers today is BIRD. Providing two route servers also adds redundancy, as if one fails the peering sessions will go down.
- Lastly, some IXPs offer NTP services as part of the IXP to their community. This is a very good service, and something that the community should make use of. However, for a new start-up IXP, NTP services are quite expensive to provide and definitely not needed at the first stage.
Managing my IXP and members
The guys over at INEX have created an amazing tool called The IXP Manager, which provides you with all the software you and your members need to manage your IXP. This is really an amazing resource that you should use for setting up the IXP!
How do I know if my IXP is a success?
What makes an IXP successful varies.
A small land-locked country where all the ISPs or Internet operators are forced to use satellite capacity for Internet access will benefit enormously from an IXP no matter how little traffic is exchanged, as each bit is a huge saving!
The value of peering is always in the eye of the beholder. Access to one single route or Internet prefix can be extremely valuable to one ISP, while another might need large traffic volumes before it creates value to them.
In the end, a successful IXP is defined by the membership. If there is a community that uses and continues to see benefit from the IXP, they will continue to support it. By being stable and sustainable, the IXP is successful!
Other resources
Below is a list of IXP associations in each of the five RIR regions:
Each of these associations provides lots of useful information for people wanting to set up and manage an IXP. The IXP Toolkit also has a useful collection of resources and the video below by Euro-IX is a great resource for explaining in a non-technical manner how an IXP works and what it does.
Finally, you are always welcome to contact me for questions and advice!
This post has been adapted from an original post by Kurtis on LinkedIn.
Kurt Erik “Kurtis” Lindqvist has 20 years’ experience in engineering, business development and training of ISPs and worldwide carrier networks. He has been the chairman of Euro-IX since 2003 and is currently the CMO at London Internet Exchange (LINX).
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.
Hi Kurtis, thank you so much for the informative post. I appreciate the time you spent writing this. I am in the process of developing an isp in a developing country and the only current ixp is politically compromised to say the least, what are the typical costs of setting up my own ixp for my own company? Thanks again!
Maybe the world need an open “IXP in a box” kit to help decentralize things so this kind of model can really take off outside of big cities:
http://www.cbc.ca/news/technology/wifi-nyc-mesh-new-york-city-1.4617106