The Myanmar Internet Exchange (MMIX) was established in September 2017 in Yangon City, Myanmar. Following its successful operation and increasing interest from various Internet Service Providers (ISPs), we made the strategic decision to establish another Internet Exchange Point (IXP) in Mandalay, the second-largest city in Myanmar. In June 2023, we successfully launched the Mandalay Internet Exchange Point (IXP), further expanding our network infrastructure and fostering connectivity in the region.
Setting up
In conceptualizing the new IXP in Mandalay as an independent entity, we needed to acquire all new resources. This included acquiring a public Autonomous System Number (ASN) for route servers and securing public IP addresses for the peering LAN and local content. Recognizing the absence of compelling content in Mandalay, we engaged potential Content Distribution Networks (CDNs) and other content providers. To enhance the offerings, we also made the strategic decision to extend some content from the established Yangon IXP to the newly launched Mandalay IXP.
The most important factor in this decision was the cost of domestic long-distance carriage. For Mandalay users, the whole cost must not be more than the existing local IP transit cost. To achieve this, we forged a partnership with a long-distance provider, negotiating a fair and competitive price that allowed us to advance the project while maintaining cost-effectiveness for Mandalay users.
Selecting a data centre (DC) became a fundamental factor in our decision-making process. Unlike Yangon, Mandalay City lacked a dedicated service location in its central area. Our priority was to identify a location that could seamlessly integrate with existing networks and also appeal to potential new connections. The absence of a suitable DC in the desired location led to a delay of over a year to the project before a new facility was set up at an ideal location.
Meeting the demand for necessary equipment was another crucial aspect of this expansion. Similar to MMIX Yangon, we purchased the required peering switches. We applied for two servers through the collaborative program involving APIX, APNIC, APNIC Foundation, and Internet Society for the route server services and other management applications. Our application was successful, and we were selected for the award. Subsequently, one of the awarded servers was deployed in Mandalay to support the infrastructure of our newly established IXP.
Getting technical
From a technical standpoint, our approach involves the implementation of VXLAN over EVPN and BGP technology to extend the peering LAN seamlessly from one city to another. Given budget constraints, we opted for a resource-efficient strategy by utilizing a single device for multiple functions. This singular device serves both as the Layer 2 infrastructure for the Peering LAN network and as a Layer 3 transit gateway, facilitating connections to CDN boxes and other management networks. To maximize efficiency, we employ multiple virtual instances within this single device.
Our VXLAN network configuration isn’t as simple as other setups, extending beyond a typical Layer 2 network. It encompasses Layer 3 functionalities, including route import and export between virtual routers. This configuration required a substantial investment of time in learning, testing, and implementing that wouldn’t have been possible without the advice and support from friends within the community contributing to the technical development and refinement of our network infrastructure.
Technical challenges and solutions
Meeting our requirements posed several technical challenges. Initially, we used a single ASN for the Yangon route servers and the Yangon Internet Gateway (IGW) router, with these devices running iBGP between them. However, when extending VXLAN via EVPN BGP to Mandalay, the limitations of iBGP became apparent with a single ASN.
To address this, we changed the ASN of the IGW router. This transition impacted multiple content providers (downstream), IP transit providers (upstream), as well as their parent and grandparent upstreams. Negotiating this change with all parties involved was crucial. The migration process took longer than a month to organize.
To prepare content for Mandalay users, we requested dedicated nodes for Mandalay from existing content providers. Recognizing that some providers would face limitations in providing dedicated nodes, we proposed they connect to both the Yangon and Mandalay peering LANs, thereby serving both networks. To meet that requirement, we extended the Mandalay Peering LAN to Yangon. This extension allowed content nodes located in the Yangon Point of Presence (POP) to seamlessly connect to both the Yangon and Mandalay peering LANs, ensuring efficient and shared access for network operators in both cities.
These efforts resulted in Mandalay users now able to enjoy the same content as Yangon, though with different pricing to accommodate long-distance costs. MMIX communicated these details transparently to members.
Simultaneously with the establishment of the Mandalay IXP, MMIX expanded its services to Naypyitaw. Despite being the capital of Myanmar, Naypyitaw had just a few potential ISPs, with only one joining MMIX. To work around this situation, MMIX set up an extended POP of the Yangon IXP in Naypyitaw. This involved carrying all content from Yangon to Naypyitaw.
In Mandalay, where a significant portion of the content is available locally, prices are only slightly higher than in Yangon. However, in Naypyitaw, where all content is carried from Yangon, prices are comparatively higher. Despite this, ISPs in Naypyitaw are willing to connect to MMIX due to the reliability and cost-effectiveness offered by our services.
Lessons learned
Based on our experience, prioritizing a well-defined plan is crucial for the success of any business. In the context of peering, effective negotiations with potential partners are key. To ensure the success of a new IXP, fees must facilitate future growth.
Technically, it is advisable to use a distinct ASN for your router, separate from your route server, especially if you’re delivering content to peer members. In our case, some CDNs operate under our ASN and treat our router as a peering member, similar to other network operators, which has proven beneficial. While using a single ASN may seem suitable initially, it’s important to be aware that this approach could lead to challenges down the line, as we experienced.
Perhaps the most crucial requirement is to thoroughly comprehend the needs of potential members and maintain communication with all relevant communities. This engagement helps garner support and encourages contributions, fostering the overall development of the IXP.
Thein Myint Khine is the founder and CEO of Myanmar Internet Exchange (MMIX). She is an Industrial Advisor with over 20 years of professional experience, 15 of those in the ISP environment.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.