International mobile roaming is a key feature of cellular networks, enabling mobile subscribers to seamlessly use cellular services worldwide. It supports international travellers, who can access data roaming at a limited cost (or no cost), often using applications with stringent Quality of Experience (QoE) requirements (for example, VoLTE, video streaming).
Mobile Network Operators (MNOs) infrastructure now offers the basic technological support for cellular Internet of Things (IoT) and boosts Machine-to-Machine (M2M) platforms as global connectivity providers. Major MNOs exploit international roaming to ensure worldwide connectivity to IoT providers, which ship their devices internationally (from wearables to cars and shipping containers) with prearranged cellular service (that is, a provisioned SIM card for the IoT device). Given the pervasiveness of these critical services, very little is known about the underlying roaming infrastructure.
Wanting to address this, my colleagues and I at Telefónica Research, Universidad Carlos III de Madrid and Northwestern University, recently analysed the central role that IP Packet Exchange (IPX) Providers (IPX-Ps) play in interconnecting all MNOs worldwide in a tight mesh that enables any device to have connectivity from anywhere it might be physically located. This was the first detailed analysis of operations in a real-world large IPX-P, with data collected directly from the IPX-P’s operational signalling and data roaming infrastructures.
The IPX ecosystem
Under the IPX model, MNOs interconnect with each other for roaming purposes by leveraging IPX-Ps. Specifically, with only one connection and one agreement, IPX-Ps offer their customers (for example, MNOs) interconnection for worldwide data roaming, and support for a variety of emerging applications, including IoT verticals, VoLTE and video streaming. No IPX-P on its own can provide connections on a global basis (for example, single-handedly interworking with all MNOs).
In an earlier study, we provided a tutorial-style description of the IPX ecosystem. Given the opaque nature of the IPX ecosystem, we cannot capture its characteristics from the public Internet by using public Internet routing data nor with active end-to-end measurements (for example, traceroute). Instead, we had to analyse a mix of private routing and peering data from an operational IPX-P, as well as market surveys and reports from third parties to build an exhaustive list of IPX-Ps active in the current ecosystem.
From this study we identified a set of 29 IPX-Ps that form the so-called IPX Network, interconnecting all IPX-Ps. This is an isolated network that bypasses the public Internet, ensuring global, secure, SLA-compliant services.
For more details on the characteristics of the IPX ecosystem, please read A First Look at the IP eXchange Ecosystem.
View from an operational IPX provider
The IPX-P we analysed during our recent study was a Tier-1 Internet Service Provider (ISP) operating one of the world’s largest backbone networks. As part of its interconnectivity products, the carrier operates an IPX infrastructure that runs on top of its vast Multiprotocol Label Switching (MPLS) network. The IPX-P infrastructure integrates more than 100 PoPs in 40+ economies with a particularly strong presence in America and Europe.
There remains a heavy reliance on 2G/3G technology
First, we studied the main services the IPX provides, namely, the signalling services for data roaming, and zoomed into the specific support for IoT customers. Our goal was to first establish which are the most popular solutions (that is, corresponding to the different Radio Access Technologies) the IPX-P offers, and what are the implications of the operational reality for the evolution of the ecosystem.
We observed that the 2G/3G signalling infrastructure is one order of magnitude more loaded than 4G. This heavy reliance on 2G/3G incurs high costs to operators in maintaining legacy radio networks and highlights a lack of consistency in deploying latest generation technologies worldwide.
Device mobility correlates with available IPX infrastructure and population migration trends
Our analysis shows that the extent of the IPX-P’s underlying infrastructure correlates with its operational breadth. Specifically, given that the IPX-P leverages access to important trans-oceanic infrastructure connecting the Americas and Europe (for example, Brusa subsea cable connecting Brazil and USA, Marea subsea cable connecting the US and Spain, or the SAm-1 subsea cable with various landing points from US to Argentina), we note that these are the main markets where it operates. Specifically, US, UK, MX and BR emerge as the main mobility hubs for devices that depend on this IPX-P to operate.
At the same time, the IPX-P (through the IPX Network) enables all customer devices to connect anywhere in the world, and, conversely, allow anyone in the world to connect to their customers’ networks. Overall, we found that the IPX-P’s infrastructure serves devices from MNOs from more than 220 (home) economies, operating in more than 210 (visited) economies.
In this context, we also found that the international mobility of mobile devices correlates with socio-economic migration trends. For example, we captured the effect of the migration between Venezuela and Colombia, with 71% of the subscribers from Venezuela travelling to Colombia during the period we captured. Inversely, we found that 56% of all Colombian outbound roamers we captured in our dataset travel to Venezuela. The Venezuela-Colombia border is one of the most active in the world, and Colombia is the primary destination for most Venezuelan migrants, which capture Venezuelans with different statuses ranging from economic migrants to refugees.
IoT load can strain networks
We also analysed the variety of device types that the IPX-P’s customer base integrates. Specifically, we captured the impact of IoT devices that benefit from the IPX-P’s global infrastructure.
We found that most operate as permanent roamers, and their long roaming sessions contribute significantly more load to the IPX-P system than smartphone devices. Moreover, synchronous traffic patterns from IoT devices with similar behaviour put a very high stress on the IPX-P platform, resulting in periodic high error rates and impact on the IPX-P performance.
The design of the IoT devices (which likely ignores the GSMA standards around flow sequences for registration, retries, and so forth) creates the synchronous pattern affecting the IPX Network. The large proportion of IoT devices within the IPX-P’s customer base also explains why the mobility restrictions economies imposed to tackle the COVID-19 emergency did not heavily affect the IPX-P customer base (≈10% drop in number of devices active, compared to ≈20% MNOs reported).
South America’s silent roamers
We further exposed and evaluated the emerging patterns of data communication that IPX-P’s end-users generate. For this, we focused our analysis on the dynamics of the data roaming service of the IPX-P.
We discovered a large fraction of silent roamers from the South America region, which is the direct result of the high costs for roaming services that is still on offer from operators in the region (in contrast, for instance, with Europe and its ‘Roam Like at Home’ regulation).
Interestingly, traffic patterns of silent roamers are similar to IoT devices and generate traffic on the signalling infrastructure but very little or no data traffic.
Juggling roaming and local requests
We finally tackled one of the most important aspect of the operational analysis, namely, the performance of the IPX-P platform while enabling the data roaming service and fulfilling its main functionality (that is, setting up and tearing down GPRS Tunnelling Protocol (GTP) tunnels for data communication in roaming). For this, we analysed the data roaming dataset to reveal statistics and capture patterns of how the IPX-P platform activates GTP tunnels for data roaming communication for its end-user requests.
The majority of the data roaming traffic is TCP or UDP used for the web (that is, HTTP/HTTPS) and DNS, respectively. In terms of performance, the quality of services strongly depends on the roaming configuration (that is, home routed or local breakout), and is impacted by the geographical location of the users, or by the applications/IoT verticals and remote servers.
We observed that the IPX-P takes full advantage of the flexibility if the IPX model, offering tailored solutions to its customers, to satisfy their requirements in terms of roaming configuration and quality of service.
To learn more about our study read our paper Insights from Operating an IP Exchange Provider, which we presented at the 2021 ACM SIGCOMM Conference.
Contributors: Diego Perino (Telefonica Research), Marcelo Bagnulo (Universidad Carlos III de Madrid), Fabián E. Bustamante (Northwestern University).
Andra Lutu is a Senior Researcher at Telefonica Research, in Madrid, Spain.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.