Authors: Shyam Krishna Khadka (University of Twente), Suzan Bayhan (University of Twente), Ralph Holz (University of Twente and University of Münster), and Cristian Hesselman (SIDN Labs and University of Twente)
Critical Infrastructures (CIs) such as banks and telecom operators increasingly rely on cloud-based email services from Microsoft or Google. However, CIs often have limited insight into the security status of the paths their traffic might follow across the Internet to reach these cloud providers, which we consider a supply chain risk. We therefore developed a generic method that finds plausible Internet paths to clouds and other endpoints and identifies to what extent the Autonomous Systems (ASes) on a path support Route Origin Validation (ROV). We used our method for a case study to find secure paths from four CIs in the Netherlands to Microsoft’s cloud-based email service.
This blog post is a summary of our paper at the Applied Networking Research Workshop 2024.
Cloud-based email services are becoming more dominant
An article published in March 2024 in NOS (a Netherlands mainstream news organization), reports that 63% of the approximately 3,800 largest companies in the Netherlands rely on Microsoft or Google for cloud-based email services. CIs are no exception and many CIs such as ING Bank, KPN (a big telecom operator in the Netherlands), and Schiphol Airport (the main international airport of the Netherlands) rely on cloud services (such as email) for their daily operations.
The problem: Limited insight into the security of paths towards clouds
At the same time, CIs typically have limited insight into the security status of the paths that their traffic might follow across the Internet to reach the email infrastructure of their cloud provider. For example, a CI might not know their traffic passes through Autonomous Systems (ASes) that do not implement Route Origin Validation (ROV). As a result, the CI’s traffic is vulnerable to prefix hijacks, which can render the cloud operator unavailable to the CI or breach the confidentiality and integrity of the CI’s data. Even if a single AS on a path does not deploy ROV, it can remain vulnerable to BGP hijacks, known as ‘collateral damage’.
We think of routing hijacks as a supply chain risk for CIs. The risk is real because BGP prefix hijacks are common incidents on the Internet and are well-known to have been used for malicious purposes, for instance, to attack payment systems, steal cryptocurrency, disrupt traffic, and create Distributed Denial-of-Service (DDoS) attacks.
Our question, therefore, is what it takes to enable CIs to get more insight into the security status of each AS on a path towards their cloud provider or other destination.
Our approach: Calculate the ROV score of valid Internet paths
We combine BGP routing data collected from public route collectors (RouteViews and RIPE RIS) with ROV scores to assess path security. Using Microsoft as our example cloud-based email service, we go through the following four steps to make such assessments.
1. Path collecting
For a CI’s AS, we look for all its BGP announcements while for Microsoft’s AS we look for BGP announcements from their email service, which has prefix 52.96.0.0/12, originated by Microsoft’s AS (AS8075). In this way, we have two types of paths as seen from the route collectors — from the CI’s AS to the route collectors and from Microsoft’s AS to the route collectors.
2. Path stitching
We create an undirected graph from the two types of paths, in which the nodes are AS Numbers (ASNs) and tuples of ASes on the AS-PATH from the edges. The common node between the two paths is the AS that dumps routing data to the route collectors, which is the stitching point for us to join the two paths.
3. Path sanitizing
We check if a stitched path is valid, meaning the prefix of a source AS (CI) can reach the destination AS (cloud-based email service) and vice versa. We accomplish this through Gao Rexford’s model for route export and valley-free conditions. We first obtain the relationship between ASes on stitched paths using CAIDA AS rank: Customer to Provider (C2P), Provider to Customer (P2C), and Peer to Peer P2P). Next, we select stitched paths that meet the route export condition — routes learned from customers are exported to any providers or peers, routes learned from providers are exported only to customers and routes learned from peers are exported only to customers. Finally, we filter out stitched paths that meet the valley-free condition — at most one P2P link exists in the path, the P2C link must not be followed by a C2P or P2P link and the P2P link must not be followed by a C2P link.
4. Security scoring
We use the ROV scores of ASes on valid paths from the ROVista API to assess the security of an Internet path. ROVista determines the score based on the number of RPKI-invalid prefixes reachable by an AS. The ROV score varies from 0% to 100%, where 0% means no filtering of RPKI- invalid prefixes.
Case study in the Netherlands
We use our method of calculating the ROV score of paths for four CIs in the Netherlands (ING Bank, water supplier Vitens, Eneco Energy, and ABN-AMRO Bank), with their destination as Microsoft email service. Figure 1 shows the number of valid paths for each CI and their security statuses (in terms of ROV scores).
As seen in Figure 1a, AS15625 (ING Bank) has the highest number of paths, which might be because it is a multi-homed AS with four upstream providers, and more providers means more ways that an AS gets BGP routes. In total, there are 40 ROV-unprotected paths (0% ROV in Figure 1a) and 20 fully ROV-protected paths (100% ROV) for path lengths of one, two, and three AS hops.
For the case of AS56517 (Figure 1b), there exists a total of 10 ROV-unprotected paths (0% ROV) and 3 fully ROV-protected paths (100%), again summed up for path lengths of one, two, and three AS hops. This shows that there is quite a significant number of paths that are prone to BGP prefix hijacking, which is a risk for the CI’s email traffic as it might take these less ROV-secure paths toward their mail server at Microsoft.
Figure 1 also illustrates that the number of additional ROV-protected paths increases significantly if only a few upstreams implement ROV. For example, if AS40985’s upstream would implement ROV, then that would result in all its 14 valid paths towards Microsoft (one, two, or three AS hops) becoming fully ROV-protected instead of 0, as shown in Figure 1c. Similarly, if AS15916’s upstream provider implements 100% ROV rather than around 62%, it will give ABN-AMRO Bank nine fully ROV-protected paths out of 20 valid paths. If these upstreams do not implement ROV, then there are no 100% ROV-protected paths from AS40985 and AS15916 toward Microsoft’s email infrastructure.
ASes’ coalition to forward CI’s traffic
Table 1 shows the number of unique ASes on valid paths to and from the four CIs that have a 100% ROV score. As an example, there exists a total of 85 different valid paths between AS15625 and Microsoft’s AS, with 15 unique ASes on these paths having an ROV score of 100%. If these 15 ASes would form a coalition (for example, a trust zone), they would have more fully ROV-protected paths to forward traffic among each other. However, the actual number of such 100% ROV-protected paths will depend on traffic forwarding agreements that the members of the coalition will set up.
In the future, we envision that ASes could also offer such concepts as a value-added service to customers who need more insight into the (ROV-based) security of ASes that handle their traffic (for example, through visualizations) and control over such paths.
| CI’s AS Number | Number of unique ASes | Number of valid paths |
| 15625 | 15 | 85 |
| 56517 | 10 | 13 |
| 40985 | 12 | 14 |
| 15916 | 13 | 15 |
Summary and future work
Although we used our method to calculate the ROV scores of Internet paths from CIs to Microsoft’s mail service in the Netherlands, it can be used to assess the ROV security of paths from any source AS to any destination AS with or without a particular destination IP prefix. Our case study shows that multiple paths are 100% ROV-protected and multiple others with less or even without ROV protection, which might introduce a supply chain security risk for CI operators. However, our analysis also reveals that implementing ROV fully by upstream providers of CIs will increase the number of fully ROV-protected paths toward the Microsoft mail service by 72.5% on average.
Our future work includes calculating a path’s security status based on security metrics other than ROV (such as the availability of DDoS protection), which AS operators could consider when making inter-domain routing decisions. We publicly provide our analysis code on GitHub.
This work was conducted as part of the CATRIN project, which received funding from the Dutch Research Council (NWO).
Shyam Krishna Khadka is a PhD student at the University of Twente, with interests in Internet routing security and Internet measurements. He has over a decade of working experience in software development and network technologies across various companies.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.