The Border Gateway Protocol (BGP) is the mechanism to construct routing tables across the Internet. Unfortunately, the original BGP protocol lacked many security features, making BGP vulnerable to many attacks such as prefix hijacking and route leaks.
To address these security limitations, Resource Public Key Infrastructure (RPKI) was developed and deployed by Regional Internet Registries (RIRs) in 2011. Although controversial in its early years, with issues surrounding BGP announcements being marked as invalid, its popularity has increased significantly in the last few years.
In this post, I’m going to share results from a comprehensive joint study that I presented at IMC 2019 into the historical and current status of RPKI deployment and the quality of BGP announcements to see if RPKI has overcome its early teething issues and is ready for the big screen.
Key points:
- RPKI deployment varies significantly between RIRs: between 1.38% (ARIN) and 15.11% (RIPE NCC) of ASes are included in one or more Validated ROA Payloads (VRPs), and between 2.7% (AFRINIC) and 30.6% (RIPE NCC) of the total IPv4 address space administered by RIRs is covered by VRPs.
- Between 9.98% and 11.28% of BGP announcements are covered by VRPs in the study.
- There has been a significant reduction in the number of BGP announcements marked as invalid; more than 94.3% of announcements covered by ROAs are valid.
- Outreach by RIRs and IXPs has helped with this reduction.
- Many unauthorized BGP announcements are not necessarily suspicious attempts.
What is RPKI?
In a nutshell, RPKI uses hierarchical Public Key Infrastructure (PKI) which binds Internet number resources (INRs), such as IP addresses, to a public key via certificates. The corresponding private keys are used to make attestations for these INRs. For example, a resource owner can generate objects called Route Origin Authorizations (ROAs), which authorize an AS to announce certain IP prefixes.
These ROAs are signed by their certificates, which are ultimately signed by one of the RPKI trust anchors (equivalent to a root certificate in other PKIs) managed by the RIRs and published in so-called RPKI repositories (Figure 1).
RPKI validating software can retrieve all RPKI objects from the repositories and performs cryptographic validation of the content to produce a set of valid ROAs. Validating routers can simply use these ROAs to verify incoming BGP announcements.
Learn — How to: Creating RPKI ROAs in MyAPNIC
Learn — How to: Installing an RPKI validator
Considering that RPKI is still young and in its early phase compared to other PKI protocols such as DNSSEC and HTTPS, we hoped to understand the current status of RPKI and the challenge for its greater adoption.
What is exactly inside a ROA and how do they help validate BGP announcements?
First, an Autonomous System (AS) can put its IP prefixes in a ROA that it wants to announce through the BGP. For example, AS 4385 (Rochester Institute of Technology), which is the holder of 129.21.0.0/16, can create a ROA containing a list of its IP prefixes 129.21.0.0/20, 129.21.16.0/20, … 129.21.240.0/20 to let other routers drop the BGP announcements originated from ASes other than AS 4385.
If the AS wants to announce other prefixes such as a /17 or a /18 for traffic engineering purposes, the AS must update its ROA to reflect the IP prefixes that it is going to announce.
Alternatively, the AS can use the MaxLength attribute in the ROA, which specifies the longest prefix length for the authorized IP prefix that the AS may announce.
Continuing with the example, the AS could instead publish a single ROA containing 129.21.0.0/16-20 where 20 is the MaxLength attribute, which authorizes AS 4385 to announce any of the sub-prefixes of 129.21.0.0/16 in CIDR blocks of length between a /16 and a /20.
Now, let’s see how routers use this information to do Route Origin Validation (ROV). In fact, it is not routers that fetch all repositories and do cryptographic verification. Routers use so-called Relying Party (RP) software in order to download and validate RPKI objects.
From all of the ROAs, RP software first cryptographically verifies RPKI objects and extracts a set of tuples (ASN, ROA prefix, prefix length, MaxLength), which are called Validated ROA Payloads (VRPs). Validating routers can fetch them using RP protocol to validate BGP announcements.
When such a router receives a BGP announcement, it attempts to validate the announcement using VRPs. First, it determines if the announced IP prefix is covered by any VRP (IP prefix address and the VRP IP prefix address are identical for all bits specified by the VRP IP prefix length).
If it finds at least one VRP that: (1) covers the announcement’s IP prefix; (2) has the identical ASN; and (3) its length of the announcement’s prefix is no greater than the MaxLength in the VRP, then the BGP announcement is considered to ‘match’ a VRP.
Hence, each BGP announcement can have three RPKI validity states:
- Valid: the BGP announcement is matched by a VRP.
- Invalid: the IP prefix in the BGP announcement is covered by a VRP, but no VRP matches the announcement.
- Unknown: the IP prefix in the BGP announcement is not covered by any VRP.
They can use these states to make a decision on a given BGP announcement. As you can tell, routers do not need to do any cryptographic verification to perform this analysis. The validation process is done purely on the basis of VRPs obtained from RP software; RPKI has been expected to be supported and deployed by many routers.
So, how has RPKI been deployed?
In order to understand the current status of RPKI, we need to consider two angles:
- How network operators have published their ROAs to protect their network resources, and
- How much of the actual BGP announcements have been covered by ROAs?
How has RPKI been used by network resource holders?
As mentioned, each of the five RIRs maintain a repository with RPKI data. Thanks to the RIPE NCC, which has maintained a daily archive of the five repositories since its adoption, we are able to measure how RPKI has been adopted.
Looking at Figure 2, we can see an increasing trend in all three graphs indicating a significant and increasing adoption of RPKI both in terms of the number of ASes that have VRPs and the fraction of IP space covered by a VRP.
We also observe that overall RPKI deployment varies significantly between RIRs: between 1.38% (ARIN) and 15.11% (RIPE NCC) of ASes are included in one or more VRPs in our latest snapshot, and between 2.7% (AFRINIC) and 30.6% (RIPE NCC) of the total IPv4 address space administered by RIRs is covered by VRPs.
Interestingly, a few registries have a rapidly growing RPKI coverage. For example, the fraction of the total IP space covered by VRPs rose from 19.2% to 30.6% between 1 January 2017 and 27 February 27, 2019.
These trends are very encouraging, especially when taking into account the results of one study conducted in 2016, which reported that 84% of network practitioners were not interested in deploying RPKI at all.
What percentage of BGP announcements are actually covered by RPKI?
Validating routers use VRPs to check if the incoming BGP announcements can be verified. Using these routers, we can see how many actual BGP announcements are covered by VRPs over time.
To this end, we leveraged two public datasets — RouteViews and RIPE-RIS. Note: one of the limitations of these databases is that they rely on a relatively limited number of vantage points to collect BGP announcements. Thanks to Akamai Technologies, we were able to obtain a much larger BGP dataset collected from thousands of vantage points.
A key observation from Figure 3 is that the number of BGP announcements that are verifiably using RPKI is consistently increasing across all datasets: between 9.98% and 11.28% of BGP announcements are covered by VRPs in our latest snapshot.
In summary, we observed a surprising level of deployment for RPKI, both in terms of the number of RPKI objects (ROAs) that protects the IP prefixes and the actual BGP announcements that can be validated using RPKI objects.
Does this mean all RPKI-covered announcements are valid?
Having only covered BGP announcements, we continued to validate all of them over the entire history of our dataset to not only understand the fraction of valid and invalid announcements today, but to also understand the overall trends.
First, we observed that a large fraction of covered BGP announcements were actually invalid at the very early stages of RPKI deployment across all datasets. For example, during 2011, 8,005,538 out of 16,363,056 (48.92%) covered announcements were invalid (in the RouteViews dataset).
From 2012, the situation became significantly better: in our final snapshot, only between 2.25% and 5.39% of the covered IP prefixes were invalid (depending on the dataset).
We believe the sharp decrease in the fraction of RPKI invalid announcements is due to the RIRs who improved their hosted services and RPKI training from 2012. For example, the RIPE NCC-hosted interface started to show BGP announcement validity to prefix owners and offered the option to operators to receive alerts about invalid announcements. The interfaces of LACNIC and APNIC were similarly modified to show invalid announcements to their users more clearly.
Second, when we focus on the last 12 months of our measurement period, we also notice that the overall percentage of invalid prefixes has further been decreasing since September 2018. We believe this is due to recent efforts from IXPs who’ve adopted RPKI as a service. Some networks started to drop RPKI invalids either by using this service or by deploying RPKI validation themselves; for example, DE-CIX deployed RPKI and started to drop RPKI invalid prefixes in 2019. Thus, the prefix owners who published invalid RPKI prefixes had two choices to prevent their announcements from being filtered by either:
- Removing their invalid ROAs, or
- Fixing them to match their announcements.
Read: IXP Manager adds RPKI support in new release
Why are some RPKI-covered BGP announcements invalid?
BGP announcements can be marked as invalid primarily due to two reasons:
- Too-specific: An announcement would be considered invalid if the announcement is otherwise valid but the announced IP prefix is too specific. In other words, the IP prefix is covered by at least one VRP, and the origin in the announcement is identical to the VRP AS number (ASN), but the announced IP prefix does not exactly match with the VRP IP prefix. In such a case, it is likely due to misconfigurations rather than malicious attempts such as prefix hijacking as the origin AS and VRP ASN are identical.
- Wrong ASN: The announced prefixes are covered by at least one VRP, but none of the VRPs match the ASN in the announcement. These announcements could be malicious as the announcing AS is not supposed to announce such a prefix. However, it may also be a configuration error (for example, ROAs that were not updated when the IP prefixes were moved to another ASN, or AS multihoming where the ROAs were created for one ASN only).
Interestingly, we observe that on average 48.0% to 51.5% of the invalid announcements are too-specific and the others are the wrong ASN.
For the too-specific announcements, because their announcement is more specific than the IP prefix in ROAs, they can be easily fixed either by setting a more specific prefix length in the MaxLength attribute or publishing separate ROAs that cover the IP prefixes that they announced.
For wrong ASN announcement, we do not know for sure if they are hijacking attempts.
There could be a number of causes, including many representing misconfigurations:
Two different ASNs (announced ASN and VRP ASN) are managed by the same operator.
An operator that owns and manages multiple ASNs may update the IP prefixes without updating the ROAs, thus making the originating AS in the BGP announcement mismatch with the ASN in the VRP.
For example, we found that Telmex Colombia S.A that manages two ASes, AS 10620 and AS 14080, announced 1,518 IP prefixes (RouteViews dataset) from AS 10620 between 8 June 2011 and 2 March 2012 (9 months!).
Incorrect announcement between a provider and a customer.
An AS can sub-allocate part of its IP prefixes to its customers. In such a case, if the AS publishes ROAs containing the sub-allocated IP prefixes with its ASN instead of the customer’s ASN, the BGP announcements originated from the customer will be invalid.
For example, AS 6128 (Cablevision Systems Corp.) who sub-allocated its IP prefixes to nine different ASes has ROAs that cover all of the sub-prefixes but set the ASN as their own ASN, thus making the IP prefixes announced from all of their customer ASes invalid from 27 October 2013 to our latest snapshot.
DDoS protection origin ASes may outsource ‘scrubbing’ of their traffic by using traffic diversion to a DDoS protection service (DPS).
These services usually announce IP prefixes on behalf of owners, which may cause their BGP announcements to be invalid if the prefix owner has not updated their ROAs.
However, we rarely see announcements from DDoS protection ASes. We found only 15 IP prefixes (in the RouteViews dataset) in our latest snapshots: AS 26415 (Verisign Global) announcing six IP prefixes owned by AS 13285 (TalkTalk Communications), AS 19905 (Neustar) announcing an IP prefix owned by AS 21599 (Cable Onda) and three ASes from Level3 announcing eight IP prefixes owned by three different ISPs.
Other
If none of the prior cases hold, we do not know the exact cause of the invalid announcement. This case would include attempted (sub-)prefix hijacking, as such announcements would not likely fall under the three categories above.
For example, from 12 January 2017 to 9 March 2017 (in the Akamai dataset), AS 395561 (Absolute Connections) announced more than 28,322 IP prefixes owned by 694 other ASes, which suggests that it had attempted to hijack many IP prefixes. Also, AS 37468 (Angola Cables) announced more than 3,500 IP prefixes originally owned by 82 ASes on 11 May 2018. They did so again on 19 July 2018 by announcing more than 15,000 IP prefixes owned by 1,554 ASes.
As shown, many unauthorized BGP announcements are not necessarily suspicious attempts. Rather, they are likely to be due to misconfigurations such as setting ROA IP prefixes too wide making BGP announcements too-specific or announcing prefixes from a different ASN managed by the same ISP. However, we also show some possibilities that RPKI could detect suspicious hijacking attempts — our simple classification methods allow us to (loosely) estimate what fraction of invalid BGP announcements is possibly due to misconfigurations.
So, is RPKI ready for the big screen?
We believe the answer is yes.
Since its adoption, RPKI has seen very significant deployment to the extent that globally 12.1% of the IPv4 address space is now already covered by ROAs.
Even though in the early days of RPKI deployment the number of misconfigurations that could lead to BGP announcements being marked as invalid was significant, they have been improved dramatically over the years to the point where nowadays over 94.3% of announcements (in the RouteViews dataset) covered by ROAs are valid.
Of course, there are many misconfigurations making BGP announcements invalid such as too-specific announcements. However, considering the data quality also improved dramatically over the years, we do believe RPKI is ‘ready for the big screen’ and operators can start relying on RPKI to drop invalid announcements. And we are not alone in this; a number of prominent operators have already started dropping invalids (for example, AT&T).
This work was done in collaboration with colleagues from Rochester Institute of Technology, the RIPE NCC, Northeastern University, the University of Maryland, Duke University, Max Planck Institute, University of Twente, Akamai Technologies, NLNetLabs and Cloudflare, and presented at IMC’19.
Watch: Taejoong Chung present ‘RPKI is Coming of Age: A Longitudinal Study of RPKI Deployment and Invalid Route Origins’ at IMC 2019.
Finally, how many routers actually do validation?
This is an excellent question that was beyond the bounds of our research. Measuring how many ASes actually do validation is very challenging because it is really hard to identify and reason why specific BGP announcement are being dropped, and on which router. Luckily there are some ongoing research projects (including ours) to measure which AS currently drops RPKI-invalid BGP announcements:
- RIPE NCC created a webpage to allow you to test if your network drops invalid RPKI BGP announcements. The trick behind this is that it fetches content from two test prefixes where one is covered by a valid ROA and the other one from an invalid ROA (for test purposes). Thus, if your network can fetch the content only served from the valid ROA, it is a strong signal that your network is dropping invalid RPKI BGP routes.
- An Internet researcher, Ben Cox, used ICMP messages to infer the RPKI dropping policy on a given network. He sent ICMP messages (ping) to IPv4 space from two IP addresses where one is covered by a valid ROA and the other one is covered by an invalid ROA. If the ping response from a target IP address only comes to the source IP address covered by valid ROA, it is also highly likely the target AS and the target IP address is mapped to is dropping invalid RPKI BGP routes. He also presented his work in the NLNOG 2018 and 2019 meetings, so if you’re interested in this technique, please take a look!
Taejoong Chung is an Assistant Professor at the Computer Science department in B. Thomas Golisano College of Computing and Information Sciences at the Rochester Institute of Technology. His work focuses on Internet security, privacy implications, and measurement.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.
The author asserts that outreach by IXPs was instrumental in popularizing RPKI, but I recall things differently. Even worse, I know of many IXPs that fought tooth and nail *against* BGP filtering (and RPKI).
It was Network Operator Groups (NOGs) that drove the current wave of RPKI enthusiasm. ISPs are the ones who are driving innovation.
Small editorial correction: “unknown” is not a validation state, it is “notfound” in RFC 6811