This is the first of two posts in which we present a new attack vector on the routing infrastructure of the Internet using BGP communities.
The Internet as we know it relies on a vast combination of different protocol stacks, protocol extensions, tools and best practices. Additional features and extensions, such as Border Gateway Protocol (BGP) communities, have been added over the years to make Internet operations feasible.
BGP communities (as defined in RFC 1997) are used to signal reachability information between Autonomous Systems (ASes). In principle, BGP is used to distribute routes amongst peers, but for network managing purposes it is possible to attach additional properties to each routing update. These properties are encoded within the communities attribute in BGP.
Most network operators happily work with BGP communities on a daily basis and, from what we have gathered, do not see them as a potential security issue. However, under some circumstances, BGP communities can be used to drop and redirect traffic. In principle, the same effect can be achieved via BGP hijacks, but community-based attacks are way harder to spot and almost impossible to attribute.
We, as measurement researchers, took a closer look to understand what could possibly go wrong when dealing with BGP communities.
Key Points:
- BGP community usage is increasing, so too the risk of communities being used as an attack vector.
-
Community-based attacks are hard to spot and almost impossible toattribute, if communities are not protected, for example by RPKI.
- 10% of the observed communities have an AS-hop-count of more than six different ASes and more than 50% traverse more than than four ASes.
BGP community usage is increasing
When looking at the publicly available BGP data provided by services such as RIPE NCC’s Routing Information Service (RIS), RouteViews, PCH and Isolario, not only are the number of ASes and prefixes increasing, but also the usage of BGP communities. Between 2010 and 2018 the number of communities has almost tripled (as shown in Figure 1 below). Such behaviour makes us researchers very curious.
Figure 1 — Historic development of BGP data: RIPE Routing Information Service (RIS) and RouteViews.
Before we explore this phenomenon, let’s first define what communities are.
A community is a single 32-bit field (an integer) and an optional attribute in BGP messages. By convention, the integer is split into two 16-bit values, interpreted as an AS number and a value, where the AS referred to in the community is usually defining the meaning of the value.
Here we run into the first problem: today we have ASes with 32-bit AS numbers that will not fit into that scheme and therefore cannot use communities in a sensible way. To mitigate this, RFC 8092 defines BGP Large Communities, a 96-bit (12-byte) field and allows these ASes to use communities as well.
The only problem here (from an Internet measurement perspective) is that as BGP Large Communities are quite young, adoption is still slow (worse than that of IPv6). We currently see only 51 global administrators (which can be compared to the “AS-part” of BGP ‘legacy’ Communities) using them.
For a detailed description of the development and usage of BGP Large Communities, see this excellent article by Job Snijders in the Internet Protocol Journal, Volume 20, Issue 1.
Use of BGP communities
All BGP communities fall in either of the following categories:
- Informational communities with passive semantics.
- Action communities with active semantics.
While passive semantics only tag an announcement with some additional information, for example, the location where a router has learned that route or some value signalling the RTT on that specific link, action communities with active semantics directly influence routing policy decisions on a peer. They are used to signal some intent from one AS to another, for example, to execute path prepending, change the local preference/MED or to apply Remote Triggered Blackholing (RTBH) on a prefix (more on that in the next article).
The problem here is: there are almost no standardized communities. Without documentation, you can not tell if a community is active or passive — you don’t even know what it’s being used for in the first place!
BGP communities as an attack vector?
Given the increasing popularity of BGP communities and the ability to remotely trigger actions and relay information, one question arises: To what extent can BGP communities be leveraged for attacks?
One necessary condition for community-based attacks is that communities are propagating from one peer to another.
We observed that 14% of transit providers are propagating received communities to their peers. At first this number — 2,200 out of 15,500 ASes — seems to be rather small. But given the ongoing flattening of the Internet topology and the highly connected AS graph, this number is actually sufficient for a wide propagation of communities.
The propagation behaviour of BGP communities is specified in two separate RFCs:
- RFC 1997 defines communities as a transitive optional attribute and
- RFC 7454 states an AS should scrub communities used internally but forward foreign communities.
Still, many people do not expect communities to propagate that widely — including authors of this work!
Propagated communities might trigger actions multiple hops away beyond the direct peer they are announced to. There is no way of knowing if the observed behaviour is intended, for example, for traffic management, or not. It is even unknown what AS has added the community to the announced prefix (more on that later).
Our assessment here is that there is a high risk for misuse or even attacks!
Let’s look into some measurements!
In order to assess how widely BGP communities actually propagate throughout the Internet, we analysed the available BGP routing data on community propagation behaviour.
Our data set consists of BGP dumps provided by four projects (RIPE RIS, RouteViews, Isolario, and PCH) for the full month of April 2018. These were almost 39 billion BGP messages to process; and we found more than 63,000 different communities in these messages.
More than 75% of all BGP announcements have at least one BGP community set. 5,659 ASes are using communities. That is about 9% of the total number of ASes.
We found that 10% of the observed communities have an AS-hop-count of more than six different ASes and more than 50% traverse more than four ASes. We even found a community that is forwarded across 11 ASes. If you compare this with the mean distance of AS-paths on the Internet of 4.7, or the median of 5 from our data sets, this means that communities are pretty much propagated globally.
But how can we measure the AS-hop-count of a community while proclaiming one cannot say which AS has added a community in the first place? Well, to be honest, we cannot. We can only infer from the routing data which AS on the AS path has probably added it, which gives us a lower bound for the travel distance. This means that the travel distance can be even higher.
Consider this simple example (Figure 2):
Figure 2 — Inferring the travel distance of a community (AS hops).
We infer that AS2 has added the community 2:303 while AS3 has added the community 3:123, while in fact, AS2 has also added community 3:123. This gives us an AS-hop-count of 1 for community 3:123 and two for community 2:303, which yields a lower bound.
Leaking BGP communities
Another interesting observation we made is ‘leaking’ of communities ‘off-path’. This refers to communities found on the route collectors that contain ASNs in the first 16 bits which are not present in the AS path of the announcement, that is, they have not traversed the AS denoted in the community (Figure 3 illustrates this scenario).
Figure 3 — Communities for ASes not on the AS path / ‘leaking’ off-path.
Why is this the case? Some ASes add communities to trigger actions with one of their upstreams to the announcements towards all of their peers and not just the upstream AS where this would be required. Furthermore, if a community is directed at an AS multiple hops away the direct peer might propagate the prefix to other peers, including all communities received. We thus distinguish on-path and off-path communities.
More about this and RTBH in our next post.
On-path versus off-path
When comparing off-path and on-path communities the distribution of values (the 16-bit right-hand part of the communities) we find distinct differences (as you can see in Figure 4 below).
Figure 4 — Distribution of values for on-path and off-path communities.
In the off-path community values on the left you find, for example, values used to trigger blackholing in upstream networks while in the on-path communities on the right you’ll notice that ‘nice’ numbers are dominant which are easy to remember. We think that blackholing communities are so prominent on the off-path side because ASes that do not implement blackholing will not touch blackholing communities and just forward them and propagate them to all peers. On the other hand, the on-path values look like operator assigned numbers for traffic engineering — limited manual checks with community documentation of some ASes confirm that suspicion.
In our next post, we will discuss the results of our experiments into RTBH, which is commonly used to mitigate Denial of Service (DoS) attacks and provide recommendations as to what providers can do to prevent external parties interfering with their networks.
Contributors: Franziska Lichtblau
Florian Streibelt is a PhD candidate at the Max Planck Institute for Informatics in Saarbrücken, Germany. His research interests include DNS, Internet measurement, protocol design, interdomain routing, security and data analytics.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.
Thank for you broadly explained..