This is the second of two posts in which we present a new attack vector on the routing infrastructure of the Internet using BGP communities.
In order to investigate to what extent BGP communities can be leveraged for attacks, we conducted two kinds of experiments: lab experiments and active measurements to validate our lab results. The latter required careful analysis of potential side effects as well as support and consent from the involved operators.
In this post, I will focus on the experiment with the strongest real-world impact: Remotely Triggered Blackholing (RTBH), which is commonly used to mitigate Denial of Service (DoS) attacks.
RTBH
During RTBH, an AS signals to its upstreams that traffic towards specific network prefixes or single IP addresses should be dropped and not forwarded, even though, for example, a less specific prefix is still announced and reachable.
To implement this technique, the upstream AS, providing the blackholing service, and the customer AS, using RTBH to drop unwanted traffic, need to agree upon a signalling mechanism to determine which prefix should be blackholed. Usually, the normal BGP session is used by adding the blackholing community of the respective provider to a regular announcement. When the blackholing provider receives this community, the routers will null route traffic towards the tagged prefix.
An example is given in Figure 1, where AS2 is providing RTBH for its customer AS1, that announces a prefix p, tagged with the blackholing community 666 of AS2.
Figure 1 — An example of how Remote Triggered Blackholing (RTBH) works.
While AS2 keeps announcing the IP space, especially if there is a less specific covering prefix, it will drop all traffic that is directed towards prefix p from other peers, for example, AS3, AS4 and AS5, at its border routers. Therefore, the link between AS1 and AS2 is freed from unwanted traffic towards p.
Blackholing providers have to ensure that customers are only able to blackhole their own prefixes. Policies for customers and peers on which prefixes are valid to be blocked may differ. RFC 7999 gives recommendations on how blackholing announcements should be forwarded, that is, adding NO_ADVERTISE and NO_EXPORT.
RTBH: confirming an attack in the wild
Figure 2 shows an experiment we developed to confirm if RTBH can be exploited to launch Denial of Service attacks itself.
Figure 2 — AS2 triggering blackholing in AS3 for prefix announced by AS1.
In this experiment, AS3 is providing RTBH for AS1. AS2 is a second upstream for AS1 but is not on the best path for AS4. Thus, traffic from AS4 towards AS1 traverses AS3, not AS2.
In the experiment, we noticed that it was possible for AS2, a valid upstream for AS1, to tag a prefix p originating in AS1 with the blackholing community for AS3. And although AS2 is currently not on the best path, traffic from AS4 to AS1 was dropped in AS3. This allowed AS2 to attack by not only dropping traffic traversing itself but also completely isolating AS1.
A similar attack was also possible when AS2 was not even an upstream of AS1 nor on a valid AS path towards AS1. By hijacking the prefix p and adding the blackholing community, we were able to let AS3 drop valid traffic towards AS1. Arguably, when the prefix has already been hijacked the attacker has already taken control of the traffic directed at the hijacked prefix. But by adding the blackholing community, the attacker is able to null route the traffic, which is harder to detect than ‘just’ a redirection.
Both attacks work over multiple hops and the attacks are hard to spot. Some attacks are possible due to intrinsic design flaws, and some due to configuration mistakes.
After presenting our work at the RIPE 77 meeting, we were informed that, in fact, bad actors are already starting to use BGP communities. A description (Dyn) of an attack on 10 July 2018 reads as follows:
”For about 30 minutes, these hijack prefixes weren’t propagated very far. Then they
were announced again at 23:37:47 UTC for about 15 minutes but to a larger set of
peers — 48 peers instead of 3 peers in the previous hour.
It appears a change of BGP communities from 24218:1120 to 24218:1
increased the route propagation.”
Challenges using BGP Communities
The notation of communities as ‘ASN:value’ is just a convention. Because there are no defined semantics, the values can mean anything and it is up to the ASes to define the functionality.
BGP communities are simultaneously used to signal and trigger actions. As such, no distinction can be made without documentation of the AS using the communities. Furthermore, the communities attribute is not cryptographically protected nor are they covered by RPKI. Attribution of changes in BGP communities is futile as it is impossible to find out which AS on an AS-Path has added, removed or changed a community value on an announcement.
Large communities, in principle, have similar limitations, although they are more strictly defined.
We acknowledge that BGP communities are the only feasible way to do signaling between ASes and that implementing complex security mechanisms to address their shortcomings is not the solution. Nevertheless, we would like to highlight that secure usage of BGP communities requires good operational knowledge and diligence. While many operators are familiar enough with these challenges, based on experience we cannot rely on that globally.
In order to realize RTBH, blackholing providers need to accept very small prefixes in IPv4 (for example from a /24 to a /32). For this to occur, an exception for prefixes tagged with blackholing communities has to be added to the configuration. That said, this configuration path often lacks proper prefix validation — we found that some providers generally check the blackholing community before any prefix filters are applied, which means providers might accept invalid blackholing announcements.
We’ve also found that some providers do not add the NO_ADVERTISE, NO_EXPORT communities, so blackholing announcements are propagated further than desired. Further to this, one of the biggest issues is that there is no validation for the origin of a community present on an announcement — we were also able to execute a traffic redirection attack on the Internet, which we detail in our paper.
Recommendations
To prevent external parties interfering with your network, we recommend you:
- Filter incoming informational communities with your ASN at your border routers.
- Publish up-to-date documentation on your communities to enable debugging and filtering to enable your customers to filter communities received from their customers that would trigger unwanted actions in your AS.
- Encourage downstreams to filter action communities for your AS on ingress if necessary
- Provide a looking glass that also shows communities.
Although BGP communities are widely used, they have limitations: their usage relies heavily on mutual trust in capabilities of third parties; there is no authenticity or security by design; attribution is impossible; and attacks are hard to detect. This makes them an ideal playground for malicious actors.
This is supported by the fact that while the prefix hijacks we did — with operators’ consent — had been reported publicly, no one noticed or reported our community attacks that led to blackholing and redirection of traffic.
Contributors: Franziska Lichtblau
Florian Streibelt is a PhD candidate at the Max Planck Institute for Informatics in Saarbrücken, Germany. His research interests include the DNS, Internet measurement, protocol design, interdomain routing, security and data analytics.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.