Criminal threats executed over the Internet are multifaceted, with the dominant threat from an infrastructure perspective remaining Denial-of-Service (DoS) attacks aiming at forcing a service or system to become unavailable by consuming all available network bandwidth.
Using distributed traffic sources (DDoS), attacks of up to 1.7 Tbps have been observed in the wild.
1.7 Tbps DDoS Attack 💣 — Memcached UDP Reflection Attack Sets New Recordhttps://t.co/6xNEfwdTIS
Just 5-days after #Github hit with record-high 🚀1.35 Tbps attack. pic.twitter.com/mJnTqWeg24
— The Hacker News (@TheHackersNews) March 6, 2018
One common technique to counter these attacks is Remote Triggered Blackholing (RTBH). While working well for transit providers, RTBH has major limitations when applied at Internet Exchange Points (IXPs). This post discusses RTBH’s shortcomings at IXPs and proposes a new approach: Advanced Blackholing. We provide insights into a prototypical implementation of Advanced Blackholing at a major IXP.
Limitations of blackholing at IXPs
RTBH, also referred to as BGP blackholing, is an operational DDoS mitigation technique. ASes under attack can signal to drop traffic to specific IP prefixes. At IXPs the process is commonly implemented at the IXP’s route servers as shown in Figures 1a/b.
Figure 1a depicts a simple example of an IXP during an attack without RTBH. The upper part shows the control plane distributing reachability information among three ASes. AS1 announces a /24 prefix which is redistributed by the route server to AS2 and AS3. Thus, AS2 and AS3 can send traffic to AS1 via the IXP’s switching platform (data plane, lower part).
If an attacker sends DDoS traffic from AS2 and AS3 to a service hosted in AS1, the traffic can congest a link or render the service of AS1 unavailable for legitimate users.
Figure 1a/b — RTBH at IXPs and illustration of limitations.
To react to the attack, AS1 announces an RTBH prefix at the IXP’s route server as shown in the upper part of Figure 1b. Usually, this is a more specific prefix (that is, a single IP, in the case of IPv4 a /32 prefix) with a special and standardized BLACKHOLE BGP community attached to the prefix (see RFC 7999), which is redistributed by the route server to AS2 and AS3.
Routers receiving this announcement are expected to drop any traffic routed towards this IP. However, in many cases RTBH prefixes are not honoured by receiving ASes for different reasons. For example, they chose not to participate in RTBH, they do not accept updates for more specific prefixes than /24, or they misconfigured their router.
In the simple example in Figure 1b, AS2 accepts the prefix but AS3 does not. This has direct implications on the data plane (lower part of Figure 1b) showcasing the two central shortcomings of RTBH at IXPs:
- The traffic from AS2 is dropped in an all-or-nothing fashion; even benign traffic from legitimate users is dropped. Thus, RTBH causes collateral damage.
- The traffic from AS3 is not dropped at all and is still traversing AS1’s link to the IXP. The load on the link may have reduced, but the service in AS1 may still not reachable via the IXP. Thus, RTBH is not effective at IXPs.
Can we prove these effects with measurement data?
We addressed this question with two experiments.
First, we examined an IXP link under a real DDoS attack. The attack was a memcached reflection attack, which lasted several hours and reached a peak volume of up to 40 Gbps. Figure 2a shows the normalized traffic mix on the link, clustered by source transport port number.
Figure 2a/b — Experiments showcasing collateral damage (left) and lack of efficiency of RTBH at IXPs (right).
The attack started at around 20:20. Before the attack, we observed a normal web service traffic mix mainly containing HTTP and HTTPS traffic. After 20:20 the memcached attack consumed about 70% of the mix, the remaining 30% of legitimate traffic remained in the mix. If the AS announced an RTBH prefix, the legitimate part of the traffic would have dropped as well. This demonstrates the order of magnitude of collateral damage caused by RTBH.
For the second experiment, we set up a dedicated measurement AS peering with the IXP’s route server and announced a dedicated /24 prefix for the experiment. Afterwards, we launched a DDoS attack targeting an IP address from the /24 by buying DDoS traffic from a DDoS-as-a-service website. To prevent damage we took a number of security measures such as coordinating the experiment with the IXP’s Network Operation Center (NOC) and being prepared to withdraw the /24 prefix immediately if any problems occurred.
The results of the experiment are depicted in Figure 2b. The x-axis shows the duration of the experiment, the red line shows the DDoS traffic received and the blue line shows the number of peers sending traffic.
Around 400 seconds into the attack we announce an RTBH prefix for the IP under attack. As a result, the DDoS traffic falls from around ~800 to ~600 Mbps and the number of peers sending traffic falls from ~40 to ~30 peers. The remaining peers did not honour the RTBH prefix, thus showing the lack of efficiency of RTBH at IXPs.
Advanced Blackholing
To design an Advanced Blackholing mechanism, two design goals are desirable:
- Allow for better granularity to increase the selectivity of the mechanism, minimizing collateral damage of legitimate traffic.
- The number of involved parties to increase the efficiency of filtering. Ideally, only two parties are involved in the process: the attacked AS and the IXP operator.
The main idea behind the design of the Advanced Blackholing mechanism is to extend the BGP interface of RTBH while performing the filtering directly in the IXP’s switching fabric. An overview of the concept is discussed below.
Figure 3a/b — Advanced Blackholing interaction (left) and implementation of blackholing controller (right).
From an AS perspective, the mechanism does not change much except for a different blackholing community that is attached to the prefix (Figure 3a). This community encodes which type of filter should be applied to the prefix, thus addressing the need for higher granularity.
When a prefix with the community appears at the route server, it is not redistributed to other ASes at the IXP. Instead, the prefix is forwarded to a dedicated blackholing controller peer always accepting the prefix.
The blackholing controller (Figure 3b) maintains a BGP session with the route server(s) and maintains the announcement’s state in a custom Routing Information Base (RIB). Changes of the RIB requiring the application of new filters or the deletion of existing filters are converted into configuration changes, which are added to a queue.
The queue is required to maintain a reasonable rate limit of configuration changes applied to the data plane. Thus, configuration changes are only de-queued with a configurable frequency and forwarded to a network management system applying the changes to the IXP switching platform; that is, to the port configuration of the involved ASes. Subsequently, the switching platform starts dropping or shaping DDoS traffic intended for AS1.
Evaluation
We evaluated the effects of the Advanced Blackholing mechanism by repeating the DDoS experiment conducted above. The only change to the experiment’s setting was the announcement of an Advanced Blackholing route instead of RTBH.
Figure 4 — Advanced Blackholing results. The red line shows the received traffic volume and the blue line shows the number of peers sending traffic again.
Similar to the experiment before, we request a DDoS attack from a DDoS-as-a-service website to attack a single IP in our dedicated research AS.
Around 300 seconds into the experiment, we announce an Advanced Blackholing route shaping the DDoS traffic to a rate of 200 Mbps. The shaping takes immediate effect while the number of peers remains constant as expected.
After ~500s an Advanced Blackholing prefix including the BGP community for dropping the traffic is announced. As a consequence, the DDoS traffic levels drop as well as the number of sending peers close to zero.
It’s time to update your mitigation solutions
Network attacks are more frequent and voluminous than ever. Recent attacks have taken everybody by surprise, as attack traffic volume is in the order of Tbps, and still growing to set new records.
As such, there is a real need for new attack mitigation techniques which inherit the scalability and low cost of blackholing, do not require the cooperation of network operators, and provide fine-grained filtering. We’ve demonstrated that Advanced Blackholing has these abilities and should be considered as a future mitigation solution.
This blog summarizes our research paper Stellar: Network Attack Mitigation using Advanced Blackholing. The research paper was published in the proceedings of the International Conference on emerging Networking EXperiments and Technologies (CoNEXT) 2018. The original authors are Christoph Dietzel (TU Berlin / DE-CIX), Matthias Wichtlhuber (DE-CIX), Georgios Smaragdakis (TU Berlin), and Anja Feldmann (MPI Saarbrücken).
Contributors: Christoph Dietzel
Matthias Wichtlhuber is a member of the DE-CIX research team and works on product development, system security, future network architectures for Internet exchange points, and large-scale network data analysis.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.