DNS operators have very few intelligent real-time tools that enable them to monitor their anycast services, for instance during a distributed denial-of-service (DDoS) attack.
In this post, we describe the challenges associated with measuring anycast services and propose a tool called the Border Gateway Protocol (BGP) Tuner. By using our open-source tool, operators can see in advance how changes in their BGP policies may impact the traffic load distribution over the anycast sites. This post is a short description of our technical report available here.
Anycast services
Anycast is a way of using the Internet’s routing system so that an incoming request can be routed across multiples sites, with the routing system dynamically selecting the one with the smallest number of hops from the client.
Anycast can reduce latency, improve network resilience, and defend against DDoS attacks. It is used extensively by root DNS operators, DNS service providers, and content distribution networks (CDNs), including Google, Facebook, and Cloudflare.
Why monitor anycast networks?
Operations teams need to be able to respond to various issues in a real-world setting, for instance, those related to:
- Traffic engineering: managing the way the Internet routing system distributes clients across anycast nodes.
- DDoS mitigation: attacks may overload anycast sites; operators can reconfigure the anycast service and shift the load across the available sites.
- User experience: ensuring clients are served by the most-suitable anycast site.
To this end, we should answer the following questions:
- How do clients on the Internet reach my anycast service?
- How can we change/shift the load of traffic reaching my anycast sites?
How can we monitor an anycast service?
In IP anycast, the BGP routes each network to a particular anycast site, dividing the world into catchments. The behaviour of an anycast service is very tightly coupled to the BGP.
To monitor how the BGP routes clients to our anycast service, we have to consider a few approaches. Broadly speaking, there are two possible approaches:
- Using client-side vantage points such as RIPE Atlas to determine which server each client is reaching; or
- Using the open-source tool Verfploeter.
In this investigation, we used the Verfploeter tool because it has a broader coverage of clients (see details here). To measure the client’s distribution, we carried out real-world measurements using the Tangled Anycast Testbed.
Anticipated effects on anycast services
The client distribution over anycast sites is tied to BGP routing visibility. More than evaluating the changes in routing visibility, it is important to develop a systematic way to measure and map the side effects of each BGP configuration. For example, what are the load distribution implications of implementing an AS-Path prepend?
With this in mind, we developed a systematic way of measuring and mapping the side effects of each BGP configuration performed. The following steps were defined:
- Build a baseline: measure and map the distribution of clients across our anycast sites using the regular BGP configuration.
- Apply changes: change the BGP policy and redo the measurement.
- Traffic shift: redo the measurement and carry out mapping to compute how clients are distributed across our sites after the change (that is, the deviation from the baseline).
BGP Anycast Tuner
The BGP Tuner is a prototype graphical interface that presents the distribution of clients over the anycast sites using a predetermined BGP configuration.
We used Verfploeter to carry out the measurements on the Tangled Anycast Testbed, preprocessed them, and afterwards used them as the input for our interface.
This interface intuitively assists the operator, allowing them to ‘equalize’ the traffic volume or catchments on each site in a simple way. An interactive display shows the traffic distribution across the sites when the previously measured BGP policy is applied. The central graph shows the distribution of clients across the respective sites.
The operator can play around with the sliders to increase/decrease the number of clients per site. Note, the sliders do not make linear adjustments; instead, every mark on the slider scale corresponds to a previously measured BGP policy. For some sites, the marks are spread along the full length of the slider scale, while for others the marks are more concentrated. The differences reflect the various sites’ ability to shift clients between anycast nodes. Some sites can exert more influence because of their relationships with their neighbors and their traffic agreements.
Our tool enables an operator to find the best available policy for that operator’s preferred outcome, that is, to increase or decrease the number of clients reaching a particular site.
Operators can also use more advanced policies. For example, on the left side (drop-down menu) there are options such as ‘Bring traffic to Europe’ and ‘Reduce traffic to the USA’. Such options will help operators to identify which of the presented policies are most suitable.
The BGP Anycast Tuner is an open-source tool and available here.
Acknowledgements
The findings of this investigation and the tools developed from it form part of the SAND project, a joint effort by SIDN Labs, NLnet Labs, and the University of Twente that started in April 2018 and finished in April 2020. Read the full report here.
Adapted from original post which appeared on RIPE Labs.
Joao M. Ceron is a Network Security Research Engineer at SIDN Labs, the research arm of SIDN, the .nl registry.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.