The Domain Name System (DNS) is a protocol that translates human-readable domain names (such as example.com) into IP addresses (for example, 93.184.216.34). Unfortunately, the DNS is mostly deployed over the User Datagram Protocol (UDP) making it prone to various types of manipulation. In November 2021, users from Mexico could not access whatsapp.net and facebook.com — it turned out those queries were routed towards one of the local instances of the K-root and got intercepted by some middleboxes on the way.
In this blog post, we at the Université Grenoble Alpes (France) analyse that event from the RIPE Atlas point of view and, more broadly, evaluate the extent of DNS manipulation when sending queries to DNS root servers. We have recently presented our findings at the Passive and Active Measurement Conference (PAM 2023).
Two problems: BGP route leaks and DNS manipulation
Any DNS resolution starts at one of the 13 root servers. To cope with huge query loads, each announces two prefixes (IPv4 and IPv6) using BGP anycast but distributes the traffic among more than 1.6k individual instances located worldwide. Some of the local instances are only meant to serve a limited number of clients (such as a single ISP or economy) and their routes should not propagate to the entire Internet. This is usually achieved with NO_EXPORT or NOPEER BGP community attributes (see RFC 4786 for further details).
Yet, sporadic route leaks may advertise local root server instances worldwide. Such events would stay transparent to end users unless clients experienced difficulties reaching certain domain names. For example, when an out-of-region I-root instance leaked in 2010, end users received bogus responses for twitter.com, facebook.com, and youtube.com. Similarly, in 2021, the K-root route leak diverged DNS queries from out of the region and triggered a response injection for whatsapp.net and facebook.com. As root server operators do not serve bogus data, some middleboxes must have intercepted user requests and injected responses.
We were wondering how many end users were impacted by the November 2021 route leak and how long it stayed unnoticeable. More broadly, in this blog post, we analyse to what extent queries sent to DNS root servers are getting manipulated, even when no route leaks or other anomalies occur.
Takeaway #1: Root server route leaks may stay unnoticed
We used one of the built-in RIPE Atlas measurements to verify how far the November 2021 route leak propagated. As it turned out, a local K-root instance was reachable outside its economy at least two months before being reported — 57 RIPE Atlas probes located in 15 economies (AU, UA, CO, HK, LK, CH, FR, US, KR, DK, MX, ZM, BE, GB, NP, KE) had their DNS queries routed towards that local K-root. Even after being fixed, 12 (RU, IL, MX, DK, HK) probes would occasionally reach the out-of-region instance in the following nine months.
Takeaway #2: DNS manipulation is persistent and omnipresent
We set up a series of 312 non-recursive DNS measurements towards all the root server letters with alternating IP versions, transport protocols, query types, and domain names (see Figure 1 below). These were run twice per day from all the connected RIPE Atlas probes (about 11k).
We then divided measurement results into two broad categories: (i) non-injected, if the answer section of the response was empty; and (ii) injected, if we got responses to our queries. We recall that DNS root servers are not authoritative for any of the queried domain names, so we only expect to see referrals to .com/.net TLD nameservers and the corresponding glue records.
As shown in Figure 2, between 3% and 4% of RIPE Atlas probes per week receive injected DNS responses when communicating with root servers. The overall ratio of manipulated measurements does not exceed 1%. At the same time, roughly 20% of participating RIPE Atlas probes experienced response manipulation during all 9 months of the experiment (see Figure 3).
Takeaway #3: Inserted responses are not always bogus
We received more than 11M individual injected responses of five different resource record types: A (7M), AAAA (4M), URI (43k), SOA (7k), and CNAME (5k). Interestingly, those were not always bogus. For example, 49% of facebook.com and 89.6% of google.com responses contained valid A records belonging to requested domain names. The ratio of valid AAAA responses was even higher — 64.4% for facebook.com and 98.3% for google.com. All the CNAMEs were pointing google.com to forcesafesearch.google.com — Google’s service to remove explicit content from search results. Only URI and SOA responses would completely prevent access to requested domains, as those did not contain any valid addresses.
Several entities are behind those injected responses — apart from national censors, we might have encountered DNS filtering services, transparent forwarders that relay DNS requests to alternate resolvers, or DNS servers that serve the root zone locally. Therefore, corporate policies may not allow end users to contact arbitrary DNS servers.
Mitigation
Overall, the response injection affected 7% of all the 14.3k RIPE Atlas probes tested in February to October 2022. The problem could further be exacerbated in case of route leaks, as clients from the outside could also be affected. Some of the countermeasures below will help minimize the risks or avoid the manipulation altogether:
- BGP communities — one can uniquely identify anycast instances using BGP communities, for example, by encoding geographical coordinates in BGP announcements. Routers at the destination networks would then choose the closest instance.
- QNAME minimization — it is not necessary to reveal the full domain name when querying DNS root servers, especially as it might trigger middleboxes.
- Encrypted DNS — this will prevent sniffing plaintext DNS traffic but needs to be deployed on the entire resolution chain (end clients to resolvers and resolvers to authoritative nameservers).
- DNSSEC — validating resolvers will reject bogus responses, provided the domain names in question are also signed.
View further details on this study in the full paper, Intercept and Inject: DNS Response Manipulation in the Wild.
Yevheniya Nosyk is a PhD student at Université Grenoble Alpes (France), working on DNS and network security from a large-scale Internet measurements point of view.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.
The blog refers to the paper from PAM 2023 – which is an excellent read & is well-researched and written very clearly.
That said, as a follow-up project, I might suggest the authors consider differentiating between DNS manipulation that seems user-driven (e.g., the noted use of Google safe search to limit certain content for parental control reasons) at relatively low rates (i.e., 6% adoption) vs. clearly nation-state-driven manipulation at levels north of 50%.
Keep up the good work!