Are large DNS messages falling to bits?

The Domain Name System (DNS) provides one of the core services of the Internet. DNS employs both UDP and TCP as transport protocols, but most responses are sent over UDP given it is fast at one Round Trip Time (1 RTT). However, UDP is not always suitable to deliver large DNS responses as packets can be dropped and fragmented. There is a risk that clients will not receive the answers, which can lead to unreachability.

To determine how serious the problem of large messages in DNS can be, we analysed 164 billion DNS queries and responses, collecting three full months of data (July 2019, July 2020, and July 2021), at the authoritative servers of the Netherlands’ .nl ccTLD. This blog will present the main results of the paper we published at the Passive and Active Measurements Conference 2021 (PAM 2021).

Nobody really likes to wait for a page to be loaded on the Internet, and the DNS can be one of the causes of slow load times. Domain names need to be resolved before pages can be loaded. Faster responses are obtained with DNS over UDP (DNS/UDP), which require one round-trip time (1 RTT). However, given UDP provides no delivery guarantees by design, the DNS can also be used with TCP (DNS/TCP), which takes two RTTs to retrieve the same responses (TCP requires an extra RTT due to its handshake).

DNS/UDP is faster than DNS/TCP, but it has a tough time handling large messages. The original DNS specification limited UDP messages to 512 bytes, which proved insufficient in many cases, so in 1999 EDNS0 was proposed, allowing the extension of UDP message sizes up to 64k bytes.

With EDNS0, DNS clients (resolvers) can advertise their UDP buffer to the authoritative servers, which would use that value as an upper limit when sending responses. If a response was larger than the EDNS0 buffer advertised by the client, then the authoritative server would truncate it and mark it TC bit, so the resolver would use that signal to request the query again, using DNS/TCP.

The issue was that all of this was done at the application layer, which is agnostic to the networking layer. In other words, these buffer negotiations did not consider the maximum transmission unit (MTU) of the path between client and authoritative server — and the most common MTU at the core of the Internet is 1,500 bytes. If DNS responses were larger than the path MTU, then these packets would be simply fragmented or discarded along the way. IPv4 fragmentation is so poorly designed that nowadays it is considered fragile and should be avoided. The worst case is when responses are silently discarded, and clients never receive a DNS response, which effectively blocks them from reaching their desired URL.

While other work has investigated this issue, we look at it from a different vantage point: two anycast authoritative servers of the Netherland’s ccTLD (.nl). We analyse 164 billion queries, collected with our DNS big data analysis tool ENTRADA, as shown in the table below:

Table 1 — Datasets from the .nl zone.

Datasets

We collected data from two .nl anycast authoritative servers, NS1 and NS3. Although each is run by a different anycast provider, they’re combined in Table 1. We took yearly snapshots in July 2019, July 2020, and October 2020, the first month after DNS Flag Day 2020.

From our vantage points, we see that a small fraction of responses is truncated: 2.93% to 7.15%, depending on the year and IP version. This is the starting point of our analysis.

Large responses are rare

The first analysis we did was to calculate the distribution of the response sizes the servers sent. As indicated by the vertical dashed line in Figure 1, 99.99% of the responses from the .nl servers are smaller than 1,232 bytes, which is the size proposed by the DNS Flag Day 2020. One could argue that data is only valid for the .nl zone, but Google Public DNS, the largest public resolver service on the Internet, reports that 99.7% of their traffic is also smaller than 1,232 bytes.

Contrary to what we expected, the largest responses were for A and AAAA records of the .nl authoritative servers, not DNSSEC records. The size of the responses also changed per server: NS1 is configured to return minimal responses, while NS3 is not. Therefore, minimum responses effectively prevent extra records being added in the additional section, reducing the message response size.

Fragmentation rarely occurs on the server side

IP fragmentation can happen on the server side, but only for IPv4 as IPv6 forbids in-network fragmentation. For each server and IP version, we analysed the number of fragmented responses sent by our servers. As shown in Figure 2, relatively few responses were fragmented at less than 10,000 per day, but we saw 1 to 2 billion daily queries by comparison, in Table 1. In the paper, we discussed an active measurement with RIPE Atlas to measure in-network fragmentation and found that 4.4% of queries are fragmented at the network level in the wild over IPv4.

Truncation

The data in Table 1 shows that 2.93 to 7.15% of the UDP responses were truncated, so we investigated. Figure 3 shows the Cumulative Distribution Function (CDF) of both response sizes and EDNS0 buffer sizes for NS1. We see that most DNS/UDP queries are truncated to values under 512 bytes, independent of the IP version.

In Figure 3, the left dashed vertical line also shows us that most buffer sizes are equal to 512 bytes, which is rather small. Oddly, the purple line for IPv4 shows that NS1 receives 13% of its queries without EDNS0 extension. We found that this was from two ASes, which has an odd behaviour and only queries NS1, which is a sticky resolver.

So, when a resolver receives a truncated response, it should ask the same query again using DNS/TCP. We found that this happens in 80% of the cases, as shown in Figure 4.

DNS Flag Day 2020 uptake

The DNS Flag Day 2020 proposed that resolver operators configure their EDNS0 buffer sizes to 1,232 bytes. That, in turn, would reduce the large buffer sizes we see in Figure 5, and avoid both fragmentation and truncation. When we use the October 2020 dataset and compare it against July 2020 to measure the uptake of DNS Flag Day, all resolvers are seen on both datasets, and we see how many have migrated to EDNS 1,232 bytes.

From 1.85 million resolvers (unique IP addresses), we see only 11,338 that adopted 1,232 bytes compared to July 2020, suggesting that Flag Day didn’t cause operators to change their settings immediately, but operators adapt slowly.

We also investigated the daily distribution over a 1.5-year period, as shown in Figure 5. By the end of May 2021, we saw 9% of the resolvers announcing 1,232 bytes — twice as many as one year earlier. However, the majority still announces either 4,096 bytes or other values.

Falling to bits?

This study complements previous ones on DNS fragmentation and truncation. While rather rare, large responses exist in the DNS, and they can be prevented by the increased adoption of smaller buffer sizes. Server-side fragmentation is very rare, for both IPv4 and IPv6, but in-network fragmentation is still present at 4.4% for IPv4.

The DNS Flag Day 2020 had some impact, but DNS operators adopted its recommendations only slowly.

This blog is based on a peer-reviewed paper, which can be downloaded here.

Giovane Moura is a Data Scientist with SIDN Labs.

Moritz Müller, Marco Davids, Maarten Wullink, and Cristian Hesselman contributed to this work.

Rate this article

---

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.