Recently, we at SIDN Labs launched rssac002.sidnlabs.nl, a new website where we publish daily statistics on the DNS traffic that reaches the authoritative nameservers for .nl. All the data is also available on GitHub.
What we publish
Every day, the .nl nameservers answer billions of DNS queries from all over the world. We summarize that traffic into a small set of daily metrics, such as query and response volumes, response codes and the number of unique sources.
Publishing DNS traffic statistics is not new in itself. We have shared .nl figures on stats.sidnlabs.nl since 2015, CZ.NIC publishes statistics for cz, and .ie includes traffic figures in its annual reports. But every registry uses its own metrics, definitions, and formats, so the numbers can’t be compared with each other or with the root servers.
That is why we now publish in the same format the Root Server Operators use: RSSAC002, a specification from ICANN’s Root Server System Advisory Committee (RSSAC). Root server operators have published these files for years. To our knowledge, .nl is the first Top-Level Domain (TLD) to publish its data in this format as well.
On the website, you will find the daily YAML files, ready-made graphs, a description of each metric and our methodology.
Why this is useful
The DNS is the Internet’s address book: Before your browser can open a .nl website, a resolver asks our name servers where to find it. Our data shows how that system is used one level below the root. It is the same kind of view the root servers give, but for one of the world’s largest CC domains.
Because the format is identical, .nl traffic can be compared directly with the root servers. Figure 2 is an example: On a typical day, the .nl name servers together handle about as much traffic as D-Root, one of the 13 root servers, and 50% more than B-Root.
With these numbers, anyone can see, for example:
- How the Internet is changing: How fast IPv6 and TCP are being adopted, and whether DNS responses are getting larger. Larger responses matter for DNSSEC and can cause problems when packets are fragmented.
- How much traffic is noise: The share of queries for domain names that don’t exist, a common sign of misconfiguration or abuse.
- When something unusual happens: Sudden peaks or drops can point to outages, attacks or changes at large resolver operators.
Researchers can use the data for DNS measurement and long-term trend studies. Operators can compare their own traffic with a large ccTLD and with the root. Existing tools built for root server data, such as the RSSAC002 charts, can process .nl data without changes.
Why we do this
At SIDN Labs, we believe that measurement data about the Internet’s core infrastructure should be open. Researchers can use it for DNS measurement and long-term trend studies, and operators can compare their own traffic with a large ccTLD and with the root. We have used the root servers’ RSSAC002 data in our own research, so we know how valuable it is. Now we are giving something back.
The whole pipeline is open source too. The data comes from ENTRADA, our open-source DNS data warehouse. The metrics are computed with our tool, a fork of the original built by computer science students at TU Delft. They developed it in the Software Project course, where teams of bachelor students spend three months building working software for an external client. For us, it is a good example of how academic collaboration can turn a real operational need into open-source software that others can use as well. Many thanks to the students.
Any ENTRADA user can run the tool and produce the same statistics for their own servers. We hope other TLD operators will publish their datasets too.
No personal data
The files contain only aggregated, daily statistics. No Personally Identifiable Information (PII) is shared: No IP addresses, no individual queries, and no looked-up domain names.
What’s next
For now, we are publishing the data in batches. Our next step is to fully automate the process, so that new files are released continuously, every day, without manual work.
Links
- Website
- Datasets
- Tool (a fork of the original by TU Delft students)
Using the data? We’re curious to know what for. Get in touch via sidnlabs@sidn.nl.
Giovane Moura is a Data Scientist with SIDN Labs and an Assistant Professor at TU Delft.
Originally published at SIDN Labs.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.