In my last post, I talked about our experience at DNS-OARC with updating the DNS Statistics Collector data tool (dsc-datatool).
I mainly focused around the Grafana ‘face-lift’ we’d given it, to be more user friendly for monitoring, but we’ve also been working on new features to make it a one-stop-shop for your DNS monitoring needs. I’m referring to DNSTAP support.
A quick search shows there’s been very little (no) talk about DNSTAP here on the APNIC Blog, so let’s first catch up those not familiar with what it is before discussing the benefits of its support in our humble monitoring tool.
What is DNSTAP?
DNSTAP is an open-source format, developed by Farsight Security, and uses protocol buffers to encapsulate events that occur inside DNS software. These events are then transported using Frame Streams (fstrm) to receivers which can choose to log the events or do other analysis of them.
Let’s take a look at the DNSTAP architecture to understand what that means.
First, a copy of the raw DNS data from the DNS server (Sender) is collected and encapsulated into DNSTAP. This is then transported over a reliable byte-stream socket (fstrm) from the sender to the receiver software where it can be logged for analysis.
Separating the DNSTAP byte stream input/output (I/O) from the DNS server operation is a defining feature of the system as it overcomes logging and performance issues associated with collecting DNS data directly from the source DNS server.
A second, and increasingly important feature, is the fact that it collects raw DNS data, which includes encrypted DNS — something we’re starting to see (or not see) a lot more of with the introduction of DNS over TLS (DoT) and DNS over HTTPS (DoH).
So, the DNSTAP system, in its basic form, is both a format and software to encapsulate and transport DNS. The next step is the analysis, which is where DSC comes in.
Visualize all of DNS
As of v2.9.0, DSC is able to receive DNS messages over DNSTAP instead of capturing it on an interface. These messages come directly from the DNS server/daemon/software itself and because of that, you can get statistics on encrypted DNS transmissions.
If you go with DNSTAP there are a few different inputs supported by DSC: file, UNIX socket, TCP, or UDP. So far we’ve tested it successfully with BIND 9 and Unbound, using the available DNSTAP transport(s) of each software.
Ultimately, DNSTAP provides a second source of data, which in some respects is more pure than the data that can be captured on interfaces.
DNSTAP is not network traffic
One important thing to clarify is that you only see the data that comes via DNSTAP, namely that which is passed along from the DNS server, and not that at the network layer — this is a limitation you might need to be aware of if you are also interested in all the weird things that happen there.
Because DNSTAP comes directly from the DNS server, DSC will not be able to generate some of the statistics it otherwise would. For example, DSC can’t generate interface statistics for captured and/or dropped network packets because the information is not included in DNSTAP.
DNSTAP is optional
Another point to clarify is that almost all data points in DNSTAP are optional, so it depends on the software sending DNSTAP, which data it will include in the messages. This means that certain statistics in DSC might not be available because the data needed for them simply doesn’t exist, and other statistics will need supplementing information (provided in DSC config) to be able to be generated.
As always, we’d love to get feedback on this new feature, so please reach out or comment below if you have any issues or suggestions.
DNS-OARC seeks to improve the security, stability and understanding of the Internet’s DNS infrastructure. This has resulted in a diverse, motivated and highly collaborative community, which has been able to work together cohesively on pertinent issues facing the DNS, the worth of which has been highlighted in recent years against the backdrop of the growing number of malicious incidents directed towards the DNS. Read: Rallying a community to protect core Internet service.
Jerry Lundström is a software engineer with DNS-OARC.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.