How to: Managing DNS zones using Git

Using zone files for managing Domain Name System (DNS) zone data is the most basic and flexible way to manage authoritative DNS data. In this post, we will describe how the Git version control system can be used to store zone files’ history and protect against errors.

Traditional zone files are still quite a popular way of keeping DNS zone data, especially for Internet Service Providers; where managing DNS is not the core business and the demand for changing data quickly is not high. The most basic solution is to edit a zone file directly on the primary authoritative DNS server and use the standard DNS zone transfer (AXFR/IXFR) to sync zone content from primary to secondary servers. In such a setup, care must be taken not to introduce syntax errors into the zone files and to keep the zone serial number increased after every change, otherwise the zone will not transfer to the secondary servers and a situation called split-brain will occur.

This is the main reason why hidden primary DNS servers are deployed. A hidden primary server holds the zone data but is not listed as the name server in the parent zone. Its sole purpose is to transfer zone data to the public-facing secondary servers using zone transfer. This way, zone data is guaranteed to be error-free and synchronous among all public-facing servers. In case of DNSSEC deployment, it’s also practical to keep private DNSSEC key material out of the public-facing servers.

It’s always useful to track the history of zone files. Using comments directly in the zone files doesn’t scale very well and is superseded by modern revision control systems such as Git. Committing every single change to a Git repository, on the other hand, adds another overhead to the work of the operator in charge. Even though all sub-tasks are rather simple, doing them manually is prone to errors. Fortunately, Git supports custom-made hooks that can be set up to intervene with various stages of its proceedings, so most sub-tasks, as well as error-checking, can be automated. In the end, the system could look like this:

Figure 1 — Custom made hooks can be set up in Git to automate sub-tasks and error checking.

Most components of this system are readily available as open-source software from multiple vendors. The only exception is the integration of the Git repository and the DNS server. I discovered only one project, named GitZone, which combines DNS server integration and Git repository management in one Perl script. I wanted something more lightweight that could be used with existing and well-tested Git repository managers like Gitolite. After some research, I decided to create my own lightweight and universal solution.

Introducing dzonegit

Project dzonegit is a set of Git hooks to automate management of DNS zones. It’s written in Python 3.5 and depends only on named-compilezone binary, that is part of the BIND name server.

In the pre-commit hook, it tries to compile all changed zone files. If they don’t compile, the commit is refused. If they compile, their serial numbers are compared against previous versions. If serial numbers haven’t been increased, the commit is refused as well.

Since all Git hooks are local to the repository instance, they cannot be enforced on a foreign repository. Therefore, similar checks have to be conducted on the DNS-server repository upon receiving commits. This is done by either a pre-receive or update hook provided by dzonegit. That ensures all zone files committed to the repository are valid DNS zone files and their serial number increases every time they are changed.

Another hook provided by dzonegit is post-receive. It is run after every update of the repository. First, it checks out the most recent revision of the repository to get zone files accessible for the DNS server. Then it generates configuration file snippets to be included into the DNS server configuration. For instance, if using the BIND server, each hosted zone requires a configuration snippet like this:

 zone "example.com" { type master; file "/path/to/example.com.zone"; };

In order to be a universal solution, these snippets are generated from a JSON template. For the example above, the template could look like this:

{
"header": "# Autogenerated by dzonegit on $datetime. Do not edit.\n",
"item": "zone \"$zonename\" { type master; file \"$zonefile\"; };"
}

The last step in the post-receive hook is to let the DNS server know that some zone files are changed. Two commands can be defined for that:

• zonereloadcmd, which is run for each changed zone and has the zone name appended as the last command line argument, and
• reconfigcmd, which is run whenever some zones are added or withdrawn.

All configurable parameters of dzonegit are stored as Git configuration options under dzonegit namespace. Using standard git config utility, they can be set either per-repository, per-user, or system-wide.

DNSSEC signing

In the early days of DNS Security Extensions (DNSSEC) technology, the only signing solution available was dnssec-signzone utility from BIND. It reads a zone file, signs it and outputs another zone file. Such a utility could be used manually before a zone file is committed to the Git repository, however, such a setup would require manual periodic signature refreshing.

Today, there are a few good implementations of automatic DNSSEC signing and key management like BIND, OpenDNSSEC, PowerDNS or Knot DNS.

Read: GeoIP in Knot DNS 2.7

In order to make the system as modular as possible, it’s a good idea to separate DNSSEC signing and key management to a dedicated functional block, in a setup known as ‘bump-in-the-wire’ signing. That means unsigned zones are transferred to the signer, signed there, and then transferred out to the public-facing secondary servers.

I have chosen Knot DNS as the signer; my arguments supporting this decision are basically similar to those explained by RIPE NCC a while ago. During testing, I discovered a few bugs in the implementation of key management. They were promptly fixed by the developers, but those fixes are available only in the newest version of Knot. Therefore, I strongly recommend using packages provided directly by developers, instead of possibly outdated distribution packages.

The setup of Knot DNS is pretty easy. In the main configuration file, we set up a zone template and signing policy:

 template:
       - id: default
         storage: "/var/lib/knot"
         zonefile-load: none   # do not use zone files at all
         zonefile-sync: -1
         journal-content: all  # keep whole zone contents in the journal
         master: primary       # reference to the server serving unsigned zone
         acl: acl_primary      # allow NOTIFY from the primary
         acl: acl_secondary    # allow AXFR from secondary servers
         notify: secondary     # send NOTIFY to secondary servers
         dnssec-signing: on
         dnssec-policy: ecdsa

policy:
       - id: ecdsa
         algorithm: ecdsap256sha256
         zsk-lifetime: 30d
         rrsig-lifetime: 30d
         rrsig-refresh: 15d
         nsec3: on

Then we include a config snippet, generated again by the post-receive hook of dzonegit — it is capable of generating more than one config snippet. In this case, we can make use of per-zone variables to use different signing policies for different zones, so we can, for instance, sign only selected zones or adjust signing policy per zone.

To share or not to share

In a web-hosting environment, at least in the Czech Republic, it’s quite common to use one set of DNSSEC keys for all hosted zones. For a large number of hosted zones, this dramatically simplifies operational workload. Also FRED, the open-source domain registry developed and operated by CZ.NIC, makes sharing of DNSSEC keys between domains easy by using so-called Keyset objects. These objects hold a public DNSSEC key, which can be associated with any number of domain objects. This is pretty different to commonly adopted solution defined in RFC 5910 where the DS records are submitted as attributes of each domain object. Since a DS record is made by hashing a public key and domain name, it cannot be shared between zones.

Last year, CZ.NIC deployed automated keyset management, so the secure delegation from the .cz registry can be established and kept in sync by in-band signalling defined in RFC 7344. This reduces the operational workload to keep secure delegations in sync to zero, no matter whether keys are shared or not.

From the perspective of key management, keeping a separate set of keys for each zone is easier. Unless you manage thousands of zones, it’s probably better to keep keys separate.

Automate all the things!

Automated management of secure delegation makes running DNSSEC easy. After proper initial setup, there is no periodic task to be taken care of. Unfortunately, there are only a few parent zones supporting automated secure delegation – .cz, .cr, .ch and .li. For other parent zones, Key Signing Key (KSK) rollover still requires manual intervention.

Ignoring the problem is obviously the easiest solution. With state-of-the-art DNSSEC based on the ECDSA algorithm with a 256-bit key, the keys are so strong they can be left unchanged for years.

Read: Measuring ECDSA in DNSSEC: An Update

Another solution is to use CDS/CDNSKEY records produced by the DNSSEC signing software as a universal signalling solution and implement automatic management for other registries on your own. Securing reverse records delegated from the RIPE NCC looks like the lowest-hanging fruit, as there is a pretty simple and well-documented REST API to read and update the RIPE database, which holds the secure delegation data. A similar API is available for the ARIN database. For APNIC, any changes to DS records have to be submitted via the MyAPNIC portal, which is quite problematic to automate.

I’ve created a proof of concept code that compares DS record data stored in the RIPE database with DS record data obtained by DNSSEC-validated queries for CDS records of the zone itself. If new record data is found, the database gets updated. During RIPE 77, this got some attention in the DNS working group, so maybe in the future, similar software would get implemented directly by the RIPE NCC. A similar approach can be used with other domain registries, provided you find a registrar that supports DNSSEC and has an API to allow DS record updates.

The biggest challenge is how to secure delegations from zones hosted at your own platform. This would require either some sort of automatic commits into the Git repository whenever DS records have to be changed, or some hidden intermediate step of adding DS records after checking out the Git repository, before the zone is loaded into the DNS server — perhaps some sort of smudge filter.

Editing DNS records is now easier

After a few months of testing, we switched the DNS provisioning system of all domains hosted by CESNET (Czech National Research and Education Network) to the presented solution at the beginning of September 2018. The transition to Knot DNS allowed us to migrate from shared RSA keys to independent ECDSA keys. The reduced workload and a safety belt made editing DNS records a much easier task for human operators.

There is still some work to be done. Everything is open-sourced, contributions are very welcome. Also, if you think automated updates of secure delegations are a clever idea, speak to your registry, registrar or raise your voice in the RIPE DNS working group or similar working group in your regional or national registry.

Ondřej Caletka is a network services administrator and developer at CESNET, National Research and Education Network of Czech Republic.

Rate this article

---

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.