Secure Shell (SSH) is a popular network protocol for accessing a shell on (remote) systems. Users might not pay enough attention to detail when asked to verify the system’s authenticity on the first connection to a system, thereby risking a network-level security breach.
In this article, I will explain how SSHFP DNS records can help mitigate such risks and share the results of our large-scale analysis. At the time of measurement, only 1 in 10,000 domains used SSHFP records and less than 50% used a correct configuration.
My colleague and I from Technische Universität Berlin conducted some research on the SSHFP record type and its usage and security in early 2022. The resulting paper was presented at CANS 2022 and published soon after. I also provided a little sneak-peak at DNS-OARC 39. All code and data are already public to allow for further research.
SSH host key verification
Before we dive into our analysis and results, we need to understand what host key verification is and why it is important. When connecting to an SSH for the very first time, the OpenSSH client will ask the user to verify the authenticity of the server as seen in Figure 1.
There exists anecdotal evidence that many users just blindly answer ‘Yes’ without considering potential security risks. A network-based attacker might be able to perform malice-in-the-middle (MITM) attacks leading to credentials leaks or session-hijacking.
There are multiple methods for the host key fingerprint verification according to the SSH standard (RFC 4251):
- Manually, by the user
- Automated, via DNS
- Automated, via CA
With the manual process, the user needs to compare the server’s host key fingerprint (such as SHA256:jq3V6…1dc in Figure 1) against an out-of-band obtained fingerprint. This usually requires contacting the administrator responsible for the server.
Furthermore, this approach entails the risk of human error when comparing fingerprints.
Automated, via CA
RFC 4251 also touches on the possibility of using a certificate authority to verify host key fingerprints. However, this approach requires the deployment of a root-CA key to all user devices and signing all host keys with them. The former might not be achievable or suitable in certain scenarios and we, therefore, disregard this method in our research.
Automated, via DNS
Another approach that we analysed in more detail is the distribution of the host key fingerprints via a special resource record. This removes any user interaction, potential human error, or presumed configuration of a user’s device because OpenSSH-client can query the DNS and do the comparison. It only requires the administrator to publish a host’s fingerprints as a set of SSHFP records, which can be securely obtained (DNSSEC). If the latter holds, MITM attacks are mitigated, and overall security is improved.
SSHFP DNS records
RFC 4255 defines the SSHFP’s resource record schema. In representation format it looks like this:
SSHFP <KEY-ALGO> <HASH-TYPE> <FINGERPRINT>
The KEY-ALGO is a numeral representing the key’s algorithm (Table 1), whereas HASH-TYPE defines the used hash algorithm (Table 2). The fingerprint is the hash’s hexadecimal digest.
In the real world, the records will look like this:
$ dig SSHFP someserver.tld +noall +answer ; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.10 <<>> SSHFP someserver.tld +noall +answer ;; global options: +cmd someserver.tld. 3600 IN SSHFP 1 1 09F6A01D2175742B257C6B98B7C72C44C4040683 someserver.tld. 3600 IN SSHFP 1 2 4158F281921260B0205508121C6F5CEE879E15F22BDBC319EF2AE9FD 308DB3BE someserver.tld. 3600 IN SSHFP 3 1 91CAC088707D2C61D2F0FDA132D6F13CAE57BCD3 someserver.tld. 3600 IN SSHFP 3 2 65564C015FCA69E82E9B9CEF35380955720C2345E660C39176782E67 06E7FDD0 someserver.tld. 3600 IN SSHFP 4 1 F416A804701190D53B31C5A1EFC2F09104C6391B someserver.tld. 3600 IN SSHFP 4 2 C6DE2110F23A123691D49E94EA71DC18BD6F7277D7A7F9FC2E76F423 89DCAB70
As can be seen in the listing, someserver.tld has three different keys (RSA, ECDSA, ED25519) and a SHA1/SHA256 fingerprint for each of them. These records can also easily be generated using OpenSSH’s built-in tools:
$ ssh-keyscan -D someserver.tld ; someserver.tld:22 SSH-2.0-OpenSSH_8.9p1 Ubuntu-3 someserver.tld IN SSHFP 1 1 09f6a01d2175742b257c6b98b7c72c44c4040683 someserver.tld IN SSHFP 1 2 4158f281921260b0205508121c6f5cee879e15f22bdbc319ef2ae9fd308db3be ; someserver.tld:22 SSH-2.0-OpenSSH_8.9p1 Ubuntu-3 someserver.tld IN SSHFP 3 1 91cac088707d2c61d2f0fda132d6f13cae57bcd3 someserver.tld IN SSHFP 3 2 65564c015fca69e82e9b9cef35380955720c2345e660c39176782e6706e7fdd0 ; someserver.tld:22 SSH-2.0-OpenSSH_8.9p1 Ubuntu-3 someserver.tld IN SSHFP 4 1 f416a804701190d53b31c5a1efc2f09104c6391b someserver.tld IN SSHFP 4 2 c6de2110f23a123691d49e94ea71dc18bd6f7277d7a7f9fc2e76f42389dcab70 ; someserver.tld:22 SSH-2.0-OpenSSH_8.9p1 Ubuntu-3 ; someserver.tld:22 SSH-2.0-OpenSSH_8.9p1 Ubuntu-3
So, we believe that deploying host key verification fingerprints in the DNS is almost trivial. Also, an update is only required should the host keys change. However, the SSHFP records must reach the client in a secure fashion, otherwise, the MITM could happen on the DNS instead of the SSH layer.
Finally, OpenSSH-client needs to be told to use DNS for fingerprint verification using the -o VerifyHostKeyDNS=yes option. Although this feature has been implemented in OpenSSH for many years, it is unfortunately still not enabled by default. The debug output tells us what will happen in the background:
$ ssh -o UserKnownHostsFile=/dev/null -o VerifyHostKeyDNS=yes -v someserver.tld g 2>&1 [...] debug1: Connecting to someserver.tld [IP.IP.IP.IP] port 22. debug1: Connection established. [...] debug1: Server host key: ssh-ed25519 SHA256:xt4hEPI6EjaR1J6U6nHcGL1vcnfXp/n8Lnb0I4ncq3A debug1: found 6 secure fingerprints in DNS debug1: verify_host_key_dns: matched SSHFP type 4 fptype 1 debug1: verify_host_key_dns: matched SSHFP type 4 fptype 2 debug1: matching host key fingerprint found in DNS [...]
As can be seen, OpenSSH establishes a connection to the server, obtains the server-side host key fingerprint, then queries the domain for SSHFP records and compares them. It confirms that the fingerprints are secure and matching, thus continuing with the connection.
In order to measure the prevalence of SSHFP records on the Internet, we first built a measurement system that works as follows:
- Query a domain for SSHFP records.
- If SSHFP records exist, query the domain for their A records.
- If A records exist, try to obtain the server-side host key fingerprints using ssh-keyscan.
- Compare SSHFP fingerprints and server-side fingerprints to find (mis)matches.
- If matches exist, repeat the SSHFP query, but with a DNSSEC-validating resolver.
We ran this analysis on two different sets of domains:
- Tranco 1M domain list
- >515M domains from certificate transparency logs over the course of 26 days.
Our paper dives deep into the details of the results, but the most interesting parts in my opinion are:
- Only 1 in 10,000 domains have an SSHFP record set.
- Less than 50% of the records are secured with DNSSEC.
- In many cases not all KEY-ALGO and HASH-TYPE combinations exist as SSHFP records (or are outdated), thus leaving it to chance whether a DNS-based verification is successful or a fallback to other methods (that is, manual verification) is done.
Although there is room for improvement when looking at the adoption of SSHFP records, a 50% mark of secured records is unfortunate, because in such setups there is no security benefit, as the MITM-attack shifts from the SSH layer to the DNS layer.
There might be a ‘chicken-and-egg’ situation between SSHFP and DNSSEC here — one requires the other, but the other is not as widely adopted as it could be. However, internal usage of SSHFP records is not measurable from the outside, so the actual number might be somewhat larger.
Nonetheless, we believe that proper deployment and usage of SSHFP records could provide security benefits at little cost in many situations.
Here’s a three-point action plan if you want to start using SSHFP records:
- Enable DNSSEC for your domain(s).
- Collect and deploy the SSHFP records for all your SSH-accessible systems using ssh-keyscan.
- Add VerifyHostKeyDNS=yes to your ~/.ssh/config.
If you are interested in more details and in-depth analysis, have a look at our paper.
Sebastian Neef is a PhD candidate and Research Assistant at the Technische Universität Berlin’s ‘Chair for Security in Telecommunications’, with a focus on network and web security.
This post is based on the accepted and soon-to-be-published paper “Oh SSH-it, what’s my fingerprint? A Large-Scale Analysis of SSH Host Key Fingerprint Verification Records in the DNS” (Sebastian Neef and Nils Wisiol, preprint).
Continue the conversation on this topic at Orbit.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.