Delivering emails is not a straightforward task. As Figure 1 shows, the number of email-related standards has more than quadrupled over the past two decades.
The fundamental protocol for email delivery, Simple Mail Transfer Protocol (SMTP) was specified in 1982 and originally built without authentication. Anyone could relay emails through a Mail Transfer Agent (MTA). This led to an increasing amount of unsolicited bulk emails also known as SPAM.
A lot of today’s standards try to overcome the gaps that SMTP was originally not designed to fulfil. However, in a world of ubiquitous networking, we cannot redesign the email ecosystem from scratch. Email providers have to decide which standards to implement and which configurations to use. We put a lot of trust in providers to make the right choices to store and exchange emails in a secure manner.
In this post, we will look at how providers do this and how you can verify whether their setup works according to their promises.
Email-related standards
My fellow researchers from the Max-Planck Institute for Informatics, University of Vienna, TU Wien, and SBA Research run an open SMTP and DNS testbed (Figure 2) that allows you to verify your provider’s email delivery capabilities or your own setup in the case of self-hosting email.
Why does this testbed include different DNS setups when testing email delivery? A lot of standards use DNS as a second channel to exchange information during email delivery. DNS requests are both used by outgoing and receiving email servers. The outgoing email server relies on DNS to perform default lookups such as MX, A, and AAAA, but also to verify if the receiving email server supports encryption before initiating a STARTTLS connection with DANE or MTA-STS.
The incoming server relies on DNS to verify if the sender is valid via SPF, DKIM, or DMARC. To guarantee the integrity and authenticity of DNS responses, DNS Security Extensions (DNSSEC) were introduced. Next to different setups for our authoritative DNS servers, our setup includes several email servers. The underlying network layer is undergoing a change from IPv4 to IPv6. Successful email delivery depends on email servers supporting the same IP version. The ongoing fight against SPAM has led to greylisting, that is, responding with a temporary error to unknown senders — a valid sender will then try to redeliver the email.
With the introduction of STARTTLS, SMTP was extended to enable transport encryption for emails. Email servers have to support at least one common TLS version and cipher combination in order to succeed. In case of any error, the connection will fall back on plaintext communication. Also, the sending email server does not know in advance if the receiver supports encryption or not.
This was heavily exploited by stripping the STARTTLS option from the server options. A total of 96% of emails from Tunisia to Gmail were found to be downgraded. While transport encryption is currently mostly opportunistic (expired or self-signed certificates), an increasing amount of email servers support strict TLS configurations.
For servers with strict TLS configurations, an error during the TLS handshake or certificate mismatches will cause the non-delivery of emails. However, the advantage is that strict TLS configurations are actively preventing downgrade attacks and thus protect your email communication. Our testbed enables keeping track of these protocols.
Let’s examine five current challenges in email delivery.
1. DNS resolution
We found that a lot of providers rely on closed DNS resolvers for their email servers, that is, Gmail does not rely on Google’s public DNS infrastructure, but uses different resolvers that do not validate DNSSEC. Related work found that closed resolvers are more likely to run older software versions and, while scarce, some are still vulnerable to the Kaminsky Attack by relying on non-random source ports.
When relying on a closed resolver, make sure to always keep it up to date! When outsourcing email, make sure to verify how the provider handles DNS resolution. Our testbed can be used to test a provider’s DNS resolution. It includes the organization name for the DNS resolver, verifies if the resolver performs DNSSEC validation, and tests IPv6 capabilities.
2. Security while keeping reachability
Next to the discrepancy between public and closed resolvers we also see a difference between larger and smaller providers. By ranking email providers based on passive DNS, we separate larger from smaller providers. We found that larger providers prioritize delivery, that is, are less likely to implement strict security mechanisms. While 68% of large providers do not validate DNSSEC, we found that only 32% of smaller providers do not.
3. IPv6 support
A common argument against IPv6 support is that IPs are easier to acquire in IPv6 than in IPv4. Thus IP-based reputation services have to be adapted for IPv6. However, this argument is only valid for receiving emails. Sending emails over IPv6 is not affected by these restrictions. We currently see that email is lacking behind DNS in IPv6 support. While more than 60% of providers are able to resolve DNS records over IPv6, only 40% of email servers allow delivery to IPv6 destinations.
4. Spam
Our testbed enabled us to measure the email delivery capabilities of spammers. We re-registered 50 expired domains and pointed them to our measurement addresses. We then were able to monitor changes in spam volume in comparison to our IPv4 plaintext measurement address. We found that more than two-thirds of spammers rely on RFC-confirm software and try to redeliver emails in case of an error. However, greylisting can still be considered effective as it reduced the initial spam volume by 37% and only affects emails from unknown senders.
While we did not expect a high volume for our IPv6-only measurement targets, 46% of spam volume was seen on our IPv6-only resolvable target. This indicates that several spammers rely on public resolvers for DNS resolution.
In general, the amount of spam for our IPv6 measurement target was very low. Only 7% of the original spam volume was measured for IPv6. Another option to reduce spam would be to enforce TLS. A lot of spammers (66%) failed to perform TLS handshakes, as they are usually not required by receiving email servers. This would also affect legitimate emails from servers, which do not support TLS. However, thanks to opportunistic encryption, TLS adoption in email has reached a steady state. We find less than 10% of regular providers still do not support transport encryption.
5. Strict transport security
With 90% of regular providers supporting transport security, opportunistic encryption for email delivery showed great success. The high adoption rate raises the question of whether TLS can be enforced or not. Each organization should decide for itself if the benefit of avoiding unencrypted email delivery outweighs the loss of some destinations.
The next question would be if these destinations are of any value to the organization, and if yes, how exceptions for specific targets can be made. Opportunistic encryption leaves email delivery vulnerable to spoofing and Man-in-the-Middle (MITM) attacks.
For enforcing TLS on the email server, DANE and MTA-STS are two solutions that can be used to protect email delivery to your domain and prevent these attacks. However, if DNSSEC is already supported, DANE should be the preferred option. It relies on DNSSEC to safely put certificate information in the DNS. This way a sender can request if encryption is available by the receiver before initiating a STARTTLS connection and is also able to verify the certificate presented by the receiving email server.
In order to measure support for DANE we created a measurement address with a TLSA record not matching the server’s certificate. With a fifth of providers validating the DANE mismatch, we can derive that currently, a fifth of email providers in our data set already support DANE. MTA-STS is the recommended solution for domains for which DNSSEC is unavailable.
How to participate
Measure your email provider/server! We are always looking to extend the set of measured email providers. Especially the heavy tail of smaller providers is of interest to us.
For a more in-depth look at email standards and the related measurements, read our paper.
The first part of our testbed is available here. We are currently working on documenting the rest, so you can also run it yourself.
Florian Holzbauer is a researcher at SBA Research and PhD student at the University of Vienna focusing on network security and network infrastructure analysis. He is interested in measuring the adoption and compliance of Internet standards.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.