The rise of cloud providers has enabled users to quickly allocate and release resources such as compute-power, storage, virtual machines (VM) and also IP addresses. Additionally, with the progressing exhaustion of IPv4, trading and exchanging IP address blocks has become more frequent.
In this article, you will learn how attackers can abuse IP address resources after you have released them, by using forgotten DNS records in your domains’ zones. We call these attacks ‘IP-use-after-free’ attacks and we show how much potential for their abuse exists for IP addresses managed by cloud providers alone.
What exactly is an IP-use-after-free attack, and how does it actually work? Think about setting up a test VM with your favourite cloud provider, for example, to test out a new version of the Content Management System (CMS) that you are using. Let’s assume you are setting it up as ‘test.example.com.’ So, you click yourself a small VM instance on a cloud provider, and along with it, an IP address, like 198.51.100.23. You then point the test A record from your zone example.com to this address:
test 3600 IN A 198.51.100.23
You go about testing and find that the new version of the CMS fulfills your every need. You shut down the VM instance, deploy the CMS to your production environment, and call it a day.
The issues only start to appear when somebody else creates a VM instance with the same cloud provider, and also receives the same IP address to which test.example.com still points. If the person’s intentions are benign, nothing bad will happen. An attacker, however, suddenly has a multitude of malicious options.
Really? Stale DNS records?
The natural question is whether people are actually forgetting to delete their DNS records that point to cloud setups (or other IP addresses that a third party may be able to gain control over. In this article, we focus on cloud setups because they provide a lower bound and are easier to measure accurately).
To better understand this problem, we rely on the Farsight dataset to extract a list of domains that are actively being looked up by clients across the Internet. We then resolve these domains ourselves, and if they point to IPv4 addresses that belong to commonly known cloud providers, like Amazon EC2, Digital Ocean, or Microsoft Azure, we then check whether they are reachable via ICMP or any of the most used ports (see Figure 1). If a host is responsive, we mark it as being online. If it was not, we consider that the IP address was up for grabs.
Naturally, ingress firewall rules could prevent our test from succeeding and, thus, our estimation is an upper-bound. One might expect it to be a gross over-approximation because cloud VM instances have traditionally received public IP addresses. Nowadays, however, this is not necessarily the case: cloud instances that do not need a public IP address can and generally do live in cloud-only internal networks. Furthermore, by default, many cloud instances respond to ICMP ping requests or allow for secure shell (SSH) access via TCP on port 22.
Figure 1: Ports and protocols used for IP address liveness checking.
Using our methodology, we discovered 702,180 unique domains that are vulnerable to IP-use-after-free attacks in major cloud providers alone!
Can you get the IP address you want?
An IP-use-after-free attack can only be successfully launched by an attacker if they are able to allocate the specific IP address that the domain name still points to. This means that an attacker needs to allocate a specific IP address within a cloud provider’s large space of addresses. Therefore, we need to understand if the address allocation strategies are random or follow an exploitable pattern.
We put this to the test. We allocated 10 (new) elastic IPv4 addresses at a major cloud operator and then repeatedly freed and allocated new addresses, thus we churned through their IP address pools (for each availability zone).
As you can see in Figure 2, it hardly took us more than a day to churn through whole address pools, and at most up to two weeks. Considering that we limited our allocation to two addresses every second, a real attacker could be significantly faster: for some providers, the limit is 10,000 API requests per second. At the same time, an attacker does not incur significant cost, as long as the allocated IPs are not actually used (and released quickly after allocation). In fact, our experiments cost us less than USD 50.
Figure 2: Time passed between the allocation of the same IP address.
Is this practical?
To understand if the attack is really practical, we put it to the test and attempted to exploit it in the wild in a controlled experiment (because we did not want to take over other operators’ domain names). We set up a cloud VM instance with one cloud account, and then pointed a DNS record to the host’s IP address. We then shut down the VM and used a different cloud account to take over the IP address. With an allocation rate of just two IP addresses per second, we successfully gained control over the right IP address within 27 minutes and 55 seconds. We then proved ownership of the domain name and obtained a corresponding TLS certificate, which is tracked in CT logs.
Yet another academic attack…
Considering the problem and attack more broadly, we find that the attack is not purely academic, but that it occurs in the wild. In fact, just recently at the end of September 2018, the PowerDNS foundation was hit by such an attack: the DNS record for web-rtc1.powerdns.com pointed to an IP address at one of their hosting providers, even though the service itself was discontinued. An attacker realized it, and once they were allocated the same IP address, quickly obtained a valid TLS certificate from Let’s Encrypt. Subsequently, they started exploiting residual trust in the domain name and served malware.
For more details, you can find their post-mortem on the PowerDNS Twitter account.
What can we do about It?
There are multiple lessons here. First, you should always delete stale DNS records. If you stop controlling the resource your domain name points to, you should remove the record. In addition, you might want to enable a service like Google Search Console across your domains to be alerted as soon as something happens.
Second, cloud and other providers utilizing IP addresses might want to subscribe to a service like the Farsight Security DNS channels to see which names (still) resolve to de-allocated and unused IP addresses in their network, to then inform customers of these misconfigurations. Finally, we are currently working on an IETF draft to describe precautions that certificate authorities can take to reduce the chances of issuing a certificate in such takeover cases.
For more details on the attack, and a possible defence for certificate authorities, our academic paper ‘Cloud Strife: Mitigating the Security Risks of Domain-Validated Certificates’ appeared at the Internet Society’s 25th Network and Distributed Systems Security Symposium (NDSS) and is available here.
Tobias Fiebig is an assistant professor in the Information and Communication Technology section at the faculty of Technology, Policy and Management of TU Delft.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.
Don’t most services wait at least a couple of days before reassigning an IP address? At least, I believe that was the case a few years ago.
Well, depends on the service. And this does not really help you when you forget to delete the record for days.
Great article, Tobias! We conduct apex-level audits of corporate domain portfolios of the F1000 and find virtually no company without dozens (if not hundreds) of domains lacking start of authority records…which leads to the dangers you outlined.
It seems that when allocating a new IP, cloud providers cycle through the same few IPs per account, which severely limits this attack. Was that the case in 2019?