Content Distribution Networks (CDNs), such as CloudFront and CloudFlare, are used to improve the performance and security of public-facing websites. Features of CDNs can include IP firewalling, client and server authentication, and Web Application Firewall (WAF) filtering for protecting origin (backend web application) servers. These features present obstacles for an attacker when trying to exploit security vulnerabilities that may exist in the underlying application.
Restricting the ability of attackers to bypass the CDN and access the origin server is critical to the effective implementation of the security features CDNs offer. Public origin access is a common misconfiguration as any request sent directly to the origin will not be filtered by the CDN.
A common mitigation is to implement IP allow-listing from the CDNs IP range. In this post, I’ll focus on similar attack vectors that are introduced when attempting when using this mitigation. Since CDN IP ranges are shared by all customers, IP allow-lists are insufficient to restrict access to the origin when traffic traverses the intended CDN network through a secondary account.
To demonstrate this issue, I created two tools: cdn-proxy and cdn-scanner.
- cdn-proxy deploys the required infrastructure for this attack.
- cdn-scanner uses the infrastructure of cdn-proxy to scan for affected origins.
In-depth information on these tools can be found on the cdn-proxy GitHub repository.
How the attack works
This attack is straightforward if you understand that CDNs operate above layer 4 of the Open Systems Interconnection (OSI) model. At this layer, TCP connections are terminated on the CDN network and separate TCP connections are then made to the origin, acting as a proxy for client requests. From the origin perspective, the source of these requests is IP ranges from (and common to all customers of) that CDN.
With this in mind, consider the following attacker scenario:
- The IP address of the targeted origin has been identified.
- A new account using the same CDN as their target is created.
- Security features are disabled, and the origin is configured to point to the IP identified previously.
With this new account, the attacker avoids many of the security restrictions present on the original. In general, CDNs are unable to prevent this attack when IP allow-listing is relied on due to the following reasons:
- Security features implemented by the CDN are applied only to requests that pass through the intended account; a secondary account on the same network is not subject to the same security restrictions as the original.
- CDNs do not restrict the origin configuration of an account and instead rely on the customer to block unintended access at the origin.
Exploitation conditions
All the following conditions must be met for this attack to work:
- Preventing access to the origin is done solely with IP allow-listing.
- The IP address of the origin is known to the attacker.
IP-based access control is a common, often considered ‘best’ practice. The second condition presents a hurdle, but a lower bar than one might initially believe. See the ‘What does this enable? – origin scanning’ section that covers automated techniques for discovering and verifying origin IPs when direct access to the origin is blocked.
What does this enable?
What is possible depends on the features the CDN grants to any one customer but in many cases means a complete bypass of the hosting CDN’s security features, including WAF, IP restrictions, rate limiting, or authentication.
For example, the WAF can be disabled in both CloudFlare and CloudFront in the account, making it vulnerable to this attack when filtering is relied upon at the CDN. However, the WAF can be attached to Application Load Balancers (ALBs) rather than the CDN in CloudFront. In this case, disabling the WAF in the attacker’s account will not work because requests will still be filtered at the origin.
CDN features often get added or changed in various ways and are not guaranteed to not change the impact of this attack. While the impact may depend on implementation, it is safest to assume the attacker has full control over their own requests to the origin. Below are some additional ways (beyond bypassing WAFs, IP restrictions, rate-limiting or authentication) this attack can be abused in some instances.
What does this enable? Origin scanning
One of the more significant prerequisites for successfully implementing this attack is determining the IP of the origin. However, many techniques exist even when the origin is not directly accessible.
For a good case study on the difficulty of finding origin IPs, consider reading Maneuvering Around Clouds: Bypassing Cloud-based Security. The research covered in the paper found that about 70% of analysed domains exposed their origin IPs, and accounting for attacks that required direct access, the number described in the report appears to be closer to 60%.
However, a problem with automated tools like this is sorting out false positives when direct access to the origin is impossible. The cdn-proxy tools partially sidestep this issue by updating the origin on the fly for each request, making it possible to detect origins relying on the IP allow-web mitigation. For example, using CloudFront, a list of arbitrary IPs can be scanned from the shared IP range as fast as the CDN will accept requests. The same is also possible through CloudFlare, however, with a considerably slower throughput. For more details on how this works, see the cdn-proxy documentation.
What does this enable? IP spoofing
The X-Forwarded-For (XFF) HTTP header presents a unique opportunity for spoofing IPs if an attacker can fully control requests to the origin. For an overview on XFF, Adam Pritchard’s blog post covers it comprehensively. I highly recommend reading that if you are not familiar with XFF or are interested in all the ways it can go wrong.
In the case of CloudFront, this alternate domain routing attack introduces an additional way for an attacker to manipulate the XFF header when the request is passed to the origin. In many cases where the CDN can be configured to add, modify, or pass through the XFF header to the origin, spoofing any IP will be possible.
There are no limitations on what IP can be spoofed, or how often this IP can be changed. An attacker can set a random IP for every request as demonstrated by the cdn-proxy. This means in addition to bypassing security restrictions at the CDN, it may be possible to do the same in the web application itself.
The feasibility of this attack, like the others mentioned earlier, is dependent on the features exposed by the CDN to any given customer as well as the customer’s configuration. For example, this works well with CloudFront when the customer uses the XFF header to determine the client’s IP. CloudFlare on the other hand does not allow full control over this header for any requests passing through their network, effectively mitigating this type of IP spoofing.
Bypassing CDN WAFs with alternate domain routing
In the video below, I show the process of bypassing restrictions placed on the admin panel of a WordPress site by using the cdn-proxy tools.
Our demo WordPress uses CloudFront and AWS WAF to deny access from all IPs to the path /wp-admin.php. Any user (or attacker) that browses our admin login panel will receive a custom ‘403: Can’t touch this’ message returned from AWS WAF.
We’ve used due diligence to ensure that an attacker can’t access the server directly to bypass our restrictions on /wp-admin.php in AWS WAF. The firewall on the origin server is implemented using IP allow-listing mitigation to restrict access from non-CloudFront IP ranges.
Attack mitigation options
While I focused on CloudFront and AWS WAF in this example, this misconfiguration can arise on any CDN that uses a shared pool of IPs for origin requests.
In the case of AWS CloudFront, the documentation provides recommendations for restricting access to the origin when using an Amazon S3 bucket or Application Load Balancer (ALB). In other cases, you can use an approach similar to what is recommended for ALBs, setting a unique header at the CDN that can be checked for and enforced by the origin.
In the case of CloudFlare, the documentation has several recommendations for limiting access to the origin. While all of these are important, the ones that will prevent this specific attack are CloudFlare Tunnels or Authenticated origin pulls.
If you decide to use CloudFlare Authenticated Origin Pulls, it is important to note there are three different ways to configure the client certificate. One of these options uses the same client certificate across all clients (Zone-Level Authenticated origin pull using Cloudflare certificates) and will not protect against this attack. When setting up Authenticated origin pulls, ensure you use one of the following options:
- Zone-Level Authenticated Origin Pull using customer certificates
- Per-Hostname Authenticated Origin Pull using customer certificates
Inspiration and other work
Aidan Steele initially tweeted about the idea behind this attack back in 2020, which is why I was aware of the issue when I started working on this project:
I also want to thank my co-workers at Rhino Security Labs as well as Kubilay Kocak for providing feedback and review of this work (I still claim responsibility for any errors, of course).
Interestingly, some work similar in implementation was done by Mingkui Wei for Usenix and the Tor Project. That work approaches the same idea from a privacy perspective rather than security. Personally, I find it interesting that the same concept can have very different results depending on how you think about it.
Around the time I published this, I came across Adam Pritchard’s thorough post on the XFF and reached out to him about this issue. Read his notes on how this affects the X-Forwarded-For (XFF) header on CloudFlare. In short, CloudFlare treats this as a protected header, so the IP spoofing attacks described in this post, in general, will not work on CloudFlare if you are using that header. Please do not rely on this though, as Adam said in his post: “Always do strong verification of your CDN!”
Lastly, but very importantly, Dhruv Ahuja helpfully pointed out that CloudFlare has three types of Authenticated origin pulls, and not all of them will protect against this attack. I added this information to the mitigations section.
Ryan Gerstenkorn is an independent offensive security researcher with a background in DevOps, focusing primarily on Cloud infrastructure.
This post was originally published on Ryan’s Blog.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.