Containers are a rapidly growing part of the IT infrastructure landscape. They’re a kind of ‘micromachine’, typically deployed in large numbers, and each with their own networking stack and IP address. They’re also typically deployed in highly-available swarms across multiple physical or virtual machines, requiring traffic between containers to be routed in, out, and around.
When you need lots of IP addresses, it’s natural to turn to IPv6, with its inexhaustible supply of IP addresses in each /64 subnet. Such a large subnet size also allows for sparse, probabilistic allocations, eliminating the need for centralized IP address management. Further, standard features like router advertisements can be used in smaller deployments to provide dynamic routing that allows automatic provisioning.
All of this makes IPv6 and containers sound like a perfect fit — and they really are. I’ve been running fully-containerized infrastructures on entirely IPv6-based internal infrastructures for the last couple of years, and it’s wonderful. I get a /48 block from my upstream provider, use one subnet for inter-machine communication, and give every machine its own /64, from which all the containers on that machine take their addresses. For routing, radvd is doing quite well advertising routes, and I know that if my routing needs become more complicated in the future, I can switch over to another standards-based routing protocol that happily supports IPv6.
This setup means that every container can talk directly to any other container, and, potentially, the Internet, if the firewall policy allows it. Eliminating layers of network address translation (NAT) simplifies the service discovery problem, because I don’t need to keep track of ports mapped from the host to particular containers. It really is a pleasure to manage this infrastructure’s network.
Unfortunately, all of these benefits require good IPv6 support from infrastructure providers, and the public cloud providers are woefully lacking.
None of the ‘big three’ cloud providers (AWS, Azure, and GCP) have an IPv6 feature set that is capable of reasonably supporting containers. Their support ranges from non-existent (yep, it’s 2018 and there are still public cloud providers with zero IPv6 support!) to ‘it’s just like IPv4, but with colons!’.
Smaller providers are similarly lacking, typically providing you with, at most, a handful of IPv6 addresses, or seemingly not being aware that IPv6 exists.
The most frustrating part of all this is that if you move away from the ‘IPv4 but bigger’ mindset, built-in IPv6 features make deploying a container-friendly IPv6 setup very practical.
DHCP Prefix Delegation, for instance, provides a straightforward way for a cloud instance to dynamically request a subnet (or block of subnets) for its own use. There’s no need to even have vendor-specific APIs to request subnets — just fire up 'dhcp6c' and include the one line of config that tells it to request a delegated prefix. It really is that simple.
Until the glorious day, though, when cloud providers wake up to the fact that IPv6 is here, and it is very useful, I’m strongly preferring to run my infrastructures in dedicated IPv6-friendly data centres (shout-out to Hurricane Electric!), and holding my nose and wrestling with NAT and overlay networks when I’m forced into the cloud.
One thing is for sure though: the first major cloud provider who provides a complete, container-friendly IPv6 network is going to be seeing a lot of my future business.
Watch Matt’s presentation on IPv6 and containers at LinuxConfAu earlier this year.
Matt Palmer is a beardy Unix guy who has been convincing computers to do things they would rather not do for a very long time. Part developer, part sysadmin, and part manager, he has seen a lot of things, and if you’re quick you can get away before he tells you all about all of them.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.
One additional advantage of IPv6 in this scenario is using RFC8273 (Unique IPv6 Prefix per host), so you can get a single /64 for the server, and have one address per each VM or container. No need to use DHCPv6-PD.