OTE Group (AS6799) is the largest ISP/Telco in Greece and one of the leading telecom groups in Southeast Europe with 2.6 million fixed-line, 1.8 million broadband, and 8 million mobile customers.
In an effort to mitigate the impending effects of public IPv4 address pool exhaustion, the engineering team undertook a feasibility study in 2014 on moving towards an IPv6-only residential service — IPv4 was to be treated as a service over the IPv6 network.
The desired characteristics of an IPv6-only service were: its stateless nature, distributed deployment, the possible complete removal of IPv4 in the future, the use of network function virtualization for scalability and cost efficiency, and a simple design.
Having considered the transition mechanism work that had been undertaken by the IETF, MAP-E (RFC 7597) and Lightweight 4over6 (RFC 7596) were recommended as the appropriate means to mitigate the address exhaustion issue. Being a part of Deutsche Telekom Group (DT), Lightweight 4over6 (lw4o6) was chosen due to DT’s involvement with the technology.
Below is a summary of a presentation we gave at RIPE 76 recently, including a brief explanation of the technology, challenges we had with deploying it to the network, and ongoing work.
Lightweight 4over6 overview
lw4o6 is an extension to DS-Lite which moves the Network Address and Port Translation (NAPT) function from the centralized DS-Lite AFTR to the tunnel client located in the Customer Premises Equipment (CPE).
This removes the requirement for a Carrier Grade NAT (CGN) function in the tunnel concentrator and reduces the amount of centralized state that must be kept to a per-subscriber level. Port-restricted, shared, public IPv4 addresses are allocated to the CPEs along with necessary parameters, via DHCPv6.
Read: lw4o6: One-step further than dual-stack networks
See the figures below for descriptions of incoming and outgoing IPv4 and IPv6 traffic.
IPv6 traffic leaving the network follows the existing native path, and all outgoing IPv4 traffic is encapsulated in IPv6 and directed to a tunnel terminator (lwAFTR) where decapsulation is performed (Figure 1).
Figure 1 — IPv6 traffic leaving the network follows the existing native path, and all outgoing IPv4 traffic is encapsulated in IPv6 and directed to a tunnel terminator (lwAFTR) where decapsulation is performed.
Figure 2 — Incoming IPv4 traffic hits the lwAFTRs where it is encapsulated in IPv6 and forwarded to the end-user.
lwAFTR VNF
lwAFTR is the softwire termination point and performs only encapsulation/decapsulation of traffic based on binding table rules. Since it performs no NAT function, it is considered a stateless function (strictly speaking, it maintains a minimal state by means of a binding table).
The lwAFTR maintains a binding table containing the binding between the CPE’s (lwB4) IPv6 address, the allocated (public) IPv4 address, and the restricted port set.
As part of our deployment, we issued a RFQ process about an lwAFTR VNF running on COTS H/W that could accommodate multi-10 Gbps traffic with predictable performance as a deliverable. No solutions were mature at the time (very short time after the RFC publication) so development was done alongside the RFQ process.
The selected solution uses an open source, fast data plane networking toolkit called Snabb. It follows the user-mode networking approach; Ethernet I/O with no kernel overhead (‘kernel bypass’ mode). lwAFTR deployment is packaged as a Docker container running on Linux commodity servers.
Deployment at production network
With around 32,000 active users in production and 12 Border Network Gateways (BNGs) at the time of writing, the deployment to the production country-wide IP network involved many components, including:
- CPE support (focusing on the one with the highest user penetration)
- BNG configuration (we requested several DHCPv6 features from our vendor)
- RADIUS configuration per user (we use freeradius open source software; we control all aspects of the deployment)
- Central DHCPv6 in two data centre locations in Athens (using ISC dhcpd)
- lwAFTR in the same data centre location (the stateless nature of lwAFTR, allows for a resilient anycast deployment)
- Monitoring/measurements (we use Juniper’s open-nti open source software)
- Provisioning/automation scripts (developed internally in Python/Perl)
Challenges
This was a multi-year project with several challenges:
- We faced issues on the CPE implementations and we still have a few cases that remain unresolved.
- There were some lwAFTR-related performance issues, resolved with the help of our vendor — lwAFTR performance improvement is ongoing.
- In terms of DHCPv6, our deployment’s scalability remains unknown.
- We also faced some expected issues with users having services over IPv4 (mostly IP cameras).
On the plus side, no dependency exists between IPv4 and IPv6 addressing, which allows for an incremental deployment approach. No noticeable MTU/fragmentation issues were observed and our anycast lwAFTR deployment provides flexibility on traffic routing.
Finally, capacity increases in the lwAFTR setup can be easily achieved.
Future work
A significant amount of work is pending. We need to globally expand the lw4o6 service in all our BNGs and lighten/offload the current CGN setup.
We also need to improve our central DHCPv6 deployment that will also enable ‘static’ IPv6 prefixes to subscribers. Our radar view also includes the implementation of different classes of service (different-sized port range sets), standardized provisioning via YANG, and exposing the IPv4 port ranges of users. These would be part of a future blog post.
We are actively seeking cooperation with other operators in similar service setups.
Contributor: Yannis Nikolopoulos
Kostas Zorbadelos is a Senior Systems and Network Engineer in the IP Network Engineering Division of OTE Group (Greek Telecommunications Organization).
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.