It’s time to replace MD5 with TCP-AO

By on 28 Jul 2021

Category: Tech matters

Tags: , , ,

Blog home

Over the last decade, there has been much discussion on the topic of replacing MD5 when authenticating control plane protocols such as Border Gateway Protocol (BGP), Label Distribution Protocol (LDP), or any other long-lived Transmission Control Protocol (TCP) session. The long-expected MD5 successor — TCP Authentication Option (TCP-AO) — has only recently become viable since multiple routing equipment vendors have implemented the feature.

The TCP authentication story started in 1998, when Andy Heffernan authored RFC 2385, which describes how to protect BGP speakers against TCP reset (RST) attacks by leveraging the MD5 Signature Option. Let’s quickly recap what a TCP reset attack looks like.

Why do we need TCP security?

Imagine an Internet Exchange Point (IXP) peering LAN with three connected routers — A, B and C. Routers A and B maintain a TCP-based BGP session and are exchanging routing information over that session. The operator of router C is a competitor to A and B and has malicious intentions; they are looking for ways to disrupt the communication between A and B.

Obviously, the operator of router C could have configured a wrong interface IP address by accident as well. Not all hijacks are intentional; so-called ‘fat fingers’ happen way more often than intentional hijacks on IXP.

In this case, the operator of router C chooses to conduct a TCP reset attack against router B by sending packages to it with router A spoofed source address using destination port TCP 179, having the TCP RST flag set and guessing the sequence number which they obviously don’t know because they weren’t part of the communication thus far.

Image: TCP segment header
Figure 1 — TCP segment header with RST flag highlighted. Source.

Initially, nothing will happen, as router B will discard the packets because the sequence numbers won’t match its expectations. However, at some point, router C will guess the sequence number right and router B will accept the packet. As the RST flag is set in the header, it will reset the TCP session. As a result, the BGP session between routers A and B will go down, which could result in serious traffic interruptions.

Authenticating TCP sessions with TCP MD5

This problem was already well understood and mitigated back in 1998 with the introduction of the TCP-MD5 option, which is still very widely deployed in many networks today.

To quote RFC 2385, MD5 works as follows.

Every segment sent on a TCP connection to be protected against spoofing will contain the 16-byte MD5 digest produced by applying the MD5 algorithm to these items in the following order:

  1. The TCP pseudo-header (in the order: source IP address, destination IP address, zero-padded protocol number, and segment length)
  2. The TCP header, excluding options, and assuming a checksum of zero
  3. The TCP segment data (if any)
  4. An independently-specified key or password, known to both TCPs and presumably connection-specific.

Upon receiving a signed segment, the receiver must validate it by calculating its own digest from the same data (using its own key) and comparing the two digests. A failing comparison must result in the segment being dropped and must not produce any response back to the sender.

RFC 2385

Today, MD5 authentication still works well. However, over time, some of the requirements have changed and in 2004, MD5 was declared deprecated due to security concerns.

New requirements

As with every (pre-shared) key-based security mechanism, operators will want to change the key every now and then. MD5 does not allow for dynamic key rollover and because of that, changing the key means the TCP session will reset. In practice, it turned out that some keys were never changed as no one wanted to break that ‘super important’ BGP peering session.

Additionally, the TCP MD5 option does not support multiple cryptographic algorithms to compute the Message Authentication Code (MAC), which is used to authenticate a segment and its headers. As TCP MD5 option only supports the MD5 cryptographic algorithm, there was a need to support newer and multiple authentication algorithms.

The TCP Authentication Option

At NANOG 37 (2006), Ron Bonica presented Authentication for TCP-based Routing and Management Protocols during which he proposed for TCP MD5 to be replaced with something that would cater to the new requirements offering hitless key-rollover and support for stronger cryptography while keeping the good parts of TCP MD5, such as protection against TCP header attacks.

Roughly six years later, RFC 5925 ‘The TCP Authentication Option‘ was published. According to the abstract:

“This document specifies the TCP Authentication Option (TCP-AO), which obsoletes the TCP MD5 Signature option of RFC 2385 (TCP MD5).”

RFC 5925

Read: What are the best practices for the Internet?

The concept behind TCP-AO is very similar to TCP MD5 but with a couple of added features:

  • Each node is configured with one (as in MD5) or multiple (new in TCP-AO) Master Key Tuples (MKTs)
  • Each node derives traffic keys from each MKT — The traffic keys and an authentication algorithm (multiple supported in TCP-AO) are used to calculate the Message Authentication Code (MAC)
  • The MAC is included in each TCP segment — The TCP-AO Signature Option includes MAC, KeyID (key for the current package) and RNextKeyID (the key used for the next packet, this allows for hitless key-rollover)
  • The receiving node calculates the MAC for each received TCP segment using the configuration specified algorithm and traffic key associated with the received KeyID
  • The receiving node discards the packet if the calculated MAC does not match the received MAC

Now what?

The migration plan is relatively simple: Upgrade the routers in your lab to a version that includes the TCP-AO code and then configure them using these example configurations.

If you don’t have a lab with hardware available, you can find the TCP-AO code in virtualized routing appliances, including Juniper Networks’ vMX Virtual Routers.

In the meantime, speak with your peers, transits, and customers. Spread the word and ask them to upgrade to a version that supports TCP-AO, as well. After successfully testing, start authenticating your EBGP sessions with TCP-AO. Of course, there’s no issue even if your peers don’t support TCP-AO yet, TCP MD5 is still supported!

Melchior Aelmans is a Consulting Engineer at Juniper Networks, where he has been working with many operators on the design and evolution of their networks.

Rate this article

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.

Leave a Reply

Your email address will not be published. Required fields are marked *