Developing a common interface between transport and application layers

The last decade has seen a revival of innovation in the Internet’s transport
layer. For example, the rise of mobile devices that are often connected to the
Internet via multiple interfaces led to the deployment of multipath TCP. The development and standardization of the QUIC transport protocol as the basis for HTTP/3 has already seen significant deployment in the Internet as well.

The lowest common denominator in interfaces between the application and
transport layer, however, remains Berkeley sockets. While Berkeley sockets
do provide abstractions for services provided by the transport layer, the
different APIs for different transports do not necessarily behave consistently.

This results in a binding between an application and the transport protocol(s) it uses being made at design time, instead of through an interface that works consistently with multiple transport protocols. Inconsistent transport APIs and early binding have been a barrier to deployment of new transports in the past; SCTP, for example, defined an additional API, without which it was difficult for applications to access the benefits of the new protocol.

The transport services (TAPS) working group in the IETF is working to define a common, flexible, reusable, language-independent abstract interface between the application and the transport layer. This interface is designed to move the binding of applications to the transport protocol stack from design-time to runtime, based on the protocols available at the endpoints and the properties of the networks that connect them to the Internet. This interface is built atop a reference architecture describing the interactions among objects in a transport services system.

The transport services architecture

The TAPS architecture abstracts the transport layer behind a transport services interface. This interface allows the application to express its preferences for desired properties of the transport protocol stack (for example, reliable transport, message boundary preservation) and the underlying network connection in the pre-establishment phase. When a connection is established (whether passively, actively, or through rendezvous, a simultaneous-open action supporting NAT traversal approaches like ICE), the transport services interface will gather candidate stacks meeting the properties and may race multiple candidate stacks to find the best one for the application’s needs and the underlying network conditions. Once a connection is established, the application interacts with it asynchronously.

The transport services interface also provides for state caching (remembering
not only network conditions and preferences but also security properties and
resumption tokens, where appropriate) as well as local policies about which
stacks are preferred.

Features of the interface

The transport services interface is designed following a few first principles. First, network interactions are inherently asynchronous, so sending and receiving data in TAPS is asynchronous and event driven. In part due to this, and because different languages have different threading models, the interface is explicitly abstract: it suggests some names for actions and events but not how those should be implemented in the idioms of any particular platform or language.

In contrast to sockets, name resolution (getaddrinfo()) and address binding (connect() / bind()) are not separated, and the application need not
explicitly resolve names before using them. This allows the implementation of
the interface both to parallelize name resolution and address binding and to
speculatively fetch related information, further decreasing chances for
connection establishment to block.

A TAPS connection does not necessarily represent a single underlying transport connection with a single underlying flow; a TAPS connection can abstract both multistreaming protocols like SCTP and QUIC as well as multipath protocols like MP-TCP. This is exposed through the concept of entangled connections. Here, the application indicates to the interface that a set of connections should share the same endpoints and properties, but provide separate message streams. When a multistreaming transport protocol is selected, this can be implemented via separate streams, while it would be implemented with separate flows when these protocols are not available (for example, with TCP).

Like most application-layer interactions, TAPS is explicitly message-oriented. When running over a stream-oriented transport protocol that does not preserve message boundaries (for example, TCP), the interface exposes framers and deframers to the application, which can translate between message-streams and octet-streams inside the interface itself.

A TAPS connection begins life as a preconnection, which implements the pre-establishment phase. A preconnection contains the application’s requirements for an eventual connection, and can be used for passive open (Listen()), active open (Initiate()) or simultaneous open (Rendezvous()). The InitiateWithSend() action supports protocols that allow data in the first packet, like QUIC 0-RTT or TCP Fast Open. In any case, actions to initiate connections return immediately, and the application is notified that a connection is ready via an appropriate event. Once active, the connection can be used for sending messages (Send()) and for receiving them via events; the application exposes that it is ready to receive via an action, which allows backpressure to be exposed to the transport layer through a more direct mechanism than buffer sizing.

Form over detail

One may ask what use an abstract interface is, if it doesn’t mandate the
particulars of the implementation in various programming languages? Setting
aside the fact that the IETF is perhaps not the best venue to design APIs in a
variety of programming languages (each of which has its own standardization
process), our insight here is that the shape of the interface to the transport
layer is more important than the implementation details. With the publication of a reference abstract architecture, we aim to ease the development and deployment of new APIs and platforms for network programming, and to incentivize application development further down the stack to be less synchronous and less imperative about its interaction with the transport layer. Applications currently on sockets will have to be rewritten once to take advantage of the features of new transports: with TAPS, this rewrite may be the last, though.

History and future

TAPS was originally chartered in 2014 with a less expansive vision: we didn’t
really set out to reinvent sockets. The first problem we addressed was that of
dynamic transport selection, based on the initial insight, within the EU Horizon
2020 NEAT project, that design-time binding of applications to transport protocols would always have the effect of slowing the deployment of new transports. After cataloguing the transport layer behaviours that could feed into such a dynamic selection function, though, the working group realized that applications using dynamic selection would also have to sit atop an abstraction over all supported transport behaviour, or be constrained to the least common denominator behaviour (SOCK_STREAM).

The core architecture and a nearly-compliant implementation of the
interface are realized in Apple’s Network Framework for macOS and iOS devices; this framework will slowly converge with TAPS after publication. In addition, a reference implementation for Python asyncio is available under the MIT license.

The TAPS API documents are nearly complete, with publication expected in 2021; comments and discussion are welcome on the mailing list and the GitHub repository.

Brian Trammell is an active contributor to the IETF’s transport area. He co-chairs the IRTF Path Aware Networking RG and the RIPE Measurement, Analysis, and Tools WG.

Rate this article

---

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.