Modern data centre applications are commonly deployed using service meshes, like Istio or Linkerd. Service meshes simplify application development by abstracting away the networking layer, making applications easier to deploy and manage. Under the hood, service meshes rely upon service proxies (or ‘sidecars’), such as Envoy, to enforce policies (for example load balancing, rate limiting) at the transport (Layer 4, or L4) and/or at the application layer (Layer 7, or L7).
While convenient, service proxies tend to significantly slow down service meshes: previous studies have shown that they can increase request latency by up to 185%. To reduce this overhead, state-of-the-art service meshes, such as Cilium or Calico, offload L4 policies to the kernel using extended Berkeley Packet Filter (eBPF). L7 policies, on the other hand, are still enforced in user space.
The case for an L7 fast path
Unfortunately, L7 policies constitute the bulk of the policies in modern deployments. As an illustration, Alibaba reports that up to 95% of their customers use L7 policies in their service meshes. Today, these deployments are left largely unoptimized. We visualize this overhead in Figure 1, where we measure the average request latency of a realistic application deployed on top of a service proxy with a minimal HTTP policy. The figure compares three different proxy configurations:
- Envoy without any optimizations adds a significant overhead of 2.6ms per request.
- State-of-the-art service meshes, which offload L4 policies using eBPF, reduce the total request latency by 17%.
- Beeline enforces L7 policies directly in the kernel, improving the request latency by 46%.
State-of-the-art service meshes process all L7 policies in user space because they are too complex to process in eBPF. Indeed, despite recent advances to improve L7 support in the kernel, such as strparser or Kernel Connection Multiplexor (KCM), processing L7 protocols remains impractical within eBPF’s stringent limitations. This is because of the way the kernel verifies the safety of an eBPF program: It performs an exhaustive execution path traversal that becomes intractable for programs with complex control flows.
This is a problem when processing L7 protocols, like HTTP, which use self-describing structures and require more complex parsing logic and more state than protocols with an implicit structure.
But there is an opportunity: While many L7 protocols are complex to parse, the logic needed to enforce L7 policies tends to be simple. We show this by analysing the L7 policies used in 2,417 open-source projects on GitHub. We analyse 4,699 distinct Envoy configurations, categorize them according to their functionality, and manually inspect each policy type to determine their eBPF-compatibility. We find that the logic of 89% of all deployed L7 policies can be implemented in eBPF without kernel changes. We visualize this in Figure 2.
This observation enables a split design, in which:
- The vast majority of L7 policies are enforced safely in the kernel, using eBPF.
- The few more complicated L7 policies are transparently handled in user space by existing service proxies.
Realizing this split design is challenging as it requires overcoming eBPF’s stringent limitations. We show how to navigate these limitations by automatically mapping L7 policies into restricted eBPF templates that support the required parsing logic and actions, while being guaranteed to pass eBPF verification.
Despite being restricted, our templates are expressive enough to capture most of the L7 policies observed in practice.
Beeline
In our recent work “Enforcing Application-Layer Policies in eBPF“, we addressed this problem with an eBPF-based fast path called Beeline. For the vast majority (89%) of L7 policies in the wild, Beeline can eliminate the service proxy from the critical path. This reduces the average request latency of realistic web applications by up to six times and increases the throughput by three times. It does this without any coordination with the service proxy. Beeline is transparent, and thus able to accelerate any service proxy.
Beeline employs two techniques to enforce L7 policies within eBPF’s limitations:
- It synthesizes the data plane specifically for each policy, allowing it to eliminate complexity that is otherwise needed in a generic data plane.
- It extracts only the headers that are necessary to enforce the policy, eliminating the need for complex parsing logic.
Data plane synthesis
Beeline synthesizes code for eBPF-compatible policies using a small set of simple, yet expressive, eBPF templates. Each template contains placeholders Beeline replaces with policy-specific parameters. Figure 3 visualizes this. The service policy specifies that /feed requests should be redirected. Beeline uses a predefined routing template and inserts the IP address of the forwarding target. The resulting code is inserted into the data plane before it is compiled. Because each template passes eBPF verification, the resulting data plane does, with some limitations, too.
Protocol parsing
Beeline only extracts the information it needs to process each request. For example, to enforce the policy in Figure 3, Beeline only needs to extract the HTTP path to identify requests to the /feed endpoint. To this end, Beeline constructs a Deterministic Finite Automaton (DFA). It can do this on startup, in user space, which reduces the complexity of the eBPF program significantly. Beeline’s data plane uses it to identify the relevant data segments in the raw message buffer and perform message delineation.
Where to go from here
Beeline is open source and currently supports HTTP/1.1 and HTTP/2. While this blog post details how Beeline accelerates service meshes, the high-level ideas can be applied to many more applications. For example, network operators can leverage Beeline’s parser to collect application-layer telemetry directly from the kernel, without any changes to the application. If you want to try this out yourself, please refer to Beeper, a nicely packaged version of the parsing stage.
Laurin is a PhD student in the Networked Systems Group at ETH Zürich. His main interest is the network stack on the end host, in particular, its interaction with the application layer.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.