Envoy Proxy: Security, Caching, Wasm, HTTP/3, and more

Published in

Ambassador Labs

4 min readAug 1, 2019

Since its release Envoy Proxy has gained enormous traction in the market. Envoy was a classic case of the right product at the right time:

Organizations were building increasingly sophisticated cloud applications and found they needed a new approach to manage L7 traffic
Envoy had the right set of features and performance to address this need. Some of these features included a runtime API for configuration & management, dynamic configuration, gRPC & HTTP/2 support, automatic retries, traffic shadowing, and robust observability systems.
Critically, Envoy had proven its stability and feature set in the real-world at Lyft

These factors created critical mass, driving rapid adoption. Envoy also quickly found a neutral home in the Cloud Native Computing Foundation, and, led by Lyft, quickly grew into a vibrant developer community, with organizations such as Airbnb, Google, Lyft, Pinterest, Verizon, and others contributing.

Envoy’s success has not gone unnoticed by the competition.

A year after Envoy’s announcement, HAProxy added a runtime API, hitless reloads, and HTTP/2 support in HAProxy 1.8, and, in June 2019, shipped HAProxy 2.0 with support for L7 retries, traffic shadowing, and gRPC.
11 months after Envoy’s announcement, NGINX added a runtime API and shadowing in NGINX Plus R13, and in March 2018, added native support for gRPC in open source NGINX.

More about Envoy Proxy

At the same time, with hundreds of developers working on it, Envoy has continued to add new capabilities at a staggering pace. So what’s new with Envoy in the past year?

Broader L7 Protocol Support

Envoy has continued to add support for new L7 protocols, including Dubbo, ZooKeeper, MySQL, and Redis. These filters enable users to take advantage of Envoy’s resilience, routing, and observability capabilities on broader types of L7 traffic. For example, Lyft sends 40M requests per second to its Redis clusters via Envoy Proxy today.

Security Hardening

Envoy Proxy is being deployed in some of the world’s largest cloud environments. Many engineers have been working on scrutinizing Envoy for security issues and contributing to a security response team. Harvey Tuch at Google has led an extensive fuzzing effort that helped harden Envoy.

Envoy Proxy TAP filter

The TAP filter is used to interpose and record HTTP traffic. Using the TAP filter, users can inspect HTTP traffic for debugging and analysis. Some of the use cases include performance analysis (record real production traffic, and replay it later for reproducible performance benchmarks) or debugging difficult-to-reproduce issues.

Dynamic forward proxy

In its original incarnation, Envoy Proxy was designed to route to known backends (e.g., microservices or databases). With the dynamic forward proxy, Envoy Proxy can forward HTTP requests to any upstream service. When using Envoy as a dynamic forward proxy, Envoy will forward requests to known hosts. Envoy will also pause requests to unknown hosts and asynchronously resolve DNS (caching the result) to ensure high performance.

Envoy Web Assembly

WebAssembly (Wasm) is a fast, efficient, portable binary instruction format. Wasm has gained enormous popularity as a common target for embedded execution environments such as web browsers. The Envoy team is working on integrating Wasm support directly into Envoy as an alternative to the current embedded Lua support. With Wasm, users can code plugins in virtually any language that can then be run as part of the Envoy process.

HTTP/3 (under development)

HTTP/3 is a major revision of HTTP designed to improve performance, especially over lossy networks. Based on the QUIC transport protocol developed by Google, HTTP/3 is the application mapping of HTTP to QUIC. When complete, users will be able to deploy Envoy at the edge to serve HTTP/3 traffic to compatible clients in a way that is transparent to backend services.

Envoy Universal Data Plane API

From the beginning, Envoy provided a set of management APIs that could be used to control fleets of Envoy proxies. Initially written in REST/JSON, these APIs have evolved into proto3-based APIs that support bidirectional streaming. The Universal Data Plane API is an effort to create a common set of APIs to manage any data plane, bridging disparate systems.. Both Envoy Proxy and gRPC-LB have committed to supporting the UDPA.

Caching API

Envoy is adding support for HTTP caching with eCache, a multi-backend HTTP cache. eCache is a caching HTTP filter that handles the complexity of HTTP caching while delegating the storage of HTTP responses to an external service.

Envoy Mobile

Envoy Mobile brings Envoy Proxy to iOS and Android. Announced in June, Envoy Mobile allows applications running mobile devices to take advantage of all the features of Envoy in connecting to the network to improve robustness and performance. This includes modern protocol support, e.g., gRPC and HTTP/3; observability, e.g., distributed tracing and metrics; and resilience, e.g., circuit breakers and retries.

Kafka Support (under development)

Kafka is a real-time streaming platform. Similar to Envoy’s support of Redis, MySQL, ZooKeeper, and Dubbo, development is in progress on adding support for routing and managing the Kafka application protocol.

Envoy and the community

The speed of Envoy development is only accelerating. Today, Envoy has become the de facto standard for the cloud-native data plane. Projects such as Ambassador Labs (edge control plane built on Envoy), AWS AppMesh (service mesh built on Envoy), HashiCorp Consul (another mesh built on Envoy), and Istio (mesh built on Envoy) are taking advantage of Envoy’s capabilities and bringing them to end users in ways that are easy to manage, control, and deploy.