Envoy Gateway 0.4.0 Release

User experience improvements and Envoy Proxy customization

This is the second in our ongoing series on Envoy Gateway releases, covering the April 2023 release of v0.4.0. Our goal with this series is not just to rehash the release notes or cover them in full but to highlight major and minor changes we think are important and provide context. This release includes

• Helm support
• A new egctl command-line tool
• Ability to customize aspects of the Envoy Proxy deployment and service
• Control plane extensions

If you need a refresher on what Envoy Gateway is and the project goals, check out my post on the v0.3.0 release.

User Experience Improvements

This release includes two great new tools that make it easier to get started and inspect what you have:

• Helm chart
• egctl CLI

Helm Support

Prior to v0.4.0, installing Envoy Gateway involved manually applying YAML files. Making Helm, the defacto Kubernetes package manager, the new recommended way of installing is great for ease of use, not just for quick trials, but for upgrading as new versions become available. It’s also much better for installing using Infrastructure-as-Code tools like Terraform and Pulumi.

Getting started with Envoy Gateway is now one simple command:

helm install eg oci://docker.io/envoyproxy/gateway-helm --version v0.4.0 -n envoy-gateway-system --create-namespace

You can customize many aspects of the installation, particularly resource requests, replicas, and the cluster domain, using the available values in the chart’s documentation.

The only improvement I’d like to see in the future is the chart being available in a traditional Helm repository. It’s currently stored on Docker Hub. You can directly install and pull charts from there, but you can’t add anything there as a repo which makes the discoverability of versions worse and pulls it out of my normal Helm workflow.

CLI

Because Envoy Gateway is in such an early stage still, the commands available in theegctl CLI are mostly useful for debugging.

For example, you can use it to translate Gateway API resources (i.e. an HTTPRoute object) installed in the cluster into their underlying Envoy Proxy configuration to check if the output that Envoy Gateway is going to create matches your expectations.

You can also use it to do a dry run of your Gateway API resources to validate them. The example given in the docs is an HTTPRoute with a missing field. By using $ egctl x translate --from gateway-api --type route --to gateway-api you can check if the resulting resource that would get created if you installed it would error out before installing it. A huge time saver as you experiment with Envoy Gateway.

Customizing Envoy Proxy

Envoy Gateway is essentially a control plane over deployed pods of Envoy Proxy. It reads Gateway API resources in the cluster, turns them into an Envoy config, and deploys Envoy Proxy pods that use the created configuration.

Emissary-Ingress and our commercial version Ambassador Edge Stack do the same thing, but the Envoy Proxy containers are embedded with the control plane rather than in a separate deployment. There are limitations to this approach:

• The control plane and proxy plane cannot be scaled separately
• Every instance is replicating the work of the others: watching the same resources and recomputing the same config
• The configuration of the control plane and proxy plane are comingled, and it’s not always clear which you are affecting

In contrast, Envoy Gateway is a separate deployment that watches for GatewayClass and Gateway resources. The Gateway resources, when implemented by Envoy Gateway, have deployments that contain the actual Envoy Proxy containers using the configuration. This architecture solves the above issues. Customizing the Envoy Proxy containers is done by providing custom configurations via a new EnvoyProxy config object. You can see examples at the above link, but this new config object allows you to customize:

• Pod annotations
• Resource requests and limits
• Service annotations
• Envoy Proxy bootstrap config

The first three of those items feel like things I would want to modify quite often as a platform engineer. The last one feels like something I would only do while on a chat with my support representative, as the whole reason I use projects like Emissary-Ingress and Ambassador Edge Stack (and Envoy Gateway in the future) is to avoid needing to muck around in Envoy config. The common ethos of these projects is creating a human-friendly API for configuring Envoy as a gateway. I personally would leave customizing the bootstrap config to the experts. However, I know there is a group of power users out there who need direct access to the bootstrap config and/or settings that aren’t exposed elsewhere. And there’s an opportunity to see what people use this access for that can inform what else to expose in the EnvoyProxy config object.

Control Plane Extensions

I’ve saved the feature with the biggest potential for last. Prior to v0.4.0 there has been no way to extend Envoy Gateway with new functionality beyond maintaining a fork of the project, which is not workable for most organizations to give it the kind of abilities you expect from a fully-featured API gateway.

With the new control plane extension system, it’s possible to create external extensions using gRPC hooks that Envoy Gateway uses to modify the underlying Envoy Proxy’s xDS resources. This is a fantastic solution as it stays within the existing systems in the Envoy and Gateway API ecosystems and will allow organizations using Envoy Gateway to add their own functionality! I highly recommend reading the full design doc for this feature to get a sense of what is possible.

There are tradeoffs with any approach, and this one only supports one extension, as reconciling changes that two extensions may want to make to a given xDS resource is a very difficult problem to solve.

A bit of clarification, though! I think it could sound like supporting one extension would only allow for one extended feature like authentication or request modification. Envoy Gateway will only be able to communicate with one extending application via the hooks, but that application could provide all kinds of additional features by modifying various xDS systems.

The Next Envoy Gateway Release

The next release of Kubernetes Gateway API will be v0.5.0 with a theme of “Observability and Scale.” The majority of work being done is visible in the 0.5.0-rc.1 milestone on GitHub. We’ll provide an update when that release is available!