Next-Level Testing with an API Gateway and Continuous Delivery
To learn what to consider when choosing an API Gateway for your cloud native application, check out the first article in this series; Selecting an API Gateway for Continuous Delivery of Cloud Native Applications.
A modern API gateway like the open source Ambassador Edge Stack API gateway can help with testing on many levels. For example, you can deploy a new service or an upgraded version of a service into production and hide, or “cloak,” this service from end-users via the gateway. This effectively separates the deploy and release process, allowing you to run acceptance and nonfunctional tests on the cloaked service, for example, load tests and security analysis. You can also canary test a service by allowing a small amount of user traffic to flow to this new deployment.
There is also potential to use a gateway to “shadow” (duplicate) real production traffic to the new version of the service and hide the responses from the user, and “shift” traffic around to focus load on a specific cluster of your system. Finally, you can also use an API gateway to implement (and control) chaos testing. These techniques allow you to learn how your service will perform under realistic use cases, load, and failure scenarios.
Traffic Shadowing
Unicorn organisations regularly use the technique of shadowing traffic (or “dark launching”) features. For example, Facebook famously tested the release of the username registration service by directing real user traffic at the service and hiding the data that was returned. Twitter has also talked about their creation and use of the internal “Diffy” tool that acts as a proxy, multicasts requests, and then compares or “diffs” the response. GitHub used a tool called “Scientist!” that allowed them to carefully refactor critical code paths during a migration from one technology or architecture to another.
Christian Posta has written a great article about traffic shadowing with “east-west” traffic within your services ecosystem using a service mesh, “Traffic Shadowing With Istio: Reducing the Risk of Code Release,” and as Istio uses the Envoy proxy as its data plane, there is potential for API gateways that also use this technology, like Ambassador, to perform traffic shadowing on ingress or “north-south” traffic, i.e., requests from your users. As Christian discusses in his article, you do have to take care of shadowing PII or encrypted data, and the side effects a user request has within the system (such as database modifications), but this can be a powerful testing tool.
Traffic Shifting
Adrian Colyer wrote a great summary paper written by the Facebook team that talked about their “Kraken” load testing tool. In a nutshell, the Kraken tool integrates tightly with the Facebook gateways. It can “shift” (or route) part of its global traffic to systems (or data centers) under test and monitor the results — reverting the traffic shifting if monitoring systems show an error. So, for example, if Facebook wants to stress test a new data center that has just opened in Germany, they can shift all of the European traffic to this center in a controlled and gradual fashion and watch what happens.
I appreciate that not all of us are on Facebook, but I think this is a very interesting technique nonetheless. It helps me think differently about how I can utilize an application gateway.
Chaos Testing — Chaos Engineering: Break it to Make it Better
The final topic of testing that I can’t resist talking about is referred to by many names: chaos engineering, chaos testing or “resilience testing.” This type of testing increased in popularity as teams built distributed systems and bumped into the realities and complex failure scenarios when working within this domain. Chaos testing allows a team to hypothesize how a system will react to failure, design and run the experiment, and monitor what happens.
The Netflix team has historically been the pioneer within this space, and I’m sure many of you will have heard (or even used) the Chaos Monkey and Simian Army. Their second evolution of these tools inspired “Failure Injection Testing,” where failure could be injected into specific requests (perhaps for a test user or cohort of tolerant end-users) monitored the results. Target requests were identified and modified via an application gateway.
Data planes like the Envoy proxy allow header manipulation, so an API gateway like Ambassador can be used to generate and inject request IDs or “chaos headers.” Of course, internal services may need to propagate this header through the service call stack, but this would be a small price to pay for the ability to run failure injection tests.
Ambassador Labs Community
Ambassador Labs team and its community are constantly adding more features to the Ambassador Edge Stack API gateway to support the features discussed in this article.
The article: “Selecting an API Gateway for Continuous Delivery of Cloud Native Applications” covers the important features you should consider when choosing an API Gateway. Or learn more about Continuous Delivery with Kubernetes
You can learn more about implementing continuous delivery with an API gateway in our blog. If you want to learn more about the underlying Envoy proxy technology, we have also created a web page for this!
