Testing Microservices: How to Share Staging Environments without Tripping Over Each Other

In the previous two parts of this series on Testing Microservices, you have learned about the concept of request isolation and explored how this enables multiple developers to “share” a single staging environment and still get the fast development/testing feedback loops. You also experienced implementing request isolation within a microservices-based system running on Kubernetes using the CNCF Telepresence tool.

In this final part of the series, you will see how to adapt what you’ve learned so far to your services, development environments, and staging environments, and critically, to your developer workflow.

Creating a good developer experience and building an effective microservices testing strategy (including both manual and automated testing) isn’t easy, but one thing I know from my previous experience is that having a good selection of tools (and techniques) in your testing toolbox can really make a difference in the day-to-day life of a busy developer!

Quick reminder: Goals for testing microservices

As a quick summary, in the previous articles I suggested that when building and testing microservices you will typically be optimizing for the following three properties:

• Time-to-feedback is low (latency for exploring, verifying functionality, and testing). This includes making code changes, building, deploying, asserting, observing, etc. for both the inner and outer development loops and also quality assurance (QA) loops.
• The quality of the deployment environment is high (production realism). We want to minimize the WTFs when finally deploying to your production environment.
• An ability to work in isolation within a single staging environment (at the individual, pair, or team level) and also securely share our work in progress widely across the organization without causing conflicts or breaking other developers’ work-in-progress.

With the previous articles covering a lot of the theory, this post focuses on how to implement the above three properties with the request isolation pattern and your services running in a local environment and a remote Kubernetes cluster using Telepresence and Ambassador Cloud.

Infrastructure prerequisites for request isolation: smart routing and context propagation

There are several prerequisites that must be in place before you can utilize the request isolation pattern for sharing a staging environment for testing your microservices-based application.

The first is smart routing. Each service needs the ability to make dynamic routing decisions based on request headers. This can be implemented with a modern cloud native proxy like Envoy (which is exactly how Telepresence implements smart routing under the hood).

The second prerequisite is context propagation. In order for the smart routing functionality to know when to route traffic differently to a service under test, a header (think “isolation feature flags”) must be passed to the service under development or test. And with a microservices-based system, there is often a “call chain” of services that take part in serving a user’s request e.g. service A calls B, B calls C and D, and D calls E, etc. This means that the request isolation header that is injected at ingress must be propagated, or passed, from service to service.

Context propagation can often be implemented easily with modern monitoring and observability libraries and frameworks, such as Open Telemetry. This technology uses request-based context propagation to correlate an end user-generated request with all of the other corresponding internally-generated service requests.

In summary, here are the prerequisites and recommended implementations:

• Smart routing: Install Telepresence into your cluster for transparent injection of smart (Envoy) proxies that are capable of making routing decisions based on request header content e.g. x-telepresence-intercept-id
• Context propagation: Add a language-specific observability or distributed tracing library that supports context propagation of headers (and corresponding data or “baggage”) to all of your existing services. You will also need to configure your ingress and/or API gateway for forwarding selective headers from external requests to internal services.

It’s worth mentioning that you most likely will want to implement selective external header forwarding with some additional security checks, rather than a “forward all” for headers at ingress, otherwise, this could be an easy attack vector for nefarious users. For example, to prevent abuse of an accidentally leaked secured development URL, all users of Ambassador Cloud are, by default, forced to log in with a single-sign-on account that shares a common organization. This can be overridden on a case-by-case basis if you do want to share access to a wider audience.

Once the prerequisites are in place you’re ready to implement request isolation in your own staging environment.

Implementing request isolation in your staging environment

In the previous article, you used Ambassador Cloud, Telepresence, and a local Docker development container to experiment with request isolation when modifying the emojivoto demo app. Now, I want to help you implement request isolation with your own staging environment, and will do this by building on the previous demo.

Rather than using a local container to install and configure everything, you will instead clone the emojivoto repo to your local machine, where you can use your favorite editor, and run a setup script to install and configure Telepresence on your local machine, allowing easy use with other projects.

You will still use a local development container (which includes a hot reload mechanism) to build and run the app, but your code and Telepresence are installed outside of the container. Once you’ve succeeded in implementing request isolation for the emojivoto app, it should be a relatively straightforward task to replicate the process and the setup script for your own code, services, and staging environment.

Acquire a remote K8s cluster for testing

First, head over to Ambassador Cloud and follow the quickstart instructions to get a free remote Kubernetes cluster and configure everything I’m going to introduce below.

The quickstart will first ask you to clone a modified version of the emojivoto code repo to your local machine:

$ git clone git@github.com:datawire/emojivoto.git

And cd in the JavaScript app directory:

$ cd emojivoto/emojivoto-web-app/js

Next, you will export the an AMBASSADOR_API_KEY to synchronise your local machine with the remote K8s cluster provisioned in Ambassador Cloud.

$ export AMBASSADOR_API_KEY=…

And you will run the setup script

$ ./setup_dev_env.sh

After the setup script runs successfully, be sure to follow the instructions on the terminal and export the KUBECONFIG that points to your remote K8s cluster.

$ export KUBECONFIG=./emojivoto_k8s_context.yaml

If you have VS Code installed an editor should have opened on your local machine. If not, you may have seen an error mentioning this that you can ignore — simply open the Vote.jsx file that is located in the current directory in your IDE of choice.

Now, look back at your terminal. During the setup output, you should see Telepresence display a preview URL. Copy this to your clipboard and paste it into your web browser. This is your secure development URL that routes to the remote emojivoto app running in the K8s cluster you provisioned earlier — with the key difference being that this URL automatically injects a request header.

As the emojivoto application has been instrumented to propagate all request headers from ingress through to all the services in the call chain, Telepresence can make the “smart routing” decision to send traffic destined to the remote web-app to your local dev machine.

To see the request isolation in action, modify the <h1> text within Vote.jsx to something different than “EMOJI VOTE” and save the file locally.

As this file is volume-mounted into the local dev container, this will trigger and re-build or hot-reload within the container. If you refresh your browser with the secured development URL, you should see your change. Note that the secured URL is directing your request to the ingress and services on the remote cluster, and Telepresence is selectively routing your request to the web-app to your local machine.

The web-app service on your local machine can call into the remote cluster and access all of its dependencies — we don’t need to run the entire app locally

Anyone else accessing the application in the remote cluster via that main URL won’t see your changes. You can test this out by getting the EXTERNAL-IP address of the Ambassador Edge Stack in your remote cluster.

$ kubectl get svc edge-stack -n ambassador
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
edge-stack LoadBalancer 10.43.86.14 34.132.143.276 80:30245/TCP,443:31770/TCP 52d

As the remote emojivoto app is being served from the root, I can view the application by accessing the IP — in this example, 34.132.143.276.

Note that the H1 title in this version of the application is still “EMOJI VOTE”. Your local change is not visible to anyone accessing the application via this URL, as the Telepresence smart proxy does not route these requests (without the request header set) to your local machine. This is what the request flows look like:

Replicating request isolation for your services

With your free remote K8s cluster available for three more hours (before it is reclaimed) and Telepresence installed locally, you can now deploy your application or a sample application that supports header context propagation into the cluster and begin experimenting.

Or you can simply change your local Kubernetes context to your staging environment and telepresence connect (checkout kubectx to make switching contexts/configs easier). You can then telepresence list to see which services can support testing via request isolation, and telepresence intercept my_svc to begin routing remote traffic to your local machine. Just spin up the service locally, begin coding, and see the results via your secured development URL.

If you want to automate this process, you can also look at and modify our setup script for your own needs:

https://github.com/datawire/emojivoto/blob/main/emojivoto-web-app/js/setup_dev_env.sh

Scroll to the bottom of the script and you will see the order in which the other functions are called. You can then read each function in turn and modify them to configure your app:

has_cli
set_os_arch
check_init_config
run_dev_container
connect_to_k8s
install_telepresence
connect_local_dev_env_to_remote
open_editor
display_instructions_to_user

Alternatives and complementary technology

As mentioned several times in this series, I always recommend learning as much as possible and building out a toolbox of tools and techniques with any process in software development. I believe the request isolation pattern is very powerful, and it is especially useful when testing assumptions against real services and dependencies, and also when trying to iterate very fast on a service without the need for the rigor of something like Test-Driven Design (TDD). However, sometimes you do need an extra level of verification and correctness, and other approaches can provide alternatives and/or be complementary.

I liked this tweet from Bjørn Sørensen at Lunar (and be sure to check out our podcast chat with his colleague, Kasper Nissen — the Lunar folks are awesome!)

Testing services in complete isolation is a valuable approach, and many of the techniques I will mention here can be used both for local development and within a staging environment. Of course, you will need to recognize the limitations and assumptions you will be introducing, e.g. a mock doesn’t capture the actual current state of an external service, only your mental model and assumptions of the service.

I encourage you to read Toby Clemson’s Testing Strategies in a Microservices Architecture for more information, and tools to focus on here are language specific-mock, stub, and double support and frameworks.

When writing Java I use Mockito, with JavaScript I like Jest (and other framework-specific mocking libraries), and when (occasionally) coding in Go I use GoMock.

If mocks aren’t quite powerful enough or don’t offer the variety or volume of data required, you can look at service virtualization technologies like Wiremock, Mountebank, and Hoverfly (which I worked on back at SpectoLabs). You can also spin up in-memory lightweight replicas of message queues (AMQP) with Apache Qpid and Apache Kafka with embedded-Kafka, and most cloud vendors provide something like the LocalStack AWS service emulators. If you want to run middleware or data stores within ephemeral containers, there’s always the ever-useful Testcontainer project that makes doing this within a test context very easy.

Many of these technologies and techniques are complementary, and I would encourage you to pick the approach that best suits your use case and provides the fastest feedback with minimum coupling.

A sign you’ve picked the wrong approach is spending too much time waiting for tests to complete or too much energy on test maintenance e.g. you’re constantly reconfiguring mocks due to API changes, or you’re endlessly re-capturing service virtualization due to data changes, or you’re waiting for 5 minutes to download and spin up a bloated ephemeral container.

Wrapping up the series

I hope this series on Testing Microservice with request isolation has been useful. There are many more development and testing patterns I can share for working effectively with microservice-based systems, so please reach out if you have any questions.

Check out the full Testing Microservices series:

• Part 1: You’re Thinking About (Environment) Isolation All Wrong
• Part 2: Isolating Requests (Not Environments) with Telepresence
• Part 3: How to Share Staging Environments without Tripping Over Each Other