03 Jun 2018  |  Culture

Mocking Out External Services in a Docker Based Microservice Architecture

5 minute read

In the world of service oriented architecture, you’ll quickly discover the need to have many microservices up and running to operate a single functioning service. More importantly, when it comes to writing automated test suites you can find yourself being dependant on external services. Let's see how they operate.

In the world of service oriented architecture, you’ll quickly discover the need to have many microservices up and running to operate a single functioning service. More importantly, when it comes to writing automated test suites you can find yourself being dependant on external services.

For our test suites we utilise three layers:

  1. Unit (PHPUnit)
  2. End to End (Cypress)
  3. Behavioural (Behat)

The Unit layer is fairly simple, very granular tests that mock out any unrelated classes/3rd party services. We use PHPUnit for testing and Mockery for mocking out dependant classes. This layer does not touch the database and can be run without an internet connection.

The End to End tests are designed to cover multiple services including the frontend JS applications, so we are not as concerned about mocking here (although we still do mock things like payment gateways for regular testing).

It’s the Behavioural tests suite where we really want to make heavy use of mocking. This test suite uses Behat and will make remote (generally HTTP) calls to our service and validate the response. Without having any access to modify the runtime environment (like you can in PHPUnit) we need to think about a new approach to mocking connected services.

The Challenges

  1. We want to easily turn mocking on/off - so developers can quickly test with mocks and do more in-depth testing when needed.
  2. We don’t want to change our code to enable mocking. Changing URLs in the code, turning off SSL or skipping a HTTP call would not be a real test of the code.
  3. We want this to work locally and on our continuous integration (CI) environment without extra work (no unique, handmade servers in sight).
  4. We don’t want developers to spend more than a few hours to be able to completely mock out a new service.

The Solution

Like most companies developing microservices, we use docker to facilitate the running of multiple services which can all be using different environments. During local development and CI we use docker-compose to orchestrate our services.

There are 3 key elements to successfully mocking out services that meet our requirements

  1. A docker container that acts as a mocked service. It should be capable of receiving a HTTP request and returning a pre-defined response.
    • Needs to listen on many ports.
    • Needs to return a self-signed certificate.
    • Needs capabilities to dynamically change responses - for example when a unique token needs to be returned on each call.
  2. Generating self-signed certificates that can be trusted by the caller.
    • Our main caller service will need to trust a self-signed certificate, so that when we redirect traffic for https://service123.paddle.com to our mock server it does not throw a certificate mismatch error.
  3. Network host redirection.
    • Without changing a line of code in the service, we want to be able to redirect all traffic for https://service123.paddle.com to our mock server.

We are using mountebank to mock our services. This is the heart of our mocking strategy and provides a quick and easy way for developers to mock the responses that we would expect from any service. You can use anything you like here, we picked mountebank because it provided everything we needed with very little setup time.

We then make use of the links functionality in docker-compose to redirect all traffic to the mock container.

The Walkthrough

There are quite a few parts that I may have glossed over here, like how do we create self-signed certificates and make the host trust them? How do we toggle mocking on and off? I cover all of these and more in the in-depth walkthrough below.

The whole sample project can be found on GitHub

Docker Files

To begin with, we need to create a Dockerfile for mountebank. You will need to modify the ports you would like the container to listen to — we only expose them on the local docker-compose network so they should not interfere with your other projects.

Next we set up an nginx and php images—these could be any web server/language. The only really important thing is the entrypoint.sh that the php container uses — make sure the container that executes your code runs this entrypoint script. It looks for the presence of /tmp/ssl/, if found it will load those certificates onto the OS and from that point onwards they will be trusted. Our PHP image in this example uses debian, if you use a different flavour of linux then the entrypoint script may need to be modified to work with your operating system.

We then use docker-compose to orchestrate the containers and create our network. There are two docker-compose.yml files, one is the default which will not mock anything, and then the second which, if included, will spin up our mountebank container and reroute all network traffic for the URIs in links to the mountebank container.

To run without mocks use:

docker-compose -f docker-compose.yml up -d

And to run with mocking enabled use:

docker-compose -f docker-compose.yml -f docker-compose.mocks.yml up -d

If you open http://localhost:8888/?service=1 in a browser when mocking is enabled you will get the mocked data, without that you’ll get a timeout because the service isn’t running.

Self Signed Certificates

The example project sets up two mocked services in mountebank, one uses SSL and the other uses plain HTTP using a non-standard port (for no reason other than as an example).

Firstly you will need to generate your own self signed certificate — this is purely for the mock testing and not for use in production.

openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout paddle_cert.key -out paddle_cert.crt -subj "/C=GB/L=London/O=Paddle/OU=Engineering/CN=service1.paddle.com"

You will need to copy the key and certificate file into docker/mountebank/ssl as will as copy/pasting the contents into docker/mountebank/resources/imposters/imposters.ejs. This step could easily be automated.

Build Automation

To make things easier, we like to use a Makefile to wrap repetitive tasks in simple build commands. If you have the sample project checked out, just run make build to build the docker images and then either make start-with-mocks or make start-without-mocks

A good example of how this Makefile could be extended would be to add the self signed certificate generation to it. It could generate the certificates as well as create the required config items in mountebank so that a developer only has to create the actual mock resources they need without worrying about anything else.