75Running a microservice system

On the staging system, you can measure the behavior of a microservice in terms of its adherence to the message flows of the system. Ensuring that the correct messages are sent by the service, and the correct responses given, is also a binary pass/fail test, which you can score with a 0 or 1 The service must meet expectations fully. Though these message interactions are tested via unit tests in development, they also need to be tested on the staging system, as this is a closer simulation of production.

Integrations with other parts of the system can also be tested as part of the staging process. Those parts of the system that aren’t microservices, such as stand-alone data- bases, network services such as mail servers, external web service end-points, and oth- ers, are simulated or run in small scale. The microservice’s behavior with respect to these can then be measured. Other aspects of the service, such as performance, resource consumption, and security, need to be measured in a statistical way, taking samples of behavior, and using these to predict risk of failure.

Finally, even in production, the risk of failure continues to be measured. Even before going into production, you can establish manual gates—formal code reviews, penetration testing, user acceptance, and so forth. These may be legally unavoidable, but they can still be integrated into the continuous delivery mind-set.

Running services can be monitored and sampled. Key metrics, particularly those relating to message flow rates, can be used to determine service and system health. A great deal more will be said about this aspect of the microservice architecture in Chapter 6.

5.7

Running a microservice system

The tooling to support microservices is developing quickly, and new tools are emerging at a high rate. It isn’t useful to examine in detail that which will be out-of-date soon. This chapter focuses on general principles, allowing you to compare and assess tools and select those most suitable for your context. You should expect and prepare to build some tooling yourself. This isn’t a book on deployment in general, and it doesn’t discuss best practices for the deployment of system elements, such as database clusters, that aren’t microservices. It is still recommended that these be subject to automation, and if possible controlled by the same tooling. The focus of this chapter is on deployment of your own microservices, encoding the business logic of the system, and making them subject to a higher degree of change compared to other elements.

5.7.1 Immutability

It’s a core principle that, of the approach described here, microservice artifacts are immutable. This preserves their ability to act as primitive operations. A microservice artifact could be a container, a virtual machine image, or some other abstraction1_{. The}

essential characteristic of the artifact is that it can’t be changed internally, and only has two states, active, and inactive.

The power of immutability is that it excludes side-effects from your system. The behavior of the system, and microservice instances, is much more predictable because you can be sure that they aren’t affected by changes that you are unaware of. An immutable artifact contains everything the microservice needs to run, at fixed ver- sions. You can be certain that your language platform version, and your libraries, and other dependencies, are exactly as you expect. Nobody can manually login to the instance, and make un-audited changes. This predictability allows you to calibrate your risk estimations more accurately.

Running immutable instances also forces you to treat your microservices as dispos- able. An instance that’s developed a problem, or that contains a bug, cannot be "fixed". It can only be deactivated, and replaced by the activation of a new instance. Matching capa- city to load isn’t about building new installations on bigger machines, it’s about running more instances of the artifact. No individual instance is in any way special. This approach is a basic building block for building reliable systems on unreliable infrastructure

MICROSERVICEDEPLOYMENTPATTERNS

This section is a reference overview of microservice deployment patterns. You’ll need to compare these patterns against the capabilities of the automation tooling that you use. Unfortunately you should expect to be disappointed, and you’ll need to augment your tooling to fully achieve the desired benefits of the patterns.

Feel free to treat this section more as a recipe book for cooking up your own pat- terns, rather than a prescription. You can skim the deployment patterns without guilt1_. Name Rollback

Motive Recover from a deployment that’s caused one or more failure metrics to exceed their thresh- olds. This enables you to deploy new service versions where the estimated probability of failure is higher than your thresholds, whilst maintaining overall operation within thresholds.

Description Apply the activate primitive for a given microservice artifact to a system. Observe failure alerts. Deactivate the same artifact. Deactivation can be manual or automatic. Logs should be pre- served. Deactivation should return the system to health. Recovery is expected but may not occur in cases where the offending instance has injected defective messages into the system (for this, see the Kill Switch pattern).

77