A recent research report from Canonical highlights how we are still in the very early days of the shift to ‘Cloud Native’ computing. Pioneered by Netflix, this is an approach to development based on the use of containers, Continuous Delivery and Microservices software architecture, and represents the most significant evolution of enterprise IT that will impact software testing.
Sauce Labs explores this in What Does Kubernetes Mean for Software Testing.
First, they highlight that the main functionality of a containerised application isn’t different from a non-containerised one, so the functional testing will remain the same. The main evolution is the ability to leverage the environment itself to support the activities of testing.
Packaging an application doesn’t change how these scripts run, but if you have a set of automated tests built, they can be incorporated into the continuous build pipeline used to produce your containers so tests can be performed and reported on with more consistency and much more timely results.
Writing for TechBeacon Glenn Buckholz highlights how Kubernetes enables complete application deployment models to be defined in YAML, enabling the environment to be easily replicated for testing purposes, bringing critical benefits to testers including a decreased dependence on shared environments, direct access to logs, the ability to get to a complicated failure state quickly and cheap disposable environments.
In short, it empowers testers with the same dynamic Cloud-powered capabilities the developers enjoy and enables both to unify their work around the same consistent environment.
This includes tools like Terratest, a Go library that makes it easier to write automated tests for your infrastructure code, which Yevgeniy Brikman explains in this talk, providing a thorough walkthrough of how to write automated tests for infrastructure code, including the code written for use with tools such as Terraform, Docker, Packer, and Kubernetes. Topics covered include unit tests, integration tests, end-to-end tests, dependency injection, test parallelism, retries and error handling, static analysis, property testing and CI/CD for infrastructure code.
Matt Young describes the fundamental challenge of developing this capability in this blog, most notably cultivating an integrated toolchain that can handle all of these different layers and activities:
“Combining related software development tools in one package is a trend which results in a testing framework like Cypress for example, which is an extension of Selenium, JMeter, Jenkins, and Cucumber, sewn together like Frankenstein’s monster. Until a truly intelligent solution is adopted, we can claim that this is a necessary evil.”
Netflix offers a great tutorial on the first part, in their 2016 presentation Microservices at Netflix Scale.
In the Netflix presentation Ruslan describes that their organizational priorities are ranked 1) velocity of innovation, 2) reliability and 3) efficiency, as their core goals. Microservices software architecture is combined with a DevOps team model where developers take full life-cycle ownership of their services, a loosely coupled approach that enables each to release new features much faster, increasing the velocity of innovation.
They recommend a series of QA practices for adopting the same type of culture as Netflix, various ways to test for failure and recovery as they do, and using tools such as OpenTracing.
Fernando Mayo explores the modernisation of testing for this new microservices world, highlighting practices like property-based testing, fuzz testing, and mutation testing, that can help detect a wider range of defects in an automated way.
To understand how testing can be adapted to this new paradigm, Martin Fowler, a renowned guru on the topic, offers this excellent presentation resource, a primer on microservices software testing, where he explains in detail how practices like Unit, Integration and Component testing are applied to this new architecture.
Serverless Testing
In their blog ‘Don’t get TechCrunched: performance testing for your HTTP APIs’ they provide a detailed walk through of how their technology can be utilized for API testing, including a recipe for implementing it on AWS Lambda serverless computing.
As they describe in this Internet age a simple mention on a site like TechCrunch could drive a huge spike of traffic to your website and you need to know it is prepared to cope, and they share a comprehensive guide to how best to achieve this.
They explain how to get started with test automation, establishing a process for functional and integration testing and then moving on to look at performance testing to gain a deeper understanding about the stability, reliability, and availability of your platform.
This covers an analysis of performance testing, how it is broken down into stress testing, to determine the system’s ability to handle a heavier than normal workload, endurance testing, to determine the system’s ability to handle a specified workload for a prolonged period of time, and load testing, subjecting an API to a workload that approaches and possibly exceeds the limits of its specifications.
They also document how to define API performance metrics so that you can structure SLAs to suit your business needs, how to assign the work responsibility across your team and importantly, how to integrate it into your Continuous Integration workflow.
For example, a fundamental characteristic of microservices is their inter-communication through APIs, which can be tested using tools like Postman, which can manage APIs available via a browser, making it easier to share and inspect APIs. Previously, Postman required DevOps teams to download a desktop application to access the Postman platform.
Writing on DevOps.com Mike Vizard says:
“Now that more organizations have adopted an API-first approach to building and deploying applications in the age of microservices, those APIs need to be managed as discrete projects. Postman provides a means of achieving that goal now through a browser interface that can be accessed more easily by members of DevOps teams working remotely.”
Postman themselves offer this article, explaining their own in-house journey to a microservices architecture. This is highly informative, exploring the challenges of DevOps team dynamics, as well as adapting their CI pipeline.
j6jmbk