How DevOps is changing software testing

The move to the DevOps methodology can help production teams bring software to their customers faster.
11 October 2018
Quality assurance
Test automation
The article by a1qa

Once upon a time, software QA was done by teams taking small aspects of a piece of software and testing every conceivable variation. Times have changed and so has the software industry. We no longer think of software as being packaged in a box and bought off store shelves. Old-style massive full versions of software are no longer the norm; now most companies deliver software via the internet in small bursts. Unless there is to be a major change, the updates are just made and instantly applied without users having a clue.

The move from gigantic disc-based software to these incremental on-the-fly updates required a great deal of thinking behind the scenes. First, the programmers became Agile and set themselves small achievable goals. They would spend a week, maybe two, working on adding some features or fixing a couple of bugs as in these images:

Bug Queue image courtesy of
Features Backlog image courtesy of

While the old mindset had developers creating software and handing it off to the operations team to maintain, the newer DevOps approach has those programmers working together with their operations team throughout the process.

The key is that now much of the company is both working and planning to get the product to an agreed-upon state. Instead of waiting for the programmers to push their possible final build to the testers after, those testers are now involved from the get-go.


To make a build cycle take as little time as possible, its scope is recursively reduced. Here is an example of one of those tasks, or stories, being further refined:


This shows an attempt to break down and refine the goal into the smallest bites possible. These smaller and smaller bites make it easier to plan software testing. QA engineers can better estimate the time needed for testing and fixing any defects discovered.

Number of tests

When talking about testing in Agile environments, the team has to be able to trust that the code will perform as expected. That trust, or its lack, can be easily understood if you look at it in light of the number of tests. More testing means higher probability that your program will build and deploy successfully. Your confidence about the application will be much higher if it passes 295 out of 300 tests than if it passes two out of three.

Size of tests

The other side is the size of the tests you need to do. With large, multifaceted applications, it may be difficult to test the whole program after every build. In those cases, you test only the parts that are important or that have been changed. There is never a set-in-stone group of required tests, your teams can add or subtract as needed.

Frequency of tests

How often you test can depend on the types of tests and what resources they require. For an application that consists of a bunch of smaller programs, it is pretty straightforward to test just what has been changed. If, however, your program is huge, you may not be able to run all your tests after each section of code is committed. In that case, you run the subset of tests that you need to and then run your full test at night. Basically, if the team decides that a test needs  to be run, its size or runtime will not matter. For a test that takes three days to run, you just automate it to run every three days at a certain time.

Automation and continuous delivery

In the past, testing engineers focused only on one part of the software development cycle—testing the code and pushing it on to the programmers. Although with DevOps, that has changed—with continuous integration and delivery, testers are now required to take responsibility for quality throughout the entire development process, not just the old QA phase.

Building more tests into the software

In the world of DevOps, it is important to have regular deployments—the automation of processes can be a big help in making that happen. That said reduced deployment time and more frequent software releases mean more testing will need to be done. If there is no automation, large numbers of test cases must be run manually, which will slow down the whole process. Teams need to increase their testing and will have to automate more and more of it.

Automation for testing environments (virtualization)

A test environment is an internal setup of software and hardware that emulates the environments where the finished product would eventually deploy. The team can then test the software on virtual systems to catch defects before it rolls out to production. Cap Gemini’s included diagram shows several varieties of testing environments.

Test Environments image courtesy of

If testing environments are automated effectively, the amount of manual intervention will be greatly reduced. However, if the automation is done incorrectly, the QA team will have to fix test environments manually, which will ultimately increase the amount of manual work—not something you would expect from automation.

Effect on teams

In the old model, QA wasn’t in touch with the development and delivery processes. DevOps get them working together.

Increased communication and collaboration

In DevOps, QA engineers must actively collaborate with other departments. They attend planning sessions and communicate to the rest of the project team what will be tested next, when, how, and by whom. This may require the QA to step out of their comfort zones, but it will be necessary for effective work.

The right project management solution is critical to keeping all teams aligned by giving everyone clear visibility on the project’s progress between meetings.

Fail faster, fix faster

Coding is a practice that often requires extreme concentration. Getting back on track, or more specifically, getting back to where you were mentally when you added a bit of code can be very difficult.

For this and other reasons, many teams work on the Fail Faster, Fix Faster principle. Jim Shore said in his 2004 IEEE article on the subject that, “failing fast is a non-intuitive technique. Failing immediately and visibly… makes bugs easier to find and fix.” This method makes it very easy for an error to stay in context for the developer. They will not have to search for hours to fix it.

Some software is given the ability to compensate for errors. This can result in a bigger mystery fail. Fail fast suggests that the error should be allowed to happen right away so that it can be detected and fixed faster. These bugs appear sooner and don’t reach production, thus reducing costs.

Closing thoughts

The move to the DevOps methodology can help production teams bring software to their customers faster. In addition to having the developers work closely with the operations team, we now have the QA testers pulled into this collaborative effort. All three teams work and plan the steps for the full rollout—keeping in mind that this consists of lots of little rollouts. The QA team is now there in the thick of things automating and testing their little hearts out. No longer are they necessarily the last stop feeling left out in the cold or blamed for the product release being late.

Author: Steve Medeiros, a writer for with an extensive background in technology, software, and customer support.

More Posts

debated technologies
30 May 2023,
by a1qa
3 min read
a1qa tech voice: Managing director at a1qa, North America, discusses pros and cons of much-debated technologies
Nadya Knysh, Managing director at a1qa, North America, puts a spotlight on 6 current technologies, discussing their positives and negatives.
Test automation
17 April 2023,
by a1qa
4 min read
Why do bugs get missed? Learn the problems and tips to avoid them
Still, finding overlooked bugs after the app goes live? Let’s find out why this happens and how to fix it.
Quality assurance
Test automation
Mobile app testing
15 February 2023,
by a1qa
4 min read
Mobile app testing guide: win the race with five-star software
Which aspects of mobile apps to test first to produce a really high-quality product? Find the answer to this and other questions related to mobile app testing in the article.
Cybersecurity testing
Functional testing
Mobile app testing
Performance testing
Test automation
Usability testing
31 January 2023,
by a1qa
5 min read
Best of 2022 by executives: 8 most visited a1qa blog posts
Let’s look back and revisit the most visited a1qa articles of 2022!
Quality assurance
Test automation
12 January 2023,
by a1qa
4 min read
The future of software testing: top 4 impactful trends that will dominate in 2023
Consider the major industry trends for the upcoming year to know how to improve current QA strategies and stay ahead of the curve.
QA trends
Quality assurance
Test automation
7 December 2022,
by Dileep Marway
3 min read
Release at pace with test automation: What, why, and how to measure success?
An automation-first approach is key to enhancing testing capabilities and increasing overall operational efficiency. However, I would suggest justifying its implementation, so that it can deliver the promised value.
Quality assurance
Test automation
2 December 2022,
by Dileep Marway
3 min read
What is a сulture of happiness for a QA company?
Great culture drives employee engagement and satisfaction, contributes to an impressive work output, and improves organisational performance. But how do we build a culture of happiness? In this blog, I will share more on the answer to this.
Quality assurance
30 November 2022,
by Dileep Marway
3 min read
Agile QA – what is needed for greater flexibility and speed?
What should your QA team focus on to become truly agile, enable quality at speed, and contribute to lasting performance improvement? In addition to introducing test automation, I suggest considering shared responsibility for software soundness.
Quality assurance
28 November 2022,
by a1qa
9 min read
Interview with Dileep Marway on a series of articles “Agility and speed: Supercharging your business strategies with QA”
We cooperated with the VP of Engineering and Quality at SHL to present you with a series of his blog posts on: culture of happiness, test automation, and Agile-driven QA. Happy reading!
Quality assurance
Software lifecycle QA
Test automation

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.