Blog

How DevOps is changing software testing

The move to the DevOps methodology can help production teams bring software to their customers faster.
11 October 2018
Quality assurance
Test automation
The article by a1qa
a1qa

Once upon a time, software QA was done by teams taking small aspects of a piece of software and testing every conceivable variation. Times have changed and so has the software industry. We no longer think of software as being packaged in a box and bought off store shelves. Old-style massive full versions of software are no longer the norm; now most companies deliver software via the internet in small bursts. Unless there is to be a major change, the updates are just made and instantly applied without users having a clue.

The move from gigantic disc-based software to these incremental on-the-fly updates required a great deal of thinking behind the scenes. First, the programmers became Agile and set themselves small achievable goals. They would spend a week, maybe two, working on adding some features or fixing a couple of bugs as in these images:

Bug Queue image courtesy of monday.com
Features Backlog image courtesy of monday.com

While the old mindset had developers creating software and handing it off to the operations team to maintain, the newer DevOps approach has those programmers working together with their operations team throughout the process.

The key is that now much of the company is both working and planning to get the product to an agreed-upon state. Instead of waiting for the programmers to push their possible final build to the testers after, those testers are now involved from the get-go.

Granularity

To make a build cycle take as little time as possible, its scope is recursively reduced. Here is an example of one of those tasks, or stories, being further refined:

SourceStackExchange

This shows an attempt to break down and refine the goal into the smallest bites possible. These smaller and smaller bites make it easier to plan software testing. QA engineers can better estimate the time needed for testing and fixing any defects discovered.

Number of tests

When talking about testing in Agile environments, the team has to be able to trust that the code will perform as expected. That trust, or its lack, can be easily understood if you look at it in light of the number of tests. More testing means higher probability that your program will build and deploy successfully. Your confidence about the application will be much higher if it passes 295 out of 300 tests than if it passes two out of three.

Size of tests

The other side is the size of the tests you need to do. With large, multifaceted applications, it may be difficult to test the whole program after every build. In those cases, you test only the parts that are important or that have been changed. There is never a set-in-stone group of required tests, your teams can add or subtract as needed.

Frequency of tests

How often you test can depend on the types of tests and what resources they require. For an application that consists of a bunch of smaller programs, it is pretty straightforward to test just what has been changed. If, however, your program is huge, you may not be able to run all your tests after each section of code is committed. In that case, you run the subset of tests that you need to and then run your full test at night. Basically, if the team decides that a test needs  to be run, its size or runtime will not matter. For a test that takes three days to run, you just automate it to run every three days at a certain time.

Automation and continuous delivery

In the past, testing engineers focused only on one part of the software development cycle—testing the code and pushing it on to the programmers. Although with DevOps, that has changed—with continuous integration and delivery, testers are now required to take responsibility for quality throughout the entire development process, not just the old QA phase.

Building more tests into the software

In the world of DevOps, it is important to have regular deployments—the automation of processes can be a big help in making that happen. That said reduced deployment time and more frequent software releases mean more testing will need to be done. If there is no automation, large numbers of test cases must be run manually, which will slow down the whole process. Teams need to increase their testing and will have to automate more and more of it.

Automation for testing environments (virtualization)

A test environment is an internal setup of software and hardware that emulates the environments where the finished product would eventually deploy. The team can then test the software on virtual systems to catch defects before it rolls out to production. Cap Gemini’s included diagram shows several varieties of testing environments.

Test Environments image courtesy of capgemini.com

If testing environments are automated effectively, the amount of manual intervention will be greatly reduced. However, if the automation is done incorrectly, the QA team will have to fix test environments manually, which will ultimately increase the amount of manual work—not something you would expect from automation.

Effect on teams

In the old model, QA wasn’t in touch with the development and delivery processes. DevOps get them working together.

Increased communication and collaboration

In DevOps, QA engineers must actively collaborate with other departments. They attend planning sessions and communicate to the rest of the project team what will be tested next, when, how, and by whom. This may require the QA to step out of their comfort zones, but it will be necessary for effective work.

The right project management solution is critical to keeping all teams aligned by giving everyone clear visibility on the project’s progress between meetings.

Fail faster, fix faster

Coding is a practice that often requires extreme concentration. Getting back on track, or more specifically, getting back to where you were mentally when you added a bit of code can be very difficult.

For this and other reasons, many teams work on the Fail Faster, Fix Faster principle. Jim Shore said in his 2004 IEEE article on the subject that, “failing fast is a non-intuitive technique. Failing immediately and visibly… makes bugs easier to find and fix.” This method makes it very easy for an error to stay in context for the developer. They will not have to search for hours to fix it.

Some software is given the ability to compensate for errors. This can result in a bigger mystery fail. Fail fast suggests that the error should be allowed to happen right away so that it can be detected and fixed faster. These bugs appear sooner and don’t reach production, thus reducing costs.

Closing thoughts

The move to the DevOps methodology can help production teams bring software to their customers faster. In addition to having the developers work closely with the operations team, we now have the QA testers pulled into this collaborative effort. All three teams work and plan the steps for the full rollout—keeping in mind that this consists of lots of little rollouts. The QA team is now there in the thick of things automating and testing their little hearts out. No longer are they necessarily the last stop feeling left out in the cold or blamed for the product release being late.

Author: Steve Medeiros, a writer for TechnologyAdvice.com with an extensive background in technology, software, and customer support.

More Posts

13 May 2021,
by a1qa
4 min read
How to attain in-sprint test automation to reinforce development processes
Learn how in-sprint test automation improves business workflows. Consider the top 5 essential steps to introduce it smoothly.
Test automation
12 April 2021,
by a1qa
5 min read
Watch out for 6 telecom trends and QA tips to implement them
In our article, we unleash the upcoming telecom trends and reveal how to be certain in introducing them correctly by applying QA practices.
Agile
Test automation
31 March 2021,
by a1qa
4 min read
QA scenario to introduce 6 eCommerce trends in 2021
Discover what trends will rule the eCommerce industry in 2021 and how QA can help implement them with confidence and ease.
Cybersecurity testing
Test automation
25 February 2021,
by a1qa
4 min read
9 QA points for delivering high-quality SaaS-based solutions
In the article, we’ve gathered 9 QA factors relying on the SaaS specifics that may help to perform SaaS testing with ease.
Cloud-based testing
Cybersecurity testing
Functional testing
Performance testing
Test automation
29 January 2021,
by a1qa
4 min read
3 do’s and 3 don’ts in BFSI software testing
Considering BFSI to be a fast-paced industry, how to keep up with such velocity? We’ve prepared 3 do’s and 3 don’ts that help sustain the rush and high software quality.
Functional testing
Mobile app testing
Test automation
18 December 2020,
by a1qa
4 min read
Top 5 QA and software testing trends in 2021: responding to a global situation with ease
Rapidly approaching 2021, a1qa proposes to have a closer look at the most influential trends empowering QA teams to perform even more effectively.
Agile
QA trends
Test automation
30 November 2020,
by a1qa
5 min read
Acumatica: ensuring sound business operations with well-tested ERP system
Internal business activities are advancing, while ERP systems’ usage is growing rapidly. Explore how to ascertain their accurate work through timely applying QA.
Big data testing
Cybersecurity testing
ERP testing
Functional testing
Performance testing
Test automation
13 November 2020,
by a1qa
5 min read
QA for media and entertainment
Read the article to explore why QA is a must for the media and entertainment sector and how to perform software testing effectively.
Functional testing
Mobile app testing
Performance testing
Test automation
Usability testing
28 October 2020,
by a1qa
5 min read
eHealth software testing: taking the digital Hippocratic oath
Medicine has broken new ground. However, there’s still no room for errors. Get to know more information about effective testing approach in the health sector. 
Big data testing
Functional testing
Performance testing
Test automation

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.