Regression testing: essence, stages, approaches

The article sheds light on the regression testing, which is often taken for granted and doesn’t pose many questions to the QA team. We hope IT project managers and product owners will find this information helpful in getting a deeper understanding of the process.

Let’s start. Software development is impossible without code changes. New features, upgrades, enhancements happen very often and influence the already existing features. This is the main reason why it’s vitally important to make sure the recent code changes haven’t affected the old functionality. And this is objective of regression testing.

Regression testing goal is to assure that the recent code changes haven’t altered or broken the existing bug-free functionality.

Let’s take an example. Say, your team has developed a flashlight app for mobile devices. The app has only two functions: switching the flashlight on and switching it off. You perform functional tests to make sure that the app operates as it should. Testing gives positive results and you have nothing to worry about.

As time goes by, you come to think that your app may also turn on when a user shakes the phone. You implement the feature and carry another round of testing. The new feature works correctly, but you also have to retest the switch on/off feature to make sure that the new functionality hasn’t hampered it.

This is a vivid example of regression testing.

Regression testing and project management methodologies

Agile

If you manage software development projects, you know that in Agile it is divided into iterations or sprints. A sprint normally lasts four-six weeks and results in the delivery of a specific piece of software that can perform some functions. Regression testing should be carried out at the end of every sprint before the piece of software is reviewed and demonstrated to the stakeholders.

Challenge. Strict deadlines and stumbling blocks in development make no room for thorough regression testing.

Solution. Partial regression testing is carried out after every sprint, and full testing before major releases.

Waterfall

The principle of the waterfall model states that each phase of the software development lifecycle must be completed before the next phase will start. Thus, testing must begin only once the development is over.

Challenge. It’s difficult to debug the product and introduce any code change. Not surprisingly, this model can be applied only when the software requirements are clear from the very beginning.

Solution. Before the development starts, make sure the functional requirements are testable, unambiguous and possess all other qualities of good requirements. Third-party business analysis professionals may be invited to perform impartial requirements testing.

Hybrid

The hybrid methodology combines Agile with waterfall principles. The planning and requirements phases coincide with the waterfall approach. The design, development and testing follow the Agile methodology. So, as we mentioned above, full and partial testing should be performed to detect all the bugs.

Contact us to get more information on how our services can help your software deliver the expected value to your business.

7 steps to execute regression testing

No matter what methodology you follow, with any change introduced to the app (even a minor one) the following steps should be taken to perform thorough regression testing:

1. Analyze the changes requirements and detect possible areas that might be affected.
2. Compile a suite of relevant regression test cases.
3. Run a first round of regression testing.
4. Report bugs.

Any bug is to be reported in a clear way with a detailed description and steps to reproduce it. If possible, videos, screenshots, logs should be attached to the report.

Developers fix the reported defect.

6. Bugs verification.

At this stage, QA engineers verify that the reported defect has been fixed. If the problem stays, engineers report it back to developers. In some cases, it is worth to discuss if the bug should be fixed. Critical and major defects require fixing. While some of the minor bugs that take a lot of time and efforts to fix may stay. Especially, when they won’t be noticed by the end users.

7. Run a second round of regression testing.

Our experience shows that you’ll average with three rounds of regression testing to address all the bugs and stabilize the whole application.

Manual regression testing vs automated

Regression tests are good candidates for automation as they run regularly and take a lot of QA efforts that can be freed up to conduct exploration of a product and verify its work in the most unusual and rare scenarios.

However, manual testing is still alive.

Manual testing

On small projects that last for a couple of months only it’s not worth introducing test automation. Then QA engineers perform regression testing manually with no special tools required except for bug tracking (JIRA, for example).

As the project scales, the application will get a new feature with every succeeding release, which keeps increasing the scope of regression testing. In this case, automated testing should be included in the QA scenario.

Automated testing

The stakeholders should decide on automating regression testing, while the job of automation will fall on shoulders of professional test automation engineers with good knowledge of a programming language. The code-based testing has many advantages that can’t be underestimated:

• Improved product final quality
• Accelerated time-to-market
• Optimized total cost of ownership (TCO)
• Reduced overall QA costs

Here is a good example of how the a1qa team saved up to 40% of manual regression testing efforts through automation.

When deciding on automating regression testing, the main point is to take into account project specifics, assess risks and calculate opportunity costs to choose a well-fit ratio of manual and automated techniques.

In a nutshell

 According to the World Quality Report 2017, the proportion of IT budget spent on QA is about 26%. Based on our experience, we know that 40-70% of the QA efforts belong to regression testing.

If you calculate the value in real money, you’ll see that it would be reckless to leave regression testing to luck. The sound strategy is required to make it effective and guarantee the high quality of your product.