Software testing: how to choose what to test

To test or not to test isn’t a question in this digital age of ours. Now the focus has shifted towards...
27 December 2018
QA consulting
Quality assurance
The article by Nadya Knysh
Head of US Office

Nadya Knysh – Managing Director at a1qa – on how to decide on the right amount of QA and choose the right test strategy.

Is it possible to choose the single right amount of quality assurance? The simple answer here is NO. And here is why.

There are multiple factors that impact the scope of testing and your testing strategy.

#1 Solution scope

In my experience, about 80% of software projects are delivered using Scrum as the project methodology.

Scrum (like most Agile frameworks) is all about change. New requirements are coming in every sprint, priorities are changing, development is following business needs…

What does that mean for QA?

Well, your planning horizon is pretty limited. Of course, you’ll define your team capacity after a few sprints.

However, you cannot be 100% sure of what skill set may be required in the next sprint – are we testing APIs or UX? Will we have new devices in the office by the time we need to test against them? Will the development team’s capacity impact the deliverables dates? All these questions impact your ability to plan and predict how much QA you’ll need and when. That doesn’t mean you shouldn’t plan, though.

#2 External factors

Imagine you have a one-year roadmap to deliver a brand-new solution to the market.

It’s December 2018 when you start developing, which means Apple has already released its 2018 updates while Samsung is still working on its presentation for February 2019.

Oh, and by December 2019, when you plan to release, Apple will have delivered another presentation on 2019’s new features, hardware, screen sizes, and who knows what else.

So, when planning QA, you should look at up-to-date statistics on what devices your target audience is using to interact with your software.

By the way, the iPhone 7 is still pretty popular, so your first thought might be to include it on the list of supported devices.

However, when thinking from a strategic perspective, by December 2019, the iPhone 7 will be three generations old. So the question is – do you care about it? And what if you do all your development and testing around the Apple Watch and then, a few months before the release, Apple recalls all the devices due to some critical issues?

So, my recommendation is: plan but be flexible. It is not the strongest of the species that survives, nor the most intelligent. It is the one that is most adaptable to change.

What are the most important quality aspects to be tested?

That’s a very good question! But I have found a very simple answer.

A long time ago, I was talking to a solution architect who was developing the architecture for a SaaS solution: highly secure, customizable, and with (hopefully) high performance.

When he was walking me through the architecture ideas (in other words, a draft version), he was explaining why he needed this feature and that feature here and there through quality attributes – a concept well defined in ISO 25010.

The full list of quality attributes is now available all over the Internet. So now, when defining a QA strategy for any of our projects, I recommend reviewing this list again and asking a question: is that important for my product now? In most cases, your answer will be YES in terms of functionality and UI/usability. That’s why you’ll plan functional, UI, compatibility, and usability tests.

Compatibility is the one that is often overlooked.

Always remember that you cannot make your audience use the same browser or smartphone that you do. You have to accept their choice and support whatever they like.

Side note: check your audience geography; people in China use very different smartphones from those used by people in the US.

Let’s say you develop an internal accounting solution for your CFO and two to three other analysts or accountants.

In this case, if one report that you only generate annually takes five minutes to generate, you may not care that much about the solution performance. But what if your solution is a stock market software? Performance is critical here: the market changes in nanoseconds and five minutes will cost you a fortune.

Security is another big topic. Do you store personal information (like SSN and DOB) or your customers’ credit card details? Oh yeah, security testing is a must.

And believe me, you’d rather be compliant with HIPAA, FDA, PCI-DSS, and other regulations than go to court against your customers.

Summing up

To test or not to test isn’t a question in our digital age. Now the focus has shifted toward choosing the right testing strategy that will meet the requirements of the software developed, end-users, and business stakeholders.

Flexible planning and setting priorities related to your product is what will help you make the smart choice.

More Posts

31 August 2022,
by a1qa
4 min read
Why do bugs get missed? Learn the problems and tips to avoid them
Still, finding overlooked bugs after the app goes live? Let’s find out why this happens and how to fix it.
Quality assurance
Test automation
21 February 2022,
by a1qa
5 min read
Continuous testing 101: a comprehensive guide
So many companies take a chance to introduce continuous testing. What is it like, and how to smartly implement it? Let’s get to know in the article.
Quality assurance
Agile and DevOps in eCommerce QA_mini
30 September 2021,
by a1qa
5 min read
Agile and DevOps: Boosting the quality of eCommerce apps
What benefits do Agile and DevOps bring to eCommerce business, and how QA helps with that? Find it out in the article.
Quality assurance
AR/VR testing infographics mini
30 August 2021,
by a1qa
< 1 min read
AR/VR testing in retail: turning challenges into opportunities
Welcome to read the infographic on AR/VR in retail: new shopping experiences, issues, and how to address them with QA.
Quality assurance
6 October 2020,
by Dmitry Tishchenko
4 min read
A clear view of smart team scalability
Get to know how to scale your team sagely and gratify end-user needs and fast-paced tech-market requirements.
Quality assurance
6 August 2020,
by Elena Yakimova
5 min read
How to arrange fruitful joint work with an outsourcing QA team
The head of the web apps testing department sheds light on how to establish more transparent and effective work on the project cooperating with a remote QA team.
Quality assurance
4 June 2020,
by Vitaly Prus
4 min read
SAFe vs. Scrum, and PI planning essentials
Let's shed some light on the SAFe differences from Scrum that are to be considered by the development and QA teams who have migrated from Scrum.
Quality assurance
29 April 2020,
by a1qa
4 min read
5 lessons we learned from COVID-19
Here are five key lessons the businesses need to learn during this pandemic to somehow achieve the planned outcomes. 
Quality assurance
17 April 2020,
by a1qa
5 min read
QA-focused retrospective: identifying and solving project problems
The a1qa experts came up to consider an effective approach to identify project bottlenecks and get rid of problems successfully.
Quality assurance

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.