Blog

Software testing: how to choose what to test

To test or not to test isn’t a question in this digital age of ours. Now the focus has shifted towards...
27 December 2018
QA consulting
Quality assurance
The article by Nadya Knysh
Head of US Office

Nadya Knysh – Managing Director at a1qa – on how to decide on the right amount of QA and choose the right test strategy.

Is it possible to choose the single right amount of quality assurance? The simple answer here is NO. And here is why.

There are multiple factors that impact the scope of testing and your testing strategy.

#1 Solution scope

In my experience, about 80% of software projects are delivered using Scrum as the project methodology.

Scrum (like most Agile frameworks) is all about change. New requirements are coming in every sprint, priorities are changing, development is following business needs…

What does that mean for QA?

Well, your planning horizon is pretty limited. Of course, you’ll define your team capacity after a few sprints.

However, you cannot be 100% sure of what skill set may be required in the next sprint – are we testing APIs or UX? Will we have new devices in the office by the time we need to test against them? Will the development team’s capacity impact the deliverables dates? All these questions impact your ability to plan and predict how much QA you’ll need and when. That doesn’t mean you shouldn’t plan, though.

#2 External factors

Imagine you have a one-year roadmap to deliver a brand-new solution to the market.

It’s December 2018 when you start developing, which means Apple has already released its 2018 updates while Samsung is still working on its presentation for February 2019.

Oh, and by December 2019, when you plan to release, Apple will have delivered another presentation on 2019’s new features, hardware, screen sizes, and who knows what else.

So, when planning QA, you should look at up-to-date statistics on what devices your target audience is using to interact with your software.

By the way, the iPhone 7 is still pretty popular, so your first thought might be to include it on the list of supported devices.

However, when thinking from a strategic perspective, by December 2019, the iPhone 7 will be three generations old. So the question is – do you care about it? And what if you do all your development and testing around the Apple Watch and then, a few months before the release, Apple recalls all the devices due to some critical issues?

So, my recommendation is: plan but be flexible. It is not the strongest of the species that survives, nor the most intelligent. It is the one that is most adaptable to change.

What are the most important quality aspects to be tested?

That’s a very good question! But I have found a very simple answer.

A long time ago, I was talking to a solution architect who was developing the architecture for a SaaS solution: highly secure, customizable, and with (hopefully) high performance.

When he was walking me through the architecture ideas (in other words, a draft version), he was explaining why he needed this feature and that feature here and there through quality attributes – a concept well defined in ISO 25010.

The full list of quality attributes is now available all over the Internet. So now, when defining a QA strategy for any of our projects, I recommend reviewing this list again and asking a question: is that important for my product now? In most cases, your answer will be YES in terms of functionality and UI/usability. That’s why you’ll plan functional, UI, compatibility, and usability tests.

Compatibility is the one that is often overlooked.

Always remember that you cannot make your audience use the same browser or smartphone that you do. You have to accept their choice and support whatever they like.

Side note: check your audience geography; people in China use very different smartphones from those used by people in the US.

Let’s say you develop an internal accounting solution for your CFO and two to three other analysts or accountants.

In this case, if one report that you only generate annually takes five minutes to generate, you may not care that much about the solution performance. But what if your solution is a stock market software? Performance is critical here: the market changes in nanoseconds and five minutes will cost you a fortune.

Security is another big topic. Do you store personal information (like SSN and DOB) or your customers’ credit card details? Oh yeah, security testing is a must.

And believe me, you’d rather be compliant with HIPAA, FDA, PCI-DSS, and other regulations than go to court against your customers.

Summing up

To test or not to test isn’t a question in our digital age. Now the focus has shifted toward choosing the right testing strategy that will meet the requirements of the software developed, end-users, and business stakeholders.

Flexible planning and setting priorities related to your product is what will help you make the smart choice.

More Posts

why-do-bugs-get-missed
17 April 2023,
by a1qa
4 min read
Why do bugs get missed? Learn the problems and tips to avoid them
Still, finding overlooked bugs after the app goes live? Let’s find out why this happens and how to fix it.
Quality assurance
Test automation
a1qa-articles
31 January 2023,
by a1qa
5 min read
Best of 2022 by executives: 8 most visited a1qa blog posts
Let’s look back and revisit the most visited a1qa articles of 2022!
Quality assurance
Test automation
qa-trends
12 January 2023,
by a1qa
4 min read
The future of software testing: top 4 impactful trends that will dominate in 2023
Consider the major industry trends for the upcoming year to know how to improve current QA strategies and stay ahead of the curve.
QA trends
Quality assurance
Test automation
test-automation
7 December 2022,
by Dileep Marway
3 min read
Release at pace with test automation: What, why, and how to measure success?
An automation-first approach is key to enhancing testing capabilities and increasing overall operational efficiency. However, I would suggest justifying its implementation, so that it can deliver the promised value.
Quality assurance
Test automation
what-is-a-culture-of-happiness
2 December 2022,
by Dileep Marway
3 min read
What is a сulture of happiness for a QA company?
Great culture drives employee engagement and satisfaction, contributes to an impressive work output, and improves organisational performance. But how do we build a culture of happiness? In this blog, I will share more on the answer to this.
Quality assurance
agile-qa
30 November 2022,
by Dileep Marway
3 min read
Agile QA – what is needed for greater flexibility and speed?
What should your QA team focus on to become truly agile, enable quality at speed, and contribute to lasting performance improvement? In addition to introducing test automation, I suggest considering shared responsibility for software soundness.
Agile
Quality assurance
interview-with-dileep
28 November 2022,
by a1qa
9 min read
Interview with Dileep Marway on a series of articles “Agility and speed: Supercharging your business strategies with QA”
We cooperated with the VP of Engineering and Quality at SHL to present you with a series of his blog posts on: culture of happiness, test automation, and Agile-driven QA. Happy reading!
Agile
Quality assurance
Software lifecycle QA
Test automation
optimizing-budget-with-qa
31 October 2022,
by a1qa
4 min read
Optimizing telecom budgets with QA outsourcing: everything you need to know. Part 2 
Welcome to Part 2 of our blog on QA outsourcing and optimizing telecom budget with it. Let’s delve deeper into the topic!
QA trends
Quality assurance
optimizing-budget-with-qa
27 October 2022,
by a1qa
4 min read
Optimizing telecom budgets with QA outsourcing: everything you need to know. Part 1
Learn how to optimize telecom’s quality assurance expenses by relying on a trusted QA partner.
QA trends
Quality assurance

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.