Blog

Assuring quality of mobile app. stage 1: strategy

Learn how to develop effective mobile app testing strategy and avoid misunderstandings with the Customer.
31 January 2017
Mobile app testing
The article by Pavel Novik
Head of testing department at a1qa

a1qa Company specializes in assuring quality of software products. Mobile apps testing is one of the main services with more than 220 company’s employees certified to engage in mobile testing projects.

Today we offer you a series of articles prepared by Pavel Novik, head of the Mobile Testing Center of Excellence. The series will be made up of 5 articles. One article is mapped to one project stage. We’ll start with the proper strategy development. Then we’ll touch upon test documentation preparation and testing itself. Product release and associated testing activities will be the subjects of the fourth piece. And to make it more interesting for you to cope with all this amount of theory, we’ll tell you how we’ve decided to test the popular mobile game of 2016 and what bugs we’ve discovered.

So let’s imagine that we’ve been addressed to assure quality of some mobile application. The customer’s goal is to release the product to the market without defects.

We’ll start with a strategy. In our context strategy is the selection of testing devices and planning of all testing activities. The strategy should be presented and agreed upon with the customer.

The first step is to analyze the product and get answers to the following questions:  

  • What are the project goals?
  • What is the target audience of the product?
  • What are the app’s technological specifications?

The answers to these questions may help estimate the project scope and assign it to the employees with the right set of skills.

What devices to test on?

Once we’ve received questions to the answers listed above, we have to compile a list of the rights devices to test on.

Consider the following factors:

  • Customer’s preferences
  • Target markets/regions
  • Target audience
  • Application specs (oldest and latest OS versions supported, app requirements to hardware, etc.)
  • Budget limits and timeframes.

Let’s analyze every point separately.

  1. If the app is already available live, the customer may have gathered stats about the user’s devices. Some customers also have their own preferences about the testing devices. Check if there are any in your case.
  2. Target region also matters. If the app is aimed at the Asian region, give attention to the Chinese smartphones Xiaomi and Meizu. In Europe Samsung devices are more popular, while in the USA Google (Nexus) and HTC devices hit the market. However, making guesses isn’t a good way. Analyze statistics to make the informed decision.

There are numerous free resources that enable to see, compare and analyze the list of the most and least popular devices. Consider several of them and compile the final list. Some of the most popular are Antutu, Akamai, DeviceAtlas, Futuremark, etc.

For example, the most widely used in North America in December 2016 are the following:

  1. Knowing target audience will help to limit the number of devices according to their “age”. Statistically, the youth prefer to use the latest devices, while the elder generation is loyal to the smartphones and tablets of 3-4 years on the market.
  2. Analyze the hardware and software specifications and remove the outdated or, vice versa, most recent devices that aren’t supported by the app from the list.
  3. And the last but not least important point is to comply with the budget and time limits. Sometimes you’ll find it necessary to test on 10-20 devices to provide full testing coverage. However, it may be impossible due to tough deadline or limited budget.

At this first stage you may come to face some odd questions from the customer. You’ll have to deal with them to avoid misunderstandings and make sure that the final results respond to their expectations.

Most commonly the customer will ask you about testing devices. “Why do you need so many devices?” “Why will it take you so long to click through the app?” Don’t grudge the time to answer them. A brief presentation about testing process, number of checks and the consequences of reducing the number of devices may be of help.

Initiate discussion on other options if you don’t have the required device for testing. Can the customer provide you with it? Can it be replaced by something else?

After addressing all the questions and gathering stats, we start drawing up a schematic matrix of configurations we want to test on. A test configuration is a combination of variable values (type of device, operating system, browser, CPU, database, etc.)

Every device type (smartphone, tablet, wearable devices) should be represented by various screen sizes and operating system types. Most commonly specific defects will depend on the operating system type. That’s why we recommend including the latest versions of OS in configurations (9.0 an 10.0 for iOS, 4.0, 5.0 and 6.0 for Android). Also don’t forget about beta updates. Include them into test plan.

Questions to ask the customer before testing

Is the app integrated with the third party services? What is the application monetization strategy? If it’s applicable, schedule integration and payments testing.

Was the application tested before? Previous testing results will be useful to estimate the quality of the app and to analyze the problems that were addressed by the predecessors. Moreover, ask the customer whether he was satisfied with those results.

Does the app support multiple languages? Then it may be worthwhile to perform localization testing. Negotiate it with the customer.

Finally, don’t forget to request access to the back-end and test accounts. It can be especially useful if you’re engaged in testing game app. Very often it’s not enough to pass the game to the end. Accessing the backend, you’ll be able to generate various playing scenarios and in-game events, say nothing about saving time.

When we’ve discussed all questions it’s time to start planning future testing activities, i.e. preparing of test documentation and assigning QA engineers. This is what we’ll clarify next week.

More Posts

5G impact
31 May 2021,
by a1qa
4 min read
5G network impact on mobile app testing
Check out what 5G connectivity will bring to the IT world and how it will modify mobile app testing.
Cybersecurity testing
Mobile app testing
Performance testing
15 March 2021,
by a1qa
4 min read
Mobile app performance testing: getting high software efficiency
Explore 3 cornerstones of mobile app performance testing and QA steps on how to execute it successfully.
Mobile app testing
Performance testing
29 January 2021,
by a1qa
4 min read
3 do’s and 3 don’ts in BFSI software testing
Considering BFSI to be a fast-paced industry, how to keep up with such velocity? We’ve prepared 3 do’s and 3 don’ts that help sustain the rush and high software quality.
Functional testing
Mobile app testing
Test automation
13 November 2020,
by a1qa
5 min read
QA for media and entertainment
Read the article to explore why QA is a must for the media and entertainment sector and how to perform software testing effectively.
Functional testing
Mobile app testing
Performance testing
Test automation
Usability testing
24 July 2020,
by a1qa
4 min read
OWASP as a guide to mobile apps security testing
More apps, more sensitive data, higher security levels... Learn how companies address the challenge of providing secure solutions harnessing unbiased safety recommendations.
Cybersecurity testing
Mobile app testing
12 December 2018,
by a1qa
3 min read
Benefits of native mobile app testing on real devices
At the early stage of every QA project, the team should make a choice: run tests on emulators, simulators, or real devices.
Mobile app testing
14 November 2018,
by a1qa
4 min read
How to use computer vision to improve your test automation
Learn how computer vision helps automate Android and iOS mobile application testing.
Mobile app testing
Test automation
20 July 2018,
by a1qa
4 min read
How QA providers help telecom companies overcome 21st century challenges
Let's examine the most vital telecom challenges and ways to overcome them with the help of Software Testing and QA companies.
Mobile app testing
Quality assurance
24 January 2018,
by a1qa
4 min read
Testing trends for 2018
What trends will mark the software quality assurance of 2018? Read the article not to miss on the potential benefits while shaping your QA strategy. 
Agile
Big data testing
Cybersecurity testing
Mobile app testing
Performance testing
QA trends
Test automation

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.