Assuring quality of mobile app. stage 3: testing

The strategy is developed, test documentation is prepared, the team is ready to begin.

Time to run tests. As we’ve previously said, mobile apps should be tested against multiple devices. However, we’ve noticed that the majority of defects are not sensitive to the environment and will be reproduced on any device. Accordingly, it’s not necessary to perform all tests on every device.

The first testing round shall include the full test and compatibility test.

• The full test is aimed at checking the entire functionality of the application. Testing is carried out in two different ways: applying positive and negative test cases. At this stage an engineer should check how the app interacts with the supporting back-end infrastructure. Specific mobile checks should also be performed in full. Moreover, during the full test the app’s compliance with iOS and Android Interface Guidelines should be verified (iOS Human Interface Guidelines, Android User Interface Guidelines).
• The compatibility testing includes checking of the main functionality and testing for displays of the devices with rare software and hardware specifications. This test presupposes only positive test cases and checking the most common use cases. It doesn’t include testing against unthinkable usage scenarios.

Mobile apps testing is a widely discussed topic. As our article is targeting QA managers, we won’t go into the most obvious issues and better specify some of the tips that can be neglected or undervalued at times.

1. Set up Wi-Fi hotspots

It’s very important to check mobile apps at different connection types and speeds.

Setting up multiple Wi-Fi spots will help save time for testing, emulate real-world working conditions and discover defects that could have been missed otherwise.

Have a look at the spots examples:

2. Perform testing on more than one device at a time

This done, you’ll check two or more interfaces for the same aspects simultaneously.

3. Conduct education of the project specialists

For example, at a1qa company there’s a Mobile Applications Testing Center of Excellence. It provides regular trainings and workshops for the in-house team.

More experienced senior testers share their expertise, talk about differences in mobile operating systems, show real bugs detected in real software. It’s also useful to learn how to apply utility software that will help gather logs, take screenshots, install build, etc.

4. Take advantage of the technical audit services

Do not have the complete certitude in your testing process? If not, take advantage of the technical audit services to assess “technical health” of your project.

To put it simply, experienced consultants will arrive to examine your workflow, ask relevant questions on how you test, report results, assign people, etc. As a result, they’ll come up with recommendations for improvement.

Solving ongoing issues

The project proceeds as it was planned? The customer doesn’t interfere and let you do your job? Great! However, it’s a rare case. More often, you’ll have to deal with customer’s questions, proposals, new requirements. Do you know how to cope up with them?

For example, you’ve offered the customer to perform testing on too many devices and asks you to cut the number down. You should explain the possible consequences and fulfil their requirements. Using the collected stats, remove the less popular configurations from the list.

Another case that may require discussion is the customer’s request to run tests against a device that you don’t have at your disposal. Find out the reasons of such a need and try to offer another option that has similar software and hardware specifications.

The customer’s development team can’t reproduce the bug you’ve reported? Sometimes it happens. If you reset the device, the bug may not be reproduced on your device as well. But it doesn’t mean it doesn’t exist. To avoid such disputable situations, accompany defects with video or screenshots. It will ensure that the defects won’t be returned to you marked as “To be reformulated”.

Test ending

At the end of every test iteration, the following artifacts should be generated:

• Defects submitted to the bug-tracking system.
• Completed test documentation and quality report. The quality report usually contains estimation of the application current version quality level and is based on the number of defects and their severity.
• Most critical issues description (from the point of view of the end user).
• Statistical data on the number of defects and their nature. Statistics helps to monitor product quality dynamics and estimate the quality of developers’ work.
• Recommendations for improvements.
• Conclusion on the product’s readiness for release.

Once testing is finished, don’t rush saying goodbye to the customer. Product release is coming and it won’t happen without quality assurance and control specialists’ participation.

Next week we’ll talk about how testers should support the product release.