Portfolio
Quality assurance of app for running community with primary focus being on performance testing
a1qa was reached out to provide the quality of the solution for the global running community.
Overview
The client was the software development vendor from London, who elaborated a location-based running app that would connect the global running community.
The app combined the founders’ love of the sport and their desire to enhance the social side of it.
The developed app allows sharing details of upcoming runs, then searching for and connecting with likeminded runners.
One of the main requirements was that the app should be available to a large number of concurrent users with no compromises on its performance characteristics.
Services offered
Functional testing
Performance testing
Project scope
The a1qa team performed one iteration of functional testing and two iterations of performance testing.
Within the scope of functional testing works, the engineers prepared test documentation (Test Survey), conducted testing on 5 devices (2 for iOS, 3 for Android), and made an informative report on the product quality.
Having functional defects fixed, the performance testing engineers got down to work. They had to simulate users’ most common navigation paths to provide information on the API requests.
To get informative and fact-based testing results the team fulfilled the following tasks:
- Thoroughly investigated the app functionality
- Developed realistic testing scenarios that covered the key functional modules
- Developed six performance testing scripts for Android and iOS versions of the app
- Established the mechanism of filling the system with test data
- Set up online monitoring of the system hardware resources
- Run stress, load, and scalability testing.
Test Documentation
Upon the project completion, the customer was provided with the following testing artifacts:
- The Quality Report with the description of all defects by their priority, devices, and functional modules of the app where they were detected
- List of the conducted functional tests
- Structured recommendations on the system environment and app quality improvement.
Challenges and solutions
The app was functioning in the UK locale only:
- The engineers employed Xcode and MockLocation tools to change their geolocation and conduct testing from their premises outside the UK.
When the scripts were recorded, a number of requests from the API documentation were missing or functioned incorrectly:
- The a1qa team added the lacking requests manually adhering to the implemented requests structure.
It was important to make sure that test data (users’ passwords, phone numbers, credit cards data, etc.) was valid:
- The team developed a Python script that generated test data in the correct format.
Complex logic of scripts, parameterization and correlation of the dynamic data:
- The engineers made use of the JMeter advanced functions – cycles execution, splitting, variable randomization. Java code for JMeter was implemented.
There was no app version with WebSockets for the client and the scripts were to be developed in tough compliance with the project documentation:
- The communication strategy was enhanced: all the WebSockets related issues were accumulated and discussed with the Dev team.
Technologies & tools
Results
-
Performance testing results showed that the system wouldn’t cope with the necessary number of users.
-
Hardware bottlenecks were detected.
-
The a1qa engineers also detected a number of functional defects that would manifested themselves only under the applied load.
-
It was found out that the WebSocket would close the connection under huge load emitting an error to the user.
In numbers
3
months of the project duration
7
engineers on the project
1
round of functional testing
2
rounds of performance testing: before and after the app code and environment optimization
85
defects detected
15
of the detected defects affected the app performance