Case Studies

Complex Testing of Wellness Portal and Mobile Apps

A1QA was approached by the software development company to perform complex testing of the online wellness portal and mobile apps. The customer's main business objective was to have the product live within the shortest time possible.

Services Offered

  • Functional Testing
  • Mobile Testing
  • Compatibility Testing
  • Localization Testing
  • Integration Testing
  • QA consulting
  • Test Automation

Team

  • 25 QA engineers

Duration

  • Ongoing project (since 2014)

Tools and Technologies

  • Html/CSS, JavaScript, jQuery
  • Java
  • Apache
  • VMware Workstation, Virtual Box
  • Atlassian JIRA, Atlassian Confluence
  • OracleDB
  • Paradigm
  • TestRail
  • Fiddler, SoapUI
  • Internet Explorer, Firefox, Google
  • Chrome, Safari, EDGE
  • Windows 7 – 10, Mac OS X, iOS 6-9,
  • Android 4.0 – 6.0

Product Overview

The portal is designed to engage employees of various companies in healthy lifestyle behavior. The registered employees get total health assessment based on their lifestyle and clinical measures (height, weight, age, blood pressure, etc.).

Once the assessment is performed, the system provides a user with his/her health age as compared to the chronological age and generates a comprehensive plan to improve the person's well-being. The plan is a complex guideline with a number of goals to achieve by the specified date. Once goals achievement is proved, an employee gets some points and is rewarded by his/her employer. In such a manner, employers motivate their staff to adhere to the healthy lifestyle.

Four dedicated teams were assembled at A1QA to complete the tasks: three of them ensured quality of web services and one ran tests on mobile devices.

Project Scope

The development of the portal is a continuous process. At the very start there was a single web service with a limited number of features. As time passed and user base increased, the product was scaled to the set of different components.

As a result, it grew into several interrelated products: user's questionnaire, training courses and webinars promoting healthy lifestyle, online health store, healththeme game, mobile application. Each of the components had to be thoroughly examined and tested.

The portal is aimed at users from the world over so its content and UI objects are translated into multiple languages. A1QA performs localization testing to check the localized versions for all culture and locale settings. Further on, the portal acquired responsive design to provide top-notch user experience. In this context, QA team had to run UI testing.

Active application of the CMS like system to manage the contents of the website attracted lots of white label partners. This enhanced the customer's stature due to the extensive advertising campaign by one of the WL partners and led to the increase of testing required.

HIPAA Compliance

When assuring quality of a healthcare domain software, it's integral that the team understands all specific regulations set force by the Health Insurance Portability and Accountability Act (HIPAA) which defines standards for safeguarding sensitive and condential data.

All A1QA engineers on the project were HIPAA-certified specialists. To ensure full compliance with HIPAA, they took the regulations into account when creating testing plan and strategy.

Test Environment

The product's complexity determined the necessity to perform testing in 27 environments and against various mobile platforms and operating systems.

While testing, A1QA engineers used custom-developed tools to prepare test data, emulate real-life scenarios and save time.

Mobile and Wearable Apps Testing

Later on, the customer asked A1QA engineers to address mobile app testing as well.

Our team started with ensuring quality of two mobile applications (1 for iOS and 1 for Android). Nowadays the number of applications tested and released monthly has increased to 8 apps.

Every application is designed for different clients, which became possible due to white labeling (the same code base is used for different applications and specific features can be turned on/off for different clients).  

All testing is performed using real mobile and fitness devices to prevent emulation programs issues.

To pick right set of mobile devices for testing, A1QA team collects statistics of the device usage worldwide (using Google Analytics) and assembles the device matrix. The selecting approach is very flexible: once additional coverage is needed, additional devices can be added.

Test Automation

A1QA has implemented flexible test automation solution that covers 95% of checks without any tool limitations and easily maintains the scripts in case of product changes. It helps to save up to 40% of manual testers' efforts during regression testing, as well as providing results faster for critical features. The solution implementation has reduced costs of testing.

Integration Testing

Mobile apps are integrated with Google/Apple Maps, Foursquare, HealthKit, Google Fit, Apple Watch, such applications as Calm and Headspace.

The portal itself is integrated not only with native mobile application but with different third party services:

  • Google Analytics
  • Map My Fitness
  • Salesforce
  • Amazon
  • Hyatt
  • Gyft
  • Vindicia

Integration quality is verified by A1QA team on a regularly basis.

QA Consulting

To optimize the testing processes, QA consultants from A1QA have been addressed. They've investigated the testing process, analyzed all QA artifacts and have developed comprehensive recommendation. Once implemented, the recommendations have significantly contributed to the testing process optimization.

A1QA consulting specialists have delivered the following works:

  • Formulated testing strategy, test model management strategy, and incidents response strategy.
  • Developed key testing efficiency indicators.
  • Defined responsibilities, roles, and duties of different team members.
  • Established release cycle and build completion process.
  • Suggested corrective and preventive measures.
  • Assembled probable risks register and developed measures to address them.

Defining and Overcoming Challenges

Being a complex portal, the product had many shortcomings our team had to get rid of.

The product has been developing for 5 years by distributed teams from different countries and continents. The process was poorly organized and QA importance was hugely undervalued. Before the A1QA team got on board, the compliance with tight deadline was given a higher priority than the quality of the delivered works.

Challenges A1QA solutions
Development team didn't allow enough time for QA, which lead to missing deadlines
  • Spent time tracking
  • Planning with all parties involved (QA, UAT, 3rd party services)
  • Requirements testing & story enter criteria introduction
  • PDCA cycle implementation for estimation process
  • Risk management implementation
Low quality of the product
  • Story exit criteria introduction
  • Expected level of quality defined
  • Transparent deployment process
  • Quantitative management implementation
Production environment specific issues
  • Up and running of Staging environment with the same configuration as Production to catch environment specific issues earlier than Production deployment
Fast growth of the team
  • Squadification as a team structure adjustment
  • Regular cross-training
  • Knowledge base creation
  • Standard knowledge transfer plan for newcomers
Need to develop quality management principles
  • Statistics collection
  • Quality metrics development
Huge amount of test cases to manage
  • Using TestRail instead of separate Google Docs for test cases
Lack of mobile testing devices
  • Purchasing of specific devices to run test only on real devices and not emulators

Results

  • The project has started in 2014 and runs up to date. During this time A1QA team on the project has increased more than six fold (from 4 QA engineers to 25). The specialists have been executing all tests cases within the specified time frame and budget.
  • All tested products are currently available live and are appreciated by 3 mln unique users. The customer, completely satisfied with the delivered results, has prolonged cooperation with A1QA.