Blog

Ten million steps to release, or difficulties in wearables testing

Testing on wearable devices may be quite challenging, but never boring. 
8 November 2016
Web app testing
The article by a1qa
a1qa

Wearable devices are tech gadgets connected to the smartphone and worn in the user’s pocket or on the body creating constant and seamless hands-free access to the smartphone. As wearable devices are rapidly advancing in terms of technology and functionality, there appear more and more real-time applications that use the data gathered by a wearable device: most commonly these are health and fitness apps that track user’s physical activities thus helping stay fit.

In this article, Veronica Yanchenco, QA-specialist at a1qa, shares her experience of testing apps for wearables and gives solutions to the possible difficulties that a software tester might face.

The article looks at a real-life example of our mobile application testing services that we’ve delivered for one hotly debated project. The project includes a website and mobile app that help users keep healthy way of life motivating and rewarding them. For instance, a user shares another fitness achievement and gets points that can be spent on purchasing real fitness objects. Initially, wearable devices were integrated with the website. Later on, the integration of the mobile app with Apple Watch was added and the author has been working on it.

Complex test coverage

Too many wearable devices from different manufacturers – that’s a challenge. Obviously, it’s time and efforts consuming to test on all devices on the market. Supporting any additional vendor means additional development and testing integration that may be too costly for a customer. How is it possible to pick up the right devices to deliver holistic testing?

Solution: To test on the right devices, address the statistics and consider the preferences of the target audience, if possible.

Absence of the emulators

In order to get the required data, you’ll need to produce this data yourself, i.e. run, walk, raise your pulse rate, have a nap, etc. But keep in mind that as soon as you reconnect the device to another smartphone, all obtained data will be lost and you’ll have to generate it again.

Solution: Think wider to come round the app. If you need to produce thousands of steps (which was our case), try making movements that will increase the steps production or imitate them. In some cases, shaking hands with the smartphone might help.

Dependence on third-party applications

The integration issues may also block your testing process. If the application you use to receive data has some shortcomings, it might impact the whole system. It is also necessary to take into account that wearable devices wouldn’t always collect data correctly; the accuracy of reading may be far from perfect.

Solution: If you have found out that the accuracy of measurement suffers, try to figure out how it affects the user’s application behavior and come up with the questions to the following answers:

  • Is it real to handle the inaccurate data and prevent it from getting into the app?
  • Is it possible to change the business logics of the app to prevent the improper use of the data to get the in-app reward, for example? If there are such loopholes in the application, it may bring losses to the customer.

Multiple user scenarios

Some of the user scenarios can be predicted (i.e. the data coming gradually during a day or two devices running simultaneously), while others can hardly be foreseen. At times, it is difficult to guess where your user will log into the app and how it will impact the data collection.

Solution: Get into your user’s shoes or become a user yourself: use the devices in your daily life monitoring how it reacts to the changing usage environment.

To overcome any challenge you might face, stay creative and use your imagination to come up with nontrivial testing decisions as our team did in the following real cases:

Testing on towels

On one of the projects we discovered that if you put on your Apple Watch and open the heart rate screen and then take it off and place it on a towel or a pair of jeans (the fabric really matters), the pulse rate will reach about 200 beats per minute. Too much for a towel, agree?

When the customer learnt about our finding, he asked us to check how this behavior would affect the application functionality. We tried to reproduce that scenario, but had neither tissue in relief nor towels in the office. Do you know what helped us? A stuffed toy! When you placed the Apple Watch on it, the pulse reached 200 beats per minute.

The question arises: Is this situation life-real?

In fact, quite real: the user finishes his training in the gym, take the wrist-worn device off and put it on the pair of jeans or on a towel. However, it’s very difficult to foresee it and include it in your testing scenario.

 #10000000 steps

On another project the user complained that the app wouldn’t synchronize the completed steps. After studying the problem, we found out that it happens when the number of steps contains more than seven digital symbols, which should at least be 10,000,000. Pay attention that when the app is launched for the first time, it uploads all the steps taken over the past 90 days.

After simple calculations, we got about 111,000 steps per day, which is approximately 80-90 kilometers (50-55 miles). Is it real? To answer this question, we decided to find out how much real users walk a day. And here popular Instagram service turned out to be helpful as many people eagerly share the number of completed steps in their accounts.

Looking through multiple posts marked with the hashtag #steps, #activity, #healthapp, we came to the conclusion that the average number of daily steps lied in between 7,000 and 10,000. The most active users can take up to 20,000 but it’s next to impossible to complete more than 35,000 steps.

Another possible solution is to search for a more precise hashtag, #10000000 in our case. When we worked on the project there was only one sole photo with such a hashtag. Judging by the caption, someone has walked across the Great Wall of China. Unfortunately, there was no time period specified. Anyway, it is unlikely that someone could cover 80-90 kilometers (50-55 miles) for 90 consecutive days. The bug was fixed but the question who and how managed to get 10,000,000 steps in a day is still open.

Summing up, I can say that testing on wearable devices may be quite challenging, but never boring. After all, how can one get bored when running tests on a towel?

More Posts

7 January 2020,
by Performance R&D
6 min read
How to enhance performance of your web software product?
In this article, we are highlighting the aspects that can help get an objective picture of the performance health of your software product and make it more high-quality.
Performance testing
QA consulting
Web app testing
18 September 2019,
by a1qa
4 min read
Client-side performance: make your end users stay
Let's have a wider look at performance testing and discover how client-side testing can improve the customer experience of a software product.
Performance testing
Web app testing
26 October 2018,
by a1qa
4 min read
Slow website speed? Performance testing is not the only solution
Strive to deliver best possible user experience with your web solution? Consider client-side testing then.
Performance testing
Web app testing
9 August 2018,
by Pavel Novik
3 min read
Accessibility testing: the basics you would like to know
Do you want to increase your user base? Consider performing accessibility testing of your software. 
QA consulting
Web app testing
12 June 2018,
by a1qa
3 min read
Web application testing: short guide
What kind of testing suits your web project best? Check it out in the post.
Cybersecurity testing
Functional testing
Localization testing
Performance testing
Web app testing
11 May 2018,
by a1qa
3 min read
Testing point of sale: challenges and solutions
If the POS doesn’t function as expected it’s likely to cause significant troubles...
Functional testing
Web app testing
7 March 2018,
by Julia Ilyushenkova
4 min read
Parallel testing to ensure billing system migration quality
What needs to be considered and actioned as part of the planning and execution of a successful Parallel Testing for the Telecom industry.
Migration testing
Quality assurance
Web app testing
22 February 2018,
by a1qa
4 min read
Integration testing: real-life example
Websites, ERP, CRM, CMS systems... Smooth communication among them guarantees proper users’ requests handling, service delivery efficiency, and overall business success.
Quality assurance
Web app testing
29 December 2017,
by a1qa
3 min read
OWASP Top 10 2017: what’s new?
Aleksey Abramovich, Head of a1qa Security Testing department, comments on the recently published OWASP Top 10 Most Critical Web Apps Vulnerabilities.
Cybersecurity testing
Web app testing

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.