Ten million steps to release, or difficulties in wearables testing

Wearable devices are tech gadgets connected to the smartphone and worn in the user’s pocket or on the body creating constant and seamless hands-free access to the smartphone. As wearable devices are rapidly advancing in terms of technology and functionality, there appear more and more real-time applications that use the data gathered by a wearable device: most commonly these are health and fitness apps that track user’s physical activities thus helping stay fit.

In this article, Veronica Yanchenco, QA-specialist at a1qa, shares her experience of testing apps for wearables and gives solutions to the possible difficulties that a software tester might face.

The article looks at a real-life example of our mobile application testing services that we’ve delivered for one hotly debated project. The project includes a website and mobile app that help users keep healthy way of life motivating and rewarding them. For instance, a user shares another fitness achievement and gets points that can be spent on purchasing real fitness objects. Initially, wearable devices were integrated with the website. Later on, the integration of the mobile app with Apple Watch was added and the author has been working on it.

Complex test coverage

Too many wearable devices from different manufacturers – that’s a challenge. Obviously, it’s time and efforts consuming to test on all devices on the market. Supporting any additional vendor means additional development and testing integration that may be too costly for a customer. How is it possible to pick up the right devices to deliver holistic testing?

Solution: To test on the right devices, address the statistics and consider the preferences of the target audience, if possible.

Absence of the emulators

In order to get the required data, you’ll need to produce this data yourself, i.e. run, walk, raise your pulse rate, have a nap, etc. But keep in mind that as soon as you reconnect the device to another smartphone, all obtained data will be lost and you’ll have to generate it again.

Solution: Think wider to come round the app. If you need to produce thousands of steps (which was our case), try making movements that will increase the steps production or imitate them. In some cases, shaking hands with the smartphone might help.

Dependence on third-party applications

The integration issues may also block your testing process. If the application you use to receive data has some shortcomings, it might impact the whole system. It is also necessary to take into account that wearable devices wouldn’t always collect data correctly; the accuracy of reading may be far from perfect.

Solution: If you have found out that the accuracy of measurement suffers, try to figure out how it affects the user’s application behavior and come up with the questions to the following answers:

• Is it real to handle the inaccurate data and prevent it from getting into the app?
• Is it possible to change the business logics of the app to prevent the improper use of the data to get the in-app reward, for example? If there are such loopholes in the application, it may bring losses to the customer.

Multiple user scenarios

Some of the user scenarios can be predicted (i.e. the data coming gradually during a day or two devices running simultaneously), while others can hardly be foreseen. At times, it is difficult to guess where your user will log into the app and how it will impact the data collection.

Solution: Get into your user’s shoes or become a user yourself: use the devices in your daily life monitoring how it reacts to the changing usage environment.

To overcome any challenge you might face, stay creative and use your imagination to come up with nontrivial testing decisions as our team did in the following real cases:

Testing on towels

On one of the projects we discovered that if you put on your Apple Watch and open the heart rate screen and then take it off and place it on a towel or a pair of jeans (the fabric really matters), the pulse rate will reach about 200 beats per minute. Too much for a towel, agree?

When the customer learnt about our finding, he asked us to check how this behavior would affect the application functionality. We tried to reproduce that scenario, but had neither tissue in relief nor towels in the office. Do you know what helped us? A stuffed toy! When you placed the Apple Watch on it, the pulse reached 200 beats per minute.

The question arises: Is this situation life-real?

In fact, quite real: the user finishes his training in the gym, take the wrist-worn device off and put it on the pair of jeans or on a towel. However, it’s very difficult to foresee it and include it in your testing scenario.

 #10000000 steps

On another project the user complained that the app wouldn’t synchronize the completed steps. After studying the problem, we found out that it happens when the number of steps contains more than seven digital symbols, which should at least be 10,000,000. Pay attention that when the app is launched for the first time, it uploads all the steps taken over the past 90 days.

After simple calculations, we got about 111,000 steps per day, which is approximately 80-90 kilometers (50-55 miles). Is it real? To answer this question, we decided to find out how much real users walk a day. And here popular Instagram service turned out to be helpful as many people eagerly share the number of completed steps in their accounts.

Looking through multiple posts marked with the hashtag #steps, #activity, #healthapp, we came to the conclusion that the average number of daily steps lied in between 7,000 and 10,000. The most active users can take up to 20,000 but it’s next to impossible to complete more than 35,000 steps.

Another possible solution is to search for a more precise hashtag, #10000000 in our case. When we worked on the project there was only one sole photo with such a hashtag. Judging by the caption, someone has walked across the Great Wall of China. Unfortunately, there was no time period specified. Anyway, it is unlikely that someone could cover 80-90 kilometers (50-55 miles) for 90 consecutive days. The bug was fixed but the question who and how managed to get 10,000,000 steps in a day is still open.

Summing up, I can say that testing on wearable devices may be quite challenging, but never boring. After all, how can one get bored when running tests on a towel?