Localization testing strategy for fostering project success

Localization testing is an indispensable part of every software localization project. As most of the software developing companies aim to reach audience from the world over, they can’t neglect testing the product’s localization. Let’s take a closer look at the current status and best practices of localization testing.

This type of testing is performed by software QA engineers (you’ll see later on why we stress this seemingly obvious fact) after the product localization and before its release. The main goals are to ensure the functional and linguistic accuracy of the software under test and make sure that no issues have been introduced during the localization process.

What problems may occur during the l10n testing?

Most of the problems occur just because some localizers do software translation using the tools “outside” of the actual software application. As a rule, localizers receive a table of strings. Of course, they are provided with all reasonable information about the strings’ context, but in practice, this information is far from complete: the strings may appear dynamically in many user situations or in different parts of the interface. These different parts may look differently.

Who should perform localization testing: a QA engineer or a linguist?

There has always been the dilemma over which is more important: the engineering capabilities of a tester and his or her product knowledge, or his or her linguistic and language background.

A decade ago, localization testing was equaled to linguistic testing. It was widely believed that l10n testing should be performed by the target language native speakers or at least by the speakers with an advanced level of language competency. However, today the situation is completely different. It’s not efficient for QA companies to hire native speaking testers to ensure localization quality of a project. Even if found, such specialists may be very expensive to maintain.

And there is no 100% guarantee that a linguist will cope with projects that complicate business logic. Sometimes, it can be helpful to differentiate between functional defects and non-translated strings. The defects are reported to the dev team, and non-translated strings are sent to the localization team.

Ideally, there should be a team of linguists or localization engineers to ensure the 100% project delivery.

But regardless of who performs testing, localization depends on dev and QA teams. It is vital that localization testing shouldn’t start until the core product has reached a certain level of stability.

Otherwise, localizers may come across multiple defects at the same time as the functional testing team finds them. As a result, several bugs will be reported for the same issue and bug fixing efforts will increase.

What are the duties of a localization engineer?

A localization engineer checks all translatable files for in-context accuracy and performs UX testing to make sure all global users can run the app and complete tasks like making a payment or filling out a form. The main personal qualities of a good localization tester are still the same: curiosity, analytical skills, and attentiveness.

Localization testing best practices

1. Pseudo-localization testing

This technique is designed to verify the localizability of strings prior to the localization itself. It is normally executed as part of the internationalization testing phase. During the pseudo-localization the standard display characters are replaced by potentially problematic ones. Such pseudo-localized software may then be tested to verify its general functionality. In average, 70-80% of bugs that would be normally reported later on during localization testing are found at this earlier stage as a result of pseudo-localization.

2. Automation of localization testing

The main goal of localization testing is to ensure the product adaptation to all target regions. To reduce the time required for adding new localization and perform regression testing on long-term projects, localization testing can be automated.

The benefits of test automation are the following: significant reduction of efforts, costs and time compared with resource-demanding manual testing, especially with a larger number of languages.

How a1qa automated localization testing for Kaspersky Lab

Kaspersky Lab addressed a1qa to perform comprehensive testing of the My Kaspersky portal and to automate the localization and internationalization testing.

Kaspersky team worked in collaboration with a1qa testing team to deliver 31 localizations.

a1qa test automation engineers developed 64 automated test scripts for localization screenshotting, which covered more than 90% of the pages a user could see. The scripts where parameterized to be executed on the specified scope of locales (different combinations of country/language), and they produced screenshots and “metadata” for each screenshot.

The so-called metadata was the xml-file with the LocalizedString items, including text, unique key and position of this text on the screenshot. Using the special parsing tool, localization engineers where able to achieve two main goals:

1. Compare the same pages of different product versions for regression.
2. Compare the same pages in different localizations.

After the analysis, the collected data was sent to third-party localization agencies. When it was returned and committed to the portal, the corrections were made.

The automated tests created by a1qa were reusable and easy to maintain. They served well not only localization purposes but also enabled proper work of the covered functionality.

• With the participation of a1qa testing team, the My Kaspersky portal has been localized and adapted to 31 languages.
• a1qa test automation has created a set of reusable tests that can be further maintained and used by the customer’s team to localize newly added strings and functionality.

Generally, the success of localization testing greatly depends on how companies approach the whole issue of internationalization and localization and how they integrate automation into the entire process.

But there is no doubt that software localization may pose challenges, and testing should be conducted prior to the software release to the global market.