Blog

Using back-to-back testing in the telecom industry

First, let’s have a “helicopter view” on what comparative test provides and which cases are appropriate for back-to-back in Telecom industry. Most commonly, comparative testing is used in the case of the full replacement of OSS/BSS solution, which is absolutely crucial for any Telecom business. Rarely, new version of existing OSS/BSS is installing and requires verification.
4 November 2014
Quality assurance
The article by Julia Ilyushenkova
Head of Telco and Web applications testing department

First, let’s have a “helicopter view” on what comparative test provides and which cases are appropriate for back-to-back in Telecom industry.

Most commonly, comparative testing is used in the case of the full replacement of OSS/BSS solution, which is absolutely crucial for any Telecom business. Rarely, new version of existing OSS/BSS is installing and requires verification.

Comparative testing checks two identical OSS/BSS with the same input data in order to reveal incorrect data processing.

Three main goals to use comparative tests

  1. Detection of functional defects in Rating and Billing systems;
  2. Detection of migration errors when user data transferred from one system to another;
  3. Configuration defects identification (setting tariffs in both systems).

The general workflow for comparative test (proposed by the author while implementing projects in Telecom) is shown at the picture below.

As you can see, two identical systems are required for the testing process: the master (“Golden”) and the test system itself.

Sometimes the input data for “Golden” system cannot be accepted by the test system without additional pre-processing. Then, test data must undergo further conversion phase for compatibility with the test system. After getting results from both systems, they have to be compared – the essence of comparative test – that is, as a rule, an automated process implemented independently from environments. The final step is an empirical data analysis activity that is performed by the tester.

When we analyze the efficiency of comparative test regarding OSS/BSS solutions on real Telecom projects, we must have a critical view and list some of the shortcomings:

  • The test involves a lot of empirical work, which is difficult to automate (final stages of divergence analysis are performed manually).
  • During late iterations of the test, a lot of records with numerical discrepancy that cannot be considered as defects can be revealed. These records must be filtered from comparison
  • Sometimes the test indicates good results while in reality the situation is bad. For instance, during the collection of final statistics, two critical defects might neutralize each other when one of them increases and the other decreases the final amount.
  • Completely relevant comparative test should be performed for at least one full billing period (about one month), which is not usually the case as the test lasts less than 30 days.

As for the billing systems comparative tests requires achieving a sufficiently high level of coincidence of the output results for both systems (for instance, 99.99%). Achieving this result is a bit tricky, due to the following factors:

  • Different rounding of floating-point data in two systems;
  • Incorrect development of mapping tables (service A in the system 1 corresponds to service B in the system 2);
  • Different order of records processing in two systems may lead to divergence in tariff discounts application;
  • Peculiarity of some data types in database (example – rating results for call forwarding);
  • Dynamism of the master (“Golden”) environment. In the real life, clients are frequently changing tariff plans, phone numbers, SIM-cards, contract statuses etc. It’s almost impossible to synchronize all these changes in the test system;
  • Confirmed changes in business-logic, that is the differences in behavior between the two systems envisaged by Telecom operator requirements to the new system;
  • Uncertainty and indistinctness of the system requirements;
  • Inability to implement the requirements of the system under test.

All these factors lead to numerical discrepancies in the results, which may be up to 25% from reference values.

Nevertheless, based on my experience I would advise implementing back-to-back testing due to the many advantages that this method provides:

  • High level of test coverage for the system;
  • The opportunity to get additional metrics of the system quality;
  • Availability of implementing migration and configuration tests, not only functional tests;
  • High automation level of the testing process.

This is confirmed by the picture below illustrating results of several testing iterations on a real Telecom project.

These statistics contains three metrics:

  1. Defects – the number of functional, integration and configuration defects in the test system
  2. Discrepancy in records, % – the percentage of data units which gave different output results, relative to the originally loaded amount
  3. Discrepancy in amounts, % – the ratio of the total discrepancies amount between the master and the test system relative to the total amount of charges in the master system

Conclusion

We can note high efficiency of this type of test in relation to OSS/BSS. A good practice is to use a back-to-back test together with other traditional strategies for testing, since they are not mutually exclusive and able to detect defects of different classes of the same functionality. You should also keep in mind that an adequate strategy must be developed for back-to-back tests, taking into account all advantages and shortcomings presented in the article.

The article was published at SoftwareTestingMagazine.

More Posts

Agile and DevOps in eCommerce QA_mini
30 September 2021,
by a1qa
5 min read
Agile and DevOps: Boosting the quality of eCommerce apps
What benefits do Agile and DevOps bring to eCommerce business, and how QA helps with that? Find it out in the article.
Agile
Quality assurance
AR/VR testing infographics mini
30 August 2021,
by a1qa
< 1 min read
AR/VR testing in retail: turning challenges into opportunities
Welcome to read the infographic on AR/VR in retail: new shopping experiences, issues, and how to address them with QA.
Quality assurance
6 October 2020,
by Dmitry Tishchenko
4 min read
A clear view of smart team scalability
Get to know how to scale your team sagely and gratify end-user needs and fast-paced tech-market requirements.
Agile
Quality assurance
6 August 2020,
by Elena Yakimova
5 min read
How to arrange fruitful joint work with an outsourcing QA team
The head of the web apps testing department sheds light on how to establish more transparent and effective work on the project cooperating with a remote QA team.
Quality assurance
4 June 2020,
by Vitaly Prus
4 min read
SAFe vs. Scrum, and PI planning essentials
Let's shed some light on the SAFe differences from Scrum that are to be considered by the development and QA teams who have migrated from Scrum.
Agile
Quality assurance
29 April 2020,
by a1qa
4 min read
5 lessons we learned from COVID-19
Here are five key lessons the businesses need to learn during this pandemic to somehow achieve the planned outcomes. 
Quality assurance
17 April 2020,
by a1qa
5 min read
QA-focused retrospective: identifying and solving project problems
The a1qa experts came up to consider an effective approach to identify project bottlenecks and get rid of problems successfully.
Agile
Quality assurance
31 March 2020,
by Dmitry Tishchenko
4 min read
QA outsourcing – the respond to unprecedented global challenge
How can companies meet their business-critical needs without health risks? QA outsourcing is the answer. Get to know why it is the right decision in this time of need.
QA consulting
Quality assurance
10 March 2020,
by a1qa
6 min read
Dedicated team model in QA: all you should know about it
Check on everything you should know about when to apply, how to run and pay for a dedicated team in QA.
Interviews
QA consulting
Quality assurance

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.