OSS/BSS testing
Blog

Back-to-back testing in OSS/BSS

Which product to choose for testing? Which tariff plan to choose for testing? What kind of charges to check, and what type of subscribers to use?
5 November 2013
Functional testing
Quality assurance
Web app testing
The article by Julia Ilyushenkova
Head of Telco and Web applications testing department

Julia Liber is head of Telecom and Web application testing department at a1qa . In this role, she manages the Internet applications and telecom systems testing team and provides consulting for wireless operators. She also assists with organizing the testing process and the acceptance phase for modification or new billing solution implementations.

Would you say testing in OSS/BSS is a trivial task? Definitely not

It usually takes at least a couple of days just to test functionality. While implementing a new system/subsystem or changing functionality, there are so many questions to be answered. Which product to choose for testing? Which tariff plan to choose for testing? What kind of charges to check, and what type of subscribers to use?

It is obviously impossible to cover all possible options and combinations during functional testing, so it is necessary to select the most important products and services.

At this point, the thought comes to mind to turn to the good old engineering approach of back-to-back testing, which is based on the law of large numbers. The point is simple: It is necessary to compare system behavior using the same data. Imagine that we have two environments:

  1. Production — your live system serving subscribers
  2. Testing — your environment intended for testing.

First, at a specific date/time, usually after a billing period has been completed, we transfer copies of user data (migration data) and the product catalog (configuration data) to the testing stand. At the end of the reporting period (month or week, depending on the timing of the invoice), a copy of the input data for each transaction (payments, charges, maintenance fees) is loaded into the testing environment.

First, the output data has being processed from both environments. Second, it is placed on a comparison server. Then, specially developed script checks if the number of transactions and the charges from both environments match. The matching and unmatched numbers, the results of this comparison, are the input data for testers.

What’s next? The testing team goes to work. Testers analyze the records based on any discrepancies between the two sets of results. The causes for the differences are identified, the records are grouped into causes, and we get the results in the following format:

Numerical discrepancies:

  • 5 percent of records could not boot due to a functional defect 1
  • 3.5 percent of records could not be loaded due to a functional defect 2
  • 1 percent of records could not boot due to a functional defect 3
  • 0.5 percent of records could not be loaded due to a functional defect 4

Discrepancies in the amount of write-offs:

  • 7 percent of records are processed improperly because of a defect in the configuration No. 5
  • 1 percent of records are processed improperly because of a defect in the configuration No. 5
  • 0.5 percent of records processed improperly because of a defect in the configuration  No. 5
  • 0.1 percent of records processed improperly because of a defect in the configuration No. 5

What is the outcome of this approach?

  1. It provides complete coverage of the product catalog, through the activities of real users. The system tests exactly what is used by subscribers;
  2. It checks the quality of the configuration and system migration, as well as the most critical functionality for OSS/BSS parts: rating, billing and payments;
  3. It helps clearly prioritize defects that are present in the system based on the needs of the business. Let’s say, it is more important to correct defect No. 1, compared to defects No. 3 or No. 4, since defect No. 1 does more damage to the business;
  4. Because testing takes place on a large volume of data, this is a good way to test how well the new version of the system can withstand real-world loading.

Of course, there are limitations to this approach.

First, the data comparison is always dependent on what type of OSS/BSS you use. It will be necessary to develop a unique script for your system to compare and select data to analyze. Second, in the ideal case, the test environment must comply with the product environment. Otherwise, there is a risk that you won’t meet the deadline because the test environment processes transactions too slowly.

More Posts

qa-trends-in-telecom
30 September 2022,
by a1qa
5 min read
4 telecom trends for 2023 and how to painlessly implement them with QA
It’s time to explore the telecom trends for the upcoming year. Let’s look at them together and also see the value that QA brings for their smooth deployment.
Cybersecurity testing
Migration testing
QA trends
Quality assurance
Test automation
why-do-bugs-get-missed
31 August 2022,
by a1qa
4 min read
Why do bugs get missed? Learn the problems and tips to avoid them
Still, finding overlooked bugs after the app goes live? Let’s find out why this happens and how to fix it.
Quality assurance
Test automation
black-friday
29 July 2022,
by a1qa
4 min read
Get ready for Black-Friday-to-Cyber-Monday shopping: 5 testing types to include in your QA strategy
What’s your nightmare during Black Friday and Cyber Monday shopping? If it’s a loss of sales, read about the ways to prevent this in the article.
Cybersecurity testing
Functional testing
Localization testing
Performance testing
Usability testing
Mobile app testing
31 May 2022,
by a1qa
4 min read
Mobile app testing guide: win the race with five-star software
Which aspects of mobile apps to test first to produce a really high-quality product? Find the answer to this and other questions related to mobile app testing in the article.
Cybersecurity testing
Functional testing
Mobile app testing
Performance testing
Test automation
Usability testing
31 March 2022,
by a1qa
5 min read
The A to Z guide to functional testing
Get a broad overview of functional testing: reasons to introduce it and 5 main phases for a well-tuned process.
Functional testing
21 February 2022,
by a1qa
5 min read
Continuous testing 101: a comprehensive guide
So many companies take a chance to introduce continuous testing. What is it like, and how to smartly implement it? Let’s get to know in the article.
Quality assurance
streaming services
30 November 2021,
by a1qa
4 min read
Ho ho ho! QA to enable crash-proof streaming platforms for Holidays season
Isn’t it high time to forget about the Christmas rush and truly enjoy the pre-holidays season? Read the article about how to put peak-load anxiety behind by applying rigorous testing of your streaming solution.
Cybersecurity testing
Functional testing
Performance testing
Usability testing
Agile and DevOps in eCommerce QA_mini
30 September 2021,
by a1qa
5 min read
Agile and DevOps: Boosting the quality of eCommerce apps
What benefits do Agile and DevOps bring to eCommerce business, and how QA helps with that? Find it out in the article.
Agile
Quality assurance
AR/VR testing infographics mini
30 August 2021,
by a1qa
< 1 min read
AR/VR testing in retail: turning challenges into opportunities
Welcome to read the infographic on AR/VR in retail: new shopping experiences, issues, and how to address them with QA.
Quality assurance

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.