OSS/BSS testing
Blog

Back-to-back testing in OSS/BSS

Which product to choose for testing? Which tariff plan to choose for testing? What kind of charges to check, and what type of subscribers to use?
5 November 2013
Functional testing
Quality assurance
Web app testing
The article by Julia Ilyushenkova
Head of Telco and Web applications testing department

Julia Liber is head of Telecom and Web application testing department at a1qa . In this role, she manages the Internet applications and telecom systems testing team and provides consulting for wireless operators. She also assists with organizing the testing process and the acceptance phase for modification or new billing solution implementations.

Would you say testing in OSS/BSS is a trivial task? Definitely not

It usually takes at least a couple of days just to test functionality. While implementing a new system/subsystem or changing functionality, there are so many questions to be answered. Which product to choose for testing? Which tariff plan to choose for testing? What kind of charges to check, and what type of subscribers to use?

It is obviously impossible to cover all possible options and combinations during functional testing, so it is necessary to select the most important products and services.

At this point, the thought comes to mind to turn to the good old engineering approach of back-to-back testing, which is based on the law of large numbers. The point is simple: It is necessary to compare system behavior using the same data. Imagine that we have two environments:

  1. Production — your live system serving subscribers
  2. Testing — your environment intended for testing.

First, at a specific date/time, usually after a billing period has been completed, we transfer copies of user data (migration data) and the product catalog (configuration data) to the testing stand. At the end of the reporting period (month or week, depending on the timing of the invoice), a copy of the input data for each transaction (payments, charges, maintenance fees) is loaded into the testing environment.

First, the output data has being processed from both environments. Second, it is placed on a comparison server. Then, specially developed script checks if the number of transactions and the charges from both environments match. The matching and unmatched numbers, the results of this comparison, are the input data for testers.

What’s next? The testing team goes to work. Testers analyze the records based on any discrepancies between the two sets of results. The causes for the differences are identified, the records are grouped into causes, and we get the results in the following format:

Numerical discrepancies:

  • 5 percent of records could not boot due to a functional defect 1
  • 3.5 percent of records could not be loaded due to a functional defect 2
  • 1 percent of records could not boot due to a functional defect 3
  • 0.5 percent of records could not be loaded due to a functional defect 4

Discrepancies in the amount of write-offs:

  • 7 percent of records are processed improperly because of a defect in the configuration No. 5
  • 1 percent of records are processed improperly because of a defect in the configuration No. 5
  • 0.5 percent of records processed improperly because of a defect in the configuration  No. 5
  • 0.1 percent of records processed improperly because of a defect in the configuration No. 5

What is the outcome of this approach?

  1. It provides complete coverage of the product catalog, through the activities of real users. The system tests exactly what is used by subscribers;
  2. It checks the quality of the configuration and system migration, as well as the most critical functionality for OSS/BSS parts: rating, billing and payments;
  3. It helps clearly prioritize defects that are present in the system based on the needs of the business. Let’s say, it is more important to correct defect No. 1, compared to defects No. 3 or No. 4, since defect No. 1 does more damage to the business;
  4. Because testing takes place on a large volume of data, this is a good way to test how well the new version of the system can withstand real-world loading.

Of course, there are limitations to this approach.

First, the data comparison is always dependent on what type of OSS/BSS you use. It will be necessary to develop a unique script for your system to compare and select data to analyze. Second, in the ideal case, the test environment must comply with the product environment. Otherwise, there is a risk that you won’t meet the deadline because the test environment processes transactions too slowly.

More Posts

Navigating the future: QA trends that will define 2024. Part 2
30 January 2024,
by a1qa
4 min read
Navigating the future: QA trends that will define 2024. Part 2
We continue exploring QA trends, helping businesses remain competitive in 2024.
Cloud-based testing
Cybersecurity testing
QA trends
Quality assurance
Navigating the future: QA trends that will define 2024. Part 1
15 January 2024,
by a1qa
4 min read
Navigating the future: QA trends that will define 2024. Part 1
Discover topical software testing trends that will shape 2024 and empower companies to smoothly implement advanced technologies.
Agile
QA trends
Quality assurance
Test automation
2023-year-end-recap:-a-journey-through-the-a1qa-milestones
20 December 2023,
by a1qa
4 min read
2023 year-end recap: a journey through the a1qa milestones 
As we bid farewell to 2023, join us in recalling noteworthy achievements and unforgettable moments that have defined this year!
General
Quality assurance
The year in valuable conversations: recapping 2023 a1qa’s roundtables for IT executives 
8 December 2023,
by a1qa
3 min read
The year in valuable conversations: recapping 2023 a1qa’s roundtables for IT executives 
From dissecting novel industry trends to navigating effective ways of enhancing software quality — let’s recall all a1qa’s roundtables. Join us!
Big data testing
Cybersecurity testing
Functional testing
General
Interviews
Performance testing
QA trends
Quality assurance
Test automation
Usability testing
Web app testing
a1qa has been included in the Next-Generation Quality Engineering Services PEAK
29 November 2023,
by a1qa
2 min read
a1qa has been included in the Next-Generation Quality Engineering Services PEAK Matrix® Assessment 2023 by Everest Group
Explore how a1qa secured a proud spot in the prestigious PEAK Matrix® by Everest Group.
General
Quality assurance
black-friday
22 November 2023,
by a1qa
4 min read
Get ready for Black-Friday-to-Cyber-Monday shopping: 5 testing types to include in your QA strategy
What’s your nightmare during Black Friday and Cyber Monday shopping? If it’s a loss of sales, read about the ways to prevent this in the article.
Cybersecurity testing
Functional testing
Localization testing
Performance testing
Usability testing
na-st-awards-23
16 November 2023,
by a1qa
3 min read
a1qa shines as the finalist in three categories of the North American Software Testing Awards
a1qa is a triple finalist at the North American Software Testing Awards.
General
Quality assurance
Test automation
6 top reasons why business should invest in software quality
9 November 2023,
by a1qa
4 min read
6 top reasons why business should invest in software quality
We congratulate you on the World Quality Day with the article by Alina Karachun, Account director at a1qa, having 10+ years of QA expertise. Delve into it to explore the reasons why businesses should prioritize software quality.
Cybersecurity testing
Functional testing
General
Interviews
Performance testing
Quality assurance
3 November 2023,
by a1qa
4 min read
From idea to buying: 7 testing types to make your mobile eCommerce solutions flawless
Read the article to discover 7 QA activities helping boost mobile eCommerce solutions quality and provide end users with exceptional buying experiences.
Functional testing
General
Quality assurance
Test automation
Usability testing

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.