OSS/BSS testing
Blog

Back-to-back testing in OSS/BSS

Which product to choose for testing? Which tariff plan to choose for testing? What kind of charges to check, and what type of subscribers to use?
5 November 2013
Functional testing
Quality assurance
Web app testing
The article by Julia Ilyushenkova
Head of Telco and Web applications testing department

Julia Liber is head of Telecom and Web application testing department at a1qa . In this role, she manages the Internet applications and telecom systems testing team and provides consulting for wireless operators. She also assists with organizing the testing process and the acceptance phase for modification or new billing solution implementations.

Would you say testing in OSS/BSS is a trivial task? Definitely not

It usually takes at least a couple of days just to test functionality. While implementing a new system/subsystem or changing functionality, there are so many questions to be answered. Which product to choose for testing? Which tariff plan to choose for testing? What kind of charges to check, and what type of subscribers to use?

It is obviously impossible to cover all possible options and combinations during functional testing, so it is necessary to select the most important products and services.

At this point, the thought comes to mind to turn to the good old engineering approach of back-to-back testing, which is based on the law of large numbers. The point is simple: It is necessary to compare system behavior using the same data. Imagine that we have two environments:

  1. Production — your live system serving subscribers
  2. Testing — your environment intended for testing.

First, at a specific date/time, usually after a billing period has been completed, we transfer copies of user data (migration data) and the product catalog (configuration data) to the testing stand. At the end of the reporting period (month or week, depending on the timing of the invoice), a copy of the input data for each transaction (payments, charges, maintenance fees) is loaded into the testing environment.

First, the output data has being processed from both environments. Second, it is placed on a comparison server. Then, specially developed script checks if the number of transactions and the charges from both environments match. The matching and unmatched numbers, the results of this comparison, are the input data for testers.

What’s next? The testing team goes to work. Testers analyze the records based on any discrepancies between the two sets of results. The causes for the differences are identified, the records are grouped into causes, and we get the results in the following format:

Numerical discrepancies:

  • 5 percent of records could not boot due to a functional defect 1
  • 3.5 percent of records could not be loaded due to a functional defect 2
  • 1 percent of records could not boot due to a functional defect 3
  • 0.5 percent of records could not be loaded due to a functional defect 4

Discrepancies in the amount of write-offs:

  • 7 percent of records are processed improperly because of a defect in the configuration No. 5
  • 1 percent of records are processed improperly because of a defect in the configuration No. 5
  • 0.5 percent of records processed improperly because of a defect in the configuration  No. 5
  • 0.1 percent of records processed improperly because of a defect in the configuration No. 5

What is the outcome of this approach?

  1. It provides complete coverage of the product catalog, through the activities of real users. The system tests exactly what is used by subscribers;
  2. It checks the quality of the configuration and system migration, as well as the most critical functionality for OSS/BSS parts: rating, billing and payments;
  3. It helps clearly prioritize defects that are present in the system based on the needs of the business. Let’s say, it is more important to correct defect No. 1, compared to defects No. 3 or No. 4, since defect No. 1 does more damage to the business;
  4. Because testing takes place on a large volume of data, this is a good way to test how well the new version of the system can withstand real-world loading.

Of course, there are limitations to this approach.

First, the data comparison is always dependent on what type of OSS/BSS you use. It will be necessary to develop a unique script for your system to compare and select data to analyze. Second, in the ideal case, the test environment must comply with the product environment. Otherwise, there is a risk that you won’t meet the deadline because the test environment processes transactions too slowly.

More Posts

why-do-bugs-get-missed
17 April 2023,
by a1qa
4 min read
Why do bugs get missed? Learn the problems and tips to avoid them
Still, finding overlooked bugs after the app goes live? Let’s find out why this happens and how to fix it.
Quality assurance
Test automation
Mobile app testing
15 February 2023,
by a1qa
4 min read
Mobile app testing guide: win the race with five-star software
Which aspects of mobile apps to test first to produce a really high-quality product? Find the answer to this and other questions related to mobile app testing in the article.
Cybersecurity testing
Functional testing
Mobile app testing
Performance testing
Test automation
Usability testing
a1qa-articles
31 January 2023,
by a1qa
5 min read
Best of 2022 by executives: 8 most visited a1qa blog posts
Let’s look back and revisit the most visited a1qa articles of 2022!
Quality assurance
Test automation
qa-trends
12 January 2023,
by a1qa
4 min read
The future of software testing: top 4 impactful trends that will dominate in 2023
Consider the major industry trends for the upcoming year to know how to improve current QA strategies and stay ahead of the curve.
QA trends
Quality assurance
Test automation
test-automation
7 December 2022,
by Dileep Marway
3 min read
Release at pace with test automation: What, why, and how to measure success?
An automation-first approach is key to enhancing testing capabilities and increasing overall operational efficiency. However, I would suggest justifying its implementation, so that it can deliver the promised value.
Quality assurance
Test automation
what-is-a-culture-of-happiness
2 December 2022,
by Dileep Marway
3 min read
What is a сulture of happiness for a QA company?
Great culture drives employee engagement and satisfaction, contributes to an impressive work output, and improves organisational performance. But how do we build a culture of happiness? In this blog, I will share more on the answer to this.
Quality assurance
agile-qa
30 November 2022,
by Dileep Marway
3 min read
Agile QA – what is needed for greater flexibility and speed?
What should your QA team focus on to become truly agile, enable quality at speed, and contribute to lasting performance improvement? In addition to introducing test automation, I suggest considering shared responsibility for software soundness.
Agile
Quality assurance
interview-with-dileep
28 November 2022,
by a1qa
9 min read
Interview with Dileep Marway on a series of articles “Agility and speed: Supercharging your business strategies with QA”
We cooperated with the VP of Engineering and Quality at SHL to present you with a series of his blog posts on: culture of happiness, test automation, and Agile-driven QA. Happy reading!
Agile
Quality assurance
Software lifecycle QA
Test automation
optimizing-budget-with-qa
31 October 2022,
by a1qa
4 min read
Optimizing telecom budgets with QA outsourcing: everything you need to know. Part 2 
Welcome to Part 2 of our blog on QA outsourcing and optimizing telecom budget with it. Let’s delve deeper into the topic!
QA trends
Quality assurance

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.