OSS/BSS testing
Blog

Back-to-back testing in OSS/BSS

Which product to choose for testing? Which tariff plan to choose for testing? What kind of charges to check, and what type of subscribers to use?
5 November 2013
Functional testing
Quality assurance
Web app testing
The article by Julia Ilyushenkova
Head of Telco and Web applications testing department

Julia Liber is head of Telecom and Web application testing department at a1qa . In this role, she manages the Internet applications and telecom systems testing team and provides consulting for wireless operators. She also assists with organizing the testing process and the acceptance phase for modification or new billing solution implementations.

Would you say testing in OSS/BSS is a trivial task? Definitely not

It usually takes at least a couple of days just to test functionality. While implementing a new system/subsystem or changing functionality, there are so many questions to be answered. Which product to choose for testing? Which tariff plan to choose for testing? What kind of charges to check, and what type of subscribers to use?

It is obviously impossible to cover all possible options and combinations during functional testing, so it is necessary to select the most important products and services.

At this point, the thought comes to mind to turn to the good old engineering approach of back-to-back testing, which is based on the law of large numbers. The point is simple: It is necessary to compare system behavior using the same data. Imagine that we have two environments:

  1. Production — your live system serving subscribers
  2. Testing — your environment intended for testing.

First, at a specific date/time, usually after a billing period has been completed, we transfer copies of user data (migration data) and the product catalog (configuration data) to the testing stand. At the end of the reporting period (month or week, depending on the timing of the invoice), a copy of the input data for each transaction (payments, charges, maintenance fees) is loaded into the testing environment.

First, the output data has being processed from both environments. Second, it is placed on a comparison server. Then, specially developed script checks if the number of transactions and the charges from both environments match. The matching and unmatched numbers, the results of this comparison, are the input data for testers.

What’s next? The testing team goes to work. Testers analyze the records based on any discrepancies between the two sets of results. The causes for the differences are identified, the records are grouped into causes, and we get the results in the following format:

Numerical discrepancies:

  • 5 percent of records could not boot due to a functional defect 1
  • 3.5 percent of records could not be loaded due to a functional defect 2
  • 1 percent of records could not boot due to a functional defect 3
  • 0.5 percent of records could not be loaded due to a functional defect 4

Discrepancies in the amount of write-offs:

  • 7 percent of records are processed improperly because of a defect in the configuration No. 5
  • 1 percent of records are processed improperly because of a defect in the configuration No. 5
  • 0.5 percent of records processed improperly because of a defect in the configuration  No. 5
  • 0.1 percent of records processed improperly because of a defect in the configuration No. 5

What is the outcome of this approach?

  1. It provides complete coverage of the product catalog, through the activities of real users. The system tests exactly what is used by subscribers;
  2. It checks the quality of the configuration and system migration, as well as the most critical functionality for OSS/BSS parts: rating, billing and payments;
  3. It helps clearly prioritize defects that are present in the system based on the needs of the business. Let’s say, it is more important to correct defect No. 1, compared to defects No. 3 or No. 4, since defect No. 1 does more damage to the business;
  4. Because testing takes place on a large volume of data, this is a good way to test how well the new version of the system can withstand real-world loading.

Of course, there are limitations to this approach.

First, the data comparison is always dependent on what type of OSS/BSS you use. It will be necessary to develop a unique script for your system to compare and select data to analyze. Second, in the ideal case, the test environment must comply with the product environment. Otherwise, there is a risk that you won’t meet the deadline because the test environment processes transactions too slowly.

More Posts

17 June 2024,
by a1qa
5 min read
Shifting to test automation to maximize software quality
Explore in the article why businesses should move from manual testing to test automation.
Quality assurance
Test automation
RPA in QA
28 May 2024,
by a1qa
4 min read
Embracing robotic process automation to drive efficiency in QA
Discover how the convergence of robotic process automation helps reshape software testing practices.
General
Quality assurance
Test automation
QA for fintech
7 May 2024,
by a1qa
5 min read
Navigating the fintech frontier in 2024: QA’s role in delivering high-quality financial software 
Unveil the future of fintech innovations and learn to refine their quality with the help of software testing.
Blockchain app testing
Cybersecurity testing
Quality assurance
Shift-left testing for better software performance
25 April 2024,
by a1qa
4 min read
Optimizing software performance with shift-left testing
Still in doubt whether to include performance testing from the initial development stages? Learn the benefits companies obtain with shift-left performance testing.
Performance testing
QA consulting
Quality assurance
Test automation
Telecom trends 2024
15 April 2024,
by a1qa
5 min read
QA’s role in adopting telecom trends for 2024 
Let’s dive into the transformative trends set to redefine the telco industry in 2024 and discover QA strategies to adopt them with precision.
Cloud-based testing
Cybersecurity testing
Functional testing
General
Migration testing
Performance testing
QA trends
Quality assurance
Test automation
Zero trust in retail
27 March 2024,
by a1qa
4 min read
Fortifying retail security posture: embracing zero trust to protect customer data
Why adopt zero trust in the retail sector? How can companies ensure increased resilience to cyber incidents? Find out the answers in this article.
General
Quality assurance
Advancing QA and software testing processes with AI
14 March 2024,
by a1qa
4 min read
Advancing QA and software testing processes with AI
Uncovering the benefits companies gain when revolutionizing QA practices with the help of AI and tips to implement it.
General
Quality assurance
On the pulse of 2024: optimizing the adoption of eHealth trends with QA
15 February 2024,
by a1qa
4 min read
On the pulse of 2024: optimizing the adoption of eHealth trends with QA
Generative AI, cybersecurity, AR/VR — come and explore how these trends are reshaping the future of healthcare and how QA helps implement them with confidence.
Cybersecurity testing
Functional testing
Performance testing
QA trends
Navigating the future: QA trends that will define 2024. Part 2
30 January 2024,
by a1qa
4 min read
Navigating the future: QA trends that will define 2024. Part 2
We continue exploring QA trends, helping businesses remain competitive in 2024.
Cloud-based testing
Cybersecurity testing
QA trends
Quality assurance

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.