The client owns a multinational media company that specializes in providing data outlining international business and world affairs. Its principal activities are focused on magazines, newspapers, conferences, and market intelligence.
Following its endeavors to deliver improved customer experience to multinational users and migrate from an outdated QSS third-party system to an in-house framework customized in line with proper requirements, the client was in need of a time-tested QA partner to rely on. Broad testing expertise, process maturity, and desire to drive the client’s solution forward outweighed towards choosing a1qa.
The client’s website represents a news portal that offers weekly content for multinational readers with a focus on politics, business, science, technology, and art.
Apart from news sections describing the development of various regions (Europe, Middle East, North America, etc.), the readers have access to blogs covering topical themes and the range of other perks like films, podcasts, internationally recognized events, and more.
Subscription to the client’s solution provides an all-access pass to extra benefits like reading offline on mobile devices or receiving daily briefing newsletters. Each time the reader creates a subscription order, it’s automatically sent to QSS for processing and further storage.
To check the a1qa specialists’ domain expertise, the client requested two engineers for a one-month deadline to fulfill a pilot project. It involved testing the back-office part of the subscription portal (filled in with the content previously well-designed by the marketers). Positive feedback was followed by a full-time business proposal continuing for the moment.
The workload increased over time and apart from assuring the quality of the subscription portal and its further migration from QSS, the engineers were testing a newly designed user account. It allows the readers to manage personal data, although part of the functionality (e.g. regarding the chosen subscription plan) is still supported by the old account in QSS.
Even in an Agile world, the teams are required to move faster. Therefore, one of the fundamental client’s demands concerned profiting from shifting testing left on the SDLC and assigning cross-functional team members able to take a sharp curve anytime and replace colleagues from other departments.
For a more effective team scale-up, all newcomers were undergoing a knowledge transfer process. To cherry-pick the candidates, all of them were to pass an interview with the QA manager on the client side.
While providing testing services, the QA engineers were closely cooperating with the client’s DevOps teams to clarify any arising issues and ensure just-in-time delivery of the planned functionality.
To support the client in rolling out defect-free software while migrating to a new system version, the a1qa engineers were performing multiple QA activities listed below.
Prior to carrying out any testing activities, the engineers created test scenarios with the help of an Xray plug-in.
Functional testing was implemented in terms of both front-end and back-office (including UI) of the portal. While searching for bottlenecks in the front-end part, the QA team was checking each new build applying smoke tests to make sure the basic functionality complied with all the requirements.
Back-office QA activities included testing the possibility to create new features/products in the appropriate catalogs ascertaining that all the fields possess the relevant content, specifically designed items contain the necessary fields, and many more.
GUI testing was performed to ensure the solution has the best appearance in strict accordance with the design mock-ups.
The presence of a mobile version encouraged the necessity to introduce mobile QA service performed according to the prepared test documentation to verify that all features complied with the requirements and worked as expected.
Before testing itself, the engineers together with the client specified the most sought-after devices and OSs applied by users in multiple regions based on statistical data.
Quality assurance activities were focused on front-end regression testing of the subscription portal and were performed on real items only taken from the a1qa corporate lab. Prior to each release, the engineers rotated mobile devices and carried out testing in turn.
The QA engineers applied a user behavior approach to simulate the work of end users. With the help of Apache JMeter, the team designed scripts based on the real-life user journeys applying the data gathered from Google Analytics and proceeded with fulfilling tests.
The scope of testing services provided by a1qa comprised client-side and server-side (including serverless stack).
Client-side testing encompassed ongoing monitoring set to determine the time required for opening a web page as well as all the activities that supplemented the workflow. The collected data was analyzed and transformed into statistics.
Server-side testing covered the following areas:
Moreover, the engineers were conducting system checks after changes in the back-end. Firstly, the QA team set up client-side monitoring after all releases. Each time new functionality was rolled out in the production environment, the engineers were spotting any changes and initiating a fixing process (e.g. RT of opening the web page increased). To streamline the workload even more, performance QA engineers switched the monitoring to an automated mode.
Secondly, after the most important releases defined by the managers and developers on the client side, the QA team was testing the server side with the same configurations of servers and comparing results before and after the release.
Serverless testing applied for assuring the quality of a user account was fulfilled with the help of serverless AWS stack (including SQS, SNS, DynamoDB, Lambda, API Gateway) to determine the maximum number of simultaneously working users.
The major performance testing challenge for the a1qa team concerned the implementation of MQTT over WebSockets – the protocol supported by Apache JMeter. The engineers customized it by writing a proper plug-in.
The approach to delivering performance testing services contributed to identifying capabilities of the system and detecting issues and the ways for improvement that were reflected in a detailed report.
The tested solution possesses a large scope of business functions allowing its end users to perform numerous activities – from creating catalogs with available products and subscribing to the desired ones to tracking invoices and generating events related to automatic payments.
The support of this extensive functionality scope was provided by the following third-party systems: Salesforce – customer relationship management and Zuora – subscription management software for SaaS.
While ensuring their defect-free integration with the client’s solution, the a1qa team ascertained that:
Later on in the process, automated E2E scenarios were added to reduce the time required for regression testing.
On the way to addressing the primary objective concerning the upgrade from QSS to a proper, advanced system, a1qa proceeded with migration testing to guarantee 100% data transfer.
A database to migrate was voluminous and comprised millions of users. The most challenging part about the process was the number of other integrated systems each possessing particular records about end users. Therefore, the engineers were sorting out this data so that each record belonged to the appropriate system.
Having fulfilled that, the QA team was returning to the main system to submit the order from a newly migrated user to ascertain data consistency between multiple systems didn’t breach.
To help the client streamline testing activities, a1qa kick-started test automation applied as regards the subscription portal and the private user account.
In terms of the subscription portal, the back-end part included tests that were creating items that later were gathered into user journeys. Frontend part presupposed passing all those journeys, verifying the correctness of submitting the subscription.
Within the user account, the tests were checking the accuracy of displaying user data, the possibilities of changing the information, the correctness of following the right link, and more.
The QA automation team developed test scripts automating UI and API checks applying C# and JavaScript programming languages.
In addition, the a1qa team transferred C# pack to a .NET Core, so that the tests could also be executed on Linux OS, not only on Windows. This helped the client reduce the operational cost due to free of charge Linux software.
The client was seeking more than just a provider of quality assurance services but a reliable and proactive business partner always caring for how things work on its side.
Therefore, at some stage of the project, a1qa experts performed an internal QA audit to help the client eliminate bottlenecks in the process by
The improvements that were put forward received positive feedback which helped introduce performance testing service in due time.
The client marked the responsible approach from the a1qa side. The partnership continues and develops until now.
