a1qa provides extensive QA support to a large US-based manufacturer of home appliances

PRODUCT OVERVIEW

The client is one of the leading US-based manufacturers of home appliances operating globally and possessing several trademarks offering an extensive blend of products.

The customer’s software under test incorporates B2B and B2C dimensions.

B2B solutions embrace:

  • A corporate portal for clients to help them trace business activities
  • A website offering replacement parts & accessories to support the functioning of goods
  • A commercial service allowing businesses to open laundries by renting equipment or leasing it.

B2C software encompasses:

  • Seven brands with miscellaneous locales. Amid them, five brands serve commercial purposes, although, the presence of a purchasing option depends on the locale.
  • A resource for the client’s team members and partners allowing them to purchase items at reduced prices.

Aiming to reduce the hosting cost and improve overall organizational performance, the client was undergoing migration to SAP Hybris Commerce Platform. To assure its quality by thorough UAT, introduce test automation best practices, build performance testing from scratch, and shape QA processes, the client pitched on a1qa.

PROJECT SCOPE

a1qa jump-started delving into the technical core of the solutions, which is based on the seamless interaction of back- and front-end.

Front-end involves:

  • AEM — a content management system responsible for delivering content across tested solutions.

Back-end incorporates:

  • SAP — contains data about all the products available in warehouses and delivered to end users, transactions performed, accounting, and more.
  • Hybris — stores catalogs and data about the products and registered users, introduces promotions and bundles, etc. It provides the possibility to administer websites allowing businesses to customize the content according to specific needs. It also redirects a huge flow of requests between servers to eliminate the possibility of SAP breaking down.
  • Accertify — serves to prevent any fraudulent orders. It gathers data entered during placing an order, analyzes it, and provides the assessment of the current order. Considering it, the system can accept the order, place it to pending, or reject it.

Having learned all products’ bells and whistles, the engineers got down to quality assurance scope.

Functional testing & UAT

Functional testing regarding the Hybris customizable features encompassed:

  • Fine-tuning of a promotion module to help businesses easily create, edit, and control campaigns aimed to increase conversion rate.
  • Assuring quality of the newly developed configurable bundle module for both small and major products. It allows defining specific pricing of an item depending on whether it’s sold separately or in a bundle.
  • Elevating software soundness of subscriptions, novel post-purchase features, unlocked prices, and more.
  • Support cases that covered prompt fixing of all detected incidents.

In terms of AEM, the engineers were ensuring the proper operation of the newly developed templates and components.

UAT incorporated verifications to make sure the whole functionality scope was operating as it was intended, and the correct information was passed between various system components. Test cases representing real user scenarios included detailed steps from opening the website to buying a certain product. The QA team also developed scenarios that comprised the actions of business users like checking multiple configurations in Hybris.

Besides, specific reference concerns the business type of verifications carried out on the SAP side, which include deeper test coverage (accuracy while creating invoices, disposal of goods, transfer of products between warehouses, etc.).

Test automation

Test automation with its primary objectives of speeding up time to market and reducing operational costs was the pivotal tool to streamline the testing scope.

Automated smoke tests were performed against the operation of the above-mentioned architecture. The essential part of tested functionality concerned placing orders. To do this, certain preconditions should be met (e.g., the product of a definite type, possessing a certain function).

The verification of selecting this product looks the following way. The automated test requests Hybris through the API and searches for the necessary item. Afterward, it’s chosen on the website and placed as an order. As soon as the transaction is performed, SAP creates the message about the successful operation, the presence and details of which are checked by this test.

Test automation introduction was effective: only in the past two months, it contributed to saving 50 business days of the manual testing team consisting of 4 people. All in all, test automation by a1qa curtailed the time required for smoke testing by 90%.

Performance testing

The client already had the team responsible for this direction. To increase its efficiency, a1qa built the testing process from scratch and leveraged a user behavior approach.

The engineers prepared user journeys emulating realistic load, which later became the baseline for scripts. Then, the team fine-tuned the basic approach to developing scripts to enhance the level of detail. If even less than 1% of users fulfilled a certain activity, it was added to the list. Scripts were updated before each release due to the ongoing functionality augmentation.

Afterward, all solutions were verified before each release by harnessing the activities listed below:

  • Client-side testing — to measure the load and render time for web pages. Applying an open-source tool WebPageTest, the engineers assessed the current performance of front-end elements and collected special metrics describing the most basic aspects like the architecture of the web page, the process of downloading its elements, the ways of optimization.
  • Load testing — to ensure the solutions would cope with the target load for a 5-hour period. The QA team chose the necessary number of concurrently working users based on the statistics from Google Analytics. Periodically, the load/client-side testing workload was supplemented by specific checks like proper compression of CSS elements.
  • Stress testing. Performance testing was carried out on the QA environment shared with the QA team due to the inability to provide a separate environment. Its hardware capabilities were lower compared to those of the production. Stress testing helped overcome this gap and define the maximum capacity of the environment. After scripts validation and deploying the build to the QA environment, test execution with further reporting was fulfilled.
  • Black Friday support — to guarantee the solutions would cope with the increased number of concurrent users. The team carried out 8-hour load tests, and the number of potential users was increased by 30%.

Suggested performance testing improvements:

  • a1qa proposed to introduce open-source tools covering the same functionality scope to help the client reduce operational costs.
  • The team offered to implement a certain RT parameter applicable to each solution type considering its specifics.
  • a1qa implemented the rule to synchronize the workload and start testing only after the whole data is delivered to avoid failing tests because of content issues.

Integration testing

It was intended to guarantee seamless integration of:

  • Ready-made payment platforms
  • Correct distribution of taxes.

Each payment platform possesses issues that may occur during money transactions. After entering, credit data is sent to the system for validation. The type of response depends on the variants of the issues spotted or the situations of accepting the data. The tests check that all those types are considered by the system and are validated correctly. For testing on production environment, the team harnessed only real credit cards provided by the client.

Considering complex taxation in the USA, it was vital to make sure the correct tax is assigned to a certain state. Taxes are calculated in the following way. The request for a certain tax is sent to Vertex, then the number received is transferred to an order. After this order is created in SAP, the process repeats once again to ensure the maximum accuracy. Testing helped guarantee the absence of software glitches.

Improving processes

Apart from testing, a1qa contributed to reinforcing the range of QA activities:

Release support scheme

The client delivers new functionality once every 2-3 weeks. To ensure the quality releases and compliance with deadlines, a1qa designed a training course for all the talents joining the project. Should any necessity to augment the team preparing for the release arises, the engineers are swiftly redeployed to support the process.

Smart team scalability

To cover workload in terms of multiple subprojects, the number of engineers was augmenting to handle the increased testing scope and decreasing when the workload was reducing.

Process alignment

Due to multiple subprojects, each group followed its own habitual patterns that resulted in a desynchronized testing process. To increase organizational performance, a1qa unified the standard pool of tasks carried out by the QA experts — from defect validation to new feature testing.

Transition to Agile

Initially, only UAT followed Agile practices. However, having demonstrated its perks like efficient defect fixing and communication with developers, faster time to market, clean code, etc., a1qa contributed to shifting QA support scope to Kanban.

Quality level assessment

The work fulfilled by multiple teams was evaluated according to miscellaneous systems, which impeded the overall assessment of system quality. To standardize it, a1qa set up a unified approach to an impartial evaluation of the quality level within teams.

PI planning

At times, user stories weren’t detailed, which caused the lack of requirements for preparing test cases or the necessity to raise additional questions while conducting a test run. Therefore, a1qa joined the client’s regular meetings to plan the workload, provide and discuss insights on the proper system operation. A team of business analysts was also assigned to identify and clarify the requirements.

Adjusting to the client’s time zone

To reach the maximum overlap in time, sync up, and solve process questions together, a1qa set the rule for all its members to move additional activities (self-education) to the second part of each business day.

Mitigating risks

To eliminate the possibilities of downtime, a1qa developed a roster for all the QA teams. Continuous monitoring and defect fixing were observed due to introducing 4-hour shifts.

Strengthening organizational performance

Particular efforts encompassed fixing pitfalls that occurred on the side of 3rd party vendors, which led to an increase in operational costs and testing time. Having analyzed their root causes, a1qa created a report describing them. It allowed the client to reassess the efficiency of all the teams involved in product development and testing to eliminate bottlenecks.

CHALLENGES AND SOLUTIONS

Bumps in the road occurred due to software architecture hallmarks:

Problems within Accertify

Its configuration is close to the production environment. Therefore, the number of orders placed by the engineers for 1 day from 1 IP address was limited due to business requirements. To bypass them, a1qa added extra addresses to the lists.

Data glitches

They involve the cases of incorrect or insufficient data filling in while working in the admin panel. As a result, products aren’t configured correctly, the data is corrupted that causes errors during test runs. Therefore, a1qa applied BAPI calls in SAP used by the website during product placement to validate goods. In case of success, the product is included into tests.

SERVICES OFFERED
  • Functional testing & UAT
  • Test automation
  • Performance testing
  • Integration testing
TECHNOLOGIES & TOOLS
  • VPN GlobalProtect
  • Citrix Workspace
  • WebPageTest
  • Apache JMeter
  • Dynatrace
  • Postman
  • SAP GUI
  • SAP Java Connector
  • Splunk
  • PuTTY
  • Selenium
  • Cucumber
  • G Suite
RESULTS
  • Strengthened QA strategies due to testing process alignment within multiple teams, the release support scheme adoption, and the introduction of unified quality assessment applied by all client’s vendors.
  • Improved solutions’ performance owing to building the test approach from the ground up, decreasing the download time by 30%, and ensuring unhindered operation of all websites during Black Friday.
  • Faster time to benefit and reduced operational cost due to test automation implementation, which saved hundreds of business days of manual testing teams.
  • Enhanced organizational performance accomplished through reducing testing costs by decreasing the number of incoming issues missed by the third-party vendors.
IN NUMBERS
  • 2+
    years of the project duration
  • 33
    QA experts involved
  • 4,000+
    test cases prepared
  • 90%
    of manual testing efforts saved in terms of smoke testing due to test automation introduction
  • 30%
    reduction in loading speed of all eCommerce solutions
QA news and tips delivered right to your inbox
We’ll send you one newsletter a month, jam-packed with amazing QA offers, hottest industry news, and all kinds of Software Testing goodness.