Portfolio

a1qa provides extensive QA support to a large US-based manufacturer of home appliances

a1qa was identified as the ideal partner to help the client deliver upscale software solutions leveraging a holistic QA package, including test automation and performance testing.
Acceptance testing
Agile testing
eCommerce
Functional testing
Integration testing
Other
Performance testing
Test automation

Overview

The client is one of the leading US-based manufacturers of home appliances operating globally and possessing several trademarks offering an extensive blend of products.

The customer’s software under test includes B2B and B2C dimensions.

B2B solutions embrace:

  • A corporate portal for clients to help them trace business activities.
  • A website offering replacement parts & accessories to support the functioning of goods.
  • A commercial service allowing businesses to open laundries by renting equipment or leasing it.

B2C software encompasses:

  • Seven brands in different locales. Amid them, five brands serve commercial purposes, although, the availability of some functions depends on the country.
  • A resource for the client’s team members and partners allowing them to purchase items at reduced prices.

Aiming to reduce hosting costs and improve overall organizational performance, the client was undergoing migration to SAP Hybris Commerce Platform. To assure its quality by thorough UAT, introduce test automation best practices, build performance testing from scratch, and shape QA processes, the client partnered with a1qa.

Services offered

Functional testing
UAT testing
Test automation
Performance testing
Integration testing

Project scope

a1qa started by delving into the technical core of the solutions, which is based on the seamless interaction of the back- and front-ends.

Front-end involves:

  • AEM — a content management system responsible for delivering content across tested solutions.

Back-end incorporates:

  • SAP ECC contains data about all the products available in warehouses and delivered to end users, transactions performed, accounting, and more.
  • SAP Commerce Cloud stores catalogs and data about the products and registered users, introduces promotions and bundles, etc. It provides the possibility to administer websites allowing businesses to customize the content according to specific needs. It also redirects a huge flow of requests between servers to eliminate the possibility of SAP ECC breaking down.
  • Accertify — serves to prevent any fraudulent orders. It gathers data entered during placing an order, analyzes it, and provides the assessment of the current order. Considering it, the system can accept the order, place it to pending, or reject it.

Having learned all products’ nuances, the engineers got down to the quality assurance scope.

Functional testing & UAT

Functional testing was applied to the SAP Commerce Cloud customizable features:  

  • A promotion module: Out-of-the-box functionality didn’t meet the client’s business needs, and the development team had to spend numerous efforts on the customization and fine-tuning of a promotion module. With thorough software testing, a1qa helped businesses to create, edit, and control various campaigns aimed to increase conversion rate.  
  • A bundle module. The client’s team developed this functionality from scratch. Testing was a priority. It allowed the client to perform and fine-tune various marketing campaigns to boost sales, offer lucrative prices to end users, and avoid mistakes.  

UAT incorporated verifications to make sure the whole functionality scope was operating as it was intended, and the correct information was passed among various system components. Test cases representing real user scenarios included detailed steps from opening the website to buying a certain product.  

Testing scenarios that covered SAP ECC had deeper test coverage to verify vital business details (accuracy while creating invoices, disposal of goods, transfer of products between warehouses, etc.).

Test automation

Test automation with its primary objectives of speeding up time to market and reducing operational costs was pivotal to streamlining the testing scope.

Automated release verification tests were performed against the operation of the entire project’s portfolio. The essential component of tested functionality concerned placing orders. Many aspects could influence this process (e.g., product type, location, registration, delivery, availability in a warehouse), requiring certain preconditions for each order — which were out of the QA team’s control. To adapt to constant changes, the QA experts created the following workaround to address that challenge: 

  • Designed specific automated scripts that request SAP Commerce Cloud through the API and searches for the product that meets the preconditions required for a test case by specific parameters. 
  • Afterward, it’s selected on the website and placed as an order.    

As soon as the transaction is performed, SAP ECC creates a message about the successful operation, the presence and details of which are checked by this test. 

a1qa designed a special approach to prevent lost time on analyzing false positive results of automated tests. If during step 1 the product wasn’t found, the test was specifically highlighted. The QA experts could easily understand what was missing, quickly fix the problem, and rerun a test suit.  

Test automation introduction was effective: in just two months, it contributed to saving 50 business days of the manual testing team consisting of 4 people. All in all, test automation by a1qa curtailed the time required for release verification testing by 90%. The QA team is constantly working on expanding test coverage. 

Performance testing

To help the client generate high conversion rates, increase revenue, and boost customer experience, the a1qa engineers should ensure the software would operate under high loads during Black Friday, Cyber Monday, and other periods of increased influx of customers. 

The client already had a team responsible for this purpose, but their Black Friday testing strategy wasn’t fully defined and didn’t consider all variations in user behavior during this period. Therefore, the experts prepared user journeys emulating realistic loads, which later became the baseline for scripts.  

While creating them, a1qa enhanced the website load profile. The QA engineers took into account different user behavior models and included both the most widespread and the least common actions to verify the results of placing an order, adding products to a wish list, and scrolling through a website. The team developed around 200 performance test scripts. 

Afterward, all solutions were verified before each release by harnessing the activities listed below:

  • Client-side testing — to measure the load and render time for web pages. The QA engineers assessed the current performance of front-end elements, validated if CSS elements were compressed properly, analyzed website page loading process, collected special metrics to compare testing results with requirements, and prepared a report with defined ways of optimization, including but not limited to the architecture of the web page and the process of loading its elements.
  • Stress testing. Performance testing was carried out on the QA environment shared with the QA team due to the inability to provide a separate environment. Its hardware capabilities were lower compared to those of the production. Stress testing helped overcome this gap and define the maximum capacity of the environment. Then, the engineers extrapolated results on hardware capabilities of the production environment, taking into account the difference in settings or the presence of a load balancer. The client got an approximate understanding of the maximum capacity of the production environment and the ways of its further enhancement. 
  • Black Friday support — to guarantee the solutions would cope with the increased number of concurrent users during the high sales season. The team carried out 8-hour load tests, and the number of potential users was increased by 30%.

Suggested performance testing improvements:

  • The team offered to implement a certain response time (RT) parameter applicable to each solution type considering its specifics. 
  • To avoid the situations with failing tests due to content issues, they first ran automated scripts that checked the completeness of all product details on a QA environment and only then started Black Friday testing activities. 

To decrease release delay risks, a1qa recommended the client fulfill performance testing during the sprints that deliver features prone to performance bottlenecks, increase sprint length, and cover not only functional testing but also the required performance testing verifications. 

When a1qa joined performance testing activities, the client was using an expensive tool. To help the client optimize operational expenditure, a1qa’s engineers identified the set of features that the tool should have and proposed switching to an open-source solution with the same capabilities. After its successful usage during a pilot period of several testing rounds, the entire performance testing team started using it on a regular basis. 

Integration testing

It was intended to guarantee seamless integration of:

  • Ready-made payment platforms
  • Distribution of taxes. 

Each payment platform (Paymetric, PayPal, Affirm) possesses a set of rules to manage possible issues that may occur during money transactions. After entering, credit data is sent to the system for validation. The type of response depends on the variants of the issues spotted or the situations of processing the data. The tests check that all those types are considered by the system and are validated correctly. For testing on production environment, the team harnessed only real credit cards provided by the client.

The distribution of taxes 

Considering complex taxation in the USA, it was vital to make sure the correct tax is assigned to a certain state. Taxes are calculated in the following way. The request for a certain tax is sent to Vertex, then the number received is transferred to an order. After this order is created in SAP ECC, the process repeats once again to ensure the maximum accuracy. Testing helped guarantee the absence of software glitches and correct calculation of taxes.

Improving processes

Apart from testing, a1qa contributed to the following:

Provided full release support  

The client delivers new functionality once every 2-3 weeks. To ensure the quality releases and compliance with deadlines, a1qa designed an approach to training and sharing constantly expanding expertise on all subprojects so that all QA team members knew the nuances of any software module within all subprojects. Should any necessity to augment the team preparing for the release arise, the engineers are swiftly redeployed to support the process.

Ensured smart team scalability 

To cover changing workload in terms of multiple subprojects and optimize QA costs, a1qa ensured a knowledge retention process, enabling a1qa to ramp teams up or down upon client’s request. 

When the client started a new subproject, a1qa dedicated specialists with a vast project background from other subprojects or onboarded new members and helped them quickly delve into the client’s systems. 

Aligned processes 

Due to multiple subprojects, each group followed its own habitual patterns that resulted in a desynchronized testing process. To increase organizational performance, a1qa unified the standard pool of tasks carried out by the QA experts. For that, they:

  • Created unified templates for opening tickets and describing test cases using a set of best practices and standardized structure. 
  • Developed regulations to align the definition of severity and priority of defects between different teams.  
  • Introduced a bug triage practice into the workflow of several teams, explained how it could contribute to the overall process and what information could help speed up such sessions, as well as assisted in scaling the practice further.  
  • Regularly adjusted processes of bug triage and describing defects adopting them to changes on the client’s side (a new manager or software) for the convenience of those concerned. 

Helped shift to Agile 

Initially, only UAT followed Agile practices. However, having demonstrated its perks like efficient defect fixing and communication with developers, faster time to market, and clean code, a1qa contributed to shifting overall development process to Scrum.

Developed approach to assess the quality level 

The work fulfilled by multiple teams was evaluated according to miscellaneous systems, which impeded the overall assessment of system quality, release readiness, and specifying the definition of done. To standardize it, a1qa set up a unified approach to an impartial evaluation of the quality level within teams.

Proactively participated in PI planning

At times, user stories weren’t detailed, leading to critical defects that occurred late in the development process. Therefore, a1qa joined the client’s regular meetings to plan the workload, provide and discuss insights on the proper system operation. A team of business analysts was also assigned to identify and clarify the requirements.

Adjusted to the client’s time zone to the fullest extent 

To reach the maximum overlap in time, sync up, and solve process questions together, a1qa set some rules for all its members regarding the distribution of work activities during the day. Thus, isolated project activities not requiring other parties’ involvement were conducted during a certain period of time. Other time slots were available for cooperating with the client’s R&D teams and project stakeholders if required without the need to rearrange or postpone planned activities or work overtime. 

Mitigated business risks

To eliminate the possibilities of downtime, a1qa developed a roster for all the QA teams. Continuous monitoring and defect validation were observed due to introducing 4-hour shifts.

Strengthened organizational performance

Particular efforts encompassed fixing pitfalls that occurred on the side of 3rd party vendors, which led to an increase in operational costs and testing time. Having analyzed their root causes, a1qa created a report showcasing the current state of things together with the list of improvements required. It allowed the client to reassess the efficiency of all the teams involved in product development and testing to eliminate bottlenecks and enhance project workflows further. 

Due to the complicated interoperability of these systems, the QA engineers faced process challenges: 

Problems within Accertify 

Its configuration is close to the production environment. Therefore, the number of orders placed by the engineers for 1 day from 1 IP address was limited due to business requirements. To bypass them, a1qa became familiar with Accertify regulations and how to configure them. Finally, the engineers updated their work environment and Accertify’s settings in such a way that allowed the tests to run without any blockages.  

Data glitches 

They include desynchronization in the interaction of SAP ECC and SAP Commerce Cloud. The data from SAP ECC is transferred to SAP Commerce Cloud at night. Sometimes, the request for a product chosen from SAP Commerce Cloud can’t be processed as SAP ECC states that the item is out of stock. This results in a failed test.  

Content issues  

They involve cases of incorrect or insufficient data filling while working on the admin panel. As a result, products lack descriptors, which also causes errors during test runs.   

To help minimize the influence of content and data on the stability of automated tests and reduced the number of false positives, a1qa introduced automated scripts (that created the selection of products from SAP ECC, SAP Commerce Cloud, and websites), analyzed them, and presented a list of items that lacked sufficient data for carrying out tests.  

Technologies & tools

  • VPN GlobalProtect
  • Citrix Workspace
  • WebPageTest
  • Apache JMeter
  • Dynatrace
  • Postman
  • SAP ECC
  • SAP Java Connector
  • Splunk
  • PuTTY
  • Selenium
  • Cucumber
  • G Suite

Results

  • Strengthened QA strategies due to testing process alignment within multiple teams, the release support scheme adoption, and the introduction of unified quality assessment applied by all client’s vendors.
  • Accelerated time to benefit due to test automation best practices, which contributed to cutting the release verification testing time by 90%.
  • Reduced operational costs by 120K per year due to the selection of a more economical testing tool.
  • Improved solutions’ performance and stability owing to building the test approach from the ground up, decreasing the download time by 30%, and ensuring unhindered operation of all websites during Black Friday.
  • Faster time to benefit and reduced operational cost due to test automation implementation, which saved hundreds of business days of manual testing teams.
  • Enhanced organizational performance and reduced testing costs due to fixing requirements issues at the level of PI planning, implementing a unified approach to quality level assessment and performance KPIs across all QA vendors, aligning processes.

In numbers

4+
years of the project duration
33
QA experts involved
5000+
automated tests cases prepared
30%
increase in loading speed of all eCommerce solutions
200+
performance scripts written

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.