The client is an internationally acknowledged front-runner in innovative analytics and data usage with its focus on driving human health forward by providing pharmaceutical companies with a full cycle of professional services.
The client collects, analyzes, and provides big data related to sales operations all over the world to its end users.
To automate the testing process, accelerate t2m, and ascertain data integrity, consistency, and compliance with the set business logic and requirements, the client was seeking a QA partner to team up with and entrust software testing guidance and support.
a1qa took the mission over and contributed to turning the client’s need into a success story.
Organizing the workflows
To process gathered big data, a special infrastructure was developed on the project. Partnering with a wide range of end clients, it was necessary to exclude the required data from the sources and group it according to business requirements. Then, the client’s account managers sent the adjusted output DBs to a particular customer and updated them within the pre-defined timeline (weekly, monthly, quarterly, etc.).
To facilitate the process, the client divided the project into subprojects serving particular end customers. a1qa assigned two dedicated specialists for one of such subprojects, which supported a multinational pharmaceutical company by providing high-quality information. The QA engineers were to supplement the Dev and testing teams to eliminate big data faults and deliver high-quality information to the end customers.
Data life cycle
Prior to performing testing activities, the team delved into specifics of the big data transformation process that encompassed 2 phases: its development and testing.
The database creation phase, in turn, was split into the following steps:
Having received this voluminous database, the QA team initiated its testing process.
Services delivered
The a1qa engineers checked that data met the following criteria:
To make sure that big data doesn’t contain duplicates, formatting issues, blank rows, and invalid values. Moreover, the QA team verified that the set of business rules (business logic) are applied in accordance with project requirements.
To check the compliance of data in source files with the information in the output database after data transformation (e.g., no incomplete data, no data loss happening due to the data transformation process).
To ensure that delivered information in the database is up to date.
To ensure delivery of only relevant data to end clients according to the proper database structure.
To ascertain that data refresh doesn’t trigger any error while enabling timely data delivery upon end clients’ requests.
The a1qa specialists supported the client throughout the entire data life cycle. On staging/consolidation/output database level, the QA team also checked portions of specific cases like displaying data in the listing format, correct calculations of measures added based on the source data, groups of data, etc.
Automating the big data testing process
The team handled a substantial amount of source files containing voluminous data, and their testing using SQL queries was time-consuming.
Therefore, to facilitate and speed up the process, the a1qa engineers applied automated QA files and reshaped them within the project requirements and the end customer’s expectations.
The mechanism for performing automated checks was the following:
By applying Macros, the testing workflow (from uploading the data to checking testing results) was automated within each file. In case of data or DB updates, all the downloaded information, SQL queries, formulas, etc. are also changed automatically.
During a subsequent stabilization stage, all the bottlenecks detected by the QA experts were transmitted to the Dev team for fixing (data was missing/trimmed, contained unexpected characters, was downloaded partially, the format was written incorrectly, etc.) on the database level.
Only after polishing the revised version, it was sent to production.
Improving processes
Each subproject had processes established in various ways, so the customer didn’t have a centralized QA workflow that could adversely impact the teams’ efficiency.
To help the client refine QA activities, the a1qa specialists suggested introducing an Agile board. This initiative made the voluminous number of tasks more structured and transparent while every project member could easily grasp their current statuses and assignees.
The a1qa experts put other Agile practices forward. The client embarked on conducting retrospectives to inspect past processes and create a plan for improvements.
Thus, a1qa provided the customer with complete visibility and a well-tuned QA process on the entrusted subproject while contributing to the enhanced performance of other subteams.
When all is said and done, a1qa helped assure the quality of big data encompassing hundreds of source files while delivering information to the end customers timely and within their expectations.
The project was successfully completed by revealing bottlenecks and strengthening the overall quality. Now, cooperation with the client continues within a range of other projects.
