What testers need to know about user acceptance testing
For more than two years, a1qa has been engaged in the project of the payment system development for a huge banking organization. At the very beginning of the project, the a1qa engineers were responsible for testing the system. However, some time ago the customer asked our team to plan, perform user acceptance testing (UAT), and present the results.
In today’s post, we’ve decided to build the basis for effective user acceptance testing, shed light on its nuances, and answer all possible questions that the QA team may have.
What is user acceptance testing?
When it comes to any new concept, it may be useful to analyze its name first. In technical context (as that of the quality assurance) the name will be literal and will give you the first understanding of the issue.
UAT stands for user acceptance testing. Let’s explain the definition word by word.
1. Acceptance = approval, validation.
2. A user = either the end consumer of the product or the customer who ordered the product development.
UAT literally means that the software will be tested by the user to find out whether it can be accepted for further development and production.
Indeed, long before the product is released to its end users, in most cases the customer wants to know that the product has been developed considering all the requirements and specifications and will work correctly in its real environment.
UAT must be an indispensable part of the projects where poorly developed software can cause huge financial losses.
Thus, the objectives of UAT are the following:
- Check that all preconcerted requirements have been satisfied and the product fits for business purposes.
- Detect last-minute mistakes that could have occurred at the development stage.
- Verify that the product is production-ready.
Who and when performs UAT?
Usually, UAT takes place right before the product is delivered to end users and after the QA team have finished their job. However, in some cases, the customer may want to follow the product development (especially, when the project is too costly and any mistake may result in great financial losses). Then, UAT may be performed twice or thrice per project to validate the right course of work.
During UAT, the product is tested either by end users who provide their feedback or someone who has built the software through an independent software vendor.
As for QA, their involvement may differ and come down to one of the following options:
- Not involved (a very rare case).
- Assistance in UAT – QA engineers may be asked to teach users how to use the software, submit defects, etc.
- Directly involved – The QA team evaluate the software and present the results to the customer who decides whether the product has been developed as expected.
If your QA team undertakes the responsibility to perform UAT, then start with selecting those specialists who possess excellent knowledge of the product and business objectives. He or she should be able to think of the software as a guest user.
After the responsible team member is selected, start with preparing a thorough acceptance test plan (ATP). It will regulate and facilitate the process of acceptance testing planning and performing.
Acceptance test plan: main sections
Usually, the specially trained personnel draft the document. Technical writers, for example. However, the QA team needs to fill it in with relevant data and keep it updated.
Let’s look through the contents of the typical ATP and briefly outline the contents of each section.
Introduction – the name of the product under test, background information, and product functionality.
UAT objectives – generally, these are the ones we’ve stated above.
Scope of requirements – the list of environments that must be verified during the testing procedure.
Pay attention that the requirements that are enumerated in this section must be demonstrated during the testing procedure (or at least, they should be planned for). If 100% of requirements have been implemented but only 70% can be demonstrated, then the customer will consider the remaining 30% as non-implemented, until the opposite is proved.
UAT tools and procedure – the list of all technical software and hardware tools applied during testing and the procedure. We recommend to specify all the tools that will be applied during the acceptance testing: databases, consoles, log files, automated tests – all this should be mentioned in this section. Otherwise, you won’t be allowed to use them during the acceptance testing process.
UAT methods – specifies how the demonstration is conducted, what methods are used to verify the implementation of a particular requirement and the expected result for each verification.
UAT exit criteria
As a rule, the criteria that must be met to formally end acceptance testing are determined at the stage of contract signing. One of these criteria can be the percentage of successfully passed test cases.
For example, if 100 test cases have been implemented and the pass rate is agreed on 80%, then 80 test cases must be successfully passed. Otherwise, the product won’t go through UAT and won’t be admitted to production.
However, most often the success of UAT is evaluated by the set of criteria. For example, the percentage of implemented requirements and the number of defects of certain priorities. Based on these results, the customer will judge the readiness of the product.
This ends the necessary theory that the QA team should be aware of before committing to UAT. Jump to this article to learn about 4 main challenges you might face and ways to overcome them.
Next week we’ll tell you how our team demonstrated the results to the customer and ensured the product successful release. Stay tuned!
