The client is one of the leading games powerhouses operating globally and creating an extensive blend of products and services within gaming industry.
Client’s product is a mobile application developed for Android and iOS platforms, which represents an online casino game integrated with Facebook.
The client reached out to a1qa as it was crucial to ensure that the system would operate under the load of 10,000 concurrent users with the response time (RT) not more than 0.2 sec.
To carefully meet client’s requirements and help deliver a stable product, a1qa assigned a skilled team of professionals who learned all the bells and whistles of the solution and elaborated effective testing methodology to independent assessment of current product state.
Namely, before getting down to testing itself, the a1qa team prepared a set of real-life user journeys to emulate a more realistic load.
Close cooperation with the Dev team made it possible to consider every single aspect of their realization.
Hence, having agreed upon particular details with the client, the a1qa team proceeded to the next step – load scripts development.
In this regard, the specialists applied Apache JMeter to create scripts with complex logic. In addition, all of them alongside associated remarks were added to GitHub so that the client could track the progress in real time.
Having fulfilled all the above-mentioned steps and aligned further activities with the client, the a1qa team proceeded with executing tests.
They were conducted on a pre-production environment, with the software and hardware configuration as close as possible to the production to obtain reliable results.
Performance testing tool was installed on the load generators – Google Cloud instances – used to create a load on the system with a minimized distortion level caused by network delays.
The a1qa engineers started from stress testing to understand the upper limit of the solution capacity and analyze its dependence on the number of concurrent users, requests, and transactions.
The results of the first check helped reveal the following system condition:
- The load was gradually increasing until the system reached its critical level, which initially was much lower than required by the client (1,200 users).
- RT enlarged considerably as well (up to 5 sec).
- The number of HTTP errors increased dramatically.
- The whole system crashed.
However, the client had the endeavor to develop the solution able to handle tenfold enhancement in load.
Working in close cooperation with the Dev team, the a1qa specialists moved on to the next step on the path to clients’ core objective.
These verifications are aimed at revealing the system ability to augment its performance proportionally to the increase of the hardware system resources.
Google Cloud Server Autoscaling was tuned on to meet this goal. The whole process was organized in the following way. When the total load level of initially allocated machines, or instances, surpassed 70%, additional machine came into play, and the overall load was evenly distributed.
Thus, the desired load level was reached gradually, and the team wasn’t restricted in resources as the whole process was automated.
However, the a1qa specialists managed to introduce the range of improvements to the process:
- During the analysis of testing results, it was stated that the RT was changing occasionally toward growth. Therefore, a1qa suggested that the number of initial machines was enlarged to minimize these fluctuations.
- In the scope of scaling, the team provided the client with the exact number of instances necessary to support the desired load level (turned out to be five times more than the initial one).
- The rent of instances within GCP isn’t free of charge. To help the client estimate the approximate price for renting them all, the a1qa specialists measured the cost during testing and produced predictions regarding further expenses for supporting such an environment on an ongoing basis.
The next step in preparing the solution to meeting all the requirements involved load testing, which allowed determining whether the system was capable to cope with the target load for an extended period of time.
Autoscaling was also tuned on. Having reached the necessary load level, the team maintained it for 5 hours. This particular time interval was chosen based on the statistics from analytical services regarding the average time spent while working with the application.
Load testing results showed that RT was remaining constant and within the normal range throughout the whole period.
To summarize all the mentioned above, performance testing carried out by the a1qa team helped the client assess current load level the system can withstand.
The engineers developed and adopted a well-thought-out testing strategy that helped swiftly put the client’s expectations into effect.
Detailed results with all the recommendations were compiled in the final report.
Furthermore, the data was presented to stakeholders in a comprehensible and lucid manner, which they specially highlighted afterwards.
A few weeks later a1qa was contacted once again to fulfill similar activities for one more company product.
- Performance testing
- Stress testing
- Load testing
- Scalability testing
TECHNOLOGIES & TOOLS
- Apache JMeter
- Google Cloud
The system performance proved to withstand the desired level of 10,000 concurrent users with RT for each user action < 0.2 sec.
The client received the preliminary assessment of the cost for supporting the environment on a full-time basis.
Stakeholders assessed transparency of presenting testing results provided by a1qa.
10,000constant user flow the application can seamlessly handle after performance testing
< 0.2sec – response time from the server
7weeks of the project duration
2performance testing experts involved