Lecture
Modern software is simply obliged to work smoothly under enormous loads. Any kind of problems associated with poor performance can cause customers to stop using your software. In this regard, the conduct of high-quality load testing should be mandatory, to ensure the stability of your applications.
Now let's answer the question: “ What is load testing? ”, And we will try to describe in more detail the process of conducting load testing .
Load testing (Load Testing) or performance testing (Performance Testing) is an automated testing system that simulates the operation of a certain number of business users on a shared (shared) resource.
Starting work in the field of load testing, you should clearly understand that this is not just a record and a run (Record and Playback) of scripts, but a more complex process:
Next, I suggest you, item by point, go deeper into performance testing :
We also recommend reading the articles of stress testing specialists :
Of course, reading all this is not enough to say: “I have become a connoisseur or expert in performance testing,” for this you will need more than one year and more than one project. From ourselves we can only wish you good luck on this difficult path. If you have any questions, you can feel free to contact us. Specialists from the ProTesting group can consult on software testing and quality assurance.
To discuss approaches to load testing and problems solved with its help, we suggest starting with terminology. Understanding the same entities under different terms, one can speak “one language”.
Load testing or Performance testing is an automated testing that simulates the operation of a certain number of business users on a shared (shared) resource. As an example, we can cite the work of employees of a modern bank in which everyone works with the same software applications installed on banking servers. Or the use of a web shop software application, in this case, the visitors who load the server will be Internet users.
The load is modeled using special products and techniques. What, what and how are we going to model? In order to understand this and you need to decide on the terminology:
Now consider how these entities are related. Expressing the intensity through the time interval between iterations, we see that the increase in the intensity of the operations performed is a reduction in the time interval. The increase in load is proportional to the increase in intensity. It is also natural that with increasing intensity, productivity increases. This increases the degree of use (workload) of resources. From a certain moment the productivity growth stops (and the load can continue to grow), saturation occurs and then the system degrades. In addition, it can be noted that during testing, the change in the intensity of operations may obey a law (for example, Poisson) or be uniform throughout the test.
The main goals of load testing are:
Note that within the same goal, different types of performance and load tests can be used, for example, for the first, second and third goals, both performance testing and stability testing should be performed. But when planning load testing it is more logical to build on the technical objectives (and not commercial ones listed above) that are achieved as a result of testing and classify tests according to them:
The main thing is to understand the goals of a particular test and try to achieve them.
Considering the stages of load testing , I would like to note the following, in our opinion mandatory:
Having decided on the types of load testing, goals and terminology, we turn to the main task of load testing - the development of a load model .
To do this, define the following:
The list of tested tasks should include operations that are critical from a business point of view, as well as from a technical point of view:
We would like to emphasize once again that by the degree of criticality of an operation, we mean its impact on the business process and the performance of the system . For example, creating a report that completely loads the database server at night will not be a high priority for optimization, and during business hours it will have the highest priority.
Next, I invite you to familiarize yourself with the following sections, step by step, revealing the specifics of developing a load model:
We will call the application under test as an “application.” To select parts of the application, namely the operations that will be tested, it is necessary to carry out work related to the study of the application. Application developers should be very helpful in this case, if we are talking about testing in the development process, or business users and system administrators, if the application is in operation. In the course of this work it is wise to take such steps:
Once again I would like to note that a survey of business users or a joint study with the developers and administrators of the system can greatly facilitate the task. If the application is in operation, you can monitor the load of the hardware components of the server (processor, memory, disks) and analyze the system logs of web servers (remove stats pack if you use Oracle as a database server, for example). Syslogs can show peaks of high user activity during the day and give a quantitative estimate of how many transactions (hits) are performed per unit of time. According to Pareto’s law or the 20/80 principle, 20% of application operations generate 80% of the system load, so you should try to select these 20% of operations for modeling.
The key point in the load model is the operation or load profile selected for testing. Naturally, these operations in the test should be performed simultaneously. There can be several load profiles for the application and this is justified. After all, business users can perform different sets of operations at different times. For example, the beginning of the trading day and the end of the day, the beginning of the month (quarter) and, accordingly, the end may differ. Thus, we obtain different sets of application operations that are performed simultaneously and respectively create different loads. By the way, not only the operations themselves can change , but also their intensity . In the first approximation, the load model is a set of load profiles, where each profile differs from another by either a set of operations or the intensity of these operations.
An example of the load profile , which includes 5 operations, the value of n can be different for each operation:
<Load Profile>
Since there are usually several operations in the load profile, this means that we will have several user groups. It is desirable to model each operation by a separate group of virtual users (although in life it is often the other way around, one business user may be responsible for performing several operations). However, if you assign one virtual user to perform a single operation, then it is easier to sustain a certain intensity (and, accordingly, performance) for this operation in the test than in the case when a sequential chain of operations is assigned to the virtual user. Knowing the intensity of the operation, you need to determine the number of virtual users in the group performing this operation. The ideal case when working with an application is similar to the work of a factory pipeline and there are accurate estimates of how many operations a user does per day . Most often it is not so and only the total number of operations performed during the day is known. It may also turn out that the intensity of the operation performed by each user is very low, for example, one user performs the operation once a day or every other day.
Simulation of work with recalculation of intensities in this case can be illustrated by the following example.
EXAMPLE : (for one operation)
Number of operations = 200 for 4 hours
Number of business users = 20
1. Determine the number of operations for each user.
200/20 = 10
2. Every user performs 2.5 operations per hour.
10/4 = 2.5
3. Determine the repetition period of the operation (intensity) for each user.
60 / 2.5 = 24 minutes
4. To reduce the test time to an hour, and at the same time get at least some statistics on the execution of operations, as well as improve the "mixing" of operations during the test, you can increase the intensity by 4 times, while reducing the number of users just 4 times .
24/4 = 6 min. = 360 seconds
Thus, 5 virtual users, performing an operation with a period of 6 minutes, will provide for 4 hours a given performance equal to 200 operations.
What gives such a recount :
First , it is possible to reduce the test time without losing the statistics on the performance of operations, and hence the reliability of the test results.
Secondly , it is possible to reduce the number of simulated users where their number goes beyond several thousand and thus lower the requirements for the number of resources of the loading computers (1VU may require up to 10MB of operating memory of the loading computer), and in some cases comply with the terms of the license agreement (license cost for test tools depends on the number of virtual users)
What are the limitations :
For some applications, it may be critical to the number of simultaneously open connections between the client and the server. There were situations when requests to the database started to work much slower if, when working with an application, the number of different users (different logins) increases and not the intensity of requests. And finally, the increase in the intensity of operations, should not lead to a situation when the repetition period becomes less than the time of the operation itself.
In any case, despite some limitations, such a calculation may allow for variation in the number of virtual users and the intensity of their operations. At the same time, performance and, accordingly, the load should not change and be different from the set one. So, the load point includes the calculations of virtual users and intensities for all load profile operations.
I would like to note that the calculation of the load point, described in the section Calculation of load points based on statistics collected for a running application (or on the expected workload for a newly developed application), is the starting point for further load growth, and the calculated load point can be considered a baseline or baseline point . Now you can increase the load, moving with a certain step, while increasing only the number of virtual users in groups, without changing the intensity of operations for a single virtual user.
A full load model is a set of load profiles with all load points for each profile . When developing test scenarios, all load points must be correctly implemented. I would also like to add that the load points for each profile must be at least three, in order to be able to evaluate the dependence of the response times of the operations performed on the growth of the load. Obviously, the more linear such dependence, the better the scalability of the application and the higher the predictability of its behavior under load.
Commercial tools for automated load testing:
| Hewlett-Packard (Mercury Interactive) | HP Performance Center (includes HP LoadRunner) |
| IBM Rational | Rational performance tester |
| Borland (Segue) | Silk performer |
| SmartBear | LoadComplete Web Load Testing |
| Neotys | NeoLoad |
A separate item will highlight the free tools for automated load testing:
Comments
To leave a comment
Quality Assurance
Terms: Quality Assurance