Web Performance Testing Metrics
Performance
Partial overview of web performance testing Generally speaking, the processing of a web request includes the following steps:
(1) The client sends a request
(2) The web server receives the request and processes it;
(3) The web server obtains data from the DB;
(4) The webserver generates the user's object (page) and returns it to the user. The time from the start to the last byte of the request sent to the client is called the response time (the third step is not included in each request processing).
1. Transaction
In web performance testing, a transaction represents a process of "sending a request from the user -> the web server receives the request and processes it -> the web server obtains data from the DB -> generates the user's object (page) and returns it to the user", General response times are for transactions.
2. Request response time
Request response time refers to the time it takes for the process to start from a request initiated by the client to the end when the client receives the response returned from the server. In some tools, the response is usually called "TTLB". That is, "time to last byte", which means the time it takes from the initiation of a request to the time the client receives the last byte of response. The unit of response time is generally "seconds" or "milliseconds". A formula can represent: response time = network response time + application program response time. Standards can refer to foreign 3/5/10 principles:
(1) Within 3 seconds, the page responds to the user and displays it, which can be considered "very good";
(2) Within 3 to 5 seconds, the page responds to the user and displays it, which can be considered "good";
(3) Within 5 to 10 seconds, the page responds to the user and displays it, which can be considered "barely acceptable";
(4) If it takes more than 10 seconds, it will make people a little impatient, and the user will probably not continue to wait;
3. Transaction response time
A transaction may be composed of a series of requests. The response time of a transaction is mainly aimed at users, which is a macro concept and is proposed to explain the business response time to users. For example: the response time of cross-bank withdrawal transaction is composed of a series of requests. Transaction response time is a parameter that directly measures system performance.
4. Number of concurrent users
Concurrency is generally divided into two situations. One is concurrency in the strict sense, that is, all users do the same thing or operation at the same time, and this operation generally refers to doing the same type of business. For example, in the credit card approval business, a certain number of users submit the completed approval business at the same time; there is another special case, that is, all users perform exactly the same operation, for example, in the credit card approval business, all users can apply together business, or modify the same record.
Another kind of concurrency is broad scope concurrency. The difference between this kind of concurrency and the previous one is that although multiple users have issued requests or performed operations on the system, these requests or operations can be the same or different. For the whole system, there are still many users operating the system at the same time, so it also belongs to the category of concurrency.
It can be seen that the latter concurrency includes the former concurrency. And the latter kind of concurrency is closer to the actual usage of users, so for most systems, only a small number of users perform "strict concurrency". For WEB performance testing, these two concurrency situations generally need to be tested, and the usual practice is to conduct a strict concurrency test first. User concurrency in the strict sense generally occurs in frequently used modules. Although the probability of occurrence is not very high, once a performance problem occurs, the consequences are likely to be fatal. Concurrency testing in the strict sense is often associated with functional testing, because exceptions encountered in concurrent functions are usually program problems, and this kind of testing is also part of robustness and stability testing.
The number of concurrent users: There are two common misconceptions about the number of concurrent users. A wrong view is to understand the number of concurrent users as the number of all users using the system, because these users may use the system at the same time; another view that is closer to the correct view is to understand the number of online users as the number of concurrent users. In fact, online users may not necessarily be concurrent with other users. For example, users who are browsing the web have no impact on the server. However, the number of online users is one of the main bases for calculating the number of concurrent users.
5. Throughput
Refers to the sum of the amount of data transmitted on the network during a performance test. Throughput/transmission time is the throughput rate,
6. TPS (transaction per second)
The number of transactions or transactions that the system can process per second. It is an important indicator to measure the processing power of the system.
7. Click rate
The number of HTTP requests submitted by users to the WEB server per second. This indicator is a unique indicator of WEB applications: WEB applications are in the "request-response" mode. Once a user sends an application, the server must process it once, so a click is the smallest unit of transaction that a WEB application can handle. If each click is defined as a transaction, click rate and TPS are a concept. It is easy to see that the greater the click-through rate, the greater the pressure on the server. The click-through rate is only a performance reference indicator, and it is important to analyze the impact of clicking. It should be noted that the click here does not refer to a click operation of the mouse, because in one click operation, the client may send multiple HTTP requests to the server.
8. Resource utilization
It refers to the degree of usage of different system resources, such as the CPU utilization rate of the server, disk utilization rate, etc. Resource utilization is the main basis for analyzing system performance indicators and then improving performance, so it is the focus of WEB performance testing.
Resource utilization is mainly aimed at WEB servers, operating systems, database servers, networks, etc., and is the main reference for testing and analyzing bottlenecks. In the WEB performance test, more according to the need to collect the corresponding parameters for analysis.
Common Indicators
(refers to the required test items for web application server and database server)
| Indicator | Description |
|---|---|
| ProcessorTime | Server CPU usage, generally when the average reaches 70%, the service is close to saturation |
| Memory Available Mbyte | The number of available memory. If you find that the memory has changed during the test, you should also pay attention. If it is a memory leak, it is more serious |
| Physicsdisk Time | Physics disk read and write time |
Web server metrics
| Indicator | Description |
|---|---|
| Requests Per Second (Avg Rps) | Average number of responses per second = total request time / seconds |
| Avg time to last byte per terstion (mstes) | The average number of iterations of business scripts per second, some people will confuse the above one |
| Successful Rounds | Successful Requests |
| Failed Requests | Failed Requests |
| Successful Hits | |
| Failed Hits | Number of failed hits |
| Hits Per Second | Hits Per Second |
| Successful Hits Per Second | Successful Hits Per Second |
| Failed Hits Per Second | Number of failed hits per second |
| Attempted Connections | Number of attempted connections |
Database server performance indicators
| Indicator | Description |
|---|---|
| User 0 Connections | The number of user connections, that is, the number of connections to the database |
| Number of deadlocks | Database deadlocks |
| Butter Cache hit | Database Cache hits |
System bottleneck definition
| Performance Items | Commands | Metrics |
|---|---|---|
| CPU limit | vmstat | when %user+%sys exceeds 80% |
| Disk I/O Limit | Vmstat | When %iowait exceeds 40% (AIX4.3.3 or later) |
| Disk limit applied | Iostat | When %tm_act exceeds 70% |
| Less virtual memory space | Lsps, -a | When the activity rate of the paging space exceeds 70% |
| Paging limit | Iostat, stat | When the virtual storage logical volume %tm_act exceeds 30% of I/O (iostat), and the activated virtual storage rate exceeds 10 times the number of CPUs (vmstat) |
| System failure | Vmstat, sar | Paging increase, CPU wait and run queue |
Stabilize the resource state of the system
| Capabilities | Resources | Reviews |
|---|---|---|
| CPU usage | 70% | OK |
| CPU usage | 85% | bad |
| CPU usage | 90%+ | very poor |
| Disk I/0 | <30% | Good |
| Disk I/0 | <40% | Bad |
| Disk I/0 | <50%+ | very poor |
| Network | <30% Bandwidth | Good |
| Run Queue | <2*Number of CPUs | Good |
| memory | no paging | good |
| Memory | 10 paging per second per CPU | Bad |
| memory | more paging | poor |
Popular understanding
- Daily visits
- The maximum concurrent number of common pages
- Number of people online at the same time
- access corresponding time
case
Recently, a project of the company, which is a portal website, needs to be tested for performance. According to the characteristics of the project, the main test items and test plan are determined:
One is to test the maximum number of concurrency acceptable to several commonly used pages (user name parameterization, set collection point strategy)
One is to test whether the user can operate normally under long-term pressure on the server (parameterized user name, iteratively running scripts)
One is to test whether the server can accept 100,000 users to operate online at the same time. If IIS is used as an application server, the maximum number of concurrency that a single server can withstand cannot reach 100,000. The machine is used for load balancing; if an application server such as websphere is used, the maximum number of concurrency that can be tolerated by a single machine can reach 100,000, but for performance considerations, it is still necessary to use a cluster to do load balancing through multiple machines. Realization; usually there is a simple calculation method, one connection generates one session, and each session has a memory space setting on the server, which is 3M on NT, so 100,000 concurrency requires 300G memory, of course the actual Considering that other programs also take up memory during use, the amount of prepared memory is required to be more than this. There are also 100,000 users online at the same time, which are two completely different concepts from 100,000 concurrent users. It's already been said above. But how to do this conversion to convert 100,000 simultaneous online users into how many concurrent numbers? This requires a large amount of historical log information to support it. The system log needs to have the log information of the number of concurrent online users and the log information of the number of user operations. The ratio of these two data is the ratio of your simultaneous online users to the number of concurrent users. In addition, according to empirical statistics, for a WEB system developed by JAVA (I haven’t counted others, I can’t give the data), generally, the maximum number of concurrency that can be supported on a server with dual CPUs and 2G memory does not exceed 500 (this In this state, most operations are timed out and errors are reported, and the server is easy to go down, which is of no practical significance), and the maximum number of concurrency that can be used normally (the waiting time for single-step non-large data volume operations does not exceed 20 seconds) does not exceed 300. Assuming that your 100,000 simultaneous online users convert 9,000 concurrent users, then you need at least 18 such machines, and no less than 30 are recommended. Of course, if you buy a large server with 200 CPUs, 256G memory, and gigabit fiber bandwidth, even if there are 100,000 concurrent users, the speed will definitely be soaring.
In addition, under the severe cold, it takes nearly 6 hours for all the settings to enter the running state. Specifically, you can take a system to press and see, the following situations may occur:
- The server is down;
- The client is down;
- From a certain time, the server rejects the request, and all errors are displayed on the client;
- The test is barely completed, but the network is congested or the test result takes a long time to display. Assuming a 100M bandwidth between the client and the server, 100M/10000=10K, then each user can only get 10K, which is close to the speed of a 64K MODEM;
In addition, none of the above analyzes considered the background of the system, such as databases, middleware, etc.
- In terms of servers: 50 PC SERVERs as mentioned above are needed;
- Network aspect: According to 50K per user, at least 5 100M bandwidths are exclusive, and the network delay alone is estimated to be at the second level;
- If there is a database, at least Oracle, preferably SYSBASE, SQLSERVER will definitely not be able to withstand it. The database server requires at least 10 machines with 4CPU and 16G memory;
- If there is CORBA, then prepare at least 10 machines with 4CPU and 16G memory; plus load balancing, firewalls, routers and various software, etc., in short, without a capital investment of 10 million, it will definitely not work.
Since such a portal system has user rights, it is not mostly static pages as Jackie said. But as long as it is a multi-server cluster, then we can calculate the load capacity of multiple machine clusters through the test results of one machine. At most, additional consideration should be given to load balancing and routing pressure, such as bandwidth, speed, delay, etc. . But if it is all changed on one machine, then we can only do calculations on some indicators. From these indicators, we can simply judge whether it is not feasible. For example, if 100,000 concurrent users only have one 100M bandwidth, then we can It is obviously not feasible to calculate that each user has only 1K bandwidth. But the actual results still need to be tested to know, after all, the system pressure and the number of users do not change linearly.
The widespread and mature use of this type of system, as well as the fact that many software can roughly estimate the performance characteristics of the system after the design of the scheme, have led to a small proportion of the system in terms of software performance tuning (of course, it is not completely ruled out for Some codes and configurations are optimized for further performance improvement), and more are considered from the hardware aspect, such as increasing memory, hard disks for RAID, increasing bandwidth, and even adding machines.
The 10M bandwidth in network technology refers to the calculation in bits, which is 10M bit / second, and the download speed is calculated in bytes (Byte), so 10M bandwidth is converted into bytes The theoretically fastest download speed For: 1.25 M Byte/sec!