Web Application s Performance Testing

(1)

Web Application’s Performance Testing

B. Election Reddy (07305054) Guided by

N. L. Sarda

April 13, 2008

(2)

1 Introduction

In Software Engineering Process Models, Testing is an important phase to guarantee the quality of the product. In most of the software projects the greatest part of the testing effort goes to functional testing and some projects don’t do performance testing at all.

For Web Applications, performance testing is crucial as the functioning of the application highly depends on the load conditions. The application may perform well at average load but may crash at high loads. So before actual deployment, performance testing will enable the developer to identify the bottlenecks in the system and can be rectified.

2 Objectives

• Assess production readiness: Performance testing enables to pre- dict the performance characteristics of an application in production and evaluate whether or not to address performance concerns based on those predictions. These predictions are also valuable to the stake- holders who make decisions about whether an application is ready for release or capable of handling future growth, or whether it requires a performance improvement/hardware upgrade prior to release.

• Evaluate against performance criteria: Calculated Performance metrics like response time, throughput, resource utilization can be eval- uated against desired values. In general, response time is a user concern, throughput is a business concern, and resource utilization is a system concern.

• Compare performance characteristics of multiple systems or system configurations: Performance metrics can be used to compare web applications. A web application’s performance can be compared with different configurations so that best configuration can be identified.

• Find the source of performance problems: Bottlenecks in the web application can be identified by performing different types of test- ings. Bottlenecks for web applications can be code, database, network, peripheral devices.

• Support system tuning: By providing information related to scalability and stability of a product prior to the production release, Perfor-

(5)

mance testing enables to make informed decisions about whether and when to tune the system.

3 Performance Indicators

• Resource utilization: The percentage of time a resource(CPU, Mem- ory, I/O, Peripheral, Network) is busy

• Throughput: The number of event responses that have been com- pleted over a given interval of time

• Response time: The time elapsed between a request and its reply.

It is a measure of how responsive an application or subsystem is to a client request.

• Database access rates: The number of times database is accessed by web application over a given interval of time

• Scalability: The ability of an application to handle additional workload, without adversely affecting performance, by adding resources such as processor, memory, and storage capacity

4 Types of Performance Testing

Performance tests are broadly divided into following categories [6]

• Performance testing: This type of testing determines or validates the speed, scalability, and/or stability characteristics of the system or application under test. Performance is concerned with achieving response times, throughput, and resource-utilization levels that meet the performance objectives for the project or product.

• Load testing: It focuses on determining or validating performance characteristics of the system or application under test when subjected to workloads and load volumes anticipated during production operations.

• Stress testing: It focuses on determining or validating performance characteristics of the system or application under test when subjected to conditions beyond those anticipated during production operations.

It also tests the performance of the system under stressful conditions, such as limited memory, insufficient disk space, or server failure. These

(6)

tests are designed to determine under what conditions an application will fail, how it will fail, and what indicators can be monitored to warn of an impending failure.

• Capacity testing: It is used to determine how many users and/or transactions a given web application will support and still meet performance goals.

5 Workload Creation

Workload characterization consists of a description of the workload by means of quantitative parameters and functions. The objective is to derive a model able to show, capture, and reproduce the behaviour of the workload and its most important features.

It is sometimes referred to Operation Profile in software engineering liter- ature. Operation Profile is a probability distribution describing the frequency with which selected important operations are exercised. It described how the system has historically been used in the field and thus is likely to be used in the future[3].

5.1 Steps involved in workload characterization

1. Model the software system

2. Collect the data, analyse it, identify the key parameters and probability distribution

5.2 Developing Performance Test Cases

• Identify the software processes that directly influence the overall performance of the system

• For each such process determine the input parameters that will most significantly influence the performance of the system

• Determine realistic values for these parameters by collecting and analysing existing usage data

• If there is no historical data available then estimate the reasonable data based on experience

(7)

• If for a given parameter, the estimated values form a range, then select representative values from within this range that are likely to effect the system performance more. Each selected value will form a separate test case

6 Performance Testing Activities

Performance testing is a complex activity that can not be shaped into “one- size-fits-all” approach. But most of the tests will include the following com- mon activities[6].

1. Identifying the Test Environment: Identify the physical test environment and the production environment as well as the tools and resources available to the test team. The physical environment in- cludes hardware, software, and network configurations. Thorough un- derstanding of the entire test environment at the outset enables more efficient test design and planning and helps to identify testing chal- lenges early in the project. In some situations, this process must be revisited periodically throughout the project’s life cycle.

2. Identifying Performance Acceptance Criteria: Identify the response time, throughput, and resource utilization goals and constraints.

In general, response time is a user concern, throughput is a business concern, and resource utilization is a system concern. Additionally, identify project success criteria that may not be captured by those goals and constraints; for example, using performance tests to evaluate what combination of configuration settings will result in the most desirable performance characteristics.

3. Planning and Designing Tests: Identify key scenarios, determine variability among representative users and how to simulate that variability, define test data, and establish metrics to be collected. Consol- idate this information into one or more models of system usage to be implemented, executed, and analyzed.

4. Configuring the Test Environment: Prepare the test environment, tools, and resources necessary to execute each strategy as features and components become available for test. Ensure that the test environment is instrumented for resource monitoring as necessary.

5. Implementing the Test Design: Develop the performance tests in accordance with the test design.

(8)

6. Executing the Test: Run and monitor your tests. Validate the tests, test data, and results collection. Execute validated tests for analysis while monitoring the test and the test environment.

7. Analysing Results, Reporting and Retesting: Analyze the data both individually and as a cross-functional team. Reprioritize the re- maining tests and re-execute them as needed.

7 Tools

7.1 OpenSTA

OpenSTA(Open System Testing Architecture) is a distributed software testing architecture designed around CORBA. It has the capability of performing scripted HTTP and HTTPS heavy load tests with performance measure- ments. It is available only for Windows OS.[4]

7.2 JMeter

Apache JMeter is a pure Java desktop application designed to load test functional behaviour and measure performance. It was originally designed for testing Web Applications and further expanded to other test functions. It is Platform independent.[5]

Used to

1. test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more)

2. simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types

3. test server/script /object behaviour under heavy concurrent load 4. make a graphical analysis of performance

(9)

7.3 Comparison of LoadRunner and OpenSTA

Item Description LoadRunner OpenSTA

Playback functions

Replaying of the script and script debugging facilities

Extended logging, supports view of parameter values and Server messages. Also view and comparison with recorded version of web page view and client response messages. Debug- ging facilities in script gen- erator, step and break- points.

Similar playback facilities, but no in- tegrated comparison function. The debugging functions are in the controller, includ- ing set break points and single stepping.

Cookie Man- agement

Detection, record- ing and playback of HTTP cookies.

Both tools need additional code to manage JavaScript

HTTP header cookies are managed automatically and can be manipulated manually if required.

HTTP header cookies are managed automatically and can be manipulated manually if required.

IP Spoofing The ability to emulate the behaviour of different IP addresses ac- cessing a system. Par- ticularly useful with Load balancing systems.

Supports IP spoofing in- cluding automated router updates for IP forwarding.

No inbuilt features to enable IP spoofing.

Caching The ability to emulate the caching of pages as performed by a web browser.

Can control browser cache emulation during playback and control setting for each individual vuser.

No specific faculties, although can be em- ulated with simple script code.

User connection speed emulation

The ability to emulate the different network speeds that can be used by real users

Can emulate different network speeds during playback

No inbuilt features to emulate user- connection speed emulation.

Cost Purchase costs of software and licenses, ex- cluding upgrades or support.

Controller costs $22000.

Additional charge for each protocol and monitored resource and for virtual users.

Can be downloaded freely from net.

System requirements

The operating systems required to host the tool.

Linux, Windows, Unix, So- laris

Windows

Hardware requirements

The hardware requirements to host the tool.

Min: Pentium 350, 128MB RAM - Load generators:

Pentium 1GHz, 1MB per VU.

Min : Pentium 200 et 80MB RAM. Pref:

Pentium 500MHz+

128MB+ RAM.

(10)

7.4 GNU/Linux Commands for Performance Testing of a System

• ps - reports a snapshot of the current processes in the system

• top - provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel

• sar - displays performance data for CPU, run queue, disk I/O, paging (swap), memory, CPU interrupts, networking, and more

• uptime - gives the current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, and 15 minutes

• vmstat - reports virtual memory statistics, information about processes, memory, paging, blockIO, traps, and cpu activity

• iostat - reports CPU statistics and input/output statistics for devices and partitions

• free - displays the amount of free and used memory in the system

• slabtop - displays the kernel slab cache information in real time

I have analysed the output of above commands on systems running Applica- tion Server and Database Server. Data is provided by my guide. The data and report on that data is available at[8].

8 Conclusion

Performance testing is crucial for web applications. Passing the functional testing does not guarantee the success of web application since its performance depends on the load. Functionally well tested application may also sometimes fail after deployment. So performance testing is important for the success of a web application. Performance testing enables the developer to identify bottlenecks in the web application before deployment so that he/she can improve its performance before release. Many tools are available to test the performance of web applications. These tools will automatically analyse and report the results of performance tests. Workload characterisation is the main issue in performance testing. The goal of performance testing is not finding bugs, but to remove the bottlenecks from the application and improve the efficiency.

(11)

References

[1] Rukma Prabhu Verlekar, Varsha Apte A Methodology and Tool for Per- formance Analysis of Distributed Server Systems

[2] Thomas Zwanzinger Testing and Improving Web Application Perfor- mance

[3] Xiang Gan Software Performance Testing [4] OpenSTA web page visited on 06-03-2008

http://www.opensta.org/

[5] Apache web page visited on 06-03-2008 http://jakarta.apache.org/jmeter/

[6] Microsoft web page visited on 10-04-2008

http://msdn2.microsoft.com/en-us/library/bb924375.aspx [7] etest associates web page visited on 9-04-2008

http://www.etest-associates.com [8] My home page

http://www.cse.iitb.ac.in/~electionreddy/seminar/performanceTesting/

Web Application s Performance Testing