Web Application Testing. Web Performance Testing

Web Application Testing

Objectives of Performance Testing

●

Evaluate runtime compliance to performance 

requirements

●

Check different properties such as 

● throughput (bits/sec, packets/sec) ● response time (mean time for getting a response after an  input) ● memory utilization ● input/output rates ● etc

Types of Performance Testing

●

Stress testing ­ focus on system behavior at, near 

or beyond overloading conditions 

● push system to failure ● emphasis near specified load, volume boundaries ● checks for graceful failures, non­abrupt performance  degradation ●

Load Testing ­ verifies handling of  a particular 

load while maintaining acceptable response times

Web Performance Testing

● Performance is key to the success of a web application

● According to a survey by Zona Research (2001) ● Most users click away after 8 seconds

● Potential cost for poor web performance cost $25 billion US dollar revenue lost (in the USA)

Web Site Load

(Number of concurrent users) Web Site Response Time Good (0 - 8 sec.) Borderline (9 – 15 sec.) Unacceptable (15 – 30 sec.) Virtually Offline (> 30 sec.)

The 8 Second Barrier Crash!

Source: Presentation by Lloyd Taylor, VP Technology & Operations, Keynote Systems

Web Performance Testing

●

Test Objectives

● How many users can the system handle while maintaining a “reasonable” response time ?

Web Performance Testing

●

Help determine:

● The maximum capacity that can be handled – while keeping an acceptable performance

● Whether hardware and/or software performance tuning is needed

Web Performance Testing Phases

1.

Planning phase

2.

Testing phase

Performance Testing Process

Planning phase

● Define objectives, deliverables, expectations ● Gather system and testing requirements – environment and resources – workload (peek, low) – acceptable response time ● Select performance metrics to collect ● e.g. Transactions per second (TPS), Hits per second, Concurrent connections,  Throughput, etc ● Identify tests to run and decide when to run them – Scenarios/Transaction to run ● Decide on a tool/application service provider option  – to generate loads  ● Write test plan, design user­scenarios, create test scripts

Performance Testing Process

Testing Phase

●

Testing phase

● generate test data ● set­up test bed  – SUT, test environment performance monitors ● run tests  ● collect result data

Performance Testing Process

Analysis Phase

● Analyze results to locate source of problems – software problems – hardware problems ● Change system to optimize performance – software optimization – hardware optimization ● Design additional tests (if  test objective not meet)

Tools for performance testing of

Web Applications

●

Necessary to simulate loads

● Large number of concurrent users

●

Record scenario/transaction scripts

● For regression testing

●

Report results in a way that help analysis

●

Example of tools:

● Apache JMeter

● IBM Rational Performance Tester ● HP Loadrunner

JMeter

● GUI desktop application designed to load test functional behavior and measure performance

● command line execution also possible

● Originally designed for testing Web Applications but has since expanded to other test functions.

● Open Source

● Has a rich graphical interface ● Built in Java

● Part of Apache Jakarta project ● Can be distributed

JMeter

● Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types.

● http://jakarta.apache.org/jmeter/ ● current version 2.3.4

Features of JMeter

● Graphical Analysis / Exporting Test Results ● Remote Distributed Execution

● If you want to generate load using multiple test servers. You can run

multiple server components of JMeter remotely. And you can control it by a single JMeter GUI to gather results.

– http://jakarta.apache.org/jmeter/usermanual/remote-test.html

● Highly Extensible

● Custom Additions (Write your own samplers / listeners) ● PlugIns

What Can You Do With It?

●

JMeter lets you set up test plans that simulate

logging into a web site, filling out forms,

clicking buttons, links, etc.

●

You can simulate the number of users doing

Test Plan

●

Describes a serie of steps to be executed by

JMeter

●

Consists of one or more Thread Groups, logic

controllers, samplers, listeners, timers,

assertions, and configuration elements.

●

Represented as an Ordered Tree

● The whole tree or subtrees can be saved in XML

Scoping Rules

● Some elements are primarily ordered (e.g. controllers and samplers).

● Other elements are hierarchical. An Assertion, for example, is hierarchical in the test tree. If its parent is a request, then it is applied to that request. If its parent is a Controller, then it affects all requests that are descendants of that Controller.

Elements of a Test Plan

●

Thread Group

● controls the number of threads JMeter will use to execute the test

● used to simulate concurrent connections to the server application (number of concurrent users) ● a ramp-up period may be defined to specify how

long it takes to start the full number of threads in the group

● It is also possible to define a number of times a test is to be executed

Elements of a Test Plan

●

Controllers

● drive the processing of a test

● include Samplers and Logical Controllers

● Samplers tell JMeter to send requests to a server and wait for a response

Elements of a Test Plan

● Samplers

● Tell JMeter to send requests to a server and wait for a response

● By default, processing is done according the order of appearance in

the test plan

– logical controllers used to alter default ● Typical samplers:

– FTP Request – HTTP Request – JDBC Request

– Java object request – LDAP Request

– SOAP/XML-RPC Request – WebService (SOAP) Request

Elements of a Test Plan

Elements of a Test Plan

●

L

ogic controllers let customize the logic that JMeter

uses to decide when to send requests.

● Can change the order of requests coming from their child elements.

● Can modify the requests themselves, cause JMeter to repeat requests, etc.

Elements of a Test Plan

● Examples of logic controllers

● Simple Logic Controller ● Loop Controller

● Only Once Controller ● Interleave Controller ● Random Controller ● If Controller ● While Controller ● Switch Controller ● ForEach Controller ● ....

Elements of a Test Plan

● Listeners

● provide access to the information gathered by JMeter while

running a test plan

● can direct the data to a file for later use

● Sample listeners

– Graph Results listener

– Assertion Results visualizer – View Results Tree

– View Results Table – Aggregate report

– Summary report

Elements of a Test Plan

● Timer

● introduces a delay between each request that a thread

makes

● can be used to account for “think time” ● Assertion

● check responses obtained from server

● "test" that application is returning the expected results ● can be specified using Perl style regular expressions

Elements of a Test Plan

● Configuration Element

● can add to or modify requests (e.g. by providing variable

definitions)

● Pre-Processor

● executes some action prior to a Sampler Request being made

● can be used to modify settings, update variables

● Post-Processor

● executes some action after a Sampler Request has been made

● can be used to process the response data, often to extract

Execution order

●

Timers

●

Pre-Processors

●

Sampler

●

Post-Processors

●

Assertions

●

Listeners