UKCMG TEC 2010 Track P - Performance

(1)

UKCMG TEC 2010

Track P - Performance

Session P7

Low Cost Performance Testing

Paul Seaton-Smith

UKCMG TEC 2010 Track P - Performance Low Cost Performance Testing

Agenda

• Performance Testing Costs

• Test Environment

• Scripting and Testing Resources • Automated Test Tools

• Installation • Test Plan Hierarchy • Thread Groups

• Recording a Browser Session • Script Parameterisation

• Regular Expression Extractor • Response Assertions • Assertion Results • CSV Data Set Config • View Results Tree • Simple Data Writer • View Results in a Table • Summary

(2)

Performance Testing Costs

• Test Environment

• System Under Test • Load Generators

• Scripting and Testing Resources • Load Testing Tools

Test Environment

• Test environment does not need to be production size • Industry benchmarks

• Single full size server of each type • Test against production servers prior to delivery • Test against virtualised test environments

• Resource controls to guarantee CPU/Memory • Use virtualised servers as load injectors

• If collecting response times at least one physical load injector is required

(3)

Scripting and Testing Resources

• Ease of scripting

• Access to support and online help • Availability of skilled testers • Time to train testers

Automated Test Tools

• Automated test tools are required to conduct performance testing, including:

• Stress Testing • Load Testing • Volume Testing • Soak Testing

• However, commercial automated test tools can be expensive • This remainder of this presentation will discuss how Apache JMeter

may be used to conduct performance testing and avoid costs associated with Automated Testing Tools

(4)

Installation

• JMeter is a free load testing tool from Apache that runs on Java Platform Standard Edition

• Download from

http://jakarta.apache.org/site/downloads/downloads_jmeter.cgi

• Management of test data, scripts and test results can be difficult

Test Plan Hierarchy

• Test Plan

• HTTP Cookie Manager • HTTP Request Defaults • Thread Group

• HTTP Request

• Regular Expression Extractor • Response Assertion

• Assertion Results • Simple Data Writer • View Results Tree • Workbench

(5)

Thread Groups

• Each Thread Group should represent one use case or saga

• JMeter runs Thread Groups in parallel but runs the contents of each

Thread Group sequentially

• Each Thread Group can be controlled separately by • Number of Threads

• Ramp up period (time period to introduce all threads) • Loop count

• Scheduler

• 10 threads with ramp up period of 60 seconds means introduce 1 thread every 6 seconds

• 100 threads with ramp up period of 1 second does not take 100 seconds to ramp up

P7-9

(6)

Test Ramp Up

• Ramp up refers to increasing the number of threads during a test • Introducing all the threads at the same time is unrealistic and

could cause the service to fail or suffer poor performance • We can determine best-case performance on a lightly loaded

system

• Allow some time for the system to achieve a steady-state • Allow measurement of multiple points on the response time

curve

• Additional control could be achieved through multiple identical

Thread Groups with different start and end times

P7-11

Thread Groups - Scheduler

• Startup delayoverrides Start Time

• Durationoverrides End Time

(7)

HTTP Request Defaults

• Use HTTP Request Defaultsto specify a default URL for HTTP Requests

P7-13

Recording a Browser Session

• Add a Thread Group to the Test Plan

• Add a Recording Controller to the Thread Group

• Add a HTTP Proxy Server to the WorkBench

• Edit your LAN Connection to use a proxy on localhost:8080 • Start the HTTP Proxy Server

• Open a browser and navigate through the use case • Browser actions are recorded in the Recording Controller

• Stop the HTTP Proxy Server

• Delete the HTTP Proxy Server

(8)

Internet Explorer and Chrome LAN Settings

P7-15

Firefox Connection Settings Settings

(9)

Recording a Browser Session

P7-17

Recording a Browser Session

• It is possible to use URL Patterns to filter out images, CSS, etc. using URL Match Strings and URL Patterns to include or exclude certain content

• It is typically safer to record all requests and responses and then manually delete unwanted calls from the script afterwards

(10)

Remove Unwanted HTTP Requests

• Typically unwanted • .js files • .css files • .gif images • .jpg/.jpeg images • Typically wanted • .asp files • .php files • .htm files • .html files P7-19 © Capacitas 2002-2010

Remove Unwanted HTTP Requests

(11)

HTTP Header Manager

• Automatically added during the recording process • Should not need editing

P7-21

Demonstration

• Record the following use case into capacitas.jmx

1. Navigate to http://www.capacitas.co.uk 2. Enter ‘virtualisation’ into the search box and

click search

(12)

http://www.capacitas.co.uk/results.php

P7-23

Demonstration

• Delete unwanted .css, .js, .gif, .jpg, .jpeg etc. HTTP Requests

• Move important HTTP Requests (.asp) under Thread Group

• Rename Thread Groupto capacitas.co.uk Search

(13)

Demonstration

P7-25

Script Parameterisation

• Many web sites manage the relationship between client and web server using session data

• Need to record the session data to send back to the web server • Important to check that the expected response is returned • May also wish to request different information in the scripts

(14)

P7-27

Regular Expression Extractor

• Use regular expressions to reference any data • (.+)

• Any brackets in the string (or )must be preceded by a \

(15)

Jakarta Regular Expressions

• JMeter uses is the Jakarta regular expression package

• For example, searching AaaaAaaaAfor A(.+)A matches aaaAaaa rather than aaa

• The solution is to say A([^A]+)A, which will match any number of NOT A

•http://jakarta.apache.org/regexp/applet.html is an applet that lets you test Jakarta regular expressions

P7-29

Response Assertions

• Response Assertions are used to check whether the response contains the expected result

• Pattern matching can be used to ensure that specific text either is, or is not displayed

• This is required to ensure the expected web page was returned when making a request

• Be wary when using Response Assertions to check that error messages are not returned, as they may be returned to a browser but not rendered on the screen

(16)

Response Assertions

P7-31

Assertion Results

• Allows you to check visually if the Response Assertions passed or failed

• If the Response Assertions passed it will just write a line with the name of the transaction

• It is possible to only write failures and ignore successes

(17)

Assertion Results

P7-33

HTTP Cookie Manager

• Nearly all web testing should use cookie support • Add an HTTP Cookie Manager to each Thread Group

• This will ensure that each thread gets its own cookies, shared across all HTTP Requests

(18)

HTTP Cookie Manager

P7-35

CSV Data Set Config

• Fully qualify the file location • Specify a .csv extension

(19)

CSV Data Set Config

• searchterm in searches.csv

P7-37

(20)

View Results Tree

• View Results Tree allows the user to see the response from the service during the tests

• Typically easiest to look at the Render HTML view under the

Response Data tab

P7-39

View Results Tree

:

Response Data

(21)

Simple Data Writer

• Used to save the results of the test

• Fully qualify the file location and specify a .csv extension

• D:\JMeter Tests\Capacitas Website Testing\Results\CAPACITAS_03.csv

P7-41

Saving Results

• To ensure results are written in .csv format rather than XML uncheck all of the XML options

(22)

Saving Results

• To write in a date format convenient for Excel uncomment the following line in jmeter.properties

#JMeter.save.saveservice.timestamp_format=yyyy/MM/dd HH:mm:ss.SSSS in JMeter.properties in jmeter.properties

P7-43

View Results in a Table

• Response times for HTTP Requests are collected by default

(23)

Summary

• Still need a test environment

• System under test • Load injectors

• Still require skilled resources for scripting and testing • Test management is difficult

• Jmeter is free

• Scripting language is simple • Skills are in the marketplace

UKCMG TEC 2010 Track P - Performance

UKCMG TEC 2010

Track P - Performance

Session P7

Low Cost Performance Testing

Paul Seaton-Smith

Agenda

Performance Testing Costs

Test Environment

Scripting and Testing Resources

Automated Test Tools

Installation

Test Plan Hierarchy

Thread Groups

Test Ramp Up

Thread Groups - Scheduler

HTTP Request Defaults

Recording a Browser Session

Internet Explorer and Chrome LAN Settings

Firefox Connection Settings Settings

Recording a Browser Session

Recording a Browser Session

Remove Unwanted HTTP Requests

Remove Unwanted HTTP Requests

HTTP Header Manager

Demonstration

http://www.capacitas.co.uk/results.php

Demonstration

Demonstration

Script Parameterisation

Regular Expression Extractor

Jakarta Regular Expressions

Response Assertions

Response Assertions

Assertion Results

Assertion Results

HTTP Cookie Manager

HTTP Cookie Manager

CSV Data Set Config

CSV Data Set Config

View Results Tree

View Results Tree

:

Response Data

Simple Data Writer

Saving Results

Saving Results

View Results in a Table

Summary

Thank You