Web Application Testing
Objectives of Performance Testing
●Evaluate runtime compliance to performance
requirements
●Check different properties such as
● throughput (bits/sec, packets/sec) ● response time (mean time for getting a response after an input) ● memory utilization ● input/output rates ● etcTypes of Performance Testing
●Stress testing focus on system behavior at, near
or beyond overloading conditions
● push system to failure ● emphasis near specified load, volume boundaries ● checks for graceful failures, nonabrupt performance degradation ●Load Testing verifies handling of a particular
load while maintaining acceptable response times
Web Performance Testing
● Performance is key to the success of a web application
● According to a survey by Zona Research (2001) ● Most users click away after 8 seconds
● Potential cost for poor web performance cost $25 billion US dollar revenue lost (in the USA)
Web Site Load
(Number of concurrent users) Web Site Response Time Good (0 - 8 sec.) Borderline (9 – 15 sec.) Unacceptable (15 – 30 sec.) Virtually Offline (> 30 sec.)
The 8 Second Barrier Crash!
Source: Presentation by Lloyd Taylor, VP Technology & Operations, Keynote Systems
Web Performance Testing
●
Test Objectives
● How many users can the system handle while maintaining a “reasonable” response time ?
Web Performance Testing
●
Help determine:
● The maximum capacity that can be handled – while keeping an acceptable performance
● Whether hardware and/or software performance tuning is needed
Web Performance Testing Phases
1.
Planning phase
2.
Testing phase
Performance Testing Process
Planning phase
● Define objectives, deliverables, expectations ● Gather system and testing requirements – environment and resources – workload (peek, low) – acceptable response time ● Select performance metrics to collect ● e.g. Transactions per second (TPS), Hits per second, Concurrent connections, Throughput, etc ● Identify tests to run and decide when to run them – Scenarios/Transaction to run ● Decide on a tool/application service provider option – to generate loads ● Write test plan, design userscenarios, create test scriptsPerformance Testing Process
Testing Phase
●Testing phase
● generate test data ● setup test bed – SUT, test environment performance monitors ● run tests ● collect result dataPerformance Testing Process
Analysis Phase
● Analyze results to locate source of problems – software problems – hardware problems ● Change system to optimize performance – software optimization – hardware optimization ● Design additional tests (if test objective not meet)Tools for performance testing of
Web Applications
●
Necessary to simulate loads
● Large number of concurrent users
●
Record scenario/transaction scripts
● For regression testing
●
Report results in a way that help analysis
●Example of tools:
● Apache JMeter
● IBM Rational Performance Tester ● HP Loadrunner
JMeter
● GUI desktop application designed to load test functional behavior and measure performance
● command line execution also possible
● Originally designed for testing Web Applications but has since expanded to other test functions.
● Open Source
● Has a rich graphical interface ● Built in Java
● Part of Apache Jakarta project ● Can be distributed
JMeter
● Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types.
● http://jakarta.apache.org/jmeter/ ● current version 2.3.4
Features of JMeter
● Graphical Analysis / Exporting Test Results ● Remote Distributed Execution
● If you want to generate load using multiple test servers. You can run
multiple server components of JMeter remotely. And you can control it by a single JMeter GUI to gather results.
– http://jakarta.apache.org/jmeter/usermanual/remote-test.html
● Highly Extensible
● Custom Additions (Write your own samplers / listeners) ● PlugIns
What Can You Do With It?
●
JMeter lets you set up test plans that simulate
logging into a web site, filling out forms,
clicking buttons, links, etc.
●
You can simulate the number of users doing
Test Plan
●
Describes a serie of steps to be executed by
JMeter
●
Consists of one or more Thread Groups, logic
controllers, samplers, listeners, timers,
assertions, and configuration elements.
●
Represented as an Ordered Tree
● The whole tree or subtrees can be saved in XML
Scoping Rules
● Some elements are primarily ordered (e.g. controllers and samplers).
● Other elements are hierarchical. An Assertion, for example, is hierarchical in the test tree. If its parent is a request, then it is applied to that request. If its parent is a Controller, then it affects all requests that are descendants of that Controller.
Elements of a Test Plan
●
Thread Group
● controls the number of threads JMeter will use to execute the test
● used to simulate concurrent connections to the server application (number of concurrent users) ● a ramp-up period may be defined to specify how
long it takes to start the full number of threads in the group
● It is also possible to define a number of times a test is to be executed
Elements of a Test Plan
●
Controllers
● drive the processing of a test
● include Samplers and Logical Controllers
● Samplers tell JMeter to send requests to a server and wait for a response
Elements of a Test Plan
● Samplers
● Tell JMeter to send requests to a server and wait for a response
● By default, processing is done according the order of appearance in
the test plan
– logical controllers used to alter default ● Typical samplers:
– FTP Request – HTTP Request – JDBC Request
– Java object request – LDAP Request
– SOAP/XML-RPC Request – WebService (SOAP) Request
Elements of a Test Plan
Elements of a Test Plan
●
L
ogic controllers let customize the logic that JMeteruses to decide when to send requests.
● Can change the order of requests coming from their child elements.
● Can modify the requests themselves, cause JMeter to repeat requests, etc.
Elements of a Test Plan
● Examples of logic controllers
● Simple Logic Controller ● Loop Controller
● Only Once Controller ● Interleave Controller ● Random Controller ● If Controller ● While Controller ● Switch Controller ● ForEach Controller ● ....
Elements of a Test Plan
● Listeners
● provide access to the information gathered by JMeter while
running a test plan
● can direct the data to a file for later use
● Sample listeners
– Graph Results listener
– Assertion Results visualizer – View Results Tree
– View Results Table – Aggregate report
– Summary report
Elements of a Test Plan
● Timer
● introduces a delay between each request that a thread
makes
● can be used to account for “think time” ● Assertion
● check responses obtained from server
● "test" that application is returning the expected results ● can be specified using Perl style regular expressions
Elements of a Test Plan
● Configuration Element
● can add to or modify requests (e.g. by providing variable
definitions)
● Pre-Processor
● executes some action prior to a Sampler Request being made
● can be used to modify settings, update variables
● Post-Processor
● executes some action after a Sampler Request has been made
● can be used to process the response data, often to extract
