Get t i n g St art ed, Ru n n i n g a Load Test , an d
An al y zi n g Resul t s
Technical support is available from our Technical Support Hotline or via our FrontLine Support Web site.
Technical Support Hotline:
1-800-538-7822
FrontLine Support Web Site:
http://frontline.compuware.com
This document and the product referenced in it are subject to the following legends:
Access is limited to authorized users. Use of this product is subject to the terms and conditions of the user
’
s
License Agreement with Compuware Corporation.
©
1998-2008 Compuware Corporation. All rights reserved. Unpublished - rights reserved under the Copyright
Laws of the United States.
U.S. GOVERNMENT RIGHTS
Use, duplication, or disclosure by the U.S. Government is subject to restrictions as set forth in Compuware
Corporation license agreement and as provided in DFARS 227.7202-1(a) and 227.7202-3(a) (1995), DFARS
252.227-7013(c)(1)(ii)(OCT 1988), FAR 12.212(a) (1995), FAR 52.227-19, or FAR 52.227-14 (ALT III), as
applicable. Compuware Corporation.
This product contains confidential information and trade secrets of Compuware Corporation. Use, disclosure, or
reproduction is prohibited without the prior express written permission of Compuware Corporation.
Compuware, ActiveAnalysis, ActiveData, Interval, QACenter, QADirector, QALoad, QARun, Reconcile,
TestPartner, TrackRecord, and WebCheck are trademarks or registered trademarks of Compuware Corporation.
Acrobat
®Reader copyright
©
1987-2002 Adobe Systems Incorporated. All rights reserved. Adobe, Acrobat, and
Acrobat Reader are trademarks of Adobe Systems Incorporated.
This product includes software developed by the Apache Software Foundation (http:// www.apache.org/)
This product includes software developed by Teodor Danciu (http://jasperreports.sourceforge.net)
This product includes software developed by the University of California, Berkeley and its contributors.
All other company or product names are trademarks of their respective owners.
US Patent Nos.: Not Applicable.
Doc.
CWQLIX570
iii
Tabl e Of Con t en t s
QALoad On lin e Help ... 1
W elcom e t o QALoad ... 2
Script Developm en t W orkben ch ... 2
Con duct or ... 2
Player ... 2
An alyze ... 2
How t o Use t h e On lin e Help ... 3
Get t in g St art ed ... 4
Th e Load Test in g Process... 4
Devel opin g Script s... 5
Validat in g Script s... 5
Run n in g a Load Test an d An alyzin g Result s ... 6
Run n in g a Load Test ... 6
M on it orin g a Load Test ... 6
St oppin g a Load Test ... 7
An alyzin g Test Result s... 7
Addin g Post -t est Com m en t s... 8
1
Get t in g St art ed, Run n in g a Load Test , an d An alyzin g Result s
2
W el co m e t o QA Lo ad
W it h QALoad, you can si m ulat e t h e load gen erat ed by t h ousan ds of users wit h out t h e expen se of act ual en d users or t h eir equi pm en t . QALoad en ables you t o quickly devel op t est scri pt s, con t rol t h e con dit ion s for t h e t est , creat e t h e virt ual users t h at sim ulat e t h e load, in it iat e an d m on it or t h e t est , an d report t h e result s.
Th e followin g com pon en t s of QALoad con t rol differen t aspect s of t h e load t est in g process:
Script Development Workbench
Devel op your scri pt s. Capt ure session s an d con vert t h e t ran sact ion s in t o C++ or Java-based script . For m ore in form at ion , see Overview of t h e Script Developm en t W orkben ch.
Conduct or
Con t rol session act ivit y. In it iat e an d m on it or t h e t est . For m ore in form at ion , see Overvi ew of t h e QALoad Con duct or.
Player
Sim ulat e user act ivit y based on com m an ds given from t h e Con duct or. For m ore in form at ion , see Overview of t h e QALoad Player.
Analyze
View sum m ary report dat a an d creat e ot h er st at ist ical report s from t h e t est . For m ore in form at ion , see Overview of QALoad An alyze.
Before you begin usin g QALoad, you sh ould fam iliarize yourself wit h load t est in g con cept s. For m ore in form at ion , see Th e Load Test in g Process.
QALoad On lin e Help
3
Ho w t o Use t h e On l i n e Hel p
QALoad's on lin e h elp in cl udes a Table of Con t en t s (TOC), an in dex, an d a search in put field. W h en you click on a t opic t it le in t h e TOC, t h e t opic appears in t h e righ t -h an d pan e of t h e on lin e h elp. Topics in cluded in t h e Table of Con t en t s are syn ch ron ized wit h t h e TOC. Th is m ean s t h at wh en you click a t opic lin k in t h e righ t -h an d pan e, t h e t opic t it le is h igh ligh t ed in t h e TOC in th e left -h an d pan e.
Use t h e In dex t ab t o search for a specific referen ce by t ypin g a keyword in t h e in dex en t ry field. Use t h e Search t ab t o search for associat ed t opics. Type a keyword in t h e search en t ry field. All t opics t h at con t ain t h e keyword are list ed.
W h en you are workin g in QALoad, click t h e H el p but t on on an in dividual di alog box or press F1 t o view a descript ion of each field i n t h e dialog box. For t h ese t opics an d ot h ers t h at do n ot appear in t h e Table of Con t en t s, t h e syn ch ron izat ion fun ct ion wit h t h e TOC is disabled.
Get t in g St art ed, Run n in g a Load Test , an d An alyzin g Result s
4
Get t i n g St ar t ed
The Load Test ing Process
You begin t h e t est in g process by det erm in in g t h e t ypes of applicat ion t ran sact ion s you wan t t o em ulat e. You t h en develop t h ese t ran sact ion s in t o QALoad script s by creat in g t h e sam e t ypes of request s t h at your applicat ion s in voke on t h e server. Each t ran sact ion becom es it s own scri pt . Th e QALoad Script
Devel opm en t W orkben ch let s you easily creat e full-fun ct ion script s.
W h en you plan your t est , you n eed t o decide wh ich t ran sact ion s t o run , t h e n um ber of sim ulat ed clien t s t h at will run each t ran sact ion script , an d t h e frequen cy at wh ich each script will run . W h en you run th e t est on a workst at ion wit h t h e QALoad Player com pon en t , you can specify t ran sact ion rat es as well as fixed an d ran dom delays t o bet t er em ulat e real-syst em act ivit y. QALoad con siders t h ese fact ors a t est scen ari o an d st ores t h em in a session ID file.
W h ile a t est is run n in g, t h e t est operat or can dyn am ically view overall run t im es as well as in dividual t ran sact ion perform an ce. QALoad's Con duct or com pon en t collect s t h is dat a for an alysis at t h e con clusion of t h e t est .
Aft er execut in g t h e t est , sum m ary report s sh ow t h e respon se t im es t h at t h e em ulat ed clien t s experien ced durin g t h e t est . In dividual an d global ch eckpoin t s let you view an d iden t ify specific areas of syst em
perform an ce. You can export all t est out put dat a t o spreadsh eet an d st at ist ical packages for furt h er an al ysis or use QALoad's An alyze com pon en t t o creat e presen t at ion -qualit y report s an d graph s.
As sh own in t h e followin g im age, a t ypical load t est set up con sist s of a QALoad Con duct or, on e or m ore QALoad Players, an d t h e syst em un der t est .
QALoad Conduct or
QALoad's Con duct or con t rols all t est in g act ivit y such as set t in g up t h e session descript ion files, in it iat in g an d m on it orin g t h e t est , an d report in g an d an alyzin g t est result s.
QALoad Player
A QALoad Player creat es virt ual users t h at sim ulat e m ult iple clien t s sen din g m iddleware calls t o a server un der t est . In a t ypical t est set up, on e or m ore QALoad Player workst at ion s run un der an y W in dows 32-bit plat form (W in dows XP or 2000) or UNIX. For large t est s (t h ousan ds of sim ulat ed clien t s), you can con n ect m ult iple Players t o QALoad's Con duct or. Th e Con duct or an d Players com m un icat e usin g TCP/ IP.
Th e h ardware an d soft ware capabilit ies of t h e Player m ach in e are t h e on ly fact ors t h at lim it t h e capacit y of an in dividual QALoad Pl ayer. Th e m axim um n um ber of virt ual users per Player m ach in e is depen den t on t h e syst em un der t est , t h e ch aract erist ics of t h e scri pt , an d t h e t est scen ario. You can specify h ow m an y t h readed- an d process-based virt ual users t o assign on a m ach in e in t h e M ach i n es win dow of t h e Opt i on s dialog box in t h e Con duct or. Th e Con duct or calcul at es h ow m an y virt ual users will be act ive per 64 M B of RAM , based on t h e values you provide in t h ese fields. For m ore in form at ion , see M ach in es Dialog Box.
Syst em Under Test
Th e servers you t est are t ypically product ion syst em s or a duplicat e of a product ion syst em t h at is set up at a t est facilit y. If you perform an y kin d of syst em select ion or perform an ce st ress t est , t h e syst em un der t est m ust use t h e sam e h ardware an d soft ware (in cludin g curren t version s) as t h e product ion en viron m en t . Com puware h as foun d t h at even subt le ch an ges h ave profoun d effect s on perform an ce.
QALoad On lin e Help
5
Developing Script s
Use t h e QALoad Script Developm en t W orkben ch t o develop t est script s. It con t ain s facilit ies for capt uri n g session s, con vert in g capt ured session s in t o script s, an d m odifyin g an d com pilin g script s. On ce you com pile your script , use QALoad's Con duct or an d Player com pon en t s t o t est your syst em .
Record Faci l i t y: QALoad's Record facilit y, wh ich you can access t h rough t h e QALoad Script Devel opm en t W orkben ch , records t h e t ran sact ion s t h at your t erm in al, browser, or clien t m akes. It st ores t h ese
t ran sact ion s in a capt ure file.
Con vert Faci l i t y: QALoad's Con vert facilit y, wh ich you can access t h rough t h e QALoad Script
Devel opm en t W orkben ch , con vert s capt ure files in t o script s. It gen erat es a on e-t o-on e correspon den ce of t ran sact ion s from t h e origin al session t o your QALoad scri pt .
Vi sual Navi gat or: Visual Navigat or for W W W is QALoad's easy-t o-use visual in t erface t o QALoad's powerful script developm en t t ools. Visual Navigat or for W W W ren ders your recorded C++ -based t ran sact ion in a t ri-pan ed, browser-like en viron m en t sim ilar t o popular visual ly-orien t ed developm en t t ools, wit h icon s represen t in g all t h e elem en t s of your script .
Validat ing Script s
Before you con duct an act ual load t est , you sh ould i n dividually validat e t h e script (s) you plan t o use by run n in g a sim ple load t est . If t h e script run s t o t h e en d wit h out an y errors an d run s m ult iple t im es wit h out errors, it is valid t o use in a load t est .
If t h e script abort s on an error, debug t h e script an d run it t h rough a sim ple t est again . You can validat e script s from t h e QALoad Script Developm en t W orkben ch , t h e QALoad Player, or t h e QALoad Con duct or: Validat in g a Script in t h e Script Developm en t W orkben ch
Validat in g a Script in t h e Player Validat in g a Script in t h e Con duct or
Get t in g St art ed, Run n in g a Load Test , an d An alyzin g Result s
6
Ru n n i n g a Lo ad Test an d A n al yzi n g Resu l t s
Running a Load Test
Aft er validat in g a scri pt , it is safe t o run a load t est wit h t h at script .
To st ar t a l oad t est :
In t h e Con duct or, click t h e Run but t on on t h e con figurat ion an d set up t oolbar, or from t h e Act i on s m en u, ch oose Run . W h ile a t est is run n in g, th e Con duct or's In t erface ch an ges t o provide you wit h real-t im e t est opt ion s. For m ore in form at ion , see Run t im e W in dow In t erface.
Not e: While any window on the desktop is re-sizing or re-positioning, all Windows applications pause. Do not click and hold on a window caption or border for extended periods during a load test because it delays message handling and may have an impact on test results.
W h ile a load t est is run n in g, t h e Con duct or’s t oolbar ch an ges from t h e Con figurat ion an d Set up Toolbar t o t h e Run t im e Toolbar. Th e Run t im e Toolbar but t on s let you con t rol t h e t est an d access det ailed
in form at ion about t h e t est wh ile it is run n in g. For m ore in form at ion , see M on it orin g a Load Test. Th is gives det ailed in form at ion about wh at t o expect from t h e QALoad Con duct or wh ile a t est is run n in g — in cludin g descript ion s of t h e Run t im e Toolbar but t on s
Running a Series of Test s
You can also run a seri es of t est s — a bat ch t est . A bat ch t est com prises m ult iple session ID files. W h en you run a bat ch t est , t h e session files are execut ed sequen t ially un t il all of t h em are execut ed. Th e Con duct or en ables you t o run m ult iple bat ch t est s wit h out operat or in t erven t ion . For m ore in form at ion , see Run n in g a Bat ch Test.
M onit oring a Load Test
W h en you st art a t est , t h e QALoad Con duct or’s in t erface ch an ges t o an in t eract ive t est con t rol st at ion , referred t o as t h e Run t im e W in dow. Th e Run t im e W in dow displays in form at ion about t h e script s, m ach in es, an d virt ual users t h at are execut in g t h e load t est . From t h e Run t im e W in dow, you can observe t h e progress of in dividual script s an d Player m ach in es, creat e an d view real-t im e graph s, an d st art or suspen d scri pt s an d Players from a run n in g t est t o bet t er sim ulat e t h e un predi ct abilit y of real users. For m ore in form at ion , see Run t im e W in dow In t erface.
In addit ion t o t h e t est dat a sh own by default on t h e Run t im e W in dow, you can access det ailed t est in form at ion usin g t h e QALoad Con duct or’s Run t im e Toolbar But t on s. You can :
! View statistics for a single virtual user
! View the activities of a virtual user in a browser-like window (WWW only) ! Step to the next request (WWW only)
! View the current datapool record
! Display the script running on a single virtual user
! Display messages sent from a Player workstation to the QALoad Conductor ! Display statistics about Conductor/Player communication
! Show/hide the Runtime Tree or Runtime Control Panel ! Exit, abort, or quit the test
QALoad On lin e Help
7
St opping a Load Test
A load t est is com plet e wh en all virt ual users exit . A virt ual user aut om at ically exit s wh en on e of t h e followin g occurs:
! A script encounters an EXIT command. ! A script completes its transaction loop.
! A QALoad function fails and Abort on Error is set in Error Handling
To st op a l oad t est :
Click t h e Ex i t Al l Vi rt ual User s but t on or click t h e Qui t Curren t Test but t on . Th e Virt ual User icon ch an ges t o an d t h e m essage, "Session abort ed by User", displays.
Adding Post -t est Comment s
If you select ed t h e Di spl ay post t est com m en t s d i al og opt ion wh en you con figured t h e Con duct or, t h e Post Test Com m en t s win dow open s wh en you click t h e Qui t Curren t Test but t on . Type an y com m en t s, wh ich are saved t o t h e t est ’s Sum m ary Report t h at you can view in QALoad An alyze.
Analyzing Test Result s
Aft er you set up a load t est an d run it , you can an al yze t h e result s from t h e t est usin g QALoad An alyze. An im port an t part of t h e l oad t est in g process is viewin g an d st udyin g t h e result s of a t est . You can view load t est result s n ot on ly on a m ach in e wh ere QALoad is in st alled, but on an y m ach in e wit h a W eb browser.
W h en run n in g a t est usin g a part icular session ID fi le (set up in t h e Con duct or), each Player com piles a local t im in g file t h at com prises a series of t im in g records for each ch eckpoin t of each scri pt run on t h at Player. Each t im in g record in t h e file con sist s of a respon se t im e/ el apsed t im e pair of values. Th ese val ues specify t h e am oun t of t im e it t ook a cert ain ch eckpoin t t o fin ish (respon se t im e) at a specific t im e in t h e t est (elapsed t im e).
At t h e en d of a t est , Player t im in g files are sen t t o t h e Con duct or an d are m erged in t o a sin gle t im in g file, called t h e Prim ary t im in g file, for an alysis. If you set up in t egrat ion wit h Com puware’s ServerVan t age product , t h e Con duct or collect s t im in g dat a from t h e ServerVan t age cen t ral con sole an d also m erges t h at dat a in t o t h e t im in g file.
W h en you open a t im in g file, QALoad gen erat es a workin g folder t h at con t ain s all support in g files, report s, an d im ages gen erat ed from t h at t im in g file. Th e folder is locat ed in t h e \Program
Files\Compuware\QALoad\TimingFiles\xxx.xml.source direct ory, wh ere <xxx> is t h e n am e of t h e
t im in g file.
Cust om Report s
Use QALoad An alyze t o creat e cust om report s usin g XM L (Ext en sible M arkup Lan guage), XSL (Ext en sible St yle Lan guage), an d HTM (Hypert ext M arkup) files. QALoad An alyze provides a set of files in .h t m , .xm l, an d .xsl form at s in addit ion t o t h e .t im file. QALoad An alyze aut om at ically gen erat es a X M L (*.xm l), X SL (*.xsl), an d HTM (*.h t m ) file wh en you open a t im in g file.
Get t in g St art ed, Run n in g a Load Test , an d An alyzin g Result s
8
Pre-defined Report s
QALoad An alyze provides pre-defin ed report s so you can receive im m ediat e load t est result s wit h out m an ipulat in g an y dat a. Al l t h e files n ecessary for t h ese report s are locat ed in t h e direct ory \Program
Files\Compuware\QALoad\Timing Files\Reports.
Not e: The pre-defined reports that are available depend on the data collected in the timing file. This is determined by the QALoad Conductor option you select when you run the load test.
Adding Post -t est Comment s
By set t in g t h e appropri at e opt ion s wh en you con figure t h e Con duct or, you can add com m en t s t o a com plet ed t est . Th e com m en t s appear in t h e t est 's Sum m ary Report in QALoad An alyze.
To con f i g u r e t h e Co n d u ct or f or ad d i n g p o st -t est com m en t s: 1. Select Tools>Options. The Options dialog box appears.
2. In the Dialog section of the Conductor Sessions page , select Display post test comments dialog. 3. Click OK. The Conductor is now configured so that you can add comments when a test completes.
To ad d p ost -t est com m en t s:
1. In the Test Completed dialog box, click Exit Test. OR
In t h e Run t im e t oolbar, cl ick t h e Qui t curren t t est but t on . Th e Post Test Com m en t s dialog box displays.
2. Type any comments in the dialog box, then click OK. Your comments are saved in the Post Test Comments field of the Summary Report in
9
