Performance test execution is a complex job. Experience and gut feeling play a major role during test execution. Performance test execution is not like functional testing execution.
In functional test execution, setting pass/fail criteria is easy. Defects are easy to detect and analyze. In performance testing execution, setting performance parameters and monitoring are complex jobs. Here setting pass/fail criteria is difficult because the defect is the bottleneck for better performance and identifying the bottleneck takes a long time.
Complete test execution must be completed before attempting to identify the bottleneck.
Test execution is completed only when all exit criteria are passed. However, a meticu-lously planned test execution yields more returns than an unplanned test execution. The following guidelines provide direction to plan a proper test execution:
•
Define explicitly the testing team, roles and responsibilities starting from planning to execution phase.•
Estimate test execution effort before the beginning of the test.•
Configure the tools properly and check version compatibility.•
Check all additional tools used in testing for version and compatibility.•
Resources from all the different groups (such as network specialist, database specialist, domain specialist, etc.) must be synchronized.•
Run PT more than once with different levels of background activity (noise) to get the true picture.•
Conduct PT with production sized databases for a real life simulation.•
Do not conduct PT in a development environment which leads to inaccurate results.•
Monitor performance counters continuously and take action suitably.•
Monitor test duration continuously and take action suitably.•
Avoid external interference during test execution.•
Monitor the test execution by probing the client to check whether the system under test behaves properly or not.•
Log the results with timings for further analysis.The above guidelines provide direction for a tester to plan and conduct the test properly.
However, each organization may adopt their own methods to test the system effectively.
Finally, though test execution is a complex activity, a methodical approach simplifies the job more comprehensively.
Summary
Test execution is an important activity in the PT life cycle and needs the same attention as that of other activities. Normally, testers do not plan the activities of test execution because this results in poor utilization of resources and time. To avoid such problems, it is suggested that meticulous planning is required during test execution.
For any test to be successful, entry and exit criteria must be defined properly. These criteria must be followed strictly during execution. Next, an elaboration test must be conducted to find in advance the various issues that may creep in during the final test execution. The Elaboration test provides indepth information about the complexity of the system under test. Based on the complexity of the system, one can plan the test execution properly to optimize resources and time.
The self satisfaction test provides the rich experience of professionals who have addressed similar projects previously. Their experience on planning, scheduling tests, and analyzing results always helps to complete the performance testing effectively. It also helps in assessing the possible bottlenecks during execution.
PT must be carried out multiple times as one test may not give proper results. To run multiple tests, proper test execution guidelines must be followed. Finally, meticulous planning and skilled resources ensure the success of the performance tests.
References
A slightly skeptical view on scripting languages. (2004). Retrieved November 22, 2004, from http://www.softpanorama.org/Scripting/index.shtml
Alcott, T. (2003). One or many applications per application server? Retrieved June 19, 2003, from http://www-106.ibm.com/developerworks/Websphere/techjournal/
0211_alcott/alcott.html
From inception to elaboration. (2003). The Metropolitan State College of Denver.
Retrieved August 11, 2003, from http://clem.mscd.edu/~hasze/csi3280/notes/
Chapter8.html
Incremental testing. (2004). Retrieved June 12, 2004, from http://homepages.nildram.co.uk/
~worrelli/cont343.htm
Joung, P. (2004). General network performance testing methodology. Retrieved Septem-ber 21, 2004, from http://www.spirentcom.com/documents/1065.pdf
Khanine, D. (2004). Tuning up ADO.NET connection pooling in ASP.NET applications.
Retrieved May 6, 2004, from http://www.15seconds.com/issue/040830.htm Kim, S. (2004). Safe session tracking. Retrieved November 2, 2004, from http://
www.sdmagazine.com/documents/s=818/sdm0103h/
Netcraft Ltd. (2004). UDP denial of services. Retrieved November 21, 2004, from http:/
/www.netcraft.com/presentations/interop/dos.html
Test execution. (2004). Retrieved June 17, 2004, from http://www.donald-firesmith.com/
Components/WorkUnits/Tasks/Testing/TestExecution.html
Additional Sources
Adirake, P. (2004, August). Research on software configuration management in middleware systems. Retrieved January 17, 2005, from http://www.jaist.ac.jp/
library/thesis/is-master-2005/paper/p-adirak/abstract.pdf
Applying eXtreme Programming Techniques to Automated Testing. (2004). Retrieved January 17, 2005, from http://www.bjss.co.uk/press/BJSS_Applying_XP_
To_Automated_Testing.pdf
Department of Defense (2003, November). Testing guide. Retrieved November 22, 2004, from http://www.eitoolkit.com/tools/implementation/build/testing_guide.doc Mar, W. (2004). Performance testing and planning. Retrieved November 21, 2004, from
http://www.wilsonmar.com/perftest.htm
Marick, B. (2004). Classic testing mistakes. Retrieved June 19, 2004, from http://
www.testing.com/writings/classic/mistakes.pdf
Pauli, K. (2003). Pattern your way to automated regression testing. Retrieved January 7, 2003, from http://www.javaworld.com/javaworld/jw-09-2001/jw-0921-test.html Test guidelines. (n.d.). Retrieved January 17, 2005, from http://www.w3.org/QA/WG/
2003/10/TestGL-20031020.html