Project 2
Performance Testing
Group # 1
Group Members :
Tahmina Binte Mnasur
ID # 062292044
Samia Jafrin
ID # 063604044
Performance Testing
Performance testing is the process of determining the speed or effectiveness of a
computer, network, software program or device. This process can involve quantitative tests done in a lab, such as measuring theresponse time or the number of MIPS (millions of instructions per second) at which a system functions. Qualitative attributes such
as reliability,scalability and interoperability may also be evaluated. Performance testing is often done in conjunction with stress testing.
Performance testing can verify that a system meets the specifications claimed by its
manufacturer or vendor. The process can compare two or more devices or programs in terms of parameters such as speed, data transfer rate, bandwidth, throughput, efficiency or reliability. Performance testing can also be used as a diagnostic aid in locating communications bottlenecks. Often a system will work much better if a problem is resolved at a single point or in a single component. For example, even the fastest computer will function poorly on today's Web if the connection occurs at only 40 to 50 Kbps (kilobits per second).
Slow data transfer rate may be inherent in hardware but can also result from software-related problems, such as:
Too many applications running at the same time A corrupted file in a Web browser
A security exploit
Heavy-handed antivirus software
The existence of active malware on the hard disk.
Effective performance testing can quickly identify the nature or location of a software-related performance problem.
Load testing is the process of putting demand on a system or device and measuring its response.
When the load placed on the system is raised beyond normal usage patterns, in order to test the system's response at unusually high or peak loads, it is known as stress testing. The load is usually so great that error conditions are the expected result, although no clear boundary exists when an activity ceases to be a load test and becomes a stress test.
There is little agreement on what the specific goals of load testing are. The term is often used synonymously with software performance testing, reliability testing, and volume testing
The term load testing is used in different ways in the professional software
testing community. Load testing generally refers to the practice of modelling the expected usage of a software program by simulating multiple users accessing the program concurrently. As such, this testing is most relevant for multi-user systems, often one built using a client/server model, such as web servers. However, other types of software systems can also be load tested. For example, a word processor or graphics editor can be forced to read an extremely large document; or a financial package can be forced to generate a report based on several years' worth of data. The most accurate load testing occurs with actual, rather than theoretical, results.
Load and performance testing is to test software intended for a multi-user audience for the desired performance by subjecting it with an equal amount of virtual users and then monitoring the performance under the specified load, usually in a test environment identical to the
production environment, before going live. For example, if a web site with a shopping cart is intended for 100 concurrent users who are doing the following functions:
25 Virtual Users (VUsers) are browsing through the items and logging off
25 VUsers are adding items to the shopping cart and checking out and logging off 25 VUsers are returning items previously purchased and logging off
25 VUsers are just logged in without any activity Some times it is also referred to as Non-Functional Testing
Using various tools available to generate these VUsers, the application is subjected to a 100 VUser load as shown above and its performance is monitored. The pass fail criteria is different for each individual organization and there are no standards on what an acceptable criteria should be, across the board. As a matter of fact, in the list above item 1 the each of the 25 Virtual users could be browsing through unique items that other virtual users will not browse through.
It is a common misconception that these are record and playback tools like regression
testing tools. Load testing tools work at the protocol level whereas most regression testing tools work at the GUI object level. For example, a regression testing tool will simulate a mouse click on an OK button on the browser, but a load testing tool will send out the hypertext that the browser will send after the user clicks the OK button, and again it will send out the hypertext for multiple users each having a unique login ID and password.
Various tools are also available to find out the causes for slow performance which could be in the following areas:
Database
Network
Client side processing Load balancer
With the popularity of the web, especially the acceptance of Business-to-business (B2B)
applications, there are sometimes Service Level Agreements (SLA) involved. B2B applications' popularity has resulted in more and more applications moving to Web Services, where there is exchange of information without a browser interface. An example of a typical service level would be a travel agency inquiring the Airline's Web Service on availability of tickets for a particular flight from Chicago to Dallas, and the Airline would be bound by the SLA to respond within 5 seconds. Often, there are large penalties involved if SLAs are not met.
Popular Tools :
Tool Name Company Name Notes
OpenSTA 'Open System Testing Architecture' Open source web load/stress testing application, licensed under the Gnu GPL. Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.
IBM Rational
Performance Tester IBM
Eclipse based large scale performance testing tool primarily used for executing large volume performance tests to measure system response time for server based applications. Licensed.
JMeter An Apache Jakarta open source project Java desktop application for load testing and performance measurement.
LoadRunner HP Performance testing tool primarily used for executing large numbers of tests (or a large number or virtual users) concurrently. Can be used for unit and integration testing as well. Licensed.
SilkPerformer Micro Focus Performance testing in an open and sharable model which allows realistic load tests for thousands of users running business scenarios
across a broad range of enterprise application environments.
TPTEST
The purpose with TPTEST is to allow users to measure the speed of their Internet connection in a simple way. TPTEST measures the throughput speed to and from various reference servers on the Internet. The use of TPTEST may help increase the consumer/end user knowledge of how Internet services work.
TPTEST was originally developed by the Swedish ICT-commission, then later by the Foundation for Internet Infrastructure (iis.se), the Swedish Consumer Agency
(konsumentverket.se), and the Swedish National Post- and Telecom Agency (pts.se). The latest development has been to separate the platform-independent test method software (the test
engine) from the platform-dependent user interface software in order to make it easier for anyone to write a test client or server that uses the TPTEST testing method. The test engine code is to be regarded as a library module and is released under the LGPL license while the reference
client/server applications is released under the GPL license.
TPTEST 3.0 does the following:
Measures TCP throughput, incoming and outgoing Measures UDP throughput, incoming and outgoing Measures UDP packet loss, incoming and outgoing Measures UDP roundtrip times
Measures UDP out-of-order packet reception
Transmit and Receive
Transmit means that TPTEST sends data packets to the test server and when the
transmission is done the server reports how long it took to receive the packets. TPTEST can then calculate the throughput speed we got when sending data.
Receive means that the test server sends data packets to TPTEST and TPTEST measures the time it takes to receive the packets. When all packets have been received, TPTEST calculates the throughput speed and reports the results.
Test Result
The test result is a close approximation of the highest throughput speed achieved. The best result achieved since the program was started will also be shown.
