Extending Open Source solution for Performance testing of Web (http\https)
application
ABSTRACT
In this research paper we examine the need for load testing and highlight the shortcomings of open source load testing solutions, vis-à-vis their commercial counterparts and present our own approach to design plug-ins over open source tools that can bring them a step closer to COTS. Our proposed solution enhances the capabilities of Open Source testing tools by providing an in-depth performance monitoring of web, application and database servers. This helps in determining bottleneck of the application and enables the user to proactively correct potential problems before they become critical. We also propose here, the implementation of IP spoofing and bandwidth simulation, which would be the add-on features of this tool.
KEYWORDS:
performance, monitoring, load testing, analysis, OpenSTA, WAN emulation, bottleneck, Analyzer, Profiler, Graphs, Overlays, Load Runner1. INTRODUCTION
For any web application, the end to end response time for a user is a critical issue. As the number of users accessing the same web application increases, the performance of the application decreases due to slowdown [7]. Load testing
refers to checking the performance of a system under multi-users workload. It is a very critical part for any web application before it goes live. There are many open source load testing solutions available today, but they do not match up completely to their commercial counterparts. Commercial tools on the other hand prove to be very expensive option for small scale industries. The challenges of successfully conducting an online business are as follows:
1) high user volumes
2) low response times for web requests 3) the overall reliability of the service
Monitoring the performance requires an efficient analysis of the various parameters associated with performance measures. Our major concern lies with the determination of the bottleneck in the complete n-tier application that comprises of a client, a network and different servers involved (web, application, database). Thus, a detailed analysis for every performance parameter from all the components involved under Load condition is required for diagnosing the complete application being monitored.
2. EXISTING TECHNOLOGIES
2.1 CURRENT STATE OF ART 2.1.1 Commercial tools
The most popular commercial tools available for Load testing and diagnostics are
a) Mercury LoadRunner
b) IBM Rational Performance tester c) IBM Rational Robot
d) Borland Silk Performer
2.1.2 Popular Open source Testing Solutions
For the purpose of this research paper, we list two of the most popular tools for performance testing of HTTP\HTTPS applications
a) Apache JMeter b) OpenSTA
2.2 SHORTCOMINGS
Open source tools lag from their commercial counterparts in the following
a) Effective Integrated Monitoring and analysis capability
b) User network speed simulation c) IP Spoofing capability
3. PROPOSED APPROACH OF THE
SOLUTION:
Figure3.1.1: Integrated Monitoring Model
First, OPENSTA tool is used to generate virtual user load on the machines (servers) to be monitored. For the collection of the performance parameters on different server machines like IIS, SQL, Weblogic, ORACLE etc., we design lightweight listening components (LC) and deploy them on different Servers (Web, Application, Database) with different OS environments.
The key requirements of the LC(s) are:
a) They should be lightweight and should not consume large memory.
b) They should continuously gather
Configurable Data Points
(parameters) on the server machine by
polling the logs or using the APIs of the server and the OS at a configurable polling interval and relay it back to a central request component (CC). c) They should run in daemon mode. The central request component (CC) which acts as a Performance Data Collector Server
connects to the installed LC(s), the network and the load generator tool (OPENSTA), and stores the data for every synchronized pulse into a database. Dashboard, a term which defines the
console at CC side, keeps on fetching the data from the database and helps the user to visualize the real time variations in the performance parameters and locate the areas that have been the cause for erroneous or unusual results. The key requirements for the DASHBOARD are:
a) Display of simultaneous graphs with feature of graph overlays.
b) Maintenance of a PROFILER which allows user to reconfigure the saved sessions. Informing user of any abnormality in the system and giving him the option to execute necessary measure. c) In real environment, the values lying out
of bounds are taken care off by the
ANALYZER, which then diagnoses the
fed data (input) and tries to point out the cause of the anomaly. We will have a facility at the Dashboard console that can send system alerts through emails etc. Since the LC(s) have to be deployed on different platforms, we decided to use a platform independent language like PERL for the designing of Listening components as well as Central Component. For communication and transfer of data values between the CC and LC, we have used Socket Programming in Perl. For designing of the Graphical User Interface and the Dashboard, we have used VB.net.
3.1.1 DASHBOARD
3.1.1.1 USER INTERFACE:
The User Interface (UI), designed using VB.NET, provides a console on which the user
could design and set a session for monitoring and can see the results in form of REAL TIME GRAPHS. These real time graphs are obtained corresponding to each machine participating in the monitoring session as well as from the machines which are used for generating the load using OPENSTA. The UI runs the scripts placed at backend prepared using PERL scripting language. Following are the steps of interaction between CC and LC:
1. The CC broadcasts START request, and the LC(s) respond back by sending their system details.
2. The CC requests the selected LC(s) to send their list of parameters.
3. The CC sends the configured list of parameters to be monitored to each LC, including the configured test to be run on OPENSTA machines.
4. CC broadcast a “START MONITORING” message to all selected LC(s) at the same instant. The user can end the session whenever he wants to by sending a “STOP” broadcast.
The REAL TIME GRAPHS are displayed by plotting the parameters of the machines being monitored along with the parameters provided by the OPENSTA machines such as virtual users and response time.
It provides us with an ‘integrated monitoring console’ (at CC) to view correlation of different
parameters of different machines The key features of real time graphs are:
Add Graph: displays the real time variation of a
single parameter from any of the machines being monitored in the session.
Add Parameter: Overlays the parameter to see
the relative variation and correlation of two or more parameter values with time.
Remove Parameter: Removes the display of
any undesired parameter from the running graphs.
Delete Graph: Deletes a graph from the graph
window.
Changing Legend Properties: Changes the
legend properties (color, width, style, scale).
The graphs are scaled on a scale of 100, and the data is interpolated for missing values.
Figure 3.1.1.1: The UI with the real time graphs
3.1.1.2 SAVED PROFILER:
After the completion of every monitoring session, the user can save the current session.
A dump of the current session in .sql format is then created at user specified location.
If at any time, user has to create a monitoring session with the same configurations, then instead of making an entirely new session and configuring it again, the user can avail this facility of saved profiler, by recreating a new session with modifications to an old session.
3.1.1.3 ANALYZER:
The ANALYZER analyzes the complete monitoring session to come out with the results of the performance measures and generate a report. The saved session dump is loaded back to the database and graphs for the monitored machines are displayed. Along with the features as in Real time graphs, we have also incorporated the following additional features:
Show Data: displays the accurate value of the
parameters at any instant of time (also has the option to select the RANGE OF VALUES).
Exporting Data to Excel: exports the data directly
to the excel sheet.
Making User Defined Graphs: provides an option
to see a particular range of data at any instant of time (by specifying the RANGE as well as INTERVAL for the data on the X-axis).
Report Generation: prepares a report and exports
the graphs to Microsoft word.
The parameters on a graph have been scaled with respect to the maximum valued parameter. The data is also interpolated before being plotted on the graph surface.
Figure 3.1.1.2: The Analyzer to analyze a saved session
3.2 BANDWIDTH SIMULATION MODEL
Till now open source load testing tools do the monitoring without considering the bandwidth availability and latency at the client’s side. The performance of applications largely depends on the availability of bandwidth and latency at client’s side. For instance, different clients might be working at different bandwidths (33kbps, 56 kbps, 1Mbps so on…). The server might be sending data at a particular bandwidth say 100Mbps, and might have a response time of sending a page equal to,
say, 26 seconds. However, due to different availability of bandwidth the time taken in displaying the page will be different for different clients.
Considering this fact, it would be necessary for real-time to simulate a network bandwidth against every task-group in OPENSTA. For network simulation we have readily available tool such as WAN Emulator tool [8].
To achieve this real time monitoring on network performance measures we need to map the load generated from OPENSTA to the network traffic on WAN emulator. For this we can design an interface to be placed between OPENSTA and WAN emulator that would call OPENSTA APIs to start the load test and simultaneously call the APIs of WAN Emulator to generate appropriate traffic so that required bandwidth on the network can be achieved.
Figure 3.2: The WAN emulation model
3.3IP SPOOFING
A major problem for load testing team is that any genuine attempt to load the server with web-requests can be perceived as DOS attack by firewall and blocked thus failing to load the server. To bypass this problem, load generation tool should be able to send requests spoofing their source IPs so that these requests can bypass defense mechanism and can actually reach server. Open source load testing tools such as OpenSTA do not have an inbuilt mechanism to achieve IP spoofing.
4. COMPARATIVE STUDY
The chart below shows comparative study of our tool with the leading commercial tool, LOAD RUNNER and the most popular open source tool, OPENSTA.
Item LoadRunner OpenSTA Our Tool
HTTP/HTTPS Protocols √ √ √
Other Protocols √
×
×
Commercial Tool √
×
×
Monitoring on Windows based servers √ √ √
Monitoring on other platform(Linux) based servers √
×
√Profiler (i.e. reconfiguring previous sessions) √
×
√Analysis of simple charts and graphs √ √ √
Overlaying graphs capability (Sophisticated & large range of charts and graphs with overlay facilities)
√
×
√IP Spoofing √
×
√Effective Integrated Monitoring and analysis capability √
×
√ Extensibility(the user can modify the code according to his needs)
×
√ √
5. CONCLUSION:
In a short time, e-business has proven to be a viable business model for dotcoms and brick-and mortars alike. With the number of Internet users growing exponentially, it is therefore critical for these businesses to prepare themselves for high user volumes.
The approach mentioned in this paper, is a step towards enhancing the existing Open-source load testing technologies, which can be used by small industries which cannot afford high-priced commercial tools.
However, the various demerits of OPENSTA at load generation and script recording level have not been combated. Enhancements can be made by implementing protocols other than HTTP/ HTTPs.
