Results Contains a summarization of Performance Test activities and final results.
Dependency: Following is an example of a dependency list:
ID Dependency Description Impact
1 Pre-Production Build
Completed The pre-production environment should contain a stable and tested build, the emphasis behind this is to minimize any impact on re-work of performance test scripts.
H
2 Pre-Production Access
Provided Access to relevant databases, application is provided to Vendor. Resources would be predominantly used to record the necessary test scripts identified for performance testing.
H
3 Performance Test Tools
Deployed/Installed All the relevant tools are deployed, installed,
and smoke tested. H
4 Test Data requirements
agreed A set of appropriate and valid test data is
loaded. H
5 Early Visibility of build
to Pre-Prod Early indication of the build that is expected to be deployed to pre-production. M 6 Cycle 1 Test Data Loaded Test data loaded and sanity tested prior to
commencement of cycle 1 testing. H 7 Drop X Deployed to
Pre-Prod Deployed build successfully completed in pre-production environment, sanity tested and meets the Entry criteria for Cycle 1 test execution.
M
Now that we have taken a detailed look at the Performance testing planning, we should now see how we implement the script creation and implementation using SoapUI.
Performance testing using SoapUI
We have already seen in Chapter 1, Introduction to SOA Testing, how to create a performance testing script using SoapUI so let's move on further from there.
We have four load generation strategies in SoapUI:
• Simple
• Thread
[ 59 ]
• Variance
• Burst
Let's go through each of them in detail.
Simple strategy: This strategy addresses three types of performance test:
• Baseline test
• Load test
• SOAK test
The simple strategy generates a specific amount of virtual user load on the application with a ramp up time between each thread to give breathing space to the application under test.
Following screenshot shows the configuration window where the setting for the simple strategy is set.
The following window is common for each strategy type but works differently for each load generation strategy:
Let's take a detailed look at the previous screenshot:
• Threads: To specify the amount of virtual user load on the application in the preceding example we have used five threads which means five virtual users.
• Test Delay: The delays you want to keep between each thread, for example 10 threads and 10 sec delay will result in 1 sec wait after each thread.
[ 60 ]
• Random: If you want to vary the wait time, this feature is used. For example, if you put a random of .5 with a delay of 20 seconds then you will have a random delay between threads of 5 to 20 seconds.
• Limit: This is for time duration of the Test.
Variance: This technique is used usually for two types of performance testing:
• Recovery testing
• Stress testing
This strategy varies the number of threads over time. Set the Interval to the desired value and the Variance to how many the number of threads should decrease and increase.
Let's have a look at the following example.
If we start with Threads of 20, and an Interval of 60 and Variance of 0.8, the number of threads will increase from 20 to 36 within the first 15 seconds, then decrease back to 20 and continue down to four threads after 45 seconds, and finally go back up to the initial value after 60 seconds.
[ 61 ]
The Variance strategy window has the following options and looks like the following:
Following are the configuration details:
• Threads: The amount of VUser load
• Interval: The time after which the threads would start changing up
• Variance: The variance defines how much the change would be after the specified set of interval
• Thread Strategy: This strategy can be used for the following:
° Identifying the threshold point ° Analyzing a particular bottleneck
° Identifying the load after which functional bugs start to occur
[ 62 ]
Following is the screenshot which shows the configuration window of the thread strategy:
Following are the configuration values of the window:
• Threads: Specifies the number of users to run this test case
• Start Threads: The number of threads to start the test
• End Threads: The number of threads to end the test
• Limit: The time till which the test will run in seconds
• Burst Strategy: This strategy can be used for the following:
° Recovery testing ° Stress testing
° Bottleneck identification
The burst strategy simulates a scenario where you suddenly have high traffic for some duration on an application. Let's see an example:
In a holiday season, some sites like Amazon give a user more response time than usual because the number of users will be more, similarly a university website at the time of graduation will work much slower than usual as the number of users will peak when the results are announced.
These types of tests are simulated by Burst Strategy. Let's have a look at its configuration window:
[ 63 ]
• Threads: Specifies the number of threads or VUsers
• Burst Delay: Delay between each burst
• Burst Duration: The duration of the test
We have now gone through all four types of strategy for load generation using SoapUI. For more advance performance testing we can integrate SoapUI with LoadUI, we will see that in the following sections of this chapter.
Now we know how to generate load on any SOA application, let's see how to validate the response with the help of assertions.