4. Implementation and Requirements
5.2 Performance Test
The load balancer web service is deployed on tomcat 6.0 using java 1.6. Soap Ui tool is used to send the request to the load balancer web service, and load strategy is used to carry out the performance test. Various strategies were used to test against the load balancers (original and proposed), to find which load balancing algorithm was behaving better.
We will be running three different load tests to compare our performance test results. The load tests that will be run will be simple, thread and variance mode. The thread level will be adjusted in each run of the load test along with the time too. The Parameters that will be compared will be minimum time, average time, and tps (transactions per second). The main focus of each test will be to see whether the new proposed load balancing algorithm have better tps (transactions per second) rate than original. Higher the tps better will be the algorithm performance.
5.2.1 Simple Strategy
This strategy runs by adding definite amount of thread with a definite delay
between each thread run for server. For example: “for a test to run with test with 5 threads with 5 seconds delay, we initialize thread array to 5, delay to 5000ms. [18]
Below are the results obtained after running tests against algorithm”.
Average
Time (ms) Maximum
Time (ms) TPS
Thread
Count Time
(Sec) Original
Algorithm Proposed
Algorithm Original
Algorithm Proposed
Algorithm Original Algorithm
Proposed Algorithm
5 60 232 231 456 450 5.01 5.05
10 80 234 230 337 350 10.06 10.13
15 100 233 232 309 325 15.01 15.13
20 120 234 231 357 340 20.19 20.3
25 150 235 233 392 355 25.3 25.4
Table 1: Simple Strategy results comparison
The table was plotted with graphs with left axis contains average time (ms) v/s right axis as thread count.
Figure 9: Average Time (Strategy Mode)
By running the tests for Strategy mode against both algorithms, it was seen average response time of proposed algorithm performs better than the algorithm [3].
Response time described here is from client end. Lower response time is one of the features that are most wanted, so by analyzing the result from the graph, it can be said that proposed algorithm behaves better.
Figure 10: Maximum Time (Strategy Mode)
This test is to find the outliers and to find whether the outlier’s time is reduced or not. From the graph, we can analyze that by increasing the thread count; the
maximum time keeps on decreasing, thereby concluding the maximum time reduces with increase in thread when compared proposed v/s original algorithm [3].
Figure 11: TPS (Strategy Mode)
The above test compares tps for the proposed algorithm against original algorithm.
For this test, there is not much change in tps, but if we it in graph proposed
algorithm performs marginally better, and it is linearly increasing with increase in number of threads passed against each algorithm.
The above graphs compares performance of algorithms ran against simple strategy mode. And proposed algorithm has performed better in average time, max time and little better for transaction response time (it can be checked from table entries).
228 230 232 234 236
0 5 10 15 20 25 30
Original Algorithm Proposed Algorithm
0 100 200 300 400 500
0 5 10 15 20 25 30
Original Algorithm Proposed Algorithm
0 20 40
0 5 10 15 20 25 30
Original Algorithm Proposed Algorithm
The next set of performance test ran with thread strategy against both algorithms.
The table was plotted with graphs with left axis contains average time (ms) v/s right axis as thread count. This test increases the amount of thread from one level to another over the run test duration. The main purpose of this strategy is to find at which thread count the test can attain the highest transaction per second. I ran the test against thread count till 50.
Figure 12: Average Time (Thread Mode)
Average time is termed as response time for the client, and from this test it is concluded original algorithm performs marginally better than proposed algorithm, but in real time response time almost looks equal if removed one outlier.
This test is to find the outliers and to find whether the outlier’s time is reduced or not. From the graph, we can analyze that by increasing the thread count; the
maximum time keeps on decreasing, thereby concluding the maximum time reduces with increase in thread when compared with original algorithm [3].
Figure 14: TPS (Thread Mode)
The above test shows proposed algorithm behaves much better than the original algorithm [3]. It can be seen with the increase in number of threads the tps of both algorithms is increasing considerably but the rate at which tps proposed algorithm increased is more than if compared against original algorithm [3]. Higher the tps better is the performance of algorithm, and also with increase in number of threads tps is increasing which also make it more scalable. This tests helps in summarizing that proposed algorithm is much more scalable than original algorithm [3].
The next set of performance test ran with variance strategy against both algorithms.
5.2.3 Variance Strategy (variance 0.5)
Average
Time (ms) Maximum
Time (ms) TPS
Thread
Count Time
(Sec) Original
Algorithm Proposed
Algorithm Original
Algorithm Proposed
Algorithm Original
Algorithm Proposed Algorithm
10 60 230.04 232.04 280 290 36.74 36.42
20 80 236.5 234.91 283 280 92.8 92
30 100 234.14 238.68 391 347 69.2 69.45
40 120 235.2 233.8 398 377 84.13 124.01
50 150 239.2 236 422 388 18.93 29.32
Table 3: Variance Strategy results comparison
The table was plotted with graphs with left axis contains average time (ms) v/s right axis as thread count. The test was performed with a variance of 0.5 for all cases.
0 50 100 150 200
0 10 20 30 40 50 60
Original Algorithm Proposed Algorithm
Figure 15: Average Time (Variance Mode)
By running the tests for variance mode against both algorithms, it was seen average response time performs much better for proposed algorithm than the original algorithm [3] with increase in thread count. Response time described here is from client end. Lower response time is one of the features that are most wanted, so by analyzing the result from the graph, it can be said that proposed algorithm behaves better when the number of thread increases which is a nice feature.
Figure 16: Maximum Times (Variance Mode)
This test is to find the outliers and to find whether the outlier’s time is reduced or not. From the graph, we can analyze that by increasing the thread count; the
maximum time keeps on decreasing, thereby concluding the maximum time reduces with increase in thread when compared with original algorithm [3]. This test show proposed algorithm behaved better than original [3].
Figure 17: TPS (Variance Mode)
“Transactions per second (tps)” for proposed algorithm are much better than the original algorithm. Higher the tps better is the performance. By increasing the number of threads, tps increases at a much higher rate for proposed algorithm.
Performance degrades for both algorithms at thread count of 50.
225 230 235 240
0 10 20 30 40 50 60
Original Algorithm Proposed Algorithm
0 200 400 600
0 10 20 30 40 50 60
Original Algorithm Proposed Algorithm
0 50 100 150
0 10 20 30 40 50 60
Original Algorithm Proposed Algorithm
5.2.4 Testing Using HP Load Runner Tool
I download HP Load runner trial version [21] to test the performance of algorithms in terms of 99%tile of response time and TPS. 99%tile response time will indicate which algorithm performs always better in terms of response time.
99%tile
Response Time (ms) TPS
VUsers Time
(min) Original
Algorithm Proposed
Algorithm Original
Algorithm Proposed Algorithm
20 5 259 259 76.6 79.05
30 5 261 258 115.27 117.1
40 5 268 263 151.3 157.4
50 5 273 265 182.5 192.3
60 5 293 275 202.4 222.6
70 5 269 267 262.1 275.2
Table 4: HP Load Runner Results Comparison
The table was plotted with graphs with left axis contains average time (ms) v/s right axis as VUsers.
Figure 18: 99%tile Response Time (HP LoadRunner)
The graph was plotted for the 99%tile response time for both algorithms, and it is evident from the graph that proposed algorithm performed much better as
compared. 99% of the time, it responded faster by 5-‐8 ms. Lower the response time, better is the performance considered.
250 260 270 280 290 300
0 20 40 60 80
Original Algorithm Proposed Algorithm
Figure 19: TPS (HP LoadRunner)
The graph indicates that tps rate is better for proposed algorithm against original algorithm [3]. The better tps rate means algorithm can process more transactions per second, and is scalable.
From the above two graphs, its concluded that proposed algorithm has better tps, and better 99% tile response time, which states proposed algorithm is performing better compared to original algorithm for the same load and criteria.
From all the performance tests (simple, thread, variance and HP Load Runner) conducted against both original algorithm [3] and proposed algorithm [3,4], it is seen that proposed algorithm has always been better in respect to increase in tps, lower response time, and low maximum value compared to original algorithm. All the three tests (average time, 99%tile, maximum time, and tps) performed better for proposed algorithm.
0 50 100 150 200 250 300
0 20 40 60 80
Original Algorithm Proposed Algorithm