ST-1 HTML test

The first static test consists in return a 168 bytes HTML only web page, where it is shown the phrase: “This is a webserver test page”.

Before starting with the charts, it is important to highlight three well different results that were obtained, being the first one about do not enabling keep-alive. Using HTTP 1.0 protocol affects the overall performance, as it lowers the server CPU load but, is limited by how speedier the server opens new connections. Afterwards, it was enabled keep-alive functionality, using HTTP 1.1 protocol, which increased the CPU load in order to maintain connections opened, but also increased the performance. The importance of this test is to appreciate the performance variation between both cases, and the relative performance difference within web servers.

The third result that is important to mention in this test case, is the increase of the performance when tuning event-driven web servers (e.g., Lighttpd or Nginx). The default configuration for those servers is only one process with a single thread; but, it can be increased to take advantage of the multi process architecture of the server. It will be shown different charts varying the number of processes, in order to find which the best-suited configuration is. Finally, it will

be compared the best performer web servers of this section in order to get the best web server for this test.

Fig. 4.1: HTML chart no keep-alive test results

Fig. 4.1 shows the HTML chart between the different tested web servers. In this case, average performance is between 8,000 and 11,000 requests per second, only Hiawatha and Yaws being far behind the others. Apache Worker (multi process multithread) and Event (event-based) configurations perform in the same way. Only Apache’s Prefork (multi process single thread) configuration performs better, about 3,000 requests per second more, in the best case. When reaching 500 simultaneous clients the performance of the three Apache configurations is similar, about 8,000 requests per second.

Lighttpd and Nginx web servers have a very similar performance, both reaching 10,000 requests per second with 100 clients at the same time. Performance difference starts to appear at 500 simultaneous clients, Nginx performance decreases quicker than Lighttpd. In this case, both Lighttpd and Nginx were tested with only one process running. On the other hand, multithread web servers such as Cherokee and Tomcat have a similar performance behavior, having Cherokee slightly superior performance in front of Tomcat. Both have the same performance decrease when reaching above 250 simultaneous clients, and with 1,000 clients the performance is almost the same, around 8,000 requests per second.

Mongrel2 gets better performance than Hiawatha, which is a very good notice as it is a newer web server using a very different architecture approach. Yaws gets the worst performance rate, less than 4,000 requests per second. This can be caused by the fact that without keep-alive, Yaws only uses one processor, which limits its performance.

Fig. 4.2: HTML chart keep-alive test results

Enabling keep-alive feature, HTTP 1.1 protocol, increases the performance of the system, as it shares connections between requests. Fig. 4.2 shows the result increase that, in some cases, reaches up to 30,000 requests per second. Apache has a very similar performance for the three different configurations. Only at the beginning of the test, between 1 and 100 simultaneous clients, performance difference is around 5,000 requests per second. When reaching 100 simultaneous clients, the results of the three configurations are around 15,000 requests per second. The performance increase by using keep-alive for Apache against not using it is more than 5,000 requests per second, which represents an increase of more than 50%.

Event–driven web servers got almost the same result, reaching 17,000 requests per second for Nginx and 16,500 for Lighttpd, in the best case. Nginx seemed to perform more stable when increasing simultaneous clients; while Lighttpd decreases its performance near 15,000 requests per second. Speaking about multithread web servers, Cherokee and Tomcat, both obtained performance around 30,000 and 27,000 requests per second, respectively. Cherokee result increased almost 270% by enabling keep-alive feature, which is a large increase. It is possible to see that Tomcat reaches its maximum performance range slowly than Cherokee, but maintains it as simultaneous clients increase.

Hiawatha only reaches 10,000 requests per second, although it doubles its previous result, but it is still 50% slower than Apache, Lighttpd and Nginx. Mongrel 2 and Yaws had a very low performance, which is not a good outlook for them. On one hand, Yaws uses the two available processors, but falls far ahead from best performing web servers. On the other hand, Mongrel2 does not take profit of the whole CPU capabilities, which leaves it as the worst performing web server of this test.

Event-driven based web servers can also be configured to take advantage of multi process systems by enabling the ability to fork its main process. As the aim of the test is to show the best performing web server for this scenario, it is

configured Lighttpd and Nginx with different number of active processes. Taking into account previous results, performance without keep-alive feature enabled is almost the same (see Appendix A.2.1) for any configuration. The only difference is that with more than one Lighttpd worker, the top performance is reached earlier (11,000 requests per second in the best case), but performance is not as stable as before when increasing simultaneous clients.

Fig. 4.3: Lighttpd HTML keep-alive chart

But, when enabling keep-alive we get very different results. Fig. 4.3 shows the performance obtained during this test. It is possible to see a big increase when configuring Lighttpd to fork in more than one process. Performance increase is about 100%, increasing from 15,000 requests per second with one process up to 30,000 requests per second with more than one process. Lighttpd does not recommend forking as it could break some of the kernel modules of the web server, but for increasing the performance it is advisable. Also, they recommend using twice the processes as the number of processors that the machine has, in this case it would be 4 processes because the machine is a multicore system.

The other event-driven web server is Nginx. The results were expected to be similar to the previous one, Lighttpd. No keep-alive HTML test, results in a top performance of 10,000 requests per second for any configuration (see Appendix A.2.1). As in the case of Lighttpd, there is no much difference between configurations as they behave very similar.

Fig. 4.4 shows the scalability of Nginx in different worker situations enabling keep-alive feature. It is possible to see that configuring the web server by default, with only one process, has a measurable impact on the performance. Meanwhile, using more than one process increases performance up to 28,000 requests per second, in the best case. Forking the main process to more than 2 workers results in a very similar performance, so there is no special performance gain when using even more processes. Nginx official documentation recommends forking the main process to be equal than the number of available processors of the system. In this case, as it is shown in Fig. 4.4, the best result occurs when having 2 Nginx workers at the same time.

Fig. 4.4: Nginx HTML keep-alive chart