During the work on the master thesis the author of the thesis learned the theory connected with ESB, and performance testing, to formalized the scenarios and added additional scenarios.
The most complicated part of the work for the author was connected with scenarios implementation for JBoss Fuse and their automation.
8. Results During the work on the thesis the author learned all the technologies used for implementation of scenarios. Apart from implementing the scenario, the author tweaked the implementation to make it optimal from performance perspective. This part required deep knowledge and understanding in the technologies used for implementation. In some cases scenarios were rerun in different configuration to find out which has the best performance.
The automation using SmartFrog also wasn’t easy for the author.
She learned how SmartFrog works in detail. It has its uses its own lan-guage to define the interaction between components. During the work on automation, the author implemented SmartFrog component to start and stop JBoss Fuse in Java, and wrote other components for automa-tion using Groovy language. SwitchYard runs on JBoss Applicaautoma-tion Server, and there was already SmartFrog component for SwitchYard implemented in Java.
During the work on the scenario Method GET of a RESTful web service the following issue was encountered. The test runs for 1 minute in the warm up phase, reaches the performance of 28,342 iterations/sec, and then the client throws the following exception:
java.net.NoRouteToHostException: Cannot assign requested address
Apart from that on the client host 28,223 connections are in the TIME WAIT status. On the server host there are maximum five waiting connections, and no exception. This situation happens because entire ephemeral port range was used on the client machine. Ephemeral port is commonly used by protocols UDP and TCP as a port on the client side in client–server communication to the specified port on the server side.
The exception was thrown because there were no more available free ephemeral ports on the client host when PerfCake wanted to create new connections with the server. To solve this problem some network tuning is required, including making more ephemeral ports available, and en-abling fast recycling TIME-WAIT sockets. Unfortunately, allowance to tune the network was not granted, as it was used by other applications.
Besides, in the earlier version of JBoss Fuse there was an issue con-nected with low performance test result for scenario SOAP implemen-tation of a web service using JAX-WS secured by WS-Security. The performance of that scenario was 144 iterations/sec, and after fixing the issue it increased to 12,487 iteration/sec. The issue was fond due to
8. Results the work on this thesis, and it is described in the issue tracking system3.
3. https://issues.jboss.org/browse/ENTESB-1473
9 Conclusion
The aim of this thesis was to compare performance of JBoss integration platform implementations.
During the work on the thesis two integration platforms were con-sidered: JBoss Fuse and SwitchYard. In the thesis the notions service oriented architecture and enterprise service bus are described.
From the perspective of testing, methods of performance testing, and tests automation were characterized. Based on that knowledge the author contributed to design of performance scenarios for comparison JBoss Fuse and SwitchYard. She has formalized and defined scenarios which were already implemented for SwitchYard, and also introduced new scenarios:
• HTTP exposed Services implementing Splitter-Aggregator pat-tern
• Method GET of a RESTful web service implementation
• Method POST of a RESTful web service implementation All fifteen scenarios were implemented for JBoss Fuse using Perf-Cake and tweaked to be optimal from the performance point of view.
The performance was measured using a tool for measuring performance named PerfCake.
Performance test execution was automated in distributed environ-ment using Jenkins and SmartFrog, a tool used to manage test run in Client-Server environment. After multiple automated test execution, the author of the thesis made sure that results are stable and collected them. The technologies used for implementation of performance tests for JBoss Fuse, included Apache Camel, Apache ActiveMQ, Apache Karaf, Apache CXF.
The analysis of the results is persented in the Chapter 8. It can be concluded that for most of the scenarios the results are comparable. In general, it appears that performance of integration framework is better in JBoss Fuse, as most of the scenarios connected with routing work faster for it. But the scenarios connected with web services, especially proxying the third party web service run faster in SwitchYard.
9. Conclusion Two issues were encountered during the work on the thesis. They are described in Section 8.2. One issue deals with performance of JBoss Fuse scenario JAX-WS secured by WS-Security, which is already fixed by developers of JBoss Fuse.
The automated tests developed during this thesis can be used to measure performance of different releases of JBoss Fuse in order to make sure that there was no drop in performance in some release.
Bibliography
[1] Jonathan Anstey. Open source integration with apache camel and how fuse ide can help. DZone, 2011. Available at http://java.
dzone.com/articles/open-source-integration-apache.
[2] The Apache Software Foundation. Apache Karaf Manual. Avail-able at http://karaf.apache.org/manual/latest/index.html.
[3] Naveen Balani and Rajeev Hathi. Apache CXF Web Service De-velopment. Packt Publishing, 2009.
[4] Brian Goetz. Java theory and practice: Dynamic compilation and performance measurement. DeveloperWorks, 2004. Available at http://www.ibm.com/developerworks/library/j-jtp12214/.
[5] Steven Haines. Pro Java EE 5 Performance Management and Op-timization. Apress, 1 edition, May 2006.
[6] HP Labs. SmartFrog Documentation. Available at http://www.
smartfrog.org/display/sf/SmartFrog+Home.
[7] Claus Ibsen and Jonathan Anstey. Camel in Action. Manning Publications Co., Stamford, 2011.
[8] JBoss Community. JBoss Application Server Documentation.
Available at http://jbossas.jboss.org/.
[9] JBoss Community. SwitchYard Documentation. Available at https://docs.jboss.org/author/display/SWITCHYARD11/
Home.
[10] Eric Jendrock, Ricardo Cervera-Navarro, Ian Evans, Kim Haase, and William Markito. The Java EE 7 Tutorial. Ora-cle. Available at http://docs.oraOra-cle.com/javaee/7/tutorial/
doc/index.html.
[11] Jaideep Khanduja. What is a testing environment for software testing? TechTarget, 2008. Available at http://www.infoq.com/
articles/ESB-Integration.
9. Conclusion [12] Ian Molyneaux. The Art of Application Performance Testing.
O’Reilly, 2009.
[13] Jakob Nielsen. Response times: The 3 important limits. Nielsen Norman Group, 1993. Available at http://www.infoq.com/
articles/ESB-Integration.
[14] The OPEN Group. The SOA Source Book. Available at http:
//www.opengroup.org/soa/source-book/soa/soa.htm.
[15] Opensourcetesting.org. Performance test tools. Available at http:
//www.opensourcetesting.org/performance.php.
[16] OSGi Alliance. The OSGI Architecture. Available at http://www.
osgi.org/Technology/WhatIsOSGi.
[17] Red Hat. Fuse Documentation. Available at https:
//access.redhat.com/site/documentation/en-US/Red_
Hat_JBoss_Fuse/6.1/.
[18] John Ferguson Smart. Jenkins: The Definitive Guide. O’Reilly Media, Inc., 2011.
[19] Bruce Snyder, Dejan Bosanac, and Rob Davies. ActiveMQ in Ac-tion. Manning Publications Co., Greenwich, CT, USA, 2011.
[20] Kai W¨ahner. Choosing the right esb for your integration needs.
InfoQ, 2013. Available at http://www.infoq.com/articles/
ESB-Integration.
10 Appendix
Figure 10.1: HTTP exposed custom service
10. Appendix
Figure 10.2: SOAP exposed custom service
Figure 10.3: JMS exposed custom service
10. Appendix
Figure 10.4: HTTP exposed content based routing using XPath
Figure 10.5: HTTP exposed content based routing using RegEx
10. Appendix
Figure 10.6: HTTP exposed content based routing using Rules
Figure 10.7: HTTP exposed Services implementing Scatter-Gather pat-tern
10. Appendix
Figure 10.8: HTTP exposed Services implementing Splitter-Aggregator pattern
Figure 10.9: Service orchestration
10. Appendix
Figure 10.10: SOAP exposed XML message transformation using XSLT
Figure 10.11: SOAP implementation of a web service using JAX-WS
10. Appendix
Figure 10.12: SOAP implementation of a web service using JAX-WS secured by WS-Security
Figure 10.13: SOAP web service proxy
10. Appendix
Figure 10.14: Method POST of a RESTful web service implementation