TEST DESIGN

Recently, Web services security has witnessed a significant impetus as several specifications have been developed and implemented to meet the security challenges of Web services. However, the performance of the security mechanisms is fraught with concerns due to additional security contents in SOAP messages, the higher number of message exchanges needed to establish trust as well as extra CPU time to process these additions. See Appendix A for a comparison between four SOAP messages (Simple, Secured with UA, Secured with MCS, and Reliable Message) with an initial data of one character.

This test focuses mainly on the overall performance of WSIT security and reliability profiles. Therefore, the following discussion does not essentially cover

the performance of underlying WSS specifications implemented within the security profiles.

4.2.1 Test Subjects

This section describes the METRO/WSIT project (Sun Microsystems Inc., 2007) and its security profiles that were selected to be benchmarked by the experiments. In‎ order‎ to‎ improve‎ Web‎ services’‎ Quality‎ of‎ Service‎ (QoS)‎ and‎ to‎ enable‎ interoperability between Java and .Net Web services, the Web Services Interoperability Technologies (WSIT) (Sun Microsystems Inc., 2007) has been developed as joint effort between Sun and Microsoft. WSIT is an implementation of a number of open Web services specifications to support enterprise features, such as message optimisation, reliable messaging, and security.

Web services have relied on transport-based security such as SSL to provide point-to-point security. WSIT implements WS-Security so as to provide interoperable message content integrity and confidentiality, even when messages pass through intermediary nodes before reaching their destination endpoint. WS- Security, as provided by WSIT, is an addition to the existing transport-level security, which may still be used.

WSIT provides a number of security profiles that can be applied to secure Web services. Each profile represents a set of pre-defined security specifications and configurations. Using security profiles reduces the development time and allows Web services developers to focus their effort on identifying the security requirements of their systems rather than going into the complexity of understanding the several security standards and finding the right combination that fulfils the security needs of the developed systems. However, while applying WSIT security profiles to enhance the security of Web services, this may also result in increasing the size and number of the exchanged SOAP messages, which may in turn lead to an increase in the time of processing these messages and transmitting them over the network.

Currently, there are many security profiles that can be implemented to secure Web services, see section ‎2.10.3 . We have deliberately selected seven of them for this experiment. The selection was based on identifying the generic profiles, which cover the general configuration types and represent the different security methods, as illustrated in Figure ‎4-1. The selected profiles are:

 Username Authentication with Symmetric Key (UA).  Username with digest passwords (UDP).

 Mutual Certificates Security (MCS).  Transport Security using SSL (SSL).  SAML Authorisation over SSL (SA).

 SAML Sender Vouches with Certificates (SV).  STS Issued Token (STS).

Figure 4-1: METRO Security Profiles

Username Authentication Security Mutual Certificates Security SAML Third Party STS Message Layer Security Transport Layer Security

Test Case/ Security Profile Security Method No Security UA UDP STS MCS SV SA SSL Layer

We also test the impact of applying Reliable Message Delivery and Deliver Messages in Exact Order, as‎provided‎by‎Metro’s‎ WSIT,‎on‎the‎performance‎of‎ Web services.

4.2.2 Test Scenario and Cases

Echo Web service (Simple structure/ Dynamic payload): This JAX-WS echo application consists of a Web service and a client, which represents the peer-to- peer mode test; the client sends different size auto-generated messages (from 1 Byte to 1MByte) and the Web service echoes (send back) the same message received. The test was run with and without applying security profiles, using different initial message sizes: 1byte to 1 Mbyte. This Web service was used to test the performance of security profiles because using a simple payload reduces the side effects of unrelated processing of the business logic. In addition, as security profiles employ encryption algorithms, which are used to decipher the exchanged data, the encryption/decryption process depends on the size of the message. Therefore, a dynamic size payload was selected for this experiment. Book details Web service (Complex structure/ static payload): The client sends a sequence of messages; each contains a one-element array of book details objects (Figure ‎4-2). The Web service replies by sending a simple string response for each message it receives. The complex structure/ static payload was used for the performance of reliable messaging methods because reliability guarantees the delivery of the message, as a whole, regardless of the actual payload. If a message is lost, the sender resends the message until its receipt is acknowledged by the receiver. If these messages are received out of order, the receiver can rearrange the messages into the correct one.

<?xml version="1.0" encoding="UTF-8"?> <S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"> <S:Body> <ns2:bookOrder xmlns:ns2="http://service.testproject/"> <book> <isbn>1-11-111111-1</isbn> <author>Author_1</author> <name>Book_1</name> <pages>111</pages> <publisher>Publisher_1</publisher> </book> </ns2:bookOrder> </S:Body> </S:Envelope>

Figure 4-2: Complex Payload SOAP-Message

4.2.3 Test Environment and Settings

In this chapter, the focus is on the increment of processing time when applying security profiles instead of the network latency. As a result, the data were collected from a local machine; the Web service and the client were deployed on a Dell machine (Pentium D CPU 2.80 GHz / 3GB of RAM) Running Microsoft XP. NetBeans IDE 6.5 was used to develop the Web service and the client. The Web service was developed as a Web application and deployed on a GlassFish 2.2 application server. A Java SE application was used to represent the client. The initial data sent from the client to the service were randomly generated before sending the message to avoid any caching.‎Metro’s‎WSIT‎Web service stack 1.4 was used to apply security profiles to the tested Web service.

4.2.4 Evaluation Metric

The time spent in requesting and responding on the client side was measured as round trip time (RTT) using Java’s System.nanoTime(). For the reliable messaging experiment, we also measured the average response time, maximum response time and the maximum throughput using the Web services monitor in the Glassfish Admin Console. Every test was repeated 1000 times and the average,

then repeated on 10 different occasions, and used the average results after eliminating the highest and lowest scores to reduce the noise results. The results were then compared using the Round Trip Time Increment Percentage (Tang et al., 2006; Chen et al., 2007) (RTTIP) in order to evaluate the performance overhead for a specific security profile deployment:

% 100 0 0    RTT RTT RTT RTTIP i (1) Where:

 RTT0 is the round trip time without applying any security profile

deployment.

 RTTi is the round trip time of the Web service with a specific security

profile i deployment.