Chapter 4: A Performance Evaluation of Security Profiles for Web services
4.4 RELIABLE MESSAGING
As illustrated in Table 4-4, adding reliable messaging to our Web service resulted in increasing the average response time by around 30 % when applying reliable messaging alone, and more than twice its value when we applied the Exact Order Delivery (which drops the throughput to less than half its value).
1 10 100 1000 10000 100000 M ax R o u n d Tr ip Ti m e – R TT (m ill isec o n d s)
Initial Message Size (byte)
No Security UA UDP MCS SSL SA SV STS
Ave ra g e R esponse Time ( mi ll isec onds ) Max R esponse T im e (mi ll isec onds ) Thr oughput (re sponse/se cond) Ave ra g e R TT (mi ll isec onds )
Simple (No reliability) 16 32 62.5 107.42
Reliable Messaging (RM) 21 312 47.62 135.11
RM + Order 35 18329 28.57 155.94
Table 4-4: Reliable Messaging Results
The maximum response time recorded in this experiment for Web services with reliable messaging implemented was more than 10 times the maximum value of Web services without reliability assurance. When applying the exact order delivery assurance, the maximum value increased by more than 570%. This is because when a message is lost, the sender endpoint resends the message until its receipt is acknowledged by the receiving endpoint. If these messages are received out of order, the receiving endpoint can rearrange the messages into the correct order. These processes have a huge impact on the performance of Web services, as demonstrated in the previous table.
4.5 Summary
In this chapter, we have compared the performance of several security profiles for Web services. The performance evaluation has shown that profiles that use transport level security are always faster than the message-level security ones. In addition, Message level security protocols have a scalability problem if large messages are exchanged, unlike SSL-based profiles. Within message level security profiles, username authentication based profiles perform better than mutual certificates security. However, the difference is insignificant when using very large size messages. Using digest passwords instead of encrypting the whole
penalty of using SAML is very small and depends primarily on the underlying security profile. In addition, the performance of STS security profiles is massively less than non-STS profiles and should only be used when the service and its clients are on different domains. Finally, reliable messaging is very important in critical systems because it enables them to overcome the failure of losing messages in transit or delivering them out of order. However, reliability comes at the expense of the performance of Web services, as discussed in the previous section.
Several studies, such as (Shirasuna et al., 2004; Moralis et al., 2007; Novakouski et al, 2010) reached similar conclusions regarding the high performance of SSL when compared to message-layer security mechanisms and the little impact of changing the security token on the performance of Web services. On the other hand, the main difference between our approach and the previous work is that we focused on the performance of security profiles rather than its underlying standards and protocols. We argue that our approach simplifies the security usage and increases the usability when making a decision on which security profile to select for a given application. This is because it shields the developer from the complexity and variety of the different security standards, and allows them to focus on the overall performance of the easier-to-understand profiles. Moreover, we studied the impact of changing the message size on the performance of Web services and analysed not only the average values, but we also considered the maximum and standard deviation values in order to reach a better understanding of the overall performance.
This chapter represents the second iteration of the DSR cycle (see section 3.6.2 ); the results gathered from these experiments provide the basis for the AHP model that is developed in the next chapter. The discussion provided in this chapter may also be used as a performance guideline to be consulted when selecting a security profile for a Web services application.
Chapter 5: Selecting Web Services Security
Profiles: A Multi-Criteria Decision
Making Approach
5.1 Overview
As discussed in the previous chapter, there are several XML-based security profiles and mechanisms that may be used to satisfy the security requirements of a particular application. The task a Web service developer in an ideal situation is to select the one profile that satisfies all the requirements. On many other occasions, there may be several solutions that satisfy most of the requirements, but not all. This can be considered as a decision problem, where an informed decision should be made by prioritizing the requirements and ranking the available options accordingly to select the profile that matches the most important requirements. The selection of a candidate solution depends on evaluating the available security profiles against several characteristics derived from the requirements of the given application. A Web services application developer specifies a set of requirements according to the service provider’s security preferences and technological constraints, taking into consideration the quality of service expected from the service consumer. The requirements can be divided into several dimensions according to different independent criteria and sub-criteria to create a multi- dimensional model. Each criterion and sub-criterion is given a weight according to its level of importance in the application. Each candidate solution is also weighted against each sub-criterion according to what extent this particular solution matches the requirement represented by this sub-criterion.
Generally, service providers set these weights on the basis of judgment. When there are a small number of selection criteria, then direct judgments may be a possibility. Nevertheless, when the number is large, the judging process may lead to improper selection due to the bias towards key elements only (Godse, Sonar & Mulik, 2008). In such a multi-criteria decision making model, it is important to have quantifiable values that are rational and consistent.
In this chapter, an Analytical Hierarchy Process (AHP) (Saaty, 1982) approach is used to solve the problem of assigning weights to selection features and enable a quantitative basis for selecting a security profile for a Web service. AHP is a well- established method to make these types of decisions. It is a quantitative approach based on relative judgments, with the assurance of consistent output. AHP is able to handle tangible as well as intangible attributes while monitoring the consistency in judgment (Godse, Sonar & Mulik, 2008; Roper-Lowe & Sharp, 1990) .
This chapter also provides a scenario-driven approach to demonstrate situations where the decision making framework is used to make informed choices to rank various service security patterns and select the best possible one to meet the requirements of these scenarios.
Accordingly, the structure of this chapter is as follows: Section 5.2 discusses the decision making concept, while section 5.3 provides an overview of the common methods used in Decision making. Section 5.4 explains the Analytical Hierarchy Process (AHP) and the steps of its implementation. The AHP framework for selecting security profiles for Web services is introduced in section 5.5. The framework is then validated using three different scenarios in Section 5.6. Finally, concluding remarks are presented in section 5.7.