Evaluation and Discussion

The above experiments have comprehensively tested the performance of the prototype while handling all typical types of service operations including single SIR, single SMR, multiple concurrent SIRs, multiple concurrent SMRs, and multiple chained SMRs. The analysed experiment results illustrate significant performance differences between the proposed approach and the standard web portals. Specifically, for single service operation tasks, the prototype demonstrates solid success rate while executing a diversity of operation commands, whereas there is small chances of failures while using the web portals. Although single SIR executions may take a little longer (usually 1 seconds) than the ordinary web interface, SMR operations can complete much faster (1/3 less time needed) with the prototype. Meantime, considering the multiple service operation executions, the prototype also demonstrates a better performance in overall. Although it shows that the concurrent SIR operation response times are still relatively slow while using the prototype, there is not much differences if more operations are involved; on the other hand, accessing via web portal tends to consume more and more time as the number of operations increases. On the other hand, simultaneous SMR operations can be executed much quicker in the prototype whilst the execution times are fairly stable; as a contrast, the web portal executions typically consume twice the times whilst the completion times varies significantly. Subsequently, for chained service operations, the series of sequenced SMRs can be completed sooner for prototype implementation methods. Especially, the more the chained operations are involved, the better the performance the prototype can achieve.

As illustrated in the EC2 case study, SAMOS framework can adequately model a wide range of operations. Its classifications of cloud service entities and operations enable structured specification presentation layout. The relevant operation element specifications reveal sufficient details for operation executions. As implemented in a wider service domain and across multiple CSPs, these would drive cloud service interoperability and composition (a further PaaS case example is illustrated in Appendix D). Further, to evaluate

164

SAMOS against other well-established cloud (service) specification frameworks/models, it provides the data comparison with OCCI, TOSCA and mOSAIC. Shown in Table 7.15, the four approaches involve dissimilar core/base model concepts with different specification semantics. They adopt distinct management tools/APIs as cloud service interfaces and enable service orchestration with own solutions. In contrast, SAMOS achieves a distinguished outcome for service management and orchestration tasks due to the flexible choices of API libraries and the lightweight operation reasoning assistances.

Meanwhile, the performance evaluation with USAMS involve covers a wide range of typical service operations. Obtained experiment results illustrate significant performance differences between the proposed approach and the standard web portals. Specifically, for single service operation tasks, the prototype demonstrates solid success rate regardless of the type/nature of operations; there is a small chance of failure while using the web portals. Although USAMS may consume a little more time (approximately 1 second) while handling single SIR operations, it facilitates SMR operations more efficiently (1/3 less time needed). Additionally, considering multiple service

Table 7.15 Comparison of Cloud Service Specification Frameworks Approach Syntax/

Semantic s

Model Core/Base Concepts Management

Interface Service Orchestrati on

OCCI OCCI

Grammar Category, Kind, Mixin, Resource Instantiation, Collections, Discovery /Entity, Resource, Link, Action) [108]

Testing tool, doyouspeakOCCI, OCCI API

OCCI client

TOSCA YAML Topology Templates, Plans /Service, Node, Relationship, Requirement, Capability, Artifact, Policy, Cloud Service Archive [148] OpenTOSCA, jclouds and PyTosca API Pre-defined Plans

mOSAIC OWL Environment, Infrastructure, Resource, Runtime Component, Stateful Component, Stateless Component/etc. [103, 104]

mOSAIC API mOSAIC

Cloud Agency SAMOS OWL Entity and operation classifications,

Entity data type specifications, Entity operational relationship Specifications /etc.

USAMS prototype tool, flexible choice of API libraries via OCSO API

Lightweight automatic reasoning

165

operations, USAMS demonstrates a better overall performance. For concurrent SIRs, despite the slower responses for a small number of operations, there is no perceptible time increase despite more tasks involved. In contrast, accessing via web portal tends to consume increasingly more time as the number of operations arises. On the other hand, simultaneous SMR operations can be executed much more efficient through USAMS whilst the execution times appear to be stable. As a contrast, the web portal executions typically consume twice of the times whilst the completion times varies significantly. These results suggest the proposed approach a competent solution to enable effective and efficient cloud service operations.

7.3 Summary

This chapter has demonstrated a series of real-world cloud service case studies to validate the modelled service specifications and the enabled cloud service assistance functions. Considering the range of service recommendation, retrieval and evaluation functions, the proposed AoFeCSO is capable of comprehensively describing the wide range of cloud service features, characteristics and properties. Utilising such as the knowledge source, the CSR sub system can display comprehensive service descriptions and evaluations and enable effective service search, recommendation and comparison tasks. On the other hand, the SAMOS approach is able to model the granular aspects of cloud service operations regardless of the service provides or types. As the CSAMO and USAMS sub system are deployed based on the approach, they can provide a unified interface for efficient cloud service remote management and orchestration tasks. Accordingly, these validate the proposed ontologies and approaches with solid experiments and evaluations.

166