State of the Art

This subsection presents the state of the art on resilience evaluation and ubiquity sup- port assessment in Ubiquitous Computing (UbiComp) Systems.

Resilience has been evaluated in different ways and for various protocols. Both the Resilience-Differentiated Quality of Service (RD-QoS) framework [Autenrieth, 2003a] and Quality of Resilience (QoR) [Cholda et al., 2008, 2009] assess the resilience sup- port of MPLS. However RD-QoS does not assess the recovery cost and only includes a time analysis based on the ITU-T M.495 model [ITU-T, 1993]. QoR combines Quality of Service (QoS) metrics (e.g., packet loss, delay) with resilience metrics (e.g., steady- state availability, mean downtime). Moreover, QoR employs histograms that can be mapped to user satisfaction. Nonetheless, the evaluation methodology is tied to MPLS, lacking a broader applicability to other kind of protocols.

Other kind of evaluation includes recovery efficiency and the protection model supported (e.g., 1+1 or 1:N) [Pioro and Medhi, 2004]. Nevertheless, the evaluation relies on non deterministic methods, which depends on the application requirements. Evaluation methodologies fail to provide a complete evaluation of resilience support, as the example of proposals that only assess availability [Autenrieth, 2003a; Akella et al., 2003; Huang et al., 2007].

Other evaluation frameworks consider qualitative (e.g., risk) and quantitative char- acteristics of resilience and related-terms (i.e. dependability, fault-tolerance) in sys- tems [Al-Kuwaiti et al., 2009]. Nonetheless, they consider physical aspects of archi- tectures, which limits their applicability to protocols in other layers of the OSI model.

With a different perspective, other proposals only address the optimal configurations of failover mechanisms for protocols supporting resilience, such as Stream Control Transport Protocol (SCTP) [Eklund et al., 2009; Budzisz et al., 2008], operating at the transport layer.

Table 3.1: Resilience evaluation summary.

Proposal Generic Objective Thorough

RD-QoSa X only MPLS X X only time analysis

QoRb X only MPLS X √

Pioroc _{X depends on application X} √

Akellae _{X only physical aspects X} √

Huangf X X X only availability

Al-Kuwaitig X X X only protection

Eklundh _{X only SCTP X X}

Budzsiszi _{X only SCTP X X}

a_{[Autenrieth, 2003a]} b_{[Cholda et al., 2009]} c_{[Pioro and Medhi, 2004]} d_{[Akella et al.,} 2003] e_{[Huang et al., 2007]} f_{[Al-Kuwaiti et al., 2009]} g_{[Eklund et al., 2009]} h_[Budzisz et al., 2008]

Table 3.1 summarizes the proposals analysed by the candidated, regarding require- ments fullfillment, namely Generic, Objective, and Thorough, previously, introduced in subsection 3.1.1. Other requirements, such as Flexible were not meet by the related proposals.

Ubiquitous computing (UbiComp) is a model where computers live in the world of people, aware of their location, but in a transparent form to the user [Symonds, 2009]. Modern devices and networks to some degree implement this vision for ubiquitous computing as they enable users to access online services 24-hours a day at “any time” and from “any place”. In principle, the choice of technologies, system architecture and protocols must be considered in the design phase of UbiComp systems. However, the evaluation approaches taken for UbiComp Systems so far [Resatsch, 2010; Stevenson et al., 2009] typically consider user-perspective ratings only, or assess a limited set of functionalities. For example, some assessments follow a prototype-based approach and thus have high development costs while not always being fully representative of the final system [Resatsch, 2010]. Others rely on user surveys requiring at least a partially complete and functional system [Stevenson et al., 2009]. Moreover, when multiple choices for the software components are available, said approaches do not provide insights for the selection of the best ones during the design phase.

3.1 Introduction

capabilities (i.e. technical characteristics) and the level of extensions [Kwon and Kim, 2006; Scholtz and Consolvo, 2004]. The assignment for each capability/item usually relies on interviews with experts in the field (e.g., with ubiquitous computing expe- rience), giving a classification in the range {1, 2, ..., 7}. These solutions require the involvement of experts.

Ubiquitous Computing Application Development and Evaluation Process Model (UCAN) is a ubiquitous computing application development and evaluation process model [Resatsch, 2010], which evaluation includes different stages and methods, for instance, the original idea can be evaluated using interviews, while the prototype is assessed through user acceptance methods. Whilst UCAN requires prototypes and is tailored for applications relying on user satisfaction metrics, other proposals allow the evaluation of the overall system [Kwon and Kim, 2006].

Ontonym [Stevenson et al., 2009] is a framework that allows the evaluation of per- vasive systems. The framework models context based on ontologies. For instance, people are modeled by using classes with different attributes such as Name and Reli- giousName. The evaluation considers three aspects: design principles, (e.g., extensibil- ity and documentation); content (e.g., clarity and consistency); and purpose (in which domain the evaluation is performed). Despite using established standards, Ontonym focuses on the context representation problem, and therefore does not provide objec- tive and comparable metrics to evaluate UbiComp systems.

In a UbiComp system, the mobility management is a key factor to enable the al- ways connected paradigm. As such, considering the software component of UbiComp systems, there is a need to establish the degree of mobility supported by IP mobil- ity management protocols. For instance, how a certain protocol manages mobility. This evaluation may consider different performance metrics. Usually, metrics include packet delivery cost, handover delay, location update cost, signalling cost, multiple interfaces and simultaneous mobility support. The packet delivery cost metric, for instance, determines the cost (e.g., processing or transmission) of the different packet delivery mechanisms (e.g., tunnel, direct) [Wang and Abu-Rgheff, 2006]. The han- dover delay metric includes movement detection, address configuration, security op- erations and location registration [Kong et al., 2008; Liu et al., 2007]. The signalling cost is a compound metric that combines the packet delivery cost and the handover cost, commonly designated by location update cost [Wang and Abu-Rgheff, 2006]. The location update cost is determined according to the network model (e.g., number of hops, number of domains, wired and wireless links), message rate and respective mes- sage length. The difference between these proposals resides on the fact that some of

them include the functions of each involved entity (e.g., home agent, correspondent node), while others only include the mobile node or the cost of specific operations (e.g., tunnelling) [Makaya and Pierre, 2008]. The support of simultaneous mobility metrics is often neglected in evaluations, assuming fixed correspondent nodes [Wang and Abu-Rgheff, 2006; Makaya and Pierre, 2008], although some consider the prob- ability of simultaneous movement [Wong et al., 2007]. Paging efficiency is another metric to consider when evaluating mobility management protocols in a ubiquitous environment, specially due to energy efficiency. Paging support evaluations assess the power consumption cost, the paging delay cost, but in a technology-dependent or application-dependent way [Lee et al., 2008; Do and Onozato, 2007; Tang et al., 2007].

Table 3.2: Ubiquity evaluation summary.

Proposal Generic Objective Thorough

UCANa √ √ _{X only user perspective}

Ontonymb √ √ X only user perspective

Kwonc √ √ _{X only characteristics and exten-}

sions

Scholtzd √ √ _{X only characteristics and exten-}

sions

Wange _{X only Mobile IP X X only mobility’s degree}

Kongf _{X only Mobile IP X X only mobility’s degree}

Liug _{X only Mobile IP X X only mobility’s degree}

Makayah X only Mobile IP X X only mobility’s degree

Leei √ √ _{X only paging}

Lo Onozatoj √ √ _{X only paging}

Tangk √ √ _{X only paging}

a_{[Resatsch, 2010]} b_{[Stevenson et al., 2009]} c_{[Kwon and Kim, 2006]} d_{[Scholtz and}

Consolvo, 2004] e[Wang and Abu-Rgheff, 2006] f[Kong et al., 2008] g[Liu et al., 2007] h_{[Makaya and Pierre, 2008]} i_{[Lee et al., 2008]} j_{[Do and Onozato, 2007]} k_{[Tang et al.,} 2007]

Table 3.2 summarizes the proposals evaluating UbiComp performance or a specific function in UbiComp systems, regarding the requirements fulfillment. Only the first three requirements (Generic, Objective, Thorough) were considered.

3.1.4 Open Issues

From the analysis of the state of the art on resilience evaluation, summarized in Ta- ble 3.1, the following open issues were identified: