Influences from State of the Art

This section discusses the influences from the state of the art that were instrumental in realising the InSitu Model Framework, Test Process and Toolset. These influences led to a set of technical requirements that must be satisfied in order to realise the situation-based testing approach. For the purpose of discussion, these are divided into two sections, the first focuses on the Model Framework, and the second set focuses on the Test Process and Toolset.

4.3.1

Influences on the InSitu Model Framework

At the core of all of the situation-based approaches discussed in the state the art review, is a situation specification model. Situations are abstractions of context and can be defined through aggregation of contextual information. Situation specification enables situations to be formally defined through logical and spatial relationships between elements of the physical deployment space. Examples can be found in UbiREAL [80], CML [50] and SituVis [27].

Of these examples, UbiREAL provides the only example of a situation-based testing approach, however a limitation of the UbiREAL approach is over-specification of test-cases. This is prohibitive for testing that aims to uncover unanticipated outcomes arising in the deployment environment. Situation specification must be generalised in order to accommodate testing for the unanticipated.

This leads to the following requirement:

Requirement R1: A situation specification model must feature in the Model Framework, which supports generalised definitions, and which can specify the logical and spatial relationships between elements in the deployment environment.

Requirement R1 addresses the issue of writing situation specifications that are capable of testing the unanticipated and unexpected. However, it is also necessary that unanticipated situations have the opportunity to arise. Chalmers [107]

suggests that model-driven simulations of users and computers can assist with the examination of ubicomp systems, particularly for emerging or unanticipated situations. Tabak and de Vries [110] are more specific in noting the current trend in activity based modelling for building usage simulation.

Activity-based approaches reflect the scheduling of activities in time and space. Routes taken through a building are a result of the activities that occupants need or want to perform, and the locations associated with those activities. Activity based modelling provides a more accurate approximation of human behaviour than purely random models [102], without requiring complete fidelity and so can be a lower investment approach. The inclusion of an activity-based model addresses one of the design considerations defined in the background chapter, user activity. An activity-based model should be included in the model framework to drive simulations that approximate the natural work-flows or activities of occupants.

The second design consideration is the physical environment and in particular its spatial relations. Spatial relationships are central to situation specification and so a situation-based testing approach must be able to reason about this aspect of a building. This is primarily because often the structure of the physical environment cannot be changed, except in terms of the operational use of the environment, and so a system must be designed to behave appropriately for the target environment.

Hierarchical models best represent the natural containment relationship in the environment, and there are many examples in the literature [25, 95]. Hybrid hierarchical models are more flexible by accommodating extra spatial relationships, as well as physical location coordinates, and there are also a number of examples in the literature [57, 10].

Proximity regularly appears in the literature in relation to sensing technologies, due to the relevance of a user’s immediate surroundings. Henricksen and Indulska [50] include it in CML, along with containment, as a key spatial relationship for situation specification. Bandini et al. [10] identify three key spatial relations for ubiquitous computing as containment, proximity and orientation.

These findings lead to the following requirement:

Requirement R2: An activity model must be included to drive simulation of user activity; a model of the built environment must be included to promote relevancy of testing by representing the target deployment environment.

The situation specification model, from requirement R1, must be able to test situation specifications using an up to date view of the state of the environment. A benefit of simulation noted in the last section is the ability to capture both simulated state and sensed information. An environment state model is necessary that can represent a unified view of this information, and which includes the main elements of situations and the spatial relationships in the environment.

This leads to the following requirement:

Requirement R3: A testable model of environment state must be defined and maintained.

4.3.2

Influences on the InSitu Test Process and Toolset

Simulation-based ubicomp test-beds generally support prototype systems to remain independent from the test-bed platform. To enable this, test-beds need to provide sources of context information to the prototype under test. Papers on the seminal simulation-based ubicomp test-beds, UbiWise [13, 14] and QuakeSim [21], set out requirements for these type of test-beds.

The requirements from QuakeSim focus on the need for realistic, configurable and extensible deployment environment simulations, featuring multi-user interactions and simulated sensors to generate location information. The requirements from UbiWise focus on user-centric assessment of ubicomp technology prior to a product-ready prototype, testing at lower costs, and providing sufficiently realistic simulated deployment environments.

Consolidating the requirements put forward by QuakeSim and UbiWise, the following requirement is put forward for a simulation-based ubicomp testbed:

Requirement R4: A simulation based test-bed for ubicomp should be configurable, extensible, reusable and scalable, with relatively low investment and should support heterogeneous context generation.

Core to the situation-based testing approach is the necessity to maintain separation between sensed context and environment state. A ubicomp system should only receive a sensed view of the environment based on the information generated by sensors embedded in the deployment environment. However, it is necessary for the situation-based testing approach that a state view of the environment is available, which represents the actual conditions that exist in the deployment environment.

Simulated sensed context should only be supplied to the ubicomp system in order to drive system behaviour. State information is supplied only to the testing engine and makes it possible to test the effect of ubicomp system behaviour by examining the situations that arise in the deployment environment.

This leads to the following requirement:

Requirement R5: Separation between simulated sensed context and environment state information must be maintained.

The challenge in assessing the effectiveness and appropriateness of system behaviour in ubiquitous computing environments lies in monitoring the distributed nature of their effect. Therefore, monitoring ubicomp deployments must be performed at a global level, i.e. across the whole space. Examples of visual monitoring tools were found in the literature, in which the designer can observe simulations unfold and watch for problems to arise, e.g. DiaSim [59]. However, visual monitoring is not particularly suitable for thorough testing. Testing is a tedious computational task, better suited to a computer than to a human. Video recording of live test-beds provides another option for monitoring ubicomp deployments, however the time investment required to process the recordings is prohibitive to both rapid iteration and low investment testing.

Also, as already noted in this section, in order to test for unanticipated and unexpected outcomes a monitoring engine must be able to apply abstract test

definitions. Test definitions should be represented using a situation specification model so they can be tested against the state of the environment.

This leads to the following requirement:

Requirement R6: A monitoring engine must be able to test environment state, which can be distributed in the ubicomp space, to automatically identify instances of inappropriate situations, without prior knowledge of the sequences and scenarios that will arise at run-time.

Structured feedback is particularly useful as part of an iterative prototyping cycle in which successive iterations produce incremental, testable changes, as discussed by Tang et al. [111]. Evaluation feedback, particularly for a prototyping cycle, is largely an open issue for ubiquitous computing and one that is actively being researched, e.g. the DiaSim [59] and AmISim [46] projects.

On identification of situational instances, the designer needs to be able to trace the causal factors. Generalisation of Weber et al.’s definition [116] resulted in a definition of traceability as: attributing events and actions to that which caused them. According to this definition, traceability should support identification of both the user activity and system activity that can effect change in the state of the environment.

This can in turn be provided as structured feedback. Although the user should not be held accountable for inappropriate situations arising in the environment, user activity that is instrumental to these outcomes should be used to inform the design refinement process.

This leads to the following requirement:

Requirement R7: Structured feedback must be generated, which can be analysed and traced for the purpose of investigating the causal factors leading to specific situations.