Sensor Layer

On an architectural level sensors are components that can provide information about the world. No discrimination is made between physical sensors and logical sensors. Physical sensors are hardware components that measure parameters in the environment and provide the information on electronic level, typically as analog

output or as digital signals. Logical sensors are components that provide information that is not directly taken from the environment but represents information about the real world, e.g. a clock as a sensor that offers time and a server offering the current exchange rate. These sensors supply the information most often as digital signal over a common interface such as a serial data connection or an HTTP-connection.

More formally each sensor Si is regarded as a time dependent function that returns a

scalar, vector, or a symbolic value (X). A set (finite or infinite) of possible values (domain D) for each sensor is defined. This is a common way of describing sensors [Brooks,98]

Si: t Æ Xi

t is the time (discrete), Xi ∈ Di, i is the identification of the sensor

To simplify calculations and regarding the fact that processing is done on digital systems the time is considered as a discrete variable.

To achieve exchangeability and hence flexibility for each sensor, a physical and logical interface is defined. The physical interface consists of the mechanical specification of the connector and the electrical specification. In Table 5 an example of these descriptions for three different sensors is given. For high level sensors the

optional Cue Distribution Platform / Network optional Context Distribution Platform / Network

Sensor 1 Sensor 2 Cue 1,1 Sensor n Cue 1,2 Cue 1,i Cue 2,1 Cue 2,2 Cue 2,j Cue n,1 Cue n,2 Cue n,k Context

Applications and Scripting

level of detail for the specification is straightforward (e.g. as for a GPS receiver), whereas for low level electronic components the specification could be extended to include all the information available about a sensor. However if the specification is too detailed it fits only one sensor. To make it feasible to exchange and replace sensors with compatible ones the specification should only denote the really essential parameters for its operation.

The specification of a sensor always reaches a level where the output data can be directly accessed from the software running on the systems, e.g. on the microcontroller. The method of access is usually done over an interface, such as accessing an analog-to-digital converter or reading from the serial line.

3.8.2 Cue Layer

The concept of Cues provides an abstraction of physical and logical sensors. As seen from Figure 3, each cue is dependent on one single sensor, but using the data of one sensor, multiple cues can be calculated. In contrast the term feature is used, a general concept of abstraction where the input data does not necessarily originate from the same sensor.

GPS-Location sensor (e.g. Garmin Etrex Vista)

Light sensor based on a light to voltage converter (e.g. TSL250) Service on the internet that provides foreign exchange rate Mechanical specification Female Sub-D connector 9 Pins, RS232 pinout

three pin hole- through soldering connection with 2.54mm grid where pin 1 is GND, pin 2 is Vdd, pin 3 is Vout, Vout is connected to an analog input of the MCU. Ethernet connection Electrical specification +12/-12 Volt, RS232 specification GND is 0V, Vdd is 5V and Vout is between 0 and Vdd, Ethernet specification

Logical interface NMEA format, 0V is dark,

increasing voltage, in full light ouput is Vdd,

TCP / HTTP protocol spec. URL to get the exchange rate, data type of the answer.

Software access Serial line API Analog/digital conversion API

TCP/HTTP API Table 5: Sensor Specification Examples.

A cue Cij is regarded as a function taking the values of a single sensor i up to a certain

time t as input and providing a symbolic or sub-symbolic output Y. A set (finite or infinite) of possible values (domain E) for each cue is defined.

Cj: Si(t) x Si(t-1) x … x Si(t-n) Æ Yij

t is the time (discrete), Yij∈ Eij, t≥0 n≥0, j is the identifier for the cue

As seen from the definition, each cue is dependent on one single sensor but using the data of one sensor, multiple cues can be calculated. Cues are one way to reduce the amount of the data provided by the sensors. For physical sensors, introducing a layer for cues also can ease the calibration problem.

To implement cues all perception methods introduced earlier applied to the data from a single sensor can be used. In particular basic statistical functions and time domain analysis proved to be of great value. They can either provide a summary of the values over time or they help to extract features from the raw data that characterise the data over the last period of time.

The decision of what time window to use as basis for calculating the features, has implication on the meaning of the feature as well as for the implementation. In general longer time windows need more storage and the features have a longer delay before they are available. But longer time windows allow more analysis over time and hence often offer valuable information. When generating multiple cues that need data collected over a longer time window a buffer can be shared between all these cues calculated from one sensor [Chen,99]. The buffer is then a layer between the sensor and the cues. A reasonable time window depends on the sensor and ranges between 100ms and a few minutes.

When designing cues the properties of the sensor should be taken into account. Working on relative values or on changes in the signal is often useful to help eliminate effects introduced by de-calibration of sensors.

In building prototypes, the concept of cues proved to be very useful to make changes of the hardware transparent for the context recognition layer. When including new

sensors with different characteristics, only changes in the corresponding cues must be adapted.

3.8.3 Context Layer

As defined earlier a context is a description of the current situation on an abstract level that can be matched against previously specified situations. These descriptions consisting of conditions may or may not be human readable; in general they can be regarded as a function that takes cues as input and evaluates to true or false stating whether the current situation belongs to a context or not. The context layer holds all possible context descriptions that are relevant and modelled for a given scenario or a particular application. In a certain situation none, one, or more of the contexts may satisfy the relation that the situation belongs to the context. The evaluation of conditions to true or false can be enhanced by assigning a certainty to the decisions on the relationship.

Depending on the dimensionality of the data available from the output of the cue-layer different perception algorithms can be deployed to evaluate contexts. To realise a system working with just a few well understood cues and a small set of contexts, simple rules (e.g. first-order logic) can be used to determine whether or not the current situation belongs to a context.

The hardware constraints (e.g. memory, processing power) and the anticipated behaviour of the recognition algorithm (e.g. fixed, able to learn, and reaction time) also have to be taken into account when deciding on which method to use.

3.8.3.1 Learning

How to incorporate learning capabilities and learning behaviour into context-aware systems is a major design decision, see Table 6. Basically three concepts can be discriminated:

• No learning after the development is completed • Dedicated learning phase during use

When the contexts that should be recognised and the conditions that indicate these contexts are independent of where the system is applied, then no learning is required after the system has been developed and deployed. Based on collected data algorithms can be found beforehand that recognise these contexts. The conditions can be hard- coded; typically this can be done using rule based algorithms and supervised neural networks.

Learning capabilities can be incorporate into the system. In a specific learning phase knowledge is acquired. When contexts differ depending on the usage of a device, but are stable over time, a system using a dedicated learning process is required. An example is a device that offers different behaviour in the contexts ‘at home’ and ‘at work’. After purchasing a device it has to learn what the contexts are like in this specific case. This requires that the learning algorithms are implemented on the actual device. In this phase the conditions for a context are determined.

Having continuous learning built into a system is an option for devices that operate on contexts that may (slightly) change over time but still have the same meaning. To implement this, flexible algorithms are required and also mechanisms that supply, at least from time to time, the labels for recognised contexts. Algorithms such as the self-organising map can be used to realise this form of adaptive behaviour.

In chapter 6 this architecture is extended with capabilities for the distribution of sensing, processing and context usage.

Concept of

Learning/adaptation

Usage Algorithms examples

No learning, fixed Contexts are globally valid

Design time data analysis

Static Rule based systems,

Preset Supervised NN

Learning phase Contexts are stable but different depending on the use case

Training and/or data analysis capabilities built in Dynamic Rule based systems Supervised Neural nets Fully adaptive, always learning Contexts are changing over time

Adaptive algorithms SOM,

ISL [Hagras,02] Table 6: Learning and adaptation.

3.9 Discussion

Having the right sensing infrastructure is just a prerequisite, to make use of it, perception and the ability to match the stimuli acquired with patterns that indicate actions is also an essential part. Senses, as we know them from biological systems, incorporate all these steps, whereas sensors in a technical context only provide very basic information.

When considering sensing and perception in Ubiquitous Computing it becomes inevitable to assess the cost introduced vs. the gain provided. In nearly all cases sensing is not for free, typical costs introduced are: higher power consumption, more fragile devices, and more complex hardware and software. For many applications sensing can provide new qualities that would otherwise not be possible.

Considering the appropriate sensing infrastructure to get a particular context is tricky, often it is easily feasible to detect a context when there are no restrictions (e.g. large number of sensors and unlimited processing power). In real settings the central issue is to create a minimal sensing and perception system that can provide the information required.

When using sensors to spot contexts and to detect situations, one has to be aware that sensing systems have limitations. As contexts often do not have well defined boundaries there will always be cases where it is not clear whether or not a situation belongs to a context or not. This is not necessarily a problem of a poor perception system; this problem may even apply when having several human observers who have to make the decision. Using a human observer as a simulation for the optimal context sensing system is motivated by this observation and discussed in more detail in chapter 8.

A further problem that can lead to ambiguities is that when creating systems, only a limited number of possible situations that should be matched by a context are assessed. Usually these are the situations that are expected by the designer. In real use however it is likely that situations no one considered before,will appear and possibly result in a context that was not desired. If the system is well engineered these exceptions are rare but still it will be not possible to guarantee 100% reliability.

In contrast to standard automation systems and also to applications in robotics sensing in Ubiquitous Computing has to consider the human in the loop. In process automation the tasks and requirements are usually well understood. Also the question as to what has to be sensed and how the sensor reading should be interpreted is clearly defined by the process. A process that is set up usually stays the same with all operational functions that were anticipated at design time. In robotics and in particular for autonomous robots sensing is also clearly related to a task, such as finding a path or trying to avoid hitting an obstacle. In Ubiquitous Computing where sensing is deployed, providing the system with a world model that relates to the user’s world model, the sensing and perception task is less focused and often more general.

3.10 Summary

In the beginning of this chapter the concept of sensing and context acquisition is motivated by biology and in particular by looking at the senses that are around in nature. A terminology for situation, context, and their relationship is introduced. Context is considered as an abstract description of a type of situation. For a particular situation it can be determined whether or not this situation belongs to a context. The overall hypothesis that the sensory stimuli received in situations of the same type, at least in their discriminating features, are similar is stated.

From a more technical perspective the requirements on sensing with a specific focus on Ubiquitous Computing are analysed. Based on these requirements and drawn from the experience gathered in various projects, an overview on sensors and sensing systems that are interesting for context-aware computing is outlined. This is complemented by the consideration how the deployment of multiple sensors can ease perception tasks. A selection of perception methods with respect to systems with very limited resources is assessed. The sensors and algorithms discussed here are the foundation for the building blocks and libraries introduced in chapter 5.

In the final part of the chapter a layered perception architecture is introduced. Discriminating between sensors, cues, and contexts the architecture provides a framework for context acquisition systems for Ubiquitous Computing. This architecture is the groundwork for device and software frameworks introduced later.

Modelling and Prototyping

Sensing and perception provides means to acquire context. In this chapter it will be investigated how context can be modelled and in particular how situation and context relate to entities. This results in a conceptual model, suggesting context as entity- based information. The corresponding implementation model is based on to the approach of embedding context and context-awareness into artefacts.