study, the application’s display size has a signicant effect on the se- mantic (i.e., unguided) navigation tasks, SDAZ signicantly reduces the task completion times in navigation tasks using HALO (Baudisch and Rosenholtz, 2003) to guide the users and a non-signicant increase in task completion times when SDAZ is used in unguided navigation tasks.
Murray-Smith et al. created a dynamic systems-basedstate-space model for SDAZ (Eslambolchilar and Murray-smith, 2004; Eslambolchilar and Murray-Smith, 2008c) and discussed its application in a tilt-based 1D text browsing interface for PDAs. In their approach, the SDAZ “cam- era” is modeled as a physical object in an environment simulating mass and friction, which is coupled to the user’s input. Using this model al- lows the creation of automatic zooming behaviors that are modeled on physical analogies, which are likely to be more easily understood by the users. In addition, the State-Space Model allows developers to pre- cisely tune the interface’s behavior using a small set of parameters. For our contributions in Chapter 3 we adapted this previous work by ex- tending the model proposed by Murray-Smith et al. to support 2D map scrolling with automatic zooming.
2.2 Sensor–Based Interaction and Around-Device In-
teraction
Interaction in the vicinity of a mobile device, Around-Device Interaction (ADI), has been discussed in previous work and is also the focus of on- going research efforts. Our contributions in Chapter 4 were developed to further explore the design space of ADI and sensor-based interaction.
Our presentation of related work focuses on two important areas in the space of ADI: sensing inputin relation to the deviceand sensing inputin the environment(including the user’s body).
2.2.1
Sensing Input in Relation to the Device
The Gesture Pendant (Starner et al., 2000) is a chest-worn device that consists of a wireless camera with IR illumination. This set-up was used to control a home automation system via in-air hand gestures. An additional use was monitoring for pathological tremors through analy- sis of the characteristics of the hand gestures.
(Cassinelli et al., 2005) developed a system for nger tracking in free space based on a steerable laser beam. This system is highly suited
22 2 Background and Related Work
for deployment in a mobile device since the laser, imaging sensor and micro mirror are very compact relatively low-cost components. The tracking and depth resolutions of the system are comparable to current depth imaging cameras.
MagiTact,AbracadabraandNenya(Ketabdar et al., 2010b; Harrison and Hudson, 2009b; Ashbrook et al., 2011) are systems for around-device motion detection that use a magnetometer to sense the presence of a magnetic token worn on the user’s nger. Magnetometers are present in many current mobile devices. However, it is difficult to derive precise 3D position information from these sensors. Furthermore, magnetometer-based techniques force users to use an input artifact equipped with a magnet, such as a magnetized ring or a stylus with a magnetic tip. Another limitation of the such systems is that only a single object can be tracked simultaneously.
SideSight(Butler et al., 2008b) used IR distance sensors to implement multi-touch input to the sides of a mobile device, when the device is placed on a at surface.
2.2.2
Sensing Input in the Environment and on the Body
A number of projects have explored the concept of sensing input in the environment in vicinity of the user, and on the users themselves. The premise is that input areas are ubiquitous—we only need to nd out how to sense user input on them. All of the following techniques can be incorporated into mobile devices.
Acoustic input via scratching on a mobile device was proposed by (Murray-Smith et al., 2008). (Harrison and Hudson, 2008) explored the use of arbitrary surfaces such as walls, tabletops or fabrics can be instrumented with a microphone. Using machine learning techniques, different gestures can be classied from the acoustic signature of the scratch.
Using the body as an antenna for ambient electromagnetic noise, (Cohn et al., 2011) could classify the location of touch inputs on surfaces near a source of electromagnetic noise, such as a wall outlet or light switch. Measurements of very low voltages on the user’s body, which are caused by external electromagnetic noise, were used as input fea- tures for classication. The scope of this concept was further extended to detect free-space body gestures in environments with electromag- netic noise (Cohn et al., 2012). Moving parts of the body alters its properties as an antenna, which in turn modies the voltages measured due to electromagnetic noise. Although the techniques discussed pre- viously look promising, they are currently only experimental, and it is
2.3 Rear-of-Device Input 23
unclear how they would perform in environments with high amounts of electromagnetic noise or highly varying amounts of noise, e.g., in large buildings, industrial settings or in electried public transport vehicles.
SkinPut(Harrison et al., 2010) uses the transmission of sound via the body’s skin and the skeletomuscular system to localize on-body input events. Harrison developed a sensing device worn on the upper arm that could locate touches at arbitrary positions on the user’s arm. The device used a set of piezo sensors tuned to a set of resonant frequencies transmitted from touches on the user’s arm through body tissue and the skin.
OmniTouch (Harrison et al., 2011) uses a shoulder-mounted projector and depth camera combination to enable touch interaction on arbitrary everyday surfaces. The depth camera is used to automatically acquire surfaces for output, detect touch events and to warp the projected im- age to approximate the orientation of the projection surface.
2.3 Rear-of-Device Input
HybridTouch (Sugimoto and Hiroki, 2006) was one of the rst mobile device prototypes with two-sided input capabilities. The authors also developed dual-sided interaction techniques for map navigation.
Rear-of-device input was further explored by (Wigdor et al., 2007c). The prototype Wigdor et al. developed consisted of a display with two resistive touch screens, one on the front and one on the back. In addi- tion, a camera was mounted at a distance from the rear touch screen, in order to detect the presence of the user’s hands such that a “shadow” image of the hand could be displayed to the user. This shadow image could also be used to represent a hover state for rear-of-device input. In case of missing visual feedback of the hand or nger position when interacting behind interactive surfaces, Wigdor demonstrated that min- imum target sizes of around 4.5 cm are required for users to reliably acquire them (Wigdor et al., 2006).
Baudisch et al. explored rear-of-device input on very small devices (Baudisch and Chu, 2009). Display-based techniques for compensating occlusion due to thefat nger problem(Siek et al., 2005), e.g., Shift (Vo- gel and Baudisch, 2007), aren’t applicable below certain screen sizes due to the missing screen space for showing additional information such as callouts. Baudisch et al. show that with rear-of-device interac- tion, complex tasks can still be accomplished, even on devices with a screen size smaller than one inch.
24 2 Background and Related Work
Wobbrock et al. conducted an extensive study input performance for front-of-device and back-of-device interaction (Wobbrock et al., 2008). The authors specically look at performance of Fitts’ Law perfor- mance, feedback mechanisms for back-of-device interaction and com- pare stroke-based text entry performance in front and on the rear of the device.