Furthermore, the preliminary study presented in this section needs to be repeated as a full user study with the suggested improvements. Be- cause of the low number of participants (N=4), the results presented here can only be considered indicative and not statistically signicant.
The user feedback also suggests that dynamic zooming behavior may a combination of FLICK and TOUCH should be considered for future studies not be needed at all times. A further technique that should be stud-
ied in the future is a combination of FLICK and TOUCH. In this variant, the UI would exhibit normal scrolling behavior for slow and short touches, allowing precision manipulation. For faster and longer touches, i.e. icks, the UI would scroll using speed-dependent auto- matic zooming, as in FLICK. This hybrid technique would thus com- bine the familiarity of TOUCH with the advantages of FLICK.
3.3 Summary
In this Chapter, we introduced SAZ and Flick-and-Zoom, two input techniques designed to help users navigate mobile map interfaces more effectively. Our initial goals of reducing the number of user inputs have been achieved by SAZ and Flick-and-Zoom, but our expectations on their usability gains have only been partially met.
We have shown that SAZ signicantly outperforms SDAZ, but does SAZ is a viable technique whenever multi-touch is not available or when provided for expert use
not outperform multi-touch. One of the problems is that tilt-based continuous input is not intuitive for every user and also that it has a signicant learning curve. We suggest conducting a longitudinal study to analyze this learning behavior. SAZ could certainly outperform a multi-touch when used by an expert, as has been demonstrated by some of our test subjects. What should be noted is that SAZ is a preferred technique for devices that do not have a multi-touch enabled screen, i.e. not supportingpinch-to-zoom.
In order to evaluate if automatic zooming for mobile maps can be re- alized without tilt-based input (which can be difficult for some users to master), we implemented a ick-based (Semi-)Automatic Zooming approach,Flick-and-Zoom. The preliminary user study we conducted to evaluate this technique showed that there still are some issues that need to be addressed, such as the calibration of the parameters for the state-space model.
The results of the preliminary user study of Flick-and-Zoom should Flick-and-Zoom received promising comments but requires further re nement and a full-scale user study be regarded as indicative. We obtained numerous suggestions for im-
provements of future versions of this technique, and we still need to conduct a full-scale user study of this technique. As with SAZ, our results indicate a steeper learning curve using Flick-and-Zoom. This
64 3 Continuous Interaction and State-Space Systems
may be a general property of all user interfaces using automatic zoom- ing, because the users need to generate a suitable mental model of the system in order understand the effects their inputs—aGulf of Evaluation in Norman’s sense (Norman, 2002). The positive feedback obtained by the study participants does make us condent that we will in the fu- ture be able to develop a Flick-and-Zoom-based mobile map navigation interface that outperforms the current standard UI for map navigation.
In summary, the results of our exploration of mobile map navigation interfaces based on continuous input and a state-space models that au- tomatically control certain parameters such as scroll speed and zoom level, show that the performance of multi-touch has been surpassed by neither SAZ nor Flick-and-Zoom. However, on devices lacking multi- touch, techniques such as SAZ can offer a real advantage over normal user interfaces (i.e., which use buttons and digital joysticks for navi- gation). Tilt-based techniques seem to have a steeper learning curve than touch, but our results indicate that users tend to perform very well once they have understood the concept.
65
Chapter 4
Around-Device
and
Sensor-
Based Interaction
“Basically, an input device is a transducer from the physical properties of the world into logical parameters of an application.” —Ron Baecker and Bill Buxton
A generally observable trend over the past 10 years is that new gener- novel user interface concepts need to be developed for sensor–equipped mobile devices ations of high–end mobile devices are being equipped with more types
of embedded sensors than previous generations. In the previous chap- ter, we looked at possibilities of using the existing sensors in mobile devices to develop novel mobile user interfaces. In this chapter, we ex- plore several types of additional sensor technologies that can be used to implement novel mobile user interfaces. This exploration enables us to draw conclusions about what types of sensors could be used in future mobile devices in order to improve the usability of those devices and to enable novel interaction concepts.
This chapter is structured as follows. Section 4.1 introduces the con- cept of Around-Device Interaction, a sensor–driven approach to expand- ing the interaction capabilities of mobile devices. The design space af- forded by Around-Device Interaction is detailed further in Section 4.2. HoverFlow, discussed in Section 4.3, is a prototype featuring Around- Device Interaction that uses a small number of simple distance sen- sors. PalmSpaceextends the concept of HoverFlow, allowing more ne- grained interaction using a depth camera (Section 4.4). In Sections 4.5 and 4.6 we examine pressure-enabled dual-sided multi-touch interac- tion using theiPhone Sandwich, a prototype that we developed.
66 4 Around-Device and Sensor-Based Interaction
4.1
Around-Device Interaction
The idea behind Around-Device Interaction (ADI) is to expand a mobile device’s interactive space on and beyond the device’s physical bound- aries to permit richer and more expressive forms of interaction. There has been relatively little previous work in Around-Device Interaction (ADI) for mobile devices. ADI is of particular interest for mobile in- teraction, as the size of mobile devices—and thus their interaction possibilities—is limited.
Figure 4.1: Interacting with very small devices via coarse gestures. The gestures are detected by an array of proximity sensors extending in radial direction from the device. (?).
Using sensors, the interaction space of small mobile devices can be ex- sensors enable
expanding the interaction space beyond the physical boundary of the device
tended beyond the physical boundary of mobile devices to include the full 3D space around them. Around-device interaction can be a bene- cial addition to standard interface elements of mobile devices, such as keypads or touch screens. This is particularly attractive for very small devices, such as wristwatches, wireless headsets, and future types of wearable devices such as digital jewelry (Figure 4.1). With these kinds of devices, it is extremely difficult or even impossible to operate small buttons and touch screens. The space beyond the device, however, can easily be used, no matter how small the device may be. Such wearable devices can also serve as easily accessible controllers for appliances in the environment or for wireless communication applications.
In a smart home environment, for example, a gesture tracked by the device could dim the light or control the volume of the entertain- ment system. In mobile use scenarios, an incoming call could casually be forwarded to the voice mailbox or an incoming message could be acknowledged using different gestures. For mobile phones or tablet PCs—whether handheld, placed on a table, or placed in a cradle in the car—ADI could open up a range of 3D interaction possibilities. Coarse movement-based gestures could control tablet applications, such as turning pages in an electronic book. In a calendar application moving to the next day or month could be controlled by specic gestures, such
4.1 Around-Device Interaction 67
Figure 4.2: An overview of the hand and nger gestures that can be recognized by the HoverFlow prototype (see Section 4.3). (?).
as sweeping with the palm or with the edge of the hand, respectively (Figure 4.2).
Such coarse gestures do not require the activation of a user interface widget and can be executed without visual feedback. This is especially benecial for devices for which command selection via visual feedback is difficult, because the device is not in the line of sight, such as digital jewelry or wireless headsets. More ne-grained gestures could have a natural spatial mapping to 3D objects on the screen. Moving the hand closer to the device or rotating the hand could be mapped to zooming along the z-axis or rotating 3D objects. In order to mitigate occlusion, such gestures do not necessarily have to be performed on top of the device display. If infrared proximity sensors are used, they can be ori-
68 4 Around-Device and Sensor-Based Interaction
ented in such a way, that the interaction does not occlude visibility of application objects on the screen.