Study Conditions

The following sections provide details on the implementation of the four study techniques. 5.2.1 Touch Screen (Touch Scroll and Touch Flat)

My implementations for touch screens replicated a user interface typically found in today’s smart phones, phablets, and tablets. A typical procedure for making a selection using a one of these devices would be to first unlock the screen to access the home menu, then start the UbiComp interaction app, and finally find and select the appropriate icon to tap on. Figure 57 (left) demonstrates how a typical home screen might look like; the bottom-right icon starts the

114

mobile devices, the lack of screen real-estate poses a problem when the number of selection options exceeds the available space for proxy icons. Touch Scroll is designed after the

aforementioned scrollable list design, which can be found, for example, in the Netflix UI for iOS and Android. Figure 55 shows a close-up of the interface where only three of six TV shows are visible at the same time. In order to select other shows, users have to click on the right arrow to scroll the menu. Scroll button are 1.0 × 2.5 𝑐𝑚2 _{of size, selection buttons 2.3 × 2.5 𝑐𝑚}2 _(width × height).

Figure 55: Example of horizontal scrolling in Touch Scroll

Figure 56: Example for flat design in Touch Flat

Touch Flat is designed after the aforementioned flat design, which can be found, for example, on

the Google Android home screen. Here my solution is to decrease the size of the buttons, so that all six of them fit in one row. On one hand, the reduced size makes buttons more difficult to hit,

115

and on the other hand, the permanent availability of all buttons renders scrolling unnecessary and thus reduces selection time (see Figure 56). With 1.4 × 2.0 𝑐𝑚2 _{(width × height) selection} buttons in Touch Flat have half the area of the ones in Touch Scroll.

Figure 57 shows an overview of the touch screen interface. The home screen (left) is the first screen participants encounter. From there, they have to tap on “Room Control” to open up the UbiComp control window. Depending on the condition, this window either shows all selectable items in scrollable lists (Touch Scroll, Figure 57 center) or with compressed buttons (Scroll Flat, Figure 57 right)

Figure 57: Home screen (left), Touch Scroll (center), and Touch Flat (right)

5.2.2 Screen Pointing

Screen Pointing follows the classic window / pointer UI design and has been implemented in

commercial products, such as the Nintendo Wii Remote and the Microsoft Kinect. People select digital artifacts by moving an on-screen cursor over an icon (selection proxy) and confirming the selection. The main differences to touch-screen interactions is that input is now indirect. There are two major questions to answer when designing for full-arm pointing at screen-based proxies. First, a designer has to decide which of the 6 input dimensions (𝑥, 𝑦, 𝑧, 𝜑, 𝜃, 𝜓) to use. One possibility is using pointing direction (𝜑, 𝜃), where yaw and pitch are mapped to on-screen 𝑥-

116

and 𝑦-coordinates (e.g., Nintendo Wii); another is using location (𝑥, 𝑦), where 𝑥- and 𝑦- coordinates of, for example, the user’s hand is mapped to on-screen 𝑥- and 𝑦-coordinates (e.g., Microsoft Kinect). I decided to use pointing direction because pointing is widely used in human communication to refer to out-of-reach objects (Ekman and Friesen, 1981) and it allowed a direct comparison to Room Pointing.

Second a designer has to decide on a 𝑐 𝑑⁄ -ratio between limb and cursor movement. There are multiple types of 𝑐 𝑑⁄ -ratios (e.g. below, equal, or larger than one; constant, linear, or higher- level functions; and time- or location-dependent) (Blanch et al., 2004); their choice depends on factors like user age or expertise (Smith et al., 1999). For this study, I expected participants to have little experience with full-arm pointing interaction techniques. Subsequently, I minimized the 𝑐 𝑑⁄ -ratio by allowing the maximal input range of ±16° of for radial / ulnar wrist deviation (cursor: up / down) and ±40.0° for wrist flexion / extension (horizontal cursor movement). In general, these values follow the recommendations from existing literature (Liskowsky and Seitz, 2014). I decided, however, to use an extension value below the possible maximum (62°) to accommodate both left- and right-handed users. This means that each icon has a size of ~6.5° horizontally and ~5.0° vertically in physical input space. In participant’s visual field, each icon is ~3.4° (vertical) times ~2.6° (horizontal).

5.2.3 Room Pointing

One factor that sets Room Pointing apart from Screen Pointing is the difference in selection proxy design: instead of on-screen icons, Room Pointing uses real-world objects as proxies. This changes the procedure on how people select digital artifacts. Instead of remembering the

navigation path to an artifact and visually searching for it on screen, people have to remember the association between artifact and real-world proxy, remember the location of the real-world proxy in the environment, and perform a pointing gesture toward it. Although it might appear that this procedure requires high cognitive load, human memory is specialized to perform all three steps exceptionally well (see Chapter 3).

In Room Pointing, real-world objects fulfill a dual role of being selection proxies as well as mnemonic devices. The richness of meaning that people can associate with real-world objects (e.g., their shape, color, location, history, name) increases their utility as mnemonic device and

117

can help people remember the association between real-world proxy and digital artifact better (see 2.5.5).

Figure 58: Screen Pointing (top) and Room Pointing (bottom)