In this section, we discuss evaluations conducted on gesture systems, and the various problems and motivations that they have attempt to solve. Figure2.8shows that almost half of the systems, interactions, applications and devices that we reviewed do not include any form of system evaluation or experimentation. While this was identified as a problem with the existing research, we next discuss potential reasons for this.
2.6.1 Point Designs
Many of the systems that we reviewed for this work present point designs, or systems that either implement or propose novel interaction techniques or applications for ges- tures (Hinckley et al., 1998; Turk, 2004). However, given the nature of these point designs, extensive evaluations are often not conducted, providing a possible explanation into why they remain point designs, and rarely contribute empirical results to increase our knowledge about gesture interactions. Some examples of point designs are novel applications such as the shaking-gestures used as a security measure for mobile comput- ing in public displays (Patel et al.,2004), or applications that utilise infrared pointers
Figure 2.9: A summary of the different forms of evaluations we reviewed in the
literature.
(Swindells et al., 2002), touch interactions (Rekimoto, 1997) or the physical bumping of devices to initiate a file transfer (Hinckley,2003). While there are many examples of point designs in gesture research, we suggest that this may be one of the problems that could contribute to our understanding of why after over 25 years, work like (Bolt,1980) has still not experienced uptake into everyday computing.
2.6.2 Evaluating Gestures
We noted several highly prominent issues that could lead to a better understanding of gestures. For example, the graph2.11shows almost half of the research reviewed in this paper has not presented any form of evaluation or study that is relevant to their systems or applications. Since all of the literature included in this pie refer to implemented systems, applications or interfaces, it is surprising that most of them have not performed any form of evaluation or provided any results about the affects of usability in terms of accuracy or any other features of their system. Although the research presents novel work, it would seem that there should be some form of study to contribute knowledge towards advancing gesture research. In the next section, we discuss some approaches that were used to study gestures (see Figure2.9).
Crickard et al.,2003b;Wild et al.,2004;Czerwinski et al.,2004), there is little attention towards determining how different input modes can effect interactions. We begin to address this issue in Chapter3, but next, we consider the scenarios in which these tasks are situated.
Computing scenarios. Our review of the literature suggests that most interaction scenarios are designed to accommodate what the technology can handle, leading to systems that require the user to adapt to the technology, rather than learning what the user actually requires from the technology. Figure 2.10 presents some of the main motivations for exploring gestures, which include creating more natural, intuitive, or simple interactions. However, these assumptions are not always based on empirical evidence, as we did not find research that specifically addresses ways to determine if a scenario can benefit through the use of gestures. We present several studies within this dissertation that begin to demonstrate how these scenarios can be determined and evaluated, in Chapters 3 and4.
System performance and user tolerance. Again, there are few if any studies noted in this literature review that investigate system performance characteristics and their effect on user tolerance. Some evaluations attempt to determine accuracy rates of systems, but researchers typically conduct only short user trials as shown in Figure
2.9. Results from these studies only present results that are relevant to individual systems and do not contribute to advances in the field. We begin to address the issue by investigating user tolerance levels for different system performance issues in gesture recognition in Chapter4.
2.6.3 Analysis of Evaluations in Gesture Research
As discussed in the previous sections, we observe several trends that persistently mo- tivate the use of gestures to create natural, simple, intuitive, human-to-human style interfaces and interactions. We also noted that a motivation for developing novel in- teractions whenever new technology or new application domains are introduced (Cao & Balakrishnan,2003;Paradiso,2003;Wilson & Shafer,2003;Lenman et al.,2002a;Fails & Olsen,2002; Swindells et al.,2002). However, before addressing some of the perfor- mance issues involved in these new technologies, or considering appropriate scenarios
Figure 2.10: Motivations and problems addressed through the gesture research.
for application domains, we propose that existing research may contain the information that can assist in determining many of the functions and constraints of interactions if we could understand the relationships that exist between the technology and the humans. This is the intended application of our classification, and our framework, presented in Chapter 5.
2.7
Gestures: Addressing and Revealing Problems
We examine some of the motivations for considering gestures as an interaction tech- nique, summarised in Figure 2.10. The graph shows that along with solving specific problems and generally exploring gestures, the main motivations are to create more nat- ural, novel and improved interactions. However, gestures also create problems due to their implementation, which we discuss next.
Natural interactions. One of the major motivations for using gestures is the cre- ation of natural computer interactions. Gesticulation is referred to as a natural gesture, however researchers are a long way away from understanding how to interpret gesticula- tion in the context of speech before such a system can be implemented (e.g.Wexelblat,
Simplifying interactions. The call for simpler and more intuitive interactions with computers through coverbal or multimodal speech and gesture interfaces has dominated the literature since at least the 80’s (e.g. Bolt, 1980; Kobsa et al., 1986; Hauptmann,
1989;Weimer & Ganapathy,1989;Bolt & Herranz,1992;Koons & Sparrell,1994;Quek,
1994;Wexelblat,1994;Cohen et al.,1997;Gandy et al.,2000;Quek et al.,2002;Eisen- stein & Davis, 2004; Kopp et al., 2004), combining speech and gestures as a means of creating a human-to-human approach to computing. But while this approach assumes that human-to-human interactions, when applied to a computer would be useful, there is little evidence to support this. Stroke and mouse gestures do however demonstrate examples of gestures creating more simplified interactions, enabling hand writing to be interpreted as input, and mouse gestures for shortcuts to menu access.
General improvements for interactions. A general problem that motivates ges- ture research is its potential to improve interactions. Gestures can improve interactions by enabling meaningful pen strokes for drawing and for introducing quick commands to control applications (Buxton et al., 1983; Rubine, 1992;Cohen et al., 1997). Gestures for 3D graphic interactions enable additional degrees of freedom over the mouse, by us- ing hand gestures to control virtual or real objects (Segen & Kumar,1998a;Zeleznik & Forsberg,1999), and to support creative, lightweight interactions for smart room envi- ronments (Streitz et al.,1999;Gandy et al.,2000;Rekimoto,1997). Adaptive interfaces also demonstrate some potential improvements where gestures can control wheel chairs or enable text input without having to use a keyboard (Pausch & Williams,1990;Keates & Robinson,1998; Ward et al., 2000). Additional improvements are suggested in per- vasive and mobile computing domains (Amento et al., 2002) and gaming applications (Freeman et al.,1996), creating intuitive interactions based on using real-world objects. .