Chapter 2: Literature Review
2.8 Summary
This chapter presented the literature review concerning the gestural interaction in smart environments. Section 2.1 introduced the importance of this communication modality, which was and still is fundamental for the human beings to interact with the surrounding environment. Section 2.2 and Section 2.3 presented the different categories for gesture classification, in both psychological and computer science domains; it is highlighted that the most important categories for gesture classification in HCI derived from the psychological researches. Although these different categories are perfectly suitable to differentiate the different kinds of gestures that exist, they do not provide a method to design gestures for “natural” gestural interfaces. Some researches coped with this issue and presented the main characteristics that gestural interface should have in order to be as “natural” as possible for the users. Other researchers presented some methods that can help to design the gestures in order to enable designers and developers to implement a “natural” gestural interface; in the subsection 2.3.3, some examples such as the role- playing, Wizard of Oz and self-definition are reported. These techniques can be used as tools to facilitate the gesture design but they have been conceived to associate a gesture to a single function. Mapping one gesture to a single function can be a problem, in particular in the current interaction scenarios, where the smart environments are designed to provide many different services. In fact, mapping one gesture to one function means creating a gesture taxonomy that is very vast if the smart environment is able to provide many functions. A vast taxonomy is a problem for the user since it reduces its usability and learnability; these two are among the most important characteristics identified by the researchers that should characterize a “natural” gestural interface. Therefore, a novel method for the gesture-to-function mapping is presented in this thesis. This method allows reducing the number of different gestures to be associated to the functions in order to create gesture taxonomies that can be easily learnt
64
by the users. This method involves the implementation of a framework for the context-aware gesture recognition.
Section 2.4 analyzed the concept of ubiquitous computing, which is the fundamental principle that is inspiring the current research in computer science. The ubiquitous computing is the general frame where the modern research about gestural interfaces is situated in. In particular, three different paradigms were identified for the design and development of gestural interfaces: environmental, wearable and pervasive. These paradigms have different advantages and disadvantages; in this thesis, a particular type of pervasive paradigm is presented, which aims at opportunely mixing the wearable and environmental paradigms in order to merge the advantages of these paradigms. This paradigm is applied to the gesture recognition in smart environments of the ubiquitous computing era.
Section 2.5 presents the different solutions for the dynamic gesture recognition in smart environments that are present in the scientific literature. The literature shows that the most used approach for the dynamic gesture recognition in smart environments is vision-based. Unfortunately, this technology has a particular problem, which is the dependency between the user position with reference to the camera and the recognition accuracy. Usually, the user position severely affects the system performance reducing the gesture recognition accuracy. A smart environment should enable the user to interact with full freedom of movement and for this reason some researchers presented some solution in order to develop a technique that can leverage a view-independent gesture recognition system. Some solutions are interesting but the best performance is provided in (Holte et al., 2010), where the authors developed a technique that allows recognizing four arm-gestures with good accuracy allowing the user to freely change position (i.e., varying the user’s angulation with reference to the optical axis of the camera lens) between -45° and +45°. In this thesis, a novel techniques based on two calibrated depth cameras is presented in Chapter 4;
65
this technique allows recognizing deictic and dynamic gestures with an augmented freedom of movement comprised between -90° and +90°.
Section 2.6 reports two examples to show the relationship between the gesture meaning and the context. The role of context is fundamental to decode the meaning of a specific gesture in the human society; this is also true in the interaction with smart environments. Indeed, Section 2.7 defines the meaning of context in computer science and presents the models present in literature to develop context-aware systems. This analysis focuses on the frameworks for the development of context-aware gestural interfaces. In particular, Greenberg et al.’s work addresses this issue providing a model to describe the spatial relationship between the different entities that are present in the same room (i.e., users and interactive screens) as shown in (Greenberg et al., 2011). Althoguh this approach is very effective for the development of context-aware touch-enabled surfaces, it does not take into account a distance interaction with gestures performed in the air. Since there is a lack of a high-level framework that can be adapted to all different types of gestural interaction (i.e., touch gestures, tangible gestures and gestures performed in the air), this thesis presents a novel framework for the modeling of gestural interactions in smart environments of the ubiquitous computing era. This framework aims at facilitating the development of gestural interfaces with a special regard to the optimization of the gestures taxonomies as it will be further explained in Chapter 3.
66