Examples of Applying the Framework

One of the key contributions of the frameworks is its capacity to enable relationships to be formed for the individual elements of the framework. Each parameter setting of the framework will have an effect on the others, and the relationships that are formed based on those settings supports a more informed process. There are two levels of relationships

Figure 6.1: The diagram shows the fundamental relationship structure that can be

drawn from applying the framework to individual systems for specific applications. The outer circle shows how we would approach relationships between element of an individ- ual system. The inner circle represents different theories in HCI, and demonstrates how they can be incorporated into the framework for more general applications to designing

gesture systems.

building that the framework supports; a specific level, where individual systems and the relationships between the parameter settings can inform design, and a general level, where existing HCI theories and methods can be incorporated into the framework.

Specific relationships. In Figure 6.1, the outer circle represents the process of examining each parameter within the categories with respect to the other, and forming an understanding about the different effects that can result. For example, we can look at the scenario that was used in the experiment presented in Chapter4, and build a table to specify the different relationships that can be drawn between the variables considered in the experiment. This is demonstrated in Table6.1 and discussed next.

General relationships. Figure6.1also allows for broader theories used in HCI to be applied to the design of gesture interactions within the structure of the framework. For example, the inner circle of Figure6.1suggests that the task artifact cycle, as introduced byCarroll & Rosson(1992), can be incorporated into the design of gesture systems under

the categories of application domain to represent tasks, and system response to represent artifacts. Thus, the existing methods and approaches within the task artifact cycle can be incorporated into the design of a gesture system under this framework. Additional theories can be developed within the structure of the framework as well. The cycle that is shown in Figure 6.1 that runs between gesture and input could potentially be extended to form a similar theory as the task artifact cycle, only with reference to the gesture-input cycle. We do not discuss this further, but suggest that this would be a relevant direction for future research.

6.2.1 Assessing System Performance

The framework can be applied as a tool to inform on the potential effects of using different settings of the parameters in a system design. In addition, systems that are under design consideration can be evaluated using the values provided in this framework to rate the design before it is implemented. Table 6.1 demonstrates how metrics that describe the parameters can be used to provide comparison ratings of different system implementation. Metrics that are used to measure values of parameters can be set and cross referenced using a table format, to determine what an optimal configuration would be based on the information provided for those parameters. For example, in Table6.1, we use four parameters to evaluate a potential gesture interaction system design (error rates, input mode, system response and task characteristics). By cross referencing each of the parameters against the others, we can determining a cumulative rating, based on the results of those combined parameters and settings, that can be used to predict the performance of that hypothetical system configuration. We can apply the information from the table in the following manner:

1. Select a parameter and setting from the row down the left side of the table (error rate = high).

2. Look along the columns on the top of the table, and locate the value for the first parameter and find its corresponding value for the appropriate value in the system response column (system response = fast).

3. Find the value to determine the rating for this particular configuration (med).

This process can be performed for all the parameters and their settings, and then assessed based on the collective ratings. In the above example, we see that a system with high error rates, and fast system response can at best provide mid-range interaction benefits. If we repeat the process and also consider task criticality and error rate, we get an additional rating of medium. We can then combine our collective ratings to determine how the system affects the interaction, making changes to reflect our design or goals. We can extend this table to incorporate the features of the interaction that designers are

Table 6.1: This table presents several task characteristics and ratings to suggest their appropriateness for use in the given scenarios in relation to the other parameters that were evaluated in the experiment conducted in Chapter4. The ratings are based on user feedback and results from the experiment, where we set low, medium, or high

values based on the effects of the different settings for the variables.

considering, and use the resulting values from the framework to build the table. This is only one approach to using the framework, others can be adapted to a specific project or approach as required.

A new perspective for old research We propose that the framework can enable existing research to be incorporated into its current structure. For example, if we con- sider previous research by Brewster et al.(2003), who investigated gestures for mobile computing, we can apply the framework to impose a structure onto the existing system, and begin to understand its relationships with similar systems. To demonstrate, we demonstrate how we would apply the framework to describe Brewster work as follows:

• Gestures: Style:Semaphores, object:none, Set:Unspecified

• Application Domain: Context:mobile, multitasking. Tasks:non-critical

• Enabling technology: Touch Screen, Interaction zone:0. Mobility:1

• System response: Audio output, visual display (PDA), audio recognition feedback on,

Though we have used a reduced set of parameters to describe the system, we can chose to provide a much more in-depth analysis of the system provided that we have access to the information. Upon examination of the results obtained in Brewster’s study, we see

that here, semaphoric gestures reduce the level of distraction during while multitasking, and support eyes-free interactions. With this information in place, we now turn to a second application of the framework.

6.2.2 An old perspective for new research

If we again look at Brewster’s work, we learn that touch gestures offer similar benefits to vision, however touch interactions provide greater accuracy than computer vision, and can potentially recognise a greater number of gestures. While it is not clear if we can always transfer the results from one interaction scenario to another, in this case, we see that one of the main benefits of using semaphoric gestures applies across interaction domains and enabling technologies. Of course, additional factors should be considered within the context of the interaction, and there are additional features that were not discussed in the research, however we do gain a general sense of the ability to view different systems within the framework parameters. To apply the framework, we consider our hypothetical system described in Chapter 4, which has a similar arrangement to Brewster’s system:

• Gestures: Style:Semaphores, object:coloured objects. Set:5 gestures

• Application Domain: Context:ubiquitous, multitasking. Tasks: non-critical

• Enabling technology: Touch Screen, Interaction zone:1. Mobility:1

• System response: Audio output, visual display, recognition feedback: audio

If we want to improve our system, and enable a greater set of gestures, then we know that touch also provides minimal distraction, and can consider upgrading our system along these lines. We would also have to consider the trade-off of the interaction zone of vision for the for accuracy of touch, however this example demonstrates how we can make specific improvements and changes to systems based on comparisons with similar systems.

6.2.2.1 Framework Validation, Extension and Directions

An important contribution of the framework are its ability to proved a structure from which to inspire new directions for research. Within specific application domains, we can extend our research to consider interactions with individual systems. For example, while part of our research focus is on secondary task interactions using gestures, we noted that notification systems are also concerned with reducing distraction to tasks during multitasking situations, with a focus on the attention-utility theme to reduce the level of attention required from users while increasing the utility of that interactionMcCrickard

and test if alternative interaction modes could affect notification system interactions. We discuss this experiment in section6.3. We also present an experiment to test propositions about reflexive feedback and add knowledge about this parameter, in section 6.4, and finally, we discuss new directions for the framework in the application domain of CSCW and details of a formative study we conducted to investigate strategies for group gesture interactions in section3.1.