Figure 42: Gesture capture – Red crosses indicate time-sampled points along the drawn gesture.
3.2 Gestures
We have examined previous work in gesture-based input (Rubine 1991), sketch recognition and beautification of hand drawn sketches (Freeman and Plimmer 2007) (Kara and Stahovich 2004) (Gusaite 2006) in order to decide on an effi-cient and intuitive input mechanism for drawing schematics. For the purpose of our application, only a small fixed set of gestures need to be supported. The application does also not need to provide a method for learning new gestures.
To meet these requirements, we chose to use a number of metrics to identify drawn gesture shapes rather than use a learning algorithm. Figure 43 illustrates the supported gestures to input the various objects defined in Section 3.1.1.
Although full sketch recognition can provide more advanced functionality than gestures by supporting multi-stroke symbols, it comes with a performance overhead in recognition as well as a mechanism to determine when the user has finished one symbol and moved onto the next. A common mechanism to implement this is the use of a waiting time between pen strokes (Gusaite 2006) but this hinders the input flow of the user. Simpler gesture recognition, where a single-stroke gesture corresponds to an object, can provide the required func-tionality whilst ensuring the user is not disturbed by workflow pauses.
Gestures are recorded as a sequence of time-stamped coordinates. As a ges-ture is drawn, SchemaSketch will record the current position of the touch mo-tion ten times a second. Figure 42 shows a visual example of this sampling pro-cess upon a junction gesture. Red crosses indicate sampled coordinates, starting
from p0(the initial point of contact) up to the final point pn. Although some ac-curacy of the gesture is lost, providing a high enough sample rate is used there is still enough information to determine the drawn gesture. Using these sample coordinates, SchemaSketch will attempt to recognise the gesture based on the metrics explained in the following subsections.
3.2.1 Minimum direct length to be classified as an edge
Direct length refers to the absolute distance between the start and end coordi-nates of the gesture. For a gesture to be an Edge object, this distance must be
≥45 pixels.
3.2.2 Minimum straightness to be classified as an edge
The straightness of the gesture G is measured using Equation 7.
straightness(G) = dist(Gstart, Gend)
actualLength(G) (7)
where actualLength(G)is calculated using Equation 8.
The straightness calculation will produce a value between 0 and 1. A value of 1 is a perfectly straight line. For a gesture to be classed as an edge,
straightness(G) must be ≥ 0.9. If a gesture passes the minimum direct length test and minimum straightness test, it can be classified as an edge, otherwise it is potentially either a junction, station, or bend point. Differentiating between these last three is performed by the following four features.
3.2.3 Minimum actual length to be classified as a station
Actual length refers to the length of the gesture if it was straightened out, and is calculated using Equation 8, where n is the number of points in the gesture and piis the ithsample point along the gesture.
actualLength(G) =
n−1 i
∑
=0dist(pi, pi+1) (8)
actualLength(G) must be ≥ 10 pixels for the gesture to be evaluated. This means any gesture shorter than 10 pixels will not be recognised and nothing will be added to the schematic. This is useful for discarding unintentional screen touches.
3.2.4 Minimum straightness to be classified as a station
The straightness is once again checked using Equation 7 and if straightness(G) ≥ 0.5 then it will be classified as a station. Although stations and edges are both straight line gestures, edges require a higher straightness(G) value because the longer a gesture, the easier it is to obtain a high straightness(G)value.
3.2.5 Minimum number of sharp bends to be classified as a bend point
The number of sharp bends along the length of the gesture is measured using Equation 9. If sharpBends(G) ≥1 then the gesture is classified as a bend point.
A sharp bend is defined as a difference in edge angle that is ≥ 100◦ between any two adjacent edges in the gesture.
sharpBends(G) =
n−2 i
∑
=0( 1 if angle(pi, pi+1, pi+2)≥100
0 else (9)
3.2.6 Minimum average radius to be classified as a junction
If there are no sharp bends, this last check is performed to identify a junction gesture. We calculate the average radius of the shape (we know the shape is curved, as straightness(G)is low). First we calculate the centre point of the ges-ture, by averaging x and y co-ordinates across all points. We can then calculate the average radius using Equation 10, where n is the number of points in the gesture and piis the ithpoint along the gesture.
radius(G) = ∑
ni=0dist(Gcentre, pi)
n (10)
(a) Junction. (b) Bend. (c) Edge. (d) Station.
Figure 43: Schematic object gestures. Gestures start at the×.
If radius(G) ≥ 10 pixels, this gesture can now be classified as a junction, oth-erwise the gesture will not be recognised and nothing will be added to the schematic.
These rules result in junctions being drawn by a circular shape with start and end points close together (Figure 43a); bend points by a line across an edge with at least one sharp bend (Figure 43b); edges by a long, straight gesture (Fig-ure 43c); and stations by a short, straight gest(Fig-ure across an edge (Fig(Fig-ure 43d).
Figure 44 illustrates the gesture interpretation process in a flow chart.
Although no studies were performed to objectively evaluate the effective-ness of our input mechanism, during extensive personal use it has proven to be an effective method for the construction of metro map schematics. Gestures are interpreted accurately and allow quick and easy schematic creation, providing a solid proof of concept for gesture-based schematic construction.
There are a number of accessibility issues associated with touch and gesture-based input systems, for example for individuals with visual or dexterity dis-abilities. Our current implementation system does not provide additional sup-port for these users, however we have ensured that the gestures used for cre-ating schematics are as simple and intuitive as possible. There are also no time limitations on gesture input, meaning the system will read gestures with iden-tical accuracy when drawn at any speed. As mentioned later in this chapter, implementation difficulties prevented the addition of zoom and scroll function-ality, however this is another feature that could greatly further aid users who may find it difficult to use the gesture input system accurately.
Figure 44: Gesture interpretation flow diagram.