Chapter 12 Measuring Throughput in a Text Entry Task
12.1 Experiment 1 – Fitts 1966 Redux
Twelve unpaid volunteers participated in this study (nine females, three males). They ranged in age from 24 to 40, with an average age of 33.25 years. Eleven were right-handed; one was left-handed (as reported by the participants).
12.1.2
Apparatus and Software
Java software that mimicked Fitts’ (1966) choice reaction time task was presented to the participants on an HP laptop (see Figure 23).
A Java program presented stimuli to participants and time-stamped and recorded their responses to a log file for subsequent analysis. In writing the software particular attention was paid to ensure the accuracy of the final time-stamps. The software’s user interface was designed to be easy to interpret with high stimulus- response compatibility (for example, the trial being displayed in Figure 24 is instructing the participant to simultaneously depress the “x”, “v” and “n” keys). Most of the software’s interface used a grey colour scheme; the activated stimulus indicators, however, displayed red letters on a bright yellow background, so as to be highly visible and stimulating.
Figure 24 - A screen-print of the Fitts (1966) Redux Experiment software
12.1.3 Procedure
Prior to performing the experiment, each participant received an orientation session in which the task was explained verbally. Some survey data was collected regarding the participant including their handedness, age, gender, and whether they frequently played any musical instruments, or video games. During the orientation session, the purpose of the study was explained to the participants as “an
investigation to determine how their rate of errors changes as they speed up or slow down”.
Participants were seated before the laptop and encouraged to make themselves comfortable. Four keys on the keyboard (two for each hand) were used for the participants to make their responses. The participants’ first and second fingers (not thumbs) of their left hand were positioned gently on the “v” and “x” keys respectively, and correspondingly the first two fingers of the right hand were positioned over “n” and “<” keys.
Each trial began with a randomly generated stimulus being presented to the participant via the stimulus indicators. The participant responded to the stimulus indicators by simultaneously depressing from 1 to 4 keys attempting to match the keys indicated by the stimulus. The stimulus, participant’s reaction time and response were recorded to a data file by the software. The software carefully monitored the timing of the participant’s keypresses – to be considered a single response, the necessary keys had to be depressed within 50 milliseconds (total time from first to last keypress). If this limit was exceeded, then the trial was marked as “Time Exceeded”, and the same stimulus would be presented to the participant again in the future in a random order; thus, “within time limit” observations for all stimuli were assured. Precisely 2 seconds after the participant had released the keys from the previous trial, the next trial would commence
A block of trials consisted of 200 trials. The software would pause every 25 trials to allow the participant an opportunity to take a break.
12.1.4 Design
Following the participant’s orientation, the participants performed a single block of 200 trials to familiarise them with the task. The participants were informed that the first block was a “throw-away” block performed solely for their practice.
Participants were encouraged to find their own speed – a comfortable pace that was their own.
Following the practice block, participants performed one block of 200 trials in the Normal condition. They were told that they would be asked to go faster and slower than this speed in future blocks, and that they should proceed at their own comfortable pace.
Following the normal block, Speed and Accuracy conditions (each consisting of 200 trials) were presented to the participants in a balanced fashion. So, all participants performed the practice and normal blocks first, then half performed the speed condition followed by accuracy, and the other half performed accuracy then speed. For the accuracy condition the participants were instructed to proceed as quickly as they could, but slow enough that they could achieve no errors. For the speed condition, participants were told that they would have to proceed at a rate that was at least 10% faster than the average speed they achieved in their normal condition block. In practice the participants had no trouble speeding up, most achieving more than a 20% speed increase.
All four blocks, practice, normal, speed and accuracy, were run in a single sitting.
Following some data analysis from the early participants, it was decided to add an extreme speed condition, called the Turbo condition. The reason this condition was added was that the data were not exhibiting the “linear relationship” that Fitts had observed, and we realised that the data points tending toward the centre of Figure 19 represent points of extreme speed with a correspondingly high error rate. We decided to see if our subjects could be made to exhibit the same behaviour. As this Turbo condition was added after the experiment had begun, not all participants could be run for this condition, and this condition was run at a separate occasion (a couple of days later). In total, 8 of the original participants performed the turbo
condition. This condition was very difficult to execute, as it represents an exceedingly high speed with a correspondingly high error rate that made all of our subjects uncomfortable. It was only with much cheerleading and cajoling on the part of the examiner, that the subjects could exceed the speeds they had achieved in the speed condition. The turbo condition data points definitely represent a speed of performance far beyond what anybody would naturally do.
12.2
Results
Because the 4-bit stimulus signals and response chords were generated randomly and statistically independently, it was possible to calculate exact values of the informatic quantities of interest, source entropy, equivocation and throughput for this experiment. Source entropy was calculated by applying Equation 23 to the frequencies of the stimulus signals. Throughput was found by applying Equation 26. Equivocation was calculated by finding the conditional entropy of the responses given the stimuli; Shannon (1998, pages 20-21) provides the necessary formula,
(
)
(
)
( )
( )
2( )
( )
2( )
1 1 1|
,
,
log
,
log
N N N i j i j j j i j jE H S R
H S R
H R
p s r
p s r
p r
p r
= = ==
=
−
⎡
⎤
⎡
⎤
=∑∑
⋅
⎣
⎦−∑
⋅
⎣
⎦
(40)where si and rj denote the individual source and response signals, respectively, and p(·) represents the probability of observing that particular stimulus or response.
A)
0 1 2 3 4 5 6 7 8 9 0 2 4 6 8 10 12 14Source Entropy, I (bits/s)
E q ui vocati on, E ( b it s/ s)
B)
0 1 2 3 4 5 6 7 8 9 0 2 4 6 8 10 12 14Source Entropy, I (bits/s)
E qui voca ti on, E (bi ts/ s)
Figure 25 - Results Data from the Fitts (1966) Redux Experiment
Part A of this figure shows the data from the first session (including only the normal, speed and accuracy conditions). Part B of this figure adds the data from the turbo condition as well. The axes of the two graphs have deliberately been kept identical to one another, and square (i.e., 1 bit/second along the horizontal axes is represented by the same distance as 1 bit/s along the vertical).
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 4 5 6 7 8 9 10 11
Source Entropy, I (bits/s)
Equivocation, E (bits/s)
Figure 26 - A Closer Look at the Results from the First Session of the Fitts (1966) Redux Experiment
This figure shows the data from the first session (including only the normal, speed and accuracy conditions) but the axes have been chosen so as to provide a more detailed view of the data.
0 2 4 6 8 10 0 2 4 6 8 10 Equivocation, E (bits/s) Thr oughp ut , R ( bi ts/ s)
Figure 27 - Data from Experiment 1 Plotted as Throughput versus equivocation for comparison with Fitts’ (1966) Data (see Figure 19)
The discussion of this data is postponed until after the presentation of the data from the second experiment.