Tasks as Stream Content

Mapping tasks to streams is a conceivable alternative to a view-stream design. The dis- play could render every task as an audible stream with acoustic properties distinguishing it from all other tasks. The problem with such as design is that the possible number of running tasks is unbounded. Ensuring an unlimited number of streams are uniquely identifiable by their positions, loudness, pitch, timbre, and other acoustic characteristics is difficult, if not impossible. Even if an auditory display could provide the necessary distinction of streams, users would still have a hard time identifying a particular task- stream from its potentially numerous counterparts based on relative judgments of its properties (Miller, 1956).

Tasks are better considered content objects within an information space, accessible through a finite number of view-streams. In such a design, the primary view and preview could act in concert to give both detailed and summary information about the active task. The overview could provide persistent reminders about the identity of the active task. The peripheral view could report on the state of the remaining, inactive tasks. The benefit here is that the number of potential streams is fixed by the number of views while the user is free to access an unlimited number of tasks (Table 3.4-18).

The display must support navigation among the various running tasks. Commands for starting, resuming, and ending tasks could provide such navigation. When a task starts, it might immediately become the active task, the focus of the primary view, and the target of all user input (Table 3.4-19). When an inactive task resumes, the currently active task might move to the periphery while the inactive task enters the primary view. When a task ends, the display could remove it from all views and then resume the most recently visited task. Meanwhile, the overview might encode the execution of these commands using a demarcation sound separating tasks. For example, the overview could use earcons with timbre encoding the kind of task (e.g., entering information, reading a document) and melody giving the type of change (e.g., start, stop, resume) (Brewster et al., 1993).

The display must also provide a method of directing start and resume commands at a particular task. One possibility is to send all resume task commands to the last task to make an announcement. For example, if a Web browser announces that a download is complete, the next resume task command might activate the Web browser. Unfortunately, this design does not account for the case where two streams give output simultaneously for two different tasks. Continuing the same example, assume an email program announces an incoming message at the same time as the download finished report. In this case, it is unclear whether the next user resume command should activate the email client or Web browser. A slight improvement requires that the user select which

concurrent stream is the target of the command before issuing it. However, the moving target problem remains: another event can occur in the stream before the user reacts to the one of interest. Enabling the use of the resume command during a review solves the timing problem, but this mechanism limits the user to picking tasks that already exist and have made recent announcements.

Presenting the user with a list of selectable named tasks mitigates all of these prob- lems. The display could consider the act of selecting a task a main menu that the user can activate using a special command (Table 3.4-4). When the menu is active, the dis- play might treat it like any other task in the primary, preview, and overview streams. The primary view could announce the names of the available tasks. The preview could give a summary of each task. The overview could play a continuous sound unique to the menu, and easily recognizable among all other overview sounds. For example, a musical, rhythmic sound can fit this purpose and contrast with the non-repeating, environmental sounds for other tasks. Because the user is likely to browse such a fixed menu for brief periods of time, a rhythmic, looping sound is not likely to become too distracting.

The cost of using an explicit task menu for navigation tasks is that it introduces a modal element into the display. The user must leave his or her active task to visit the menu. The design of the display must weigh the ability of this solution to provide a single, consistent, time-independent method of starting and resuming tasks against the size of the interruption it causes in the user’s workflow. Still, an explicit, modal menu is likely to be more usable than forcing the listener to synchronize his or her commands with output from one of many streams.

Browsing Actions

The primary view should take responsibility for reporting text content and properties as the user browses information within a task (Table 3.4-1). Navigation by character could result in a spoken announcement of each character, and, depending on the verbosity

preferences of the user, whole words as they are encountered. Navigation by words might produce spoken reports of each word in the text. Navigation by chunk could result in a report based on the task-dependent definition of a chunk. For instance, in the task of browsing emails in a mailbox, the display might reasonably define a single message as a chunk. In this case, a user might hear the sender, subject, and other header info announced in the primary view (Table 3.4-12).

The preview should give the user an idea of where he or she is browsing in the current task, and how much content remains unexplored. For instance, a preview of an email folder could report the index of the currently selected message and the total number of messages. Such information helps a user decide on the most efficient way to go about completing a task. If, for example, a mailbox contains five unread messages, the user might prefer to read them all. On the other hand, if the mailbox has five hundred new messages, the user might wish to first prioritize or filter them before traversing them.

The peripheral view should connect the content the user is browsing in the active task to relevant content. For instance, the peripheral view could summarize where a hyperlink leads when the user passes over it on a Web page. Likewise, as the user browses appointments on a calendar, the peripheral view might mention other appointments on the same day.

Searching Actions

The primary view should also take responsibility for supporting searches within a task (Table 3.4-2). The primary view could give immediate feedback when the user begins defining search terms. One primary view stream might announce the first word or chunk matching the search query every time the query is updated. A second primary view stream could confirm the terms the user has already entered. At the same time, the preview could summarize the position of the current match within the entire body content. The searching functions could respect commands for navigating forward and

backward through search matches, and provide the same level of feedback after each movement. Together, these features should provide a means for filtering content so that it may be traversed by points of interest (Table 3.4-6).

Switching between searching and browsing must be transparent to the user (Ta- ble 3.4-3). At any point during browsing, the display should allow the user to initiate a search by defining search terms. Likewise, when a search reveals a match of interest, the display must allow the user to start browsing content at that location. The au- ditory display should make this transition seamless such that output during browsing and searching is similar, if not exactly the same. For instance, the primary view could report the name and details of a file in the same manner regardless of whether the user reached the file by browsing or searching. The preview, however, might make a verbose announcement when searching and no report at all when browsing. Searching for a file could cause the user to skip hundreds or thousands of files that do not match the search criteria. The display should inform the user of such a large jump. In contrast, browsing from one file to the next is a relatively small change. The display should not overwhelm the user with announcements about such trivial navigation.

Editing Actions

Finally, the primary view should announce changes during text editing. When entering or deleting text, the first speech stream could report words while the second should echo characters. The view streams could play sounds when the user encounters text with special states as well as when special editing events occur. An auditory icon, for example, might represent text acting as a heading in a document, a misspelled word, or emphasized text. Likewise, an earcon might succinctly indicate when the user splits an existing line in two with a line break, or joins two lines into one by removing a break.

Spoken feedback echoing user input assumes that the speech echo is not in conflict with the input method. For example, the echoing of words and characters is appropriate

when input is given using the keyboard. To the contrary, when user input is spoken, shadowing everything the user says with synthesized speech is unnecessary and likely confusing. When input is spoken, auditory icons and earcons might provide effective feedback, for example, about voice recognition confidence.