Seeing the wood for the trees: A framework for the specification of metaphor in interface design

Seeing the wood for the trees: A framework for the

specification of metaphor in interface design

Simon Crowle

Royal London House Old Christchurch Road

Bournemouth England [email protected] tel: +44 1202 702755 fax: +44 1202 595314

Linda Hole

We examine the influence of metaphors in user interface design and its treatment with respect to the interface specification stage of the software development process. Recent advances in computer hardware places inexpensive and powerful graphical technologies in the hands of interface designers who will no longer be bound within the constraints of the ubiquitous desktop metaphor. However, effective interactive systems development requires a multidisciplinary approach, integrating a variety of design concepts and notations. Currently, specification and integration of metaphor design is weak and informal. A metaphor specification language is proposed to support the development of new and innovative forms of human-computer interaction. The de-coupling and abstraction of metaphor from contemporary component-based specification of interfaces is discussed.

KEYWORDS: Metaphor, interface specification, integration

INTRODUCTION

History repeatedly shows the gradual rise, evolution and decline of a variety of interaction paradigms; command line and menu-driven dialogs are examples of interactions subsumed by the modern graphical user interface (GUI). Xerox's Star Interface [22] is widely regarded as the precursor to this interaction style, later made popular by Apple's Finder interface and closely followed by many other personal computer hardware and software vendors. User interfaces found on personal computers today have evolved from a legacy of progressively refined designs - a process that is continuous and on going. However, substantial ‘macro scale’ changes in the current interaction paradigm have not occurred since the introduction of the ubiquitous desktop user interface. The creation of the GUI was a synergy of new hardware technologies and a then radical approach to designing software for users who were not part of the computer science fraternity. Whilst major computer operating system vendors are still innovating in the extant paradigm, many of the changes that we see are more cosmetic than conceptually ground breaking.

We are about to see history repeat itself however: unprecedented graphics technology, particularly in the field of three-dimensional imaging, is sufficiently powerful and inexpensive to foster an entirely new human-computer interaction paradigm, recent experimental forms of which can be found in the literature [4][20]. What remains uncertain is what final form this new interaction will take, how existing users will adapt to it and what (if anything) will happen to the desktop metaphor experience of today.

Here, we examine the influence of metaphors in user interface design and its treatment with respect to the interface specification stage of the software development process. It will be argued that current metaphor design practice, if it is active at all, is a craft-based, impoverished and foreign part of the formal software design specification (and consequent development). Additionally, we will examine the need for an abstracted and de-coupled description of the interface metaphor and a framework for its integration into existing HCI design methods. Finally, we briefly overview ISML (Interface Specification Meta-Language) - a specification language in development that attempts to address these issues. ISML has been developed to support the specification and development of metaphorical and innovative forms of interface design.

In natural language, metaphors frequently form the basis of a description or explanation of real world phenomena where otherwise such concepts would be difficult to express [7]. In an attempt to make computers accessible to the general public, every day objects and actions used in tasks that are to be automated by the computer are borrowed and represented at the computer interface to reduce the ‘conceptual gulf’ between the user and the computer system [18]. Today, these objects and actions have settled down into a relatively stable, if rather mixed and partial set of metaphors for users to adopt as a means of understanding how the computer supports their particular tasks. This ubiquitous desktop metaphor is now an implicit part of the software developer’s toolkit, providing a library of ‘off-the-shelf’ components (such as icons, buttons, windows and menus) that can be

rapidly assembled and compiled. As we shall discuss later, whilst this arguably reduces software production time, it ultimately leads to a narrow view of interface design and stifles innovation.

DESIGN PRACTICE

Despite lacking a robust and established theory of how metaphors contribute to the usability of a user interface, they continue to pervade the thinking of interface innovators in a variety of different problem domains. The use of metaphor has been employed in a number of problem domains including programming environments [11], information systems [4], hypertext systems [23] and computer games. Designing good metaphors for human-computer interaction is difficult. A number of guidelines and case-studies can be found in the literature [15][10][23] as well as evidence to suggest that metaphors do not play as strong a role in supporting usability as we’d like to think [9][17].

Development of effective interactive systems requires an interface design team whose members comprise of individuals with expertise in psychology, usability engineering, graphic design and software engineering [14]. Throughout the design process, a sample of the expected users of the finished product will be repeatedly consulted and help evaluate the product. A large number of methodologies have emerged during the maturation of the HCI discipline that attempt to formalise this design process, the proper treatment of which is beyond the scope of this paper. Instead, we shall focus on some of the common models that are produced from existing HCI methodologies and examine how they inform the early stages of design: requirements and specification. It is important to emphasise that HCI practices should be applied throughout the software design life cycle and is not just an evaluation exercise conducted toward the end [1]. Therefore it is imperative that metaphor and interface design is addressed early and

continuously during development. However, as we shall

argue later, metaphor design is difficult and is either frequently ignored or implicitly subsumed within the narrow context of a specific application running in a particular GUI environment.

EXTANT UI SPECIFICATION METHODS

Development of the user interface is a synthesis of a number of different design perspectives including task, user and problem domain modelling as well as architectural descriptions of user interface presentation and dialogue control. Treatment and emphasis of these entities in user interface design varies widely in the literature, however a number of attempts have been made to normalise and integrate these views of interface design. Many practitioners in the field outline methods of eliciting task and domain models (see [5] and [21]); An evaluation of the products (modelled entities) of a variety of task analysis methods has also recently be summarised [2]. Toolkits have also been developed that either support a particular modelling paradigm’s [12] or integrate a few of these models [21] and in some cases also allow development of a prototype interface [19].

Traditionally, abstractions of the presentation and dialogue components of interface design have remained only loosely coupled with the ‘user-oriented’ models already described. It is common for prototypes to take the form of paper-based, annotated storyboards or ‘throw-away’, executable mock-ups developed with multimedia presentation tools such as Macromedia’s

Director or other dedicated prototype builders such as SILK [13]. Whilst this technique allows both developers

and users involved in the design process to rapidly visualise design solutions, it provides only an informal description of the intended interface design; very little in detail is stated regarding the underlying metaphor model that supports these solutions.

As the means of specifying human behaviour in the HCI discipline has matured, so have the formalisms for modelling the presentation and behavioural responses of the computer to the human interaction. Early frameworks such as Seehiem and Arch [8] provide a high-level of abstraction of the transformation of physical interaction with the hardware to the functional core of the computer system. More recent work in the literature demonstrates a much more refined and diversified treatment of this problem. Abstractions are expressed in many different forms including algebraic specifications, petri nets, Backus-Naur form [3] and ‘interactor’ compositions [16].

Mappings between the ‘real world’ of user, task and domain modelling and that of the abstracted user interface mechanics are being developed. TRIDENT [21] and MOBILE [19] are toolkits that combine domain and task models to generate conventional GUI interfaces in this manner. Toolkits such as these allow the design cohort to explicitly state the relationship between the models that the design process produces and a generic set of presentations and interactions for that problem. This is an implicit treatment of metaphor design since

the solution is embedded within a generalised human-computer interaction paradigm (in the above cases, the

conventional GUI). This is not surprising; using an established set of interface components makes compilation easier and is likely to be an easily recognisable set of concepts for the end user. However, there are a number of inter-related problems with this approach that will hinder innovation in interface design, particularly with regard to metaphor generation:

Scalability And Reusability

At present, the particular machinery of buttons, menus, windows and other components for a specific problem is a ‘custom made’ specification. That is to say that all the components specified have both their metaphoric and

application specific abstractions bound together. For

example, when we specify a button that saves a document to a pre-defined destination, we specify its appearance, behaviour, the message types it sends and receives from other components and its particular semantic behaviour relative to the problem in the same logical space. For all but the most trivial of designs, it is

clear to see that specifying the behaviour of an application in this fashion is a daunting task. In addition, specifications are not easily reused; the appearance and behaviour of the component is closely tied to the problem domain, blurring the distinction between the metaphor and the assembly of GUI components that provides a solution (we will revisit this issue later).

Subtle Behavioural Variance

Seemingly elementary interface components, such as buttons or icons, have subtle behaviours that may be over-looked using an informal view of the contemporary graphical user interface. In fact, in some systems, both these components react subtly with other elements of the environment (such as the desktop and the mouse pointer). Closer examination of focusing behaviour, for example, reveals that a mouse click does not necessarily have to directly hit an icon to focus it. Similarly, hidden behaviours specific to certain GUIs and not others (such as menu-item moving in some versions of Microsoft

Windows) may produce unexpected behaviour for

some users of the system.

Implicit Treatment Of Metaphor

Lastly and most importantly, contemporary specification techniques implicitly define the nature of the metaphor employed in the design solution’s interface. This is perhaps a symptom of our prolonged exposure to the ubiquitous desktop interface – we can no longer see the wood for the trees. Implicit definition means that is difficult to examine and evaluate the design of any particular metaphor in isolation. This means we have no formal method of examining how a metaphor lends itself to a) the problem domain and tasks the user already understands and b) the technical solution (both hardware and software) that will be used in development. One other consequence of the blurred distinction between the user interface solution and the underlying metaphor is the difficulty in being able to express design solutions that do not rely on the iniquitous desktop convention.

ISML FRAMEWORK

We have argued the case for a more rigorous and formal approach to metaphor design that can be integrated with existing HCI development methodologies and formalisms. It is important to state here that we are not proposing a unified development methodology and notation encompassing all design issues. Indeed, it would be naïve to imagine that such a tool could possibly exist, at least in the near future. Instead, we propose an interface specification framework within which some of the commonly used HCI design concepts can be integrated with a more formal description of user interface metaphor.

To this end, we present a high-level overview of the Interface Specification Meta-Language (ISML) that attempts to address the issues discussed above. Space does not permit a detailed discussion of ISML, so we will focus on its treatment of metaphor as a part of the interface specification. The ISML breaks down an

interface specification into five categories (figure 1 depicts their mappings):

• Device definition

• Component definition

• Meta-object definition

• Interactor definition

• Task definition

We will review all but the metaphor definition (declared in the meta-object section) in only brief detail.

Figure 1: ISML framework

Device Definition

Input/output devices, such as the keyboard, mouse and graphics adapter are abstractly defined in terms of a) data types in-coming or out-going and b) available functions - declared from a pre-defined, generic library.

Component Definition

This category can be broadly thought of as the presentation definition of the interface; at run-time, components are rendered resulting in either some output displayed or some input stored locally in the component instance. Components may be assigned attributes, more than one set of rendering instructions and may also maintain a state-chart.

Interactor Definition

The interactor definition specifies an interface design solution within the context of a metaphor-based environment (described later). Interactor types are based on one meta-object abstraction, combined with one or more display and controller part abstractions (all defined in the meta-object definition). ISML allows the definition of interactor-style entities such as can be found in the literature [16]. Design solutions are then specified as an inter-communicating network of interactor instances. Interactors may contain private attributes, a state-chart and event-handling routines (ISML uses a sub-set of the C programming language for procedural operations).

Task Definition

ISML allows a simple, hierarchical definition of tasks using an HTA-style specification [6]. Abstract objects, actions and events in the ‘task world’ are defined using meta-object notation and then used in description of the HTA. In addition to the usual iteration description, a sub-task may be iterated until a particular state of an object class is reached. Finally, objects and actions declared in the task definition are explicitly mapped (in a

D ev ices M e ta -O b jec ts C o m p o n e n ts In te rac to rs T asks

sub-section called ‘metaphor’) to interactor instances defined previously.

Meta-object Definition

Before proceeding with a more in-depth examination of the abstracted metaphor specification, it is important to note that the framework used here is not intended to represent any psychological theory of metaphor. The philosophy behind ISML is to integrate existing HCI design models with an explicit abstraction of the underlying objects, actions and ‘rules’ that express the

practical effect of a particular metaphor design. ISML is

a marked-up, textual specification that can be transformed to produce code that can be compiled by Inprise’s C++ Builder version 4 to produce an executable prototype.

The meta-object definition is core to the ISML specification; it abstracts a world of objects, constraints, actions and events that forms the basis for the underlying metaphor of the GUI. In addition, the meta-object definition declares display and controller parts that are then bound to object abstractions and used as interactor class declarations (space does not allow this to be covered here). Meta-object definition is divided into two sections: semantic and syntactic (see figure 2).

Figure 2: Components of the meta-object definition Mapping-constraints are descriptions of either a) some direct or translated mapping of one object’s attributes to another or b) a set of conditions that one or more attributes of an object can be tested against (with an optional procedural outcome if a condition is violated). Object abstractions include a set of attributes and a state-chart. Additionally, a whole or partial set of the mapping-constraints described above are specified as either potentially affective by the object or effective to the object. Each mapping-constraint specified for that object has a set associated with it containing (at run-time) references to either the objects it is affecting or the objects that are affecting it. Special tests can be made on whole or parts of these sets to determine whether they satisfy the particular constraint. In this manner, we can specify rules about the metaphorical world we construct; for example it is possible to say, “A trash-can object may

contain objects” where contain is a reference to a

mapping-constraint. To further refine the metaphor, we can then say, “A file object may be contained by an object”; both parts of the proposition must be declared to

allow a legal containment of the file object by the trash-can object.

Action-events are specified in a similar fashion action; these can be effectively viewed as communications of events that occur between objects. The action-events may have an associated set of parameters passed with them (including references to mapping-constraint sets of another object). Each object may declare use of an action-event either as a sender, a receiver or both. Statements such as “A pointing object focuses an icon” and “an icon may be focused by a pointing object” may then be constructed determining how one object may send actions to another.

DISCUSSION

We have briefly outlined a framework for interface design that places at least as much emphasis on the specification of metaphor as extant GUI component-building development methods enjoy. ISML is currently in development and will shortly be evaluated using a number of case studies. Principally, we are concerned with whether or not the abstraction of metaphor in this manner is a) beneficial to the design process (what insights do we stand to gain from using it?) and b) is such a specification practically feasible in an interface design project? Initial small-scale studies with ISML have brought to light design issues that we hope to explore and share with the design community in the near future. Here, we briefly discuss a few of the principal issues associated with our treatment of metaphor specification.

Platform Independent Metaphor Specification

ISML makes a clear distinction between the metaphorical description of the user interface and its eventual rendering. It is perhaps tempting to think of the desktop metaphor purely in terms of the graphical objects that are presented to the user. However, as we have already observed, the underlying metaphor is frequently more subtle than is readily apparent from appearances. Additionally, many different hardware platforms (including personal digital assistants, legacy fixed character array interfaces and virtual 3D environments) all present the user with common metaphors. The slider bar is a good example of this; we understand the nature of a slider bar as it relates to other components within the specific environment, but our actual interaction with, and perception of it, varies from system to system. We hope to be able to demonstrate that ISML’s abstractions will allow improved cross-platform design and evaluation of the metaphor and application solution combination.

Separating Metaphor From GUI Component Solution

Initial investigations have already demonstrated that, even for simple desktop descriptions, an ISML specification spans several pages. De-coupling the metaphor from the component-based description of the interface design initially appears to generate considerable over-head. The impact of this on a single project or a number of different projects sharing the

Mapping-Constraints Action-Events Object Type 1 Object Type n Display Parts Controller Parts Interactor Type n Interactor Type 1 Syntactic Semantic

same or similar metaphor environments is unclear. In addition, the meta-object definition of this simple environment appears to demand a certain degree of consideration regarding the type of hardware it will eventually be deployed on. For example, if a two-dimensional display is being used to mimic a three-dimensional environment (this is sometimes referred to as the 2.5D desktop display), one cannot avoid abstracting some kind of invisible ‘stack’ associated with the desktop. A three-dimensional rendering of the same metaphor would not require such an abstraction. Again, the trade-off between hardware technology and metaphor abstraction remains unclear.

CONCLUSION

We believe that user interface design will shortly experience a substantial paradigm shift as powerful and inexpensive new interface technologies provide users with new forms of interaction. Up until recently, the interface metaphor has been a colourful but ultimately understated craft-based tool in the HCI developer’s toolkit. If existing and future HCI theory and methodology is to be effectively applied to the next interaction paradigm, we need to promote and integrate a metaphor specification framework that will extend beyond the desktop.

REFERENCES

1. Bevan, N. Quality in Use: Incorporating Human Factors in the software engineering lifecycle. In

Proceedings of the Third International Symposium and Forum on Software Engineering Standards, ISESS’97 Conference, August 1997

2. Breedvelt,I., Paternò, F. and Severiins, C. Reusable Structures in Task Models. In Proceedings of

Design, Specification, Verification of Interactive Systems ’97, Granada, June 97. Springer Verlag

pp.251-265

3. Brun, P. and Beaudouin-Lafon, M. A Taxonomy

and Evaluation of Formalisms for the Specification of Interactive Systems. In Proceedings of HCI’95,

People and Computers X, Huddersfield, U.K.

Cambridge University Press, 1995. pp 197-212 4. Card, S. K., Mackinlay, J.D. and Shneiderman, B.

Readings in Information Visualization – Using Vision to Think. Morgan Kaufmann Publishers Inc, San Francisco, 1999

5. Carroll, J.M. Scenario-Based Design – Envisioning Work and Technology in System Development. John Wiley and Sons, Inc. New York, 1995 6. Dix, A., Finlay, J., Abowd, G. and Beale, R.

Human-Computer Interaction, 2nd Edition. Prentice Hall, 1998

7. Eberts, R.E. User Interface Design Prentice-Hall, 1994

8. Edmonds, E.A. The Separable User Interface. Academic Press, 1992.

9. Gillan, D.J. and Bias, R.G. Use and Abuse of

Metaphor. In Human-Computer Interaction. In

Systems, Man, and Cybernetics 1994.2-5 Oct 2-5

Vol 2. pp 1434-1439

10. Gurak, L.J. Evaluating the Use of Metaphor in

Software Design: A Rhetorical Approach In

Professional Communication Conference, 1991 Proceedings. 30 Oct-1 Nov 1991. Vol 1 & 2. pp

267-271

11. Kahn, K ToonTalk - An Animated Programming

Environment for Children. In Journal of Visual

Languages and Computing (1996), Vol 7. pPp

197-217

12. Khalifa, M. and D. Kira A Graphical Task Analysis Language (GTAL). In Human-Computer Interaction (1992) Vol 31(2): pp 65-79.

13. Landay, J.A. and Myers, B.A. Sketching Interfaces: Toward More Human Interface Design. In

Computer. IEEE 2001. Pp 56-64

14. Lanning, T.R. Let the Users Design! In Taking

Software Design Seriously. John Karat (Ed).

Academic Press 1991

15. Lovgren, J., How to choose good metaphors. IEEE

Software (May 1994) Vol 11, Issue 3 pp 86-88

16. Markopoulos, P. A compositional model for the formal specification of user interface software.

Doctoral Thesis. Department of Computer Science,

Queen Mary and Westfield College, University of London 1997.

17. Nardi, B.A. and Zarmer, C.L. Beyond Models and Metaphors: Visual Formalisms in User Interface Design. In Journal of Visual Languages and

Computing, 1993, Vol 4, pp 5 – 33

18. Norman, D. A. and S. W. Draper (1986). Cognitive Engineering. User Centred System Design. D. A. Norman and S. W. Draper, Lawrence Erlbaum Associates. 1: 31 - 61.

19. Puerta, A.R., Cheng, E. Ou, T., Min J. MOBILE: User-Centred Interface Building In ACM

Conference on Human Factors in Computing Systems (CHI ’99). Pittsburgh May 1999.

20. Robertson, G., van Dantzich, M., Robbins, D., Czerwinski, M., Hinckley, K., Risden, K., Thiel, D. and Gorokhovsky, V. The Task Gallery: A 3D Window Manager. In Proceedings of CHI 2000 Conference on Human Factors in Computer Systems. ACM Press. 2000 pp 494-501

21. Schlungbaum, E. (1998). (Knowledge-based) Support of Task-based User Interface Design in TADEUS. Short paper from CHI’98 .

http://www.uni-paderborn.de/fachbereich/AG/szwillus/chi98ws/schl ung.html

22. Smith, D, Irby, C, Kimball, R and Verplank, B Designing the Star User Interface. In

Human-Computer Interaction. J. Preece & L. Keller (eds), Prentice-Hall, 1990

23. Zajicek, M, and Windsor, R. Using Mixed

Metaphors to Enhance the Usability of an Electronic Multimedia Document. In Human-Computer

Interface Design for Multimedia Electronic Books.