Simple() Select( Text ) Select( Line ) Select(type) ClickSel(type) AreaSel(type)

Interaction in Really Graphical User Interfaces

G.Viehstaedt M.Minas

Lehrstuhl fur Programmiersprachen Universitat Erlangen-Nurnberg Martensstr. 3, 91058 Erlangen, Germany email: ^fviehstae,minas^g@informatik.uni-erlangen.de

Abstract

Employing diagrams in the UI causes problems that don't exist in so called Graphical User Interfaces. We are im- plementing a tool for generating editors for (a certain class of) diagrams from a specication. Diagram Edi- tors are needed in really graphical UIs and interactive vi- sual language environments. This paper addresses three aspects of the specication of diagram editors.

After highlighting diculties in user interaction with diagrams event automata are suggested as a model for specication of dialogs. Furthermore, it is discussed how objects selected by the user can automatically be assem- bled into semantically meaningful groups. These are needed for more complex editing operations. An easy and versatile way of specifying layout for diagram editors by constraints has been introduced in [1] and is generalized here.

1 Introduction

So called Graphical User Interfaces (GUIs) are actually not graphical, but based on widgets. We share the opin- ion voiced in a recentCACM issue on GUIs:

What the market considers a GUI is little more than a gloried menu system, having no graphics. This leaves the graphical representation of the application domain as an exercise for the developer,^::: [2].

Diagrams in addition to widgets and interaction with diagrams make `G'UIs more graphical, but also give raise to problems that don't exist in widget based UIs.

Using a general purpose drawing editor isn't an ad- equate way for dealing with diagrams. Generally, di- agrams are needed as component of a GUI or interac- tive visual language environment and have semantics relevant to the application. Thus, we can't permit any composition of pimitive graphical objects, but have to keep the diagram within some class of diagrams.

Adiagram editor is a syntax-directed editor tailored to a certain class of diagrams, e.g., Nassi-Shneiderman

diagrams (NSDs). Layout and complex editing opera- tions can be supported by a diagram editor, which is hardly possible for a general drawing program.

Similar to the Synthesizer Generator [3] for syntax- directed text editors, diagram editors should be built from a specication using a tool. Currently, we are implementing the Diagram Editor Generator DEG. An editor's specication includes the denition of the dia- gram class by means of aconstraint hypergraph grammar (CHG) [1]. This kind of grammar permits easy speci- cation of layout for graphic objects.

In the rst section, we focus on specifying dialog con- trol for editing diagrams. After highlighting the specic problems, event automataare introduced. Abstraction for events is realized since transitions can be caused by user-dened, high level events. Event automata are based on nite state machines and therefore have an intuitive visual representation. Parallel composition of automata supports multi-threaded dialogs.

The next section deals with the problem of assem- bling a set of selected objects into groups. This is done automatically, taking into account the specication of a group's structure. Such a specication is based on the grammar given for a diagram class.

CHGs and the incremental algorithm for constraint evaluation presented in [1] make possible integration of automatic diagram layout with user dened modica- tions. Constraints consisting of linear equations and in- equalities specify layout. We show how this concept can be generalized to include nonlinear constraints.

2 Event Automata

2.1 User interaction with diagrams

Widgets as available in the most prevalent toolkits can be parameterized, their behavior, however, is rather xed. The widget is responsible for sensing primitive events such as pressing a mouse button. The user of a widget doesn't have to deal with these primitive

Figure 1: Example diagram with selected line for sec- tion 2.1, resp. NSD^go()referenced in section 2.3 events, but receives a notication, if a higher level event (e.g., selection of a menu item) has happened. Usually, this is done by invoking a callback function that has been registered by the user. The behavior of a widget is determined by the callback functions that can be regis- tered.

Toolkits provide for composition of widgets into more complex ones, e.g., a couple of menu items make up a menu. Since component widgets don't overlap, it is clear where primitive events go. Even if there is an overlap of widgets during interaction (consider pulldown menus), the behavior is well dened since only the topmost wid- get receives the events.

The situation is entirely dierent for diagrams. First, we do not have xed behavior. Instead, it has to be spec- ied how primitive events are composed into higher level events. Second, diagram objects often overlap. Even if output objects don't, this can be the case for areas sen- sitive to input.

Consider the NSD example in Fig. 1, which is a snap- shot from our system. A diagram is composed of^lines and^textblocks. A^linecan be selected by clicking.

Diagram objects consist of primitive components for output and handles for indicating selection. Areas sen- sitive to input, subsequently calledreceptors, are needed as well. In DEG, diagram objects are terminal symbols (TSs) in the grammar for NSDs and in the simplest case dened as follows:

"text"(ul,ll,ur,lr) = // ul: upper left node { Text < > (ul,ll,ur,lr);

Handle < arity = 4 > (ul,ll,ur,lr);

RectangleReceptor < receptor_type = "Text" >

(ul,ll,ur,lr);

};

"line"(n1,n2) = { Line (n1,n2);

Handle < arity = 2 > (n1,n2);

LineReceptor < receptor_type = "Line", tolerance = 5 > (n1,n2);

};

TSs are hyperedges connected to a number of nodes, e.g., ^text is connected to the four corners of a text block. Each node (^ul, ^:::) represents a point in the plane and is used for specifying layout (see [1] for de- tails). ^toleranceindicates the width of a^linereceptor in number of pixels as needed for clicking. This width causes overlap with^textreceptors.

Abstraction is a major concern for specication of di- alogs. Selection of a ^line, e.g., can be seen as a high level event. In contrast to low level events such as press- ing a mouse button, visual feedback is given. Thus, high level events can have a dual nature of being actions as well.

2.2 Models for interaction

The most prevalent models for interaction arestate tran- sition systems andevent systems. In order to elucidate their inherent characteristics, we will rst compare the basics of these models. Then Statecharts [4] and the multiagent model, which are extensions of state transi- tion resp. event systems [5, 6] are addressed.

State transition systems are based on the well-known nite state machines (FSM). An event system consists of a collection of rules having the form `^if condition

then action'. Rules are red, if the system's current state satises the condition part. In the action part, the system's state can be changed and/or new events can be sent, i.e., high level events can be generated.

The strength of the event model is that there is no sequencing inherent to the rules. This is essential for widget-based UIs, where some user actions can often be performed in any order. Finite state machines, in contrast, exhibit sequential behavior.

The event model's strength is also its weakness. Fir- ing a rule has global consequences since its action can turn any other rule's condition part totrue. Due to the lack of explicit transfer of control it is dicult to track down the system's behavior, esp. if the condition part of more than one rule is fullled. This case can eas- ily lead to nondeterminism, if two actions modify the same global data item. Generally, the execution engine

Select(“Text”) Select(“Line”)

Edit() Move()

Simple()

Select(type)

ClickSel(type) AreaSel(type)

Figure 2: Simple automata for describing selection of items. ^Text or ^Line objects can be selected in

Simple(). ^Select()permits two ways for selection.

of an event system will enforce some order, but the user doesn't know if such unwanted con icts occur. In or- der to nd out about these con icts, an analysis of the entire system is necessary. If we want to tell whether a FSM is deterministic or not, alocal analysis (i.e., for each state) is sucient.

Except from Harel'sStatecharts, there have been sev- eral earlier suggestions for extending FSMs [7, 8]. We consider Statecharts here since it has gained some pop- ularity and is the rst to incorporate parallelism. State- charts is a very powerful model where states can be arranged in a hierarchy and actions permit generating user-dened, high level events.

The multiagent model adds encapsulation to the event model by using techniques from object-oriented programming. This includes hierarchical composition and a local state that can serve for limiting the state changes caused by an event to a well-dened part of the system. If events coming from the level below need to arrive in a particular sequence, the agent has to keep track of this using the local state. I.e. some implemen- tation of a FSM is needed inside the agent.

Obviously, the extensions diminish the dierences be- tween the basic models. A major point, however, is that Statecharts is a visual formalism, whereas the multi- agent approach is more like a special purpose program- ming language. (The model-view-controller model [9]

is an example.) In our opinion, a visual formalism is preferable for the domain of user interaction with dia- grams. Thus, we will now highlight what makes using Statecharts troublesome in this application.

Consider a very simple example for interaction in an NSD. Suppose updates to the layout are made by se- lecting a^linethat can then be moved. Also, we may require that a ^textbe selected prior to editing. The most natural solution is to have a ^Select() that can be specialized to the type of receptor (input area) we want to select and a state from which both ^Select() edges start, as in^Simple() in Fig. 2. Dierent target nodes make it easy and visual to allow for operations

that depend on the item selected.

In Statecharts, only transitions labeled `event<sup>(</sup>condi- tion<sup>)</sup>' are permitted, but an event can be a high level event generated by another automaton running in par- allel to<sup>Simple()</sup>. Thus, we have to revert to something like `Select(x = "Line")' with a global variable^xor to two dierent events^SelectLineand^SelectTextto accommodate both cases. The main problem, however, is that due to the overlap of receptors both events may be generated. This results in a nondeterministic system and is an erroneous specication from the user's point of view [4]. To avoid nondeterminism, both^Select()s can be combined into a single automaton or a global vari- able might be used for communication between both au- tomata. In any case, the solution doesn't leave the two

Select()s independent of each other.

A similar problem shows up on the next level of our example in Fig. 2, where ^ClickSel() and ^AreaSel() are possible to select an item by clicking resp. indicat- ing a selection area. Assume that the same mouse but- ton is used for both kinds and area selection is done in the usual way with a selection box being displayed.

This time the problem isn't due to overlap of recep- tors, but results from the event sequence ^ButtonDown,

ButtonUp for ^ClickSel()being a subsequence of that for^AreaSel().

In Statecharts, an automaton ^A stays in its cur- rent state, if an event doesn't t any arc emanating from this state. One reason for this solution is that events intended for an automaton^A0 running in paral- lel to ^A have no eect and that ^A and ^A0 don't have to be modied only because they run in parallel. Ac- tually, the same notion of staying in the current state exists in the event model, as, e.g., in [10]. Thus, both

ClickSel() and ^AreaSel() may occur. Feedback as given by ^AreaSel() will result in confusion if more than one action is involved. Again, this diculty has to be solved in Statecharts by giving up independence of ^ClickSel() and ^AreaSel() at an early stage, i.e., before both are used in^Select().

It is easy to imagine that this lack of independence is undesirable and esp. complicates extensions or changes to dialog specications. One of the reasons for these dif- culties is that events in Statecharts abandon some of the advantages of FSMs¹. In particular, a global analy- sis of the system has to be performed to ensure determin- istic behavior, since a high level event can come from any other automaton running in parallel. The other reason is that for diagrams a limited form of nondeterminism is highly desirable, if we want to allow for abstraction.

This is not provided by Statecharts.

1Furthermore, Harel mentions in [4] that events make denition of the semantics very dicult.

Select(“Text”) Select(“Line”)

Edit() Move()

Simple()

Select(type)

ClickSel(type) AreaSel(type)

Deselect() Deselect()

<otherwise>

Figure 3: Event automata for selection as explained in Fig. 2. The little arrowheads on the borderline of states indicate a priority for outgoing arcs.

2.3 The event automata model

We try to avoid the aforementioned problems by tem- porarily permitting nondeterminism. If necessary, non- determinism is resolved by the execution engine in a lucid way.

An event automaton can be a simple orparallel au- tomaton. A parallel automaton is composed of simple automata and will be addressed later. The example given above can be written down as event automata

Simple()and^Select()in an intuitive way, see Fig. 3.

Edges are labeled with conditions that include primi- tive events or with subautomata as in the example. A sequence of events leading to successful execution of a subautomaton can also be seen as high level event.

Initial states are indicated by grey ll pattern, - nal states by double circles. There can be more than one nal state or none², as in ^Simple(). When, e.g., ^Simple() is started, it is tried to reach another state from the initial state. Thus, all subautomata on edges emanating from the initial state are started.

Low level events coming from the window system are processed by the active subautomata (those that have been started, but have not yet terminated execution).

Simple() enters a new state, if one of these subau- tomata reaches a nal state. Due to overlap of recep- tors, this might happen for more than one at the same time. To ensure deterministic behavior, a priority is as- signed to all edges starting from some node. Priority is indicated by the small arrow head attached to the start- ing point of the edge with highest priority. This gives

Select("Line")priority over subsequent arcs.

In the second part of the example, ^AreaSel() in- cludes transitions producing output for feedback. If a transition's action (not shown here) includes an output operation, other branches such as ^ClickSel() in our case are terminated. This is another way how nondeter-

2Thus, edges are indeed hyperedges with a single source and a number of target nodes.

Figure 4: Subdiagram ^goTryTrans of NSD ^go() in Fig. 1. The handles in this snapshot are displayed for selected items and serve as an example in section 3.

Figure 5: Subdiagram ^goSetState of NSD ^go() in Fig. 1. Some details of the actual algorithm have been omitted.

minism is resolved. In ^Simple() we obviously assume that^Select()nds an object of the desired type. This means that ^ClickSel()(and^AreaSel()as well) must not reach a nal state, unless such an object is found. To prevent subautomata from `getting stuck' after a couple of events, they can signal unsuccessful termination by a transition into a give-up state. This is a (non-nal) state without emanating edges. If all subautomata are unsuccessfully terminated and there is an<otherwise>- transition, this is taken. In our example, not nding an appropriate object in^ClickSel()or^AreaSel()leads to unsuccessful termination of^Select(). Since there isn't any<otherwise>in^Simple(), this automaton stays in its current state (the initial state) and the next primitive events arriving may cause a transition.

The NSDs in Figs. 1, 4, and 5 give an idea of how the execution engine for simple event automata works.

When a low level event is delivered from the window

system, it is propagated downwards in the hierarchy of currently running automata by^go(event) (Fig. 1). A C++-like notation is used, thus^A.xrefers to a member function or instance variable^xof^A. The value of^status is returned by ^go() and can either be <NoStepDone>

for no progress in ^A or one of ^A's subautomata, or

<StepDone> for some progress, or <GiveUp> if ^A has reached a give-up state, or one of^A's nal states if^Ahas entered such a state. These return values are relevant to deciding on the level above which of the nondeterminis- tic branches shall be followed in the next step.

A value of ^{new state} dierent from ^undefined in- dicates that a transition in ^A has been completed. In the^for-loop in Fig. 1 the transitions starting from^A's current state are tried in the sequence of their priority until the rst has completed a transition in^Aor all have been tried. This way, execution is made deterministic.

If a new state has been reached in^A, ^A's current state and the return value^statusare updated according to subdiagram^goSetState. If the new state isn't a nal state, the^until does not yet nish, and further tran- sitions are tried. (Due to having setcurrent eventto

undefined, only transitions with conditions that don't require an event can then be made.)

In subdiagram^goTryTrans(Fig. 4) it is tried to reach a target node of transition^t. ^t can be a simple or a complex transition. In the former case, ^t consists of a condition (which can include a primitive event) and an action. A complex condition references some subau- tomaton^S. If an additional condition is given, this has to be true only when^Sis started. The return value re- ceived fromS.go(event)may dene new values for^A's return value^statusand for^{new state}.

In^goSetState(Fig. 5)^A's current state^A.stateand

status may be set. If a new state has not yet been reached in^Asince there are still some subautomata run- ning, it is decided here which of the nondeterministic branches shall be followed in the next step. These are kept as the list ofactive transitions, which is deter- mined by the return values from the subautomata. The intention is to follow only those branches that `t' the se- quence of primitive events received so far and terminate the others. (These branches are also redened by ac- tions that cause output, as explained before.) In case all subautomata have unsuccessfully terminated execution,

A's state depends on the existence of an<otherwise>- transition.

We will now brie y explain parallel automata. Apar- allel automaton^Aconsists of a list ofconstituents that are simple automata, a list ofnal states, and a termi- nation table. By means of this table, termination of^Ais dened with respect to (successful) termination of ^A's constituents. If all of them terminate, this ensues ter-

Buttons()

Quit()

OtherButtons()

Figure 6: Basic automaton for the buttons on the lhs of Fig. 1 with subautomata ^Quit() for quitting and

OtherButtons()for other buttons actions.

mination of ^A. On the other hand,^Acan terminate as soon as at least one of its constituents terminates. This way interrupt like eects of an event can be specied, as is possible in Statecharts. ^go(event)for a parallel au- tomaton simply works by mapping to the constituents that did not yet terminate and by checking the termi- nation table each time one of the constituents nishes execution.

Suppose, interaction for the buttons on the left hand side of Fig. 1 is specied by automaton^Buttons(), de- picted in Fig. 6. Then automaton^Main()can be dened as parallel composition of ^Buttons() and ^Simple().

Main()will have a single nal state and its termination table denes that^Main()nishes as soon as^Buttons() terminates via^Quit(), regardless of^Simple()'s current state. This avoids a multitude of transitions for^Quit() that would be needed without having parallel composi- tion and user specied termination.

3 Groups

When dealing with diagrams, everybody will expect to be able to deal with a set of (primitive) objects in some action. Groups address this problem. In a general pur- pose drawing editor the user can arrange objects hier- archically into groups. This is somewhat inconvenient since an explicit action is required and the result de- pends on the sequence of actions for selection and group- ing. Furthermore,any objects can be put together into a group, i.e., a group isn't more than a set of unrelated objects. Complex editing operations, however, can re- quire a special structure for a group. When, e.g. a se- quence of statements in the NSD example is arranged into a group, we want to deal with it as a sequence and not as a set of unrelated statements.

This is possible in the Synthesizer Generator [3], where a group is assembled automatically based on the objects selected by the user. The structure groups can have is limited to entire subtrees of the derivation tree andlists. For lists, the productions have to have a spe- cial form. User interaction is based on editing a number of buers in a cut-and-paste style. Just a single selec-

tion is possible in each buer, with the consequence that grouping is not independent of the order of select oper- ations. This looks acceptable for text editors, but for diagram editors a more general concept is desirable.

Attribute grammars lend themselves to a declarative specication of groups. Assembling selected objects into groups can be achieved by attribute evaluation in an order-independent way. This approach would oer high- est exibility, but has two major disadvantages. First, specication will get rather tedious since several aspects of a group (structure, eects of (de)select, visualization) have to be specied by attribution. Second, when us- ing groups in transformations of the diagram, a group's shape (i.e., the way a group looks like in the derivation tree) is needed. For general attribution any shape can emerge which can't be handled in transformations.

In DEG we adopted a model for groups that is based on attribute grammars and oersdeclarative andorder- independent grouping. The structure of a group is spec- ied by agroup pattern consisting of a number ofrules. Groups aredened automatically in a hierarchical way according to these rules. This results in restricted shapes which can be used in transformations without any trouble. Starting point for group denition are the terminal symbols (TSs) currently selected, mirrored by an attributeSel for each TS. TSs form subgroups; TSs and subgroups form new, bigger subgroups. A subgroup denes a group, if it is not contained in another sub- group. Composition of groups is done bottom-up in the derivation tree that represents a diagram in DEG. Thus, a group is bound to some NTS^N in the derivation tree and all members of the group can be found in the sub- tree with root ^N. The rules that caused denition of a group tell which TSs and subgroups are member of a (sub)group. A rule is associated with some production in the grammar given for the diagram class.

There is a default groupSel extending selection to NTSs resp. indicating selection of an entire subtree. For the NSD example productions and group patterns for groupsSel andSeq (for a sequence of statements) are:

NSD(n1,n2,n3,n4) ::=

P1: Sts(n1,n2,n3,n4)

"v_line"(n1,n2) ... // more terminal symbols

< group "Sel": 1; > {} . // (1) Sts(n1,n2,n3,n4) ::= // statement sequence P2: St(n1,n2,n3,n4) {}

|

P3: St(n1,n5,n3,n6) Sts(n5,n2,n6,n4)

"h_line"(n5,n6)

< group "Sel": 1,2; > {} .

St(n1,n2,n3,n4) ::= // statement P4: "text"(n1,n2,n3,n4)

{ ... // Constraints for layout }

Figure 7: Single group after further selection for the diagram in Fig. 4. The diagram has been clipped to the relevant part.

...

group Seq = { P2(Sel); // (2)

P3(Sel, !Seq, ?);

P3(Sel, Seq, ?);

};

We won't go into the details, but try to explain the concept for group patterns. For group^Sela straightfor- ward default rule for assembling bigger ^Sel-subgroups tells that there must be a ^Sel-subgroup for all symbols on a production's rhs in order to form a ^Sel-subgroup for the production's lhs. The default can be overridden as, e.g. in ^P1, where just the ^Sel-subgroup for the rst symbol on^P1's rhs,^Sts, is required (see⁽¹⁾).

The group pattern for^Seqconsists of three rules, see

(2). The rst says that there is a^Seq-subgroup for^P2's lhs, if we have a ^Sel-subgroup for ^Ston the rhs. This is a pretty small sequence of statements with just one element. The same applies for the second rule, where a

Sel-subgroup for the rst symbol on^P3's rhs,^St, must exist and there must not be ^Seq-subgroup for the sec- ond. In the third rule, a ^Sel- and a ^Seq-subgroup are combined into a bigger^Seq-subgroup. ³

The eects of these specications will be illustrated by a tiny example. We start from Fig. 4, where we have two ^Seq-groups indicated by handles. The upper one consists of two consecutive statements, the lower one of a single statement. Suppose now that there is a

`click' on the^textbetween these groups and this causes a^select()action for the^text-TS. Then groups are au- tomatically restructured according to the new selection.

Here, both ^Seq-groups are combined into a single big- ger one, resulting in the image depicted in Fig. 7. The result is independent of the order of selection.

Groups dene temporary diagram objects that can be used for interaction like primitive objects (i.e., TSs).

In particular, groups catch events as well as receptors of TSs. Primitive objects contained in a group are still accessible (in contrast to the primitivegroups in drawing editors). The denition of groups thus doesn't cause any

3Although the impression of a pure bottom-up procedure arises, this is actually not the case, but can't be addressed here.

restriction for interaction, but rather oers compound objects with a certain structure.

Visualization of selected objects can be confusing, if many are selected and each is marked by handles. To avoid this, handles are displayed sparingly by not dis- playing handles for TSs that are member of some group.

For groups, only nodes on the group's `border' (i.e., nodes connected to a TS that is member of the group as well as to a TS not in the group) carry handles. The system takes care of this, based on the group patterns dened. For a<sup>select()</sup>action issued for a TS it is fur- thermore possible to `propagate selection' such that a whole set of TSs can be selected due to a single primi- tive action. E.g. we could dene that a single `shift-click' on the header of an NSD selects the entire diagram. Se- lection in the Synthesizer Generator doesn't oer such exibility.

4 Layout

Since layout of diagram objects frequently is a matter of personal taste, automatic layout should be integrated with user-dened modications. This is possible in an easy and versatileway usingconstraint hypergraph gram- mars [1]. A diagram's structure is dened by a gram- mar as illustrated in section 3, p.6, with TSs as in 2.1, p.2. Symbols are hyperedges, i.e., edges connected to a number of nodes. Each node stands for a point in the plane having an^x- and^y-attribute. Layout is specied by a set of constraints for these attributes. For user- dened modications to the system's layout, inequality constraints are needed. They have been introduced in [1] together with an incremental algorithm for solving these constraints, but were restricted to linear equa- tions and inequalities. Nonlinear constraints had to be avoided. Even for the NSD example in [1], however, nonlinear constraints make sense: the size needed for a text block should be large enough to show its text en- tirely. In terms of its width^wand height^h(in number of lines), this results in a nonlinear constraint. Strongly simplied, the constraint is ^{w
h  c
n}, where ^c is a character's width andⁿis the number of characters.⁴ Searching for tight bounds for^was well as^his a discrete optimization problem. However, the constraint can be written as^hf^h(^w) with a nonlinear function f^h. There are two possibilities how to deal with these constraints, presented in the following.

4This constraint is nonlinear since^wand^hare unknown vari- ables. Linear constraints have the form^c1
x1+^:::+^cn
xnc0 with constants^ci.

4.1 Avoiding nonlinear constraints

The simplest possibility is to not consider the nonlin- ear constraints during constraint solving. These con- straints are rather restricted to a sharper linear con- straint. Within the NSD example, this is the way used so far: each text block has a predened, constant width

w

0. This implies a constraint ^{w  w0}. Each time when a text block is edited, f^h(^w0) is evaluated and an appropriate height-constraint^hf^h(^w0) is inserted.

Both constraints are obviously a linear restriction of

hf^h(^w). Thus, each solution using these sharper lin- ear constraints is a solution of the original constraint system.

This procedure has the disadvantage, that a text block's width can never shrink below^w0 unless special measures are taken. Furthermore, a text block's height is never less than f^h(^w0), even if its width is fairly greater than ^w0. This may lead to text blocks that use only a small space fragment for text. To adjust the minimal width or height, a special method must be provided.

Adjustments can be made by using this method in ac- tions of an event automaton.

These special methods, the inconvenience of using them, and the xed limits for width and height are a consequence of excluding nonlinear constraints during constraint system solving. The following idea includes nonlinear constraints in the solving process, however in a dierent way than the linear ones.

4.2 Partitioning the constraint system

In the NSD example, each linear constraint restricts only^x-attributes or only^y-attributes, but never simul- taneously ^x- and^y-attributes. Only the nonlinear con- straints combine ^x- and ^y-attributes. Thus, it is ob- vious to partition the whole constraint system (which, of course, depends on the current NSD) into an ^x- and a ^y-system, each consisting of only linear constraints.

The nonlinear constraints link both systems, but don't belong to any of them.

A solution of the whole constraint system, includ- ing the nonlinear constraints, can be found as follows:

rst, the ^x-system is solved using the known algo- rithm. During this procedure, the nonlinear constraints are included as linear constraints projected to their ^x- attributes. As the second step, the ^y-system is solved.

While doing so, the nonlinear constraints with their^x- attributes xed are considered as linear^y-constraints.

This solving procedure works due to two reasons:

 The nonlinear constraints have linear projections in their^x- and ^y-attributes. In the rst step, each

nonlinear constraint's projection is of the form ⁵

x

2 ,x

1

min wheremin is a constant depending on the text. In the second step, each constraint has the projection^{y1,y2 }f^h(^x2,x1) where^x2 and

x

1 are constant values after the rst step.

 Choosing an^x-system solution during the rst step can't prevent the second step from nding a ^y- system solution as long as at least one solution for the whole system exists.

Following this procedure, the constraint solving algo- rithms presented in [1] can be used together with non- linear constraints without any modication of the algo- rithms. Optimal adaption of a text block's size to its text is thus possible without special methods provided for just this purpose.

The only preconditions to make this procedure appli- cable are the ability to partition the original constraint system and the two properties mentioned above. Thus, this solving procedure isn't restricted to the NSD ex- ample. It can rather be used in the general case: the constraint system is partitioned into several subsystems consisting of merely linear constraints. The subsys- tems are linked by nonlinear constraints. Provided the nonlinear constraints have the two properties described above, solving the whole system can be performed by subsequently solving the subsystems while including ap- propriate projections of the nonlinear constraints.

5 Conclusions

Interaction with diagrams causes specic problems that don't exist in widget based UIs. Approaches to solving these problems have been presented in this paper and are part of our diagram editor generator DEG.

To cope with the diculties of dialog control when the user interacts with diagrams,event automata have been introduced. The model oers a limited form of nondeterminism being resolved by the execution engine.

This is done in a way visible in the specication of an automaton and supports abstraction. The multiagent model surely is more general, but is not visual and doesn't oer help for the special diculties with dia- grams. For a more general concept, event automata could be incorporated into the multiagent model such that an agent's state is represented by an event au- tomaton. The structure of the event automata model ts better for this composition than Statecharts does.

Since we have lots of objects in diagrams, it should be possible to compose them into semantically meaningful

5In the following, we replace the width^wby^x2,x1and the height^hby^y1,y2to emphasize the^x- and^y-attributes.

groups. The method for specication presented here is a bit more tedious than in the Synthesizer Generator, but also more general. Multiple selections are possible and the result is independent of the order of select actions.

Furthermore, it has been shown how nonlinear con- straints can be included into the layout algorithms for diagram objects described in [1].

The components described in this paper except from nonlinear constraints have been implemented for DEG, but some other parts are still missing. As soon as they have been added, interactive diagrams generated from a specication can be included into GUIs and ease the implementation of visual languages.

References

[1] M. Minas and G. Viehstaedt, \Specication of di- agram editors providing layout adjustment with minimal change",in Proc. 1993 IEEE Symposium on Visual Languages, pp. 324{329. IEEE Computer Society Press, 1993.

[2] A. Morse and G. Reynolds, \Overcoming current growth limitsin UI development",Communications of the ACM, vol. 36, pp. 73{81, Apr. 1993.

[3] T.W. Reps and T. Teitelbaum, The Synthesizer Generator, Springer, New York, 1989.

[4] D. Harel, \Statecharts: A visual formalismfor com- plex systems",Science of Computer Programming, vol. 8, pp. 231{274, 1987.

[5] J.A. Larson, Interactive Software, Yourdon Press Computing Series. Yourdon Press, Englewood Clis, New Jersey, 1992.

[6] L. Bass and J. Coutaz,Developing Software for the User Interface, The SEI Series in Software Engi- neering. Addison-Wesley, 1991.

[7] R.J.K. Jacob, \A state transition diagram language for visual programming", Computer, vol. 18, pp.

51{59, Aug. 1985.

[8] A.I. Wasserman, \Extending state transition dia- grams for the specication of human-computer in- teraction", IEEE Transactions on Software Engi- neering, vol. 11, pp. 699{713, Aug. 1985.

[9] G.E. Krasner and S.T. Pope, \A cookbook for using the model-view-controller user interface paradigm in Smalltalk-80",Journal of Object Oriented Pro- gramming, pp. 26{49, August/September 1988.

[10] R.D. Hill, \Supporting concurrency, communica- tion, and synchronization in human-computer in- teraction { The Sassafras UIMS", ACM Transac- tions on Graphics, vol. 5, pp. 179{210, 1986.