• No results found

Determining Verb Phrase Referents in Dialogs

N/A
N/A
Protected

Academic year: 2020

Share "Determining Verb Phrase Referents in Dialogs"

Copied!
16
0
0

Loading.... (view fulltext now)

Full text

(1)

D e t e r m i n i n g V e r b Phrase R e f e r e n t s in Dialogs I

A n n E. R o b i n s o n

Artificial Intelligence C e n t e r SRI International M e n l o Park, California 9 4 0 2 5

This paper discusses two problems central to the interpretation o f utterances: deter- mining the relationship between actions described in an utterance and events in the world, and inferring the "state o f the world" from utterances. Knowledge o f the language, knowledge about the general subject being discussed, and knowledge about the current situation are all necessary for this. The problem o f determining an action referred to by a verb phrase is analogous to the problem of determining the object referred to by a noun phrase.

This paper presents an approach to the problem o f determining verb phrase referents in which knowledge about language, the subject area, and the dialog itself is combined to interpret such references. Presented and discussed are the kinds of knowledge necessary for interpreting references to actions, as well as algorithms for using that knowledge in interpreting dialog utterances about ongoing tasks and for drawing inferences about the task situation that are based on a given interpretation.

1. Introduction

This p a p e r discusses two p r o b l e m s c e n t r a l to the interpretation of utterances: determining the relation- ship b e t w e e n actions d e s c r i b e d in an u t t e r a n c e a n d events in the world, and inferring the current world- state f r o m u t t e r a n c e s . K n o w l e d g e of the language, knowledge a b o u t the general subject area, and knowl- edge a b o u t the current situation are all necessary for this. The p r o b l e m of determining an action r e f e r r e d to by a v e r b phrase is analogous to the p r o b l e m of deter- mining the object r e f e r r e d to by a noun phrase. Al- t h o u g h c o n s i d e r a b l e a t t e n t i o n has b e e n given to the latter (Donellan, 1977; G r o s z , 1977a, 1977b; Sidner, 1979; W e b b e r , 1978), little has b e e n done with the former. 2

The need to identify an action is o b v i o u s in utter- ances containing verbs like " d o " , " h a v e " , and " u s e " , as in " I ' v e done it", " w h a t tool should I u s e ? " , or "I

1 This research has been funded under three-year NSF Con- tinuing Research G r a n t No. MCS76-22004. This paper and the research reported in it have benefited from interactions with all the members of the natural language research group at SRI. Barbara Grosz, Jerry Hobbs, Gary Hendrix, and Jane R o b i n s o n have been particularly helpful in the preparation of the paper.

2 A problem related to determining verb phrase referents - - interpreting verb phrase ellipsis - - has been investigated by W e b b e r (1978).

have it". In these u t t e r a n c e s the v e r b does not n a m e the action, but rather refers to it m o r e generally, m u c h as p r o n o u n s or " n o n s p e c i f i c " n o u n s (e.g., " t h i n g " ) refer to objects. E v e n w h e n m o r e specific verbs are used, c o m p l e x reasoning m a y be required to ascertain the particular action being r e f e r r e d to. F o r example, the u t t e r a n c e " I ' v e glued the pieces t o g e t h e r " c a n refer to different steps in a task - - depending on w h a t o b j e c t s " t h e p i e c e s " refers to, b e c a u s e e a c h gluing action is a different step. Similarly, the v e r b " c u t " refers to different types of cutting actions w h e n used with different objects, as in "cut g r a s s " , "cut w o o d " , or "cut c a k e " (Searle, 1978).

A v a r i a n t of this p r o b l e m is deciding w h e t h e r a v e r b is intended to refer to a general or a specific ac- tion. F o r example, "cutting w o o d " can refer to the general activity of cutting m a n y pieces of w o o d or it can refer to the action of cutting a particular piece. (Werner, 1966)

This p a p e r presents an a p p r o a c h to these p r o b l e m s in which k n o w l e d g e a b o u t language, the subject area, and the dialog itself is c o m b i n e d to i n t e r p r e t r e f e r - ences by verbs. P r e s e n t e d and discussed are the kinds of k n o w l e d g e necessary for interpreting r e f e r e n c e s to actions, as well as algorithms for using that k n o w l e d g e in interpreting dialog u t t e r a n c e s a b o u t ongoing tasks

Copyright 1981 by the Association for Computational Linguistics. Permission to copy without fee all or part of this material is granted provided that the copies are not made for direct commercial advantage and the Journal reference and this copyright notice are included on the first page. To copy otherwise, or to republish, requires a fee a n d / o r specific permission.

0 3 6 2 - 6 1 3 X / 8 1 / 0 1 0 0 0 1 - 1 6 5 0 1 . 0 0

(2)

Ann E. Robinson Determining Verb Phrase Referents in Dialogs

a n d f o r d r a w i n g i n f e r e n c e s a b o u t the task situation that are based on a given interpretation. The algor- ithms have b e e n i m p l e m e n t e d and tested in a c o m p u t e r s y s t e m ( T D U S ) that participates in a dialog a b o u t the a s s e m b l y of an air c o m p r e s s o r ( R o b i n s o n et al., 1980). The s y s t e m acts as an expert, guiding an a p p r e n t i c e through the steps of the task. T h e k n o w l e d g e availa- ble will be described first, followed by a detailed de- scription of the algorithms f o r v e r b i n t e r p r e t a t i o n , then by a discussion of a sample dialog in which the system participated.

2. K n o w l e d g e N e e d e d

I n t e r p r e t i n g any u t t e r a n c e and relating it to a task requires k n o w l e d g e a b o u t the language and the task, as well as the relationships b e t w e e n them. This p a p e r will c o n c e n t r a t e on k n o w l e d g e n e e d e d to identify ac- tions. It builds directly on the c o n c e p t s of global and i m m e d i a t e focusing, t h r o u g h which certain entities are highlighted ( G r o s z , 1977a, 1977b, 1978; Sidner, 1979). G e n e r a l familiarity with that r e s e a r c h will be assumed. M o r e detailed descriptions of other aspects of the k n o w l e d g e n e e d e d f o r i n t e r p r e t i n g u t t e r a n c e s c a n be f o u n d e l s e w h e r e ( G r o s z , 1977a; H e n d r i x , 1977, 1979; R o b i n s o n et al., 1980; J. R o b i n s o n , 1980).

2.1 A c t i o n s a n d E v e n t s

I n t e r p r e t i n g v e r b p h r a s e s requires k n o w i n g a b o u t events that have occurred, are occurring, or can occur. Such k n o w l e d g e typically includes the steps n e c e s s a r y to p e r f o r m the actions associated with the events, the possible participants, the conditions that m u s t be true b e f o r e the actions can be p e r f o r m e d , and their effects. K n o w l e d g e a b o u t actions and e v e n t s includes b o t h general knowledge a b o u t possible actions and events and m o r e specific k n o w l e d g e a b o u t those t h a t occur during a particular task.

We h a v e d e v e l o p e d a formalism, process models, for e n c o d i n g i n f o r m a t i o n a b o u t a c t i o n s ( G r o s z et al., 1977). This f o r m a l i s m enables the specification of a hierarchical d e c o m p o s i t i o n of actions into subactions, as well as the description of individual types of ac- tions. It is an e x t e n s i o n of the n e t w o r k f o r m a l i s m used for r e p r e s e n t i n g other k n o w l e d g e a b o u t o b j e c t s a n d relationships, as d e s c r i b e d b y H e n d r i x ( 1 9 7 9 ) . T h e description of each action type includes i n f o r m a - tion a b o u t its p a r t i c i p a t i n g actors and o b j e c t s , the p r e c o n d i t i o n s f o r its e n a c t m e n t , its e f f e c t s , a n d the a l t e r n a t i v e sequences of substeps that m a y be follow e d to accomplish it. A sequence of substeps m a y be par- tially ordered. This d e c o m p o s i t i o n of actions builds u p o n earlier research on the hierarchical d e c o m p o s i - tion of the p l a n n i n g p r o c e s s ( S a c e r d o t i , 1977) and u p o n the w o r k by H e n d r i x (1973, 1975) on modeling a c t i o n s a n d processes. M a n y of the actions f o r a

p u m p - a s s e m b l y task have b e e n e n c o d e d in this f o r m a l - ism for use in the T D U S system.

Figure 1 illustrates a p r o c e s s m o d e l f o r a p u m p - a t t a c h i n g process. T h e n e t w o r k n o d e A T T A C H P U M P r e p r e s e n t s the set of p u m p - a t t a c h i n g actions. T h e large b o x depicts a s e p a r a t e space in the n e t w o r k in which the schema of the A T T A C H P U M P action is r e p r e s e n t e d . T h e D E L I N arc links the s c h e m a to the A T T A C H P U M P node. T h e s c h e m a specifies the p a r t i c i p a n t s in the a t t a c h o p e r a t i o n , m a r k e d b y the M A J O R P A R T , M I N O R P A R T , and A G E N T arcs. T h e description of the action, an e l e m e n t of the set of E V E N T D E S C R I P T I O N S , includes the P R E C O N D I - T I O N S t h a t m u s t be true f o r the action to be p e r - f o r m e d , the E F F E C T S of p e r f o r m i n g the action, and the P L O T or steps b y which the action is p e r f o r m e d . E a c h step in the plot ( e n c o d e d on a s e p a r a t e space) is in turn f u r t h e r described b y a p r o c e s s model. In this e x a m p l e , the s u b s t e p s of a t t a c h i n g are positioning and bolting the pump. Their ordering is indicated b y the S U C (successor) link. T h e plot steps h a v e m a n y of the same participants as the main action. In addition the s e c o n d plot step, " s e c u r e with b o l t s " , introduces a n o t h e r set of participants, B O L T S , indicated by the F A S T E N E R arc.

During a task, a record of p r o g r e s s is k e p t b y filling in, or instantiating, the s c h e m a f o r an action as that action is p e r f o r m e d and t h e n i n c o r p o r a t i n g the newly c r e a t e d piece of n e t w o r k into the m o d e l of the c u r r e n t situation. R e c o r d s of actions are linked b o t h t e m p o - rally b y a time lattice a n d t h r o u g h their t a x o n o m i c relationships with o t h e r events and o b j e c t s in the task. E a c h instantiated action has associated with it a time interval. T h e interval can be past, present, or future, and it can be b o u n d e d by t w o times: a start time and an end time. F o r events t r e a t e d as points, the start and end times are identical. F o r events whose start a n d / o r end time is n o t p r e c i s e l y k n o w n , the values m a y be left unspecified or r e p r e s e n t e d b y p a r a m e t e r s t h a t are b o u n d e d a b o v e a n d / o r b e l o w b y k n o w n points in the time lattice.

O n c e an instance of an e v e n t is recorded, it can be used in s u b s e q u e n t d e d u c t i o n s a n d is available f o r answering questions a b o u t past events. This provides a m e a n s of maintaining an u p - t o - d a t e record of a s s e m - bly progress. Such a r e c o r d c o m p r i s e s an essential p a r t of the d o m a i n c o n t e x t within which u t t e r a n c e s are i n t e r p r e t e d a n d q u e s t i o n s a n s w e r e d . A t a n y given m o m e n t the d o m a i n c o n t e x t indicates w h a t a s s e m b l y a c t i o n s h a v e a l r e a d y o c c u r r e d ( a n d in w h a t o r d e r ) , w h a t actions are in progress, and w h a t actions can be initiated next.

We have d e v e l o p e d p r o c e d u r e s f o r r e a s o n i n g a b o u t process models. T h e s e p r o c e d u r e s build u p o n those that e m b o d y general k n o w l e d g e a b o u t logical d e d u c - tion (Fikes and Hendrix, 1977). T h e s e new p r o c e -

(3)
(4)

Ann E. Robinson Determining Verb Phrase Referents in Dialogs

a goal is current or achieved, and h o w goals are repre- sented. 4 In the following sections, we will see h o w these goals are used for interpreting verbs.

2.2.1 Recognizing Goals in TDUS

T h e T D U S system handles two kinds of goals: do- main goals and certain k n o w l e d g e - s t a t e goals. D o m a i n goals c o n c e r n states to be a c h i e v e d by t a s k - r e l a t e d actions, while k n o w l e d g e - s t a t e goals c o n c e r n states to be achieved by acquiring a specific piece of i n f o r m a - tion.

Figure 2 illustrates the relationship b e t w e e n actions and goals. T h e hierarchy shown is a simplification of a p o r t i o n of the a s s e m b l y task h i e r a r c h y c u r r e n t l y e n c o d e d in T D U S . 5 E a c h n o d e r e p r e s e n t s an action a n d its a s s o c i a t e d goal. T h e h i e r a r c h y e n c o d e s the substep relationships: child nodes r e p r e s e n t substeps of their p a r e n t nodes. T h e top-level node in the tree, n o d e 1, r e p r e s e n t s the action of a t t a c h i n g a p u m p whose associated goal is that the p u m p be attached. N o d e s 2 and 3 r e p r e s e n t s u b s t e p s of this a t t a c h i n g p r o c e s s - - the actions of p o s i t i o n i n g the p u m p and tightening the bolts, with the associated goals that the p u m p be positioned and that the bolts be tight. T h e action of locating bolts r e p r e s e n t e d by n o d e 4 is not an explicit step in the task, but is n e c e s s a r y for its p e r f o r m a n c e . N o d e 4 has an associated k n o w l e d g e - state goal: " k n o w the location of the bolts". All these goals have associated actions that, in the process model formalism, are specific instantiations of actions, not action schemata.

We distinguish two classes of goals: direct goals

a c h i e v e d b y actions the a p p r e n t i c e has explicitly or implicitly said are being performed now or have been performed and potential goals m e n t i o n e d by either p a r - ticipant that have not been acted upon but might possi- bly be. Both d o m a i n and k n o w l e d g e - s t a t e goals can be either direct or potential although the current imple- m e n t a t i o n of T D U S d o e s n o t s u p p o r t p o t e n t i a l k n o w l e d g e - s t a t e goals.

In the c o n t e x t of the task steps s h o w n in Figure 2, "I am a t t a c h i n g the p u m p " states that an a t t a c h i n g action (node 1), is being p e r f o r m e d . This establishes the direct d o m a i n goal that the p u m p be a t t a c h e d . "Should I tighten the b o l t s ? " indicates that the tight- ening action (node 3) might be p e r f o r m e d , establishing the potential d o m a i n goal that the bolts be tight.

4 The current implementation of goals in T D U S is an exten- sion and partial revision of one by Sidner described in her disserta- tion (1979).

5 A l t h o u g h the a s s e m b l y task currently e n c o d e d in T D U S involves strong structuring of actions and goals, the representations and procedures we have developed are applicable to less structured subject areas.

(2)

I

(1)

I ATTACH PUMP

goal: ATTACHED

(4)

POSITION PUMP

goal: IN POSITION

TIGHTEN BOLTS

goal: TIGHT

I

LOCATE BOLTS

[image:4.612.326.581.50.250.2]

goal: KNOW LOCATION

Figure 2. G o a l / a c t i o n tree.

A direct k n o w l e d g e - s t a t e goal c a n be established, for e x a m p l e , b y the u t t e r a n c e " w h e r e are the b o l t s ? " , which establishes the k n o w l e d g e - s t a t e goal " k n o w the l o c a t i o n of the b o l t s " ( n o d e 4). A p o t e n t i a l k n o w l e d g e - s t a t e goal would be established b y an utter- ance such as " I ' d like to r e a d m o r e P l a t o " which im- plies the p o t e n t i a l k n o w l e d g e - s t a t e goal of k n o w i n g m o r e a b o u t the p h i l o s o p h y of Plato.

Direct and p o t e n t i a l goals are distinguished f r o m one a n o t h e r b e c a u s e of the different roles they play in the i n t e r p r e t a t i o n of verbs. Basically, direct goals are those that are k n o w n to be current or f o r m e r goals a s s o c i a t e d with actions t h a t are being or h a v e b e e n p e r f o r m e d . P o t e n t i a l goals are possible n e a r - t e r m goals associated with possible future actions. D e p e n d - ing on the type of u t t e r a n c e , o n e or the o t h e r class of goal might be considered first. T h e different roles of the two goal classes will be illustrated w h e n the inter- p r e t a t i o n of verbs is discussed in detail in Section 3.

In the T D U S system, a p o t e n t i a l goal can be intro- duced either by the a p p r e n t i c e w h o is p e r f o r m i n g the task or b y the s y s t e m which is acting as an e x p e r t advisor. T h e s e goals c a n be i n t r o d u c e d in at least three different ways.

(1) T h e a p p r e n t i c e can introduce a potential goal b y m e n t i o n i n g a possible future action, while not ex- plicitly stating that it will be p e r f o r m e d . This distin- guishes b e t w e e n "I am going to take the lid o f f n o w " and " s h o u l d I take the lid o f f n o w ? " T h e f o r m e r ex- p r e s s e s a direct goal b e c a u s e the s p e a k e r explicitly says s / h e is planning to p e r f o r m the action. T h e latter expresses a potential goal b e c a u s e the s p e a k e r has not m a d e a c o m m i t m e n t to p e r f o r m i n g the action, but implies that s / h e might. W h e n a potential action is m e n t i o n e d in this way, if it is an a p p r o p r i a t e next step

(5)

Ann E. Robinson Determining Verb Phrase Referents in Dialogs

in the task the s y s t e m will establish the associated goal as a potential goal. F o r example,

"Should I tighten the bolts n o w ? "

will cause the s y s t e m to establish the p o t e n t i a l goal " t h a t the bolts be tight" if the a p p r o p r i a t e reply is " y e s " .

(2) The expert can introduce a potential goal b y telling the apprentice w h a t actions to perform. T h e goal is p o t e n t i a l and not direct, b e c a u s e the e x p e r t c a n n o t , on the basis of the u t t e r a n c e alone, a s s u m e that the apprentice will p e r f o r m the action. F o r e x a m - ple, the e x p e r t ' s reply to

" W h a t should I do n o w ? "

will cause e s t a b l i s h m e n t of the p o t e n t i a l goal - - or goals if there are multiple possibilities - - associated with the action in the reply.

(3) The apprentice can also introduce a potential goal by indirectly m e n t i o n i n g an action in the task. F o r example, if the apprentice says

"I found the pulley."

in a situation in which one of the next steps is to in- stall the pulley, but n e i t h e r the installation n o r the pulley has b e e n m e n t i o n e d before, the potential goal " t h a t the pulley be installed" will be inferred f r o m the reference to the pulley and the knowledge that it is a possible next step. This f o r w a r d r e f e r e n c e to an ob- ject implicitly focuses the o b j e c t and the step it is a s s o c i a t e d with. Previously, algorithms for shifting focus caused a shift to the step a s s o c i a t e d with the object (Grosz, 1977b). H o w e v e r , this is p r o b l e m a t i c b e c a u s e the s p e a k e r m a y not intend to p e r f o r m the step or even discuss it, but rather intends to talk a b o u t the object. Establishing the step in which the o b j e c t participates as a potential goal highlights the step but does not force a shift of focus to it. This change has p r o v e d to be i m p o r t a n t , as will be seen during discus- sion of the algorithm.

U t t e r a n c e s can introduce direct and potential goals simultaneously. In the e x a m p l e s in items 1 and 2 a b o v e , direct k n o w l e d g e - s t a t e goals are also b e i n g introduced. In particular, the k n o w l e d g e - s t a t e goals are " k n o w i n g w h e t h e r tightening the bolts is the next step" and " k n o w i n g the action to p e r f o r m " .

2.2.2 R e c o g n i z i n g t h e S t a t e of a G o a l

As i m p o r t a n t as recognizing a goal, is recognizing w h e t h e r the goal is the current one, one that has al- r e a d y b e e n achieved, or one that has b e e n a b a n d o n e d . Recognizing when goals are no longer potential is also important.

m direct goal is a s s u m e d to be c u r r e n t w h e n an u t t e r a n c e states that an action that will a c h i e v e the

goal is in progress. achieved either

A goal is a s s u m e d to have b e e n

(1) w h e n an explicit s t a t e m e n t such as "I h a v e a t t a c h e d it" or " I ' m d o n e " or " O K ''6 indicates the c o m p l e t i o n of the action achieving the goal;

(2) w h e n an explicit s t a t e m e n t indicates an action i n t e n d e d to a c h i e v e the goal is finished; or

(3) w h e n the start of a new action implies c o m p l e t i o n of the current one and thus a c h i e v e m e n t of the associated goal. 7

A goal is a s s u m e d to h a v e b e e n a b a n d o n e d following an u t t e r a n c e such as " n e v e r m i n d " .

Potential goals c a n n o t be achieved as such. R a t h - er, t h e y can either b e c o m e direct goals t h r o u g h the m e c h a n i s m s for establishing direct goals or they disap- pear w h e n a new potential goal is recognized.

2.2.3 R e p r e s e n t i n g G o a l s in T D U S

T h e structure of goals in a dialog a b o u t a task is related b o t h to the structure of the task and to the structure of the dialog. T h e structure of tasks and the s t r u c t u r e of dialogs h a v e b e e n discussed e l s e w h e r e ( G r i m e s , 1980; G r o s z , 1977a, 1977b, 1978; H o b b s , 1978; R e i c h m a n , 1978; Sacerdoti, 1977; Sidner, 1979; Wilensky, 1978). O p e n questions r e m a i n a b o u t the structure of the goals that arise and h o w they should be represented.

In T D U S direct goals are r e p r e s e n t e d in a single list, acting like a l a s t - i n - f i r s t - o u t stack. B o t h k n o w l e d g e - s t a t e and domain goals are e n t e r e d on the same list. This simplification has p r o v e d a d e q u a t e for current purposes.

In general, there can be only one potential goal at a time. The e x c e p t i o n is w h e n two possible actions are i n t r o d u c e d at once, as in "install the a f t e r c o o l e r or install the b r a c e " . Because it is simplest to view a potential goal as a single item, h e r e a f t e r references to

the potential goal can be r e a d as referring to the possi- ble c o n j u n c t i o n or disjunction of potential goals w h e n a p p r o p r i a t e .

2.3 K n o w l e d g e about Language

To interpret verbs and infer the current task and dialog situation, the knowledge outlined a b o v e must be c o m b i n e d with knowledge a b o u t the language includ- ing w h a t is g e n e r a l l y c h a r a c t e r i z e d as syntactic, se- mantic, and discourse knowledge.

6 See the discussion in Grosz (1977a) of the roles of OK. 7 As Sidner (1979) points out, in the first two cases the information comes from the utterance, while in the third case it is from the task model.

(6)

A n n E. Robinson Determining Verb Phrase Referents in Dialogs

2.3.1 Syntactic Knowledge

O n e of the m o s t i m p o r t a n t e l e m e n t s of s y n t a c t i c knowledge n e c e s s a r y for interpreting verbs - - and the one discussed here - - is k n o w l e d g e a b o u t tense and aspect. T e n s e and a s p e c t are used to indicate the relative time of an event and w h e t h e r it is or was oc- curring or completed.

T e n s e and a s p e c t are indicated s y n t a c t i c a l l y b y auxiliaries a n d / o r certain v e r b forms. In T D U S , utter- ances are a n a l y z e d and m a r k e d for tense (past, pres- ent, or future) and for progressive (event in progress) and perfective (event c o m p l e t e d ) aspect.

The following are some e x a m p l e s of the v e r b f o r m s T D U S can interpret along with their tense and aspect markings:

I am going. I had gone. I had b e e n going. I will be going.

present, progressive past, perfective

past, perfective, progressive future, progressive

In d e t e r m i n i n g r e f e r e n t s , the tense and a s p e c t of the u t t e r a n c e restrict the alternatives within the task m o d e l and limit the goals t h a t might be c o n s i d e r e d . G e n e r a l l y , p r e s e n t tense a n d p r o g r e s s i v e a s p e c t are used w h e n referring to a new action, indicating that it has b e e n started. Only if the u t t e r a n c e is s o m e h o w m a r k e d , as in " I ' m s t i l l tightening the b o l t s " , will the v e r b phrase refer to an action that already has b e e n m e n t i o n e d as in progress. Similarly, past tense a n d / o r p e r f e c t a s p e c t indicate t h a t an action has b e e n fin- ished. H o w e v e r , the h e a r e r m a y or m a y n o t h a v e k n o w n that the action was in progress.

So far, we h a v e c o n s i d e r e d primarily v e r b s t h a t r e f e r to e v e n t s r a t h e r t h a n states, a n d to the usage that is m o s t c o m m o n in dialogs a b o u t tasks, such as r e f e r e n c e s to single o c c u r r e n c e s of actions. H o w e v e r , the analysis and r e p r e s e n t a t i o n are c o m p a t i b l e with analyses that c o n s i d e r o t h e r kinds of usage ( L e e c h , 1976).

2.3.2 Semantic Knowledge

T h e i n t e r p r e t a t i o n of r e f e r e n c e s to actions and events requires k n o w l e d g e of the relationship b e t w e e n words for actions or events and the internal r e p r e s e n t - ations of the c o r r e s p o n d i n g classes of a c t i o n s or events; s it also requires k n o w l e d g e of the relationship b e t w e e n nouns and entities in the domain. F o r e x a m - ple, the " S E L L I N G " action is an action whose partici- pants include a buyer, seller, some o b j e c t being sold, and s o m e m o n e y . Semantic k n o w l e d g e a b o u t selling would include the i n f o r m a t i o n t h a t f o r an u t t e r a n c e

8 Note that at the beginning of a dialog only the relationships b e t w e e n w o r d s and classes of c o n c e p t s is known. The p r o b l e m a d d r e s s e d here is how to identify the particular action or event referenced in a particular utterance.

whose main v e r b is "sell" in the active voice, the syn- tactic s u b j e c t is the "seller" in a selling event, the syntactic o b j e c t is the item sold, the indirect o b j e c t is the one to w h o m the item is sold, and the o b j e c t of the " f o r " p r e p o s i t i o n is the selling price. T h e infor- m a t i o n n e c e s s a r y to m a k e this m a p p i n g and to build the a p p r o p r i a t e r e p r e s e n t a t i o n is e n c o d e d with the verb. ( H e n d r i x in Walker, 1978; Konolige, 1979).

2.3.3 Discourse Knowledge

Discourse k n o w l e d g e is k n o w l e d g e a b o u t h o w the d o m a i n and dialog contexts in which an u t t e r a n c e oc- curs c o n t r i b u t e to and are influenced by the interpre- tation of the utterance. A l t h o u g h we h a v e included it here u n d e r k n o w l e d g e a b o u t l a n g u a g e , discourse k n o w l e d g e m a y be v i e w e d as s p a n n i n g k n o w l e d g e a b o u t language and a b o u t the domain.

2.3.3.1 Focusing

During a dialog, the participants focus their a t t e n - tion on only a small p o r t i o n of w h a t e a c h of t h e m k n o w s or believes. B o t h w h a t is said and h o w it is i n t e r p r e t e d d e p e n d on a shared u n d e r s t a n d i n g of this n a r r o w i n g of a t t e n t i o n to a small highlighted p o r t i o n of what is known.

Focusing is an active process. As a dialog p r o g - resses, the participants continually shift their focus a n d thus f o r m an evolving c o n t e x t within which u t t e r a n c e s are p r o d u c e d and interpreted. A s p e a k e r provides a hearer with clues of w h a t to look at and h o w to look at it - - w h a t to focus on, h o w to focus on it, and h o w wide or n a r r o w the focusing should be. We h a v e de- v e l o p e d a r e p r e s e n t a t i o n f o r discourse f o c u s i n g (or

global focusing), p r o c e d u r e s for using it in identifying o b j e c t s r e f e r r e d to b y n o u n phrases, and p r o c e d u r e s f o r d e t e c t i n g a n d r e p r e s e n t i n g shifts in f o c u s i n g ( G r o s z , 1977a, 1977b, 1978, 1980).

F o c u s e d o b j e c t s are highlighted in the n e t w o r k m o d e l b y placing t h e m in s e p a r a t e " f o c u s s p a c e s " . Several f o c u s e d o b j e c t s m a y a p p e a r in one space. F o - cus spaces are a r r a n g e d in a hierarchy that reflects the degree of focusing. T h e m o s t p r o m i n e n t space is c o n - sidered primary focus. As f o c u s i n g shifts, the hier- a r c h y is c h a n g e d accordingly and n e w spaces m a y be c r e a t e d f o r the newly highlighted o b j e c t s , while old ones m a y disappear.

In addition to global focusing, we h a v e i n c o r p o r a t - ed the c o n c e p t of immediate focus (Sidner, 1979) t h r o u g h which o n e entity a m o n g those f o c u s e d is sin- gled out. This is a m o r e localized focusing p h e n o m e - non that is closely related to the use and recognition of a n a p h o r a , as well as to c h a n g e s in global focusing.

T h e n o t i o n of f o c u s i n g has b e e n u s e d e l s e w h e r e and is related to notions such as topic, c o m m e n t , giv- en, and new. E a c h of these reflects an a t t e m p t to

(7)

A n n E. R o b i n s o n Determining Verb Phrase Referents in Dialogs

identify the roles of certain sentential e l e m e n t s within a discourse. See Sidner (1979) for a discussion of the relationship b e t w e e n focus and these other concepts.

2.3.3.2 Common-background and Communicated Knowledge

In o u r f r a m e w o r k , the dialog p a r t i c i p a n t s are as- sumed to share knowledge a b o u t processes in the task model 9 and the history of the task p e r f o r m e d to date, along with knowledge a b o u t direct and potential goals and focused entities. We view this shared k n o w l e d g e as c o m p o s e d of at least t w o parts: (1) common- background knowledge - - knowledge a b o u t the world that is assumed to be shared by the participants inde- p e n d e n t l y of the dialog, based on their c o m m o n b a c k - ground and experience, such as the processes in the task m o d e l and the h i s t o r y of its p e r f o r m a n c e ; (2)

communicated knowledge - - knowledge a b o u t the goals and focusing, which is assumed to be shared as a result of the dialog. The steps of the task that are explicitly m e n t i o n e d are c o m m u n i c a t e d knowledge, as are o t h e r focused entities that have b e e n mentioned. We will distinguish these two types of shared knowledge and their roles in the i n t e r p r e t a t i o n of utterances.

We distinguish as c o m m u n i c a t e d knowledge essen- tially what Clark and Marshall (1980) distinguish as the m u t u a l k n o w l e d g e t h a t results f r o m "linguistic c o - p r e s e n c e . " O u r use of the t e r m c o m m o n - b a c k g r o u n d k n o w l e d g e c o v e r s the m u t u a l k n o w l e d g e they describe as resulting f r o m "cultural c o - p r e s e n c e " and a limited f o r m of "physical c o - p r e s e n c e " .

To help clarify our distinction b e t w e e n c o m m o n - b a c k g r o u n d and c o m m u n i c a t e d knowledge, consider a dialog a b o u t assembling a pump. The dialog partici- pants share knowledge a b o u t actions used in a s s e m b l y (inserting objects, tightening bolts), a b o u t parts (nuts, bolts, washers), a b o u t tools, and a b o u t t e r m i n o l o g y for talking a b o u t them. All this is c o m m o n at the begin- ning of the dialog. During the dialog additional knowledge is c o m m u n i c a t e d . Consider the following e x c h a n g e b e t w e e n an e x p e r t (E) and an a p p r e n t i c e

(A):

E: First, put the bolts in the holes. A: H o w m a n y and what size? E: 4 bolts, each 3 / 4 " .

A: OK. A: T h e y ' r e in.

C o m m o n - b a c k g r o u n d k n o w l e d g e here includes k n o w - ing a b o u t aligning holes and inserting bolts. Following

9 N o t e t h a t the a p p r e n t i c e k n o w s n e i t h e r all the steps in the task nor their o r d e r i n g - - o t h e r w i s e there would be no n e e d for the expert. H o w e v e r , the a p p r e n t i c e d o e s k n o w how to p e r f o r m most of the basic actions, such as b o l t i n g and tightening.

the e x p e r t ' s first u t t e r a n c e it has b e c o m e c o m m u n i c a t - ed k n o w l e d g e that the first step is to put the bolts in the holes and that doing so is a potential goal of the apprentice. The e x p e r t ' s second u t t e r a n c e c o m m u n i - cates the f a c t that 4 bolts should be used. T h e a p p r e n t i c e ' s r e s p o n s e t h e n adds to c o m m u n i c a t e d k n o w l e d g e the fact that the action has t a k e n place. T h e fact that the holes were aligned and the p r o p e r bolts found can be a s s u m e d b y the expert, drawing on knowledge of the task. Since these actions were not m e n t i o n e d , t h e y are p a r t of c o m m o n - b a c k g r o u n d knowledge but not c o m m u n i c a t e d .

A s s u m p t i o n s a b o u t things t h a t are c o m m u n i c a t e d knowledge play a critical role in the i n t e r p r e t a t i o n and p r o d u c t i o n of utterances (Clark and Marshall, 1980), as the use of a n a p h o r a illustrates. P r o n o u n s and pro- verbs (when used felicitously) always refer to c o n c e p t s in c o m m u n i c a t e d k n o w l e d g e , so that a n y u t t e r a n c e c o n t a i n i n g a p r o n o u n or p r o - v e r b m u s t d r a w u p o n c o m m u n i c a t e d knowledge. In the e x a m p l e above, if the a p p r e n t i c e ' s second u t t e r a n c e had b e e n " I ' m put- ting t h e m in n o w " followed b y " I ' v e done it", the "it" could have referred only to the insertion step, which has b e e n c o m m u n i c a t e d , not to any substep which has not been.

A similar o b s e r v a t i o n a b o u t the use of a n a p h o r a has b e e n m a d e by H a n k a m e r and Sag (1976). T h e y d i f f e r e n t i a t e the linguistic and nonlinguistic c o m p o - nents of c o m m u n i c a t e d k n o w l e d g e , using the t e r m " p r a g m a t i c e n v i r o n m e n t " to refer to the nonlinguistic e n v i r o n m e n t - - which is limited in our situation since there is no shared visual information. H a n k a m e r and Sag state that "the conditions on insertion (and inter- p r e t a t i o n ) are that the s p e a k e r p r e s u m e s the c o n t e n t of the a n a p h o r to be r e c o v e r a b l e , either f r o m linguistic c o n t e x t (in which case the a n a p h o r has an ' a n t e c e d e n t ' in linguistic structure, a fully specified linguistic f o r m with the same s e m a n t i c c o n t e n t ) or f r o m the p r a g m a t i c e n v i r o n m e n t . " (Pg. 422). T h e algorithms we h a v e d e v e l o p e d for interpreting verbs draw on these o b s e r - vations and distinguish b e t w e e n u t t e r a n c e s containing and not containing a n a p h o r a , relying m o r e heavily on c o m m u n i c a t e d k n o w l e d g e w h e n a n a p h o r a is present.

Entities that f o r m part of c o m m u n i c a t e d knowledge

c a n be referred to anaphorically, but they are not al-

ways, as is d e m o n s t r a t e d by the use of definite noun phrases to refer to f o c u s e d objects. In the foregoing example, the bolts are f o c u s e d and are thus part of c o m m u n i c a t e d k n o w l e d g e after the e x p e r t ' s first utter- ance - - but w h e n the expert refers to t h e m the second time, a noun phrase is used instead of a pronoun. The d e g r e e of focusing, which influences the choice of a n a p h o r a or a definite n o u n phrase to refer to some entity in c o m m u n i c a t e d knowledge, has b e e n discussed e l s e w h e r e (Sidner, 1979; G r o s z , 1977b; R e i c h m a n ,

1978).

(8)

A n n E. Robinson Determining Verb Phrase Referents in Dialogs

W h e n r e f e r r i n g to s o m e t h i n g not a s s u m e d to be c o m m u n i c a t e d knowledge, a s p e a k e r not only c a n n o t use a n a p h o r a , but must draw on o t h e r shared knowl- edge and supply e n o u g h i n f o r m a t i o n to e n a b l e the h e a r e r to i n t e r p r e t the r e f e r e n c e correctly. In o u r example, if the apprentice had asked where to find the bolts, the e x p e r t could h a v e said "in the c a b i n e t " , assuming the a p p r e n t i c e was generally familiar with the s u r r o u n d i n g s and k n e w w h e r e the c a b i n e t was. The expert could n o t h a v e said "in it" unless the cabi- net had already b e e n m e n t i o n e d and c o m p r i s e d a high- ly f o c u s e d part of c o m m u n i c a t e d knowledge.

3. D e t e r m i n i n g V e r b P h r a s e R e f e r e n t s

In this section we address issues that arise in apply- ing d o m a i n and linguistic k n o w l e d g e to interpret v e r b phrases and to infer the current situation on the basis of the interpretation. M a n y of the e x a m p l e s in this section are t a k e n f r o m the sample dialog in Section 4.

T h e possible r e f e r e n t s of a v e r b p h r a s e are c o n - strained by b o t h the c o n t e x t and the u t t e r a n c e itself. C o o r d i n a t i o n of the constraints is necessary for inter- preting verbs. Contextual constraints are derived f r o m two sources: the dialog and the subject area, particu- larly the task being p e r f o r m e d . U t t e r a n c e constraints are derived f r o m the syntax and semantics, particularly tense and aspect i n f o r m a t i o n and the type of action d e n o t e d b y the verb.

T h e search for the r e f e r e n t of a v e r b phrase can be c o n d u c t e d either t o p - d o w n or b o t t o m - u p . T h e t o p - d o w n search uses c o n t e x t u a l c o n s t r a i n t s to find the place in the task t h a t the u t t e r a n c e fits a n d it uses u t t e r a n c e c o n s t r a i n t s to limit alternatives. T h e b o t t o m - u p m o d e uses i n f o r m a t i o n f r o m the u t t e r a n c e , such as v e r b type, to find its relationship to the task. If the t o p - d o w n search is successful, the action and its place in the task are identified simultaneously.

F o r the a s s e m b l y dialogs in which all the u t t e r a n c e s are directly related to the task and in which the s y s t e m has already e n c o d e d all the r e l e v a n t steps to be per- f o r m e d , t o p - d o w n c o n s t r a i n t s are s t r o n g e n o u g h to allow a t o p - d o w n search to be c o n d u c t e d first - - and only if that fails is a b o t t o m - u p search conducted. In dialogs where less structure is p r o v i d e d b y the task, a b o t t o m - u p search will clearly play a m o r e central role. This search can be i m p r o v e d b y m o r e extensive rea- soning b a s e d on the v e r b in the utterance.

O n e of the limitations of o u r p r e v i o u s n a t u r a l - language s y s t e m s h a s b e e n a lack of c o o r d i n a t i o n of the strategies for identifying r e f e r e n t s of n o u n phrases and p r o n o u n s with one a n o t h e r or with the i n t e r p r e t a - tion of the verb. In fact, except for the p r o n o u n reso- lution p r o c e d u r e that used a very simple goal recogni- tion algorithm (Sidner, 1979), the v e r b p h r a s e was not e v e n t a k e n into account. H o w e v e r , since the interpre-

ration of each of these u t t e r a n c e e l e m e n t s c a n n o t be carried out in isolation, the p r o c e d u r e s for identifying noun phrase and p r o n o u n r e f e r e n t s described in G r o s z ( 1 9 7 7 a ) and Sidner ( 1 9 7 9 ) h a v e b e e n m o d i f i e d to coordinate the search for noun phrase and a n a p h o r i c r e f e r e n t s with the search for the v e r b phrase referent.

3.1 The Top-down Algorithm

D i f f e r e n t types of u t t e r a n c e s can draw u p o n differ- ent contextual constraints. T h r e e m a j o r f a c t o r s are considered b y the i n t e r p r e t a t i o n algorithm in d e t e r m i n - ing which c o n t e x t u a l constraints to d r a w upon. T h e f a c t o r s are: (1) w h e t h e r or not a p r o n o u n is p r e s e n t in the u t t e r a n c e ; (2) w h e t h e r or n o t all the n o u n phrases in the u t t e r a n c e r e f e r to f o c u s e d entities; and (3) w h e t h e r or not the main v e r b is " d o " . F o r the first factor, the p r e s e n c e of a p r o n o u n indicates that c o m m u n i c a t e d k n o w l e d g e , p a r t i c u l a r l y goals a n d im- m e d i a t e focus, is being d r a w n upon. If no p r o n o u n is present, these f a c t o r s m a y still be r e l e v a n t but o t h e r f a c t o r s weigh m o r e heavily in d e t e r m i n i n g constraints. F o r the s e c o n d f a c t o r , w h e n all the definite n o u n phrases r e f e r to f o c u s e d entities, focusing i n f o r m a t i o n is also k e y in interpreting the verb. If not all the re- f e r e n t s are focused, k n o w l e d g e a b o u t the task and its structure m u s t be used. F o r the third factor, w h e n " d o " a p p e a r s as the main verb, c o m m u n i c a t e d knowl- edge plays a m o r e central role t h a n w h e n o t h e r verbs are used. T h e particular usage of " d o " , as signalled by the other constituents, indicates which aspects of c o m m u n i c a t e d k n o w l e d g e are m o s t i m p o r t a n t .

We will discuss the i n t e r p r e t a t i o n a l g o r i t h m by e x a m i n i n g the i n t e r p r e t a t i o n of u t t e r a n c e s resulting f r o m various c o m b i n a t i o n s of these factors. T h e utter- ances we will discuss are t h o s e c o n t a i n i n g the v e r b " d o " , those c o n t a i n i n g v e r b s o t h e r t h a n " d o " a n d p r o n o u n s , and those containing v e r b s o t h e r than " d o " and definite n o u n phrases.

Within the first t y p e of u t t e r a n c e s , those containing " d o " , we f u r t h e r distinguish u t t e r a n c e s like " I ' v e done it" f r o m u t t e r a n c e s like " I ' v e d o n e the screws." In the f o r m e r , " d o " refers to the general action of p e r f o r m - ing an action and "it" refers to the action. In the latter, " d o " r e f e r s to a p a r t i c u l a r action, such as " r e m o v e " . O u r discussion will first c o v e r these two types of u t t e r a n c e s containing " d o " , t h e n u t t e r a n c e s with o t h e r verbs and p r o n o u n s , t h e n u t t e r a n c e s with o t h e r verbs and definite noun phrases.

3.1.1 D o a n d P r o n o u n s

In interpreting v e r b phrases such as " d o it", knowl- edge a b o u t the c o n t e x t is used first to d e t e r m i n e possi- ble r e f e r e n t s . If " i t " has b e e n used felicitously, it m u s t r e f e r to an action in c o m m u n i c a t e d knowledge. As we h a v e discussed, c o m m u n i c a t e d k n o w l e d g e in T D U S is r e p r e s e n t e d b y goals and focusing. G o a l s are

(9)

Ann E. R o b i n s o n D e t e r m i n i n g V e r b Phrase R e f e r e n t s in Dialogs

a subset of all f o c u s e d entities and, b y definition, those actions that could possibly be p e r f o r m e d by the apprentice. C o n s e q u e n t l y , possible r e f e r e n t s are c o n - tained in the subset of c o m m u n i c a t e d k n o w l e d g e rep- resented by the m o s t current direct goals and by the potential goal.

T h e main u t t e r a n c e c o n s t r a i n t s are derived f r o m the tense and aspect, which limit the goals whose asso- ciated actions could be referents. T h e three cases we distinguish are past tense, p r e s e n t tense and p r o g - ressive aspect, and future tense.

P a s t - t e n s e utterances can refer to either direct or p o t e n t i a l goals. F o r such u t t e r a n c e s , the a l g o r i t h m examines the m o s t recent direct goal first. If it is as- sociated with a t a s k - r e l a t e d action (i.e., not a k n o w l e d g e - s t a t e goal), the action is t a k e n to be the r e f e r e n t of "it" because it is the action k n o w n to be in progress. U t t e r a n c e 10 f r o m the sample dialog illus- trates such a reference to a task goal.

A: I ' m d o i n g t h e b r a c e now. (9) E: O K

A: I ' v e d o n e it. (10)

H e r e "it" refers to the action of installing the brace, the action associated with the current goal.

Because of current i m p l e m e n t a t i o n restrictions, the m o s t recent direct goal is not considered as a r e f e r e n t if it is a k n o w l e d g e - s t a t e goal. I n s t e a d , the action associated with the potential goal is t a k e n to be the o n e r e f e r r e d to since it is always an action of the task.10 Clearly, if p o t e n t i a l goals were e x t e n d e d to include k n o w l e d g e - s t a t e goals, a m o r e s o p h i s t i c a t e d test would be required.

U t t e r a n c e s 12 through 15 f r o m the sample dialog illustrate reference to a potential goal.

A: W h a t s h o u l d I d o n o w (12) E: I n s t a l l t h e a f t e r c o o l e r e l b o w

o n t h e p u m p .

A: I ' v e d o n e it (13)

E: O K

A: S h o u l d I i n s t a l l t h e a f t e r c o o l e r (14) E: y e s

A: I ' v e d o n e it (15)

The a p p r e n t i c e ' s utterance 12 establishes a direct k n o w l e d g e - s t a t e goal of k n o w i n g what action to per- form, while the e x p e r t ' s reply establishes a potential goal that the a f t e r c o o l e r elbow be installed. U t t e r a n c e

10 This is a limitation that should be removed as linguistic and representational capabilities improve. An example of "it" referring to a knowledge-state goal would be "I wanted to learn Spanish and I've done it", where the goal was a k n o w l e d g e - s t a t e goal of ' K N O W I N G S P A N I S H ' .

13 refers to the potential goal. U t t e r a n c e 14 similarly establishes a direct k n o w l e d g e - s t a t e goal of k n o w i n g a b o u t the action - - in this case, w h e t h e r the action is installing the aftercooler; here the a p p r e n t i c e ' s utter- ance establishes the potential goal that the a f t e r c o o l e r be installed. U t t e r a n c e 15 refers again to the potential goal.

An u t t e r a n c e that is p r e s e n t - t e n s e and progressive (e.g., " I ' m doing it") refers to an action that has b e e n p r e v i o u s l y m e n t i o n e d but only just started. As we have seen, a potential goal is associated with such an action, so that the latter is t a k e n as the referent. F o r example, u t t e r a n c e 15 could have b e e n " I ' m doing it", referring to the action of installing the aftercooler.

F o r a question referring to a future or a h y p o t h e t i - cal action (e.g., " W h a t should I do n o w ? " ) , no a t t e m p t is m a d e to identify the action as part of the i n t e r p r e t a - tion. Instead, the reasoning process m a k e s use of the task model to identify the a p p r o p r i a t e reply.

3.1.2 D o a n d D e f i n i t e N o u n P h r a s e s

F o r the use of " d o " in which " d o " refers to an action (e.g., " I ' m doing the s c r e w s " ) , the h e a r e r must be able to infer the action f r o m the context. O n e case of this is w h e n the action type is part of c o m m u n i c a t - ed knowledge but no specific action is being r e f e r r e d to. F o r e x a m p l e in the sequence:

I ' v e a t t a c h e d t h e p u m p . I ' m d o i n g t h e p u l l e y n o w .

the first u t t e r a n c e adds the a t t a c h i n g action for the p u m p to c o m m u n i c a t e d k n o w l e d g e . In the s e c o n d utterance, " d o " refers to a n o t h e r attaching action, but this o n e is a t t a c h i n g the pulley, a s e p a r a t e action. " D o " is not referring to the same

specific

action, but rather to the same

type

of action, " a t t a c h i n g " .

T h e r e are o t h e r o c c u r r e n c e s of " d o " in which the action is implicit f r o m the c o n t e x t and the action t y p e has not b e e n mentioned. T h e algorithm currently only handles the situation in which the action type has b e e n mentioned.

T o interpret these u t t e r a n c e s , the contextual knowl- edge used is c o m m u n i c a t e d k n o w l e d g e and k n o w l e d g e a b o u t the task. T h e c o m m u n i c a t e d k n o w l e d g e used is focusing information, b e c a u s e an action of the same type as the one r e f e r r e d to should be focused, it T h e i n t e r p r e t a t i o n a l g o r i t h m s e a r c h e s a m o n g f o c u s e d ac- tions to find one that is of a type c a p a b l e of having the newly m e n t i o n e d participating objects. F o r e x a m - ple, the algorithm might find " a t t a c h p u m p " as a fo- cused action, d e t e r m i n e that it is an " a t t a c h " and t h e n

It Goal information could be used by examining the types of the actions associated with domain goals. However, access to the action type is more direct through focusing information.

(10)

Ann E. Robinson Determining Verb Phrase Referents in Dialogs

that a pulley can also participate in an " a t t a c h " action. If an action is found, task knowledge is used to deter- mine if an action of that type with the participants indicated is an appropriate action in the current situa- tion. Thus, if attach + pulley is an appropriate action, "attach pulley" is taken as the referent of " d o " .

Tense and aspect i n f o r m a t i o n from the u t t e r a n c e help determine which actions in the task model are appropriate. As we noted, a present-progressive utter- ance indicates initiation of a new step, whereas the past tense could be used either with a new step or with one in progress.

Utterances 8 and 9 of the sample dialog illustrate a related situation.

A: S h o u l d I i n s t a l l t h e p u l l e y n o w (8) E: No. T h e n e x t s t e p is:

i n s t a l l t h e a f t e r c o o l e r e l b o w o n t h e p u m p , o r

i n s t a l l t h e b r a c e o n t h e p u m p . A: I ' m d o i n g t h e b r a c e n o w (9)

Here two steps have been mentioned and are essential- ly equally focused and both potential goals, so " d o it" could not refer unambiguously to one of the actions. H o w e v e r , b o t h actions are "install" actions, so " d o " can refer to an "install" type action. The interpreta- tion algorithm outlined above works for this case as well.

3.1.3 P r o n o u n s w i t h V e r b s O t h e r T h a n Do

F o r u t t e r a n c e s c o n t a i n i n g verbs o t h e r than " d o " and p r o n o u n s , c o n t e x t u a l constraints also stem f r o m c o m m u n i c a t e d knowledge, since the object or objects referred to by the p r o n o u n must be c o m m u n i c a t e d k n o w l e d g e - - in our case, m e n t i o n e d in the dialog. The way the referent of the p r o n o u n was introduced into the dialog affects the interpretation of utterances with pronouns. The distinction we make is whether the object w a s m e n t i o n e d as a participant in an action

that is part of the task, (e.g., "I attached the p u m p . " )

or w a s n o t m e n t i o n e d as a participant in an action

(e.g., " W h e r e is the p u m p ? " ) . In the first case, if the object has been m e n t i o n e d as participating in an ac- tion, the action will be recognized as a direct or p o t e n - tial goal and all its participating objects will be fo- cused. In the second case, if no action has been m e n - tioned but the o b j e c t is a participant in some task action, the action will be inferred t h r o u g h the potential-goal recognition mechanism and will b e c o m e a potential goal. H o w e v e r , in this case only the object m e n t i o n e d will be focused and not the other partici-

pants in the action. A n example of the second case is:

Where are the bolts?

[Immediate focus = bolts]

[Potential goal = T H E B O L T S A R E B O L T E D ]

I've tightened them with the wrench. [with the wrench not in focus]

In this situation, the first reference to the bolts has established the potential goal that the bolts be bolted.

In b o t h these situations the o b j e c t m e n t i o n e d is f o c u s e d and, w h e n appropriate, an action it partici- pates in is established as a goal. The difference be- tween the two is whether the actions and the other participating objects are also focused. This difference affects the interpretation of successive utterances con- taining pronouns.

Three cases are distinguished in the algorithm: (1) If there is a p r o n o u n and there are no definite noun phrases, the actions associated with the most recent direct goal and the potential goals are considered as possible referents of the verb, since either of the two cases described above could obtain. (2) If there are definite n o u n phrases, all of which refer to f o c u s e d entities, then the actions associated with the most recent direct goal and the potential goal are the most likely referents. Since all the objects are focused, the action was presumably m e n t i o n e d as in the first case described above. (3) If there is a p r o n o u n and there are also definite n o u n phrases, but not all the definite n o u n phrases refer to focused entities, then only an action associated with a p o t e n t i a l goal is a possible referent. Since a direct goal associated with this o b - ject could not have been established, only the second case described above could obtain.

In all three cases, u t t e r a n c e i n f o r m a t i o n a b o u t tense and aspect and a b o u t action type ( f r o m the verb) is used either to verify that the action associated with the goal is a possible r e f e r e n t or to c h o o s e a matching action type a m o n g possible referents.

3 . 1 . 4 N o P r o n o u n o r D o

W h e n there is no a n a p h o r a in the u t t e r a n c e , the c o n t e x t u a l k n o w l e d g e used for i n t e r p r e t a t i o n c o m e s from focusing and the task model. F o c u s i n g is used to determine the relationship b e t w e e n the utterance and f o c u s e d entities, including the c u r r e n t action. The task model, including the record of task progress, is used to determine which actions can r e a s o n a b l y be talked about in the current context. First, focusing information is used to determine if the referents of any definite n o u n phrases associated with the verb are currently focused.

(11)

Ann E. Robinson Determining Verb Phrase Referents in Dialogs

3.1.4.1 All N o u n P h r a s e s in C u r r e n t F o c u s

The presence of all noun phrase referents in focus indicates that the action involves objects currently being discussed by discourse participants and that the action is related to the current step ( b e c a u s e it in- volves the same objects). The task model provides information about actions the apprentice can perform and has p e r f o r m e d . Tense and aspect i n f o r m a t i o n from the utterance and the verb type restrict alterna- tives within the task model.

Since present and progressive utterances generally refer to newly started actions, the actions considered in the task model are those that are closely related to the most recent action p e r f o r m e d and that involve objects referred to in the utterance. Possible actions might be: a substep of the last step started but not completed; the potential goal; or a step not involving any different objects that is closely linked in the plan to the last step started or completed (i.e., a step that is a substep of or successor to the last step, or succeeds a parent of the last step).

U t t e r a n c e 1 in the sample dialog ( " I am attaching the p u m p " ) illustrates a present-progressive utterance with a noun phrase referring to a focused object. In this instance, the p u m p - a t t a c h i n g step is a substep of the last step started - - installing the pump.

F o r utterances that are past tense a n d / o r perfective aspect, actions in the task model k n o w n to have been in progress and those that could be next steps are pos- sible referents. The alternatives considered during interpretation are: a step in progress; the potential goal; a substep of the last step started; a substep of any step in progress; and a step closely linked to the last step started or completed. U t t e r a n c e 7 ( " I atta- ched the p u m p " ) shows a reference to a c o m p l e t e d action that was a step in progress - - attaching the pump. The verb in utterance 11 ( " I ' v e installed the pulley") refers to a completed action which was the next step to perform, but was not explicitly m e n t i o n e d as having been started - - installing the pulley.

3.1.4.2 Not all N o u n P h r a s e s in C u r r e n t F o c u s

If the referents of the noun phrases are not cur- rently focused, the focusing hierarchy is searched be- cause the hierarchy indicates previously f o c u s e d o b - jects that might become focused again. If the n o u n phrase referents are identified somewhere in the focus- ing hierarchy, the action n a m e d in the u t t e r a n c e is matched against any action occurring at that place in the hierarchy.

If the utterance contains noun phrases referring to objects participating in the action and those objects c a n n o t be identified a m o n g f o c u s e d entities, the ac- tions associated with direct goals are eliminated as possible referents of the verb. This happens because

all actions associated with direct goals have been men- tioned, which has caused all their participants to be focused.

Possible referents of such verb phrases include: the action associated with the potential goal; a substep of the current step in progress; a substep of all the steps in progress (if the utterance is past a n d / o r per- fective); or any action which can achieve some current goal (e.g., knowing a location - > f o u n d the object). Since the objects described in the noun phrases and the action b o t h have to be tested when examining the substeps, the algorithm first checks the objects de- scribed by the noun phrases to see if they are partici- pants in any of the substeps and if so, it then examines the actions to ascertain whether one of them matches the input action.

3.2 B o t t o m - U p S e a r c h

C u r r e n t l y the b o t t o m - u p algorithm consists of a search for the most specific occurrence of an event in the model whose participants are compatible with those in the utterance. This strategy is being expand- ed to include a search for a more general event that can then be found in the task. This can be either the most specific event type that is compatible with all the elements in the u t t e r a n c e , or a more general or 'similar' event type that is c o m p a t i b l e and can be f o u n d in the task. An example of the first is an utter- ance c o n t a i n i n g " t i g h t e n the bolt". The verb " t i g h t e n " refers to a general tightening action, that can have more specific uses - - such as tighten screws, tighten bolts, etc. F r o m the knowledge that one kind of tightening is bolt tightening and f r o m the occur- rence of "bolts" in the utterance, it can be inferred that the " t i g h t e n bolts" action is intended. In the s e c o n d case, a more specific verb might have been used (e.g., bolt the pump) to mean securing the bolts. The verb " b o l t " might be initially interpreted as refer- ring to a specific action of tightening bolts. H o w e v e r , the task model may not have "tighten bolts" e n c o d e d as an explicit step. Instead, perhaps it is implicit in some more general securing step. F r o m the bolting action and knowledge of the more general actions of which it is a subset (e.g., securing), its relation to the task model can be found.

3.3 S e t t i n g L i m i t s t o a S e a r c h

Knowing when to stop searching for a referent of a verb phrase is a n o t h e r important part of interpreting it. In general, the extent to which a verb phrase refer- ence is interpreted depends on the type of utterance. For example, a verb phrase m a y refer to an action that does not fit into the current task context, such as one that could not or should not be p e r f o r m e d at that time. If the verb phrase is contained in a question (e.g., "Should I cut the end off n o w ? " ) , a reasonable

Figure

Figure 2. Goal/action tree.

References

Related documents

On the other hand, the rule MINLFT ± with a single-project approach ± and the software Time Line with the application of the corresponding feasible schedule improving algorithm,

Technical Excellence Award 124 The Feeling Betty Jeans Dance Studio. Outstanding Costume Award 325 Infinity Betty Jeans

The benefit of such a program is also maintained in long-term vocational follow-up for patients admitted many years post-injury [113, 114, 115]. This program offers a signi-

Assim, acredita-se que este estudo possa ter oferecido duas contribuições: 1 a verificação da influência positiva dos mecanismos de governança sobre a condução do processo

As observed for GFP-Mad2, and unlike mRFP- Nup107 or Mtor, Mad1 was mainly nuclear in early em- bryos, and its NE localization only became apparent in late preblastoderm or

The Nortel Networks adoption of the IEEE 802.3ae WAN-compatible 10 GE provides seamless connectivity for data and optical networks without incurring any additional networking delays

It is a companion piece to three other Beekman Wealth Advisory, LLC White Papers on private equity investing, which provide additional detail about evaluating and choosing