Incremental Interpretation: Applications, Theory, and Relationship to Dynamic Semantics

I N C R E M E N T A b I N T E R P R E T A T I O N :

A P P L I C A T I O N S , T H E O R Y , A N D RF, L A T I O N S H I P T O D Y N A M I C S E M A N T I C S *

D a v i d M i l w a r d & R o b i n C o o p e r Centre for Cognitive Science, University of Edinburgh 2, Buccleuch Place, Edinburgh, EH8 91,W, Scot, land, [email protected]

A B S T R A C T

Why should computers interpret language incremen- tally? In recent years psycholinguistic evidence for in- cremental interpretation has become more and more compelling, suggesting that humans perlTorm semantic interpretation before constituent boundaries, possibly word by word. However, possible computational ap- plications have received less attention. In this paper we consider various potential applications, in parti- cular graphical interaction and dialogue. We then re- view the theoretical and computational tools available for mapping from fragments of sentences to flflly sco- ped semantic representations. Finally, we tease apart the relationship between dynamic semantics and in- creinental interpretation.

A P P L I C A T I O N S

Following the work of, for example, Marslen-Wilson (1973), .lust and Carpenter (1980) and Altma.nn al]d Steedrnan (1988), it has heroine widely accepted that semantic i11terpretation in hnman sentence processing can occur beibre sentence boundaries and even before clausal boundaries. It is less widely accepted that there is a need for incremental interpretation in com- putational applications.

In the [970s and early 1980s several compntational implementations motivated the use of' incremental in-. terpretation as a way of dealing with structural and lexical ambiguity (a survey is given in Haddock 1989). A sentence snch as the following has 4862 different syntactic parses due solely to attachment ambiguity

(Stabler 1991).

1) I put the bouquet of flowers that you gave me for Mothers' Day in the vase that you gave me for my birthday on the chest of drawers that you gave me lbr Armistice Day.

Although some of the parses can be ruled out using structural preferences during parsing (such as [,ate C'losure or Minimal Attachment (Frazier 1979)), ex traction of the correct set of plausible readings requi- res use of real world knowledge. Incremental inter- pretation allows on-line semantic tiltering, i.e. parses of initial fragments which have an implausible or an- olnalous interpretation are rqiected, thereby preven-

*'.lPhis research was s u p p o r t e d by the UK Science and Gn- glneerlng l~.esearch Council, H, esearch G r a n t 1tR30718.

Ling ambiguities from multiplying as the parse pro- c e e d s .

However, onqine semantic filtering for sentence pro- cessing does have drawbacks. Firstly, for sentence processing using a serial architecture (rather than one in which syntactic and semantic processing is perfor- lned in parallel), the savings in computation obtained from on-line filtering have to be balanced against the additional costs of performing selnan~ic computations for parses of fl:agments which would eventually be ru- led out anyway from purely syntactic considerations. Moreow~r, there are now relatively sophisticated ways of packing ambiguities during parsing (e.g. by the nse of graph-structured stacks and packed parse forests (2blnita 1985)). Secondly, the task of judging plausi- bility or anomaly according to context and real world knowledge is a difficult problem, except in some very lilnited domains. It, contrast, statistical techniqnes using lexeme co-occurrence provide a relatively sim- ple mechanism which can imitate semantic filtering in many cases. 1,br example, instead of judging bank as a lhmncial institution as more plansible than bank as a riverbank in the noun phrase the rich bank, we can cornpare the number of co-occurrences of the le- xemcs r i c h and bank1 (= riverbank) versus r i c h and bank2 (= financial institution) in a semantically ana- lysed corpus. Cases where statistical techniques seem less appropriate arc where plausibility is affected by local context. For example, consider the ambiguous sentence, The decorators painted a wall with cracks in tim two contexts 517~c room was supposed to look" run- down vs. The clients couhln't afford wallpaper. Such cases involve reasoning with an interpretation in its immediate context, as opposed to purely .judging the likelihood of a particular linguistic expression in a gi- ven application domain (see e.g. Cooper 1993 for dis- cussion).

Although the usefulness of on-line semantic filtering during the processing of complete sentences is deba- table, filtering has a more plausible role to play in in- teractive, real-time environments, such as interactive spell checkers (see e.g. Wirdn (1990) I'or arguments for incremental parsing in such environnlents). IIere the choice is between whether or not to have semantic ill- tering at all, rather than whether to do it on-line, or at the end of the sentence.

q'he concentration in early literature on using in- cremental interpretation for semantic filtering has perha.ps distracted f'roln SOlne other applications which provide less controversial applications. We will

consider two in d e t a i l here: g r a p h i c a l interfaces, ~md d ialogttc,

T h e I,'ounda.tions for I n t e l l i g e n t Cral)hics I'roje(:{; (I,'l(l) I (:onsidered various wa,ys in which n a t u r a l hm-- gu;~ge i n p u t could be used w i t h i n eoml}uter a.idcd de- sign syste, m s ( t h e i}~rl;ieula.r al)plicai;ion s t u d i e d was eoull)ul;er aided k i t c h e n design, where users would n o t uecessarily I)e professional designers). I n c r e m e n t a l in- t e r p r e t a t i o n was considered to be useful in e n a b l i n g imme(li;m~ visual feedl)aek. Visual feedback could be used to l}rovide Colllh:lna.I;ion ([or ex;tlnl)le~ l)y hig- h l i g h t i n g a.n olo.ieet ].el'erred to by a stleeeSSFlll deli- nite des('xiption), or i{; could be used to give the tiber a.n iml)roved ch3.l]{;e of a.('.llievillg suc{:essi'ttl r{?l'{~'l'elice. I,'or Cxmul)le, if sets o[ possil}le ref(:l'en~s ['or a dellnit.e n o u n p h r a s e a.re h i g h l i g h t e d chlrillg wol'(I I)y word i)1'o eessillp; t h e n the user knows how mue], or how little infornt~t,ion is re{luire(l for su(:{:esslid refepenee. :~'

I l u m a , n dia.logue, in ]);n'tieular, I;ask oriented (lialo- guc is eha.rac:terised I)y a. large numl)ers of sell-rel)airs (l,eve[t 198:I, ( ' a r l c t t a et ;d. [9!}:1), such as hesita.ti-. ()]IS} il/;~el;tiol.ls} ;lll(l ix~.[)]~l/;elllelltS, [1, iS ~L]SO Cot|lll[{)]t to l l n d i n t e r r u p t i o n s reqltesting exti:a ela.rilieati(m, or d i s a g r e e m e n t s I)cl'ore I;he (211(I o f ~t. Sell[ell(;{'. I(, iS evelt l)ossil)le for seutenecs stnr{.cd I) 5, one (lialogtte 1)acl, ici - p a n t to b e [in[shed 1)), a.nother. Al)l}lieations involving I;hc u n d e r s t a , tlcLing o f dia.logues inehn(le informal.ion e x t r a c t i o n I¥om eonversal, ional (latal)ases, or eoml)U- ter m o n i t o r i n g of conversal.ions. It. also m&~y I)e useful to include SOllle ['(20~Ltll?es o f ]l/ll[i;I.ll dialogue in m a n m~u:hine (lia.logue. I,'ov ('.Xaml)le, inl.,~rru[}t:ions (::m I)e IlSe(-1 D)]' e a r l y s i g n a l l i n g of e r r o r s a n d P~.nll)iguit,ies.

I,(21; us tirsl; eonsidcl' SOl[l{2 eXaml)les of sel[2rel)air. IllSel:l;iOllS a,dd extl'~l, infol'nmtion, usually uto(li[iel:S (2 .~.

2) We sLarl, in {,].le m i d d l e wil, h .... in the mid(lie of the i)N)er w i t h a. I)h]e disc (I,evelt 1983:ex.3) l]e])l~cements corr('.et l)ieees o[ infotnlal,ion e.g.

3) (I() from left, a g a i n to uh ..., from pink ap;~iu to I)lue (I,evelt 1983:ex.2)

In s(n ne eases informal;ion from [.he eorre{¢e{l nlaterial is ineorl)oI'a.ted il~l:,) [,he fin;d messnp.;e, l,'or examl)le , {:ot]s[([er ;~ :

d) a T h e three m a i n sources of d a t a come., uh ..., they can I}e Found in the vet'erenee~

b ,Iohn t)ol.ic~(t t h a t the (,[{l m a u and his wife, uh L.[oIIIL (JOtlllCiIs Initi~tive in Cognitive L;cieuce/ll(JI, (h':mt; 8826213, IBdCAAI) and ()entre for (Mgni{:ive Science, /]ldver- sity o[ l,Minburgh.

2Thls exmnple was inspired by the work of I1;uldock (1 !)87) on in{:pcmenl~t[ hltcrl)retatlon of de[[nile nouu l)hrases, llad- d[)cl,[ iI~;(!d ;l~li ill(21'(!lli(!ll{;:ll C{}li~;I,iNtilll~ I};]~st1{t al)[}ro;~{ h following Mellish {198s) t;o l}r(wide ;m explanaCi(~tl of why it i:~ i}{,~sil}le {.o use I;he llOllll i)hl'~ts(! [h{" ~'a6~)it £7l t],~: t~a[ e'¢C:ll whell t:hm'e are I.w{} h;M.s, bul. only on(: h;tt wilh ~ P;d}h]l in il.

8 [,;Xallll)le (;t) iS i'ecollslJrll(:lJed i'l'onl ttli ;IC|,(ID~] iil;tet;~.llCe. J')x D.|llpl(2b (I)) D~lld ({i) *,V{'I'¢: COIIS{|'/IC{;{2d.

.... t h a t the m a n got i n t o t,he ear a n d the wife was w i t h h i m w h e n fl]ey left t h e house e ]!',very boy took, u h ..., he s h o u l d h a v e takeA| a

w a t e r 1)ottle wil.h h i m

In (a.), the corrected ma.terial the thre, .main .sou.rcc.s of data come, provides {,he a n t e e e d e n t for the pronon]~ the:]. In (b) the corrected m:~terial tells us t h a t the ma.n is boCh old mid has a. wife. In (e), the p r o n o u n he is b o u n d I)y [,he qmmtifier ever:/boy.

l"or ~ s y s t e m to u n d e r s t a n d dia.logues i n v o l v i n g s e l l repa.irs such as {,hose iu (d) would seem t,o require. either a.n ~d)ility to interl)ret increment, a.lly, or the use of a g r a m m a r which in(:ludes self repa.ir as a synta.etic {:onst, r u e t i o n a.kin to n o n - c o n s t i t u e n t coor- d i n a t i o n (the relationshil)I)ctwec, n coordin;~t,ion a n d s(;li2eorrection is noted I)y I,evelt (1983)). 1:or a. sy-- stenl to g e n e r a t e self tel)airs m i g h t also I:equire in- ePel/iCltta] i lltel?[)lX;[,atiOll] aSStllnilt~ ~/ ])Poeess whelx2 t h e s y s t e l n i)erI'oP1]ts o n - l i n e n | o l l i t o r i l l g o f its o l l l ; p u t (a.kiu to l,evelC's Inodel of the h u l m m sell2rel)a.ic me- eha.uism). I1; has been suggested t.hat geltel'al;iOll of seir tel)airs is useful in eases where t h e r e are severe t i m e

const::~ints,

A m o r e cOral)ell[rig al'gttmetlt for i n e r e n t e n t a I i n t e r i)rcta.tiolt is in'ovided I)y c o n s i d e r i n g dialogues invol r i n g illtel?rtll)tiOllS. Consider tit(' following dialogue from [,he TILAINS COl?pits ((:II'()S,S el, al. 1 1!)9!1):

5) A: so we should m o v e the engin0 a.t Avon, engine l!;, l:o ...

B : e n g i n e E I A : 1')1 l~: okay

A: {mgiue I'21, [,(} B a t h ...

T h i s re{luires hlterl)l:ctatiol| by Sl){'.aker I~ I)el'ore the end o[' A's sente,lee to allow ol).iection to the al)po. siLion, lhe cn:/mc al Auon, {m:lbzc 1¢. A l l exa.ml)Ie o[' the p o t e n t i a l use of iltterl'uI)tiolls in Illllllat] eotnl)uter i n t e c a c t i o n is the followiug:

6) U s e r : Pitt {;lie plllleh ()lit() ...

( ] O l l l I ) l l | ; ( w : T h e i)1111(211 (;;lll~l; 1)(' IIv.)ved. It}s b()]- t,ed to the lloor.

lit t h i s e x a l n l ) l e , interl)ret, a.tion lllUSL lie[ Oll]y /)e I)e- 12)r( the end o[' l:he Sellt,ell('e, I)ul; I)el'ol:e a (:otmt.il, uent, I)otm(lary (Idle vePI)I)hrase in the user's e o m n t a n d has not yet I)e.eu {:Oml)lel.e(I ),

( J U R R E N ' I ? T O O L S

1.. S y n t a x t o S { m m n t i ( : l h ~ i ) r e s e n t a t i o n

(of some sort) for f r a g m e n t s of sentences, and be able to extract s e m a n t i c representations from these.

One possibility, which has been explored m a i n l y within the Categorial G r a m m a r t r a d i t i o n (e.g. Steed- m a n 1.988) is to provide a g r a m m a r which can treat most if not all initial f r a g m e n t s as constituents. 'Phey then have full s y n t a x trees from which the semantics can be calculated.

However, an a l t e r n a t i v e possibility is to directly link the partial s y n t a x trees which can be %rmed fol: nOl>COnstituents with flmctional s e m a n t i c representa- tions. For example, a fragment missing a noun phrase such as John likes can be associated with a seman- I, ies which is a function from entities to t r u t h values. Ilence, tam partial s y n t a x tree given in Fig. 14,

S

/

\

np vp

John

/

\

v np~

likes F'ig. I

call be associated with a semantic representation, Ax. l i k e s ( j o h n , x ) .

13oth Categoria l approaches t;o incremental inter- p r e t a t i o n and approaches which use partial s y n t a x trees gel; into difficulty in cases of left recurs|on. Con- sider the sentence fragment, Mary thinks dohn. A possible partial s y n t a x tree is provided by Fig. 2.

S

/

\

np vp

M a r y / \

v S

thinks / \

np vp}

J o h n Fig. 2

llowever, this is lie| the only possible partial tree. la fact there are infinitely m a n y cliff>rent trees possi- ble. T h e completed sente.nce may have an arbitrarily large n u m b e r of i n t e r m e d i a t e nodes between the lower s node and the lower hi). For exarnple, J o h n could be e m b e d d e d within a gerund e.g. Mary thil&s John leaving here was a mistake, and this in t u r n could be enfl)e(lded e.g. Mary thinks John leaving here being a mistake is surprising. J o h n could also be embed- ded within a sentence which has a sentence modifier requiring its own s node e.g. Mary thinks John will go home probably 5, and this can be flu'ther embedded

4'Phe d o w n a r r o w n o t a t i o n for m i s s i n g c o n s t i t u e n t s is a d o p - t e d f r o m S y n c h r o n o u s T r e e A d j o i n i n g ( } r a m m m " ( S h l e b e r & S c h a b e s 1990).

5'['he t r e a t m e n t of p r o b a b l y as a m o d i f i e r of a s e n t e n c e is p e r h a p s c o n t r o v e r s i a l . I l o w e v e r , t r e a t m e n t of it: as a v e r b p h r a s e m o d i l i e r w a u l d m e r e l y s h i f t t h e potenl, ia] left r e c u r s | o n ~o I h e v e r b p h r a s e n o d e .

e.g. Mary thinks John will go home Frobablg because he is tired.

T h e problem of there being an a r b i t r a r y mmg)er of different partial trees for a particular f r a g m e n t is re- fleeted in most current approaches to incrementM in- terpretation being either incomplete, or not flflly word by word. For example, incomplete parsers have been proposed by Stabler (11991) and Moortga.t (1988). Sta- bler's system is a simple top-down parser which does not deal with left recursive g r a m m a r s . M o o r t g a t ' s M - S y s t e l n is based on the L a m b e k (~ah:ulus: the pro- blem of an infinite l m m b e r of possible tree ka.gments is replaced by a corresponding problem of initiM fl:ag- m e n t s having an infinite n u m b e r of possible types. A colnplete incremental parser, which is not fully word by word, was proposed by P u l l n a n (1986). T h i s is ba.~ sed on arc-eager left-corner parsing (see e.g. l{esnik

To elmbIe complete, fully word by word parsing re

quires a way of encoding an intinite nmnber of partiM

l, rees. There are several possibilities. 'Fhe first is to use a language describing trees where we can express the fact t h a t ,]ohn is donfinatcd by the s u o d e , but do not have to speciiy what it. is ilmnediately domina- ted by (e.g. D-Theory, Marcus et ah 198a). Semantic representations could be tbrmed word by word by ex- tracting 'default' s y n t a x trees (by s t r e n g t h e n i n g do- minance links into i m m e d i a t e d d o m i n a n c e links whe- rever possible).

A second possibility is to factor out recursive struc- tures from a g r a m m a r . T h o m p s o n et al. (1991) show how this can be done for a phrase s t r u c t u r e gram- m a r (creating an equivalent 'Pree Adjoining ( ; r a m m a r (,Ioshi I987)). The parser for the resulting g r a m m a r allows linear parsing tbr an (infinitely) parallel sy- stem, with Cite absorption of each word performed in c o n s t a n t time. At each choice point, there are only a finite n u m b e r of possible new partial TAG trees (the TAG trees represents the possibly inlinite n m n - bet of trees which can be forlned using a d j u n c t | o n ) . It should agMn be possible to extract ' d e f a u l t ' seman- tic values, by taking the semantics from the TA(I tree (i.e. by assuming t h a t there are to be ,to adj unctions). A s o m e w h a t similar system has recently been propo- sed by Shieber and J o h n s o n (191t3).

The third possibility is suggested by considering the semantic representations which are appropria.te during a word by word parse. Although there are any n u m b e r of dill'trent partial trees for the f r a g m e n t Mary thinks John, the semautics of the f r a g m e n t can be represented using j u s t two l a m b d a expressions6:

AP. thinks(mary,I)(john)) AP. AQ. Q ( t h i n k s ( m a r y , P(john)))

Consider the tlrst. The l a m b d a a b s t r a c t i o n (over a (;Two representa~,ions are a p p r o p r i a t e if t:here a r e no V P - m o d i f i e r s as it, d e p e n d e n c y g r a m m a r . If V 1 ) - m o d i f i c a t l o n is Mlowed, I, wo m o r e e x p r e s s i o n s a r e r e q u i r e d :

AP. AR,. ( I I , ( k x . t h i n k s ( m a r y , x ) ) ) ( P ( j o h n ) ) a n d

5 p . a n . .

aQ Q((ll,(Xx.thhlks(mary,x)))(P(john))).

fimctional item of type e--}l;) can 1)e t h o u g h t of as ~t way of e n c o d i n g mt intinite set of p n r t i a l sema.ntie (tree) s t r u c t u r e s . For c x m n p l e , t h e e v e n t u a l s e m a n t i c s t r u c t u r e m a y e m b e d j o h n at ~my d e p t h e.g.

t hinks(nm.ry,sleeps (j oh n))

t h i n k s ( n m r y , p o s s i b l y ( s h ' . e p s ( j o h u ) ) ) etc.

T h e second exl)re.ssion (~ fimctiona] i t e m over t y p e e-+t; a n d t - + t ) , allows for e v e n t u a l s t r u c t u r e s where the m a i n senten<:e is embedde.d e.g.

l>ossibly(l, h i n k s ( n m r y , s l e e p s ( j o h n ) ) )

T h i s third possibility is therefor<; to l>rovide a, syntac- tic c o r r e l a t e of l a m b d a expressions. In l)rm:tice, ho-- wever, l)rovided we are o n l y interested in mai)l)ing f r o m ~ s t r i n g of words to ~ s e m a n t i c representa.tion, ~md d o n ' t need explicit synta.x trees I.o be eonstru(> I;e(|, we (:tin ]nerely use the types of the 'synta(:- tic lambda, expressions', ra~ther them the expressions themselves. T h i s is essentially the a p p r o a c h t a k e n in Milward (]992) in order to provid(; eontplete, word l)y word, i n c r e m e n t M i n t e r p r e t a t i o n using s i m p l e ]e- ×ieMised gr~umna.rs, snch as a lexiealised version of f o r m a l dependen<'y g;ral-lnrlar and s i m p l e eategorial gra.lll lllar 7 .

2. L o g i ( : a l F o r m s t o S m n a n t i c F i l t ; e r i n g In l)ro(:essing the sent(race A,larg introduced John to Susan, a, w o r d - b y - w o r d ;~l)l)roach such as M i l w a r d (1992) provides t h e following logical fornls alTte, r the eorresl>OlMing sentence f r ~ g m e n t s are al~sorbe(l:

Mary M ). l ) ( mary )

Mary introduced Ax. Ay.inl.r (mary,x,y) M~wy introduced John Ay.intr(mary,john,y) Mary iul;rodu(:ed .John to Xy.inLr(mm'y,john,y) Mary introdn(:ed .John to Su<: ini.l'(mary,,john,sne) li':mh i n p u t level rel)res(mtatiotl is apl)ropria.tc for the m e a l f i n g ol7 ~n incomplel, e senl;enee, I)eing e i t h e r ~ pro- positi(m or a, f u n c t i o n i n t o a p r o p o s i t i o n .

/ n Cha.ter et al. (1990) it is argue(1 t l m t l, he i n e r e m e n t M l y derived m e a n i n g s are not .indged [br p l a u s i b i l i t y directly, b u t i n s t e a d ~re first tur~ ned i n t o e x i s t e n t i a l l y (luantified p r o p o s i t i o . s , l,br exa.mp[e, i n s t e a d of .ju(lghtg tim plausiiMity of )~x..~y.int:r(Inary, x , y ) , we j u d g e the plausil~ility of _ ~ ( x , q ' , 3 ( y , T , i n t r ( m a r y , x , y ) ) ) s. T h i s i~ j u s t the p r o p o s i t i o n Mart introduced something to something using ~ g e n e r a l i z e d q u a n t i l i e r n o t ~ t i o n or the t b r m Q u a n t i t i e . r ( V a r i a b l e , R ( : s t r i ( ' t o r , B o d y ) .

A | t h o u g h t h e lambd:~ exl)ressions are built ul) mo- notonieMly, word by word, the, l)rOl)ositions [brined 7Whe version of categorial grammar used is AP. (Sttcgorial C, rmnmar with Associativity.

~'[hc prol)oSil;ion '.P is alw~Lys true. See Chatter et ~tl. (IDg,t) for discussion (ff whether it; is m o r e al)prol)ri~tl:(: to use ~t lit-HI-

trivial rcsl, rictor.

f r o m t h e m m a y need to be r e t r a e t e d , a.long with all t h e r e s u l t i n g infi~relmeS. I,'or examl)le, Mart intro- duced something to something is ina.pl)ropriate if the thml sentence is Marg introduced noone to anybodg. A rough a l g o r i t h m is as follows:

l. Parse a. new word, W o r d /

2. l"orm ~ new l a m b d a expression by e o m l ) i n i n g the lambda~ exl>ression f o r m e d after p a r s i n g W o r d i _ 1 w i t h the lexieal s e m a n t i c s h)r W o r d /

3. F o r m a. p r o p o s i t i o n , Pi, by existentia.lly quantit):-. ing over the la.mbda a/)stracted va.riables.

4. Assert Pi. If Pi does n o t ent~dl Pi-1 r e t r a e t I)i_ ~md all conclusious m a d e ft:om it s.

5. J u d g e the. phmsilfility of Pi. If iml)hmsible, block this del:iwlLio,.

It is worth u o t i n g tht~t the need for r e t r a c t i o n is not due to ~x failure to e x t r a c t the eorrect q e a s t eolnDlit- menC' propositiotl from the s e m a u t i c (:ontent of the f r a g m e n t Mary introduced, 'l'hi~ is due to tim fm:t t h a t it, is I)ossible to find pairs of l)ossible e o n t i n u a t i - ous which m:e the negation e l each o t h e r (e.g. M(rrg introd'ltccd noonc to anybody a n d Mary inl,'rodltced so- meone to somebody). T h e only p r o p o s i t i o u comps> tibk', w i t h b o t h a p r o p o s i t i o n , p , a n d its n e g a t i o n , ~1 ) is the trivial p r o p o s i t i o n , "P (see (.:hater et al. for further discussion).

3. I n e r e m e i H ; a l Q u a n t i t h ~ r S e e p i n g

So fa.r we have only considered s e m a n t i e r(~presental.i- ons which do not involve (lll~uttiliel'S (except I'or the exist(mtial q u a n t i f i e r i n t r o d u c e d by t h e m e c h a n i s t ,

~d)ove).

In senten(:es with two oF m o r e qmmtiliel;s, there is generally ~m ~mabiguity eon(:erning w h i C | q u a n t i f i e r has wider s(:ope. 1"or exm:nple, in sentence (a) below tim preferred reading is lbr the s a m e kid to have ('Aim- bed every tree (i.e. the ml.iversal q u a n t i l i e r is w i t h i n the scope of the existeutia.I) whereas in sentence (b) the preferred reading is where the universal q u a n t i f i e r has scope over the existential.

7) a A I, ireless kid eliml)ed every tre.e. b T h e r e was ~ tish on every l~latc '.

Scope prefiwenees s o m e t i m e s seem to I)c esl, al)lished bel'ore the end of tz sentence. ],'or e x a m p l e , in s e u t e n e e (a) below, t h e r e s('.ellla a l)referell(:e for all Oll(,er seol)e r e a d i n g for the first q u a n t i f i e r as soon as we inl;erl)rel; child. Ill (13) the i)refereu(:e, by the t i m e we get to e.g. gram.mar, is [~.)r adl ituwr scope re~ding for the lh:st qu a.ntiller.

8) a A te~eher gave every child a g r e a t deal of he-. m e w o r k Oll gralflnlar.

b Every gM in the class showed a rather strict new teacher the results of her attempt to get the g r a m m a r exercises correct.

This intuitive evidence can be backed up by consi- dering garden path effects with quantifier scope tun- biguities (called jungle paths by Barwise 1987). The original examples, such ~s the fbllowing,

9) Statistics show that every 11 seconds a man is mugged here in New York city. We are here today to interview hiln

showed that preferences for a particular scope are established and are overturned. 'Po show that pre- ferences are sometimes established before the end of' a sentence, and before a potential sentence end, we need to show ga.rden path effects in examples such as the following:

10) Mary pttt the inIbrmation that statistics show that every 11 seconds a man is mugged here in New York city and that she was to interview him in her diary

Most psycholinguistic experimentation has been con- cerned with which scope preferences are made, rather than the point at which the preferences are establis- hcd (see e.g. K u r t z m a n and MacDonald, 1993). Given tile illtuitive evidence, our hypothesis is that scope preferences can sometimes be established early, befbre the end o f a sentence. This leaves open the possibility that in other cases, where the scoping inIbrmation is not particularly of interest to the hearer, preferences are determined late, if at all.

3.1 I n c r e l n e n t a l Q u a n t i f i e r S c o p i n g : h n p l e - lnent, a t i o n

Dealing with quantifiers incrementally is a rather si- mila.r problem to dealing with h'aglnents of trees incre- mentally, a , s t as it is in-,possible to predict the level of embedding of ~r noun phrase such as John from tile fragment Mary thinL's John, it is also impossible to predict the scope of a quantifier in a fragment with respect ~o the arbitrarily large number of quantiliers which might appear later in the sentence. Again the problem can be avoided by a tbrm of pacldng. A par- ticularly simple way of doing this is to use unseoped logical forms where qmmtifiers are left in situ (silni- lar to the representations used by Hobbs and Shieber (1987), or to Quasi Logical Form (Alshawi 1990)). For example, the fl'agment Every man gives a boot" can be given the tbllowing representation:

I1) kz.gives(< V,x,nlan(x)>,< ~,y,book(y)>,z) Each qnantitied term consists of' a quantitier, a va.ria- ble and a restrictor, but, n o body. To convert lambda expressions to unscoped propositions, we replace an occurrence of each argument with an empty existen- tia.l quantitier term. In this case we obtain:

12) gives(< V,X,ITIall(X)>,< 3,y,book(y)>,< -~,z,'l'>)

Scoped propositions can then be obtained by using an outside-in quantifier scoping algorithm (Lewin, 1990), or an inside-out algorithm with a free w~riable con- straint (IIobbs and Shieber, 1987). The propositions fbrlncd can then be judged for plausibility.

To imitate jungle path phenomena, these pla.usi o bility judgements need to feed back into the scoping procedure for the next fragment. For example, if' every man is taken to be scoped outside a book after proces- sing the fragment l?vcry man ga~c. a book, [;hen this preference should be preserved when deterlnining the scope for the full sentence l?very uza~t gave a book lo

a child. Thus instead of doing ~dl quantitier scoping

at the end of the sentence, each new quantilier is sco- ped relative to the existing quantifiers (and operators such as negation, intensional verbs etc.). A prelimi- nary irnplemenl, ation achieves this by annotating the semantic representations with node nantes, a.nd re- cording which quantifiers are 'discharged' at. which nodes, and in which order.

D Y N A M I C S E M A N T I C S

l)ynamic semantics adopts the view that "the mea-- ning of a sentence does not lie ill its truth conditi- ons, but rather in the way ill which it changes (tile representation of) the in[brmation of the intcrl)reter" (Groencndijk and Stokho[', ] 991). At first glance such a. view seems ideally suited t.o incremental interpreta- tion. Indeed, Groenendijk and Stokhof claim that the compositional nature o[' l)ynamic Predicate Logic en- ables one to "interpret a text ir~ an on-line ntauner, i.e., incrementally, processing a.nd interpreting each basic unit as it comes along, in the context created by t h e interpretation of the t.ext s o fa.r'.

Putting these two quotes together is, however, mis- leading, since it suggests a more direct mapping bet- ween incremental sem~mtics and dyna.mh: semantics than is actually possible. In an incremental semantics, we would expect the informtttiou state, of an interpre- ter to be updated word by word. In contrast, in dyna- mic semantics, the ol:der in which states are updated is determined by semantic st;ructure, not by left-to- right order (see e.g. I,ewiu, 1992 [br discussion). For example, in 1)ynanfic Predicate Logic ((~roenendijk ,~ Stokhof, 1991), states are threaded from the antece- dent of a conditional into I, he conseque~d~, and from a restrictor of' a quantitier into I;he body. Thus, in interpreting,

13)

away is the output state from the a.ntecedent a ear

hnp,vsses hhn. in this ease the threading through

semantic structure is in the opposite order to the order in which the two clauses appear in the sentence.

Some intuitive justification for the direction of threading in dynamic semantics is provided by cou- sidering appropriate orders for evaluation of proposi- tions against a database: the natural order in which

l,o cvMual;e a, conditiona,1 is first, 1,o add the antecedenl;, illl([ thell see if I.he COllSC(lUOlll, c{i.ll 17c: l)roveli. ]to is ()lily ai, tile sentence lew;l iu ,siniple na,rrative texLs 0ha,t I,he l)l;esenl;al, ion ordor itlld I,ho iw, l, ur~d order o[ 4"wahl3J;ion necx~s,<sarily coincide.

T h e o r d e r h i g o f a,n~tl)hors and theii: autel:edent, s is o[l;en used i n l ' o r i n M l y / , o jusl,ify lefl,-l,o-riglll; i, h r e a d i u g or thi:eadilig t h r o u g h selllaait, ic sl;rtlC(,llrO, llowew',r, (,]iroa(lingj fl;Olll ]el'[,-to-righ{ disa,llows cX;/llllJlcs (3]" Ol)

(;tonal c;d;aphoi;a,, as ill e x a m p l e (1'3), mid examI)les

o[' c:oiilpll|sory c,a.l;a,p}lora, a,s ill :

14) I lesitle her> every girl could see a, large cl:acl~

Sitnihu:ly, l;]irei~ding li'oin the aui;ccedeui,s o1' conditio-- ua, lS i n t o t;he COllSl'x[llOll[; fails f'()r (!X31ill)JeS sHCh &s:

15) EVC'L'y b o y will be a.I)lc t,(i s(!(', olLi, <)[' a w i n d o w i l

he walll;s I,()

I{, is also 1)ossilTle l;o gel; ,SOli{,ellCCS w i t h <([Olil,:cy'

rea,dhlgs, })ill; whei:c the inde[inil;c is hi I, hc conse-

qllOlil,:

[6) A 81,11de, tit w i l t a, tt e u d the COli['creliCe it' we C(~II ~2~(]0 l;ogei,hoi: e n o u g h lliOli(?y [or her air [are

T h i s ,qCli[;OliCO ,q(!oHis [;o g(;1L, ;i, reading where we aro liOt, l a l k i n g &l)ollb it particul~u' sl0tl(i()tll, (~/11 O[IL(H' exist, on

0ial), or ~d)Olil; i~ typic~d stu(]cul, (a geiloric, reading).

Mol;0Ovo, i;, as noi;ed by Zcewd; (] {).90), t;}10 115(! of ~:llly k i n d o[' o.rch?rcd i;hl:0a,dhig w i l l tend 1;o tail Ibr l~ach- I)el;ers s(}ii[a'~lll':os~ sllc, h {ts:

17) I,;vcry m a n w h o loves lwl: a.pl~reciai,es a wonlan who liw~s w i t h h i m

I"or tliis l,:hid of e×ainple, il, is still possible t,o il~;C ii

sl,a, n d a r d t l y i i t t l n i c SOliH'~.lll;]c,q~ I)lll; o n l y if i, hei!e is SOlile p r i o r level o f FOt'OI!el]CC rcsohll;ioil which reorders the

a.iil;ccedonl, s alld a.lla])llors appropriately. FOI: c.X;llll

I)le, [[' (I 7) is converl,ed i n t o t, he ~(ionl,:ey' selil,el]ce:

1 8) I,]vel'y ill&It w h o loves a. wollH/,ll w h o lives w M i hiin al)preciai, es hot

Whe, n we consider t]lreading (7[' possible wor]ds, as hi (JIMal, e Seuia,ui;]cs ( V e ] t m a n 19{)0)> l:he noed I;o (]i- si, inguish bci;woen L]le orcler o[ eva,[uai, ion aitd l;he oi; (ler of I)resentat, ion I)ccolues inore cl(~ar cut,. (ton,sider I,i:yiug LO 17e]'l()r[n i;hroa,dh]g hi le['D-l;o-rigiil, Ol:dO..r t l i l ring in/;el:ITrel, al;iol] o [ l,he s(~ll|;(~ltCO~ ,loire h:fl i f Marg

l<ffl,. Afi, cr processiltg t h e 1)roposition ,ioh, left I;he set, o[' worlds is refined d o w n to those, worlds in which .lolu~ left, Now c o n s i d e r processing 0 M a r y lejT. [Iere we wa,nl; l,o ix;ilti;rodtlce 8OlllC' Wt)i:i([S, [;]lOSO in which noit, hor Ma.ry or & ) h n le['L I lowever, 1,tits is nor allo-

wed by {Jpc[atc ,qenlanl,ics w h M l is ~limilmtiv< cacti

new piece o f in[orma.t, ion cAlli o u l y l'tii'l,[lei: i'e[hic l,hc set; of worlds.

It is woiq;h liOl;ing t, hal; I.he diliii:nll,ic!,~ in l;l'3'illg

1;0 c, onll)ine c]iuiinal;ive seiuaili;ics w i t h le['tq,o-righl, tlu:e~t(linp; a.pply t,o c, ousl;i:a.illl, basc(l ,Seliiaili,ic.s as welt

as I;o Upda,te Sema.ntics. t l a d d o c k (1987) uses iucre.- inentM re{inenient of sets of possible ret~rents, l,'or ex ainl)le , the ell'cot, of processing t/w rabbit in I;he ttOllli plll'aSc Utc ~ rabbil ht l/to hat is to p r o v i d e it set of all l;~dol)ii,s. T h e pi:oces,sing o[' i~t I't?IIII(!S this sel~ to ra.b- bil;s w h i c h are ill SOlllCl,liillg;. h'inaHy, proccsSillg o17 [hc ]lal relines the sol; t,o i:~d)lfits w h i c h al:(} ill a. ha, t. I I o "W(]veI' 1 I1()'~¥ COllSid(;r ]l.i'oces,sing th<~ rclblTit 71l #loltc o f thc bomcs. Ily tile {inie Uu: rabbit in has been proces- so'd, the only ra.bloits i:eula,ining in c.onsidcra,l;ioll are r~d)bits which arc in solncl;hing. T h i s im:orrecLly l:ule,~ (ml, the possibilii, y of t h e uoun p h r a s e referring to a, I:~d)l)ii, which is in n o t h i n g a.l, a.[l. T h e case is *u'l;ually a parMle/ to t h e earlier e x a m p l e o f M a r y introduced som, e, olm, to somethin,q being iimpl)rOl)rial;e if t h e final sontcncc is Mar.Is i*~h'odtu:cd noonc to asqjbodg.

A/ldlough t, hi,s discussion has a r g u e d {haA i~ is no1. possilTle to i;hread the sti~t('s which are us(:d by a, (lyua- lille (71" eliHlimigive setlta.ill, ics troll] 1(;['1, 1;o i'ip;[il,, word by w o M , t;liis sliould not; Im t a k e n as a.ll a, rgtlfiiOllt, against 1,he ilSf! 0|' SIlC,[I ;I, ,S(~,IIItI, III;IcS ill il/creiitelll;a[ iit- l;erpretal, ion. \¥hai, is rcqltired is it slight, ly IIIOl'O i n d i - rect al)proach, [11 the I)l:f.',ql'!ltl, iUll)ielllenlLa, l, ioll , SOlllall- I;IC .M,I;tlCI, III'(;S ( a k i n tt7 logica,l f(71HIIS) ;11:(] b u i l t word

I)y word, allel (;ach StlTIlCtltre i,~ then ewdlta.l;ed indc-

pclldcntly tiSillg ~t dynalnit: SOlilltlll, i(:8 ( w i t h t;hl;C/i..dili{{ l)erf'ornied ~c<:ording I,o t;he sl,rHct;ure o f 1;he logica.I ['Orlrl).

I M P I , E M E N T A T I O N

At, p r e s e n t there, is ~r Iimii;cd implenicn/,al;ion, which i)ertbrui;~ ~t i u a p p i n g from sent, cm'e ['ragmcut.s I;o l'ully scol)ed Iogi('iL[ rel)resenta.t, ions. '1'o illustrate its ope- ration, (:ousider the, Ibllowing discourse:

19) London has a lower. Every pa, reni; shows it ....

We a s s u m e ghat, the [irst so'hi;once h a s 1oceu processed, a m l coilcentra.l,e on i)ro(:esshig i, he l"ra,glnen{. T h e iul-- [)l(:lil(',lll;a,l, iOll COllSiSi,,s (7[ flVO lnodules:

I. A w o r d - J ) y - w o r d hicreiricltl, a,i parser ['or a, lexicldi- sod vcrsioli oi c' del)el/denc, y gl:alltlrlal: ( M i l w a r d , 1992). 'l'hi,s I;akes f]:aginettts o f sentences and liiaps t h e m I,o

Ilnscoped logical forlns.

IN I>UT: E w ' , r y | l a r o l l l ; s h o w s i t OIJ'I'I>UT:

X z . s h o w ( < V , x , p a r e i i t (x) > , < p r o n o u u , y > , z )

2. A n i o ( h d e w h i c h replaces la.inl)di~ a.bstra.ci, c'd varia-. bk~s wil,h cxist;cnl;ial quanl, i[iers in sil;u.

INI)tJT: ()ll|;[)llt frolli ].

()[I'I'I~{ST: s h o w ( < V,x,l)ltrO.lll;(x)),<prOllfTIllt,y>,< _t,z,T>)

~{. A I)l;OiK)Ull c o i n d e x i n g procedlu:e which replace,s i)lX)liOlln varia,1)l<'s w]l;ll a. val:ia,I)le [rolll ghe, s;tlnc son- ICll(;(;~ or froltt the 17recediug COligOxt,.

IN I>lJ'l': ()lll,lTlli;(s) ['i:onl :2 a.ll(I a list o[' wu'ia.bles ava,i

lablo ['roln tile c, ontcxl;.

4. A n o u t s i d e - i n q u a n t i f i e r s e o p i u g a l g o r i t h m b a s e d on l,ewin (1990).

[ N P U T : O u t p u t fl:om 3.

O U ' P P U T I : V ( x , p a r e n t (x),3 ( z , T , s h o w ( x , w , z ) ) ) CUll'PUT2: 3 ( z , T , V ( x , p a r e n t ( x ) , s h o w ( x , w , z ) ) )

5. A n ' e v a l u a t i o n ' p r o c e d u r e b a s e d on L e w i n (I 992), w h i c h t a k e s a logical f o r m c o n t a i n i n g free v a r i a b l e s ( s u c h as t h e w in t h e L F a b o v e ) , a n d e v a h l a t e s it u s i n g a d y u a l n i e s e , n a n t i c s in t h e e o n t e x t g i v e n by t h e pre- c e d i n g s e n t e n c e s . T h e o u t l ) u t is a n e w logical fol:m r e p r e s e n t i n g t h e c o n t e x t as a whole, w i t h all v a r i a b l e s correcLly bottlld.

I N P U T : O u t p u t ( s ) fi'om d, a n d t h e eolltext, 3 ( w , m , t o w e r ( w ) &: h a s ( l o n d o n , w ) )

O U q ) P U T I : ~ ( w , T , t o w e r ( w ) & h a s ( l o n d o n , w ) 8z V ( x , p a r e n t ( x ) ,3 ( z , T , s h o w ( x , w , z ) ) ) )

OUTPU'I?2: 3 ( w , T , ~ ( z , T , t o w e r ( w ) &: h a s ( h m d o n , w ) & V ( x , p a r e n t ( x ) , s h o w ( x , w , z ) ) ) )

.;\~ p r e s e n t , t h e c o v e r a g e of m o d u l e 5 is l i m i t e d , and m o d u l e 3 is a n a i v e c o i n d e x i n g p r o c e d u r e w h i c h al- lows a p r o n o u n to be c o i n d e x e d w i t h a n y q u a n t i f i e d v a r i a b l e or p r o p e r n o u n in t h e c o n t e x t or t h e c u r r e n t Selltence.

C O N C L U S I O N S

T h e p a p e r d e s c r i b e d s o m e p o t e n t i a l a p p l i c a t i o n s 0]7 in- c r e m e n t a l i n t e r p r e t a t i o n . It t h e n d e s c r i b e d t h e series of s t e p s r e q u i r e d in m a p p i n g fi'Oln initial f r a g m e n t s of s e n t e n c e s to p r o p o s i t i o n s w h i c h c a n I)e j u d g e d for p l a u s i b i l i t y . I,'inally, it a r g u e d t h a t t h e a p p a r e n t l y close r e l a t i o n s h i p b e t w e e n t h e s t a t e s u s e d in incre- m e n t a l s e m a n t i c s a n d d y n a m i c s e m a n t i c s fails to hold I)elt)w t h e s e n t e n c e level, a n d briefly p r e s e n t e d a m o r e i n d i r e c t w a y o f u s i n g d y n a m i c s e m a n t i c s in i n c r e i n e n - tal i n t e r p r e t a t i o n .

R E F E R , E N C E S

Alshawi, II. (1990). Hesolving Quasi Logical Forms. Co'mpu- tational Linguistics, 16, p,133-144.

Alt;lnann, (].T.M. and M.J. Steeclinan (1988). Interaction wiH~ Context during Hmnan Speech Coml)rehension. Cognition,

30, 11.191-238.

Barwise, J. (11987). Noun Phrases, Generalized QuanLifiers and Allaphors. ]n P. Gardenfors, F,d., Generalized Quanti- tiers, 11.1-29, Dordrecht: l{eidel.

(Jarletta, J., H. Caley and S. Isard 11993). A Collection of Self-repairs from the Map Task Corl)us. ]{esearch Heport, HGI{C/TR-,tT, Uniwq'sity of Eclinburgh.

Chater, N., M.J. Pickerillg and ]).H. Milward 11994). What is Increnaental Interpretation? ms. '15 appear in Edinburgh Working Papers in Cognitive Science.

Cooper, H. (1993). A Note on t;he l/elal;ionship between Lin- guistic :['heory and Linguistic Engineering. l{esem'ch Heport, IICI~C/ItP-,t2, University of 13clinburgh.

Frazier, L. (1979). On Cornprehendiny .5'e~tences: Syntactic Parsim.] Strategies. Ph.D. Thesis, lJnive.rsil~y of Connecticut. l:)uhlished by lilclim,a University I,inguistics Club. Groenendijk, J. and M. Stokhof (1991). l)ynamic Predicate

Logic. Linguistics and Philosophy, 1~, I).39-100.

Gross, 1)., J. Allen and I). 'lYaum (1993). The THA[NS 91 Dialogues. TllAINS Technical Nol~e 92-.:l, Computer Selence 1)el)i;., University of Hochester.

IIadd0ck, N.J. (1987). Incremental semantic interpretation and incremental syntactic analysis. Ph.l). q'hesis, Univer- sity of Edinburgh.

]laddock, N.J. (:[989). Computational Models of hmrementa] Sentant;ic ]nterpretation. LarLquaqe a~d Cognillve Proces- ses, It, (3/41, Special issue, p.337-368.

Hobbs, J.[I.. and S.M. Shieber (t987). An Algorithm fin' (]e- nerating Quantifier Scoping. Computational Linguistics, 8, t)47-63.

Joshi, A,K. (1987). An Introduction to Tree Adjoining C~rammars. In Mmmster-l/alner, Ed., Mathematics of Lan~ guage, Amsterdam: John Benjamins.

Just, M. and P. Carl)enter 119801. A Theory of Heading from Eye Fixations to Comprehension. Psychological Re- view, 87, p.329-354.

Kurtzman, H.S. mad M.C. MacDonMd 119931. l{esolution of Quantifier Scope Ambiguities. Cognition, 48131, p.243- 279.

l,ewin, i. (1990). A Quantifier Scoplng Algorithm wlthcmt a Free Variable Constraint. In Procccdinos of COLING 90, Helsinki, vol 3, p. 190-194.

Lewin, i. (19921. Dynamic Quantification in Logic and Com- putathJnal Senlantlcs. l~esearch reporl;, Centre for (:ogni- tive Science, University of Edinl)urgh,

Levelt, W.J.M. (1983). Modelling alld Self-Hepair in Speech. Cognition, 1~, p.,t1-10,1.

Marcus, M., D. IIindle, and M. Fleck (1983). D:I'heory: qhlking about Talking about Trees. Ill Procecdisi.qs of the

gist A CL, CamlJridge, Mass. p.129-130.

Marslen-Wilson, W. (1973). Linguistic Structure and Speech Shadowing at Very Short ],atencies. Nature, 244 , p.522- 523.

Mellish, C.S. 119851. Computer lnterprclation of Natural Language Descriptions. Chlchester: Ellis Horwood. Milward, D.IL (1991). Axiomatic (@ammar,

Non-Constituent; Coordination, and incremental Interpre- tation. Ph.D. Thesis, University of Cambridge.

Milward, D.R. (t992). Dynamics, ])ependency (3rammar and [ncremental Interpretation. [n Proecedinqs of CO- LIN(~ 92, Nanies, vo] ,t, i).1095-1099.

Moortgat, M. (t988). Catcgorial lnve.stigatlons: Logical a,d Linguistic Aspects o/the Lambek Calculus, Dordrecht: Fo- ris.

Puhnan, S.G. (1986). (-~rannnaI'S, Parsers, and Memory IA- mitations. Language ,nd Co~lnitiv~ l'rotesses, 1(.?), 1).197- 225.

Hesnik, P. (19921. Left-corner Parsing and Psychological Plausibility. In Proceedings of COLING 92, Nantes, vo[ 1, p.191-197.

Shieber, S.M. and M..Iolu~on (1993). Variations on Incre- lnental Interpretation..lour~*al of l~sycholinguistic Rese- arch, 2212), I).287-318.

Shieher, S.M. and Y. Schabes (19901. Synchronous Tree- Adjoining Grammars. In Proceedings of COLING 90, [[el- sinki, vol 3, p.253-258.

Stabler, E.P. 1199[). Avoid the pedestrian's paradox. In tL Berwick, S. Abney, and C. Tenny, Eds., Principle-Based Parsing: Computation and Psgcholinguisties. l(luwer. Steedman, M. (1988). Colnbinators and Grammars. In

H. Oehrle et al., Eds., Catcgorial Grammars and Natu- ral Language Structures, p a l %442.

Thompson, II., M. Dixon, and J. Lalnping (1991). Compose- Heduce Parsing. In Proceedings of the 29th A CL, 11.87-97. Tomita, M. (19851. Efficient Parsing for Natural Languaqe.

Kluwer.

Veltman F. (1990). ])efaults in Update Semantics. In H. Kaml), Ed., Conditionals, Defaults and Belief Rcvi.sion, DYANA Report 2.5.A, Uentre for Cog,fitive Science, Uni- versity of Edinburgh.

Wirdn, M. (1990). Incremental Parsing and l{eason Main- ienance. In Proceedi'a9 s of COLIN(; 90, Helsilfld, vol 3, p.287-292.

ZeevaG II. (1990). S[.al.ic Semantics. in ,J. van Benthem, F,d., Partial and Dynamic Semantics I, I)YANA Iieport 2.I.A, Centre for Cognitive Science, LTniversity of ]gdlnburgh.