An Integrated Understander

(1)

R o g e r C. S c h a n k M i c h a e l L e b o w i t z L a w r e n c e B i r n b a u m

D e p a r t m e n t of C o m p u t e r S c i e n c e Y a l e U n i v e r s i t y

N e w H a v e n , C o n n e c t i c u t 0 6 5 2 0

A new type o f natural language parser is presented. The idea behind this parser is to map input s e n t e n c e s into the deepest form o f the representation o f their meaning and inferences, as is appropriate. The parser is not distinct from an entire understand- ing system. It uses an integrated conception o f inferences, scripts, plans, and other knowledge to aid in the parse. Furthermore, it does not attempt to parse everything it sees. Rather, it determines what is most interesting and concentrates on that, ignoring the rest.

1. Overview of Conceptual Dependency Parsing O v e r the course of the last ten years, researchers in o u r p r o j e c t h a v e d e s i g n e d and p r o g r a m m e d a large n u m b e r of parsers. T h e task of these parsers was the initial m a p p i n g of natural language into an internal representation. ( N o t e that the first p h a s e of the u n d e r s t a n d i n g process traditionally refers to the discovery of the syntactic f o r m of the input. H o w e v - er, the t e r m " p a r s i n g " can just as meaningfully be applied to w h a t e v e r the first phase of u n d e r s t a n d i n g might be.) In this p a p e r we will discuss s o m e of the p r o b l e m s which h a v e arisen in the d e v e l o p m e n t of parsers, and p r e s e n t a new t h e o r y of the w a y parsing w o r k s in the n o r m a l r e a d i n g process. W e will describe a p r o g r a m which i m p l e m e n t s this t h e o r y and u n d e r s t a n d s n e w s p a p e r stories a b o u t terrorism.

All o u r parsers were p r o g r a m s that m a p p e d E n g - lish s e n t e n c e s into the C o n c e p t u a l D e p e n d e n c y ( C D ) r e p r e s e n t a t i o n of their m e a n i n g . U n d e r l y i n g their c o n s t r u c t i o n was always the m e t h o d o l o g i c a l a s s u m p t i o n that the parsing algorithm that t h e y were to e m p l o y was to be as psychologically c o r r e c t as possible. Thus, our parsers are intended to m o d e l the w a y we believe p e o p l e parse. This m e t h o d o l o g i - cal a s s u m p t i o n b r o u g h t with it an operating principle which was (with one exception to be discussed later) always followed, n a m e l y t h a t the parsing algorithm was a left-to-right, o n e - p a s s o p e r a t i o n without b a c k - tracking. T h e s e parsers were not designed to handle true " g a r d e n p a t h " sentences w h e r e p e o p l e h a v e to b a c k t r a c k .

T h e first parser t h a t we w o r k e d on (Schank and Tesler, 1969), used w h a t we called " r e a l i z a t i o n rules" to m a p English syntactic structures into CD. (This t e r m was t a k e n f r o m L a m b (1966) and signi- fied t h a t we were m a p p i n g f r o m one linguistic level to a n o t h e r . ) T h e p r i m a r y p r o b l e m with this parser was t h a t it violated our m e t h o d o l o g i c a l goal of m o d - eling h u m a n processes. F o r m a n y English sentences t h a t were ambiguous, the algorithm we used exhibit- ed no clear p r e f e r e n c e f o r one i n t e r p r e t a t i o n o v e r another, e v e n though people clearly had such p r e f e r - ences.

In Schank et al. (1970) we p r o p o s e d a solution to r e m e d y this p r o b l e m in the design of a new p a r - ser which we called S P I N O Z A II. S P I N O Z A II was to use the C D r e p r e s e n t a t i o n itself to drive the parse. T h a t is, during the parsing process, the m e a n - ing that h a d b e e n u n d e r s t o o d up to a n y point would help in the d e t e r m i n a t i o n of the m e a n i n g of the rest of the sentence. This idea b r o u g h t with it the c o n - c o m i t a n t idea that, since m e a n i n g would be driving the parse, we really might n o t h a v e to rely v e r y m u c h on syntax to do our parsing. (Wilks (1973) was w o r k i n g on a similar idea at the same time a n d his view h e l p e d to s u p p o r t o u r o w n belief in the feasibility of the idea.) W e did n o t believe that we could a v o i d s y n t a x altogether. R a t h e r , it was o u r view t h a t relying on m e a n i n g c o n s i d e r a t i o n s first would drastically reduce our p a r s e r s ' d e p e n d e n c e on syntax.

Copyright 1980 by the Association for Computational Linguistics. Permission to copy without fee all or part of this material is granted provided that the copies are not made for direct commercial advantage and the Journal reference and this copyright notice are included on the first page. To copy otherwise, or to republish, requires a fee and/or specific permission.

0 3 6 2 - 6 1 3 X / 8 0 / 0 1 0 0 1 3 - 1 8 5 0 1 . 0 0

(2)

Roger C. Schank, Michael Lebowitz, and Lawrence Birnbaum An Integrated Understander

While these parsers were being developed, o t h e r researchers were devising methods for parsing natural language input into an internal r e p r e s e n t a t i o n . Most of these m e t h o d s c o n c e n t r a t e d on the syntax of sentences. One very popular technique has been the A u g m e n t e d Transition N e t w o r k ( A T N ) . Parsers of this sort have been discussed in T h o r n e , et. al. (1967), B o b r o w and Fraser (1969), Woods (1969) and Kaplan (1975). A parser strongly related to A T N ' s is in Winograd (1972). A T N ' s have t e n d e d to deal primarily with syntax, perhaps occasionally checking a few simple semantic properties of words. E v e n more closely tied to syntax are the parsers based on T r a n s f o r m a t i o n a l Grammar, such as Plath (1974). A more recent parser which views syntax as an isolatable sub-part of language understanding is in Marcus (1979). T h e i m p o r t a n t thing to n o t e a b o u t all of these programs is that they view syntactic parsing as a process totally isolated from the rest of understanding. Syntax drove these systems, although in some cases the syntactic parser was al- lowed to request semantic information. Some speech u n d e r s t a n d i n g systems, H e a r s a y - I I in particular ( C M U C o m p u t e r Science Speech G r o u p , 1976) use a more integrated approach, but they are only mar- ginally c o n c e r n e d with the level of processing that we are interested in. Our view has always stressed the integration of semantics and syntax in parsing.

S P I N O Z A II was only partially finished w h e n it was a b a n d o n e d for r e a s o n s o t h e r t h a n academic ones. A few o t h e r attempts were made at starting it up again until Chris Riesbeck designed a parser that was similar in spirit, but different in form from SPI- N O Z A II (see Riesbeck, 1975). His program was based on what he t e r m e d requests, a form of p r o d - uctions (see Newell, 1973). Requests were activated w h e n e v e r expectations a b o u t some syntactic or semantic i n f o r m a t i o n could be made, and were exec- uted if the expectations they e m b o d i e d came true. Thus, e x p e c t a t i o n s guided the parse, making Riesbeck's system, later called ELI, very top d o w n (see Riesbeck and Schank, 1976).

E L I was used as a f r o n t end to SAM our script- b a s e d u n d e r s t a n d i n g system (Schank et al., 1975 and Cullingford, 1978), and was c o m b i n e d with G e r s h m a n ' s ( 1 9 7 7 ) work on noun groups to provide a parser that could handle very complex sentences.

One of the m a j o r problems with E L I is its fragili- ty. G r a n g e r (1977) designed F O U L - U P , an adjunct to E L I which determines the meaning of u n k n o w n words in context, and this p r o d u c e d a more robust parser. But, in actual d a y - t o - d a y use, students have o f t e n f o u n d it simpler to design special purpose parsers p a t t e r n e d after E L / that are less c u m b e r s o m e and easier to use. C a r b o n e l l ' s ( 1 9 7 9 ) P O L I T I C S program, a m o d e l of subjective u n d e r s t a n d i n g of political events, for example, uses a parser that is

similar to ELI, but was written by Carbonell him- self.

P e r h a p s the m o s t i m p o r t a n t f e a t u r e of Carbonell's work is that it has pointed out to the rest of u s a major flaw in our reasoning behind the design of large u n d e r s t a n d i n g systems. We have always leaned in the direction of modularity in the design of our programs, b o t h because this has always b e e n considered good programming style, and because, since our systems are very large, each sepa- rate module has o f t e n b e e n the work of a different person.

But, this m o d u l a r i t y has caused a n u m b e r of problems. A n y understanding system that we build, for example, should ideally use E L I as a f r o n t end. But E L I is a very large and c u m b e r s o m e program to work with. F u r t h e r m o r e , there is a n o t h e r practical problem, n a m e l y that the v o c a b u l a r y for a n y new domain to be handled by some system we set up is unlikely to be already present in ELI. Since in E L I the definitions of words are in a sense programs themselves, any new system will require the writing of a large part of its parsing p r o g r a m from scratch in any case. This practical p r o b l e m leads to a much more interesting issue. In the same way that we realized years ago that it was i m p o r t a n t to take advantage of the p o w e r of the CD r e p r e s e n t a t i o n s available to us to build a more integrated parsing system, any new parser designed for a new system should, in principle, take advantage of the higher level understanding processes that are a part of the new system. Thus, P O L I T I C S can parse more effectively if it can use n o t only the partially c o n s t r u c t e d CD r e p r e s e n t a t i o n of what it has already u n d e r s t o o d , but also its place in the ideology it is using, its over- all significance, and so on. That is, modularity is, in an i m p o r t a n t sense, a disadvantage. W h y not capi- talize on everything that is available to help parsing along? People are no more likely to use only syntax and some particular notions of m e a n i n g (but not others) to help in the parsing than they are to use

only syntax. Understanding is a completely inte-

grated process. T h e idea of building m o d u l a r systems has h a m p e r e d advances in parsing, because the full range of o u r k n o w l e d g e should obviously be available to help disambiguate, find a p p r o p r i a t e word senses, and just as importantly (as we shall see later in this paper) to help us know what to ignore.

It should be emphasized what we mean when we say that modularity was a handicap in parser devel- opment. Clearly from a programming point of view our parsers must be modular. H o w e v e r , if modules seem to be spending all of their time communicating with each other, then the particular modularization scheme must be suspect - the modules really form an integrated unit. Since communicating amongst modules tends to be hard, the t e n d e n c y is to avoid it.

(3)

This c a n result in p r o c e s s e s which should i n t e r a c t strongly b e c o m i n g isolated f r o m e a c h other. This is w h a t h a p p e n e d with the m o d u l a r i z a t i o n s t r a t e g y we h a v e described.

2. Paying Attention To Less Than Everything

O n e of the m a j o r p r o b l e m s with SAM, and also with E L I as a p a r t of SAM, was its inability to h a n - dle texts for which it was u n p r e p a r e d . A new v o c a b - ulary item, d o m a i n of discourse, or previously u n e n - c o u n t e r e d s y n t a c t i c c o n s t r u c t i o n could, a n d o f t e n would, t h r o w things into disarray. O n e of the out- puts p r o d u c e d b y S A M was a s u m m a r y of w h a t it had read. It s e e m e d to us t h a t we could p r o d u c e essentially the same output with a m u c h m o r e r o b u s t and m u c h faster p r o g r a m , F R U M P ( D e J o n g , 1977). F R U M P does not process e v e r y w o r d of e v e r y story that is input. R a t h e r , it has e m b o d i e d within it a t h e o r y of skimming that guides it in w h a t it is reading. F R U M P skims for w h a t it is interested in - usually the items of i n f o r m a t i o n it wishes to include in its s u m m a r y f o r a n y particular d o m a i n t h a t it has k n o w l e d g e about. F R U M P is thus a highly t o p - d o w n s y s t e m and f o r this r e a s o n it c a n n o t be considered as a r e p l a c e m e n t for SAM. S A M could, in principle, r e s p o n d to inputs it was u n p r e p a r e d for, although in practice this did not h a p p e n v e r y often! F R U M P c a n n o t r e s p o n d to aspects of stories it is u n p r e p a r e d for; but t h e n n e i t h e r is it u n a b l e to p r o c e s s such stories at all.

F o r our p u r p o s e s here h o w e v e r , w h a t is particu- larly interesting a b o u t F R U M P is that it is an e x a m - ple of a working, robust, integrated (that is, n o n - m o d u l a r ) system. F R U M P ' s parser is virtually indis- tinct f r o m its i n f e r e n c e r . T h e r e a s o n is simple. F R U M P k n o w s w h a t it needs to find in a story. It has rules f o r h o w to find these things, which c a n be either i n f e r e n c e rules or p a r s i n g rules. But such rules are really just the low-level m a n i f e s t a t i o n s of higher level decisions t h a t h a v e b e e n m a d e on the basis of m a n y c o n s i d e r a t i o n s , only s o m e of which are related to parsing. F R U M P works as well as it does b e c a u s e its interests guide w h a t it looks for. It can ignore w h a t it is n o t interested in and c o n c e n - trate on w h a t it w a n t s to know.

N o w let's consider h o w a normal, literate adult reads a story, a n e w s p a p e r story, for instance. We h a v e considered seriously the question of w h e t h e r a h u m a n reads in detail, like SAM, or skims, like F R U M P , in his n o r m a l reading mode. And, although we possess no hard evidence one w a y or the other, we n o w feel that a h u m a n is m o r e F R U M P - l i k e t h a n we previously believed. If this is true, it has i m p o r - tant implications a b o u t w h a t a parser ought actually to l o o k like. We are n o t suggesting t h a t F R U M P ' s p a r s e r is adequate. Clearly it is not, as it misses significant aspects of m a n y stories. O n the o t h e r hand,

s o m e kind of c o m b i n a t i o n of F R U M P parsing and E L I p a r s i n g m i g h t m a k e f o r a v e r y p o w e r f u l a n d r o b u s t s y s t e m for story understanding.

3. Time of Processing in Parsing

One of the major factors to be considered in

discussions of the design of a h u m a n - l i k e parser is the speed with which h u m a n s c a n r e a d text. C o n s i d - ering all the i n f e r e n c e s a n d bringing in of b a c k - ground k n o w l e d g e and o t h e r p r o b l e m s that an un- d e r s t a n d e r m u s t deal with in the course of reading or listening to a sentence, people are very fast at the job. T h e y finish understanding, for the m o s t part, as soon as the s e n t e n c e they are hearing is finished being uttered. This implies that the a m o u n t of time that they h a v e available for inferencing and knowledge a p p l i c a t i o n c a n n o t wait until the end, a f t e r parsing is finished. R a t h e r , such additional processes m u s t be going on at the same time as parsing is going on. This is c o n f i r m e d b y psychological evi- d e n c e such as M a r s l e n - W i l s o n ( 1 9 7 5 ) which uses errors in the shadowing of sentences to show high- level p r o c e s s i n g m u s t b e o c c u r r i n g t h r o u g h o u t the reading of a sentence. Such a conclusion certainly m a k e s the a r g u m e n t we w e r e stating a b o v e m u c h m o r e significant. It implies that h u m a n processes are highly integrated. T h a t is, p e o p l e m u s t b e inferring f r o m the early p a r t s of a s e n t e n c e b e f o r e they e v e n h e a r the latter parts of the sentence. If this is so, t h e n it also follows t h a t p e o p l e will m a k e use of w h a t e v e r else t h e y discover, thus allowing w o r d sense identification, etc., to b e a f f e c t e d b y higher level processes. Thus, as models of h u m a n p r o c e s s - ing, parsers that first do their j o b c o m p l e t e l y and t h e n send their results off to inferencers m a k e no sense.

T h e r e is a f u r t h e r c o n s e q u e n c e to this as well. We must ask ourselves w h e n this n o n - p a r s i n g t y p e processing takes place. T h e r e are t w o possible answers. Either p e o p l e e m p l o y parallel processes and it all goes on at the same time, or if processing is serial, space m u s t be being m a d e to do this w o r k at the e x p e n s e of s o m e t h i n g else. This s o m e t h i n g else is likely to be the c o m p l e t e processing of e v e r y w o r d that is seen. T h a t is, in the serial view, not all words are equal. S o m e words get a lot of the processing time (those t h a t h a v e great syntactic, semantic and inferential i m p o r t a n c e for e x a m p l e ) a n d others h a r d - ly get noticed.

N o w the question of w h e t h e r the serial or parallel e x p l a n a t i o n is c o r r e c t is really n o t r e s o l v a b l e here. H o w e v e r , e v e n with s o m e parallel processing, it seems plausible t h a t the total processing capability available at a n y o n e time for use in u n d e r s t a n d i n g m u s t h a v e s o m e b o u n d s , and t h a t the s p e e d of input m u s t o f t e n o v e r w h e l m those bounds. Thus we are

(4)

still left with the necessity of processing s o m e words at the e x p e n s e of others.

The serial explanation, then, p r e s u m e s a m o d e l of parsing which is in s o m e places incomplete. As we h a v e discussed e l s e w h e r e (Sehank, 1975, f o r in- s t a n c e ) , the p r o c e s s of doing a c o m p l e t e p a r s e is e x t r e m e l y complex. Simply stated, it takes n milli- seconds to r e a d a w o r d and it takes m milli-seconds to c o m p l e t e l y process a word. Since it seems quite likely t h a t m is m u c h larger t h a n n in o r d i n a r y s p e e c h and reading, a n d since w o r d s c o m e in s t r e a m s in o r d i n a r y speech and reading, t h e n it is o b v i o u s that p e o p l e c a n n o t be c o m p l e t e l y processing e v e r y w o r d t h e y hear. W h a t is m o r e likely the case is that t h e y are deciding w h a t to p a y serious a t t e n - tion to a n d w h a t to p a y casual a t t e n t i o n to as they go. Such decisions can be explained on the basis of m a n y factors. O n e m o s t o b v i o u s o n e is interest. T h a t is, p e o p l e are liable to p a y a t t e n t i o n to ( t h a t is, d e v o t e their processing time t o ) w h a t interests them. W e h a v e discussed the c o n c e p t of interest and its r a m i f i c a t i o n s f o r the i n f e r e n c e p r o b l e m in S c h a n k (1978). T h e m a i n conclusion there was that inference is controlled b y interest. This is likely also to be true in this revised view of the parsing process t h e n b e c a u s e we are n o w viewing the entire understanding process as an i n t e g r a t e d p h e n o m e n o n .

C o n s i d e r the following sentence:

A small twin engine airplane carrying federal marshals a n d a convicted m u r d e r e r w h o was b e i n g t r a n s p o r t e d to L e a v e n w o r t h c r a s h e d during an e m e r g e n c y landing at O ' H a r e Air- p o r t yesterday.

Intuitively, s o m e p a r t s of this s e n t e n c e are m o r e i n t e r e s t i n g t h a n others. But m o r e t h a n that, it is crucial, according to the idea stated a b o v e with re- s p e c t to the a m o u n t of p r o c e s s i n g time available, t h a t the p r o c e s s i n g of s o m e words m u s t t a k e less time t h a n the time it takes to r e a d or h e a r them. N o w at first glance this m a y s e e m a bit bizarre. H o w can a word be p r o c e s s e d in less time t h a n it takes to r e a d or h e a r it if reading or hearing it is a p a r t of that processing? Yet we are in precisely this p a r a d o x i c a l situation if we hold to the idea that the processing of a n y o n e w o r d in a s e n t e n c e can take longer t h a n the time it takes to r e a d or h e a r it, since it takes no longer to process an entire s e n t e n c e t h a n it d o e s to h e a r it a n d since the individual w o r d s c o m e in at such a rate that there is no time b e t w e e n t h e m in which to process. (This is obviously the case since just finding the w o r d b o u n d a r i e s in a sen- t e n c e is a v e r y c o m p l e x t a s k b e c a u s e the s p e e c h s t r e a m is continuous.)

Since the a m o u n t of processing time available is limited b y the rate of flow of the input (which is continuous for speech), t h e n s o m e words are p r o b a -

bly not being p r o c e s s e d at all (or in a n y case they are p r o c e s s e d so partially that t h e y are hardly seen). Since the m o s t i m p o r t a n t words o f t e n c o m e at the end of a phrase, the p r e c e d i n g words m a y be virtually ignored until t h e y can be ' g a t h e r e d u p ' right to left. T h e n t o p d o w n processing helps the u n d e r s t a n - der k n o w w h a t to ignore. A c c o r d i n g to this scheme, words are stored in a b u f f e r a n d virtually ignored until a w o r d that initiates processing is found. W h e n such a w o r d is found, the words in the b u f f e r are g a t h e r e d up a n d their analysis c o m p l e t e d . W o r d s which initiate processing usually a p p e a r at the end of phrases or b r e a t h groups.

In o r d e r to p r o c e s s a n o u n p h r a s e such as "small twin engine a i r p l a n e " then, we m u s t assume that a p r o c e s s o r virtually ignores all the w o r d s until ' a i r p l a n e ' , simply m a r k i n g their e x i s t e n c e in s h o r t t e r m m e m o r y f o r retrieval a f t e r the h e a d n o u n is found. O n c e we k n o w t h a t ' a i r p l a n e ' is the subject of the sentence, e x p e c t a t i o n s can be g e n e r a t e d that allow us to h a v e a b e t t e r idea of w h a t to look f o r ( a n d t h e r e f o r e of w h a t to ignore). F o r example, the m e a n i n g of ' c a r r y i n g ' c a n be virtually ignored, since as we are only beginning to recognize w h a t word it is, we h e a r a b o u t the marshals and the m u r d e r e r and decide to p a y a t t e n t i o n to those items.

T h e point here is that we are really not seeing things one w o r d at a time, but r a t h e r since we are seeing a continuous s t r e a m we can pick out w h a t we find interesting, go b a c k to discover just those rela- tionships t h a t c o n n e c t t o g e t h e r w h a t we are interested in and virtually ignore the rest. D o we care that the v e r b ' c a r r y i n g ' was used instead of ' c o n t a i n i n g ' , or t h a t the c o n s t r u c t i o n used was not "in which they were f l y i n g " ? We have already p r e d i c t e d that the r e l a t i o n s h i p b e t w e e n the p e o p l e a n d the airplane was c o n t a i n m e n t b e c a u s e t h a t is w h a t it ordinarily is. We n e e d only c o n f i r m the fact that nothing c o n - tradicts this prediction and this c a n be d o n e on the fly. U n d e r this t h e o r y there is little w o n d e r at the fact that u n d e r s t a n d e r s f r e q u e n t l y c a n n o t r e m e m b e r the actual w o r d s t h a t t h e y read. T h e y m a y n e v e r h a v e actually r e a d t h e m at all!

A t h e o r y of partial parsing then, says that m o s t words are barely noticed until s o m e r e a s o n is f o u n d to p a y a t t e n t i o n to them. A m a j o r issue in our t h e o - ry then is h o w we k n o w w h a t words we m u s t p a y a t t e n t i o n to and begin to p r o c e s s seriously.

If certain e l e m e n t s of language are s k i p p e d en- tirely, or p r o c e s s e d o n l y slightly in the c o u r s e of n o r m a l understanding, o n e might ask w h y t h e y ap- p e a r at all. W h y d o e s n ' t the a u t h o r just omit t h e m ? T h e a n s w e r to this is t h a t the s a m e s t o r y c a n be u n d e r s t o o d b y different readers with varying degrees of c o m p l e t e n e s s . It is possible for a r e a d e r paying a t t e n t i o n to e v e r y detail of a s t o r y to d i s c o v e r n u a n c e s that a partial u n d e r s t a n d e r like the one we

(5)

are describing would miss. H o w e v e r , it is our belief that m o s t of the time, w h e n dealing with m e d i a such as n e w s p a p e r s , p e o p l e do not do extensive processing, and y e t are able to e x t r a c t the vast m a j o r i t y of i n f o r m a t i o n of interest. T h a t is the process we are a t t e m p t i n g to m o d e l here. W e do not claim that it r e p r e s e n t s the only level of processing a p e r s o n can use, b u t we do believe it is a v e r y i m p o r t a n t and widely used level of understanding.

( T h e r e is an aside that is w o r t h making at this point. W e h a v e t a l k e d o v e r the y e a r s a b o u t h o w e x p e c t a t i o n s drive various parts of the u n d e r s t a n d i n g process (Riesbeck, 1975 and Cullingford, 1978 for example). T h e c o n t r a s t here is b e t w e e n e x p e c t a t i o n - b a s e d processes and interest-driven processes. O b v i - ously the m o s t p o w e r f u l and i m p o r t a n t m e c h a n i s m s available to an u n d e r s t a n d e r are b o t h e x p e c t a t i o n and interest driven at the same time.)

R e a s o n s f o r c o m p l e t e l y processing a given w o r d occur at all levels of the system. S o m e of these are: parser expectations: if the p a r s e r expects a certain kind of word, the s a t i s f a c t i o n of t h a t e x p e c t a t i o n can be t a k e n as an e x t r e m e l y i m p o r t a n t force in the parser. Thus, a parser might function best t h a t ex- p e c t e d certain syntactic or c o n c e p t u a l types to the extent t h a t it ignored everything else until it f o u n d them. This is again a violation of the idea of left to right parsing since a parser might not b e c o m e interested in s o m e t h i n g until it h a d a l r e a d y p a s s e d it, ignored it, and t h e n seen an item that caused e x p e c - tations to be raised t h a t could only be satisfied b y checking b a c k w a r d s .

syntax: m a i n n o u n s in a n o u n p h r a s e c a n cause a p r o c e s s o r to try to gather up its modifiers for which there is a need or interest. C e r t a i n function words cause words to be paid a t t e n t i o n to if their interest value has b e e n predicted. Thus, ' t o ' is noticed to the extent that it c a n focus a t t e n t i o n on the following h e a d n o u n if it has already b e e n d e t e r m i n e d that a location is e x p e c t e d and desired.

interest values: h o w does the p a r s e r decide w h a t i t wants to pursue? O b v i o u s l y we n e e d a fully inte- g r a t e d s y s t e m w h e r e the p a r s e r and m e m o r y talk during the parsing of a sentence. W i t h o u t such integration, there would be no overriding r e a s o n s f o r noticing o n e thing and not another. It is the role of episodic m e m o r y and world knowledge to i n f o r m the parser of w h a t to p a y a t t e n t i o n to. I n t e r e s t values are stored in m e m o r y as p a r t of the k n o w l e d g e associated with concepts. C e r t a i n c o n c e p t s are nearly always interesting, others are interesting in certain circumstances. M o r e i m p o r t a n t l y , certain things are interesting on the basis of w h a t has p r e c e d e d t h e m - i n t e r e s t i n g n e s s is a d y n a m i c p r o p e r t y . Thus, the object of a shooting might be e x p e c t e d to be m o r e interesting if the s h o o t i n g t o o k place in an e m b a s s y as o p p o s e d to a generally l o w - i n t e r e s t location such

as a bar. (But of course, c o n t e x t s can be c r e a t e d where bars c a n be very i m p o r t a n t . This is w h y it is n e c e s s a r y to h a v e a d y n a m i c m e m o r y available as o p p o s e d to just a dictionary.)

top level expectations: if we are r e a d i n g a b o u t an event t h a t fits into a high level k n o w l e d g e structure such as a script or a plan, p r e d i c t i o n s f r o m t h a t script or p l a n c a n focus interest during the p r o c e s s - ing of a sentence. Thus, we c a n k n o w that the target of an assassination and the identity of the assassin are of critical i m p o r t a n c e in reading a story a b o u t an a s s a s s i n a t i o n a n d we c a n thus focus in on t h o s e items as t o p - d o w n predictions during parsing.

T o see h o w all this is used consider the s e n t e n c e f o r m " X w e n t to visit Y " : M e m o r y is accessed to see if X is interesting b e c a u s e it is a m a i n n o u n and b e c a u s e it is a p e r s o n . W h e n no i n f o r m a t i o n is f o u n d , the p r o c e s s i n g should b e f a s t e r t h a n w h e n i n f o r m a t i o n is found. Thus, w h e n X is ' J o h n ' or ' S a m ' we p r o c e e d quickly. If X w e r e ' H e n r y K i s s i n g e r ' or ' y o u r m o t h e r ' we would p r e s u m a b l y p r o c e e d m o r e slowly b e c a u s e m o r e e x p e c t a t i o n s a b o u t their b e h a v i o r t h a t are of interest would b e found.

' W e n t ' is an item t h a t urges us to continue p r o c - essing since it has no specific m e a n i n g in isolation. ( T h a t is, we could have ' w e n t c r a z y ' , ' w e n t fishing', ' w e n t to B o s t o n ' , and we c a n ' t do anything until we see the next words. T h e t h e o r y here is w h y specu- late at all, just go on.) 'Visit' is a w o r d that calls up a script ( $ V I S I T ) if the o b j e c t of the visit is an equal or a family m e m b e r . But o t h e r scripts can be called up b y the w o r d visit t h a t are distinct f r o m $ V I S I T . If the o b j e c t of visit is ' m u s e u m ' or ' C o n g r e s s ' we would get quite d i f f e r e n t scripts or e v e n no available script at all. ( W h a t script d o e s ' w e n t to visit a m o r t u a r y ' bring up?) Obviously, the p r o b l e m here is that 'visit' is also almost totally ig- n o r e d since it t o o m e a n s v e r y little. Its real p u r p o s e is to get us i n t e r e s t e d in the o b j e c t t h a t follows next. T h a t is, we d o n ' t really start p r o c e s s i n g this s e n t e n c e d e e p l y until we see w h a t Y is. Then, if Y m e e t s certain criteria we instantiate $VISIT. If Y is a m e m b e r of the o p p o s i t e sex, we have an a m b i g u - ous s e n t e n c e in t e r m s of script (and thus processing). In t h a t case, either $ V I S I T or the R O - M A N C E script would be applicable, a n d we will n o w w a n t to figure out which.

N o t i c e that while m o s t of the e x a m p l e s we h a v e presented, and those we will present, describe stories in t e r m s of scripts, there is no r e a s o n o u r processing ideas could n o t b e used to handle stories b e t - ter r e p r e s e n t e d b y other k n o w l e d g e structures, such as plans and goals (Schank a n d A b e l s o n , 1977, Wil- ensky, 1978). In fact, we b e l i e v e t h e y will a p p l y g e n e r a l l y to w h a t e v e r is r e p r e s e n t e d in m e m o r y . Scripts are p r o m i n e n t in o u r early d e v e l o p m e n t of

(6)

J

o u r parsers, b e c a u s e the domain we are c o n c e n t r a t - ing on, n e w s p a p e r stories, tends to involve m a n y stereotypical situations. H o w e v e r , we expect to ex- t e n d this w o r k to handle stories requiring different t y p e s of representations.

4. An Example

T h e following is a s e n t e n c e t a k e n f r o m a f r o n t page story in the N e w Y o r k Times:

An A r a b i c s p e a k i n g g u n m a n shot his w a y into the Iraqi E m b a s s y h e r e (Paris) y e s t e r d a y m o r n i n g , held h o s t a g e s t h r o u g h m o s t of the d a y b e f o r e surrendering to F r e n c h p o l i c e m e n and t h e n was shot b y Iraqi security officials as he was led a w a y b y the F r e n c h officers.

We will n o w e x a m i n e this s e n t e n c e w o r d b y w o r d a n d consider the kind of processing we desire in an i n t e g r a t e d u n d e r s t a n d i n g s c h e m e . T h e p r o g r a m we will describe later does s u c h processing. O u r m o d e l will skip the u n i n t e r e s t i n g p a r t s a n d build up its r e p r e s e n t a t i o n w h e n n e c e s s a r y , a t t e m p t i n g to be c o m p l e t e l y finished with e a c h s u b - p a r t at the right time. T h a t is, we desire that the m o d e l we p r o p o s e b e finished processing only slightly a f t e r the input has b e e n received - just as a p e r s o n would do.

O n e i m p o r t a n t point here is that although we will discuss this s e n t e n c e in a l e f t - t o - r i g h t w o r d - b y - w o r d fashion, there is no real r e a s o n to believe that hu- m a n u n d e r s t a n d i n g goes one w o r d at a time. A c t u a l - ly, words e n t e r in chunks, b o t h visually (in reading) a n d aurally (in speech). We can thus process the s a m e way. Thus, f o r e x a m p l e , the n e x t w o r d to be r e a d is available f o r a n y w o r d u n d e r consideration. Such an a s s u m p t i o n c a n simplify the p r o b l e m of disambiguation.

* * * * * * * * * * * * * * * * * * * * * * * * * * * * *

An A r a b i c s p e a k i n g gunman...

A N is a w o r d t h a t n e e d only be saved initially. This m e a n s t h a t it is l o o k e d up in the d i c t i o n a r y , a n d w h a t is f o u n d t h e r e are instructions to go to the n e x t w o r d and place A N in s o m e f o r m of short t e r m m e m o r y ( S T M ) to be e x a m i n e d later.

A R A B I C is listed in the dictionary as a w o r d that is skippable w h e n it has b e e n p r e c e d e d b y a skippable word, so it is s k i p p e d and placed in STM. ( T h a t A R A B I C is skippable has a l r e a d y b e e n d e t e r m i n e d and is simply l o o k e d up. T h e p r o c e d u r e f o r d e t e r - mining w h a t can be s k i p p e d is obviously one of the interesting p r o b l e m s in the issue of the d e v e l o p m e n t of language ability. (See Schank and Selfridge, 1977, f o r a discussion of these issues.) In general, adjectives can be skipped, though not all can be. In p a r - ticular, ' R u s s i a n ' could n o t b e s k i p p e d b e c a u s e it c a n also be an actor. Also, adjectives designated as

interesting m a y n o t be skipped, i. e. 'disgusting', ' m u r d e r o u s ' , ' l e c h e r o u s ' , etc.)

S P E A K I N G is also skippable as long as no potential actors h a v e b e e n so far e n c o u n t e r e d . A search for A C T O R s in S T M finds none, so this w o r d is also skipped.

G U N M A N is m a r k e d as an A C T O R , as a N O U N , and as a H I G H I N T E R E S T A C T O R . T h e f a c t that we h a v e a H I G H I N T E R E S T w o r d c a u s e s us to create t o p - d o w n requests to fill in certain i n f o r m a - tion, in particular we n o w w a n t to k n o w the answers to the following questions:

W H O is he? . . . . causes us to g a t h e r up s t a c k e d adjectives (e.g. A R A B I C ) and add t h e m to the m e m o r y t o k e n f o r this G U N M A N

W H A T did he do? . . . . this is a n s w e r e d b y an item f o u n d on the t o k e n f o r G U N M A N , n a m e l y S H O O T . Thus, an i n f e r e n c e t h a t the g u n - m a n shot or will s h o o t s o m e b o d y is m a d e here b e f o r e a n y t h i n g else c o m e s in as input W H O M did he s h o o t ? . . . . causes us to be i n t e r e s t e d

in the syntactic o b j e c t of the v e r b which we assume will be S H O O T

W H Y did he s h o o t ? . . . . causes us to look f o r a rea- s o n

W H E R E did this h a p p e n ? . . . . causes us to l o o k for a location

W H A T S C R I P T S might this instantiate? . . . . G U N - M A N can itself cause a script to be instantiated. P r i m e c a n d i d a t e s are $ R O B B E R Y , $ T E R R O R I S M and $ K I D N A P . W e c a n n o w look f o r c o n f i r m a t i o n in the rest of the sentence.

T h e f o r m u l a t i o n of the a b o v e questions (as requests, see R i e s b e c k , 1975) n o w guides the parsing of the rest of this sentence. H e r e it is i m p o r t a n t to point out that m u c h m o r e t h a n just parsing is being guided at this point b y these requests. This i n f o r m a - tion is w h a t we are i n t e r e s t e d in as understanders. W e are a c t u a l l y p e r f o r m i n g the entire p r o c e s s of u n d e r s t a n d i n g this s t o r y T h e s e r e q u e s t s r e l a t e to m a t t e r s of parsing a n d inference a n d scripts application and goal pursuit as well.

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

shot his w a y into the Iraqi E m b a s s y . . .

S H O T is e n c o u n t e r e d a n d i m m e d i a t e l y is f o u n d to satisfy an e x p e c t a t i o n t h a t was derived f r o m G U N - M A N . Satisfying an e x p e c t a t i o n of this sort is the w a y that c o n c e p t u a l structures are built and we n o w build the first one, n a m e l y a S H O O T action with the g u n m a n t o k e n as a c t o r and an unfilled final direction f o r the shooting. This unfilled slot is m a r k e d as the s a m e o n e t h a t satisfies the a n s w e r to the W H O question asked b e f o r e a n d the p a r s e r n o w is inter-

(7)

ested in satisfying that r e q u e s t b y looking for the next main n o u n in the n e x t n o u n phrase.

H I S is skipped and held in S T M as b e f o r e .

W A Y does n o t satisfy the e x p e c t a t i o n to fill the e m p t y slot. W A Y is also listed as b o t h skippable and pointing to a direction or location. A request is set up for the location and we a t t e m p t to skip until we find it.

I N T O dictionary entry says to k e e p on going and it is skipped.

T H E is saved and skipped. I R A Q I is saved a n d skipped.

E M B A S S Y is f o u n d to be a location and is set up as the l o c a t i o n of the S H O O T event. F u r t h e r m o r e , E M B A S S Y is m a r k e d as interesting a n d a place of political significance. This latter piece of i n f o r m a t i o n satisfies the r e q u e s t f o r i n s t a n t i a t i n g the $ T E R R O R I S M script that we h a d predicted ( a m o n g others) f r o m G U N M A N . Since E M B A S S Y is interesting, its requests are activated. O n e of these is for a c o u n t r y w h o s e E M B A S S Y it is. I R A Q I is thus f o u n d in S T M as filling this request and is picked up.

Setting up $ T E R R O R I S M causes us to lose interest in the r e p r e s e n t a t i o n of the sentence as such and focuses us on setting up a n d filling r e q u e s t s f r o m that script for the r e p r e s e n t a t i o n for the entire story. Thus, we n o w expect answers to the following requests:

W e r e H O S T A G E S t a k e n ?

W h a t d e m a n d s w e r e m a d e ? ( m o n e y ; free political prisoners?)

W a s a n y d a m a g e d o n e ?

W h a t m e a s u r e s to c o u n t e r a c t the terrorist w e r e m a d e ?

( r e t u r n fire; arrest; free hostages?)

* * * * * * * * * * * * * * * * * * * * * * *

here y e s t e r d a y morning...

* * * * * * * * * * * * * * * * * * * * * * *

H E R E always refers to the dateline l o c a t i o n in a news story. It adds this location to the story r e p r e - sentation.

Y E S T E R D A Y is f o u n d to be a time w o r d a n d is thus added to the time slot of the event.

M O R N I N G is also handled in this m a n n e r .

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

held hostages t h r o u g h m o s t of the d a y ....

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

H E L D is skipped since it m a t c h e s n o n e of the requests. It m a t c h e s n o n e of t h e m b e c a u s e the infor- m a t i o n f o u n d a b o u t H E L D in the part of the dic- t i o n a r y we l o o k at at this point is just that it is a verb. N o verbs were p r e d i c t e d so we skip it. W h a t could h a v e c h a n g e d this would have b e e n some interest m a r k i n g under H E L D or o t h e r item of signifi-

cance. N o t e t h a t H E L D is a highly a m b i g u o u s w o r d t h a t p r e v i o u s l y m i g h t h a v e c a u s e d us to m a k e a g r e a t m a n y p r e d i c t i o n s a n d l o o k f o r e v i d e n c e of w h a t sense was intended. With an integrated u n d e r - standing s y s t e m we n e e d n o t do t h a t at all. T h e r e a - son for this can be seen in w h a t h a p p e n s in subse- quent processing of this phrase.

H O S T A G E S is i m m e d i a t e l y f o u n d to satisfy an ex- t a n t request. T h e T A K E H O S T A G E S scene of $ T E R R O R I S M is instantiated. At this point a check is m a d e on the s t a c k e d v e r b to see if doing this is okay. If the stacked v e r b were ' s h o t ' , for example, this instantiation would not work. H E L D is found to b e precisely the kind of w o r d that fits here. T h e i m p o r t a n t point is that the m e a n i n g of H E L D n e v e r h a d to be d e t e r m i n e d in isolation, which is nice b e - cause words like H E L D really do not h a v e a n y p a r - ticular meaning. . Its m e a n i n g is derived f r o m its c o n - nection to H O S T A G E S , and H O S T A G E S is u n d e r - s t o o d t h r o u g h $ T E R R O R I S M . Thus integrated understanding plus " s a v e a n d skip" parsing facilitates processing t r e m e n d o u s l y .

T H R O U G H O U T is f o u n d to point to either a time or place, so a request is m a d e f o r a time or place word. H o w e v e r , at this point our u n d e r s t a n d i n g sys- t e m k n o w s w h a t it is interested in. In particular, satisfying the r e q u e s t s t h a t are still active is v e r y i m p o r t a n t b e c a u s e t h e y are d e a t h - r e l a t e d r e q u e s t s (see Schank, 1978). Thus we virtually ignore the rest of this phrase due to lack of interest.

M O S T is saved and skipped. O F is skipped.

T H E is saved a n d skipped.

D A Y is r e c o g n i z e d and ignored. It also satisfies the l o w level request f o r a time w o r d and this i n f o r m a - tion is a d d e d to w h a t we k n o w a b o u t time to b e used later if we ever get interested in w h a t we h a v e n o w decided is uninteresting.

b e f o r e surrendering to F r e n c h policemen...

B E F O R E is a time ordering w o r d that p r e p a r e s us to set up a n e w e v e n t a n d m a r k its time relative to the preceding event.

S U R R E N D E R I N G is a w o r d t h a t is b o t h m a r k e d as of high interest a n d as part of a n u m b e r of scripts including $ T E R R O R I S M . T h e s u r r e n d e r scene of $ T E R R O R I S M is instantiated and requests are fired o f f concerning the reasons f o r this action, his c a p - tors etc. C e r t a i n w o r d s are m a r k e d as indicating which of these might follow. Thus, ' b e c a u s e ' m a r k s off reasons, and ' t o ' m a r k s o f f captors.

T O tells us that c a p t o r s is coming. F R E N C H is held in STM.

(8)

Roger C. Schenk, Michael Lebowitz, and Lawrence Birnbaum An Integrated Understander

P O L I C E M E N is marked as a noun that can be an A C T O R , so STM is consulted to gather up its relevant components. P O L I C E M A N also is a possible captor (because it is b o t h a h u m a n and an institu- tion, either of which would do), so it satisfies two extant requests.

and then was shot b y Iraqi security officials .... ******************************************

A N D says w h e n e v e r an event has just ended, a new event m a y be coming.

T H E N orders the time of the event.

W A S specifies that the actor stored in STM is the c o n c e p t u a l o b j e c t of the new event. This sets up requests for the actor and the action.

S H O T is f o u n d to be interesting and is treated simi- larly to the w a y that G U N M A N was, except that we do n o t expect the things that were particular to G U N M A N as o p p o s e d to the action he was per- forming. Thus we have:

W H O M did he shoot? . . . . A R A B G U N M A N

W H O shot? . . . . not answered

W H Y did he shoot? . . . . not answered W H E R E did this h a p p e n ? . . . . already k n o w n

W H A T S C R I P T S does this instantiate? . . . . S H O O T can also cause a script to be instantiated. P r i m e c a n d i d a t e s are $ R O B B E R Y , $ T E R R O R I S M and $ K I D N A P , ordinarily.

But we are in a c o n t e x t set up b y

$ T E R R O R I S M . N o n e of the above are nor-

mal continuations of $ T E R R O R I S M . This

causes us to look for plans and goals.

W H A T were the R E S U L T S of this action? -- A request is set up to find the results. If this request is not satisfied the usual results of this action are inferred. In this case death for the object.

Since S H O T is interesting, we need to explain it. N o scripts are available here, so we need to ask w h o would want to kill the G U N M A N and why. These requests are added to the active requests.

BY tells us the actor is to follow.

I R A Q I is stacked and skipped.

S E C U R I T Y is stacked and skipped.

O F F I C I A L S is used to end the processing of the n o u n group. It satisfies the requests for W H O did the shooting, and, as we n o w have an actor, we ask a b o u t w h y he would kill a T E R R O R I S T . This causes us to examine the themes we have for w h y T E R - R O R I S T S might be killed after capture. A t this point we might make the c o n n e c t i o n b e t w e e n I R A - Q I S E C U R I T Y O F F I C I A L S and I R A Q I E M B A S S Y , but that does not lead to an explanation. This causes us to be surprised by this event. We seek to explain it by postulating a R E V E N G E or S H U T H I M

UP type theme, but we are certainly not sure of it.

as he was led away by F r e n c h officers. * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

We are basically done n o w as no further requests need to be satisfied immediately. (We k n o w this after we have seen a period and f o u n d no new requests.) We are still interested in the goals of each of the actors, however, so W H Y requests are still alive.

AS is k n o w n to be a time c o - o c c u r r e n c e word. Since we are not interested in anything that occur- red at the same time unless it is itself interesting, we can n o w skip a h e a d looking for actions or actors that are interesting. AS can also indicate causality, but in that case the semantic predictions set up earlier will find the cause.

H E is skipped.

W A S is skipped.

L E D is uninteresting and is r e c o g n i z e d and then skipped.

A W A Y is skipped.

BY is skipped.

T H E is skipped.

F R E N C H is skipped.

O F F I C E R S is skipped because there are no requests asking for it.

The period tells us we are done.

The final representation for this sentence is:

$ T E R R O R I S M

A C T O R - A r a b g u n m a n P L A C E - Paris, Iraqi E m b a s s y S C E N E S

$ H O S T A G E S - some $ C A P T U R E

A C T O R - F r e n c h policemen O B J E C T - A r a b gunman P L A C E - Paris, Iraqi E m b a s s y

U N E X P E C T E D R E S U L T : A C T O R - Iraqi officers A C T I O N - S H O O T O B J E C T - A r a b g u n m a n R E S U L T

A C T O R - A r a b g u n m a n S T A T E - dead

5. P r o c e s s i n g in I n t e g r a t e d P a r t i a l P a r s i n g

We have written a p r o g r a m which implements the t h e o r y of parsing illustrated above - an I n t e g r a t e d Partial Parser ( h e r e a f t e r IPP). In this section we shall look at h o w this parser works. It was written to handle a limited class of stories, namely n e w s p a - per stories a b o u t terrorism a n d related areas. We have not tried to address all the issues involved in

(9)

parsing. R a t h e r we have c o n c e n t r a t e d on the areas which are crucial to IPP. One obvious problem we have not addressed is words with multiple senses. F o r t u n a t e l y , in the class of stories we are processing, most words, especially the interesting ones, have one strongly p r e f e r r e d sense. IPP has successfully processed over 200 stories t a k e n directly from various newspapers. M a n y of these were processed sight unseen. IPP current has a vocabulary of over 2000 words. The parser is written in LISP, and runs on a D E C System 2 0 / 5 0 .

T h e program's limitations center around its vocabulary and knowledge of the world. 2000 words, a sizable vocabulary for a typical AI program, is still a bit too small, even for stories about terrorism. We are c u r r e n t l y expanding the vocabulary. T h e program is also limited to u n d e r s t a n d i n g stories for which it has appropriate world knowledge. We have c o n c e n t r a t e d on stories that are script-based, b u t as mentioned earlier, we believe the general techniques of IPP will e x t e n d to o t h e r f o r m s of knowledge. IPP's ability can be increased b o t h b y adding more script-type i n f o r m a t i o n , similar to that it already has, and by considering these o t h e r types of world knowledge.

A n o t h e r limitation IPP has is that it has problems with stories which are subtly phrased - those where jumping to a conclusion causes problems. But these are exactly the kinds of stories people have trouble with when they are reading quickly. The projected solution for this problem is to give IPP the capability of going back and reading text in a more careful m o d e than it normally does.

The parsing scheme implemented in IPP is based on classifying the words in the dictionary in terms of what the parser should do with each word as it reads it. Thus, labels such as noun, verb, etc. only make sense in a parser if they cause different processing d e p e n d e n t on seeing such classifications.

It is very well to say, as we have, that a given word should be skipped or saved or whatever. We must make these determinations b e f o r e h a n d , however. Thus the key issues in the realization of this parser are, first, the establishment of a set of categories for the words in the dictionary that will be useful in such a scheme; and, second, a procedure for deter- mining what category a given word fits into. As we will see shortly, the category a word is assigned to may be domain dependent.

L o o k i n g back at the example in the last section, we can see that there are basically three different things that can be done with a word w h e n it is read. It can be skipped, it can be saved and then skipped, or it can be completely processed immediately.

T h e first possibility is that it may simply be skipped. T h e r e are m a n y words which have no signifi-

cant conceptual c o n t e n t for normal reading. E x a m - ples f r o m the story in the last section include the Words 'most', 'way', and 'held'.

The second possibility that we can see f r o m the example is that a word may be saved in some kind of short term m e m o r y and t h e n skipped. Words for which this processing strategy seems a p p r o p r i a t e have some functional purpose or significant conceptual c o n t e n t of a rather dull and uninteresting sort. Nevertheless, we c a n n o t simply ignore them, because their meanings m a y be important in elaborat- ing our knowledge of the events or things that we are interested in. F o r example, they may be used to fill roles in the c o n c e p t u a l structures r e p r e s e n t i n g interesting events. T h e y m a y a l s o n e v e r be used again. M a n y of the words in our example are processed this way. E x a m p l e s include the words 'Arabic', 'Iraqi', and 'his', as well as all articles.

T w o things can h a p p e n with these words. Either their meaning does help elaborate something interesting, in which case that meaning will be incorpo- rated in the representation, or it doesn't. F o r example, the meaning of the word ' F r e n c h ' in the phrase (1) b e f o r e surrendering to F r e n c h police

is i n c o r p o r a t e d into the r e p r e s e n t a t i o n because we are interested in w h o m the terrorist surrendered to, i.e. 'police'. On the o t h e r hand, the meaning of the word ' F r e n c h ' in the phrase

(2) as he was led away by the F r e n c h officers is not i n c o r p o r a t e d into the meaning representation because we n e v e r b e c o m e interested in 'officers'.

Words with some conceptual c o n t e n t will o f t e n also have some associated processing information, in the form of expectations, which can help to elaborate o n their o w n meaning. M a n y of the words which are processed by the "save and skip" strategy are objects of these sorts of expectations. F o r example, it seems quite plausible that the word ' e m b a s s y ' has an e x p e c t a t i o n which looks f o r the name of the c o u n t r y which the embassy represents, and that the words 'police', 'officers', and 'officials' have expectations for the name of the governmental a u t h o r i t y in w h o s e name t h e y operate. But if a word is subject to the " s a v e and skip" strategy, these e x p e c t a t i o n s should not be applied until we k n o w that the c o n c e p t associated with the w o r d actually elaborates o n our knowledge of something interesting. If it turns out that we d o n ' t care a b o u t the concept, we d o n ' t want to have done unneces- sary processing. L e t ' s c o m p a r e our processing of 'police' in phrase (1) above with our processing of 'officers' in phrase (2). Since it turns out that the c o n c e p t of police in the first case adds to o u r knowledge of an interesting event, it seems plausible that the expectation that the word 'police' has for the a u t h o r i t y governing the police would b e used. In the second case, since the c o n c e p t of 'officers'

(10)

does not add to our knowledge of anything interesting, there is simply no point in applying any similar rule.

T h e third possible processing strategy we can apply to a word is to process it immediately, i.e. pay attention to its meaning and the expectations it gen- erates. This is the strategy that we apply when the word has a significant and interesting c o n c e p t u a l content. It is these concepts and their associated expectations that drive the analysis. Examples from the s t o r y of the last section include the words

'gunman', 'shot', and 'hostages'. The expectations

which these words generate include the same kind of simple elaborative, or "slot-filling", e x p e c t a t i o n s associated with some of the words for which a "skip and save" strategy is appropriate. F o r example, it is quite plausible that one e x p e c t a t i o n g e n e r a t e d b y the word 'gunman' looks for the nationality or political affiliation of the gunman.

T h e s e words can also g e n e r a t e e x p e c t a t i o n s which operate at a much higher level. F o r example, when we read the word 'gunman', we e x p e c t to read that he m a y have p e r f o r m e d the action of shooting a weapon. We also expect the events associated with several possible scripts, including $ R O B B E R Y and $ T E R R O R I S M . T h e s e e x p e c t a t i o n s o p e r a t e in a m a n n e r s o m e w h a t akin to script application (see Cullingford, 1978), in that they serve to recognize events, and so recognize that they are sensible in the given context. So, as described in the example of the last section, once we k n o w that the gunman is quite likely a terrorist, we expect that he m a y hold hostages, that he may shoot or kill some people, and

that he m a y make demands. We also know that

there are only a small n u m b e r of possible outcomes of the episode: the terrorist might be captured, he might surrender, he might be killed, or he might escape. These high level expectations help us decide what is i m p o r t a n t in the text in a v e r y t o p - d o w n way. T h e analysis process depends crucially on this. But its flexibility also depends on its ability to pursue questions a b o u t interesting things and events, even if they were not anticipated.

The expectations used by IPP are implemented in the form of requests (see Riesbeek, 1975). A request is a form of production, or test-action pair. If the test of an active request is checked and f o u n d to be true, then the corresponding sets of actions are performed. The list of requests is o r d e r e d so that when the active requests are considered, the most r e c e n t l y activated are c o n s i d e r e d first, since t h e y r e p r e s e n t n e w e r , and so p r o b a b l y b e t t e r , e x p e c t a - tions.

While in t h e o r y the tests and actions which requests p e r f o r m could be arbitrary, in our system we have f o u n d that only a restricted set is necessary. Requests m a y do the following:

build new c o n c e p t u a l structures -- usually a given request will only build one such structure;

fill a slot in some c o n c e p t u a l structure with some o t h e r c o n c e p t u a l structure -- for example, filling the A C T O R slot of $ S H O O T with the t o k e n for the gunman;

activate o t h e r requests -- these will o f t e n be requests trying to fill slots in the structure built by the activating request; they can also be expectations for possible actions, states, or more complicated episodes which m a y follow; d e - a c t i v a t e requests -- requests are able to d e a c t i v a t e requests, including themselves, when they are no longer appropriate.

T h e r e are three types of tests which requests per- form:

checks for specific lexical items -- for example, function words tend to be specific to a given c o n s t r u c t i o n ; so in the p h r a s e " s u r r e n d e r to F r e n c h p o l i c e " , the requests a s s o c i a t e d with ' s u r r e n d e r ' (or $ S U R - R E N D E R ) , can look for the o c c u r r e n c e of the word :to' to p r e c e d e the authority to whom the surrender is taking place;

checks for lexical items satisfying some p r o p - e r t y -- for example, words which activate a specific script;

look for tokens or events of a specified type -- this might be as simple as matching a particular structure; or it m a y involve use of semantic tests such as ' h u m a n ' or 'authority'. The fact that requests can look for specific lexical items is very i m p o r t a n t in reducing processing time. This savings is realized b o t h b y requests looking for function words to fill slots (such as 'to' or ' b y ' ) , and b y requests which look for more substantial events. O f t e n a word m a y create expectations which look for specific words which indicate what script is ap-

plicable. As an example, gunman creates expecta-

tions which look for the terrorist, hijack, and rob- b e r y scripts. T h e r e q u e s t looking f o r the hijack script m a y include tests for specific words (or phrases), such as ' d i v e r t e d ' , 'hijack', ' t o o k o v e r ' , all of which indicate the hijack script. Requests will normally have checks at the c o n c e p t u a l level as well. The request activated by 'gunman' which checks for the terrorist script looks at the location of the action. If that location is the location of a political entity, such as an e m b a s s y or the office of some political organization, that is a good clue that the terrorist script m a y be relevant. A sample request is shown on the next page.

Within the b r o a d categories of words that are p r o c e s s e d immediately, and words that are saved and skipped, t h e r e are subcategories that help to