Flexible Parsing

Philip J. Hayes

George V. Mouradian

C o m p u t e r S c i e n c e D e p a r t m e n t C a r n e g i e - M e l l o n U n i v e r s i t y P i t t s b u r g h , P e n n s y l v a n i a 1 5 2 1 3

W h e n people use natural language in natural settings, they o f t e n use it ungrammatical- ly, leaving out or repeating words, breaking o f f and restarting, speaking in fragments, etc. Their human listeners are usually able to cope with these deviations with little difficulty. If a computer system is to accept natural language input from its users on a routine basis, it should be similarly robust. In this paper, we outline a set o f parsing flexibilities that such a system should provide. W e go on to describe FlexP, a bottom-up pattern matching parser that we have designed and implemented to provide many o f these flexibilities for restricted natural language input to a limited-domain computer system.

1. The I m p o r t a n c e o f Flexible Parsing

W h e n people use natural language in natural con- versation, they often do not respect grammatical nice- ties. Instead of speaking sequences of grammatically w e l l - f o r m e d and c o m p l e t e sentences, people o f t e n leave out or repeat words or phrases, break off what they are saying and rephrase or replace it, speak in fragments, or use otherwise incorrect grammar. The following example conversation involves a n u m b e r of these grammatical deviations:

A: I want ... can y o u send a m e m o a message to to Smith

B: Is that J o h n or John Smith or Jim Smith A: Jim

Instead of being unable or refusing to parse such un- grammatical utterances, h u m a n listeners are generally unperturbed by them. Neither participant in the above dialogue, for instance, would have any difficulty.

W h e n c o m p u t e r s a t t e m p t to interact with people using natural language, they face a very similar situa- tion; the people will still tend to deviate from whatev- er grammar the c o m p u t e r system is using. The fact that the input is typed rather than spoken makes little difference; grammatical deviations seem to be inherent in s p o n t a n e o u s h u m a n use of language whatever the modality. So, if computers are ever to converse natu- rally with humans, they must be able to parse their

1 This research was sponsored by the Air Force Office of Scientific Research under Contract F49620-79-C-0143.

inputs as flexibly and robustly as humans do. While considerable advances have been made in recent years in applied natural language processing, few of the sys- tems that have been constructed have paid sufficient attention to the kinds of deviation that will inevitably occur in their input if they are used in a natural envi- ronment. In m a n y cases, if the user's input does not c o n f o r m to the s y s t e m ' s g r a m m a r , an indication of i n c o m p r e h e n s i o n f o l l o w e d by a request to rephrase m a y be the best he can expect. We believe that such inflexibility in parsing severely limits the practicality of natural language c o m p u t e r interfaces, and is a major reason w h y natural language has yet to find wide ac- ceptance in such applications as database retrieval or interactive c o m m a n d languages.

In this paper, we report on a flexible parser, called FlexP, suitable for use with a restricted natural lan- guage interface to a limited-domain c o m p u t e r system. We describe first the kinds of grammatical deviations we are trying to deal with, then the basic design char- acteristics of FlexP with justification for them based on the kinds of problem to be solved, and finally more details of our parsing system with w o r k e d examples of its operation. These examples, and most of the others in the paper, represent natural language input to an electronic mail system that we and others [2] are con- structing as part of our r e s e a r c h on user interfaces. This system employs FlexP to parse its input.

Copyright 1981 by the Association for Computational Linguistics. Permission to copy without fee all or part of this material is granted provided that the copies are not made for direct commercial advantage and the Journal reference and this copyright notice are included on the first page. To copy otherwise, or to republish, requires a fee and/or specific permission.

2. T y p e s o f Grammatical Deviation

T h e r e are a n u m b e r of distinct types of g r a m m a t i - cal deviation, and not all types are f o u n d in all types of c o m m u n i c a t i o n situation. In this section, we first define the restricted type of c o m m u n i c a t i o n situation t h a t we will be c o n c e r n e d with, that of a limited- d o m a i n c o m p u t e r s y s t e m and its user c o m m u n i c a t i n g via a k e y b o a r d and display screen. We then p r e s e n t a t a x o n o m y of g r a m m a t i c a l deviations c o m m o n in this context, and b y implication a set of parsing flexibilities needed to deal with them.

2.1 C o m m u n i c a t i o n w i t h a L i m i t e d - D o m a i n S y s t e m

In the r e m a i n d e r of this paper, we will focus on a restricted t y p e of c o m m u n i c a t i o n situation, t h a t b e - t w e e n a limited-domain s y s t e m and its user, and on the parsing flexibilities n e e d e d b y such a s y s t e m to cope with the user's inevitable g r a m m a t i c a l deviations. E x a m p l e s of the type of system we have in mind are d a t a b a s e retrieval systems, e l e c t r o n i c mail s y s t e m s , medical diagnosis systems, or any systems o p e r a t i n g in a domain so restricted that they can c o m p l e t e l y under- stand any relevant input a user might provide. T h e r e are several points to be made.

First, a l t h o u g h such s y s t e m s can be e x p e c t e d to parse and u n d e r s t a n d a n y t h i n g r e l e v a n t to their d o - main, their users c a n n o t be e x p e c t e d to confine t h e m - selves to relevant input. As B o b r o w et. al. [3] note, users o f t e n explain their u n d e r l y i n g m o t i v a t i o n s or otherwise justify their requests in terms quite irrele- vant to the domain of the system. T h e result is that such systems c a n n o t expect to parse all their inputs e v e n with the use of flexible parsing techniques.

Secondly, a flexible parser is just part of the c o n - versational c o m p o n e n t of such a system, and c a n n o t solve all parsing p r o b l e m s by itself. F o r example, if a p a r s e r can e x t r a c t two c o h e r e n t f r a g m e n t s f r o m an otherwise i n c o m p r e h e n s i b l e input, the decisions a b o u t w h a t the system should do next must be m a d e b y an- other c o m p o n e n t of the system. A decision on w h e t h - er to jump to a conclusion a b o u t what the user intend- ed, to present him with a set of alternative i n t e r p r e t a - tions, or to profess total confusion, can only be m a d e with i n f o r m a t i o n a b o u t the history of the conversation, beliefs a b o u t the user's goals, and m e a s u r e s of plausi- bility for any given action by the user. (See [10] for m o r e discussion of this b r o a d e r view of graceful inter- action in m a n - m a c h i n e c o m m u n i c a t i o n . ) Suffice it to say that we assume a flexible parser is just one c o m p o - n e n t of a larger system, and t h a t a n y i n c o m p r e h e n - sions or ambiguities that it finds are passed on to an- other c o m p o n e n t of the system with access to higher- level i n f o r m a t i o n , p u t t i n g it in a b e t t e r p o s i t i o n to decide what to do next.

Finally, we assume that, as is usual for such sys- tems, input is typed, r a t h e r t h a n s p o k e n as is n o r m a l in h u m a n conversations. This simplifies low-level proc- essing t r e m e n d o u s l y b e c a u s e k e y - s t r o k e s , unlike speech w a v e - f o r m s , are u n a m b i g u o u s . On the other hand, p r o b l e m s like misspelling arise, a n d a flexible p a r s e r c a n n o t e v e n a s s u m e t h a t s e g m e n t a t i o n into words b y spaces and carriage returns will always be correct. H o w e v e r , such input is still o n e side of a c o n v e r s a t i o n , r a t h e r t h a n a polished text in the m a n n e r of m o s t written material. As such, it is likely to c o n - tain m a n y of the same type of errors n o r m a l l y f o u n d in s p o k e n conversations.

2.2 Misspelling

Misspelling is p e r h a p s the m o s t c o m m o n class of error in written language. Accordingly, it is the f o r m of u n g r a m m a t i c a l i t y that has b e e n dealt with the m o s t by language processing systems. P A R R Y [14], L I F E R [11], and n u m e r o u s other s y s t e m s h a v e tried to correct misspelt input f r o m their users.

An ability to c o r r e c t spelling implies the existence of a d i c t i o n a r y of c o r r e c t l y spelled w o r d s ( p o s s i b l y a u g m e n t e d b y a set of m o r p h o l o g i c a l rules to p r o d u c e derived f o r m s ) . A n input w o r d not f o u n d in or deriva- ble f r o m the dictionary is a s s u m e d tO be misspelt and is c o m p a r e d against each of the dictionary words and their derivations. If one of these words c o m e s close enough to the input word according to some criteria of lexical matching, it is used in place of the input word.

Spelling correction m a y be a t t e m p t e d in or out of context. F o r instance, there is only one r e a s o n a b l e correction f o r " r e l a v e n t " or for " s e p e r a t e " , but for an input like " u n " some kind of c o n t e x t is typically nec- essary as in "I'11 see y o u un A p r i l " or " h e was shot with the stolen u n . " In effect, c o n t e x t can be used to reduce the size of the dictionary to be s e a r c h e d for correct words. This b o t h m a k e s the search m o r e effi- cient and reduces the possibility of multiple m a t c h e s of the input against the dictionary. T h e L I F E R [11] system uses the strong constraints typically p r o v i d e d b y its s e m a n t i c g r a m m a r in this w a y to r e d u c e the range of possibilities for spelling correction.

A particularly t r o u b l e s o m e kind of spelling e r r o r results in a valid word different f r o m the o n e intended, as in " s h o w me on of the m e s s a g e s " . Clearly, such an e r r o r c a n o n l y be c o r r e c t e d t h r o u g h c o m p a r i s o n against a contextually restricted subset of a s y s t e m ' s vocabulary.

2.3 N o v e l W o r d s

decide it was a colour f r o m " a fawn c o l o u r e d sweater". If one just knew the word as referring to a y o u n g deer, one might conclude that it was being used to mean the colour of a y o u n g deer. In general, be- y o n d making direct inferences about the role of un- k n o w n words from their immediate context, v o c a b u - lary learning can require arbitrary a m o u n t s of real- world knowledge and inference, and this is certainly b e y o n d the capabilities of present day artificial intelli- gence techniques (though see Carbonell [5] for work in this direction).

There is, however, a very c o m m o n special subclass of novel words that is well within the capabilities of present d a y systems: u n k n o w n proper names. Given an appropriate context, either sentential or discourse, it is relatively s t r a i g h t f o r w a r d to classify u n k n o w n words as the names of people, places, etc.. Thus in " s e n d copies to Moledeski Chiselov" it is reasonable to conclude from the local context that " M o l e d e s k i " is a first name, " C h i s e l o v " is a surname, and together they identify a person (the intended recipient of the copies). Strategies like this were used in the P O L I T I - CS [6], F R U M P [8], and P A R R Y [14] systems.

Since novel words are b y definition n o t in the k n o w n vocabulary, h o w can a parsing system distin- guish them from misspellings? In most cases, the nov- el words will not be close enough to k n o w n words to allow successful correction, as in the above example, but this is not always true; an u n k n o w n first name of " A I " could easily be corrected to "all". Conversely, it is not safe to assume that u n k n o w n words in contexts which allow proper names are really proper names as in: " s e n d copies to al m a n a g e r s " . In this example, " a l " p r o b a b l y should be corrected to "all". In order to resolve such cases it m a y be n e c e s s a r y to c h e c k against a list of referents for proper names, if this is k n o w n , or otherwise to consider such factors as whether the initial letters of the words are capitalized.

As far as we know, no systems yet c o n s t r u c t e d have integrated their handling of misspelt words and u n k n o w n proper names to the degree outlined above. H o w e v e r , several applied natural language processing systems, including the C O O P [12] system, allow sys- tematic access to a database containing proper names w i t h o u t the need for inclusion of the words in the system's parsing vocabulary.

2.4 Erroneous s e g m e n t i n g markers

Written text is segmented into words by spaces and new lines, and into higher level units by commas, peri- ods and other p u n c t u a t i o n marks. Both classes, espe- cially the second, m a y be omitted or inserted specious- ly. Spoken language is also segmented, but by the quite different markers of stress, intonation and noise words and phrases, which we will not consider here.

I n c o r r e c t segmentation at the lexical level results in t w o or more w o r d s being run together, as in " r u n t o g e t h e r " , or a single word being split up into two or more segments, as in " t o g e t h e r " or ( i n c o n v e n i e n t l y ) " t o get h e r " , or c o m b i n a t i o n s of these effects as in " r u n t o geth er". In all cases, it seems natural to deal with such errors by extending the spelling correction mechanism to be able to recog- nize target words as initial s e g m e n t s of u n k n o w n words, and vice-versa. As far as we know, no current systems deal with incorrect segmentation into words.

The other type of segmenting error, incorrect punc- tuation, has a much b r o a d e r impact on parsing m e t h o - dology. C u r r e n t parsers typically work one sentence at a time, and assume that each sentence is terminated by an explicit e n d - o f - s e n t e n c e marker. A flexible parser must be able to deal with the potential absence of such a marker, and recognize that the sentence is being t e r m i n a t e d implicitly b y the start of the next sentence. In general, a flexible parser should be able to take a d v a n t a g e of the i n f o r m a t i o n p r o v i d e d b y p u n c t u a t i o n if it is used correctly, and ignore it if it is used incorrectly.

I n s t e a d of p u n c t u a t i o n , m a n y interactive systems use c a r r i a g e - r e t u r n to indicate s e n t e n c e termination. Missing sentence terminators in this case c o r r e s p o n d to two sentences on one line, or to the typing of a s e n t e n c e w i t h o u t the t e r m i n a t i n g return, while spe- cious terminators c o r r e s p o n d to typing a sentence on more than one line.

2.5 B r o k e n - O f f a n d R e s t a r t e d U t t e r a n c e s

In spoken language, it is very c o m m o n to break off and restart all or part of an utterance:

I want to - - Could you tell me the name? Was the m a n - e r - the official here y e s t e r d a y ? Usually, such restarts are signalled in some way, by " u m " or " e r " , or more explicitly by "let's back u p " or some similar phrase.

In written language, such restarts do not normally occur because they are erased by the writer before the reader sees them. Interactive c o m p u t e r systems typi- cally provide facilities for their users to delete the last character, word, or current line as though it had never b e e n typed, for the very p u r p o s e of allowing such restarts. Given these signals, the restarts are easy to detect and interpret. H o w e v e r , users sometimes fail to make use of these signals. I n p u t n o t c o n t a i n i n g a c a r r i a g e - r e t u r n can be spread o v e r several lines by intermixing of input and output, or a user m a y simply fail to type the "kill" character that deletes the input line he has typed so far, as in:

where the user p r o b a b l y intended to erase the first two words and start over. A flexible parser should be able to deal with such non-signalled restarts.

2.6 F r a g m e n t a r y and O t h e r w i s e Elliptical Input

Naturally occurring language o f t e n involves utter- ances that are not c o m p l e t e sentences. O f t e n the ap- p r o p r i a t e n e s s of such f r a g m e n t a r y u t t e r a n c e s d e p e n d s on c o n v e r s a t i o n a l or physical c o n t e x t as in:

A: Do y o u m e a n Jim Smith or F r e d Smith?

B: Jim

A: Send a message to Smith

B: O K

A: with copies to Jones

A flexible parser must be able to parse such f r a g m e n t s given the a p p r o p r i a t e context.

T h e r e is a question here of what such f r a g m e n t s should be parsed into. Parsing systems that h a v e dealt with the p r o b l e m h a v e typically a s s u m e d t h a t such inputs are ellipses of c o m p l e t e s e n t e n c e s , and t h a t their parsing involves finding that c o m p l e t e sentence, and parsing it. Thus the sentence c o r r e s p o n d i n g to " J i m " in the example a b o v e would be " I m e a n J i m " . Essentially this view has b e e n t a k e n b y the L I F E R [11] and G U S [3] systems. A n alternative view is that such f r a g m e n t s are not ellipses of m o r e c o m p l e t e sen- tences, but are themselves c o m p l e t e u t t e r a n c e s given the context in which they occur, and should be parsed as such. C e r t a i n s p e e c h parsers, including H E A R - S A Y - I I [9] and H W I M [20], are oriented t o w a r d s this m o r e b o t t o m - u p view of fragments, and this view is also the basis of our a p p r o a c h to f r a g m e n t a r y input, as we will explain m o r e fully below. C a r b o n e l l [personal c o m m u n i c a t i o n ] suggests a third view a p p r o p r i a t e for some fragments: that of an e x t e n d e d case frame. In the s e c o n d e x a m p l e a b o v e , for instance, A ' s " w i t h copies to J o n e s " f o r m s a n a t u r a l p a r t of the case f r a m e established b y " s e n d a message to Smith". Yet a n o t h e r a p p r o a c h to f r a g m e n t parsing is t a k e n in the P L A N E S system [15] which always parses in t e r m s of m a j o r f r a g m e n t s rather t h a n c o m p l e t e utterances. This technique relies on there being only one w a y to c o m - bine the f r a g m e n t s thus o b t a i n e d , which m a y be a r e a s o n a b l e assumption for m a n y limited d o m a i n sys- tems.

Ellipses can also occur without regard to context. A type that interactive s y s t e m s are particularly likely to face is crypticness in which articles and o t h e r non- essential words are o m i t t e d as in " s h o w messages after June 17" instead of the m o r e c o m p l e t e " s h o w me all m e s s a g e s d a t e d a f t e r J u n e 17". Again, there is a question of w h e t h e r to consider the cryptic input c o m - plete, which would m e a n modifying the s y s t e m ' s g r a m - mar, or to c o n s i d e r it elliptical, and c o m p l e t e it b y

using flexible techniques to parse it against the c o m - plete version as it exists in the s t a n d a r d g r a m m a r .

O t h e r c o m m o n f o r m s of ellipsis are associated with c o n j u n c t i o n as in:

J o h n got up and [John] b r u s h e d his teeth.

M a r y saw Bill and Bill [saw] Mary.

F r e d r e c o g n i z e d [the building] and [Fred] walked t o w a r d s the building.

I

Since conjunctions can s u p p o r t such a wide range of ellipsis, it is generally i m p r a c t i c a l to r e c o g n i z e such u t t e r a n c e s b y a p p r o p r i a t e g r a m m a r extensions. E f - forts to deal with c o n j u n c t i o n have t h e r e f o r e d e p e n d e d on general m e c h a n i s m s that s u p p l e m e n t the basic pars- ing strategy, as in the L U N A R s y s t e m [19], or that m o d i f y the g r a m m a r t e m p o r a r i l y , as in the w o r k of K w a s n y and S o n d h e i m e r [13]. We h a v e not a t t e m p t e d to deal with this type of ellipsis in our parsing system, and will not discuss f u r t h e r the t y p e of flexibility it requires.

2.7 I n t e r j e c t e d Phrases, O m i s s i o n , and S u b s t i t u t i o n

S o m e t i m e s people interject noise or other qualify- ing phrases into w h a t is otherwise a n o r m a l g r a m m a t i - cal flow as in:

I w a n t the message d a t e d I t h i n k J u n e 17

Such interjections can be inserted at almost a n y point in an u t t e r a n c e , and so m u s t be dealt with as t h e y arise by flexible techniques.

It is relatively s t r a i g h t f o r w a r d for a s y s t e m of limit- ed c o m p r e h e n s i o n to screen out and ignore s t a n d a r d noise phrases such as " I t h i n k " or " a s far as I can tell". M o r e t r o u b l e s o m e are interjections that c a n n o t be recognized b y the system, as might for instance be the case in

Display [just to r e f r e s h m y m e m o r y ] the m e s s a g e d a t e d J u n e 17.

I w a n t to see the m e s s a g e [as I f o r g o t w h a t it said] d a t e d June 17.

where the u n r e c o g n i z e d interjections are b r a c k e t e d . A flexible parser should be able to ignore such interjec- tions. T h e r e is always the chance that the u n r e c o g - nized part was an i m p o r t a n t p a r t of w h a t the user was trying to say, but clearly, the p r o b l e m s that arise f r o m this c a n n o t be handled b y a parser.

Omissions of words (or phrases) f r o m the input are closely related to cryptic input as discussed above, and one w a y of dealing with cryptic input is to treat it as a set of omissions. H o w e v e r , in cryptic input only ines- sential i n f o r m a t i o n is left out, while it is conceivable that one could also omit essential i n f o r m a t i o n as in:

Display the m e s s a g e J u n e 17

17. (We assume that the system addressed can display things immediately, or not at all.) If an omission can be n a r r o w e d d o w n in this way, the parser should be able to g e n e r a t e all the a l t e r n a t i v e s ( f o r c o n t e x t u a l resolution of the ambiguity or for the basis of a ques- tion to the user). If the omission can be n a r r o w e d to one alternative then the input was merely cryptic.

Besides omitting words and phrases, people s o m e - times substitute incorrect or u n i n t e n d e d ones. O f t e n such substitutions are spelling errors and should be c a u g h t b y the spelling c o r r e c t i o n m e c h a n i s m , but s o m e t i m e s t h e y are i n a d v e r t e n t substitutions or uses of equivalent v o c a b u l a r y not k n o w n to the system. This t y p e of substitution is just like an omission except that there is an u n r e c o g n i z e d w o r d or phrase in the place where the o m i t t e d input should have been. F o r in- stance, in " t h e m e s s a g e o v e r June 17", " o v e r " takes the place of " d a t e d " or " s e n t a f t e r " or w h a t e v e r else is a p p r o p r i a t e at that point. If the substitution is of v o c a b u l a r y t h a t is a p p r o p r i a t e b u t u n k n o w n to the system, parsing of substituted words can provide the basis of v o c a b u l a r y extension. Wilks [17] has devel- o p e d t e c h n i q u e s f o r relaxing s e m a n t i c c o n s t r a i n t s w h e n t h e y are a p p a r e n t l y violated b y relations implied b y the input, as in the previous example, where " J u n e 1 7 " and " t h e m e s s a g e " do not fit the n o r m a l semantic constraints of the " o v e r " relation.

2.8 Agreement Failure

It is not u n c o m m o n f o r p e o p l e to fail to m a k e the a p p r o p r i a t e a g r e e m e n t b e t w e e n the various parts of a n o u n or v e r b p h r a s e as in :

I wants to send a messages to Jim Smith.

T h e a p p r o p r i a t e action is to ignore the lack of agree- m e n t , and Weischedel and Black [16] describe a m e - t h o d for relaxing the predicates in an A T N g r a m m a r which typically check for such agreements. H o w e v e r , it is generally not possible to conclude locally which value of the m a r k e r ( n u m b e r or p e r s o n ) for which the clash occurs is actually intended.

2.9 I d i o m s

Idioms are phrases whose i n t e r p r e t a t i o n is not w h a t would be o b t a i n e d b y parsing and interpreting t h e m c o n s t r u c t i v e l y in the n o r m a l way. T h e y m a y also not adhere to the s t a n d a r d syntactic rules. Idioms must thus be p a r s e d as a whole in a p a t t e r n m a t c h i n g kind of mode. Parsers b a s e d purely on p a t t e r n matching, like t h a t of P A R R Y [14], are able to p a r s e idioms naturally, while others m u s t either add a p r e p r o c e s s i n g phase of p a t t e r n m a t c h i n g as in the L U N A R s y s t e m [19], or mix specific p a t t e r n s in with m o r e g e n e r a l rules, as in the w o r k of K w a s n y and S o n d h e i m e r [13]. Semantic g r a m m a r s [4, 11] provide a relatively natural w a y of mixing idiomatic and m o r e general patterns.

2.10 User-Supplied Changes

In n o r m a l h u m a n conversation, once s o m e t h i n g is said, it c a n n o t be changed, except indirectly b y m o r e words that refer b a c k to the original ones. In interac- tively t y p e d input, there is always the possibility that a user m a y notice an error he has m a d e and go b a c k and c o r r e c t it himself, w i t h o u t waiting for the s y s t e m to pursue its own, possibly slow and ineffective, m e t h o d s of correction. With a p p r o p r i a t e editing facilities, the user m a y do this w i t h o u t erasing i n t e r v e n i n g words, and, if the s y s t e m is processing his input on a w o r d - b y - w o r d basis, m a y thus alter a w o r d that the s y s t e m has already processed. A flexible parser m u s t be able to take a d v a n t a g e of such u s e r - p r o v i d e d c o r r e c t i o n s to u n k n o w n words, and to p r e f e r t h e m o v e r its o w n cor- rections. It m u s t also be p r e p a r e d to change its parse if the user changes a valid w o r d to a n o t h e r different but equally valid word.

3. A n A p p r o a c h to F l e x i b l e P a r s i n g

M o s t c u r r e n t p a r s i n g s y s t e m s are u n a b l e to c o p e with m o s t of the kinds of g r a m m a t i c a l deviation out- lined above. This is b e c a u s e typical parsing s y s t e m s a t t e m p t to apply their g r a m m a r s to their inputs in a rigid way, and since deviant input, b y definition, does not c o n f o r m to the g r a m m a r , they are unable to p r o d - uce a n y kind of parse for it at all. A t t e m p t s to parse m o r e flexibly h a v e typically involved parsing strategies to be used a f t e r a t o p - d o w n parse using an A T N [18] or similar transition net has failed. Such e f f o r t s in- elude the ellipsis a n d p a r a p h r a s e m e c h a n i s m s of L I - F E R [11], the p r e d i c a t e relaxation techniques of Weis- chedel and Black [16], a n d several of the devices for extending A T N ' s p r o p o s e d b y K w a s n y and S o n d h e i m - er [13]. An i m p o r t a n t e x c e p t i o n to this o b s e r v a t i o n is the case of p a r s e r s , including H E A R S A Y - I I [9] and H W I M [20], designed for s p o k e n input with its inher- ent low-level uncertainty. H W I M , in particular, incor- p o r a t e s techniques to a p p l y an A T N in a b o t t o m - u p style, a n d thus can capitalize on w h a t e v e r points of c e r t a i n t y it can find in the input. F r a g m e n t a r y input, h o w e v e r , is typically the only t y p e of u n g r a m m a t i c a l i t y dealt with b y speech parsers.

will note those of its techniques that seem unlikely to scale up to use with m o r e c o m p l e x g r a m m a r s with wider coverage.

We h a v e a d o p t e d in FlexP an a p p r o a c h to flexible parsing b a s e d not on A T N ' s , but closer to the p a t t e r n matching techniques used in the P A R R Y s y s t e m [14], possibly the m o s t robust natural language processing system yet constructed. At the highest level, the de- sign of FlexP can be characterized b y the following three features:

• pattern matching: This provides a c o n v e n - ient w a y to r e c o g n i z e idioms, a n d also aids in the d e t e c t i o n of omissions a n d substitutions in n o n - i d i o m a t i c phrases.

• b o t t o m - u p rather than top-down p a r s i n g : This aids in the parsing of f r a g m e n t a r y utterances, and in the recognition of in- terjections and restarts.

• parse suspension a n d continuation: This is i m p o r t a n t for dealing with interjections, restarts, and implicit terminations.

In the rest of this section, we e x a m i n e and justify these design characteristics in m o r e detail, and in the process, give an outline of F l e x P ' s parsing algorithm.

3.1 P a t t e r n M a t c h i n g

We have chosen to use a g r a m m a r of linear p a t - terns rather than a transition n e t w o r k b e c a u s e p a t t e r n matching meshes well with b o t t o m - u p parsing, b e c a u s e it facilitates recognition of utterances with omissions and substitutions, and b e c a u s e it is ideal for the recog- nition of idiomatic phrases.

The g r a m m a r of the parser is a set of rewrite or p r o d u c t i o n rules; the l e f t - h a n d side of o n e of these rules is a linear p a t t e r n of constituents (lexical or high- er level) and the right-hand side defines a result con- stituent. E l e m e n t s of the p a t t e r n m a y be labelled op- tional or allow for r e p e a t e d matches. We m a k e the a s s u m p t i o n , certainly true f o r the g r a m m a r we are presently working with, that the g r a m m a r will be se- mantic rather t h a n syntactic, with p a t t e r n s c o r r e s p o n d - ing to idiomatic p h r a s e s or to o b j e c t and e v e n t de- scriptions meaningful in some limited domain, rather t h a n to general syntactic structures.

L i n e a r p a t t e r n s fit well with b o t t o m - u p parsing b e c a u s e they can be indexed by any of their c o m p o - nents, and because, once indexed, it is s t r a i g h t f o r w a r d to c o n f i r m w h e t h e r a p a t t e r n m a t c h e s input a l r e a d y processed in a w a y consistent with the w a y the p a t t e r n was indexed.

P a t t e r n s help with the d e t e c t i o n of omissions and substitutions b e c a u s e in either case the relevant p a t - tern can still be i n d e x e d b y the r e m a i n i n g e l e m e n t s that a p p e a r correctly in the input, and thus the p a t t e r n

as a whole can be recognized e v e n if s o m e of its ele- m e n t s are missing or incorrect. In the case of substi- tutions, such a technique c a n actually help focus the s p e l l i n g - c o r r e c t i o n , p r o p e r - n a m e - r e c o g n i t i o n , or v o - c a b u l a r y - l e a r n i n g techniques, whichever is a p p r o p r i a t e , b y isolating the substituted input and the p a t t e r n con- stituent that it should have m a t c h e d . T h e ( o f t e n high- ly selective) restrictions on w h a t c a n m a t c h the c o n - stituent c a n t h e n be used to r e d u c e the n u m b e r of possibilities c o n s i d e r e d b y these relatively e x p e n s i v e lexical correction techniques.

3.2 B o t t o m - U p Parsing

O u r choice of a b o t t o m - u p s t r a t e g y is b a s e d on our need to recognize isolated s e n t e n c e fragments. If an u t t e r a n c e t h a t would n o r m a l l y be c o n s i d e r e d only a f r a g m e n t of a c o m p l e t e s e n t e n c e is to be recognized t o p - d o w n , there are two s t r a i g h t f o r w a r d a p p r o a c h e s to take. First, the g r a m m a r can be altered so that the f r a g m e n t is recognized as a c o m p l e t e u t t e r a n c e in its o w n right. This is undesirable b e c a u s e it can cause e n o r m o u s e x p a n s i o n of the g r a m m a r , and b e c a u s e it b e c o m e s difficult to decide w h e t h e r a f r a g m e n t a p - pears in isolation or as p a r t of a larger utterance, espe- cially if t h e r e is the possibility of missing e n d - o f - s e n t e n c e markers. T h e second option is for the p a r s e r to infer f r o m the c o n v e r s a t i o n a l c o n t e x t w h a t g r a m - matical c a t e g o r y (or sequence of s u b - c a t e g o r i e s ) the f r a g m e n t might fit into, a n d t h e n to do a t o p - d o w n parse f r o m that s u b - c a t e g o r y . This essentially is the tactic used in the G U S [3] a n d L I F E R [11] systems. This s t r a t e g y is clearly b e t t e r t h a n the first one, but has t w o problems: first, of predicting all possible sub- c a t e g o r i e s which m i g h t c o m e next, a n d secondly, of inefficiency if a large n u m b e r are predicted. K w a s n y a n d S o n d h e i m e r [13] use a c o m b i n a t i o n of the t w o strategies b y t e m p o r a r i l y modifying an A T N g r a m m a r to accept f r a g m e n t categories as c o m p l e t e u t t e r a n c e s at the times they are contextually predicted.

B o t t o m - u p parsing avoids the p r o b l e m of trying to p r e d i c t w h a t g r a m m a t i c a l s u b - c a t e g o r y a s e n t e n c e f r a g m e n t should be p a r s e d into. T h e d a t a - d r i v e n na- ture of b o t t o m - u p p a r s i n g m e a n s t h a t a n y sub- c a t e g o r y c a n be p a r s e d as an isolated unit in exactly the same w a y as a c o m p l e t e sentence, so that no pre- diction a b o u t w h a t s u b - c a t e g o r y to e x p e c t is n e c e s - sary. O n the o t h e r h a n d , if a given input c a n be p a r s e d as m o r e t h a n o n e s u b - c a t e g o r y , a b o t t o m - u p a p p r o a c h has no g o o d w a y of distinguishing b e t w e e n them, e v e n if only one would be predicted t o p - d o w n .

c o u p l e d with s u s p e n d e d parses, are also helpful in recognizing interjections and restarts.

While well suited to parsing f r a g m e n t a r y input, pure b o t t o m - u p parsing (as, for instance, described b y C h e s t e r [7]) can result in the generation of an unnec- essarily large n u m b e r of i n t e r m e d i a t e s t r u c t u r e s t h a t c a n n o t f o r m part of a c o m p l e t e d parse. To reduce these inefficiencies, F l e x P does n o t require all ele- m e n t s of a p a t t e r n to be p r e s e n t b e f o r e including the p a t t e r n in the parse structure, and is thus similar to the l e f t - c o r n e r algorithm also described b y Chester. ( F o r greater detail see A h o a n d Ullman [1].) In fact, FlexP is m o r e restrictive t h a n the l e f t - c o r n e r algorithm. In n o r m a l left-to-right processing, if a new w o r d is ac- c o u n t e d for b y a p a t t e r n that has already b e e n partial- ly m a t c h e d b y previous input, other p a t t e r n s indexed b y the n e w w o r d are n o t c o n s i d e r e d as possible m a t c h e s for that word. This is a heuristic designed to limit the n u m b e r of partial parses that need to be in- v e s t i g a t e d , and could lead to missed p a r s e s , b u t in practical experience, we have not f o u n d this to be a p r o b l e m . E x a c t l y h o w this heuristic o p e r a t e s will be m a d e clear b y the description of the parsing algorithm in the following section.

3.3 Parse S u s p e n s i o n and Continuation

FlexP e m p l o y s the technique of suspending a parse with the possibility of later continuation to help with the recognition of interjections, restarts, a n d implicit terminations. T o m a k e clear w h a t this means, it is first n e c e s s a r y to sketch the o p e r a t i o n of the parsing algorithm as a whole. This will also serve to clarify the discussion of b o t t o m - u p parsing in the p r e v i o u s section. E x a m p l e s of the algorithm in action are given in Section 4.

F l e x P ' s parsing algorithm maintains a set of partial parses, each of which accounts for the input already p r o c e s s e d but not yet a c c o u n t e d f o r b y a c o m p l e t e d parse. T h e parser a t t e m p t s to i n c o r p o r a t e each new input word into each of the partial parses b y one of the following methods:

1. fitting the word directly into one of the p a t t e r n slots available for m a t c h i n g at the right-hand edge of the partial parse; 2. finding a chain of n o n - t e r m i n a l g r a m m a r

s u b - c a t e g o r i e s that allow the word to fit indirectly at the r i g h t - h a n d edge of the partial parse;

3. finding a c o m m o n s u p e r - c a t e g o r y of the input w o r d and the s u b - c a t e g o r y at the top of the partial parse, so that the partial parse c a n be e x t e n d e d u p w a r d s a n d the input w o r d will then fit into it b y either m e t h o d 1 or 2 a b o v e ;

4. s a m e as 1, b u t b a s e d on flexible p a t t e r n matching;

5. same as 2, but b a s e d on flexible p a t t e r n matching;

6. s a m e as 3, b u t b a s e d on flexible p a t t e r n matching.

Flexible p a t t e r n m a t c h i n g is e x p l a i n e d in Section 4. As the description of m e t h o d 3 implies, FlexP does not build the partial parses a n y higher t h a n is n e c e s s a r y to a c c o u n t for the input already processed. In particular, FlexP does not try to build e a c h partial parse up to a c o m p l e t e u t t e r a n c e unless a c o m p l e t e u t t e r a n c e is n e e d e d to a c c o u n t for the input seen so far.

Which of the six m e t h o d s is used to i n c o r p o r a t e the new w o r d into the existing set of partial parses is de- t e r m i n e d in the following way. T h e parser first tries m e t h o d 1 on e a c h of the partial parses. F o r e a c h of them, it m a y succeed in one or m o r e t h a n o n e w a y or it m a y fail. If it succeeds on a n y of t h e m , the ones on which it fails are discarded, the ones on which it suc- ceeds are e x t e n d e d in all the ways that are possible, the extensions b e c o m e the n e w set of partial parses for possible e x t e n s i o n b y the n e x t input word, a n d the o t h e r five m e t h o d s are not tried. If m e t h o d 1 fails for all partial parses, the same p r o c e d u r e is r e p e a t e d for m e t h o d 2, and so on. If no partial parse can be ex- t e n d e d b y a n y of the six m e t h o d s , the entire set of partial parses is saved as a s u s p e n d e d parse, and the input is used either to start a c o m p l e t e l y n e w set of partial p a r s e s , or to e x t e n d a p r e v i o u s l y s u s p e n d e d parse. Clearly, the policy of not a t t e m p t i n g the m o r e c o m p l i c a t e d m e t h o d s if the simpler m e t h o d s succeed can result in some parses being missed. H o w e v e r , in practice, we have f o u n d it heuristically a d e q u a t e for the small d o m a i n - s p e c i f i c g r a m m a r we h a v e b e e n us- ing, and m u c h m o r e efficient t h a n trying all m e t h o d s regardless of the o u t c o m e s of the others. O n the o t h e r hand, if c o m p l e t e n e s s b e c a m e i m p o r t a n t , it would be simple to change FlexP always to try all m e t h o d s .

T h e r e are several possible e x p l a n a t i o n s f o r input m i s m a t c h , i.e. the failure of an input to e x t e n d the currently active set of partial parses.

• T h e input could be an implicit t e r m i n a - tion, i.e. the start of a n e w t o p - l e v e l ut- terance, in which case the previous utter- ance should be a s s u m e d complete,

• T h e input could be a r e s t a r t , in which case the active parse should be a b a n d o n e d and a n e w parse s t a r t e d f r o m that point.

It is not possible, in general, to distinguish b e t w e e n these cases at the time the m i s m a t c h occurs. If the active parse is not at a possible t e r m i n a t i o n point, t h e n input m i s m a t c h c a n n o t indicate implicit t e r m i n a t i o n , but m a y indicate either restart or interjection. It is necessary to suspend the active parse and wait to see if it is continued at the next input mismatch. On the other hand, if the active parse is at a possible t e r m i n a - tion point, input m i s m a t c h does not rule out interjec- tion or e v e n restart. In this situation, our algorithm t e n t a t i v e l y a s s u m e s t h a t t h e r e has b e e n an implicit termination, but suspends the active parse a n y w a y for subsequent potential continuation.

Finally, it m a y be worthwhile to note why we im- p l e m e n t e d FlexP to o p e r a t e in a b r e a d t h - f i r s t m o d e , carrying ambiguous alternative parses along in parallel, rather t h a n investigating t h e m individually depth-first. This choice follows naturally f r o m a decision to parse each input t o k e n i m m e d i a t e l y after it is typed, which in turn follows f r o m our desire to deal with implicit t e r m i n a t i o n of input strings (see Section 2.4). A b r e a d t h - f i r s t a p p r o a c h allows the parser to m a k e best use of the time during which the user is typing. A depth-first i m p l e m e n t a t i o n could p o s t p o n e considera- tion of some alternatives until the input had b e e n ter- m i n a t e d b y the user. In such cases, u n a c c e p t a b l y long delays might result. N o t e that the possibility of im- plicit t e r m i n a t i o n also p r o v i d e s justification f o r the strategy of parsing each input word i m m e d i a t e l y a f t e r it is typed. If the input signals an implicit termination, then the user m a y well expect the s y s t e m to r e s p o n d i m m e d i a t e l y to the input thus terminated.

4. FlexP in Operation

This section describes through a series of examples of gradually increasing complexity h o w F l e x P ' s parsing algorithm operates, and h o w it achieves the flexibilities discussed earlier. T h e i m p l e m e n t a t i o n used to run the examples has b e e n used as the p a r s e r for an intelligent interface to an electronic mail s y s t e m [2]. T h e intelli- gence in this interface is c o n c e n t r a t e d in a User Agent that m e d i a t e s b e t w e e n the user a n d the u n d e r l y i n g mail s y s t e m to ensure t h a t the i n t e r a c t i o n goes s m o o t h l y . T h e A g e n t does this by, a m o n g o t h e r things, checking that the user specifies the operations he wants p e r f o r m e d and their p a r a m e t e r s correctly and unambiguously, conducting a dialogue with the user if errors or ambiguities arise. T h e role of FlexP as the A g e n t ' s parser is to t r a n s f o r m the user's input into the internal r e p r e s e n t a t i o n s e m p l o y e d b y the A g e n t , re- solving as m a n y of the errors or potential ambiguities that it can, so as to minimize the a m o u n t of interac- tion b e t w e e n A g e n t and user n e c e s s a r y to arrive at a correct and u n a m b i g u o u s version of the input. Usually the u s e r ' s input is a r e q u e s t f o r action b y the mail

s y s t e m or a description of objects k n o w n to the mail system. O u r examples are d r a w n f r o m that context.

4.1 P r e l i m i n a r y E x a m p l e

Suppose the user types

display new messages

Parsing begins as soon as a n y input is available. T h e first w o r d is used as an index into the store of rewrite rules. E a c h rule gives a p a t t e r n and a structure to be p r o d u c e d w h e n the p a t t e r n is matched. T h e c o m p o - nents of the structure are built f r o m the structures or words that m a t c h the elements of the pattern. T h e w o r d " d i s p l a y " indexes the rule:

(pattern: (Display MessageDescription)

result: (StructureType: OperationRequest

Operation: Display

Message: (Filler MessageDescription) ) )

N o t e that the n o n - t e r m i n a l s in the p a t t e r n of this and s u b s e q u e n t rules are specific to the m e s s a g e s y s t e m domain, so that. the g r a m m a r being used is semantic rather t h a n syntactic. Using this rule the parser c o n - structs the partial parse tree

(Display MessageDescription)

I I

di splay

We call the partially i n s t a n t i a t e d p a t t e r n t h a t labels the u p p e r node a hypothesis. It r e p r e s e n t s a possible i n t e r p r e t a t i o n for a s e g m e n t of input.

T h e next w o r d " n e w " does not directly m a t c h the hypothesis, but since " n e w " is a MsgAdj (an adjective that can m o d i f y a description of a m e s s a g e ) , it indexes the rule:

(pattern: (?Det *MsgAdj MsgHead *MsgCase)

result: (StructureType: MessageDescription

Components : . . . ) )

H e r e , " ? " m e a n s optional, and " * " m e a n s r e p e a t a b l e . F o r the sake of clarity, we have o m i t t e d other prefixes t h a t distinguish b e t w e e n t e r m i n a l a n d n o n - t e r m i n a l p a t t e r n elements. T h e result of this rule is a structure of t y p e M e s s a g e D e s c r i p t i o n that fits the current hy- pothesis, and so extends the parse as follows:

(Display MessageO~scription)

i l

1 (?Det *MsgAdj MsgHead *MsgCase)

f I

I

di splay new

T h e third input m a t c h e s the c a t e g o r y M s g H e a d (head noun of a message description) and so fits the c u r r e n t h y p o t h e s i s . This m a t c h fills the last n o n - optional slot in that pattern. By doing so it m a k e s the c u r r e n t h y p o t h e s i s a n d its p a r e n t p a t t e r n

potentially

complete.

W h e n the parser finds a potentially c o m - plete phrase whose result is of interest to the A g e n t (and the p a r e n t phrase in this e x a m p l e is in that cate- gory), the result is c o n s t r u c t e d and sent to the Agent. 2 H o w e v e r , since the p a r s e r has not seen a t e r m i n a t i o n signal, this parse is kept active. T h e input seen so far m a y be only a prefix for s o m e longer u t t e r a n c e such as " d i s p l a y new m e s s a g e s a b o u t A D A " . In this case " a b o u t A D A " would be r e c o g n i z e d as a m a t c h f o r M s g C a s e (a prepositional phrase that can be part of a m e s s a g e d e s c r i p t i o n ) , the p a r s e would be e x t e n d e d , a n d a revision of the p r e v i o u s s t r u c t u r e sent to the Agent.

4.2 Unrecognized Words

W h e n an input word c a n n o t be f o u n d in the dic- tionary, FlexP tries to spelling-correct the input word against a list of possibilities derived f r o m the current hypothesis. F o r example:

display the new m e s s a e g s

p r o d u c e s the partial parse

(DisplaYl MessageD~scription)

I I

l (?Det *MsgAdj MsgHead *MsgCase)

I I I

I I I

display the new

T h e lower p a t t e r n is the current hypothesis and has t w o elements eligible to m a t c h the next input. A n o t h - er MsgAdj could be matched. A m a t c h for M s g H e a d would also fit. Both e l e m e n t s have associated lists of words that m a t c h t h e m or occur in phrases that m a t c h them. T h e one for M s g H e a d includes the w o r d " m e s s a g e s " , and the spelling c o r r e c t o r passes this b a c k to the main part of the parser as the m o s t likely inter- pretation.

In some cases the spelling c o r r e c t o r p r o d u c e s sever- al likely alternatives. The parser handles such a m b i g u - ous words using the s a m e m e c h a n i s m s that a c c o m m o - date p h r a s e s with a m b i g u o u s i n t e r p r e t a t i o n s ; t h a t is, alternative i n t e r p r e t a t i o n s are carried along until there is enough input to discriminate those that are plausible f r o m those that are not. T h e details are given in the next section.

2 W h a t h a p p e n s to the result w h e n the Agent receives it is beyond the scope of this paper. However, we should note that the Agent is not obliged to act on the result right away. One strategy is for the Agent to p e r f o r m immediately " s a f e " actions, such as the identification or display of a set of messages, but to wait for explicit termination of " u n s a f e " c o m m a n d s , such as those to send or delete messages.

The user m a y also c o r r e c t the input text himself. These changes are handled in m u c h the same w a y as those p r o p o s e d by the spelling corrector. O f course, these u s e r - s u p p l i e d c h a n g e s are given priority, and parses built using the f o r m e r version must be m o d i f i e d or discarded.

4.3 Ambiguous Input

In the first e x a m p l e there was only one hypothesis a b o u t the structure of the input. M o r e generally, there m a y be s e v e r a l h y p o t h e s e s t h a t p r o v i d e c o m p e t i n g i n t e r p r e t a t i o n s a b o u t w h a t has already b e e n seen and w h a t will a p p e a r next. Until these partial parses are f o u n d to be inconsistent with the actual input, they are carried along as p a r t of the active parse. T h e r e f o r e the active parse is a set of partial parse trees each with a t o p - l e v e l h y p o t h e s i s a b o u t the overall s t r u c t u r e of the input so far and a c u r r e n t hypothesis c o n c e r n i n g the n e x t input. T h e actual i m p l e m e n t a t i o n allows sharing of c o m m o n s t r u c t u r e a m o n g c o m p e t i n g h y - p o t h e s e s and so is m o r e efficient t h a n this description suggests.

The input

were there a n y m e s s a g e s on ...

could be c o m p l e t e d b y giving a date ( " . . . o n T u e s d a y " ) or a topic ( " . . . o n A D A " ) . C o n s e q u e n t l y , the sub- phrase " a n y m e s s a g e s o n " results in two partial pars-

(?Detl *MsgAdj MsgHead *MsgCase)

I I I

any messages (On Date)

I I

o n

e s :

(?Detl *MsgAdj MsgHead *Msg~ase)

I I I

any messages (On Topic)

I I

on

If the next input were " T u e s d a y " it would be consist- ent with the first parse, but not the second. "Since one of the alternatives does a c c o u n t for the input, those that do not m a y be discarded. On the o t h e r hand, if all the partial p a r s e s fail to m a t c h the input, o t h e r action is taken. We consider such situations in the section on s u s p e n d e d parses.

4.4 Flexible Matching

• relax consistency constraints, e.g. n u m b e r a g r e e m e n t

• allow o u t - o f - o r d e r m a t c h e s

C o n s i s t e n c y c o n s t r a i n t s are p r e d i c a t e s t h a t are a t t a c h e d to rules. T h e y assert relationships that must hold a m o n g the items which fill the pattern, e.g. n u m - ber agreement. Although such relationships can usual- ly (it d e p e n d s on the particular relation) also be ex- pressed through c o n t e x t - f r e e rewrite rules, they can be expressed m u c h m o r e c o m p a c t l y through consistency constraints. The c o m p a c t n e s s that can be achieved in this w a y has o f t e n b e e n exploited b y a u g m e n t i n g c o n t e x t - f r e e p a r s e r s to deal with c o n s i s t e n c y c o n - straints on their c o n t e x t - f r e e rules. T h e flexibility achieved by relaxing such constraints in A T N parsers [18] has b e e n explored by Weischedel and Black [16] and b y K w a s n y and Sondheimer [13]. This technique would fit s m o o t h l y into FlexP b u t has n o t actually b e e n n e e d e d or used in our current application.

On the other hand, o u t - o f - o r d e r matching is essen- tial f o r the p a r s e r ' s a p p r o a c h to e r r o r s of omission, t r a n s p o s i t i o n , and substitution. E v e n w h e n strictly interpreted, several elements of a p a t t e r n m a y be eligi- ble to m a t c h the next input item. F o r example, in the p a t t e r n for a M e s s a g e D e s c r i p t i o n

(?Det *MsgAdj MsgHead *MsgCase)

each of the first three elements is initially eligible but the last is not. O n the other hand, once M s g H e a d has b e e n m a t c h e d , only the last e l e m e n t is eligible under the strict i n t e r p r e t a t i o n of the pattern.

C o n s i d e r the input display new a b o u t A D A

T h e first two words parse normally to p r o d u c e (DisplaYl MessageDesc~iption)

I I

I (?Det *MsgAdj MsgHead *MsgCase)

I I

I L

display new

The next word does not fit that hypothesis. T h e two eligible elements predict either a n o t h e r message adjec- tive or a M s g H e a d . T h e word " a b o u t " does not m a t c h either of these, n o r can the parser construct a n y p a t h to t h e m using intermediate hypotheses. Since there are no o t h e r partial parses available to a c c o u n t for this input, and since normal matching fails, flexible m a t c h - ing is tried.

First, previously skipped elements are c o m p a r e d to the input. In this example, the e l e m e n t ? D e t is consid- ered but does not match. Next, elements to the right of the eligible elements are considered. Thus M s g C a s e is c o n s i d e r e d e v e n t h o u g h the n o n - o p t i o n a l e l e m e n t M s g H e a d has not b e e n matched. This succeeds and allows the partial parse to be e x t e n d e d to

(Display

I (?Det *MsgAdj

I I I

display new

MessageDesc~iption)

MsgHead *Msg~ase)

I (About topic)

I about

which correctly predicts the final input item. 3 As al- r e a d y described, flexible m a t c h i n g is applied using the s a m e t h r e e m e t h o d s used f o r the initial n o n - f l e x i b l e m a t c h i n g , i.e. first f o r direct m a t c h e s with p a t t e r n e l e m e n t s in the current partial parses, then for indirect m a t c h e s , and t h e n for m a t c h e s that involve extending the partial parses upwards.

U n r e c o g n i z a b l e s u b s t i t u t i o n s are also h a n d l e d b y this flexible m a t c h i n g mechanism. In the p h r a s e

display the new stuff a b o u t A D A

the w o r d " s t u f f " is n o t f o u n d in the d i c t i o n a r y , so spelling correction is tried b u t does not p r o d u c e a n y plausible alternatives. H o w e v e r , the remaining inputs can be p a r s e d b y simply omitting " s t u f f " and using the flexible m a t c h i n g procedure. T r a n s p o s i t i o n s are h a n - dled t h r o u g h o n e a p p l i c a t i o n of flexible m a t c h i n g if the first e l e m e n t of the t r a n s p o s e d pair is o p t i o n a l , two applications if not.

4.5 S u s p e n d e d P a r s e s

Interjections are m o r e c o m m o n in s p o k e n t h a n in w r i t t e n language b u t do o c c u r in t y p e d input s o m e - times. T o deal with such input, our design allows for b l o c k e d parses to be s u s p e n d e d r a t h e r t h a n m e r e l y discarded.

Users, especially novices, m a y embellish their input with words and phrases that do not provide essential i n f o r m a t i o n and c a n n o t be specifically anticipated. Consider two examples:

display please messages d a t e d J u n e 17

display for me messages d a t e d June 17

In the first case, the interjected word " p l e a s e " could be recognized as a c o m m o n noise p h r a s e that m e a n s nothing to the A g e n t except possibly to suggest that the user is a novice. T h e s e c o n d e x a m p l e is m o r e difficult. B o t h w o r d s of the i n t e r j e c t e d p h r a s e c a n a p p e a r in a n u m b e r of legitimate and meaningful con- structions; they c a n n o t be ignored so easily.

F o r t h e l a t t e r e x a m p l e , p a r s e s u s p e n s i o n w o r k s as follows. A f t e r t h e first w o r d , t h e a c t i v e p a r s e c o n t a i n s a s i n g l e p a r t i a l p a r s e :

(Display

l

display

MessageDescription)

T h e n e x t w o r d d o e s n o t fit this h y p o t h e s i s , so it is s u s p e n d e d . I n its p l a c e , a n e w a c t i v e p a r s e is c o n - s t r u c t e d . It c o n t a i n s s e v e r a l p a r t i a l p a r s e s i n c l u d i n g

(For Person) and (For Timelnterval)

I I

I I

for for

T h e n e x t w o r d c o n f i r m s t h e first of t h e s e , b u t t h e f o u r t h w o r d " m e s s a g e s " d o e s n o t . W h e n t h e p a r s e r f i n d s t h a t it c a n n o t e x t e n d t h e a c t i v e p a r s e , it c o n s i d - ers t h e s u s p e n d e d parse. S i n c e " m e s s a g e s " fits, t h e a c t i v e a n d s u s p e n d e d p a r s e s are e x c h a n g e d a n d t h e r e m a i n d e r of t h e i n p u t p r o c e s s e d n o r m a l l y , so t h a t t h e p a r s e r r e c o g n i z e s " d i s p l a y m e s s a g e s d a t e d J u n e 1 7 " as if it did n o t c o n t a i n " f o r m e " .

5. C o n c l u s i o n

W h e n p e o p l e u s e l a n g u a g e n a t u r a l l y , t h e y m a k e m i s t a k e s a n d e m p l o y e c o n o m i e s of e x p r e s s i o n t h a t o f t e n r e s u l t i n l a n g u a g e t h a t is u n g r a m m a t i c a l b y s t r i c t s t a n d a r d s . I n p a r t i c u l a r , s u c h g r a m m a t i c a l d e v i a t i o n s will i n e v i t a b l y o c c u r in t h e i n p u t of a c o m p u t e r s y s t e m t h a t a l l o w s its u s e r s to e m p l o y n a t u r a l l a n g u a g e . S u c h a c o m p u t e r s y s t e m m u s t , t h e r e f o r e , b e p r e p a r e d to p a r s e its i n p u t f l e x i b l y , if it is to a v o i d f r u s t r a t i o n f o r its users.

I n this p a p e r , we h a v e o u t l i n e d t h e m a i n k i n d s of f l e x i b i l i t y t h a t s h o u l d b e p r o v i d e d b y a n a t u r a l l a n - g u a g e p a r s e r i n t e n d e d f o r n a t u r a l use. W e h a v e also d e s c r i b e d a b o t t o m - u p p a t t e r n m a t c h i n g p a r s e r , F l e x P , w h i c h e x h i b i t s m a n y of t h e s e f l e x i b i l i t i e s , a n d w h i c h is s u i t a b l e f o r r e s t r i c t e d n a t u r a l l a n g u a g e i n p u t t o a l i m i t e d - d o m a i n s y s t e m .

A c k n o w l e d g e m e n t s

T h e a u t h o r s t h a n k t h e r e f e r e e s f o r t h e i r e x t r e m e l y d e t a i l e d c o m m e n t s a n d M i c h a e l M c C o r d a n d G e o r g e H e i d o r n f o r t h e i r h e l p f u l e d i t i n g s u g g e s t i o n s .

R e f e r e n c e s

1. Aho, A. V. and Ullman, J. D. The Theory of Parsing, Transla-

tion, and Compiling. Prentice-Hall, Englewood Cliffs, 1972.

2. Ball, J. E. and Hayes, P. J. Representation of Task- Independent Knowledge in a Gracefully Interacting User Inter- face. Proc. 1st Ann. Mtg. of the AAAI, Stanford University, August, 1980, pp. 116-120.

3. Bobrow, D. G., Kaplan, R. M., Kay, M., Norman D. A., Thompson, H., and Winograd, T. "GUS: a Frame-Driven Dialogue System." Artif. Intell. 8 (1977), 155-173.

4. Burton, R. R. Semantic Grammar: An Engineering Technique for Constructing Natural Language Understanding Systems. TR 3453, Bolt Beranek and Newman, Inc., Cambridge, Mass., December, 1976.

5. Carbonell, J. G. Towards a Self-Extending Parser. Proc. 17th Ann. Mtg. of the ACL, La Jolla, Ca., August, 1979, pp. 3-7. 6. Carbonell, J. G. Subjective Understanding: Computer Models of

Belief Systems. Ph.D. Thesis., Yale University, 1979.

7. Chester, D. "A Parsing Algorithm That Extends Phrases."

Am. J. Comp. Ling. 6, 2 (1980), 87-96.

8. DeJong, G. Skimming Stories in Real-Time. Ph.D. Thesis., Computer Science Dept., Yale University, 1979.

9. Erman, L. D., and Lesser, V. R. HEARSAY-II: Tutorial Intro- duction and Retrospective View. Tech. Report, Computer Science Department, Carnegie-Mellon University, 1978. 10. Hayes, P. J., and Reddy, R. Graceful Interaction in Man-

Machine Communication. Proc. 6th IJCAI, Tokyo, 1979, pp. 372-374.

11. Hendrix, G. G. Human Engineering for Applied Natural Lan- guage Processing. Proc. 5th IJCAI, MIT, 1977, pp. 183-191. 12. Kaplan, S. J. Cooperative Responses from a Portable Natural

Language Data Base Query System. Ph.D. Thesis, Dept. of

Comp. and Inform. Science, University of Pennsylvania, 1979. 13. Kwasny, S. C. and Sondheimer, N. K. "Relaxation Techniques

for Parsing Grammatically Ill-Formed Input in Natural Lan- guage Understanding Systems." Am. J. Comp. Ling. 7, 2 (1981), 99-108.

14. Parkison, R. C., Colby, K. M., and Faught, W. S. "Conversational Language Comprehension Using Integrated Pattern-Matching and Parsing." Artif. Intell. 9 (1977), 111-

134.

15. Waltz, D. L. "An English Language Question Answering System for a Large Relational Data Base." Comm. ACM 21, 7

(1978), 526-539.

16. Weischedel, R. M. and Black, J. "Responding Intelligently to Unparsable Inputs." Am. J. Comp. Ling. 6, 2, (1980), 97-109. 17. Wilks, Y. A. Preference Semantics. In Formal Semantics of

Natural Language, Keenan, Ed., Cambridge Univ. Press, 1975.

18. Woods, W. A. "Transition Network Grammars for Natural Language Analysis." Comm. ACM 13, 10 (Oct. 1970), 591- 606.

19. Woods, W. A., Kaplan, R. M., and Nash-Webber, B. The Lunar Sciences Language System: Final Report. TR 2378, Bolt Beranek and Newman, Inc., Cambridge, Mass., 1972.

20. Woods, W. A., Bates, M., Brown, G., Bruce, B., Cook, C., Klovstad, J., Makhoul, J., Nash-Webber, B., Schwartz, R., Wolf, J., and Zue, V. Speech Understanding Systems - Final Technical Report. TR 3438, Bolt Beranek and Newman, Inc., Cambridge, Mass., 1976.

P h i l i p J. H a y e s is a research c o m p u t e r scientist in the D e p a r t m e n t o f C o m p u t e r S c i e n c e at C a r n e g i e - M e l l o n

University. H e received the D . S c . degree in c o m p u t e r

science f r o m the E c o l e p o l y t e c h n i q u e f e d e r a l e de L a u - s a n n e in 1977.

George V. M o u r a d i a n is a research p r o g r a m m e r in the D e p a r t m e n t o f C o m p u t e r S c i e n c e at C a r n e g i e - M e l l o n

University. H e received the M . P h . degree in c o m p u t e r