Meta rules as a Basis for Processing Ill Formed input

(1)

M e t a - r u l e s as a Basis f o r P r o c e s s i n g

I l l - F o r m e d I n p u t 1

Ralph M . W e i s c h e d e l 2

D e p a r t m e n t of C o m p u t e r & I n f o r m a t i o n S c i e n c e s U n i v e r s i t y of D e l a w a r e

N e w a r k , DE 19716

N o r m a n K. S o n d h e i m e r

U S C / I n f o r m a t i o n S c i e n c e s I n s t i t u t e

4 6 7 6 A d m i r a l t y W a y M a r i n a del Rey, C A 9 0 2 9 2

If natural language processing systems are ever to achieve natural, cooperative behav- ior, they must be able to process input that is ill-formed lexically, syntactically, semantical- ly, or pragmatically. Systems must be able to partially understand, or at least give specific, appropriate error messages, when input does not correspond to their model of language and o f context.

W e propose m e t a - r u l e s and a control structure under which they are invoked as a framework for processing ill-formed input. The left-hand side o f a meta-rule diagnoses a problem as a violated rule of normal processing. The right-hand side relaxes the violated rule and states how processing may be resumed, if at all.

Examples discussed in the paper include violated grammatical tests, omitted articles, homonyms, spelling/typographical errors, unknown words, violated selection restrictions, personification, and metonymy. An implementation o f a meta-rule processor within the framework of an augmented transition network parser is also described.

1. Introduction

N a t u r a l l a n g u a g e u n d e r s t a n d i n g s y s t e m s h a v e i m - p r o v e d m a r k e d l y i n r e c e n t y e a r s , a n d n a t u r a l l a n g u a g e i n t e r f a c e s h a v e e v e n b e g u n to e n t e r t h e c o m m e r c i a l m a r k e t p l a c e , f o r e x a m p l e , t h e I N T E L L E C T s y s t e m of A r t i f i c i a l I n t e l l i g e n c e C o r p o r a t i o n ( H a r r i s 1 9 7 8 ) . T h e s e s y s t e m s p r o m i s e to m a k e m a j o r i m p r o v e m e n t s in the ease of use of d a t a b a s e m a n a g e m e n t a n d o t h e r c o m p u t e r s y s t e m s . H o w e v e r , t h e y h a v e o n l y b e g u n to

l This material is based upon work supported in part by the National Science Foundation under Grant Nos. 1ST-8009673 and IST-8311400 and in part by the Defense Advanced Research Pro- jects Agency under Contract No. MDA 903-81-C-0335, ARPA Order No. 2223. Views and conclusions contained in this paper are the authors' and should not be interpreted as representing the official opinion or policy of DARPA, the U.S. Government, or any person or agency connected with them.

2 Currently visiting at the Department of Computer & Informa- tion Science, University of Pennsylvania, Philadelphia, PA 19104.

c o n s i d e r t h e p r o b l e m s of t r u l y n a t u r a l i n p u t . T h e e m p h a s i s has b e e n , a n d c o n t i n u e s to b e , o n t h e u n d e r - s t a n d i n g of w e l l - f o r m e d i n p u t s . T r u e n a t u r a l l a n g u a g e i n p u t is o f t e n i l l - f o r m e d in t h e a b s o l u t e s e n s e of b e i n g filled w i t h m i s s p e l l i n g s , m i s t y p i n g s , m i s p u n c t u a t i o n s , t e n s e a n d n u m b e r e r r o r s , w o r d o r d e r p r o b l e m s , r u n - o n s e n t e n c e s , e x t r a n e o u s f o r m s , m e a n i n g l e s s s e n t e n c e s , a n d i m p o s s i b l e r e q u e s t s . I n a d d i t i o n , n a t u r a l i n p u t is i l l - f o r m e d in t h e r e l a t i v e s e n s e of c o n t a i n i n g r e q u e s t s t h a t are b e y o n d the limits of e i t h e r t h e c o m p u t e r sys- t e m or t h e n a t u r a l l a n g u a g e i n t e r f a c e .

E v i d e n c e i n d i c a t e s t h a t a b s o l u t e l y i l l - f o r m e d i n p u t r e g u l a r l y o c c u r s in a d a t a b a s e q u e r y e n v i r o n m e n t . F o r i n s t a n c e , in a n e x t e n s i v e s t u d y ( T h o m p s o n 1 9 8 0 ) i n c l u d i n g 1615 i n p u t s , o n l y 1093 w e r e p a r s a b l e , a n d a n o v e r a l l t o t a l of 4 4 6 c o n t a i n e d v a r i o u s k i n d s of e r - rors: 161 w i t h v o c a b u l a r y p r o b l e m s , 72 w i t h p u n c t u a - t i o n e r r o r s , 62 w i t h u n g r a m m a t i c a l i t y , a n d 61 w i t h

Copyright 1984 by the Association for Computational Linguistics. Permission to copy without fee all or part of this material is granted provided that the copies are not made for direct commercial advantage and the Journal reference and this copyright notice are included on the first page. To copy otherwise, or to republish, requires a fee and/or specific permission.

0 3 6 2 - 6 1 3 X / 8 3 / 0 3 0 1 6 1 - 1 7 5 0 3 . 0 0

(2)

spelling errors. F u r t h e r m o r e , 211 inputs were frag- m e n t a r y , including 91 p a r s e d terse replies and 67 p a r s e d terse questions.

In a n o t h e r e x p e r i m e n t ( E a s t m a n and M c L e a n 1981), 693 English queries to a data b a s e system were analyzed. C o - o c c u r r e n c e violations, including subject- v e r b disagreement, tense errors, a p o s t r o p h e p r o b l e m s , and p o s s e s s i v e / p l u r a l errors o c c u r r e d in 1 2 . 3 % of the queries. O m i t t e d words, e x t r a n e o u s w o r d s / p h r a s e s , telegraphic ellipsis, and i n c o m p l e t e sentences arose in 1 4 % of the queries.

O u r c o n j e c t u r e is t h a t w h e r e v e r n a t u r a l l a n g u a g e i n t e r f a c e s are e m p l o y e d , i l l - f o r m e d input will occur. A n y natural language interface, w h e n faced with ill- f o r m e d input, must either intelligently guess at a user's intent, request direct clarification, or at the very least a c c u r a t e l y i d e n t i f y the i l l - f o r m e d n e s s . As Wilks ( 1 9 7 6 ) states, " U n d e r s t a n d i n g requires, at the v e r y least, ... s o m e a t t e m p t to interpret, r a t h e r t h a n merely reject, what seems to be ill-formed u t t e r a n c e s . "

T h o u g h experience has s h o w n that users c a n a d a p t to the limitations of the s y s t e m ' s w e l l - f o r m e d antici- p a t e d input (Harris 1977b, H e n d r i x et al. 1978), we feel that relying on such user a d a p t a t i o n ignores one of the m o s t p o w e r f u l m o t i v a t i o n s for English input: e n a b l i n g i n f r e q u e n t users to access d a t a w i t h o u t an i n t e r m e d i a r y and without extensive practice. E v e n the p e r s o n w h o f r e q u e n t l y uses such a s y s t e m will be exas- p e r a t e d if it c a n n o t explain w h y it m i s u n d e r s t a n d s an input.

In some circumstances, ill-formed input m a y be less frequent. F o r instance, F i n e m a n (1983) r e p o r t s that in an e x p e r i m e n t where users were c o n s t r a i n e d to dis- crete speech, ill-formedness o c c u r r e d in as little as 2 % of the input. A n o t h e r unusual e n v i r o n m e n t can be created by informing the users that the s y s t e m c a n n o t really u n d e r s t a n d n a t u r a l language, thus biasing language use.

In addition to natural language interfaces, p r o c e s s - ing ill-formed input is critical to correcting language use. P r o t o t y p e systems have already b e e n investigated in the l a n g u a g e - l e a r n i n g e n v i r o n m e n t ( W e i s c h e d e l et al. 1978) and in the office a u t o m a t i o n e n v i r o n m e n t of d o c u m e n t p r e p a r a t i o n (Miller et al. 1981). E v e n in published text unintentional ungrammaticalities occur.

M o s t natural language u n d e r s t a n d i n g systems deal with a few types of ill-formedness. O u t of o u r o w n work, and that of others, we h a v e p r o d u c e d a f r a m e - w o r k for processing ill-formed input. This a p p r o a c h treats ill-formedness as rule-based. First, natural language interfaces should process all input as p r e s u m a b l y w e l l - f o r m e d until the rules of n o r m a l p r o c e s s i n g are violated. At t h a t point, e r r o r handling p r o c e d u r e s b a s e d on meta-rules relating ill-formed input to well-

3 The purpose of these meta-rules is therefore quite distinct from that of Gawron (1982).

f o r m e d structures t h r o u g h the modification of the violated normal rules should be e m p l o y e d . T h e s e m e t a - rules c o r r e s p o n d to types of errors. 3

T h e rest of the p a p e r argues f o r this r u l e - b a s e d a p p r o a c h . Section 2 c h a r a c t e r i z e s b o t h the types of ill-formed input, and the types of possible a p p r o a c h e s to them, including our proposal. Section 3 gives examples of m e t a - r u l e s for processing ill-formed input. Section 4 describes h o w s o m e heuristics d e v e l o p e d b y others fit within our paradigm. A n i m p l e m e n t a t i o n is s k e t c h e d in Section 5. Section 6 discusses limitations of the proposal. Sections 7 and 8 p r e s e n t directions for future work and conclusions.

2 . A p p r o a c h e s t o I I I - f o r m e d n e s s

This section i n t r o d u c e s the p r o b l e m of i n t e r p r e t i n g i l l - f o r m e d input. First, we discuss the t y p e s of ill- f o r m e d input briefly. T h e n we consider the range of a p p r o a c h e s that have b e e n tried f o r allowing for such input.

I l l - f o r m e d n e s s p h e n o m e n a can be divided into two sets. T h e first defines w h a t we call absolute ill-formedness. An u t t e r a n c e is absolutely ill-formed if the typical listener considers it ill-formed. T h e definition u n f o r t u n a t e l y a p p e a l s to s u b j e c t i v e e v a l u a t i o n s ; these are k n o w n to differ widely (Ross 1979). But it s e e m s to include the m a j o r i t y of typical cases a n d exclude the m a j o r i t y of t y p e s of g o o d English sentences.

The second set defines relative ill-formedness. This is ill-formedness with r e s p e c t to the n o r m a l processing rules of the f o r m a l c o m p u t i n g s y s t e m including the natural language interface and the underlying application system. T h e set of ill-formed inputs for an interface can be defined as the union of these two sets for that interface.

The set of ill-formed input c a p t u r e d by these defi- nitions can also be seen t h r o u g h the f o u r typical phas- es of i n t e r p r e t a t i o n in n a t u r a l l a n g u a g e i n t e r f a c e s : lexical, syntactic, semantic, and p r a g m a t i c processing. In lexical processing, absolute ill-formedness can c o m e f r o m misspelling, mistyping, a n d m i s p r o n u n c i a t i o n ; relative ill-formedness can arise f r o m u n k n o w n words. In syntactic processing, absolute ill-formedness is seen in faulty s u b j e c t - v e r b a g r e e m e n t , w o r d o r d e r errors, o m i t t e d words, r u n - o n s e n t e n c e s , etc; relative ill- f o r m e d n e s s is seen in g r a m m a t i c a l c o m b i n a t i o n s of words that exceed the i n t e r f a c e ' s g r a m m a r .

Semantic processing c a n be d e f i n e d as the interpretation of the input in isolation. K n o w l e d g e of the task d o m a i n can be applied, but the c o n t e x t of input with respect to previous interactions and the state of the u n d e r l y i n g c o m p u t i n g s y s t e m are o n l y c o n s i d e r e d in p r a g m a t i c processing.

A b s o l u t e ill-formedness in s e m a n t i c s includes omitting n e e d e d i n f o r m a t i o n and violating of selectional restrictions. A b s o l u t e i l l - f o r m e d n e s s in p r a g m a t i c s

(3)

Ralph M. Weischedel and Norman K. Sondheimer Mete-rules as a Basis for Processing Ill-Formed Input

includes b r e a k i n g the rules of conversation, as w h e n answering a question with a question, having p r e s u p - positions of the s p e a k e r fail, and failing to m a k e clear an a n a p h o r i c r e f e r e n c e . R e l a t i v e i l l - f o r m e d n e s s in b o t h cases includes " o v e r s h o o t " , requesting capabili- ties or i n f o r m a t i o n not c o v e r e d by the s y s t e m in its current state, and p a r e n t h e t i c a l expressions i n c o m p r e - hensible to the system.

2.1. Four a l t e r n a t i v e a p p r o a c h e s to

ill-formedness

T h e r e are at least five a p p r o a c h e s o n e c a n t a k e to ill-formedness. This section outlines the four alternatives to the a p p r o a c h we have f o r m u l a t e d ; our ap- p r o a c h is c o v e r e d in Section 2.2. In describing the five a p p r o a c h e s , we use the following informal n o t a - tion. SYSTEM[s] refers to a system designed to process a set of sentences s. WELL-FORMED is a set of w e l l - f o r m e d utterances; ILL-FORMED is a set of ill- f o r m e d utterances. Naturally, an a p p r o a c h that covers the b r o a d e s t range of linguistic b e h a v i o u r should be preferred.

O n e a l t e r n a t i v e is to t r e a t the p r o c e s s i n g of ill- f o r m e d and w e l l - f o r m e d inputs identically, by ignoring constraints. T h a t is, o n e designs SYSTEM[WELL- FORMED U ILL-FORMED]. Schank et al. (1980) and Waltz (1978) have t a k e n this a p p r o a c h t o w a r d g r a m - matical constraints. CASPAR ( H a y e s and C a r b o n e l l 1981) exhibits this a p p r o a c h f o r g r a m m a t i c a l c o n - straints as well. Since there is m u c h r e d u n d a n c y in language, the practice of not using certain constraints will o f t e n work. H o w e v e r , this will fail on m a n y utterances, since it ignores rules that not only constrain search but also eliminate u n i n t e n d e d i n t e r p r e t a t i o n s . O n e can see this b y c o n s i d e r i n g s u b j e c t - v e r b a g r e e - ment, a g r a m m a t i c a l constraint that people s o m e t i m e s violate and that is often left out of natural language systems. T h o u g h other constraints, such as semantic (selection) restrictions b e t w e e n a v e r b and its subject, o f t e n indicate the intended interpretation, it is easy to think of e x a m p l e s w h e r e s u b j e c t - v e r b a g r e e m e n t is crucial to u n d e r s t a n d i n g . C o m p a r i n g e x a m p l e s (1) and (2) below, s u b j e c t - v e r b a g r e e m e n t is crucial to determining w h e t h e r the c o m p a n y or assets were pur- chased.

(1) List the assets of the c o m p a n y that was pur- chased by XYZ Corp.

(2) List the assets of the c o m p a n y that were pur- chased b y XYZ Corp.

A s e c o n d a p p r o a c h is to build s y s t e m s for well- f o r m e d input and for ill-formed input together; that is, o n e designs SYSTEM[ILL-FORMED] m e r g e d with SYSTEM[WELL-FORMED]. Unlike the first a p p r o a c h , w e l l - f o r m e d n e s s c o n s t r a i n t s are e m p l o y e d on well- f o r m e d input. LUNAR ( W o o d s et al. 1972), an early English interface to a q u e s t i o n - a n s w e r i n g system, and

SOPHIE ( B u r t o n and B r o w n 1977), an intelligent tu- toring s y s t e m with an English interface, b o t h used this a p p r o a c h . The p r o b l e m with this a p p r o a c h is that it does not reflect the fact that constraints indicate p r e f - erences in interpretation. F o r instance, though e x a m - ple (3) below has two legitimate syntactic i n t e r p r e t a - tions, the one that violates our m o d e l of the world is rejected, causing us to reject the " g a r d e n p a t h " interpretation.

(3) I saw the Statue of L i b e r t y flying to N e w York.

As a n o t h e r e x a m p l e , the t w o p r o n o u n s in " H e shot h i m " are n o r m a l l y c o n s i d e r e d to r e f e r to d i f f e r e n t people; the alternative that the s p e a k e r m e a n t " H e shot h i m s e l f " does not arise unless t h e r e are s t r o n g e x p e c t a t i o n s a h e a d of time t h a t t h a t is the c o r r e c t proposition.

A third a p p r o a c h is to build two systems, but to use

SYSTEM[ILL-FORMED] only if SYSTEM[WELL-

FORMED] finds no i n t e r p r e t a t i o n . A c o m m e r c i a l l y available English interface to data bases (Harris 1977) has t a k e n this a p p r o a c h . T h e EPISTLE p r o j e c t ( J e n s e n a n d H e i d o r n 1983, Miller et al. 1981) e m p l o y s this a l t e r n a t i v e f o r g r a m m a t i c a l violations. DYPAR ( C a r b o n e l l et al. 1983) has t a k e n this a p p r o a c h in an interface to an e x p e r t system. K a p l a n (1978) devel- o p e d a strategy to give m o r e useful r e s p o n s e s w h e n a data base query yields a negative response, for e x a m - ple, w h e n no e n t i t y satisfies the desired conditions. C h a n g (1978) c r e a t e d a heuristic for inferring missing joins in i n c o m p l e t e queries to relational d a t a bases. T h e defect in this m o d e l is that there is no m e a n s of relating strategies for processing ill-formedness explicitly to the strategies f o r p r o c e s s i n g w e l l - f o r m e d n e s s . We argue here that one can explicitly relate the t w o classes of strategies.

A f o u r t h a p p r o a c h is to build o n l y one s y s t e m , SYSTEM[WELL-FORMED], and to e m p l o y a metric to m e a s u r e h o w f a r a p o s t u l a t e d i n t e r p r e t a t i o n is f r o m satisfying all w e l l - f o r m e d n e s s constraints. C h a r n i a k (1981) has a d v o c a t e d this for g r a m m a t i c a l processing; Wilks ( 1 9 7 5 ) has m a d e this the basis of s e m a n t i c processing during the i n t e r p r e t a t i o n phase. O f course, the notion of weighing alternatives and using metrics has b e e n used for p h e n o m e n a o t h e r t h a n ill-formedness, such as p a r s i n g ( R o b i n s o n 1982) and s p e e c h u n d e r s t a n d i n g ( W a l k e r 1978, W o o d s et al. 1976). Clearly, r a n k i n g a l t e r n a t i v e i n t e r p r e t a t i o n s is n e c e s - sary. H o w e v e r , if one relies

solely

on a metric and SYSTEM[WELL-FORMED], t h e n an a c c o u n t of the fact that the ill-formedness o f t e n has specific implications is still needed. In e x a m p l e (4), the selection restriction that " l i k e " requires a n i m a t e agents is violated; a rea- sonable inference is that the s p e a k e r s o m e w h a t per- sonifies the c o m p u t e r in question.

(4) M y h o m e c o m p u t e r d o e s n ' t like to run BASIC.

(4)

N o r does a metric reflect the fact that there are clear patterns of error, such as those that have been report- ed in linguistic studies ( F r o m k i n 1973) and in application studies ( T h o m p s o n 1980, E a s t m a n and M c L e a n 1981).

Table I summarizes these four approaches.

2 . 2 . O u r a p p r o a c h

Based on previous work, b o t h our o w n and that of others, we propose a f r a m e w o r k employing meta-rules to relate the processing of ill-formed input to well- formedness rules. This f r a m e w o r k m a y be stated as follows:

1. Process the input using SYSTEM[WELL-FORMED]. 2. If no i n t e r p r e t a t i o n is f o u n d by SYSTEM

[WELL-FORMED], apply a meta-rule to the well- formedness rules, based on a ranking of the alternatives, in order to

a) diagnose the problem, that is, the rule that is violated and h o w it is violated,

b) relax the rule,

c) add a " d e v i a n c e n o t e " to the interpretation re- cording the violation,

d) resume processing via the well-formedness rules, if possible.

3. R e p e a t step 2 as necessary.

E a c h meta-rule should c o r r e s p o n d to a pattern of ill- formedness and should a c c o u n t for utterances corre- sponding to only that pattern. SYSTEM[ILL-FORMED] is therefore implicit in the meta-rules.

This f r a m e w o r k has advantages lacking in one or more of the other approaches. Well-formedness constraints, whether syntactic, semantic, or pragmatic, are e m p l o y e d to eliminate u n i n t e n d e d interpretations.

Well-formed interpretations are always preferred. Ill- formedness processing is explicitly related to the well- formedness rules. Only the constraint that seems to be violated is relaxed; all o t h e r w e l l - f o r m e d n e s s constraints are still effective. F u r t h e r m o r e , the deviance notes record the aspect that deviates f r o m well- f o r m e d n e s s , thus allowing p r a g m a t i c inferences by later processing.

In the next two sections, we propose a handful of primitives for s y n t a c t i c and s e m a n t i c p r o b l e m s and also propose a formalism for writing meta-rules. As supporting evidence, we state meta-rules for a n u m b e r of problems and describe a p p r o a c h e s for several others. These p h e n o m e n a include the following:

failed grammatical tests, word confusions, spelling errors, u n k n o w n words, restarted sentences,

resumptive p r o n o u n s and n o u n phrases, contextual ellipses,

selection restriction violation, m e t o n y m y ,

personification, and presupposition failure.

3. E x a m p l e s o f M e t a - r u l e s

To further argue for recta-rules as a uniform framework for processing ill-formed input, we describe a wide variety of meta-rules in this section and the next. We adopt the following n o t a t i o n for meta-rules in this paper:

Approach 1

Characterization:

Flaw:

Approach 2

Characterization:

Flaw:

Approach 3

Characterization:

Flaw:

Approach 4

Characterization:

Flaw:

Do not encode well-formedness constraints.

Well-formedness rules convey meanings by constraining interpretations.

Design systems for well-formed input and ill-formed input together.

This gives no preference of well-formed interpretations over ill-formed ones.

Search for well-formed interpretations prior to considering any ill-formed ones.

This does not explicitly relate handling ill-formedness to processing well-formedness.

Use a metric to rank ill-formedness interpretations, and select the one that comes closest to satisfying all constraints.

This does not state what the deviation is so that one may draw inferences from the ill-formedness, nor does it capture actual patterns of error.

Table 1. Four Rejected Approaches.

(5)

Ralph M. WeischedeI and Norman K. Sondheimer Meta-rules as a Basis for Processing Ill-Formed Input

C1 C2 ... C n - - > A1 A2 ... A m

The left-hand side (LHS) diagnoses what the problem might be; the right-hand side (RHS) states how to relax the failed constraint. The Ci are conditions on the computational state of the system; all must be true if the meta-rule is to apply. The Ai are actions stating h o w to rewrite the violated c o n s t r a i n t and resume processing; all will be e x e c u t e d if the rule applies. M a n y of the actions we suggest here can be viewed as rewriting a rule of the normative system, for example, a grammar rule or case frame. H o w e v e r , some are more appropriately viewed as changing the c o m p u t a - tional state when blockage occurs; an example is correcting the spelling of a word. In Section 5 we will argue that it is best to implement all of the actions as modifications of a blocked alternative.

Naturally, in rewriting a rule, p a t t e r n - m a t c h i n g and substitution are fundamental. We adopt the following definition of patterns. A p a t t e r n is a LISP s- expression. A t o m s preceded b y a question mark are variables. Expressions p r e c e d e d by a dollar sign are evaluated using the LISP rules; their values are treated as patterns. 4 If a period appears before a pattern variable that is the last item in a list, that pattern variable matches the tail of a list. All other items in patterns are literals. The scope of a pattern variable is the whole meta-rule. The first time a variable is b o u n d in a meta-rule, it retains the binding t h r o u g h o u t the rule.

Potentially, there may be many places where relaxation can occur. If a meta-rule applies to more than one configuration, it will be applied to each in turn, creating a list of possibilities for processing after recovery is complete. Consequently, the meta-rules will refer to only one failed configuration at a time.

3 . 1 . M e t a - r u l e s r e l a t e d t o s y n t a x

First, let us consider meta-rules dealing with the grammar. M a n y of our examples here are reformulations of our earlier work (Weischedel et al. 1978, K w a s n y and Sondheimer 1979, Weischedel and Black 1980) within the uniform f r a m e w o r k of meta-rules. All meta-rules pertaining to syntax should have a first c o n d i t i o n which is (SYNTAX-FAILED?); this is true iff the parser is blocked. Since all rules in this section would con- tain that predicate, we will not include it in the examples.

M a n y syntactic formalisms have a similar framework for expressing rules: these include c o n t e x t - f r e e grammars, a u g m e n t e d phrase structure g r a m m a r s ( H e i d o r n 1975), P r o g r a m m a r ( W i n o g r a d 1972), the linguistic string parser (Sager 1981), Lifer (Hendrix et al. 1978), and a u g m e n t e d transition networks (ATNs) (Woods 1970). In fact, all of these can be viewed as formally equivalent to ATNs, and we will describe our techniques in that framework.

Figure 1 gives several predicates that should be useful in the LHS of meta-rules. The LHS of the meta- rule is matched against the e n v i r o n m e n t in which an ATN arc failed. The environment is called a configuration and includes the current ATN state, the arc, all ATN registers, and the remainder of the input string.

4 The expression $expr could be implemented as (*EVAL* expr). The pattern variable ?atom could be implemented as (*VAR* atom).

(IN-STATE? state)

(CAT? category)

(WRD? list)

(NEXTCAT? category)

(NEXTWRD? list)

(FAILED-TEST? pattern)

(FAILED-ARC? pattern)

(HOLDLIST-NOT-EMPTY?)

(CONFUSION-WORD? x)

Did the configuration block in state?

Is the current word in category?

Is the current word a member of list?

Is the word after the current one in category?

Is the word after the current one in list?

Is the pattern a predicate expression in the test of the arc and did both the expression and the test evaluate to false?

Does the failed arc match pattern?

Is the hold list empty?

Is x a word frequently confused with another? If so, the related word is returned.

Figure I. Useful Conditions for Syntactic Meta-rules.

(6)

( E M P T Y - H O L D )

( F A I L E D - C O N S T R A I N T pattern)

( S U B S T I T U T E - I N - A R C p a t t e r n l pattern2)

( R E P L A C E - * x)

defines the hold list to be empty.

adds the instantiation of the pattern to a list of violated constraints stored in the configuration. The position of the parser within the input string will automatically be recorded as well.

changes the arc in the failed configuration by replacing all expressions matching p a t t e r n l by pattern2.

makes x the current word in the blocked configuration.

Figure 2. Useful Actions for Syntactic Meta-rules.

State

s~

S / N P

s / v

S / P O P

N P /

N P / D E T

Arcs

(PUSH N P / T ...

(SETR S U R F A C E - S U B J E C T *) (* We think this is the subject) (TO S / N P ) )

(VIR NP T (SETR S U R F A C E - S U B J E C T *)

(* In relative clauses, this identifies a noun phrase from the hold list as surface subject) (TO S / N P ) )

(Figure 3.)

( C A T VERB ( S U B J E C T - V E R B - A G R E E ? ( G E T R S U R F A C E - S U B J E C T ) *)

(* The predicate S U B J E C T - V E R B - A G R E E ? is true iff the number and person of the subject and verb are compatible)

... (SETR VERB *) (TO S / V ) )

(JUMP S / P O P ( I N T R A N S I T I V E - V E R B ? ( G E T R V E R B ) ) ) ( C A T A D V T ... (TO S / V ) )

(PUSH NP T (SETR O B J E C T *)... (TO S / P O P ) ) (VIR NP T (SETR O B J E C T *)

(* Identifies a noun phrase from the hold list as the direct object in relative clauses) (TO S / P O P ) )

(POP ( B U I L D Q ...)

( A N D ( T R A N S I T I V E - V E R B ? ( G E T R V E R B ) )

(NOT ( R E Q U I R E S - I N D I R E C T - O B J ? ( G E T R V E R B ) ) ) ) )

( C A T PRO T ... (SETR PRO *) (TO N P / P O P ) ) ( C A T DET T ...

(SETR DET *)

(SETR N U M B E R ( G E T N U M B E R DET)) (TO N P / D E T ) )

( C A T ADJ T ...

(* collecting adjectives before head noun) (TO N P / D E T ) )

( C A T N T ... (SETR N *) (TO N P / N ) )

(7)

Ralph M. W e i s c h e d e l and Norman K. Sondheimer Meta-rules as a Basis for Processing Ill-Formed Input

NP/N (CAT N T ...

(ADDR COMPOUND (GETR N)) (SETR N *) (* a possible nominal compound)

(TO NP/N) (JUMP NP/POP

(DET&NOUN-AGREE? (GETR NUMBER) (GETR N))

(* The predicate DET&NOUN-AGREE? is true iff the determiner used is incompatible with the head noun)

. H )

(PUSH R E L C L / T

(SENDR TRACE (TRACE))

(* This sends a trace of a noun phrase to a relative clause) (SETR RELATIVE-CLAUSE *) (TO NP/POP))

NP/POP (POP (BUILDQ ...) T)

R E L C L / (CAT RELPRO T (HOLD (GETR TRACE)) ... (TO S/))

(PUSH NP T (SETR SURFACE-SUBJECT *) (HOLD (GETR TRACE))

(* This allows for relative clauses where the subject is present) (TO S/NP))

(JUMP S/NP (SETR SURFACE-SUBJECT (GETR TRACE)) (* Allows for elided subjects in reduced

relative clauses))

Figure 3. A Simple ATN Graph.

A n i m p o r t a n t a c t i o n f o r the RHS is NEW-CONFIGURATION, which defines a new parser configuration, thus replacing the failed c o n f i g u r a t i o n that the meta-rule matched. It may take any n u m b e r of arguments which set parts of the configuration. F o r example, SETR will define the new value of an ATN register. A list of useful a r g u m e n t s to NEW-CON- FIGURATION is given in Figure 2. Failed constraints fill the role of the deviance notes of K w a s n y and Son- dheimer (1979). All parts of the failed configuration that are not explicitly changed in NEW-CONFIGURA- TION remain the same. O u r i m p l e m e n t a t i o n assumes that there is only one NEW-CONFIGURATION per meta-rule, t h o u g h one could generalize this so that executing NEW-CONFIGURATION n times in a meta- rule gives n new configurations to replace the failed one. If a new configuration is generated, the parse can be resumed.

Figure 3 gives a trivial ATN which will be used for the sample meta-rules. The start state is S / . A list beginning with an asterisk in the actions of an arc is a comment.

3.1.1. Simple grammatical t e s t s

Our earlier work showed how to relax tests that appear on ATN arcs. In one study ( E a s t m a n and Mc- L e a n 1981), s u b j e c t - v e r b d i s a g r e e m e n t o c c u r r e d in 2 . 3 % of the English queries. Meta-rule (i) relaxes that agreement test. The new configurations here are the result of replacing the a g r e e m e n t test in each failed arc by the predicate T. Since a new configuration is generated, parsing is resumed using it. T h o u g h the s u b s t i t u t i o n was trivial in this case, SUBSTITUTE-IN-ARC is a general p a t t e r n - m a t c h i n g and substitution facility. As an example, consider " A curious problem showing unusual conditions a p p e a r ..." A t o p - d o w n , left-to-right parse using a grammar such as the one in Figure 3 would block at the word " a p p e a r " . One of the blocked configurations would c o r r e s p o n d to the a g r e e m e n t test failure in the arc leaving state S/NP; meta-rule (i) would apply, allowing the sentence to be parsed.

(i) (FAILED-TEST? (SUBJECT-VERB-AGREE? ?X ?Y)) --> (NEW-CONFIGURATION

(FAILED-CONSTRAINT (SUBJECT-VERB-AGREE? ?X ?Y)) (SUBSTITUTE-IN-ARC (SUBJECT-VERB-AGREE? ?X ?Y) T))

(8)

3 . 1 . 2 . O m i t t e d a r t i c l e s

A n o t h e r frequently occurring p r o b l e m is omitting required articles f r o m c o u n t nouns. In the study by E a s t m a n and M c L e a n (1981) this o c c u r r e d in 3 . 3 % of all queries. In the g r a m m a r of Figure 3, b l o c k a g e would o c c u r at N P / N b e c a u s e of the test DET&NOUN-AGREE? on an e x a m p l e such as " P r i n t price of P27 o v e r the last five y e a r s " . Rule (ii) relaxes the test. W h e n the p a r s e r starts on the new configuration, the modified test will be checked, verifying that no d e t e r m i n e r is present. If none is, the m e s s a g e f r o m FAILED-CONSTRAINT is available for error recovery.

T h e m e t a - r u l e a p p r o a c h allows for m o r e sophisticated actions. Suppose that a linguistic study of utterances with missing d e t e r m i n e r s s h o w e d that a default a s s u m p t i o n of definite r e f e r e n c e is a g o o d heuristic. In this case, one could simply add the action (SETR DET 'the) to the actions in NEW-CONFIGURATION.

O n e could argue that, in a data b a s e e n v i r o n m e n t , the g r a m m a r should simply t r e a t o m i t t e d d e t e r m i n e r s as a n o r m a t i v e construction. E v e n though d e t e r m i n e r s are f r e q u e n t l y o m i t t e d in d a t a b a s e contexts, p r e f e r - ring w e l l - f o r m e d i n t e r p r e t a t i o n s c a n eliminate s o m e ambiguities in c o m p l e x n o u n phrases such as " a m a - chine running p r o g r a m s " . T h e d e t e r m i n e r constraint suggests t h a t " r u n n i n g p r o g r a m s " m o d i f i e s the h e a d n o u n " m a c h i n e " r a t h e r than " m a c h i n e " and " r u n n i n g " b o t h modifying " p r o g r a m s " .

3.1.3. C o n f u s i o n w o r d s

A n u m b e r of word pairs are f r e q u e n t l y confused, such as h o m o n y m s and " g o o d " for " w e l l " . M e t a - r u l e (iii) allows for such errors, since REPLACE-* will m o d i f y the current w o r d in the b l o c k e d configuration. M R - S E T Q binds the value of its second a r g u m e n t to the p a t t e r n v a r i a b l e a p p e a r i n g as its first a r g u m e n t . H e n c e , " Y o u p e r f o r m e d g o o d " would b l o c k at S / V , and the m e t a - r u l e would substitute " w e l l " for " g o o d " .

3 . 1 . 4 . R e s u m p t i v e p r o n o u n s

A n o t h e r kind of ill-formedness is r e s u m p t i v e p r o n o u n s and r e s u m p t i v e n o u n phrases. T h e s e occur in relative clauses w h e r e the e n t i t y r e f e r r e d to b y the relative p r o n o u n is i m p r o p e r l y r e p e a t e d in the relative clause as a p r o n o u n or n o u n phrase. A n e x a m p l e is " J o h n ' s friend M a r y m a r r i e d the m a n that she p l a n n e d to m a r - ry h i m " , since there is no syntactic slot in the relative clause for the relative p r o n o u n " t h a t " to fill. A typical ATN s t r a t e g y f o r interpreting relative clauses is to put a place holder or trace on a " h o l d list"; the ATN p r o c e s s o r p r e v e n t s POPping f r o m a level if the hold list is n o n - e m p t y . T h a t test p r e v e n t s a c c e p t i n g clauses where traces are n o t used. M e t a - r u l e (iv) p r o v i d e s f o r r e s u m p t i v e p r o n o u n s a n d r e s u m p t i v e n o u n p h r a s e s . O n e can imagine m o r e c o m p l i c a t e d tests, since there are specific c o n d i t i o n s ( K r o c h 1981) u n d e r which r e s u m p t i v e p r o n o u n s and r e s u m p t i v e n o u n phrases are m o r e likely.

(ii) (FAILED-TEST? (DET&NOUN-AGREE? ?X ?Y)) --> (NEW-CONFIGURATION

(SUBSTITUTE-IN-ARC (DET&NOUN-AGREE? ?X ?Y) (NULL ?X))

(FAILED-CONSTRAINT (DETERMINER&NOUN ?Y -- MISSING DETERMINER)))

(iii) (MR-SETQ ?X (CONFUSION-WRD *)) ---> (NEW-CONFIGURATION

(REPLACE-* ?X)

(FAILED-CONSTRAINT (?X SUBSTITUTED FOR *)))

(iv) (FAILED-ARC? (POP ? V A L U E . ?Z)) (IN-STATE? S/POP)

(HOLDLIST-NOT-EMPTY?) ---> (NEW-CONFIGURATION

(FAILED-CONSTRAINT (Resumptive Clause $?VALUE)) (EMPTY-HOLD))

(v) (IN-STATE? S/POP)

--> (PRINT-RESPONSE-PATTERN

(I DO NOT UNDERSTAND YOUR USE OF THE VERB $(GETR VERB) /. WOULD YOU LIKE EXAMPLES OF WHAT I UNDERSTAND?))

(SELECTQ (READ)

((YES Y) (PRINT-EXAMPLES (GETR VERB))) NIL)

(9)

Ralph M. Weischedel and Norman K. Sondheimer IVleta-rules as a Basis for Processing Ill-Formed Input

(FAILED-SEMANTIC-TEST? pattern)

(MANDATORY-CASE-MISSING?)

(TOO-MANY-FILLERS?)

(SEMANTIC-CLASS? class)

(MATRIX-TYPE? class)

(VIEWABLE? x y z)

(CASE? case)

Is the pattern a predicate expression (on a constituent) and is the predicate false?

Is a required case absent?

In trying to fill case, does it already have the maximum number assigned?

Is class a semantic class predicate, for example, human?

Is the matrix to which we are trying to assign the current constituent in class?

Can entity x be viewed as a y in the context z? (This is a question for the pragmatic component.)

Is the constituent supposed to fill role case?

Figure 4. Predicates for Semantic Meta-rules.

(FAILED-CONSTRAINT pattern)

(SUBSTITUTE-IN-CASE patternl pattern2)

(SUBSTITUTE-FOR-CASE case)

adds the instantiation of the pattern to a list of violated constraints stored in the configuration.

replaces patternl by pattern2 everywhere in the constraint.

tries assigning the constituent as case.

Figure 5. Actions for Semantic Meta-rules.

3.1.5. Error m e s s a g e s

Of course, the parser m a y not be able to recover at all due to either absolute or relative ill-formedness. Weischedel and Black (1980) p r e s e n t e d a technique for associating error messages with states where the parser blocked. The only way to block in S/POP is if the verb complement expected for the main verb is not present. Meta-rule (v) could handle this simple case. Notice that this is a different class of meta-rule, for it does not resume computation. Naturally, such rules should be tried only after no o t h e r meta-rules are available. One could define different classes of meta- rules by appropriate declarations; alternatively, this class can be recognized easily, since none of the actions resume processing.

This is not the only alternative in the face of failure to parse even with relaxation; Jensen and Heidorn (1983) present heuristics for what to pass to the semantic interpreter in this case, given b o t t o m - u p parsing.

3 . 2 . M e t a - r u l e s r e l a t e d t o s e m a n t i c s

In addition to these syntactic examples, semantic problems can also be addressed within the formalism. If some semantic tests are included in the parser, say a certain arc test contains calls on the semantic c o m p o - nent, specific semantic tests can be relaxed by the general mechanism we described for relaxing tests on ATN arcs.

Instead, suppose that semantic constraints are en- coded in a separate c o m p o n e n t . Semantic constraints may be expressed in several formalisms, such as semantic nets ( B o b r o w and W e b b e r 1980a,b; Sondheim-

er et al. 1984), first-order logic, and production rules (for example, PROLOG, Warren et al. 1977). It is generally agreed that all are formally equivalent to first-order logic. F o r the purposes of this paper, we assume that the selection restrictions are e n c o d e d in first-order logic.

One of the most c o m m o n designs for a semantic interpreter is based on selection restrictions and case frames (Bruce 1975). At least five kinds of constraints may be violated:

1) what may fill a given case,

2) which cases are required for a complete constituent,

3) which m a y have multiple fillers without c o n j u n c - tion,

4) which are allowed for a given case frame, and 5) what order cases m a y appear in.

Figure 4 lists tests useful for diagnosing failures in such a semantic interpreter. Assume that any predicate on the semantic class of a constituent is e n c o d e d simply in LISP notation, for example, (HUMAN x) is true iff x is of class human. All meta-rules in this section can be assumed to include an initial test (SEMANTICS-FAILED?.)

F o r convenience, we have used the same names for some of the actions as in the syntactic cases (for example, FAILED-CONSTRAINT, NEW-CONFIGURA- TION, etc.). W h e n implemented in a particular system, different names m a y be used, since the c o n c e p t of configuration, blockage, etc., is usually different for the types of processing (for example, lexical, syntactic, semantic, and pragmatic). Figure 5 lists several actions useful in semantic meta-rules.

(10)

CONCRETE

A N I M A T E I N A N I M A T E

/ , , ,

H U M A N A N I M A L

Figure 6. A Fragment of a Semantic Hierarchy.

3.2.1. Universal relaxation of semantic class M e t a - r u l e (vi) is a very general rule. A s s u m i n g that s e m a n t i c class tests are o r g a n i z e d in a h i e r a r c h y , it states that the failed test is to be replaced b y its p a r - ent in the h i e r a r c h y , yielding the next m o s t g e n e r a l test.

An e x a m p l e of the use of m e t a - r u l e (vi) is " M y car drinks gasoline". T h e restriction on the AGENT case could be the predicate ANIMATE. A f r a g m e n t of a s e m a n t i c h i e r a r c h y a p p e a r s in Figure 6. In that, ANIMATE has a p a r e n t predicate of CONCRETE that would include cars. T h e failure of the initial s e n t e n c e a n d the s u b s e q u e n t p r o c e s s i n g using the m e t a - r u l e would accept a s e n t e n c e with the special deviance n o t e identifying the semantic oddity.

3.2.2. Personification

In a w a y similar to our a r g u m e n t s against a p p r o a c h 4 in Section 2.1, we feel that general m e t a - r u l e s such as (vi) will p r o v e less v a l u a b l e t h a n specific rules. A particular test that can be relaxed is the r e q u i r e m e n t for a h u m a n ; for instance, the verbs of saying and those of propositional attitude, such as " b e l i e v e " and " t h i n k " , normally h a v e a restriction that their agent be h u m a n . N e v e r t h e l e s s , such a c o n s t r a i n t is regularly v i o l a t e d t h r o u g h p e r s o n i f i c a t i o n of pets, higher animals, machines, etc.

Since personification is infrequent c o m p a r e d to the n o r m of descriptions designating humans, a case c o n - straint of " h u m a n " can trim the search space. Since p e r s o n i f i c a t i o n c o n v e y s p a r t i c u l a r i n f e r e n c e s ( L a k o f f and J o h n s o n 1980), a relaxation rule that records the d e t e c t e d personification can trigger a p p r o p r i a t e inference processes. Figures of speech certainly are not absolutely ill-formed; we argue here that it is useful to treat t h e m as relatively ill-formed.

M e t a - r u l e (vii) is one simple relaxation for personi- fying animals. M o r e specific ones m a y p r o v e p r e f e r a - ble, if classes of p e r s o n i f i c a t i o n are t a x o n o m i z e d .

3.2.3. M e t o n y m y

T h e r e are at least seven classes of m e t o n y m y ( L a k o f f and J o h n s o n 1980), including a part for the whole, the p r o d u c e r for the product, the o b j e c t for its user, the

controller for the controlled entity, the institution for the p e o p l e r e s p o n s i b l e , the place for the institution, and the place f o r the event. This analysis suggests two kinds of strategies. A particular class of descriptions m a y occur in exactly the same linguistic e n v i r o n - m e n t s as their class of m e t o n y m o u s descriptions. F o r instance, institutions and p e o p l e a p p e a r i n t e r c h a n g e - able as the logical subject of the v e r b s of saying and of propositional attitude. T h a t can be e n c o d e d directly in the case f r a m e s of those verbs.

H o w e v e r , m a n y types of m e t o n y m y are c o n d i t i o n e d on a highly specialized relationship. F o r instance, places can be used m e t o n y m o u s l y for events only if the s p e a k e r believes an e v e n t is identifiably associated with the location. F o r instance, c o m p a r e the following examples:

Pearl Harbor caused us to enter the war.

*Fifth and Lombard caused us to reconsider graduated income taxes.

T h e r e f o r e , a m e t a - r u l e such as (viii) seems a p p r o p r i a t e to p r e f e r the normal, but accept m e t o n y m o u s descriptions of events by places. In m e t a - r u l e (viii), we have a s s u m e d that there is a variable FILLER of the s e m a n - tic i n t e r p r e t e r t h a t holds the c o n s t i t u e n t to be assigned. Also, in the call to VIEWABLE?, CURRENT indicates that the p r a g m a t i c c o m p o n e n t should use its current context.

3.2.4. Phrase ordering

Failure in selectional restrictions c a n indicate o t h e r s e m a n t i c errors. T h e s e include ordering p r o b l e m s , for example, " J o h n killed with a gun M a r y " , and u n e x - p e c t e d prepositions, for e x a m p l e , " J o h n killed M a r y by a g u n " . The LHS of the a p p r o p r i a t e m e t a - r u l e s would begin with identification of selectional restriction failures b u t would also include o t h e r tests. T h e RHS would change the a s s u m e d case. A rule for the first e x a m p l e is (ix). H e r e the a s s u m p t i o n is that the ordering p r o b l e m will be first n o t e d w h e n " M a r y " is tried as a time modifier. Using SUBSTITUTE-FOR- CASE p o s t u l a t e s the c o n s t i t u e n t " M a r y " to fill the object case and a t t e m p t s to do so.

[image:10.612.51.557.60.169.2]

(11)

Ralph M. Weischedel and Norman K. Sondheimer Meta-rules as a Basis for Processing Ill-Formed Input

(vi) (FAILED-SEMANTIC-TEST? (?Y ?Z)) (SEMANTIC-CLASS? ?Y)

- - > (NEW-CONFIGURATION

(FAILED-CONSTRAINT (?Y TOO RESTRICTIVE -- USING PARENT $(PARENT-OF ?Y))) (SUBSTITUTE-IN-CASE (?Y ?Z) ($(PARENT-OF ?Y) ?Z)))

(vii) (FAILED-SEMANTIC-TEST? (HUMAN ?Z)) --> (NEW-CONFIGURATION

(FAILED-CONSTRAINT

((HUMAN ?Z) -- ASSUMING PERSONIFICATION OF ANIMAL)) (SUBSTITUTE-IN-CASE (HUMAN ?Z)

(viii) (FAILED-SEMANTIC-TEST? (EVENT ?X)) (LOCATION FILLER) (VIEWABLE? FILLER 'EVENT 'CURRENT) --> (NEW-CONFIGURATION

(SUBSTITUTE-IN-CASE (EVENT ?X) T) (FAILED-CONFIGURATION (METONYMY PLACE-FOR-EVENT FILLER)))

(ix) (FAILED-SEMANTIC-TEST? (TIME ?Z)) (MATRIX-TYPE? 'CLAUSE) (CASE? 'TEMPORAL)

--> ( N E W - C O N F I G U R A T I O N (FAILED-CONSTRAINT (ORDERING PROBLEM -- OBJECT CASE AS- SUMED))

3 . 3 . G e n e r a l i t y o f t h e a p p r o a c h

T h o u g h we h a v e e x p e r i e n c e in i m p l e m e n t i n g our f r a m e w o r k for ATN p a r s e r s only, we believe the f r a m e w o r k to be applicable o v e r a b r o a d r a n g e of parsers. It a s s u m e s only t h a t a " c o n f i g u r a t i o n " or " a l t e r n a t i v e " representing a blocked, partial interpretation can be stored, modified, and restarted. N o as- s u m p t i o n regarding the direction of p r o c e s s i n g ( f o r example, left to right), the nature of search (for e x a m - ple, t o p - d o w n vs. b o t t o m - u p ) , nor the class of p r o b l e m (for example, lexical, syntactic, or semantic) is made. F o r instance, the design of an i m p l e m e n t a t i o n for semantic meta-rules as in Section 3.2 is complete. T h e underlying semantic c o m p o n e n t is b a s e d on searching case f r a m e s b r e a d t h - f i r s t with b o t h t o p - d o w n and b o t t o m - u p characteristics. E x c e p t for the one m e t a - rule regarding i n c o r r e c t p h r a s e o r d e r i n g in Section 3.2.4, the s e m a n t i c m e t a - r u l e s t h e m s e l v e s are inde- p e n d e n t of w h e t h e r p r o p o s i n g a p h r a s e f o r a given case in a f r a m e is b a s e d on syntactic considerations or other criteria (for example, Schank et al. 1980). N a t u - rally, the primitive conditions and actions of a given set of rules will d e p e n d on a particular formalism. In the next section, we relate our f r a m e w o r k to a variety of parsers and problems.

4. A d d i t i o n a l S u p p o r t i n g E v i d e n c e

M a n y natural language interfaces have some heuristics for processing one or m o r e classes of ill-formed input. Since an exhaustive analysis would be impossible here, we will review only a handful of techniques that have inspired us to d e v e l o p the m e t a - r u l e f r a m e w o r k . We describe each technique by showing h o w it could be p h r a s e d as a m e t a - r u l e within our paradigm.

T h e LADDER s y s t e m ( H e n d r i x et al. 1978) imple- m e n t s three m a j o r techniques f o r processing ill-formed input. All fit within the f r a m e w o r k we suggest. O n e deals with r e c o v e r y f r o m lexical processing. In this system, the d e v e l o p e r of a q u e s t i o n - a n s w e r i n g s y s t e m p r e p a r e s only a dictionary of w e l l - f o r m e d words. If a s e n t e n c e contains a word that is not in the dictionary, the parser will fail. T h e system localizes the area of failure to the ATN s t a t e a s s o c i a t e d with the partial i n t e r p r e t a t i o n that has p r o c e e d e d r i g h t m o s t in the input and that is shallowest (in terms of i n c o m p l e t e d ATN PUSH arcs). C a n d i d a t e s for the c o r r e c t spelling are limited to the words that would permit the parser to p r o c e e d and that are close to the spelling that appears. An equivalent m e t a - r u l e would check in the LHS that the parser failed. T h e RHS would c o m p u t e a list of words e x p e c t e d next for each type of arc leaving that state, for example, the c a t e g o r y m e m b e r s and literal words e x p e c t e d next. T h e next action would a p p l y the Interlisp spelling c o r r e c t i o n a l g o r i t h m to postulate a k n o w n word that was e x p e c t e d next. This w o r d would replace the u n r e c o g n i z e d one in the input and parsing would resume. A similar heuristic is running in our current i m p l e m e n t a t i o n , with the addition that, if the u n r e c o g n i z e d w o r d a p p e a r s to h a v e an inflected ending, spelling c o r r e c t i o n is p e r f o r m e d on the possible root.

A second technique in LADDER deals with under- standing contextual ellipsis, if no parse for the input is found. This heuristic i n t e r p r e t s " t h e f a s t e s t s u b m a r i n e " as " T o what c o u n t r y does the fastest sub- marine b e l o n g " , if it occurs after a q u e r y such as " T o what c o u n t r y does each m e r c h a n t ship in the N o r t h A t l a n t i c b e l o n g " . In W e i s c h e d e l a n d S o n d h e i m e r (1982), we e x t e n d e d that heuristic to allow for turn- taking in dialogues and to allow e x p a n s i o n s as well as

(12)

substitutions, such as the elliptical f o r m " L a s t m o n t h " following " D i d you go to C h i c a g o ? "

A third t e c h n i q u e in LADDER is the printing of error messages, in the same sense that m e t a - r u l e (v) a b o v e prints a message w h e n all a t t e m p t s have failed. We could phrase this heuristic as a m e t a - r u l e whose LHS would check that the parser has blocked. This m e t a - r u l e would be o r d e r e d strictly after the ones f o r spelling c o r r e c t i o n and c o n t e x t u a l ellipsis. A state would be p o s t u l a t e d as the locale of the p r o b l e m by the same heuristic as for spelling correction. T h e RHS would print for e a c h arc that leaves that state the c a t e - gory, constituent, or word that was e x p e c t e d b y that arc.

H a y e s and M o u r a d i a n (1980) e m p h a s i z e r e c o v e r y t e c h n i q u e s f o r b l o c k i n g during l e f t - c o r n e r p a r s i n g ( A h o and Ullman 1972). Their strategies are i n v o k e d only if the parser blocks. T w o of t h e m can be r e f o r - m u l a t e d as m e t a - r u l e s as follows. O n e m e t a - r u l e would check in its LHS that the p a r s e r was b l o c k e d and t h a t a special p a r s e r v a r i a b l e (call it BLOCKED-PARSE) was e m p t y . T h e RHS would save the b l o c k e d c o n f i g u r a t i o n in BLOCKED-PARSE, and start parsing as if the current w o r d were the first w o r d of the input. This would enable the s y s t e m to ignore initial strings that could not be u n d e r s t o o d . A useful e x a m p l e of this is r e s t a r t e d inputs, such as " C o p y all print all h e a d e r s of m e s s a g e s " . A second m e t a - r u l e is related. T h e LHS would check w h e t h e r the p a r s e r was b l o c k e d and BLOCKED-PARSE had a c o n f i g u r a t i o n in it. F u r t h e r m o r e , the LHS would c h e c k to see t h a t a n o t h e r parser variable (call it DONE-ONCE) was NIL. If so, the RHS would set DONE-ONCE to T. T h e RHS would t h e n s w a p the c u r r e n t c o n f i g u r a t i o n with BLOCKED-PARSE and w o u l d try r e s u m i n g the p a r s e f r o m the current w o r d with that configuration. This heuristic is designed to ignore i n c o m p r e h e n s i b l e m a t e - rial in the middle of an input. F o r instance, it would enable skipping the p a r e n t h e t i c a l material in " L i s t all messages, assuming there are any, f r o m B r o w n " .

In the area of pragmatics, solutions that could fit within o u r p a r a d i g m h a v e b e e n s u g g e s t e d f o r t w o classes of problems. O n e p r o b l e m is the failure of p r e s u p p o s i t i o n s of an input. In the e n v i r o n m e n t of an intelligent t u t o r f o r c o m p u t e r - a s s i s t e d l a n g u a g e in- struction, a technique suggested in Weischedel et al. (1978) could be f o r m u l a t e d as a m e t a - r u l e as follows. T h e LHS would check w h e t h e r processing was b l o c k e d due to a p r e s u p p o s i t i o n being false. Since that s y s t e m would h a v e a m o r e c o m p l e t e k n o w l e d g e of language t h a n a b e g i n n i n g s t u d e n t of a f o r e i g n language, the s y s t e m could treat the input as absolutely ill-formed. A sophisticated RHS could p a r a p h r a s e the false presupposition for the student and indicate which w o r d or syntactic c o n s t r u c t i o n was used inappropriately. Thus, the tutor could point out m i s t a k e s such as " D a s F r a e u - lein ist S t u d e n t " , indicating t h a t the s t u d e n t should

look up the m e a n i n g of " S t u d e n t " (which applies only to males).

K a p l a n (1978) suggests an alternative heuristic for false extensional p r e s u p p o s i t i o n s in a d a t a base envi- r o n m e n t . O n e can r e f o r m u l a t e it as a m e t a - r u l e w h o s e LHS would check that the query h a d r e q u e s t e d a set as a r e s p o n s e a n d t h a t the set was e m p t y . T h e RHS would c o m p u t e queries c o r r e s p o n d i n g to subsets that the original q u e r y p r e s u p p o s e d w o u l d h a v e a n o n - e m p t y extension. T h e RHS would p a r a p h r a s e the m o s t general such q u e r y with an e m p t y r e s p o n s e set, r e p o r t - ing to the user that the s y s t e m k n e w of no such enti- ties.

5. I m p l e m e n t a t i o n

We i m p l e m e n t e d a g r a m m a t i c a l m e t a - r u l e p r o c e s s o r first for an ATN i n t e r p r e t e r and m o r e r e c e n t l y for an ATN c o m p i l e r ( B u r t o n and B r o w n 1977). O u r experi- m e n t s h a v e used RUS ( B o b r o w 1978), a b r o a d - c o v e r a g e g r a m m a r of English with calls to a s e m a n t i c c o m p o n e n t to b l o c k a n o m a l o u s i n t e r p r e t a t i o n s p r o - p o s e d by the g r a m m a r .

Design a n d i m p l e m e n t a t i o n of a m e t a - r u l e p r o c - essor for violation of s e m a n t i c c o n s t r a i n t s is currently u n d e r w a y in t w o different s e m a n t i c interpreters. In one, case constraints are e x p r e s s e d as sets of logical f o r m u l a s ; in the other, KL-ONE is used to e n c o d e case f r a m e s ( S o n d h e i m e r et al. 1984).

F o u r design issues are c o n s i d e r e d in the following sections.

5 . 1 . A p p l y i n g m e t a - r u l e s

T h e set of m e t a - r u l e s dealing with the g r a m m a r or s e m a n t i c s y s t e m could be viewed f o r m a l l y as a func- tion f f r o m a c o m p o n e n t ' s rules S to a n e w c o m p o n e n t ' s rules S ' .

f(S) = S'

S t is the transitive closure of applying e v e r y m e t a - r u l e pertaining to the s y s t e m rules in e v e r y possible way. (Since it is the transitive closure, S is c o n t a i n e d in S ' ) . T h e r e are three alternatives. O n e is to c o m p u t e S' and use it, r a t h e r t h a n S, as the basis of processing, assuming that the transitive closure S' is a finite closure. T h e second is to apply m e t a - r u l e s only as n e e d - ed, thus m a k i n g S ~ a virtual system. T h e third a l t e r n a - tive is a c o m b i n a t i o n of applying s o m e m e t a - r u l e s as n e e d e d and applying others in advance.

The first alternative is superficially similar to ap- p r o a c h 2 of Section 2.1, where i l l - f o r m e d n e s s p r o c e s s - ing is e m b e d d e d in the n o r m a t i v e system; h o w e v e r , S' will m a i n t a i n the p r e f e r e n c e f o r n o r m a l i n t e r p r e t a t i o n s o v e r ill-formed ones. We h a v e r e j e c t e d this alternative b e c a u s e of the c o m b i n a t o r i a l g r o w t h of rules n e e d e d f o r S ' . F o r instance, o n e c a n write m e t a - r u l e s f o r handling relaxation of w o r d categories and relaxation of p r e d i c a t e s on ATN arcs. Since b o t h c a n o c c u r

(13)

Ralph M. Weischedel and Norman K. Sondheimer Meta-rules as a Basis for Processing Ill-Formed Input

t h r o u g h o u t the grammar, they should not be expanded ahead of time. A similar argument is used to justify treating conjunction processing as a separate process rather than building it directly into the g r a m m a r (Woods 1973). Since the classes of ill-formedness can occur in combination, the n u m b e r of relaxed rules in S v can be very large. Furthermore, since utterances where many, m a n y c o m b i n a t i o n s of errors o c c u r should be rare, computing the transitive closure seems uncalled for.

The second alternative, generating a relaxed rule each time it is needed, is the one we implemented first in the context of an ATN interpreter. This alternative provides a kind of virtual system and avoids the in- creased m e m o r y necessary to hold S w.

The third alternative, applying some rules ahead of time and using others only as needed, offers the great- est flexibility and a variety of alternatives. We have implemented a version in which the underlying parser is the output of an ATN compiler. W h e n the meta-rule processor applies a meta-rule at a given arc, the relaxed version of the arc is compiled and saved. 5 If the meta-rule is to be tried by the meta-rule processor at that arc again, the form of the relaxed arc need not be re-computed; it can simply be executed.

This third alternative also offers the potential of adapting the system to the idiosyncrasies of an individual's language and also the potential of extend- ing its own model of language. Obviously, this is an area for future research.

Alternatives two and three assume only that the p r o c e s s o r applying well-formedness rules is able to store a " c o n f i g u r a t i o n " in a queue or agenda. No assumption about the type of processing (for example, b o t t o m - u p or t o p - d o w n ) , nor the class of violated rule (for example, lexical, syntactic, semantic, or pragmatic) is necessary.

5 . 2 . W h a t t o s t o r e

W h e n a c o n f i g u r a t i o n blocks because of the well- formedness rules, should the blocked configuration be stored or the results of applying each relevant meta- rule? Both of the implementations in the ATN environment save only the blocked configuration, namely, a b l o c k e d arc at the end of a path. The n u m b e r of blocked configurations can be large. At present, there is insufficient evidence to determine whether a well- tuned set of meta-rules will yield a substantially larger (or smaller) n u m b e r of relaxed c o n f i g u r a t i o n s c o m - pared to the set of blocked configurations.

Some types of problems, for example, subject-verb agreement, m a y be so c o m m o n , and some types of

5 The current implementation is limited somewhat; it saves the relaxed arc only if the RHS of the meta-rule modifies only the arc itself. Our misspelling meta-rule, for example, does not modify the arc at all, but rather the input string.

relaxation, for example, an unrecognized word, m a y be so diagnostically clear that the c o r r e s p o n d i n g m e t a - rules should be applied immediately. In the case of subject-verb agreement, hand-compiling the meta-rule into the g r a m m a r may be appropriate (that is, writing an arc whose test is that subject-verb agreement failed and whose action places the new configuration on a queue that is tried only after all normal configurations have failed).

5 . 3 . L o c a l i z i n g t h e p r o b l e m

W h e n processing ill-formed inputs, some means of ranking alternatives is appropriate, since the system must determine what is intended in the face of violated constraints and possible error. Also, the n u m b e r of relaxed configurations may be large, even with a set of well-tuned meta-rules designed to o p e n the search space minimally.6 The ideal solution is that the ranking of alternatives should be based on syntactic, semantic, and pragmatic evidence, in addition to the diagnosis and recovery strategy.

The current i m p l e m e n t a t i o n uses only some of those bases and employs a rather simple ranking. Since b o t h grammatical constraints and selection restrictions are employed while parsing with RUS, both syntactic and semantic evidence is used. Blocked configurations are ordered on the a m o u n t of input proc- essed; there is also a partial order on the meta-rules.

O n e of our students, Amir Razi, is designing an experiment to collect data on the p e r f o r m a n c e of the system. The current system can be run in one of two modes: saving all blocked configurations or using only ones that p r o c e e d e d rightmost in the input. One aspect of the experiment is to determine the f r e q u e n c y with which the interpretations covering the most input in a left-to-right parse block at the true source of the problem. Some preliminary evidence (Weischedel and Black 1980) indicates that this heuristic f r e q u e n t l y does indicate where the problem is, if the normative system is nearly deterministic, for example, because the grammar is a fairly constrained subset of English or because semantic criteria filter out parses that have no meaning in the application domain.

O u r l o n g - t e r m goal is accurate d e t e r m i n a t i o n of both the problem in an ill-formed utterance and what was intended. The current implementation represents the first step toward that by employing both syntactic and semantic evidence. We are investigating the use of pragmatic evidence for that purpose as well. In addition, we wish to explore techniques for examining both the left and right contexts of a blocked interpretation, for instance, by employing b o t t o m - u p processing.

6 However, it is not clear whether the combinatorics alone for typical inputs will be a problem, given the rapid increase in processor power/cost and the prospect of multi-processing.