Natural Language Input for Scene Generation

(1)

N A T U R A L LANGUAGE INPUT FOR SCENE GENERATION M

Giovanni Adorni, Mauro Di Manzo

Istituto di Elettrotecnica, University of Genoa Viale F.Causa 13, 16145 Genoa, Italy

Giacomo Ferrari

Istituto di Linguistica Computazionale, CNR Via della Faggiola, 56100 Pisa, Italy

ABSTRACT

In this paper a system which understands and conceptualizes scenes descriptions in natural language is presented. Specifically, the following components of the system are described: the syntactic analyzer, based on a Procedural Systemic Gram- mar, the semantic analyzer relying on the Conceptu- al Dependency Theory, and the dictionary.

I INTRODUCTION

In this paper a system is presented, which under stands and conceptualizes scenes descriptions in natural language (Italian) and produces simple stat ic images of the scenes. It is part of a larger project that aims at understanding the description of static scenes, reasoning (in case of incom- pleteness or inconsistency) and dialoguing about them, and finally generating and displaying them.

The Input Analyzer (IA) of the system is the most stable end experimented component and it is the topic of this paper. It consists of a Syntactic Analyzer, a Cognitive Data Base (CDB) and a Seman- tic Interpreter.

II SYNTACTIC ANALYZER

The syntactic analysis is performed by means of a Procedural Systemic Grammar (PSG) (McCord,77). The main characteristics of the PSG parser is that the operation flow is highly structured, since different levels of the analysis are associated to the syntactic units of the sentence. Five processes can be activated (CLAUSE, COMPL.GR, NOUN.GR, ADJ.GR and VERB.GR) devoted to recognize respectively:

(i) the sentences, (ii) the propositional phrases, comparatives, quantification and noun phrases, (iii) the components of the noun phrases, (iv) the adjectives and their modifiers, (v) the verb and its modifiers.

Fig.l shows how these processes can interact in our parser: double arrows indicate message passing and

Work

supported

by M.P.I. under grant 27430/81

single

I CLAUSE

[.-

,.1 . . .

. . .

h

! v - - . o .

arrows indicate reading from input. Each

I N°UN'G"~I-'-

I

i ADJ.GR I-

I-

®

Fig.l - Levels of Syntactic Analysis

level is activated by the superior one, as shown in Fig.l, and returns to its caller the results of its computation as a message. A feature network is associated to each process, which is activated together with its corresponding processes.

In a PASCAL-like language the feature network can be defined as follows:

type FEATURE . . . . (.LIST OF FEATURES.)...; LINK=^NODE;

NODE=record

NAME:FEATURE; VALUE:boolean;

FATHER,NEXT_BROTHER:LINK; FIRST_SON,ALTERNAT:LINK; end;

FEATURE NETWORK:array(FEATURE) of LINK; Each NODE represents s feature identified by its NAME; the ALTERNATE pointer allows the connection in a Circular list of mutually exclusive features as in SHRLDU (Winograd,72). Each process gives as output a fragment of the FEATURE NETWORK manipu- lated to describe the input; this is performed by means of a set of functions which test the presence

(2)

of a feature in the FEATURE_NETWORK, add and erase features, as described in McCord ('77). The process is divided into a set of sequential routines,called SLOTs, analyzing the functional components of a Syntactic Unit. In the function:

function FILLER(ARGI:PROCESS,

A R G 2 : S E T O F _ F E A T U R E S ) : b o o l e a n ; ARGI activates the appropriate process to fill the caller slot; the second argument of the function selects the set of features to which the called process must be inizialized.

This last f e a t u r e s - p a s s i n g mechanism is absent in the original PSG; from our experience, we found it usefull in all the cases in which a choice in a syntactic level is determined by the s y p e r i o r level or by a more larger context.

Thus, for instance, the set of features character- izing a prepositional phrase is determined at the corresponding syntactic level by the preposition and the features of the nominal phrase; but further and not less important selection criteria can be imposed by the verb which is found in the upper level. The output of a simple analysis is shown in Fig.2; it gives an idea of the syntactic representation.

I N P U T : IL M ~ O G A T T O S T A M A N G I A N D O {my c a ~ is e a t i n g ) R E S U L T O F T H E A N A L Y S I S :

T I P O D I C H I A R A T . T V A A T T I V A (dec l a r a ~ i v e a c t i v e } IL M I O G A T T O S O G G E T T O { s u O J e ¢ ~ }

S T A M A N G I A N D O V E R B . G R

F E A T U R E N E T W O R K :

- - C L A U S E - ! - P R O P O S I Z I O N E - ! - P R I N C - ! - D I C H I A R A T I V A - - I - V E R B . G R . . . ! - P E " S - - ! - T E R Z A - -

! I - N U M --- ! -SINGOLARE--

! ! - M O U O - - ! - E S P L I C I T O - ! - I N D I C A T I V O - -

I ! - T E M P O - ! - P R E S E N T E - -

I ! - A C T - - - f - T R A N S I T I V A - - ! - F O R M A - ! - S T A R E - -

! f - G E N E R I C O - -

! - C O M P L . U R . . . ! - P E R S - - ! - T E R Z A - -

! -GEN--- ! -MASCHILE--

- g U M - - - ~ - S Z N G O L A R Z - -

- N O U N . C ~ - ~ - . O N E - -

f - T I P O - I - C O M U N E PSRS--

F I G . 2 - Result of a Sentence Analysis

The choice of PSG is mainly motivated by the possibility of parallel computation. A control structure allowing the parallel computation is:

cobegin ... coend;

It is a single input-output structure, very usefull to handle alternative choices for the same computational level. In the case of mutually exclusive alternatives only one of the "n" processes activated by a cobegin control structure can end suc- cessfully. In the case of not mutually exclusive alternatives, it is still possible to use the cobegin control structure , but it is necessary to define a strategy for the selection of the most suitable alternative when the coend occurs.

An experimental implementation in terms of para~ lel computation has been made on a m u l t i p r o c e s s o r system (Adorni et ai.,'79). Another version of this

parser has been implemented in PASCAL (DiManzo et ai.,'79} and a version in FranzLisp is in progress.

III STRUCTURE OF THE COGNITIVE DATA BASE The organization o f knowledge, in this system, is based on a set of THOUGHTs. A THOUGHT is a frame like structure within which new data are interpret- ed in terms of concepts acquired through previous experience (Minsky,'75), (Schank,Abelson,'77). Every THOUGHT has a TYPE w h i c h determines a set of operations applicable to it. The following predefined types are allowed (Adorni,DiManzo,'83): - DESCRIPTIVE, that defines the complete description of a

physical,abstract,animate

or not,object. - PROTOTYPE, that defines the structural part of a physical object in terms of g e n e r a l i z e d cones (Marr,Nishihara,'78). An example of definition of simple prototype object is given in Fig.3.

- JOINT, that defines the element of c o n n e c t i o n between physical objects, in order to build more complex objects or scenes (Fig.4).

- SPATIALREL, that defines spatial relationships like "on,near,on the left of,..." between objects. All the linguistic relationships like "above,under, behind", and so on, are reduced into quantitative geometrical relationships between the coordinates of some points of the involved objects; this choice is motivated by the possibility of deriving a set of very general inference rules from analytic geom- etry (Adorni et ai.,'82), (Boggess,'79), (Boggess, Waltz,'79). The coordinates of an indefinite point P are given in the form:

COORD K OF P (REFERRED_TO A)=H

where K is a group of possible coordinates, H a set of values for these coordinates and A is the THOUGHT of the object to which the reference system used is connected. Fig.5 shows the THOUGHT for an use of the preposition "on".

A spatialrel type THOUGHT can contain conceptualizations and prototype THOUGHTs; a joint type can contain only its description; a prototype type can contain joint or prototype THOUGHTs or descriptions in terms of g e n e r a l y z e d cones;all these types can be enclosed in a descriptive type which can contain conceptualizations and all the types of THOUGHTs, previously introduced. A descriptive type can include the following fields (Adorni,DiManzo,

'83), (see Fig.6):

- DESCR, contains all the basic not specialized knowledge about the object;

- LEVELS, contains a d e s c r i p t i o n of the shape of the object (in terms of prototype THOUGHTs) divided in different levels of detail hier- archically organized;

- USE, contains the descriptions of the most common activities involving the use of the object, in terms of spatialrel between prototype THOUGHTs;

(3)

F I G . 3 - E x a m p l e o f D e f i n i t i o n o f a S i m p l e P r o t o t y p e

F I G . 4 - D e f i n i t i o n o f a S i m p l e J o i n t i n g E l e m e n t a n d Use o f t h i s E l e m e n t to b u i l d a More C o m p l e x O b j e c t

(4)

- POSITION, gives the most common spatial relations between the described object and other objects in standard scenes, in terms of a spatialrel between prototype THOUGHTs;

- SUPPORT, contains the indication, in terms of descriptive THOUGHTs, of the objects which are supported in standard situations; - COLOR and MADE, describe the possible set of col-

ors and materials, while WEIGHT contains information about the range of possible weights;

- CONTENT, says, in terms of descriptive THOUGHTs, that the normal use of the object is a con- tainer for other objects;

- DYNAMIC, contains the current expectations about the boundaries of the dimensions of the objects; it can be dinamically updated every time a new object of the same class enters the system's CDB.

IV SEMANTIC INTERPRETER

The Semantic Interpreter of the IA interacts with the Syntactic Analyzer and operates on a set of rules in order to build the concepts a sentence was intended to mean. The output of this module is a Conceptual Dependency Network (Schank,'75), in which every nominal is substituted by a complex descriptive THOUGHT instantiated from the CDB.

Let us illustrate the procedure of analysis con- sidering the following sentence (the translation is word by word in order to reproduce the problems of Italian):

(i) "l'uomo dai capelli grigi e' andato a Roma con l'auto di Giuseppe"

(the man with the grey hair has gone to Rome with the car of Joseph)

The procedure of analysis has several steps:

A. Analysis of Words and Simple Phrases

During this step the entities which take part into the c o n c e p t u a l i z a t i o n are identified. In fact an indexed identifier Xi is associated to each object referred to in the sentence (each nominal), which points to one or more conceptualizations, contained in the field "descr" of each nominal in the CDB. The adjectives contained in the noun phrases are also analyzed during this step. Each o f them adds some c o n c e p t u a l i z a t i o n s which contribute to further individuate the nominal. During this step personal pronouns are identified as:

Xi ~=--> ISA(HUMAN)

Temporal and local adverbials are also analyzed in this phase in order to assign to the sentence c o n c e p t u a l i z a t i o n a time and place identification according t o certain rules described in (Adorni et al.,'81).

At the end of this step the sentence (i) is represented as follows:

identifier nominal conceptualization Xl uomo (man) Xl < = ~ I S A ( H U M A N ) X2 capelli (hair) X2<==>ISA(HAIR) X3 Roma (Rome) X 3 ~ = > I S A ( C I T Y ) XS<==>NAME(ROME) X4 auto (car) X4<==>ISA(CAR) X5 Giuseppe (Joseph) X5<==>ISA(HUMAN)

X5~-->NAME(JOSEPH) The sentence (i) can then be read:

(2) "XI da X2 e' andato a X3 con X4 di X5" (XI from X2 is gone to X3 with X4 of X5)

B. Analysis of Modifiers

The simple phrases of a sentence can either fill conceptual cases of a main conceptualization, thus serving as 'picture producer' (PP), or further ind !

ON is spatial~el {AOHB} thought

begin

COORD X,Y OF P REFERReD_TO M = COORD X,Y OP 0 R E F E R R E D T O M and

P P A R T ( A ) a n d P NUM(>®) and O P A R T ( B ) e n d

a n d h e r i n

COORD Z OF P ~ COOR9 Z OF Q and P ~ U ~ T ( 1 )

e n d and

begln

B~=~PROPEL ~-- OBJ(FOPCE(H)) ~-- DIR((FROH{N~L))(TO(A))) end

e n d .

{ e x i s t s , a t l e a s t , ~ p o i n ? P w h i c h i s p a r t O f t h e THOUGhT(A) and a

p o i n t ~ w h i c h i s p a r = o f r.he THOUGHT(B) and f o r a.ny paL." Op p o i n t s P and 0 i s Z ( P ) >. ~ ( Q ) . More, t h e r e i s an t s s e r t i o n a b o u t t h e f a c t t h a t t h e THOUGHT(B) s u p p o r ~ t h e THOUGHT(A)).

FIG.5 - Example of Use of a Spatial Relationship in a Case Like "a man is on a chair"

(5)

v i d u a t e a P P . T h e r e f o r e t h e y c a n b e c l a s s i f i e d a c - c o r d i n g t o w h e t h e r t h e y m o d i f y :

a) the n o m i n a l that p r e c e d e s ( a l s o n o t i m m e d i a t e l y ) ; "i l i b r i di C a r l o "

^

(the b o o k s o f C h a r l e s )

b) the s u b j e c t o r o b j e c t i n d e p e n d e n t l y f r o m t h e i r p o s i t i o n ;

" M a r i a e' a n d a t a a R o m a c o n A n n a " ^

(Mary h a s g o n e to R o m e w i t h Ann) c) the a c t i o n ;

" M a r i a e' a n d a t a a R o m a c o n la m a c c h i n a " ^

(Mary h a s g o n e t o R o m e w i t h the car)

C ~ I R I S d e s c r i p t i v e

~hOU~ht

d e s c r

ISA(rd~.~TU~

) e n d / . / ' /

l e v e l s O f l: B3X_X

2 : CMAIR_I, e n d

use O f 1: HL~(AN B~ING ON CHAZR m

end

S u p p o r t O f I : H U ! I C ~ 8 £ I N C ~ . ~

2 : . . .

end

p o s l ¢ l o n _ o f i : C F ~ I R REAR T A B L E

e n d

c o l o r O f I : LIGHT BROWN

2 : . . .

end

made o f .: ~OCD

2: ... end

w e l g h : 2kg -:- 8kg end

dynamic ~ 3 0 ;

• max: 5Gc~,5~Cm. IC~Jcm; min: 3 5 c = . 3 5 c m . 8 O c m ; end

e n d .

F Z G . 3

C~.~ZR I s pr'otot)"we t h O U g ~ . . . e n d .

)

FIG.5

HUMJL~BEING Is descelp~Ive]

thought ... end. . )

NF.A~ IS $vatlalrel t~ought ... end.

)

F I G . 6 - D e f i n i t i o n o f the D e s c r i p t i v e T H O U G H T o f a C h a i r

T h e t r e a t e m e n t o f the m o d i f i e r s in b) a n d 4) r e - q u i r e s that the s t r u c t u r e o f the s e n t e n c e is e n - t i r e l y k n o w n a n d c a n n o t , in any case, be p e r f o r m e d b e f o r e the v e r b has b e e n a n a l y z e d ( s u b j e c t a n d o b - j e c t are c o n s i d e r e d t y p e c) m o d i f i e r s ) . T h e m o d i - f i e r s in a), o n the c o n t r a r y , h a v e a l o c a l role, l i m i t e d to the P P they a r e to m o d i f y , a n d t h e i r r e l a t i o n to the s e n t e n c e s t r u c t u r e is m a r g i n a l . T h e y are, t h e r e f o r e , i m m e d i a t e l y a s s o c i a t e d to t h e i r c o r r e s p o n d i n g n o m i n a l s . In (2) " d a X2" a n d "di X5" a r e o f this k i n d a n d a r e c o n s e q u e n t l y l i n k e d t o X1 and X4 p r o d u c i n g :

(3) "XI e' a n d a t o a X 3 c o n X 4 " (XI has g o n e to X 3 w i t h X4)

In the " d e s c r " f i e l d o f T H O U G H T s Xl a n d X4 t h e f o l l o w i n g i n f o r m a t i o n is added:

X2 < . . . . P A R T O F ( X 1 ) X 5 <===> O W N E R S H I P ( X 4 )

The e m b o d y i n g o f a m o d i f i e r c r e a t e s c o m p l e x PPs o r C L U S T E R s . E a c h C L U S T E R h a s a s its H E A D a b) or c) m o d i f i e r , a c o n c e p t u a l i n d e x n o d e m o d i f i e d

b y the a c c e s s o r y c o n c e p t s .

In o u r e x a m p l e " l ' u o m o d a i c a p e l l i neri", "a Roma", a n d " c o n l ' a u t o di G i u s e p p e " a r e C L U S T E R s , in w h i c h the h e a d is a l w a y s the l e f t m o s t n o m i n a l .

T h e d e c i s i o n a b o u t the e m b o d y i n g o f a m o d i f i e r into its h e a d is r e l a t e d to the c l a s s i c a l p r o b l e m o f the p l a c e m e n t o f PP's. I n fact, it is n o t a l w a y s t h e c a s e t h a t a p r e p o s i t i o n a l p h r a s e m o d i f i e s a c o n c e p t u a l i n d e x node; i t is o f t e n p o s s i b l e t h a t it h a s to b e e m b o d i e d i n t o a n o t h e r a c c e s s o r y m o d i - fier, as in:

"il l i b r o d e l l ' u o m o dal c a p p o t t o b l u " (the b o o k o f the m a n w i t h the b l u e c o a t ) I f it is d e f i n e d :

m d ---> the c u r r e n t p h r a s e ;

m d - i ---> the i m m e d i a t e l y p r o c e e d i n g p h r a s e ; m d - 2 ---~ the p h r a s e t h a t i m m e d i a Z e l y p r o c e e d s

md-l;

the s o l u t i o n is o b t a i n e d b y r e c u r s i v e l y d e c i d i n g w e t h e r m d is to b e e m b o d i e d i n t o md-i o r md-2. R e - c u r s i o n is f r o m the l o w e r level.

T h i s d e c i s i o n is m a d e b y a s e t o f s t a n d a r d g e n e r a l p r o c e d u r e s a s s o c i a t e d to p r e p o s i t i o n s (di, da, con, p e r . . . . ) a n d a d v e r b s (sopra, sotto, d a v a n t i , d i e - fro, ...). N o n - s t a n d a r d s p e c i a l i z e d a l t e r n a t i v e s a r e a c t i v a t e d by p a r t i c u l a r n o u n s a n d v e r b s in o r - d e r to t r e a t h i d i o s y n c r a s i e s . T h e s e p r o c e d u r e s are w r i t t e n as t h r e e - s t e p s p r o g r a m s , w h i c h a c c o m p l i s h the o p e r a t i o n s of:

1 - L O O K I N G for c o m p a t i b i l i t y o f c e r t a i n f e a t u r e s o f m d , m d - l , a n d md-2. T y p i c a l f e a t u r e s are s u p e r s e t a n d p a r t - o f r e l a t i o n s o f m d ' s . A r u l e m a y s t a t e t h a t "IF m d has a p a r t - o f r e l a t i o n to m d - 2 T H E N m d m a y be e m b o d i e d into md-2". E x a m p l e :

"il l i b r o del b a m b i n o d a l l a c o p e r t i n a r o s s a " / - - - m d - 2 . . . m d - I . . . m d . . . /

(the b o o k o f the c h i l d w i t h r e d cover) 2 - D e c i d i n g w h e t h e r M E R G I N G c a n be p e r f o r m e d . T h i s is m a d e by i m p o s i n g f u r t h e r r e s t r i c t i o n s o f the t y p e d e s c r i b e d above. A l s o the m a i n c o n c e p t u a l - i z a t i o n a n d o t h e r l i n g u i s t i c p e c u l i a r i t i e s a r e t a k e n into a c c o u n t .

3 - A c t u a l LINKING. In o u r e x a m p l e , the s t r u c t u r e : m d <===> P A R T O F ( m d - i OR md-2)

" l ' u o m o dai c a p e l l i g r i g i " / - - m d - I . . . md . . . / H A I R <===> P A R T _ O F ( M A N )

is p r o d u c e d b e c a u s e m d " c a p e l l i " c a n b e p a r t - o f m d "uomo".

S h o u l d it n o t h a v e b e e n the case, the f o l l o w i n g s t r u c t u r e w o u l d h a v e b e e n p r o d u c e d :

(md-I O R m d - 2 ) < .... P O S S ( m d ) " l ' u o m o dal v e s t i t o s c u r o " / - - m d - i . . . m d . . . /

[image:5.612.51.281.95.434.2]

(6)

(the man with the dark dress) MAN c===, POSS(DRESS)

L~4PADA DA TA~DLO is descriptive t h o u ~ t

descr

X.e=COND~ USE ,~-.- OBJ (LAMPADA) and

]I4PLICAT(LA~ADA ON TAVOLO) end

o.. e n d .

(it is an object such that if x Use the lamp in a standard way,then the lamp is on the table)

FIG.7 - THOUGHT of the Table Lamp

C. Construction of the Main C o n c e p t u a l i z a t i o n The nucleus of a main conceptualization can be associated in the CDB both to a noun indicating an action, state or change of state and a verb. In our example, we find the THOUGHT of fig. 8. A time identification always related to the present (T@), is taken from the syntactic analysis and connected to this conceptualization, thus resulting into:

X ~===~ PTR~NS .... OBJ(X) .... DIR((FROM(Z))(TO(Z))) A and

INSTR(CONC) and T1 ~ T@

If a lexical ambiguity arises, the features assumed by the nominals in the previous steps will help to desambiguate.

A~ARE is descriptive |

thou~t ,. descr

X~.:==~PTR~--- OBJ(X)~--- DIR((FR(~(Y))(TO(Z)) and

D~TR(C~C) end end.

FIG.8 - THOUGHT of the verb "andare" (to go).

At this step "splitting" of a c o n c e p t u a l i z a t i o n often occurs. In the sentence:

"Giovanni d~ un colpo a Maria" (lit. John gives a blow to Mary)

although two nuclei are present (d~ & colpo),never- theless the correct interpretation is "Giovanni colpisce Maria" (John hits Mary), instead of "Gio- vanni trasferisce il possesso dell'oggetto colpo a Maria" (John tansfers the ownership of the object

'blow' to Mary)!!!

We have observed that this phenomenon involves conceptualizations based on the primitives of "state", "action", and "spatial relationship" and relies only on the pairs ACTION-STATE, ACTION-SPATIAL RE-

LATIONSHIP, and ACTION-ACTION. The regularities ruling the formation of these pairs have been found to depend only upon those conceptual primitives. This keeps the number of rules to be evaluated rea- sonably small, if compared with the number of CDB entries ( ~ 6 0 0 entries in the present implementation (Adorni et al.,'81))~

An example will illustrate the m e c h a n i s m of reduc- tion of the conceptual "splitting" as well as of disambiguation.

The pair ACTION-SPATIAL R E L A T I O N S H I P may be represented by:

"tirare su il braccio"

^ A

ACTION SPATIAL R E L A T I O N S H I P ( l i f t the arm)

The compound "tirare su" has the two meanings: - innalzare, a l z a r e , . . . ( l i f t , r a i s e . . . . );

- confortare, dare sollievo psiehico,...(encourage, console .... );

which can be c o n c e p t u a l i z e d respectively:

X ... PTRANS .... OBJ(Y) .... DIR ( (FROM (K))(TO(H) ) ) and

((COORD Z OF H - COORD Z OF K) and

R(X ... PROPEL .... OBJ(Y) .... DIR((FROM(NIL)) ( T O ( N I L ) ) ) ) ) X ~ == ~ DO

S(Y(CHANGE STATE((FROM(HAPPINESS(N)}) ( T O ( H A P P I N E S S ( N ) ) ) ) ) ) The context helps disambiguation.

In our example, the object of the spatial relationship being a physical object, the first alternative is selected. The rule performs a further control, discovering that the physical object is, in this case, PART OF(HUMAN); the PROPEL primitive is then substituted by the MOVE primitive.

D. Case Fillin~ in the Main C o n c e p t u a l i z a t i o n The next step performed by the semantic module is the filling of the conceptual cases of the main conceptualization with the THOUGHTs instantiated during the previous steps. Again, standard rules are associated to prepositions and adverbs and hidiosyncrasies are also treated. These rules make use of messages sent by the syntactic component and look at the conceptual syntax of the main conceptualization. Through these rules the cluster"con X4" turns out to be 'instrumental' and the following conceptualization is then produced:

(4) X1 . . . . USE .... OBJ(X4)

Since the filler of the instrumental case of the main conceptualization h a s to be a conceptualization, the rule activated by the "con" modifier fills the instrumental case with (4).

In (3), 'a X3' is placed in the destination of the directive case of the main conceptualization, be- cause preposition 'a' is stated to indicate the

(7)

'destination' if the main conceptualization contains a PTRANS,PROPEL or MOVE,with empty directive case; otherwise it indicates 'state'. "Andare a Roma" is thus distinguished from "essere a Roma"

(to be in Rome). The result, for our example, is: XI< .... PTRANS~--- OBJ(XI)~--- DIR((FROM(NIL))

(TO(IN X3)) ) The directive case,as shown in the above example is not simply filled with a md; it is filled with a "spatial_relationship-md" pair. This is a general rule for our system, e m p h a s i z i n g the change of coot dinates caused by an action. In our example this means that the primitive PTRANS has moved the object to a point whose coordinates are defined with in the city of Rome. The result of the analysis of (I) is given in Fig.9.

X6 Is de$crlptLve

t h o u g h t

XI ~ - - - * PT.RAN$ ,~--*OB3 ( X I ) ,,~--D IR ( (FRON(NI L) ) (TO( I;[ 13 ) )

T_T l.~_-- ( T I< T

/ I N S T R I X I ¢ . - ~ U $ . ~ - - or-JlX 1)

/

,o0//

/

• X4 is de~c,~iptive X3 is descPiptlve

thOUght thOUgh:

d e s o r d e S c l -

ISA(CAR) ISA(CITY)

end N A ~ ( RCI,IE )

•.. end

en~ . . . e n d .

X 2 iS descriptive X5 is d e s c r i p t i v e

t hOU~ : I thought

descr 1 ~.escr

: SA ( M-a- !R " ~ I -~A ( h--7~'l )

PAINT GFfXI) IPOSS(X4)

e n d end

c~;or of

end

end.

r

i

i ..,

end.

FIG.9 - Result of the Analysis of the Sentence (i)

E. Conceptual Analysis of Complex Sentences

The process of semantic interpretation is ap- plied to every clause in the sentence, identified by a verb or a noun indicating an action. Seg- mentation into such clauses or nominalized clauses is obviously performed by the syntactic component, which has also non-standard rules for specific classes of (modal) verbs like: dovere (must),volere (to want),potere (can),incominciare (to start) .... These verbs constitute a single main conceptualization together with the embedded infinitive. Simple composition rules have been defined to com- bine the meaning of clauses (sentences).

Thus for conjunction, as in

"si alzo',si mise il cappello e a p r i ' la porta"

(he stood up,put on its hat and opened the door) the main conceptualizations associated to every proposition are connected by an 'and' relationship.

(si alzo') . . . T1 and

(si mise il cappello) T2 > T I and

(apri' la porta) T3 > T 2

A time indication is also associated to every main conceptualization to emphasize the execution order of every action.

Conceptual analysis of each single clause (sentence) is activated by this top level structure and at the end the resulting conceptualizations are linked one to the other.

V CONCLUSIONS

In this paper a system for understanding a natural language input to a scene generator has been described. It makes use of a conceptual dependency semantic model, substantially modified in as much as syntax is kept apart from semantic interpretation and a fully formalized dictionary is used, much more complex than the one embodied in Schank's theory. The dictionary is particularly oriented to the generation of scenes, and the stress is on the representation of the structure of objects. The awareness of the structure of the objects is often intimately related to our capability of under standing the. meaning of spatial relationships and other complex linguistic expressions. For instance, the meaning "the cat is under the car" is clear, even if it may depend on the state of the car, moving or parked; on the contrary, the sentence "the cat is under the wall" is not clear, unless the wall is crashed or it has a very particular shape.Our model tries to account t~is understanding activity by means of the following features:

- an object is described at several levels of de-

tails; in some cases, only a rough definition of the object dimensions can be sufficient, while in other cases a more sophisticated knowledge about the structure of the object itself is re- quired;

- the characteristic features of an object are emphasized; the recognition of a feature allows the activation of particular rules and the generation of hypotheses about the presence of an object;

- the typical relationships among objects are described.

The interaction between syntactic and semantic analyzers seems rather complex, but it provides some valuable solutions to certain crucial points of computational linguistics, like PP's placement, conceptual splitting, idioms and preassembled

(8)

The syntactic analyzer, working top-down, yelds a representation of the input sentence in which information about gender, number, person and tense are recorded and for each function such as subj, obj, time, etc.., the ccrresponding filler is identified, or a list of fillers is given in case of ambiguity. These two kinds of information are exactly what is usefull for semantic interpretation and are picked up in various steps of the interaction by the semantic analyzer in order to build the main conceptualization and to fill its role. Also MARGIE(Schank,'75) makes some use of syntactic knowledge distributed among lexical definitions of words. This solution gives the entire control to the semantic interpreter and no syntactic functional representation is used. It seems,however, that an

intermediate

step, keeping the syntactic output separate from the semantic one, has the

advantage

of avoiding the multiplication of single

pieces of syntactic knowledge. It also provides a simpler way of dealing with syntactic variants of the same sentence and a help in identifying coreferences.

The semantic interpreter works fundamentally bottom-up and, although much is still to be at- tempted, it seems that it can usefully cooperate with a top-down parser to find the correct interpretation. These practical advantages will be taken into account also in the future development of the system. In fact it seems that, although no definite solution has been given to many linguistic problems, the interaction between two fully developped mecha- nisms controlling each other can provide an indication and a frame into which a more compact system can be built.

In the present version of the system the interaction between the two modules is strictly sequential. In a more compact analyzer, syntactic specialists, i.e. simplified pieces of grammar specialized in particular syntactic phenomena, will be called by semantic interpreter according to opportunity. This second version is still being designed.

VI ACKNOWLEDGEMENTS

The autors would like to thank Dr. Lina Massone for her contributions and assistance in the prepa- ration of this paper.

VII REFERENCES

G.Adorni,F.Cavagnaro,M.DelCanto,M.DiManzo,O.Giuffre

and L.Stringa, "Un Analizzatore Sintattico del Linguaggio naturale Italiano per l'Elaboratore Multi-Mini Associativo EMMA", DOC-ERI-050, ELSAG SpA, Genoa, 1979.

G.Adorni,W.Ansaldi,M.DiManzo and L.Stringa,"NAUSICA: NAtural language Understanding System; the

Italian language Case Analyzed", Rivista di Informatica ii, 1981, 39-88.

G.Adorni,A.Boccalatte and M.DiManzo, "Cognitive Models for Computer Vision", Proc. COLING '82, Prague, 1982, 7-12.

G.Adorni and M.DiManzo, "Top-Down Approach to Scene Interpretation", Proc. CIL '83,Barcellona,1983. L.C.Boggess, "Computational Interpretation of

English Spatial Prepositions", Tech.Rep. T-75, Coordinated Laboratory, University of Illinois, Urbana, 1979.

L.C.Boggess and L.Waltz, "Visual Analog Represen- tation for Natural Language Understanding",Proc. IJCAI '79, 1979, 9 2 6 - 9 3 4 .

M.DiManzo,L.Stringa and G.Zano, "Un Approccio proce durale all'Analisi Sintattica dell'Italiano". Rivista di Informatic~ 9,' 1979,: 257-284.

D.Marr and H.K.Nishihara, "Representation and Re- cognition of the Spatial Organization of 3-D Shape", Proc. R.Soc. London, 1978, 289-294. M.C.McCord, "Procedural Systemic Grammars", Int.J.

o £ Man-Machine Studies 9, 1977, 255-286.

M.Mi{sky, "A Framework for Representing Knowledge", in The PsycholoF~y of Computer Vision, ed. P.H.Winston, McGraw-Hill, New York, 1975, 211- 277.

R . C . S c h a n k , Conceptual Information Processing,North Holland, Amsterdam, 1975.

R.C.Schank and R.P.Abelson, Scripts, Plans, Goals, and Understanding, Lawrence Erlbaum, Hillsdale, NY, 1977.

T.Winograd, Understanding Natural Language,Academic Press, 1972.