Co occurrences of Antonymous Adjectives and Their Contexts

20 

Co-occurrences of Antonymous Adjectives and Their Contexts. John S. Justeson* Slava M. Katz* IBM T. J. W a t s o n R e s e a r c h C e n t e r . Charles and Miller propose that lexical associations between antonymous adjectives are formed via their co-occurrences within the same sentence (the c o - o c c u r r e n c e h y p o t h e s i s ) , rather than via their syntactic substitutability (the s u b s t i t u t a b i l i t y h y p o t h e s i s ) , and that such co- occurrences must take place more often than expected by chance. This paper provides empirical support for the co-occurrence hypothesis, in a corpus analysis of all high-frequency adjectives and their antonyms and of a major group of morphologically derived antonyms (e.g., i m - possible, u n - h a p p y ) . We show that very high co-occurrence rates do appear to characterize all antonymous adjective pairs, supporting the precondition for the formation of the association; and we find that the syntactic contexts of these co-occurrences raise the intrinsic associability of antonyms when they do co-occur. We show that via one of these patterns, mutual substitution within otherwise repeated phrases in a sentence, the co-occurrence hypothesis captures the generalizations that were the basis for the substitutability hypothesis for the formation of antonymic associations.. 1. A n t o n y m i c A s s o c i a t i o n . M u c h c u r r e n t r e s e a r c h in linguistics is c o n c e r n e d w i t h t e x t u a l o r d i s c o u r s e b ases fo r linguistic s t r u c t u r e ; w i t h i n lexical s e m a n t i c s , s u c h r e s e a r c h is d i r e c t e d at p a r t i c u l a r lex- ical relations a n d at c o r r e l a t i o n s b e t w e e n s y n t a x a n d s e m a n t i c s . This p a p e r a d d r e s s e s t h e t e x t u a l u n d e r p i n n i n g s of a n t o n y m y b e t w e e n p r e d i c a t i v e adjectives, f o l l o w i n g u p r e s e a r c h r e p o r t e d b y C h a r l e s a n d Miller (1989).. A n t o n y m y is a special lexical a s s o c i a t i o n b e t w e e n w o r d pairs. T h a t it is lexical a n d n o t s i m p l y s e m a n t i c f o l l o w s f r o m t h e fact t h a t d i f f e r e n t w o r d s f o r t h e s a m e c o n c e p t can h a v e d i f f e r e n t a n t o n y m s ; f o r e x a m p l e , big-little a n d large-small are g o o d a n t o n y m pairs, b u t large-little is not. T h e classic w o r k o n a s s o c i a t i o n s a m o n g adjectives, a n d b e t w e e n a n t o n y m o u s adjectives in particular, is b y D e e s e (1964, 1965), a n a l y z i n g t h e res ults of s t i m u l u s - r e s p o n s e w o r d - a s s o c i a t i o n tests. 1 C h a r l e s a n d Miller (1989) a r g u e , c o n t r a r y to m o r e c o m p l e x p s y c h o l i n g u i s t i c theori es, t h a t t h e p r i m a r y s o u r c e o f t h e s e as sociations is a t e n d e n c y t h e y h y p o t h e s i z e for a n t o n y m s to c o - o c c u r w i t h i n t h e s a m e s e n t e n c e s in discourse. This p a p e r s u p p o r t s a n d e x t e n d s t h e i r h y p o t h e s i s . . D e e s e ' s w o r k o n w o r d a s s o c i a t i o n f o r adjectives w a s b a s e d o n a list o f t h e 278 m o s t f r e q u e n t adjectives in t h e T h o r n d i k e - L o r g e (1944) c o u n t , t h o s e h a v i n g a f r e q u e n c y of 50 p e r m i l l i o n or m o r e . T h i r t y - f o u r pairs of these w o r d s h a d a r e c i p r o c a l p r o p e r t y : e a c h m e m b e r of the p a i r w a s the m o s t f r e q u e n t r e s p o n s e to t h e o t h e r o n w o r d - a s s o c i a t i o n . • Natural Language Group, IBM T. J. Watson Research Center, P.O. Box 704, Yorktown Heights, New York 10598. 1 When we use the term association, we use it in the general sense of associative pairing, not for response frequencies in word-association tests.. (~) 1991 Association for Computational Linguistics. Computational Linguistics Volume 17, Number 1. tests. These pairs were all antonymic (e.g., good-bad, big-little, large-small). Essentially reciprocal were 5 additional a n t o n y m pairs overlapping with these 34: easy-hard (cf. hard-soft), heavy-light (cf. light-dark), left-right (cf. right-wrong), long-short (cf. short-tall), and new-old (cf. old-young). For these pairs, m e m b e r A has B as its m o s t frequent response, while B is the second :most frequent response to A (A's m o s t c o m m o n response being its a n t o n y m in the original list of 34 pairs).. A general finding of word-association studies is that the part of speech of the response usually agrees with that of the stimulus. Deese's data are consistent with this, and suggest further that at least the m o s t frequent a n t o n y m o u s adjectives are associated more directly with each other than with other adjectives. Deese (evidently) did not find antonymic responses to be the m o s t frequent for s o m e 200 of the 278 ad- jectives he studied. These w e r e generally of lower frequency than those that w e r e each other's m o s t c o m m o n responses; Deese (1962, 1964:347) s h o w e d that the most c o m m o n responses to lower-frequency adjectives are typically n o u n s that those adjectives often modify, as in administrative decision, cited b y Deese and actually u s e d in the Brown C o r p u s (see Section 2.1). In addition, the w o r d s with reciprocal responses were almost all morphologically simple and etymologically native (i.e., of Old English origin).. In h u m a n contingency j u d g m e n t studies (see Shanks and Dickinson 1987 for a recent review), the formation of associations a m o n g events is f o u n d to d e p e n d u p o n temporally close occurrence of the associated events. Lexical associations, and in par- ticular the w o r d pairs linked as stimulus and response in word-association tests, have thus been accounted for b y positing frequent temporal proximity of lexical access of the w o r d s involved; Charles and Miller contrast t w o such accounts, the substitutabil- ity hypothesis, introduced b y Ervin-Tripp (1961, 1963), and their o w n co-occurrence hypothesis.. Because responses to higher-frequency stimuli in association tests tend to agree in part of speech, and because parts of speech are defined linguistically b y m u t u a l substitutability within a context, psycholinguistic accounts have sought explanations for antonymic associations as side-effects of part-of-speech associations; this is an eco- nomical kind of account, since part-of-speech agreement requires an explanation in a n y event. The k e y idea of the substitutability hypothesis is that the context in which one w o r d occurs in a sentence m a y call to m i n d other w o r d s syntactically substi- tutable in that context, so that substitutable w o r d s are activated mentally in close temporal proximity. U n d e r such a hypothesis, the greater the syntactic and semantic appropriateness of a substitutable w o r d in a context, the greater its mental activation. Part-of-speech agreement often suffices for syntactic appropriateness; and, intuitively, the use of an adjective that characterizes the position of a n o u n ' s referent in one region of a semantic dimension seems sensible mainly w h e n another region cannot be ruled o u t as inconsistent. Accordingly, a n t o n y m o u s adjectives should frequently be called to m i n d in each other's environment.. Charles and Miller (1989) d i s p u t e the substitutability of a n t o n y m o u s adjectives, s h o w i n g that their sentential contexts normally leave only one m e m b e r of the pair plausible. They suggest instead the simplest form of associative explanation, that the forms in question d o in fact tend to co-occur in close temporal contiguity - - in partic- ular, that a n t o n y m o u s adjectives occur more frequently than expected b y chance in the same sentence. As s u p p o r t i n g evidence they provide counts s h o w i n g a higher than expected n u m b e r of sentences containing both m e m b e r s of each of the a n t o n y m pairs big-little, large-small, public-private, and strong-weak; 2 they conclude that it is sentential. 2 They also h y p o t h e s i z e t h a t adjective pairs t h a t are semantically opposite b u t not antonymic, one. Justeson and Katz Co-occurrences of Antonymous Adjectives. co-occurrence alone that is responsible for the associative pairing of a n t o n y m o u s adjec- tives. This p a p e r confirms that, for all adjectives frequent e n o u g h to judge, a n t o n y m o u s adjectives d o co-occur within the same sentence m u c h more often than is expected b y chance. On the other hand, something like the substitutability hypothesis m u s t also be involved: w h e n a n t o n y m s d o co-occur, they usually substitute for one another in clausal or phrasal contexts that are otherwise w o r d - f o r - w o r d repetitions of each other, apart from pronominalization or ellipsis in some cases. We propose that, while train- ing for a n t o n y m association takes place via co-occurrence, it is their substitution in repeated contexts that makes this training particularly effective.. 2. Co-occurrences among Antonyms. This section s h o w s that adjectives d o indeed tend to occur in the same sentence as their antonyms far more frequently than expected b y chance. Using the 1,000,000- w o r d Brown Corpus, Charles and Miller (1989) tested this hypothesis for four pairs of antonyms: big-little, large-small, strong-weak, and public-private, the first three pairs being a m o n g Deese's list of reciprocally-associated antonyms. We verify their results for these four pairs, and extend the demonstration to encompass all of the Deese antonyms. On a class basis, w e extend it to encompass all pairs of a n t o n y m o u s adjectives for which at least one m e m b e r has an adjectival frequency comparable to the Deese a n t o n y m s and to a major g r o u p of morphologically a n t o n y m o u s adjectives. For all three groups of adjective pairs, a n t o n y m s turn o u t to co-occur sententially roughly 10 times as often as expected b y chance. These results are obtained mainly from a version of the Brown Corpus tagged b y part of speech, and have been checked against an untagged 25,000,000-word corpus of general literature.. 2.1 Test Corpora Our w o r k is based primarily on the Brown Corpus, a database containing 1,000,000 w o r d s of English text balanced across 15 general categories, d i v i d e d into 500 text extracts of a b o u t 2,000 w o r d s each; for a detailed description, see Francis and Ku~era (1982). In the version w e use, all w o r d s are tagged essentially b y part of speech. Thus, although the w o r d back is u s e d not only as an adjective b u t also as a noun, verb, and adverb, w e are able to recover exclusively adjectival uses of the w o r d from the corpus. The tagging system actually e m p l o y s some distinctions finer than simple part of speech, so certain adjectives receive special tags; for example, first and last are tagged as ordinal n u m b e r s rather than as adjectives. On the other hand, s o m e part-of-speech labels are u s e d more b r o a d l y than u n d e r most conventional grammatical descriptions. For example, past participles of verbs are tagged as such in b o t h verbal and adjectival use; thus, married is not marked as an adjective in the corpus although it does occur adjectivally (e.g., a married couple). For simplicity, consistency, and replicability, w e consider only those adjectives m a r k e d with the standard adjective tag in the corpus.. To test h y p o t h e s e s concerning sentential co-occurrence of words, the corpus m u s t be divided into sentences. The Brown Corpus is divided b y a special sentence-ending tag into a sequence of "pseudo-sentences." We eliminate those pseudo-sentences that are labeled as "headlines," since they rarely consist of full sentences. Quantitatively. m e m b e r being a s y n o n y m of an a n t o n y m of the other, are not directly associated, and suggest that these "indirect antonyms" have no special tendency to co-occur sententially. They confirm this suggestion for four pairs they take to be good indirect antonyms: powerful-weak, strong-faint, big-small, large-little. In none of these cases do they find a substantial excess of sentential co-occurrences over what is expected by chance. For further discussion see Gross, Fischer a n d Miller (1989).. Computational Linguistics Volume 17, Number 1. this h a s little effect; t h e i r i n c l u s i o n w o u l d a d d less t h a n 1% to t h e 54,717 s e n t e n c e s t h a t r e m a i n . In a s a m p l e c h e c k e d f o r i n c o r r e c t s e n t e n c e d i v i s i o n s , w e f o u n d t h a t all w e r e cases of p r e m a t u r e d i v i s i o n , n o n e of s e n t e n c e joins; t h e s e e r r o r s are m a i n l y list e l e m e n t s , s e p a r a t e d b y s e m i c o l o n s , t h a t are t r e a t e d as s e n t e n c e s , a n d s u b j e c t - v e r b seg- m e n t s t h a t are s e p a r a t e d f r o m q u o t e d s p e e c h b y e i t h e r e x c l a m a t i o n p o i n t s o r q u e s t i o n m a r k s . . As a r e s u l t of t h e s e e r r o r s , s o m e of t h e rat i o s w e r e p o r t o f o b s e r v e d to e x p e c t e d n u m b e r s of s e n t e n t i a l c o - o c c u r r e n c e s of a n t o n y m s m a y b e s o m e w h a t h i g h e r o r l o w e r t h a n t h e y s h o u l d be. We h a v e m a d e n o a t t e m p t to c o r r e c t f o r a n y i n a c c u r a t e s e n t e n c e d i v i s i o n s , f i n d i n g t h a t a n y increase is too slight to affect o u r r e s u l t s ( a n d a n y d e c r e a s e will bias a g a i n s t the c o - o c c u r r e n c e h y p o t h e s i s ) . If t h e r e p o r t e d rat i o s are h i g h e r t h a n t h e y s h o u l d be, p r e l i m i n a r y a n a l y s i s s h o w s t h a t t h e i n c r e a s e is p r o b a b l y b y n o m o r e t h a n 3%; if t h e y are l o w e r ( w h i c h is less likely in g en eral ), t h e d e c r e a s e is a l w a y s b y m o r e t h a n 3% since t h e largest n u m b e r of c o - o c c u r r e n c e s in o u r s a m p l e is 28. T h e c o - o c c u r r e n c e h y p o t h e s i s calls f o r h i g h o b s e r v e d / e x p e c t e d ratios; a 3% i n c r e a s e in t h e s e ratios d o e s n o t s i g n i f i c a n t l y bias in f a v o r o f t h e h y p o t h e s i s , since t h e rat i o s w e calculate are all m o r e t h a n 2:1 a n d a v e r a g e a r o u n d 10:1.. We h a v e also c h e c k e d o u r m a i n results o n t h e A P H B C o r p u s , a m u c h l a r g e r b u t g r a m m a t i c a l l y u n t a g g e d c o r p u s of 25,000,000 w o r d s , o b t a i n e d f r o m t h e A m e r i c a n P u b - lishing H o u s e for the Blind a n d a r c h i v e d at IBM's W a t s o n R e s e a r c h Center. It consists of stories a n d articles f r o m b o o k s a n d g e n e r a l c i r c u l a t i o n m a g a z i n e s , s u c h as Reader's Digest, Datamation, a n d Fortune. S e n t e n c e s e p a r a t i o n is less reliable t h a n in t h e B r o w n C o r p u s , a n d w o r d s are n o t t a g g e d b y p a r t of s p e e c h . In a d d i t i o n , since t h e c o r p u s is u n t a g g e d , c o m p u t a t i o n s of t h e e x p e c t e d n u m b e r s o f c o - o c c u r r e n c e s o f a n t o n y m i c adjectives are n e c e s s a r i l y i n f l a t e d e x c e p t w h e n n o n a d j e c t i v a l u s e o f b o t h a d j e c t i v e s is rare. We u s e d this c o r p u s p r i m a r i l y to v e r i f y t h e r e s u l t s d e r i v e d f r o m t h e B r o w n C o r p u s , a n d f o r p r o b l e m s r e q u i r i n g a s a m p l e size m u c h l a r g e r t h a n 1,000,000 w o r d s . . 2.2 The D e e s e Antonyms F o r 4 of t h e a n t o n y m p a i r s i d e n t i f i e d b y D e e s e (1964) w e h a v e n o e v i d e n c e b e c a u s e at least o n e m e m b e r of t h e p a i r d o e s n o t o c c u r w i t h t h e s t a n d a r d a d j e c t i v a l t ag in the B r o w n C o r p u s : alone-together, few-many, first-last, a n d married-single. T h e s e h a v e n o effect o n o u r counts. For t h e r e m a i n i n g 35 p a i r s w e h a v e d e t e r m i n e d t h e n u m b e r o f s e n t e n c e s c o n t a i n i n g e a c h m e m b e r of the pair; t h e n u m b e r c o n t a i n i n g b o t h m e m b e r s ; the n u m b e r e x p e c t e d to c o n t a i n b o t h m e m b e r s ; t h e r a t i o o f t h e o b s e r v e d to e x p e c t e d c o - o c c u r r e n c e s ; the r a t e of s e n t e n t i a l c o - o c c u r r e n c e ; a n d t h e p r o b a b i l i t y t h a t as m a n y o r m o r e s e n t e n c e s w o u l d c o n t a i n b o t h m e m b e r s as w e r e f o u n d to. T h e results, g i v e n in Table 1, s h o w a n o v e r w h e l m i n g excess of o b s e r v e d o v e r e x p e c t e d n u m b e r s o f co- o c c u r r e n c e s . T h e f o l l o w i n g s e n t e n c e s are t y p i c a l o f t h e a n t o n y m c o - o c c u r r e n c e s : . The group sets the styles in clothing, the kind of play engaged in, and the ideals of right and wrong behavior.. Soil redeposition is evaluated by washing clean swatches with the dirty ones.. Originals are not necessarily good and adaptations are not necessarily bad.. Overall, a n t o n y m c o - o c c u r r e n c e takes p l a c e in m o r e t h a n 8.6 t i m e s as m a n y sen- tences as e x p e c t e d , 3 a n d c o - o c c u r r e n c e s are f o u n d f o r 30 o u t of t h e 35 p ai rs, in sp i t e. 3 This estimate is probably too low. For one pair (old-new) the expected number of sentences (10.4) amounts to 39% of the total number (27.0) expected, and it happens to have the lowest ratio of. J u s t e s o n a n d K a t z C o - o c c u r r e n c e s o f A n t o n y m o u s A d j e c t i v e s . Table 1 D e e s e ' s adjective pairs and their sentential co-occurrences in the tagged B r o w n Corpus. Sentential occurrences o f a n a d j e c t i v e is t h e n u m b e r o f s e n t e n c e s i n w h i c h t h e a d j e c t i v e o c c u r s i n t h e c o r p u s ; t h i s n u m b e r is g i v e n i n c o l u m n s 1 a n d 3, o n c e f o r e a c h m e m b e r o f a n a n t o n y m p a i r . Observed is t h e n u m b e r o f s e n t e n c e s i n w h i c h b o t h a d j e c t i v e s o c c u r ; expected is t h e n u m b e r e x p e c t e d t o h a v e b o t h a d j e c t i v e s b y c h a n c e ; ratio is t h e r a t i o o f o b s e r v e d t o e x p e c t e d c o - o c c u r r e n c e s ; rate 1 / n i n d i c a t e s t h a t o n e s e n t e n c e o u t o f n t h a t h a v e t h e l e s s f r e q u e n t a d j e c t i v e p r o d u c e s a c o - o c c u r r e n c e w i t h i t s a n t o n y m ; a n d probability is t h e p r o b a b i l i t y o f o b s e r v i n g b y c h a n c e a s m a n y o r m o r e c o - o c c u r r e n c e s t h a n a r e a c t u a l l y o b s e r v e d . . s e n t e n t i a l o c c u r r e n c e s o f . i n d i v i d u a l a d j e c t i v e s . s e n t e n t i a l c o - o c c u r r e n c e s . o b s e r v e d e x p e c t e d r a t i o r a t e p r o b a b i l i t y . 85 a c t i v e 11 p a s s i v e 2 0.01709 117.0 1 / 5 . 5 1.30 x 10 -4 55 a l i v e 157 d e a d 2 0.15781 12.7 1 / 2 7 . 5 1.10 × 10 -2 25 b a c k 76 f r o n t 3 0.03472 86.4 1 / 8 . 3 5.79 × 10 -6. 125 b a d 682 g o o d 16 1.55802 10.3 1 / 7 . 8 5.06 x 10 -12 316 b i g 273 l i t t l e 12 1.57662 7.6 1 / 2 2 . 8 8.18 x 10 - s 146 b l a c k 243 w h i t e 22 0.64839 33.9 1 / 6 . 6 2.84 × 10 -27. 3 b o t t o m 69 t o p 0 0.00378 - - - 46 c l e a n 36 d i r t y 1 0.03026 33.0 1 / 3 6 . 0 2.98 × 10 -2. 136 c o l d 119 h o t 7 0.29578 23.7 1 / 1 7 . 0 2.23 x 10 - s 147 d a r k 61 l i g h t 5 0.16388 30.5 1 / 1 2 . 2 6.89 × 10 -7. 83 d e e p 14 s h a l l o w 0 0.02124 - - - 52 d r y 45 w e t 2 0.04277 46.8 1 / 2 2 . 5 8.54 x 10 -4. 109 e a s y 150 h a r d 0 0.29881 - - - 63 e m p t y 215 f u l l 1 0.24755 4.0 1 / 6 3 . 0 2.20 x 10 -1 35 f a r 16 n e a r 1 0.01023 97.7 1 / 1 6 . 0 1.02 × 10 -2 30 f a s t 48 s l o w 1 0.02632 38.0 1 / 3 0 . 0 2.60 x 10 -2 89 h a p p y 32 s a d 1 0.05205 19.2 1 / 3 2 . 0 5.08 x 10 -2. 150 h a r d 59 s o f t 3 0.16174 18.5 1 / 1 9 . 7 5 . 8 7 × 10 -4 107 h e a v y 61 l i g h t 1 0.11929 8.4 1 / 6 1 . 0 1.13 x 10 -1 4 0 7 h i g h 137 l o w 20 1.01904 19.6 1 / 6 . 9 3.95 × 10 -2o. 6 i n s i d e 38 o u t s i d e 0 0.00417 - - - 347 l a r g e 504 s m a l l 26 3.19623 8.1 1 / 1 3 . 3 4.33 x 10 -16 122 l e f t 231 r i g h t 28 0.51505 54.4 1 / 4 . 4 1.27 × 10 -4o 508 l o n g 187 s h o r t 12 1.73613 6.9 1 / 1 5 . 6 2 . 2 l × 10 -7. 60 n a r r o w 113 w i d e 2 0.12391 16.1 1 / 3 0 . 0 6.92 × 10 -3 1001 n e w 569 o l d 28 10.40936 2.7 1 / 2 0 . 3 3 . 0 7 × 10 -6. 569 o l d 357 y o u n g 17 3.71243 4.6 1 / 2 1 . 0 2.83 × 10 -7 101 p o o r 69 r i c h 7 0.12736 55.0 1 / 9 . 9 5.80 x 10 - l l . 40 p r e t t y 20 u g l y 0 0.01462 - - - 231 r i g h t 112 w r o n g 8 0.47283 16.9 1 / 1 4 . 0 2.91 x 10 - s . 40 r o u g h 35 s m o o t h 1 0.02559 39.1 1 / 3 5 . 0 2.53 × 10 -2 187 s h o r t 55 t a l l 1 0.18797 5.3 1 / 5 5 . 0 1.72 x 10 -1. 1 s o u r 62 s w e e t 1 0.00113 882.5 1 / 1 . 0 1.13 × 10 -3 189 s t r o n g 29 w e a k 3 0.10017 29.9 1 / 9 . 7 1.39 × 10 -4. 63 t h i c k 90 t h i n 1 0.10362 9.7 1 / 6 3 . 0 9.86 × 10 -2. o f t h e s m a l l s i z e o f t h e c o r p u s . F o r m o s t o f t h e 3 0 a n t o n y m p a i r s e x h i b i t i n g s e n t e n - . t i m c o - o c c u r r e n c e s , t h e n u m b e r s o f c o - o c c u r r e n c e s a r e s t a t i s t i c a l l y s i g n i f i c a n t : 2 5 a r e . observed to expected co-occurrences a m o n g all pairs for w h i c h a n y co-occurrence w a s observed. Excluding this p a i r as a n outlier yields a n overall ratio of a b o u t 11.0 times as m a n y sentences as expected w i t h the co-occurrence.. Computational Linguistics Volume 17, N u m b e r 1. s i g n i f i c a n t a t t h e .05 level; 20 a t t h e .01 level; a n d in 18, t h e p r o b a b i l i t y o f o b t a i n i n g s o m a n y c o - o c c u r r e n c e s is less t h a n 10 - 4 . T h e o c c u r r e n c e o f n o n s i g n i f i c a n t r e s u l t s r i s e s as t h e e x p e c t e d n u m b e r o f c o - o c c u r r e n c e s declines. T h i s s u g g e s t s t h a t t h e n o n s i g n i f - i c a n t r e s u l t s a r e a t t r i b u t a b l e in p a r t to s a m p l e size. I n d e e d , t e s t i n g in o u r l a r g e r b u t u n t a g g e d c o r p u s o f 25,000,000 w o r d s , all 35 p a i r s d o y i e l d h i g h l y s i g n i f i c a n t n u m b e r s o f c o - o c c u r r e n c e s . 4. A n t o n y m c o - o c c u r r e n c e is a m a s s p h e n o m e n o n a f f e c t i n g t h e e n t i r e D e e s e list; it is n o t d u e m a i n l y t o t h e c o n t r i b u t i o n s of a f e w c a s e s w i t h i d i o s y n c r a t i c a l l y h i g h co- o c c u r r e n c e r a t e s as a r e s u l t o f t h e i r h a v i n g s o m e s p e c i a l p r o p e r t y . W e ( o v e r ) c o n t r o l f o r t h e p o s s i b i l i t y o f u n d u e effect o f h i g h c o - o c c u r r e n c e n u m b e r s f r o m p a r t i c u l a r a n t o n y m p a i r s b y c o m p u t i n g t h e p r o b a b i l i t y t h a t 30 o r m o r e o f t h e 35 p a i r s w o u l d c o - o c c u r in a t l e a s t o n e s e n t e n c e ; in this c o m p u t a t i o n , h i g h n u m b e r s o f c o - o c c u r r e n c e s h a v e n o effect. T h i s p r o b a b i l i t y t u r n s o u t t o b e n e g l i g i b l e , 7.1 x 10 -23. A s i m u l a t i o n p r o v i d e d a p i c t u r e o f t h e d i s t r i b u t i o n of t h e n u m b e r o f p a i r s h a v i n g a t l e a s t o n e c o - o c c u r r e n c e b y c h a n c e ; o n l y 9.4 a n t o n y m p a i r s w e r e e x p e c t e d to e x h i b i t a c o - o c c u r r e n c e , a n d i n n o n e o f a m i l l i o n trials d i d e v e n 20 p a i r s d o so.. To g e t a n i d e a o f t h e a c q u i s i t i o n a l c o n s e q u e n c e s o f t h e s e r e s u l t s it is u s e f u l to c o n s i d e r t h e rates o f c o - o c c u r r e n c e b e i n g f o u n d . T h e m a x i m u m n u m b e r o f p o t e n t i a l s e n t e n t i a l c o - o c c u r r e n c e s is t h e m i n i m u m o f t h e n u m b e r o f s e n t e n c e s c o n t a i n i n g e a c h m e m b e r o f t h e a n t o n y m p a i r ; c a l c u l a t i n g t h e r a t e o f c o - o c c u r r e n c e as t h e p r o p o r t i o n o f o b s e r v e d to p o s s i b l e c o - o c c u r r e n c e s , w e f i n d a n o v e r a l l r a t e of o n e c o - o c c u r r e n c e e v e r y 14.7 s e n t e n c e s . A t s u c h r a t e s , t h e l a n g u a g e l e a r n e r is r e p e a t e d l y e x p o s e d to t r a i n i n g f o r t h e a s s o c i a t i o n . . T h e c o - o c c u r r e n c e r a t e s a r e h i g h e r if o n e c o n t r o l s f o r w o r d sense. F o r e x a m p l e , o n l y 24 o f t h e 62 s e n t e n c e s c o n t a i n i n g t h e a d j e c t i v e s w e e t r e f e r t o t a s t e o r s m e l l a n d t h u s r e a d i l y c o n t r a s t w i t h sour; t h e o t h e r s r e f e r t o a t t i t u d e s , p e r s o n a l i t i e s , a n d m u s i c . T h i s is m o s t a p p a r e n t in t h e c a s e o f t h e f i v e a d j e c t i v e s t h a t o c c u r t w i c e in t h e t a b l e , e a c h w i t h t w o d i f f e r e n t a n t o n y m s d e p e n d i n g o n t h e sense. T a k i n g a c c o u n t o f this, t h e l o w r a t e o f I o c c u r r e n c e o f s h o r t p e r 55 s e n t e n c e s c o n t a i n i n g tall rises to I o c c u r r e n c e o f tall p e r 6 s e n t e n c e s c o n t a i n i n g s h o r t , s i n c e o n l y 6 o f t h e 187 s e n t e n c e s c o n t a i n i n g s h o r t u s e it to r e f e r to h e i g h t . Similarly, i n o n l y 310-321 s e n t e n c e s d o e s old r e l a t e to n e w n e s s , as o p p o s e d to y o u t h ; s r e s t r i c t e d to t h e s e s e n t e n c e s , t h e c o - o c c u r r e n c e r a t e i n c r e a s e s to a t l e a s t o n e o c c u r r e n c e o f n e w p e r 11.1-11.5 s e n t e n c e s c o n t a i n i n g a r e l e v a n t s e n s e o f old, a n d t h e o b s e r v e d / e x p e c t e d r a t i o f r o m 2.7 to a m o r e t y p i c a l 4.8-4.9. T h e c o - o c c u r r e n c e r a t e f o r o l d - y o u n g a l s o rises, to o n e p e r 12.2-12.8 s e n t e n c e s , a n d t h e o b s e r v e d / e x p e c t e d r a t i o r i s e s to 12.0-12.6.. 2.3 Other High-Frequency Antonym Pairs I n a d d i t i o n to t h e w o r d s in D e e s e ' s list, w e t e s t e d f o r s e n t e n t i a l c o - o c c u r r e n c e a s e t o f a n t o n y m p a i r s o n e o f w h o s e m e m b e r s h a d a f r e q u e n c y o f a t l e a s t 50 in t h e 1,000,000- w o r d B r o w n C o r p u s b u t t h a t w e r e n o t a m o n g t h e D e e s e a n t o n y m s . T h i s set w a s c o n s t r u c t e d as f o l l o w s . F r o m a list o f all a d j e c t i v e s w i t h a f r e q u e n c y o f a t l e a s t 50, all. 4 The lower rates in the case of low-frequency adjectives cannot produce statistically significant results in a corpus of only 1,000,000 words. Twelve of the 15 pairs that failed to produce statistically significant numbers of co-occurrences in the Brown Corpus were among the 14 (40% of) pairs having the lowest co-occurrence rates in the 25,000,000-word corpus. This is exacerbated by a weaker concentration of 13 of these 15 pairs among the 21 (60% of) pairs having the lowest minimum adjective frequencies.. 5 The range is due to ambiguity in assignment of certain cases of old, as in He came to a stretch of old orange groves, the trees dead, some of them uprooted, and then there was an outlying shopping area, and tract houses.. Justeson and Katz Co-occurrences of Antonymous Adjectives. adjectives were removed that did not seem to have a n y clear antonym, or that were comparative or superlative. For the remainder, one or more a n t o n y m s were placed on a candidate list. This list was given to a professional lexicographer, who confirmed most of the judgments and revised others. The original list, expanded by the additions but not reduced by the deletions m a d e by the lexicographer, was then reviewed by a linguist specializing in lexical semantics. Finally, the list as expanded by her judgments was similarly critiqued by an elementary school teacher.. The set of 35 adjective pairs judged to be good lexical antonyms by all three reviewers we are confident in labeling as antonymic; analysis was restricted to these, at the expense of possibly excluding a h a n d f u l of plausible candidate pairs. 6 Eleven of these a n t o n y m pairs 7 are m o r p h o l o g i c a l a n t o n y m s , that is, one member is derived from the other by a prefix of negation, i n - (also in the forms il-, i m - , ir-) or u n - . For a word pair whose a n t o n y m y is morphologically marked, acquisition of the a n t o n y m y relation for that pair does not require an associative hypothesis involving those particular words. We therefore treat morphological antonyms separately (Section 2.4), and exclude them from the analysis of high-frequency, morphologically arbitrary a n t o n y m pairings. This leaves a set of 24 new a n t o n y m pairs to test, at least one of whose members occurs adjectivally in no fewer than 50 sentences in the Brown Corpus (see Table 2). Two of the 24 pairs on the list have no effect on the results. Because past participles in adjectival use are not labeled in the corpus as adjectives, we lack data for the pairs o p e n - c l o s e d and theoretical-applied. For the 22 pairs for which the corpus provides data, 14 co-occur sententially. The overall co-occurrence rate (once per 13.3 sentences) is comparable to that (14.7) for the Deese adjectives; there is no significant difference as judged by the X 2 statistic. Again, word-by-word statistics confirm about half of the pairings to be statistically significant. Again, nonsignificant results are for pairs with very low expected co-occurrences, and their sentential co-occurrence rate is similar to that for the significant (but more frequent) pairs. Finally, the result is again not due to any concentration of the observed cases in a few vocabulary items: for most pairs the number of co-occurrences is quite small, and the probability that as m a n y or more than 14 of the pairs w o u l d exhibit at least one co-occurrence is negligible, 2.1 x 10 -1°. Simulation shows that the number of pairs expected to show a co-occurrence was just 2.9.. A high rate of co-occurrence for antonyms is therefore validated not only for those adjectives that Deese showed to have the strongest reciprocal associative structure, but also for antonyms involving frequent adjectives in general.. 2.4 Morphological Antonyms This section addresses morphological antonyms, adjective pairs in which one member is derived from the other by an affix of negation. In particular, we treat those morpho- logical antonyms in which the derivation is by any of the most productive prefixes of negation (a-, ab-, an-; dis-; il-, i m - , in-, ir-; u n - ; and n o n - ) . 8 Morphological antonyms require separate treatment because their lexical a n t o n y m y is recoverable morpholog- ically, hence their lexical association can be acquired differently from that between nonmorphological antonyms. We show here that morphological a n t o n y m s show the same tendency for greater than chance numbers of co-occurrences that character-. 6 In our judgment, the best candidates among those not accepted by all three of our judges were conservative-liberal, open-shut, primary-secondary, religious-secular, safe-dangerous, and typical-atypicaL. 7 Able-unable; complete-incomplete; common-uncommon; fair-unfair; legal-illegal; personal-impersonal; practical-impractical; responsible-irresponsible; safe-unsafe; useful-useless; and usual-unusual.. 8 We exclude morphological affix pairs that are themselves antonymous, e.g., -ful and -less, pro- and anti-.. C o m p u t a t i o n a l Linguistics Volume 17, N u m b e r 1. Table 2 Other high-frequency adjective pairs and their sentential co-occurrences in the tagged B r o w n Corpus. Sentential occurrences of a n adjective is the n u m b e r of sentences in which the adjective occurs in the corpus; this n u m b e r is g i v e n in c o l u m n s 1 a n d 3, once for each m e m b e r of a n a n t o n y m pair. Observed is the n u m b e r of sentences in which b o t h adjectives occur; expected is the n u m b e r expected to have both adjectives by chance; ratio is the ratio of observed to expected co-occurrences; rate 1/n indicates that one sentence out of n that have the less frequent adjective produces a co-occurrence w i t h its a n t o n y m ; a n d probability is the probability of o b s e r v i n g b y chance as m a n y or more co-occurrences t h a n are actually observed.. sentential occurrences of. i n d i v i d u a l adjectives. sentential co-occurrences. observed expected ratio rate probability. 21 absent 223 present 0 0.08559 - - - 62 ancient 184 m o d e r n 4 0.20849 19.2 1/15.5 5.93 x 10 -5. 122 b e a u t i f u l 20 u g l y 0 0.04459 - - - 82 b r o a d 60 n a r r o w 1 0.08992 11.1 1/60.0 8.61 x 10 -2 55 b u s y 10 idle 0 0.01005 - - - 58 complex 158 simple 5 0.16748 29.9 1/11.6 7.63 × 1 0 - 7 48 cool 64 w a r m 1 0.05614 17.8 1/48.0 5.47 × 1 0 - 2 . 2 crooked 55 straight 0 0.00201 - - - 160 difficult 109 easy 3 0.31873 9.4 1/36.3 4.11 × 1 0 - 3 26 d u l l 71 sharp 0 0.03374 - - -. 240 early 128 late 12 0.56143 21.4 1/10.7 5.83 × 1 0 - 1 3 28 false 220 true 5 0.11258 44.4 1/5.6 9.15 x 10 -s. 287 general 113 specific 4 0.59270 6.7 1/28.3 3.05 × 1 0 - 3 52 i n n e r 28 outer 6 0.02661 225.5 1/4.7 2.03 × 1 0 -13 14 l o u d 59 soft 0 0.01510 - - -. 101 lower 66 u p p e r 9 0.12183 73.9 1/7.3 5.86 × 1 0 -15 217 major 43 m i n o r 6 0.17053 35.2 1/7.2 1.96 x 10 -s. 64 m a x i m u m 36 m i n i m u m 4 0.04211 95.0 1/9.0 9.74 x 10 -8 49 negative 71 positive 11 0.06358 173.0 1/4.5 2.18 x 10 -22. 6 noisy 62 quiet 0 0.00680 - - - 174 p r i v a t e . 264 public 13 0.83952 15.5 1/13.4 3.89 × 1 0 - 1 2 50 sick 13 well 0 0.01188 - - -. i z e s n o n m o r p h o l o g i c a l a n t o n y m s . T y p i c a l s e n t e n c e s c o n t a i n i n g t h e s e c o - o c c u r r e n c e s . a r e . Plato feels that man has two competing aspects, his rational faculty and his irrational.. Choose carefully between c o n t r i b u t o r y or n o n - c o n t r i b u t o r y pension plans.. J u d g m e n t of a n t o n y m y is n o t n e a r l y as s u b j e c t i v e a n i s s u e i n t h e c a s e of m o r p h o - l o g i c a l a n t o n y m s , so w e c o m p o s e d t h e l i s t to b e t e s t e d o u r s e l v e s . W e i d e n t i f i e d 662 a d j e c t i v e s i n t h e B r o w n c o r p u s as b e i n g m a r k e d n e g a t i v e b y o n e of t h e s e p r e f i x e s a n d as h a v i n g a b a s e t h a t a l s o f u n c t i o n s a d j e c t i v a l l y . F o r 27 of t h e s e p a i r s (e.g., famous- infamous), o u r j u d g m e n t w a s t h a t t h e i r m o r p h o l o g i c a l a n t o n y m y d o e s n o t e n t a i l s e m a n - tic o p p o s i t e n e s s ; 9 t h e s e s p u r i o u s p a i r s w e r e e x c l u d e d f r o m c o n s i d e r a t i o n . F o r n e a r l y . 9 Clear cases rejected on these grounds are ionic-anionic; pathetic-apathetic; septic-aseptic; stringent- astringent; trophic-atrophic; graceful-disgraceful; material-immaterial; memorial-immemorial; passive-impassive; pertinent-impertinent; perturbable-imperturbable; different-indifferent; famous-infamous; fluent-influent;. Justeson and Katz Co-occurrences of Antonymous Adjectives. h a l f of t h e 635 r e m a i n i n g a n t o n y m p a i r s w e h a v e n o e v i d e n c e in t h e B r o w n C o r p u s , since the u n n e g a t e d m e m b e r of the p a i r fails to occur. O u r an al y si s t h e r e f o r e c o n c e r n s t h e 346 s e m a n t i c a l l y a n d m o r p h o l o g i c a l l y a n t o n y m o u s ad j ect i v e p ai rs f o r w h i c h e a c h m e m b e r o c c u r s in the c o r p u s . . Overall, adjectives o n this list h a v e v e r y l o w f r e q u e n c i e s ; in fact, 36 (10%) o f t h e m o r p h o l o g i c a l l y p o s i t i v e a n d 154 (45%) of the m o r p h o l o g i c a l l y n e g a t i v e adjectives oc- c u r o n l y o n c e in the c o r p u s . With f r e q u e n c i e s so low, b o t h t h e e x p e c t e d a n d o b s e r v e d s e n t e n t i a l c o - o c c u r r e n c e s of t h e s e a n t o n y m s are too l o w to test t h e c o - o c c u r r e n c e h y - p o t h e s i s o n a w o r d - b y - w o r d basis, so w e test t h e m i n s t e a d o n a class basis. In this re- spect, the excess of o b s e r v e d o v e r e x p e c t e d c o - o c c u r r e n c e s a g a i n t u r n s o u t to b e h i g h l y significant. T h e o v e r a l l r a t i o of o b s e r v e d to e x p e c t e d c o - o c c u r r e n c e s , 34.5 (74/2.1), is m u c h h i g h e r t h a n the e s t i m a t e s of 8 to 11 t i m e s the e x p e c t e d n u m b e r o f c o - o c c u r r e n c e s s u g g e s t e d b y the h i g h e r - f r e q u e n c y sets of a n t o n y m pairs. F u r t h e r m o r e , 48 o f t h e s e a n t o n y m s c o - o c c u r in at least o n e s e n t e n c e in the c o r p u s . C o m p u t a t i o n o f t h e p r o b - ability of o b s e r v i n g c o - o c c u r r e n c e s for at least 48 o f t h e 346 p a i r s is intractable, so w e e s t i m a t e it u s i n g s i m u l a t i o n ; o n l y 2.0 a n t o n y m p ai rs are e x p e c t e d to co-occur, a n d in o n l y o n e of 1,000,000 trials d i d as m a n y as 11 p a i r s co-occur. T h e p r o b a b i l i t y o f 48 or m o r e pairs c o - o c c u r r i n g b y c h a n c e is t h e r e f o r e negligible; since t h e tail o f t h e d i s t r i b u t i o n d r o p s rapidly, t h a t p r o b a b i l i t y is c e r t a i n l y far less t h a n 10 -6 . In s u m m a r y , the c o - o c c u r r e n c e of m o r p h o l o g i c a l l y - e x p r e s s e d a n t o n y m s as a g r o u p is h i g h l y sig- nificant. N o t e t h a t t h e f r e q u e n c i e s are so l o w f o r m o s t ad j ect i v es t h a t t h e r e c a n b e n o t r a i n i n g b y t e x t u a l c o - o c c u r r e n c e f o r m o s t m o r p h o l o g i c a l ad j ect i v e p a i r s d u r i n g a h u m a n lifetime, so t h e class p h e n o m e n o n is a c q u i s i t i o n a l l y p e r t i n e n t at m o s t to t h e m o r p h o l o g i c a l p a t t e r n . . 2.5 C o - o c c u r r e n c e R a t e s T h e analysis a b o v e s h o w s t h a t a n t o n y m s c o - o c c u r s e n t e n t i a l l y far m o r e o f t e n t h a n e x p e c t e d b y chance. It d e m o n s t r a t e s this b o t h o n a p a i r - b y - p a i r basis, fo r ad j ect i v e p a i r s w h o s e m e m b e r s are f r e q u e n t e n o u g h , a n d o n a class basis, fo r e a c h g r o u p o f a n t o n y m s w e i n v e s t i g a t e d . . T h e r e is s u b s t a n t i a l v a r i a t i o n a m o n g a n t o n y m pairs, f o r t h e D e e s e g r o u p a n d for the o t h e r h i g h - f r e q u e n c y adjectives, in the rat es o f s e n t e n t i a l c o - o c c u r r e n c e o f a n t o n y m s ; a m o n g p a i r s for w h i c h fairly stable e s t i m a t e s can b e m a d e (explicitly, t h o s e h a v i n g 10 or m o r e c o - o c c u r r e n c e s ) , c o - o c c u r r e n c e s t ak e p l ace a n a v e r a g e of o n c e e v e r y 12.5 s e n t e n c e s , w i t h a r a n g e f r o m o n c e p e r 4.4 to o n c e p e r 22.8 sentences. N o n e t h e l e s s , b o t h of t h e a n t o n y m g r o u p s i n v o l v i n g h i g h - f r e q u e n c y ad j ect i v es as w e l l as t h e ( m o s t l y l o w - f r e q u e n c y ) m o r p h o l o g i c a l a n t o n y m s h a v e q u i t e similar o v e r a l l c o - o c c u r r e n c e rates: o n c e p e r 14.7 s e n t e n c e s f o r the D e e s e g r o u p , o n c e p e r 13.3 s e n t e n c e s f o r t h e o t h e r h i g h - f r e q u e n c y a n t o n y m s , a n d o n c e p e r 18.2 s e n t e n c e s fo r t h e m o r p h o l o g i c a l a n t o n y m s . T h e s e rates d o n o t differ significantly; c o m p a r i n g all t h r e e y i e l d s a X 2 o f 4.3947 w i t h 2 d e g r e e s of f r e e d o m (p - 11%). 1° F u r t h e r m o r e , t h e r e is s o m e c o n s t a n c y e v e n to t h e . human-inhuman; terminable-interminable; conscionable-unconscionable; founded-unfounded. Perhaps more debateably excluded were measurable-immeasurable; calculable-incalculable; credible-incredible; definable-indefinable; subordinate-insubordinate; respective-irrespective; canny-uncanny; clean-unclean; easy-uneasy.. 10 Separating the morphological adjectives from the others as possibly different in kind, the difference between the Deese group and the other antonym pairs involving high-frequency adjectives yields a X 2 of 0.6959 with one degree of freedom (p ~ 40%); grouping these two against the morphological antonyms yields a X 2 of 3.6663 with one degree of freedom (p ~ 5.6%). This second difference is on the borderline of statistical significance, suggesting that nonmorphological antonym pairs may co-occur at a slightly higher rate than do morphological adjectives.. Computational Linguistics Volume 17, Number 1. variations in co-occurrence rate characterizing specific a n t o n y m pairs: rates in the APHB Corpus are in good agreement with those in the Brown Corpus, after adjusting for the inclusion of nonadjectival instances of some of the w o r d s (e.g., little is u s e d adverbially about twice as often as it is u s e d adjectivally) and for a slightly l o w e r overall co-occurrence rate in the larger corpus. Co-occurrence rates therefore seem to be a relevant w a y of characterizing the co-occurrence p h e n o m e n o n ; they are also intu- itively clear, being sample conditional probabilities for the occurrence of an adjective given the occurrence of its less frequent antonym.. In contrast, the three groups s h o w substantially different overall ratios of o b s e r v e d to expected co-occurrences: 8.6 for the Deese group, 23.5 for the other high-frequency antonyms, and 34.5 for morphological antonyms. These differences, given the essen- tially constant co-occurrence rates, are d u e to differences in the overall frequencies of adjectives in the three groups. Let N be the total n u m b e r of sentences in the cor- pus, nl,i the n u m b e r containing the more frequent adjective in pair i, n2, i the n u m b e r containing the less frequent adjective in that pair; and let fm,i = n ~ , J N be the cor- responding relative frequencies, w h e r e m = 1 or 2. Then for a n y given a n t o n y m pair i, the rate r~ and the ratio p~ are related as r~ = f~#p~. Using this relation, simple algebraic manipulation s h o w s that: the overall rate ~ and the overall ratio ~ are re- lated as ~ = f l P , w h e r e fl = Y~i f l # ( d 2 # / ~ j h , j ) (the average of the higher a n t o n y m frequencies w e i g h t e d b y the l o w e r a n t o n y m frequencies). A n t o n y m sets w i t h system- atically higher fl,i distributions (which are positively correlated with the f2,i distri- butions) therefore have lower o b s e r v e d / e x p e c t e d ratios. This again suggests that the relatively constant co-occurrence rates are a more appropriate w a y to characterize the co-occurrence p h e n o m e n o n . . 3. Syntactic Contexts o f Co-occurrences. Charles and Miller (1989) d e m o n s t r a t e d that b o t h the sentential and the n o u n phrase contexts of an adjective differ from those of its a n t o n y m in sentences in w h i c h only one or the other appears; in such sentences, antonymic adjectives are not readily sub- stitutable for one another. However, Charles and Miller did not address sentences in which a n t o n y m s d o co-occur. This section examines these sentences for the a n t o n y m s discussed in Section 2. We find that, in sentences containing b o t h m e m b e r s of an a n t o n y m pair, the antonymic adjectives are usually syntactically paired, and in these cases they are c o m m o n l y found in conjoined phrases that are identical or nearly iden- tical, w o r d for word, except for the substitution of one a n t o n y m for the other:. That was one more reason she didn't look forward to Cathy's visit, s h o r t or long;. Under normal circumstances, he had a certain bright-eyed all-American-boy charm, with great appeal for y o u n g ladies, old ladies, and dogs.. There was good fortune and there was bad and Philip Spencer, in handcuffs and ankle irons, knew it to be a truth.. The Brown C o r p u s contains 229 sentences w i t h co-occurrences of b o t h m e m b e r s of at least one of the Deese a n t o n y m pairs. Six of these sentences have m e m b e r s of t w o pairs, as in. Pre-decoration, low-cost molds, and the freedom to form large and small, t h i c k and thin materials make plastics tailor-made for the industry.. 10. Justeson a n d Katz Co-occurrences of A n t o n y m o u s Adjectives. a n d t w o h a v e t w o i n s t a n c e s o f t h e s a m e pair. T h i s y i e l d s 237 s e n t e n c e / c o - o c c u r r e n c e t o k e n s t o c l a s s i f y s y n t a c t i c a l l y . A c l a s s i f i c a t i o n o f a n t o n y m c o - o c c u r r e n c e s a c c o r d i n g t o s u r f a c e s y n t a c t i c s i m i l a r i t y is g i v e n i n Table 3. T h e s e n t e n c e s i n w h i c h a n t o n y m s c o - o c c u r h a v e d i v e r s e s t r u c t u r e s , a s w e l l a s d i v e r s e s t r u c t u r a l p o s i t i o n s f o r t h e a n t o n y m s . I n s p i t e o f t h e d i v e r s i t y , w e f i n d a s t r o n g t r e n d f o r t h e a n t o n y m s t o o c c u r i n s y n t a c - t i c a l l y p a r a l l e l a n d u s u a l l y l e x i c a l l y i d e n t i c a l s t r u c t u r e s . E i g h t e e n c o - o c c u r r e n c e s w e r a t e as " a c c i d e n t a l , " 219 a s i n v o l v i n g d i r e c t c o n t r a s t . 11 E x c l u d i n g t h e a c c i d e n t a l c a s e s , 63% ( 1 3 9 / 2 1 9 ) o f a n t o n y m c o - o c c u r r e n c e s a r e i n l e x i c a l l y i d e n t i c a l s t r u c t u r e s . I n 42% ( 5 8 / 1 3 9 ) o f t h e s e c o - o c c u r r e n c e s , t h e a n t o n y m s t h e m s e l v e s a r e s i m p l y c o n j o i n e d : 12. S h e f e l t c o l d a n d h o t , s t i c k y a n d c h i l l y at the s a m e time.. 39% ( 5 4 / 1 3 9 ) o f t h e m o c c u r i n r e p e a t e d n o u n o r p r e p o s i t i o n a l p h r a s e s , w o r d - f o r - w o r d i d e n t i c a l a p a r t f r o m s u b s t i t u t i o n o f t h e a n t o n y m s f o r e a c h o t h e r a l o n g w i t h o p t i o n a l d e l e t i o n o r p r o n o m i n a l i z a t i o n o f s o m e r e p e a t e d w o r d s : . . . . o n e o f l o w a n i o n i c b i n d i n g capacity a n d o n e o f h i g h a n i o n i c b i n d i n g capacity.. Table 3 Syntactic contexts o f a n t o n y m co-occurrences i n t h e t a g g e d B r o w n Corpus: N o n p a r e n t h e s i z e d n u m b e r s are for sentential co-occurrences; parenthesized n u m b e r s are for those that occur in immediately conjoined phrases of the t y p e specified, and in conjoined larger structures containing such phrases, respectively.. syntactic sentential co-occurrences. context Deese other frequent m o r p h o l o g i c a l r a n d o m . adjective conjunction adjective 58(58 + 0) 22(22 ÷ 0) 30(30 + 0) 12(12 + 0) identical n o u n phrases 49(16 + 17) 14(3 + 6) 7(3 + 2) 1(1 + 0) identical prepositional phrases 5(3 + 2) 2(2 + 0) 1(1 + 0) 0(0 + 0) identical head n o u n s 18(3 + 10) 8(3 + 0) 8(4 + 1) 1(1 + 0) identical predicates 9(7 + 1) 0(0 + 0) 1(1 + 0) 2(2 + 0) other 80(0 + 47) 36(0 + 26) 27(0 ÷ 13) 217(0 + 81). s u b t o t a l 219(87+77) 82(30+32) 74(39+16) 233(16+81) accidental 18 3 0 -. total 237 85 74 233. 11 "Accidental" is a cover term meant to convey the absence of direct semantic contrast in the uses of the two adjectives. Most of these cases appear simply fortuitous, e.g., The work uses the old eighteenth century tradition of giving the part of a young inexperienced youth to a soprano, and It is possible that especially large anacondas will prove to belong to subspecies limited to a small area. More difficult to classify are five or six examples in which the use of the polar terms does not involve their semantic contrast, though deliberate selection of the adjectives based on their antonymy is plausible or probable, e.g., That cold, empty sky was full of fire and light, and Its high impact strength, even at low temperatures, resists chipping, cracking, and crazing, according to DuPont.. 12 Many antonym pairs overwhelmingly favor a particular order of presentation when they co-occur in the special patterns. For example, all 16 instances of good co-occurring with bad, which all occur in adjective-conjunction-adjective or in conjoined identical short noun phrases, are in the order good, bad; similarly, in all 30 cases of immediately conjoined morphological antonyms in our sample, the unmarked-marked order is exceptionless. In general, the more frequent adjective precedes its less frequent antonym and, correlatively, the unmarked precedes the marked where markedness clearly applies. This is consistent with Fenk-Oszlon's (1989) results concerning order and frequency in frozen phrases consisting of two conjoined words: in 84% of pairs, the higher-frequency member precedes the lower.. 11. Computational Linguistics Volume 17, Number 1. Together, these t w o patterns a m o u n t to 51% (112/219) of all co-occurrences. Fully 164 (75%) of these 219 co-occurrences a p p e a r in conjoined syntactic struc-. tures. The pattern is even more characteristic of the 139 identical repeated phrases, 117 (84%) appearing in conjoined structures. Eighty-seven (63%) of the 139 identical phrases appear in an even m o r e striking pattern, the phrases themselves occurring in immediate conjunction with each other, comprising 40% of the total 219 co-occurrences.. More than half of the remaining 80 co-occurrences are similar to the 139 in lexically identical structures in that they are f o u n d in highly parallel and strongly contrastive phrases. For example, 16 of the co-occurrences could have b e e n g r o u p e d with the identical structures except that t w o contrasting n o u n s also substitute for each other, as in. The old shop adage still holds: a g o o d mechanic is usually a bad boss.. Frequently he must work long hours in the h o t sun or cold rain.. The pain seems short and the pleasure seems long.. This special g r o u p occurs in conjoined structures at the s a m e rate as the lexically identical group. Even the less tightly parallel examples exhibit quite strong contrastive parallelism, e.g.,. For example, a boy may inherit a s m a l l j a w from one ancestor and large teeth from another.. In their search for what turned out to be the right breakfast china but the w r o n g table silver, they opened every cupboard door.... There was nothing specifically w r o n g with Edythe, but there was absolutely nothing right about her either.. We have separately classified the other a n t o n y m s involving high frequency adjectives and the morphological a n t o n y m s according to these categories (see corresponding columns in Table 3). For the high-frequency group, there is no statistically significant difference in the proportions of cases that occur as an a n t o n y m pair joined b y con- junction, lexically identical phrases with a n t o n y m substitution, or conjoined phrases. For morphological antonyms, a n t o n y m pairs joined b y conjunction are almost twice as frequent, accounting for 42% (31/74) of the co-occurrences. Controlling for this difference, the proportion of co-occurrences in otherwise identical phrases and the proportion in conjoined phrases is the same as for the other t w o groups.. Analysis of a sample of r a n d o m adjective co-occurrences s h o w s that the distribu- tion of a n t o n y m o u s adjectives across sentence structures is atypical of adjective pairs generally; see Table 3. To determine this, w e selected 250 sentences at r a n d o m from a m o n g those having at least t w o adjectives, and r a n d o m l y selected t w o adjectives from each of those sentences. Seventeen sentences w e r e r e m o v e d from this list. One w a s in fact an a n t o n y m pair, not pertinent to examination of n o n a n t o n y m i c usage. In 16 of these 250 sentences, the t w o adjectives selected w e r e premodifiers within the same n o u n phrase; usually this structure is semantically incongruous for antonyms, and it w a s not observed a m o n g o u r a n t o n y m co-occurrences, so these 16 were also excluded. This left 233 sentences w i t h r a n d o m adjective co-occurrences to compare with the a n t o m y m i c cases. The result: of the 233 adjective pairs selected from these. 12. Justeson and Katz Co-occurrences of Antonymous Adjectives. 233 sentences, only 7% (16/233) occurred in lexically identical constituents, only 42% (97/233) occur in structures joined b y a conjunction. These percentages are drastically lower than observed for any of the a n t o n y m groups.. In summary, w e have s h o w n that antonyms co-occur sententially mainly b y sub- stituting for one another in otherwise identical or near-identical phrases. Repeated phrases, which are b o u n d to be linked to one another during processing, yield a w o r d - for-word alignment - - a pairing of their repeated or anaphorically related w o r d s that induces a direct pairing of the substituting a n t o n y m s with one another. Most w o r d s co-occurring in a sentence are not directly paired b y any mechanism highlighting their co-occurrence; the direct pairing of antonyms gives them more salience as potential associates, amplifying the effect of co-occurrence in the formation of a lexical associ- ation b e t w e e n the antonyms. Arguably, the mechanism of a n t o n y m pairing via w o r d alignment produces an immediate, short-term association that with repeated training m a y stabilize into a long-term association.. 4. D i s c u s s i o n . 4.1 A Co-occurrence Theory o f A n t o n y m A s s o c i a t i o n Charles and Miller (1989) d e m o n s t r a t e d that antonymic adjectives are generally not substitutable for one another in sentences containing only one m e m b e r of the pair; and they pointed out that a lexical association b e t w e e n antonyms w o u l d result from frequent co-occurrence of antonymic adjectives in the same sentence, according to general association theory. Grammatical class associations that m a y form on the basis of the usual sentential contexts of adjective occurrences evidently d o not account for the formation of the lexical association b e t w e e n antonyms specifically.. We have verified the conjecture that lexical a n t o n y m s co-occur sententially far more than w o u l d be expected b y chance. We have also s h o w n that this is a v e r y general phenomenon: it characterizes all a n t o n y m pairs involving a frequent adjective, w h e t h e r or not they s h o w strongly reciprocal responses in word-association tests. These results s u p p o r t the co-occurrence hypothesis for the formation of the lexical association b e t w e e n nonmorphological antonyms, in that both the existence of the p h e n o m e n o n and its generality are crucial to this acquisitional hypothesis.. Accordingly, sentences with a n t o n y m co-occurrences d o seem to be the crucial ones for understanding the formation of antonymic associations. Based on our analysis of these sentences, w e are able to elaborate the original co-occurrence hypothesis: w e n o w characterize the co-occurrence p h e n o m e n o n not simply in terms of its excess over chance expectation, b u t in terms of regular syntactic patterns that they exhibit; and w e have identified a mechanism for association formation, a n t o n y m alignment via phrasal substitution.. This is the essence of the theory w e propose: co-occurrence takes place via sub- stitution, substitution yields a n t o n y m alignment, and alignment leads to association. It is crucially a co-occurrence theory - - the i m p r o b a b l y high rates of co-occurrence of a n t o n y m s result in the formation of associations b e t w e e n them; furthermore, co- occurrence takes place mainly b y substitution in repeated phrases, and phrasal rep- etition with a substitution of antonyms evidently occurs mainly w h e n these phrases occur v e r y near one another, particularly in the same sentence. Perhaps less crucially, it is also a s u b s t i t u t i o n (not substitutability) theory: phrasal substitution provides a mechanism, a n t o n y m alignment, that yields an explicit pairing of the a n t o n y m s and enhances the efficacy of training on the association b e t w e e n them.. Finally, these investigations have changed our u n d e r s t a n d i n g of w h a t a n t o n y m y is. To the semantic criterion for antonymy, opposition in meaning, w e n o w a d d a lexical. 13. Computational Linguistics Volume 17, Number 1. criterion: i m p r o b a b l y frequent substitution in nearby, otherwise essentially identical phrases. Together, the semantic and lexical criteria define antonymy. 13. 4.2 Acquisitional Implications of Textual Co-occurrence Additional results of this research p r o v i d e further s u p p o r t for the co-occurrence hy- pothesis, while suggesting three factors connected with associative pairing as having a p r i m a r y influence on the formation of w o r d associations generally and of a n t o n y m o u s pairing in particular. The suggested factors are the overall rate of occurrence of a w o r d with the stimulus word; the improbability of the extent of pairing u n d e r a hypothesis of chance; and the inherent associability of the w o r d pairs w h e n they d o co-occur, a pattern effect. 14. 1. One implicit a s s u m p t i o n of a co-occurrence hypothesis for acquisition of a lexical association is a high e n o u g h frequency of co-occurrence to p r o v i d e adequate training for the association. We find mostly quite high rates of occurrence of adjectives w i t h their antonyms, averaging a b o u t once per 15 sentences having the less frequent m e m b e r of an a n t o n y m pair; and overall, a b o u t 1 sentence in 150 includes the co-occurrence of an a n t o n y m pair from o u r study.. 2. Such high rates should normally suffice to yield associations, b u t this is not the case w h e n they can occur b y chance. For example, most sentences containing a given adjective also contain the w o r d the, so that the article occurs with that adjective at a higher rate than does a n y genuinely associated w o r d , including an antonym. Conversely, l o w e r rates of co-occurrence m a y be equally effective (provided they achieve s o m e threshold assuring adequate training) if these co-occurrences are substantially more surprising (i.e., improbable u n d e r a hypothesis of chance co-occurrence).. 3. Finally, s o m e contexts in w h i c h w o r d s co-occur highlight the fact of their co-occurrence, increasing their inherent potential for forming associations; such contextual enhancement of associability m a y be required for forming associations b e t w e e n infrequently occurring (or co-occurring) words. We have f o u n d that a n t o n y m s habitually substitute for one another in otherwise essentially identical phrases within the same sentence, and that this p o w e r f u l l y s u p p o r t s their association via a n t o n y m alignment. This is p r o b a b l y the single m o s t important pattern increasing the efficacy of co-occurrence in fostering the d e v e l o p m e n t of a lexical association b e t w e e n antonyms, and p r o b a b l y b e t w e e n m e m b e r s of a n y one grammatical class. Other patterns raise associability as well; adjacency, for example, is especially effective in raising the associability of m e m b e r s of different grammatical classes, such as n o u n s with their modifying adjectives.. 13 J u d g m e n t s o f t h e a n t o n y m y o f specific w o r d p a i r s are s o m e t i m e s subjective a n d u n c e r t a i n ; often, r e s p o n s e s a r e t h a t a w o r d p a i r is s o m e w h a t a n t o n y m o u s o r t h a t t h e w o r d s a r e fairly g o o d o r n o t s o g o o d a s a n t o n y m s . T h e lexical c r i t e r i o n a c c o m m o d a t e s s u c h u n c e r t a i n t y a n d gradability, s i n c e i m p r o b a b i l i t y a n d c o - o c c u r r e n c e rate are g r a d e d factors.. 14 A n u n u s u a l n u m b e r o f t e x t u a l c o - o c c u r r e n c e s , as m e a s u r e d b y t h e i r o b s e r v e d to e x p e c t e d ratio, is c o m i n g to b e w i d e l y u s e d b y c o m p u t a t i o n a l l i n g u i s t s as e v i d e n c e for w o r d associations. F o r e x a m p l e , Wilks et al. (1989) u s e this ratio a s a c r i t e r i o n for e s t a b l i s h i n g links b e t w e e n w o r d s in a s e m a n t i c n e t w o r k ; C h u r c h a n d H a n k s (1989) u s e t h e l o g a r i t h m of this ratio a s a m e a s u r e f o r w o r d association.. 14. Justeson and Katz Co-occurrences of Antonymous Adjectives. Under this formulation of the co-occurrence theory, acquiring the lexical relation of a n t o n y m y requires a certain a m o u n t of training for the association, and as the frequency of adjectives declines, so must the frequency of training for its associations. On the whole, then, very infrequent training should result in weaker associations; more generally, adjective frequency should correlate with the strength of lexical associations. One consequence of this correlation for the formation of a lexical association between semantically opposed adjectives is that, as the frequency of adjectives declines, so does the proportion of those adjectives that have good a n t o n y m s (apart from morphological antonyms, which can be derived by rule with no associative training); we have verified this gradient for adjectives in the Brown Corpus. Another consequence is that semantic (especially synonymic) associations should be stronger relative to the lexical (especially antonymic) associations for lower-frequency adjectives. This m a y account for Deese's (1962:82) finding that the frequency of an adjective stimulus on a word-association test correlates very strongly with the proportion of adjective responses that are antonymic.. 4.3 W o r d - A s s o c i a t i o n Tests a n d Textual C o - o c c u r r e n c e We believe that frequencies of response to a stimulus on word-association tests can be accounted for in large part on the basis of patterns of textual co-occurrence of the associates of the stimulus word. In particular, these frequencies depend not only on the strength of a n t o n y m associations but on those of the entire range of words with which the stimulus has formed associations. We illustrate the p h e n o m e n o n using the a n t o n y m pair good-bad; they co-occur 16 times in the corpus, which is highly improbable as a chance effect (having probability 5 x 10-12), and all 16 co-occurrences are in highly associable syntactic patterns. Finally, the co-occurrence rate as defined in Section 2, which is the conditional probability of encountering good in a sentence containing bad, is quite high (16/125), about once per 8 sentences. All three of the conditions discussed in Section 4.2 are satisfied, promoting the formation of a strong association between these words. No other open-class word has so high a probability of being encountered in a sentence containing bad. In contrast, with five times as m a n y sentences containing good as bad, the conditional probability of encountering bad in a sentence containing good is only one-fifth as high (16/682). Furthermore, several other open-class words occur more frequently than bad in sentences containing good. A m o n g adjectives, none has a significantly greater than chance number of co-occurrences, and none occurs in the high associability pattern of substituting adjectives in conjoined and otherwise identical phrases, so no other adjective competes strongly with bad as an associate of good. A m o n g nouns, however, a few (man, people, time, day) co-occur with good more often than bad does, and do so with greater than chance numbers of co-occurrences. While they cannot substitute phrasally for good in these sentences, as adjectives can, all of them often occur immediately following the adjective; we suppose that this close association facilitates association formation, t h o u g h probably not as dramatically as direct substitution in context. This leaves bad as the single strongest associate for good, but with more competition from other candidates than good had from other associates of bad. In Deese's word-association tests, bad was indeed a m u c h weaker response (13/100) to good than good was to bad (43/100), though each was the other's most common response. This pattern is typical of Deese's antonyms: it is usual, t h o u g h not exceptionless, for the more frequent adjective in an a n t o n y m pair to exhibit the lower response frequency to its antonym; a substantially higher frequency normally entails a larger number of significant textual associates.-. We presume that this competition among a set of unequal lexical associates, and of these associates with semantically but not lexically associated words, can account n o t only for the high proportion of antonymic responses to high-frequency adjectives,. 15. Computational Linguistics Volume 17, Number 1. b u t also for the low proportion of antonymic responses to low-frequency adjectives. In particular, w e have already suggested that nonmorphological antonymic associa- tion m u s t be quite w e a k in the case of v e r y infrequent adjectives, d u e to insufficient training opportunities; and morphologically negative antonyms of low-frequency ad- jectives are almost always m u c h lower in frequency than the base form. A s s u m i n g that such infrequent adjectives typically have quite limited semantic domains of rel- evance, they are apt to s h o w repeated co-occurrence with n o u n s in n o u n phrases, especially in a particular t y p e of corpus or a particular person's sphere of activity and experience. Accordingly, it will usually h a p p e n that a f e w n o u n s premodified b y a given rare adjective have the highest co-occurrence rate and improbability factor of any open-class word, and in corpora w h o s e size reflects the language learner's early experience with language, the a n t o n y m (which can hardly be other than a morpholog- ical antonym) will rarely or never have appeared. In contrast, the highest-frequency adjectives have extensive training opportunities, resulting in a great deal of experience with a n t o n y m co-occurrence, w h o s e associative strength has the s u p p o r t of phrasal substitution; and competition from n o u n s and verbs is m u c h more diffuse than in lower-frequency adjectives d u e to a usually b r o a d range of applicability. Accordingly, the three factors governing associative strength, together with competition a m o n g as- sociates of varying strengths, induce a correlation b e t w e e n the frequency of a stimulus adjective and the proportions of adjective vs. n o u n responses on word-association tests. Deese interpreted this correlation as reflecting a greater effectiveness of paradigmatic (same part of speech) associations at higher frequencies, and of syntagmatic (typical or prototypical accompaniments) at l o w e r frequencies. It n o w appears that the o p p o - sition b e t w e e n paradigmatic and syntagmatic is not directly relevant to the process of association formation. Rather, these are descriptively different effects of competition a m o n g all associated words; apart from possible differences in the efficacy of sub- stitution vs. adjacency in raising the associability of co-occurring words, and in the relative strengths of semantic vs. lexical associations in the paradigmatic and syntag- matic cases, the o u t c o m e of the competition is b a s e d on the same kinds of textual co-occurrence preferences for both paradigmatic and syntagmatic associates.. 4.4 C o - o c c u r r e n c e a n d S u b s t i t u t i o n The factors discussed here, and their relations to word-association response frequen- cies, are quite general and thus likely to a p p l y to other major w o r d classes. In fact, the t w o t y p e s of syntactic patterns characterizing most a n t o n y m co-occurrences o u g h t to p r o d u c e associations of frequent w o r d s with other w o r d s having the same part of speech, because these contexts of w o r d co-occurrence are specific to w o r d s agreeing in part of speech, is (1) Syntactically, conjunctions are explicit pairing devices, and the w o r d s or constructions they join necessarily agree in grammatical class; the conjoining of t w o w o r d s within a phrase thus provides training for a lexical association b e t w e e n w o r d s of a single class. (2) Similarly, w h e n a given environment is literally repeated in a sentence, except that one m e m b e r of a w o r d pair occurs in one of them and an- other occurs in the same slot in the other, this usually guarantees grammatical class agreement b e t w e e n the substituting words; and the repetition with w o r d substitution results in direct pairing of the substituting w o r d s and provides training for a lexical association b e t w e e n them. Both contexts also substantially raise the associability of the co-occurring words, so the training they receive should be effective.. 15 These syntactic contexts m a y also aid in the acquisition of w o r d morphology, t h r o u g h part-of-speech substitutions t h a t exhibit the morphological variations. We h a v e s h o w n t h a t m o r p h o l o g i c a l a n t o n y m y as a relation is a m p l y trained via co-occurrence, t h o u g h m o s t i n d i v i d u a l pairs receive n o training.. 16. Justeson and Katz Co-occurrences of Antonymous Adjectives. We conclude that the acquisition of a n t o n y m y in adjectives takes place by a process that m a y contribute to the acquisition of associations between other words agreeing in part of speech: co-occurrences of words of a single grammatical class in two special types of syntactic context that are characteristic of a n t o n y m co-occurrence. Such a uni- fied account for antonymic and part-of-speech associations was the motivation behind the substitutability hypothesis. The syntactic contexts that typify the co-occurrences of a n t o n y m s introduce a restricted and especially noticeable form of substitutability into the co-occurrence hypothesis - - actual substitution of associated words in a repeated context.. Appendix: Statistical Calculations. We model the co-occurrence problem using the hypergeometric distribution for com- putation of expected values and probabilities of occurrence of as m a n y co-occurrences as observed. Suppose there are N sentences in the corpus, nl of which contain adjec- tive 1 and Tt 2 of which contain adjective 2, and that k sentences contain both adjectives. To model this situation, we suppose that we have selected at r a n d o m the nl sentences to contain adjective 1 and are now selecting the sentences to contain adjective 2. Since we have k co-occurrences, we must select k sentences from among those containing adjective 1, and the remaining n 2 -- ]~ sentences from among those not containing adjective 1. There are. ways of choosing k sentences from among nl, and. N - n 1 ) n2 - k. ways of choosing n2 - k sentences from among N - nl; since the choices of sentences from those containing adjective 1 are independent of the choices from those not con- taining adjective 1, the probability p(k) of choosing at r a n d o m k sentences to contain both adjective 1 and adjective 2 is. ( /¢ ) ( N - n 1 n2 ) ' nl n 2 _ k ) / ( N . These probabilities form the hypergeometric distribution. The expected value of k is nln2/N; this formula is used to compute the expected n u m b e r of co-occurrences in Tables 1 and 2.. The probability P(k) of observing at least k co-occurrences is. min(nl ,n2) k-1 E p(k) = 1 - Z p ( k ) . i=k i----0. The values in the probability column of Tables I and 2 are computed using this formula for P(k); this allows us to determine whether the observed number of co-occurrences is unlikely to have been produced by chance.. Using the above formula, the probability that the a n t o n y m pair in question has at least one co-occurrence is P(1) = 1 - p ( 0 ) . Let j refer to the j t h a n t o n y m pair, and pj (k) and Pj (k) the probabilities of k co-occurrences and at least k co-occurrences. 17. Computational Linguistics Volume 17, Number 1. respectively, a n d r b e the total n u m b e r of a n t o n y m p ai rs b e i n g c o n s i d e r e d . N o w , select s pairs, { J l , . . . , Js}, o u t of the total of r pairs to h a v e at least o n e c o - o c c u r r e n c e each, a n d the r e m a i n i n g r - s p a i r s { J ~ + l , . . . , J r } as h a v i n g n o c o - o c c u r r e n c e s . Call this p a r t i c u l a r c o m b i n a t i o n c, a n e l e m e n t of the set C~ o f c o m b i n a t i o n s o f s o u t o f t h e r pairs. T h e p r o b a b i l i t y qc t h a t this p a r t i c u l a r c o m b i n a t i o n w o u l d b e f o u n d b y c h a n c e is. T h e o v e r a l l p r o b a b i l i t y Qs t h a t e x a c t l y s p a i r s will c o - o c c u r b y c h a n c e is t h e s u m ~ c c ¢ ~ qc; it consists of. (:) s u c h t e r m s , o n e f o r e a c h c o m b i n a t i o n c in Cs. Finally, t h e p r o b a b i l i t y t h a t at least t p a i r s h a v e at least o n e c o - o c c u r r e n c e is ~ = t Qs.. This is the f o r m u l a u s e d to c o m p u t e t h e p r o b a b i l i t i e s of g e t t i n g at least as m a n y p a i r s c o - o c c u r r i n g as w e o b s e r v e , in Tables 1 a n d 2. H o w e v e r , t h e n u m b e r o f t e r m s r e q u i r e d f o r c o m p u t i n g the p r o b a b i l i t i e s Q~ b e c o m e a s t r o n o m i c a l w h e n r is l arg e a n d s is n o t close to 0 o r to r. This w a s the s i t u a t i o n in o u r an al y si s o f m o r p h o l o g i c a l a n t o n y m s (r = 346, s = 48) in Section 2.4. In s u c h cases, s i m u l a t i o n m u s t b e u s e d to e s t i m a t e ~ = t Qs. T h e s i m u l a t i o n p r o c e e d s as follows. In e a c h trial, r r a n d o m n u m b e r s are g e n e r a t e d , u n i f o r m l y d i s t r i b u t e d b e t w e e n 0 a n d 1. If t h e i t h n u m b e r is g r e a t e r t h a n p~(O), t h e n t h e trial p r o d u c e s a c o - o c c u r r e n c e f o r p a i r i. We k e e p t r a c k of t h e n u m b e r of s u c h c o - o c c u r r e n c e s g e n e r a t e d f o r e a c h trial. A f t e r 1,000,000 trials, w e h a v e a n a p p r o x i m a t i o n to the p r o b a b i l i t y d i s t r i b u t i o n fo r t h e n u m b e r o f c o - o c c u r r e n c e s e x p e c t e d to b e p r o d u c e d b y chance. If k o r m o r e p a i r s n e v e r c o - o c c u r in 1,000,000 trials, the p r o b a b i l i t y of so m a n y p a i r s c o - o c c u r r i n g is v e r y l i k el y to b e s m a l l e r t h a n 10-5; o t h e r w i s e , the e x p e c t e d n u m b e r of o b s e r v a t i o n s w o u l d b e at least 10, t h e p r o b a b i l i t y of h a v i n g n o trial w i t h k or m o r e p a i r s c o - o c c u r r i n g w o u l d b e less t h a n 0.003.. Acknowledgments This paper owes its existence to productive interaction with George Miller, who brought the phenomenon of antonym co-occurrence to our attention and provided very useful critical comments on the paper. We have also received helpful comments from Ted Briscoe, Roy Byrd, Martin Chodorow, Judith Klavans, and Yael Ravin. We thank Sue Atkins, Claudia Justeson, and Yael Ravin for evaluating candidate antonyms for high-frequency adjectives.. References Charles, Walter G. and Miller, George A.. (1989). "Contexts of antonymous adjectives." Applied Psycholinguistics, 10(3):357-75.. Church, Kenneth W. and Hanks, Patrick (1989). "Word association norms, mutual information, and lexicography." Proceedings, 27th Annual Meeting of the. Association for Computational Linguistics, Vancouver, 76-83.. Deese, James E. (1962). "Form class and the determinants of association." Journal of Verbal Learning and Verbal Behavior, 1(3):79-84.. Deese, James E. (1964). "The associative structure of some common English adjectives." Journal of Verbal Learning and Verbal Behavior, 3(5):347-57.. Deese, James E. (1965). The Structure of Associations in Language and Thought. The Johns Hopkins Press.. Ervin, Susan (1961). "Changes with age in the verbal determinants of word-association." American Journal of Psychology 74:361-72.. Ervin, Susan (1963). "Correlates of associative frequency." Journal of Verbal Learning and Verbal Behavior, 1(6):422-31.. Fenk-Oczlon, Gertraud (1989). "Word frequency and word order in freezes." Linguistics, 27(3):517-56.. 18. Justeson and Katz Co-occurrences of Antonymous Adjectives. Francis, Winthrop N. and Ku~era, Henry (1982). Frequency Analysis of English Usage: Lexicon and Grammar. Houghton Mifflin.. Gross, Derek; Fischer, Ute; and Miller, George A. (1989). "Antonymy and the representation of adjectival meanings." Journal of Memory and Language, 28(1):92-106.. Shanks, David R. and Dickinson, A. (1987). "Associative accounts of causality judgment." In The Psychology of Learning and Motivation, edited by Gordon H. Bower, 21:229-61, Academic Press.. Thorndike, Edward L. and Lorge, Irving (1944). The Teacher's Word-Book of 30,000 Words. Teacher's College, Columbia University.. Wilks, Yorick; Fass, Dan; Guo, Cheng-Ming; McDonald, James E.; Plate, Tony; and Slater, Brian M. (1989). "Machine tractable dictionaries as tools and resources for natural language processing." In Computational Lexicography for Natural Language Processing, edited by Branimir Boguraev and Edward J. Briscoe, 193-228, Longman.. 19

New documents

– Make plans and ideally let the clinic/doctor/nurse know in advance that you wish to talk about your future care plans, so they can be prepared and make time for it... tAlKinG About

1Department of Surgical Oncology, Sir Run Run Shaw Hospital, Zhejiang University College of Medicine, Hang- zhou, Zhejiang, China; 2Department of Thyroid and Breast Surgery, Changhai

2016, with some minor differences: Be- cause all data for the shared task was parallel, a constrained model was built by employing a source side matching approach inspired by stan- dard

Resident Care Plan Audit An audit of ten residents’ dying domain page care plans and medical charts from Le Chéile was carried out to determine the range and level of information

The system uses three types of features: the features based on the target-language model estimates provided by the baseline system, linguistic features concerning the source word

Table 1: The final best BLEU score after the runs of the inner loop for PSO without and with the termination condition with 16 and 64 threads respectively and standard Moses MERT

In the present study, results showed the VEGF expression in the adjacent normal endometrial tissues of EP patients was significantly lower than in normal endometrial tissues of mid-

When you have an advancing neurological illness, planning for the future may also include talking with your family and healthcare team about your wishes and preferences in case you

The system-level correlation of metrics on evaluation translation out of English on WMT15 Metrics task.. Metrics de-en cs-en fr-en fi-en ru-en

Lifestyle intervention refers to patients with glucose tolerance reduce stage for planned, systems guidance and control such as diet and exercise, and to establish and maintain healthy