Other Experiential Factors

As in the previous section, I conclude with a collection of remarks on other types of experience that might systematically affect judgments of grammaticality. The most obvious would be the amount of experience with the language in question.

There have been numerous studies of metalinguistic skill in nonnative learners of a second language, as part of the second-language teaching literature, which is beyond the scope of this investigation (see Ellis (1991) for a review). Clearly, one would expect nonnative speakers to differ from their native counterparts in judgments as well as in language use, but the results of a third experiment in Snow & Meijer’s (1977) study (see also Sections 4.2 and 4.4.1) suggest that native intuitions may be acquired independently of native skill in language use.

This experiment involved the same procedure as was used in Snow & Meijer’s first and second studies, this time with nonnative speakers of Dutch as subjects.

Their within-subject consistency was at least as good as that of native speakers, but predictably they showed more between-subject disagreements, since their degree of familiarity with Dutch was not matched. Nonetheless, their pooled judgments agreed somewhat better with the native speaker group than those of the linguists did. And, surprisingly, the three virtually bilingual non-natives did not match the native group better than the remaining poorer Dutch speak-ers (as measured by correlations in rank-ordering). The authors interpret this to mean that one’s skill in speaking a language can improve without one’s syntac-tic intuitions becoming more nativelike.³³Conversely, they suggest that classics scholars, for instance, show the opposite: they develop strong intuitions without being able to speak the language. Together with the large amount of variation in judgments among native speakers found in the first two experiments, Snow &

Meijer’s results lead them to conclude that speaking and understanding involve a different language faculty from judging, since skill in one is not a good pre-dictor of skill in the other. On the other hand, Coppieters (1987) claims that his subjects appeared to have achieved native levels of production and comprehen-sion, and yet their judgments were significantly different from those of native speakers. But, as discussed in Section 3.5, Coppieters’s study had not actually shown that the two groups were identical in their use of the crucial forms, but

33Chaudron (1983) points out that there were only eight nonnative subjects in this experiment altogether, so due caution is advised in interpreting the results.

only on unrelated general measures of fluency, mastery of various constructions, and so forth. Thus, we have no basis for concluding that nonnative speakers dis-play differences unique to their judgments. More likely, their grammars simply differ from those of natives on the points investigated, and this would show up in everyday use as well if these constructions occurred. It has also been proposed that experience in another language (e.g., bilingualism) leads to differences in metalinguistic ability (Van Kleeck 1982; Bialystok 1986; see Section 5.3.2).

One would expect certain types of nonlinguistic experience to influence judg-ments as well, for example, factual world knowledge, and cultural and social experiences and beliefs. Greenbaum’s (1977) review cites correlations between judgments and occupation or socioeconomic class, without elaborating. Svartvik

& Wright (1977) found differences on judgments by 14-to-17-year-olds concern-ing the use of ought correlated with the different academic standconcern-ing of three groups of English schools they attended. I am not aware of any studies show-ing that these variables can affect structural judgments. A purported example of how world knowledge is relevant to grammaticality is provided by Belletti (1988).

According to her, the following two sentences involving subject postposing con-trast in grammaticality in Italian:

The crucial difference here is claimed to be that we can assume people normally own only one wallet, but the same is not true for a plant. If this is true, some-one from a different culture presumably would not show this distinction. Un-fortunately, according to one native speaker I consulted (Mirco Ghini, personal communication), while (9b) does require the presupposition of a unique plant that the speaker is referring to, it is structurally fine. In fact, it represents the unmarked word order for expressing this idea, and is clearly better than other starred examples given by Belletti that do seem to violate structural constraints.

Evidently, a systematic investigation of this point is called for. G. Lakoff (1971) has argued that the well-formedness of a sentence can never be assessed without reference to a set of presuppositions about the nature of the world, and cites nu-merous sentences where people differ in this regard. For example, whether My cat enjoys tormenting me is grammatical depends on whether one believes cats to have minds. In cultures where events are believed to have this property, the equivalent of My birth enjoys tormenting me is perfectly normal. Similarly, Lakoff

has argued that grammaticality judgments of John called Mary a Republican and then she insulted him depend on the speaker’s beliefs, and perhaps even on John’s and Mary’s. Chomsky (1972) argues instead that such sentences should be con-sidered grammatical regardless of anyone’s beliefs, and that it should be left to the semantic component of the grammar to specify the presuppositions they re-quire. (See also Bar-Hillel (1971), who argues against those who feel “obliged to force a clearly pragmatic matter into a syntactico-semantic straitjacket.”)

4.5 Conclusion

The studies reviewed in this chapter show that a considerable proportion of indi-vidual differences in grammaticality judgments can be attributed to specific lin-guistically relevant features of the person, be they inborn or the result of experi-ence. Nonetheless, we can be fairly certain that there remains much variation that we cannot factor out in this way. In this regard grammaticality judgments are like most other forms of behavior, including other metalinguistic tasks such as am-biguity detection (Kess & Hoppe 1983). A common genetic endowment provides for a certain degree of commonality, and certain gross parameters of variation, but beyond that differences abound. This state of affairs, however immutable, presents frustrating problems once we acknowledge that the study of grammar, while in principle a study of each individual’s mental structures, must appeal to the judgments of many individuals. However, before we resign ourselves com-pletely, we should consider that not all the variation that shows up within and across experiments is attributable to real differences between subjects. Subtle dif-ferences in procedures or in the sentences themselves can add error to the actual variation. In the next chapter I turn my attention to such confounding sources.

Grammaticality Judgments

Meander (a linguist): I have a theory that everybody’s eyes are colourless.

Simplon (a psychologist): But, Meander, everybody’s eyes look brown, blue or green to me.

Meander: That’s because they are actually wearing contact lenses to color their eyes.

Simplon: But, Meander, I know that I don’t wear contact lenses, and when I look in the mirror my eyes look blue to me.

Meander: Ah: but then, there’s a lot we don’t know about mirrors.

(Bever 1974)¹

5.1 Introduction

By now it should not be a surprise to find that grammaticality judgments may vary depending upon the procedure by which they are obtained and properties of the stimulus items that are presented. In fact, the latter assertion might seem tautologous. Obviously, if judgments are to be of any value they must vary de-pending on the sentences being judged. My focus here, therefore, will be on vari-ation caused by factors that are irrelevant to the concept that we are trying to access through grammaticality judgments, namely grammaticality. Clearly there is room for disagreement here, since what should count towards grammaticality is a matter of theoretical assumption or fiat. Similarly, whether an experimental procedure interferes with grammaticality judgments depends on one’s view of how best to obtain them. For the most part, the variables I examine in this chapter would be uncontroversially labeled as confounds by the majority of linguists.

Even where there is disagreement, for instance, as to whether context is a nui-sance or an integral part of the grammaticality of a sentence, systematic study

1Tom Bever (personal communication) states that this is a dialogue version of a joke attributed to Haj Ross.

should lead to a better understanding of the phenomenon, and thus improve the linguist’s chances of designing effective elicitation procedures. The recommen-dations I make on the basis of the research reviewed here reflect my own point of view on the nature of grammaticality.

This chapter is essentially a top-down survey of the experimental literature.

Section 5.2 covers features of the elicitation process as a whole, beginning with what subjects are asked to do, that is, how the procedure of judging grammat-icality is explained to them (Section 5.2.1).² This issue will pervade the entire chapter, since differences in instructions could be largely to blame for the stag-gering discrepancies among experiments, and much of the existing literature is undermined because vague instructions make many of the results virtually un-interpretable. ³Then I examine the effects of the order in which sentences are presented for judgment (Section 5.2.2). The next three subsections largely follow the research program of one experimenter, Hiroshi Nagata, whose initial work looked at the effects of repeated exposure to the same sentences (Section 5.2.3).

This also raises the issue of intrasubject consistency, which was a subsidiary concern of several other experimenters mentioned in this chapter. Later work by Nagata and others brought in mental state manipulations and their interac-tion with repetiinterac-tion (Secinterac-tion 5.2.4), and sought support for his hypothesized ex-planations by correlation with subjects who were explicitly told to use certain judgment strategies (Section 5.2.5). The subsequent subsection explores the least-studied procedural variable, which nonetheless could arguably have the great-est impact on judgments – namely, the presentation modality (spoken versus written). Closely tied up with this is the matter of register, since together these two factors define to a large degree the nature of the discourse situation, and hence how grammatically strict or permissive we are liable to be as listeners (Section 5.2.6). Finally, I take a brief look at speed of judgment (Section 5.2.7).

Section 5.3 takes a closer look at the properties of stimulus materials them-selves. I begin with the role of the context in which the sentence is situated. I restrict the term context to the purely linguistic context, ignoring social factors

2I do not concern myself here with the much larger question of the range of tasks one might use to elicit information about acceptability. This was discussed to some degree in Section 3.2;

further exploration is beyond the scope of this book, since my focus is on judgments.

3One feature of the task instructions not covered explicitly in this chapter is the type of judg-ment required: good/bad or numeric rating or relative ranking. This issue is discussed in Sec-tion 3.3.4. I also omit menSec-tion of certain standard confounding effects that psychologists typi-cally seek to avoid but that do not seem to have any special impact in the domain of language judgments. Some of these are mentioned in conjunction with the methodological proposals in Section 6.3.

that obviously influence acceptability in the broader sense (see van Dijk (1977) for a discussion of social factors). Nevertheless, the term context is used in at least four different ways (Section 5.3.1). My next concern is the extent to which the meaning of a sentence, or the apparent lack of meaning, affects people’s judgments of grammaticality, in cases where (we assume) it is an orthogonal is-sue (Section 5.3.2). I then ask the same question about how easily a sentence is parsed (Section 5.3.3) and the (perceived) frequency of occurrence of sentences of the same type (Section 5.3.4). The final two subsections concern the level of indi-vidual words: what happens when one word is replaced by another that is gram-matically equivalent (Section 5.3.5), or with one that is gramgram-matically identical (Section 5.3.6). Some potential stimulus variables do not have separate headings devoted to them in this chapter. Intonation is mentioned briefly in conjunction with modality in Section 5.2.6. Its written counterpart, punctuation, does not ap-pear to have been studied for its effects on grammaticality judgments in general (but see Levelt (1974) for some discussion of its possible effects), although it is occasionally mentioned anecdotally in other types of psycholinguistic studies.

The most detailed work I am aware of on the linguistic significance of punctua-tion is by Steegar (1975), who analyzes its relapunctua-tion to prosody. Finally, Secpunctua-tion 5.4 summarizes the major implications of the reviewed research.