Conceptual Representation of Actions in Sign Language

Download (0)

Full text

(1)

Conceptual Representation of Actions in Sign Language

Christian Dobel*, Stefanie Enriquez-Geppert, Marja Hummert, Pienie Zwitserlood, Jens Bo¨lte

Westfa¨lische Wilhelms–Universita¨t Mu¨nster

Received January 14, 2010; revisions received December 17, 2010; accepted December 30, 2010

The idea that knowledge of events entails a universal spatial component, that is conceiving agents left of patients, was put to test by investigating native users of German sign language and native users of spoken German. Participants heard or saw event descriptions and had to illustrate the meaning of these events by means of drawing or arranging toys. Two types of verbs were tested, differing in the way they are signed. Verbs with a horizontal transient are typically signed with a left-to-right directionality, from the addressee’s point of view. In contrast, verbs with sagittal transients display transitions moving toward or away from speaker. Signers showed a direct mapping preference for verbs with horizon-tal transients, by putting agents at the same position in space as in the signed message (i.e., mirroring signing space). No such effect was found for verbs with sagittal transients. In all, the data fit with the idea that interpretations of signed or spoken languages are modulated by task and culture as well as language-related factors and constraints.

The potential interdependence of language and thought has fascinated researchers ever since Sapir and Whorf (Whorf, 1956) proposed that the language one speaks influences thinking in general and, in par-ticular, semantic/conceptual representations. These representations are the starting points in speaking and the targets in understanding language. They are taken to be quite abstract, represented, for example, in terms of interconnected propositional networks (e.g., Anderson, 1976, 1983). Concepts are the build-ing blocks of explicit long-term memory and pro-cessed actively and consciously in working memory (Baddeley, 1986). Executive mechanisms of working memory serve to connect conceptual–semantic and lexical representations, for purposes of speech

produc-tion and comprehension. The activaproduc-tion of concepts determines what is going to be said and in which order to fulfill a communicative intention. In addition, the world knowledge (knowledge about states and events of the world and its inhabitants), the discourse model (an individual’s model of a discourse so far), and the theory of mind (the attribution of mental states to oneself and others) also play a major role.

How can conceptual knowledge vary as a function of language? The Sapir–Whorf hypothesis, also known as linguistic relativity, states that ‘‘(1) languages vary in their semantic partitioning of the world; (2) the structure of one’s language influences the manner in which one perceives and understands the world; (3) therefore, speakers of different languages will perceive the world differently’’ (Gentner & Goldin-Meadow, 2003; p. 4; Whorf, 1956). Whereas some authors rejected this hypothesis (e.g., Li & Gleitman, 2000; Pinker, 1994), other authors provided convincing evi-dence in favor of linguistic relativity (e.g., Davidoff et al., 1999; Levinson, 2003a; Lucy, 1996). Opponents of linguistic relativity (Pinker, 1994) embrace the idea of language universals, for example, that the syntax of language is fundamentally universal and innate or se-mantics are based on an innate ‘‘language of thought’’ (Fodor, 1975).

In view of recent overviews on the relationship of language and thought, which summarize arguments in favor of and against linguistic relativity, it seems that this issue is far from resolved (cf. Gentner & Goldin-Meadow, 2003). Although syntactic and se-mantic universals are hard to come by (Levinson, 2003b), Chatterjee introduced one for action repre-sentations (see Chatterjee, 2001). He proposed that agents of actions are mentally represented as being *Correspondence should be sent to Christian Dobel, Institute for

Bio-magnetism und Biosignalanalysis, Westfa¨lische Wilhelms–Universita¨t Mu¨nster, Malmedyweg 15, 48149 Mu¨nster, Germany (e-mail: cdobel @uni-muenster.de).

Ó The Author 2011. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com

doi:10.1093/deafed/enq070 Advance Access publication on February 21, 2011

by guest on June 6, 2013

http://jdsde.oxfordjournals.org/

(2)

left to patients of the event.1 This fits well with the notion of innate, preverbal spatial primitives, upon which linguistic devices operate (Jackendoff, 1996). Ini-tial evidence came from an aphasic patient, who assigned the agent role to the left person in action events, independent of whether this person was actually agent or patient of the action (Chatterjee, Maher, Gon-zalez Rothi, & Heilman, 1995). This observation was corroborated by data from unimpaired native speakers of English, who showed a clear left-to-right action tran-sient when depicting action events by different ways of drawing (Chatterjee, Southwood, & Basilico, 1999). Chatterjee founded this left-to-right preference in functional brain asymmetries and assumed that ‘‘the left [hemisphere] mediates schematic and the right im-agistic representations. Language that relies on sche-matic representations, as in relational concepts encoded in verbs and locative prepositions, might rely on an intact left hemisphere’’ (Chatterjee, 2001, p. 60). Opponents of innate cognitive universals saw a number of potential confounds for this phenomenon. An obvious confound is reading and writing direction. Chatterjee tried to minimize the influence of reading and writing by asking participants to use their non-dominant hand or by employing writing independent sentence–picture matching tasks. To further explore this issue, Maass and Russo (2003) compared Italian and Arab speakers with different levels of expertise in Italian. With tasks similar to those by used Chatterjee and colleagues (1999), native speakers of Italian showed a left-to-right strategy, whereas Arab speakers tended to place agents on the right. This effect was modulated by expertise in Italian. Native Arab speak-ers who spent considerable time in Italy were less in-clined to place agents on the right than Arab speakers living in Arab-speaking countries. Such results seem to provide strong evidence against Chatterjee’s univer-sality assumption. However, because the left-to-right tendency in Italians was more strongly expressed than the right-to-left tendency in Arabs, the authors favored a weak innate left-to-right predisposition that is modulated by cultural factors. Consequently, we assumed that children, being influenced by cultural factors to a lesser extent than adults, should display the left-to-right tendency independent of cultural upbringing.

We recently investigated the interplay of culture and age (or expertise) by comparing preschoolers and adults whose native language was either German (left-to-right reading and writing system) or Hebrew (right-to-left reading and writing system; Dobel, Diesendruck, & Bo¨lte, 2007). The two preschooler groups did not differ in agent placement. Interestingly, the preschoolers showed no preference. They posi-tioned agents left and right to an almost equal degree. But Hebrew-speaking adults placed agents more often on the right than German adult speakers, who placed them more often on the left. This nicely corroborated the results of Maass and Russo (2003). In addition to the influence of spatial factors such as directionality of reading and writing, we also looked at linguistic factors. Participants had to depict actions expressed in two different syntactic ways such that either agents or patients/recipients were mentioned first. Although this factor had no influence on Hebrew speakers, German preschoolers and adults were influenced by syntactic structure, and thus, by order-of-mention. When agents were mentioned first, German speakers were more likely to place them on the left than when recipients were mentioned first. These results show that action representations are flexible and influenced by cultural and linguistic factors, such as reading and writing, and order of mention in the verbal input.

Thus, action representation in languages such as English or German turned out not to be a good test case for innate linguistic or conceptual universals. In this article, we turned to sign language, which seems particularly well-suited to explore agent-left concep-tualization. We reasoned that Chatterjee’s hypothesis might receive stronger support in languages that ac-tually use space to convey information. The mapping of spatial primitives (agent is left of patient) onto a lin-guistic spatial surface might well be more straightfor-ward in sign language than in spoken languages because the code used on the surface of the utterance bears more similarities to the underlying conceptual representation than in spoken languages.

Investigating the use of space when speakers and addressees talk about spatial relations is not a new idea (for a summary, see Emmorey, 2002a). When signers converse about a jointly perceived environment, they make use of what Emmorey and Tversky (2002) called

by guest on June 6, 2013

http://jdsde.oxfordjournals.org/

(3)

shared space. Whereas speakers of English usually adopt the addressee’s perspective (Mainwaring, Tversky, & Schiano, 1996) even when the addressee is absent (Schober, 1993), signers do not. We ask here, however, how signers represent events nonlinguisti-cally that were signed to them.

Signers usually avoid mental transformations when describing a jointly observed physical layout; they rather prefer a direct mapping of physical and signing space. If a signer describes an environment that is only visible to the addressee, choice of perspec-tive is almost balanced between speaker and addressee (Experiment 2: Emmorey & Tversky, 2002). If two signers talk about an environment invisible to both of them (e.g., a previously seen map), they both de-scribe the environment as if they were looking at it (Emmorey, 2002b). Apparently, signers use a shared space for describing absent environments. It is not the case that signers prefer a direct mapping of the envi-ronment to signing space because of difficulties with mental rotations in general. Signers can perform men-tal (nonlinguistic) transformations even faster than nonsigners (Emmorey, Kosslyn, & Bellugi, 1993).

Thus, there are two opposing predictions for sign-ers. From the perspective espoused by Chatterjee (2001), signers should place agents to the left of patients, as has been suggested for languages such as English and Italian. If the observations by Emmorey and Tversky (2002) generalize to action scenes, signers should show a direct mapping of signed and physical space, as they do when describing maps. The location of an agent in a physical arrangement such as in a drawing should depend on the signed direction. We are aware that an agent-left bias might be mediated by several factors such as handedness of the speaker and/or comprehender, the orientation of both (do they face each other or not), and whether specific argu-ments of an utterance have been already used or not. These factors stress the necessity to use a strict exper-imental approach in which such factors are controlled and which allows to compare the strength of specific effects against each other and not only their mere qualitative presence or absence.

In order to explore these issues, we used in this article a design similar to Dobel and colleagues (2007). Event descriptions were signed, in German Sign

Lan-guage, to a group of signers. The same utterances were verbally described to a group of native speakers of spoken German. All participants had to draw the events or to arrange toy figures in such a way that they depicted the events. We used two types of verbs for event descriptions. The first type are verbs that are typically signed from right to left by a right-handed signer of German sign language (i.e., from left to right for an opposite addressee; examples are: ‘‘to ask’’, ‘‘to give’’, ‘‘to lend’’). We will nominate these verbs as verbs with a ‘‘horizontal’’ transient (or more loosely horizontal verbs). The signing of sentences containing such verbs starts typically where the agent is signed (e.g., to the right of a signer) and it ends spatially where the patient is signed (e.g., correspondingly to the left of the signer). For the second verb type, the main transient goes typically toward or away from the signer, which we will refer to as verbs with a ‘‘sagittal’’ transient2(or sagittal verbs). In such verbs, the move-ment direction marks iconically onset and end points of a figurative or physical motion assigning semantic roles such as agent and patient to these points. Seman-tic roles are not consistently located in space. The distinction of these verb types corresponds to ‘‘direc-tional’’ (here: horizontal) or ‘‘motion’’ (here: sagittal) verbs (for an overview of these and similar terminol-ogy, see Liddell, 2003).

We expected that the usage of horizontal and sag-ittal verbs has a differential effect on signers. In order to balance for handedness of the speaker, sentences produced by a right-handed signer were digitally mir-rored in order to have the same sentences signed from a ‘‘left-handed’’ speaker. Events with horizontal verbs should bring about more left to right arrangements than sagittal verbs. On the other hand, Chatterjee’s predicted spatial preference might be equally strong for both types of verbs. In that case, the conceptual representation would not depend on its actual expres-sion at the surface.

Methods Participants

Eight deaf signers and eight hearing nonsigners, matched with respect to gender (6 women, 10 men), age (Mage 5 30.5 years, age range from 12–47 years),

by guest on June 6, 2013

http://jdsde.oxfordjournals.org/

(4)

education, and handedness according to the Edin-burgh Handedness Scale (Oldfield, 1971), participated in the experiment. All signers were prelingually deaf and ranged in age from 12 to 47 years (mean age 5 31 years). Half of the deaf participants came from deaf families and were exposed to German sign language (Deutsche Geba¨rdensprache [DGS]) from birth. The other four deaf participants had hearing families and learned DGS at kindergarten and primary school. All deaf participants used DGS as their primary and pre-ferred language and had nine or more years of educa-tion including reading and writing German. The deaf participants are regular visitors of events organized by the Culture Centre for Deaf in Mecklenbeck, Mu¨nster (approximately four to eight events per month). Hear-ing nonsigners were all native speakers of German and were all personally known to the investigators. None of them had any proficiency with or exposure to any sign language.

Stimulus Material

A right-handed, professional DGS interpreter was videotaped with a Panasonic NV-S88E camcorder while signing stimulus material and instructions. Each sentence was digitally stored as a separate movie clip. These movie clips had a mean duration of 6.95 s, ranging between 5 and 8 s.

There were four stimulus conditions, crossing verb types (verbs with horizontal transient vs. verbs with sagittal transient), and hand use for signing (right vs. left hand, created by mirroring stimuli). In the condition ‘‘horizontal, right,’’ the directional verbs ask, giveand lend were used in active sentences, signed with the right hand. A clear transient was thus pro-duced, going from right to left, from the interpreter’s point of view. The condition ‘‘horizontal, left’’ was generated by mirroring the movie clips, using Avid-Xpress 3.5 Software. Thus, movie clips were exactly the same as in ‘‘horizontal, right,’’ but this time going from left to right, as if signed by a left-handed signer. In the condition ‘‘sagittal, right,’’ we used motion verbs with transients going toward or away from the right-handed speaker (push, pull, and open). These movie clips were again mirrored to mimic a signer using her left hand (‘‘sagittal, left’’).

Each verb was used three times, but in the context of different arguments (see Appendix for a list of all event descriptions). The stimuli were read in active voice to the German speakers (see Appendix).

Tasks

Participants performed two tasks: drawing pictures and arranging toy figures. In the drawing task, they had to draw the content of the signed or spoken sen-tence on a sheet of paper and to mark the position of the agent by a cross. In the arrangement task, partic-ipants had to arrange toy figures (PlaymobilÒ), such as a fire fighter or a knight, in order to depict the content of the signed or spoken sentence. Location of the agent was again indicated by the participant and registered by the experimenter.

As these tasks and conditions were all strongly spatially biased by the very nature of signing a com-plete sentence or having the agent first in an active spoken sentence, we included a control condition for drawing and arranging actions. Participants were given cards with a verb written on it. Toys, to be used as potential roles of the action expressed by the verb, were laid down above the card, arranged one above another, from the participant’s perspective (e.g., ask, diver, knight). Participants should arrange these fig-ures into a coherent scene (arrangement task) or draw a picture on the basis of the verb and the figures (drawing task) and were asked to indicate afterwards the side the agent was on. Given that no signed or spoken utterance was provided, participants were free to mentally choose any syntactic or argument struc-ture. This condition thus served as a baseline to assess the preferred physical layout.

Design and Procedure

All signers made 18 drawings and 18 arrangements. Each verb with horizontal or sagittal transient was

encountered six times (drawing/arranging 3

mirrored/nonmirrored/control) by each participant. If verbs appeared in sentence context, it was paired each time with different arguments. Movies were pre-sented on a laptop with a screen size of 15’’. Window sizes for movies were held constant with a size of

by guest on June 6, 2013

http://jdsde.oxfordjournals.org/

(5)

12.07 3 15.24 cm. Speakers encountered each verb four times because the factor Mirroring was not relevant.

All participants first performed the control condi-tions (order of verbs was randomized for each individ-ual), followed by the sentence/signed conditions. Arranging always preceded drawing because signers tend to repeat the position of a specific sign denoting an argument if it was used before. Therefore, drawing might bias the arranging task such that action transi-ents going away or toward the participants would be less likely. The control conditions preceded all other ones for the same reason.

Participants were tested individually, all but two signers were tested at home, and all speakers and the two signers were tested in the Department of Psychol-ogy. Three experimenters were present during testing of signers, one running the experiment, one register-ing the results, and one interpreter. Sessions lasted about 40 min. The proportion of agents drawn or placed on the left was determined for each participant in each condition. These proportions were inverse arcsine transformed for statistical analyses (Kirk, 1982). For sake of comprehensibility we report, how-ever, proportions in Table 1. We also report effect sizes, with eta squared (g2) and the square root r for results of the repeated measurements analysis of var-iance (ANOVA) and r values based on t values and degrees of freedom (df) (Rosenthal, 1991) for post hoc t tests. A widely accepted benchmark suggests (Cohen, 1988, 1992) that r values of .10 can be con-sidered as weak effects, values of .30 as medium, and of .50 and larger as strong effect.

Some basic requirements of parametric tests might not be met by our sample. Therefore, we ran non-parametric tests on all significant results with fre-quency as the dependent variable as a precautionary measure. In order to keep the number of reported results manageable, only significant results of the para-metric tests that were backed up by nonparapara-metric tests are reported.

Results

Analyses of the data from the control conditions (with Group, Task, and Verb Type as factors) are presented first, because they are identical for signers and speak-ers, and unbiased by order information present in all other conditions. There were no significant main effects or interactions. Comparing all cells of the con-trol conditions for both groups against chance revealed that there was only a trend for signers to place agents more on the left in the drawing task for horizontal verbs, t(7) 5 2.2, p 5 .06, r 5 .64.3

Next, we analyzed the data for signers separately because signers had conditions that were not present (Mirroring) or not directly relevant (Verb Type) in spoken speech. A repeated-measures ANOVA with the factors Task (drawing vs. arranging), Verb Type (horizontal vs. sagittal), and Mirroring (agent is signed left vs. agent is signed right from the addressees’ view-point) was run. Again, there was a trend for Task, with slightly more agent-left placements in the drawing task than in the arrangement task, F(1,7) 5 3.88, MSE 5 1.118, p 5 .09, g25 .36, r 5 .60. There was a strong

Table 1 Mean proportions (SE in parentheses) of agent-left placements, broken down by Group (signers and speakers), Verb Type (horizontal and sagittal), Task (drawing and arranging), and Mirroring of the sign (for users of spoken German only active sentences were used)

Control Normal orientation Mirrored

Group Task Task Task

Users of DGS Verb Type Drawing Arranging Drawing Arranging Drawing Arranging Horizontal 0.75 (0.10) 0.42 (0.15) 0.96 (0.04) 0.75 (.55) 0.46 (0.14) 0.29 (0.28) Sagittal 0.58 (0.16) 0.46 (0.15) 0.83 (0.13) 0.42 (0.15) 0.63 (0.16) 0.33 (0.15)

Control Spoken active sentences

Users of spoken German Drawing Arranging Drawing Arranging

Horizontal 0.79 (0.13) 0.58 (0.15) 0.75 (0.12) 0.58 (0.12) Sagittal 0.58 (0.15) 0.54 (0.15) 0.63 (0.15) 0.33 (0.13)

by guest on June 6, 2013

http://jdsde.oxfordjournals.org/

(6)

influence of Mirroring, F(1,7) 5 20.17, MSE 5 .363, p 5.003, g25 .74, r 5 .86. If addressees saw an agent placed on their own left (i.e., on the right from the signer’s view), they also put the agent on the left (74%), whereas they did so in only 43% of the cases when the agent was signed to their right. Thus, this result lends more support to a ‘‘direct mapping’’ than to an ‘‘agent-left’’ hypothesis. This effect of Mirroring was modulated by Verb Type, F(1,7) 5 10.77, MSE 5 .178, p 5 .013, g25 .61, r 5 .78. Although there was a difference between agent placement (drawing and arranging) for signed sentences in normal versus mir-rored versions—86% vs. 38%; t(7) 5 25.49, p 5 .001, r 5 .81—for verbs with horizontal transients, this did not hold for verbs with sagittal transients (63% vs. 48%, not significant).4Note that for sagittal verbs, agents were often not explicitly positioned in space at all. The percentage of left placements for verbs with horizontal transients differed significantly from chance if they were signed to the left of the addressee (for both drawing and arranging: all t . 6, all p , .01).

Finally, we compared the performance of signers and controls in conditions comparable between groups. For signers, these were the unmirrored cases, with agents of horizontal verbs placed on the left from the addressee’s perspective. The equivalent condition for German speakers entailed spoken active sentences with agents mentioned first. This comparison best captures differences between languages when com-plete utterances are given.5

As in the control condition, there was a main effect of Task—F(1,14) 5 6.85, MSE 5 .571, p 5 .02, g25 .33, r 5 .57—with more agent-left placements in drawing (.79) than in arranging (.52). There was also a main effect for Verb Type—F(1,14) 5 10.95, MSE 5.418, p 5 .006, g25 .43, r 5 .65—with more frequent agent-left placements for horizontal verbs (76%) than for sagittal verbs (55%).6 This effect was expected for signers because consistent position-ing in space only held for verbs with a horizontal tran-sient. It was somewhat surprising for speakers of a spoken language, for whom the agent always was men-tioned first in the active sentences, because neither a horizontal nor a sagittal transient is expressed in spo-ken language. Although the percentage of left

place-ments was quite high for signers in the normal orientation condition, there was no main effect of Group and no interaction between Verb Type and Group. Thus, even though verb types were selected according to sign language criteria, speakers of German were similarly affected by verb type as signers. Discussion

We tested the idea put forward by Chatterjee (2001), that there is a general preference to place agents of events on left of patients or recipients. We investigated whether the modality of language input, signing ver-sus speaking, affects the spatial representation of actions. Native users of German sign language and native speakers of German made drawings or toy arrangements of action events that were signed or spoken to them. Two types of verbs were tested: verbs that signers typically sign with a horizontal transient (i.e., from left-to-right or right-to-left from the sign-er’s point of view) and verbs that they typically sign with a sagittal transient moving toward or away from the signer. Movies of such signed descriptions were digitally mirrored so that utterances were also signed from right-to-left instead of left-to-right for horizon-tal verbs (i.e., with the left hand). As a baseline, sign-ers and speaksign-ers of German performed the two tasks without any signed or spoken sentences, merely on the basis of verbs and participants of the action. Mirroring did not affect signing direction for sagittal verbs. It altered only the hand with which signing was done.

In fact, the baseline condition provided only min-imal support for Chatterjee’s hypothesis. When no bias was present in the input towards a specific tran-sient, signers and speakers positioned agents almost equally on the left as on the right even though a non-significant left advantage was observed. There was only a slight left bias of signers for the culturally more trained drawing task. With complete, signed utteran-ces, there was a small left advantage, but input tran-sients of horizontal verbs had a clear impact. In both tasks, left-signed agents were placed left and right-signed agents were placed right (from the viewpoint of the participant). No such direct mapping was found for utterances with sagittal verbs, that is transients in the sagittal plane produced no specific spatial prefer-ence for agents. Keep in mind that agents were seldom

by guest on June 6, 2013

http://jdsde.oxfordjournals.org/

(7)

clearly positioned in space for verbs with a sagittal transient. The fact that signers map the signed and physical space for horizontal verbs fits well with the findings of Emmorey and Tversky (2002). Note that an alternative view might be that signers do not per-form automatically mental rotations to reflect the com-prehender’s perspective. We consider this less likely because signers can perform mental rotations very well and even better than speakers of spoken languages (Emmorey et al., 1993).

In contrast to horizontal verbs, when sentences with sagittal verbs are signed to signers, they distrib-ute agents left and right, both in drawings and arrangements. This could be a task-dependent effect or a general property of this verb type that has been referred to as motion verbs. One possible reasoning is that the mental representation of motion verbs does not specify the position of the participants of the action, at least not on a left-to-right axis.

Although this interpretation may seem clear-cut so far, we would like to point out that the ‘‘direct map-ping’’ effect was not perfect (i.e., agents were not always placed left if signed left). The lack of such clarity is unfortunately quite common in empirical studies and it demonstrates that more than one factor influence the chosen outcome measures. In this article, the signers displayed a preference to directly map the placement of the signers, but sometimes they did not. Obviously there are more factors that determine the location of agents than only a ‘‘direct mapping’’ in-clination. Such factors might be situated in the listener who is going to respond (e.g., ease of articulation, handedness), in the model of the speaker (i.e. what does the listener assume about him/her) or in the discourse (what has been said so far). Although the systematic control or exploration of such effects is beyond the scope of a single study, the present design can certainly serve as a starting point to investigate which other factors determine whether the point of view of someone is adopted or not.

Surprisingly, we found rather little influence of one’s native language on representations of actions. There were no overall group differences, not even for the different verb types. Although there are obvious and visible differences in signing sentences with horizontal or sagittal transient verbs, this is not so for spoken

lan-guage. Nevertheless, we found no evidence for different action representations in speakers of a language that extends in space (and time) and speakers of a language that extends only in time. Obviously, the investigated verb types differ semantically in similar ways for the speakers of German sign language and those of spoken German. This fact is hard to explain by theories that treat language as a symbolic manipulation system (see above and e.g., Anderson, 1976), but less for those that propose a grounding of meaning in bodily and percep-tual activity (e.g., Barsalou, 1999; Glenberg & Robert-son, 2000). In such a vein, meaning derives from ‘‘the biomechanical nature of bodies and perceptual sys-tems’’ (Glenberg & Kaschak, 2002, p. 558). To support such an argument empirically, Glenberg and Kaschak (2002) performed an experiment in which they used stimulus material similar to ours, that is sentences im-plied a motion toward or away of the body (e.g., ‘‘Close the drawer!’’). Participants had to judge whether sen-tences were sensible by making a manual response re-quiring moving toward or away from their body. It turned out, that sensibility judgments were slowed down if the implied movement direction of the verb interfered with the manual response. Thus, this finding supports ‘‘embodied’’ theories of meaning that relate the meaning of sentences to actions. As such, we regard the lacking difference between the investigated lan-guages as compatible with such a view.

We observed a slight (statistically insignificant) left preference for agents when a sign: for example, a visual cue on the left of the addressee, a position in the spoken sentence, or a cue on a left-to-right arrow of time were given to the signers. This lends minimal or no support to Chatterjee’s spatial universal.

Note, however, that this left preference is strength-ened or weakstrength-ened by a number of factors. First, we observed effects of task, that is, assignment of agents to the left is more strongly expressed in drawing, a cul-turally well-trained task. Second, the left preference was influenced by factors pertinent to the specific lan-guage used. Here, there was a direct mapping effect in signing. For speech and writing, agents most often take the sentence subject position in the languages studied and consequently are named first in signs and senten-ces. Furthermore, these languages are written from left to right (note again that right-to-left writing direction

by guest on June 6, 2013

http://jdsde.oxfordjournals.org/

(8)

does weaken the agent-left preference: Dobel et al., 2007; Maass & Russo, 2003). Speakers might thus be ‘‘trained’’ or ‘‘used’’ to place (draw) persons who are mentioned first on the left. Clearly, this first in-first out strategy serves to reduce working memory load.

In sum, our findings give only very little support to an ‘‘agent-left’’ language universal, either in spoken nor in signed languages. It seems rather that interpre-tations of signed or spoken language depend on task demands, on the type of language, and on its specific constraints and the current language input including the agent position in the sentence. Our data support a view that spatial preferences for agents are strongly determined by cultural and language-specific factors.

Notes

1. Agent and patient are semantically defined notions. In a given sentence, the agent is the participant who carries out the action whereas the patient is the one upon whom the action is carried out. Agent and patient are thematic roles and should not be confused with syntactic roles such as subject and object.

2. Note that the terms ‘‘horizontal’’ and ‘‘sagittal’’ do not make sense for spoken languages. But as we made this distinc-tion on the basis of a sign language, we apply it also to spoken German for the sake of comprehensibility.

3. A Wilcoxon test confirmed that signers tended to place agents on the left in case of horizontal verbs in the drawing task (z 5 21.87, p 5 .062).

4. The Wilcoxon tests for Mirroring (z 5 22.512, p 5 .012) and for Task (z 5 21.970, p 5 .049) were significant, corroborating the results obtained with the ANOVA. The

in-teraction was tested with the help of a Friedman test (chi25

11.3, df 5 3, p 5 .01). Verbs with a horizontal transient differed significantly from each other (z 5 22.555, p 5 .011) whereas this was not the case for sagittal verbs (z 5 21.361, p 5 .174). 5. Note that ‘‘mirroring’’ agents and recipients/patients in sign space has no parallel in spoken German. Here, reversing the temporal order of agents and patients would result in passive sentences. In the mirrored signed sentences, only handedness and spatial locations of agents and patients are reversed.

6. Again, both results proved to be significant in the non-parametric tests as well (Wilcoxon task: z 5 22.614, p 5 .009; Verb Type: z 5 22.729, p 5 .006).

Funding

German Research Council, the Deutsche Forschungs-gemeinschaft (Do 711/4-1).

Conflicts of Interest

No conflicts of interest were reported.

Acknowledgements

We cordially thank the ‘‘Kulturzentrum fu¨r Geho¨r-lose’’ in Mecklenbeck for their kind support and Tom Temming for his valuable advice and help. We are also grateful to Pamela Perniss, Meike Vaupel and Rachel Rosenstock for reading earlier versions of the manuscript as well as Simone Schmidt for her help. We would also like to thank the editor, Claudia Pagliaro, and two anonymous reviewers for their help-ful comments and suggestions.

References

Anderson, J. R. (1976). Language, memory, and thought. Hill-sdale, NJ: Lawrence Erlbaum Associates.

Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press.

Baddeley, A. (1986). Working memory. Oxford: Clarendon Press. Barsalou, L. W. (1999). Perceptual symbols systems. Behavioral &

Brain Sciences, 22,577–660. doi: 10.1017/S0140525X99002149.

Chatterjee, A. (2001). Language and space: Some interactions.

Trends in Cognitive Sciences, 5,55–61. doi:

10.1016/S1364-6613(00)01598-9.

Chatterjee, A., Maher, L. M., Gonzalez Rothi, L. J., & Heilman, K. M. (1995). A syntactic thematic role assignment: The use of a temporal-spatial strategy. Brain & Language, 49, 125–139. doi: 10.1006/brln.1995.1024.

Chatterjee, A., Southwood, M. H., & Basilico, D. (1999). Verbs, events and spatial representations. Neuropsychologia, 37, 395–402. doi: 10.1016/S0028-3932(98)00108-0.

Cohen, J. (1988). Statistical power analysis for the behavioral

sciences(2nd Ed.). New York: Academic Press.

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159. doi: 10.1037/0033-2909.112.1.155.

Davidoff, J., Davies, I., & Roberson, D. (1999). Colour catego-ries in a stone-age tribe. Nature, 398, 203–204. doi: 10.1037/0096-3445.129.3.369.

Dobel, C., Diesendruck, G., & Bo¨lte, J. (2007). How writing sys-tem and age influence spatial representations of actions—A developmental, crosslinguistic study. Psychological Science, 18, 487–491. doi: 10.1111/j.1467-9280.2007.01926.x.

Emmorey, K. (2002a). Language, cognition and the brain. Mah-wah, NJ: Lawrence Erlbaum Associates.

Emmorey, K. (2002b). The effect of modality on spatial lan-guage: How signers and speakers talk about space. In R. P. Meier, D. G. Quinto, & K. A. Cormier (Eds.), Modality and structure in signed and spoken languages (pp. 405–421). Cambridge, UK: University Press. Emmorey, K., Kosslyn, S., & Bellugi, U. (1993). Visual imagery

and visual-spatial language: Enhanced imagery abilities in deaf and hearing ASL signers. Cognition, 46, 139–181. doi: 10.1016/0010-0277(93)90017-P.

Emmorey, K., & Tversky, B. (2002). Spatial perspective in ASL.

Sign Language and Linguistics, 5, 3–26. doi: 10.1075/

sll.5.1.03emm.

by guest on June 6, 2013

http://jdsde.oxfordjournals.org/

(9)

Fodor, A. J. (1975). The language of thought. New York: Crowell. Gentner, D., & Goldin-Meadow, S. (2003). Whither Whorf. In D. Gentner & S. Goldin.-Meadow (Eds.), Language in mind (pp. 3–14). Cambridge, MA: MIT Press.

Glenberg, A. M., & Kaschak, M. P. (2002). Grounding lan-guage in action. Psychonomic Bulletin & Review, 9, 558–565.

Glenberg, A. M., & Robertson, D. A. (2000). Symbol grounding and meaning: A comparison of high-dimensional and em-bodied theories of meaning. Journal of Memory &

Lan-guage, 43,379–401. doi: 10.1006/jmla.2000.2714.

Jackendoff, R. (1996). The architecture of the linguistic-spatial interface. In P. Bloom, M. A. Peterson, L. Nadel, & M. Garret (Eds.), Language and space (pp. 1–30). Cambridge, MA: The MIT Press.

Kirk, R. E. (1982). Experimental designs: Procedures for the behavioral sciences. Pacific Grove, CA: Brooks/Cole. Li, P., & Gleitman, L. (2002). Turning the tables: Language and

spatial reasoning. Cognition, 83, 265–294. doi: 10.1016/ S0010-0277(02)00009-4.

Levinson, S. C. (2003a). Space in language and cognition: Explo-rations in cognitive diversity. Cambridge, UK: Cambridge University Press.

Levinson, S. C. (2003b). Language and mind: Let’s get the issues straight. In D. Gentner & S. Goldin-Meadow (Eds.),

Lan-guage in mind(pp. 25–46). Cambridge, MA: MIT Press.

Liddell, S. K. (2003). Grammar, gesture and meaning in American sign language. Cambridge, UK: Cambridge University Press.

Lucy, J. A. (1996). The scope of linguistic relativity: An analysis and review of empirical research. In J. J. Gumperz & S. C. Levinson (Eds.), Rethinking linguistic relativity (pp. 37–69). Cambridge, UK: Cambridge University Press.

Maass, A., & Russo, A. (2003). Directional bias in the mental representation of spatial events: Nature or

culture?Psycho-logical Science, 14,296–301. doi: 10.1111/1467-9280.14421.

Mainwaring, S., Tversky, B., & Schiano, D. (1996). Perspective choice in spatial descriptions. Palo Alto, CA: Interval Research Corporation. IRC Technical Report 1996-06.

Oldfield, R. C. (1971). The assessment and analysis of handed-ness: The Edinburgh inventory. Neuropsychologia, 9, 97–113. doi: 10.1016/0028-3932(71)90067-4.

Pinker, S. (1994). The language instinct. New York: Morrow. Rosenthal, R. (1991). Meta-analytic procedures for social research

(2nd Ed.). Newbury Park, CA: Sage.

Schober, M. F. (1993). Spatial perspective-taking in conversation.

Cognition, 47,1–14. doi: 10.1016/0010-0277(93)90060-9.

Whorf, B. (1956). Language, thought, and reality. In J. B. Carroll (Ed.), Selected writings of Benjamin Lee Whorf. Cambridge, MA: MIT Press.

Appendix A Stimulus Materials

(a) Sentences containing verbs with a horizontal transient

Der Taucher fragt den Ritter. (The diver asks the knight a question.)

Die Oma fragt den Indianer. (The grandmother asks the native American.)

Die Krankenschwester fragt den Opa. (The nurse asks the grandfather.)

Der Mann gibt der Frau den Blumenstrauß. (The man gives the flower bouquet to the woman.)

Der Polizist gibt der Oma den Blumenstrauß. (The police officer gives the flower bouquet to the grandmother.)

Der Junge gibt dem Ritter die Kanne. (The boy gives the mug to the knight.)

Die Krankenschwester leiht dem Ma¨dchen die Gita-rre. (The nurse lends the girl the guitar.)

Der Opa leiht dem Jungen die Gitarre. (The grand-father lends the boy the guitar.)

Der Mann leiht der Oma die Tasche. (The man lends the grandmother the bag.)

(b) Sentences containing verbs with a sagittal transient

Die Krankenschwester schiebt den Rollstuhl.

(The nurse pushes the wheelchair.)

Die Frau schiebt den Kinderwagen. (The woman pushes the pram[stroller].)

Der Taucher schiebt die Schubkarre. (The diver pushes the wheelbarrow.)

Der Ritter zieht den Kinderwagen. (The knight pulls the pram [stroller].)

Der Ritter zieht die Schublade auf. (The knight pulls the drawer.)

Das Ma¨dchen zieht die Kiste. (The girl pulls the crate.) Der Cowboy o¨ffnet die Truhe. (The cowboy opens the chest.)

Der Pirat o¨ffnet die Truhe. (The pirate opens the chest.)

Der Ritter o¨ffnet die Truhe. (The knight opens the chest.)

by guest on June 6, 2013

http://jdsde.oxfordjournals.org/

Figure

Updating...

References