Differences between Direct and Indirect Cues

and (Holler et al., 2014), two cues do not seem to be more helpful than one. However, the nature of these two cues differs greatly. Due to the diverse nature of different kind of cues we do not yet know if it is always the case that direct and indirect cues cannot be used cumulatively to ease utterance interpretation. Moreover, the age at which we begin to see the first evidence for the comprehenders ability to process and use them seems to differ depending on the type of visual cue. These processing differences become especially evident when looking at the ability to process faces and emotional facial expressions.

3.6 Differences between Direct and Indirect Cues

These two kinds of cues, i.e., direct referential cues such as depicted objects and more indirect social cues, are not just processed differently with regard to language comprehension, but also seem to be processed in partially distinct brain regions.

Looking at the face processing literature, this becomes evident. The ability to recognize faces compared to other objects is widely seen as an exceptional process.

Whereas common objects, such as cars, toasters and computers are visually very distinct, human faces, although differing in detailed features, all share uniform and general features (Grelotti, Gauthier, & Schultz, 2002). Even though we are able to discriminate faces on the basis of their detailed features, most people are not able to do the same with individual objects belonging to a certain category, such as discriminating different types of racing cars or songbirds (Archambault, O'Donnell, &

Schyns, 1999; Diamond & Carey, 1986). However, we interact with people on a daily basis and can hence be called experts in face processing and recognition. This is on the other hand not the case for non-face objects, as we rather discriminate them into separate general categories like bird or dog as opposed to Robin or Great Dane (Grelotti et al., 2002). Yet, if the motivation is strong enough, we seem to be able to become experts in virtually any field.

The motivation for becoming an expert in the recognition and processing of faces is likely socially motivated, allowing us to interact and communicate with each other (Grelotti et al., 2002). Nevertheless, arguably because not everyone is an expert on everyday common objects, indeed, distinct brain regions have been found to be active during the perception of objects in contrast to faces. Pitcher, Charles, Devlin, Walsh, and Duchaine (2009) delivered transcranial magnetic stimulation (TMS) over

!

different brain regions of participants, while they performed face, body and object discrimination tasks. They found that face discrimination was impaired when the right occipital face area was stimulated but not when the right lateral occipital area was stimulated. By contrast, object discrimination was impaired when the right lateral occipital area was stimulated but discrimination was not impaired when TMS was delivered over the right occipital face area. These results led to the conclusion that faces, objects and bodies are processed in functionally different brain areas (Pitcher et al., 2009).

A single case study of a patient with developmental deficits in object but not face recognition supports this assumption (Germine, Cashdollar, Düzel, & Duchaine, 2011). The patient did not suffer from brain damage or memory impairment and underwent an otherwise normal development. Still, in a series of face and object recognition tests, her performance in the object recognition tests across different object categories was impaired. Her face recognition scores by contrast did not show any deficits (Germine et al., 2011).

Scherf, Behrmann, Humphreys, and Luna (2007) looked at the organization of category-selective regions in the brain from a developmental perspective. In an fMRI study, they showed children (5-8 years), adolescents (11-14 years) and adults (20-23 years) videos of natural faces, objects and places and measured their brain activity in the ventral visual cortex. Adults and adolescents both showed activation in classical face-related areas, i.e., the fusiform face area, the occipital face area and the superior temporal sulcus. Children by contrast, although showing adult-like activation for places and objects, did not show activation in any of the classical face-related areas.

Hence, these results suggest that category-selectivity develops differently depending on the class of visual objects. Moreover, Scherf et al., (2007) see their results as compatible with the notion of expertise and the acquisition of expertise for face recognition.

In light of these studies, it becomes clear that not all visual cues are processed in the same way and that they also differ in terms of when we develop the ability to process them. With regard to language processing, it is then not surprising that direct visual cues such as objects and indirect visual cues such as faces might be used to different extents in order to comprehend language.

!

However, being experts in the recognition of faces due to social motivation (Grelotti et al., 2002) and extensive practice throughout our lives does not tell the whole story when it comes to integrating social cues such as faces into language processing because faces as social cues seem to be particularly complex. When talking, we do not just “have a face” so that our conversation partner can recognize it, but we arguably use our face conversationally to (consciously or unconsciously) convey a non-verbal message alongside our verbal message. In turn, the comprehender interprets the facial expression of the conversation partner (mostly unconsciously) and tries to integrate it into the linguistic input in order to correctly understand and interpret it or even to facilitate sentence processing (Carminati &

Knoeferle, 2013).

This facial expression is often not neutral but portrays some kind of emotion depending on the emotional valence of the utterance. However, the use of emotional facial expressions as a cue is not an easy task, as the processing, recognition and interpretation of emotions is highly complex in and of itself.

Recall that the studies we will present in this thesis (Sections 6-9) investigated the use of emotional facial expressions as an indirect visual social cue on real-time language comprehension and thematic role assignment across the lifespan. Hence, the following Section (3.7) will focus on emotion processing in more detail. We will demonstrate that emotions are processed differently depending on the age of the comprehender and will discuss potential underlying reasons. While younger adults seem to prefer negative over positive emotional stimuli, older adults and children appear to show an opposite emotional bias⁸, i.e., they favor positive over negative emotional material. This is especially relevant in the light of our results (see Sections 6-9) and will be discussed further in Section 10.6 taking our results into account.

Moreover, these age dependent processing differences of emotional facial expressions as an indirect social cue have clear implication for the development of a more complete real-time language processing account and hence need to be taken into account. This will also be discussed further in Section 10.8.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

8!! Throughout this thesis we will use the terms (emotional) bias and preference interchangeably and define them as synonyms. For a more detailed discussion on this matter see Murphy & Isaakowitz (2008).!

!