Syntactic and semantic processing in the brain

Understanding a sentence requires interpreting each word in a syntactically and semantically coherent way. Many studies have investigated the process of combining words into a

37

degree of complexity in syntactic combinations. For example, the degree of syntactic complexity and local ambiguity varies across different sentences in a natural language

environment. Several studies have manipulated the syntactic complexity of sentences without violating the grammar (Just, Carpenter, Keller, Eddy & Thulborn., 1996; Caplan, Alpert & Waters., 1998; Caplan, 1999) to investigate the neural substrates of syntactic processing in healthy subjects. Consistent with the traditional view of Broca’s area as a “syntax-region”, these studies showed that syntactically more complex sentences (e.g. object relatives structure like “The reporter who the senator attacked admitted the error”) elicited stronger activation in Broca’s area (L-BA44/45) than less complex sentences (e.g. subject relatives structure like “The reporter who attacked the senator admitted the error”). Moreover, Rodd et al. (2010) manipulated the syntactic ambiguity of a sentence similar to that in Tyler & Marslen-Wilson (1977) and Tyler et al. (2013):

a) He noticed that landing planes frightens some new pilots (high-ambiguity) b) She thought that renting flats requires a large deposit (low ambiguity)

In an fMRI study of healthy subjects, they found that the posterior portion of LIFG and LpMTG were strongly activated for high-ambiguity sentences compared to low-ambiguity sentences. In conjunction with the evidence from the morphological studies, these studies emphasized the functional role of the commonly activated left-lateralized LIFG-LpMTG network in syntactic processing at both lexical and sentential levels.

In contrast to the regions involved in syntactic processing, a more bilateral and distributed network is involved in semantic processing including temporal cortex, inferior parietal cortex and inferior frontal cortex (Binder et al., 2009; Price, 2010, 2012). To investigate brain regions involved in semantic processing, a number of fMRI studies have contrasted the brain activity associated with semantically plausible and implausible sentences. For example, Roder, Neville, Bien & Rosler (2002) found that meaningful sentences elicited greater activation in perisylvian cortex including LIFG and both anterior and posterior temporal regions than pseudo-word sentences (stronger in left). This pattern of results has been observed in other studies using similar experimental manipulations (Narain, Scott, Wise, Rosen & Leff., 2003; Crinion et al., 2003).

The functional role of posterior temporal regions in speech comprehension has long been demonstrated by patients studies (Bates, Wilson, Saygin et al., 2003; Gorno-Tempini, Dronkers, Rankin et al., 2004) and by neuroimaging studies on healthy subjects (Binder,

38

Frost, Hammeke et al., 1997; Miglioretti & Boatman, 2003). Moreover, several studies have reported the engagement of posterior ITG in spoken word processing (Binder et al., 2000) and semantic ambiguity resolution (Rodd et al., 2005). For example, by manipulating the semantic ambiguity of a spoken word in a sentence, Rodd and colleagues (2005) found increased activation in bilateral anterior IFG (BA45) and left posterior ITG when processing semantically ambiguous sentences (e.g. “She saw a hare/hair while she was skipping across the field) compared to unambiguous sentences. Taken together, these studies suggest that the posterior temporal lobe is involved in the lexical-semantic processing of a spoken word with or without context. In conjunction with the abundant evidence for the involvement of

Heschl’s gyrus (HG) and posterior STG in acoustic-phonetic processing (Naatanen,

Lehotokoski, Lennes et al., 1997; Morosan, Rademacher, Schleicher et al., 2001; Formisano, Kim, Di Salle et al., 2003; Mesgarani, David, Fritz & Shamma, 2008), these studies showed evidence for a functional role of this posterior temporal region as a phonological-semantic interface (see Hickok & Poeppel, 2004; 2007).

Also, another consistently reported region in semantic processing studies is the bilateral inferior frontal gyrus. For example, Kang, Constable, Gore and Avrutin (1999) investigated the brain regions involved in processing two-word phrases in one of the three conditions (normal, syntactically anomalous or semantically anomalous) in an fMRI study without an explicit task. They reported significant activation in bilateral IFG when processing

semantically anomalous phrases whereas syntactically anomalous phrases elicited activation only in LIFG (L-BA44). More recent studies have reported that the strong activity in bilateral IFG reflects increased semantic competition or conflicting semantic information inconsistent with the semantic constraints (Vartanian & Goel, 2005; Peelle, Troiani & Grossman, 2009). For example, a spoken word recognition fMRI study which varied the degree of cohort competition (a number of competing word candidates) showed significant activation of bilateral anterior IFG (BA45/47) with increased cohort competition (Zhuang, Tyler, Randall, Stamatkis & Marslen-Wilson, 2012). Given that increased activation in this region has also been observed for processing semantically ambiguous sentences (Rodd et al., 2005), this bilateral IFG region may play an important role in semantically constraining the target word based on the context, selecting the likely candidates and integrating the target into the context. According to Binder et al. (2009), the most consistently reported region across 120

functional imaging studies regarding semantic processing is the left angular gyrus (LAG) located in the inferior parietal cortex. For example, Obleser and Kotz (2009) showed that

39

LAG activation was only observed when successful speech comprehension was accomplished either by increased signal quality or by strong semantic constraints. Activation in LAG was reported when semantically anomalous words were embedded in a sentence (Ni, Constable, Mencl et al., 2000) and when processing a coherent narrative compared disconnected

sentences (Xu, Kemeny, Park, Frattali & Braun, 2005). Similarly, Humphries, Binder, Medler and Liebenthal (2007) showed that processing semantically coherent sentences elicited activity in LAG compared to semantically random sentences (e.g. “The freeway on a pie watched a house and a window”). A recent single word recognition study demonstrated that trial-wise variability both in the degree of cohort competition and in the ease with which the semantic features are integrated generated patterns consistent with multivariate activity patterns in LAG (Kocagoncu et al., 2017). These various lines of evidence suggest that this region is involved in conceptual representation and integration at word, sentence and discourse levels.

Lastly, the anterior temporal lobe (ATL) has been suggested as a core semantic processing region from studies of patients with semantic dementia (Mummery, Patterson, Price et al., 2000; Gorno-Tempini, Rankin, Woolley et al., 2004), a virtual lesion study using repetitive transcranial magnetic stimulation (picture naming and word comprehension; Pobric, Jefferies & Ralph, 2007), a meta-analysis of 97 functional imaging studies elucidating a functionally unified bilateral ATL system (Rice, Lambon-Ralph & Hoffman, 2015). Other functional imaging studies which contrasted the neural activity between sentences and word-lists (or sounds) also showed strong activation in bilateral anterior (superior/middle) temporal regions (Mazoyer, Tzourio, Frak et al., 1993; Schlosser, Hutchinson, Joseffer et al., 1998). Consistent with these results, ATL is involved in syntactic structure building during natural language comprehension (Brennan, Nir, Hasson et al., 2012) and damage in this region has been associated with deficits in understanding complex syntactic structures (Dronkers, Wilkins, Van Valin et al., 2004), suggesting the role of this region in combinatorial processing in natural language comprehension (Hickok & Poeppel, 2007). Rogalsky and Hickok (2008) tested if activity in ATL is modulated by syntax, compositional semantics or both using a selective attention paradigm with an error detection task (either syntactic or semantic). By specifying the sentence-specific ATL region responding to sentences compared to noun-lists, they showed that this region is sensitive to both syntactic and compositional semantic

functions (except for a small proportion of this area that is only sensitive to semantic functions).

40

In summary, semantic processing during language comprehension recruits an extensive bilateral fronto-temporo-parietal network in contrast to syntactic processing which involves a left-lateralized fronto-temporal network. Inside this extensive network, four different regions including bilateral posterior and anterior temporal, inferior frontal and left inferior parietal areas (LAG) have consistently been reported. From these studies, the functional role of each of these areas has been suggested; 1) the posterior temporal regions are involved in lexical analysis of a word by mapping it onto its semantics, 2) the inferior frontal regions are involved in resolving competitions during the process of constraining the interpretation, 3) LAG is involved in representation of conceptual semantics and 4) the anterior temporal regions are involved in combinatorial processing such as semantic composition. In the following section, I describe a number of neurobiological models of language processing in humans which are built upon the rich evidence from these studies.