Neural basis of basic composition: what we have learned from the red boat studies and their extensions

(1)

royalsocietypublishing.org/journal/rstb

Opinion piece

Cite this article: Pylkkänen L. 2019 Neural basis of basic composition: what we have learned from the red –boat studies and their extensions. Phil. Trans. R. Soc. B 375:

20190299.

http://dx.doi.org/10.1098/rstb.2019.0299

Accepted: 4 September 2019

One contribution of 16 to a theme issue

‘Towards mechanistic models of meaning composition ’.

Subject Areas:

neuroscience

Keywords:

semantics, syntax, conceptual combination, left anterior temporal lobe,

magnetoencephalography

Author for correspondence:

Liina Pylkkänen

e-mail: [email protected]

Neural basis of basic composition: what we have learned from the red –boat studies and their extensions

Liina Pylkkänen

^1,2,3

1Department of Linguistics, and²Department of Psychology, New York University, New York, NY, USA

3NYUAD Institute, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates LP, 0000-0002-6332-9378

Language is our mind’s most powerful generative system for the expression of meaning and thought. What are the neural mechanisms of our ability to compose complex meanings from simpler representations? This question is impossible to answer unless we decompose the notion of‘meaning composition’ in some theoretically guided way and then begin to assess the extent to which brain activity tracks the posited subroutines. Here, I summarize results from a body of MEG research that has begun to address this question from the ground up, first focusing on simple combinations of two words.

The work sets off with a hypothesis space offered by theoretical linguistics, positing syntactic and logico-semantic composition as the main combinatory routines, but then reveals that the most consistent and prominent reflection of composition, localized in the left anterior temporal cortex at 200–250 ms, cannot be described with this toolkit. Instead, this activity tracks a much more conceptually driven process, robustly sensitive to the density of the conceptual feature space of the composing items. I will describe our functional understanding of this activity and how it may operate within a broader‘combinatory network.’

This article is part of the theme issue‘Towards mechanistic models of meaning composition’.

1. Introduction

The purpose of language comprehension is to extract meaning. The purpose of language production is to convey meaning. The same is true of much non-linguistic behaviour: we compose meaning from our visual and auditory percepts and produce many meaningful actions, even without language. This special issue is devoted to endeavours to understand the composition of linguistic meaning in mechanistic ways. Language is indeed an ideal window into meaning composition, given that linguists have developed exquisitely detailed representational theories about the semantic properties of natural language. Thus, at the algorithmic level [1], we have a rather good idea about what we are looking for in the brain.

Here, I will summarize the results of a research programme that has varied the combinatory properties of linguistic expressions in ways guided by linguistic theory. This research programme follows a hypothesis-killing strategy: initial results are compatible with many accounts but then each subsequent study narrows down the hypothesis space more. I will summarize the work in three

‘steps’, each consisting of many experiments. The first step gets the project off the ground by identifying some possible correlates of composition. The second step rules out large classes of hypotheses, here syntactic and logico-semantic composition, leaving a more conceptually based process on the table. The third step involves a more detailed hypothesis space that begins to get closer to mechanism, examining exactly what type of conceptual combination this activity may implement. I will close with brief comments about the broader ‘combinatory network’ and some open questions for the future.

The work described here uses magnetoencephalography (MEG) to measure neural activity. MEG provides both centimetre-level spatial and millisecond-

Downloaded from https://royalsocietypublishing.org/ on 05 November 2021

(2)

level temporal information about the electric activity created by neurons and is, therefore, ideal for measuring rapidly unfolding language processing. Ultimately, it will be important to understand how MEG findings on language relate to those obtained by haemodynamic methods (offering detailed spatial but poor temporal resolution) and EEG (offering detailed temporal but blurry spatial information).

Establishing these correspondences continues to be methodo- logically challenging. In this brief summary, I will not branch out to this topic much but direct the reader to Schoffelen et al.

[2] and Matchin et al. [3] for projects aimed at elucidating the MEG–fMRI relationship and to Neufeld et al. [4] for an EEG study using the basic composition (red–boat) paradigm which my discussion here mostly focuses on.

2. Step 1: identifying possible neural correlates of composition

In order to study a process in the brain, one first needs to discover something in the brain that might reflect it. For the process of meaning composition, we are faced with an immedi- ate methodological challenge: the composition of meaning is governed by the syntactic structure of the expression, and the steps of composing that structure are tightly—perhaps even completely—intertwined and correlated with the semantic combinatory steps. But if we want to understand meaning composition in a mechanistic way, we must understand the neurobiological relationship between syntactic and semantic composition.

Given this, our first step was to focus on specific constructions in which the tight correlation between syntax and semantics breaks down, or at least cracks a bit, with the aim of trying to identify something specifically semantic, as opposed to syntax. Because the overwhelming generalization is that syntax and semantics go hand in hand—this is the key insight behind Frege’s Principle of Compositionality—cases that appear to violate this principle have driven the vast majority of theoretical research on the relationship between the syntactic and semantic structures of language (syntax–semantics interface).

One well-studied family of constructions in this literature is so-called‘coercion’ constructions, involving an extra bit of meaning that is not expressed in the syntax (or at least the overtly pronounced syntax; for more detailed discussion, see [5]). In these expressions, two elements combine in the syntax but do not go together semantically. In principle, such situations should result ill-formedness, but interestingly, natural language has an inventory of routines that salvage such mismatches in certain cases—those routines are called

‘coercions.’ For example, in the sentence The little boy finally finished his pasta

the verb‘finished’ describes the completion of an event or activity. However, here, its direct object does not describe an event or activity, but rather just an entity or a thing, Surpris- ingly, the expression sounds just fine. But what our brains do here—according to both intuition and much theoretical and psycholinguistic research [5,6]—is insert that missing meaning into the structure. As a result, we interpret this sentence as saying that the little boy finished eating his pasta, even though there is no mention of eating in the sentence.

It is those implicit meaning insertions that our first set of

studies set out to investigate, as a possible case of semantic composition that is not accompanied by parallel syntactic composition ([7–10]; for EEG versions, see [11] and [12]).

This work turned out to be a useful background for the

‘red–boat’ studies which most of this article focuses on.

Our findings from the coercion work were surprising in two ways. First, the localization of the effect was quite focal;

conceivably, a complex process like coercion might recruit many brain regions. Second, coercion affected neural activity in a surprising place, outside any traditional version of the

‘language network’. In our data, the presence of coercion correlated with increased activity in ventromedial prefrontal cortex (vmPFC), a region typically associated with decision making, theory of mind, social cognition—a wide array of different aspects of higher cognition. A follow-up study using fMRI did also implicate the left inferior frontal cortex for coercion sentences [13], showing that at some point in the sentence, this more traditional language region is also recruited. Our vmPFC effects in MEG occurred rather consist- ently at around 400 ms after the onset of the word that triggered coercion ( pasta in the example above), making it plausible that the effect, in fact, reflected coercion.

The‘non-language’ localization of the MEG coercion effects could have been due to an underlying mechanism for some type of domain general mismatch resolution. To address this, we drastically simplified the stimulus, now just measuring the processing of simple adjective–noun combinations, such as

‘red–boat’, involving no syntax–semantics mismatch or complex processing [14]. If comprehending these phrases also engaged the vmPFC, this would rule out mismatch-specific explanations and leave on the table an account in terms of more general processes relating to semantic composition.

The red–boat studies compellingly showed that vmPFC engagement did not require a syntax–semantics mismatch.

A straightforward step of composition between an adjective and a noun was enough to elicit a similarly timed increase in MEG activity localizing to the vmPFC. In haemodynamic research, it was also becoming clear that the vmPFC is part of the general ‘semantic system’ of the brain, based on a meta-analysis of a large set of neuroimaging studies [15,16].

Importantly, the simple composition of adjectives and nouns also elicited another consistent effect, in fact more consistent that the vmPFC effects, which we sometimes have not replicated (e.g. [17]). The second effect localized in the left anterior temporal lobe (LATL) and occurred earlier, at around 200–250 ms after the onset of a word in its combinatory context (figure 1). It was already well-known that in haemodynamic measures, a contrast between sentences and unstructured lists elicits an increase in the LATL for the sentences (e.g. [19–22]), and we had also observed this effect in MEG [23]. Since our red–boat studies were in a sense a mini-version of the popular sentence versus list paradigm, a LATL effect made sense and usefully showed both the timing of this activity—not obtain- able from fMRI or PET—and that the LATL is engaged by just a single step of composition, as opposed to needing a full sentence as a stimulus. When simple composition is measured with EEG, combinatory effects are seen in time-windows similar to the LATL and vmPFC effects observed in MEG [4].

Altogether, the coercion and red–boat studies were a two-pronged approach to Step 1: this work defined some dependent measures whose contribution to composition one can then study further. In particular, subsequent research has systematically narrowed down the hypothesis space capable

ro yalsocietypublishing.org/journal/rs tb Phil. Trans. R. Soc. B 375 : 20190299

2

Downloaded from https://royalsocietypublishing.org/ on 05 November 2021

(3)

of explaining the initial red–boat effects in the LATL, as summarized in ‘Step 2’ below. Our understanding of the vmPFC is more elusive. While in comprehension, vmPFC effects are relatively late, at around 400 ms, they onset at around 150–200 ms when the task is picture naming and we are measuring the planning stages of phrase production [18,24,25]. Thus, it is quite compelling to consider the vmPFC as a representational site for the end product of composition in comprehension and the starting point for it in production—that is, in production, the vmPFC may represent the combinatory message which is then packaged into lexical items. However, our replication rate for the vmPFC effects has been poorer than for LATL effects (e.g. [17]), making systematic research on this activity harder.

There are many possible reasons for this. The activity is close to the eyes and thus prone to artefacts. Also, as late activity (in comprehension), it may be quite task sensitive. Thus, the rest of my discussion will focus on the LATL, as we have learned more about it.

3. Step 2: distinguishing between large classes of hypotheses (syntactic, logico-semantic versus conceptual composition)

To understand the functional role of an effect, we need to characterize both generalizability (how the effect extends to new contexts) and computational limits (how the effect disappears). As regards generalizability of the combinatory effect in the LATL, we have shown that it occurs in reading [14,26], listening [17] and language production [18], suggesting a rather modality independent function. Crosslin- guistically, the LATL effect generalizes not only to languages that are typologically far from English, like Arabic [27], but also to American Sign Language [25], which in addition uses a different articulator. Across constructions, we know

that the effect obtains not only for noun phrases but also for verb phrases [27,28] and noun–noun compounds [29,30]. These extensions are crucial for understanding function, but in the remainder of this text I shall focus on the computational limits of this activity, that is, the situations in which the effect disappears.

(a) Distinctions within basic composition

Within the general landscape of cognitive neuroscience, much of which focuses on large networks and complex tasks, neural correlates of combining two words together may seem like a rather specific finding. But for a linguist, such a finding is still completely ambiguous: it could reflect any aspect of composition. Returning to the simple case of‘red–boat’, the composition of these two words potentially involves many correlated but distinct combinatory processes. One of them is the syntactic combination of the adjective and noun categories into a noun phrase. In principle, syntactic combination does not care about the meanings of the input items: for the purpose of the syntax,‘red boat’ and ‘married boat’ are equally good noun phrases and will do the job of a noun phrase in any syntactic context that has a‘noun phrase slot’. And conversely, semantic well-formedness cannot salvage syntactic ill-formedness:‘quickly boat’ is no good as a noun phrase even though we could probably work out that this combination might describe a quick boat, which is a perfectly fine concept.

But ultimately, our brains are always trying to extract meaning from language and the syntax is really just there to help us extract or express meaning. As regards possible subroutines of meaning composition, theoretical research in formal semantics and behavioural research in cognitive psychology lead us to expect at least two classes of semantic combinatory operations, one more logical and the other more conceptual in nature (figure 2a). The more logical system is mainly a composer of predicate–argument 4

vmPFC

comprehension production

LATL

12

0

nAnA nA

12

0

nA

3 2 1

0 100 200 300 400 500 0 200 400 600

ms

0 200 400 600

ms ms

red boat composition

red, blue list 0 100 200 300 400 500

4 3 2 1

ms

red boat (composition) xtp boat

cup, boat (list) xtp, boat

Bemis & Pylkka¨nen [14] Pylkka¨nen et al. [18]

(b) (a)

Figure 1. Neural correlates of basic composition as revealed by the red –boat paradigm. During reading ((a) 0 ms is the onset of the noun), a phrasal stimulus elicits an early LATL increase at 200 –250 ms followed by a later vmPFC increase [14]. During the planning stages of picture naming ((b) 0 ms is the onset of a picture), the same regions show higher activity for phrases than for lists, but now the two effects both onset early, around 200 ms, and show a more sustained effect profile [18].

(Online version in colour.)

ro yalsocietypublishing.org/journal/rs tb Phil. Trans. R. Soc. B 375 : 20190299

3

Downloaded from https://royalsocietypublishing.org/ on 05 November 2021

(4)

structure, and in some cases, that structure diverges from the way the syntax ‘cuts the pie’, thus motivating a separate system (e.g. [33]). For example, although adjective–noun combinations and adverb–verb combinations are syntactically different, the semantic composition of both (at least in some cases) has been hypothesized to involve the same conjunction rule, such that‘red–boat’ yields a description of an individual that is both red and a boat while‘slowly walks’

describes an event that is both a walking event and slow (e.g. [34]).

Conversely, semantic combinatorics can differ between syntactically parallel phrases. We have already seen this with the coercion cases, but here is another example: while

‘she liked my eye colour’ and ‘she guessed my eye colour’

are parallel in surface syntax, the semantic restrictions of

‘like’ and ‘guess’ are very different [35]. Consequently, we can swap‘eyes’ for ‘eye colour’ in the like-sentence and get another grammatical sentence: ‘she liked my eyes’. But not so in the guess-sentence: ‘she guessed my eyes’ sounds strange. This is because‘guess’ semantically needs a prop- osition/question as its argument (she guessed what my eye colour was) or something that can easily be converted into a question. Nouns describing relations, like colour, value, age or name, turn out to be such, but entity-denoting nouns like

‘eyes’ are not. Formal semantic theories in linguistics are theories about these combinatorics and the semantic argument structures that drive them. This domain of inquiry is distinct from syntax, though the two are intricately connected.

Finally, although syntactic and formal semantic theories offer elaborate accounts of the possible distributions of words and phrases, capturing a vast range of data about our intuitions of well-formedness, they still do not speak to the actual conceptual content of linguistic expressions. The formal systems compose‘black cat’ and ‘black hole’ exactly the same even though these are obviously not the same thing. Thus, we still need a proper conceptual system as well. Though the conceptual aspect of composition has not been much of a topic in linguistics, it has been studied in cognitive psychology, with a focus on adjective–noun and noun–

noun combinations [36–38]. Thus, there is a contrast in the

scope of the conceptual work in psychology and logico- semantic research in linguistics, the latter basically addressing the entire language system while the former has focused on specific constructions. But the basic prediction is that the syntactic and logico-semantic systems should be rather insensitive to the conceptual details of the input items, taking dogs, cats, clouds and houses as equal input whenever a noun or an entity is needed. But for the conceptual system, a cloud and a house should be rather different things.

(b) Sensitivity of the LATL to conceptual content

Nothing about the LATL findings as presented so far discrimi- nates between the three modes of composition just laid out (figure 2a). When we combine an adjective with a noun, all three combinatory steps are expected to occur. In the Step 1 phase of our research, a syntactic function seemed quite likely and we made speculations in that direction in our original report [14]. For example, in another study, we had observed a correlation between syntactic node count in a narrative and blood oxygen level-dependent (BOLD) signal in the LATL (e.g. [39]). Also, a hypothesis proposing that phrase-structure is initially processed in a similar time window to our LATL findings (around 200 ms) was prominent in the literature [40].

But we soon learned that our LATL effect had a distinctly non-syntactic, and non-logico-semantic, nature. Converging evidence for a non-syntactic role for the LATL began to mount from neuroimaging and neuropsychological research, as well [41–43].

We discovered that the LATL does not show a combinatory effect every time an adjective and a noun compose, as one would expect from a syntactic combiner. Instead, the effect was heavily modulated by the nature of the concepts described by the words, in particular, by the specificity of their meaning. Here, I am referring to hierarchical relations between concepts such as poodle being a type of dog and dog being a type of mammal and so forth. In other words, poodle is more specific than dog and dog is more specific than mammal. We learned that the LATL effect goes away if we swap boat in red–boat to a specific instance of a boat, syntactic,

logico-semantic versus conceptual composition

lx.red(x) and boat(x)

A lx.red(x) N ly.boat(y) NP

red boat

(b) (i) (ii)

(a)

LATL effect

blue boat blue canoe

tomato dish vegetable dish no LATL effect

Figure 2. Components of composition and the conceptual modulation of the LATL. (a) Possible subroutines of basic composition, derived from linguistics and cognitive psychology. Distinguishing between the different subroutines is a core challenge for understanding the neural activity that is sensitive to composition.

(b) Examples of combinatory stimuli that do (i) and do not (ii) elicit measurable LATL effects during the processing of the second word [31,32]. Black dots represent features contributed by the first word and grey dots features contributed by the second word. More specific meanings are represented with more dots (since canoe is a type of boat, canoe gets more dots than boat). The percentage of black dots within the full feature set of the phrase is higher on the left than the right side. It is hypothesized that this ratio is important for the LATL combinatory effect. (Online version in colour.)

ro yalsocietypublishing.org/journal/rs tb Phil. Trans. R. Soc. B 375 : 20190299

4

Downloaded from https://royalsocietypublishing.org/ on 05 November 2021

(5)

such as canoe [31,44]. We observed no measurable LATL effect for red canoe. The specificity of the modifier mattered, too. But now a more specific modifier was the one that led to a larger LATL effect: when composing lamb with stew to yield lamb stew, the LATL showed the usual combinatory effect after the onset of stew, but if the modifier is more general, like meat, it did not [32]. So no measurable LATL effect for meat stew. This cannot be explained with syntax.

We did not expect this pattern at all; rather, we ran these studies to try to connect our composition findings to an extant haemodynamic and neuropsychological literature on the LATL, suggesting a role for it in the processing of single concepts, in a specificity modulated way [45–47]. In our MEG data, single word specificity by itself did not reliably affect LATL activity, although we saw weak trends. Instead, specificity interacted with composition.

This interaction seems to be driven by the way the inte- gration of the first word affects the feature space of the second word. The sparser the feature space of the second word (i.e. the more general its meaning), the more notable both the addition of new features both (cf. Weber’s Law [48]).

Thus, a more general head would show a larger composition effect. Similarly, a more specific modifier can be thought of as contributing a more notable addition of features to the head, resulting in a bigger effect for more specific modifiers. This idea is shown in picture form in figure 2b. In the future, we hope to test this hypothesis in a more computationally precise way.

To summarize: our conceptual specificity studies substan- tially narrowed down the hypothesis space capable of

explaining the LATL effect. Now syntactic and logico-semantic composition was effectively off the table, at least in any familiar variety. Neither of them should be sensitive to conceptual specificity in the way just outlined. But it was also not the case that there was some extant model on the shelf that was capable of accounting for the findings. The general process of‘conceptual combination’ does not explain the data, either, since clearly the mechanism does not operate equally for all cases of conceptual combination. A more fine-grained investigation is needed.

4. Step 3: distinguishing between varieties of conceptual combination

(a) Lexical versus functional meanings

If the LATL reflects some function that combines concepts into more complex conceptual representations, what types of meanings can serve as the input to this function? One obvious question is whether the input items must be open class words, like dog, house, love or run, or whether conceptually un-meaty function words would do as well, like the, as or some? While the question is obvious, it is methodologi- cally tricky, since the functional lexicon is indeed a‘closed class’—that is, it is quite restricted and each function word is potentially its own unique element. For example, figuring out what open class item would create a good comparison for the very short and frequent‘the’ is challenging. Neverthe- less, we have managed to make some headway on this general topic (figure 3).

stimulus utterance green twos

thirty-two

three, twos

three, two

LATL effect during utterance planning

modification: LATL increase over List

complex number: LATL increase over List

numeral quantification:

no LATL increase over List

List control

same star

2122

2122 3218

3218

production of number words

comprehension of functional adjectives (b)

(a)

Figure 3. Composition of function words. Though sensitive to conceptual specificity, the LATL can also operate on items from the functional lexicon. For example, positive LATL effects have been shown for the planning of complex number terms in production (a) [49] and for the comprehension of functional adjectives such as same (b) [50]. The adjective same imports features from the context to the subsequent noun. Here, the context is a picture of a green striped star and thus same imports the features green and striped onto the noun star. (Online version in colour.)

ro yalsocietypublishing.org/journal/rs tb Phil. Trans. R. Soc. B 375 : 20190299

5

Downloaded from https://royalsocietypublishing.org/ on 05 November 2021

(6)

One pair of studies used picture naming (recall that LATL effects are observed in production as well), in which subjects named pictures of one, two, three or four coloured objects either as colour–object phrases (blue cups) or as number phrases (two cups). As we measured MEG activity between picture onset and articulation onset, the planning of the number phrase did not engage the LATL at all, while prepar- ing the colour–object phrase yielded the usual ‘composition effect’ [51]. We also learned that the disinterest of the LATL in the number phrase was not because it did not like numbers: when asked to name sequences of numbers as either quantificational phrases (12335→ ‘two threes’), as complex number terms (12335→ ‘twenty-three’) or as just lists (12335→ ‘two, three’), the complex number terms did engage the LATL while the quantificational phases did not, when compared with the lists [49]. This tells us that what matters is the mode of composition, not the inherent nature of the input items to composition. When the numbers were used for quantification, the LATL was not engaged, but when a number term provided a type of‘feature’ for another number term, it did. One issue in research on conceptual combination has been that the term‘conceptual combination’

is quite ill-defined—partly because the term ‘concept’ is so hard to define. But here, the brain seems to be telling us how it cuts the pie. For the purposes of LATL-housed conceptual combination,‘twenty-three’ is in, but ‘two threes’ is out.

The number studies taught us that for the LATL, what matters is the way a word contributes to the meaning of the whole, as opposed to the inherent properties of the word itself. And, in fact, it turns out that the word itself can be totally void of conceptual features, as long as it can reference conceptual features in the context, importing them into the combination. This was shown by Zhang & Pylkkänen, who studied the functional adjective ‘same’ and its ability to serve as an input item to LATL composition [50,52]. If the LATL only took open class conceptual items as input, same would probably not be able to drive it, as it has no conceptual features of its own. However, same can function as a powerful‘importer’ of conceptual features from the context, despite its own semantic lightness. If I am wearing a red silk scarf with white polka-dots and you tell me you have the same scarf, I know a lot about your scarf although you just used one adjective, same. Thus, if the LATL does not care about the inherent semantic properties of the input items but rather just their impact on the conceptual combination, then same might be a robust driver of LATL composition. This is exactly what we found: when same imported both a colour and a size feature from a picture context when composing with car, same car elicited a larger LATL effect than green car, in which only the colour feature is added.

As we progress towards ‘mechanism’, we now know that upon encountering a word in its linguistic context, the computation reflected in the LATL seems to

(a) consult the syntax (so it can draw a difference between

‘twenty threes’ and ‘twenty-three’);

(b) be insensitive to the source of the conceptual features carried by the linguistic context (i.e. the features can come from, say, a picture, as in the study on same);

(c) combine conceptual feature spaces as opposed to

‘words’, such that it is the changes in the feature space that primarily modulate the measured MEG activity (as evidenced by our specificity findings). More robust changes yield a bigger LATL signal.

(b) Open questions

Many questions remain open. Here I will list a few of them.

(i) What is a‘feature’? Going forward, it will be important to understand what actually counts as a‘feature’ for LATL computation. So far we have no systematic investigation of this. Though the original red–boat paradigm used highly concrete and imageable concepts (colours and objects), later studies have shown similar effects in rather different domains, including manner modification of verbs (slowly walks—still concrete but less so [28]) and the formation of complex number terms (twenty-three [49]). Finding restrictions within open class concepts that are not explainable in terms of specificity (which we to some extent understand) would be fascinating.

(ii) Language specificity. Another important question is the extent to which the LATL computes conceptual meaning across domains, or whether it needs its concepts to be conveyed by words. So far, in our work, the latter has been true: with just pictorial stimuli, we have not been able to elicit combinatory effects in the LATL [24,25,53].

(iii) The right ATL. We will also need to characterize the relationship of the LATL to its right hemisphere homol- ogue, which I have not discussed here but which does respond to composition with a somewhat different profile [28,54]. Relating to the domain generality question above, a large meta-analysis of haemodynamic studies on conceptual processing found that ATL activity is more likely to be left lateralized when the input is a word [55]. This somewhat converges with our inability to engage the LATL without words [24,25,53].

(iv) Function as informed by connectivity. Our model will also need to be consistent with the general connectivity profile of the ATL. The connectivities of the left and right ATLs are somewhat different [56]. Though both have robust connections to the occipital pole, middle and inferior temporal cortex, and to the ventral prefrontal cortex, the connectivity of the right ATL is stronger to ventral prefrontal cortex than that of the left ATL. The left ATL, on one hand, connects more strongly to the Inferior frontal gyrus, consistent with an increased role in language.

Ultimately, this connectivity profile should inform our functional understanding of the ATLs within language and beyond.

5. Zooming out and scaling up (a) Larger expressions

Having gained some understanding of the neural correlates of phrasal composition in minimal phrases, a natural next question is how these effects manifest in full sentences. In one such‘scaling up’ study, we investigated the interaction of conceptual specificity and composition in a sentence context, varying the specificity of a complex subject and testing for the effect of that manipulation on LATL activity elicited by a subsequent verb [50,52]. Activity at a sentence final verb indeed showed a pattern similar to what we had seen for phrases: when more specific subjects composed with the verb, LATL activity was higher (cf. figure 2b). This study also showed that the calculation of the relevant specificity value not only takes into account the specificity of the head noun in the subject phrase (say boat versus canoe) but is also

ro yalsocietypublishing.org/journal/rs tb Phil. Trans. R. Soc. B 375 : 20190299

6

Downloaded from https://royalsocietypublishing.org/ on 05 November 2021

(7)

affected by negation, which we built into the determiner position of the subjects (a boat versus no boat). Thus, as already indicated by the results on same discussed above, it is not the lexical/functional contrast that matters for the LATL, but whether the items affect the specificity/information density of the expression.

The LATL has also been studied in several experiments using naturalistic, narrative stimuli, i.e. ‘real’ language. In this approach, running text is tagged word by word for a variety of linguistic computations, ideally covering all representational levels. For proper reviews, see Brennan [57]

and Pylkkänen & Brennan [58], but the upshot is that the LATL clearly correlates with the number of combinatory steps word by word in a narrative, both when measured with fMRI [39,59] and with MEG [59]. When different models of composition are compared with each other, models with some amount of bottom-up information fare better than fully top-down ones in accounting for activity in the LATL and surrounding regions [60]. This finding is broadly compatible with our results on minimal phrases, which also are bottom-up results in that they reflect activity on the second word of the phrase and not predictive activity on the first word. It has also been shown that models incor- porating richer and more abstract grammatical information account for LATL activity better than ones with a simpler phrase-structure grammar [61]. Relating this finding to our minimal phrase work would require more detailed investi- gations into exactly what properties of the more abstract grammars are driving the better fits. In general, naturalistic studies have so far solely operated with models that are

already on the ‘shelf’ as opposed to creating new models, guided by the brain data. To move towards more mechanistic understanding, this work should always take the best model, probe into it further—what are its strengths and weak- nesses?—and try to create a new model that beats it. This would be parallel to how we have proceeded in the con- trolled designs: new experiments probe finer contrasts than the previous ones. In our data, the LATL does not track a computation that is already on the ‘shelf’, so comparing extant models will probably not get us to the finish line.

Instead, we need to formulate new hypotheses about the nature of the computation.

(b) LATL within the ‘language network’

Finally, what is the general shape of the broader network within which the LATL operates? The most systematic work on the ‘language network’ has been conducted by Evelina Fedorenko’s group using fMRI and contrasts such as sentences versus non-words or narratives versus degraded speech to extract a spatial representation of the whole language network [62]. This method systematically reveals a left temporal swathe of activation extending from the left anterior temporal cortex all the way to the angular gyrus as well as left frontal activation including areas of both inferior and middle frontal cortex.

As a comparison, figure 4 summarizes effect loci of various specific computations studied with MEG in our laboratory, shown here for the visual modality and depicted along a time-line starting from word onset. The left temporal swathe and inferior frontal cortex are seen here as well, with the effects of

argument structure

LATL combinatory effects (sem)

combi

combi reference

dep

visual word form recognition

(fusiform gyrus)

structtururureee e ord ion rm )

mapping of form to meaning (middle posterior

temporal cortex)

vmPFC combinatory

effects (sem) long distance

dependencies

effects of reference resolution

0 100 200 300 400 MSEC

word

pM/STG combinatory effects (syn?)

arg combi meaning form lateral

medial

Figure 4. Summary of MEG results on various stages of language comprehension during reading. The LATL combinatory activity is situated in the middle of a cascade of temporally overlapping processes, including activation of argument structure representations in the angular and supramarginal gyri (dark blue, [63]), initial stages of retrieval operations in left inferior frontal cortex if the current item is the second element of a long-distance dependency (orange, [64]) and posterior temporal activity potentially reflecting more structurally based combinatory operations ( purple, [65]). This complex processing stage at 200 –250 ms is preceded by visual word form recognition in the fusiform gyrus, including form-based decomposition of morphologically complex words (turquoise, [66]), and followed by N400 type lexical access effects in middle temporal cortex, reflected in MEG by so-called M350 activity (light green, [67]). If the ‘centre of gravity’ for lexical–semantic effects is around 350 ms, then the sensitivity of the LATL to word meaning and argument structure at approximately 200 ms is a puzzle. One possibility is that semantic access is, in fact, a gradual process unfolding over time, with initial stages occurring well before the middle temporal ‘M350’ [28,44]. On such an account, the light green processing stage would reflect ‘deeper’ semantic access (cf. [16]). Slightly after the M350 lexical access stage, we observe effects of linking word meanings to referents in the context, i.e. reference resolution (yellow, [68,69]), and overlapping with this, the second stage of semantic combinatory effects in the vmPFC discussed in the main text (brown) (syn = syntactic; sem = semantic). pM/STG, posterior middle/superior temporal gyrus. (Online version in colour.)

ro yalsocietypublishing.org/journal/rs tb Phil. Trans. R. Soc. B 375 : 20190299

7

Downloaded from https://royalsocietypublishing.org/ on 05 November 2021

(8)

LATL corresponding to the work on composition discussed in this paper, middle temporal cortex representing effects of lexical level manipulations [67] and the most posterior centre a recent result aiming to vary syntactic composition while keeping constant the string of concepts described by open class items [65]. In our work, the left IFG (LIFG) has shown effects of retrieval during the processing of long-distance dependencies (red blob in figure 4; [64,70]), consistent with a large haemodynamic literature, but no effects of basic composition, at least with the analysis methods we have used so far.

This is in contrast with two-word composition work in fMRI [71,72], but in addition to the difference in the measured signal, these studies have used grammaticality judgements as of the experimental task and therefore, despite similarities in the designs, there are still many differences between the two sets of studies. To unpack the relationship, one would need at least parallel MEG and fMRI studies using both designs. But even this is complicated since the picture match- ing task used in the MEG studies would not be appropriate in fMRI, as it would not be possible to distinguish the task por- tion of the trial from the processing of the linguistic stimulus. The more straightforward part of the project would be assessing whether LIFG effects might emerge in MEG when a grammaticality judgement task is used.

The two additional regions revealed by our work in contrast with the fMRI language map are the vmPFC, as discussed above, and the medial parietal cortex, revealed in a series of studies aimed at identifying neural correlates of reference resolution [68,69]. Importantly, both of these regions also belong to the default network, comprising a set of regions acti- vated when subjects are left to just think to themselves [73].

Meta-analyses focused on fMRI tasks of semantic processing have further shown that the default network and the‘semantic network’ are highly similar [15]. Thus, despite the absence of these regions in classic views of the language network, we have a sizeable body of work implicating both anterior and posterior regions of the medial wall in higher-level—perhaps domain general—processes that participate in language.

(c) Localized versus distributed representation and processing

Finally, the work discussed here only represents a subset of the possible analyses one can conduct on neural measures and thus the emerging picture is accordingly limited. In particular, the focus above has been on relatively narrowly defined effects that show a reasonably high degree of localization, that is, they are not distributed across large sets of regions. This type of work needs to be complemented by characterizations of more distributed processes and representations. For example, lexical representations are known to have highly

distributed characteristics and thus to understand their time course and evolution during composition we will need tech- niques such as multivariate pattern analysis. Early work using this approach has characterized, for example, how the representations of adjectives activate and deactivate during composition with subsequent nouns [74]. This work uses vector representations of word semantics, based on the co- occurrence of words in documents across a dataset of millions of webpages [75]. A goal for future work is to understand the interplay between localized, integrative hubs and distributed representations of stored lexical meanings (cf. [76]).

6. Conclusion

This summary article has addressed the question‘If you look for composition in the brain, armed with a linguist’s initial set of assumptions and biases, what do you find?’ The answer we have today is that we have not found evidence of the computations that we primarily set out to look for, that is, syntactic and logico-semantic composition, the modes of composition that linguistic theories are mostly about. Instead, as our most robust result, we have found evidence of a much more conceptually driven process, localized in the LATL. This forces us to think about the system differently from our initial inclinations and shows that the neural basis of syntax is a much harder problem than one might have initially thought.

Thus, for syntactic composition, we are still at Step 1 (cf. [65]).

One reason to believe that basic composition may be the type of robust and redundant system discussed here—with multiple combinatory routines contributing to the composition of even a single phrase—is that it is rarely impaired by focal brain damage. Though impairments of lexical access and the processing of more complex syntax are classic patient profiles, the ability to form simple two- word phrases usually remains, unless the patient has massive damage leading to so-called global aphasia, in which all of language is gone. Though impressionistic, this may be an important clue to the functional organization of the basic composition system.

Data accessibility.This article has no additional data.

Competing interests.I declare I have no competing interests.

Funding.Support for the writing of this review and research summarized within it was provided by the National Science Foundation grant no. BCS-1221723 and grant no. G1001 from the NYUAD Institute, New York University Abu Dhabi.

Acknowledgements.The red–boat paradigm was a simple and elegant idea by my former student Douglas Bemis, without whom most of the work described in this article probably would not have hap- pened. Other critical contributions are thanks to the creativity and curiosity of Jonathan Brennan, Masha Westerlund, Linmin Zhang, Songhee Kim, Esti Blanco-Elorrieta and Graham Flick.

References

1. Marr D. 1982 Vision: a computational investigation into the human representation and processing of visual information. San Francisco, CA: W. H.

Freeman.

2. Schoffelen JM, Oostenveld R, Lam NH, Uddén J, Hultén A, Hagoort P. 2019 A 204-subject multimodal neuroimaging dataset to study

language processing. Scient. Data 6, 17. (doi:10.

1038/s41597-019-0020-y)

3. Matchin W, Brodbeck C, Hammerly C, Lau E. 2019 The temporal dynamics of structure and content in sentence comprehension: evidence from fMRI- constrained MEG. Hum. Brain Mapp. 40, 663–678.

(doi:10.1002/hbm.24403)

4. Neufeld C, Kramer SE, Lapinskaya N, Heffner CC, Malko A, Lau EF. 2016 The electrophysiology of basic phrase building. PLoS ONE 11, e0158446.

(doi:10.1371/journal.pone.0158446)

5. Pylkkänen L. 2008 Mismatching meanings in brain and behavior. Langu. Linguist. Compass 2, 712–738.

(doi:10.1111/j.1749-818X.2008.00073.x)

ro yalsocietypublishing.org/journal/rs tb Phil. Trans. R. Soc. B 375 : 20190299

8

Downloaded from https://royalsocietypublishing.org/ on 05 November 2021

(9)

6. Pylkkänen L, McElree B. 2006 The syntax-semantics interface: on-line composition of sentence meaning.

In Handbook of psycholinguistics, 2nd edn (eds M Traxler, MA Gernsbacher), pp. 539–579. New York, NY: Academic Press. (doi:10.1016/B978-012369374- 7/50015-8)

7. Pylkkänen L, McElree B. 2007 An MEG study of silent meaning. J. Cogn. Neurosci. 19, 1905–1921.

(doi:10.1162/jocn.2007.19.11.1905)

8. Brennan J, Pylkkänen L. 2008 Processing events:

behavioral and neuromagnetic correlates of aspectual coercion. Brain Lang. 106, 132–143.

(doi:10.1016/j.bandl.2008.04.003)

9. Pylkkänen L, Martin AE, McElree B, Smart A. 2009 The anterior midline field: coercion or decision making? Brain Lang. 108, 184–190. (doi:10.1016/

j.bandl.2008.06.006)

10. Brennan J, Pylkkänen L. 2010 Processing psych verbs: behavioural and MEG measures of two different types of semantic complexity. Langu. Cogn.

Process. 25, 777–807. (doi:10.1080/

01690961003616840)

11. Baggio G, Choma T, Van Lambalgen M, Hagoort P.

2010 Coercion and compositionality. J. Cogn.

Neurosci. 22, 2131–2140. (doi:10.1162/jocn.2009.

21303)

12. Kuperberg GR, Choi A, Cohn N, Paczynski M, Jackendoff R. 2010 Electrophysiological correlates of complement coercion. J. Cogn. Neurosci. 22, 2685–2701. (doi:10.1162/jocn.2009.21333) 13. Husband EM, Kelly LA, Zhu DC. 2011 Using

complement coercion to understand the neural basis of semantic composition: evidence from an fMRI study. J. Cogn. Neurosci. 23, 3254–3266.

(doi:10.1162/jocn_a_00040)

14. Bemis DK, Pylkkänen L. 2011 Simple composition:

a magnetoencephalography investigation into the comprehension of minimal linguistic phrases.

J. Neurosci. 31, 2801–2814. (doi:10.1523/

JNEUROSCI.5003-10.2011)

15. Binder JR, Desai RH, Graves WW, Conant LL. 2009 Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb. Cortex 19, 2767–2796. (doi:10.1093/

cercor/bhp055)

16. Binder JR, Desai RH. 2011 The neurobiology of semantic memory. Trends Cogn. Sci. 15, 527–536.

(doi:10.1016/j.tics.2011.10.001)

17. Bemis DK, Pylkkänen L. 2012 Basic linguistic composition recruits the left anterior temporal lobe and left angular gyrus during both listening and reading. Cereb. Cortex 23, 1859–1873. (doi:10.1093/

cercor/bhs170)

18. Pylkkänen L, Bemis DK, Elorrieta EB. 2014 Building phrases in language production: an MEG study of simple composition. Cognition 133, 371–384.

(doi:10.1016/j.cognition.2014.07.001)

19. Mazoyer BM et al. 1993 The cortical representation of speech. J. Cogn. Neurosci. 5, 467–479. (doi:10.

1162/jocn.1993.5.4.467)

20. Stowe LA, Broere CA, Paans AM, Wijers AA, Mulder G, Vaalburg W, Zwarts F. 1998 Localizing components of a complex task: sentence

processing and working memory. Neuroreport 9, 2995–2999. (doi:10.1097/00001756-199809140- 00014)

21. Humphries C, Binder JR, Medler DA, Liebenthal E.

2006 Syntactic and semantic modulation of neural activity during auditory sentence comprehension.

J. Cogn. Neurosci. 18, 665–679. (doi:10.1162/jocn.

2006.18.4.665)

22. Rogalsky C, Saberi K, Hickok G. 2009 Temporal and structural contributions to activation of anterior temporal sentence processing regions: an fMRI study. Structure 55, 9. (doi:10.1016/s1053- 8119(09)71759-8)

23. Brennan J, Pylkkänen L. 2012 The time-course and spatial distribution of brain activity associated with sentence processing. Neuroimage 60, 1139–1148.

(doi:10.1016/j.neuroimage.2012.01.030) 24. Blanco-Elorrieta E, Ferreira VS, Del Prato P,

Pylkkänen L. 2018 The priming of basic combinatory responses in MEG. Cognition 170, 49–63. (doi:10.1016/j.cognition.2017.09.010) 25. Blanco-Elorrieta E, Kastner I, Emmorey K, Pylkkänen

L. 2018 Shared neural correlates for building phrases in signed and spoken language. Scient. Rep.

8, 5492. (doi:10.1038/s41598-018-23915-0) 26. Bemis DK, Pylkkänen L. 2013 Flexible composition:

MEG evidence for the deployment of basic combinatorial linguistic mechanisms in response to task demands. PLoS ONE 8, e73949. (doi:10.1371/

journal.pone.0073949)

27. Westerlund M, Kastner I, Al Kaabi M, Pylkkänen L.

2015 The LATL as locus of composition: MEG evidence from English and Arabic. Brain Lang. 141, 124–134. (doi:10.1016/j.bandl.2014.12.003) 28. Kim S, Pylkkänen L. 2019 Composition of event

concepts: evidence for distinct roles for the left and right anterior temporal lobes. Brain Lang. 188, 18–27. (doi:10.1016/j.bandl.2018.11.003) 29. Brooks TL, Cid de Garcia D. 2015 Evidence for

morphological composition in compound words using MEG. Front. Hum. Neurosci. 9, 215. (doi:10.

3389/fnhum.2015.00215)

30. Flick G, Oseki Y, Kaczmarek AR, Al Kaabi M, Marantz A, Pylkkänen L. 2018 Building words and phrases in the left temporal lobe. Cortex 106, 213–236.

(doi:10.1016/j.cortex.2018.06.004)

31. Westerlund M, Pylkkänen L. 2014 The role of the left anterior temporal lobe in semantic composition vs. semantic memory. Neuropsychologia 57, 59–70.

(doi:10.1016/j.neuropsychologia.2014.03.001) 32. Zhang L, Pylkkänen L. 2015 The interplay of

composition and concept specificity in the left anterior temporal lobe: an MEG study. Neuroimage 111, 228–240. (doi:10.1016/j.neuroimage.2015.

02.028)

33. Partee BH. 1987 Noun phrase interpretation and type-shifting principles. In Studies in discourse representation theory and the theory of generalized quantifiers (eds J Groenendijk, D de Jongh, M Stokhof ), vol. 5, p. 21. Dordrecht, The Netherlands: Foris.

34. Parsons T. 1990 Events in the semantics of (Current Studies in Linguistics. vol. 334). Cambridge, MA: MIT Press.

35. Nathan LE. 2006 On the interpretation of concealed questions. Doctoral dissertation, Massachusetts Institute of Technology.

36. Hampton JA. 1997 Conceptual combination. In Knowledge, concepts, and categories (eds K Lamberts, D Shanks), pp. 133–159. Hove, UK:

Psychology Press.

37. Murphy GL. 1990 Noun phrase interpretation and conceptual combination. J. Mem. Langu. 29, 259–288. (doi:10.1016/0749-596X(90)90001-G) 38. Smith EE, Osherson DN. 1984 Conceptual

combination with prototype concepts. Cogn. Sci. 8, 337–361. (doi:10.1207/s15516709cog0804_2) 39. Brennan J, Nir Y, Hasson U, Malach R, Heeger DJ,

Pylkkänen L. 2012 Syntactic structure building in the anterior temporal lobe during natural story listening. Brain Lang. 120, 163–173. (doi:10.1016/

j.bandl.2010.04.002)

40. Friederici AD. 1995 The time course of syntactic activation during language processing: a model based on neuropsychological and neurophysiological data. Brain Lang. 50, 259–281. (doi:10.1006/brln.

1995.1048)

41. Baron SG, Osherson D. 2011 Evidence for conceptual combination in the left anterior temporal lobe.

Neuroimage 55, 1847–1852. (doi:10.1016/j.

neuroimage.2011.01.066)

42. Coutanche MN, Thompson-Schill SL. 2014 Creating concepts from converging features in human cortex.

Cereb. Cortex 25, 2584–2593. (doi:10.1093/cercor/

bhu057)

43. Wilson SM, DeMarco AT, Henry ML, Gesierich B, Babiak M, Mandelli ML, Miller BL, Gorno-Tempini ML. 2014 What role does the anterior temporal lobe play in sentence-level processing? Neural correlates of syntactic processing in semantic PPA. J. Cogn.

Neurosci. 26, 970–985. (doi:10.1162/jocn_a_

00550)

44. Ziegler J, Pylkkänen L. 2016 Scalar adjectives and the temporal unfolding of semantic composition: an MEG investigation. Neuropsychologia 89, 161–171.

(doi:10.1016/j.neuropsychologia.2016.06.010) 45. Warrington EK. 1975 The selective impairment of

semantic memory. Q. J. Exp. Psychol. 27, 635–657.

(doi:10.1080/14640747508400525)

46. Hodges JR, Patterson K, Oxbury S, Funnell E. 1992 Semantic dementia. Progressive fluent aphasia with temporal lobe atrophy. Brain 115, 1783–1806.

(doi:10.1093/brain/115.6.1783)

47. Rogers TT, Hocking J, Noppeney U, Mechelli A, Gorno-Tempini ML, Patterson K, Price CJ. 2006 Anterior temporal cortex and semantic memory:

reconciling findings from neuropsychology and functional imaging. Cogn. Affect. Behav. Neurosci.

6, 201–213. (doi:10.3758/CABN.6.3.201) 48. Fechner GT, Howes DH, Boring EG. 1966 Elements of

psychophysics, vol. 1. New York, NY: Holt, Rinehart and Winston.

49. Blanco-Elorrieta E, Pylkkänen L. 2016 Composition of complex numbers: delineating the computational role of the left anterior temporal lobe. Neuroimage 124, 194–203. (doi:10.1016/j.neuroimage.2015.

08.049)

ro yalsocietypublishing.org/journal/rs tb Phil. Trans. R. Soc. B 375 : 20190299

9

Downloaded from https://royalsocietypublishing.org/ on 05 November 2021

(10)

50. Zhang L, Pylkkänen L. 2018 Composing lexical versus functional adjectives: evidence for uniformity in the left temporal lobe. Psychon. Bull. Rev. 25, 2309–2322. (doi:10.3758/s13423-018-1469-y) 51. Del Prato P, Pylkkänen L. 2014 MEG evidence for

conceptual combination but not numeral quantification in the left anterior temporal lobe during language production. Front. Psychol. 5, 524.

(doi:10.3389/fpsyg.2014.00524)

52. Zhang L, Pylkkänen L. 2018 Semantic composition of sentences word by word: MEG evidence for shared processing of conceptual and logical elements. Neuropsychologia 119, 392–404. (doi:10.

1016/j.neuropsychologia.2018.08.016)

53. Bemis DK, Pylkkanen L. 2013 Combination across domains: an MEG investigation into the relationship between mathematical, pictorial, and linguistic processing. Front. Psychol. 3, 583. (doi:10.3389/

fpsyg.2012.00583)

54. Poortman EB, Pylkkänen L. 2016 Adjective conjunction as a window into the LATL’s contribution to conceptual combination. Brain Lang.

160, 50–60. (doi:10.1016/j.bandl.2016.07.006) 55. Rice GE, Lambon Ralph MA, Hoffman P. 2015 The

roles of left versus right anterior temporal lobes in conceptual knowledge: an ALE meta-analysis of 97 functional neuroimaging studies. Cereb. Cortex 25, 4374–4391. (doi:10.1093/cercor/bhv024) 56. Papinutto N et al. 2016 Structural connectivity of

the human anterior temporal lobe: a diffusion magnetic resonance imaging study. Hum. Brain Mapp. 37, 2210–2222. (doi:10.1002/hbm.23167) 57. Brennan J. 2016 Naturalistic sentence

comprehension in the brain. Langu. Linguist.

Compass 10, 299–313. (doi:10.1111/lnc3.12198) 58. Pylkkänen L, Brennan JR. 2019 Composition:

the neurobiology of syntactic and semantic structure building. In The cognitive neurosciences, 6th edn (eds M Gazzaniga, R Mangun, D Poeppel), pp. 861–869.

Cambridge, MA: MIT Press.

59. Brennan JR, Pylkkänen L. 2017 MEG evidence for incremental sentence composition in the anterior temporal lobe. Cogn. Sci. 41, 1515–1531. (doi:10.

1111/cogs.12445)

60. Nelson MJ et al. 2017 Neurophysiological dynamics of phrase-structure building during sentence processing. Proc. Natl Acad. Sci. USA 114, E3669–E3678. (doi:10.1073/pnas.1701590114) 61. Brennan JR, Stabler EP, Van Wagenen SE, Luh WM,

Hale JT. 2016 Abstract linguistic structure correlates with temporal activity during naturalistic comprehension. Brain Lang. 157, 81–94. (doi:10.

1016/j.bandl.2016.04.008)

62. Scott TL, Gallée J, Fedorenko E. 2017 A new fun and robust version of an fMRI localizer for the frontotemporal language system. Cogn. Neurosci. 8, 167–176. (doi:10.1080/17588928.2016.1201466) 63. Williams A, Reddigari S, Pylkkänen L. 2017

Early sensitivity of left perisylvian cortex to relationality in nouns and verbs. Neuropsychologia 100, 131–143. (doi:10.1016/j.neuropsychologia.

2017.04.029)

64. Leiken K, McElree B, Pylkkänen L. 2015 Filling predictable and unpredictable gaps, with and without similarity-based interference: evidence for LIFG effects of dependency processing.

Front. Psychol. 6, 1739. (doi:10.3389/fpsyg.

2015.01739)

65. Flick G, Pylkkänen L. 2018 Isolating syntax in natural language: MEG evidence for an early contribution of left posterior temporal cortex.

BioRxiv, 439158. (doi:10.1101/439158)

66. Zweig E, Pylkkänen L. 2009 A visual M170 effect of morphological complexity. Lang. Cogn. Process. 24, 412–439. (doi:10.1080/01690960802180420) 67. Pylkkänen L, Marantz A. 2003 Tracking the time

course of word recognition with MEG. Trends Cogn.

Sci. 7, 187–189. (doi:10.1016/S1364- 6613(03)00092-5)

68. Brodbeck C, Gwilliams L, Pylkkänen L. 2016 Language in context: MEG evidence for modality-

general and -specific responses to reference resolution. eNeuro 3, ENEURO.0145-16.2016.

(doi:10.1523/ENEURO.0145-16.2016)

69. Brodbeck C, Pylkkänen L. 2017 Language in context:

characterizing the comprehension of referential expressions with MEG. Neuroimage 147, 447–460.

(doi:10.1016/j.neuroimage.2016.12.006) 70. Leiken K, Pylkkänen L. 2014 MEG evidence that

the LIFG effect of object extraction requires similarity-based interference. Lang. Cogn.

Neurosci. 29, 381–389. (doi:10.1080/01690965.

2013.863369)

71. Schell M, Zaccarella E, Friederici AD. 2017 Differential cortical contribution of syntax and semantics: an fMRI study on two-word phrasal processing. Cortex 96, 105–120. (doi:10.1016/

j.cortex.2017.09.002)

72. Zaccarella E, Friederici AD. 2015 Merge in the human brain: a sub-region based functional investigation in the left pars opercularis. Front.

Psychol. 6, 1818. (doi:10.3389/fpsyg.2015.01818) 73. Buckner RL, Andrews-Hanna JR, Schacter DL. 2008

The brain’s default network. Ann. N Y Acad. Sci.

1124, 1–38. (doi:10.1196/annals.1440.011) 74. Fyshe A, Sudre G, Wehbe L, Rafidi N, Mitchell TM.

2019 The lexical semantics of adjective–noun phrases in the human brain. Hum. Brain Mapp. 15, 4457–4469. (doi:10.1002/hbm.24714)

75. Fyshe A, Murphy B, Talukdar P, Mitchell T. 2013 Documents and dependencies: an exploration of vector space models for semantic composition. In Proc. 17th Conf. Computational Natural Language Learning, Sofia, Bulgaria, August 2013 (eds J Hockenmaier, S Riedel), pp. 84–93. Stroudsburg, PA: Association for Computational Linguistics.

76. Mollo G, Cornelissen PL, Millman RE, Ellis AW, Jefferies E. 2017 Oscillatory dynamics supporting semantic cognition: MEG evidence for the contribution of the anterior temporal lobe hub and modality-specific spokes. PLoS ONE 12, e0169269.

(doi:10.1371/journal.pone.0169269)

ro yalsocietypublishing.org/journal/rs tb Phil. Trans. R. Soc. B 375 : 20190299

10