Multivoxel pattern analysis

Univariate results revealed that whereas representations for hands and tools share anatomical territories in the left LOTC and left aIPS suggesting a close functional link between these two areas, in the lFG representations for these categories are largely independent of each other reflecting the well documented dissociation of animate versus inanimate domains (Kriegeskorte, et al., 2008; Mahon, et al., 2009). Here, to examine whether distribution of response patterns of these object categories reflects univariate results, I used multivoxel pattern analysis (MVPA) to investigate similarity of hand, tool, body, and non-graspable objects responses in the left LOTC, left aIPS and left FG. As described in the previous chapter, methods such as MVPA are well suited to investigate similarities in representations between object categories that share overlapping territories. To this aim MVPA will be applied to investigate the following experimental questions

115

in more details. (1) Is the large anatomical correspondence between hand and tool representations in the left aIPS (reported in current Study 3) also reflected in a similar distribution of their response patterns? (2) Conversely, as suggested by univariate results (see Figure 4.6), are hands and tools represented independently in lFG? In addition, as shown in Study 2, distribution of response patterns of tools and hands are expected to be highly correlated with each other relative to other categories (e.g., bodies) in the left LOTC.

To test these experimental questions, the left LOTC, left aIPS and left FG were defined in each individual subject using contrasts described in the methods section (Materials and Methods). Figure 4.9a shows the three ROIs in a representative subject. Mean cluster size (mm3) and Talairach coordinates (x, y, z) of each ROI were as follows: left LOTC (3384 mm3; x y z = -48, -71, -2), left aIPS (2327 mm3; x y z = -40, -45, 43) and left FG (5912 mm3; x y z = -38, -43, -19). Within these ROIs, we then correlated the voxel-wise patterns of activity between each of the conditions of interest across the two runs (e.g., hands run1 – tools run2). Figure 4.9c shows the 4x4 correlation matrices averaged across subjects, and Table 4.3 reports mean correlation values (averaged across the two runwise comparisons).

Results in the left LOTC replicated findings reported in Study 2 showing that, although overall responses to hands and bodies were significantly higher than all other categories (see Figure 4.9 b, for all comparisons p<.001), high correlation was found between response pattern distribution to hands and tools (r= 0.09). Indeed, pairwise t-tests revealed significant higher correlation between tools and hands relative to tools and bodies (t(15) =7.99, p<.001, r= -0.57), hands and bodies (t(15) =2.84, p<.01, r= -0.22), hands and non-graspable objects (t(15) =5.06, p<.001, r= -0.34). Interestingly, in the left LOTC, tools were also highly correlated with non- graspable objects (r= 0.20). Indeed, the correlation between tools and non-graspable objects was significantly higher than the correlation between hands and non-graspable objects (t(15) =8.30, p<.001), but did not differ from the correlation found between hands and tools (t(15) =1.38, p>.1). In the left aIPS response patterns to hands and tools did not correlate with each other (r= -0.05, see Figure 4.9 c). Instead, in this region, tools did correlate significantly more strongly with non- graspable objects than with hands (t(15) =2.80, p<.01, r= 0.16). Finally, in the left FG results match univariate analyses showing high correlation between the two animate categories (hands and bodies, r= 0.39) and the two inanimate (tools and non-graspable objects, r= 0.14) categories. Correlations between animate categories (hands and bodies) were significantly higher than all other comparisons (for all comparisons p<.001, see Figure 4.9 c). Similarly, the correlation between

116

inanimate (tools and non-graspable objects) were significantly higher than all other comparisons (for all comparisons p<.006, see Figure 4.9 c).

Figure 4.9. Multi-voxel pattern analysis (MVPA) in Study 3. a. ROIs used for MVPA in a representative subject. ROIs were functionally defined in each individual subject by contrasting the average response to the conditions of interest (hands, tools and bodies) relative to non-graspable objects (left LOTC and left aIPS) and hands, tools, bodies and non-graspable objects relative to baseline (left FG). b. Average activity (%BSC) for each stimulus category extracted from the ROIs used in the MVPA. c. Multivoxel correlation matrices in the left LOCT, left aIPS, and left FG. Activity patterns for each condition were correlated with each other across runs. Each cell of the matrix represents the correlation value (averaged across subjects) for the between-category (off-diagonals) and within-category (diagonal) correlations. Warm colours represent positive correlations and cold colours represent negative correlations.

Overall, multi-voxel pattern analyses confirmed similarity (see also study 2), between distribution of hand and tool response patterns in left LOTC. Instead, within both the left aIPS and left FG representations for hands and tools appear to be largely independent of one another.

117

Table 4.3. Results of multi-voxel pattern analysis in Study 3.

bodies hands tools non-grasp objects

left LOTC bodies 0.89 -0.25 -0.56 -0.17

hands 0.82 0.09 -0.34

tools 0.59 0.20

non-grasp objects 0.46

left aIPS bodies 0.43 0.05 -0.02 0.24

hands 0.65 -0.12 0.00 tools 0.47 0.16 non-grasp objects 0.23 left FG bodies 0.71 0.39 -0.46 -0.48 hands 0.83 -0.20 -0.78 tools 0.45 0.14 non-grasp objects 0.82

Mean correlation values for within-category (e.g., hands-hands) and between-category (e.g., hands-tools) comparisons in the left lateral occipitotemporal cortex (LOTC), left anterior intraparietal cortex (left aIPS) and left fusiform gyrus (FG).

4.5 Discussion

This study investigated whether category related activations for vision of hands and tools are present in cortical areas other than LOTC. Connectivity analysis of the LOTC region, reported as part of Study 2, suggests association to the front-parietal network. The aim of this study was therefore to reveal whether, similar to the left LOTC, hands and tools are represented in parietal regions. Results document three main findings: (1) overlap between hand and tool representations in both the left LOTC and left aIPSv; 2) differential profile for hand responses in parietal areas aIPSd and aIPSv, and (3) anatomical and functional dissociation between hand and tool representations in the fusiform gyrus. I will now expand on each point separately. Firstly, results revealed that the left LOTC and left aIPSv show similar anatomical and functional hand/tool correspondence. The functional connectivity analysis further confirms the finding reported in Study 2, showing existing patterns of functional connectivity between the left LOTC and left aIPSv hand/tool regions. This result further supports the hypothesis advanced in Study 2: functional organisation of hand and tool representations in high-level visual cortex is partly determined by the

118

type of information these objects provide, and reflects non-visual object properties (maybe action- related) processed by specialised regions within the frontoparietal network. This functional specialisation is reflected in functionally connected parietal regions. Interestingly, MVPA results show that in spite of close anatomical overlap between hands and tools in both the left LOTC and left aIPSv, these regions encode hand and tool representations differently. Indeed, whereas hand and tool response patterns are very similarly distributed in the left LOTC (see Study 2 and Study 3), these representations appear to be independent from each other in the left aIPSv. Therefore, although results clearly speak in favour of these regions being part of a common computational network, left LOTC and left aIPSv hand/tool regions might instead represent differential stages of hand/tool information processing. For instance, form-related object representations might be processed within overlapping visual substrates (e.g., left LOTC) and subsequently sent via common pathways to those brain regions that specify the motor repertories associated with hand/tool actions (e.g., left aIPSv). Secondly, visual presentation of hands furthermore elicits distinct fMRI responses in the dorsal and the ventral portion of the aIPS. Interestingly, parietal hand activations, unlike the tool responses, did not show exclusively left hemispheric lateralisation. Moreover, the differential functional profiles of ventral and dorsal aIPS suggest that the two regions subserve different cognitive tasks. On the one hand, given the role played by the superior parietal lobe in visually guided actions, I advance the hypothesis that the dorsal aIPS (hand-responsive) might be involved in providing visual feedback of hands during reach-to-grasp movements. On the other hand, given the role played by the inferior parietal lobe in tool-use, I advance the hypothesis that the ventral aIPS (hand/tool responsive) might be involved in storage of hand/tool action representations. Thirdly, the FG was found to show a large scale dissociation between hand and tool representations. This is in agreement with previous reports showing that object dimensions stored in the fusiform gyrus follow the category domain distinction (i.e. animate versus inanimate).