Crossmodal face recognition

(1)

LEABHARLANN CHOLAISTE NA TRIONOIDE, BAILE ATHA CLIATH TRINITY COLLEGE LIBRARY DUBLIN OUscoil Atha Cliath The University of Dublin

Terms and Conditions of Use of Digitised Theses from Trinity College Library Dublin

Copyright statement

All material supplied by Trinity College Library is protected by copyright (under the Copyright and Related Rights Act, 2000 as amended) and other relevant Intellectual Property Rights. By accessing and using a Digitised Thesis from Trinity College Library you acknowledge that all Intellectual Property Rights in any Works supplied are the sole and exclusive property of the copyright and/or other I PR holder. Specific copyright holders may not be explicitly identified. Use of materials from other sources within a thesis should not be construed as a claim over them.

A non-exclusive, non-transferable licence is hereby granted to those using or reproducing, in whole or in part, the material for valid purposes, providing the copyright owners are acknowledged using the normal conventions. Where specific permission to use material is required, this is identified and such permission must be sought from the copyright holder or agency cited.

Liability statement

By using a Digitised Thesis, I accept that Trinity College Dublin bears no legal responsibility for the accuracy, legality or comprehensiveness of materials contained within the thesis, and that Trinity College Dublin accepts no liability for indirect, consequential, or incidental, damages or losses arising from use of the thesis for whatever reason. Information located in a thesis may be subject to specific use constraints, details of which may not be explicitly described. It is the responsibility of potential and actual users to be aware of such constraints and to abide by them. By making use of material from a digitised thesis, you accept these copyright and disclaimer provisions. Where it is brought to the attention of Trinity College Library that there may be a breach of copyright or other restraint, it is the policy to withdraw or take down access to a thesis while the issue is being resolved.

Access Agreement

By using a Digitised Thesis from Trinity College Library you are bound by the following Terms & Conditions. Please read them carefully.

(2)

(3)

Crossmodal Face Recognition

by

Sarah Jane Casey

A dissertation submitted for the degree o f Doctor o f Philosophy o f the

U niversity o f Dublin, Trinity College, DubHn 2, Ireland.

This research was conducted in the School o f Psychology and the Institute

o f Neuroscience.

(4)

Summary

A lthough all faces share the sam e basic configuration o f features, we are capable o f distinguishing betw een, and recognising hundreds, and even thousands o f faces with rem arkable skill and precision. The perceptual processes, cognitive m echanism s, and neural substrates underpinning visual face recognition have been studied extensively. H ow ever, it is now acknow ledged that the form ation o f coherent percepts o f objects in the environm ent benefits from the m erging o f com plem entary sources o f sensory inform ation about that object. Therefore, researchers have been inspired to venture beyond vision, and exam ine how inform ation from other sensoiy m odalities can facilitate face recognition.

Recently, it has been discovered that the haptic system is capable o f representing and recognising faces. M oreover, one study has provided evidence that inform ation about hum an faces can be shared betw een the visual and haptic systems. This thesis has sought to explore further the extent to w hich face inform ation is shared across the visual and haptic m odalities. The prim ary aim s o f this thesis w ere to gain a better insight into visual and haptic processing sim ilarities that can facilitate crossm odal face recognition, as well as to gain insight into the nature o f face inform ation that can be represented in m em ory across vision and touch.

In this thesis I have used a num ber o f sim ple variations o f the basic crossm odal m atching and recognition paradigm s to investigate crossm odal visual and haptic face recognition. The findings o f this body o f research dem onstrate that face inform ation encoded in one m odality can indeed be readily shared w ith the other m odality for the purposes o f face recognition. Furtherm ore, this sharing o f face inform ation is affected by face fam iliarity, and configural face inform ation appears to m ediate the efficient sharing o f face inform ation across the m odalities. The experim ents reported in this thesis also provide behavioural evidence to suggest that face recognition is not a

(5)

representations o f face shape was not obtained, the experiments herein do provide indirect evidence to suggest that crossmodal face recognition may be mediated by multisensory representations o f face shape. The theoretical implications o f these research findings are discussed in Chapter 5.

Crossmodal face recognition is an emerging area o f research with huge potential and scope for future work. It is clear that a number o f exciting challenges lie ahead for

(6)

Acknowledgements

Throughout the course o f this thesis I have been fortunate to receive guidance, friendship, support, and assistance from the most wonderful group o f people. Therefore I would like to extend my warmest thanks to the following, who have helped me so much over the last few years;

Firstly, I would like to thank my supervisor. Dr. Fiona Newell. She has been a constant source o f encouragement and advice. I have benefited greatly from her support and guidance, and under her watchfial eye I have developed a number of skills, which will no doubt stand me in good stead for many years to come.

Secondly, I would like to thank my Viva Voce Examiners P rof Susan Lederman and Dr. Hugh Garavan, and Dr. John Davenport, Chair, for providing a challenging yet friendly environment for my thesis defence. They put me at ease from the very first moment which made it one o f the most pleasant and rewarding exams I have ever taken!

In addition, I extend my thanks to my appraisers, Drs. Stuart Smith and Hugh Garavan, for their enthusiastic ideas, insights, and advice.

I would like to thank our lab group (Jason, Kylie, Andy, Achille, & Gary) for listening patiently to my various ideas, and providing me with helpful comments and feedback. A special thanks goes to Andy, who had the interesting experience o f sharing an office with me. He has been a good friend (even on the very rare occasions when I was grumpy!), and under his tutelage 1 have become very proficient in Photoshop - so everybody watch out! ©

Three years ago I embarked on this journey with a little group o f fellow explorers. To Fiadhnait, Clare, Julie, and Michelle, thank you for sharing the ups and downs o f PhD life with me.

(7)

Thanks are also ow ed to all those who volunteered their faces for the purposes o f scientific discovery. W ithout you this research w ould not have been possible! Thank you to Tanja K hosraw i for her assistance in creating the haptic face stim uli.

O utside o f T rinity I have been blessed w ith the friendship and support o f m any friends. Thank you to everyone, but m ost especially to Claire, Fiadhnait, Caroline, Orla, Sinead, Ciara, and Ciara C., for their friendship, loyalty, support, patience, and

optim ism .

Finally, I w ould like to thank my fam ily, and m y parents M ichael and Mary. T hey have w orked tirelessly to support me in every possible way, and for that I am m ost gratefiil. Furtherm ore, they have alw ays believed me capable o f achieving my goals, and have instilled in me the m otivation and determ ination to see them through.

(8)

Publications arising from the present work

Published Papers

C asey, S.J. & N ew ell, F.N. (2006) Are representations o f unfam iliar faces independent o f encoding m odality? Neuropsychologia. In press. [Chapter 3].

Casey, S.J. & N ew ell, F.N. (2005). The role o f long-term and short-term fam iliarity in visual and haptic face recognition. E xperim ental Brain Research, 166(3-4), 583- 589. [C hapter 2 J.

Casey, S.J. & N ew ell, F.N. (2003). Haptic ow n-face recognition. Proceedings o f EuroH aptics 2003, Dublin, Ireland, July 2003. (pp. 424-429, peer review ed).

Published A bstracts

Casey, S.J. & N ew ell, F.N. (2005). M odality-specific and m odality-independent processing in face recognition. The Irish Psychologist, 32 (2), p . 119.

Casey, S.J. & N ew ell, F.N. (2005). Face inform ation prom oting efficient crossm odal m atching. The Irish Psychologist, 31(9), p .255.

Casey, S.J, & N ew ell F.N. (2005). E vidence o f configural face processing in crossm odal face m atching. A cta N eurohiologiae Experim entalis, 65, (Supplement), 32.

Casey, S.J. & N ew ell F.N. (2005). W hat inform ation prom otes efficient crossm odal face m atching? Proceedings o f the W orldH aptics Conference, Pisa, Italy, 18'^-20''’ M arch 2005.

C asey, S.J. & N ew ell, F.N. (2004). Fam iliarity and crossm odal face recognition. Proceedings o f the Touch, Temperature, Pleasure, Pain (TTPP), M anchester, UK,

(9)

Casey, S.J, & Newell, F.N. (2004). The role o f familiarity in visual and haptic face recognition. Proceedings o f the 5'^ International Multisensory Research Forum, Sitges, Barcelona, June 2004.

Casey, S.J. & Newell, F.N. (2004). Crossmodal face recognition Proceedings o f the Experimental Psychology Society London Meeting, London, UK, January 2004,

th Casey, S.J. & Newell, F.N. (2003). Haptic own-face recognition. Proceedings o f the 4

(10)

List o f Figures

Figure 1.1; A graphical depiction o f the B ruce and Y oung (1986) fiinctional m odel o f face recognition.

Figure 1.2: An illustration o f the lA C m odel o f face recognition, after Bruce and Y oung (1993).

Figure 1.3: The distributed neural system for face perception after Haxby et al.

(2000).

Figure 1.4: M odels o f crossm odal face recognition.

Figure 1.5: An illustration ol the general m atch-to sam ple crossm odal paradigm adopted m this thesis.

Figure 1.6: M odel o f convergence (M illar, 2000).

Figure 2.1: A sam ple o f the haptic face stimuli used in all of the experim ents in this thesis.

Figure 2.2: M ean percentages o f correct recognition o f one's ow n face across the haptic and visual conditions.

Figure 2.3: M ean percentages o f correct 'hit' responses in the crossm odal m atching task.

Figure 3.1: Plot show ing the m ean percentage o f correct responses for face recognition both w ithin and across m odalities.

Figure 3.2: A n illustration o f the type o f visual stim uli used in the learning and test phases o f E xperim ent 4.

Figure 3.3: Plot show ing the m ean percentage o f co ireci responses for the w ithin m odal and ciossm odal recognition o f faces.

Figure 3.4: An illustration o f the type o f visual face stim uli used in Experim ent 5.

Figure 3.5: Plot show ing the m ean reaction tim es for the correct responses to each o f the visual face types.

Figure 4.1: Plot show ing the m ean percentage o f c o n e c t 'fam iliar' responses for

the crossm odal and within m odal conditions and the correct 'unfam iliar' responses to unfam iliar face stim uli.

(14)

[image:14.551.41.485.52.814.2]

Figure 4.2; Plot show ing the m ean reaction tim es for 'fam iliar' responses for the crossm odal and within m odal conditions and the correct 'unfam iliar' responses to unfam iliar face stim uli. 82 Figure 5.1; M illar's (2000) m odel o f convergence as a m odel o f crossm odal

(15)

List of Tables

Table 5.1: T alairach coordinates (Talairach and T oum oux, 1988) as reported in

[image:15.552.65.540.34.720.2]

(16)

Chapter 1

Introduction

In term s o f evolution, the ability to process faces is an im portant skill, not only for hum ans, but for other species as well. Faces m anifest a vast array o f inform ative signals; they index em otion, facilitate social com m unication, and are a m arker o f personal identity. For exam ple, direction o f eye gaze is indicative o f social attention, facilitating interaction and com m unication betw een individuals (Perrett, Rolls, & Caan, 1982). E m otive facial expressions can tell us if a person is agreeable or poses a threat. They also w arn if som ething in the environm ent m ight be harm ftil, for instance a grim ace o f disgust m ight indicate that available food has spoiled Perhaps m ost im portantly, each face also conveys personal identity, allow ing us to discern friends from foes. A face can be classified into a variety o f categories that correspond to diverse properties o f the individual - age, race, sex, etc. M oreover, all faces share the same basic configuration o f features, yet we are capable o f distinguishing betw een, and recognising, hundreds or even thousands o f faces w ith rem arkable skill and precision. The ability to detect the m inute and subtle differences that allow us to identify one face from another is a testam ent to the form idable capacity o f the hum an brain. This ability appears to be acquired very early during developm ent and does not seem to require formal training or cultivation through special techniques (Sergent and Signoret, 1992).

(17)

has been dem onstrated that the haptic system ' ' is capable o f representing and recognising faces (K ilgour, de G elder, & Lederm an, 2004; K ilgour and Lederm an, 2002) This thesis is concerned with how faces are processed and recogniscd across the visual and haptic m odalities Specifically, I am interested in w hether the same inform ation is processed for recognition by both the visual and haptic system s, and if so, w hether a single com m on m em ory representation underpins recognition in both m odalities. In this chapter I will set the context for my experim ental work by outlining the benefits o f crossm odal recognition relative to unim odal recognition, and by providing a b rie f review o f the central findings from visual and haptic face recognition research.

1.1 C om bining the senses

Traditional approaches to the study o f sensory perception stemmed from a b elief that sensory processing is highly m odular. Thus, a wealth o f literature exists in which the processes o f the visual, auditory, and haptic system s have been studied in isolation. A lthough each sense acquires inform ation that differs in term s o f its physical properties (i e. light waves in vision, sound w aves in audition, and m echanical vibrations and kinesthetic input in haptics are each detected as m odality-specific sensory stim uli), our experience o f objects or events in the environm ent generally does not com prise a collection o f separate sensations (M eredith, 2002). A single event can evoke a num ber o f spatially and tem porally congruent sensory stim uli (e.g. a bouncing ball) and these sensory stim uli are processed sim ultaneously by different sensory receptors, yet we perceive a coherent event. This im plies that our experience o f the w orld reflects the com bination and integration o f m ultiple sources o f sensory inform ation in the brain, resulting in a unified, robust, perception o f objects or events (Calvert, Bram ner, & Iversen, 1998; Driver and Spence, 2000; Ernst and Biilthoff, 2004; Stein and M eredith,

(18)

1993). The ability to integrate inform ation across the senses is advantageous as it allow s us to be m ore certain about objects or events in the environm ent, thus influencing our subsequent responses. For instance, identifying an object in a dim ly lit room is difficult and error prone i f using vision alone, how ever, com bining the im poverished visual inform ation w ith inform ation obtained by actively exploring the object using haptics increases the likelihood of accurately identifying the object. In other w ords, inform ation from both m odalities helps to m inim ise the uncertainty in the inform ation associated w ith either m odality alone, to yield the best approxim ation o f the object (D river and Spence, 2000; Ernst and Biilthoff, 2004).

/. 1 .1 C om bin ation a n d integration o f se n so ry inform ation

Researchers have turned to investigating the convergence o f sensory inform ation in the brain w ith a view to understanding the m echanism s underpinning the integration o f sensory inform ation into coherent percepts. A num ber o f studies investigating brain activity in hum ans in response to different sensory inputs have found areas o f com m on brain activation when inform ation from different sensory sources relates to the same object or event, such as perceiving speech, or recognising objects using vision and haptics (e.g. A m edi, M alach, H endler, et al., 2001; C alvert, C am pbell, & Bram m er, 2000; Sathian, Z angaladze, H offm an, & G rafton, 1997; Jam es, H um phrey, G ati, et al., 2002; Prather, V otaw , & Sathian, 2004). H ow ever, it is not clear w hether these areas o f activation reflect populations o f neurons that respond to m ultiple types o f sensory input, or sim ply an interm ingling o f different types o f sensory cells w ithin the same cortical area. A nim al studies have revealed that architecturally, convergence can occur in three w ays (M eredith, 2002; 2004). A real convergence refers to the convergence o f inputs o f tw o or m ore different m odalities in the same brain region, but w ithout inform ation from these m odalities term inating on a single neuron. N euronal convergence on the other hand, refers to the convergence o f input from m ultiple m odalities on a single neuron. The resultant m ultisensory neuron can respond to each m odality in turn, and also integrate sim ultaneous input across the m odalities. In this way, m ultisensory neurons are attuned to features o f objects or events encoded through any m odality. The final type o f convergence is a com bination o f the first tw o, and m ost neural areas related to m ultisensory perception tend to incorporate both unim odal and m ultisensory neurons (M eredith, 2002). Feedforw ard and feedback connections serve to relay inform ation

(19)

betw een unim odal and m ultim odal areas o f cortex (e.g. C lavagnier, Falchier, & K ennedy, 2004; Foxe and Schroeder, 2005).

O ne can extrapolate from anim al m odels o f convergence and integration to explain how hum ans generate m ultim odally derived percepts at a very low level through excitatory and inhibitory activation in neuronal circuits (M eredith 2002). How ever, in m any w ays it is difficult to understand m ultisensory qualia sim ply by investigating patterns o f neuronal responses. For instance, does activation in a m ultisensory neuron underpin a veridical or an illusory percept? Insights into the nature o f m ultisensory perception have com e from behavioural studies exam ining the perception o f illusions that rely on input from at least tw o m odalities, for exam ple, the M cG urk effect, the ventriloquism illusion, or illusiory flashing, as they can be system atically induced and studied in a controlled environm ent (Bertelson, 1999; M cGurk and M acD onald, 1976; M orein-Zam ir, Soto-Faraco, & K ingstone, 2003; M unhall, Gribble, Sacco, & W ard, 1996; Sham s, K am itani, & Shim ojo, 2002; Shim ojo and Sham s, 2001). From these studies it is clear that tw o factors in particular influence m ultim odal perception, nam ely the spatial co-location and close tem poral proxim ity o f tw o or m ore sensory events. If these criteria are met then sensory stim uli are generally perceived as em anating from a com m on source (e.g. B ertelson and Radeau, 1981). For instance, the ventriloquism illusion arises w hen the perceived location o f a sound shifts tow ards a visual stim ulus at a slightly different position. This illusion is com m only perceived at the cinem a; the sound o f a voice is perceived as com ing from an actor on the screen, w hen it is actually com ing from a speaker m ounted on the wall. The tem poral proxim ity o f audio and visual stim uli is crucial to the success o f this effect, as the effect disappears when the audio-visual asynchrony exceeds approxim ately 300 ms (Slutsky and R ecanzone, 2001).

(20)

(1999) proposed that sensory inform ation is com bined in an optim ally efficient m anner, that is, one m odality can be w eighted m ore heavily in the com bination if it produces a greater reduction in uncertainty. In other words, the m ore inform ative m odality guides perception, w hich m ay result in a capture effect. U sing psychophysics, Ernst and Banks (2002) expanded on this idea o f w eighted sensory input. Their data indicated that integration is a bottom -up process and that the direction o f sensory capture can be predicted according to the inform ativeness o f the encoded inform ation. They proposed that the nervous system com bines inform ation (in this instance visual and haptic inform ation) in a statistically optim al m anner sim ilar to a m axim um likelihood integrator. Thus, in a task w herein a judgem ent m ust be m ade about the height o f an object, vision dom inates perception w hen the variance associated w ith the noise in the visual signal associated with height is less than that associated w ith the noise in the haptic estim ation o f height. This statistical account o f integration has becom e know n as the m axim um likelihood estim ation (M LE) m odel, and has been used to account for other m ultisensory phenom ena (e.g. B resciani, Ernst, D rew ing et al., 2005).

A lthough it is possible to com bine sensory inform ation from a num ber o f senses w hen perceiving objects and environm ental events, the visual and haptic m odalities are particularly com patible for 2- and 3-dim ensional object recognition. This com patibility stem s from the fact that they are the only m odalities capable o f encoding inform ation about geom etrical shape. Therefore, each m odality can provide inform ation about the shape o f an object, as well as contributing unique sensory inform ation, to provide a rich description o f the object for recognition or subsequent action (e.g. taking m ore care w hen handling the blue glass bauble than when handling the red w ooden one). Indeed, Ernst and B iilthoff (2004) distinguished betw een 'sensory com bination' and 'sensory integration' processes during m ultisensory perception. Sensory com bination refers to the interactions betw een sensory inform ation that are m odality specific (e.g. colour in vision, or texture in haptics) to provide a richer description o f an object than could be gained through either m odality alone. Sensory integration, on the other hand, refers to the interactions betw een redundant signals from both m odalities (e.g. the geom etrical structure o f the object) to provide the best approxim ation o f object shape. This bim odal redundant inform ation is that w hich m ay be integrated according to the M LE model.

(21)

It has been know n for som e tim e now that the haptic system is capable o f recognising 3-dim ensional objects with ease and speed (K latzky, Lederm an, & M etzger,

1985). As in vision, shape inform ation is a diagnostic property for object recognition (Lederm an and K latzky, 1990). H ow ever, there are also a num ber o f differences

betw een the visual and haptic m odalities in term s o f how they respectively process object related sensory input, and this m ay com e to bear on how bim odal visual-haptic percepts are generated. O f course such disparities need not necessarily have a negative im pact on object perception and recognition, indeed they often result in a richer, m ore robust object percept.

A lthough fine-grained object inform ation is obtained by rapidly scanning across the object's surface in a series o f foveal saccades (e.g. Yarbus, 1967), the visual system is capable o f sim ultaneously carrying out a coarser analysis using peripheral receptors in the retina, thus vision is capable o f encoding the global stnicture o f an object quickly and easily. In contrast, unless an object falls w ithin the scope o f the fingertip, the haptic system m ust encode an object in a serial fashion over tim e, and integrate this serially acquired inform ation to form a coherent object percept, placing a considerable dem and

1 2

on working m em ory (Loom is and Lederm an, 1986) . Furtherm ore, the haptic system appears to em ploy a series o f specialised exploratory procedures (EPs) to encode object inform ation (for a m ore com plete description refer to Lederm an and K latzky, 1987). Each EP is associated with a particular object property, and if the appropriate EP is not perform ed, then the associated property is o f m inim al use for recognition (e.g. Lederm an, K latzky, & Reed, 1993). EPs differ in term s o f m easurem ent precision and the speed in w hich data is acquired, for instance, there are tw o EPs used for encoding shape inform ation; enclosure, and contour following. Enclosure is used to gain a quick but rough estim ate o f shape and size, and is executed by m oulding the hands to the surface o f the object. A m ore detailed description o f shape can be obtained through contour follow ing, that is, tracing the contour o f the object with the fm ger(s), and

(22)

although m ore precise, this EP requires the integration o f local feature inform ation over time.

W hile the geom etrical properties o f objects are im portant for recognition in both vision and haptics, the processing o f the m aterial properties o f objects (e.g. surface texture, com pliance, and tem perature) has also been show n to be im portant for haptic object recognition (e.g. Klatzky et al., 1985; 1993; K latzky, Lederm an, & Reed, 1987). Indeed, each m aterial property has its ow n specific EP (L ederm an and K latzky, 1987). For instance, K latzky, Loom is, Lederm an, et al. (1993) found that recognition perform ance, as a result o f exploring objects through heavy gloves, was significantly enhanced w hen the tips o f the gloves w ere cut o ff to expose the fingers to the object's surface texture, divined through 'lateral m otion' o f the fingers across the surface.

T he differences in the spatial bandw idth o f sensory receptors in vision and haptics suggest that a m odality encoding bias exists, with vision preferring to encode m ore global shape properties, and haptics preferring local shape inform ation. How ever, although a greater im portance for local features is evident early in haptic object processing, global features becom e m ore im portant as processing tim e increases (L akatos and M arks, 1999). Also, w hen the cognitive dem ands o f haptic perception are contrived in vision by restricting the visual field o f view so that object properties must be encoded in a serial m anner, recognition perform ance in vision has been found to be sim ilar to haptic perform ance (Loom is, Klatzky, & L ederm an, 1991). Thus, despite different receptor capacities, the visual and haptic m odalities appear to be ftinctionally equivalent in term s o f representing and recognising objects.

T here is som e debate as to how objects, including faces, are actually represented in the brain. One account posits that representations arc view point-independent (e.g. B iederm an and G erhardstein, 1993; M arr, 1982), such that they are stored as structural descriptions o f objects sim ilar to 3D com puter m odels. R ecognition o f an object is a reconstructive process: it begins by extracting local features and progressively com bining them into m ore com plex structural descriptions. H ow ever, it is difficult to generate a 3D representation o f som e (e.g. unfam iliar) objects given their 2D images on the retina. Indeed, even fam iliar objects are not recognised equally well across all view points (e.g. Bruce, V alentine, & Baddeley, 1987). Therefore, a second account has been proposed w hich suggests that objects are represented as a collection o f view- dependent im ages (for a review see T arr and Btilthoff, 1999). In this instance

(23)

recognition is based on m atching the retinal image to the corresponding stored view o f the object. It is not necessary to store every conceivable view, in m any cases a single 'canonical' view can contain sufficient inform ation to allow m atching to a num ber o f different retinal im ages, such as the 3/4 view o f a face (Bruce et al., 1987).

This thesis is not concerned with the exact nature o f the representations underpinning crossm odal face recognition, rather, w hether or not inform ation can be shared betw een the tw o m odalities for recognition'^. As both vision and haptics can encode the geom etrical properties o f faces and this inform ation is stored regardless o f w hether the representation is view point-independent or view point-dependent, it is likely that both m odalities share the same representational format. For instance, efficient recognition o f objects in both m odalities appears to be view point-dependent (e.g. Edelm an and Biilthoff, 1992; H arm ann and H um phrey, 1999; Newell, Ernst, Tjan, & Biilthoff, 2001; N ew ell and Findlay, 1997)’

An increasing num ber o f studies are investigating the way in which inform ation from the haptic and visual m odalities is integrated to form coherent percepts o f objects in the environm ent. B ehavioural evidence has found that crossm odal transfer o f shape inform ation can occur betw een the visual and haptic system s, but that this sharing o f inform ation is view point dependent (Newell et al., 2001). Physical lim itations on m anual exploration predispose the haptic system to encoding the back o f objects, whilst the visual system encodes the front. Thus to m axim ise recognition across m odalities, it is necessary to rotate the object so that the test m odality encodes the object from its preferred view. How ever, in term s o f representing a w hole object in m em ory, both m odalities provide com plem entary inform ation that, w hen com bined, serves to increase the inform ation content regarding that object, allow ing the form ation o f a richer object representation ( c f Ernst and Biilthoff, 2004).

G iven that inform ation regarding object shape is necessary for object recognition in both vision and haptics, and that geom etric inform ation encoded in one m odality can

A lthough the term 'structural' is typically associated with the view point-independent m odel, in this thesis the w ords structural and structure are used to connote the geom etrical shape o f objects and faces, w hich can be represented regardless o f whether the underlying representation is 3-dim ensional or im age-based.

(24)

be utilised for recognition in the other, it is possible that the two m odalities share the same underlying representation o f object shape. A num ber o f studies have investigated the existence o f com m on geom etrical object representations using a crossm odal prim ing

paradigm (Easton, Greene, & Srinivas, 1997; Reales and Ballesteros, 1999). Prim ing refers to the facilitation o f a response (in term s o f accuracy or speed) to a stim ulus to which an individual has recently been exposed (Tulving and Schacter, 1990). In a crossm odal prim ing task objects are prim ed in one m odality and then identified in another m odality. Evidence from the visual literature suggests that object prim ing is m ediated by structural descriptions o f objects (e.g. Cave, Bost, & C obb, 1996; C ooper, Schacter, Ballesteros, & M oore, 1992). Therefore, if visual and haptic object recognition is underpinned by shared structural object representations, crossm odal prim ing effects should be sim ilar in m agnitude to within m odal prim ing effects. Data supporting this prediction has been provided by both Easton et al. (1997) and Reales and B allesteros (1999). It could be argued that such shared representations com prise redundant shape inform ation encoded by vision and haptics.

In a natural progression, research has turned to identifying the neural substrates that m ight underpin visual and haptic object recognition. The question arises as to w hether a single brain area m ight subserve object recognition in both m odalities. A num ber o f functional m agnetic resonance im aging (fM RI) studies have identified the lateral occipital com plex (LO C ) as a possible substrate o f not only visual, but also crossm odal object recognition. Previously im plicated in visual object recognition (Grill- Spector, Kourtzi, & K anw isher, 2001; K anw isher, Chun, M cD erm ott, & L edden, 1996; M alach, Reppas, Benson, et al., 1995), a region o f the LOC has been found to respond w hen participants recognise objects through touch (A m cdi, M alach, Pendler, et al., 2001; G rill-Spector, K ushnir, H endler, et al., 1998; G rill-Spector, K ourtzi, & Kanw isher, 2001; L em er, H endler, H arel, & M alach, 2001). Also, haptics-to-visual prim ing o f novel objects has been found to elicit activation in the LOC that is equivalent to that o f vision-to-visual prim ing o f the sam e objects (Jam es, H um phrey, Gati, et al., 2002). Furtherm ore, evidence suggests that haptically induced activation o f the LOC is not due to verbal m ediation or visual im agery (see Jam es et al., 2005). These studies suggest that visual and haptic object representations share the same neural substrate.

In sum m ary, it is clear from the evidence outlined above that the visual and haptic system s are both capable o f representing 3-dim ensional inform ation about general

(25)

objects, and, w hat's m ore, this inform ation can be shared efficiently betw een the two m odalities. This sharing is possibly m ediated by a com m on object representation subserved by shared neural substrates.

1.2 Visual and haptic face recognition

It is argued that one advantage o f generating m ultisensory object percepts is that it allow s us to use sensory inform ation interchangeably so that object recognition can be m aintained should one sense be lost (Calvert, 2001). G iven that a face constitutes an ecologically im portant stim ulus, the question arises as to w hether person recognition can occur outside o f the visual dom ain. It seems likely that voice inform ation can serve as a m arker o f personal identity in the absence o f vision. V isual and auditory inform ation for person identification generally occur in close tem poral proxim ity, that is, auditory speech and face m ovem ents occur as tem porally synchronous inform ation inputs. T herefore, this inform ation is likely to be integrated and generate a robust m ultisensory representation o f the speaker. In contrast, visual and haptic face inform ation do not autom atically co-occur in the real w orld, therefore it m ay take substantial learning to associate these two types o f face inform ation. On the other hand, the visual and haptic system s are both capable o f processing shape inform ation, therefore the haptic system m ay well be capable o f recognising faces and representing face inform ation in a format conducive to crossm odal face recognition.

(26)

num ber o f new, interesting questions, specifically, how are faces processed by the haptic system , and are these processes sim ilar to those underpinning visual face recognition? Subsequent studies have draw n on the visual face recognition literature to inform both their conceptual and m ethodological approach to investigating haptic face recognition.

1.2.1 Is fa c e recognition special?

One debate that recurs tim e and again throughout visual face recognition literature centies on the notion that faces constitute a 'special' class o f objects. On the one hand, face processing can be considered dom ain-specific, that is, faces are processed by m echanism s separate from those m ediating the recognition o f all other objects (e.g. Bentin and Deoueli, 2000, K anw isher, 2000; K anw isher, M cD erm ott, & Chun, 1997; M oscovitch, W inocur, & Behrm ann, 1997). On the other hand, face recognition m ay be considered a function o f expertise, so that processes that appear to be specific to faces are actually em ployed in the processing o f any object class w ith w hich an individual is expert (D iam ond and Carey, 1986; G authier and Tarr, 1997; G authier, W illiam s, & Tarr, 1998; G authier, Tarr, M oylan, et al., 2000a). E vidence from behavioural, neuropsychological, and im aging studies has been gathered in support o f each perspective. A detailed account o f this debate is beyond the scope o f this thesis. It is sufficient to note that regardless o f w hether face recognition is innately specified or due to expertise, it is clear that face processing is facilitated by configural face inform ation, is orientation sensitive, and m ediated prim arily by cortical structures in the right hem isphere.

A lthough the haptic system is m uch practised in term s o f processing and recognising com m on objects, for instance w hen m oving through a darkened room or searching for an item at the bottom o f a bag, it is generally not used for recognising faces. Therefore investigations o f haptic face recognition prove interesting m light o f the specificity debate. G iven that face recognition is non-expert in the haptic m odality, if haptic face recognition involves sim ilar processes and utilises the sam e neural substrate as visual face recognition, then it could be argued that face recognition processes are indeed underpinned by a dom ain-spccific face m echanism . An overview o f the general findings concerning visual face processing is provided in the follow ing

sections, and these findings are discussed in relation to recent evidence from haptic face recognition studies.

(27)

1.2.2 Configural processing in object and fa ce recognition

B iederm an (1987) proposed that objects are represented and recognised in term s o f their constituent parts (term ed "geons"), and that the spatial arrangem ent o f those parts is unim portant. A lthough featural inform ation is im portant for face recognition (e.g. C ollishaw and Hole, 2000; Schw aninger, Lobm aier, & Collishaw, 2002), we appear to have a preference for processing faces in term s o f configural inform ation'^. For instance, in a study by Y oung, H ellaw ell, and H ay (1987), the top and bottom halves o f different fam ous faces w ere com bined and participants were presented with the new com binations in a face recognition task. They found that when the two halves were closely aligned participants had great difficulty in nam ing the fam ous individuals to whom the top halves o f the com posites belonged. How ever, identity perform ance for each face h a lf was m uch better w hen the halves w ere not closely aligned. This finding supports a configural processing strategy, since by closely aligning the two halves o f the faces, this effectively created a 'new' face w ith different configural relationships which interfered with the ability to identify the top halves o f the original faces.

For adults, the recognition o f individual faces is m ore affected by inversion than the recognition o f any other type o f object (Yin, 1969, 1970; D iam ond and Carey, 1986). The ‘face inversion effect’ refers to the im paired ability to recognise inverted faces relative to other inverted non-face objects or upright faces, and the effect m anifests as reduced accuracy scores and increased response latencies. A plethora o f experim ents using varied m ethodologies have dem onstrated that inversion disrupts configural face processing (e.g. Bartlett and Searcy, 1993; D iam ond and Carey, 1986; Freire, Lee, & Sym m ons, 2000; Leder and Bruce, 2000; Lew is and G lenister, 2003), as when

(28)

individuals are presented with inverted faces they can readily detect m anipulations affecting recognition at the featural level, but they arc largely insensitive to changes in configural inform ation. For instance, the T hatcher Illusion (Thom pson, 1980) is often used to discern the occurrence o f configural processing. In this illusion, configural face inform ation is m anipulated such that the eyes and the m outh are inverted giving the face a grotesque appearance. H ow ever, w hen inverted, the face no longer looks abnorm al, presum ably due to the disruption o f configural processing. This illusion has also been used to dem onstrate that configural inform ation is gradually degraded during the rotation o f a face from upright to inverted (Lew is, 2001; Lewis and G lenister, 2003). A lthough it is not clear to w hat extent configural inform ation is disrupted (e.g. M urray, 2004), the processing o f featural inform ation does not appear to be particularly affected by inversion (V alentine, 1988). A s inversion does not usually affect the recognition o f non face objects (presum ably because the processing o f objects is thought to be feature- based, e.g. B iederm an, 1987), configural processing and the inversion effect are largely accepted as indicators o f a face-specific m echanism (M oscovitch, W inoeur, & Behrm ann, 1997). H ow ever, as a small num ber o f studies have also found an effect o f inversion for m ono-oriented non-face stim uli such as cars, even for non-experts (e.g. G authier, Skudlarski, Gore, & A nderson, 2000b), it is im portant to note that the inversion effect by itse lf cannot be taken as conclusive evidence o f face-speeificity, but rather m ust be interpreted in unison with evidence from neuropsychological and im aging studies, and behavioural studies using alternative paradigm s, such as blurring, for investigating configural face processing (M aurer et al., 2002).

(29)

It seem s from the evidence described that the processing o f configural inform ation is im portant for face processing in vision, regardless o f w hether it arises due to dom ain-specificity or expertise. The question for haptic researchers was w hether the haptic system processes faces in term s o f featural or configural inform ation. There are a num ber o f reasons why the haptic system m ight be expected to use featural rather than configural inform ation for face processing. Firstly, face stim uli fall outside the area confined to the fingertip, thus haptic face encoding m ust proceed in a serial fashion over tim e, and this m ay influence the m anner in w hich faces are represented in m em ory. For instance, visual face stim uli that were encoded in a piecem eal marmer were found to be less prone to the inversion effect than faces encoded holistically, im plying a shift tow ards m ore featural processing for these stim uli (Farah, Drain, & Tanaka, 1995), This suggests that haptic face processing m ay rely m ore on individual face features for identification due to encoding differences across the m odalities. Secondly, the haptic m odality is non-expert at processing faces therefore a dem onstration o f the behavioural effects (i.e. configural processing and the inversion effect) associated with expert object recognition should not be obtained.

(30)

1.2.3 Neuropsychological case studies

In neuropsychological research, double dissociations o f function after focal brain dam age offers the strongest evidence that face and object recognition are m ediated by separate m echanism s and distinct neural pathw ays. Prosopagnosia is a neuropsychological disorder and refers to the inability to recognise fam iliar faces follow ing cerebral injury, usually to the right tem poral lobe (Y oung, 1998). Prosopagnosics can distinguish faces as a category quite easily from other visual objects but are unable to identify specific faces (B odam er, 1947; B niyer, L.aterre, Seron, et al., 1983) A lthough a num ber o f prosopagnosics have dem onstrated a loss o f face processing capabilities, they have been found to have intact object processm g abilities (Farah, Levinson, & K lein, 1995; M cN eil and W arrington, 1993). In contrast, other studies have reported incidents o f object agnosia, w here patients who are perfectly able to identify faces have great difficulty recognising com m on objects (M oscovitch et al., 1997). This indicates that the two processing system s are functionally distinct and anatom ically segregated; focal brain dam age can selectively dam age either one w hilst leaving processing intact in the other system (Farah, R abinow itz, Q uinn and Liu, 2000).

As prosopagnosics can distinguish faces correctly w ithout being able to identify them , it is argued that prosopagnosia reflects an inability to process faces configurally beyond the level o f first-order relational inform ation (that is, a face com prises eyes above a nose, above a m outh) (e.g. M oscovitch et al., 1997). T herefore it m ight be assum ed that prosopagnosics perform to the sam e degree o f accuracy with upright and inverted face stimuli. How ever, studies involving prosopagnosic LH have found that he actually tends to show a face inversion superiority effect, that is, he processes inverted faces better than upright faces (de G elder, B achoud- Levi, & D egos, 1998; de G elder and Rouw , 2000, Farah, W ilson, Drain, & Tanaka, 1995). This suggests that configural processing, though disrupted, is still present to a lesser degree and interferes with identification-by-features in upright faces. Thus w hen configural processing is rem oved through inversion, a feature-based analysis can be conducted m ore easily (de G elder and Rouw , 2000).

A recent study assessed w hether LH m ight retain the ability to discrim inate faces through touch (K ilgour, de Gelder, & Lederm an, 2004). D espite intact sensorim otor

(31)

haptically, and his perform ance was significantly w orse than that o f neurologically intact controls. H ow ever, LH's m atching o f upright non-face objects (clay teapots) was significantly greater than chance, and neither he nor the controls dem onstrated an inversion effect with the non-face objects. Interestingly, as in vision, LH was found to dem onstrate a paradoxical inversion effect in the haptic dom ain. His m atching perform ance for inverted faces was significantly above chance. The results o f this study are im portant for a num ber o f reasons. Firstly, it appears to be the first docum ented case o f haptic prosopagnosia. Secondly, the results have im plications as to the neural substrate underpinning visual and haptic face recognition. It appears that dam age to the cortex that precluded face processing in the visual m odality m ay also have resulted in a deficit in face processing in the haptic m odality. M oreover, LH's haptic deficit m irrored his visual deficit in term s o f inversion superiority effects being present in both m odalities. These findings suggest that visual and haptic face processing m ay be underpinned by the sam e area o f cortex. Finally, LH's intact ability to haptically discrim inate betw een both inverted and upright non-face objects suggests that a dissociation m ight exist betw een face and object processing in the haptic m odality as has been evidenced in vision.

1.2.4 Face-specific neural substrates

Prosopagnosic case studies have indicated that face processing deficits typically occur after dam age to the right hem isphere. A num ber o f im aging techniques have been used in an attem pt to pinpoint a m ore circum scribed region o f cortex in the right tem poral lobe that m ight act as neural substrate for face recognition in neurologically intact individuals. These include positron em ission tom ography (PET), functional m agnetic resonance im aging (fM RI), and the recording o f event-related potentials (ERPs) from the scalp.

(32)

o f m illiseconds, the spatial resolution is poor, so PET and fM RI studies have been

designed and executed in order to identify specific regions o f activation in response to

face stimuli. Several studies have show n that the cortical region in which m ost

participants show a significantly greater activation for visually presented faces than

objects is the right fusiform gyrus o f the inferior tem poral lobe (H axby, H orow itz,

U ngerleider, et al., 1994; K anw isher, M cD erm ott, & C hun, 1997; Puce, A llison, Gore, &

M cCarthy, 1995; M cC arthy, Puce, Gore, & A llison, 1997; Sergent, O hta, & M cD onald,

1992), coinciding w ith the location o f greater N170 activity. One o f the sem inal works

in this area is an fM R I study conducted by K anw isher, M cD erm ott and C hun (1997),

dem onstrating strong evidence o f cortical specialisation in the fusiform gyrus obtained

by testing the responsiveness o f the sam e region o f cortex w ith m any different stim ulus

com parisons’^. The right fusiform gyrus region does not sim ply respond to any anim al

or hum an im ages o f body parts, but instead, generalises to respond to im ages o f faces

taken from different view points. K anw isher and her colleagues have dubbed this area

the "fusiform face area (FFA)". A num ber o f subsequent studies have im plicated the

FFA in the recognition o f facial identity (e.g. H offm an and H axby, 2000; Sergent et al.,

1992).

A recent fM RI experim ent dem onstrated that w hen participants arc experts with

"greebles", identity m atching o f upright m inus inverted greebles, as w ell as passive

view ing o f greebles m inus objects, both produce focal activation in the FFA , com parable

in spatial extent to that obtained with faces (G authier et al., 2000a). So, the com bination

o f categorisation level and expertise appears to provide a plausible explanation for the

strong focal activation obtained w hen faces are com pared to non-face objects. Therefore

it is possible that there is nothing unique about faces activating the fusiform gyrus as

m any other tasks also do so, indeed this suggests that the FFA subserves identification at

the subordinate level rather than at the category level (G authier et al., 2000a, 2000b).

Faces m ay elicit m ore activation sim ply because o f greater processing experience.

The prosopagnosic patient LH, w hose face processing deficit has been found to

extend into the haptic m odality, sustained dam age to the right tem poral lobe, am ong

other areas (Etcoff, Freem an, & Cave, 1991; K ilgour et al., 2004). It m ay be possible

(33)

that the cortex in the right tem poral lobe subserving visual face recognition, specifically the FFA, also facilitates haptic face processing. To date, only two fMRI studies have exam ined cortical activation for haptically explored face stim uli (Kilgour, Kitada, Sevros et al., 2005; Pietrini, Furey, Riccardi et al., 2004).

A lthough Pietrini and colleagues did not investigate haptic face processing per se, they w ere interested in w hether patterns o f category-related cortical activation for objects in visual extrastriate cortex w ere specific to the visual m odality or represented m ultisensory object form , and w hether visual experience was necessary for such category-related activations to occur. Their fMRI results indicated that visual experience w as not necessary for the developm ent o f category-related responses in the ventral visual stream as category related activation was also found for haptically explored objects and facem asks in blind individuals. For both sighted and blind participants, visual and haptic patterns o f cortical activity elicited by bottles and shoes clearly differed from patterns o f activation evoked by face stim uli (im ages or facem asks), but w ere difficult to distinguish from each other. Also, in sighted participants, tactile recognition o f objects evoked patterns o f response in the ventral visual pathw ay that w ere closely related to the responses evoked during visual recognition o f the sam e objects, indicating that representations o f m anm ade objects in visual extrastriate cortex m ay indeed be m ultisensory (see also e.g. Am edi et al., 2001; Jam es et al., 2002). C ontrary to expectations, haptic recognition o f facem asks did not evoke responses in the same area o f inferior tem poral cortex as visual recognition. On the other hand, collapsing across all conditions, earlier visual areas in the occipital cortex w ere activated during haptic recognition, indicating that perhaps only higher- order processing occurs in different areas during haptic and visual face recognition. For instance, it has been suggested that the posterior face regions that fall w ithin the region o f LOC m ay serve to encode the structure o f faces, w hereas m ore anterior parts o f the FFA subserve face recognition (e.g. C ourtney, U ngerleider, Keil, & Haxby, 1997; Sergent et al., 1992).

(34)

processing o f the facem asks explicitly as faces (i.e. it required processing o f face inform ation at the subordinate level, w hich w ould presum ably encourage configural rather than ju st featural processing). T ypically im aging studies report face-specific activation in the right fusiform gyrus. In contrast, this study found that haptic face recognition activated the left fiisiform gyrus, right hippocam pal/ parahippocam pal regions, left cingulate, left inferior parietal lobe and clusters in the right cerebellum .

As acknow ledged by the authors, it is not possible to com pletely dissociate activation due to m em ory retrieval and face processing; the activation pattern rather reflects the neural netw ork underpinning haptic face recognition. H ow ever, the right hippocam pal and parahippocam pal regions, the cingulate gyrus, and the cerebellum have previously been im plicated in visual face m em ory (K im , A ndreasen, O 'L eary et al., 1999; Leveroni, Seidenberg, M ayer, et al., 2002).

K ilgour et al. proposed that left fusiform activation m ay indicate the processing o f sequentially explored facial features due to serial haptic exploration. It is argued that ventral visual areas m ediate integration within objects (H um pherys, 1999a; 1999b). Indeed, it appears that activation o f the left fusiform gyrus is associated w ith integrative processes in visual object recognition, that is, integrating object parts into perceptual w holes (G erlach, A aside, H um phreys, et al., 2002). M oreover, this integration process seem s to be m odulated by stored know ledge about objects. For the highly fam iliar facem asks it m ight be assum ed that a representation o f global shape is stored in m em ory ( c f Lakatos and M arks, 1999), H ow ever, encoding differences betw een vision and haptics m ay lead to haptically encoded faces being processed by the left rather than the right fusiform gyrus, in a m anner m ore sim ilar to visual object recognition' ^ In any case, haptic face recognition appears to recruit cortical structures that subserve visual face and object recognition.

L3 M odels o f face recognition

The literature outlined thus far indicates that the visual and haptic m odalities process faces in a sim ilar m anner (i.e. they both appear to utilise configural face

(35)

inform ation, and both are subject to inversion effects), albeit initial im aging studies suggest that these processes are subserved by slightly different cortical areas in each m odality. It is now pertinent to ask how faces are recognised, and if possible, at what level haptic face inform ation m ight be incorporated into extant m odels o f visual face recognition.

1.3.1 Cognitive models o f face recognition

It is assum ed that people process fam iliar and unfam iliar faces differently due to the finding o f double dissociations for the two types o f processes after brain dam age to the

visual system (M alone, M orris, K ay, & Levin, 1982). Successfiil recognition requires com plete identification and consequently a range o f inform ation is required, for exam ple, a visual representation o f the person as well as some biographical inform ation about them (i.e. a nam e or occupational inform ation). Therefore, functional m odels o f face recognition serve prim arily to outline the recognition o f fam iliar faces (e.g. Bruce and Young, 1986; H. Ellis, 1987; H ay and Y oung, 1982). As the nam e suggests, such m odels can be broken dow n into a num ber o f different processing m odules, or stores, each o f w hich have distinct functions. TTiese m odules can be elim inated, isolated, or independently m anipulated during experim ents, and face recognition deficits with varying sym ptom s can arise due to brain dam age to different m odules (B ruce and Y oung, 1986).

The m ost w idely cited m odel in the visual literature is that devised by Bruce and Young (1986). In this m odel, person recognition is thought to occur independently o f the processes involved in the lecognition o f facial expressions and the analysis o f facial speech inform ation (e.g. Cam pbell, Brookes, de Haan, & Roberts, 1996; Etcoff, 1984; Y oung, M cW eeny, Hay, & Ellis, 1986; but see also Calder and Y oung, 2005) Figure 1.1 illustrates the sequence o f stages involved in face recognition under this m odel. The further one has to progress through the sequence o f stages, the longer it takes to access the desired inform ation about the individual.

(36)

corresponding to a particular familiar face. The FRUs com prise structural face information that is sufficiently abstract so as to allow recognition o f faces across m uhiple viewpoints and view in g conditions' *^. Indeed, studies using recognition priming have demonstrated that priming effects for familiar faces are strongest when the prime and test view s are the same, but are still present even when different view s o f the faces are used in the priming and test phases (A. Ellis, Y oung, Flude, & Hay, 1987).

' A s m en tio n ed earlier, the exact nature o f the representation is not the focus o f this thesis. A lthough o rig in ally the hR U s w ere considered to com prise structural representations (B ru ce and Y oung, 1986), they c o u ld also conceivably com prise collections o f canonical face view s for individuals (i.e. front, profile and 3/4 view s). T herefore, these cognitive m odels serve as a guide as to how the actual pro cess o f recognition o ccu rs, rath er than being largely concerned w ith how faces are represented in m em ory. T hus, the FRUs rem ain the first point in the recognition process, regardless o f w h eth er they co m p rise abstract 3- d im en sio n al representations o r are im age-based.

(37)

D u ' ^ v t c J ' iMUil P ro := ,> u :

Name G eneration

Structural Encoding

Face Recognition

Units (FR Us)

Person Identity Nodes

(PINS)

F ig u re 1.1, A g ra p h ic a l d e p ic tio n o f th e B ru c e a n d Y o u n g (1 9 8 6 ) fu n c tio n a l m o d el o f face re c o g n itio n . T he m o d u le s in b o ld a re in v o lv e d in the fa c e re c o g n itio n p ro c ess .

[image:37.551.15.411.76.489.2]

(38)

intluential on fam iliar face re co g n itio n (B ruce, 1983; B ruce and V alentine, 1986) and is thought to o p erate via the PIN s. T he basic effect is that re co g n itio n o f a fam ous face (e.g. H illary C linton) is faster if it is p receded by an im age o f a re la ted person (e.g. Bill C linton), than if it is p receded by an u n related or unknow n face. W hereas repetition prim ing occurs only in resp o n se to perceptually m atch ed p erso n inform ation, th at is, face im ages p rim e o th er im ages but not w ritten nam es and v ice versa (B ruce and V alentine, 1985; A. Ellis et al., 1987; A. E llis, Elude, Y oung, & B urton, 1996), associative prim in g can o cc u r across any dom ain (B urton, B ruce and H ancock, 1999).

T his m odel has since been develo p ed into the interactive ac tiv atio n com petition (TAC) m odel, ad o p tin g a co n n ectio n ist approach (e.g. B urton and B ruce, 1993; Burton et al., 1999; B urton, B ruce and Johnston, 1990) (see F igure 1.2). T h e basic m odules for face reco g n itio n have been retained, w ith the addition o f sem antic inform ation units (S IU s). T he S lU s code inform ation about know n in d iv id u als, such as nam e, occupation, and nationality, and often m any o f these SIU s w ill be shared am ong fam iliar individuals. T his is d ifferent from the B ruce and Y oung m odel in that both nam e and biographical inform ation are stored in the SIUs.

(39)

NR Us

John Brown

FRUs Sarali Casey

Tracy Grceiie

TG SC

PINs

sc

_TG

Name: >

Sarah Casey Irish SIUs

F ig u re 1.2. A n illu s tra tio n o f the lA C m o d el o f fa ce re c o g n itio n , a fte r B u rto n a n d B ru c e (1 9 9 3 ). A ll d o m a in s o f p e rs o n -re la te d in fo m ia lio n can b e se e n to c o n v e rg e at th e P IN s.

[image:39.551.12.453.58.597.2]

(40)

w hen a nam e is follow ed by a related face. A ssociative prim ing thus allow s prim ing to cross dom ains (faces can prim e nam es and vice versa) and even m odalities. For instance, H. Ellis and colleagues found that associative prim ing facilitated face-voice prim ing o f fam iliar individuals (H. Ellis, Jones, & M osdell, 1997).