The colour phi phenomenon provides an example of many different types of binding, including (i) temporal binding, (ii) motion binding, (iii) location binding, and (iv) feature binding, as well as others.119 However, the colour phi phenomenon also illustrates another
very important aspect of “encoding” mechanisms like object files: namely, their relation to “selection” mechanisms such as FINSTs. Perceiving the colour phi illusion depends on the fact that below the threshold, both dots are “indexed” by the same FINST; however, the content of the visual representation (i.e., that there is a single dot that moves and undergoes a change in colour) is due to the particular features that are listed in the object file to which that FINST is linked at any given time.120
As was argued above, both the selection and encoding versions of the problem
fundamentally concern the nature of sensory representation. The encoding problem is a problem about establishing relations between different types of sensory contents— namely, different representations of features, objects, and spatial locations. On the other hand, however, the selection problem is a problem about reference—it concerns how
119 See Treisman (1996) and Roskies (1999) for a detailed list of different types of binding.
120 Matthen (2005, p.278-282) also discusses the colour phi phenomenon in the context of the binding
sensory systems pick out and track discreet sensory individuals.121 Put somewhat differently, the encoding problem concerns how something is represented, whereas the selection problem concerns what is represented.
What is particularly important about selection mechanisms such as FINSTs and master maps of locations is that they function something like referential pointers, allowing sensory systems to track individual objects and locations without representing any of their features.122 For example, Treisman’s master map (which is scanned by attention) is
not a map of features; rather it is a map of locations, represented independently of whatever features happen to be instantiated at those locations at any given time. Similarly, visual indexes are posited precisely because (by definition) they can track objects despite changes in their visual properties (within certain constraints).
This allows sensory systems to pick out particular individuals regardless of what sensory features the individual might have at a given time. In this sense, these referential pointers function something like natural-language demonstratives. Indeed, Pylyshyn (2001) has explicitly claimed that visual indexes should be viewed "as performing a demonstrative or preconceptual reference function" (p. 127). Similarly, Clark (2000, 2004) has argued that Tresiman’s master map of locations can also act as a kind of referential mechanism, which he calls “feature-placing”. However, Clark also argues that the psychosemantic function of feature-placing mechanisms is somewhat different than FINSTs, and thus “fail in several ways to have the referential powers of visual indices.” As he explains it,
“…a FINST refers in something like the way a demonstrative term in natural language refers; a feature-map refers (or better, "indicates regions") in something like the way a map refers […] the ‘placing’ in feature-placing provides materiel
121 The selection problem concerns reference because the ability to refer to a particular sensory individual
directly (i.e., regardless of any property it has at the time) is what allows for features to be selected for binding (and what determines the features that are selected).
122 Indeed, Viger et al. (2008, p.269) describes a visual index as “a referential pointer to the object that is
without content, except for perhaps the “thatness” defined by its functional role. The content of a visual representation is recorded in the object file.” Or as Kahneman & Treisman (1992, p.216) puts it, “FINSTs are perhaps closer to Marr’s concept of place tokens-abstract markers that allow the visual system to treat filled locations independently of the particular features or objects that occupy them. For example, place tokens allow certain spatial relations, such as co-linearity, to be made explicit without reference to any other aspect of the elements between which they hold.”
with which to make the step to full-blown reference, but it alone cannot complete that step. It is proto-reference; or, if you like, protean reference”. (p.467)
In the previous chapter, I describe the specialized referential capacities of spatial representation in much greater detail. In any case however, my primary point here is unaffected: “selection” mechanisms such as FINSTs and location maps act as referential mechanisms. What’s more, the demonstrative-like nature of these types of referential mechanisms is vitally important from the point of view of the psychosemantics of sensory representation. For example, it is instructive to compare the account of sensory representation described above to a descriptivist version of sensory representation that takes reference to be fixed by some kind of descriptive content associated with a representation. Such an account is ultimately untenable for at least two reasons: First, as was mentioned above,the need for a kind of ‘direct’ reference is particularly necessary in sensory representation because sensory systems must be able to refer to, track, and re-identify objects that could be continuously changing all of their sensible features (colour, shape, pitch, temperature, smell, etc.). A description that picked out sensory individuals solely by reference to those features would be unable to perform this kind of tracking, as it provides no way to determine that two different descriptions are in fact co-extensional (since any variation in a the sensible features of a sensory individual would necessarily change the description required to pick it out).123
Second, and more importantly, descriptivism about sensory reference is untenable because it fails to explain how things inside the head manage to establish relations to things outside the head. That is, it fails to explain how relations holding between internal states can establish relations to external objects and properties. This is known as the symbol-grounding problem (Harnad; 1990). The problem (roughly) is that
representations—symbols—cannot acquire their semantic properties only from their
123 Of course, it’s true that sensory individuals have to be picked out and tracked by some set of features;
however, Pylyshyn is clear that these need not be the features that can be listed in the object file associated with that sensory individual. For example, the onset of a visual stimulus is a feature by which an object might be indexed but which need not be listed in the object file. Similarly, mechanisms like edge detectors are useful for segmenting proto-objects from the rest of a visual scene but are too low-level to be available to be a feature capable of being tagged to an object file.
connection to other symbols in the representational system, for that would lead to an infinite regress. At some point, the meanings of our representations have to be grounded via connections with the external world. (This problem was first discussed in section 3.4 above, in the context of short-armed FRS.) As Devitt (1990) puts it (perhaps unfairly), purely descriptivist theories of reference seem to have no answer to this problem other than “it’s magic”.
However, both of these issues are easily dealt with on the account of sensory
representation described above. The demonstrative-like referential pointers posited by FIT and FINST allow one to refer ‘directly’ to objects and locations in the external world (thereby grounding the intentionality of sensory states) without needing to represent those objects and/or locations under any kind of description.
Importantly however, as I will argue in the next section, the capacity to characterize a sensory individual under a description is the most fundamental type of sensory
representation in phenomenal experience. Genuine, veridicality-evaluable representation requires representing an object, property, or state of affairs under some description or other; i.e., as belonging to a certain category or falling under a certain concept, or possessing a certain feature. However, as noted above, in order to be able to do this we ultimately require some way to refer to the things which our representations are about— their intentional objects—that does not rely on a description. Hence the need for the referential pointers described above in FIT and FINST. In other words, thanks to the demonstrative-like mechanisms of sensory reference and the object file –based encoding mechanisms of binding, qualitative experience is about the world; phenomenal qualities are not experienced as properties of sensations, but as properties instantiated by external things.