The standard way of thinking about the mind as an information processor takes the digital computer as a model. (This was almost unchallenged in the early 1970s, and remains a popular view now, although we now have a much clearer sense of some alternative ways of thinking about information processing.) Digital computers store and manipulate information in a fixed format. Essentially, all forms of information in a digital computer are represented using the binary numerals 0 and 1. Each binary digit carries a single unit of information (a bit). Within the computer these units of infor- mation are grouped into words– a byte, for example, is an 8-bit word that can carry 256 units of information. This way of carrying information in discrete quantities is often called digital information storage. One feature of digitally encoded information is that the length of time it takes to process a piece of information is typically a function only of the quantity of information (the number of bits that are required to encode it). The particular information that is encoded ought not to matter. But what the mental rota- tion experiments have been taken by many to show is that there are information processing tasks that take varying amounts of time even though the quantity of infor- mation remains the same.
Exercise 2.4 Why does a byte carry 256 units of information?
In order to get an intuitive picture of what is going on here and why it might seem puzzling, look again at the experimental drawings in Figure 2.7and think about how each of them might be digitally encoded. Suppose that we think of each drawing as divided into many small boxes (rather like pixels on a television screen or computer monitor). Since the drawings are in black and white we can convey a lot of information about the drawing by stating, for each pixel, whether it is black or white. But this will not give us a full characterization, since the figures are represented three-dimensionally. This means that our characterization of each pixel that represents part of a surface will have to include a value for the surface’s degree of orientation, degree of brightness, and so on.
Now, suppose that this has been done and that we have a pixel-by-pixel description of each drawing. This will be a collection of pixel descriptions. Each pixel description is simply a set of numbers that specifies the values on the relevant dimensions at the particular pixel locations. The overall pixel-by-pixel description of each drawing puts all those individual descriptions into an ordering that will allow it to be mathematically
manipulated. One way of doing this would be to assign a set of coordinates to each pixel. In any event, the point is that each drawing can be represented by a set of numbers.
The information-processing task that the experiment requires is essentially to com- pare two such numerical descriptions to see if one can be mapped onto the other. Solving this problem is a tricky piece of mathematics that we fortunately do not have to go into, but there is no obvious reason why it should take longer to solve the problem for pairs of figures that are at greater degrees of rotation from each other than for pairs that are at smaller degrees from each other– and certainly no reason why there should be a linear relationship between reaction time and degree of rotation.
For reasons such as these, then, it has been suggested that cognitive tasks like those investigated by the mental rotation experiments involve ways of encoding information very differently from how information is encoded in a digital computer. We will be looking in more detail at different ways of thinking about information in the chapters in
Part III. For the moment we can present the distinction with relatively broad strokes of the brush. One distinctive feature of how information is represented in digital computers (what is often called digital representation) is that the connection between what we might think of as the unit of representation and what that unit represents is completely arbitrary.
There is no reason, for example, why we should use the symbol“0” to represent a black pixel and the symbol“1” to represent a white pixel, rather than the other way around. The symbol“0” represents a black pixel because that is how the computer has been set up. (As we’ll see later, it’s no easy matter to explain just how computers are set up to represent things, but we can gloss over this for the moment.)
Contrast this with how, for example, a map represents a geographical region. Here there is a large-scale resemblance between the principal geographical features of the region and the discernible features of the map– if there is no such resemblance then the map will not be much use. The weaving and winding of a river is matched by the weaving and winding of the line on the map that represents the river. The outlines of a region of forestry are matched by the edges of the green patch on the map. Undula- tions in the terrain can be mapped onto the contour lines. And so on. A map is an excellent example of what we might think of as an imagistic representation. The basic characteristic of an imagistic representation is that representation is secured through resemblance.
Exercise 2.5 Can you think of other differences between digital representation and imagistic representation?
One popular interpretation of the mental rotation experiments is as showing that at least some types of information are represented imagistically at the level of subcon- scious information processing. It is not just that we have the experience of consciously rotating figures in our mind’s eye. The shapes are also represented imagistically in the subconscious information processing that makes possible these types of conscious experience. The point of this interpretation is that certain operations can be carried
out on imagistically represented information that cannot be carried out on digitally represented information. So, for example, it is relatively straightforward to think of rotating an imagistic representation, but as we saw earlier, difficult to think of rotating a digital representation. This gives us one way of explaining what is going on in the mental rotation experiments.
The idea that the information processing in mental imagery involves operations on imagistic representations also makes sense of many of the other effects identified in the experimental literature provoked by the imagery debate. So, for example, in a famous experiment carried out by Stephen Kosslyn in 1973 subjects were asked to memorize a set of drawings like those illustrated inFigure 2.9.
Kosslyn then gave them the name of one of the objects (e.g. “aeroplane”) and asked them to focus on one end of the memorized drawing. The experiment consisted of giving the subjects the names of possible parts of the object (e.g.“propeller”) and asking them to examine their images to see whether the object drawn did indeed have the relevant part (which it did on 50 percent of the trials). The subjects pushed a button only if they did indeed see the named part in their image of the drawn object.
Kosslyn found an effect rather similar to that in the mental rotation studies – namely, that the length of time it took the subjects to answer varied according to the distance of the parts from the point of focus. If the subjects were asked to focus on the tail of the plane, it would take longer for them to confirm that the Figure 2.9 Examples of vertically and horizontally oriented objects that subjects were asked to visualize in Kosslyn’s 1973 scanning study. (Adapted from Kosslyn, Thompson, and Ganis2006)
plane had a propeller than that there was not a pilot in the cockpit. Kosslyn’s interpretation of his own experiment was that the type of information processing involved in answering the test questions involves scanning imagistic representations. Instead of searching for the answer within a digitally encoded database of information about the figures, the subjects scan an imagistically encoded mental image of the aeroplane.
Exercise 2.6 Can you think of a way of explaining the results of Kosslyn’s experiments without the hypothesis of imagistically encoded information?
The lengthy theoretical and practical debate that began with the mental rotation and scanning experiments goes to the heart of one of the fundamental issues in cognitive science. Almost all cognitive scientists agree that cognition is information processing. But what emerged in a particularly clear form in the imagery debate is that there are competing models of how information is stored and how it is processed. The mental rotation experiments were the first in a long line of experiments that tried to decide between these competing models. One of the great benefits of this lengthy experimental literature has been much greater clarity about how each model thinks about informa- tion and information processing – and about what exactly it is that we are trying to explain. We will return to these issues in later chapters.
2.3
An interdisciplinary model of vision
The mind can be studied at many different levels. We can study the mind from the bottom up, beginning with individual neurons and populations of neurons, or perhaps even lower down, with molecular pathways whose activities generate action potentials in individual neurons, and then trying to build up from that by a process of reverse engineeringto higher cognitive functions (reverse engineering being the process by which one takes an object and tries to work backwards from its structure and function to its basic design principles). Or we can begin from the top down, starting out with general theories about the nature of thought and the nature of cognition and working down- wards to investigate how corresponding mechanisms might be instantiated in the brain. On either approach one will proceed via distinct levels of explanation that often have separate disciplines corresponding to them. One of the fundamental problems of cogni- tive science (seeChapters 4and5below) is working out how to combine and integrate different levels of explanation.