4
The café wall illusion. The arrangement of the tiles on this café wall makes the rows of tiles appear to be tilted at strange angles. Although the illusion has been known about for many years, it was made famous by Richard Gregory, who used this café (in Bristol) as an illustration of the effect. The ‘English meat’ sign suggests that the illusion might well have been called the ‘butch-er’s wall illusion’. A larger version of the illusion is seen on the Digital Harbour Port 1010 building in Melbourne designed by the architects Norman Disney & Young.
CHAPTER OVERVIEW
In the last chapter we looked at the nuts and bolts of the visual parts of the brain. Most of this information has come from studies of animals, but can we tell if we humans have the same sort of visual pathways? How might we find out? For example, Figure 4.1 consists of a chequerboard with the black and white checks arranged in orderly horizontal rows and vertical columns, but that’s not the way it looks. Everyone sees the lines as very distorted.
Can we use this information to explain why sometimes our visual perception doesn’t quite match up to what we know is out there in the real world? In this chapter we shall concen-trate on how we see objects, how we determine their correct size, position, and orientation in a world where our eyes can be close to (or far away from) what we are looking at, and where our heads can be tilted to one side or the other. The active processes by which we attempt to infer what objects and patterns are in front of us are not perfect and, so, under some conditions, we can get it wrong and construct an incorrect interpretation of the world, which we might call an ‘illusion’. Perhaps we can even use our knowledge to predict new illusions and distortions.
Experiments on humans
In the previous chapter, we learnt about how Hubel and Wiesel found their oriented bar detectors in the cortex of cats and monkeys, but that doesn’t necessarily tell us anything about what we’ve got inside our heads. How can we find out what sorts of neurons are in the human area V1? Well, there is one obvious way, but bleeding-heart liberal ethics committees seem to feel that recording from the visual cortex of a few undergraduates is just not on, even in return for course credit. Fortunately, there is an alternative. In Chapter 2 we used the Hermann grid to demonstrate that we have con-centric receptive fields in the retina similar to those found in cats and monkeys. Our reasoning went something like this:
1 Cats and monkeys have these neurons with funny receptive fi elds.
2 A pattern like the Hermann grid seen with these neurons should be distorted.
3 We see just such a distortion.
4 So we must have these funny receptive fi elds too.
Actually, this argument is completely the wrong way round. The Hermann grid was discovered about a century before anyone had even thought of concentric receptive fields in ganglion cells. So the proper way of looking at this problem goes like this:
1 The Hermann grid produces a funny distortion.
2 If we had concentric receptive fi elds this is what you would expect.
3 Cats and monkeys have just such receptive fi elds.
4 So we must have these funny receptive fi elds too.
➾See Chapter 3
➾See Chapter 2 Experiments on humans 99
Now let’s get back to the oriented receptive fields in V1 of cats and monkeys. Do we have neurons like this too? Once again, the evidence that we do was known long before we knew about the neurons!
The tilt after-effect
Look at the small spot on the left of Figure 4.2. The lines above and below this spot will appear to be vertical. If they don’t, worry—because they are. Now look at the small bar on the right of the figure. The lines above and below should not appear verti-cal (because they aren’t). Continue to look at this bar while you count slowly to 60, and then look again at the small spot on the left. What do you see now? Most people report that the bars no longer appear vertical (at least for a few seconds)! The bars at the top appear tilted clockwise and those at the bottom tilted counter-clockwise. Note that this is in the direction opposite to what you were just seeing. This is known as the tilt after-effect and was discovered around 1937 (Gibson and Radner, 1937). The tilt after-effect is good evidence that we have orientation-selective neurons like those dis-covered in cats and monkeys some 20 years later, as we shall now see.
A neural explanation of the tilt after-effect
Assuming you haven’t skipped ahead in the book to this chapter without reading Chapter 3, cast your mind back to the orientation-selective neurons we introduced
Figure 4.1 An illusion of tilt. One of many brilliant illusions created by Akiyoshi Kitaoka. This really is a regular array of black and white squares.
there. You may remember that each cell responds best to a particular orientation, or tilt, of a line. Lines slightly away from the cell’s preferred orientation produce less of a response, and ones that are quite different in orientation produce no response at all.
Figure 4.3 plots the response of a real cell recorded from area V1.
Now, whereas in Figure 4.3 we plotted the response of one cell to a range of orienta-tions, in Figure 4.4a we plot the response of lots of cells to one orientation. This figure is a bit tricky so let’s go slowly. The orientation we are going to present to our neurons is a vertical bar, so we show a vertical bar at the top of the figure. Below the bar, we
➾See Chapter 3 Figure 4.2 The tilt after-effect. Stare at the bar between the two tilted sets of lines on the right for
about 60 seconds. When you move your gaze to the dot between the vertical lines they should appear distorted, tilting in the opposite direction.
140 160 180 200 220 240 260 0
5 10 15 20
Orientation of line
Response (spikes/sec)
Figure 4.3 An orientation-tuned cell from area V1.
A neural explanation of the tilt after-effect 101
(a)
Figure 4.4 (a) Before adaptation, a vertical pattern will excite cells tuned to vertical. The pattern will also excite neurons tuned to neighbouring orientations to a lesser extent. (b) A tilted pattern maximally excites neurons tuned to its degree of tilt. (c) We don’t need neurons with a peak excitation at every orientation we need to judge. By looking at the pattern of excitation over a range of orientation-tuned neurons we can make much finer discriminations of orientation. Here, we only have neurons tuned to discrete orientations of 0°, 22°, 45°, etc., but we can still identify a grating as having an orientation of 10°. (d) After a period of adaptation, the cells excited by a pattern tilted at 22° will be adapted and respond less vigorously to the pattern. Notice that the tilted pattern still looks tilted, but the cells aren’t firing as strongly. We shall consider what this means a little later.
(e) After adaptation to the tilted pattern, a vertical pattern will be distorted and maximum excitation will be found in cells tilted in the opposite direction to the adapting pattern. (f) Adapting to large tilts will not affect the perception of a vertical pattern.
show the tuning curves of a whole bunch of neurons. Each neuron responds best (or, to put it another way, is most sensitive) to one orientation (just like our cell depicted in Figure 4.3) and its response (sensitivity) drops off as the orientation moves away from this preference. That’s what each of the purple humpy lines shows. At the bottom of Figure 4.4a we show the response (think of it as the number of nerve impulses per sec-ond) of each of our neurons to the vertical bar. As you can see, the neuron tuned for vertical bars (neuron C) is excited the most, but neurons B and D which are tuned to just off vertical have about 50% of their peak sensitivity to vertical bars, and those tuned well away from vertical (neurons A, E, F, and G) do not respond to vertical bars at all. This pattern of excitation tells the brain ‘there’s a vertical line’. So we have a way of knowing what is out there in the world by knowing how these neurons fire.
Now think what happens when we look at a tilted bar (Figure 4.4b). The neuron that responds most is, wait for it, the neuron that is optimally tuned for that tilt, neuron D.
And now it’s neurons C and E that respond a bit, while the others don’t respond at all.
If that seems too difficult, have you considered transferring to a sociology course?
We could decide that the orientation of the bar is determined by which neuron fires most vigorously, but there is an obvious problem if we do this. Look again at our array of neurons in Figure 4.4c. Neuron C responds to verticals (0°) and neuron G responds best to horizontals (90°). Therefore, D responds best to about 22°, E to 45°, and F to 67°. Imagine what will happen if we are presented with a bar tilted to the right by 10°.
Both neurons C and D will respond quite well to it, with C responding slightly more; if you look closely, neuron B even responds a little to the 10° bar. Now, if orientation is determined by a winner-take-all model then we will see this bar as being vertical, as neuron C responds best to it. In other words, we would only be able to discriminate as many orientations as we have different orientation neurons. However, if we look at the pattern of activity in the whole group of neurons you can see that we can make lots of discriminations between each orientation step. In Figure 4.4c we can draw a curve through the responses of our neurons (the dotted green line) and see that the pattern of response observed suggests a bar at about—surprise, surprise—10°!
Let’s get back to the tilt after-effect. You will remember that we stared at the tilted bars for some time, so what happens to our neurons when we do this? Let’s suppose that the neurons that are most excited can’t keep up their very fast firing rate for very long; think of them as getting ‘tired’, or perhaps ‘bored’. Actually, we say that they adapt, but it means the same thing. The important point is that their response (or sensi-tivity) to the stimulus drops over time. You can see this in Figure 4.4d. After adapting to the tilted bar, the humpy purple sensitivity curves of the cells that respond to the tilted bar are reduced in height, but the cells that didn’t respond at all to the tilted bar are unaffected—they show no adaptation. Note that the more the neuron is excited the more tired, or adapted, it gets. Just like you. What happens to the perceived orientation of our tilted bar after this adaptation? Look at the responses at the bottom of Figure 4.4d. You will see that the overall level of the responses has dropped compared with Figure 4.4b, but the shape of the distribution is just the same—nice and symmetrical.
The interesting bit is when we return our gaze to the vertical lines after the adaptation A neural explanation of the tilt after-effect 103
(Figure 4.4e). The vertical bar may still excite the vertical detector more than any other but it now produces a bigger response in neuron B than in neuron D—the response dis-tribution is subtly shifted to the left, i.e. the vertical bar now appears to be tilted slightly to the left. And this is exactly what we observe.
Could we produce a more impressive effect by having the adapting lines tilted even further from the vertical? Let us see what our theory predicts (Figure 4.4f). If we were to stare at lines that were even more tilted, then we should cause activity in neurons that are tuned for these particular tilts and in turn will adapt to them. However, as these neurons are tuned for orientations that are well away from the one tuned to ver-tical they do not have any effect on the verver-tical lines. Therefore, the fact that they are now less sensitive should make no difference. This is, indeed, what is found.
Figure 4.5 shows data from just such an experiment that measured the size of the tilt after-effect as a function of the orientation of the adaptation bars (Morant and Harris, 1965). The solid line through the data points comes from a computer simulation of the model of the tilt after-effect that we have just described. You can see that the model predicts an ‘optimum orientation’ for producing the biggest after-effect, and the data agree with this prediction. This finding, that the biggest effect is found a little away (but not too far) from the test pattern, is known as the distance paradox.
Tilt-specific threshold elevation
Let’s examine our little model again. Each of the orientation-tuned neurons fires vig-orously to its preferred tilt, and less well as the orientation of the lines moves away from this optimum stimulus. Now suppose that we keep the orientation at the pre-ferred angle but change how distinct the stimulus is. We can do this by changing the contrast of the stimulus. Look at Figure 4.6 which shows a stripy pattern that goes from being rather indistinct at the top to being really stripy at the bottom. When the bars are very faint (alternating between two similar shades of grey) we say that this is a low-contrast pattern because there isn’t much difference between the light and dark bars of the pattern. When the bars have lots of difference (go from black to white), we say that it has a high contrast. Now how do we expect a cell that prefers vertical
Adaptation orientation
Magnitude of tilt after-effect
Figure 4.5 The tilt after-effect as a function of adaptation tilt. The biggest effect is found with tilts of about 10–20°. Bigger tilts reduce the effect, just as our model predicts.
patterns to respond to these different contrasts? Perhaps unsurprisingly, cells respond well to high contrasts and not so well to low contrasts.
We explained the tilt after-effect by the idea that after we have stared for some time at (or ‘adapted to’) lines of a particular tilt, the neurons that signal their presence become less sensitive. This means that these neurons don’t respond as well as they did and this should mean that the lines would become harder to see. Figure 4.7 gives a demonstration of this effect (Blakemore and Campbell, 1969). In the middle part of the figure you should be able to see some faint vertical lines. Look at the high contrast
High contrast Low contrast
Figure 4.6 A simple grating pattern whose contrast (the difference between the dark and light stripes) decreases as we go up the page. Eventually, the pattern becomes invisible—it drops below our contrast-detection threshold.
Figure 4.7 Demonstration of orientation-specific contrast threshold elevation. After adaptation to the high-contrast vertical grating (top left), the central low-contrast pattern will be invisible.
Adapting to a horizontal grating (top right) or to very different sizes of stripes (bottom) will not affect the visibility of the central test pattern.
Tilt-specific threshold elevation 105
vertical lines for about 30 seconds. Now look back at the faint (very low contrast) lines. Where have they gone? After a few seconds the bars should return. Now try adapting to the high-contrast horizontal bars. You should find that this produces no effect on the vertical test pattern. Results from a real experiment are shown in Figure 4.8. In this experiment, the contrast of the bars was adjusted to be ‘just visible’ before and after adaptation. As you can see, the effect is greatest when the adapting and test patterns have the same orientation (termed 0), and the effect becomes smaller as they become more different. Patterns that are 90° different (such as vertical and horizontal) produce no effect. So, as predicted, as we stare at the bars of a particular orientation we lose sensitivity to this orientation, but only to this and very similar orientations.
The size after-effect
Let’s go back to our oriented receptive fields in V1. Figure 4.9 shows a few vertical bar detectors of different sizes. Some of them have large receptive fields and some have small receptive fields. Clearly, to detect big fat vertical bars you need big fat vertical bar detectors, and to detect skinny little vertical bars you need . . . well, you know what you need. So, as well as having an array of receptive fields for all different orien-tations, cats and monkeys appear to have an array of different-sized receptive fields at each orientation. To detect big bars they have big receptive fields, to detect medium-sized bars they have medium-medium-sized receptive fields, and to detect little baby bars (remember the story of Goldilocks and the three bears?) they have small receptive fields.
Again, we can ask the question, how can we demonstrate that we have similar neu-rons in our heads? And again an after-effect comes to our rescue. If the same processes are taking place for the size of the lines as we have proposed for orientation, we should be able to produce a size after-effect in exactly the same way as we produced the tilt
–100 –50 0 50 100
Baseline Adapted
Angle of adaptation
Contrast needed to see pattern
Figure 4.8 Threshold elevation as a function of the orientation of the adapting grating pattern. The biggest adaptation effect occurs when the adaptation and test gratings have the same orientation.
Figure 4.9 Examples of vertical bar detectors of different sizes. In order to see small fine bars you need a small receptive field—a big bar will cover both the excitatory and inhibitory regions and produce no response. Large receptive fields are poor at detecting fine lines, as the lines will only cover a small fraction of the receptive field.
The size after-effect 107
after-effect (Blakemore and Sutton, 1971). On the left of Figure 4.10 are two sets of bars, and if you look at the spot in the middle of them the bars should all look as if they are the same thickness. On the right of the figure are two more sets of bars. If you look at the horizontal line between the bars, the ones on top should appear fatter than the ones on the bottom. Now adapt to these bars by looking at the line for about a minute. When time is up, return your gaze to the spot between the bars on the left. The bars should no longer appear the same thickness; instead, the bars at the top should look thinner than the bars at the bottom. This is the size after-effect, and the explana-tion is the same as what we gave for the tilt after-effect. The cells that were signalling fat bars when you were adapting become less active and so when you look at the
after-effect (Blakemore and Sutton, 1971). On the left of Figure 4.10 are two sets of bars, and if you look at the spot in the middle of them the bars should all look as if they are the same thickness. On the right of the figure are two more sets of bars. If you look at the horizontal line between the bars, the ones on top should appear fatter than the ones on the bottom. Now adapt to these bars by looking at the line for about a minute. When time is up, return your gaze to the spot between the bars on the left. The bars should no longer appear the same thickness; instead, the bars at the top should look thinner than the bars at the bottom. This is the size after-effect, and the explana-tion is the same as what we gave for the tilt after-effect. The cells that were signalling fat bars when you were adapting become less active and so when you look at the