Marr’s tri-level hypothesis and the integration challenge We looked at Marr’s theory of early visual processing in section 2.3 As was brought out

there, Marr’s theory was hailed at the time as a textbook example of cognitive science. Its renown was partly due to Marr’s many insights into the operations of the early visual system, which was just beginning to be understood at the time. But there is a further reason why Marr has been so celebrated as an inspiration for cognitive science. Marr’s book Vision is truly interdisciplinary, and the theoretical framework that he developed, what is generally known as the tri-level hypothesis, has seemed to many to provide a general framework and methodology for cognitive science in general.

We saw insection 2.3that the fundamental theoretical idea driving Marr’s discussion

is that cognitive systems, such as the early visual system, have to be analyzed at three different levels. Marr’s three levels differ in how abstract they are. The most abstract level of analysis is the computational level. Analyzing a cognitive system at the computa- tional level is a matter of specifying the cognitive system’s function or role. But this specification has to take a particular form.

Marr understands the role of a cognitive system in a very clearly defined and focused sense. We specify the role of a cognitive system by specifying the information-processing task that the system is configured to solve. The basic assumption is that cognitive systems are information-processing systems. They transform information of one type into information of another type. For Marr, we analyze a cognitive system at the computational level by specifying what that transformation is. Marr’s computational analysis of the early visual system is, in essence, that its role is to transform information from the retina into a representation of the three-dimensional shape and spatial arrange- ment of an object.

The next level of analysis is the algorithmic level. The form of an analysis at the algorithmic level is dictated by the analysis given at the computational level. This is because, as its name suggests, an algorithmic analysis specifies an algorithm that performs the information-processing task identified at the computational level. Information- processing algorithms are step-by-step procedures for solving information-processing problems. We will be looking at algorithms in more detail inChapters 6through9. For

the moment the important points to notice are, first, that algorithms are finite sets of instructions. It must be possible to write them down. Second, it must be possible to execute an algorithm in a finite amount of time. Finally, algorithms must be mechanical and automatic. They cannot involve either guesswork or judgment.

We can think of a computer program as the paradigm of an algorithm. A computer program is a set of instructions that“tells” the computer what to do with any input it receives. If the program is well designed and contains no bugs, then it will always respond in the same way to the same inputs. Consider a spell-checker in a word-processing program, for example. A well-designed spell-checker will always flag exactly the same words every time it is presented with a given sentence. And it does not require any further information beyond the words that it is checking. All the relevant information is programmed into it.

Exercise 5.4 Give another example of an algorithm, preferably not one that has anything to do with computers. Explain why it counts as an algorithm.

The move from the computational level of analysis to the algorithmic level is the move from identifying what information-processing task a system is carrying out to identifying the procedure that the cognitive system uses to carry out the task. The first step in giving an analysis at the algorithmic level involves deciding how information is encoded in the system. Algorithms are procedures for manipulating information. In order to spell out how the algorithm works we need to specify what it is working on. Infor- mation needs to be encoded in a way that allows it to be mechanically (algorithmically) manipulated to solve the information-processing problem.

In earlier chapters we have seen some very different ways of thinking about how information is encoded. When we looked at artificial neural networks insection 3.3, for example, we looked at an artificial neural network trained to discriminate between mines and rocks. The information-processing problem that the network is trying to solve is the problem of distinguishing between sonar echoes that come from rocks and sonar echoes that come from mines. As we saw, the network solves this problem through the backpropagation learning algorithm. Backpropagation is algo- rithmic because it works in a purely mechanical, step-by-step manner to change the weights in the network in response to the degree of“mismatch” between the actual result and the intended result. (This is the error that is“propagated back” through the network.)

But the algorithm can only work if the sonar echo is encoded in the right sort of way. The algorithm cannot work directly on sound waves traveling through water. This is why, as was explained insection 3.3, the levels of activation of the input units are used to code each sonar echo into the network. The input units are set up so that each one fires in proportion to the levels of energy at a particular frequency. Once the input information is encoded in this way, it can flow forwards through the network. This feedforward process is itself algorithmic, since there are simple rules that determine the levels of activation of individual units as a function of the inputs to those units. During the

training phase, the output from the network is compared to the desired output and the backpropagation algorithm used to adjust the weights.

Exercise 5.5 Thinking back to the historical survey inPart I, identify one other example of an algorithmic analysis of an information-processing problem.

In one sense an analysis of an information-processing problem at the algorithmic level is very concrete. If the analysis is complete, then it tells us all we need to know from the perspective of task analysis. That is, it gives us a blueprint for solving the task identified at the computational level. We know that all that the system needs to do is to follow the algorithm, however complicated it might be. Nonetheless, in another sense an algorithmic analysis remains very abstract. If one is an engineer, for example, trying to build a machine to solve a specific information-processing problem, then it is plainly not enough to be given an algorithm for the problem. One needs to know, not just what algorithm to run, but how to build a machine that actually runs the algorithm. Similarly, in analyzing a cognitive system, it is not enough simply to know what algorithm it is running. One also needs to know how it runs the algorithm.

This brings us to the final level of analysis in Marr’s approach, namely, the implemen- tational level. An analysis at the implementational level is an analysis of how the algorithm is realized in the cognitive system being studied. Analysis at the implementa- tional level takes us from abstract characterizations of inputs, outputs, and information- processing operations to detailed accounts of how the brain actually executes the algo- rithm. At the implementational level we are dealing primarily with neurobiology, neuro- physiology, and neuroanatomy. At the time at which Marr was writing far less was known than is now about how information is processed in the brain (and this is reflected in the relatively small amount of space devoted to questions of implementation in his book Vision).

Figure 5.4shows one of Marr’s implementational proposals. It represents schematically how the brain might be configured to detect zero-crossings (which are sudden changes of light intensity on the retina, so called because they mark the point where the value of light intensity goes from positive to negative, and hence crosses zero). The proposal exploits the fact that some neurons fire when the centers of their receptive fields are stimulated (these are the on-center neurons), while others (the off-center neurons) fire when there is no stimulation in their receptive field. If there are two neurons, one on- center and one off-center, with receptive fields as depicted inFigure 5.4, then both will fire when there is a zero-crossing between them. The only other thing needed for a zero- crossing detector is a third neuron that will fire only when the off-center and on-center neurons are both firing. This neuron would be functioning as what computer scientists call an AND-gate.

Despite cognitive science’s turn to brain (described inChapter 3), it remains the case that there are relatively few information-processing problems for which we have a fully

worked out implementational level analysis. Fortunately we have already looked at some examples of implementational level analyses. One is the PET study of lexical processing explored insection 3.4.

Exercise 5.6 Redescribe the model of lexical processing reported insection 3.4in terms of Marr’s three levels of analysis.

The potential relevance of Marr’s tri-level hypothesis to the integration challenge should be obvious. Marr is not simply suggesting a distinction between different levels of analysis. The key feature of his proposal for studying cognitive systems is that it gives us a way of connecting the different levels. The analyses at the three different levels are distinct but not independent of each other. Analysis at the computational level con- strains and determines analysis at the algorithmic level. The aim of the algorithms identified at the algorithmic level is to solve the problems identified at the computa- tional level. By the same token, analysis at the implementational level is dictated by analysis at the algorithmic level.

AND

P

Q

(a) (b)

Figure 5.4 A mechanism for detecting oriented zero-crossing segments. In (a), if P represents an on-center geniculate X-cell receptive field, and Q an off-center, then a zero-crossing must pass between them if both are active. Hence, if they are connected to a logical AND gate as shown, the gate will detect the presence of the zero-crossing. If several are arranged in tandem as in (b) and are also connected by logical ANDs, the resulting mechanism will detect an oriented zero-crossing segment within the orientation bounds given roughly by the dotted lines. Ideally, we would use gates that responded by signaling their sum only when all their P and Q inputs were active. (Adapted from Marr and Hilldreth1980)

It is not surprising, therefore, that many cognitive scientists have seen Marr’s tri-level hypothesis as the key to solving the problem of how to link together the different disciplines involved in cognitive sciences and the many different levels of organization that we find in human cognitive agents. We might think of high-level disciplines, such as cognitive psychology and cognitive neurospychology, as contributing to analysis at the computational level. Analysis at the algorithmic level might be carried out by computa- tional neuroscientists, for example, or by researchers in artificial intelligence. Implemen- tational level analysis might be thought of as the domain of neurophysiology and cellular neuroscience.

Certainly, this is often how cognitive science is presented – as an interdisciplinary activity unified by the fact that all its constituent disciplines and activities can be located at one or other level of Marr’s hierarchy of levels of analysis. However, as I shall be suggesting in the remainder of this section, there is a very fundamental problem with any attempt to generalize Marr’s theory into a global model for cognitive science.

This fundamental problem is a direct consequence of its most obvious and innovative feature – namely, the “recipe” that it gives for connecting up the different levels of analysis. As we have seen, the thread that ties the different levels together is the notion of an algorithm. In analyzing a cognitive system at the computational level cognitive scientists have to be very precise and determinate about the information-processing problem that the system is configured to solve. They have to be precise and determinate because the information-processing problem has to be the sort of problem that can be solved algorithmically. Similarly, cognitive scientists working at the implementational level are not simply studying neurobiological systems. They are studying neurobio- logical systems as systems that are computing certain algorithmic procedures.

Problems with the tri-level hypothesis as a blueprint