Connectionist models

Connectionist approaches to learning and knowledge representation have been gaining popularity in the cognitive sciences in the last two decades; this impact is felt perhaps particularly strongly in the field of reading research.

Most connectionist models of learning are the variants of the same core architecture. They constitute a network built of two or more sets o f basic units (representing different aspects of input and output) interlinked through adjustable connections. Information is devolved throughout the system and stored in the strength of the connections between input and output. This non-localised, parallel-distributed (Seidenberg & McClelland, 1989) mode of processing the information means that the distinction between item-specific and mle-general knowledge is effectively abolished. In the context of reading it implies the rejection of any dual-route architecture that proposes separate procedures for reading familiar and unfamiliar words (e.g. Coltheart, 1978a). Even further, it implies that the assumption of lexical entries - distinct entities stored at some specific location within the system and representing information about individual words - is not required.

Connectionist networks are capable of learning, which is based on the associative principle (repeated pairing of input with output). Some form of feedback is usually provided to the network after each trial, allowing it to adjust the weights on its connections, and thus make its performance more accurate. The network usually contains the intermediate level of ‘clean-up’ units to optimise the learning process.

The connectionist models of reading (e.g. Seidenberg & McClelland, 1989; van Orden, Pennington & Stone, 1990; Hinton & Shallice, 1991) may consist of input orthographic units (representing letters, letter triplets or onset-rime structures), output phonological units (representing phonemes, phonetic features or their triplets, onset- rimes); sometimes also semantic units corresponding to some arbitrarily specified dimension of the semantic space (such as ‘green’, ‘sw eef, ‘animate’, ‘flying’ etc.)

Arguably the most attractive aspect of connectionist models is the possibility of mathematical formalisation, which allows the models to be physically implemented (in the form of computer neuronetworks) and tested against the predictions coming from human studies. So far, parallel-distributed computer neuronetworks set up to simulate reading prove capable of replicating a range of effects observed in human studies of skilled word recognition (such as frequency by regularity interaction: Seidenberg & McClelland, 1989). Manipulating network parameters (limiting the amount of learning

experience, removing connections or hidden units, or reducing the rate at which information can be accumulated within the system) can simulate different forms of acquired and developmental dyslexia (Hinton & Shallice, 1991; Manis, Seidenberg, Doi, McBride-Chang & Peterson, 1996; Harm & Seidenberg, 1999).

From the developmental point of view, connectionist models are attractive as they effectively remove the problem of ‘bridging the gap’ between the learning process and the skilled performance. This is the problem faced by other classes of developmental models described here, and even more so by some models of skilled word recognition, which - like the dual route account - are essentially static in the sense of describing mainly the end product of the developmental process. Within the connectionist framework, learning and skilled performance are accounted for by the same structural framework of units and interconnections, and a unitary processing mechanism. Growth in competence corresponds to the gradual change and refinement in the pattern of weights. This itself can lead to qualitative change in the nature of performance, or the emergence of radically new structures. Connectionist modelling can also account for the role of phonology in reading acquisition (e.g. it can show how underspecified phonological representations impair the rate of learning: Harm & Seidenberg, 1999), individual differences in reading strategies, or compensatory mechanisms (which are understood to be the consequence o f changing network parameters, such as removing certain types of units or connections).

The blurring o f the distinction between instance-based and rule-based learning (that is, the assumption of a single processing route for all types of words) is perhaps the most hotly contested aspect of connectionist models. It is criticised both in the specific context of reading (Coltheart et. al., 1993; Zorzi, Houghton & Butterworth, 1998) and in the broader context of human language processing (Pinker, 1994) The dispute is not easy to settle in the light of experimental data on skilled human reading performance: it seems that both connectionist and dual-route models can account for the pattern of findings rather well, and no clear winner is in sight (although the protagonists of each view strive to prove otherwise: Coltheart et. al., 1993; Harm & Seidenberg, 1999). The decisive evidence is more likely to come from brain studies (neuroimaging of normal and impaired reading, and neuropsychological studies o f acquired dyslexias). It must be acknowledged, however (against the strong claims of connectionism), that the system that mimics human performance very well (such as a computer neuronetwork) may do so by employing learning mechanisms totally different from those used by the human brain. Indeed, at least the most popular class of connectionist models (implemented in

so called supervised networks: Seidenberg & McClelland, 1989) contains some elements that are not plausible in the context of normal learning to read (e.g. the assumption of immediate feedback following each and every trial) or some learning algorithms that are generally implausible biologically (e.g. back-propagation through time). Also, even those neuronetworks that are very successful in mimicking the qualitative features of human reading performance usually leam much slower (sometimes by several degrees of magnitude).

From the developmental point of view the connectionist account is problematic through its assumption of fully parallel processing. Although robust evidence from various sources (e.g. eye movement studies: Rayner, 1998) shows that skilled readers indeed process strings of letters in parallel under most circumstances, the same is not the case with apprentice readers who were often shown to use serial processing (e.g. sounding out and blending). The parallel-distributed framework, unlike other models previously discussed, does not leave obvious room for such strictly serial processes.

A solution to some of the problems discussed above may be offered by the recent development of connectionist dual process models of reading, which implement dual-route architecture (separate whole-word and sublexical processing mechanisms) in a connectionist network (Coltheart et. al., 1993; Zorzi et. al., 1998). Connectionist dual­ process models are capable of learning (they are, therefore, developmental, not static) and can mimic the acquisition of orthographic rules (i.e. grapheme-to-phoneme mappings) as well as serial decoding (e.g. through the deployment of a competitive queuing algorithm).

In the light of the general criteria for the evaluation of models of reading development adopted in the beginning of this section, no model is fully satisfactory, and every model type reveals its own unique profile of strengths and weaknesses. Stage models may be particularly adept at describing developmental changes in overt reading behaviour. Processes and resources models as well as connectionist models deal thoroughly with the relationship between phonological skills and reading. Connectionist models have the advantage of computational formalisation and deal most directly with the question of learning mechanisms. No single theoretical framework can, thus, be recommended as being fully comprehensive or generally superior to all the others. At present, some degree of theoretical eclecticism seems, then, inevitable in explaining reading acquisition. Indeed, some of the models discussed so far explicitly combined elements

taken from different frameworks (e.g. the stage framework and connectionism: Seymour, 1999; or a dual-route framework and connectionism: Coltheart et. al., 1993; Zorzi et. al., 1998).

CHAPTER 2