Support for the multiple semantics hypothesis from reaction-time

Paivio (1975; 1978) has claimed that evidence from reaction-time studies of normal subjects supports the view that there are multiple semantic systems. Paivio's dual-coding theory (1971; 1974) is very similar to the multiple semantics models that have been put forward by some authors to account for data collected from neurological patients. The theory states that knowledge is represented in independent but interconnected verbal and non-verbal systems, which are differentially accessible from verbal and non-verbal stimuli. Linguistic stimuli, such as printed words, have privileged access to the verbal system, whereas non-verbal stimuli, such as pictures, have more direct access to the non-verbal system. In addition, the two systems have different structural and functional characteristics. The verbal system is specialized for representing and processing linguistic information, whereas the non-verbal system represents and processes perceptual information concerning concrete objects and events. Furthermore, Paivio has argued that semantic representations within the non-verbal system have analog or continuous characteristics, rather than being digital or discrete. Hence he has argued that representations within this system "contain information that bears a continuous (analog) relation to the perceptual information directly aroused by the perceptual objects themselves" (1978 p. 199). Thus, for example, representations in this system would

reflect the relative sizes of the objects that they represent, rather than simply categorizing objects in discrete linguistic terms such as large, medium or small. Finally, Paivio has suggested that visual imagery is the mode in which such analog information is stored.

Paivio has claimed that a number of observations from reaction-time studies of normal subjects support the visual component of this dual-coding model. Two of these observations - symbolic distance effects and picture superiority effects - have received particular attention.

4.2.1 Symbolic distance effects

Moyer (1973) found that when subjects were shown pairs of animal names and asked to judge which animal was the larger, reaction times decreased as the difference between the rated sizes of the animals increased. Moyer and Bayer (1976) referred to this inverse relationship between the time taken to compare two symbols and the distance between their referents on the judged dimension as the "symbolic distance effect". The similarity of these results to those obtained when subjects are asked to compare the sizes of stimuli that are actually physically presented (Curtis, Paulos and Rule 1973) led Moyer to suggest that subjects make some kind of "internal psychophysical judgement" among memory representations. It was suggested that these representations have analog characteristics, in that the relationships that hold amongst them reflect the relationships that exist amongst the objects themselves.

In an extension of Moyer's study, Paivio (1975) found that reaction times were negatively related to the ratios of the rated sizes of the objects, suggesting that the memorial representations used to make such comparisons contain interval information about the degree to which objects differ in size, rather than just the ordinal rankings of objects along this dimension. This suggests that the long-term memory representations used to make such judgements have analog characteristics. Paivio has argued that the symbolic distance effects observed when subjects are asked to make size comparisons therefore support the existence of the visual component of his dual-coding model

However, Kerst and Howard (1977) argued that if comparisons along abstract dimensions also produced symbolic distance effects, then the distinction between visual and verbal processes in terms of analog versus discrete representations would be undermined. In their study, subjects were asked to compare pairs of animal, country or car names in terms of concrete and abstract dimensions. The concrete dimension used was size, while the abstract dimensions used for the three categories were ferociousness, mihtary power and cost, respectively. These authors observed a significant tendency for reaction time to decrease as the symbolic distance between members of a pair increased. In addition, it was observed that comparison times on both concrete and abstract dimensions were better predicted by the interval rather than the ordinal distance between test pairs. This suggests that comparison processes for both concrete and abstract dimensions involve analog representations. Kerst and Howard argued that the most parsimonious explanation of this pattern of results would be that both concrete and abstract knowledge is represented in a single amodal code which has analog properties. However, they also acknowledged that these data were not inconsistent with a model in which concrete perceptual information and abstract information are stored by different kinds of memorial representation (e.g. a visual imagery code and an amodal code, respectively) if it is posited that both codes have analog properties. In other words, concrete and abstract judgements could involve different mental operations which have similar temporal characteristics. However, these authors emphasized that the observation of a symbolic distance effect for abstract comparisons meant that the existence of the visual component of a dual-coding model could not be supported by the observation of symbolic-distance effects alone. The distinction between visual and verbal processes must therefore be justified by other sources of evidence if a dual-coding model is to be supported.

4.2.2 Picture superiority effects

Paivio has argued that the visual component of a dual-coding model is also supported by the observation that size judgements are made more quickly for pictures than for words (Paivio 1975). This pattern of results is predicted by the dual-coding model, as according to this hypothesis, pictures have direct access to visual-perceptual information about objects.

whereas words may only access this information indirectly, via the verbal system.

However, the claim that picture-word differences in response latencies support the distinction between visual and verbal systems has been contested. This is because picture superiority effects have also been found using comparisons on abstract dimensions Banks and Flora (1977) replicated the finding of faster reaction times for pictures than words in a size comparison task, but also found the same effect for comparisons on an abstract dimension (the intelligence of animals). These authors argued that the pictorial advantage in the latter task would not be predicted by a dual-coding model, as a response could not be computed on the basis of visual imagery. They argued instead that the picture superiority effect occurs because initial interpretation is faster for pictures than words, and that following this interpretation, processing is the same for both kinds of stimuli. A number of possible explanations for the initial advantage of pictures over words were suggested. For example, it was suggested that an advantage of this type could arise because pictures are less ambiguous than words, because they contain more information, or because they avoid a phonemic recoding stage.