Low Semantic Matrices Performance

The next task which will be considered is the low semantic matrices. This task was created to reduce the opportunity for semantic support. A first observation is that, under conditions of no interference, performance on this task is akin to the Size JND. Specifically, it is subject to rapid decay across the first 8.5seconds. This immediately suggests that the matrices may be maintained in the same system as the JND stimulus.

In contrast to the JND, performance on the low semantic matrices is impaired by a reduction in encoding time from 1500msec to 500msec. This suggests that encoding of the matrix is not equivalent to the JND, this is consistent with the matrices activating some elements of semantic memory. When STM capacity is exceeded (as

155

in the span procedure) executive resources are engaged to allow for chunking of information on the basis of semantic information (e.g. Jeffries et al, 2004). Although this allows for improved memory performance in the 1500msec encoding condition, this semantic elaboration is not sufficient to create a stable representation as it does in the high semantic task. However, the level of performance seen is still greater than that observed under executive interference, supporting the idea of some executively driven semantic elaboration. This supports a model whereby visual information and semantic information may be activated concurrently.

When performed as a 2-back task, it would be expected that the intervening stimulus would impair the visual representation (e.g. Della Sala et al, 1999). When visual capacity is exceeded participants should recruit executive resources to chunk information on the basis of semantic information. However, the 2-back procedure also denies the opportunity to recruit executive resources which are used in the aid of semantic elaboration (e.g. Jeffries et al, 2004). Under such conditions, participants are unable to represent the patterns and cannot perform the task, even at an entry level of 4. This suggests that any semantic representation that can be formed of the low-semantic visual patterns, is one which requires executive resources. Allen et al (2009) propose that representations in the Episodic Buffer may be bound by either ambient (automatic) or focussed (executively demanding) attention. It may be that the binding of information (during encoding) required for the low semantic patterns, where the semantic links are not as rich, recruits focussed attention which is demanding of executive resources.

When the intervening visual stimulus is removed in the 1-back procedure (Experiment 6), performance is severely impaired relative to control but participants are able to perform the task. However, given that access to executive resources is compromised, it is proposed that this level of performance represents the maximum capacity of the temporary visual system employed defined by only quantitative complexity (cf. Ichikawa, 1985). Ichikawa‟s definition of quantitative complexity was proposed to be defined by properties of the stimulus such as overall number of cells and certain types of gestalt property such as continuity. This type of complexity may be representative of pattern properties which can be chunked automatically. Baddeley (2007) proposes that executive resources are employed to increase capacity when slave system capacity is exceeded in verbal working

156

memory. It stands to reason that the same process could take place in visual working memory. In Experiment 6, where access to executive resources are denied the representation that is held in memory is representative of the capacity of a temporary visual memory representation, this could explain why there is a significant decay function. Performance on the low semantic under 1-back interference decays between 4.5 and 8.5 seconds, at which point it is equivalent to the high semantic matrices. It may be that a visual representation is formed, as in the Size JND but this representation is stored „offline‟ due to the executive interference and as such begin to decay. This would suggest the performance levels seen in experiment 6 on both the high and low semantic tasks are representative of the capacity of offline or unattended short term memory. It is possible in the high semantic patterns that this involves a verbal short term memory (as suggested by experiment 5) to maintain automatically chunked items.

The above suggestion is supported by the results of experiment 7, where attention is continually occupied during maintenance by the secondary task employed. Under such conditions task performance is equivalent to the 1-back task. Again, suggesting a maximum capacity of visual memory without executively driven chunking or support. This does however also suggest that the maintenance of the low semantic matrices is not dependent on sustained attention as the JND is. Continually capturing attention in experiment 7 is no more detrimental to span than the 1 back in experiment 6 where executive resources are employed only to update the contents of working memory.

As discussed above this could perhaps implicate the function of another visual memory system, and it was proposed that the Size JND is maintained online in a visual buffer (see Pearson, 2001), in the VSSP (see Baddeley, 2000) or within the focus of attention (see Cowan, 2005). It may be that in the absence of executive support for additional semantic elaboration (experiment 6), and when attention is occupied continually (experiment 7), the low semantic Matrices are stored in a different system or by different mechanisms to the Size JND. Perhaps the patterns are chunked automatically and held in an offline or passive store. For example, Pearson‟s Visual Cache, Baddeley‟s episodic buffer or outside of the focus of attention in Cowan‟s model.

157

Cowan (personal communication, August 18, 2009) proposes that the JND is maintained as a form of sensory memory. It may be that the low semantic matrices represent a different form of memory, perhaps activated LTM and are able to temporarily leave the focus of attention and be maintained in short term memory. Cowan (1988) proposed short term memory may contain a later sensory memory, which contains information that is partially interpreted but that is dependent on the sensory information and subject to rapid decay (Conway et al, 2001). This could be the system responsible for the maintenance of the JND. A second possible explanation is that this represents the function of an offline memory system such as Pearson‟s (2001) visual cache, which may contain information that is partially interpreted but that is stored without the need for attention (Pearson, 2006).

It was shown that verbal interference had no effect on the low semantic matrices in experiment 8. It is possible that the type of semantic support that is available in the low semantic matrices is not linked with a verbal STM representation; perhaps the semantic representation in this task is visual in nature, and driven by automatic processes but recruits executive resources to elaborate the representation on the basis of more abstract semantics. It is also possible that some verbal labelling of the low semantic patterns occurs, but that these labels are not necessary to facilitate performance.

DVN was shown to have a significant effect on the low semantic matrices in experiment 9. For the JND this was interpreted as interference caused by competition for the visual buffer. It may be that the matrices are stored online in the visual buffer to maintain high resolution representations or in the focus of attention. In the face of interference, it is possible that the representations can be shunted „offline‟ into a visual cache or outside of the focus of attention. The drop in performance under interference by DVN, may represent the change in the quality of the representation between online and offline representation of the matrices. In Cowan‟s model, attention may be recruited by the central executive or it may be captured by changes in the environment (Cowan, 1988). DVN may act on the latter by changing the focus of attention long enough to produce decay in „online‟ visual representations.

158

Awh et al (2007) showed that the number of representations and the resolution of the representations of a stimulus in visual working memory are not correlated, suggesting these are separable processes. They went on to show that capacity is constrained by the resolution of representations when the differences between sample and test stimuli are small and within-category. As sample-test similarity decreased, performance is less dependent on resolution. In the high semantic patterns, it is possible that the differences between sample and test stimuli are larger than in the low semantic patterns. If this is so, it may be that the low semantic patterns are more reliant on resolution and as such need to be represented in or refreshed by a similar system to the JND stimulus. It is likely that this is an online system, such as the visual buffer, VSSP or the focus of attention.

It was shown in Experiment 4 that a reduction in encoding time (and therefore a reduction in the opportunity for semantic elaboration) has an effect on the low semantic matrices. As such it may be that the „offline‟ representation is indicative of semantic representation of the matrix, which is less reliant on the high-resolution visual detail. This would also explain why DVN affected both the low semantic matrices and the JND to a greater extent than the high semantic matrices, as the latter are more able to rely on a richer LTM semantic representation when forced into offline representation.

In conclusion, the low semantic Matrices show clear evidence of activating some elements of LTM, again supporting a model where information passes through LTM or where working memory is activated LTM. The low semantic matrices, under no interference, appear to rely on the same mechanism as the JND, one which is susceptible to interference by DVN, requires attention and decays rapidly. However, 1-back executive interference and strong attentional capture have an equivalent impact on the low semantic matrices, suggesting the matrices can perhaps be represented offline more efficiently than the JND. This is thought to be representative of the matrices affording more semantic representation. Under interference by attention and 1-back interference, the offline representation may rely on LTM activation without attention. Cowan proposes this is possible, but that this activation will only be partial. The greater reliance on attentional resources in the JND supports the possibility for the low semantic matrices being represented as activated LTM, (e.g. Cowan, 1988; Phillips, 1974) and the JND stimulus being

159

represented in sensory memory. This could also represent storage in the visual cache in Pearson‟s (2001) model, which is stored separately from executive resources. A further possibility is one suggested by Allen et al (2009) where attention can be classified as ambient or focussed, and it is proposed that the episodic buffer can also be employed to store information via ambient attention, which is not dependant on executive resources. It may be that the partial semantic elaboration of the patterns under interference is due to this process, this will be discussed in greater detail below.