• No results found

4 Approaches to defining constructs for language tests

4.5 Theoretical models of language ability for testing purposes

4.5.2 Processing models

Processing models of language ability focus on the cognitive processes that people engage in when they use language. The approach is psycholinguistic and closely related to the psychological notions of memory and attentional capacity, which are relevant because language use happens in real time. Language users have limited short-term memory and limited attentional capacity, and while they use language, they are embedded in an interactive situation where both language and other activities are going on at the same time. Processing models try to specify what is going on in the language user’s cognitive system, what they pay attention to, and what their language resources are. In the field of second/foreign language ability, processing models tend to be learning-related, which means that theorists particularly focus on learning tasks, especially ones which might be considered to enhance language learning.

The processing approach to language ability for language testing purposes has recently been discussed by Peter Skehan (eg. 1998a, 1998b). The current version of his theory of second language learning builds on previous work in psycholinguistics on attention, noticing, and lexicalised processing (eg. Nattinger and DeCarrico 1992; Pawley and Syder 1983; Robinson 1995; Schmidt 1990, 1993; Van Patten 1990) as well as data

from a series of studies by Skehan and Foster (1997, 1998, Foster and Skehan 1996, 1997), in which learners were engaged in paired interactions. Skehan and Foster used different task types and varied performance conditions in terms of planning and post-task operations, transcribed the learner performances, and analysed the transcripts through a rough operationalization of fluency, accuracy, and linguistic complexity.

Skehan (1998b) begins his presentation of a processing perspective on second language learning from a theory of attentional priorities in learner performance. Building on Van Patten’s (1990) distinction of form and meaning as relatively independent features to which language learners need to pay attention, Skehan makes a further distinction within form between focus on accuracy and focus on complexity. The relative independence of the three factors is given some support by Skehan and Foster’s empirical results (1998b:71-72). Skehan presents the case that different kinds of tasks and performance conditions call for different balances of attention to the three factors. Because of this dual focus of individual processing when the individual engages with tasks, Skehan’s approach can be considered interactionalist. However, he is clearly concentrated on the fundamental processes aspect of learner factors and the processing requirements of tasks.

The task characteristics that Skehan (1998b:79) identifies as important for learner performances are familiarity of information, degree of structuring in task, number and complexity of mental operations required, complexity of the knowledge base that the learner needs to draw on to respond to the task, and degree of differentiation in task outcome. The more familiar the information on which the task is based, the more fluent the performance. Clear sequential structuring in the task, such as narrating a story or giving someone a set of instructions, leads to greater fluency and accuracy in performance than a task which lacks such structuring. If the learner needs to make transformations to the task material, such as combining pieces of background information or creating links between instances, it may require more complex language, but it will reduce the amount of attention available to accuracy and fluency. If a learner needs to consider a complex set of perceptions to explain their point of view, it will require them to use more complex language than would be needed for the expression of simple or clearly structured information. And if there is only one possible outcome for the task, the language that learners use will be less complex than in tasks where there are several different outcome options, any of which would be equally “correct”. Skehan (1998b:80)

points out that these dimensions are only likely tendencies, not laws, because there are no hard-and-fast laws in rules of language use.

The differences between Skehan’s (1998b) and Bachman and Palmer’s (1996) task characteristics are striking. While Bachman and Palmer describe task settings physically in terms of location, duration, or lighting, and task language in terms of its syntactic and textual characteristics, Skehan describes the operations which learners engage in to complete the tasks and predicts their effect on learner language. Yet both sets of characteristics can be used by test developers for the same purpose, namely to categorise tasks for the purpose of covering enough variation in language use within a test and creating parallel tasks for different versions; they result in complementary perspectives to the description of examinee ability.

Skehan’s contribution to the modelling of language ability for testing purposes is a closely data-based relationship between competence and performance. Bachman and Palmer model this through various kinds of knowledge (or competences) mediated by strategic competence, but they state (1996:62) that their constructs are not directly related to processing. Skehan, instead, focuses on processing. He takes up McNamara’s (1996) model of performance testing and extends it further by including processing-based dimensions to task description as discussed above. Furthermore, he specifies that the candidate’s underlying language competence, while probably relevant, is mediated by a dual-coding system for language as well as ability for use. With dual-coding, Skehan means speakers’ use of memory-based, lexicalised language as a default in online situations such as spoken interaction, where processing demands are quite high, and their use of syntactic processing for precise and clear expression when the task requires it and processing resources allow it. This follows the work of those who apply cognitive theories on language processing, eg. Pawley and Syder (1983), Skehan (1998a), and Widdowson (1989). The point that Skehan makes is that inferences about learners’ language competence based on their performance on a test are mediated by so many factors that interactive, processing constructs might be more useful bases of generalisation. Skehan does not advance new theories for ability for use, but restates McNamara’s call that such a model is necessary for performance assessment (1998b:84).

The implication of Skehan’s processing model for the characterisation of constructs in test development is a call to pay attention to the processing dimension. In practice, this means that processing-based task characteristics and task conditions should be considered when tests are

being developed. It might also mean that fluency, accuracy, and complexity are used as scoring criteria, at least for oral tests. These are tied to a cognitive processing model of human activity, where the main interest is on attention, performance conditions, and performance features, while the underlying competences of individual learners have a less central role (Skehan 1998a:155). If test developers choose to make processing constructs central to their test, these also form the basis of generalisation for their test results; in other words, they might be able to specify quite concretely the types of language use tasks to which the scores should generalise. However, it would be less likely that they would state the results in terms of the participant’s language competence.

Constructs like Skehan’s may be most useful in educational contexts as a means for providing diagnostic feedback to learners. It may be more difficult to use these constructs in large-scale examination contexts unless the score users are prepared to accept such highly task-related constructs. Whether test developers choose to align their thinking about the test construct along processing dimensions or not, the existence of such an alternative at least encourages them to think about why they mention the kinds of constructs they do and what the relationship is between their constructs, task characteristics, assessment criteria, and scores.