As discussed above, a cognitive model in a specific domain plays a crucial role in
developing diagnostic assessments. Hence, it is central to operationalize the test construct in any test procedure. This is especially true for a diagnostic reading test, as reading is essentially a private process that is internal to the reader, and difficult to define, describe, externalize, and observe. Thus, assessing the ability to read in L2 reading is presumably more complicated than in L1. This section defines the construct of L2 reading by presenting a brief, focused account of the relevant theories. Following a description and discussion of the construct of L2 reading, the
next chapter will demonstrate how the L2 reading construct can be operationalized in the specific context of the CDA in this study.
2.2.1 Overview of Research into L2 Reading
Whereas numerous theories and research can be drawn on regarding L1 reading
problems, this is not the case for L2 reading. However, much of the literature for L2 reading is still dependent on or derivative of the L1 literature. Several influential publications in L2 reading (Alderson, 2000; Grabe, 2009; Koda, 2005) have provided useful overviews of the main recent trends in L1 reading research and theories, along with their implications for the teaching and testing of L2 reading.
There are various ways to classify reading models in the literature; however, my focus is on two types of classifications: one aims to describe the actual process of reading and the other is concerned with the result (i.e., product) of the reading process (Alderson, 2000). While the former attempts to account for the dynamic relationship between text and reader, the latter is product-oriented and typically describes reading in static terms (Alderson & Urquhart, 1984). In the studies of reading, a distinction between product and process is meaningful. Alderson, Haapakangas, Huhta, Nieminen and Ullakonoja (2015) states that such a distinction was useful for the study of reading and reading problems, in particular for diagnostic tests of reading, which aim to diagnose examinees’ strengths and weaknesses in reading. Specifically, the product of reading is typically what readers have understood that is usually reflected as their answers to reading comprehension questions, whereas the process of reading is usually invisible and challenging to observe. Moreover, the process of reading is probably more variable than the product, because how reading comprehension can be achieved varies across individuals and occasions, even though the product might be same. For example, the overall test score can be
quite different from one test-taker to another, when we look at the individual items that yield the yield the overall score.
Moreover, the products and processes of reading can be affected by the task demands that are set to assess L2 reading. Reading tasks operate on a wide variety of texts and the choice of text (e.g., genre, topic, style, language, organization) affects task difficulty. As Grabe (2009) stated, readers are not simply bottom-up or top-down readers, since they have to be both bottom- up and top-down readers at the same time. Consequently, readers need to be able to recognize words automatically and fluently, and parse texts accurately and expeditiously. During this process, readers’ background knowledge should be used to set goals, and effective
comprehension strategies should be used to monitor reading comprehension. Obviously, all readers experience difficulty at some point during reading, thus the problem for diagnosis is to know where the difficulties exist. From this perspective, research on the design and development of a CDA-influenced test should put emphasis on the cognitive basis of L2 reading
comprehension.
2.2.2 Component Skills of Reading
As Alderson (2000) stated, researchers have different perspectives on the number and the content of the separable skills that comprise reading ability. Their perspectives on L2 reading can be generally summarized by holistic general-factor theories (Goodman, 1976; Thorndike, 1917a, 1917b, 1917c) and multiple-factor models (Davis, 1944; Gray, 1919). Both the holistic and multiple-factor models believe in different factors underlie reading comprehension, including either “vocabulary”, “decoding” or “literal reading” as the first factor and
“comprehension” or “inferential reading” as the second factor (Pettit & Cockriel, 1974; Stoker & Kropp, 1960; Vernon, 1962).
However, most reading researchers agree in general that reading skills are components of the reading process, and such skills can be isolated and tested separately, though there is no consensus on the number and content of reading skills. A traditional way this has been accomplished in L2 reading studies, especially in diagnostic tests, is to examine the skills or components of reading that are involved in reading comprehension. The componential models consider reading ability to be a separable trait, and attempt to identify the elements involved in reading.
As reviewed in Urquhart and Weir (1988), reading components were used in several studies such as Davis (1968: four skills), Munby (1978; 19 skills), and Grabe (1991; six skills). In particular, Munby (1978) proposed a multi-factor model, which argued that 19 micro-skills are required for reading comprehension, and this taxonomy has been very influential in language instructional materials and in test development. Comparing these studies reveals that although reading researchers have not reached agreement on the component dimensions of reading, considerable overlaps in reading skills exist among these studies.
Moreover, some researchers have suggested that there are hierarchical relationships among reading skill components. For example, Gray (1960) distinguished three reading skills: “reading the lines” which refers to the literal meaning of the text, “reading between the lines” which means the inferred meaning, and “reading beyond the lines” which refers to the critical evaluation of the text. This three-way distinction leads to an implicit hierarchy of levels of understanding: the literal level may be lower than the level of inferred meaning, which is again lower than the level of critical understanding. Based on this hierarchy, it is more difficult to attain the higher levels of understanding than the lower ones.
On the other hand, literal comprehension and inferential comprehension can be distinguished by using another classification: that of lower-level and higher-level cognitive processes of reading. According to Grabe (2009), reading consists of different types of processes, commonly called low-level and higher-level processes, and both are relevant to diagnostic
testing. Whereas low-level cognitive processes contribute to word recognition (e.g., lexical access, syntactic parsing and semantic processes), higher-level processes involve a set of skills and resources (e.g., background, topical and cultural knowledge), strategies, inferences, and the ability to monitor ongoing comprehension. Nevertheless, even at the same level process, the difficulty of skills varies. Clearly, literal comprehension is based on lower-level cognitive process of reading. In contrast, inferential comprehension involves using higher-level cognitive processes to construct what the text says and understanding what it is about. As Alderson et al. (2015) pointed out, it is important to distinguish between the ability to extract the meaning that the writer has attempted to convey (the text model of comprehension) and the interpretation the readers make (the situation model).
The notion that skills or components of reading can be tested is especially applied to diagnostic tests. For example, as reported in Alderson et al., (2014), three major and generally recognized skills (i.e., identifying the main idea, reading for detail, and inferencing) were isolated and tested in DIALANG, one of the first attempts at devising an online diagnostic testing system. Moreover, according to Urmston, Raquel and Tsang (2013), another recent diagnostic test, the Diagnostic English Language Tracking Assessment (DELTA), includes eight reading skills: identifying specific information, interpreting a word or phrase as used by the writer, understanding main ideas and supporting ideas, understanding information and making an inference, inferring the writer’s reasoning, interpreting an attitude or intention of the writer,
understanding grammatical relationships of words or phrases across text, identifying text type. As noted, both the DIALANG and DELTA contain items for measuring “vocabulary” separately. In addition, grammar is separately measured in DELTA as well. However, there are two issues in current research. One is they have failed to provide convincing evidence on whether expert judges generally agree on the skills being tested. Another is that no clear evidence is provided on whether the skills are organized in a hierarchical relationship, and whether a clear relationship exists between a good reader and one’s ability to master all or most of the reading skills. That is to say, the extent to which the presence or absence of such skills may explain strengths and weaknesses in L2 reading is what needs to be further investigated.
Another way that is parallel to the approach of identifying L2 reading components is to investigate the relationship between performance on L2 reading tests and tests of linguistic abilities (Alderson et al., 2015). Most commonly, reading abilities can be divided into grammar and vocabulary (Shiotsu, 2010). Weir and Porter (1994) concluded that vocabulary was generally considered a separate component from reading comprehension, though it might not be
consistently possible to identify multiple, separate reading skill components. Consistent with that conclusion, Perfetti (1985) reported that deficient word recognition was associated with poor text comprehension. Moreover, it might be challenging to maintain the simple dichotomy between vocabulary and grammar, as both are considered to be associated with good reading performance (Shiotsu, 2010). Hence, recent studies (Purpura, 1998; Urquhart &Weir, 1998) proposed that the knowledge of lexiogrammar plays a much greater role in determining reading ability at the lower levels of reading proficiency, whereas such knowledge is automatized and thus has less impact on higher levels of reading proficiency. As such, lexiogrammar has been proposed as an important component of L2 reading ability (Alderson & Kremmel, 2013). Therefore, both
vocabulary and grammar should be included as crucial components when dividing reading into separate skills, but the challenge is to investigate the nature of this relationship to find exactly what linguistic factors contribute to strengths and weaknesses in SFL reading.
In conclusion, identifying reading component skills can provide a useful framework to help in course design, instruction and test development (Lumley, 1993), though it is challenging to establish whether there are distinct component skills in reading comprehension and what those are. In addition, a reading test designed with a clear structure among skills can provide more fine-grained diagnostic information than one assuming no hierarchical structure.
2.2.3 The Cognitive Basis of L2 Reading
Reading is a cognitive activity involving skills, strategies, attentional and knowledge resources, and their integration (Grabe & Stoller, 2011). Thus, reading can be viewed as a dynamic process in which the reader decodes the written symbols to allow for the recovery of information from long-term memory to construct a plausible interpretation of the writer’s message (Kitsch, 1998). Cognitive processing theory, viewed as a foundation for examining the underlying processes of reading, remains the most influential in current work on reading ability (Tracey & Morrow, 2006).
2.2.3.1 Models of Reading Comprehension
A number of models for L2 reading comprehension have been proposed. This section selectively discusses the four most influential models: the bottom-up model, the top-down model, the construction-integration model, and the verbal efficiency theory.
The bottom-up model. In bottom-up processing, the letters, words and language features
in the text are decoded. Readers are assumed to be involved in a mechanical process where they decode the ongoing text letter by letter, word by word, and sentence by sentence (Grabe, 2009).
The text decoding includes the visual identification and combination of letters, word recognition, sentence construction via their syntactic structures and the final integration of sentences into coherent discourse. In this model, the reader’s word knowledge, contextual information, and other higher-order processing strategies play a minor role in processing information, especially at the beginning stages (e.g., Alderson, 2000; Koda, 2005). Word recognition, however, is
important, since eye movement studies have indicated that nearly every content word obtains direct visual fixation and the lack of even a single letter can be disruptive and decrease reading efficiency (e.g., Just & Carpenter, 1980, 1987). Further evidence can be provided by
developmental studies in that deficient word recognition is associated with poor comprehension (e.g., Perfetti, 1985).
The top-down model. A top-down model is based on meaning or conceptually-driven
operations. The reader’s schema (e.g., prior knowledge) for prediction is an important operation to understand in order to infer the meaning of the text. In this model, the primary goal of reading is deriving meaning from the text rather than mastery of letters, letter-sound correspondence, and words (e.g., Alderson, 2000). Readers are supposed to use meaning and grammatical cues to identify unfamiliar words, and according to this model they should be able to comprehend a passage even if they do not recognize each word.
The construction-integration model. Kintsch and his colleague (Kintsch, 1988, 1998;
Kintsch & van Dijk, 1978; van Dijk & Kintsch, 1983) proposed the Construction-Integration Model, which includes two interactive sets of processes: local text and text-modeling processes. While the former involves the processes that the reader uses to encode contextually appropriate meanings and propositions (Graesser, Millis & Zwaan, 1997), the latter is for combining the text representation with a reader’s background knowledge to fill the gaps in the propositional base
and facilitate the reader’s inference-making. Integrating the automatic lower-level with higher level reading processes, a coherent text representation at the discourse level can be created. The basic principle of this model is that text comprehension is partially constrained by the efficient operation of the local processes. Thus, skilled readers can free cognitive resources for
constructing and integrating meaning by benefitted from highly efficient word identification. In contrast, poor readers, constrained by their inefficient word-level processing, drain the limited attentional resources needed to maximize comprehension.
The verbal efficiency theory. Perfetti (1985) proposed this model as an example of an
interactive model that is highly constrained by the bottom-up view of reading (Hudson, 2007). Similar to the Construction-Integration model, efficient word-recognition skills are central to reading comprehension in this model. Essentially, this model believes that skilled readers and unskilled readers mainly differ in their efficiency at word-level processing in reading, since problems with higher-level comprehension skills originate from inefficient word-recognition skills which, in turn, stem from low-quality lexical representations. As Perfetti (2007) explained, this is probably because word recognition involves the cognitively demanding interaction of orthographic, phonological, semantic and syntactic processes, which are identified as four constituent information systems that work together and share information until a word is recognized. Consequently, readers who are good at word recognition leave much of their attentional resources free for higher-level reading processes which, in turn, results in better comprehension. Also, basic grammatical information can be extracted to support clause-level meaning and proposition formation.
While each model contributes to our understanding of the cognitive process of reading, the increasing specification of the role of cognitive processing in reading makes it possible to
understand the process of reading more clearly. The key difference between these models resides in the different emphasis they place on the text versus the reader. Each interpretation has its merits, but none is complete in and of itself in explaining reading. For example, the Bottom-up Model requires the reader to process the smallest linguistic unit (i.e., graph-phonic) first, before combining them to discover and comprehend the higher units such as sentence syntax (Alderson, 2000). This approach has been criticized by ignoring the involvement of higher-order reading skills and background knowledge in deriving and interpreting the meaning of the text. Moreover, the Top-down Model cannot identify the mechanisms readers draw on to make inferences
(Grabe, 2009). In addition, both the Verbal-Efficiency and the Construction-Integration Model take the reading process as involving both lower-level (e.g., word-recognition skills, syntactic parsing) and higher-level (e.g., making inferences) processes, and word-recognition skills seem to be important in both models. Moreover, both models treat working memory as a limited capacity pool of resources that is central and definitive in manipulating reading processes.
However, the two models differ in the nature and reading processes. The Verbal-
Efficiency appears to be more prominent in explaining only efficient word-recognition skills that result in automaticity during reading, however it is not yet clear to what extent individual
differences in working memory may explain differences in reading performance, and also whether the role of working memory may change as proficiency develops.
It should be noted that all four models reviewed here are limited to the cognitive aspects of reading that were considered most relevant to the specific context in this study. Obviously, the models demonstrate that working memory plays a central role in reading; however, due to the practical constraints on measurement, working memory is not included in the cognitive model of this study, as described in Phase One of Chapter Three. Similarly, other models that characterize
motivational and emotional aspects as important in L2 reading are also excluded from this review since it is beyond the scope of a CDA.
2.2.3.2 Multi-level Text Representation
As described above, reading comprehension is viewed as a dynamic process, which involves the building of coherent mental representations (Kintsch, 1998; van Dijk & Kintsch, 1983; Bransford & Johnson, 1972). This process can be elaborated in multi-level text
representation (van Dijk & Kintsch, 1983; Kintsch, 1998): readers form the basic idea units from words, connect the idea units in the form of propositions and retrieve associated knowledge or experience from long-term memory to construct an interrelated network.
Van Dijk and Kintsch (1983) proposed three separate mental representations during reading: the surface level, the proposition level, and the situation model. The surface level is the verbatim representation of the text that is characterized by the exact words and phrases used, thus readers only retain surface information. The proposition level is a semantic representation that describes the meaning of the text and represents the idea units directly connected in the text. The situation model is the mental representation of the situation to which the text refers, representing the knowledge schema used to assimilate it. The three levels of representation imply different levels of the processes, and the amount of information that a reader retains at a certain level can be used to judge whether strong representation at that level has been formed.
Researchers have argued that successful text comprehension requires the construction of a coherent situation model (Perfetti, 1989; van Dijk & Kintsch, 1983). They maintain that the meaning of a sentence cannot be understood completely until readers integrate this level of information into their existing system of knowledge or experience in long-term memory and thus construct a situation model. Theoretical support for such a hierarchical order of different levels
among text representation can be found in the Depth of Processing theory (Selfridge & Neisser, 1960). This theory states that perception starts with analyzing physical and sensory features and proceeds to later stages including recognizing patterns and extracting meaning out of the
stimulus, followed by even deeper semantic and cognitive analysis.
Perfetti (1979) applied the depth of processing framework to language comprehension by proposing seven levels of processing: acoustic, phonological, syntactic, semantic, referential, thematic, and functional. Comparing Perfetti’s products of these processes with Kintsch’s three- level representation, we find that Perfetti’s first three levels describe processing with Kintsch’s surface level information: the semantic level produces the meaning of a sentence, and thus corresponds to Kintsch’s text-based level; the processing of the referential, thematic, and functional levels is determined by the context and readers’ previous knowledge, which is comparable to Kintsch’s situation model.
While different levels of text representation follow a hierarchical order, text is processed