Visual word recognition

Word production and word recognition differ fundamentally in that while the input in production is predominantly top-down, starting with the abstract concept to be expressed and ending with the specification of the phonemes in the target word, in recognition the input is necessarily bottom-up as initially-available phoneme/grapheme information is used to build up to the more abstract representations. Word recognition occurs once activation at the lexical level reaches the recognition threshold and syntactic and semantic information becomes available at this point (De Groot, 2013). This implies that grammatical gender information is available only

46 upon recognition of the word, in contrast to word production in which gender information is available significantly earlier in the process.

One of the most influential models, the interactive activation model (McClelland & Rumelhart, 1981;

Rumelhard & McClelland, 1982), is a connectionist model whose architecture has served as a base for numerous subsequent models (Gomez, 2012). In the interactive activation model (Figure 3.5) there are three levels of processing, visual features, letters, and words, as well as some higher processes that provide top-down input to the word level (McClelland & Rumelhart, 1981). This model assumes parallel processing, both in the sense that more than one letter is processed simultaneously and with regard to the fact that processing occurs on multiple levels at the same time (McClelland & Rumelhart, 1981). As its name implies, the interactive activation model also

Figure 3.5. The interactive activation model (McClelland & Rumelhart, 1981) of written word recognition (adapted from McClelland & Rumelhart, 1981).

47 posits that activation is interactive such that processing is bottom-up (input from visual features perceived) and top-down (input from processing above the word level). Activation in this model can be either positive or negative, with inhibitory connections within each level decreasing the activation of incompatible nodes.

In bilingual visual word recognition, one of the earliest and most widely-cited models is the bilingual interactive activation model (BIA) (Dijkstra & Van Heuven, 1998) and its extension, BIA+

(Dijkstra & Van Heuven, 2002). The BIA model (Figure 3.6) takes its fundamental architecture from the interactive activation model (McClelland & Rumelhart, 1981; Rumelhard & McClelland, 1982), making the same basic assumptions and adding a language level containing language

Figure 3.6. The BIA model (Dijkstra & Van Heuven, 1998) of written word recognition (adapted from Dijkstra & Van Heuven, 2002).

48 nodes for each of the bilingual’s languages (Thomas & Van Heuven, 2005). This model assumes an L1-L2 integrated lexicon in which activation is non-selective and therefore visual features and letters activate word candidates in both languages. Like the interactive activation model, there is lateral inhibition at the word level such that cross-linguistic candidates inhibit each other and inhibition also occurs at the language level since the model allows for top-down as well as bottom-up processing (Thomas & Van Heuven, 2005). In the BIA model, language nodes are both representational and functional. Representationally, they provide language tags which serve to label the language to which each word belongs. On a functional level, language nodes facilitate word recognition through the inhibition of words in the non-target language as a result of activation of the target-language node.

Figure 3.7. The BIA+ model (Dijkstra & Van Heuven, 2002) of written word recognition (adapted from Dijkstra & Van Heuven, 2002).

49 The BIA+ model (Dijkstra & Van Heuven, 2002) is an extended version of the BIA model that adds phonological and semantic representations as well as a language-external control mechanism to account for non-linguistic context effects in bilingual word recognition (Figure 3.7). In this extended model, sublexical orthographic (visual features) and lexical orthographic (letters) representations activate associated phonological and semantic information (Dijkstra & Van Heuven, 2002), as well as word candidates, rather than just simply word candidates (as in the original model). Furthermore, in the BIA+ model, languages nodes are reduced to their representational purpose and their activation is an indicator of the general level of activation in the lexicon. The functional role of these nodes under the BIA model as well as the non-linguistic context effects are absorbed into a task/decision system which interacts with the linguistic process of word recognition. This system encompasses the impact of instruction, task requirements, and other participant variables (Dijkstra & Van Heuven, 2002).

Neither the BIA nor the BIA+ models specifically include a representation for syntactic word information (lemmas) or grammatical gender information, however, Dijkstra and Van Heuven (2002) mention that lemma representations would fit into the BIA+ model between the word level and the semantic information and that the language nodes would then be connected to this lemma

Figure 3.8. The representation of gender in the BIA+ model (Dijkstra & Van Heuven, 2002).

50 level. Assuming that features such as gender are attached to the lemma representation, this implies that both gender and language information would be represented as nodes connected to the lemma level (Figure 3.8).

In sum, unlike with spoken word production, the most prominent models of visual word recognition build on each other and thus make similar assumptions. The interactive activation model posits that there are three levels of processing ‒ visual features, letters, and words ‒ and that processing of multiple letters and at various levels occurs in parallel. The BIA and BIA+

models are bilingual adaptations of the interactive activation model, taking the same basic architecture and adding language nodes (BIA and BIA+) as well as a language-external control mechanism (BIA+ only). Similar to the majority of the models of spoken word production, gender information in the BIA+ model is assumed to be represented as nodes connected to the lemma level.