Creating categories based on purpose

As discussed in Chapter Two, a major issue in multimodal research is both the process of intersemiotics (i.e. the interaction between different semiotic modes) and the identification of which mode was the focal communication mode (Kress & van Leeuwen, 2006). The focal mode is the one to which other modes are deemed subsidiary (Márquez et al., 2006). One coding structure that sought to achieve this is the one shown in Figure 3-16.

Figure 3-16: Focal mode by excerpt (Márquez et al., 2006, p. 218)

Here Márquez et al. (2006) looked to analyse meaning making and meaning transmission in the science classroom and in particular how “three communicative modes – language, gesture, and image – contribute to the construction of meaning” (Márquez et al., 2006, p. 206). They started their analysis of the basic transcriptions by identifying ‘interactivity excerpts’ characterised primarily by the thematic content.

Each excerpt was in turn broken down into a semiotic space and the process adopted within that space. The thematic spaces were:

Thematic space (TS). Every meaning that is related to the topic under study, every process that gives meaning to conceptual aspects. So our thematic space is water circulation in nature.

Classroom management space (CMS). Every meaning that relates to organization of the classroom as a communicative and social space where it is necessary to organize participation, time, order of the interventions etc.

Representation management space (RMS). Every meaning that relates to the strategies used by the teacher to help students construct a water cycle diagram. (Márquez et al., 2006, p. 208)

These three spaces were then broken down to reflect the activity in each excerpt (Márquez et al., 2006, p. 209). Two of these spaces, “Processes related to properties and characteristics of water in nature” and “Processes related to water changes and causes of water circulation”, can be seen as means to describe the scientific process under investigation which in the taxonomy above is the thematic space. A third space, “processes related to the control of students’ participation”, relates to how the classroom management space was adapted as the classes progressed and different learning and teaching modes were adopted. Finally the representation management space was broken into three specific processes:

1) “Processes of naming water cycle entities”;

2) “Processes related to the management of the water cycle diagram”; and 3) “Mental processes” (Márquez et al., 2006, p. 209).

The thematic space is the underlying scientific knowledge and the representational space is the process by which the students and teacher interact to construct that understanding. Each section was then assigned to one of these modes. A secondary stage in their coding is to move away from describing the gestures in detail and instead assign the gestures to a category, with this in turn related to the relevant semiotic space. So in terms of the thematic space, gesture is often coded as indicating a dynamic process or flow, while in terms of classroom management, gesture is used to control the class, and in terms of representation, it is used to guide learning (tutor points to the diagram) or to confirm that students have given a correct answer. Overall, to achieve this they reduced the actual detailed speech and gesture to one of four semiotic resources: speech, gesture, visual language and written text (Márquez et

al., 2006). This allowed an understanding both of which modes were most dominant in which spaces and processes, and how these interacted. So, for example, speech- dominated in the thematic space but gesture was the dominant mode in the classroom management space, as shown in Table 3-3.

Table 3-3: Use of different modes for different roles (Márquez et al., 2006, p. 214)

Márquez et al. use a reduced range of descriptions to capture the semiotic resources for the process of analysis. As is shown in Table 3-3, they start from a rich, very precise description of what each gesture consists of, but for analytic purposes these are conflated into a single category ‘gesture’. Since the main purpose of the paper is not to describe in detail how gesture is built up (Xiong & Quek, 2006), but instead to discuss how semiotic resources interact in the science classroom (and which, if any, are dominant at a particular stage), this is an appropriate choice.

agreement as to which categories should be used. Kress et al. (1998) suggested that four modes of talk, images, gesture and the material apparatus were the key modes in the science classroom (Kress, Ogborn, & Martins, 1998). The high level coding scheme used for reporting is, as with Márquez et al., derived as much from the factors deemed to be of interest in building up the research as from any conventionally agreed listings.

This style of analysis allowed consideration of when the different modes were co- operative (i.e. used to present the same information) or different as they were used to present different information. In the example above, speech and gesture have a shared role in managing the functioning of the classroom (Márquez et al., 2006). However, in terms of the discussion of the knowledge content (thematic space), speech and gesture are playing different roles. While speech is presents the core information, gesture is being used to show the flows and dynamic element.

Jaipal (2009) developed this approach, and in particular the overarching stage structure offered by Márquez et al. (2006) by first positing that the entire communication serves, at various stages, one of four functions. Where Márquez et al. split this categorisation into their three spaces, Jaipal suggests four functions:

• Presentational Function (Conceptual Aspect of Meaning), which could be mapped onto Márquez et al.’s Representation Management Space;

• Orientational Function (Social Aspects of Meaning) which could be mapped onto Márquez et al.’s classroom management space, but includes elements such as “how the voice of the teacher and text position the learner in relation to science. For visual graphics, typographical tools such as italics and boldface emphasize importance and act as orientational tools” (Jaipal, 2009, p. 52);

• Organisational Function (Pedagogical Aspects of Meaning), which again could be Márquez et al.’s Representation Management Space; and,

• Epistemological Function (Nature of Knowledge) which broadly reflects Márquez et al.’s Thematic Space

Thus even before coding the actual semiotic resources for analysis, Jaipal (2009) has created an overarching coding scheme that can be roughly mapped onto that of Márquez et al. (2006), but also differs in key respects. This is not to argue that either is wrong, or that either is better in an abstract sense, but it does indicate that the process of creating typographies and categories to report on multimodal discourse is complex, and that there is a lack of predefined elements that are shared across even

similar studies, in this case both studying how the different resources are combined to create meaning in the science classroom.

Having created the categorisation scheme (the four functions), Jaipal gathered data using a mixture of audio and visual recording, interviews as well as direct observation. The first stage of categorisation was to “identify data units where multiple modalities were used to teach different concepts” (Jaipal, 2009, p. 55). This was then broken into an ‘event map’ to show how excerpts of talk are related to specific phases of the teaching cycle. The modalities captured included:

verbal language, written text, gesture, diagrams, written equations, and visual recall imagery. Visual recall imagery is a term I use to describe narratives about real events, analogies, metaphors, evocative descriptions, and demonstrations that the teacher asks students to recall, visualize, and imagine. (p. 55)

Again this is not a dissimilar scheme to that followed by other studies (Kress et al., 1998; Márquez et al., 2006), but it is different and reflects the focus of the particular study. An example of how these different modalities were related to the Epistemological Function is offered given in Table 3-4.

Table 3-4: Time sequence, focus and modalities in use (adapted from Jaipal, 2009, p. 56)

February 1 Chemosynthesis 1. Textbook diagram

2. Written equations 3. Visual diagrams on board 4. Recall everyday examples 5. Gestures

6. written definitions

Again, as with other studies, there is a process of gradual reduction from the rich raw data to a form that is suitable for further analysis. As an example, the following excerpt is summarised as a ‘Verbal comment and gesture to reinforce meaning of oxidation’ and grouped with other similar elements to indicate the phase of a lesson: “Teacher: It releases quite a bit of energy (moves both hands outward in a wide circle). You know the perfect example of that” (Jaipal, 2009, p. 57). In this case, the excerpt is split between ‘gestures’ and ‘recall everyday examples’. In turn, the lesson was summarised and mapped onto Jaipal’s four functions as shown in Table 3-5.

Table 3-5: Sequence of semiotic modalities and functions (Jaipal, 2009, p. 59)

Two papers discussed in some detail above (Jaipal, 2009; Márquez et al., 2006) offer useful insights into both the process of transcription from the raw material and the organisation of the transcribed information into categories that are useful to explain the nature of the semiotic interaction being studied. It is noticeable that both rely on verbal transcription and description to capture non-verbal and verbal modes (Norris, 2004a), rather than linking a text record to visual images (O’Halloran, 2011b). Both develop an analytic framework but they use two different constructs. Whether they use the concept of spaces (Márquez et al., 2006) or functions (Jaipal, 2009), both indicate what they see as the main building blocks to describe the interaction they will study. However, they make a distinction between the scientific knowledge under investigation, the process by which meaning making takes place and the domain (classroom) within which this will take place.

The papers both show the process by which they develop a coding scheme that moves from a rich, detailed description of a particular moment in the session up a hierarchy (losing detail at each stage) until both use a very abstract concept, such as ‘gesture’, to capture a range of different events. This process can be traced, and presumably challenged if seen to be inappropriate, from the basic material to the aggregated description. However, the key is that both studies were interested in the interaction between semiotic modes (not the detailed descriptions of how those modes were made up), and as such, a high level of abstraction was appropriate. In summary, the research focus produced the appropriate final coding scheme, not the application of an agreed classification process.