Application of the geosemiotic framework

Through analysis of the interaction order, visual semiotics and place semiotics the geosemiotic scheme outlined by Scollon and Scollon (2003) will help to account for the complexities of

expressing voice in a video conferencing environment. However, any framework requires careful application as a tool kit for analysis, as it may presume relationships within the data that do not fit with the particular context of this study. Geosemiotic frameworks have mostly been applied to the study of prefabricated instances of communication such as street signs, advertisements or films. In this analysis of children’s communication through telecollaboration meaning is constructed through purposeful interaction which can be serendipitous. In the current analysis greater emphasis is placed on the interaction order and in particular by the children’s display of personal fronts to express their voice. These aspects of the semiotic scheme are flexible and dynamic in Skype communication and reflect how children deal with the responsive nature of voice. The children’s activity through the interaction order might be in keeping with or contrasting to visual and place semiotics.

In order to identify the relationships between the different semiotic systems described above the data was first recorded and analysed from a very broad perspective. Table 1 shows a relatively low magnification of the data. These transcriptions relate to the initial data management in phases 1 and 2 of Creswell’s data management spiral (see Figure 1 above). A key function of this type of transcription is to capture a broad picture of the categories running through the video data. This requires a macro-analytical approach to reconstructing the data ‘which attempts to capture the meaning making processes of complete texts in terms of the links between the various sub units that make up a text’ (Baldry and Thibault, 2006, p. 166), using the categories of the interaction order, place semiotics and visual semiotics, which make up the geosemiotic framework (Scollon and Scollon, 2003). Thus, in the coding chart (Figure 4) the transcription focuses on reconstructing the ways in which social action (the event) is anchored to the world through these three broad geosemiotic systems. It takes into account the relationships that exist between the main semiotic systems which in turn guides a more detailed analysis of the data by looking at subsystems within each category during stage 2 of Creswell’s model for analysis.

Table1.Excerpt fromthe multimodal codinc chart

The representation of the video data would be very crude if the analysis was to go no further than the fairly flat model provided by Figure 4. Each semiotic mode works in a different way to contribute meaning to a voice depending on the media through which the mode is expressed (Baldry and Thibault, 2006). From a macro view of the data it is easy to underestimate the significance of a particular mode as it becomes part of the integrated whole of social activity.

To see the different modes as they appear in the data requires a selective form of multimodal transcription (Baldry and Thibault, 2006). For example, Figure 2 shows excerpts from the same timeframe of data transcribed and analysed using the chosen semiotic framework. This stage in the process of transcription and analysis relates to the third phase in Creswell’s diagram (Figure 2, see also Appendix 6 for an outline of the codes used in the transcription). Each table is divided into columns, with time being the foremost column. The time sequence, therefore, provides a reference point that allows each table to be compared with another. The next column provides a screen shot of the visual information at certain points within the time frame. Unfortunately every frame from the video data could not be shown so the image represents what is happening within a time period.

The subsequent columns are headed with the anonymised names for the children in the exchange.

Those highlighted in blue (Anna and Beatriz) are from Portugal and those in pink (Claire, David, Ethan, Fiona, Gary and Heidi) are from England.

Each table focuses on a particular subcategory (the examples in Tables 2 to 9 show verbal, interpersonal, time, place and modal meanings for the same timeframe of data respectively). The black written text in each column beneath the children is the verbal transcription. This text remained on all the tables as it is an important mode of communication and also provided a helpful reference point to relate one table to another and also relate each table back to the original video data. The coloured text (blue, red and green) shows the researcher’s observations of a particular category. In Table 5 the subcategory of place semiotics is annotated in different colours to distinguish aspects of the visual system that relate to the built environment (the red text) and those aspects that relate to the wider context of time and space (the green text). In Table 6 a similar approach is taken to distinguish aspects of colour and illumination (the red text) and how they are located in the visual system (the green text). (See also Appendix 6.) These micro-transcriptions of 81

N ic h o la s A u s ti n Y 2 8 7 2 1 5 6 V id e o C o n f e r e n c in g a n d M u ltim o d a l E x p r e s s i o n o f V o ic e

the data are designed to carefully reconstruct the ways in which each mode of com m unication is

interw oven with the constantly changing visual image over time. By teasing out the role played by

each mode it is then possible to understand the relationships betw een them and, thus, to determine

how they are organised through the expression of children’s voices in the data.

Table 2. Excerpt from the transcription and analysis of spoken language (timeframe 17:35 - 17:55)

T is m * Scrw n Shot

iwilo, tt

*nar»i J «x;!>0ni4or»Ou

17:35

Own*; you tfWflkyOU Own*

y o t're #e'-zorr e

vcvwr-.etwoof re'naeen*

Table 3. Excerpt from the transcription and analysis of interpersonal distance (timeframe 17:35 -17:55)

Table 4. Excerpt from the transcription and analysis of the sense of time (timeframe 17:35 - 17:55)

.

N ic h o la s A u s ti n Y 2 8 7 2 1 5 6 V id e o C o n f e r e n c in g a n d M u ltim o d a l E x p r e s s i o n o f V o ic e

Table 5. Excerpt from the transcription and analysis of place semiotics (timeframe 17:35 -17:55)

Tim* Screen Shot

i*

■ O T M >nd I'm

o u e itia w you

you're welcome a n you r a n e t w o S#

S a n ta s re iro e e rs ’

CftudOlplM

Table 6. Excerpt from the transcription and analysis of modality (timeframe 17:35 - 17:55)

h f S . IS

twneta»ULU8B£- qjeit-oni y >-;

snankyou ^you

t t W f t welcome can

you n»mt Two of

Santa's reindeeri*

(Rudolph)

The last phase of data analysis is representing and visualising the data. At this point it was

appropriate to revisit the main corpus of data and review the analysis in relation to the bigger

picture o f the recorded data. Table 7 is an example o f how this body of data was review ed and

entextualised. Each of the 14 recorded Skype sessions from both the pilot and m ain study were re­

These incidents were subject to further m icro-transcription to investigate how closely they relate to

the themes first discovered in the analysis of the social event delim ited by the initial short time

span of data. An example is Figure 30, which is the m icro-transcription for the exophoric link

identified at 04:57 in Table 7. The following chapter will describe the patterns of m ultim odal

resource use that emerged through analysis of the data and that illustrates how the children express

their voices.

Nicholas Austin Y2872156 Video Conferencing and Multimodal Expression of Voice