Applying and presenting corpora

3.6.1. Key requirements

The final stage in corpus construction concerns the application and presentation of data. In other words, it seeks to address how corpora are presented to the end user, once data has been collected, transcribed and coded. The notion of the (re)presentation of data is heavily reliant on the software used by the corpus developers, as this determines how the data, including the raw video and/or audio files; transcripts and separate coding tracks; metadata; header information and so on, is arranged within the software’s infrastructure. The software also determines how the data is navigated, searched and interrogated in screen. Again, as with previous stages of development, it would be preferable if the conventions used at this

stage were universal, however at present this is not the case as a range of different forms of corpus software exist.

Having said this, most current corpora are integrated with a key functionality which operates in a similar way across each individual database; a text concordancing tool. An example of a typical concordance output is seen in Figure 3.7 (taken from CANCODE).

Figure 3.7: An example of 3rdgeneration corpus concordance outputs.

It is this which when coupled with search and word count facilities, allows the user to research statistical or probabilistic characteristics of corpora, together with exploring specific lexemes, phrases and patterns of language usage in more detail. At the click of a button, appropriate citations of speaker information, socio-cultural context of use and further details of the specific conversation in which each search term, line and/or turn occurs (as presented

in concordance output), can usually be accessed. Some of this is the information that forms part of the metadata content of the corpus.

The key limitation with such concordancers, however, is that they are only able to interrogate transcripts and text files, and not MM and/or ubiquitous datasets, as there is a scarcity of concordancers that deal with MM corpora specifically. For the advancement of 4th generation corpora it is vital that this void is filled and capabilities for conducting corpus-based searches of MM data are enhanced. However, this process is no mean feat as with the onset of MM, multi-media datasets present a whole host of technological challenges for the synchronisation and representation of multiple streams of information.

In an attempt to construct some guidelines for software which allow for the presentation and interrogation of MM datasets, in addition to the coding, organisation and management of such, the following key requirements were established at the start of the DReSS project. Although these principles act as benchmarks that were specifically constructed with the NMMC in mind, they can be seen to be valid beyond the remit of this corpus, and act as useful prescriptions for other MM corpora (see Knight et al., 2005: 12):

 Multi-modal: Allowing for the analysis and exploration of data from a variety of multimedia (sound and visual data) simultaneously, both within a single frame and a combined frame of reference when desired.

 Accessible: It should be integrated with a user-friendly interface to access and search specific frames or sequences of frames.

 Proficient: To be able to synthesise, tag, code and transcribe large quantities of MM datasets.

 Flexible: Allowing the interrogation of specific frames or sequences of data, as well as allowing the exploration of specific modes of data.

 Systematic: It should enable accurate and systematic searches and statistical analyses of spoken and visual records to be undertaken with ease.

3.6.2. Presenting multi-modal corpora in DRS

In light of these requirements, it should be noted that what sets DRS apart from ANVIL (and the other tools mentioned above) is that it is integrated with a fully MM search and concordancing facility for text and video data. Furthermore, it is also integrated with a facility that allows users to conduct basic text-based word frequency searches of corpora. So, in addition to providing the standard mono-modal concordance facilities seen in current corpora (as depicted in Figure 3.7), this MM concordancer allows users to search for gestural codes within the output. This provides an easy point of access for analyses of patterns of behaviour across the different modes, highlighting the tool’s accessibility. No other multimodal analysis or annotation tool is equipped with this facility at present. For this reason, DRS currently exists as the most suitable tool for MM linguistic corpus development and presentation. An example of the concordancer search facility is seen in Figure 3.8.

Figure 3.8: Exploring backchannel behaviour using the DRS concordancer.

In this figure, the standard text based concordances of yeah are presented. As the ‘select code’ box, in the top right corner, is enabled and a given gesture code is selected (in this case small nods of a short duration), relevant concordance lines are highlighted indicating where the search term and the specific coded gesture co-occur in close vicinity. In this case the figure indicates that where yeah is uttered and a small nod of a short duration is also enacted at some point within this turn.

Using the DRS concordancer, it is possible to search around the immediate environment of textual concordances using the right-click mouse facility. This enables the user to directly access the time-stamped segment of a transcript, and associated position in the text and video where specific events occur or where a particular lexeme is uttered.

At present, the tool does not allow for searches of specific gesture codes directly within the concordancer. Future releases will hopefully enable this line

of enquiry, for example, allowing users to search for specific gesture codes such as <NOD> and to calculate the frequency of these in the given text(s).

Since the current thesis is corpus-based and thus relies on these sorts of concordancing applications for research, DRS is invaluable. However, given that the concordance tool and related functionalities including the word count facility, are still relatively new components, its reliability may be questioned because extensive testing of this functionality has yet to be carried out. As a result, both the case study and extended five-hour datasets used in this thesis do not utilise this frequency tool, although it noted that this application is sure to be invaluable for MM corpus research. Since no alternative MM concordancers or frequency tools exist at present, the majority of the searching and counting conducted here has been undertaken manually (see Chapters 4 and 5 for further details).

3.7. Summary

This chapter has explored the processes of developing MM corpora, drawing on a wide range of issues and methodological considerations that need to be addressed; from the process of recording MM conversational data through to its’ representation and re-use. Although this developmental methodology is by no means definitive, it has provided a context to MM CL research, by outlining some of the key practical, technological and ethical questions that are faced.

Effectively, this research provides a background for the second focus of this thesis; the actual implementation of such corpora. Chapter 4 examines this matter in more detail. The chapter outlines a refined approach, a framework, one which enables accurate and relevant analyses of MM corpus

datasets to be undertaken, taking ten minutes of case study data as a means for doing this. This analytical framework is, in turn, used as the basis for analysing 5 hours of NMMC data in Chapters 5 and 6.