Data Analysis

Data analysis is "an ongoing process" (Taylor and Bogdan, 1984:128) that starts with and goes "hand-in-hand" with data collection (Taylor and Bogdan, 1984:128). In this study, throughout data collection, field notes and transcripts were read in an attempt to make sense of the data. The process of data analysis in this study followed two approaches. First, data from the questionnaires and interviews was analyzed with an inductive approach to make sense of it without imposing pre- existing views on the meaning. Second, with observation of sessions and interactive materials a deductive approach was implemented through the use of a checklist or

criteria derived from the literature presented in Chapter Two. The specific steps that were followed to analyse data involved the following stages.

3.11.1 Organising and preparing data for analysis. The purpose of the arrangement of the raw data is to organise the data from different sources into manageable formats. Different organisation systems can be followed in this stage. In the present study, the data collected were organized from each source (Web-based materials, observational notes from online sessions, interviews, questionnaires) individually into separate folders. Student interviews were transcribed using pen and paper and saved in a folder. The details of each interview such as name of the participant and their random numerical identified, date, and duration of the interview were recorded at the beginning of each transcript.

The data from automated materials was also organised. The data was saved in two documents. The first document included screenshots and comments that were relevant to the content of the materials (e.g. writing activities, grammar exercises and so on). The second document included screenshots and comments about the resources and tools that were integrated in these materials (e.g. library, forum, etc). The narrative and analytic description of these materials was based on and illustrated by the visual data and comments in these two documents (Ellis, 1997).

The first task with the questionnaires before coding was to pull the answers to the open-ended questions together. Questionnaires were organized by question then printed and coded using pen and paper. All the questionnaires were examined and the answers grouped under the related questions. For example, one of the open-ended questions was what do you like about the English course. This question was written on a separate page followed by all the answers given by the students to this question.

This technique made analysis of data easier to carry out as it allowed analysis to consider the answers of all participants for an individual question.

3.11.2. Holistic analysis. All recovered data was read through to observe the data from a holistic perspective. This assisted familiarity with the data and reflections on overall meaning. This process involved "conversing" with the data, asking questions, and making comments (Merriam, 1988:131). The process of data analysis of the interactive materials and observation data followed a deductive approach (top-down coding). The process of data analysis of the interviews and open- ended questionnaires followed an inductive approach (bottom-up coding). The procedures followed in each type of these analyses will be outlined in the following section.

3.11.3 Coding of observations and materials. The process of data analysis of the observations and automated materials was guided by the research objectives and the research questions. Themes related to these data were arrived at deductively. Coding categories were defined in advance (Stigler et al., 2000). This analysis process included using a criteria or a checklist (Ellis, 1997; Williams, 1983) and followed a process adapted from Hutchinson and Waters (1987, as cited in Zhang, 2007: 30)

Step 1. Defining criteria for examining the data by exploring the interactive materials and observation data. Criteria included the goals that characterize traditional and new approaches of language teaching in general, and CALL, in particular.

Step 2. Examining the data according to the defined criteria to decide to what extent the materials and online sessions achieved these goals.

The process of data analysis resulted in narrative and analytic descriptions of the interactive materials and online sessions. The analysis was later used to draw conclusions about the similarities and differences between individual study and working with the faculty.

3.11.4 Defining criteria. Defining criteria for analysing the materials and observation data (Mayring, 2000) was developed following a model of deductive application. The resulting step model was used in this study as a framework for deductive data analysis. Mayring (2000: 15) stated that the main idea of these coding procedures "is to give explicit definitions, examples, and coding rules for each deductive category, determining exactly under what circumstances a text passage can be coded with a category. Finally, those category definitions are put together within a coding agenda." The criteria included five categories: Feedback, Organization of the Lesson, Virtual Setting (e.g. technology used), Stage (teaching) Content and Goals, and Role of Participants (faculty, students). From the perspective of this criteria, the process of teaching and learning was viewed as " an integrated set of characteristics, including tasks, discourse, and particular roles for faculty and students" (Stigler, 2008: 137). These elements reflected assumptions about the way students learn and the proper role of the faculty (Stigler, 2008).

The criteria served as a framework to guide and interpret the analysis. During the analysis, the categories were presented in the form of questions about the nature of teaching as described by Hutchinson and Waters (1987, as cited in Zhang, 2007). Categories were used to compare and contrast the teaching aspects of the automated materials with the teaching processes of the online sessions.

3.11.5 Examining the materials and observation data. According to the defined criteria, two sets of data were evident: the interactive materials, and the

observation data from the online sessions. The analysis of these two data categories included two approaches. The first approach of the analysis was "a straightforward, analytical matching process" Zhang (2007: 30). This approach was used with the materials and involved exploring the different levels of proficiency and categorising of these materials according to the pre-defined criteria. This type of feedback is one of the characteristics of Behaviouristic CALL (underlying pedagogy). This judgement (categorisation) was based on a straightforward analytical matching of the materials with this element from the criteria. Screenshots taken in the data collection phase of the study were categorised simultaneously using the same framework.

Observation data and matching from the online sessions was not straightforward as it was more appropriate to use a simple observational coding scheme: first, top-down coding (Stigler, et.al, 2000), second, match the categories that result from coding with the elements from the criteria. This approach included using codes that were apparent themes in the related literature (an example about how Mayring's (2000) framework was used for analysing observation data can be found in Appendix. K: Analysis of the Materials and Observation Data).

3.11.6 Coding of interviews and questionnaires. Thematic analysis (Coffey and Atkinson, 1996) was performed on the data. Themes emerged through reading of the data, first on a descriptive level, and then on an analytical level. Coffey and Atkinson (1996: 31) stated that "we can start with a simple framework for coding based on what we as researchers are interested in. Reading through data extracts, one might discover particular events, key words, processes, or characters that capture the essence of the piece." The aim of this process was to condense the bulk of data into units that could be analyzed by creating categories with and from the data.

Coding of the interviews and questionnaires was carried out separately. The coding process was done manually and involved highlighting comments and essential points as well as writing notes to assist in the development of themes, categories, and subcategories. Questionnaires and interviews were coded separately in the sense that they were dealt with as two sets of data. Coding during this stage presented an initial organization and sorting of meaning that led to the next stage of data analysis subcategories were established (Lincoln and Guba, 1985; Robson, 1993).

The main categories are family codes that could include a number of related codes that shared common properties and revolved around the same dimension. For example, the social cues category involved the following codes: forming impressions of others, engagement in lesson, depersonalization, and turn-taking. Initial categories and subcategories were revisited and revised (Simpson and Tuson, 1995, as cited in Hoepfl, 1997). The family codes of the data from faculty interviews were reviewed to determine how far they converged with certain categories in the students interview data, and vice versa. The establishment of family codes (categories) in this way allowed reflection on the perceptions of both faculty and students of the opportunities and constraints of the online environment. The first set of categories related to the constraints of the online environment. The second set of categories related to the opportunities of the online environment. The data analysis process was considered an example of triangulation of the data (Silverman, 2001, Yin, 2003). After data analysis, reflection on the answer to the research question was possible as faculty and students perceptions of the opportunities and constraints of the online environment were evident (for more details about how teachers and students views were analysed and merged see Appendix. L: Analysis of the Questionnaires and Interview Data).