Mixed methods data collection

It may be noted that ‘mixed methods’ in this study refers to mixed methods of data collection used within the frame of this evaluative multiple-case study research. The term mixed methods can be defined as ‘the mixing of quantitative and qualitative procedures’ for data collection and analysis that are introduced into the case study research. Even though questionnaires were used to collect the quantitative data in this case study, procedures for the administration of the questionnaire results followed the same rules as in the quantitative research, but the analysis, interpretation and presentation of the findings were framed as case study research. Also, this study relied more on the qualitative procedures than on the quantitative ones, so it could be represented by the symbol ‘QUAL with quan’ based on Morse’s (1991, cited in Richards et al, 2011) definition. The first section offers brief details about the usefulness and main characteristics of mixed methods, followed by the

justification of the use of mixed methods in this study. The second section relates to how the mixed-methods data collection was carried out, looking specifically at types of data collected and research instruments.

There has been an ongoing growth of interest in mixing quantitative and qualitative methods. Most researchers identify the advantages of mixed methods as providing answers to both ‘what’ and ‘why’ (research) questions (Ivankova and Creswell, 2009). The mixed methods also offer an opportunity to explore the potential of different perspectives on the research process (Richards, 2011:7), and when the quantitative data collection is followed by qualitative data collection, they examine the surprising quantitative results in more detail, (the sequential explanatory strategy: Creswell, 2009:211). Mixed methods are sometimes seen as a triangulation procedure to validate data from different sources in order to reach a conclusion. Relating to this multiple-case study, data analysis and triangulation was one of the processes used to find out whether the participants had similar or different perspectives on the effectiveness of TBLT and mediation.

Features and procedures of this study matched the descriptions of mixed methods specified by Yin (2009), and Ivankova and Creswell (2009). Ivankova and Creswell (2009) note that mixed methods can be employed in case study and action research by using different tools for data collection such as observation, interview, questionnaire, and diaries. A similar suggestion is given by Yin (2009) who points out that a case study can encompass a survey; in his discussion, it refers to a type of interview consisting of structured interview questions. In this situation, sampling procedures, design, and analysis of the results undergo similar procedures as normal surveys, but the survey is linked to other sources of evidence. In other words, survey

results are regarded as one component of the overall data set. Based on the characteristics of the mixed methods and a plan of this multiple-case study, an overall picture of the study can be represented by Figure 1 below.

Figure 1 Plan for data collection

The decision to adopt mixed methods for data collection in this study was based on three main reasons. The first reason related to the use of pre-intervention questionnaire data as the baseline data for the implementation of TBLT and mediation, as student preferences for, for example, teaching methods and receiving feedback were examined. Secondly, the design and construction of the questionnaire was quite straightforward, and with systematic analysis as in statistical analysis, the former provided reliable results. This enables a survey of the attitudes and backgrounds of all the students in one step. In contrast, the qualitative procedures were more complex but generated more detailed results. Thus, combining both quantitative and qualitative data collection methods offered both broad and in-depth perspectives, which were useful for an evaluation of TBLT and mediation. In this study, qualitative methods were used to guide the project, whereas questionnaire

EVALUATIVE MULTIPLE-CASE STUDY Quantitative data collection (pre-intervention questionnaire)

Qualitative data collection

Class observation Post-intervention interview

Teacher’s reflections Recorded class interactions

Quantitative data collection (post-intervention questionnaire)

Overall interpretation

served a secondary data set providing a supporting role, as discussed by Richards (2011) and Creswell (2009).

Another significant reason for choosing mixed methods concerned the focus of this research and research questions. Ivankova and Creswell (2009) suggest considering whether mixed methods are the best way to address the research problem. In other words, the researchers should ask themselves whether mixed methods provide the answers to the research questions. While quantitative methods can provide an overall picture of the entire population, qualitative methods can capture the voices of the participants. Both methods have strengths and weaknesses, but a combination of their strengths helps to answer questions that cannot be answered by relying solely on one or the other (Creswell and Plano Clark, 2011).

Consideration of the research questions is crucial. The use of unsuitable data collection methods might result in inadequate details affecting the credibility of the research. There were four research questions in this study (see p. 70), each of which (particularly questions 2 and 3) required different types of data, from multiple sources for evidence and claims of the effectiveness of TBLT and mediation practices. In order to answer these questions, quantitative methods were used to collect the feedback of the learners and then triangulated with qualitative results of the interviews, observations, recordings and the teacher’s reflections.

Creswell (2009) suggests identifying data to be collected that must be specific in terms of what types of data need to be gathered – both quantitative and qualitative. There are further considerations such as in the case of interviews whether they should aim at open-ended or closed-ended responses. An interview with open questions is categorised as qualitative, whereas one with closed questions is

quantitative. These suggestions are useful for the development of research instruments. Based on the four research questions (see p.72), this research was designed to collect seven kinds of data as shown in Table 7.

Type of data Method of data collect. Research instrument Phase of intervention Answering res. quest..no.

1. Learners’ attitudes and preferences for learning activities and teaching methods

Quantitative Pre-intervention questionnaire

Before 1,2,3,4

2. Four teacher-observers’ comments and feedback on the teaching

Qualitative Observation forms with guided questions

During 1, 2, 4

3. Class interactions Qualitative Video/MP3

recordings

During 2, 3

4. The teacher/researcher’s reflections

Qualitative Teacher’s journal During 1, 2, 3,4 5. Learners’ attitudes, perception

and opinions of the designed tasks, TBLT, feedback (mediation), and suggestions for improvement

Quantitative Post-intervention questionnaire

After 1, 2, 3,4

6. Four teacher-observers’ feedback on the teaching overall and suggestions for improvement

Qualitative Guided questions for the post- observation interviews

After 1, 2, 3, 4

7. Learners’ perceptions, opinions of TBLT and mediation, and suggestions

Qualitative Guided questions for in-depth interviews

After 1, 2, 3, 4

Table 7 Types of data and data collection methods

From Table 7, it can be seen that more qualitative data than quantitative data were collected. In terms of quantitative data, two questionnaires were used: the pre- intervention questionnaire and the post-intervention questionnaire. The qualitative data was gathered from observation of four teacher-observers, the researcher’s journal, recorded class interactions, and observer and learner interviews.