Mixed methods research

Mixed methods research is viewed as the third methodology (Johnson, Onwuegbuzie, & Turner, 2007, p. 129) or the third movement that has emerged alongside quantitative and qualitative approaches (Creswell & Garrett, 2008, p. 322). Johnson et al. (2007) provided the following definition of mixed methods research:

Mixed methods research is the type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches (e.g., use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the broad purposes of breadth and depth of understanding and corroboration (p. 123).

In highlighting the rationale for using mixed methods research, Creswell and Plano- Clark (2011) and Creswell and Garrett (2008) stated that the main reason for combining quantitative and qualitative approaches was to provide a better understanding of research problems than purely using either approach alone and this was the goal of employing mixed methods for this study. Quantitative methods could provide evidence of changes in learning skills and allow for statistical analysis of relationships among variables, whereas qualitative methods could provide deeper insight from the participants and add to the meaningfulness of the study.

58 Quantitative research involves measuring individual abilities, collecting scores from individuals, and employing procedures of psychological experiments and large-scale surveys (Creswell, 2008). This research approach is “a means for testing objective theories by examining the relationship among variables” (Creswell, 2009, p. 4). In quantitative research, researchers generally deal with a larger number of participants who can provide statistically meaningful data for the study, but the researchers have little personal interaction with them (Gay, Mills, & Airasian, 2009). Theories are tested deductively, and the findings from the study can often be generalised and replicated. There are a variety of research designs based on quantitative research methods. These include descriptive, correlational, causal-comparative, experimental, single-subject experimental studies, and surveys.

The current study employs a correlational research design for the quantitative approach. Correlational research is used to examine the relationships that exist between variables (Ary, Jacobs, & Sorensen, 2010; Schunk et al., 2014). This research design is in contrast to an experimental design, in which the variables in the study are manipulated to explore whether they affect each other.

This study aimed to examine the existing relationship between students’ SRL and their perceptions of the usefulness of online portfolios; therefore, the correlational research design was believed to be an appropriate approach. The quantitative findings of this study could be generalised to other groups of postgraduate students who study in a similar context to this study, while the qualitative findings could build a more comprehensive and detailed picture of SRL and students’ use of online portfolios. Qualitative research is “a means for exploring and understanding the meaning individuals or groups ascribe to a social or human problem” (Creswell, 2009, p. 4). This research approach involves collecting, analysing, and interpreting comprehensive

59 narrative data to get in-depth information about a specific phenomenon (Gay et al., 2009). Participant numbers tend to be small, but the researcher gains in-depth understanding of participants’ views. Thus, personal interaction is commonly involved in the data collection process. Examples of qualitative research include action research, case studies, ethnographic studies, and grounded theory studies.

The current study applied elements of qualitative research, especially semi-structured interviews with open-ended questions, to obtain the participants’ view of their learning behaviours and online portfolio use. Gaining students’ perspectives on changes in their learning within the online context and, more specifically, in relation to factors that impacted on online portfolio use could provide insights in relation to the questions under investigation in this study and inform the course coordinators in order to help future students learn independently. Thus, the qualitative data could provide further information on any relationships between SRL and perceptions of the usefulness of online portfolios as discovered through the correlational research.

Mixed methods research can be conducted through different designs (e.g., concurrent and sequential) (Creswell, 2009; Creswell & Plano Clark, 2011; Onwuegbuzie & Leech, 2004), depending on the research problem and questions (Creswell & Plano Clark, 2011). In the case of this study, the explanatory sequential design (Creswell, 2009; Creswell & Plano Clark, 2011) was selected. In this design, quantitative data were collected through the use of a survey and analysed first. Then the qualitative data were gathered through individual semi-structured interviews. A second quantitative questionnaire was utilised at the end of the study. The findings from all data sets were used to answer the research questions.

60 According to Creswell and Plano Clark (2011), the sequential design is normally presented as a two-phase study. This design was adapted as a three-phase study to include a follow-up quantitative component toward the end of the year.

The reason for employing this design was that the data needed to be collected and analysed in sequence: the results of the first phase (questionnaire) provided the basis for designing the approach of the second phase (semi-structured interviews) and the selection of participants for this phase. This is in contrast to concurrent designs where both quantitative and qualitative data are collected at the same time and are analysed after all the data have been collected (Onwuegbuzie & Leech, 2004).

The results from both the quantitative and qualitative methods used in this study were integrated during the discussion of the research findings. Figure 3.1 shows the mixed methods explanatory sequential design phases and the procedures used.

61

Phases Procedures Products

- Questionnaire One (N =162; n = 64) - Descriptive statistics - Numeric data - Frequency, means, - Descriptive statistics - T-test

- Pearson product moment - Selection of 3 participants each from the highest and lowest levels of SRL based on responses to Questionnaire One - Standard deviation - Comparison of means - Correlations - Cases (n = 6) - Individual semi-

structured interview with 6 participants

- Open-ended questions - Coding and thematic analysis

- Text data (interview transcripts, answers from open-ended questions)

- Codes and themes

- Questionnaire Two - Two-way ANOVAs repeated measures

- Comparison of means at two points in time

- Interpretation and explanation of the

quantitative and qualitative results

- Discussion - Implications - Future research

Figure 3.1. Procedures of data collection used in the mixed methods explanatory sequential design of this study. Adapted from Creswell & Plano Clark (2011, p. 121) Quantitative Data Collection1 Case selection: Interview question development Quantitative Data Analysis1 Qualitative Data Collection Qualitative Data Analysis Integration of quantitative and qualitative results Development of interview questions Interview protocol Quantitative Data Collection and Analysis2

62 In the diagram in Figure 3.1, boxes and circles indicate the general flow of data collection and analysis while arrows show the sequence. Dashes in the second column detail the procedures for each stage in the sequence, and dashes in the third column identify products gained in each stage. With respect to the diagram, results from each set of data collection and analysis were used in responding to the research questions, as indicated in Table 3.1.

Table 3.1

Products of Data Collection and Analysis in Relation to Research Questions

Data Collection

and Analysis Products Research Questions

Quantitative data

- Questionnaire One

- Correlation between SRL and perceptions of the usefulness of online portfolios

- Baseline for

selection of interview participants

1. Is there a relationship between students’ SRL and their perceptions of the usefulness of online portfolios?

Qualitative data

- Interviews

- Correlation between SRL and perceptions of the usefulness of online portfolios

Elaborate on the correlation between SRL and perceptions of the usefulness of online portfolios

- Open- ended questions

- Perceptions of students’ own SRL skills and online portfolio use

4. From students’ perspectives,

does the use of online portfolios increase their SRL skills?

5. What factors help or hinder

students in the construction and use of their online portfolios. Quantitative data - Questionnaire Two - Cross-lagged correlation

between SRL and the perceptions of the usefulness of online portfolios

- Changes in SRL skills and perceptions of the usefulness of online portfolios

2. Do students’ initial SRL skills relate to their subsequent

perceptions of the usefulness of online portfolios?

3. Do scores for both students’ SRL skills and their perceptions of the usefulness of online portfolios increase during the course, and, if so, do the scores for the high SRL group increase more than the scores for the low SRL group?

63 As shown in Table 3.1, initial quantitative data were used to examine students’ SRL skills and their perceptions of the usefulness of online portfolios, and the relationship between the two variables. These data were collected to provide the answer to research question one regarding the relationship between students’ SRL and their perceptions of the usefulness of online portfolios. This initial quantitative data phase also provided the basis for choosing participants to be interviewed, based on their reported levels of SRL skills (see Research Methods Phase Two, section 3.4.1). Qualitative data were collected through semi-structured interviews to provide a deeper insight into participants’ views in terms of their SRL skills and their perceptions of the usefulness of online portfolios, and this was to elaborate the findings for research question one.

Open-ended questions were used to elicit students’ thinking about whether their SRL skills in relation to the use of online portfolios had improved over the year. These data provided the answer for research question fourthat is, from the students’ perspectives, whether the use of online portfolios had increased their SRL skills. These data also related to research question five by identifying the factors that helped and hindered students in the construction and use of their online portfolios.

Another quantitative data collection method (Questionnaire Two) was used to investigate changes in students’ SRL skills and their perceptions of the usefulness of online portfolios over a two-semester online course. These data provided answers for research questions two and three  whether students’ SRL skills related to their subsequent perceptions of the usefulness of online portfolios, and whether students’ scores for both SRL skills and their perceptions of the usefulness of online portfolios increased during the double semester course. It also allowed analysis of whether the scores for the high SRL group increased more than the scores for the low SRL group; as the high SRL group were expected to be highly self-efficacious, this could have

64 influenced their SRL skills. Details of the three phases of data collection and data analysis are outlined below.