Data Analysis

This section describes how these sources of data were processed and analyzed to answer the research questions about the four qualities of the materials on source use. After a brief introduction to the general procedure of preparing the data for analyses, I will explain the analyses used to address each of the questions.

4.5.1. Preparing Data for Analysis

For the quantitative data sources, all the responses to the questionnaire surveys were processed before analyses following the procedure described by Dörnyei and Taguchi (2010) which involves coding questionnaire data, inputting data, and pre-processing data. All the responses to the 6-point Likert-scale questions in the questionnaires were directly exported from the Qualtrics survey platform for final checking.

Next, all the students’ Moodle-based logs, including their performances on the pre- and post-tests on source use, the lesson, the journal on source use, and their submitted written drafts before and after completing the materials were retrieved and de-identified. The first quantitative data from these sources included (1) scores of the tests ranging from 0 to 7 corresponding to equally weighted seven questions in the tests; and (2) letter grades on source use quality of students’ drafts ranging from D (Needs work) to A (Excellent), which were then converted into numerical data on a corresponding scale of 1 to 4. The second set of quantitative data from these sources included the time spent on the tests and the lesson which were measured in seconds. The time range was from 0 to 30 minutes for the test, and from 0 to 50 minutes for the lesson. The qualitative data from these sources consisted of (1) the students’ actual responses to the two

open-ended questions in the lesson; and (2) their responses to the prompts in the journal. These were also gathered and imported into Nvivo version 11 for coding.

Moreover, for each of the individual records of use in the web-based corpus tool, two pieces of quantitative data were retrieved. The first was the total number for each type of searches out of the sixteen types provided in the menu of the tool, which could be any number equal or greater than 0. The second was the actual time spent on the tool which could range from 0 to 50 minutes for each person. Both data sources were collected from the admin page and downloaded as an Excel file for each user. All of these data were then stored in individual folders in a password-protected Cybox account. All the instructor and student participants were

anonymized by being assigned pseudonyms before data analysis.

Finally, the two sources of qualitative data, including all the recordings of the interviews and stimulated recall sessions, were transcribed by a professional transcription service, Rev; all the transcriptions were then checked by the researcher and sent to the participants for member check before being imported into the Nvivo version 11 software.

4.5.2. Conducting Data Analysis

Corresponding to the two major types of data as presented above, two different types of data analyses were conducted in this study. A general approach to each analysis type is first presented before a detailed presentation of data analysis for each research question of the study.

For quantitative data, because the goal of the study was to seek an understanding of the learners’ learning experiences with the materials in the specific instructional context, descriptive analyses, defined as “techniques that are used to organize and summarize data for the purpose of enhancing understanding,” were chosen to be used in this study (Onwuegbuzie & Combs, 2010,

p.401). These descriptive analyses were conducted for all the student and instructor participants. Major descriptive results including mean, mode, and standard deviation were reported for each source of quantitative data. Moreover, the characteristics of quantitative data such as test scores on source use were further examined to check if a t-test would be appropriate to see if there was any significant change in knowledge about source use after completing the lesson.

As presented in the previous section, there were two groups of qualitative data. The fir st group contained the students’ written responses to the prompts of the lesson and the journal. The second group was comprised of the responses from interviews and stimulated recalls. The first group was used to provide artifacts to illustrate how the students performed on the tasks to learn the features of source use; as a result, they were examined in terms of how well they represented the students’ perceptions of their learning in the interviews and stimulated recalls.

The second group of qualitative data was used to triangulate the results from other quantitative measures by providing insight into the users’ perceptions of the four qualities of the materials and their working processes while doing the lesson. Because the qualitative data in this study were collected from different classes by different instructors with several instructional background variations, the analyses selected for this qualitative data group were within-case analyses (Onwuegbuzie & Combs, 2010) as the analyses were bounded within a single case, which is the class in this study. In other words, qualitative data from each class were analyzed individually. Each of the within-case qualitative analyses in this study also followed the common analysis process in qualitative research which consists of three stages (Lichtman, 2010;