Limitations of this Research

In this section, I will discuss several limitations of this study. Over the course of the four reflective cycles, several procedural changes were made to better suit the needs of my students. However, by making these procedural changes, the comparisons I made between intervention groups are subject to debate. In the discussion that follows I will discuss the possible limitations in the findings which were made, the data analysis which was used, and the procedures which were implemented in the intervention stage of this study.

7.2.1 Limitations of the findings.

While this study had four reflective cycles using three different groups of participants, all of the students who took part in this investigation were of the same English proficiency level (low-proficiency) and from the same country (Japan). The decision to focus on low-

proficiency students was made to address the disparity in collocation research which to date has largely focused on advanced students (Eyckmans, 2009; Jiang, 2009; Nesselhauf, 2003; Revier, 2009). However, by only focusing on low-proficiency students, the findings may not be applicable to students at a higher English proficiency level.

Another possible limitation of the findings is the fact that over the course of this study only two specific types of collocations (delexicalized verb collocations and frequent adjective + noun collocations) were investigated. These two types represent only a small portion of

the total number of collocations used in English, and the findings from this study might have been different if various types of collocations were investigated.

As seen in the findings presented in Chapters five and six, the students had a positive impression of studying collocations. However, the student responses were elicited after only one semester (reflective cycles one, two, and three) or two semesters (reflective cycle four) of explicit collocation instruction. It is possible that the positive responses are partially the result of the novelty of a different approach to vocabulary acquisition. It is unknown if the students would still have a positive impression of studying collocations over a longer period of time. Furthermore, the questionnaire included the word ‘useful’ which might have

indicated a preferred response to the students.

The post-intervention questionnaire findings for reflective cycle three can also be questioned. Of the 43 participants who took part in this reflective cycle, only 21 completed the final questionnaire. It is possible that the more motivated students were the ones to complete this questionnaire, and that these findings are not representative of the total group of participants.

7.2.2 Limitations in the data analysis.

In regard to the data analysis, there are three possible limitations. The first limitation concerns the students’ responses elicited through the four questionnaires which had open- ended items. Many of these responses were written in Japanese by the students and then translated to English by an experienced Japanese university instructor with a high level of English proficiency. However, the responses were not back translated to ensure accuracy.

The statistical analysis from reflective cycles two and three can also be brought into question. A matched pair t-test was used to analyze the difference in a pre-intervention and post-intervention spoken fluency assessment. For reflective cycle two especially, a valid argument can be made that the ANOVA would have been a more suitable statistical measurement given the fact that there were two intervention groups (receptive and

productive). Furthermore, the Fleiss’ Kappa test that was conducted to measure inter-rater reliability only had an observed agreement of 0.375.

The spoken assessment (see sections 5.3.6.1 and 6.2.4.1 for excerpts) was an elicited monologue in which the students described a series of pictures. While this procedure provided a measure of consistency in the assessment between students, it can be argued that a monologue is not the most representative speech sample for measuring fluency. A speech sample taken from a conversation might be a more accurate indicator of a student’s fluency.

Several aspects of the procedures used during the various reflective cycles can also be questioned. In Chapter seven, I compared findings from the second and third reflective cycles. While these two cycles used the same collocation list (list 2), reflective cycle three included a homework assignment, but reflective cycle two did not.

Another potential weakness in the procedures used during this study concerns the spoken assessment task. To elicit the speech sample, I used a series of pictures depicting daily routine activities. Collocation list 2, which was used for the two cycles in which a speech sample was taken, contained several delexicalized verb collocations which could be used to describe these activities. Specifically, eight of the 120 collocations (6.7%) from list two could have been used during the spoken assessment, and the excerpts seen in sections 5.3.6.1 and 6.2.4.1 show that several targeted collocations were used. Therefore, the improvements in spoken fluency might only relate to the students’ ability to use the targeted collocations as opposed to their overall fluency capabilities.

Finally, between the second and third reflective cycles, I started working at a different university. As a result, the participants used during these two cycles were also different, yet I compared the findings from the two cycles in Chapter seven. Specifically, I proposed that the productive+ tasks from reflective cycle three were superior for improving fluency to the receptive and productive tasks from cycle two. The fluency improvement seen in the findings could have been a result of using a different group of students as opposed to the change in procedure. However, this change did provide a measure of triangulation to my findings.