Data Analysis & Triangulation

Other than comparing and reconciling the results of error reduction (see Chapter 6) with students’ perceptions on teacher written feedback (see Chapter 7) obtained from the experimental group and the control group, the results gathered within this study were also exclusively compared and reconcile with those of Truscott’s study (1996), Ferris’s study (2006) and Lee’s studies (2008a & 2008b) alongside some common observations derived from some other previous research studies. The reasons for doing so are that this would help put together a fuller, more meaningful and insightful picture as to the impact of teacher written feedback on student writing revisions from various perspectives, and would also help reconcile our results with any contradictory evidence (or vice versa) obtained from Ferris’s study (2006) and Lee’s studies (2008a & 2008b) on which the methodology of this study was based.

185

This research aimed at evaluating the actual effectiveness of the electronic feedback on error reduction, when comparing to the traditional paper-based feedback. An explanatory mixed- methods research design was adopted, which was modeled on Ferris’s (2006) and Lee’s (2008b) methodologies. The data collection was divided into two phrases. The first phase was conducted by ‘error count’ with a primary objective (i.e. research questions 1 and 2) of comparing student uptake on two different forms of feedback (i.e. electronic feedback vs. paper-based feedback). The second phase was conducted by administering questionnaires with a secondary objective (i.e. research question 3) of understanding the students’ attitudes toward the feedback they received. Participants were 62 year-two Engineering students who were enrolled in the same English course with a comparable language proficiency over one semester. Convenience sampling was adopted. A number of measures was implemented to ensure the validity and reliability of the study. Quantitative results of this study were triangulated with the qualitative results of the study, as well as with the results of other studies (e.g. Ferris, 2006; Lee, 2008b; Truscott, 1996).

186

Chapter 6 Findings and Discussions: Error Reduction

This chapter covers the error reduction by examining the actual effects of the electronic feedback and paper-based feedback in reducing the frequency of errors in student writing revisions. Modeling on the approach adopted by Ferris (2006) in her study titled ‘Does error feedback help student writers? New evidence on the short term and long term effects of written error correction’, the actual effectiveness of the teacher written feedback was evaluated by counting the total number of errors, by category and frequency, students respectively from the experimental group and the control group made before and after receiving teacher written feedback. Ferris’s study (2006) on the effectiveness of teacher written feedback covered 16 error types whereas the present study extended the number of error types to 77 which were then further divided into 7 error categories (i.e. ‘Awkwardness’ errors, ‘Clausal-level’ errors, ‘Word-level’ errors, ‘Collocation’ errors, ‘Content’ errors, ‘Organization’ errors, and ‘Tone & Style’ errors). The table in Appendix A shows these 77 error types under 7 categories identified for an investigation in this study. The tables in Appendix C also show how well students in both the experimental group and control group performed in revising the various error types after receiving a particular form of feedback treatment within their own groups.

Other than comparing the results of error reduction obtained from the experimental group and the control group and drawing their implications, the results gathered within this study were also compared with those of Ferris’s study (2006) and Truscott’s study (1996) alongside some common observations derived from some other previous research (e.g. Lalande, 1982; Zamel, 1985; Robb

187

et al., 1986; Ferris & Roberts, 2001; Hyland & Hyland, 2006; Lee, 2008a & 2008b; Bitchener & Knoch, 2010b). It is noted that Ferris (2006) and Truscott (1996) took a conflicting position with regard to the effectiveness and values of teacher written feedback. The findings and analysis in this section added on to their substantial discussions not only about the controversy over the effectiveness of teacher written feedback but also about the limitations and constraints faced by teachers and students in the feedback environment. The reasons for doing so are that this would help put together a more complete, meaningful and insightful picture as to the impact of teacher written feedback on student writing revisions from various perspectives, and would also help reconcile our results with any contradictory evidence (or vice versa) obtained from Ferris’s study (2006) on which the methodology of this study was partially based.

In summary, it can be concluded that the electronic feedback (‘Mark My Words’) modeled on Nunan’s (1997) ‘Model of Framework for Developing Learner Autonomy’ and Krashen’s (1985) ‘Input Hypothesis’ is relatively more effective in error reduction across all categories, namely (1) ‘Awkwardness’ errors, (2) ‘Clausal-level’ errors, (3) ‘Word-level’ errors, (4) ‘Collocation’ errors, (5) ‘Tone & Style’ errors, (6) ‘Content’ errors and (7) ‘Organization’ errors, in both ‘between- groups’ and ‘within-groups’ comparison (see Appendix G: Summary of the Statistical Results).

It is also noteworthy to highlight that students receiving electronic feedback were more successful than those students receiving paper-based feedback in revising ‘Awkwardness’ errors and ‘Collocation’ errors, which are highly characterized as ‘L1-L2 interference’ errors and regarded as ‘untreatable errors’ by Ferris (2006). This is evident by the findings that these two error categories were the only categories which made no statistical significant revisions even in the

188

‘within-groups’ comparison of the paper-based feedback group (i.e. control group). For all seven error categories, despite the fact that the control group receiving the paper-based feedback was statistically less effective than its counterpart in remedying errors in the ‘between-groups’ comparison, at least the paper-based feedback still managed to make a statistically significant number of revisions for most error categories within its own treatment group, with the exceptions of ‘Awkwardness’ errors and ‘Collocation’ errors. These findings lend weight to Ferris’s (2006) argument that non-rule governed ‘sentence-level’ errors (which is similar to what ‘Awkwardness’ errors was defined in this study) and ‘word choice & idioms’ (which is similar to what ‘Collocation’ errors was defined in this study) are the most difficult error items for students to correct. On the other hand, the results of the questionnaires in Chapter 7.10 and Chapter 7.11 (see Q14-Q17 in Appendix E) also lend support to these findings and Ferris’s argument when students were asked to rate all error categories in terms of their level of difficulty for comprehension and correction.

189