Total Errors

‘Total Errors’ in this study refers to all 7 error categories, comprising 77 error types, identified in student writing. They include (1) ‘Awkwardness’ errors, (2) ‘Clausal-level’ errors, (3) ‘Word- level’ errors, (4) ‘Collocation’ errors, (5) ‘Tone & Style’ errors, (6) ‘Content’ errors and (7)‘Organization’ errors. An overview of error count before the statistical analysis for the top ten errors are appended in Appendix C. It is found that ‘Awkwardness’, ‘Singular-Plural Form’, ‘Missing Determiner’, ‘Sentence Fragment’, ‘Run-on Sentence’ and ‘Wrong Verb Form’ are the common error types identified in both experimental group and control group.

When examining the percentage of error reduction across all 77 error types under these 7 error categories in Appendix C, it is observed that the experimental group receiving the electronic

190

feedback achieved the highest percentage of error reduction across all error types when compared to the control group receiving the paper-based feedback. Having said that, statistical analyses (i.e. a two sample t-test) were conducted in this section and the following sections to examine if either form of feedback treatment could offer a more statistically significant difference in the total error reduction (Chapter 6.2), as well as in each of the 7 categorical error reduction (Chapters 6.3 – 6.9)

Findings I (Total Errors between the Treatment Groups)

A Two Sample t-test was conducted between experimental group and control group in order to find out if either group performed better than the other in reducing the total number of errors. The summary statistics of the experiment group and the control group on ‘Total Errors’ can be seen in Table 6.3 in Appendix E.

For the experimental group receiving the electronic feedback, a total of 615 errors were collected from the 30 students’ first drafts marked by Teacher A. There was an average of 20.46 errors per first draft (of 550 words on average – i.e. 1 error per 27 words). A total of 64 errors were collected from the 30 students’ second drafts marked by the same teacher. There was an average of 2.13 errors per second draft (of 550 words on average – i.e. 1 error per 258 words).

As for the control group receiving the paper-based feedback, a total of 614 errors were collected from the 32 students’ first drafts marked by Teacher B. There was an average of 19.16 errors per first draft (of 550 words on average – i.e. 1 error per 28.66 words). A total of 304 errors were

191

collected from the 32 students’ second drafts marked by the same teacher. There was an average of 9.5 errors per second drafts (of 550 words on average – i.e. 1 error per 57.89 words).

The average number of total errors per first draft made by experimental group and control group was 20.46 and 19.16 respectively (Table 6.3 in Appendix E) which were very close. However, the average number of total errors per second draft made by both groups was 2.13 and 9.5 respectively (Table 6.3 in Appendix E). The difference in the average number of total errors respectively made by the experimental group and control group in draft 2 was relatively widened.

The statistical results (between groups) imply that the overall language competence before treatment was the same for the electronic feedback group and the paper-based feedback group (Table 6.4 in Appendix E: p-value [draft 1] = 0.629), but the average number of total errors made by the electronic feedback group in draft 2 was significantly smaller than that made by the paper- based feedback group in draft 2 (Table 6.4 in Appendix E: p-value [draft 2] = 5.697e-06). Such difference (between groups) is educationally significant as well as being statistically significant, because the effect size (= -1.325) is large (Table 6.4 in Appendix E).

In conclusion, these ‘between groups’ results indicated that students in the experimental group receiving the electronic feedback were able to make a statistically significant number of revisions across all 7 error categories (i.e. ‘Awkwardness’ errors, ‘Clausal-level’ errors, ‘Word-level’ errors, ‘Collocation’ errors, ‘Content’ errors, ‘Organization’ errors, ‘Tone & Style’ errors); whereas students in the control group receiving paper-based feedback group did not in such ‘between groups’ comparison.

192 Findings II (Total Errors within the Treatment Groups)

Another two Sample t-test was conducted within each of the experimental group and control group in order to find out if each individual treatment group itself helped reduce the total number of errors in draft 2 when comparing to that in draft 1. The summary statistics of the experiment group and the control group on ‘Total Errors’ can be seen in Table 6.3 in Appendix E.

When testing the effectiveness of each feedback treatment in reducing the total number of errors within its own feedback group, it is found that both the electronic feedback and the paper-based feedback (within its group) were able to make a statistically significant number of revisions in the total number of errors in draft 2 (Table 6.5 in Appendix E: p-value [control group] = 7.496e-05 with an effect size = 1.07, and p-value [experimental group] = 2.736e-10 with an effect size = 2.34). However, with the smaller p-value and the larger effect size, the electronic feedback (within its group) was able to make a more statistically significant number of revisions in the total number of errors when comparing those made by the paper-based feedback.

In conclusion, these “within groups” results further support the conclusion that the electronic feedback treatment is more effective than the paper-based feedback treatment in reducing the total number of errors across all 7 error categories (i.e. ‘Awkwardness’ errors, ‘Clausal-level’ errors, ‘Word-level’ errors, ‘Collocation’ errors, ‘Content’ errors, ‘Organization’ errors, ‘Tone & Style’ errors).

193 Discussion

The positive findings obtained from the experimental group contradicted Truscott’s (1999) arguments against the effectiveness of error correction on student writing revisions, which were rested on the aspects of (1) teacher’s limitations in feedback practice and (2) student’s limitations in comprehend sion and adoption of teacher written feedback. In fact, the conflicting results between Truscott’s (1996) observations and the findings in this study could be attributed to the pedagogical approach of ‘Mark My Words’ (‘MMWs’) which is underpinned by the rationales postulated by Nunan’s (1997) ‘Model of Framework for Developing Learner Autonomy’ and Krashen’s (1985) ‘Input Hypothesis’ (please refer to Chapter 3.1 ‘Theoretical Framework and Language Theories’). In an attempt to remedy the above limitations which were believed to render teacher written feedback ineffective, this ‘three-step approach’ pedagogy empowered by its comprehensive and consistent error feedback database is synchronized into the operation of ‘Mark My Words’ (‘MMWs’).

For the aspect of teacher’s limitations, Truscott and previous researchers (cited in Ferris, 2006) argued that practical problems like incomplete, inconsistent, and inaccurate teachers’ error feedback have rendered the writing feedback ineffective. To ensure more detailed, comprehensible and standardized feedback comments to be given for each error type, seventy-seven electronic feedback comments covering content, organization and language are pre-set. This contributes to the electronic feedback database of ‘Mark My Words’ (‘MMWs’). The findings showing that the experimental group made a statistically significant number of reduction in their total number of

194

errors across all these seven categories between the first draft and the second draft do not support the claims of some previous researchers that teachers give incomplete and inaccurate error feedback and that students ignore teacher feedback or cannot utilize it effectively in their writing revisions (Cohen & Robbin, 1976; Truscott, 1996; Zamel, 1995).

As for the aspects of student’s limitations in comprehension and adoption of teacher feedback, Truscott (cited in Hyland, 2003, p.218) attributed the ineffectiveness of teacher feedback to the students’ lack of skills to understand and use the feedback. Their lack of skills to comprehend and adopt teacher feedback might possibly due to the theoretical issue raised by Truscott (1996) who argued that different types of linguistic forms may take a different sequence of L2 acquisition. This means a student’s interlanguage (i.e. proficiency) level would determine to what extent he or she can comprehend, process and utilize teacher feedback for a particular error type. To put Truscott’s theoretical issue into perspective, the sequence of second language acquisition should begin with word-level linguistic forms, and then followed by phrase-level, clausal-level and discourse level. A student who is at a point of struggling with the acquisition of clausal-level linguistic forms in his continuum of interlanguage stands a high possibility of finding teacher feedback on ‘Clausal-level’ errors (e.g. run-on sentence, sentence fragment, non-standard grammatical structure heavily interfered by L1) incomprehensible if the feedback is not made explicit for the student. Such causal relationship between students’ interlanguage level and their comprehension and adoption of teacher feedback for a particular error type lends weight to Ferris’s (1996 & 1999) argument in that students’ level of progress in error correction varied depending on error types. This implies that different feedback strategies, at various levels of explicitness, should be prescribed for different error types. However, Truscott (cited in Ferris, 2006, p.83-84)

195

argued that teachers often responded to errors of all categories in the same way without sufficient awareness that different types of linguistic forms may take different order and sequence of L2 acquisition, while ignoring variation in students’ interlanguage level to process and utilize teacher feedback successfully.

To improve students’ comprehensibility and adoption of teacher feedback, ‘Mark My Words’ (‘MMWs’) provides a feedback environment where students instead of teachers can exercise their autonomy to determine, according to their interlanguage level and interests, how much information and how explicit the teacher feedback on a particular error type they would like to receive. Hence, with its capacity of reconciling the above teacher’s and student’s limitations, the electronic feedback has rendered teacher written feedback effective for student writing revisions in this study.

6.3 ‘Awkwardness’ Errors

‘Awkwardness’ errors in this study refers to the “L1 – L2 interference” (or ‘Chinglish’) – an L2 utterance which is heavily interfered by the mother tongue’s (L1) grammatical structure, and this non-standard structure cannot be simply explained by or identified with grammatical rules (i.e. not rule-governed) and specific marking codes, but to a certain extent shares similar syntactic patterns with the Chinese equivalence. Given the proficiency level of the subjects in both experimental group and control group, their utterances produced may still be semantically understandable but they may not be syntactically correct. For example:

196

1. People will be more and more to come (Chinese syntactic structure) to walk behind the trend (Chinese syntactic structure) of iPhone 5.

2. There are three people go to school. (Chinese syntactic structure)

3. Recently, the born of (Chinese syntactic structure) an advanced communication technology is just around the corner.

4. The problem will be smaller and smaller … (Chinese syntactic structure).

Findings I (‘Awkwardness’ Errors between the Treatment Groups)

A Two Sample t-test was conducted between the experimental group and control group in order to find out if either group performed better than the other in reducing ‘Awkwardness’ errors. The summary statistics of the experiment group & control group on ‘Awkwardness’ errors can be seen in Table 6.6 in Appendix E.

The average number of ‘Awkwardness’ errors per first draft made by the experimental group and control group (between groups) was 1.57 and 2.25 respectively (Table 6.6 in Appendix E) which were very close. However, the average number of ‘Awkwardness’ errors per second draft made by both groups was 0.167 and 1.66 respectively (Table 6.6 in Appendix E). The difference in the average number of ‘Awkwardness’ errors respectively made by the experimental group and control group in draft 2 was relatively widened.

197

The statistical results (between groups) imply that the electronic feedback group and the paper- based feedback group demonstrated the same level of performance with respect to avoiding ‘Awkwardness’ errors in draft 1 before any intervention took place (Table 6.7 in Appendix E: p- value [draft 1] =0.1237), but the average number of ‘Awkwardness’ errors made by the electronic feedback group in draft 2 was significantly smaller than that made by the paper-based feedback group in draft 2 (Table 6.7 in Appendix E: p-value [draft 2] = 0.000149). Such difference (between groups) is educationally significant as well as being statistically significant because the effect size (=-1.047) is large (Table 6.7 in Appendix).

In conclusion, the “between groups” results indicated that students in the experimental group receiving the electronic feedback were able to make a statistically significant number of revisions in the average number of ‘Awkwardness’ errors; whereas students in the control group receiving paper-based feedback group did not in such ‘between groups’ comparison.

Findings II (‘Awkwardness’ Errors within the Treatment Groups)

Another two Sample t-test was conducted within the experimental group and control group in order to find out if each individual treatment group itself helped reduce the number of ‘Awkwardness’ errors between draft 1 and draft 2. The summary statistics of the experiment group & control group on ‘Awkwardness’ errors can be seen in Table 6.6 in Appendix E.