Formative feedback text analysis-results

The chart in figure 7.1 provides a comparison of the number of different types of comment received by seven students in their written end comments for the formative task (see coding categories in Section 3.6.1). Critical analysis and explanation categories were assigned after a second coding, with a further re-coding of

comments to assign the positive / negative tone, so these are not added to the overall total of comments. There is a significant absence of any comments from the

developmental reflective questions, or informational content categories.

Figure 7-3 Formative task combined feedback

On the first coding, the categories of language (examples 1 and 2) and referencing (examples 3 and 4) received more comments than content categories.

1. Your language is very readable, but plural nouns for generalities (without the) need some work, and you could edit for these in future work. (Diane) 0 5 10 15 20 25 Register Non-evaluative summary Phatic Negative content Developmental future Structure Language and expression

Explanation Developmental alternatives Positive content Reference and source use

Critical analysis Number of comments Co m m e n t typ e

163

2. You need to pay greater attention to your choice of language, e.g. an overuse of idiomatic expressions and use of ‘I’ inappropriate in academic writing (Anna)

3. You do not always reference accurately (Ethel) 4. Take care to identify surnames (Flora)

The examples above show how such comments could be relatively specific (examples 1 and 4) or more general (examples 2 and 3). Example two was the only ‘register’ comment on academic style in the sample, and it is worth noting here how the issue of voice (See Section 2.3.5) was not picked up in this feedback. Five students also received at least one comment in the structure category (e.g., Tell the

reader in the introduction what you will argue: Ethel), while only three out of a total

of seventy four comments were categorised as phatic comments that function to maintain a relationship with the student (e.g., In a strong assignment, there are only

a few points to pick up here: Diane).

Critical analysis (CA) comments from a second coding were significantly the most frequent of all categories (examples 5-7). These comments were typically framed as ‘advice’:

5. Try to include more critical comments (Ethel)

6. I would have liked to see greater analysis and assessment of the evidence used to support the writer’s points (Anna)

7. Consider referencing more to support your points (Diane)

Only Flora did not receive such comments. Equally significant in terms of the low frequency count, were the total number of comments categorised as explanation. Less than a seventh of all comments in this sample were explanations relating to earlier indicative comments (e.g., You could look at your use of ‘it’ and

‘this’, use ‘this’ when you refer back to ideas and whole clauses in the discourse:

Diane).

Less than a quarter of all comments were in the developmental categories. It is significant that only seven comments were categorised as future developmental, with only two students, Helen and Ethel, receiving five and two of these types of comment respectively. On the other hand, almost a sixth of total comments in the sample came in the alternative developmental category (example 6 above); these

164

comments were distributed fairly evenly across all participants, with only Flora again not receiving any comments in this category.

Unsurprisingly, comments on content were evenly distributed across the students, with only Flora not receiving this type of comment. Figure 7.1 shows that positive content comments were double the number of negative content comments. Analysis after re-coding of comments in all categories in terms of their tone (See Section 3.6.1) revealed a more nuanced picture, with far fewer encouraging versus critical comments. Table 7.3 below indicates that positive end comments were more in evidence than wholly negative comments. Positive comments in the margins were often simple indications such as ‘good’ or ‘good point’, while end comments often included aspects of understanding and criticality (examples 8-11). Negative end comments tended to focus on language and expression (examples 12-14):

8. You show some understanding of the central concepts (Helen) 9. You’ve displayed a good level of critical skills (Anna)

10. You provide clear evidence of trying to take a more critical stance (Betty)

11. You have a structure and develop a position (Clara)

12. You make confusing statements that don’t make sense (Helen) 13. Pay attention to paragraphing

14. There are instances where meaning is unclear due to language used (Betty)

Marginal comments in the text were more often negative in tone and these negative comments often focused on the lack of referencing or language and

expression (examples 15-18). Softened negative comments were also more frequent in marginal comments, often involving questions or couched in the language of advice (examples 19-22):

15. Inappropriate use of idiom here (Anna)

16. Ref- do not use first names of researchers (Betty) 17. Need a reference here (Clara)

18. Not clear! (Ethel)

19. How, and is this a valid concern (Betty) 20. How are these two ideas connected (Anna)

21. You should consider defining these terms earlier (Clara 22. You need to comment on the quality of evidence here (Diane)

Overall combined negative / softened negative comments were fairly similar for all students, suggesting that on balance the tone of end and marginal feedback comments was not one of encouragement and positive reinforcement, but rather

165

more diagnostic. The feedback tended to identify where requirements were not being met or improvement was needed, a finding similar to that of the preliminary study.

Table 7:3 Positive negative balance in formative feedback

(Numbers in brackets refer to marginal comments)

To conclude this section, a short extract from Flora’s task with accompanying marginal comments is provided to further illustrate points made above. Corrections are indicated where words are scored through and comments or corrections are in bold within brackets:

Figure 7-4 Flora formative feedback extracts Student Wholly (P) positive Wholly negative (N) Softened negative (SN) Combined negative (CN) End and marginal CN comments Anna 2 (1) 1 (4) 3 (5) 4 (9) 13 Betty 3 (2) 1 (5) 2 (5) 3 (10) 13 Clara 4 (2) 1 (5) 4 (3) 5 (8) 13 Diane 5 (1) 1 (1) 5 (5) 6 (6) 12 Ethel 3 (0) 2 (1) 5 (2) 7 (3) 10 Flora 1 (0) 1 (3) 2 (3) 3 (6) 9 Totals 18 (6) 7 (19) 16 (23) 28 (42) 70

Title: Teaching takes the child’s cognitive development into account (537 words) As David R Schaffer and Katherine Kipps (2007) emphasis [emphasise], “information about cognitive tools, skills and interpersonal relations are transmitted through intermediate social interaction” [Page reference]

Learning knowledge is not equal [does not necessarily mean]as absorbing knowledge passively.

Moreover, it is argued that each child understand questions in a different way to ask questions appropriate to [needs better phrasing] the child’s level, the content must be changed…

Hence the main job of the teacher is to create environments to encourage children

themselves to think critically and learn things through understanding rather than the center of the class[adopting a teacher-centred style]

166

The marginal comments in Figure 7-4 tended to be corrections to language and phrasing and technicalities of referencing (words and phrases crossed out with comments and corrections within the brackets). In fact, of a total of eight substantive marginal comments,36 three simply stated “page reference” where these were

missing from quotes (Flora used substantial amounts of direct quotation in a short piece of writing, 68 words of her 500 word text) with another comment stating “use surnames here”. Three end comments were written after the Reference list, two of these focusing again on technical referencing issues:

1. Good

2. Take care to identify surnames

3. After a direct quotation, you must give the page reference

The feedback and short extracts show how Flora, despite her pre-sessional work on referencing, was struggling with its technicalities. This feedback made no reference to criticality, but there were opportunities for positive reinforcement around Flora’s approach to argument. An example of this was her clear thesis statement which she re-stated in her conclusion:

(from Introduction) In this essay, I will argue that the schools and teachers should consider individual differences among the students, attach importance to develop the autonomy in learning during the process of teaching and strengthen interaction among students.

(from Conclusion) In conclusion, the educational institutes and teachers should pay more attention to the students’ individual differences, develop their ability of autonomy learning and cultivate the team spirit to share each other’s’ ideas.

The literature shows how students prefer praise with reasons rather than outright praise (Straub, 1997) but the bald end comment ‘good’ was not amplified in any way. Perhaps it reflected Flora’s attempt to structure an argument, but it is

significant that these strengths in the writing were not made explicit in marginal or end comment feedback.

36 _{There were a small number of direct corrections to language, such as ‘that’ crossed out and ‘about’} written above, with three examples of ‘the’ crossed out in the text.

167