Classifying types of comments and identifying them in revisions

First, I read peer response transcripts and coded each comment made by a peer responder according to a modification of a scheme developed for a peer response study that compares the kinds of comments that ESL freshman composition students make in face-to-face versus online peer response settings (Liu and Sadler, 2003). The original coding scheme considers the type of comment (evaluation, clarification, suggestion, or alternation); the area of the comment (global or local) and the nature of the comment (revision-oriented or non-revision oriented). Evaluations focus on either good or bad features of writing, clarifications probe for explanation or

justification, suggestions point out the directions for changes, and alterations provide specific changes. Global comments focus on idea development, audience, purpose, and organization of writing; local ones are oriented toward copy-editing, such as wording, grammar, and punctuation (McGroarty and Zhu, 1997, as cited in Liu and Sadler, 2003). Finally, revision-oriented

comments point to areas that need to be changed, while non-revision-oriented comments signal areas of the paper that are strong and should not be changed. For the purposes of this study, with its small number of participants relative to other studies like Liu and Sadler, I considered only the area of comments (global or local) and their nature (revision-oriented or non-revision

oriented). Table 5-4 provides an example of each of these types of comment as they were identified in the current study:

Table 5.4 Modified scheme for classifying comments (Liu and Sadler, 2003)

Global Local

Revision- oriented

This section should be longer. “These children” sounds kinda rude. Non-revision

oriented

I really like your personal connection.

I didn’t think you had any grammar problems.

The final measure used to gauge improvement from first to second drafts is a

consideration of whether or not writers used their partners’ comments when making revisions. This attempt to trace peer response suggestions from the spoken interaction to the revision phase has been used in other peer response studies (e.g., Nelson and Murphy, 1993). To complete the task for this study, I used a revision record to: (1) record each specific, revision-oriented

comment (according to Table 4.4) that was made in peer response; (2) determine from the second draft whether or not it was incorporated; and (3) record any insights about making the revision choice that were supplied in stimulated recall interviews. Table 5-5 provides an example revision record, showing a selection of the suggestions that Zelda gave Ivana for the second paper:

Table 5.5 Revision record for Ivana

Revision Suggestion Action Stimulated Recall Comments

Add personal experience to conclusion

N Ivana didn’t do this because she didn’t think it

was appropriate for an academic paper. Add a transition sentence

between first and second paragraphs

Y Ivana said she was “inattentive in the class” and

forgot that the instructor asked them to make transitions.

Expand the conclusion Y Ivana said that while she didn’t think it’s

appropriate to include personal opinion, she did think the conclusion was too short.

I completed a revision record for each participant’s second draft (twenty-six draft). Next, I calculated the number of comments and the percent of comments included in second drafts.

Chapter Seven reports the results of this analysis, presenting uptake of writer comments

according to the pattern of interaction role that he or she adopted during the conversation about

the draft. It also provides more detail about how I identified specific, revision-oriented comments in the peer response transcripts.

5.4 Researcher observation notes and classroom documents

The final data source comes from classroom observation of the sessions when peer response was conducted. I attended class sessions and took observation notes about how the instructor set up the peer response task, moderated the task while students interacted with each other, and communicated with students what they should do after the peer response task. I also audio recorded and transcribed the class session when students received a brief training about how to participate in peer response. Finally, I collected class materials related to the peer

response session. Yin (2003) would call these handouts physical artifacts, and lists them as one

of the six sources of evidence that can be used in case study research. These documents can help the researcher to contextualize the case study by fully describing the classroom setting (Duff, 2008).

In this study, these physical artifacts helped me to understand some of the language and content of students’ peer response sessions, as well as to consider how the instructor’s behavior and her framing of the task might influence their revision behavior. In the handout prepared for the third peer response session, where each student had written a research paper about a topic of his or her choice, the instructions asked the students to tell their partner something they liked and/or something they learned from the paper. Several students used the phrase “one thing I liked about your paper…” in these sessions, illustrating how class handouts directed topics and

Observation notes from class sessions showed that the instructor circulated around the room while students were reviewing each other’s papers, pausing to answer questions when necessary. Thus, a pair who could not reach consensus on the proper way to use in-text citation decided that they should “just ask Cindy”. It bears mentioning that although the instructor was present to answer questions, she encouraged students to rely on each other for the majority of their feedback, while she occasionally answered technical questions like the one mentioned above.

Finally, observation notes from peer response sessions show that the instructor always encouraged students to use their own judgment when deciding if and how to incorporate their partner’s feedback during revisions. She closed one class by asking students to make revisions after the session, reminding them that “ultimately, you are the writer, and you decide”. While none of the participants directly referenced these instructions in their stimulated recall

interviews, it is possible that students who decided not to incorporate their partners’ feedback felt justified in doing so because of the way the instructor explained the revision process at the end of class.