The first cycle of full TA and retrospective interview coding

I started the first coding using Nvivo 10, based on my preliminary list of codes from other sources and on my previous experience with the earlier two printed think-aloud sets of data and with the segmentation process. The preliminary list of codes was organised under six main categories.

The following step was an attempt to use this list of codes to code the data and to see how to best use NVivo10. In this stage, I first started by using the two printed sets of data which I had coded manually. Using this data made it easy to figure out how well the main categories worked. In addition, they helped me to deal with NVivo10 without too much difficulty. For this step, in NVivo10, I chose to create structured memos to record the non-verbal data and my analytical notes. These memos helped me to link the verbal protocol transcript with my observation and my analytical notes as well as to use the content of these memos to validate and support the coding of the think-aloud transcript (Wiredu, 2014).

At this point I moved on to code the rest of the TA data. The purpose of this step was to generate more codes and start refining and defining the existing ones. These were ongoing processes. I was flexible in adding new codes under the main categories as well as adding main categories when needed, and made necessary modifications to their names or definitions according the nature of the data.

The first cycle of coding involved a lot of further decision making. The type of decisions I made are illustrated in Table 3-4 below.

Table 3-4 Examples of decisions made during the first cycle of full coding

The issue The coding decision made Rationale

1. When the teacher reads aloud and then s/he stops for evaluation or for interpretation and then s/he does the same for the

Code as: Read part of the text.

The reading behaviour is an essential

foundation of the rating process. Hence, we shall

whole composition, reads then stops.

differentiate between reading stages. 2. When the teacher spots an

error in the Grammar category and just articulates it, e.g. this is a grammatical error, without any evaluative comment.

Code as: Classify error into types.

3. How many times to code Read part of the text when the teacher’s behavior is to read then stop?

Code it once if the rater stops only to give minimal feedback such as ok/good or articulate an error and then move on. However, if the rater stops to give detailed evaluation, I will code it a second time after that.

Since we planned to count code frequencies in the end, this seemed to be the best

compromise between obtaining artificially high numbers of this code and unrealistically low ones.

4. Some teachers read all the text at the beginning but it is not clear whether they are reading all of it or just scanning it.

Two different codes: Read whole text; Scan the text.

As I mentioned earlier reading behaviour is an integral part of the rating process, so I have to be more specific in this code. Scanning could be identified

through teacher’s verbal comments.

5. How many times to code Reread

If there is no interval between Reread and doing something else (such as Reading the task prompt) code it once. However, if there is an interval between rereadings then code it twice. If the teacher rereads for the first time and then tries to interpret the sentence to understand it and then rereads the same sentence or same paragraph for more clarification or evaluation then code it a second time. Moreover, when the teacher missed something out and recalled it and then reread the same sentence/ paragraph for more evaluative purposes, code as another rereading.

The word Reread itself implies that the material is read for the second or more time. Hence, regardless of the number of times the teacher rereads the text it should be coded once unless something substantial intervenes

6. When something is said that does not refer to teacher evaluation or actual decision making, such as teacher saying 'I will put a tick or arrow' or 'I will underline this' without classifying what the kind of error is.

I will be coded it as Teacher’s reaction.

We do not need to get into too much detail on anything written or said that is not informing us about rating criteria and decision making.

7. When the teacher writes on the script some

comments or

abbreviation that denotes error types such as SP or VF.

Not coded Because this concerns

feedback to the writer, which is not the target of the study.

8. Both in the TA for the researcher and on texts as feedback for students, teachers often indicate either positive or negative evaluation such as: 'the organization is good' or 'there is no organization'. Shall I set up two different sub-codes -

No. all these should be arranged under one single code for the aspect of writing/criterion that is in focus (e.g. organisation).

This decision will eliminate the chances of discussing feedback on writing which is a different angle from my research. We are

concerned with what criteria are used in rating, how weighted, etc., not in whether the

positive and negative codes - under every criterion?

essays are often rated positively or negatively on any criterion.

In coding teachers’ decision making during their rating process, I felt it was important to distinguish between where they give immediate responses to a specific part of the text and overall comment about the whole text, avoiding going deeper. Hence, I introduced an independent code for Overall impression.

The result of the first cycle of coding was a list of 55 codes under five main categories: I. Focus; II. Presentation; III. Task requirements; IV. Reading behaviour and V. Non- evaluative comments. This cycle produced the initial list of codes that constituted a useful start to code all the TA data and helped me to gain an overall picture of what was going on during the think-aloud process. However, these codes needed to be taken under careful review and evaluation in the next cycle of coding.

Since the main categories and rationale of the coding system and the list of specific codes later changed considerably from the one sketched here we will not describe it in more detail.