Data Analysis Methods and Metrics

As we mentioned above, evaluating outcomes was not a main focus of this study. We wanted to analyze the ‘way in which’ the authoring took place in-situ. A theoretically-driven discourse analysis was used to analyze the data collected to identify qualitative trends in the children’s creative process. The coding process was conducted thus:

40

All the audio clips recorded during the story creation sessions were transcribed with associated speaker turn numbers, emphasis points and short event descriptions (e.g. playing the animation). A reference chain analysis as described in (David McNeill et al., 2008) was performed by four coders. Each was first allotted a transcript to analyze according to the following coding scheme of speech at three levels: object, where the reference is to a task-related entity; meta, which includes references to the talk being carried out (this includes speech referencing the process of story construction and not the stories being constructed themselves); and para, which references things and persons in the environment and includes discourse about the individual’s perspective and opinions. The four coders then conferred and resolved any serious discrepancies in the different understandings of the coding scheme. We found that an object code in our case could refer either to talk about the Story or the Tool. The transcripts were then redistributed among the coders, and coded again with new understanding of the coding scheme. After the reference chain analysis, an open coding was done on the transcripts to reveal categories and themes. Examples included ‘distribution of roles’, ‘asking for help’, ‘setting up next scene’, or ‘searching the library’. All the instances of each code and theme were then grouped and analyzed for patterns using Coughlan & Johnson’s (Coughlan & Johnson, 2009) two perspectives of creative interaction as a guide.

The first perspective, structural interaction entails the practices and the structuring of the creative process. In this respect we looked at the different stages that the children went through to complete the task. The speech acts coded as Para and Meta helped in identifying patterns. The second, productive interaction is concerned with “low-level interactions”, such as the generation, externalization and evaluation of ideas while working towards the final outcome. We thus looked at the different strategies that the children employed to generate ideas for their stories by employing Guilford’s (1967) concept of convergent and divergent thinking but given our interest in the process, we analyzed the concepts in a more qualitative manner. Divergent thinking has been defined as “producing a variety of responses in which the product is not completely determined by the information given to the respondent” (Mark A. Runco & Pritzker, 1999). Several divergent thinking tests have been developed (e.g. Torrance’s Tests of Creative Thinking, Guilford’s Alternative Uses Task). In our coding process, divergence was taken to be a proposition of a story idea that departs from previously presented material, either in the stimulus story or by other group members. It was however not taken to be equivalent to disagreements. If only a disagreement was manifested without a suggestion for a change, it was not counted as an instance of divergent thinking.

Convergent thinking refers to the process of following “a particular set of logical steps to arrive at one solution, which in some cases is the ‘correct’ solution (Mark A. Runco & Pritzker, 1999), and has often been measured using the Mednick’s Remote Associates Test. For our study, an instance of convergence entailed a proposition of a story idea that draws from, builds, follows or draws from what was presented before by the stimulus story or by another child. Apart from

41

divergent and convergent thinking, we also coded a third element that we labeled as ‘seeding’. The emergence of a story idea was categorized as seeding when it consisted of a proposition that did not draw from what has been presented before in any form. It should be noted that all story ideas were considered in the analysis, including ideas that were suggested but did not eventually make it into the final story. Furthermore, no assessment was done as to the originality of the story ideas during the coding process. A convergent idea could be as original or more original than a divergent idea, but if it drew from material presented in the past, it was still coded as convergence. The coding was done mainly in terms of how an idea shaped, directed or influenced the path of the creation of the final story product.

A further analysis was carried out for the children’s retelling of their stories in the post- interviews. There were three storybooks and three animations produced. Although the stories were short, each story was first reduced to a ‘narrative digest’ (L. M. Register & T. B. Henley, 1992) before being used for analysis so that they all had comparative value. Procedures to create a ‘narrative digest’ included reviewing the story and writing down the main events that happened in the story as a list of sequential story propositions. The retelling of the story by the child was then compared to the story’s ‘narrative digest’ and checked for coverage of the propositions.

Finally, we used Amabile’s (1982) consensual assessment technique for the evaluation of the actual stories. Three teachers with experience dealing with children of the 8 to 10 year old age group were asked to be the judges. Teachers were chosen because as pointed out by Boden (2004), it is not possible for one to “decide whether or not an act is genuinely creative” without being familiar with the conceptual space of the creator. The teachers were asked to judge the stories with the following instructions: they were asked to first go through all the six stories without any judging. They then had to preview each story again one by one and fill in an evaluation form for each. The ratings scale was exactly as Amabile’s and included the dimensions of creativity, novelty, coherence, continuation and completion. Following Valkenburg and Beentjes (1997), we grouped the first two dimensions into a story Novelty component, and the last three into a story Quality component.