Third level of analysis: Prevalence

The third level of analysis included the evaluation of the prevalence of responses. For this purpose, the answers of each student were compared to the answers of the other students within the cohort. Assessments which report the performance of students in comparison to the performance of other students within a group are called norm- referenced (Koretz, 2006, p. 50). In assessments like this, an original answer in one cohort may not be an original answer in another cohort. Therefore, in the assessment

159

used in this research, scoring prevalence as a third indicator of creativity in activity 1 can be considered an item which attributed scores in a norm-referenced manner.

At this third level of analysis, there was a calculation of a score for each of the categories created in the previous level of analysis which evaluated flexibility. That score represented the number of people who mentioned this category within the overall research sample who took the assessment. Therefore, each category which already had a unique code for the flexibility analysis was not attributed a Frequency Code. Even though each category had a unique Code for the prevalence analysis, the Frequency Code could be the same for more than one category. For example, a Frequency Code 19 would mean that a category was mentioned by 19 people in the overall cohort who sat the assessment. If another category was also mentioned by 19 people, then that category would still receive a Frequency Code of 19.

As a result, each response of the students could receive a Frequency Code based on the prevalence of this response amongst all of the students in the research sample. It has to be clarified that as in the flexibility score each category was counted only once. These had two consequences. If a student mentioned twice the same unique category, that was linked only once to a Frequency Code. Hence, there was no Frequency Code higher than the number of students in the sample.

At the last stage of this analysis, the aim was to attribute a Prevalence score to the students. There was a challenge to decide on how this score would be calculated. If the Frequency Codes were used and aggregated, then the lower the score that a student got the more creative that student would be. However, what usually happens in assessments and in the previous two levels of analysis is that the more answers students would suggest, then the students would be considered more creative.

That would be a big contradiction in the measurement of creativity. However, it was desirable that the higher scores were given to the most creative students. Hence, there was a reversion of the scores and recode of the categories of the students mentioned. In order for this to be achieved, it has to be reminded that there was no Frequency code higher than the N (number of students in the sample). This is due to the fact that each category was counted only once even if mentioned more than once by the student. This means that Pk results from the flexibility and not the fluency score.

Hence, in N is overall number of the students who sat the assessment, then the Prevalence Code for each category was calculated as

160

Prevalence Code = N Frequency Code

The calculation of prevalence code for each unique category based on its Frequency Code can be found in the appendix. What should be clarified at this point are the two different ways to measure prevalence that they were established. The first measurement of prevalence was based on the overall score and it was called Prevalence Sum. The prevalence sum is the score given to the students and it is calculated as the sum of all the prevalence codes of the responses of the student. As a result student who score high in this variable should score both have mentioned many answers and answers of categories with high prevalence scores.

However, this type of calculation could offer an advantage to the students who mentioned more categories, because they were be more variables to be added. For this reason, there was also a different calculation of prevalence score for the student. That second variable was called Maximum Value and was equivalent with the category with the highest prevalence score in the responses of the students. Therefore, the score of students for this variable would be dependent on the rarest category they mentioned in their responses.

The maximum value was also calculated because a student might not have received a high score overall, but they could have offered an innovative answer. For this reason, I decided to correlate and examine both of these indicators in order to choose which one is the most appropriate to be used. Only one of the two variables would count as a prevalence indicator in the overall score of creativity and the decision is presented in the results section.

Before, discussing the prevalence score, an adjustment was necessary. Due to the fact that N was bigger in pre-test than post-test, that slightly disadvantaged the group in the post test. To be more precise, if only one student gave a unique response in the pre-test that student scored 816 in the pre-test. A student who gave a unique response in the post-test scored 737. There is an obvious disadvantage in the second group. In order to make the two scores equivalent despite the fact the difference in the sample size in pre-test and post-test, the final prevalence scores of the students (both sum and maximum value) were calculated using the formula below

161

Similarly, the maximum value was adjusted