This study follows Miles and Huberman’s (1994) three recommendations for qualitative case data analysis: (1) data reduction, (2) data display, and (3) conclusion drawing and
verification (Miles & Huberman 1994: 10-12). The data analysis and collection process was interactive and cyclical, allowing the researcher to reflect, revise, and reconsider, gaining a deeper understanding of the information collected.
III.6.1 Data Reduction
Data reduction is a process that involves selecting, focusing, simplifying, abstracting, and transforming collected data into more usable forms (Miles and Huberman, 1994). Data reduction occurred continuously throughout this study from before the project began until the final report was completed. To improve validity and assist in analysis, this research included the use of various methods for summarizing (contact summary sheets, document summaries, case analysis, and interim case summaries), different approaches to coding (at both descriptive and inferential levels), methods to assist in thinking about data (annotations, journaling, and memos), and methods for producing extended reports throughout the study.
III.6.2 Data Display
Miles and Huberman (1994) also recommend the use of matrices, graphs, charts, and diagrams designed to assemble organized information into immediately accessible, compact forms. Creating a data display was an iterative process, occurring throughout and following data collection. Additionally, displays were created to compress and order data to allow the
researcher to draw justifiable conclusions. Single-case displays were collected into a matrix, which were further condensed to permit side-by-side comparisons (Miles & Huberman, 1994: 176). Cross-case displays, composite models, and sequence analysis were particularly helpful in data analysis by allowing for ordering actions and behaviors and sorting by significant categories (Miles & Huberman, 1994: 172-187).
III.6.3 Conclusion Drawing and Verification
Data reduction and data display provided the base information for conclusion drawing and verification. Review of the data collected involved identifying key themes and drawing
conclusions by identifying cues, patterns, processes, justifications, and explanations from the obtained and observed data. Initial conclusions were kept loose and tentative until further support was gathered and solidified as the process concluded. The conclusions were then verified after analysis to improve validity.
III.6.4 Coding
All interviews were recorded and transcribed by the researcher or a professional
transcriptionist. Following the recommendations of Miles and Huberman (1994), data was coded using both descriptive and inferential codes to facilitate analysis and interpretation. An initial coding scheme for the interview transcripts of expert judges was developed based on a review of literature of various definitions and components of creativity, the measurement and assessment
of creative output, dual process models of judgment and decision making, the various elements and indicators of expertise and expert cognition, the factors and differentiators of Cognitive Experiential-Self Theory and Cognitive Continuum Theory, indicators of heuristics and potential biases, and the aspects of Naturalistic Decision Making, including three different forms of the Recognition Primed Decision model. From the literature review, 11 major codes and 32 minor or sub-codes were developed for use in the review and coding of interview transcripts. The review and coding process inductively generated 3 additional major codes and 14 additional sub- codes as several unique concepts, descriptors, and unanticipated aspects of the judging criteria and process arose from explanations provided by expert judges.
Initial transcript coding involved review and analysis of four transcripts chosen by the researcher as representative of all interviews conducted. The initial coding scheme and four selected interview transcripts were imported using NVivo 10 software and “nodes” were created using the initial coding scheme. The transcripts were then coded using the initial criteria, and any potential new codes and sub-codes that emerged were coded accordingly on appropriate transcript portions. Coding was also used to help to identify coherent themes, and initial codes were modified, expanded, collapsed, and refined as data collection and analysis progressed, as new themes and patterns emerged.
III.6.5 Check Coding
A check coding process was utilized in an effort to increase both the accuracy and trustworthiness of the analysis. The four transcripts initially selected for check coding and the accompanying coding sheet were provided to two independent researchers who had no
connection to the study. The two check coders who were selected to review the transcripts were trained researchers identified and hired through a freelancer website. Both of the coders have
doctorate degrees, one in experimental psychology and the other in public health, and both have published articles in their disciplines. The two check coders separately coded the four transcripts using the preliminary codes and added new or different codes as they felt appropriate.
Preliminary review indicated overall agreement among coders of between 64% and 85% for the four transcripts. Discussion between the coders revealed that most of the differences related to assign multiple codes to the same data and choices between closely related codes. As a result, some codes were collapsed into broader concepts while other codes were divided into more discrete elements. After several discussions, all coders reached general agreement on the main and subsidiary codes and coding of the data. All remaining transcripts were coded by the researcher using the agreed coding scheme and coding approach.
III.6.6 Ethical considerations
All participants consented to be interviewed for the study and to have their interviews audiotaped. A consent form, approved by the university Institutional Review Board, was provided to each participant before the interviews. To preserve the confidentiality of the participants, the awards program and the data provided, all interview tapes, notes from
observations and interviews, and collateral data collected were kept in a locked cabinet or in a password protected file on the researcher’s personal computer. When the interviews were transcribed, code names were used to identify the participants in place of their real names and, where reported, code names are used. All judging scores received from the awards program were anonymous, with numbers assigned to individual judges and all entries identified by item