Qualitative Analysis of Interview Data

A typological analysis of the responses to precourse and poststudy interview questions was completed based on Hatch (2002) using NVivo qualitative analysis software. The interview protocols can be found in Appendix B. The analysis began by identifying the typologies to be analyzed. These emerged from the goals of the inquiry which were to improve the disputative and deliberative argument skill of my students. The typologies identified for this analysis are (1) statements of strategic approach for argument construction, and (2) statements of knowledge regarding how to construct strong scientific arguments. Data included in the analysis were the responses students had to the interview questions only. No other utterances were analyzed. Sections of the interview transcripts were marked as belonging to one of the typologies or the other. Responses to questions that demanded argument construction were associated with the strategic approach typology, while responses to the final question in each interview, “What makes a strong argument?”, were related to knowledge about argument construction and were marked as being of the second typology.

Reading through the transcripts of the interviews several times, I subjectively felt that students addressed the questions in a manner that depended on many factors. These included, but were not limited to, extent of problem-solving experience, level of conceptual understanding of the general topic, heuristics that have been effective in the past, skill at using logic, and epistemic considerations that included methodological trust, all of which can play a role in argument construction. The term strategic approach is used here to indicate “what is going on?” in the mind of a student while constructing the argument response and to unearth features such as those listed above. However, they were not directly measured. Instead, they were hinted at in the student responses. This qualitative analysis attempted to gather those hints systematically, and draw

conclusions based on insights that the data provided. Some portion of each student response contained utterances that were not associated with strategic approach such as (1) a restatement of the question, (2) internal rambling of thought, or (3) filler words like “um”, “so”, and “OK”. These were words that perhaps set up a thought, but weren’t needed to express it. This is not to say that any of the above did not play a role in helping the student respond to the question. Certainly, repeating the question may helped the student to mentally clarify it. But, since this analysis was trying to identify those statements that objectively played a role in the construction of an argument, restatement of the question to start a response did not move the student toward anything that could be objectively analyzed. The student was still at the starting point of the work and hadn’t measurably proceeded toward an answer.

Conversely, the students made many statements (or utterances) that indicated what they were thinking, at the moment, in relation to argument construction. These statements provided clues related to the strategy a student used to respond to the questions. For purposes of this analysis, statements were defined as being one or more sentences, or sentence parts, that made one point in the argument. For example, “Those aren’t fitting the trend” or “Density is mass over volume.”

Based on a thorough review of the transcripts, it was hypothesized that statements that indicated strategic approach could be categorized as (1) those that “declared” and (2) those that “made use of…”. Importantly, no judgement needed to be made as to whether the statements were true when categorized in this way. The list below describes these statement categories:

1. Statements that declared a fact: The student stated a given piece of information such as, “Object 1 is red.”

2. Statements that declared a concept: The student stated a scientific idea such as, “Density is mass over volume.”

3. Statements that declared a procedure: The student stated either what they were doing or what they were planning to do. For example, “I'd say the first thing that I look at is the temperature that they're all sitting at right now.”

4. Statements that made use of facts (without use of science concepts): The student stated correlations, comparisons, trends, or other patterns in the given information such as, “There is no correlation between the color and the density data” or “Object A undergoes a greater temperature change than object B.”

5. Statements that made use of science concepts (with or without specific facts): The student used a scientific idea to make an assertion. Here is an example with fact usage: “If the mass is one and the density is ten, then the volume is ten.” Here is an example without fact usage: “I can find the volume if I know the mass and density.”

Students also made statements that indicated their understanding and knowledge of effective argument structure that fit the second typology. These were associated with responses to the final question of the interview protocol. Some of these statements identified citing evidence as important when constructing strong scientific arguments. For example, “A strong scientific argument is going to be based in fact.” Other statements pointed to the importance of constructing a coherent argument, as is seen in, “…and then can be easily followed by the reader in a progressive flow, so that the starting facts can then be connected all the way down.” Finally, other statements indicated that conceptual rationale, linking the evidence to the claim, was important as in, “And um, it’s supported by research. Um, I would also say that it’s in line with prior research unless it’s seeking to contradict that.”

Once the transcripts were coded to identify the various statement-types, patterns (regularities), relationships (links), and themes (integrating concepts) were identified and used to

help answer the inquiry questions. The term coverage percent was used to identified what percentage of student utterances fell into the various statement-type categories. This concept will be described in greater detail in chapter four.