Empirical Testing

Armed with theoretical sensitivity towards significant terms, categories of activity, and the potential relationships between them described in the literature, I set out to determine how these distinctions might help me make sense of classroom instruction emphasizing CPs. My goal was not to determine whether the instructional theories outlined above were effective; rather, I set out to determine whether this a priori set of categories I developed from the literature were empirically viable. In the next section, I describe in the role of empirical observation in the development of the framework.

3.1 Setting and Participants

I partnered with a practicing high school teacher whom I refer to with the pseudonym Ms. Spence. Ms. Spence frequently used contextualized problems in

instruction but was not familiar with MMP or RME as theoretical models. I observed two different sections, two to three times per week over the course of twelve weeks, for a total of 78 observations. In addition to these observations, I conducted 20 post-instruction interviews with the teacher, to gain insight into the critical implementation issues that arise from practice (Heid et al., 2006). Ms. Spence taught from a curriculum called Core- Plus Mathematics: Contemporary Mathematics in Context [commonly called Core-Plus] (Hirsch et al., 2008), developed by an academic center in the US with funds provided by the National Science Foundation.

This setting was appropriate for the development of the framework for a number of reasons. First, Ms. Spence was an ideal partner in the research because she had 13 years of experience implementing Core-Plus and had participated in a great deal of professional development specifically targeting Core-Plus implementation. Because of this significant experience with the curriculum, she was able to consciously reflect on what she perceived as the strengths and weaknesses of the curriculum as well as the ways in which she adapted, supplemented, and selected from the curricular offerings to address critical issues of practice. Second, high school instruction is under-represented in the theoretical literature around the use of CPs for developing new mathematical

understanding and in RME and MMP literature specifically, so a framework that can be used to describe CP-based instruction at this level would be valuable to the field. Finally,

Core-Plus was a particularly appropriate curriculum to observe in combination with the theoretical perspectives described above because, although mathematical modeling is a unifying theme in the curriculum and an inspiration for its instructional design, the development of this curriculum was not explicitly guided by any single particular

instructional theory like RME or MMP model-development sequences (Fey, personal communication, July 25, 2011; Hirsch, personal communication, June 15, 2011). A framework that is grounded in these particular instructional theories and developed to make sense of a curriculum that was not designed from these theories has significant potential to be generalizable.

3.2 Data Sources

The primary dataset used for the development of the framework consisted of written Core-Plus lessons and transcriptions of a subset of the observed lessons. Two full units of Core-Plus textbook were analyzed: 26 instructional days of an algebra unit focused on linear relationships and 20 instructional days of a statistics unit focused on distribution and descriptions of center for one-dimensional data sets. I recorded,

transcribed, and coded eight sessions of the algebra unit; each session was 1.5 hours long. During observations, I took field notes. I also engaged the teacher in 15 post-observation interviews. From the field notes and interviews, I identified types of tasks that were particularly meaningful to the teacher or to me as an observer.

3.3 Data Analysis

The framework was developed through an iterative data analysis process. First, I consulted relevant literature to develop theoretical sensitivity and an initial set of

conceptual categories. Each of the four levels of RME, for example, represented a potential category of activity. Next, I analyzed instructional activity in both the written

curriculum and transcripts of classroom observations. I use the term activity as an umbrella term that includes tasks, questions and statements. In order to assign codes to sections of the textbook and class transcripts, I segmented the written curriculum and enacted curriculum into topically contained turns [TCT], a variation on Mehan’s (1979)

topically related sets. In discourse analysis, a turn is the speech unit that begins when one person speaks and ends when another person speaks. Mehan defines a topically related set as a group of these turns focused on a single subject (e.g. the spelling of a word, the solution to a particular instructional exercise). To adapt this idea for the

purposes of coding the text, I decided that, in a textbook, a “turn” ends when the reader is signaled to perform some sort of task or to answer a question. A transition to a new topic or section also indicates a new turn. Each problem or task denoted by a letter or number is its own topically contained turn. Similarly, teacher utterances in the transcriptions of the enacted lessons were segmented into topically contained turns. A new TCT was indicated when a student was expected to perform some sort of task or respond in some way. A single utterance was segmented into multiple TCTs if a new topic was clearly indicated.

I analyzed the written and enacted curriculum using the constant comparative method (Glaser & Strauss, 1967). As I read each TCT, I assigned a code, or label, describing what type of activity was present. As I continued coding, if a particular activity matched the type assigned to a previous activity, it was coded with that label. If it did not fit, an existing code was modified or a new code was created. Over the course of the coding, the definitions of these codes stabilized. These codes were then categorized and organized according to the interpretive ideas from the literature and problems of

practice identified in my observation field notes and teacher interviews. This categorization was guided by the following questions: what are the dimensions or organizing concepts that connect these categories? How do they relate to each other? What types of activity are contained inside each category?

When ideas from the literature were not sufficient for describing or organizing the empirical data, I performed another round of literature review targeting any activity types that did not fit into my existing codes. Existing categories were adjusted or new

categories were established; existing codes were re-categorized. In the end, a set of 105 activity-type codes were identified and categorized into the ten categories found in the resulting framework. For example, the code “answer a context question given a mathematical representation” was placed into a broader category called “interpret

model”; the code “explain the connection between two different types of representations” was placed into a category called “focus on model”.

After the framework was established, a coding manual was developed that described the ten categories, including the various types of activity within the categories. The ten categories were validated through a test of inter-rater reliability. I trained an advanced mathematics education graduate student on the use of the coding manual, and she coded two of the Core-Plus lessons, assigning one of the ten category codes to each TCT. Comparing her codes to my own, I found the joint probability of agreement to be 85%. The resulting framework is grounded in empirical data, connected to prior theory and empirical research, and also empirically validated.

4. A Framework for Analyzing the Role of Contextualized Problems in Mathematics