This chapter provides a description of the methodology, participants, research site, data collection and analysis, and the researcher’s role. This section outlines the techniques or procedures used to gather and analyze data to explore the proposed research questions. The initial research questions and sub-questions guiding this study are as follows:
RQ1. How did teachers perceive student assessment after participating in a coaching project that focused on data-driven instruction in an underperforming urban elementary school?
RQ2. How did a PLC focused on data-driven literacy instruction influence teachers’ perceptions of literacy practices in an urban underperforming school?
RQ3. How did teacher coaching provided by an external literacy consultant influence teachers’ perceptions of literacy instruction in an urban underperforming school? There are several compelling reasons that supported the use of a qualitative approach for this study. First, qualitative case study methodology allows researchers to explore complex phenomenon—interventions, relationships, communities, or programs—in context using a variety of data sources and supports the deconstruction and subsequent reconstruction of various phenomena (Denzin & Lincoln, 2000; Yin, 2003). Second, qualitative research methods are especially useful in discovering the meaning individuals associate with social events and lived experiences (Bogdan & Biklen, 2003; Denzin & Lincoln, 2000). The purpose of this study is to
better understand how literacy teachers in an urban, high-poverty elementary school use data to improve literacy instruction.
The research design was selected after careful consideration of various research
methodologies, the study’s purpose, the research questions, and the researcher’s epistemological stance. The foundational tenets that guide qualitative research serve as the means to
contextualize and understand the research questions in the study. A qualitative approach is most appropriate for this research because it fosters a better understanding of the lived experiences of the participants, elementary school teachers, and their own understandings of how they collect, make sense of, and work with student literacy data. Furthermore, this research allows
participants to articulate the ways in which student literacy data informs teachers’ pedagogical decisions.
This study, based in a constructivist paradigm, used an exploratory case study design to capture the details of data-driven instructional decision making of 15 elementary school teachers serving mostly African American and Latino/a students and the context, social practices, and outcomes of a data-centered professional learning community and coaching interactions led by an external literacy consultant. Further, an exploratory case study approach was used to explore how teachers’ beliefs and attitudes about student data use influence their data-driven literacy practices in an urban, under-performing school.
Philosophical Foundation
The epistemological stance framing this dissertation is constructivism. Constructivism asserts that individuals construct meaning differently, even when experiencing the same event (Crotty, 1998). There are several assumptions of constructivism, many of which are essential to this qualitative study:
1. Because meaning is constructed by people as they engage with others in social interactions, qualitative researchers use open-ended questions to solicit the viewpoints of participants;
2. People engage with others in social interactions and make sense of those encounters based on their own historical, cultural, and social perspectives; and 3. Meaning is socially constructed and results from human interactions within a
community.
These assumptions help remind researchers that interpretations and findings in a qualitative study are context-specific.
Constructivism is a helpful philosophical framework for this study. According to Stake (1995), the researcher is a central instrument in the collection and interpretation of data. Stake further claimed that most contemporary, qualitative researchers promote the belief that
knowledge is a social construction rather than a discovery. Stake described constructivism as a belief that knowledge is formed by social interpretations about the world.
This study is based on the interpretations of teachers working in an urban elementary school who are responsible for planning and providing literacy instruction. Of particular interest are the ways in which these teachers interpreted their interactions within a professional learning community and coaching activities focused on student data use and literacy instruction.
Furthermore, the study explored teachers’ perceptions of the student assessments used, student achievement, and literacy practices within the school and their respective classrooms.
Participants in the study constructed their own unique understanding of their individual and shared experiences. How they made instructional decisions based on their interactions with student assessment data, the external literacy consultant, administration, and students is complex and reflects a constructivist epistemology.
In terms of analysis, the interpretive theoretical perspective provided a framework for understanding the ways in which teachers interpreted and constructed meaning around the
student assessment data that they collected and analyzed and the literacy practices they used in their classrooms. Specifically, this research is interested in discovering how an external literacy consultant supported teachers in collecting and analyzing student assessment data and how the data, if at all, informed literacy practices for elementary school students in a small, urban school. The interpretive tradition emphasizes the examination of a social setting through inquiry rather than imposing predisposed assumptions and theories on a context. A major component within interpretivism is the role of the researcher in understanding how participants construct meaning of a situation or interaction. Thus, the researcher is an instrument in understanding how
participants perceive their experiences. The strategy is inductive, and the outcome is descriptive (Merriam, 2002). In general, the interpretive approach involves researchers immersing
themselves in the natural setting that their study participants inhabit rather than beginning with a theory or preconceived idea of how the social players work together to construct, modify, and interpret their social reality (Esterberg, 2002).
Qualitative Case Study Methodology
This section provides an overview of case study research, defines case study
methodology, and examines the usefulness of a case study research design for addressing this study’s research questions. Although there are several prominent case study researchers who have written about case study research, for the purpose of this dissertation, I will rely mostly on the extensive scholarly works of modern case study methodologists Robert K. Yin, Robert E. Stake, and Sharon B. Merriam.
Stake (1995) described case study methodology as a strategy of inquiry in which the research explores in-depth a program, event, activity, process, or one or more individuals. To this end, qualitative case study methodology is appropriate for the study because the research
learning environment consists of coaching activities and a professional learning community of teachers involved with literacy instruction. The study explored teachers’ data-use practices in the context of a professional learning community and one-on-one coaching activities and, ultimately, the myriad factors influencing teachers’ adoption and application of data-driven instructional choices in the classroom.
According to Yin (2003, 2014), case studies are the preferred approach when “how” and “why” questions are posed. Yin additionally claimed a qualitative case study approach is well- suited for examining contextual conditions that appear relevant to the phenomenon being
studied, literacy teachers’ data-driven decision making. In this instance, the urban school context and the social context in which literacy teachers engage with, analyze, and interpret student data and build their pedagogical knowledge around literacy instruction is particularly important. It is in these settings, coaching activities and the professional learning community, that data-driven instructional decision-making skills are developed and practiced.
A qualitative approach allows the researcher to explore phenomena such as feelings and thought processes that are typically difficult to ascertain through conventional research methods (Strauss & Corbin, 1998). Qualitative methods emphasize the researcher’s role as an active participant in the study. For the present study, the researcher will be instrumental in the
collection, interpretation, and analysis of data. Ultimately, the intention is to explore and seek to understand participants’ beliefs, attitudes, and perceptions of their current and potentially changing data-use and literacy practices in the school and their respective classrooms.
Unit of Analysis
Stake (1995) defined a case as a specific, integrated, bounded system that becomes the unit of analysis. Cases are bounded by time, place, and activity, and researchers collect detailed
narrow the scope of research, case boundaries will be established to indicate what will and will not be studied (Miles & Huberman, 1994; Stake, 1995; Yin, 2014). For the present study, the unit of analysis will be a three-year professional development program focusing on data-driven instructional decision making for literacy teachers at an under-performing elementary school in Washington, D.C. A single case study design will be used to explore literacy teachers’
participation in various professional learning opportunities, including one-on-one coaching activities with an external literacy consultant and a data-centered professional learning
community aimed at building teachers’ data-use capacity and improving literacy instructional practice.
Socio-Political Context of District of Columbia Public Schools
In 2011, a National Research Council report on District of Columbia Public Schools (DCPS) ranked student test score averages in the city below almost every other major city from 2003 onward. According to the National Assessment for Educational Progress (NAEP), only 8 percent of the districts’ eighth graders were proficient in math and only 12 percent were proficient in reading (NCES, 2011). In sharp contrast, Chancellor Michelle Rhee found 95 percent of DCPS educators were “meeting expectations” based on the performance evaluations she examined when she was hired to reform the district’s public school system. During Rhee’s chancellorship, she put in place neoliberal reform agenda aimed at instilling accountability and improving principal and teacher quality.
Turning the ship would require strategies that had never been tried before in D.C. It was clear that those strategies would generate significant pushback, especially from those with a vested interest in the status quo, and in a system that had zero accountability in place (Rhee, 2012, p. 40).
including Baltimore, New Orleans, New York City, and Washington, DC. This work, which mainly focused on strengthening the teacher workforce in major cities by privatizing teacher preparation, helped cement her blueprint for creating a culture of accountability in DCPS based on high-stakes testing and teacher and leader performance evaluations. During Michelle Rhee’s first year as chancellor, her leadership team channeled their efforts on disrupting the central office and dismissing school leadership who appeared ineffective. According to a 2014
Mathematica study, 39 percent of DCPS’s 131 school principals left the district at the end of the 2007–2008 school year. In the subsequent year, 22 percent of school leadership left the system.
Next, Rhee’s leadership team focused their attention on improving teacher quality by building and rolling out a robust evaluation system intended to accurately assess teacher performance in the district. Eventually, the new IMPACT evaluation system would cover all 7,000 school-based staff, from custodians and aides to teachers and principals. It marked a new era of accountability in DCPS that rewarded teachers based on performance rather than seniority. To ensure IMPACT focused on what mattered most—high-impact instruction—Rhee’s
leadership team developed a new Teaching and Learning Framework for DCPS, comprised of nine instructional practice standards. The IMPACT teacher evaluation system was based on classroom observations, student achievement data, and other pertinent information about the teachers’ professional practice. Union leaders and educators commonly critiqued the new teacher evaluation system as being too complicated while others thought the rating system was overly simplistic and too blunt of an instrument to capture the full range of instructional practices and interactions teachers enact each day. “There are hundreds of human capital questions you need to answer to effectively run a school district, and now, for the first time, we have the data to answer
them,” remarked Jason Kamras, Director of Human Capital for Teachers, a new role within the newly formed human capital department (McGrummen, 2011, p. 5).
Along with the new teacher evaluation system, Chancellor Rhee brokered a new
agreement with the Washington Teachers Union to include performance-based compensation for the first time for teachers. Teachers rated “Highly Effective” were eligible for higher
compensation as well as significant bonuses while teachers rated “Minimally Effective” would receive no salary increase. Teachers receiving “Minimally Effective” or “Ineffective” ratings for two consecutive years would be terminated from DCPS. In summary, Michelle Rhee and her leadership team actualized a neoliberal agenda by implementing radical and swift changes to DCPS’s human capital policies and structures around teacher and leader recruitment, selection, evaluation, development, compensation, and retention, with the ultimate goal of increasing student achievement as measured by standardized tests. From 2007 to 2014, math proficiency scores in DCPS rose 23 percentage points and reading scores rose 13 percentage points. The district’s students made larger performance gains than their peers in any other state in the NAEP assessments.
Research Site
Fortunately, there were no barriers to identifying and accessing a research site to conduct this study. The researcher had a school contact who was involved with coordinating and
delivering professional development to literacy teachers as part of the Target Literacy Grant. The researcher knew the external literacy consultant in a professional capacity—they had previously worked together as special education teachers. Gaining access to the school was achieved by meeting with the building-level administrator, who granted permission, and coordinating with the external literacy consultant to observe one-on-one coaching sessions with teachers,
instruction. In addition, the external literacy consultant assisted with arranging teacher interview schedules. Because the nature of the study was to determine how literacy teachers improve their instructional practice and their collaborative data use, the school administrator and the
participating teachers were interested in the study results.
The Target Corporation lists as its corporate giving and philanthropic work a partnership with America’s Promise Alliance (APA), an education organization founded and led by General Colin and Mrs. Alma Powell. Target sponsored America’s Promise’s annual graduation report,
Building a GradNation, and a companion study titled Don’t Call Them Dropouts, focused on
raising the nation’s high school graduation rates. According to Target’s corporate responsibility report, “a child who can’t read proficiently by the end of third grade is four times more likely to drop out of high school than a child who can. As part of Target’s commitment to education, we’re working to help more kids become better readers and stay on the path to graduation” (Target, 2013, p. 1). In 2010, the Target Corporation committed to give $1 billion for education improvement initiatives by the end of 2015 citing the following reasons:
1) Target consumers indicated education was their primary social concern,
2) To help address America’s dropout crisis, in which one-third of African American and Latino students did not graduate on time and one-fourth of students did not graduate high school, and
3) To help create an educated workforce of future leaders, ensuring the country’s global competitiveness.
In 2011, the Target Corporation first piloted a literacy program in six elementary schools in Minneapolis, MN, which marked the beginning of the company’s work on “personalized literacy programs” and laid the groundwork to launch literacy pilot programs in Los Angeles and
Washington, DC.
across the country to receive this multi-million-dollar grant from Target Corporation for the Targeted Literacy Initiative (TLI). The Targeted Literacy Initiative was a three-year grant program designed to improve literacy instruction and expand literacy coaching and resources. The three-year grant was split among six DCPS elementary schools and funds were used to hire external literacy consultants to build teachers’ data-use capacity, improve the use of interim assessments to guide instruction, and help teachers use data management technology to access timely and disaggregated student data (DCPS Press Release, 2012).
External literacy consultants were a main component of the TLI grant and were employed and managed by Amplify to work on-site at a designated school. Specific coaching practices that the external literacy consultant was responsible for included:
Co-planning literacy instruction in response to data;
Co-teaching and modeling exemplary literacy instruction;
Observing literacy instruction in classrooms;
Serving as a reflective thought-partner before, during, and after literacy instruction; and
Developing and delivering meaningful and sequenced professional
development to build teachers’ instructional practices and data-use capacity
based on literacy needs.
Additionally, literacy consultants played an integral role in the school leadership team and worked to develop collaborative working relationships with teachers and the school community. Overall, the objective of the TLI grant was to change teachers’ data-use practices and prepare teachers to make appropriate instructional responses to student assessment data that will ultimately improve student literacy.
An integral component of improving literacy instruction in the school was building teachers’ capacity to understand and use benchmark assessment data to drive instructional change in the classroom. The external literacy consultant hired had experience working with
student data to drive education improvements. The literacy consultant led professional learning communities called Collaborative Learning Cycles (CLC) focused on guided reading, close reading, and developing literacy lesson plans with teachers during collaborative learning sessions. These professional development topics aligned with the coaching support that literacy consultants provided teachers, which also focused on guided reading, small-group instruction, literacy work stations, shared reading, interactive read-alouds, and developing literacy lesson plans for a 120-minute literacy block. Over a three-year timeframe, the external literacy consultant focused solely on developing a data-literate faculty and expanding the instructional repertoires of literacy teachers in the school.
Table 1. 2013–2014 Demographic Characteristics of the Student Population
Number Percentage Total Population Ethnicity Black students 280 194 69% Latino/a students 60 21% White students 19 6% Asian students 3 1% Multi-racial students Special Education
English Language Learners Free/Reduced Lunch 4 70 34 276 1% 25% 12% 99%
Implementation plans for Year 3 of the TLI grant included effective orientation of new teachers to the TLI program due to teacher turnover, building teacher capacity and leadership skills to sustain the literacy initiative, developing resource materials and a repository to house data collection information, and implementing Collaborative Learning Cycles with the teachers.
A key element of succession planning for when the TLI grant ends was identifying and
supporting teacher leaders to assume responsibility for planning and leading the CLCs after the external literacy consultant departed the school.
Based on the 2013–14 DCPS School Scorecard, the grade levels in the school range from pre-kindergarten to fifth grade with approximately 280 students enrolled in the school. As shown in Table 1, the average core class size was approximately 15 students. The student population was 69% Black, 21% Latino/a, 6% White, 1% Asian, and 1% multi-racial. The school received