LESSON STUDY LIBRARY: A CATALYST FOR INCREMENTAL INSTRUCTIONAL
IMPROVEMENT
A Project Presented to the Faculty of
California State University, Stanislaus
In Partial Fulfillment
of the Requirements for the Degree of Master of Arts in Education
By Carolyn Viss November 2014
CERTIFICATION OF APPROVAL
LESSON STUDY LIBRARY: A CATALYST FOR INCREMENTAL INSTRUCTIONAL
IMPROVEMENT
by Carolyn Viss
Signed Certification of Approval Page is on file with the University Library
Dr. Dawn Poole
Professor of Educational Technology
Dr. Oddmund Myhre
Professor of Educational Technology
Date
© 2014 Carolyn Viss
DEDICATION
This lesson study project is dedicated to my husband, Max Viss, without whose ongoing support and belief in me, not to mention an occasional encouraging kick in the pants, the work would not have been completed.
ACKNOWLEDGEMENTS
The completion of this project is the culmination of learning over many years. Most of my own learning about effective teaching and learning, as well as how to improve one’s professional practice and support the learning of other teachers, is the result of years of collaborative work with the team in Waterford Unified School District. Don Davis, then principal and now district superintendent, mentored me in that work and to him, I owe a debt of gratitude. My work at the Stanislaus County Office of Education, and collaboration with Erin Cross and the team, launched me into this journey of lesson study which carries to potential to incrementally change the culture of classrooms to support mathematics learners in a powerful way.
TABLE OF CONTENTS
PAGE
Dedication ... iv
Acknowledgements ... v
List of Figures ... vii
Abstract ... viii
CHAPTER I. Introduction ... 1
Statement of the Problem ... 3
Significance of the Project ... 13
Definitions... 14
Summary ... 16
II. Review of the Literature ... 17
Effect of Professional Development on Teacher Practice ... 17
Lesson Study as Professional Development ... 25
Summary ... 32
III. Description of the Project ... 33
Overview ... 33
Logistics and Design Considerations ... 33
Factors in Implementation ... 34
Overview of the Timeline and User Access to the Project ... 35
Evaluation ... 37
References ... 40
Appendix: The Structure and Use of the Lesson Study Library ... 46
LIST OF FIGURES
FIGURE PAGE
1. How to filter by criteria... 47 2. Selection by criteria ... 47
ABSTRACT
This project creates a digital archive for lesson study materials created by
mathematics teacher teams working to deepen their understanding of content and instructional strategies associated with the Common Core State Standards for Mathematics. The archive serves as a digital library, searchable by grade level, domain, content standard, standard for mathematics practice, and other criteria. Teacher teams may use this library as a starting point for their own action research, or lesson study, on a particular learning objective. In addition to retrieving existing lesson study documents, users may submit the data from their own team’s lesson study. This growing knowledge-base of mathematics resources serves to support teachers as they work to increase student achievement and deepen student
understanding of mathematics at every grade level.
CHAPTER I INTRODUCTION
U.S. students’ academic achievement in the areas of reading, science, and mathematics lags behind that of students in many other countries (Organization for Economic Cooperation and Development, 2013). Despite a decade of pressure to improve achievement through accountability systems, there is little evidence of substantive improvements in student performance. Students have been tested annually and the performance of schools and districts measured and reported to stakeholders. Specific subgroups’ outcomes, like socio-economically disadvantaged youth, English learners, and others, have been disaggregated, and schools required to raise the performance of all subgroups to proficiency or face sanctions from the state or federal government (U.S. Department of Education, 2014). By 2013, 80% of K-12, Title 1 public schools in California had been labeled as underperforming for failing to meet such criteria (Education Data Partnership, 2014). At the same time, educators have not seen hoped-for narrowing of persistent performance gaps between rich and poor, white and minority, second language and English only
students. Internationally benchmarked assessments provide no relief, revealing little change in students’ performance overall in the last decade (Organization for
Economic Cooperation and Development, 2013).
The implications for post-secondary education in the U.S. are significant, with many college freshmen failing to demonstrate college readiness in English and in
2 mathematics. Half of U.S. college students are enrolled at a two year college. Fully 60% of those students nationally are not college-ready for math, and are placed in a remedial mathematics class. Of those students, only 25% complete a degree or credential within eight years (Silva, 2013).
In their desire to support learning among children in their systems, local reform efforts have made progress on the dual challenges of raising proficiency levels for all students and narrowing the gap in achievement between student groups. In studying educational best practices, Douglas Reeves (1995) coined the phrase 90/90/90 schools to describe schools with at least 90 percent of students receiving free or reduced price lunch, at least 90 percent minority students, and at least 90 percent of students meeting or exceeding state or district proficiency targets in reading or other content areas. Reeves and others have studied the systems which have attained such successes as measured by the California Standards Test (CST), and there is guidance to be found there as to what structures contribute to effective reform. To date, however, such efforts have not been replicated at scale.
Lesson study, a staple of professional development among educators in some high performing countries as measured by the National Assessment for Educational Progress (NAEP) and the Program for International Student Assessment (PISA), is being adopted by some teacher communities in the U.S., and the structure of lesson study addressed some of the key findings of researchers on affecting teacher practice and student outcomes. Lesson study is a small scale action research protocol in which a teacher team plans and delivers a lesson. Team members serve as data
3 gatherers who collect and analyze student production triggered by the lesson, and a revision cycle occurs.
Statement of the Problem
Teachers’ ability to deliver quality instructional opportunities to learners is enhanced by professional collaboration on practice (Garet, 2001). The work of lesson study leverages research-based findings that sustain change in practice. In its basic form as nascent groups in Stanislaus County engage in the work, the final product is a lesson that has undergone a measure of action research. Double benefits to the group consist in the learning in which they engage as they collect and analyze data on students’ responses to the lesson they have planned together as well as, possibly, a well-planned and revised lesson owned by the group. The teachers in the team may use the lesson again in subsequent years. The reach of that learning, though, is limited to the original group of teachers and perhaps to colleagues with whom they work. In the absence of a change to this protocol, the means of extending the action research or sharing the outcomes with other teacher groups is lost.
Poor achievement in mathematics among U.S. students is evidenced by internationally benchmarked measures like the PISA (Organization for Economic Cooperation and Development, 2013). With U.S. student performance below the average of participating nations, providing teachers with opportunity to refine instruction through collaboration and job-embedded professional development is key. The need is even greater in California, which trails the national average in students performing Proficient or above in mathematics on the NAEP (National
4 Center for Educational Statistics, 2012). One implication of this weakness in K-12 mathematics education that merits attention is the high rate of incoming college freshmen who do not demonstrate mastery of high school mathematics and are placed in remedial courses. These courses do not earn the student college mathematics credit and enrollment in these courses correlates strongly with elevated rates of college drop-out (McCormick & Lucas, 2011). Although the vast majority of two-year colleges and most four-year degree bearing programs offer remedial courses, access may be limited to those who do not demonstrate readiness. And university readiness in the area of mathematics is increasingly aligned with the expectations of employers in the workforce (Huebner & Corbett, 2008).
The California Common Core State Standards in Mathematics (CCCSSM) create a framework for educators to see what sorts of learning their students must engage as they prepare U.S. learners to compete in a global economy. Expectations like Standard for Mathematical Practice (SMP) number 1, “Mathematically proficient students make sense of problems and persevere in solving them” (California
Department of Education, 2014, p. 6) make clear that the K-12 system must adapt. What is not clear is how that adaptation can best be supported.
In a large-scale study of instructional practices, first in the U.S., Japan, and Germany, and later with an expanded list of participating nations, the Third
International Mathematics and Science Study (TIMSS) study sought to identify differences in teaching and learning practice across nations. James Hiebert and James W. Stigler (2007) analyzed videotaped instructional segments in mathematics
5 classrooms in these participating countries. They found that while instructional practice within countries varied little, significant differences in practice were evident between countries. They found the differences to be deeply cultural. Teachers teach, in large measure, the way they were taught. U.S. teachers’ comfort with teaching procedures and discrete mathematical skills, and their celerity in doing the
mathematical thinking themselves at the first sign of students’ grappling with their own understanding are rooted in their experience as learners and in the expectations of their administrators, parents, and most immediately, of their students.
The instructional culture in higher performing nations differs in ways significant to the goal of creating “mathematically proficient students.” One
difference is the cultures’ tolerance of student grappling with important mathematical thinking. If students are to make sense of problems and persevere in solving them, they must access tasks that provide opportunity to do just that. Findings of the TIMSS video analysis (Stigler & Hiebert, 2007) reveal that while U.S. teachers do present students with meaningful tasks at a rate consistent with that of higher
performing countries, the outworking of the culture of U.S. classrooms, in every case, culminated in the teacher intervening in such a way as to remove students’ need to make sense and persevere. The existing culture of U.S. classrooms discourages teachers from allowing their students to grapple productively. If teachers push themselves and their students toward this goal, students, parents, and administrators may well push back, accusing the teacher of failing to teach.
6 Hiebert and Stigler (2007) aver that attempts to implement wholesale change are ineffective in creating the deep cultural shifts needed in classrooms. Incremental change, they say, is more likely to have an impact on the delivery of lessons within U.S. classrooms.
The quality of instruction that students receive affects achievement, both in the short and in the long term. Puchner (2006) relates increased levels of engagement on the part of students as teachers engaged in a process of lesson study. Achievement of students increased measurably in Hixon’s 2009 study focused on effective
professional development and its impact on student mastery of learning.
PISA data and other measures illuminate the gap between the achievement of U.S. students and that of their counterparts in some other countries. Two critical elements are necessary, though perhaps not sufficient, for teachers to master in order to support learners in changing the outcomes that PISA reports: deepening their own content knowledge, and improving instructional practice. Despite differences in the scope of content knowledge needed when considering primary and secondary school mathematics instruction, the needs are glaring in both grade-spans. Primary school teachers are often not mathematics specialists, and their experience with concepts and structures of mathematics varies with the quality and rigor of their own educational history and the ongoing professional development they receive on the job. Many second grade teachers, for example, understand the standard algorithm for multi-digit addition and teach those steps. Many students, shown the steps, fail to see the deeper connections inherent in the meaning of addition and in the significance of place
7 value. Teachers are less likely to have a robust understanding of how manipulatives might model this new, deeper mathematics that the Common Core Standards outline (McIntosh, 2012), and even more, how to support learners in using non-standard strategies like composing and decomposing numbers to add in a more conceptual way. The idea of teaching strategies as opposed to the standard algorithm is an expectation of the CCSSM (e.g. grade two, numbers and operations in base ten) for which elementary teachers might not be prepared. Secondary school teachers, on the other hand, are more likely to be content specialists due to the requirements of a single subject teaching credential, yet they may be the success stories of a system that, again, emphasized algorithms at the expense of application or conceptual understanding. In a typical secondary mathematics lesson in the U.S., the teacher presents a problem, often only requiring computation or manipulation disconnected from any real world context, solves the problem, and assigns similar problems for students to practice the computation or manipulation. The skill of solving non-routine and potentially ambiguous problems as directed by the SMP 4,
“mathematically proficient students model with mathematics,” (California
Department of Education, 2014, p. 7) is not a part of their own repertoire, let alone that of facilitating such activity on the part of their pupils. Their content knowledge, in short, may be limited to lecture and algorithms (Stigler & Hiebert, 2007). The lecture method of mathematics instruction outlined above provides a valid, but one-dimensional approach, while the need is for multi-faceted units of study to support deep conceptual understanding.
8 References to the CCSSM in this document, both the grade- or course-level standards and the Standards for Mathematical Practice, only scratch the surface of the paradigm shift that these new expectations embody. Although the law of the land in California since August, 2010, teachers have faced obstacles in their efforts to
transition from the former 1997 mathematics standards. Early implementation efforts were hampered by the lack of resources like appropriately aligned curriculum, sample items and tasks to exemplify the breadth and depth of the standards, and professional development both for teacher preparation programs and for veteran educators in the classroom. An additional impediment was the ongoing assessment which contributed to California’s compliance with the federally mandated No Child Left Behind Act, and the results of which were reported by the Department of Education and local news media. Students in 2nd through 11th grade continued to participate in the California Standards Test (CST) through the 2012-2013 school year, and schools started the 2013-2014 school year expecting to administer CSTs again. In October 2013, Governor Brown signed legislation to remove most requirements of CST administration. During this “twilight zone” from 2010 to 2014, teachers and
administrators were put in the unenviable position of choosing between teaching the new standards as adopted by the California Department of Education or preparing their students for the test they would take in the spring. The third option, of course, was to teach both.
Four years after adoption of the CCSSM, high quality resources are beginning to surface, although fully developed, aligned classroom curriculum continues to
9 lag. The expectations of the new standards sacrifice breadth of coverage for depth, with a three-fold call for focus, coherence, and rigor. Content that for decades has been taught in a given course or grade level has, in some cases, been shifted to provide a sound progression of mathematical learning K-12. The eight SMPs describe a reality of teaching and learning that is not a part of the skill set of many U.S. teachers. Teachers need time and structure in which to delve deeply into the new standards. An action research model of lesson study provides a potential structure to meet just such a need.
The purpose and structure of lesson study vary depending on the philosophy and goals of the organization. For the purposes of this project, the structure of lesson study is defined as it is currently implemented with teacher teams associated with the Stanislaus County Office of Education in Modesto, California. A teacher team, which includes a facilitator and a small group of five to seven grade-alike or course-alike teachers, engage in planning and implementing a mathematics lesson around a collaboratively determined learning objective. Before meeting for day one of the two-day cycle, teachers agree on an objective drawn from the CCSSM, attending to both the grade- or course-level content standards and the Standards for Mathematical Practice. On day one, the planning day, the group begins the day deepening their understanding of the mathematics and considering the implications for
instruction. The balance of the planning day provides time for the collaborative creation of a lesson designed to provide students with access to the mathematical concept(s) targeted in the learning objective, and culminating in student production to
10 assay whether the learning objective was met for all students. Teachers plan in detail both the prompts they will provide, and the responses they expect to get from
students. Logistics for day two, the implementation day of the lesson are determined. Which participant will teach the first lesson and to which group of students? Who will teach the second lesson after reflection based on evidence has generated revisions in the lesson? Facilitating this second lesson is a greater challenge, requiring the teacher to implement the team’s revisions with little time for practice. After an interval of at least several days to allow any additional planning or preparation needed, the team reconvenes for implementation day. One team member takes the role of teacher in the lesson they crafted together. The others take the role of
researcher, collecting data on student behaviors. These silent observers annotate what students say and do in response to the planned lesson elements. A teacher debrief elicits evidence-based statements about the impact of the lesson elements on
students. As team members consider the level of student mastery that resulted from the lesson, they revise the initial lesson to better target stated goals and student needs. This refined lesson is presented to a new group of students and again data are collected. The second debrief focuses on changes in outcomes for students associated with decisions the team made in the revision work. Final notes for further revision are gathered, and teachers reflect on their own learning about instruction and lesson design resulting from this cycle of lesson study. A typical group will complete two or three two-day cycles in an academic year.
11 The outcomes of lesson study as it rolls out in this model include teachers deepening their own understanding of mathematical concepts, calibrating a common language around instructional practice, and engaging in action research to assess the effect of instructional decisions on student learning. The CCSSM describe
“mathematically proficient students” as those who “look for and make use of structure” (California Department of Education, 2014, p. 7) in mathematics. Many educators in U.S. classrooms think of mathematics primarily as a set of rules useful for manipulating numbers and for getting answers. The foundational work on day one deepens conceptual understanding for teacher team members and sets the imperative of creating learning opportunities where students do the same. In their book Instructional Rounds, Elizabeth City and Richard Elmore (2009) describe observing a single lesson planned by a teacher team but presented to students by each teacher in isolation from the team. The student outcome data varied widely, and teachers were at a loss to account for the variance. The observers, however, saw how the lack of calibration of language around the instructional elements of the lesson created deeply different implementations of that single lesson, differences which could easily account for the gaps in student mastery. In the model of lesson study described here, calibration is inherent, and a common language emerges around key structures like student engagement, formative assessment, and the like.
Lesson study merits consideration as a model for incremental change of the culture of instruction in U.S. classrooms. A risk of the current implementation, however, is that the learning of teachers gained through this work be restricted to the
12 participating small group. Hypotheses and findings of the embedded action research are held only by the team, and lack any systematic way of being shared out with a wider audience of educators. Groups, then, work in isolation and recreate the mistakes made and rediscover lessons already learned by other teams. The current lack of any history or archive for lesson study limits its impact on the education community, leaving it falling short of achieving its full potential. The addition of a digital repository increases exponentially the likelihood that the valuable outcomes of this work be shared across school sites, districts, and beyond.
Lesson study with ongoing and wide access to created and revised lessons is a potential key to incrementally changing the culture of classrooms, as advocated by Stigler and Hiebert (2007). Teachers collaborate to plan and deliver instructional opportunities that support students’ deepening understanding of key concepts in math. The group implements the lesson and collects data. Student responses, silences, interactions, and persistence are recorded. The group reviews the data and revises and teaches the lesson to a different group of students, and again reviews student data, suggesting additional revisions for future work. Two or three lessons created by a grade-level team in a year become accessible to other teacher
groups. The following year brings opportunity to further refine existing lessons or to create a new set focused on critical areas for learning to support the depth of the CCSSM.
This project is intended to create a digital archive for lesson study. This digital repository will serve as a history upon which teachers will build,
13 incrementally, their understanding of the mathematical concepts and skills they are teaching, and their expertise in delivering instructional opportunities that support students’ learning of mathematics at deep levels. As grade-alike or course-alike teacher teams collaboratively plan, deliver, and analyze student data from a lesson, the quality of their lessons, over time, will increase.
The purpose of this project is to support teacher collaboration around quality lessons to affect student access to and mastery of the Common Core State Standards for Mathematics. The creation and maintenance of a digital archive of mathematical lessons that have undergone the scrutiny of a lesson study will give access to a wider audience both to revisit and revise existing work, and to add to the growing body of vetted lessons themselves.
Significance of the Project
One of the expectations of the CCSSM is that students build upon the learning of their colleagues. Learning in isolation from other learners, and from connections to the real world is inadequate to the needs of our future leaders. In the same way, teachers have a need to build upon, and contribute to, the learning of colleagues. An isolation model of teaching reduces the reach of learning about mathematical content and pedagogy. Lesson study begins the work of collaboration at the local level, but, as currently practiced in the U.S., fails to create an archive of learning from which a more extensive collaboration, both over time and distance, can continue.
The significance of this project is its aim to open the collaboration of lesson study teams. The creation of a “knowledge base” (Heibert, 2002) gives voice to the
14 learning of teams as they engage in the action research of lesson study. The team records its initial goals and strategies, what the data revealed about student responses, and how revisions based on this evidence impacted student learning. As a new team engages in a planning cycle, the archive delivers access to others’ research outcomes to inform the new team’s own learning.
The real benefits of this project come over a period of time, as teachers’ understanding increases, first of instructional methods of value, and secondly of the deeper constructs of K-12 mathematics concepts as they engage in contributing to and learning from this lesson study repository.
Definitions
The following short descriptions define terms of importance in the understanding of this document.
Critical Area. A conceptual focus presented within the California Common Core State Standards for Mathematics highlighting areas of significance for the understanding of mathematically proficient students at a given grade level. As the major work of the grade level, teachers and students expect to spend the majority of their instructional year working in these areas (California Department of Education, 2014).
Domain. In kindergarten through eighth grade CCSSM, the unit into which grade level content standards are divided. Domains encompass big ideas from which
clusters (the next level of hierarchy) and discrete content standards (the most
15 of the standards. In higher mathematics spanning ninth through twelfth grades, domains are components of conceptual categories that comprise the breadth of high school mathematics. As in K-8, domains are then composed of clusters and content standards (California Department of Education, 2014).
Lesson Study. Lesson study is an action research model of lesson design, implementation, data collection, and revision. This model of professional
development has been shown to contribute to a culture of collaboration and inquiry among teaching professionals.
Performance Task. A performance task is an extended assessment item delivered by the Smarter Balanced Assessment Consortium that gives opportunity for students to demonstrate their ability to apply their understanding of skills and
concepts. In contrast with multiple choice or selected response items, these tasks require students to construct responses to questions or activities organized around a real world problem or scenario, and may require multiple sessions over time for students to complete (Smarter Balanced Assessment Consortium, 2014).
Standards for Mathematical Practice. The California Department of Education has published mathematical practices in its CCCSSM. The SMPs are a set of eight “habits of mind” (2014, p. 2) that describe “mathematically proficient
students.” While the mathematical context in which the practices occur and what the practice looks like in the classroom setting vary with the age and grade level of students, the overarching habits, e.g. “construct viable arguments and critique the reasoning of others,” remain constant (2014, p. 6 - 8).
16 Summary
The critical need for professional development to support instruction in a way that student achievement increases is the clear call of this era. International
benchmark assessments as well as national and state data on K-12 achievement and mathematics readiness in colleges and universities underline this need. Lesson study has been proposed as a potential tool in the a hands of teachers, schools, and districts, a tool that aligns with what research has revealed as key components to this work of changing practice. Further, an archive of lessons and the action research produced by their implementation presents the possibility of moving this work from a small scale at a site to the larger scale of collaboration across time and location.
In Chapter II, a review of the literature will plumb the effect of professional development on teacher practice. Foundational elements from reviewed studies will then present the opportunity to compare with the elements of lesson study as it has been studied in the U.S. Connection and overlap between effective professional development and lesson study as professional development will be examined. The final chapter will describe the project and discuss the logistics of planning and design considerations. An overview of implementation will be outlined.
CHAPTER II
REVIEW OF THE LITERATURE
Effect of Professional Development on Teacher Practice
A number of research studies and articles form a foundation for understanding the effect of various models of professional development (PD) on the practice of teachers in their classrooms. A look at both exemplars and cautionary tales supports structuring this project in such a way as to increase the likelihood of affecting teacher practice.
The influence of pedagogy and belief on PD implementation was the focus of Howard’s (2009) case study. The three teacher participants were selected based on teaching experience and background. A pre-survey gathered data on teacher experience and beliefs. Participants enrolled in a week-long training designed to promote the use of inquiry as a learning model. Questions in the research study included what factors influence teachers’ experience of professional development, and in what ways those factors play a role. Also considered was the effect of PD on teachers’ beliefs and instructional practices. Data collected included documents like lesson plans, interviews, and follow-up communication via email. One participant, a second year teacher, was excited and immediately implemented structures and inquiry task types from the PD she received, but in a short time reverted to lecture, citing student resistance and lack of collegial and administrative support for the changes she was attempting. A second teacher expressed initial desire to move away from lecture
18 as her primary pedagogy, and found the training helpful, but after an initial attempt at engaging her students in an inquiry problem, she fell back into her former practice of lecture. Although both of these participants communicated interest in new teaching methods, implementation over time failed. A third participant already used the techniques forming the core of the PD and her practice continued to use the strategies with her existing tasks, rather than using the tasks provided in the PD. The author concluded that teacher experience and beliefs play a role in how PD is received, and theorized that new teachers may be more likely to implement new practices. In the absence of a supportive system, however, even the willing attempts at implementation were not sustained. In the big picture, individuals experience PD and implement or fail to implement based on personal practices and beliefs about learning.
The impact or lack of impact of professional development is well studied in the United States. Elmore and Burney (1997) discussed common problems that limit the reach of professional development and the potential for a resulting
“implementation gap” between training teachers have received and what actually gets implemented in their classrooms. What is needed, they said, is an unswerving focus on instruction as the only meaningful driver for school improvement. While
examples abound of failed improvement efforts due at times to changing leadership, changing goals, lack of systematic strategies at the site and district levels, other successful examples exist. Describing the work of a single school district in New York, they found an exemplar of single-minded leadership maintained over time, an “existence proof” for such a model. In addition to the need for long term vision and
19 focus, Elmore and Burney cited necessary components of effective PD. What the authors call “awareness” is key: that is, access to experts, text resources, and
examples of practice to further educators’ own learning about pedagogy. Opportunity and resources to plan, design and implement curriculum in an environment that supports such work, as well as the expectation of observation and feedback as
teachers try things out are important. Reflection with colleagues and experts with the goal of refining practice is the final element they called out (Elmore & Burney, 1997). This type of culture builds a shared expertise which, rather than targeting only the eager or the early adopters, builds capacity of all members of the
organization. As opposed to the typical model of isolation in teaching and learning, this model builds peer networks among teachers and administrators, with site visits and observation of practice normal across school sites and with both groups. A unique aspect called out in this work is the role of the administrative team, who not only monitors instruction, but engages in the PD with staff, teaches model lessons, and in all ways fully takes on the role of instructional leader.
The need to establish a culture of teacher inquiry through a cycle of learning, with an emphasis on student outcomes formed the core of a best evidence article from the New Zealand Ministry of Education. Motivated by a high achievement gap tied to student socioeconomic and language status, Alton-Lee (2011) addressed the difficulty of affecting instruction through PD, countered by the realization that such change is precisely what is needed to support all learners. Focused on two specific PD programs, the author synthesized five leadership dimensions on student
20 outcomes: clear goals, resources, planning and monitoring, leader participation in teacher learning, and supportive environment. In his synthesis of 17 effect sizes from seven studies, the highest effect value (0.84, SE = 0.14) was associated with leader participation. In addition, the author posed characteristics of effective PD having a strong effect on student outcomes. PD that is effective is focused on specific content and includes clarity on what knowledge will be assessed and how to assess it. In addition, PD needs to engage teachers’ own theories of practice, allowing them to connect new learning to existing knowledge, and offering the potential to revise their philosophy based on new work. Integration of old and new learning parallels the need for integration of new learning into practice, where teachers try new
implementation in their immediate context. Alton-Lee (2011) also called out the key role school leadership plays in professional inquiry into teacher practice.
In their study on effective professional development, Garet, Porter, Desimone, Birman, and Yoon (2001) used a national probability sample of 1,027 mathematics and science teachers. This teacher self-reported survey data focused on
characteristics of professional development and resulting effects on teachers’ learning. Three core features of professional development rose to the surface in the analysis of data as having significant, positive effects on teachers’ increase in knowledge and skills, and on changes in classroom practice. The first of these features was a focus on the development of deepening content knowledge and a facility of use of content within participating mathematics and science
21 the occasion to engage in active learning concomitant with the professional
development. The third core feature involved the coherence of the new learning or strategy or method with other learning activities in which the teachers were currently or previously engaged. In the presence of these features, teachers reported increased learning as well as increased implementation of that learning. Three structural features of the professional development provided significantly affected teacher learning. The form of the activity; for example, workshop as less effective and study group as more effective, the collective participation of a group of teachers from the same school, grade, or subject, and the duration of the activity all played a powerful role in bridging the implementation gap. This research (Garet et al., 2001) exposed the insufficient nature of the traditional workshop model of professional development in affecting teacher knowledge and practice, and named cost as a potentially serious roadblock to the building of professional development systems that embody the characteristics that research compels.
The effectiveness of professional development on the Correlated Science and Mathematics (CSM) model was the focus or Morlier’s (2012) mixed methods
study. In her work to support teachers in central Texas with this model, she engaged fifth- through eighth-grade teachers recruited by their principals. Eighteen teachers were assigned to the treatment group and additional teachers were recruited to form the control. Treatment teachers received a two week intensive training during the summer with follow-up training and collaboration days on Saturdays during the academic year. The training course focused on content in mathematics and science,
22 pedagogy, inquiry, active learning, cooperative learning, and committed time for collaborative lesson planning, as well as an ongoing lens on the components of CSM. Data collected included teacher, student, and principal pre and posttests as well as interviews and observations of teachers. The collected data revealed high value teachers placed on the training they had received. Of particular utility, teachers marked the student-centered, hands-on activities and discussions, and the focus on active learning. In many cases, these structures validated practices they already used. While principals were the initial contacts, had recruited their staff members to participate, and in many cases supported ongoing teacher collaboration through the school year (e.g. release time for planning), principals did not, in most cases, attend themselves, failing to support a goal of the study. Teacher content pre and posttests showed an increase in content knowledge, especially in science. The author
hypothesized that initial knowledge of the physics involved was low, and therefore more growth was realized. Administrative leadership in the work was not fully realized, and as a result, organizational change was limited to what teachers could bring about on their own. Classroom observation revealed limited implementation of correlated lessons, perhaps due to timing of the state assessment overlapping with the observation window. Student outcome effects were unclear. Timing, lack of student data, and limited direct administrator involvement all seem to have played a role in limiting the effectiveness of this professional development program.
In their experimental study on professional development, Carpenter, Fennema, Peterson, Chiang, and Loef (1989) describe a positive correlation between teacher
23 knowledge about the research on children’s thinking about mathematics and student achievement on both computation and problem solving. Forty first grade teachers in Wisconsin volunteered to participate in this study in which teachers were randomly assigned by school to a treatment or control group. Treatment teachers received four weeks of intensive training on the research on learning related to addition and
subtraction concepts in young children. Time was included to plan instruction based on that learning. Teachers and students were observed. In addition, teachers were tested on their ability to predict the success of individual students on specific tasks and that data compared with students’ actual outcomes. Observation revealed that treatment teachers spent more time on contextualized problem solving rather than on isolated computation. They posed more problems and listened to student processes and understanding more frequently than the control group, rather than listening only for answers. The resulting data supports the contention that teachers who had
knowledge of learning research changed their instructional practices, were better able to assess student understanding, and students’ performance benefitted.
In their work to refine understanding of effective professional development, Hiebert, Gallimore, and Stigler (2002) began their report with criteria and examples for affecting teacher practice. They referred to lesson study as a potential research tool leading, through incremental change, to a culture of educational research in the United States, and they included a call for lesson video to support implementation with fidelity. According to Hiebert et al., (2002), “There is a growing consensus that professional development yields the best results when it is long-term, school-based,
24 collaborative, focused on students’ learning, and linked to curricula” (p. 2). The lack of a “knowledge base” for the teaching profession in the U.S. contrasts with the typical approach to lesson study in Japan, where lesson study teams share findings from their action research in reports at the conclusion of a given cycle. This body of work forms a library from which teachers pull expert thinking and the outcomes of previous studies to inform their own lesson planning. Hiebert et al. (2002) see the creation and evolution of such a resource as key in attaining to the goal of improving classroom teaching in a steady, lasting way. The current perceived dichotomy between educational research and educational practice leads some researchers to theorize that research is not accessible to classroom teachers, or others to posit a disconnect between the controlled and generalizable framework in which research occurs as opposed to the “craft” knowledge teachers carry, knowledge that is practical and deeply connected to the context of their specific classroom. Such a disconnect, perhaps, prevents teachers from valuing research, perceiving it as too remote from their work, and from the rich knowledge they have accumulated over their teaching careers. Hiebert et al. (2002) propose that professional knowledge must be
public. It must be represented in a form that facilitates the accumulation and sharing of this knowledge with other members of the profession. And it must be continually verified and improved upon. The authors call for lesson study protocols to be implemented widely, and for an ongoing accumulation and sharing of knowledge.
25 Lesson Study as Professional Development
Motivated by internationally published TIMSS study’s reporting of the low mathematics performance of U.S. students, Taylor and Puchner (2002) began work on lesson study with a group of volunteer teachers in southern Illinois. Short term grant funding provided 17 local teachers with the resources to participate in lesson study in the area of mathematics. Four groups of teachers engaged in one cycle, planning and teaching lessons in 1st, 2nd, and 4th grade classrooms. The lesson objectives were centered, in each case, around a challenging problem or task for students to
solve. Mathematics specialists from district or county offices or from regional universities supported the work of the teams. Self-reported qualitative data revealed some significant changes in teacher philosophy over the course of the work. Initial suspicion on the part of the teachers focused on the ability of the team to collaborate effectively and a perception that the time allocated to planning a lesson was too long. Participants reported surprise at the realization that the first half day of collaborative planning, rather than quickly producing a lesson outline, engendered discussion about the mathematics concepts. Teachers reported “talking math” and found the
conversations strengthened and expanded their own conceptual
understanding. Lesson planning, in this context, became focused on depth and was viewed as a learning opportunity for teachers. A second shift teachers reported was away from right answers as the only goal of the lesson, and toward student
understanding of concepts and procedures. Finally, rather than planning teacher actions as the primary focus of writing a lesson, the “research lens” was directed
26 toward curiosity about children’s mathematical thinking. The benefits that
participating teachers highlighted after their first cycle of lesson study included the creation of an effective lesson that they felt worthy of using again in the future, a change in professional culture among the group moving toward collaboration, an increase in motivation and excitement about teaching and a feeling of
professionalism, and a deepening of mathematical understanding on the part of participating teachers.
In a follow-up study, the authors described the “how to” of the initial teacher groups’ experienced lesson study and the impact of that work on teacher efficacy and the tension between autonomy and collaboration (Puchner & Taylor, 2006). Teachers in this ongoing work were stipended over a period of six months. Most group
members had little or no prior experience with lesson study. Researchers used classroom observation, interviews, and collection of documents to inform their study. Early work generated frustration, but persistence and revisions led to reteaching of lessons. Later outcomes included an increase in collaboration and in enjoyment of the process of lesson study, with conversations about mathematics spilling into unplanned meetings in the hallways. Teachers were pleasantly surprised to witness a positive impact on student engagement resulting from the collaborative work, generating a high level of teacher efficacy. This efficacy was characterized by a growth mindset on the part of teachers about their own ability to develop and deliver effective lessons for student learning.
27 In her doctoral dissertation, Gail Siragusa Yamnitzky (2010) studied the connection between lesson study and effective professional development practices, specifically looking for the perceived impact the work had on instruction, teacher content knowledge, teacher knowledge of pedagogy, and teacher understanding of students’ mathematical thinking, all elements called out in the research as key
outcomes of effective professional development. Additionally, she sought to measure the degree to which lesson study as used by participants matched the research-based definition of effective PD. Yamnitzky found, in keeping with other researchers, that common barriers to lesson study implementation included, “lack of shared long-term goals across staffs, lack of curricular coherence, lack of strong content knowledge, teacher isolation and the lack of shared planning time” (p. 9). Her study included 129 teachers in western Pennsylvania who had experienced lesson study as part of their PD work. Respondents to her survey overwhelmingly reported their view that “lesson study is an effective way to continue my professional development” (p. 97). Moreover, those who had participated longer in lesson study agreed even more strongly with that statement. In the area of content knowledge, teachers reported increased confidence in their own understanding of the mathematical concepts, a better grasp of the cognitive load involved in specific mathematics tasks, and a greater sense of preparation to use and teach the skills and concepts involved in the study. In the area of pedagogy around the effective teaching of elementary
mathematics, teachers reported a strong increase in ability, as evidenced by
28 develop student conceptual understanding, to elicit student preconceptions and
misconceptions, and to support investigation and problem solving, undergirded by conjectures and hypotheses. Interestingly, the two major hindrances to effective implementation of lesson study as expressed by participants were time (both
scheduling and amount of time, expressed by 75% of teachers) and size and makeup of the group (concerns of 34.9% of respondents). In measuring alignment of the work of lesson study to research-based elements of effective PD, teachers reported
collaboration, focus on content, increased use of new teaching skills, engagement in their own learning, and extended work over time.
Reporting on a group of eight intermediate grades teachers from six schools in Florida, Jennifer Dubin (2010) encountered a group who for three years had
collaborated on lesson study. This early group then split to create and facilitate lesson study teams at their respective school sites. In the lesson reported in this work, teachers tailored their lesson to engage kids in a context with which they would readily connect, a Guitar Hero competition among their teachers. They carefully selected initial data values from the (fictitious) competition where both correct and incorrect representations would lead to the correct answer. Next they posed a “what if” example where incorrect reasoning would fail and had students grapple with why it didn’t work. Additional scenarios reinforced the correct reasoning and allowed a full understanding of both the correct and incorrect reasoning. Support for their planning included a university mathematics professor who supported them via email as they pursued the project, reviewing their findings and providing suggested
29 revisions. Dubin (2010) concludes that despite an existing culture of isolation in U.S. education, lesson study provided an effective framework for collaboration on content and student learning, as well as for supporting the powerful learning of the team.
In their qualitative study, Chong and Kong (2012) researched a link between the effective professional development elements described above and the elements of lesson study. Ten volunteer teachers spanning the content areas of science,
humanities, and mathematics, and all teachers at a gender based girls’ high school in Singapore, engaged in a seven week period of lesson study. Though experienced in using specific cooperative learning approaches, none of the participants had been involved in lesson study. Grouped by content, each team worked with a senior teacher in their content area, also a colleague at the school. The senior teacher
received training in lesson study protocols and had content expertise. Over the course of the study, teams, facilitated by their senior teacher, met to plan, implement, revise, and implement a second lesson. Planning and reflection meetings were taped and analyzed, as were observation field notes. Participants created written reflections after each session and were interviewed by their senior teacher using a protocol at the culmination of the cycle. Qualitative analysis brought out themes of increased content knowledge resulting from opportunities to collaborate and the act of building on the contributions of the team. Joint ownership of the lesson was associated with increased confidence in trying new methods with students, and an overall increased sense of teacher efficacy. Teacher beliefs were honored and provided the starting point for the planning. The composition of the teams, with years of experience
30 varying from 2 to 40 years, brought the strengths of both groups to the
forefront: experience and innovation. Alongside this willingness of the presenting teacher to risk new practices and be observed trying them out was a lowered affective filter, a confidence that constructive criticisms were about the (jointly owned) lesson, not the individual. In every case, teachers observed a heightened sense of students’ engagement in the lesson study lesson, which in turn increased teachers’ sense of efficacy in their ability to plan engaging lessons. The researchers concluded that lesson study does foster the processes that evidence suggests support increasing teacher efficacy, which in turn has been shown to enhance student achievement and sustained teacher behaviors.
In his grounded theory study, Michael Hixon (2009) considered lesson study as a structure to develop professional learning communities and collaboration. He looked at the overlap between lesson study and theories of effective PD. His research questions focused on the effect of the lesson study process on curriculum, teacher learning, and student outcomes. Participants were from a rural elementary school in the Central Valley of California and included six third-grade teachers, two special educators, and two administrators from the same school site, as well as 20
students. The academic focus was on language arts. Interviews, observations, a focus group, and artifacts from student portfolios formed the data analyzed. Analysis revealed that lesson study increased collaboration among teachers through structured time and protocol, a focused goal, and reflection on practice and student outcomes. A further outcome was a cultural shift among participants toward teamwork and a sense
31 of reflective professionalism. Despite existing implementation of peer coaching and structured teacher planning time, the school district involved was missing a key component. Teachers taught in isolation without authentic partnerships. The purpose of the study was to explore the influences of colleagues through lesson study as an additional element to coaching and structured planning. The participants found lesson study promotes teacher learning as well as focusing instructional strategies and increasing student achievement. Implementation of new learning in participants’ own classrooms occurred after the collaborative lesson was implemented and observed. This collaboration engendered a sense of shared success. One result of the initial implementation was recognition of the need for behavior management structures, which teachers observed significantly improving students’ access to the lesson on the subsequent teaching rounds. Besides increased engagement resulting from these structures, participating teachers and the researcher found increased quality in student writing over the course of the lesson study period. Planning focus changed from teaching an individual standard to a focus on what, how, and why students are learning. Participating teachers, while valuing the process and benefits accruing, were concerned about the sustainability of lesson study at the conclusion of the research study, especially with the constraints of time and expense needed for collaboration. Global outcomes for the participants from lesson study were more reflective teaching and an increased culture of collaboration and support. Students benefited from greater focus and engagement in lessons and increased writing scores.
32 Summary
The existing literature on effective professional development calls out specific elements that correlate with teacher efficacy, deepening of teachers’ content
knowledge, teacher implementation and persistence in pedagogical practices, and student achievement. This review of literature found a research base supporting the use of lesson study as an effective protocol in affecting these outcomes. In addition to documenting overlap between PD that has an impact on student learning and the protocol of lesson study, a need for supporting structures to enable collaboration and incremental change is clear. This project aimed to provide such a structure.
CHAPTER III
DESCRIPTION OF THE PROJECT
Overview
The development of this project is tied to the protocol of lesson study implemented by the Stanislaus County Office of Education beginning in 2009. The outcomes of the two-day cycle of action research were powerful, and repeating the cycle multiple times over the course of a school year gave teachers access to a building of experience with the CCSS, the conceptual understanding needed to support learning of the mathematics, and a building of expertise about what works in classrooms. In almost every case, however, the work of the team remained with the team. Teachers’ findings were built over their own professional development year, but each team was, by definition, starting at ground zero, and the ability to construct learning on the foundations of other teams was missing.
Logistics and Design Considerations
In considering a remedy to this situation, the need for an archive of resources was obvious. In Japan, as one example, teachers publish findings from their lesson study work and schools maintain large print libraries of bound reports generated from lesson studies. The refining work of lesson study builds over years, and becomes an encyclopedia for pacing, lesson planning, and teaching key concepts. The logistics of creating a physical repository of lessons and sharing them over even local agencies,
34 let alone further afield, would be onerous. An early consideration was the required capacities of a digital platform. Ideally, the archive would be searchable on multiple criteria, e.g. grade level, content standards, mathematics practice. Lessons would be editable by multiple authors or author teams. Resources in the archives would include multiple file types to allow for teacher presentations, student worksheets, work samples, video of lesson segments, etc. The archive would support
commentary, enabling teachers to annotate their findings as the revision work progressed, the reasoning behind changes made, images of student work, and the like. A history of the refinements of the lesson should allow a linear view of changes over time, as well as the opportunity for branching, where different teams take the same initial lesson in alternative directions. The branch lessons would maintain a connection back to the original, allowing a later audience to see the big picture of the progression of the work. Security concerns include prevention of unauthorized editing or deletion of resources, as well as privacy of student information. The structure of a summary report for each lesson study team’s final product will need to facilitate the learning of new teams who choose to access it.
Factors in Implementation
Free document management systems (DMS) like Dropbox, Google Drive, as well as fee-based services were considered. Finding a single product that met the most critical of the ideal conditions as well as minimizing cost was the goal. Some of the requirements of the project were accomplished through structures layered onto the selected DMS, like a naming structure to maintain the user’s view of connections
35 between lessons as they are revised. Other elements, like the ability to annotate at the point of use were non-negotiable for the project to work as intended, and in the end played a decisive role in selection. An early approach to archiving, based on the print work of lesson study done in the past, was to maintain reflections and revisions in a separate document. This model, however, would have created an extra layer of complexity for users, a feature that might have discouraged busy teachers from using the resource to best effect. A better approach was selected: to host a final report to serve in the same way as the abstract of a research project.
Another key component was the ability to index lessons and search on criteria. A model similar to one used by the author to index Smarter Balanced Assessment Consortium sample tasks proved useful. In this model, a spreadsheet houses a list of resources with additional search tags. The user designates search tags to hide data he or she isn’t interested in, and a link within the spreadsheet redirects to the actual resource. In the case of this project, the link connects the user to a folder of a particular lesson’s media. Search tags to facilitate teachers’ use of the library include grade level, critical area, domain, content standard(s), SMP(s), author
names/anonymous, teacher rating, target instructional level (presenting new learning, developing learning, demonstrating mastery), and whether the lesson engages
students in a performance task.
Overview of the Timeline and User Access to the Project
The first task in creating the archive was to digitize a small number of existing lesson study folders. Creation of the index ultimately took the form of a spreadsheet,
36 thereby providing opportunity to define the search tags and list possible tags for each field. An initial design required the team lead who submitted the study to key in naming structures to facilitate sorting; for example aligning with a critical area would have required the user to enter a structure like 2.1 to indicate grade two and critical area one. Maintaining a uniform naming structure for search tags would have simplified searches for end users. Concerns about accuracy of user naming led to a revision of this protocol and instead, the user selects from a menu to tag a lesson as aligned with a given critical area or standard. The act of tagging the initial lesson studies suggested additional tags to include. At this point, initial testing of the indexing system is taking place, with volunteer colleagues vetting the search process. As refinements continue, additional lessons studies are occurring in the county. The project author will add new content as it develops during this phase. The final design consideration was the creation of an end-user method to submit lesson study documents and artifacts. The method put in place for this piece was a Google form where a team leader provides information to support inclusion of the study in the lesson study library and selects search tags that describe it. Team leaders access instructions on lesson submission. Lessons undergo an evaluation for inclusion by the project author. The author creates a name and links new versions with previous versions of the same lesson. Published lesson studies are available as read-only files. Team leaders are required to provide their name and contact information for future work, and to receive information about new lesson studies as they are created. The project author will continue to manage the distribution list for
37 email contact, and provide support for users, perhaps using Google resources, or continuing PLCs established by SCOE using Edmodo. An advantage of Edmodo or a similar platform is the social networking aspect and resulting advertising for this work. Currently in the 2014-15 school year, a dozenteams are contracted to work with SCOE in a three-study cycle, a small enough group for the author to manage in this building phase of the project. Input from users on design considerations for the archive will guide refinements to the structure. During the 2015-2016 school year, the pilot groups will continue developing lesson studies under the aegis of their districts, and new groups will be in place as districts develop their own instructional coaches and teacher leaders to lead lesson study teams of their own.
The reader will find information detailing access to a website created to host the lesson study library in Appendix A of this document. This appendix provides links to the site and to resources within it, and gives guidance as to the use of the filtering mechanism within Google sheets by which users can quickly locate lessons of interest to them. The site also connects with a form for lesson study submissions and with instructions for submission. A small but growing number of resources for beginning lesson study teams is provided as well.
Evaluation
As the work grows, teacher ratings in categories like student attainment of the learning objective, both by the initial team and by subsequent ones, may prove beneficial to support teachers in accessing quality lessons. In the long view, this archive will provide an additional resource for that first learning phase of lesson
38 study. That is, after agreeing on a learning objective, teachers will locate studies and before beginning their own planning, benefit both from the content knowledge and the pedagogy illuminated by the work of previous teams.
The outcome of this project is a digital repository for lesson study
content. The initial users are teachers and instructional coaches within Stanislaus County who are engaged in the work of lesson study as part of their district’s plan. First use is the creation of lessons and lesson resources within the archives by these grade level teams. A specific lesson is assigned a unique name and later revisions will retain this name with additional markers to signal the relationship to previous and subsequent work. Over the course of a year and over subsequent years, the archive will be enriched with additional content as teams complete multiple lesson study rounds. If teachers find this source to be beneficial, the scope could easily expand beyond Stanislaus County. The CCSSM are national standards, and for the first time, teachers can effectively collaborate across state lines.
40
REFERENCES
Alton-Lee, A. (2011). (Using) Evidence for educational improvement. Cambridge
Journal of Education, 41(3), 303-329.
California Department of Education. (2014). Common core state standards:
Mathematics. Sacramento, CA: California Department of Education.
Carpenter, T., Fennema, E. Peterson, P. L., Chiang, C. & Loef, M. (1989). Using knowledge of children’s mathematics thinking in classroom teaching: An experimental study. American Educational Research Journal, 26(4), 499 - 531.
Chong, W., & Kong, C. (2012). Teacher collaborative learning and teacher self-efficacy: The case of lesson study. Journal of Experimental Education, 80(3), 263-283.
City, E., Elmore, R., Fiarman, S., & Teitel, L. (2009). Instructional rounds in
education: A network approach to improving teaching and learning. Cambridge, MA: Harvard Education Press.
Dubin, J. (2010). American teachers embrace the Japanese art of lesson study. Education Digest, 75(6), 23-29.
Elmore, R. F. & Burney, D. (1997). Investing in teacher learning: Staff development
and instructional improvement in community school district #2, New York City. New York, NY: National Commission on Teaching & America's
41 Education Data Partnership. (2014). Adequate yearly progress under NCLB.
Retrieved from
https://www.ed-data.k12.ca.us/Pages/UnderstandingTheAYP.aspx
Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915. Retrieved from http://ezproxy.lib.csustan.edu:2048/login?url=http://search.proquest.com/docv iew/200450497?accountid=10364
Hiebert, J., Gallimore, R., & Stigler, J. (2002). A knowledge base for the teaching profession: What would it look like and how can we get one? American
Educational Research Association, 31(5), 3-15.
Hixon, M. L. (2009). Lesson study: A proposed intervention for professional
development and student achievement. (Doctoral dissertation). Retrieved from
ProQuest Dissertations and Theses. (305079666).
Howard, C. (2009). Influencing pedagogy and belief: A case study in the professional development of teachers of mathematics. Retrieved from ERIC
http://files.eric.ed.gov.ezproxy.lib.csustan.edu:2048/fulltext/ED539456.pdf. (ED539456)
Huebner, T., & Corbett, G. (2008). Rethinking high school: Supporting all students to
be college ready in math. WestEd. Retrieved from
42 Intersegmental Committee of the Academic Senates of the California Community
Colleges, the California State University, and the University of
California. (2013). Statement on competencies in mathematics expected of
entering college students. Sacramento, CA. Retrieved from
http://icas-ca.org/Websites/icasca/images/ICAS-Statement-Math-Competencies-2013.pdf McCormick, N. & Lucas, M. (2011). Exploring mathematics college readiness in the
United States. Current Issues in Education, 14(1). Retrieved from http://cie.asu.edu/ojs/index.php/cieatasu/article/view/680
McIntosh, G. (2012). Testing instrumentation validity for measuring teachers’ attitudes toward manipulative use in the elementary classroom. Retrieved from Education Resources Information Center. (ED537025)
Morlier, R. (2012, May 1). Evaluation of the correlated science and mathematics professional development model, 2009-2010 Cohort. Online
Submission. (ED531891)
National Center for Education Statistics. (2013). The nation’s report card: A first
look: 2013 mathematics and reading. Washington D.C.: U. S. Department of
Education. Retrieved from
http://nces.ed.gov/nationsreportcard/subject/publications/main2013/pdf/20144 51.pdf
National Center for Education Statistics: Institute of Educational Sciences. (2012). Digest of Education Statistics. Washington D.C: U.S. Department of
