This section provides a typology for three types of student assessments. Their purposes, characteristics, and summaries of the research on their utility for informing teaching and learning are discussed. The research reviewed in this chapter focuses mainly on efforts to implement interim assessments and other data-based programs. However, to promote the use of student assessment data for instructional decision making, several programs also include related supports such as professional development, coaching, and guides and resources. The research on these supports is reviewed in brief.
Interim Assessments
Perie, Marion, and Gong (2009) provide a useful framework for distinguishing among assessment types, and for defining and evaluating interim assessment programs, specifically. They organize assessments into three main categories: summative, interim, and formative. Two criteria are used to distinguish among assessment types: (1) the scope (e.g., coverage, purpose), and (2) the frequency of administration (figure 2.1).
33
Summative assessments are the broadest in curricular scope, but the least frequent in administration. As part of an accountability system, the results are used to inform policy, and to determine rewards and sanctions at the classroom (i.e., teacher), school, and district levels (Perie, Marion, & Gong, 2009). However, summative assessments have limited instructional use. Results are received too late, too infrequently, and are typically not granular enough to provide teachers with the type of data needed to inform their instruction (Dembosky, et al., 2005; Stecher & Hamilton, 2006; Marsh, Pane, & Hamilton, 2006; Supovitz, 2009).
At the other end of the spectrum are formative assessments, the narrowest type in terms of scope. Formative assessments can vary widely, but they are characterized by a short assessment cycle (i.e., frequent assessments) and are often embedded within the current lesson or unit of instruction (Perie, Marion, & Gong, 2009). Their purpose is to inform teachers of students’ mastery of skills related to only one or several content standards; diagnosing student learning, gaps in understanding, and often misconceptions (Perie, Marion, & Gong, 2009). However, they are typically not standardized for
comparison across classrooms, grades, or schools.
One of the key inputs of the ANet program is interim assessments which fall between formative and summative assessments on the continuum. These assessments are also referred to as benchmark, predictive, diagnostic, or, in some cases, even formative assessments. Interim assessments that serve an instructional purpose tend to be most similar to formative assessments, but with a longer assessment cycle and greater coverage of content standards. They are “administered during instruction to evaluate students’
knowledge and skills relative to a specific set of academic goals in order to inform policymaker or educator decisions at the classroom, school, or district level” (Perie, Marion, & Gong, 2009, p. 6). These assessments are often standardized for comparison across schools and built around a bank of items aligned to standards and curriculum. Results are reported quickly and often disaggregated by student and standard. When part of an assessment system, the assessments are often paired with support for interpreting the results and making decisions about instructional interventions.
Other Program Components
Prior research contends that the effectiveness of educational reforms in general, and interim assessment programs in particular, is dependent on leaders and teachers having the necessary skills and knowledge to properly implement such programs (Borko, Mayfield, Marion, Flexer, & Cumbo, 1997; Christman, et al., 2009; Blanc, et al., 2010). Furthermore,
“[w]hile Benchmarks may be helpful, they are not in themselves sufficient to bring about increases in achievement without a community of school leaders and faculty who are willing and able to be both teachers and learners.” (Christman et al., 2009, p. 44)
Data-based instructional programs vary widely in respect to the types of support and resources offered to teachers. In her review of data-based interventions, Marsh (2012) found evidence that multiple, linked supports may be necessary to support effective data use: e.g., data systems or tools that are supported through professional development. Unfortunately, the existing research on these supports tends to be observational and
35
focused on data-use strategies specific to one program, reform effort, or district, making it difficult to connect specific supports to teacher outcomes.
Observational research has shown that the content and reported usefulness of data-based professional development (PD) varies substantially (Marsh, Pane, &
Hamilton, 2006; Means, Padilla, DeBarger, & Bakia, 2009; Stecher & Hamilton, 2006). Even when satisfied with the PD they have been offered (Means et al., 2009), some studies concluded that teachers were not offered enough content around the effective use of data or data systems (Mason, 2002; Dembosky, et al., 2005; Clune & White, 2008; Means et al., 2009; Means, Padilla, & Gallagher, 2010). Support at various points in the instructional data use cycle also appears to be lacking. In some cases, support was
insufficient during implementation (Jacobs, Gregory, Hoppey, & Yendol-Hoppey, 2009). In others, initial support on data systems access and operations was provided, but failed to assist teachers’ analysis, interpretation, and use of data (Means, Padilla, & Gallagher, 2010; Jimerson & Wayman, 2011). Particularly lacking from PD is content to help teachers bridge the gap between interpreting student assessment data and making appropriate instructional decisions (Clune & White, 2008; Goertz, Oláh, & Riggan, 2009a).
Despite extensive literature suggesting best practices for professional
development processes and content,3 there have been few studies that link specific PD
models with teachers’ use of data, instructional strategies, or student achievement. One
3 See Borko, et al., 1997; Cohen & Hill, 2000; Garet, Porter, Desimone, Birman, & Yoon, 2001; Desimone,
Porter, Garet, Yoon, & Birman, 2002; Lee & Wiliam, 2005; Wayman, 2005; Young, 2006; Goertz, Oláh, & Riggan, 2009b; Young & Kim, 2010; Jimerson & Wayman, 2011.
such study utilized a school-randomized design to explore the effects of the Classroom Assessment for Student Learning (CASL) program, a “self-executing” professional development program aimed at improving teachers’ knowledge and practices around classroom and formative assessments through learning teams (Randel, et al., 2011, p. 11). After two years, no detectable difference was found in the quality of teachers’ classroom assessment practices or in students’ math achievement. However, they did find
significant differences in teachers’ knowledge of classroom assessment, with intervention teachers answering about 2.78 more items correctly (0.42 standard deviations, p = 0.01) (Randel, et al., 2011).
Several observational studies have examined the relationship between PD and teachers’ data-based beliefs and practices. Chen, Heritage, and Lee (2005) found that training on the usage of a particular data system was related to improvements in educators’ perceived value of student data, as well as increases in their collection, analysis, and use of data for understanding student learning. Case study research in three urban, high-need districts found a similar positive relationship between support provided and teachers’ instructional data use (Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006). Means, Padilla, and Gallagher (2010) found a moderate, positive correlation between teachers’ perceptions of support for data use and the frequency they used data in various ways (r = 0.40).
Prior research cites the importance of coaching as a specific form of PD (Lachat & Smith, 2005; Denton, Swanson, & Mathes, 2007; Marsh, McCombs, & Martorell, 2010). Coaching models can vary widely, but typically include coaches who are experts
37
in one content area, instructional model, or skill (e.g., literacy, math, data use), and provide in-person support one-on-one or to small groups of teachers. Means, Padilla, and Gallagher (2010) found that 32 percent of the districts they surveyed reported providing some type of data coaching to all schools and 26 percent of districts reported that instructional coaches were required to include elements of data use in the support they provided to teachers in all schools.
As with other forms of professional support, there is a recognized need for
experimental studies that examine the impacts of coaching on various teacher and student outcomes, and identify the particularly effective models.4 Observational evidence has
shown that the frequency of coaching around data use is associated with both teachers’ self-reported changes in instructional practice and with teachers’ likelihood of attributing instructional changes to coaching (Marsh, McCombs, & Martorell, 2010). The frequency with which coaches review data with teachers has also been associated with a small, but positive and significant increase in student reading and math achievement (Marsh, McCombs, & Martorell, 2010). Effective coaches focused on teachers’ specific needs, modeled data use, observed teachers during the data-use cycle, provided feedback and expertise, supported dialogue and questions around data and instruction, and helped bridge the gap between data and instruction (Huguet, Marsh, & Farrell, 2014).
4 The author found no experimental studies of data/instructional coaching on teacher or student outcomes.
However, two RCTs offer more general support of coaching on teacher and student outcomes. Blank, Smithson, Porter, Nunnaley, and Osthoff (2006) found evidence of a positive impact of an instructional improvement professional development model on middle-school math and science teachers’ alignment of instruction with standards. Campbell and Malkus (2011) found that, over three years, math coaching had a positive impact on student achievement in grades 3 through 5.
The research on supplemental tools and resources to support teachers’ data use and instructional practices is extremely limited. These tools often include instructional materials, model lesson plans, curriculum frameworks and guides linked to the interim assessment, and protocols for organizing and analyzing student data, and developing instructional plans. Means, et al. (2009) report that only three of their ten case study districts provided tools as part of their data systems. Goertz, Oláh, & Riggan (2009a) found that the districts in their study used a data analysis protocol to set and reinforce expectations for the analysis and use of interim assessment data. Both teachers and leaders were required to complete the protocol for their respective roles. Leaders also reviewed teachers’ protocols during grade-level team meetings, often inserting a level of accountability by asking for evidence that the reteaching plan captured in the protocol actually took place (Datnow, Park, & Wohlstetter, 2007; Goertz, Oláh, & Riggan, 2009a). When part of teachers’ instructional communities, tools – such as score reports,
curriculum guides, and lesson plans – can provide a starting point for conversations about student performance, as well as structure and routine around data-based practices such as instructional planning and practices (Brunner, et al., 2005; Blanc, et al., 2010; Turner & Coburn, 2012). Datnow, Park, and Wohlstetter (2007) found that protocols helped teachers and principals interpret student data correctly, make appropriate instructional plans based on the data, and ensure follow through on reteaching.
39
RESEARCH ON INTERIM ASSESSMENTS AND OTHER DATA-BASED