Race to the Top Performance Incentives
in North Carolina
A Summative Report
Authors:
Douglas Lee Lauen and Karen Phelan Kozlowski
Carolina Institute for Public Policy, The University of North Carolina at Chapel Hill
August 2014 Consortium for Educational Research and Evaluation– North Carolina
Table of Contents
Executive Summary ... 2
Introduction ... 5
Performance Pay ... 5
North Carolina RttT Performance Incentive ... 7
Bonus-Awarded Schools ... 10
Summary of Findings from the First Report ... 13
Purpose of this Report ... 13
Data & Methods ... 15
Sample ... 15
Interviews ... 15
Data Analysis ... 15
Results ... 16
Changes in Teacher and Principal Understanding of the Program ... 16
Awareness of Program Changes ... 18
Explanations for Changes in Awareness ... 19
Program Saturation and Confusion ...19
Communication about the RttT Performance Incentive ...19
Changes in Perceptions and Role of RttT Performance Pay in North Carolina Schools ... 21
Perceptions of Disconnects between Performance Incentives and Targeted Outcomes ...22
Perceptions of Connections between Performance Incentives and Targeted Outcomes: High School Level ...27
Conclusion ... 30
Summary ... 30
How Teachers Become More Effective ... 30
Recommendations ... 31
RACE TO THE TOP PERFORMANCE INCENTIVES IN NORTH CAROLINA: A SUMMATIVE REPORT
Executive Summary
Purpose of this Report
North Carolina’s Race to the Top (RttT) program includes several initiatives designed to improve educator effectiveness and retain effective educators, all with a goal of improving student performance, particularly in the state’s lowest-performing schools. The purpose of this report is to provide a summative assessment of the RttT performance incentive. In its first two years, the incentive was a $1,500 bonus available to all certified staff in designated schools that met high growth. Beginning in 2012, a $500 individual-level incentive in addition to the $1,500 school-wide bonus was made available (and will continue to be available for incentives earned through the 2013-14 school year) to teachers of tested subjects whose classrooms exceed expected growth, regardless of school-wide performance.
Data, Sample, and Methods
Data from the report come from qualitative interviews and focus groups from seven bonus-eligible schools of various levels (elementary, middle, high) and award statuses (never awarded, awarded once, awarded twice). A total of 67 participants were interviewed in order to understand how teachers and administrators perceive and respond to the program.
Key Findings from the Previous Evaluation Report1
1. Bonus-eligible schools had disproportionately higher percentages of poor and minority students compared to North Carolina schools on average, and there was little difference in these characteristics between bonus-awarded and eligible but non-awarded schools.
2. There was little to no awareness of the performance incentive initiative in the bonus-eligible schools that were interviewed in the spring of 2012. Even among schools that received bonus pay, there was confusion about eligibility in future years.
3. Student performance and growth increased between 2010 (prior to the incentive) and 2012 for both bonus-awarded and bonus-eligible schools. The lack of awareness of the program suggests that the performance incentive had little to do with this improvement.
4. Teachers generally reported that, even if they had known about the performance incentive, it would not have changed their teaching practices, although bonus-awarded teachers
appreciated the money after having received it. The perception that a performance incentive would not change their teaching practice was true of teachers and administrators at all school levels (elementary, middle, and high) and of all bonus status types (awarded and
non-awarded).
5. Finally, when teachers were asked whether they supported a school-wide or individual bonus structure, 75 percent reported favoring a school-wide bonus and 25 percent reported favoring an individual bonus structure. Those who favored individual incentives tended to work in school environments that were less collaborative than those who favored the school-wide bonuses.
Key Findings from this Final Summative Report
1. Awareness of the program in general has increased, though after two years of eligibility, there is still some confusion among teachers about eligibility and terms of the program. Confusion appeared to be highest in schools that were saturated with other grants and programs. Awareness was highest in schools with strong communication and relationships with North Carolina Department of Public Instruction Instructional and School
Transformation Coaches.
2. Very few teachers and administrators were aware of the additional classroom-level bonus available to tested-area teachers.
3. As before, a majority of teachers (73%) said that the performance incentive did not or would not play any role in their teaching because they claimed already to be working as hard as possible for their students. However, some of these teachers did recognize that performance incentives may play a role in others’ teaching, which is consistent with the findings of the preliminary report. Approximately 24% of the sample believed that the performance incentive played some minor role, such as “extra motivation,” in either their own or their colleagues’ practice. Approximately 22% of the teachers the team interviewed reported a larger role of the performance incentive in their own or colleagues’ practice, such as extra tutoring or pulling low achievers out of elective classes to focus on tested content.
4. Aside from a few teachers, almost all elementary participants reported the performance incentive to have virtually no tangible impact on either their own or their colleagues’
teaching practice. Middle and high school teachers were more mixed, with just under 60% of teachers reporting that the incentive had affected or would affect teaching in some way at their school (though not always in a positive way).
5. When teachers reported significant improvements to either their own or their colleagues’ practice, they often attributed those changes to learning coaches, professional development and training, and collaboration and teamwork—not to the presence of the incentive.
Implications
The RttT performance incentive itself likely has had limited impact on teacher improvement and student growth. Despite growing awareness of the program, few teachers and administrators could communicate the details of the program, which suggests that the performance incentive probably was not a primary motivator for most teachers. If performance pay continues to be considered for teachers in North Carolina, careful attention to clear, concise, and consistent communication about the program must be a priority in order for the policy to be effective. Most of the teachers interviewed believed that a performance incentive had not played or would not play any role in adjusting teaching practices; only about 22 percent (almost all of whom were
high school or middle school teachers) reported incentive-driven changes to classroom teaching practices or to broader school structures with a goal of maximizing students’ test scores. In addition, of those who reported changes, not all were positive changes. For example, 20 percent of those who changed their practices reported changes like a decrease in their willingness to help their colleagues.
These findings suggest that performance incentives alone may not be the best strategy for increasing the effectiveness of teachers or the quality of schooling for low-performing students and schools. Instead, policy makers may consider directing more resources to recruiting highly-effective administrators and providing learning coaches and training opportunities—strategies that teachers and administrators from this study report as having improved the quality of teaching and learning in their schools.
Introduction
Performance Pay
As Taylor et al. (1991) note, “the principle of merit pay is deeply rooted in the American way of differentiating compensation increments based on varying levels of output or productivity.” Merit pay philosophies assume that there is little incentive, besides compensation, to do good work. An idea originating from industry, merit pay programs have been applied to the education sector for several decades and continue to be incorporated into school accountability programs today (Podgursky & Springer, 2007).
In 1996, North Carolina instituted an incentive program based on its ABCs (Accountability, teaching the Basics, and local Control) accountability model. This accountability program set growth2 and performance standards based primarily on End-of-Grade (EOG) and End-of-Course (EOC) test results for each public school in the state. If a school met “expected growth” based on what students in the school were predicted to score, all certified teachers, principals, and staff received a $750 bonus. If the school met or exceeded “high growth,” all certified staff received an additional $750 bonus, for a total of $1,500.
Though monetary bonuses were discontinued in 2009, bonuses were reinstated as a part of the Race to the Top (RttT) program for the lowest-performing schools in the state. This incentive aims to increase the performance of North Carolina’s lowest-performing schools (a) by inspiring teachers in those schools to increase their effort and/or change their teaching practices for
maximal effectiveness; and (b) by encouraging effective teachers to remain in those schools. Some studies have found positive effects of merit pay on student and teacher outcomes. Schools and districts that use performance pay tend to have higher student performance (Figlio & Kenny, 2007; Woessmann, 2011; Goldhaber & Walch, 2012), pay teachers higher incomes than teachers who do not teach in pay-for-performance districts (Belfield & Heywood, 2008; Goldhaber, 2008), and elicit greater work effort from teachers (Ahn, 2008). However, some of these studies are limited by data or methods that make causal claims untenable (Figlio & Kenny, 2007; Belfield & Heywood, 2008).
However, the highest-quality studies conducted to date suggest that traditional pay-for-performance systems (i.e., systems in which the opportunity to earn a bonus is offered at the beginning of the year and rewarded at the end of the year if a teacher meets the performance criteria) do not affect either teacher practices or student outcomes (Springer et al., 2010; Fryer, 2011; Gius, 2012; Glazerman & Seifullah, 2012; Springer et al., 2012; Yuan et al., 2013). For example, a randomized experiment of a school-based pay-for-performance program in New York City elementary, middle, and high schools found no effect of incentives on students’
2 “Growth” was calculated by standardizing students’ “academic change,” or changes in test scores (students’
normalized test score minus the average of scores from the two prior years, adjusted for the mean) on the mean and standard deviation from the first year the test was used. “Expected growth” was defined as having a collective change ratio of 0.0 or better based on results from all of the students who took the EOG or EOC tests, and “high growth” was defined as having a change ratio of 1.5 or better.
performance, attendance, high school graduation, or behavior; nor did it find evidence that incentives change teacher retention, absences, or reported behavior (Fryer, 2011). These findings are similar to those from Springer et al. (2010), who, from their analysis of Nashville’s three-year, randomized experimental merit pay program, found that large teacher-specific incentives had virtually no effect on teachers’ classroom practices or student achievement.
It is unlikely that failure to find an effect of performance incentives could be attributed to characteristics of the bonus itself. Whether bonuses are team-based or individual, traditional performance incentives do not seem to have an impact on teacher effectiveness (Fryer, 2012). Additionally, it is unlikely that these performance incentives offered incentives too small to make a difference in teaching, as the amount of money available to teachers in many of these experimental studies was quite substantial—up to $15,000 per individual teacher in Nashville and up to $3,000 per unionized teacher for each school to distribute at its discretion in New York City. 3
While there have been few significant or sizeable results in small-scale randomized-control experiments, there have been some significant effects of performance pay in conjunction with high-stakes teacher evaluation used by one whole district. Washington, D.C.’s IMPACT program, for two years, rated teachers by giving them effectiveness scores based on their value-added as well as on formal observations. If a teacher were rated “Minimally Effective” two years in a row, the teacher was forced to leave the profession. On the other hand, if a teacher were rated “Highly Effective,” she or he was awarded a large ($25,000) bonus and given an up-to-$27,000 annual base pay raise. Using a quasi-experimental (regression discontinuity) method, Dee and Wyckoff (2013) found that voluntary teacher attrition increased by 50 percent among teachers who received “Minimal Effectiveness” ratings in the first year, and for those minimally-effective teachers who chose to stay, performance increased by .27 standard deviations. Among the most effective teachers, a “Highly Effective” rating in the first year increased the teacher’s performance by .24 standard deviations in the following year. These findings are meaningful given that they result from an at-scale implementation of performance pay and teacher evaluation. However, while these are sizeable effects, it is important to note that these effects only pertain to teachers who are near the “Minimally Effective” and “Highly Effective” thresholds only, not the full population of Washington, D.C. teachers.
Overall, few studies find positive effects of performance incentives on student or teacher outcomes. However, those that examine teachers’ working conditions tend to find negative effects of performance incentives. Despite the higher pay, Heywood and Belfield (2008) found that job satisfaction also was lower for teachers who receive merit pay. The authors posit that job satisfaction is lower because of income risk or the extra pressure or effort required to maintain earnings. Yuan et al. (2013) confirm these suspicions, finding from their survey of teachers in three randomized experiments that merit pay programs were not motivating, increased the number or hours teachers worked, increased reported stress, and damaged collegiality with
3 One exception in this line of research has been a “loss aversion” study conducted by Fryer and colleagues in
Chicago Heights, Illinois (Fryer et al, 2012). In this randomized-control trial, bonuses were given to teachers at the beginning of the school year, and if teachers did not meet the performance criteria at the end of the year, they were forced to return the money. This “loss aversion” type of bonus did produce large and statistically significant gains of .2 to .4 standard deviations in math for students.
coworkers. This last result is particularly interesting, given that schools likely to use performance pay tend to be collaborative in nature (Belfield & Heywood, 2008). What is perhaps not
surprising, though, is that merit pay structures are more likely to exist in districts that have more information about individual teacher performance (high-accountability cultures), and less likely where teachers are unionized (Goldhaber, 2008), two conditions that may produce more stress for teachers.
While research finds few links between performance pay and desired student and teacher output in the United States, in other countries researchers have found much stronger links between performance pay and these outcomes. Lavy (2009), for example, found that in Israel, cash bonuses for student improvement led to increases in test-taking rates, conditional pass rates, and test scores—outcomes that he argues were mediated through changes in teaching methods and practices. Similarly, cross-national comparisons suggest that performance pay structures are significantly (.25 SD higher) related to math, science, and reading achievement (Woessmann 2011).
However, these studies assume that monetary rewards operate the same way for teachers as do those in other professions. When teachers are civil servants intrinsically motivated to develop youth (as in the United States), eva i (2009) argues that performance pay will have no impact the teaching profession is seen as an altruistic endeavor. However, when teachers are
opportunistic, using the post to advance their careers and political power (as is the case in India and many developing countries), performance pay will have a greater impact on student
outcomes and teacher effort ( eva i 2009). In other words, the assumptions that guide the economic perspective of teaching, effort, and compensation do not seem to adequately apply to cultures in which teaching is a form of altruism rather than opportunism, as is often the case in the United States ( eva i 2009).
North Carolina RttT Performance Incentive
The North Carolina RttT performance incentive is a $1,500 bonus awarded to certified
principals, teachers, and staff for meeting certain growth targets over the course of the RttT grant period (2010-2014). Those eligible for the bonus include certified staff in the lowest five percent of elementary, middle, and high schools in the state (as determined by Performance Composite and, when applicable, graduation rates).4 Performance Composites are the percentage of End-of-Grade (elementary/middle school) or End-of-Course (high school) test scores in the school at or above “grade level.” The lowest 5% of elementary schools had a Performance Composite below 52.4%; for middle and high schools, the threshold was 53.0% and 58.1%, respectively. High schools with four-year graduation rates under 60% also were eligible for the performance incentive. Performance Composites and graduation rates were calculated from 2009-10 school-year data, and at that time, 118 schools were selected for bonus eligibility. Since then, 12 of the lowest-performing schools have been closed or restructured, reducing the number of eligible schools to 106.
4
Performance incentive-eligible schools are not the same as federal Elementary and Secondary Education Act (ESEA) “priority” schools. ESEA priority schools comprise the lowest-performing 5% of Title I schools in the state (77 schools), whereas performance incentive-eligible schools comprise the lowest-performing 5% of elementary, the lowest 5% of middle, and the lowest 5% of high schools (original 118 schools at the start of RttT).
In 2010-11 and 2011-12, the $1,500 school-wide incentive was available to all certified staff in a school that made “high growth” according to the ABCs Accountability Model. Under ABCs Accountability, “growth” was calculated by standardizing students’ “academic change,” or changes in test scores (students’ normalized test score minus the average of scores from the two prior years, adjusted for the mean) on the mean and standard deviation from the first year the test was used. “Expected growth” was defined as having a collective change ratio of 0.0 or better based on results from all of the students who take the EOG or EOC tests, and “high growth” was defined as having a change ratio of 1.5 or better.
In April of 2013, the State Board of Education updated its policy governing the RttT performance incentives by making two changes that align the policy with both the newly-adopted statewide student growth model5 and the state’s plan for the final two years of the RttT grant. The first of the two changes aligns the incentive program’s technical terminology for “high student
achievement” with the terminology used in the new statewide student growth model adopted for first use in the 2012-13 school year. Specifically, the descriptor for bonus eligibility changed from making “High Growth” to “Exceed[ing] Expected Growth.” Also, growth now is determined by the state’s new approach to calculating teacher, principal, and school value-added, the Education Value-Added Assessment System (EVAAS), which measures how much students achieved relative to how much the students were expected to achieve in a particular course or grade. Under the EVAAS model “Exceed[ing] Expected Growth” means that students, on average, exceed the predicted scores associated with one year’s worth of growth at the school. Table 1 provides an overview of the number and percentage of North Carolina schools that attained the various designations under the ABCs Accountability and READY models.
Table 1. Schools’ Growth Categorizations by ABCs & READY Accountability Models, 2010-2013
ABC
Accountability 2010 2011 2012 READY 2013
Did Not Meet
Growth 300 12% 456 18% 510 21%
Did Not Meet
Growth 687 29%
Expected Growth 886 36% 982 40% 882 36% Meets Expected
Growth 1,027 43%
High Growth 1,305 52% 1,032 42% 1,091 44% Exceeds Expected
Growth 691 29%
*Note: Due to rounding, percentages may not add up to 100
The second, more significant change to the RttT performance incentive is that it offers an additional $500 above and beyond the $1,500 school-level bonus, for a total of $2,000, to
individual teachers in tested subject areas whose classes exceed “Expected Growth,” as determined by teachers’ individual value-added composites. Tested-subject teachers are those whose classes
5 The change in language reflects North Carolina’s switch from the ABCs Accountability model, which ended in
2011-12, to the READY accountability model, which began in 2012-13. The READY accountability model was a part of a comprehensive READY remodeling effort undertaken prior to RttT competition, but accelerated with the help of RttT funding. The overall READY implementation included updating the entire North Carolina Standard Course of Study (NC SCOS) at the start of the 2012-13 academic year; the updated NC SCOS includes the Common Core State Standards for English/Language Arts and Mathematics.
offer End-of-Grade, End-of-Course, or North Carolina Final Exams, or Career and Technical Education Post-Assessments in grades four and higher (Table 2). Even if teachers of tested subject areas teach in a school that does not make the $1,500 school-level bonus, these teachers still may be awarded $2,000 for exceeding individual-level “Expected Growth.” Table 3 summarizes the terms of the performance incentive and the number of recipients for each eligible year.
Table 2. Grades and Subjects Eligible for Performance Incentives (2012)
Elementary Middle High School
Grade 4 Math Grade 6 Math Agricultural Mechanics I & II Environmental &Natural Resources Grade 4 Reading Grade 6 Reading Agricultural Production I & II Equine Science
Grade 5 Math Grade 7 Math Agriscience Applications Fashion Merchandising Grade 5 Reading Grade 7 Reading Algebra I Foods I & II
Grade 5 Science Grade 8 Math Allied Health Sciences I & II Foundations of Info. Technology Grade 8 Science Animal Science I & II Fundamentals of Technology
Apparel Development I & II Furniture & Cabinet Making I & II Auto Svce. Technology I & II Health Team Relations
Biology Horticulture I & II Biomedical Technology Housing and Interiors
Biotechnology & Agriscience MS Word, Power Point & Publisher Business Law Medical Sciences I & II
Career Management Microsoft Excel and Access Communications Systems Parenting and Child Development Comp. Eng. Technology I & II Principles of Technology
Computer Programming Printing Graphics I & II Computerized Accounting Scientific & Tech. Vis. I & II Culinary Arts & Hospitality I & II Small Business/Entrepreneurship Digital Communication Systems Sports and Entertainment Marketing Digital Media I & II Structural Systems
Drafting I & II Teen Living
E-Commerce Transportation Systems Early Childhood Education Travel/Tourism & Recreation English I Welding I & II
Table 3. Performance Incentive Terms and Winners by Year
2010-11 2011-12 2012-13
Standard Meet High
Growth Meet High Growth Exceed Expected Growth Exceed Expected Growth
Type of Bonus School School School Individual
Number Eligible 115 schools;
4300 teachers 108 schools; 4200 teachers 106 schools; 3975 teachers 1000 tested teachers
Number Winners 23 schools 35 schools 35 schools 232 teachers
Bonus-Awarded Schools
In December 2011, 23 schools were awarded school-wide bonuses based on growth attained during the 2010-11 school year, and the following December, 35 schools earned school-wide bonuses for growth during the 2011-12 school year. There was not a high degree of overlap among winners between 2011 and 2012, with only 8 of the 23 winners from 2011 earning a bonus again in 2012.
In December 2013, $1,500 school-wide bonuses were awarded to certified staff in 35 schools that “exceeded growth” in the 2012-2013 academic year. Twelve of the 35 bonus-awarded schools in 2013 had been awarded a bonus in one of the two prior years, and three of the 35 bonus-awarded schools were awarded the bonus in all three years. Table 4 (following pages) provides a complete list of bonus-awarded schools from 2011 to 2013.
In addition to the school-wide bonus, individual teachers whose classrooms “exceeded expected growth” were awarded $500 on top of the $1,500 amount of the school-wide bonus, or a total of $2,000. These teachers were awarded the full $2,000 regardless of whether their schools as a whole exceeded expected growth. A total of 232 (23 percent) of eligible individual teachers earned this bonus, 112 of them (48 percent) having taught in a school that earned the wide bonus and 120 of them (52 percent) having taught in a school that did not earn the school-wide bonus. The percentage of teachers earning the individual bonus by Local Education Agency (LEA)6 ranged between 3 and 67 percent. Table 4 (following pages) also provides the total number and percentage of individual bonus winners by LEA.
Table 4. School-Level Bonus-Awarded Schools and Total Number and Percentage of Individual Bonus-Awarded Schools, by LEA
LEA 2011 School LEA 2102 School
Hickory City Schools Hickory Career & Arts Magnet High Anson County Schools Anson High Durham Public Schools Spring Valley Elementary Anson County Schools Morven Elementary Durham Public Schools WG Pearson Elementary Durham Public Schools Hillside High
Durham Public Schools YE Smith Elementary** Durham Public Schools Fayetteville St. Elementary Forsyth County Schools Philo Middle Durham Public Schools Southern High
Gaston County Schools Pleasant Ridge Elementary** Durham Public Schools YE Smith Elementary** Guilford County Schools T Wingate Andrews High** W-S/Forsyth County Sch. Forest Park Elementary Guilford County Schools Fairview Elementary Gaston County Schools Pleasant Ridge Elem.** Guilford County Schools Julius I Foust Elementary Gaston County Schools Woodhill Elementary** Guilford County Schools Oak Hill Elementary** Greene County Schools Greene Central High** Guilford County Schools Union Hill Elementary** Greene County Schools Greene County Middle Hertford County Schools Student Development Center Guilford County Schools T Wingate Andrews High** Charlotte-Mecklenburg Billingsville Elementary*** Guilford County Schools Ben L Smith High
Charlotte-Mecklenburg Martin Luther King Jr Middle School* Guilford County Schools Union Hill Elementary** Charlotte-Mecklenburg Pawtuckett Elementary Halifax County Schools Northwest High
Pasquotank County Schools PW Moore Elementary Halifax County Schools Southeast Halifax High** Pitt County Schools Pactolus Elementary*** Halifax County Schools William R Davie Middle Pitt County Schools North Pitt High Hertford Count Schools Riverview Elementary** Asheboro City Schools Charles W McCrary Elementary Charlotte-Mecklenburg Billingsville Elementary*** Richmond County Schools Mineral Springs Elementary** Charlotte-Mecklenburg MLK Jr. Middle*
Scotland County Schools SHS-Visual & Performing Arts Charlotte-Mecklenburg Sedgefield Elementary Wilson County Schools Vick Elementary Charlotte-Mecklenburg Thomasboro Elementary Wilson County Schools Beddingfield High*** Charlotte-Mecklenburg West Mecklenburg High Pitt County Schools Northwest Elementary** Pitt County Schools Pactolus Elementary*** Pitt County Schools Wellcome Middle** Robeson County Schools Fairgrove Middle** Robeson County Schools Townsend Middle Rockingham County Schools Lawsonville Ave Elem. Rowan-Salisbury Schools North Rowan High Tyrrell County Schools Columbia High** Union County Schools Rock Rest Elementary** Washington County Schools Pines Elementary Wayne County Schools Goldsboro High Wilson County Schools Beddingfield High***
LEA 2013 School LEA 2013 Individual Pct. of Eligible
Alamance-Burlington Eastlawn Elementary Anson 10 29%
Anson Morven Elementary Asheboro City 1 25%
Charlotte-Mecklenburg Billingsville Elementary*** Charlotte-Mecklenburg 43 27%
Charlotte-Mecklenburg Walter G Byers Elementary Duplin 1 13%
Charlotte-Mecklenburg Thomasboro Academy Durham 26 22%
Charlotte-Mecklenburg West Mecklenburg High Edgecombe 2 6%
Charlotte-Mecklenburg Druid Hills Academy Gaston 4 36%
Duplin Warsaw Elementary Greene 13 43%
Durham Chewning Middle Guilford 17 19%
Durham owe’s Grove Halifax 14 23%
Gaston Woodhill Elementary** Hertford 2 9%
Greene Greene Central** Hoke 2 29%
Guilford Smith High Lenoir 8 33%
Guilford Oak Hill Elementary** Nash-Rocky Mount 2 11%
Guilford Parkview Village Elementary Pitt 24 38%
Halifax Aurelian Springs Elementary Richmond 2 40%
Halifax Pittman Elementary Robeson 13 17%
Halifax Southeast Halifax High** Rowan-Salisbury 10 48%
Halifax Northwest High** Thomasville City 4 Not available
Hertford Riverview Elementary** Tyrrell 2 67%
Nash-Rocky Mount Williford Elementary Union 3 30%
Pitt Belvoir Elementary Vance 1 20%
Pitt Northwest Elementary** Washington 7 41%
Pitt Pactolus Elementary*** Wayne 1 3%
Pitt Wellcome Middle** Weldon City 2 13%
Richmond Mineral Springs Elementary** Wilson 14 64%
Robeson Fairgrove Middle** Winston-Salem/Forsyth 4 7%
Thomasville City Liberty Drive Elem
Thomasville City Thomasville Primary ** Denotes having received the bonus in two of the three years Tyrell Columbia High** ***Denotes having received a bonus in all three years
Union Rock Rest Elementary** Washington Washington County Union Wilson Beddingfield High*** Winston-Salem/Forsyth Forest Park Elementary
Summary of Findings from the First Report
In 2013, program evaluators assessed preliminary impacts of the RttT performance incentive on teachers in bonus-eligible schools. Evaluators conducted descriptive analyses of bonus-eligible and bonus-awarded schools, as well as qualitative analyses of teachers’ and administrators’ perceptions of and experiences with the RttT performance incentive. Focus groups and interviews with teachers and administrators in 12 bonus-eligible schools (approximately 80 teachers and administrators) were conducted in spring 2012, after they either had received or had not received the first bonus in 2011.
Several key findings arose from this first report:
1. Bonus-eligible schools had disproportionately higher percentages of poor and minority students compared to North Carolina schools on average, and there was little difference in these characteristics between bonus-awarded and eligible but non-awarded schools.
2. There was little to no awareness of the performance incentive initiative in the bonus-eligible schools that were interviewed in the spring of 2012. Even among schools that received bonus pay, there was confusion about eligibility in future years.
3. Student performance and growth increased between 2010 (prior to the incentive) and 2012 for both bonus-awarded and bonus-eligible schools. The lack of awareness of the program suggests that the performance incentive had little to do with this improvement.
4. Teachers generally reported that, even if they had known about the performance incentive, it would not have changed their teaching practices, although bonus-awarded teachers
appreciated the money after having received it. The perception that a performance incentive would not change their teaching practice was true of teachers and administrators at all school levels (elementary, middle, and high) and of all bonus status types (awarded and
non-awarded).
5. Finally, when teachers were asked whether they supported a school-wide or individual bonus structure, 75 percent reported favoring a school-wide bonus and 25 percent reported favoring an individual bonus structure. Those who favored individual incentives tended to work in school environments that were less collaborative than those who favored the school-wide bonuses.
In addition, one of the purposes of the incentive was to encourage educators who produce higher achievement to remain in the targeted schools. However, because few teachers were aware of the incentive, we were not able to measure that potential impact of the initiative.
Purpose of this Report
The purpose of this report is to provide a follow-up assessment of the impact of the RttT performance incentive. Because awareness of the program was quite low in the first year, the first evaluation assessed teachers’ abstract perceptions of performance pay rather than
perceptions of the RttT performance incentive in practice. This evaluation pays particular
attention to changes in teachers’ and administrators’ general program awareness and perceptions of their experiences with the performance incentive program. Also, because of the addition of the
individual tested-grade/-subject teacher bonus to the school-wide bonus, this report assesses awareness and perceptions of the program’s changes. This report uses qualitative analyses of interviews and focus groups with teachers and administrators in bonus-eligible schools to answer the following questions:
1. How has awareness of the RttT performance incentive changed among teachers and administrators?
2. How aware of changes to the performance incentive program are teachers and administrators?
3. What role has the incentive program, particularly the addition of the tested classroom incentive, played in teachers’ classroom and teaching practice? What role might an incentive program play in teachers’ classroom and teaching practice?
Data & Methods
Sample
Evaluators conducted semi-structured focus groups and interviews with 67 tested and non-tested grade/subject teachers and 12 administrators from seven schools eligible for the performance incentive. In order to have a diverse sample, evaluators purposively sampled 20 of the bonus-eligible schools to participate in follow-up interviews based on school level (elementary, middle high), how many bonuses the school had been awarded (none, one, or two), and whether or not evaluators had visited the school in the first-year evaluation.7 Seven schools agreed to participate in the study. Twelve of the 67 participants were interviewed individually, and 55 participants were interviewed in focus groups ranging in size from two to five people.
Interviews
Evaluators conducted interviews between September 2013 and January 2014—nine to 13 months after teachers had received the second RttT bonus. (In the one school in which interviews were conducted in January 2014, teachers had received the third RttT bonus only one month before.) All interviews and focus groups were audio-recorded and conducted during the school day during a teacher workday, a planning period, or a time when teachers were able to get coverage for their classes. To be considerate of time and coverage constraints for teachers, interviews with teachers as well as administrators lasted a maximum of 30 minutes, and focus groups lasted a maximum of 45 minutes, with few exceptions.
Interviews were semi-structured, which allowed evaluators to explore unanticipated themes as they arose. Evaluators asked questions about teachers’ awareness of the program—particularly whether they were aware of changes to the program, if and how the availability of the incentive affected their motivation or teaching practice, and their general perceptions of the program and of performance pay. Evaluators also asked questions about general practices, changes,
programming, and school climate or culture, particularly when significant changes—like an administration change or the adoption of a new program or curriculum—had occurred in recent years.
Data Analysis
All focus group and interview recordings were transcribed and coded for common and emergent themes.
7 Willingness to participate in the follow-up evaluation was lower than it had been in the first year, resulting in a
total Year 2 sample of seven schools—three elementary, one middle, and three high schools. Two of the schools in which evaluators interviewed had been awarded the bonus twice. One had never been awarded the bonus, one had been awarded the bonus in the first year but not the second, and two had been awarded the bonus in the second but not the first year. Additionally, four of the schools in which interviews were conducted in Year 2 were schools in which interviews has been conducted in Year 1.
Results
Changes in Teacher and Principal Understanding of the Program
1. How has awareness of the RttT performance incentive changed among teachers and administrators?
Awareness about the incentive program is growing, although there are still more teachers and principals unaware of the program than aware of it. At five of the seven schools in which interviews were conducted, there was at least one administrator or teacher who had good awareness (i.e., had a mostly accurate understanding of the policy and its terms) about the program and could talk about the details of eligibility or what it would take to attain the bonus. Overall, however, only 19 percent of interview or focus group exchanges about awareness were characterized as indicating “good” or “clear” awareness (Figure 1).
Figure 1. Proportion of Interview/Focus Group Exchanges by Level of Awareness
Here is an interchange between two tested-grade teachers in an elementary school that demonstrates the kind of understanding typical of “aware” teachers:
Interviewer: What do you know about the Race to the Top performance incentive initiative at this point?
Teacher A: That you get an incentive based on their performance. Interviewer: Based on performance?
Teacher A: Mm hmm
Interviewer: Ok, and how much is the incentive, do you know? Both: It’s $1,500
2%
17%
35%
27%
Interviewer: And do you know if it’s to individual teachers or to just tested teachers or to the whole school, or anything like that?
Teacher A: I think it was the tested? Based on the test scores? Was it 3-5? Teacher B: Uh huh. 3-5.
Interviewer: And the measurement is based on performance, you said? Both: [Nods]
Teacher B: Yes.
Interviewer: Ok, so the student performance? Go ahead, what were you saying?
Teacher A: I’m thinking, ’cause didn’t we get it, all the teachers, right? ’Cause didn’t K-2 get it?
Teacher B: Mm hmm
Teacher A: And it’s based on the school’s performance— Teacher B: At the time
Teacher A: ’Cause K-2 is not, they’re not EOG-tested Teacher B: Tested, yeah. Sometimes it’s for school. Interviewer: So it was for the school.
Teacher A: Mm hmm.
The teachers at this elementary school could communicate the terms of the program fairly well, although they were incorrect in some of the details (i.e., thinking that the program rewarded performance rather than growth). This excerpt also reflects some of the memory-jogging that was characteristic of several focus groups. It was common for teachers to claim that they knew little or even nothing about the incentive program at first, but then with more specific questions and comments from other colleagues, they started to remember some of the details. The fact that it took some pointed questions and collaborative discussion to uncover what memories the teachers had about the program suggests that the RttT performance incentive may not have been a
primary motivator for many of the teachers in eligible schools.
More common was vague, general knowledge that an incentive program existed, but an inability to communicate anything about the details of the program. At all seven schools in which
evaluators conducted interviews and focus groups, at least one teacher or principal (but usually more) had this kind of vague knowledge about the incentive. As one high school teacher said:
We were briefly told about it way, way back, and then all of a sudden no one talked about it, then all of a sudden we were a “School of Growth” and we’re getting money. . . . You know . . . for me it was always very confusing, because the growth is only measured in the three tests. I didn’t teach any of those grades, so . . . was the money just going to those teachers? Were we getting [it]? You know, it was very confusing for me. Other teachers had little knowledge of the program in general. When asked about what they knew of the performance incentive, one teacher responded:
I wasn’t aware of it. The only performance incentive I was aware of is what our state is proposing from the legisla[ture] . . . I don’t know if Race to the Top had anything to do with that.
The teacher in this focus group was one of about 13 other teachers who were unable to remember anything about the RttT performance incentive (approximately 19 percent of the sample). It is important to note that even in schools that had received bonuses before, awareness of the program among administrators and faculty, in some cases, was still quite low. In about half of the schools visited, previously-interviewed administration and faculty had turned over between receiving or being eligible for the first bonus (December 2011) and the time of the interview (May 2013 to January 2014). The following excerpt from one high school assistant principal demonstrates how turnover results in limited awareness of the program:
I’m totally not aware that . . . [there was a performance incentive, and a] couple of things happen[ed] at this school. We’re under new leadership. . . . And the Race to the Top funds that were dispersed, I guess, two years ago or a year and a half ago . . . all of that was under a different administration. . . . The administration here now is brand new from a year, I’m the longest standing administrator and I’ve only been here a year and a half. . . . And the principal’s new, and the other two—the intern and the other assistant principal—are brand new, like this week new [emphasis added].
This assistant principal later described having no information or communication about the program when he entered his position in fact, the interviewer’s email to participate in the evaluation is what prompted him to learn more about the RttT performance incentive.
Awareness of Program Changes
2. How aware of changes to the performance incentive program are teachers and administrators?
Almost every teacher and administrator in six of the seven schools in which evaluators
conducted focus groups had no awareness of a significant change in the policy: the addition of the $500 individual incentive for tested-area teachers. The following excerpt demonstrates a typical interaction between interviewer and participant when asked if s/he was aware of any changes to the performance incentive:
Interviewer: Are you aware of any changes to the Race to the Top performance incentive since it started?
Teacher: I’m not. Has there been, or . . . ?
One principal was clearly aware of the additional individual bonus and could recite its precise terms with no additional probing or pointed questions. The teachers in this principal’s school, though, were more mixed on their understanding of the program’s changes, with half of the them believing the individual bonus replaced the school-wide bonus, and the other half not realizing that there was an additional individual incentive at all.
Overall, with few exceptions, virtually none of the teachers and administrators knew about the opportunity to earn an additional individual bonus for tested-area teachers, which suggests that, in most schools, the individual bonus played little to no role in motivating or changing teacher practice.
Explanations for Changes in Awareness
Program Saturation and Confusion
Because these schools are the lowest-performing in the state, several of them had School Improvement Grants (SIG) and other types of programs designed to improve student
achievement.8 In schools that were saturated with grants and programming (two of the seven schools were recipients of SIG grants), teachers and administrators had the hardest time identifying or detailing the components of the RttT performance incentive. The inability to distinguish between programs was apparent in one elementary school focus group description of the RttT performance incentive. When the interviewer asked the focus group to describe the terms of the RttT performance incentive, they said:
Teacher A: I want to say $700, but that’s just off the top of my head.
Teacher B: We got a certain percentage if we met math versus reading, but you had to make the qualifications, you had to be here, you couldn’t miss more than five days, your kids had to grow a year and a half in reading and math. I don’t remember the numbers that we physically got for our bonus, but. . . .
Teacher A: And then on top of that, there was like a school-wide one, too. Teacher B: Yeah.
Teacher A: And I don’t know what the school-wide—I don’t know how much that was. The RttT performance incentive, which is a $1,500 bonus (not the $700 these teachers reference) rewards nothing but student growth. However, these teachers confused the RttT performance incentive with the SIG bonus, which in this school primarily rewarded teacher attendance. Communication about the RttT Performance Incentive
Regardless of program saturation or administrative turnover, the principals and teachers who had the most awareness of the program or of changes to the program were the ones who appeared to have the closest partnerships with their schools’ Instructional and School Transformation Coaches (STC) from the District and School Transformation (DST) division of the North
Carolina Department of Instruction (NCDPI). This was the case for three of the seven schools in this evaluation. Two of the three schools with regular School Transformation or Instructional Coach communication also had principals who reported doing extra personal research about the incentive and other RttT programs online. The following interchange demonstrates the level of constant communication that a very aware principal receives, particularly from the NCDPI
8 For more information about state- and local-level pay-for-performance and strategic staffing initiatives, please see
Strategic Staffing in North Carolina: A Summative Review of Local and State Implementation across the Race to the Top Period (http://cerenc.org/rttt-evaluation/equitable-supply-and-distribution-of-teachers-and-leaders/).
instructional coach (This principal also was the only principal to be aware of the addition of the individual performance incentive):
Principal: I do a lot of reading and research and also with my STC [School
Transformation] coach through DPI. . . . She and I, we talk about it frequently. She always keeps me up-to-date on, um, released information based on Race to the Top. Interviewer: Do you talk to her by phone, via email?
Principal: Mmm mmm. We actually do, we talk by phone, we text, she comes to the school, too, so she does site visits as well.
While the principal above had constant communication with the school’s NCDPI coach through a variety of channels (phone, text, in-person communication), the schools that had the least awareness of the incentive program (four of seven schools) reported receiving knowledge about the program from mass emails and/or letters from the state only, or reported receiving no communication at all. When the interviewer asked the administrative team at one of the less-aware schools how they received their information, they responded:
Principal: I have a publication that comes to me from [Race to the Top] that I get. Assistant Principal: Emails that we get.
Interviewer: OK, emails.
Assistant Principal: Other than that, it’s . . . [pause]. Interviewer: Not a lot of communication otherwise? Administrative team: [Nods]
These responses suggest that constant, interactive communication with principals is a key to program awareness, as teachers in schools with very aware principals tended to be more aware of the performance incentive program than were teachers in schools with less-aware principals. Teachers, especially in schools with less-aware principals, also mentioned desiring better, clearer, and more concise communication about what the incentive program was, who was eligible, and how teachers could get the money. Teachers from other focus groups and schools believed that emails would get lost in their inboxes, and requested workshops or panel discussions with people from NCDPI instead.
While participants’ responses about awareness and communication of program information relate explicitly to the RttT performance incentive, participants’ perceptions of the RttT bonus and its role in their practice blended so much with their general opinions of performance pay that it is not possible to disentangle the two. The following results therefore represent participants’ perceptions not only of this particular RttT incentive program, but also of performance incentives in general.
Changes in Perceptions and Role of RttT Performance Pay in North Carolina Schools
3. What role has or the incentive program, particularly the addition of the tested classroom incentive, played in teachers’ classroom and teaching practice? What role might an incentive program play in teachers’ classroom and teaching practice?
At baseline, when there was virtually no awareness of the program, evaluators found that an overwhelming majority of teachers did not think performance incentives would change their teaching behavior or practice. After two years of eligibility in the program with growing—but not yet full—awareness of the program, the reported role of the performance incentive in teaching behavior and practice has been more mixed. Specifically, about 73 percent of
interviewees and focus group members (49 out of 67 total participants) who were asked about the role of performance incentives in teaching reported that the performance incentive had no role in their own teaching (Figure 2).
Figure 2. Role of Performance Incentive in Own Teaching Practice (Percent of Respondents)
However, 12 teachers who said that the incentive played no role in their teaching did report that incentives may play or have played a role in their colleagues’ teaching practices or other aspects of their profession, such as collegiality or willingness to collaborate. About 24 percent (16 out of 67) of participants reported that the performance incentive had some (though usually small) role in their or their colleagues’ practice (Figure 3, following page). “Some role” meant that teachers found the incentive to be “extra motivation” to “push ourselves,” though when asked what that meant, they could not typically verbalize what extra motivation would look like or how it might translate to identifiable change in practice. About 22 percent (15 out of 67) of teachers reported that the incentive played a larger role in their colleagues’ teaching, meaning that practices in the classroom or within the structure of the school had changed as a result of eligibility for the performance incentive.
73%
12% 15%
Figure 3. Role of Performance Incentive in Own or Colleagues’ Teaching Practice (Percent of Respondents)
Note though, that changes to teaching practice did not always correspond to being awarded the bonus. In fact, only five teachers out of all 31 who said that in 2012 performance incentives had affected or would affect their or their colleagues’ teaching or school climate taught in a school that actually earned the bonus in 2013. The sections that follow explore these perceptions in greater detail.
Perceptions of Disconnects between Performance Incentives and Targeted Outcomes
Educators who thought that incentives had little effect on teaching highlighted four aspects of teaching in particular that they thought would not be impacted by the presence of incentives: teaching practices, motivation, collaboration, and commitment.
Disconnects between incentives and teaching practices. An overwhelming majority of
elementary teachers (71%), and several middle and high school teachers and administrators, said that performance incentives would not play any role in their teaching practice (Figure 4).
Figure 4. Role of Incentive in Own or Colleagues’ Teaching Practice, by School Type (Number of Respondents)
54%
24% 22%
No Role Some Role Big Role
71% 21% 7% 41% 26% 33%
No Role Some Role Big Role
Elementary Middle/High
As one teacher succinctly put it:
Interviewer: Do you think [the performance incentive] would change your practice . . . Teacher: Not mine.
Interviewer: . . . to be eligible [for the incentive]? Teacher: No.
Interviewer: No? Teacher: No.
While most teachers in general reported that performance incentives played no role in their teaching, some high school teachers expressed an interest in a pay structure that included performance incentives. As discussed later in this report, it is unclear why more high school teachers in particular believed the incentive played a role in their teaching.
There is, however, much more consensus about why teachers believed performance incentives would have little influence on their teaching practices. In general, teachers believed that their motivation for teaching was not tied to extra compensation, that performance incentives did not encourage the type of collaboration they felt was necessary for successful teaching, and that they already were working as hard as possible. Of those three reasons, by far, the most consistent reason that teachers gave for why performance incentives would not affect their practice was their intrinsic desire to work hard and serve students rather than to earn extra money.
Disconnects between incentives and motivation. Many of the teachers and administrators said that the performance incentive had not impacted them at all. One of the reasons why the performance incentive appeared not to have mattered to teachers was because teachers and administrators regularly said that what they do is not for money. When the study evaluator asked one principal whether she thought the incentive had impacted the teachers’ performance in any way, she said:
It just didn’t seem to make a difference. I never heard them talk about it, as a motivating factor for we need to do this, it was always about the data and always trying to increase the data from one benchmark to the next, from one formative assessment to the next. I never heard anything about the money. . . . I didn’t hear them discussing the money, the incentive pay. . . . I don’t think it ever came up after that initial meeting that I told them about it.
Then she went on to explain why she thought the teachers were not impacted by the performance incentive:
Because, I started talking about, for me, what education is, is a moral purpose. It’s a moral imperative. And, I think [the teachers at this school] understand that as part of my . . . vision for [our school] as we work towards 100% of our students being successful, that that’s part of, it’s a moral purpose for me, and I think that’s become part of our
This principal described being motivated by a “moral purpose” rather than by a monetary one, and teachers in this and other schools reported similar motivations for their practice as well, particularly “not doing it for the money.” Said one teacher:
I think it really comes down to the heart of the school and the heart of the educator. Why are they there? If you find teachers who are there just for the money, obviously when the money goes away, the teaching will decrease. You will be hard-pressed to find teachers here who are solely here for the money. That would just be a crazy career choice. Rather than money, teachers described themselves as having an “intrinsic value” of doing their work “for the sake of our goals toward the children.” “We all care very deeply and want to move our kids, so that when they move to the next grade they’re the best-prepared children,” one kindergarten teacher explained.
Disconnects between incentives and collaboration. In addition to teachers reporting that they work for the good of the students and not for extra money, teachers and administrators who reported that the performance incentive played no role in teaching also suggested that “best practices” in education are inconsistent with the structure of performance pay, particularly when those bonuses are awarded at the individual level. Specifically, teachers and administrators pointed to the importance of collaboration among faculty and staff members as being a key mechanism for identifying and sharing strategies for increasing student learning. Said one elementary school principal:
We work very, very hard to create an atmosphere of unity and collaboration, and [to] get away from that old mindset of going into my classroom, closing my door, and these are my 20 or 25, and I’m only concerned about what they do. Because I really had . . . to work on helping them to understand that, it doesn’t matter if that 25 are successful, and the rest of the 500 or so are not. That we are a team. That, when we all aren’t successful, then none of us are successful, so . . . I’ve had to work really hard to create that . . . collaborative culture.
This principal recognized that collaboration is necessary for a whole school’s success, and though this school has yet to receive a bonus, when the staff started implementing strategies to increase collaboration, the school made growth for the first time in their five-year history and was identified as one of the best elementary schools in the LEA. Similar examples of
collaborative practices were shared by participants in bonus-awarded schools (including those awarded in more than one year) as well.
Teachers and principals at most of the schools in this study recognized that individual
performance incentives could be destructive to collaboration efforts in schools, pushing teachers “back into that close-the-door-and-my-25-[students] mindset.” Additionally, particularly in high school, some participants reported that awarding incentives to some teachers and not others, particularly making the distinction between “tested” core subjects and “untested” electives and other courses, can create friction and decrease collaborative efforts to share in the responsibility of teaching tested-area content across classrooms. Said one principal:
It’s . . . divisive . . . especially in our [school] community, where . . . [we’ve] struggled academically for a number of years and . . . our efforts have all been pointed at, “ et’s bring everybody together. et’s make this happen as a group. . . .” [P]eople are gonna look at it [like], “Well, why do I put forth the effort when they’re gonna receive the reward for it, why do I give up time in my class to work on these skills when I’m not gonna get anything for it?”
While principals like the one above spoke hypothetically that individual incentives could be destructive to teacher collaboration and morale, teachers in one school who were very aware that individual bonuses would be awarded for high growth in tested grades and subjects reported already feeling the decreased morale and collaboration principals and other teachers had feared. As this tested-area middle school teacher said,
I feel like last year when everybody knew that everybody would benefit together, that we were helped more by teachers that were not in a tested area. Um, our resource teachers . . . PE and art and music, they were more helpful, they pushed the students a little more last year than they have this year. This year, if you ask for help you may or may not get it. Last year it was, “Well, we’re all one big happy family because we’re all gonna get something,” and this year it was, “You get a bonus if your scores are good.” I’ve heard that from at least one person. I was like, “Ok, I’ll get it myself, then.”
The following interchange between resource teachers in the same school who were not eligible for the individual bonuses confirms the suspicion of the tested-area teacher that they were less willing to help:
Teacher A: I really do think eventually that . . . once people realize where the money is going, that . . . there might be some professional jealousy. et’s use that term.
Teacher B: We might not be so apt to . . . Teacher C: Help.
Teacher B: . . . help with reading, right? Teacher D and C: Yeah.
Teacher A: No. Because why am I gonna help you—and it may go up like she said it, I had heard, [to] $2,500 too—why am I gonna help you get $2,500 when I maybe get a bag of peanuts?
Not many teachers were so blunt about their unwillingness to help as the teachers in this school. In fact, teachers in some of the other schools seemed to think that individual performance incentives either would not make a difference in, or even would increase, collaboration (such was the case for the majority of elementary school participants, who reported that the incentive would play “some” role in their colleagues’ practice). What makes the teachers from this middle school worth featuring, though, is that they had high—albeit slightly misinformed—awareness of the incentive program. For example, one teacher believed it was possible to earn up to $2,500, when in reality the highest bonus one could earn was $2,000. More importantly, this group of teachers believed that bonuses were only individually-based and that there was no opportunity for the school to earn a bonus for collective growth. This is the only school where teachers had
any awareness of the individual bonus. Teachers in the other six schools had little to no awareness of the mid-program addition of an individual bonus for tested-area teachers only. Disconnects between incentives and commitment. Most teachers who reported that the bonus played no role in their teaching claimed to be working as hard as they could, regardless of whether an incentive was available to them. One high school teacher noted that “If we get a $30,000 raise it would make it easier to come to work every day but I’m not going to work any harder because I feel like I work as hard as I can every day.” A middle school teacher felt the same way: “Regardless of whether there is a bonus or not, you know, we just work hard, and, and we try to be our best, and to do our best, so, that’s just the way I feel. Regardless of whether there is a bonus or not, I come to work every day and try to give it my all.” “You’re not short on motivated teachers,” another high school teacher said; “I see people who are here before the sun comes up and are here till the sun goes down, and they’re doing everything they can possibly do to motivate students.”
Additionally, teachers identified very few of their colleagues as being ineffective. Most teachers described a bell curve of teaching effectiveness at their school. Some teachers identified only one or two teachers in their school as being ineffective, whereas others believed up to 10 or 15
percent of their colleagues were ineffective. There was only one teacher who claimed that 25 percent of her colleagues were “slacker” teachers (25 percent was the highest percentage any teacher reported as being ineffective in a school).
Regardless of how many teachers participants claimed were in the “least effective” end of the distribution, only three of the teachers and one of the principals interviewed (six percent of the total sample) believed that a performance incentive would increase the efforts or effectiveness of the lowest-performing teachers in their schools.
However, even among participants who claimed that the bonus played no role in their teaching practice, teachers consistently reported being happy to receive it. In one interview, one of the evaluators spent 10 or 15 minutes asking questions about what made students grow, in this teacher’s opinion. The evaluator noted to the participant that a performance incentive was not included in this teacher’s list. When the interviewer suggested that the incentive might not be the best way to encourage student growth through more effective teaching, the respondent replied:
Teacher: I don’t want to say bonus is useless! . . . You know, it’s really nice, and I know for years, like, when I first started, all these other schools were getting all these different bonuses for growth and we didn’t get anything because we didn’t grow. And then it was like the first year that we actually grew, that bonus got axed because the money wasn’t available. We’re like, “Shoot, we still don’t get it.” So it’s nice to get it.
This seems to suggest that even if performance incentives play little role in affecting teachers’ practices, teachers still appreciate receiving the extra money and do not want to see the
Perceptions of Connections between Performance Incentives and Targeted Outcomes: High School Level
While most teachers did not think that the performance incentive had affected or would affect their own teaching, about 44 percent of teachers reported that the performance incentive had affected or could affect their or their colleagues’ practice. As indicated above in Figure 4, very few elementary teachers shared this opinion, but some middle/high school teachers did. Aside from the five middle school teachers who noticed that resource teachers would be less likely to collaborate with tested teachers, most teachers in this sample who reported that performance pay had affected or would affect teaching were high school teachers (17 out of 29 high school teachers). Nine of those teachers believed that the extra compensation would be a “good thing” for education because it would “motivate” or “energize” teachers who “maybe don’t give 110 percent” every day.
Few teachers and administrators explicitly stated that performance incentives would change teaching behaviors and practices. However, comments from eight participants seem to suggest that the performance incentive did play a role in changing some of their teaching strategies. Teachers at one school talked about introducing professional learning communities in order to get the bonus. A teacher at another school mentioned that the bonus motivated her or him to volunteer for extra tutoring during planning periods, and other teachers mentioned that “interventions” started earlier in the year to ensure that the lower achievers were getting extra assistance. These are some of the few concrete practices that teachers indicated had changed as a direct result of the performance incentive.
Several reasons emerged for why these educators thought that performance pay could or would fairly motivate teachers, some of which contrast with reasons above that other educators gave for why incentives would not be effective. However, because a smaller proportion of teachers shared this view, few of their reasons were reported consistently by multiple groups of teachers, making it less clear why some high school teachers believed performance incentives would play a role in their teaching. The three most commonly-cited beliefs—that incentives might positively impact collaboration, motivation, and self-improvement—are explored here.
Performance incentives may encourage teachers to work together. The most widely supported view (held by at least one focus group each in two of the three high schools) for why
performance incentives—specifically, the school-wide performance incentive—would play a beneficial role in teaching was that they would make teachers work “harder” and “together” as a school. In practice, teachers reported that working “harder” consisted of more teachers—
particularly non-tested or non-core class teachers—incorporating tested content into their classes in order to earn the bonus. This is precisely what a non-tested history teacher from a recent bonus-awarded school reported experiencing:
We kinda joked around, I mean joked, but serious, and said, “Hey, we’re all Algebra teachers this year. Hey, we’re all biology teachers this year. Hey, we’re all English I teachers this year.” Because those were the . . . places that we were gonna be assessed. In addition to incorporating more tested content into non-tested courses, high school teachers also reported working “together” to “pull kids in those tested areas . . . out of teachers who are
