Teacher-Level Analysis Findings: Teacher-level regression models, run separately for each district, indicated that increasingly positive
teacher responses on four variables—Openness to Feedback, Evaluator Quality, School Support Structure and Rating Alignment Scale*—were associated with small increases in observation scores, standardized evaluation scores and/or value-added scores. Each model controlled for prior performance and years of teaching experience. See Appendix Table B3 for additional details on the construction of these variables.
Observation Scores. Across a series of linear regression models, four predictors were significantly related to increases in teacher
observation scores: Openness to Feedback, Evaluator Quality, School Support Structure and Rating Alignment Scale. The number of teachers contributing data varied across models. In Districts A and C, between 1,500 and 2,700 teachers were included. In District B, between 200 and 400 teachers were included across models.
For every one-point increase on our Openness to Feedback measure, observation scores could be expected to increase by 0.72 points in District A (p<.001), 0.04 points in District B (p<.05) and 0.04 points in District C (p<.001). The more positively teachers rated the quality of their evaluators, the more their observation scores increased. A one-unit increase in the evaluator quality construct was associated with observation score increases of 0.74 points in District A (p<.001), 0.08 points in District B (p<.001) and 0.10 points in District C (p<.001). As teachers provide more positive responses on the school support structure index, observation scores could be expected to increase by 0.33 points in District A (p<.001), 0.04 points in District B (p<.01) and 0.03 points in District C (p<.001). Finally, as teachers reported ratings which were more aligned to the formal assessment of their practice in 2013-14, observation scores were expected to increase by 2.49 points in District A (p<.001), 0.17 points in District B (p<.001) and 0.05 points in District C (p<.001).
THE MIRA
GE
Standardized Evaluation Scores. Approximately the same number of teachers were included in these models as were included in
models predicting observation scores. In these models, two variables were significantly related to evaluation scores: Evaluator Quality and Rating Alignment Scale.
A one-unit increase in teacher perceptions of evaluator quality was associated with an increase in standardized evaluation ratings of 0.09 standard deviations in District A (p<.001), 0.17 standard deviations in District B (p<.001) and 0.07 standard deviations in District C (p<.001). Rating alignment was also significantly related to increases in standardized evaluation scores; as teachers reported ratings more aligned to the formal assessment of their practice in 2013-14, standardized evaluation scores could be expected to increase by 0.33 standard deviations in District A (p<.001), 0.47 standard deviations in District B (p<.001), and 0.53 standard deviations in District C (p<.001).
Value-added Scores. Notably, only two districts had enough teachers with value-added scores and survey data to conduct these
regressions (in District A, roughly 2,200 teachers contributed data, and in District C, roughly 450 teachers are included). In these models, rating alignment was the only significant predictor. As teachers reported ratings more aligned to the formal assessment of their practice in 2013-14, value-added scores were expected to increase by 0.54 points in District A (p<.001) and 0.99 points in District C (p<.001).
*Note: All teachers who received the highest rating in 2013-14 in each site were removed from the analysis to look more specifically at teachers not already identified as the highest performers.
School-Level Analysis Findings: School-level regression models, run with all districts pooled, indicated that increasingly positive
teacher responses (aggregated to the school level) on two variables—Average Number of Observations and Rating Alignment*—were associated with a small increase in the percent of improvers at a school. Each model included a thematically related subset of variables constructed by aggregating individual teacher survey responses to the school level, as well as controls related to school demographics and aggregate teacher demographics. See Appendix Tables B3 and B4 for additional details on the construction of these variables. Percent of teachers improving on observation scores. There were approximately 370 schools included in regression models
predicting the percent of teachers in a school improving on observation scores (using the “quartiles of growth” definition). For every increase in the average number of observations reported by teachers in a school, the percent of teachers identified as improvers at the school was expected to go up by 3% (p<.05). When considering teachers’ self-reported evaluation scores as compared to the formal assessments of their practice in 2013-14, for every one-unit increase in school alignment scores, the percent of teachers identified as improvers at a school was expected to increase by 10% (p<.01).
Percent of teachers improving on standardized evaluation scores. There were approximately 370 schools included in regression
models predicting the percent of teachers in a school improving on standardized evaluation scores (using the “quartiles of growth” or “fixed-split growth” definition). For every addition to the average number of observations reported by teachers in a school, the percent of teachers identified as improvers at the school was expected to go up by 3% (p<.05) or 2% (p<.05) using “quartiles of growth” and “fixed-split growth,” respectively. When considering teachers’ self-reported evaluation scores as compared to the formal assessments of their practice in 2013-14, for every one-unit increase in school alignment scores, the percent of teachers identified as improvers at a school was expected to increase by 28% (p<.01) or 25% (p<.01) using “quartiles of growth” and “fixed-split growth,” respectively.
Percent of teachers improving on value-added scores. There were approximately 200 schools included in regression models
predicting the percent of teachers in a school improving on value-added scores (using the “quartiles of growth” definition or the “fixed-split growth” definition). Only District A and C were included in the VAM analysis due to sample size limitations at the school level in District B. For every additional observation reported by teachers in a school on average, the percent of teachers identified as improvers at the school was expected to go up by 3% (p<.05), using “quartiles of growth.” As teachers at a school, on average, self-report ratings more aligned to or deflated in relation to the formal assessments of their practice in 2013-14, the percent of teachers identified as improvers at a school was expected to increase by 10% (p<.05), using “fixed-split growth.”
*Note: All teachers who received the highest rating in 2013-14 in each site were removed from the analysis to look more specifically at teachers not already identified as the highest performers.