Moreover, identifying base-level practices and common expectations helped NSI build systems they could use to improve this work. What elements of their design were most criti- cal and non-negotiable for student learning? What were the gaps and problems that their designs missed? Were there any adjustments or additions educators made to their designs, and if so were they effective at solving unforeseen challenges? With planned study around such questions, NSI learned from their members about what it takes to solidify support for the implementation of instructional practices, and to incorporate promising innovation.
Contingencies in Measuring Instructional Design and School Improvement Processes
While all of the NSI used state test results as an
important gauge of the efficacy of their instruc- tional resources, such measures alone could not supply sufficient feedback for this develop- mental learning task. To varying degrees, each built other indicators to acquire more regular and immediate information, such as teacher and leader surveys, teacher and leader advisory groups, school site walkthroughs, instructional observations, formative assessments of stu- dent learning, and more. Some established structured routines to analyze the evidence and incorporated lessons learned into their instruc- tional resources.
But there were key differences in the approach- es that these NSI used to improve their work that stemmed, in part, from their varying governance arrangements. With more direct authority over member schools, the CMOs were in a position to more readily develop and require a compre- hensive set of resources and processes at the school level, such as regular, school-based PL sessions and monitoring tied directly to the instructional practices they wanted to inculcate. One CMO created a robust series of hub- and school-level mechanisms to support and gather evidence about those who had successfully mastered instructional practices. Proficient practitioners were given leeway to experiment with adaptations or additions to the design. Each year this CMO would use released state test results to identify high performing schools and teachers, and over the course of a month hub leaders would interview and even videotape innovative practices to distribute as a resource for others, or to revise their central design. The second CMO had similarly comprehensive instructional resources and mechanisms to secure and gauge fidelity to instructional prac- tices. However, while this CMO articulated the desire for teacher adaptation and thoughtful use of their resources, hub leaders had not created
any processes to nurture and support exper- imentation, or to gather information about and incorporate positive changes. Hub leaders rec- ognized that they needed to establish more de- liberate strategies for innovation and redesign. The context for AOs and FSCs placed some constraints on their ability to develop a similar hub learning strategy. With a more powerful set of district superintendents pressing in on schools, and without their own power to man- date practice, AOs were reluctant to adopt new and intensive measures to gather information on the implementation of their instructional designs by those who had opted to use them. And, al- though they had formative assessments directly aligned to their instructional resources, these were similarly optional for teachers to use and thus not a reliable source of feedback on the work. Instead, these NSI relied much more on advisory groups and surveys of participants in their professional learning as a primary source of information about their design. They also gauged their success by the numbers of cli- ents regularly participating in their professional learning.
FSCs, too, were reliant on member districts and schools opting in to their services. Here client participation was also a key source of feedback about whether FSC supports were satisfying member needs. This fairly simple measure led to significant changes in what the hubs pro- vided. For example, during its first two years in operation, one FSC created borough-wide PL sessions based on their own analysis of what the majority of district clients needed. This process led to a search for common ground across a diverse array of instructional designs, and, ultimately, a fairly generic set of sessions. These PL events were poorly attended and were not getting strong traction within schools. In their third year, this FSC shifted the content and
46
delivery of PL so it was located in individual dis- tricts and tied to the more specific instructional initiatives in which these districts were engaged. Superintendents then reinforced this content in their own leadership meetings with principals. With improved coordination and alignment across levels of the system, teacher attendance and enthusiasm for the FSC professional learn- ing sessions increased.
Networks’ abilities to learn and improve were also dependent, in part, on their organizational maturity. Both CMOs in our sample had been op- erating for at least a decade, giving them ample time to develop information systems, evaluative routines, and communication mechanisms that together provided a relatively complete picture of their networks’ status over time. Similarly, the AOs in our sample had been operating in the district for some time despite the new pressures that the recent restructuring had placed on their organizational visions and procedures. Not only did both AOs collect substantial information about school progress throughout the year, but they also regularly analyzed this data alongside school leaders to ensure support plans were appropriate to meet schools’ shifting needs. On the other hand, SOs in their current form and FSCs were new players on the scene. They spent much of their first year or two figuring out the logistics of forming a support network. One FSC director we spoke with in February of year one had just hired the individuals necessary to provide instructional professional learning sessions to schools. Thus, it was not surprising that many of these teams used somewhat crude measures of network success in the early years; simply being able to offer professional learning and seeing teachers show up and appreciate those offerings was an improvement. However, as these organizations evolved, they too began moving toward more sophisticated means of
assessing their impact. FSC personnel spoke of using student work samples to determine the extent to which instruction had improved over the course of a professional learning cycle. One FSC director summarized this shift to more so- phisticated reflection as follows: