Doing
What
Matters
Creating the Conditions and Know‐How for Effective Data‐Driven Dialogue
around Student Performance and Behavior
Research on school improvement places data‐driven dialogue at the heart of any
successful reform effort.1 Two major challenges face educators trying to make
data‐driven dialogue work in their schools and districts: how can accurate and
timely data be made available to all who need it when they need it; and what
strategies should be used for converting data about students into knowledge and
actions that will benefit students. The Institute for Research and Reform in
Education (IRRE) offers technical assistance and support to address both of these
questions.
What
is
DWM?
Doing What Matters (DWM) is a partnership between IRRE and schools and
districts to provide new data technology, technical assistance and professional
development supports to strengthen data reporting systems, engage in a proven
process of data‐driven dialogue; and develop specific improvement strategies to
increase student performance and commitment.
Strengthen and Enhance Data Reporting Systems
Make them more accurate, useful, and accessible
Make them less institutional and more about student performance Build System Capacity around Data‐Driven Dialogue
Dialogue about student performance and commitment
Dialogue among teachers and between teachers and administrators To use data to make decisions and target strategies for improvement
at all levels of the system and then track the results.
1
Cawlti & Protheroe, 2001; Massell & Goertz, 2002; McLaughlin & Talbert, 2003; Skrla, Scheurich, & Johnson,
DWM is focused on putting in place the conditions and capacities necessary for a
culture of data use to emerge and be sustained in schools. The four building
blocks of a data‐driven culture are:
1. System Leadership – At all levels of the district (teachers, school
administrators, central office staff, senior district leadership, and board)
leaders create the conditions, implement supports, set expectations and
accountability for, and model data‐driven dialogue in order to create a culture
of data use.
2. Capacity and Structures – At all levels of the system leadership and staff are
trained and have opportunities to practice using data‐driven dialogue
processes. Teachers, counselors and administrators have the time and space in
the workday to have regular and on‐going data‐driven dialogue about student
performance, behavior and commitment.
3. Data Governance – The district implements an intentional program of active
data management and quality control processes and creates policies and
procedures to ensure that district and school data holdings are accurate, up‐
to‐date, and supported by a system of on‐going data audits. Accountability for
data integrity is distributed to identified data owners and data stewards
throughout the system.
4. Enhanced Technology – District data systems integrate disparate databases
into a single data warehouse where users can easily access up‐to‐date,
accurate, and actionable reports of student performance and behavior.
Teachers, students, parents and administrators can each access reports on a
limited number of key research‐proven indicators of student progress and
behavior through customized user dashboards.
Figure 1. Building Blocks of a Data‐Driven Educational System
What
is
the
History
of
this
Work?
Measuring What Matters (MWM) is IRRE’s data‐based continuous improvement
system and focuses educators on data collected about the quality of teaching and
learning in classrooms and the use of proven effective practices during
instruction, common planning time, and academic advocacy. MWM was
developed by IRRE to support the comprehensive high school reform framework
First Things First.2 MWM tools and processes have been developed and field‐
tested over IRRE’s ten years of experience supporting high school reform in nine
states. These applications of MWM involved diverse student information systems
and hundreds of district‐ and building‐based educators. In doing this work, IRRE
and its district and school partners learned some hard lessons – all of which have
shaped DWM. Examples include:
Too much data can undermine data‐driven decision making as much as too
2
From 2005 to 2008, MWM was supported by a project grant from the Bill and Melinda Gates Foundation. DWM is
currently underway in two school districts in Texas with support from the Michael & Susan Dell Foundation.
Culture
of
Data
Use
System
Leadership
Data
Governance
Enhanced
Technology
Structures
and
Capacity
little data.
Data must be on time as well as on point.
Data now available to school and district staff are often not actionable. For
example, average daily attendance isn’t actionable; knowing the names of
students who missed more than a day of school last week is actionable.
Teachers and administrators won’t access and use data on a regular basis
to drive their daily work unless they are expected and supported do so.
What
Are
the
Project
Activities?
Doing What Matters is designed to make student outcome data a more central
and powerful driver of dialogue and actions at every level of the system. In this
document we provide a brief description of how IRRE works with district staff
from multiple departments as well as teachers and administrators in participating
schools to craft and then use new tools and processes in order to achieve this
goal. Each of the project activities described below was designed to build upon
and strengthen existing district and school efforts toward this same goal. The
activities fall into two major categories: strengthening, enhancing and targeting
data reporting systems and building system capacity around data‐driven dialogue.
During the first several months of the project, IRRE and our technology partner
Mizuni, Inc., learns about the district’s existing data management and reporting
infrastructure and data use systems and then collaborates with district and school
staff to strengthen these systems so that 1) data integration into a single data
warehouse can occur; 2) new reports of student performance, progress and
behavior are available to users; and 3) customized entry points (i.e., performance
management dashboards) are available for parents, students, teachers and
administrators. District technology staff commit to collaborating with Mizuni to
install new servers and software and have the new data infrastructure in place
within 8 weeks of the project start date. IRRE brings to this process our extensive
experience working in and with schools around data quality and data‐driven
dialogue and the results of our national study of other districts that have
As district and school personnel work with Mizuni staff to create the data
infrastructure and bring new student outcome reports online, IRRE works
intensely with data owners and users to create a coherent data governance
approach that will ensure accurate, timely, actionable and useful data are being
captured in the new data warehouse and student outcome reports. IRRE also
provides districts a planned sequence of professional development opportunities
and several options for varying levels of supports and training to use the new data
system to maximum effect.
Training and support activities by IRRE focus on having district and school leaders
and teacher leaders learn to more effectively facilitate and participate in data‐
driven dialogue focused on student performance and behavior. Data‐driven
dialogue is expected to occur across multiple settings :
Board and cabinet‐level meetings, central office departmental meetings
School‐level teams and groups of administrators and teachers;
Professional learning community meetings (e.g., content area teams, grade
level teams, course level teams)
Teachers with individual students and/or parents.
The purposes of these discussions range from:
Designing and monitoring student‐focused action plans and interventions;
Creating and evaluating targeted professional development activities for
staff;
Planning and reviewing systemic district and building‐wide
improvement efforts.
IRRE offers two foundational training and support activities: 1) four half‐day on‐
site training sessions for district and school leaders to provide focused skills
two 2‐day on‐site coaching sessions for district and school leaders to address
specific challenges being encountered in the implementation of data‐driven
dialogue.
On‐site Skill‐Building Sessions. DWM provides to participating districts four half‐
day training sessions focused on learning a process for engaging in and facilitating
data‐driven dialogue. Participants have multiple opportunities to learn and
practice the process. Sessions focus on how to access and select data reports,
using data reports for data‐driven dialogue and decision making; and gaining
proficiency in using data to identify areas of student performance and behavior
that need improvement. IRRE assists district and building leadership in identifying
areas of concern, identifying potential improvement strategies, creating action
plans and setting targets for on‐going monitoring and evaluation of
implementation of improvement strategies.
On‐site Coaching Sessions. DWM provides one‐on‐one coaching, support and
assistance to participating district and building leaders to address barriers to
implementation of data‐driven dialogue. IRRE leaders spend time meeting with
senior leaders, teachers, and principals on creating and sustaining the conditions
that allow educators time to study, engage in collaborative dialogue, and act upon
data.
Additional
DWM
Supports
Available
Districts can choose to expand their measurement and reporting of student
outcomes and increase the capacity of district and school administrators to
engage in data‐driven dialogue with the following three options:
Implementation of student and staff surveys measuring behaviors, supports
and experiences known to be important for student academic success;
Capture, analysis and presentation of historical trends for key outcomes to
establish a baseline before new improvement strategies are implemented;
Participation in DWM Network Meetings.
Student and Staff Surveys. Districts may elect to participate in IRRE’s student and
staff surveys that measure important constructs shown to predict proximal and
distal student outcomes. A number of validated scales offer districts additional
information about student engagement, teacher expectations and support, peer
support, and quality of relationships. Staff surveys allow districts to gain valuable
information about teacher engagement, experience of support, commitment, and
use of data to make decisions. Districts will receive access to IRRE’s proprietary
web‐based surveys and be able to access reports immediately after the survey
windows have closed. Available reports provide item and construct‐level results
disaggregated at multiple organizational levels.
Capture, Analysis and Presentation of Historical Trends. District research and
technology staff work closely with IRRE’s Research and Measurement group to
secure historical achievement, attendance, discipline, and graduation data to
construct long‐term trends and to set a baseline before new improvement
strategies are implemented. Districts can then track the impact of data‐driven
decision‐making on instructional practice and ultimately on student outcomes.
DWM Network Meetings. Participating districts may elect to participate in one‐
day network meetings held each year. Participants will have the opportunity to
learn from national thought leaders and practitioners successfully implementing
data‐driven dialogue in their districts and schools. Past DWM Network Meetings
have included speakers from the Data Quality Campaign and invited guests from
the Bill and Melinda Gates Foundation, the Michael & Susan Dell Foundation, and
others. Attendees will participate in:
Interactive simulations and demonstrations of data‐driven dialogue using
Opportunities for cross‐site discussions of structures and processes to be
implemented to strengthen data‐driven dialogue around newly
implemented student outcome reports.
For more information please contact:
William Moore, Ph. D.
Director, Research and Measurement Services
