Data Management: Developing Data
Governance Structures
Welcome!
Kathryn Tout, Child Trends
Ivelisse Martinez-Beck, OPRE
INQUIRE Webinar Series
Webinar 1: Overview and Application of the INQUIRE Data Tools
• Completed on March 20, 2013
• Available at http://www.ResearchConnections.org
Webinar 2:Data Management: Developing Data Governance Structures
Webinar 3: Data Management: Best Practices for Producing High Quality Data
Purpose of Webinar #2
To illustrate the need for and benefits of building strong
ECE data governance and system-wide data management policies and practices using the example of QRIS.
Agenda
Background on INQUIRE
Data governance for QRIS
• Challenges to QRIS data quality
• State options for coordinated data systems
• Data governance’s role in producing high quality data
State Perspectives & Applications
• Mississippi
The Quality Initiatives Research and Evaluation
Consortium (INQUIRE)
Consortium of primarily researchers and evaluators who are working on projects related to Quality Rating and Improvement Systems (QRIS) or other quality
improvement initiatives or topics
Purpose of INQUIRE
• Support high quality, policy relevant research and evaluation
• Provide guidance to policymakers on evaluation strategies, new research, interpretation of research results, and
Through OPRE-funded projects and in state QRIS
evaluations, we heard from states and from
evaluators about the need for support on data.
The need for guidance on how to organize and manage the data they are collecting
The need to coordinate the efforts of the different departments and organizations collecting early care and education data
Presenters
Child Trends
• Sarah Friese, Senior Research Analyst
Oregon State University, College of Public Health and Human Sciences
• Bobbie Weber, Research Associate
Frank Porter Graham Child Development Institute, University of North Carolina-Chapel Hill
• Iheoma Iruka, Scientist Mississippi
• Michael Taquino, National Strategic Planning and Analysis Research Center
• Jill Dent, Mississippi Department of Human Services Maryland
What is data governance?
Data governance is the set of business processes, policies, and data management practices that provide guidance on the use of a single data set or compilation of multiple,
related sets. Governance promotes systematic data usage through adherence to uniform data quality and
What is the importance of data governance for the
field of early childhood?
• Early childhood data is often collected by different agencies, housed in different data systems, and managed using
different sets of rules.
• Early childhood data governance allows policymakers and practitioners to share data that describes the population and
Glossary of Terms
DATA SET A collected set of data elements collected for one program or purpose.
DATA SYSTEM A data system is a collection of data sets housed within single or multiple organizations.
Glossary of Terms
COORDINATED DATA SYSTEM A coordinated data system is one where multiple sub-data systems and sets are governed by a central body that provides guidance related to the policies and procedures for handling and sharing data.
INTEGRATED DATA SYSTEM An integrated data system builds on a
coordinated one by also provided direct assistance in the management of individual data sets housed in different sub-data systems.
Challenges to QRIS Data Quality
1. States use data from data systems governed and
administered by multiple agencies and organizations.
• QRIS ratings are generated by drawing on data from a variety of sources.
• Licensing, workforce registry, and subsidy are some of the data sources that are used with QRIS.
Challenges to QRIS Data Quality
2. Differences in database design and practices impede linkages to other data systems.
• Systems (workforce, licensing, etc.) use their own unique identifiers.
• Values my be overridden at time of updates.
• Linking data may be difficult and opportunity for error increased.
Challenges to QRIS Data Quality
3. Data practices often do not support the production of high quality data.
• Data set and system documentation is often limited.
• Departments don’t have established procedures for ensuring data quality and confidentiality and, when they do, they may conflict with those in place in other departments.
Challenges to QRIS Data Quality
4. States typically lack a governance framework for ECE data systems and management.
• Many states do have not established authorities that govern and manage the policies and practices of their QRIS.
• The policies and practices that affect data management may vary across the databases linked to the QRIS.
Challenges to Data Quality
5. States are designing or redesigning their QRIS data systems and are looking for models and guidance.
• We are in a period of rapid change for standards in the design and management of QRIS data systems.
• Now is an ideal time to support states in building QRIS data management capacity.
Moving from Multiple Independent Databases
to an Integrated QRIS Data System
•
In states with QRIS, developing a quality
rating for a program requires linking of data
on the workforce, licensing, and facilities.
•
Linking is facilitated when data is shared in a
coordinated or integrated data system.
Options for a Coordinated or Integrated Data
System Vary On Key Characteristics:
•
Data Quality
•
Data Availability
Types of Coordinated or Integrated Data
Systems
Unlinked Databases or Point Solutions
Benefits
•
Least disruptive in the short run.
Drawbacks
•
Will produce the lowest quality data
Coordinated data systems with linked
customized interfaces
Benefits
• Databases are linked one by one as needed.
Drawbacks
• Data are not based on standards.
• Interfaces are designed ad hoc and require ongoing maintenance.
Federated, shared data system
Benefits
• Data elements needed for QRIS and other purposes are extracted from databases, mapped to standards, linked to master identifiers and stored in shared repository.
• Cross-agency governance is required for shared data, but
individual databases may retain their own governance process.
Drawbacks
Comprehensive, integrated data system
Benefits
• Data silos are eliminated which also reduces the potential for redundancies.
• Data is managed according to uniform standards so quality is high.
Drawbacks
• An investment of time and resources is required including changing data management policies and processes in
Governance Essential Regardless of Data
System Option Selected
• Governance is where stakeholders come together to make decisions about what the vocabulary will be, which
nationally-recognized standard will be used for its
representation, and who will have permission to access the data.
Identifying Governance Body for QRIS Data
System
• Executive Council—sets overall mission and goals, secures funding and resources
• Strategic Committee—develops high-level plan to achieve goals
Tasks of the Governance Body for the ECE Data
System include:
• Produce standard data-sharing agreement.
• Develop documentation for databases in system.
• Have a policy on database updates.
• Ensure data are saved and system changes captured.
• Develop common data standards.
• Determine unique identifiers for children, workforce, & facilities.
State Perspectives &
Applications
STATE PERSPECTIVES
STATE PERSPECTIVES & & APPLICATIONSAPPLICATIONS
MISSISSIPPI
MISSISSIPPI
SIX BUILDING BLOCKS OF
The scope defines the purpose of the
integrated data system and provides the
general framework for supporting and
institutionalizing its use.
Example: Evaluate the effectiveness of the
Allies For Quality Care program. The overall
SCOPE SCOPE
A data stakeholder is an individual or
organization affected by information generated
from an integrated data system and aligned
with the scope
Allies data stakeholders:
Mississippi Department of Human Services
Mississippi Department of Education
DATA STAKEHOLDERS DATA STAKEHOLDERS
Data will only be used for activities
directly related to the scope
Key factors for successful applications:
Determine data availability
Data documentation through data dictionaries or
codebooks
Develop data mapping
APPLICATION APPLICATION
Ability to fulfill scope of integrated data system
• Secure data and access data
Components:
• Create a Center of Excellence through University Partnerships
• Data, technical, and research expertise
• Legal and compliance expertise
• Formal agreements
• Policies and procedures for data lifecycle for state data clearinghouse (data warehouse) or federated system
OPERATIONAL CAPACITY OPERATIONAL CAPACITY
DATA ACCESS
Leadership and Accountability
Who owns, promotes, and oversees the system? Who is responsible for making sure things are done right?
In Mississippi, a governing board provides a single point of
leadership and accountability, and a management board provides technical advice. A center of excellence provides the capacity for the system to operate.
Sustainability
In Mississippi, sustainability has been established through legal authority:
LEADERSHIP & ACCOUNTABILITY LEADERSHIP & ACCOUNTABILITY
Quality Care and Education System for
Maryland
EXCELS
Where are we now?
• Fall 2011-Spring 2012 – Pilot Phase
• Fall 2012-Spring 2013 - Field Test
Maryland EXCELS Website
Maryland EXCELS Data
Connectivity
Specific Maryland EXCELS Data
• Program quality “Check” level
• Data obtained; Date expired – archival by “cycles”
• All reviewers – and level of communication
• Rate of change in quality status
• Specific program supplied evidence demonstrating quality
Types of Questions We Seek to
Address
•
Longitudinal impact on school performance of
children by early-care experience
•
Correlations between quality elements
(accreditation, credentialing, level of technical
assistance, etc.)
Lessons Learned
•
Have a great central data architect
•
Clearly identify needs, then look at data
collection mechanisms
•
Understand the data use from multiple
perspectives
Looking Across the State Examples
The state examples provide an overview of different activities related to data governance.
The examples demonstrate the important connections that are made between data, program monitoring and policy questions of interest in the state.
Next Steps
Upcoming Webinar on Data Management
• May 16, 2013, 2:00-3:30 EST:
Best Practices for Producing High-Quality Data
Webinar recording will be available on Research Connections
Acknowledgements of Contributors to INQUIRE’s Data
Work Group
Rick Brandon, Consultant
Missy Cochenour, AEM
Iheoma Iruka, FPG, University of North Carolina
Tabitha Isner, MN Department of Human Services
Fran Kipnis, Center for the Study of Child Care Employment at UC Berkeley
Lee Kreader, National Center for Children in Poverty
Minh Le, Office of Child Care, ACF
Contact Information
Ivelisse Martinez-Beck, Office of Planning, Research and Evaluation, Administration for Children and Families
Kathryn Tout, Child Trends
