Evaluate Program Implementation
Importance of Program Evaluation
Three main purposes of evaluating the implementation of C&C: 1. Monitoring and ensuring fidelity of implementation 2. Providing information for program improvement 3. Demonstrating program impact
Regular monitoring of C&C core components can help ensure fidelity of implementation when coordinators provide immediate feedback to mentors. Evaluation activities during the first year of implementation can provide vital information for identifying professional development needs of mentors and coordinators, and may determine what, if any, adaptations will be necessary for C&C to be successful in a specific context. Once C&C is fully implemented at a site, an evaluation can demonstrate the impact on student outcomes.
Stages of Program Implementation
The implementation of any new program moves through the following stages:
■ Exploration: Sites explore the intervention or program which will best
meet their needs.
■ Installation: Sites create an implementation team, plan for
implementation, and provide training for leadership teams, coordinators, and mentors.
■ Initial Implementation: Mentors begin working with students.
Coordinators and leadership teams note successes and address barriers. Stakeholders gather data on program implementation for purposes of evaluation. It is at this stage that evaluation is most critical.
■ Full implementation: All processes for C&C implementation are fully
in place and functional, which typically occurs 2-4 years after initial implementation began.
Exploration Stage
Evaluation activities during the exploration stage include:
■ Assessing the needs of the site
■ Identifying the availability of resources
■ Determining the site’s capacity to implement C&C based on organizational
structures, and buy-in from key stakeholders
■ Analyze how C&C will fit with current initiatives
A resource for evaluation activities at this stage is the Hexagon Tool developed by the National Implementation Resource Network (NIRN). This tool can be found on the NIRN website.
http://nirn.fpg.unc.edu/learn-implementation
Installation Stage
Evaluation activities during the installation stage:
■ Define program goals
■ Develop or refine assessment tools/measures » Fidelity of implementation
» Outcomes
During the installation stage, plans are put in place for staffing, training, coaching, organizational support, leadership support, and fidelity measures. Program goals need to be specific, measurable, achievable, relevant, and timely so that outcomes can be documented. Tools to measure fidelity of implementation and outcomes are developed or refined at this stage.
Initial Implementation Stage
Evaluation activities during the initial implementation stage:
■ Fidelity of implementation ■ Program improvement ■ Initial outcomes
Monitoring fidelity is critical during the initial implementation stage. Through monthly monitoring, gaps and barriers to implementation fidelity can be identified and addressed. For example, coordinators review monthly monitoring forms or view mentor activity in the C&C App to provide feedback to mentors.
Evaluating for program improvement with a variety of assessment tools allows coordinators to modify program features and mentor practices and to plan necessary professional development.
Tracking initial outcome data (attendance, behavior, and course performance) helps assess progress towards program goals and report initial successes to stakeholders. Additional strategies for evaluating initial outcomes may include administering the Student Engagement Instrument (SEI), available through the C&C website (see http://checkandconnect.umn.edu/research), and documenting student success stories.
Tools for Evaluating Fidelity of Implementation
Tools for evaluating fidelity of implementation include:
■ C&C core components and elements self-assessment ■ C&C Monthly Fidelity Survey
■ Mentor Practice Profile and Coordinator Functions
The Check & Connect core components and elements self-assessment, on page 128 of Implementing with Fidelity, can be completed by leadership teams throughout the initial implementation stage to track progress towards full implementation. Many sites complete this on a quarterly basis during the first 1-2 years of implementation. The C&C monthly fidelity survey measures to what extent mentors are implementing the core components of C&C. Information from the C&C monthly fidelity survey can be used by leadership teams as they complete the program level self-assessment. The mentor practice profile and coordinator functions can be used as both a self- assessment and a staff performance assessment.
Program Improvement
Discussions with leadership teams and mentors as well as data from self- assessments, fidelity surveys, and practice profiles provide information to help answer the following questions:
■ What are we doing well?
■ What are the obstacles to full implementation with fidelity? ■ What needs to be changed or adjusted?
■ Where should we focus our professional development efforts?
Identifying successful practices increases mentor momentum and promotes an environment for sustaining and scaling-up implementation. Identifying obstacles will lead to discussions regarding what needs to be changed or adjusted so that C&C can be implemented with fidelity. Difficulties in implementation may also be related to professional development needs of mentors and coordinators.
Full Implementation Stage
Evaluation activities during the full implementation stage:
■ Fidelity of implementation ■ Program improvement ■ Program outcomes/impact
C&C is being fully implemented when monitoring and support systems are in place for the following:
■ Recruitment and selection of mentors ■ Initial and follow-up training for mentors ■ Coaching processes and data collection
■ Outcome data measures and reporting processes ■ Fidelity measures and reporting processes
At this stage of implementation, program outcomes and impact can effectively be measured. The impact of C&C can be illustrated by achieving program outcomes (i.e., related to program goals) as well the impact on teachers, school culture, and families. Measuring and evaluating program impact may include using teacher and family surveys, documenting individual student stories, and assessing school climate.
In addition, fidelity of implementation continues to be measured and the data are used for continuous improvement of C&C implementation.
Program Planning
Step 9: Monitor the Person-Environment Fit
■ Who are the key contact people that will support mentors with
concerns about person-environment fit?
■ What steps should a mentor take if they are concerned about
policies or practices in the school?
Step 10: Provide Mentor Support and Supervision
■ What will be the hierarchy of coordination at your site?
■ Who will be the coordinator(s)?
■ How will the coordinator monitor the fidelity of implementation
of Check & Connect?
■ How will the coordinator collect and analyze data?
■ How often will coordinators meet with mentors? Individually?
As a whole group?
■ What professional development might the coordinator provide
or arrange for mentors?
Step 11: Evaluate Program Implementation
■ How and when will you evaluate program fidelity?
■ How and when will you evaluate program impact?
■ How will you share your progress with others?
Goal: What do you hope to accomplish?
Measure: What data or measure will you use to tell if you’ve met your goal?
1.
2.
3.
Check & Connect Elevator Speech
Instructions: Pretend you are in an elevator and someone asks you, “What is Check & Connect?” You have 30 seconds to describe C&C to the other person. What would you say?Resources
See the next several pages for resources to help with your introductions to families and building relationship with students. These are all available electronically on our training documents website at http://z.umn.edu/ CnCforms We encourage you to download and adapt these and other resources to meet your needs (*Available in Spanish) —
■ C&C Brochure Template*
■ C&C Parent/Guardian Consent Form * ■ C&C Introduction Letter to Parents * ■ Getting Started – Relationship Building ■ Student Engagement Plan
What is Check & Connect?
Check & Connect is a model of ongoing, systematic intervention and support to promote students’ engagement at school and with learning.
Each student participating in Check & Connect is assigned a mentor. Check & Connect mentors seek to build long-term, sustained relationships with participating students that will connect them with their school and help them make academic progress.
Key Features of Check & Connect
■ Preventive: Check & Connect focuses
on watching for signs of withdrawal and providing timely interventions to keep students engaged.
■ Builds capacity: Check & Connect works
with students to build new skills such as problem solving and goal setting.
■ Efficient and adaptable: Check & Connect
builds on existing programs, services, and resources within the home, school, and community.
■ Provides outreach: Check & Connect
creates long-term trusting connections between schools, youth, and families.
■ Individualized: Check & Connect uses a
systematic and organized approach, focusing on the needs of each individual student.
Check & Connect in [insert District/School/ Organization]
[Insert D/S/O name] will begin implementing Check & Connect in [insert month/year] to help [insert grade] grade students be successful in school.
Participating students will receive Check & Connect [insert time frame such as, “for at least 2 years and/or until the end of their senior year”, etc. …]
Students participating in Check & Connect will [insert expectations such as, “meet with their mentors at least weekly,” etc. …]
Parents/guardians whose students are participating in Check & Connect will [insert expectations such as, “receive regular updates from their child’s mentor,” etc. …]
Check & Connect is a nationally-recognized program with proven success. Of the dropout prevention interventions reviewed by the U.S. Department of Education’s What Works Clearinghouse, Check & Connect is the only program found to have strong evidence of positive effects on staying in school.
Check & Connect was developed through a partnership between the University of Minnesota’s Institute on Community Integration and Minneapolis Public Schools. Since its inception in 1990, Check & Connect has been studied and implemented with K-12 students with and without disabilities, in suburban and urban settings. Check & Connect is currently being implemented by schools and community organizations across the U.S. as well as abroad.
For more information, contact your site’s Check & Connect coordinator (see front of brochure) or visit checkandconnect.umn. edu
Check & Connect works with
students by…
■ Building a persistent, positive, and
trusting relationship with students and their families;
■ Checking and tracking situations which
may put a student at risk of failing classes or leaving school, such as excessive absences, behavior referrals, and poor grades;
■ Connecting with students to find ways
to address disengaged behaviors and help students re-connect to school and academics;
■ Teaching students skills aimed at
improving academic success, such as problem solving and goal setting ; and
■ Facilitating home-school collaboration to
ensure students’ educational success.
district/school/organization name]
Name: Phone: E-mail: Fax: [repeat as needed] ORYour child’s Check & Connect mentor:
Name:
Phone: E-mail:
Fax:
Check & Connect Coordinator
Name:
Phone:
E-mail:
Fax:
Check & Connect is a comprehensive student engagement intervention developed at the Institute on Community Integration, University of Minnesota, and implemented by schools nationwide. For more
information, visit checkandconnect.umn.edu
The University of Minnesota is an equal opportunity educator and employer.
Student’s Name School
Parent/Guardian Name
Mentor
I hereby give my permission for my son/daughter to participate in the Check &
Connect Program, a mentoring program that helps to promote my child’s educational success. I understand that the mentor will meet with my child at least once a week during the school day, and more often if needed, to discuss attendance, academic progress, behavior, and the importance of education. The mentor and my child will also engage in problem solving, goal setting, and skill development to help my child to be more successful in school. I also understand that the mentor may contact me and my child at home by phone or email to discuss school progress, answer questions, and share resources that may be helpful.
Parent/Guardian Signature Date
Daytime phone number Email address
Check & Connect is a comprehensive student engagement intervention developed at the Institute on Community Integration, University of Minnesota, and implemented by schools nationwide. For more
information, visit checkandconnect.umn.edu.
The University of Minnesota is an equal opportunity educator and employer. Delete This Image and Replace With Your Own
[School Name]
[School Mailing Address] [Date]
[Parent/Guardian Name]
[Parent/Guardian Mailing Address]
Dear Parent or Guardian of [Student Full Name],
My name is [Mentor Name] and I work with the Check & Connect program at [School Name]. I will be working with [Student First Name] to help [him/her] be successful in school.
Check & Connect is a mentoring program that helps to promote students’ active participation in school and learning and help them feel connected to school. As a Check & Connect mentor, I will meet individually with [Student First Name] once a week in school. In these meetings we’ll discuss attendance, academic progress, behavior, and the importance of education. For areas of concern, we will problem solve together and look for possible solutions. I will also help [Student First Name] access services in the school and community that may help [him/her] to be more successful in school.
You are important to [Student First Name]’s school experience, and so I look forward to working with you to support your child’s school success. I will contact you on a regular basis to share information about [Student First Name]’s progress in school. I invite you to share the valuable knowledge you have about your child and to share your questions and concerns whenever you have them. Please expect a call from me in the next few days to discuss your goals for your child and to set up a communication plan.
I am excited to work with [Student First Name] and you this school year. Together, we can help make this a great school year for [Student First Name]! My phone number is [Mentor Phone] and my email address is [Mentor Email]. Please call or email anytime with questions or concerns. I look forward to talking with you soon!
Sincerely,
[Mentor Signature] [Mentor Name]
Check & Connect Program [School Name]
Stages of Relationship Development
All relationships go through stages or phases of development. This table is meant to help you think about your relationship from this point to the time when your role as mentor ends. (See Implementing With Fidelity, p. 97 for strengths-based interview tips.)
Stage Conceptual Features C&C Program Practices
Contemplation Anticipating and
preparing for the relationship
■ Recruiting, screening, and training
mentors
Initiation Beginning the
relationship and becoming acquainted ■ Matching mentors-students ■ Making introductions Growth & Maintenance
Meeting regularly and establishing patterns of interaction
■ Mentors connect with students at least
weekly
■ C&C coordinators provide supervision,
support, and ongoing training for mentors Decline & Dissolution Addressing challenges to relationship or ending relationship
■ Reduction in the importance or
level of closeness in the mentoring relationship (not necessarily negative!)
■ Ending the formal C&C mentoring
relationship
Redefinition Negotiating terms
of future contact or rejuvenating relationship
■ After two years or determined
completion of the program, mentors facilitate closure and transition
■ Conclude formal “checking &
connecting”
■ Agree on how the relationship will
Getting Started Relationship-Building
Confidentiality
Confidentiality is absolutely critical for
establishing trust. Students will not trust their mentor unless they believe that the mentor is trustworthy. Mentors discuss confidentiality with the student. They abide by the principle of who “needs to know.” It is important to note that mentors are mandated reporters. Should it be necessary for mentors to report a situation, they would explain the circumstance to the student.
Respect and expectation
Mentors hold clear expectations for students. They believe that students can be engaged learners and attain personal goals. The emphasis is on what is wanted from the student, not what the student has done wrong in the past. Mentors provide the supports (or see that they are provided) needed by the student to reach the stated expectation. Respect for the student is characterized by ensuring that there are supports for students to reach academic and social expectations for school performance. Role modeling
Quite simply, the mentor models the desired behavior for the student. In particular,
mentors model how to problem solve. They remain calm and professional, talk through situations, and use think-aloud procedures for both academic and behavioral concerns. Mentors model optimism — there is a way to figure this situation out and to advance the student toward greater productivity in the school setting.
Quality relationships cannot be forced; they occur naturally. Mentors need to realize that relationships will happen over time; some will develop quickly while others may take a while to establish. Ongoing positive, solution- oriented communication and outreach to stu- dents influences the relationship.
Relationship-Building Characteristics
The following set of characteristics describes how Check & Connect mentors build these trusting relationships:
Acceptance and accountability
Mentors are nonjudgmental and use non- blaming communication. They accept the student where where they are. There is no at- tempt to “fix the student.” Rather, the focus is on improving the educational performance of the student. Mentors accept students, but do not necessarily condone their behavior. They hold students accountable for their actions using a problem solving dialogue and action planning. Mentors clearly articulate expecta- tions for student behavior and model them via follow-through with the student.
Commitment and persistence
Mentors realize that trusting relationships require hanging in there with the student — even when the student’s behavior is not changing or the student seems direction- less. Mentors do not give up; they continue to problem solve, develop action plans, and communicate with the student. Persistence is integral to the relationship.
Relationship-Building Tips
The first few conversations with your mentees are all about building trust with the student and establishing a positive relationship.
■ Rather than providing a script for
the conversation, these conversation starters below are just ideas for getting to know your students and to spark a conversation.
■ You need not stick to the questions
provided or get to all questions. Try to keep the conversation flowing naturally, and if the student wants to talk more about a particular topic, spend some time exploring that subject with them.
■ Provide as much information about
yourself as you are comfortable sharing. If you ask a student a question, be prepared to answer it yourself. Establish the two-way nature of the relationship early on.
■ Keep conversations positive,
particularly the first few times you talk with a student.
■ It’s OK to get to challenges the student
faces, but don’t allow rumination on these challenges. Establish a positive, optimistic tone from the beginning.
Conversations about interests
■ What do you like to do in your free
time?
■ What are some of your hobbies?
■ What extracurricular activities are
you involved in, in school or outside of school?
■ What books do you like?
■ What music do you like?
■ What television shows or movies do
you like?