ExCEL After School Programs: Evaluation

(1)

San Francisco ExCEL Programs 2013-14

Critical Investments

in Support of Continuous

Quality Improvement

(2)

ACKNOWLEDGEMENTS

We would like to acknowledge the following people for their active contribution to the 2013-14 evaluation and report:

We are grateful to the ExCEL office and additional employees of San Francisco Unified School District for their support of this evaluation.

Mele Lau-Smith, Executive Director of Family Engagement and Community Partnerships

Karina Henriquez, ExCEL Program Administrator Lucy Hong, ExCEL Fiscal and Program Lead Tanya Avila, Quality Improvement Consultant Amy Adkins, Health and Wellness Specialist, ExCEL Jovanna Badger, ExCEL District Coordinator

Joyce Cheung, ExCEL Budget Administrator Jeanette Coleman, ExCEL Support Staff

Kitty Ou, Senior Statistician, Research, Planning and Accountability Department Karen Polk, ExCEL District Coordinator

Jennifer Quevedo-Serrano, ExCEL District Coordinator Alice Tam, ExCEL District Coordinator

Wendy Tran, ExCEL Support Staff

The Principals, Academic Liaisons and school day staff at the ExCEL sites throughout the District

We also extend our thanks to the Agency Directors, Site Coordinators, and Line Staff of the community agencies that provide services to the children and youth of San Francisco every day. Their active participation in the evaluation is key to the success of this report.

Public Profit Evaluation Team

Jessica Manta-Meyer, Senior Research Associate and Project Lead Linda Lu, Research Associate

Emily Salvaterra, Research Assistant Saili Willis, Project Assistant

Lindsay Cattell, Graduate Intern Justine Wolitzer, Graduate Intern

(3)

KEY FINDINGS

In 2013-14, ExCEL continued its critical investments to improve the quality of ExCEL after school programs with the goal of achieving improved student outcomes for participants. These investments included implementing Restorative Practices,

development of Quality Action Plans based on program evaluation data, and redefining the role of the Academic Liaison, or school day staff assigned to support the after school program and facilitate the programs’ communication with the rest of the school

community.

The evaluation found evidence that these investments may have had an impact on the youth and the programs based on youth reports of bullying and on school day alignment, school engagement, and academic behaviors. Moreover, there is evidence that

participation in ExCEL programs has a modest relationship to school day attendance and chronic absenteeism, as well as on English Learner redesignation.

Specific findings include:

• ExCEL participants attended more days, on average, in 2013-14 than in 2012-13 or 2011-12. Elementary participants attended 136 days in 2013-14 compared to 123 days in 2011-12. High school participants attended 29 program days compared to 24 in 2011-12.

• ExCEL programs may increase school day attendance, support English Learner redesignation, and reduce chronic absenteeism.

• Students who attend more days of ExCEL also attend more regular school days. Specifically, the average ExCEL participant attended 0.54 more school days per year as compared non-participants, a finding driven by middle school participants who attended 0.72 more school days per year compared to non-participants.

• Among ExCEL participants, each extra day attending ExCEL was associated with a higher likelihood of being redesignated in the spring semester. For each extra day of ExCEL attendance, English Learner students increased their likelihood of being redesignated by about 1.006 times. This result was consistent across grade levels and when controlling for gender and race/ethnicity.

• ExCEL programs have successfully implemented strategies to support restorative practices. Over two-thirds of youth (73%) report participating in restorative circles and 76% report that staff used the restorative questions with them (when applicable). • ExCEL participants continue to report positive relationships with adults in the

program and their peers (nearly 90%). However, some youth (about 20%) still do not report opportunities for leadership in their program.

(6)

• The rates of reported bullying dropped, possibly because of the introduction of Restorative Practices in most ExCEL programs. Fewer than 2 in 10 youth (17%) report they have had mean rumors or lies spread about them, while only 14% report they have been hit or pushed in their programs more than once. Although this is a marked improvement over the previous year where nearly 30% of youth reported verbal or physical bullying, it is still a point of concern.

• High school youth generally rate their programs of higher quality and report better outcomes compared to their younger peers. However, only 1,088 high school youth completed a survey although nearly 9,000 youth participated in high school

programs at some point for an effective response rate of only 12%. This suggests that ExCEL may wish to increase input from high school youth to get a more

representative picture.

• Across several dimensions, English Learners are more likely to rate their programs of higher quality and report better outcomes compared to their peers. Similarly,

participants who join the program because they want to join are more likely to rate the programs and their outcomes higher compared to their peers who joined the program because their parents or teachers told them to join.

• Youth report both increased school engagement and improved academic skills and strategies, both of which are key contributors to academic success. However, some ExCEL programs need to continue to improve in these critical areas.

• Staff survey results point to a pattern of modest after school - school day alignment. ExCEL may wish to continue to prioritize supports for this alignment as it will likely lead to additional positive outcomes for the youth and the programs

• Parents of ExCEL programs feel respected by staff, but continue to be uncertain about their opportunities to participate. Nearly 2 in 10 (17%) parents report that they do not know whether there are opportunities for parent participation in ExCEL. .

(7)

ABOUT SAN FRANCISCO EXCEL

San Francisco ExCEL is the after school programs office of the San Francisco Unified School District (SFUSD), administering and monitoring federal and state funding for school-based after school programs and aligning programming with district goals for student success.

The ExCEL after school program office is part of the Student, Family and Community Support Department (SFCSD) which supports students and families, by building school capacity to address student needs, and working to achieve District-wide academic goals. San Francisco ExCEL seeks to increase student achievement and pro-social student behaviors so all students can become high achieving and joyful learners. ExCEL seeks to reach these goals by providing comprehensive after school programs, which consist of a mandated mix of academic, recreational/physical, and enrichment components that are open to all students at the school site for low or no cost.

Principals select local community-based organizations to serve as Lead Agencies through a contract with the District. Each agency has demonstrated experience in developing and implementing after school programs and established relationships in their respective communities.

In the 2013-14 school year, 22 community-based organizations operated ExCEL

programs in 89 schools throughout San Francisco. This included 61 programs that serve elementary aged youth, 19 that serve grades 6-8, and 16 that serve high school youth.1

(8)

Theory of Action

Existing research suggests that young people who come to high quality out-of-school time programs often are most likely to demonstrate positive outcomes in a variety of dimensions, including a wide range of socio-emotional and youth development skills, engagement with school, and improved academic skills and performance.

Research also shows that youth who regularly attend high quality programs are more likely to demonstrate strong relationships with peers and with a caring adult, develop new skills, experience new aspects of the world around them, be safer, and be more engaged in school. These positive changes in turn support other positive outcomes for youth, such as enhanced school performance.

Young people are most likely to benefit from San Francisco ExCEL programs, therefore, when they regularly attend a high quality program. In this way, San Francisco Unified School District’s ExCEL programs can serve as a “launching pad” for student success, providing a unique program where young people learn and practice important skills and gain new experiences.

The figure below provides a visual model of the ways in which San Francisco ExCEL programs contribute to positive outcomes for young people. This Theory of Action informs the evaluation design and the organization of this report.

FIGURE 1. THEORY OF ACTION FOR SAN FRANCISCO EXCEL PROGRAMS 2013-14

Regular Participation Program Attendance In High Quality Programs Safe Environment Supportive Adults Positive Peer interactions Promoting Diversity Opportunities to Lead Support for Skill Building

School-day aligned

Has Direct Benefits for Youth

New skills and experiences Improved academic behaviors School engagement Contributes to Other Positive Outcomes School Day Attendance English fluency Academic performance

(9)

San Francisco ExCEL programs share common goals. Each program must meet grant-stipulated performance requirements, including enrollment and attendance. Each program also strives to provide youth with an environment that is safe, supportive, engaging, and interactive, and to have a direct impact on student’s academic behaviors and school engagement that supports the District’s goal of graduating students who are career and college ready.

In addition to sharing these goals, the programs share a common framework through which progress toward their goals can be measured. The San Francisco ExCEL evaluation combines multiple data sources to explore the extent to which sites are meeting program performance goals, providing high quality services for children and youth, and demonstrating benefits for participants.

What makes a quality program?

San Francisco ExCEL defines quality programs in alignment with the Expanded

Learning Collaborative, a city-wide effort to increase access to and the quality of out-of-school time opportunities for young people throughout San Francisco. These quality elements include:

• Safe Environment: Program spaces are physically safe and free of hazards. Moreover, programs are emotionally safe. Youth do not experience either verbal or physical bullying. Programs develop and follow restorative practices through circles, restorative questions and other program elements as a way to respond to conflict and inappropriate behavior and as a way to foster a positive emotional and social climate throughout the program.

• Positive Relationships: Programs develop strong, positive youth-adult

relationships between youth participants and adult program leaders. Programs

support positive peer interaction and the development of pro-social skills. Finally, programs build strong communities by promoting respect for diversity through both program norms and behaviors and programming that acknowledges and explores issues of difference and social justice.

• Opportunities to Learn and Lead: Programs provide strong skill-building and learning opportunities through enrichment and academic activities that move

beyond homework help. Participants have opportunities to make meaningful

choices within activities and about the program as a whole. The program provides leadership opportunities for all participants.

(10)

The table below summarizes the key data sources by report section. TABLE 1. KEY DATA SOURCES BY REPORT SECTION

Report Section Measures Data Source

Program

Participation Counts and characteristics of youth served Average days attended, attendance rates

Unique attendances (participant-days)

EMS (ExCEL’s Cityspan attendance system)

California Department of Education’s Dataquest SFUSD demographic data on participants

Program

Quality Safe environment, restorative practices Supportive adults, positive peer relationships, support for diversity Engaging learning and leadership opportunities

Experience of program quality by key participant populations

Aligning with the school day Engaging families

Youth, parent and key staff surveys SFUSD demographic data on

participants

Direct

Outcomes New skills and experiences _{Increased school engagement} Improved academic behaviors

Experience of program quality by key participant populations

Youth, parent and key staff surveys SFUSD demographic data on

participants

Contributory Outcomes

Increased school day attendance, decreased chronic absenteeism, English learner redesignation rates, CAHSEE pass rates

SFUSD academic data on

participants and non-participants at school sites with an ExCEL program

(11)

PROGRAM PARTICIPATION

ExCEL programs served a more vulnerable population

compared to the District as a whole.

ExCEL programs served 21,967 youth during the 2013-14 school year, or 38% of all students enrolled in SFUSD that year. These youth were generally similar to the SFUSD population, although there were some notable differences.

ExCEL programs served a greater proportion of youth who were English Leaners and who were eligible for free or reduced price lunch, compared to the District. More than one-third (34%) of ExCEL participants were English Learners, compared to 28% of SFUSD. Similarly, 65%% of ExCEL participants were eligible for free or reduced price lunch, compared to 61% of SFUSD students. ExCEL programs served a similar

proportion of students with Special Needs (11%) compared to the District (12%).

Moreover, ExCEL programs served a greater proportion of students from high-need zip codes as determined by the Department of Children, Youth and Their Families in the most recent Children’s Services Allocation Plan.2_{In 2013-14, 52% of ExCEL participants} were from the Mission/Bernal Heights (94110), Outer Mission/Excelsior/Ingleside (94112), Bayview/Hunters Point (94124), and Visitacion Valley (94134) (Figure 2), whereas only 47% of all District students were from these neighborhoods.

FIGURE 2. PROPORTION OF EXCEL PARTICIPANTS BY HOME ZIP CODE

Source: San Francisco ExCEL’s Cityspan (EMS) attendance records for 22,498 youth; 568 participants with zip codes outside of San Francisco were not included in this map. Darker green indicates zip codes where a larger numbers of ExCEL participants live.

2_{San Francisco Department of Children, Youth and Their Families, (May 2013), Children’s Services} Allocation Plan 2013-2016, prepared by LFA Group.

(12)

ExCEL programs also served a greater proportion of youth who scored below proficient on the California Standards Tests (CSTs) in the prior year.3_{Half of all ExCEL}

participants scored below proficient on the English Language Arts (ELA) CST and over half (53%) on the Math CST. By comparison, only 39% of all District students scored below proficient on the ELA CST, and only 42% on the Math CST. ExCEL programs served an even greater proportion, compared to the District, of students who are scored particularly low on the CSTs (below basic or far below basic).

ExCEL programs served a greater proportion of African American and Latino students compared to the District, and a lower proportion of Asian and White students. In other words, Latino and African American students were somewhat over-represented in ExCEL programs, while White and Asian students were under-represented, as compared to the District. In particular, nearly one-third (31%) of ExCEL participants are Latino, compared to 27% of District students.

Taken together, this suggests that ExCEL programs may be a critical venue where the District can reach those students who are, or are likely to be, on the wrong side of the achievement gap.

(13)

ExCEL programs serve youth with both internal and

external motivation to participate

Youth who attend ExCEL programs do so for a variety of reasons. Some may join because an adult (e.g., a parent, teacher or principal) in their life wanted them to. Others may join because their friends are in the program. Still others may join out of an internal motivation to do so.

For younger youth in ExCEL programs, a majority (57%) joined because their parents wanted them to. One-quarter (25%) reported that it was their idea to join the ExCEL program.

The proportion of youth who are internally motivated to join the ExCEL program

increases among high school youth. More than half (56%) of youth in high school ExCEL programs joined because they themselves wanted to join. Still, 19% of high school youth report joining because an adult wanted them to join.

TABLE 2. AS THEY GET OLDER, YOUTH ARE INCREASINGLY INTERNALLY MOTIVATED TO ATTEND EXCEL PROGRAMS.

Grades 3-5 n=3,534 Grades 6-8 n=1,622 Grades 9-12 n=1,088 All grades n=6,244

I joined this program because my parents wanted me to. -- 57% 9% 38% I joined this program because my friends wanted me to. -- 8% 9% 8% I joined this program because my teacher or principal

wanted me to. -- 2% 10% 5%

I joined this program for another reason. 9% 17% 12% Overall: External motivation to join program (I joined

because of parents, friends, teacher/principal, or other) -- 75% 44% 63% Overall: Internal motivation to join program (I joined

because I wanted to) -- 25% 56% 37%

(14)

Youth attended regularly in the early grades, but regular

participation dropped off in the older grades

Research shows that when youth attend structured, high quality programs regularly, they reap the intended benefits. Youth do not need to attend a specific number of days – there is no magic attendance number. However, research does suggest that at about 100 days, young people are more likely to benefit from participating in the program.

For younger youth in ExCEL programs – those in grades K-5 – about 78% attend 100 days or more. On average, elementary-aged youth attend 136 days. For middle school youth – those in grades 6-8 – the average number of program days attended is 90 (Figure 3).

Among high school youth, only 5% attended 100 days or more; the average days attended among high school youth was 29 days.

ExCEL participants attended more days in 2013-14, on average, than in prior years. In particular, youth in the lower grades (K-5) attend about 13 more program days on average than in 2011-12.

FIGURE 3: AVERAGE DAYS ATTENDED BY GRADE LEVEL

Source: San Francisco ExCEL’s Cityspan (EMS) attendance records for 22,498 youth (2013-14), 19,917 (2012-13), 21,875 (2011-12); * High overall average in 2012-13 occurs due to a temporary decrease in the number of high school programs in 2012-13. The jump in the number of high school students, who tend to attend far fewer days than other grade levels, pulled down the overall average in 2013-14.

Asian youth attended more ExCEL program days than any other racial or ethnic group, attending 93 program days on average. Multiracial youth (91), Pacific Islander youth (90) and Filipino youth (89) attended similarly high numbers of program days on average. White youth tended to attend the fewest program days, attending 77 days on average.

Youth classified as English Learners attended more ExCEL program days on average than their peers. English Learners attended 105 program days on average, whereas youth

86 29 90 136 94 26 85 134 73 24 77 123 0 20 40 60 80 100 120 140 160 180 Overall* High Schools Grades 6-8 Grades K-5

Average days attended 2011-12 Average days attended 2012-13 Average days attended 2013-14

(15)

who are native speakers of English attended 91 days on average and youth redesignated as English Fluent attended 73 days on average.

Middle school and high school programs tend to experience high degrees of turnover; that is, the program serves many youth but retains them in the program for only a small number of days. However, middle school students attended, on average, 68% of the program days between their first and last days attended. In other words, while middle school youth are participating in the program, they tend to attend regularly. For example, a student may attend for only a few months, but attend two-thirds of all program days for those few months.

This suggests that middle school programs may be able to keep youth coming back to the program regularly for a period – they are getting youth to attend at least 3 days a week – but that they may want to look for strategies to keep youth engaged in the program over several months or to follow up with youth when they first drop off in attendance. By comparison, high school youth attend, on average, only 38% of available program days between their first and last day of attendance.

(16)

ExCEL programs met grant requirements

ExCEL after school programs are expected to meet specific attendance targets based on their grant funding amounts. SF ExCEL programs must meet an 85% attendance target (at least 85% of progress toward contracted attendance) established by the California Department of Education (CDE).

Of the 74 elementary, K-8, and middle school ExCEL programs,44 elementary schools, 6 K-8 schools, and 6 middle schools met at least 85% of their total program attendance targets for 2013-14. This is similar to the 2012-13 school year, when 41 elementary schools, 5 K-8 schools, and 7 middle schools met their total program attendance targets. Of those that did not meet the attendance target in 2013-14, the average progress toward contracted program attendance was 75%.

Of the 16 high school ExCEL programs, 13 met at least 85% of their total program

attendance target for the 2013-14 school year. By comparison, in the 2012-13 school year, 4 of the 8 participating high school ExCEL programs met or exceeded their total program attendance target. However, of those programs that did not meet 85% of contracted attendance in 2013-14, the average progress toward program attendance was only 56%, similar to last year’s 57%.

FIGURE 4. PROPORTION OF SITES MEETING ATTENDANCE GOALS DEFINED BY CDE, BY PROGRAM TYPE, 2012-13 AND 2013-14

Source: San Francisco ExCEL’s Cityspan attendance records for 22,498 youth. In 2013-14 Elementary site n=55; K-8 n=6; Middle School n=13; High School n=16; in 2012-13 Elementary site n=54; K-8 n=6; Middle School n=12; High School n=8.

50% 58% 83% 76% 81% 46% 100% 80% 0% 20% 40% 60% 80% 100% HS MS K-8 ES

Met 2013-14 Total Program Attendances Goal Met 2012-13 Total Program Attendances Goal

(17)

PROGRAM QUALITY

[The program at our school] is an incredible program! It is such a valuable

experience for our students between the activities, support, and relationships with the staff and kids. The use of restorative practices and collaborating with day time staff to support behavior and academic needs is awesome and much appreciated.”

– Key Staff (no role selected)

ExCEL programs provide a safe place for students

through the use of restorative practices

The foundation of a successful after school program is an environment in which all students are physically and emotionally safe. Research has demonstrated that students’ perceptions of danger are associated with their school

performance and achievement test scores.4 That is, students perform better in schools where they feel safe. This suggests that students’ lower order needs (e.g., safety and well-being) are essential to address before students can attend to higher order tasks such as learning. Programs that provide a safe environment also allow students to feel a sense of belonging and express themselves in authentic ways. Further, environments that promote physical and emotional safety are conducive to positive peer and staff-student relationships.

In prior years, about 9 in 10 youth reported that they felt safe in their ExCEL program. However, large numbers (as high as 34% in the younger grades, report the range) reported that they experienced bullying in the programs. These

findings were comparable to the school day, suggesting that they reveal a pattern of bullying in the larger school community, not one specific to ExCEL programs.

4_{Ratner, H., Chiodo, L., Covington, C., Sokol, R., Ager, J., & Delaney-Black, V. (2006). “Violence Exposure,} IQ, Academic Performance, and Children's Perception of Safety: Evidence of Protective Effects” in Merrill-Palmer Quarterly: Journal of Developmental Psychology, 52(2), 264-287.

SFUSD Restorative Questions When responding to conflict, a restorative approach consists in asking the people involved these key questions:

1. What happened, and what were you thinking at the time of the incident?

2. What have you thought about since?

3. Who has been affected by what happened, and how?

4. What about this has been the hardest for you?

5. What do you think needs to be done to make things as right as possible?

(18)

In 2013-14, however, the rates of reported bullying dropped, possibly because of the introduction of Restorative Practices in most ExCEL programs (see the next section). Less than 2 in 10 youth (17%) report they have had mean rumors or lies spread about them, while only 14% report they have been hit or pushed in their programs more than once. Although this is a marked improvement over the previous year where nearly 30% of youth reported verbal or physical bullying, it is still a point of concern.

Most youth (nine in ten) report being able to tell an adult in their program if they see bullying occur (Table 2).

Reported incidences of bullying are higher for younger youth compared to older youth. Elementary students report the highest rates of bullying compared to other grade

groups. While less than 10% of high school students report being verbally and physically bullied, more than 15% of elementary and middle school youth report having been harassed. Elementary youth continue to need the most support in reducing bullying as more than 20% report being verbally harassed in their programs.

Reported incidences of bullying are generally higher for males than females. Females are more likely to respond that they feel safe – while 12% of females report having been physically harassed, 16% of males say the same. Moreover, 92% of females report that if they see bullying they can tell an adult, compared to 89% of males.

TABLE 3. SAFETY BY GRADE LEVEL

If I see bullying, I can tell an adult in this program. ✪v 90% 90% 92% 90% Other kids never/rarely hit or push me in this program when

they are not just playing around.* ✪v¤l 84% 85% 94% 86% Other kids at this after school program never/rarely spread

mean rumors or lies about me.* v¤l 79% 85% 92% 83%

✪ Gender difference is statistically significant (p<.05) v Motivation to join is statistically significant (p<.05) ¤ Ethnicity difference is statistically significant (p<.05) l English Learner difference is statistically significant (p<.05)

Source: Youth Surveys, Spring 2014.

*For the actual text of these survey questions, which are presented reversed from how they appear on the survey, see the California Healthy Kids Survey examples at CHKS.WestEd.org.

Males and English Learners were more likely to report bullying compared to English fluent participants (including both native speakers and those who have become English fluent). African American youth were more likely to report bullying compared to their peers while Asian youth are less likely to report bullying compared to their peers.

(19)

ExCEL participants in grades 6 and up indicated whether they joined the program because they wanted to (self-motivated to join the program) or because someone told them to join such as a teacher or parent. Those that were self-motivated were less likely to report being harassed in the program.

Parents overwhelmingly perceived their children’s programs as being safe. Nearly all parents (98%) agreed that their child’s after school program is a safe place. Similarly, 9 in 10 staff agreed that their program is a safe place for their students, and most staff reported that they are equipped to help students when they are bullied.

Results from the 2012-13 San Francisco Unified School District’s California Healthy Kids Surveys (CHKS) indicate that ExCEL elementary-aged youth in 2013-14 report

moderately higher levels of verbal harassment compared to 2012-13 reports from SFUSD. While 52% of elementary-aged youth report never being verbally harassed during their school day, 43% of ExCEL elementary-aged youth report the same. ExCEL elementary programs in 2013-14 report slightly lower levels of physical harassment than elementary-aged youth at the District level. Fifty-one percent of ExCEL elementary youth report never being physically harassed in their program compared to 48% for SFUSD. Results from the 2013-14 SFUSD’s CHKS report for middle and high school youth indicate that ExCEL consistently reports moderately lower levels of verbal harassment than SFUSD across all grade groups. As an example, 14% of youth in ExCEL middle schools report being verbally bullied compared to 23% at the District level. Similarly, 8% of high school youth in ExCEL programs report verbal harassment compared to 14% at the District level.

FIGURE 5. VERBAL BULLYING IN MIDDLE AND HIGH SCHOOLS - EXCEL AND DISTRICT COMPARISONS

Source: Youth Surveys, Spring 2014; SFUSD California Healthy Kids Survey, HS, 2013-14.

86% 77% 92% _86% 14% 23% 8% _14% 0% 20% 40% 60% 80% 100%

ExCEL SFUSD ExCEL SFUSD

Middle School

Grades 6-8 Grades 9-12 High School

Never, Rarely Sometimes, Often

(20)

Generally, rates of physical bullying are lower in high schools than middle schools. ExCEL programs report approximately the same rates (6%) of physical harassment as the District (7%). However, ExCEL middle school youth report lower rates of physical harassment (15%) than the District (19%).

FIGURE 6. PHYSICAL BULLYING IN MIDDLE AND HIGH SCHOOLS - EXCEL AND DISTRICT COMPARISONS

Source: Youth Surveys, Spring 2014; SFUSD California Healthy Kids Survey, HS, 2013-14.

Since 2009, the San Francisco Unified School District has been implementing

Restorative Practices (RP) at school sites to respond to bullying and teasing and as an alternative to suspensions and other punitive responses to negative behaviors. In 2011, the ExCEL program, with support from the RP Office, began to bring those program elements into the after school hours.

In part as a response to the bullying rates reported in prior years, ExCEL launched a full Restorative Practices initiative in 2013-14, which charged all ExCEL programs with participating in professional learning cohorts, implementing restorative circles in their programs, and using restorative practices – such as the restorative questions – in response to conflicts or other problem behaviors in the program.

85% _81% 94% 93% 15% _19% 6% 7% 0% 20% 40% 60% 80% 100%

ExCEL SFUSD ExCEL SFUSD

Middle School

Grades 6-8 Grades 9-12 High School

Never, Rarely Sometimes, Often

How many times have you been pushed, shoved, slapped, hit or kicked by someone who wasn't just kidding around?

(21)

Survey results indicate that ExCEL programs have implemented these practices in their programs, especially among elementary programs. Elementary youth reported higher participation in community circles than other grade groups. Participation levels begin to drop as youth get older. While 88% of 3rd_{to 5}th_{graders report having participated in} community circles, 60% of 6th_-8th_{graders and 43% of 9}th_-12th_{graders report the same.} Among the 6th_{through 12}th_{graders, approximately 75% report the adults having used the} restorative questions in a conflict (Table 3).

TABLE 4. RESTORATIVE PRACTICES BY GRADE LEVEL

In this after school program I have participated in

community circles one or more times. l¤ 88% 60% 43% 73%

If I have been in a conflict, the adults in this program have

used the restorative questions. -- 74% 80% 76%

English Learner students are more likely to report participating in community circles; 80% report doing so compared to only 73% of English fluent participants.

There are several contributing factors to reported lower participation for older youth. When program staff were asked what additional technical assistance and professional development they needed, many requested more support for restorative practices. As one staff wrote, “I don’t think the after school staff has been given enough PD on restorative practices.” In addition, ExCEL staff noted that some programs are using other terms for “community circles” so students may not have known exactly what the question meant.

The Restorative Practices Cohorts were GREAT. Please continue this practice and support.” – ExCEL Site Coordinator

(22)

ExCEL programs foster positive relationships –

supportive adults and positive peer relationships – and

respect for diversity

Research on youth development suggests that after school programs provide a valuable context for youth to develop positive relationships with their peers and adults.5 These relationships can provide a foundation that can encourage youth to pursue positive social connections with others outside of the program.

Additionally, opportunities to develop relationships with adults and peers can buffer youth from the negative impact of risk factors (e.g., poverty, living in violent

neighborhoods).6_{Thus, the quality of student-staff and peer relationships is a key} element in providing a supportive environment to students.

After school programs are in a unique position to further the goals of diversity, equity, access and inclusion, such as through structured activities that allow participants to learn about each other and their different cultures as well as through programming that explore critical concepts of diversity and equity such as prejudice and discrimination, justice and respect.7

We love the programs our after school program designed and implements with our students. They are so open to student ideas and bend over backwards to make them successful. The after school staff is professional and show high expectations for the students they serve, while also building close relationships with them.

– School day staff who works in the ExCEL program

5_{Dworkin, J. B., Larson, R., & Hansen, D. 2003. “Adolescents’ accounts of growth experiences in youth} activities” in Journal of Youth and Adolescence 32(1), 17-26.

6_{Masten, A. S., & Coatsworth, J. D. 1998. The development of competence in favorable and unfavorable} environments: Lessons from research on successful children. American Psychologist, 53, 205–220. 7_{California Tomorrow. (2003). Pursuing the Promise: Addressing Equity, Access, and Diversity in After} School and Youth Programs.

(23)

ExCEL programs provide caring adults in participants’ lives

A great number of youth reported the presence of supportive adults in their ExCEL program. Overall, more than 9 in 10 youth (91%) reported that they feel the adults in the program are happy to see them. Youth reported the presence of an adult who cares about them and that adults in the program tell them when they are doing a good job in

similarly positive numbers: 90% and 88%, respectively.

Recognition of the presence of supportive adults seems to vary by age, and is at its highest in high school. For example, 97% of high schoolers reported that the adults in their program are happy to see them, as compared to 91% of elementary age youth and 89% of middle school youth. High schoolers responded more positively than other age groups to the other two related items as well (Table 4).

TABLE 5. PRESENCE OF SUPPORTIVE ADULTS BY GRADE LEVEL

I feel like the adults are happy to see me here. v¤l 91% 89% 97% 91% There is an adult in this program who cares about me. v¤ 90% 94% 92% 90% The adults here tell me when I do a good job. v¤l 87% 87% 94% 88%

Participants who are self-motivated to join the program are much more likely to report supportive adults in the program.8_{English Learners were more likely to report}

supportive adults compared to their English fluent peers. Latino youth were more likely to report supportive adults than their Asian or white peers, and were more likely than their African American peers to report specifically that adults in the program are happy to see them.

Overall, parents are overwhelmingly positive regarding the presence of supportive adults in their child’s ExCEL program. In fact, at all but one site, at least 85% of parents

reported that their child feels comfortable with the after school staff, and that there is an adult their child can talk to about his or her problems. Said one high school parent, “The staff has been really great in making my child feel comfortable there and feel like adults

8_{The question about why a participant joined the program was asked only of those in grades 6 and up} (self-motivated n = 962, externally (self-motivated n=1,646).

(24)

care about her. She's also gotten a lot of great opportunities for jobs and community service through this program.”

Staff agreed. Overall, at least 9 in 10 staff agreed that the adults in their ExCEL program really care about every student (92%), want every student to do their best (93%), and listen to what youth have to say (93%). Across roles, Academic Liaisons, Site

Coordinators, and enrichment providers had the highest overall percentage of positive responses.

[The Site Coordinator] and the ASP staff have amazing and thoughtful

relationships with the students and provide an integral part to students' emotional development and sense of belonging. – Academic Liaison

ExCEL programs can support positive peer relationships

Many but not all youth report that ExCEL programs support the development of positive peer relationships and of community in the program. Over 8 in 10 report that they feel like they belong in the program. Nearly 8 in 10 report that the program helps them make friends and a similar proportion report that most youth respect each other in the

program.

There is significant variation among grade level, with high school participants much more likely to report a sense of belonging, the opportunity to make friends and that youth respect each other (Table 5). Meanwhile, middle school participants are more likely than elementary participants to report a sense of belonging and that most youth in respect each other.

(25)

Moreover, the evidence reveals wide variation among programs themselves. While at some programs, nearly 100% of youth report positive peer climate on all of these measures, at a handful of programs, fewer that two-thirds of youth report positive peer climate. Therefore, some ExCEL programs may want to take additional steps to create community and strengthen opportunities for interaction and relationship building. TABLE 6. POSITIVE PEER CLIMATE BY GRADE LEVEL

At this program, I feel like I belong. v 81% 86% 92% 84% Most youth respect each other in this program. vl 78% 81% 90% 81% This program helps me to make friends. vl 77% 76% 84% 78%

English Learners were more likely to report positive peer interaction and community-building compared to their English fluent peers. Participants who are self-motivated to join the program are much more likely to report positive peer interaction and

community-building.

[This program] gives my daughter more confidence and a greater sense of being connected to her school and community.

(26)

ExCEL programs respect diversity, but need to create more

opportunities to incorporate diversity practices.

Nearly all youth (93%) reported that their programs respect their language and cultures. Staff were also likely to report that their programs respect the language and cultures of their young people. Ninety-one percent (91%) of staff reported that their programs foster an appreciation of student diversity and respect for each other, and 93% feel their

programs emphasize showing respect for all cultural beliefs and practices.

Although most youth agreed that their programs respect their language and cultures, slightly fewer (85%) agreed that they get to do activities with people who are not a part of their normal group, and nearly one-fourth (24%) disagreed that they get to talk to other people about their race, ethnicity, gender or culture (Table 6). Taken together, these results indicate that while programs demonstrate appreciation of diversity, there could be efforts to make different diversity practices more prevalent.

TABLE 7. RESPECT FOR DIVERSITY BY GRADE LEVEL

The people in my after school program respect my language

and culture. v -- 92% 94% 93%

I get to do activities with people who are not part of my

normal group. v -- 85% 86% 85%

I get to talk to other people about my race, ethnicity,

gender or culture. v¤l -- 69% 87% 76% ✪ Gender difference is statistically significant (p<.05) v Motivation to join is statistically significant (p<.05) ¤ Ethnicity difference is statistically significant (p<.05) l English Learner difference is statistically significant (p<.05)

Source: Youth Surveys, Spring 2014; these questions only appeared on surveys for 6th_{grade and up.}

Participants who were self-motivated to join the program were much more likely to report practices the build diversity. English Learners were more likely to report that they get to talk to others about their race, culture, ethnicity or gender compared to their English fluent peers (83% and 75% respectively). African-American and Latino youth were more likely than their peers of other races and ethnicities to report opportunities to talk about elements of their identity with others.

(27)

ExCEL programs provide opportunities for youth to learn

and develop mastery, but could provide stronger

opportunities for youth voice and leadership

Programs that provide engaging activities for youth can increase youth’s sense of

mastery and accomplishment. Through project-based learning, increased challenges over time and collaborative learning, ExCEL programs can build life and work skills, self-confidence and self-efficacy.

Input and choice in programs can increase a sense of ownership, belonging and

community in a program. Coupled with opportunities to reflect on the outcome, making meaningful choices in a program can also develop critical thinking skills and increased self-knowledge.

Age-appropriate leadership opportunities can also provide appropriate challenge, and increase self-confidence and self-efficacy, while also building critical leadership skills such as public speaking, planning, and coordinating a group.

ExCEL programs promote skill-building

Most youth agreed that their after school program provides opportunities to build skills. Ninety-four percent (94%) of youth reported that adults in the program expect them to try hard to do their best, while 81% reported that the program challenges them in a good way. However, only 76% reported that the program has made them better at something they used to think was hard, a relatively low percentage (Table 7).

(28)

Across the board, high school youth had higher percentages of positive responses to these items than elementary or middle school youth, most notably in response to whether the program helps them get better at something they used to think was hard: 84% of high schoolers agreed, as compared to only 74% of younger youth.

TABLE 8. SKILL-BUILDING BY GRADE LEVEL

The adults in this program expect me to try hard to do my

best. v¤l 95% 92% 94% 94%

I am challenged in a good way. v¤l 80% 79% 87% 81% I am better at something I used to think was hard. v¤l 74% 74% 84% 76%

Participants who were self-motivated to join the program, African-American and Latino participants, and English Learners were more likely to report that they receive support for skill-building.

Many, but not all, youth have opportunities to lead and make choices

in ExCEL programs

Overall, many youth, but not all, reported opportunities for leadership in their ExCEL program. Over 8 in 10 youth reported that they get to help other students in their program and just under 8 in 10 reported that their ideas count in the program. Only seven in ten youth (72%) reported that they help to make rules or choose things to do in their program (Table 8).

(29)

TABLE 9. LEARNING AND LEADING BY GRADE LEVEL Grades 3-5 n=3,534 Grades 6-8 n=1,622 Grades 9-12 n=1,088 All grades n=6,244

I get to help other students in this program. 85% 73% 76% 81% I feel like my ideas count here. ¤✪ 80% 76% 88% 80% I help make rules or choose things to do in this program.

¤✪ 74% 69% 80% 73% ✪ Gender difference is statistically significant (p<.05) v Motivation to join is statistically significant (p<.05) ¤ Ethnicity difference is statistically significant (p<.05) l English Learner difference is statistically significant (p<.05)

Females, participants who were self-motivated to join the program and English Learners were more likely to agree that they have leadership opportunities than their peers. Latino youth were more likely than Filipino, Asian, and white youth to report helping make rules or choose things to do in the program, and to report feeling like their ideas count. Parents and staff generally agree with the youth. Among parents, more than 9 in 10 reported that their child’s program provides opportunities to try new things (95%) as well as activities that meet their child’s interests (92%). Just over 8 in 10 parents (84%) and a similar proportion of key staff (86%) report that the ExCEL program provides participants with opportunities to make a difference. Over 9 in 10 key staff reported that the ExCEL program encourages opportunities for students to decide things like class activities or rules.

Staff responses varied by site. In particular, at 19 sites, fewer than 80% of staff reported that ‘This program gives students opportunities to make a difference.’ This suggests that at least some programs can improve leadership and civic opportunities for youth.

The ASP has provided guided homework practice for students as well as

enrichment. Recently, the fun/clubs activities seem to have outweighed the academic and leadership opportunities.” – Academic Liaison

(30)

Alignment with the School Day

Many ExCEL programs support school day goals, but school day and

after school staff should continue to strengthen these partnerships

through regular meetings and support from the Academic Liaison

After school partnerships between schools and

community partners can benefit youth, the CBOs and schools. Specifically, after school programs that intentionally support academic behaviors and learning can improve academic outcomes for participants.9

After school can benefit schools by providing additional caring adults in the lives of its students, supporting a positive school climate through events and performances, provide enrichment, science, art, leadership and recreation not

available during the school day and reinforce academic concepts.10

Finally, CBOs can benefit by having the opportunity to meet their mission to support youth in most need of their services, by

accessing school day resources and expertise to

improve program quality and staff engagement, and by having access to school day information to individually support students.11

During 2013-14, ExCEL transitioned the role of certificated teachers in ExCEL programs from Lead Teachers to Academic Liaisons. This shift moves the teacher role beyond direct instruction to students to intentional collaboration with the ExCEL Site

Coordinator and School Principal to develop academic strategies after school that best support students. The Academic Liaison role includes:

• Using data to determine program offerings for students,

• Developing after school strategies to support student achievement, • Communicating between school day and after school,

• At the high school level, coordinating credit recovery efforts.

9_{Little, Priscilla. “Supporting Student Outcomes Through Expanded Learning Opportunities.” Harvard} Family Research Project. January 2009.

10_Ibid. 11_Ibid.

5 Principles that Support School – After School Partnerships

1. A shared vision for learning and success, with explicit focus on supporting academics

2. Blended staffing models that enable crossover between school and after school staff 3. School-after school partnerships

at multiple levels within the school

4. Regular and reciprocal collection and sharing of information about student progress

5. Intentional and explicit contrast between school and after school environments

(31)

Key ExCEL staff – which includes both District school site staff and after school staff employed by the partner CBO – reported that the ExCEL programs support school goals for academic success and that the CBO and District staff communicate with and respect each other.

However, staff were less likely to report that they follow specific strategies to increase school day alignment. Only 58% of key staff reported that after school and school day staff meet regularly, a key step in developing and forwarding a shared vision. Similarly, only 77% reported that the Academic Liaison – a critical link in the school day alignment chain – provides professional development for staff.

I think the site can improve more with communication overall. I feel that after school and day time staff communicate with each other occasionally. I would like to see more communication, with more effort from the day staff reaching out to the after school staff.” – Program leader working in the ExCEL program

Encouragingly, staff at the same site often agreed, regardless of their role, on the extent to which their site was aligned. For example, when Academic Liaisons reported that they provide professional development for the after school program staff, nearly all of the corresponding principals (88%) and Site Coordinators (93%) report the same.

Similarly, when Academic Liaisons reported that after school staff and school day staff did meet regularly, most of the corresponding principals (77%) and nearly all of the corresponding Site Coordinators (91%) agreed with them.

Of note, the perceptions of Principals, CBO staff and Academic Liaisons did not always align, further suggesting that communication can be improved. For example, while only 56% of Principals believed that after school and school day staff meet regularly, three-fourths of Academic Liaisons and Site Coordinators agreed (76% and 73% respectively). Similarly, while three-fourths of Principals and Academic Liaisons (77% and 74% respectively) reported that the Academic Liaison provides professional development for after school staff, only 63% of Site Coordinators reported the same (Table 9). Notably, 81% of front line CBO staff agree that Academic Liaisons provide professional

development.

At the same time, fewer staff completed the survey than anticipated, limiting the generalizability of the results. While at 13 sites, 15 staff or more completed the survey, several sites had fewer than 5, including 18 sites where no staff completed a survey. Therefore, while these results point to a pattern of modest school day alignment with some areas for growth, ExCEL may wish to ensure broad input from key staff in 2014-15 as school day alignment continues to be a critical goal for ExCEL programs.

(32)

TABLE 10. SCHOOL DAY ALIGNMENT BY STAFF ROLE Principal n=32 Academic Liaison n=48 Site Coordinator n=52 All Respondents* n=697

CBO staff and school day staff support

and treat each other with respect. 97% 96% 94% 95% The after school program supports

school goals for students academic success.

97% 94% 96% 94%

Other school day staff share

information about students with after school (CBO) staff.

94% 89% 96% 87%

After school (CBO) staff share

information about students with school day staff.

94% 91% 94% 86%

The Academic Liaison provides

guidance and strategies to improve the after school academic offerings.

89% 93% 78% 85%

The Academic Liaison facilitates the sharing of information about students in the after school program.

86% 92% 86% 84%

After school (CBO) staff share information about program activities

with school day staff. 94% 93% 92% 84% The Academic Liaison provides

professional development for after school staff.

77% 74% 63% 77%

Other school day staff provide ideas, curriculum, or support for after school activities.

90% 84% 82% 76%

After school staff and school day staff

meet regularly. 56% 76% 73% 58%

Source: Key Staff Surveys, Spring 2014. Only critical leadership roles shown by role. *‘All Respondents’ includes roles not specifically named above.

Our ASP and day staff have forged ahead quite a bit in strengthening our

relations this school year. As a relatively new Principal to this site and having hired an almost entirely new staff in the ASP we have been able to effect much needed change in relationship building among the two staffs. Our SSC has identified the ASP --> School day partnership as a high priority and has set aside funds to allow for additional hours to have ASP staff on site to inform our practices and relationship building to best serve our students. We are on our way and it feels great.” – Elementary School Principal

(33)

Family Engagement

Parents of ExCEL programs feel respected by staff, but are uncertain

about their opportunities to participate.

Parent participation in after school programs can help build a sense of community for both the program and regular school day. Parents who participate may also build skills, resources, and community connections that empower them to provide their children with the support they need to be successful both in and out of school.12

Nearly all parents reported that the after school staff treats their family with respect, recognizes them when they visit, and listen to parents when they have a question or comment. Parents of elementary and middle school-aged youth were more likely to report agreement that they are engaged with program staff, while parents of high school youth were less likely to agree (Table 10).

On a number of parent survey items, a considerable portion of parents chose ‘Don’t Know’ in response to survey statements about the after school program. Most notably, 17% of parents reported that they didn’t know if there was opportunity for parent participation in the program (16% of elementary and middle student parents and 25% high school). This finding indicates that, at all school levels, programs may want to make opportunities for parent participation more explicit.

TABLE 11. PARENTS REPORTED FAMILY ENGAGEMENT

Grades K-8 n=3,725 Grades 9-12 n=292 All grades n=4,017

This program treats our family with respect. 97% 87% 96% At least one after school program staff recognizes me when I visit. 97% 76% 95% The after school staff listen to me when I have a question or comment. 96% 82% 95% This program allows input from parents. 89% 73% 88% There are opportunities for parent participation in this program. 78% 66% 77%

Source: Parent Surveys, Spring 2014.

12_{Lopez, M. Elena & Caspe, Margaret. (June 2014). “Family Engagement in Anywhere, Anytime Learning.”} FINE Newsletter, VI(3). Retrieved from

(34)

I feel that the program does listen to parents and teachers, but it is difficult with so many students and little adults.”

– School day staff who works in the ExCEL program

Parents feel most positively towards programs where their children

feel safe, supported, and prepared academically.

Overall, parents were happy with their child’s ExCEL program. Ninety-eight percent (98%) of parents who completed surveys felt satisfied with their child’s program, and 95% reported their child tells them good things about their program. Parents of K-8th graders reported higher rates of satisfaction, whereas 86% of parents of high school youth reported their child tells them good things about their after school program; however, 11% of those parents also state “don’t know” – indicating a disconnect in communication between parents and older youth about their after school program. TABLE 12. PARENTS REPORTED SATISFACTION

Grades K-8 n=3,725 Grades 9-12 n=292 All grades n=4,017

How satisfied are you with this after school program? 98% 94% 98% My child tells me good things about this after school program. 96% 86% 95%

Source: Parent Surveys, Spring 2014.

There are a few factors that drove parents’ satisfaction. Parents who reported that they were satisfied with the program were more likely to report that the program was a safe place for their child. Parents were also likely to be satisfied with a program where their child felt comfortable with the after school staff. For parents of high school youth, parents were more likely to be satisfied with the program if they also felt that the program prepared their child for college.

The after-school program offers a broad variety of resources for my child. The program staff…are invested in the children [and] create fun programs for the children to participate in. The children understand who to go to with issues, and also to share good things with. Every child is celebrated and included, and that's saying something.

(35)

DIRECT BENEFITS TO YOUTH

High quality after school programs benefit young people directly, by building skills, expanding horizons and supporting positive academic behaviors and school engagement. Through these pathways, after school programs can also contribute to academic success.

Youth get a chance to try new things in their ExCEL

program

High quality after school programs can support learning in a variety of dimensions. In particular, after school can and should provide active and meaningful learning that builds skills and learning that expands horizons through new experiences.13,14

When asked about whether their ExCEL program has provided them with new skills and experiences, many youth reported learning new things in areas that interest and excite them. Overall, 84% of youth reported learning a new skill that they are excited about (Table 12). Elementary school youth and high school youth reported gaining new skills and experiences in larger proportions than did middle school youth.

TABLE 13. NEW SKILLS AND EXPERIENCES BY GRADE LEVEL

In this program, I got to do something that I had not done

before. v¤l 92% 83% 85% 88%

In this program, I learned new information about a topic

that interests me. vl 87% 77% 87% 84%

In this program, I learned a new skill that I’m excited

about. vl 88% 75% 82% 84% ✪ Gender difference is statistically significant (p<.05) v Motivation to join is statistically significant (p<.05) ¤ Ethnicity difference is statistically significant (p<.05) l English Learner difference is statistically significant (p<.05)

Participants who are self-motivated to join the program and English Learners were more likely to report having new experiences and building new skills compared to their peers. Additionally, Latino youth were more likely than their white, African-American, and

13_{Learning in Afterschool Project Position Statement. Retrieved from} http://www.learninginafterschool.org/documents/PositionStatement.pdf.

14_{Birmingham, Jennifer, et al. (November 2005). Shared Features of High-Performing After-School} Programs: A Follow-Up to the TASC Evaluation. Policy Studies Associates. Retrieved from

(36)

Filipino peers to report getting to do something they had not done before. Asian youth were also more likely to report this than were their African-American and white peers.

[The program at our school] is an incredible program! It is such a valuable

experience for our students between the activities, support, and relationships with the staff and kids. The use of restorative practices an collaborating with day time staff to support behavior and academic needs is awesome and much appreciated.”

(37)

Program participants feel more engaged with their

school, a critical step toward improving academic

achievement

Quality after school programs can also help participants feel more engaged in school by increasing their sense of belonging at their school and by encouraging positive attitudes toward school and learning.15_{Higher levels of school engagement have been connected} to other positive academic outcomes, including increased school day attendance, academic performance, and standardized test scores. Lower levels of engagement, however, have been associated with poor school attendance (absenteeism and/or dropping out of school) and negative behavior.16

Youth in ExCEL programs reported that the programs help them feel engaged with their school. Just over 8 in 10 reported that they feel like a part of their school and that the programs helps them care about their school (Table 13).

High school youth were more likely to report high levels of school engagement. However, as with other high school results, it is hard to generalize these findings as very few high school participants completed a survey. On the other hand, middle school youth were somewhat less likely to report school engagement; only 78% respond positively to either survey questions.

Responses varied by site, especially among the elementary and middle school programs. At 8 of the 74 elementary and middle school sites, fewer than 70% reported that the program helps them feel like a part of their school. Similarly, at 10 of these 74 sites, fewer than 70% reported that the program helps them care about their school. On the other hand, at about 10 out of 51 elementary programs, greater than 90% of youth reported increased school engagement.

TABLE 14. SCHOOL ENGAGEMENT BY GRADE LEVEL

This program helps me feel like a part of my school. v¤l 82% 78% 93% 83% This program helps me care about my school. v¤l 80% 78% 90% 81%

15_{Durlak, J. A., & Weissberg, R. P. (2007). The impact of after-school programs that promote} personal and social skills. Chicago, IL: Collaborative for Academic, Social, and Emotional Learning. 16_{Klem, A.M., Connell, J.P. (2004). Relationships Matter: Linking Teacher Support to Student Engagement} and Achievement. Journal of School Health, 74 (7), 262-273.

(38)

Participants who are self-motivated to join the program and English Learners were more likely to report improved school engagement compared to their peers. White youth are statistically significantly less likely than their peers of any other racial or ethnic group to report that their ExCEL program helps them care about or feel like a part of their school. Without additional information, it is hard to determine whether students respond

negatively because they do not feel engaged in school and the program does not help them feel more so or because they already feel engaged in school and the ExCEL program does not change that. However, since alignment with the school day is a top priority for ExCEL, the ExCEL office may wish to support programs in identifying themselves as part of the school community and using program events and

communications to increase the students’ sense of connection between the school day and the ExCEL program.

I have watched students who come into the program that struggle with certain aspects of school and have grown so much academically. They have also gained social skills and have been able to talk about their feelings and try to solve them instead of holding them in. This program really changes students for the better.”

ExCEL After School Programs: Evaluation

San Francisco ExCEL Programs 2013-14

Critical Investments

in Support of Continuous

Quality Improvement

ACKNOWLEDGEMENTS

Public Profit Evaluation Team

Table of Contents

KEY FINDINGS

ABOUT SAN FRANCISCO EXCEL

Theory of Action

What makes a quality program?

PROGRAM PARTICIPATION

ExCEL programs served a more vulnerable population

compared to the District as a whole.

ExCEL programs serve youth with both internal and

external motivation to participate

Youth attended regularly in the early grades, but regular

participation dropped off in the older grades

ExCEL programs met grant requirements

PROGRAM QUALITY

ExCEL programs provide a safe place for students

through the use of restorative practices

ExCEL programs foster positive relationships –

supportive adults and positive peer relationships – and

respect for diversity

ExCEL programs provide caring adults in participants’ lives

ExCEL programs can support positive peer relationships

ExCEL programs respect diversity, but need to create more

opportunities to incorporate diversity practices.

ExCEL programs provide opportunities for youth to learn

and develop mastery, but could provide stronger

opportunities for youth voice and leadership

ExCEL programs promote skill-building

Many, but not all, youth have opportunities to lead and make choices

in ExCEL programs

Alignment with the School Day

Many ExCEL programs support school day goals, but school day and

after school staff should continue to strengthen these partnerships

through regular meetings and support from the Academic Liaison

Family Engagement

Parents of ExCEL programs feel respected by staff, but are uncertain

about their opportunities to participate.

Parents feel most positively towards programs where their children

feel safe, supported, and prepared academically.

DIRECT BENEFITS TO YOUTH

Youth get a chance to try new things in their ExCEL

program

Program participants feel more engaged with their

school, a critical step toward improving academic

achievement