First 5 Sonoma County
Program Evaluation Brief
07/1/2011 - 06/30/2012
AVANCE Parent-Child Education Program
February 2013
Prepared For
First 5 Sonoma County
Prepared By
LFA Group:
Learning for Action
LFA Group: Learning for Action enhances the impact and sustainability of social sector organizations through highly customized research, strategy development, and evaluation services.
Executive Summary
Program Details
Program Name AVANCE Parent-Child Education Program (PCEP)
Contractors Community Action Partnership and Petaluma Adult School
Terms of Grant July 1, 2010 to June 30, 2015
2011-15 Strategic Plan Goal Area Goal 2: Supported Families and Nurturing Families
Priority Outcome Priority Outcome 2B: Strengthen family literacy
Strategic Plan Core Program Outcomes
(First 5 Sonoma County Pathways to Results)
Increase in parent knowledge of child development and parenting skills
Increase in family literacy skills, including parent-child read aloud time
Increase in parent knowledge of and access to community resources
Increase in parent interactions with children and encouragement of language acquisitionFirst 5 Sonoma County Funding
Amount
Community Action Partnership: $$555,000 Petaluma Adult School: $555,000Leveraged Funding Received During
First 5 Grant Period No additional funding has been leveraged
Level of Evidence at Start of Grant Evidence-based Program – Recognized as Tier 1 in the Portfolio of Model Upstream Programs
Key Accomplishments
AVANCE PCEP has achieved three of its four evaluation targets as well as two additional targets during the 2011-12 fiscal year:
Implemented program with fidelity to the AVANCE PCEP model: Three sites have implemented theprogram with fidelity to the AVANCE PCEP model, indicating that the programs are on track to increase parent knowledge of and access to community resources and increase parent interactions with children and encouragement of language acquisition. The fourth site has not yet been rated for fidelity.
Increased participants’ knowledge of child development and parenting skills: 90% of programgraduates reported improvement in their knowledge of child development and parenting skills.
Key Challenges and Lessons Learned
Program Challenges: 63% of graduates reported that they increased the frequency with which they read totheir children aloud, falling short of the 75% annual target.
Program Adjustments: The program experienced lower than hoped for completion/graduation rates, with61% of its participants attending 75% of classes required to graduate in 2011-12, due in part to low enrollment and attendance in an afternoon class. Thirty-three of 43 participants completed the program in 2010-11, making a total of 94 graduates (65%) for the two program years. Program staff may want to consider ways to address their participants’ barriers to regular attendance, such as transportation, work schedule challenges, and family obligations.
This evaluation brief provides an update on AVANCE PCEP progress toward key
evaluation targets including populations served and outcomes achieved during the
reporting period. This brief is intended to be a resource to guide program development
implementation and improvement. It is also intended to inform the First 5 Sonoma County
Commission of the impact of its investments and lessons learned for future funding
decisions.
1Program Description
The AVANCE Parent-Child Education Program (PCEP)
, a 40-year old program headquartered in
San Antonio, Texas,
provides education and support to Latino parents with children under the age of three. It is founded on the belief that a parent is a child’s first and most important teacher, so it is critical to train parents to have the skills to support optimal child development, literacy, and school readiness. The program operates in housing projects, community centers, and schools. AVANCE PCEP instructors guide parents through their children’s stages of physical, cognitive, social, and emotional development with special attention to the importance of reading, nutrition, expressing affection, and effective discipline.Parents attend weekly three-hour workshops for nine months. Community Action Partnership conducts the workshops in Spanish and Petaluma Adult School’s program is presented in a bilingual English and Spanish format. Both programs provide early child education programs. During the first hour, parents make an educational toy using typical household items and are taught to use the toy in activities that will promote their child’s development. Parents receive a “possibility sheet” with suggested activities and vocabulary to use at home. During the second hour, the program covers a section of a 27-lesson sequence based on community college curriculum on early childhood development. During the third hour parents are introduced to an array of community resources, services and providers. Parents receive monthly home visits beginning in the third month. After the first year, parents are encouraged to continue on to classes in literacy, English, obtaining a GED, and enrolling in college or career training.
During the 2011-12 program year, First 5 Sonoma County funded two contractors to administered the AVANCE PCEP:
Community Action Partnership of Sonoma County (Community Action): The federallydesignated anti-poverty agency in Sonoma County that partners with low-income families and individuals to help them achieve economic and social stability, build community, and advocate for social and economic justice. Community Action operated the AVANCE PCEP at:
o Kawana: An elementary school in Santa Rosa
1For a complete description of the First 5 Sonoma County evaluation approach, please see the Evaluation Plan at
http://www.first5sonomacounty.org/documents/evaluation_plan.pdf.
| LFA Group:
| February 2013
2
o The Mountain Shadows campus (Mountain Shadows): A campus comprised of a
Title 1 funded preschool program operated by the YMCA, the district’s alternative education programs and the district administration office.
Petaluma Adult School (PAS): Part of the Petaluma City School Districts, PAS offers anopportunity for students to continue their education on a high school or post-high school level suitable for adults. PAS operated the AVANCE PCEP at Petaluma’s two sister schools serving a combined school attendance area:
o McDowell Elementary School (K-3) o McKinley Elementary School (4-6)
Program Theory
Need for the Program
AVANCE PCEP meets a pressing need for programming that will promote the long-term educational success of Latino children in Sonoma County. Latino children demonstrate lower educational outcomes than their white, non-Latino counterparts; in 2011 only 27% of Latino third graders in Sonoma County scored “proficient” at reading, compared to 61% of white students in Sonoma County.2 In Sonoma County public schools, the number of English Learner (EL) students increased by 52% from 2001 to 2011.3 ”
How the Intervention Links to Outcomes
AVANCE PCEP is an evidence-based program that supports children’s school readiness by strengthening family literacy. A significant body of research has linked the AVANCE PCEP to the core outcomes addressed within this report, as well as to longer-term positive outcomes. The Carnegie Corporation of New York found, in a 1987 evaluation, that AVANCE PCEP mothers provided a more stimulating, organized, and responsive home environment, talked more frequently with their children, spent more time teaching their children, and increased their knowledge of
parenting skills.4 Additionally, a study by Project CAN PREVENT found that the AVANCE PCEP
effectively improved conditions which are often linked to the prevention of child abuse and neglect, such as parents’ self-esteem, social support network, and knowledge of child development.5
As an early childhood intervention program, AVANCE PCEP is well-positioned to support children’s school readiness and achievement. Research shows that the positive effects of parental involvement increase when involvement starts early in a child’s education, and engages parents in working
directly with their children at home.6 The AVANCE PCEP encourages early parental involvement
and promotes family literacy skills that support children’s school readiness. According to a 2006 study conducted by the Dallas chapter of AVANCE, 95% of parents reported that they play developmental games with their children and read to them always or sometimes, while 98% of parents reported discussing their child’s school experiences with them daily.7 Furthermore, Pre-K
2 California Department of Education. (2012). Standardized Testing and Reporting (STAR) Results. Retrieved from
http://star.cde.ca.gov/
3California Department of Education, Educational Demographic Unit. DataQuest EL Report: Time Series Number of
English Learners for Sonoma County.
http://dq.cde.ca.gov/dataquest/lc/NumberElCounty.asp?Level=County&cCode=49&cName=SONOMA&TheYear=2010-11. Report viewed on 2/22/12.
4 As cited in Rodriguez, G. (2003). Connecting with parents: The AVANCE experience. Children’s Research Center at
University of Illinois-Champaign. Retrieved from http://ecap.crc.illinois.edu/pubs/connecting.html. Report viewed on 2/4/13.
5Ibid.
6 Cotton, K., Wikelund, K. Parent Involvement in Education. Northwest Regional Educational Laboratory, School
Improvement Research Series.
7 (2006). Maternal Attitudes and Behaviors Supporting Education. Retrieved from
http://www.avance.org/about/our-impact/research-evaluation/maternal-attitudes-behavioral-study/. Report viewed on 2/4/2013.
| LFA Group:
| February 2013
3
to kindergarten teachers in a 2000 survey reported that AVANCE children are ready for school.8 The teachers reported that 82% of AVANCE children knew their letters, 86% knew their numbers, and 89% could express themselves verbally. Many preschool and kindergarten accomplishments, such as the oral language skills and letter identification skills, reported in AVANCE children, are closely related to “later success in conventional literacy tasks.”9 The AVANCE program’s promotion of long-term academic achievement is further supported by research which indicates that reading difficulties can be prevented by ensuring families and preschools provide experiences for language and literacy development.10
Long-Term Cost Savings
By engaging Spanish-speaking families in activities that support children’s school readiness, AVANCE PCEP will likely generate public sector cost savings by helping to close the achievement gap between white and Latino students. According to the Alliance for Excellent Education, if students of color graduated at the same rate as white students, more than $101 billion would be added to California’s economy by 2020 as a result of an increase in taxable income.11
Reaching the Target Population
AVANCE PCEP is a broadly targeted program designed to mitigate the effects of poverty on children in targeted school areas in Sonoma County. The program provides services for low-income, Latino families with children ages 0- 3 who traditionally enter the school system with limited school-readiness.
Number Served
Programs that serve the neediest families with children ages 0-3 invariably experiences steep attrition rates. Taking this into account, First 5 has set high recruitment and enrollment targets of AVANCE PCEP participants. The program sites have a local target to initially recruit 30 parents with children 0- 3 years of age and ultimately enroll 25 families.12 McDowell (PAS), which enrolled 30 parents, and McKinley (PAS), which enrolled 26 parents met the program target, while Kawana and Rohnert Park (Community Action), which enrolled 22 parents each, fell short of the target. Despite falling short of the enrollment target, the two Community Action sites actually had the highest completion rates among the four sites.
While the program also serves children, it does not have a target number of children each year. During 2011-12, the four sites served a combined total of 131 children; 84 of whom where ages 0-3
and 47 of whom where ages 0-5.13
8 (2000). School Readiness of AVANCE Children. Retrieved from
http://www.avance.org/about/our-impact/research-evaluation/school-readiness-survey. Report viewed on 2/4/2013.
9 Snow, C. E., Burns, M. S., & Griffin, P. (Eds.). (1998). Preventing reading difficulties in young children. Washington, DC:
National Academy Press. As cited in Zygouris-Coe, V. (2000). Emergent literacy. Florida Literacy and Reading Excellence Center at University of Central Florida.
10 Ibid. 11Ibid.
12
For 2012-13, AVANCE National has clarified its definition of enrollment to mean families who attend a minimum of three sessions before the enrollment period ends six weeks into the program year.
13Although the program is designed to serve children 0-3, it also serves the older siblings of the 0-3 children enrolled.
| LFA Group:
| February 2013
4
Exhibit 1: Families Enrolled at all Four Sites (FY 2011-12)
100 0 20 40 60 80 100 120 140 N um be r E nr ol le d CollectiveTarget = 100 Parents/CaregiversRace, Ethnicity and Language of Participants
AVANCE PCEP targets Spanish-speaking, Hispanic/Latino parents. During 2011-12, all parents and almost all children (97%) identified as Hispanic/Latino (Exhibit 2 and 3).
Exhibit 2:
Parents’ Race/Ethnicity
(FY 2011-12)
Exhibit 3:
Children’s Race/Ethnicity
(FY 2011-12)
Hispanic /Latino 100% n=131 n=131 n=96 Multi-Racial 3% Hispanic /Latino 97% n=131 n=131 n=131 Note: Although 100 parents enrolled in the program, program staff collected race/ethnicity data for only 96.
During 2011-12, almost all parents (98%) and almost all children (98%) identified Spanish as their primary language (Exhibit 4 and 5).
Exhibit 4:
Parent’s Primary Language (FY 2011-12)
Exhibit 5:
Children’s Primary Language (FY 2011-12)
Spanish 98% English 2% n=93 Spanish 98% English 1% 1% n=131 OtherNote: Although 100 parents enrolled in the program, program staff collected primary language data for only 96.
Special Needs Population
Of the 131 children served by AVANCE PCEP in 2011-12, eight (6%) have special needs.
Progress Achieved
In implementing the AVANCE program over the first two years of their grant, the Community Action and PAS sites have worked toward meeting the four First 5 Sonoma County Strategic Plan core outcomes that the program was intended to address:
Increase in parent knowledge of child development and parenting skills
Increase in family literacy skills, including parent-child read aloud time
Increase in parent knowledge of and access to community resources
Increase in parent interactions with children and encouragement of language acquisitionProgress toward these outcomes is highlighted in the table below.
Progress Achieved Toward Core Outcomes
1407/01 / 2011 – 06 / 31 / 2012
Core Program Outcome Specific Target Results Actual Toward Target Progress
Increase in parent knowledge of and access to community resources
100% of sites will implement the program
with fidelity to the AVANCE PCEP model, indicating that they are on track to meet these outcomes demonstrated by previous research 100% (3 of 3) sites monitored for fidelity demonstrate fidelity to the model15 Achieved
Increase in parent interactions with children and encouragement of language acquisition
100% of sites will implement the program
with fidelity to the AVANCE PCEP model, indicating that they are on track to meet these outcomes demonstrated by previous research 100% (3 of 3) sites monitored for fidelity demonstrate fidelity to the model16 Achieved
Increase in parent knowledge of child development and parenting skills
75% of parents show improvement in these
areas (57 of 63) 90% Exceeded
Increase in family literacy skills, including parent-child read aloud time
75% of parents show improvement in these
areas (26 of 41)63% 17 Not Achieved
Three sites monitored in 2011-12 implemented the program with fidelity to the AVANCE PCEP model: This indicates that the programs are on track to increase parent knowledge ofand access to community resources, and to increase parent interactions with children along with encouragement of language acquisition. The McDowell site was rated for fidelity in spring 2011 and the program improvements suggested they were implemented in 2011-12.
14
Throughout this brief, progress toward targets is measured using the following definition: Not Achieved (more than 5% below target), On Track (1-5% below target), Achieved (0-5% above target), Exceeded (more than 5% above target).
15
The McDowell, PAS site did not receive a fidelity rating in FY 2011-12 because the national office did not complete a monitoring visit at this site. However, the site achieved fidelity when monitored in FY 2010-11, and the monitor observed in FY 2011-12 that staff conducted home visits at the highest level of fidelity..
16
The McDowell, PAS site did not receive a fidelity rating in FY 2011-12 because the national office did not complete a monitoring visit at this site. However, the site achieved fidelity when monitored in FY 2010-11, and the monitor observed in FY 2011-12 that staff conducted home visits at the highest level of fidelity..
17
This analysis excludes the 22 parents who reported already reading to their child seven days a week at the beginning of the program.
| LFA Group:
| February 2013
7
One of the fathers shared how
AVANCE has changed his
expectations and his family’s life. He
told us that before AVANCE, his focus
was on material things like a house,
car, and technology for his children,
and having money to spend as he
wanted. He didn’t see time spent with
his children as important or a priority.
Now, he values and looks forward to
spending time with his children
playing, reading or just being together.
He knows that what he has learned
has made him a better father and is the
best advantage he can give his family.
Program Staff
AVANCE PCEP participants increased their knowledge of child development and parenting skills. To assess their graduates’ retention of theprogram’s lessons, sites administered the AVANCE Parenting Questionaire (APQ) at the beginning and end of the program. According to the APQ’s results, at the end of the program, 90% of graduates
increased their knowledge of child development and parenting skills, thus exceeding the program target that 75% of parents show improvement in these areas. The pre-post questionnaire measures parents’ retention of the lessons taught during the program’s weekly three hour parent education classes and is therefore used as the measure of improvement.
The program helped increase the frequency with which parents read aloud to their children, but did not reach its target. Parents at each site receivebook bag containing Spanish, English and bilingual board books to support reading at home. This is a
Community Action enhancement of the AVANCE PCEP model that has been adopted across all sites. At the end of the program, 63% of graduates reported that they increased the frequency with which they read to their children aloud, falling short of the annual target that 75% of parents increase their reading to their children. While the parents who reported already reading to their child daily at the beginning of the program were excluded from this
measurement, the single question about daily reading does not adequately measure the change in the quality of book sharing with children by parents.
In addition to the key accomplishments specifically related to First 5 Sonoma County’s Strategic
Plan core outcomes described above, the Community Action and PAS sites also worked to make
progress toward the followiong targets:
Additional Progress Achieved
07 / 01 / 2011 - 06/ 30 / 2012
Program Outcome Specific Target Actual Results Toward Target Progress
Staff trained in AVANCE model 100% of staff attend AVANCE training (20 of 20) 100% Achieved High-need parents are recruited
and enrolled into program
through door-to-door visits in 25 enrolled per site McDowell, PAS 30 enrolled Achieved school attendance areas
High-need parents are recruited and enrolled into program
through door-to-door visits in 25 enrolled per site McKinley, PAS 26 enrolled Achieved school attendance areas
High-need parents are recruited and enrolled into program
through door-to-door visits in 25 enrolled per site Community Action Kawana, 22 enrolled Not Achieved school attendance areas
High-need parents are recruited and enrolled into program
through door-to-door visits in 25 enrolled per site Community Action Mountain Shadows, 22 enrolled Not Achieved school attendance areas
Program Outcome Specific Target Actual Results Toward Target Progress
Program staff address barriers to parent participation to ensure
parents’ weekly attendance and 85% of parents successfully retained and complete the program (61 of 100) 61% Not Achieved retention in the program
Conduct monthly home visits 90% of families receive at least seven home visits At least 4 home visits N/A18
AVANCE PCEP staff focused on continual learning and professional development. All sitestaff received training and were certified in the AVANCE PCEP model at the beginning of FY 2011-12. The day-long training was a review for most attendees, with all ten PAS staff and seven of ten Community Action staff returning for a second year. Community Action followed up this training, with another training that focused on aspects of the AVANCE PCEP model that are unique to each site. Additionally, site staff had many other opportunities to engage in continual learning and professional development. Before the beginning of the program year, Community Action’s and PAS’s Parent Educators attended a train-the-trainer workshop hosted by AVANCE National in Santa Barbara at which they learned how to develop AVANCE program staff and improve implementation at their sites to assure fidelity to the AVANCE PCEP model.
Additionally, Early Childhood Education (ECE) and core program staff at the Community Action sites participated in a training hosted by an EL education specialist with expertise on how to intentionally encourage English and Spanish language development. Furthermore, two PAS staff members received training and certification on the Pasitos model, a program that engages Spanish-speaking parents and their 3- and 4-year old children in activities that encourage language and literacy development to prepare children to be successful in school.
In addition to these trainings, site staff participated in two weekly meetings throughout the program year; one to debrief following the first class of the week and the second, to train and plan for the following week’s classes. These meetings, which included the entire program staff, helped clarify and solidify roles and responsibilities and reinforced the interconnection of program elements within the AVANCE PCEP model. The all-staff meetings also promoted cross-planning among the core program activities: the parent and early childhood development education, and toy-making instruction, the community resource education, and the home visits.
Two of the four sites did not meet their enrollment target despite an extensive, on-the-ground recruitment campaign. Only two of the four sites met their target to enroll 25 parents
into the program. The two Community Action sites fell short of this target despite an intensive recruitment effort that included visiting neighborhood apartment complexes and knocking on hundreds of doors, distributing flyers outside the local markets, and placing posters at the local library, markets, churches, and Petaluma Health Center. All sites did, however, attract a number of new participants through referrals and word of mouth, indicating that the program has developed a positive reputation among parents in the community and may enjoy stronger enrollment moving forward.
Participant retention is an ongoing challenge for all sites, and program staff members have worked hard to mitigate barriers to participants’ successful graduation. The sites
did not meet their target to have 85% of their participants graduate: the two Community Action sites graduated 80% of its participants (17 of 18 at Kawana and 18 of 22 at Rohnert Park), while the two PAS sites graduated only 46% of its participants (8 of 26 at McKinley and 18 of 30 at McDowell).
18 Because home visits are not a requirement for graduation in the AVANCE PCEP, program staff did not record in detail parents’ level of participation in this aspect of the program.
| LFA Group:
| February 2013
9
According to Sonoma County First 5 staff, many low-income participants struggled to regularly attend the weekly program because of transportation costs, irregular work schedules, and family obligations. The afternoon session offered at McKinley proved to be problematic for families, conflicting with early-release days at schools and husbands’ expectations that dinner be prepared when they returned home from work. Additionally, despite the on-site childcare offered, a number of children found it difficult to remain separated from their parent during the three hour sessions. Program staff worked hard to reduce these barriers to attendance. For families dealing with the issue of separation anxiety, sites created a personalized integration plan. At the McDowell site, staff provided parents with access to an on-site supervised counseling intern through their partnership with the McDowell Family Resource Center. Despite these supports, some children were ultimately deemed not ready to participate in the program. To address other attendance challenges, sites set up make-up sessions and gave participants the option to attend class at a different site to accommodate their changing schedules. Furthermore, program staff regularly called participants to encourage them to attend upcoming classes and to check in if they missed a class.
Home visits provide an opportunity for program staff to see parents implement what they learn, and to provide parents with individual attention and feedback. The monthly
home visits allow the program staff to observe how parents are putting the weekly lessons into practice. The Home Visitor observes firsthand how parents are using the toys made in class, games, and language to interact with, encourage, and nurture their children. At regular intervals, Home Visitors videotape the parent and child interaction during the visit and show the recording to all participants at subsequent weekly sessions as a learning tool to initiate group discussion. Participation in the home visits varied by contractor; however, every
participating family received at least four home visits across all sites. PAS parents consistently honored the agreed-upon home visit schedule. However, Community Action parents regularly made last minute cancellations or missed their appointments. Despite the differences in the number of home visits, many Community Action and PAS parents reported to program staff that the personalized attention they received at the home visits was extremely helpful. Because home visits are not a requirement for graduation in the AVANCE PCEP, program staff did not record in detail parents’ level of participation in this aspect of the program.
Learning for Action: Building on Success and Lessons Learned
In the two years that Community Action and PAS have been implementing the AVANCE PCEP, and over the course of the evaluation, program staff and the evaluators have learned key lessons, including factors that contribute to a successful implementation of the AVANCE model and consideration for future programming. Select parents most in need of support to engage in family literacy activities. At the
beginning of the 2011-12 program year, 22 parents reported that they already read to their children every day and 51 parents reported already reading to their children three times per week or more. Given that this is a primary focus of the program is to serve the neediest families, AVANCE national acknowledges that the greatest change will be seen among parents most in need of support, and with the most room to grow in terms of regularly reading to their children.
Explore ways in which the program can better support families experiencing challenges to regular reading time at home. While the book bags provide age-appropriate books, parents
still face challenges of family and household obligations that may prevent daily reading.
AVANCE National is reviewing the AVANCE Parent Questionnaire to make sure questions about reading are also reflective of the improved quality of the book sharing and reading experience.
Maximize on-the-ground recruitment efforts by working with partner schools and
program alumni. The sites that enrolled the most parents into the program had the help of
their partner school sites to identify the neighborhoods and apartment complexes where they should focus recruitment. All sites should consider taking advantage of their relationships with partner schools to maximize their recruitment efforts. Additionally, given the apparent
effectiveness of word-of-mouth referrals, sites many want to consider inviting program alumni to participate in their on-the-ground recruitment drives.
Develop flexible and customized solutions to address participants’ attendance barriers early on. The elimination of the afternoon time schedule will support improved attendance and
completion rates in 2012-13. Program staff may also want to consider setting up a system at the beginning of the program to identify their participants’ possible barriers to regular attendance, such as transportation, work schedule, and home obligations. This information can help inform the time and place of the program’s regular and make-up sessions. Regular phone calls and reminders to families who miss class also help families to be reminded that they are missed and welcomed back to class even if they have missed a couple of sessions.
