Independent living programs for young people leaving the care system: The state of the evidence

(1)

Independent living programs for young people leaving the care system: The state of the evidence

Paul Montgomery^⁎, Charles Donkoh, Kristen Underhill

Centre for Evidence-Based Intervention, Department of Social Policy and Social Work, Oxford University, UK Received 27 January 2006; received in revised form 7 March 2006; accepted 22 March 2006

Available online 23 June 2006

Abstract

A substantial body of evidence suggests that young people leaving public care systems are at increased risk for low educational attainment, unemployment, homelessness, physical and mental health difficulties, dependency on public assistance, and involvement with the criminal justice system. Independent living programs (ILPs), which incorporate life skills and personal development, are one strategy frequently used to improve outcomes for young people leaving care. However, the effectiveness of ILPs remains unknown. This paper systematically reviews the current evidence for ILPs, summarizing all controlled comparisons designed to evaluate their effectiveness. Reviewers were unable to find any randomized controlled trial evaluating ILPs, but the results of eight non-randomized controlled studies suggest that some ILPs may have protective effects for youth leaving the public care system. These trends were observed for outcomes related to educational attainment, employment, housing, health, and other life skills; nevertheless, the weak methodological quality of the evidence tempers the validity and generalizability of these conclusions. Further research utilizing random assignment of participants is imperative in order to draw reliable conclusions for policy and practice.

Keywords: Young people; Independent living programs; Public care; Foster care; Life skills

1. Introduction

1.1. Young people leaving care

The population of young people in public care is substantial. Public care systems accommodated 523,000 American children in 2003 (Child Trends Databank, 2005), and 60,900

Children and Youth Services Review 28 (2006) 1435–1448

www.elsevier.com/locate/childyouth

⁎ Corresponding author. Centre for Evidence-Based Intervention, Department of Social Policy and Social Work, Barnett House, 32 Wellington Square, Oxford, OXI 2ER, UK. Tel.: +44 1865280325.

E-mail address:paul.montgomery@psych.ox.ac.uk(P. Montgomery).

doi:10.1016/j.childyouth.2006.03.002

(2)

English children in 2005 (Department for Education and Skills, 2005). There is a great deal of heterogeneity in these populations (⁎Biehal, Clayden, Stein, & Wade, 1995); however, most enter care because they are likely to have suffered maltreatment in their original homes (Department of Health, 1991), and their backgrounds are more likely to have been multiply disadvantaged (Bebbington & Miles, 1989).

Approximately 20,000 American and 6000–8000 English young people graduate annually from their respective care systems (Department for Education and Skills, 2005; United States General Accounting Office [USGAO], 1999). Many of these young adults remain severely disadvantaged and often lack the resources or life skills necessary to live independently (Barth, 1990; Cheung & Heath, 1994; Courtney, Piliavin, Grogan-Kaylor, & Nesmith, 1998; Festinger, 1983; Garnett, 1992). As compared to the general population, care leavers are more likely to be homeless, unemployed, and/or dependent on public assistance. These young adults are also at higher risk for physical and mental health difficulties, engagement in risky health behaviors, and involvement with the criminal justice system (Barth, 1990; Cook, 1994; Courtney & Dworsky, 2005; Courtney, Piliavin, Grogan-Kaylor, & Nesmith, 2001; Festinger, 1983; Fowler, Harwood,

& Meegan, 1996; Maunders, Liddell, Liddell, & Green, 1999). These detrimental outcomes are compounded by the fact that looked-after youth transition to independence earlier than their counterparts in the general population, and retain little or no financial, emotional, or social support from their families (Cashmore & Paxman, 1996; Courtney & Barth, 1996; Morrow &

Richards, 1996).

Having recognized some of the difficulties facing care leavers, policymakers and legislators have taken action to help prepare these youth for self-sufficiency. The UK has achieved this largely through the Children (Leaving Care) Act of 2000, which requires a needs assessment for each eligible child, followed by the development and implementation of an independence

“pathway plan” with a personal advisor (Department of Health, 2001). In the US, governmental actions have included the Foster Care Independence Act of 1999, part of which outlines the John H. Chafee Foster Care Independence Program (Barth & Ferguson, 2004; National Research Centre for Youth Development, 2005). This program is a set of state-run initiatives intended to prepare youth for education and employment, to offer psychological support, and to facilitate the transitional years between ages 18 and 21 (Barth & Ferguson, 2004).

1.2. Independent living programs

Both the UK and US legislation describe independent living programs (ILPs), initiatives designed to provide young people leaving care with skills that will limit their disadvantage and aid in their progress to adulthood. Although program designs differ, ILPs generally employ social skills training techniques such as instruction, modelling, roleplays and feedback. These approaches have been shown to effectively improve skills for young people in clinical and non- clinical settings (Spence, 1995).

The specific skills targeted by ILPs vary, but most programs focus on personal development and independent living. Personal development skills may include communication, decision making, and anger management; independent living skills may include job skills, budgeting, household tasks, seeking housing, obtaining legal assistance, and utilizing community resources (Cook, 1994; USGAO, 1999). Beyond these services, some ILPs provide educational and vocational support, and may continue services after youth are emancipated.

ILPs generally involve several delivery formats. Skills training is frequently conducted in group settings, while individual support such as mentoring can be provided on a one-to-one

(3)

basis (⁎Biehal et al., 1995; Meston, 1988). In the years before emancipation, ILPs often provide supervised living spaces in which young people can practise their acquired skills (Mauzerall, 1983). The service settings for ILPs are diverse, and include community centers, group homes, independent tenancies, transition placements, and supervised practice placements (⁎Biehal et al., 1995; Meston, 1988). The content, setting, and delivery of ILPs are contingent on legal and political contexts, causing programs to vary across different countries and local authorities.

1.3. Need for a systematic review

Although ILPs are widespread in public care systems, little evidence is available to gauge their effectiveness (USGAO, 1999). Furthermore, the basic premise of ILP delivery is unproven;

that is, it is unclear that youth who acquire independent living skills actually will experience a smoother transition to self-sufficiency. Previous reviews on this topic are often limited by geography, non-systematic searches, and the inclusion of studies not designed to assess effectiveness, suggesting the need for a systematic review with rigorous inclusion criteria. A recent systematic review (Donkoh, Underhill & Montgomery, 2006) found no randomized or quasi-randomized controlled studies worldwide evaluating the effectiveness of ILPs, although one such evaluation is in progress (M.E. Courtney, personal communication). These designs are the most rigorous methods for assessing effectiveness. However, despite a lack of randomization, the available research evidence may still provide informative data to researchers and practitioners (Shadish & Ragsdale, 1996). As such, studies that met all inclusion criteria for the systematic review apart from random assignment have been selected for discussion in this paper.

2. Methods

The studies described assessed the effectiveness of ILPs by comparing them to usual care, no intervention, or another intervention. Study participants were young people leaving the care system at their country's statutory age of discharge. Programs were ILPs providing skills training and/or personal development support. Programs specifically targeting youth with special needs (e.g., youth with disabilities, teenage parents, or young offenders) were excluded due to their incomparability with the general population of care leavers. Outcomes of interest included participants' educational attainment, employment, housing, health, and life skills.

To identify relevant papers, the authors searched electronic databases, contacted experts in the field, and cross-referenced bibliographies of existing evaluations. The electronic databases included the Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, CINAHL, PsycINFO, Sociological Abstracts, ASSIA, and Dissertation Abstracts. The search, which ended in October 2005, yielded 2196 citations. Two reviewers independently screened all citations for inclusion.

3. Search results

Seven of the eight identified studies were carried out in the US, while one (⁎Biehal et al., 1995) took place in the UK. A summary of each study is provided below. The full reports for three papers were unavailable; after an extensive search and at least two attempts to contact authors for the primary studies, the data for these studies were abstracted from secondary sources.

P. Montgomery et al. / Children and Youth Services Review 28 (2006) 1435–1448 1437

(4)

3.1. ⁎Baker, Olson, and Mincer (2000): WAY Scholarship Program

This was a 5-year longitudinal study in New York City evaluating the aftercare component of the Work Appreciation for Youth (WAY) initiative, an employment program in a residential treatment center for young people with severe emotional and behavioral difficulties. The aftercare component targeted those about to be emancipated, and included educational advocacy, tutoring, counseling, mentoring, work experience, and financial incentives. Services continued after discharge from care until the youth had been enrolled for 5 years. Participants were 10 cohorts of 15 to 20 young males (n = 155) recruited between 1984 and 1994, and a comparison group (n = 76) recruited between 1985 and 1990. The comparison group comprised those who had been eligible for the program but were discharged from care earlier, suggesting that they may have required fewer services. Outcome measures included educational attainment, employment, and criminality at the program's end. Dropouts (24%), defined as those leaving during the first 2.5 program years, were more likely to be older, were discharged sooner, and experienced fewer types of abuse earlier in life. The evaluation reported outcomes selectively, potentially obscuring adverse or null effects.

3.2. ⁎Biehal, et al., (1995): Moving On

This study used a quasi-experimental design to compare young people who had a key worker and used aftercare services (n = 30) to young people who did not have a key worker (n = 23).

Outcome data were collected via participant and caseworker interviews 18 to 24 months after the youth left care in three local authorities in England. Intervention services encompassed individual and group work, and included skills training, accommodation, counseling and career support.

Services varied in setting and exposure, even within individual districts. Furthermore, the treatment groups were vulnerable to non-equivalence and contamination: comparison group participants were more likely to have informal support, and some also had contact with leaving care services. Outcome measures included education, housing, employment, and support networks.

3.3. ⁎Harding and Luft (1993): The Preparation for Adult Living (PAL) Program. Abstracted from ⁎Lemon, Hines, and Merdinger (2005)

The study design for this evaluation, which took place in Texas, was unclear from the available information. It involved a survey after youth had left care, which compared a group of ILP participants (n = 30) with a group of young people who had not participated in an ILP (n = 29).

Surveys were administered by phone, by post, and via face-to-face interviews. It is unclear how much time had elapsed since youth were discharged from the program. Outcomes included completion of vocational training and the number of times participants moved housing.

3.4. ⁎Lemon, et al. (2005): Pathways to College

In nine counties of California, this study used survey data to compare 81 young people who had participated in an ILP and 133 young people who had not. It used a multiphase approach that included a postal survey of former foster youth attending a large US state university. The authors indicated that ILPs varied across the state, categorizing them into“typical” and “unique” services.

Typical ILP services, which were widely provided, included skills training, financial assistance, incentives, and information. Unique ILP services included experiential activities, role-plays, and computer training courses. Outcome measures included educational attainment, housing, health,

(5)

and problems with the law. ILP participants were significantly more likely to be of minority ethnicity, to be younger (21.6 against 24.0 years), and to have experienced a placement in non- relative foster care.

3.5. ⁎Lindsey and Ahmed (1999): The North Carolina Independent Living Program

The study evaluated the effectiveness of ILPs in North Carolina by comparing ILP participants (n = 44) against non-participants (n = 32). After an initial skills assessment, intervention youth received educational support, employment assistance, and information; control youth received usual care after the assessment. Stratified cluster sampling was used to select counties for the study. The authors reported that comparison group participants were randomly chosen from county lists, but the method of randomization was unclear. At the time of the survey, participants in the ILP group were significantly older and more likely to be female. Postal questionnaires were sent to participants one to three years after emancipation, and outcomes included educational attainment, employment, housing, and self-sufficiency.

3.6. ⁎Scannapieco, Schagrin, and Scannapieco (1995) and ⁎Scannapieco (1996): Baltimore Independent Living Programs

The authors investigated the effectiveness of ILPs in Baltimore County by comparing young people who had participated in an ILP between 1988 and 1993 (n = 44) with foster care youths who were eligible for participation but did not take part (n = 46). ILP services were provided in the community and included group-based and individual components. Specific activities focused on employment, education, advocacy, counseling, life skills instruction, and skills practice. Outcome data were abstracted from case records in 1993 (mean participant age = 19.1 years), and no data were available after case closure. Outcome measures included educational attainment, employment, housing, and self-sufficiency.

3.7. ⁎Shippensberg University Center for Juvenile Justice Training and Research (1993):

Pennsylvania's Independent Living Program for Youth. Abstracted from ⁎Lindsey and Ahmed (1999),Barth and Ferguson (2004), and ⁎Lemon et al. (2005)

A postal survey was used to compare a group of Pennsylvania youth who had participated in an ILP against a matched group of non-participants. The survey took place one year after emancipation. The response rate was low−24% for ILP participants and 41% for non-participants (⁎Lindsey & Ahmed, 1999). The low response rate is problematic, as there may have been systematic differences between responders and non-responders. Estimates of group size differ:

⁎Lindsey and Ahmed (1999) suggest that the groups numbered 32 ILP youth and 24 controls, while ⁎Lemon et al. (2005) indicate that the study included 51 ILP youth and an unspecified number of controls. Outcomes included educational attainment, employment, receipt of public assistance, and living independently.

3.8. ⁎Simmons (1990): The Preparation for Adult Living (PAL) Program. Abstracted from USGAO (1999)

Little information was available for this evaluation, which took place in Harris County, Texas, and which evaluated the same program as ⁎Harding and Luft (1993). The study involved direct

(6)

comparisons between PAL participants and a non-participant comparison group, although the numbers of participants and study design were unclear. Outcomes included full-time employment and educational attainment.

Table 1

Non-comparative ILP studies

Study Location N Methods Findings

Abatena (1996)^a

Nevada 26 Self-report

assessment 3 months after discharge

“Most participants” felt the ILP helped prepare them for independent living“to some extent.” 53% felt services were not“satisfactory.”

Center on Children, Families, and the Law (1994)^a

Nebraska 58 Survey 1 year after

discharge

85% had attained high school/GED qualification (including 100% of respondents over age 20), and 53% had completed post-high-school training.

64% were employed. 57% were living independently. 18% received one or more types of public assistance. Most had felt prepared for independent living at time of discharge.

Cook, Fleishman, and Grimes (1991), Cook (1994)

Arizona, California, Illinois, Missouri, New York, Pennsylvania, Tennessee, and the District of Columbia

810 Personal interviews 2.5–4years after discharge

Receiving instruction in each of 5 skill areas (budgeting, credit, consumer skills, education, and employment) was significantly related to maintaining a job for > 1 year, living at no cost to the community, accessing health care, and general satisfaction.

Mallon (1998) New York City 46 Case records and interviews pre-ILP, post-ILP, and follow-up on average 3 years (min. 6 months) after discharge

74% had attained high school/GED qualification at follow-up, and 17% had post-secondary training. 72% and 78%

were employed full-time at discharge and follow-up, respectively. 68% lived independently at discharge, increasing to 77% at follow-up. One participant depended on public assistance at discharge; two received welfare at follow-up.

McMillen, Rideout, Fisher, and Tucker (1997)

Missouri 25 Focus groups, average

of 2.11 years after discharge

Participants believed skills training to be beneficial, especially the stipends and subsidies in the financial skills component. Participants also valued independent living specialists for emotional support and knowledge of resources.

Moore and Associates, Inc.

(1998)^a

Wayne County, Michigan

61 Survey during participation in ILP

80% believed their quality of life had improved after they received independent living services. Housing and health care services were most beneficial; employment services were least beneficial.

a Original paper was not available. Data forAbatena (1996)andMoore and Associates, Inc. (1998)were abstracted fromUSGAO (1999). Data forCenter on Children, Families, and the Law (1994)were abstracted from ⁎Lindsey and Ahmed (1999).

(7)

Table 2

Comparative ILP studies that did not assess effectiveness

Study Location Reason for exclusion N Methods Findings

Austin and Johnston (1995)^a

Pennsylvania Compares youth graduating from the same ILP at two time points, but does not compare ILP to usual care, no intervention, or non-ILP.

278 youth in early cohort, 255 youth in late cohort

Surveys at time of ILP completion.

Compares youth who received services during the“start-up” phase (1988–1991) against youth who received services during the

“fine-tuning” phase (1992–1994).

No group differences observed in educational attainment;

later-cohort youth were less likely to drop out of high school and post-secondary education.

Christenson (2003)

Idaho Compares two different ILPs, but does not compare ILP to usual care, no intervention, or non-ILP.

78 pre-Chafee youth, 164 Chafee participants

Uses case records and reports of ILP coordinators to compare youth eligible for Idaho ILPs before and after the 1999 implementation of the Chafee Program (cohorts left care in 1996–1998 and in 2000–2002).

Favored pre-Chafee youth for attaining high school/GED (65% vs. 42%) and employment (65% vs. 23%). Chafee participants reported less homelessness (13% against 17%), teen pregnancy/childbearing (33% vs. 35%) and use of social services (62% vs. 85%).

Courtney and Dworsky (2005)

Illinois, Iowa, and Wisconsin

Ongoing study. Available outcome analyses do not compare youth who do and do not receive ILP services.

736 in entire study;

603 at Wave 2

3-wave study using interview data to compare youth discharged before and after December 2004. Only wave 2 data were available, comparing youth in the care system (n = 282) with youth already discharged (n = 321).

Outcomes generally favored youth remaining in care, but effects of ILPs were not isolated.

Waldinger and Furman (1994):

Phase I

Los Angeles, California

Compares ILP participants to non-participants, but does not present outcomes of interest for this review.

289 ILP participants, 65 non-participants

Social worker survey reports for age-eligible youths at initial assessment and after two program years

ILP participants more likely than controls to improve behavior (33% vs. 15%) and perform“at or above grade level” (51% vs. 45%).

Waldinger and Furman (1994):

Phase II

Los Angeles, California

Compares two different ILPs, but does not compare ILP to usual care, no intervention, or non-ILP.

“Universe” of 62 in ILP and 230 in enhanced ILP

All participants attended classes, but enhanced program youth were assigned social workers to integrate school/family issues with emancipation preparation.

Youth were assessed via case records and interviews 6 months after program began

No significant group differences found for current employment, feeling“ready to make it on their own,” or size of support network.

a Original paper was not available. Data forAustin and Johnston (1995)were abstracted fromBarth and Ferguson (2004). 1441P.Montgomeryetal./ChildrenandYouthServicesReview28(2006)1435–1448

(8)

3.9. Excluded studies

A number of studies were excluded from in-depth discussion because they were not designed to assess the effectiveness of ILPs. These included qualitative studies, studies without comparison groups, and evaluations comparing two different ILPs. Since these types of designs may still provide valuable insight, studies meeting these descriptions are summarized inTables 1 and 2.

4. Methodological quality of included studies

This section assesses the methodological quality of the presented studies using information available in the published versions.

4.1. Baseline differences

There were extensive baseline differences between groups in almost every study. For example,

⁎Lindsey and Ahmed (1999) reported that ILP participants were significantly older than non- participants, suggesting that age differences might have accounted for the effects observed.

Similar differences in age, ethnicity, gender, placement history, and support networks were reported by other studies, making it impossible to attribute causality solely to the intervention in any study. Given that none of the studies were randomized controlled trials, these systematic differences undermine the studies' external validity.

4.2. Sample sizes

Sample sizes were small. For instance, sample sizes for ⁎Biehal et al. (1995) and ⁎Lindsey and Ahmed (1999) were 53 and 76, respectively. Small samples make estimation and generalization of intervention effects difficult because of inadequate statistical power and the probable unrepresentativeness of the sample. Even when sample sizes were large, the participants were not necessarily similar to the entire population of care leavers. For example,

⁎Baker et al. (2000) had a sample size of 231, but all were male youth with severe emotional or behavioral problems, making it difficult to generalize the study results to other youth in public care.

4.3. Outcome measurements

Studies relied on case records and self-reported data. It is well known that self-reports are biased and that case records might be inaccurate or influenced by performance pressures.

Independent living is a process; therefore, some outcomes such as educational attainment, health, and maintenance of housing are best measured over the long term. Few studies in this review measured long-term outcomes, and follow-up times often varied within and between trial arms.

4.4. Implementation fidelity

Implementation fidelity includes factors that may influence the comparability of different programs, such as program dosage, staff adherence to protocol, and the quality of program

(9)

delivery (Dane & Schneider 1998). Several studies grouped different ILPs together for analysis (e.g., ⁎Biehal et al., 1995; ⁎Lemon et al., 2005; ⁎Lindsey & Ahmed 1999), making it difficult for trialists to report specific information on how the different programs were implemented. The absence of this information limits the applicability of the available evidence, since unclear reports may lead to misleading conclusions about which programs or program components are most effective.

5. Outcomes

5.1. Educational attainment

All but one study utilizing this outcome reported results favoring ILP participants. Three studies reported statistically significant protective effects. ⁎Lindsey and Ahmed (1999) observed that a significantly higher proportion of ILP participants than non-participants had completed high school, the GED, or a technical/vocational program at follow-up (58% against 18%, p < 0.05), and program participants were significantly more likely to be enrolled in college (16%

against 0%, p < 0.05). ⁎Scannapieco et al., (1995) also found that ILP participants were significantly more likely than non-participants to graduate from high school (50% against 13%, p < 0.05). ⁎Harding and Luft (1993) reported that ILP participants were significantly more likely to complete vocational training than non-participants (⁎Lemon et al., 2005). ⁎Simmons (1990) reported that ILP participants were more likely to complete high school at a younger age, but specific data were not available (USGAO, 1999).

⁎Baker et al. (2000) reported that of all WAY participants in cohorts 1–6 who remained in the program for at least 2 years, 80% were still in school or had graduated from high school at age 21 (including one-third who were enrolled in college); however, the lack of a comparison group weakens this finding.

Only ⁎Shippensberg (1993) observed results in the opposite direction. One year after discharge, youth who had participated in the Pennsylvania independent living program were significantly less likely than non-participants to have completed high school (59% of ILP participants against 79% of non-participants) (Barth & Ferguson, 2004; ⁎Lindsey & Ahmed, 1999).

5.2. Employment

A number of studies reported positive employment outcomes for ILP participants.

⁎Scannapieco et al., (1995) found that a significantly higher percentage of ILP participants than control participants were employed at case closure (52% against 26%, p < 0.05). Similarly, in

⁎Lindsey and Ahmed's evaluation (1999), ILP participants were nonsignificantly more likely than non-participants to be employed full-time at follow-up (41% against 22%). ⁎Shippensberg (1993) reported that ILP participants were more likely than non-participants to be employed full- or part-time at follow-up (50% against 37%) (⁎Lindsey & Ahmed, 1999). ⁎Simmons (1990) found that ILP participants obtained full-time employment earlier than non-participants, but specific data were not available (USGAO, 1999).

⁎Baker et al. (2000) reported on a subset of youth from cohorts 1–6 who had participated for at least 2 years of WAY aftercare; 80% of these young men were working in 1997 (2 to 11 years after leaving the program). Again, the lack of a control group tempers this finding, and it is unclear how this subset was chosen.

(10)

In one opposite finding, ⁎Lemon et al. (2005) found that controls were more likely than ILP participants to be employed after discharge from care (74% against 58%, p < 0.05). This effect may be confounded with age; ILP participants were significantly younger than control participants, with over 2 years' mean difference (p < 0.05).

5.3. Housing

Every study examining housing reported more favorable outcomes for ILP participants.

⁎Lindsey and Ahmed (1999) found that a significantly higher proportion of ILP participants than non-participants were living independently at follow-up (68% against 41%, p < 0.05), and paying all housing expenses while living with others (25% against 0%, p < 0.05), although the two groups did not differ in having experienced homelessness since leaving care (52% against 53%). Another significant result occurred in ⁎Scannapieco et al.'s study (1995), in which 36%

of ILP participants and 4% of non-participants reported living on their own at case closure (p < 0.05). ⁎Lemon et al. (2005) reported that ILP participants had experienced nonsignificantly less homelessness than non-participants (23% against 16%, p = 0.233). In a measure of housing stability, ⁎Harding and Luft (1993) found that ILP participants moved significantly fewer times than non-participants (⁎Lemon et al., 2005). Finally, ⁎Biehal et al. (1995) also reported favorable housing outcomes for program participants, although specific data was not available.

5.4. Health

Health outcomes were not consistently reported. In the study by ⁎Lemon et al. (2005), ILP participants were nonsignificantly less likely than non-participants to have received mental health services (34% against 31%), or to have experienced difficulty accessing medical care (58%

against 53%) at follow-up.

5.5. Other relevant life skills outcomes: criminality and self-sufficiency

This category encompasses a number of outcomes that were not uniformly used, but were relevant to independent living outcomes. Where criminality outcomes were reported, they favored program participants. ⁎Lemon et al. (2005) found that ILP participants had nonsignificantly fewer problems with the law than non-participants (16% against 12%). As reported by ⁎Baker et al.

(2000), adult criminality rates were 5% for participants who stayed in the WAY program for over 2 years (n = 118), 35% for those who completed less than 2 years (n = 37) and 15% for the comparison group (n = 76).

Outcomes were mixed in the area of independent living. ⁎Scannapieco et al., (1995) reported that ILP participants were significantly more likely than controls to be self-supporting at follow- up (48% against 17%, p < 0.05). ⁎Biehal et al. (1995) also reported favorable life skills outcomes for program participants, although specific data were not available. Alternately, ⁎Lindsey and Ahmed (1999) reported mixed effects. In this study, ILP participants were nonsignificantly more likely than non-participants to utilize housing assistance, Aid to Families with Dependent Children (AFDC), Women Infants and Children (WIC), and emergency assistance. The authors suggested that the difference between the two groups could be due to the higher proportion of females in the ILP group. However, a nonsignificantly higher percentage of ILP participants in this study also reported being completely self-sufficient at follow-up than non-participants (20%

(11)

against 16%), and a significantly smaller percentage of ILP participants indicated difficulty paying bills (5% against 25%, p < 0.05). ⁎Shippensberg (1993) reported similar findings: ILP participants were more likely than non-participants to receive one or more types of public assistance (38% against 29%), but were also significantly more likely to be living independently 1 year after discharge (50% against 17%, p < 0.05) (⁎Lemon et al., 2005; ⁎Lindsey & Ahmed, 1999). It is possible that the use of public assistance may not be a detrimental outcome, but rather a measure of ability to access available resources.

6. Discussion

In sum, the available evidence indicates that some ILPs may improve educational, employment, and housing outcomes for young people leaving the care system. The strength of this evidence, however, is tempered by weak evaluation methodology. Non-randomized controlled studies are prone to bias from systematic confounders, which may be especially likely in a group of young people with heterogeneous pre-care experiences, demographics, and personal characteristics. Without random assignment of participants, it is impossible to attribute outcome differences solely to ILP participation. The strength of the evidence is further offset by small sample sizes, baseline differences, substantial variation in ILP design, reliance on self- reported data, inadequate reporting of effect sizes and confidence intervals, and the lack of information on program delivery. Thus, while the results reported here are promising, limitations of the data set suggest that it would be inappropriate to uniformly generalize the results of this narrative review.

This paper makes a valuable contribution to the literature on ILPs, because it is the first to document a systematic worldwide literature search and methodologically appraise the current evidence for ILPs. However, this review has a number of shortcomings. It is severely limited by methodological weaknesses in the data set, as outlined above. Incomplete reporting and the inability to obtain full evaluation reports were recurring problems; information for three of the included studies (⁎Harding & Luft, 1993; ⁎Shippensberg, 1993; ⁎Simmons, 1990) could only be abstracted from secondary sources, which were contradictory in one case (⁎Shippensberg, 1993).

The literature search extended only through October 2005, meaning that subsequent publications were not included. Despite extensive efforts to identify unpublished evaluations, it is possible that some of these were not captured. The generalizability of this review is lastly limited by the geographical bias of the available evidence. No studies outside the US and UK were identified, and the impacts of ILP programs elsewhere are not known.

The results of almost every study favored ILP participants over non-participants for educational attainment, employment, housing, health, and life skills outcomes. In some studies, these findings reached statistical significance, particularly in the areas of educational attainment and housing. Although two significant results in the opposite direction were observed (⁎Lemon et al., 2005 for employment; ⁎Shippensberg, 1993 for educational attainment), these evaluations do not provide consistent evidence to suggest that ILPs cause harm. Of particular interest is the finding by two studies (⁎Lindsey & Ahmed, 1999; ⁎Shippensberg, 1993) that ILP participants were more likely to utilize public assistance, but also more likely to be living independently after discharge. The relationship between public assistance and independent living for ILP participants requires further investigation.

Notably, the outcomes reported for ILP participants compared favorably with outcomes documented among the general population of care leavers. It is estimated that 55% of this group graduate from high school (Barth, 1990), and 85% are unemployed (Courtney et al., 2001).

(12)

Various studies in this review documented more favorable outcomes for ILP participants than these figures would predict (e.g., ⁎Baker et al., 2000; ⁎Lemon et al., 2005; ⁎Lindsey & Ahmed, 1999; ⁎Scannapieco et al., 1995).

But when compared to the general population of young people, ILP participants still report extensively poorer outcomes. For example, Lindsey and Ahmed reported that 58% of the study's ILP participants completed high school or a GED (1999). According to the US National Center for Education Statistics, this figure reached 86% among the general US class of 1998 (Greene, 2002). Employment rates for ILP participants are also below the national standard, given that the average yearly US unemployment rate between 1995 and 2004 was approximately 5.1% (United States Bureau of Labor Statistics, 2005). This indicates that ILPs have not successfully put outcomes for care leavers at par with their peers in the general population.

7. Conclusions

The eight evaluations summarized here constitute the best available evidence for the effectiveness of independent living programs. According to these reports, some ILPs appear to improve a variety of outcomes for some foster care youth. However, the current evidence does not permit reviewers to identify which elements of ILPs are most effective, which youth may benefit most from ILPs, or the mechanisms by which ILPs influence outcomes (if this is indeed the case).

Further research investigating the effectiveness of ILPs is imperative, and there is a vital need for studies utilizing randomized controlled designs to establish causality. Additional research should also attempt to incorporate larger sample sizes, report information on implementation fidelity, and measure outcomes over longer and more standardized follow-up intervals. The results of more rigorous primary studies can then be compared to the current evidence to allow a more thorough assessment of ILP effectiveness. Improving the quality of the evidence base is the surest way to permit more relevant and reliable inferences for policy and practice.

Acknowledgements

The authors would like to thank Jane Dennis and Jo Abbott from the Cochrane Developmental, Psychosocial, and Learning Problems Review Group, and various primary-study authors, including Mark Courtney. This review was funded by Socialforskningsinstituttet/The Danish National Institute of Social Research.

References¹

Abatena, H. (1996). Independent living initiative program: A follow up survey report of the youth released from foster care in Nevada in 1996. Henderson, NV: Nevada Research, Evaluation, and Planning Consultants.

Austin, T., & Johnston, J. (1995). The effectiveness of Pennsylvania's independent living initiative. Shippensberg, PA:

Center for Juvenile Justice Training and Research, Child Welfare Division.

*Baker, A. J., Olson, D., & Mincer, C. (2000). The WAY to work: An independent living/aftercare program for high-risk youth. Washington, DC: Child Welfare League of America.

Barth, R. P. (1990). On their own: The experiences of youth after foster care. Child and Adolescent Social Work, 7, 419–440.

Barth, R. P., & Ferguson, C. (2004). Educational risks and interventions for children in foster care. Retrieved December 12, 2005, fromhttp://www.socialstyrelsen.se

1 Starred references indicate primary studies discussed in this review.

(13)

Bebbington, A., & Miles, J. (1989). The background of children who enter local authority care. British Journal of Social Work, 19(5), 349–368.

*Biehal, N., Clayden, J., Stein, M., & Wade, J. (1995). Moving on: Young people and leaving care schemes. London:

HMSO.

Cashmore, J., & Paxman, M. (1996). Wards leaving care: A longitudinal study. Sydney, Australia: New South Wales Department of Community Services.

Center on Children, Families, and the Law (1994). Independent living skills evaluation: Former state wards—baseline report. Lincoln, NE: Author.

Cheung, S. Y., & Heath, A. (1994). After care: The education and occupation of adults who have been in care. Oxford Review of Education, 20(3), 361–374.

Child Trends Databank. Foster care (2005). Retrieved December 16, 2005, fromhttp://www.childtrendsdatabank.org/

indicators/12FosterCare.cfm

Christenson, B. L. (2003). Youth exiting foster care: Efficacy of independent living services in the state of Idaho. Cheney, WA: Eastern Washington University.

Cook, R. J. (1994). Are we helping foster care youth prepare for the future? Children and Youth Services Review, 16(3–4), 213–229.

Cook, R., Fleishman, E., & Grimes, V. (1991). A national evaluation of Title IV-E foster care independent living programs for youth, phase 2 final report. Rockville, MD: Westat, Inc sponsored by the Administration for Children, Youth, and Families (DHHS).

Courtney, M., & Barth, R. P. (1996). Pathways of older adolescents out of foster care: Implications for independent living services. Social Work, 41(1), 75–83.

Courtney, M. E., & Dworsky, A. (2005). Midwest evaluation of the adult functioning of former foster youth: Outcomes at 19. Chicago, IL: Chapin Hall Center for Children, University of Chicago.

Courtney, M. E., Piliavin, I., Grogan-Kaylor, A., & Nesmith, A. (1998). Foster youth transition to adulthood: Outcomes 12 to 18 months after leaving out-of-home care. Madison, WI: Institute for Research in Poverty Report, University of Wisconsin.

Courtney, M. E., Piliavin, I., Grogan-Kaylor, A., & Nesmith, A. (2001). Foster youth transitions into adulthood: A longitudinal view of youth leaving care. Child Welfare, 80(6), 685–717.

Dane, A., & Schneider, B. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23–45.

Department for Education and Skills (2004). Children looked after in England (including adoptions and care leavers):

2003–05. Retrieved March 3, 2006, fromhttp://www.dfes.gov.uk/rsgateway/DB/SFR/s000615/index.shtml Department of Health (1991). Children Act 1989: Guidance and regulations. London: HMSO.

Department of Health (2001). Children (leaving care) Act 2000: Regulations and guidance. London: HMSO.

Donkoh, C., Underhill, K., & Montgomery, P. (2006). Independent living programmes for improving outcomes for young people leaving the care system. The Cochrane Database of Systematic Reviews.

Festinger, T. (1983). No one ever asked us. New York: University of Columbia.

Fowler, S., Harwood, S., & Meegan, F. (1996). Too much too young: The failure of social policy in meeting the needs of care leavers. London: Barnardo's.

Garnett, L. (1992). Leaving care and after. London: National Children's Bureau.

Greene, J. P. (2002). High school graduate rates in the United States. Retrieved December 12, 2005, fromhttp://www.

manhattan-institute.org/html/cr_baeo.htsm#14

Harding, J. T., & Luft, J. L. (1993). Outcome evaluation of the PAL (Preparation for Adult Living) program. Abstract presented at the University of Illinois invitational research conference on preparing foster youth for adult living. In E.

V. Mech, & J. R. Rycraft (Eds.), Preparing foster youth for adult living: Proceedings of an invitational research conference (pp. 144–146). Washington, DC: Child Welfare League of America.

Lemon, K., Hines, A. M., & Merdinger, J. (2005). From foster care to young adulthood: The role of independent living programs in supporting successful transitions. From foster care to young adulthood: The role of independent living programs in supporting successful transitions, 27, 251–270.

Lindsey, E. W., & Ahmed, F. U. (1999). The North Carolina independent living program: A comparison of outcomes for participants and non-participants. Children and Youth Services Review, 21(5), 389–412.

Mallon, G. P. (1998). After care, then where? Outcomes of an independent living program. Child Welfare, 77(1), 61–78.

Maunders, D., Liddell, M., Liddell, M., & Green, S. (1999). Young people leaving care and protection: A report to the National Youth Affairs Research Scheme. Melbourne, Australia: Australian Clearinghouse for Youth Studies.

Mauzerall, H. A. (1983). Emancipation from foster care: The independent living project. Child Welfare, 62(1), 46–53.

(14)

McMillen, J. C., Rideout, G. B., Fisher, R. H., & Tucker, J. (1997). Independent living services: The views of former foster youth. Families in Society: The Journal of Contemporary Human Services, 78, 471–479.

Meston, J. (1988). Preparing young people in Canada for emancipation from child welfare care. Child Welfare, 67(6), 625–633.

Moore & Associates (submitted for publication). Final report: An evaluation of the Wayne County Independent Living Coalition. Southfield, MI: Michigan Family Independence Agency.

Morrow, V., & Richards, M. (1996). Transitions to adulthood: A family matter. London: Joseph Rowntree Foundation.

National Research Centre for Youth Development (2005). The transition years: Serving current and former foster youth ages eighteen to twenty. Retrieved December 16, 2005, fromhttp://nrcys.ou.edu/nrcyd/publications/monographs/

transitions.pdf

Scannapieco, M. (1996). An independent living program: Characteristics, outcomes, and indicators for the future of foster care services. Community Alternatives, 8(2), 19–25.

Scannapieco, M., Schagrin, J., & Scannapieco, T. (1995). Independent living programs: Do they make a difference? Child and Adolescent Social Work, 12, 381–389.

Shadish, W., & Ragsdale, K. (1996). Random versus nonrandom assignment in psychotherapy experiments: Do you get the same answer? Journal of Consulting and Clinical Psychology, 64, 1290–1305.

Shippensberg University Center for Juvenile Justice Training and Research (1993). An evaluation of Pennsylvania's independent living program for youth. Shippensburg, PA: Author.

Simmons, J. (1990). PAL evaluation final report. Harris County, TX: Children's Protective Services.

Spence, S. H. (1995). Social skills training: Enhancing social competence with children and adolescents. Windsor, UK:

NFER-Nelson.

United States Bureau of Labor Statistics (2005). Labor force statistics from the Current Population Survey: (Unadj) unemployment rate. Retrieved December 16, 2005, fromhttp://data.bls.gov

United States General Accounting Office (1999). Foster care: Effectiveness of independent living services unknown, (GAO/HEHS-00-13). Washington, DC: GAO Health, Education, and Human Services Division.

Waldinger, G., & Furman, W. M. (1994). Two models of preparing foster youth for emancipation. Children and Youth Services Review, 16(3–4), 201–212.