Methodological considerations 1 Strengths

This dissertation has contributed to the small body of existing knowledge about effective interventions for homeless young adults. Until now, only a small number of studies have evaluated the effectiveness of an intervention for homeless young adults and previous studies concluded that more thorough research using rigorous designs are needed. In this dissertation, a cluster randomized controlled trial was set

7

up to investigate the effectiveness of the Houvast intervention, making this a methodological strong study. Although the most ideal design would have been a randomized controlled trial in which individuals were randomly allocated to an intervention or control group, this was both from a practical and an ethical standpoint not feasible. The number of professionals working in a shelter is relatively small and confining trained professionals within a shelter facility to consistently work with some homeless young adults and not with others would have made it unpractical. Furthermore, some of the tools of Houvast like team supervision requires the entire team of professionals to be trained in order to be effective. Providing strengths- based support to one youth in a shelter and not to another youth who resides in the same shelter would have been unethical and unfeasible.

A second strength is that there was much collaboration with professionals during the data collection. Researchers stayed in contact on a regular basis with team leaders and supervisors to be involved in the implementation process, to answer questions, to discuss and identify difficulties, and to provide information about the research progress. Furthermore, professionals provided help to make contact with homeless young adults to make an interview appointment, and to reconnect with them for follow-up interviews. This was especially helpful when homeless young adults had already left the shelter facility. This probably led to a total drop-out of only 21 percent of the homeless young adults at the follow-up.

A third strength is that the fidelity of the Houvast intervention was measured. Much effort was invested to measure the fidelity of Houvast by conducting a one-day audit to the shelter facility. The advantage of the fidelity measurements was that more reliable conclusions on the ineffectiveness of Houvast could be drawn. A second, more practical advantage of the fidelity measurements was that all shelter facilities received tailor-made guidelines on how to improve and optimize their implementation of Houvast.

Another strength of this dissertation is that a dyadic perspective was used to investigate the working alliance between homeless young adults and their professionals. By using a one-with-many design the nested structure of the data was taken into account. Thereby, in fact three perspectives were investigated, namely the perspective of the young adult, the perspective of the professional and the perspective of their unique dyad.

3.2 Limitations

This dissertation has some limitations as well. A first limitation is that the time of follow-up measurement was flexible. The follow-up interview with homeless young adults took place when homeless young adults left the shelter facility, no matter what the reason was. Experts in the field indicated that this seemed to be the best option. A fixed timeframe for the follow-up would result in invalid findings because it would

result in a variation of time between young adults leaving the shelter facility and measurement. However, as a result of this decision, the duration of exposure to Houvast or care-as-usual was not equal for each homeless young adult. Controlling for this variable was not possible because interpreting ‘duration of exposure’ is not conclusive. A long or short exposure to the intervention can be interpreted as positive or negative depending on the reasons for finishing care. For instance, a short duration of exposure could mean that homeless young adults left the shelter facility because they had achieved their goals early. On the other hand, it could also mean that homeless young adults were forced to leave the shelter facility, for example because they violated the rules of the shelter facility. Second, fidelity measurements were conducted in five shelter facilities who worked according to the Houvast intervention. No fidelity scores are available for the five shelter facilities who participated in the control condition. Therefore, it was not possible to control for this variable in the analysis concerning the effectiveness of Houvast. As fidelity measurement is an important aspect of an intervention study, future research should regularly conduct fidelity measurements on a larger scale in order to be able to incorporate fidelity data in the analysis of outcomes.

3.3 Representativeness and generalizability

To what degree are homeless young adults in this study representative for the total population of homeless young adults in the Netherlands? As described in the introduction, agencies in the Netherlands provide different types of care to homeless young adults such as outreach services and emergency or night services for homeless people in general. The present dissertation did not focus on these types of care. Instead, the homeless young adults in this dissertation represent the population of homeless young adults who either reside in shelter facilities where they receive care (e.g., social pension or social shelters) or ambulatory care (while being housed or not being housed). These types of care were designed specifically for homeless young adults. The participating shelter facilities were spread throughout the Netherlands. Therefore, the results in the present dissertation are representative for this subgroup, but no conclusions can be drawn about the effectiveness of Houvast among other subgroups.

The results in the present dissertation can be compared with both Dutch and international outcomes. But how generalizable are the results of the present dissertation to the national and international population of homeless young adults? In the few studies conducted among homeless young adults in the Netherlands, the distribution of age and gender of the participants varies widely. Most Dutch studies focused on homeless young adults with an average age between 19 and 22 years old (Beijersbergen et al., 2008; Fransen & van den Handel, 2011; Jansen, Mensink, et al., 2007) whereas many studies in the United States focused on homeless young adults

7

from about 14 years of age onward (Slesnick et al., 2009). In the Dutch literature, both an equal distribution of gender or an over-representation of male homeless young adults have been reported ( Wolf, Altena, Christians et al., 2010) such as in the present dissertation. In the international literature the distribution of males and females in the study population varies enormously (Slesnick et al., 2009). About half of the homeless young adults in the current dissertation has a Dutch background which is comparable with other Dutch studies (Beijersbergen et al., 2008; Fransen & van den Handel, 2011). The main reasons for homeless young adults to leave home in the present dissertation like family conflicts, bad relationship with (step) parents, financial problems and emotional abuse are similar to other Dutch studies (Wolf, Altena, Christians et al., 2010) and international studies (Slesnick et al., 2009; Thompson, Safyer, & Pollio, 2001; van Deth, van Doorn, & Rensen, 2009; Whitbeck et al., 2000). The broad range of problems homeless young adults in the Netherlands face are to a large extent comparable with homeless young adults in other western countries: low educational level, high debts, psychological and substance use problems, and a suspected intellectual disability (Pollio et al., 2006).

Also the care system in the Netherlands is different compared to the United States making generalizability of the results difficult. Every citizen in the Netherlands is required to pay tax and to pay for health care insurance. This system provides citizens (including homeless young adults) comprehensive coverage for a wide range of services, such as a guaranteed income, the necessary medical care (e.g., general practitioner and hospital), and medication. In the United States, the culture itself tends to be more individualistic and the amount of tax to be paid is low compared to the Netherlands. However, under the Affordable Care Act (ACA), or in other words ‘Obamacare’, it is nowadays compulsory for citizens to have qualifying health coverage or to pay a fee. But at this time there are many struggles in implementing this health care system in the United States. Furthermore, there are several federal and private programs who provide support to homeless people. Despite these programs, a previous study revealed that a more national social welfare system could prevent homelessness and near-homelessness experiences (Tompsett et al., 2003).