SELECT AND TRACK STUDY PARTICIPANTS

The study team worked with each of the sites to select and track program participants over a period of 12 months. This section describes the sample selection and informed consent process. The baseline, retrospective, and monthly data collection processes are described as well. The study team also worked with external trackers to gather information about program leavers and conducted focus groups at the close of the 12-month data collection period to elicit feedback from study participants regarding the Housing First program.

Sample Selection

During the baseline site visit to each of the sites, the study team instructed the sites to select the first 25 clients who entered the Housing First program between April and June 2004 and were unaccompanied (not part of a homeless family), severely mentally ill, and willing to participate

in the study.95 Staff at the Housing First programs confirmed that the clients included in the

study sample are representative of the overall population of the programs. During the sample selection process, each program had a number of clients refuse to participate in the study or the program chose to skip them. Although human subjects research standards prohibit asking a prospective study participant why he or she does not want to participate, program staff subsequently offered possible reasons for such refusal.

• At DESC, five clients refused to participate in the study. DESC staff believe their refusal was

related to mental illness, including suspiciousness and paranoia. One of these was a client who had entered the program and left before the study enrollment began. She was offered the opportunity to join the study, but declined.

95

Two of the study sites recruited more than 25 participants to track over 12 months. The sample at Pathways to Housing was 26 and the sample at Project REACH was 29.

• At REACH, seven clients could not consent due to the nature or acuteness of their mental illness; five clients were not homeless; 10 clients would not be cooperative with the study, as determined by REACH staff; two refused to participate in the study; and one was missing from the program.

• At Pathways to Housing, there were no refusals and two skips because the clients were

unavailable for enrollment, for unknown reasons.

Because the Housing First programs selected for study had a fairly low turnover rate and a small volume of new enrollments each month, the study relied on a largely retrospective sample. The balance of retrospective and prospective cases in the sample was determined by the volume of new placements entering the program during the 3-month enrollment period of the study. The study team also requested that program staff attempt to enroll participants who entered and left the program before study recruitment started—this was the case of one prospective study

participant at DESC who refused to participate. Approximately one-half of the study sample was retrospective (i.e., enrolled in the Housing First program prior to the study enrollment period, which began in May 2004). (See exhibit A–1.)

Exhibit A–1. Enrollment of Study Sample

DESC Pathways to _Housing REACH Total Date of Enrollment N % N % N % N % June 2003–August 2003 0 0% 6 23% 1 3% 7 9% September 2003–November 2003 8 32% 1 4% 5 17% 14 18% December 2003–February 2004 8 32% 5 19% 8 28% 21 26% March 2004–May 2004 9 36% 10 39% 13 45% 32 40% June 2004–August 2004 0 0% 4 15% 2 7% 6 8% Total 25 100% 26 100% 29 100% 80 100% Informed Consent

The study team worked with the Housing First programs to consistently obtain clients’ informed consent to agree to participate in the study, as well as to assure confidentiality of all client information gathered for the study. Informed consent covered information about the purpose of the study, use of the data, assurances of confidentiality, contact persons who would know the participant’s location, and permission to obtain information about the participant from the service provider. (See exhibit A–4 at the end of this appendix for the Informed Consent Form.)

Data Collection

The study team tracked the full study sample of 80 cases on housing status, location, and other key outcome variables for the 12-month period following their program enrollment. To track these data, the study team relied on baseline and monthly data submitted by the Housing First programs. The study team trained staff at the programs to complete the baseline and monthly tracking forms. Because most clients stayed in the Housing First programs, this approach to data collection was feasible. If larger numbers of clients had left the Housing First programs, this approach might have been unworkable. The rate of departure for those clients at REACH who either refused participation in the study or were skipped was similar to that of the sample tracked

for the study. Of the 25 clients who refused or were skipped, four have left the program (compared to six clients leaving REACH of the 29 clients who participated in the study). The study team relied on a number of different case managers as well as administrative data sources to collect the data. To ensure consistent reporting, the study team provided training and ongoing assistance; however, respondent variability constitutes a limitation of the methodology. The rationale for this method to collect data is the clear impression that case managers would provide more accurate information than self report for some variables such as level of

impairment related to psychiatric symptoms or substance use.

An additional limitation of the study methodology was collecting participant data retrospectively. Such an approach requires case managers to recollect or reconstruct information, and it thus may result in less variation across months because data for several months are reported

simultaneously. In addition, the study team requested that case managers consult case files and assessment instruments to establish the most accurate responses for each data item at each point in time. To address this limitation during the data analysis phase, the study team studied changes in monthly trends by quarter or across longer spans of time such as between the first month of tenure and month 12.

The study team provided two incentive payments to the sites to encourage timely submission of monthly tracking forms. The Housing First programs received the first payment after it enrolled its sample and the second payment after it collected a full 12 months of data on the members of the study sample who still remained in the Housing First program.

All data collection and participant tracking procedures were vetted with Abt’s Institutional Review Board prior to implementation.

Baseline Data Collection

Once the sites selected the sample and the participants signed the Informed Consent Form, the sites collected baseline information on all participants. The study sites submitted baseline data shortly following the receipt of informed consent from each of the study participants. (See exhibit A5 at the end of this appendix for the Baseline Data Collection Instrument.) Retrospective and Monthly Data Collection

For each month that a study participant was in the Housing First program, the study site submitted a standard set of monthly data. For retrospective study participants, the study sites submitted all retrospective months of data with the first prospective monthly data. These data were the same although the study team developed a retrospective data collection form so that sites could easily report 6 months of data on one form. The study sites typically submitted these data to the study team one month following the month for which the data were collected. The study sites submitted these data for every month that the study participant was in the Housing First program. (See exhibit A–6 at the end of this appendix for the Monthly Data Collection Instrument.)

Tracking Leavers

The study sites were unable to provide monthly tracking data for those study participants who left the Housing First program in less than 12 months. To gather information about the leavers, the study team hired local researchers at each of the program locations to gather information about the participants who left the program. The local researchers provided information about the circumstances of the participants’ departures and, if the local researcher was able to find such information, where the participant was living following the leave from the Housing First

program. (See exhibit A–2.)

Exhibit A–2. Client Tenure in Housing First Programs

DESC Pathways to _Housing REACH Total Date of Enrollment N % N % N % N % Leavers 5 20% 2 8% 6 21% 13 16% Stayers 20 80% 24 92% 23 79% 67 84% Total 25 100% 26 100% 29 100% 80 100% Focus Groups

To obtain participants’ perspectives on the quality of their housing, satisfaction with their housing, and their “quality of life” experience with the Housing First program during their first year of placement, the study team conducted several focus groups at each of the study sites. These groups elicited client opinion on components of each program, such as use of the representative payee strategies and the program’s housing options, to address the question of why different approaches seem to work well. The focus group moderator asked clients how they came to the Housing First program and their satisfaction with the program, as well as their

feedback on how the Housing First program compares to other programs. The rationale for using focus groups was based on the study team’s past experience in utilizing this data collection methodology effectively with the target population of this study.

Several weeks in advance of the followup site visits, the study team sent invitations to the study sites for participants to attend focus groups. The hope was that the local trackers would be able to personally contact each client who left the program and invite them to a separate focus group; however, they were not able to do so and the study team was not able to conduct a “leavers-only” focus group. To increase focus group participation among the stayers, the study team provided food and offered participants a $10 incentive. The study team conducted two focus groups at DESC and REACH and three at Pathways to Housing. The focus groups ranged in size from 1 to 11 participants, with a total of 27 participants. Pathways to Housing had the smallest number of participants, most likely because it took place at the first of the month when clients received their benefits. (See exhibit A–3.)

Exhibit A–3. Client Participation in Focus Groups

Focus Group DESC Pathways to _Housing REACH Total

Focus Group 1 7 1 11 18

Focus Group 2 2 3 3 8

Focus Group 3 1 1

The turnout for the focus groups was less than the study team had hoped, particularly at

Pathways to Housing. However, the study team did take advantage of all opportunities to speak with clients about their program experience, whether in a focus group setting or in more informal discussions with one or two people. An experienced moderator from the study team led the focus groups and followed a discussion guide that included general questions as well as probes to stimulate conversation. The moderator was accompanied by a second person to take notes, help coordinate the event, and record the sessions.

During the analysis phase of the study, the study team reviewed the focus group notes and incorporated the focus group participants’ opinions and observations in the site profiles prepared for each Housing First program. The focus group participants provided useful insights on how clients enter the programs, whether they had choices in their housing or services, and how satisfied they were with their program experiences. Large numbers of clients did not participate in the focus groups and the views of those present are not representative of all clients at each Housing First program, but their input provided useful “reality checks” for the information obtained from program staff. (See exhibit A–7 for Focus Group Discussion Guides.)