This section describes how the team will use the qualitative and quantitative data to address the input/process question posed in Section 4.2: How has the intervention been implemented within and across sites, and how has the implementation changed over time? The top section of Exhibit 4-3 summarizes the data sources for addressing this research question. The answer will be in the form of a comprehensive description of BOND’s implementation.
Researchers will use the process study data to analyze and document the implementation of BOND across the study sites throughout the demonstration period. These findings will inform our understanding of the experiences of site office staff, WIC and EWIC providers, community partners, and beneficiaries participating in the BOND demonstration. Interviews, observations, participant focus groups, and program materials will play a role in describing the implementation of BOND within the study site.
Statistics from BODS; SSA administrative data; and national, state, and community databases will supplement this material. Five primary areas will be examined:
Associates Inc.Evaluation Analysis Plan73 ND Implementation and EvaluationContract No. SS00-10-60011 Exhibit 4-3. Data Sources and Topics Used to Answer Process Study Research Questions
Research Topics DESCRIBE THE IMPLEMENTATION OF BOND
Demonstration Site
FIDELITY TO DESIGN AND DIFFERENCES BETWEEN WIC AND EWIC SERVICES Recruitment and
IDENTIFY LESSONS FOR IMPLEMENTATION, POLICY AND FUTURE RESEARCH
Lessons Learned X X
Abt Associates Inc. Evaluation Analysis Plan 74
Demonstration site selection, planning, and startup. Based primarily on the first two site visits, the team will develop detailed documentation of the site baseline and how it changed with the
implementation of BOND. This documentation will include descriptions of the administrative and staffing structure, service delivery process, and the early implementation challenges and lessons learned. It will also include documentation of how and why decisions were made and the range of options considered.
Demonstration context. Understanding the site environment may help explain what factors influence the BOND implementation, level of fidelity to the BOND model, and ability of sites to achieve impacts. This analysis will rely on descriptive data from the site visits and quantitative data from BODS (for example, demographic characteristics of treatment and control groups);
SSA administrative data (for example, number of SSDI and concurrent beneficiaries;
demographic characteristics of SSDI/concurrent beneficiaries such as gender, race/ethnicity, age group, disabled adult child, and primary and secondary diagnoses; number of beneficiaries;
number of beneficiaries working; and use of Ticket-to-Work prior to BOND); and national, state, and community databases (for example, population density; employment and unemployment statistics; and measures of the population’s socioeconomic status, education, and income statistics).
Beneficiary notification, recruitment, and enrollment. The team will document in detail notification, recruitment, and enrollment processes. This documentation will include descriptions of the level and types of community outreach. For Stage 1, this includes the process for mailing letters and informing respondents about the demonstration. For Stage 2, it includes the
recruitment and enrollment process, staffing capacity, types and amount of information
disseminated, involvement of community partners, and subjects’ perceptions of the recruitment and enrollment process.
Development and structure of EWIC and WIC organizations. A key component of the process study is description of the provision of EWIC and WIC services for Stage 2 treatment groups. The research team will use qualitative data gathered onsite to describe the structure and process for providing EWIC and WIC services. These data will include background on the WIC provider and other organizations that may have been considered as potential EWIC providers (gathered in the first site visit), and information on selection of the EWIC provider (gathered in the initial implementation site visit). Later visits will provide the information needed to assess organizational capacity to provide timely EWIC and WIC services and the quality of the service provided. Information gathered from focus groups of beneficiaries as well as the interviews with site office and provider staff will play an important role in developing the findings.
Accessing the benefit offset. Analysts will use site visit information to document the operations of the processes established to obtain earnings information, support TWP determinations, support benefit adjustments, and help treatment subjects understand the notices they receive from SSA.
Initial problems with implementation inputs are to be expected, and may be identified in the early site visits. Subsequent site visits, and later reports will explore the extent to which earlier problems have been addressed and how.
Cross-site variation in implementation inputs is also expected. The process analysis will highlight notable variations across sites. It will also consider whether site characteristics correlate with different patterns of intervention inputs—characteristics including (1) site size, (2) geographic distribution of beneficiaries, (3) economic climate, (4) availability of resources for individuals with disabilities (for example, SVRA and EN services, One Stop Employment Centers, disability support organizations), (5) level and types of involvement from community partners, (6) accessibility of counseling services for treatment and control subjects, (7) access to Medicaid Buy-in (MBI) programs, and (8) local receptivity and support for BOND.
It will not be possible to establish causality, however, with just 10 sites and eight or more characteristics to consider. However, the interpretation key informants put on linkages between site characteristics and intervention implementation will provide useful hypotheses for possible later, larger studies of the implementation of BOND-like provisions.