Work and Use of Demonstration Services - Evaluation Analysis Plan

For the second component of the participation analysis, in order to address the six research questions regarding work and use of demonstration services, the BOND team will use a conceptual framework that tracks how prospective BOND subjects move along a work path and, for treatment subjects, their use of the offset (Exhibit 3). Major markers on that path under current law appear in the top half of Exhibit

5-Abt Associates Inc. Evaluation Analysis Plan 98

3 and those under the BOND benefit offset appear in the bottom half of the exhibit. Beneficiaries may avail themselves of counseling services or obtain employment services under Ticket to Work (TTW) at any point along the path. The top half of the boxes in Exhibit 5-3 will be used to compare the work path of treatment and control subjects, while the bottom half of the boxes will be used to examine how treatment subjects use BOND services.

Exhibit 5-3. Beneficiary Work Path

At the time of enrollment into the demonstration, prospective BOND subjects may be anywhere along the current-law path after award and before the termination of benefits, which will have direct implications for their use of BOND services. The first group (Box 1) includes subjects who are not engaged in any work activities or support programs. The next two groups include subjects who are working and using employment services (Box 2) and/or have completed the TWP (Box 3), but are not yet at the point of using the offset. The final group (Box 4) includes subjects who have completed the TWP and are earning above SGA.

Treatment subjects along the work path can be classified into the following three primary groups for the participation analysis:

 Inactive subjects are those treatment subjects (T1, T21, T22) who have not engaged in any employment or employment related activities (e.g., TTW) since initial award. Presumably, these subjects are farthest away from using the offset given their work history since SSDI entry (Box 1 of the work path).

 Potential users are treatment subjects (T1, T21, T22) who did not receive BOND payments, but have engaged in activities that can lead to the use of the offset in the future (Boxes 2 and 3 in the work path).

Inactive (Not Working or Using Employment Services)

Employed and/or Using Employment

Services

TWP Completed

Earnings Above

SGA Following

TWP

Inactive Subjects (Box 1)

Potential Offset Users (Boxes 2 and 3)

Offset User (Box 4) BOND Treatment Subject Path to Becoming Offset Users

 Users are treatment subjects (T1, T21, T22) who have completed their TWP and received BOND payments for earning above the SGA (Box 4 in the work path).

5.4.1 To what extent do subjects in each treatment group work or use employment services and benefits counseling?

To address this research question, the participation analysis will include a summary of the size of the inactive, potential user, and user subgroups based on available BODS data. The BOND team will produce separate estimates for treatment subjects in Stages 1 and 2. It is likely that the size of these subgroups will differ between Stages 1 and 2 due to differences across stages in the target population and services (Exhibit 5-4). The Stage 2 potential offset user and user groups will likely be relatively larger than the corresponding groups from Stage 1 because the sample for Stage 2 includes only volunteers, and

volunteers are more likely to be working at the start of the demonstration relative to those assigned to T1.

Additionally, the type of work incentive counseling (WIC) the treatment subjects use also differs by group. As noted in Chapter Three, the T1 and T21 subjects will receive information on BOND work incentive counseling, referred to here as BOND-WIC, from the WIPA provider. This counseling is similar to that provided to the control group by the WIPA, except that WIPA providers will provide information on the BOND work incentives. The T22 subjects are expected to receive enhanced counseling services, EWIC (see Chapter Three).

The BOND team will track how the size of these groups changes over the course of the demonstration by computing work participation and employment service use rates at specific points in time relative to enrollment in the demonstration (e.g., at six months after random assignment, one year after random assignment, etc.).⁶⁰ We know from past research that beneficiaries who return to work and use the SSDI work incentives usually do so during their first few years on the rolls (Liu and Stapleton 2010). In the first year, very few beneficiaries work and use the work incentives. By the fifth full year after award, however, 23 percent worked in at least one year and 14 percent were employed during the year. Liu and Stapleton (2010) also found that over 7 percent completed the TWP by the end of the fifth year, and almost 5 percent had their benefits suspended for at least one month because of earnings above SGA. Less change is anticipated for long-duration beneficiaries. Although some of these beneficiaries will have been on the rolls for only four or five years, most of those who are assigned at Stage 1 will have been on the rolls for a much longer period. The longitudinal research shows that most beneficiaries who return to work and use the work incentives do so within their first five years on the rolls. It is certainly possible that the offset will stimulate new work activity for those in the long-duration group, but the expectation is that most users of the offset in this group will be dominated by those who are already working when they enter BOND. Results might be quite different for Stage 2 volunteers, however, because those not already engaged in work activity are much less likely to volunteer.

The BOND team will produce estimates for Exhibit 5-4 for each participation analysis report. Each completed exhibit will provide information on the current use of services, the potential use of service in future months (based on their position in the work path), and, more broadly, how many treatment subjects are on a work path. Separate statistics will be produced for short- and long-duration subjects, as well as totals weighted to reflect the differing sample weights for these short- and long-duration subjects.

60 Please refer to the timeline for the participation analysis (Exhibit 5-10) discussed in Section 5.5 for details.

Abt Associates Inc. Evaluation Analysis Plan 100 Exhibit 5-4. BOND Users and Potential Users in Stage 1 and Stage 2

Universe = Stage 1 Treatment Subjects Inactive (%)

Inactive (%)

Potential Offset Users and Offset Users

Offset Users (%)

Potential Offset Users and Offset Users

Offset Users (%) Offset and

BOND WIC

BOND WIC Only

Offset and EWIC Users

EWIC-Only BOND

WIC-only

Offset and BOND

WIC

Universe = Stage 2 Treatment Subjects

Source: TRF and MEF.

5.4.2 Who Works, Uses Counseling Services and Other Work Incentives, and Eventually Uses the Offset?

To address this research question, the participation analysis will include a descriptive summary of the current work activities of subjects within inactive, potential user, and user groups (Exhibit 5-5). This analysis will provide information on where subjects are along that work path and, hence, their potential use of the offset. It will also provide information on the work and service use characteristics of

participants in each group. This information will be useful for assessing whether subjects with certain work or service use characteristics, or living in certain areas, are more likely to be on the work path as a potential or actual offset user relative to other subjects.

This descriptive summary of work and work incentive use by those within each treatment group will build on the analysis in Exhibit 5-5 to provide a more in-depth look at where treatment subjects are relative to use of BOND services. The potential offset users group includes subgroups of treatment subjects who are at various stages along the work path. Those who have completed their TWP, especially current workers, are presumably closest to becoming offset users in the future (“high potential users”). Those who have not completed the TWP, especially those who are not currently working, presumably have lower potential for becoming users in the short-term. The offset user group includes the actual users of the offset since random assignment. The current user group provides an estimate of short-term BOND service use. The combination of current and past user groups provides a long-term view of treatment subjects who have ever used BOND services since random assignment. The past user group is important to track because these treatment subjects might return to offset use. The work activity statistics for the Stage 2 Offset-Only and Offset-EWIC treatment groups will be compared directly to determine whether these two groups are using the offset, or progressing toward offset use, at the same or different rates.

5.4.3 How did the demonstration affect the use of work-incentive counseling and the services delivered by counselors?

To address this question, the participation analysis will include a descriptive summary of work-incentive counseling services used by each control and treatment group (Exhibit 5-6). This analysis will provide information on how the use of counseling services differs by treatment status in each demonstration stage.

The analysis will produce statistics on: the percentage in each group that receive any service from a work-incentive counselor; mean service duration; the number and types of contacts made; findings from beneficiary assessments; and recommendations made by counselors. In addition, the analysis will include a summary of use of the additional services that the EWIC counselors are to provide (T22 subjects only).

Abt Associates Inc. Evaluation Analysis Plan 102 Exhibit 5-5. Work Activity Composition of Treatment Groups (T1, T21, and T22)

Characteristics

Total (weighted) Inactive

Not working or using supports Potential Users

Not completed TWP Currently working