This section (i) summarizes the evaluation’s outcome data sources and (ii) illustrates how impact findings will be presented in evaluation reports in tables indicating data sources.
6.2.1 Summary of Data Sources
As discussed in Chapter Three, the net impact evaluation will draw on the following data sources:
Administrative data from SSA and other agencies, including the Centers for Medicare &
Medicaid Services (CMS) and the Rehabilitation Services Administration (RSA);
Stage 1 and Stage 2 surveys; and
BOND Operations Data System (BODS) data, which includes data from random assignment, outreach, recruitment, intake, earnings data collection, and data exchange between BOND operational components and SSA.
The use of these sources to obtain the outcome measures in Exhibit 6-1 above is indicated in Appendix B.1. Specific examples appear in the table shells below.
6.2.2 Presentation of findings on earnings and employment
A full set of table shells for the impact analysis appears in Appendix B.2. To illustrate, Exhibits 6-3 and 6-4 provide table shells for Stage 1 and Stage 2 impacts on earnings and employment. As seen in these exhibits, SSA administrative records will allow for an examination of how impacts evolve over time. The three Stage 2 comparisons (T21 vs. C2, T22 vs. C2, and T22 vs. T21) will be shown together in the same table in order to minimize the total number of tables.
Exhibit 6-3. Stage 1 Impacts of BOND on Earnings and Employment: T1 Versus C1
Outcome
Treatment Mean
Control Mean
Impact Estimate Total Earnings
Total Earnings in RA Year and 4 Years Following $nnn $nnn $nnn
(nnn) Time Path of Earnings
Earnings in Year before RA Year $nnn $nnn $nnn
(nnn)
Earnings in RA Year $nnn $nnn $nnn
(nnn)
Earnings in First Year after RA Year $nnn $nnn $nnn
(nnn)
Earnings in Second Year after RA Year $nnn $nnn $nnn
(nnn)
Earnings in Third Year after RA Year $nnn $nnn $nnn
(nnn)
Earnings in Fourth Year after RA Year $nnn $nnn $nnn
(nnn) Employment
Any Employment in RA Year and 4 Years Following
0.nn 0.nn 0.nn
(0.nn) Number of Years With Any Employment in RA
Year and 4 Years Following
n.nn n.nn n.nn
(n.nn) Time Path of Employment
Any Employment in Year before RA Year 0.nn 0.nn 0.nn
(0.nn)
Any Employment in RA Year 0.nn 0.nn 0.nn
(0.nn)
Any Employment in First Year after RA Year 0.nn 0.nn 0.nn
(0.nn)
Any Employment in Second Year after RA Year 0.nn 0.nn 0.nn
(0.nn)
Any Employment in Third Year after RA Year 0.nn 0.nn 0.nn
(0.nn)
Any Employment in Fourth Year after RA Year 0.nn 0.nn 0.nn
(0.nn) Source: Social Security Administrative Earnings records.
Sample Sizes: T1 = 80,000; C1 = nnn,nnn.
Notes: *** = p<.01, ** = p<.05, * = p<.10 in two-tailed t-test. Robust standard errors in parentheses. Impact estimates are regression-adjusted. All statistics are computed using analysis weights reflecting sample design.
Abt Associates Inc. Evaluation Analysis Plan 120 Exhibit 6-4. Stage 2 Impacts of BOND on Earnings and Employment:
T21 Versus C2, T22 Versus C2, T22 Versus T21
Outcome
Total Earnings in RA Year and 3 Years Following Time Path of Earnings
Earnings in Year before RA Year $nnn $nnn $nnn $nnn Earnings in First Year after RA
Year Earnings in Second Year after RA
Year Earnings in Third Year after RA
Year
Any Employment in RA Year and 3 Years Following Number of Years With Any
Employment in RA Year and 3 Years Following
Time Path of Employment Any Employment in Year before
RA Year Any Employment in First Year after
RA Year Any Employment in Second Year
after RA Year Any Employment in Third Year
after RA Year Source: Social Security Administrative Earnings records.
Sample Sizes: T21 = 4,800; T22 = 3,000; C2 = 4,800.
Notes: *** = p<.01, ** = p<.05, * = p<.10 in two-tailed t-test. Robust standard errors in parentheses. Impact estimates are regression-adjusted. All statistics are computed using analysis weights reflecting sample design.
6.2.3 Presentation of findings on SSDI Benefits
Exhibits 6-5 and Exhibit 6-6 show sample table shells for Stage 1 and Stage 2 impacts on SSDI benefits and receipt.
Exhibit 6-5. Stage 1 Impacts of BOND on SSDI Benefits and Receipt: T1 Versus C1
Outcome
Treatment Mean
Control Mean
Impact Estimate Total SSDI Benefits
Total SSDI Benefits in Years 1-6 after RA $nnn $nnn $nnn
(nnn) Time Path of SSDI Benefits
SSDI Benefits in Year 1 after RA $nnn $nnn $nnn
(nnn)
SSDI Benefits in Year 2 after RA $nnn $nnn $nnn
(nnn)
SSDI Benefits in Year 3 after RA $nnn $nnn $nnn
(nnn)
SSDI Benefits in Year 4 after RA $nnn $nnn $nnn
(nnn)
SSDI Benefits in Year 5 after RA $nnn $nnn $nnn
(nnn)
SSDI Benefits in Year 6 after RA $nnn $nnn $nnn
(nnn) SSDI Receipt
Number of Years With Any SSDI Receipt in Years 1-6 after RA
n.nn n.nn n.nn
(n.nn) Time Path of SSDI Receipt
Any SSDI Receipt in Year 1 after RA 0.nn 0.nn 0.nn
(0.nn)
Any SSDI Receipt in Year 2 after RA 0.nn 0.nn 0.nn
(0.nn)
Any SSDI Receipt in Year 3 after RA 0.nn 0.nn 0.nn
(0.nn)
Any SSDI Receipt in Year 4 after RA 0.nn 0.nn 0.nn
(0.nn)
Any SSDI Receipt in Year 5 after RA 0.nn 0.nn 0.nn
(0.nn)
Any SSDI Receipt in Year 6 after RA 0.nn 0.nn 0.nn
(0.nn) Source: Social Security Administrative TRF records.
Sample Sizes: T1 = 80,000; C1 = nnn,nnn.
Notes: *** = p<.01, ** = p<.05, * = p<.10 in two-tailed t-test. Robust standard errors in parentheses. Impact estimates are regression-adjusted. All statistics are computed using analysis weights reflecting sample design.
Abt Associates Inc. Evaluation Analysis Plan 122 6.2.4 Presentation of findings on other outcomes
Table shells for impacts on other employment-related outcomes, other benefits, training-related outcomes, health and functioning, and material hardship are shown in Appendix B.2, for all the outcome measures in Exhibit 6-1 above.
Exhibit 6-6. Stage 2 Impacts of BOND on SSDI Benefits and Receipt:
T21 Versus C2, T22 Versus C2, T22 Versus T21
Outcome
Total SSDI Benefits in Years 1-5 after RA Time Path of SSDI Benefits
SSDI Benefits in Year 1 after RA $nnn $nnn $nnn $nnn
Number of Years With Any SSDI Receipt in Years 1-5 after RA
n.nn n.nn n.nn n.nn Time Path of SSDI Receipt
Any SSDI Receipt in Year 1 after RA 0.nn 0.nn 0.nn 0.nn Source: Social Security Administrative TRF records.
Sample Sizes: T21 = 4,800; T22 = 3,000; C2 = 4,800.
Notes: *** = p<.01, ** = p<.05, * = p<.10 in two-tailed t-test. Robust standard errors in parentheses. Impact estimates are regression-adjusted. All statistics are computed using analysis weights reflecting sample design.