The basic experimental design operated within the 2006–2010 NSFG 12-week quarter. During phase 1 (weeks 1–10 of each quarter), selected survey respondents aged 15–44 were offered
$40 to complete an interview. During phase 2 (weeks 11 and 12 of each quarter), a sample of approximately one-third of the remaining cases was selected. Adults aged 18–44 in quarters
2, 3, and 4 were randomly divided into two groups:
a) Group 1 received $10 prepaid in addition to the standard $40 (a total of
$50 for the main interview).
b) Group 2 received $40 prepaid in addition to the standard $40 at the beginning of the interview (a total of
$80 for the main interview).
These two groups are designated as the $10/$40 and the $40/$40
experimental conditions. Cases selected for phase 2 were sent a final letter via express mail with the prepaid incentive enclosed. The letter stated that the enclosed incentive was for the respondent to keep in appreciation for their help. (For conciseness, the $40/$40 group is referred to as the ‘‘$80 group’’
and the $10/$40 group is referred to as the ‘‘$50 group.’’)
Table I below presents the pooled results across the three quarters. The table shows counts of cases and response rates separately for household screener and main interview cases in order to evaluate the potential impact of the incentives in each group. Overall, the response rates for screener cases in phase 2 that were offered $80 for the main interview were 10 percentage points higher than for the screener cases offered $50 for the main interview (77%
compared with 67%, respectively). The response rates for main interview cases in phase 2 were 12 percentage points higher in the $80 group than the $50
group (64% compared with 52%, respectively).
NSFG study staff also wanted to know if the $80 incentive brought different types of people into the sample compared with the $40 or $50
incentives (Table II). The sample sizes in the experimental categories are small because only a one-third subsample of the remaining cases were selected for phase 2, and that subsample was randomly split into two payment plans (for women, n = 51 in the $50 group and n = 68 in the $80 group), but the sample size in the phase 1 group is 1,896. The split among men is similar:
n = 47 in the $50 group and n = 70 in the $80 group; the sample size in the phase 1 group is 1,432.
Despite the small subsamples in the phase 2 groups, 9 of the 13 differences between the phase 1 $40 and $80 groups were significant using two-tailed t tests. Only 3 of the 13 comparisons between the $40 group and the $50 group were significant. These results suggest that the $80 incentive was recruiting different people into the sample compared with the $40 group, but the $50 incentive was much less effective in that respect. Given due caution about the sample sizes, the patterns are clear:
For women:
+ Sixty percent of the $80 group was childless at the date of interview,
compared with 41% of women who received $40.
Because the principal outcome variable of NSFG is fertility and birth rates, this is a critical finding:
the $80 incentive was more effective in including childless women in the survey. (There was no significant difference in the group that received a $50 incentive.)
+
+ Among women who received $80, only 24%
lived in multiunit structures (e.g., apartments and condominiums), compared with 38% of women who received $40. (This was also true for men: 26%
compared with 37%, respectively.) These findings suggest that both men and women living in single-family homes were more likely to respond to the higher incentive.
For men:
+ Hispanic men were a larger percentage of men in the
$80 group (37% of men
Table I. Pooled quarters 2, 3, and 4 unweighted case counts for phase 2 incentive experiment outcomes, response rates, and simple random standard errors: National Survey of Family Growth, 2006–2010
Screener interview cases in phase 2
Experiment
Sample size
Completed screener
interviews Refusal Noninterview Nonsample
Response
Main-interview cases in phase 2
Experiment
Sample size
Completed main
Interviews Refusal Noninterview Nonsample
Response
NOTES: Teenagers aged 15–17 were not included in the experiment. Their token of appreciation for the interview was never more than $40. Randomized assignment of phase 2 cases to treatment groups was made on the segment level (e.g., all cases in a segment were assigned to the same treatment group). Therefore, the simple random sample standard errors are likely to underestimate the true standard errors.
groups of men. Given strong policy and program interest in representing Hispanic men in national surveys, this is also a key finding. phase 1 group. Given the strong public health-related interests in these behaviors, this is also a critical finding.
+
+ Among men who received
$80, 42% had incomes of
$75,000 or more a year, compared with 25% of men who received $40.
This suggests that the $80 incentive was more effective in drawing high-income men into the sample.
Conclusions
Despite relatively small samples in the two experimental groups, consistent results were obtained across three consecutive quarters: the $80 incentive raised response rates and recruited different people into the sample than did the phase 1 effort alone ($40 incentive).
Further, the results are broadly consistent with findings from the 2002 NSFG. The results suggest that childless, high-income people living in single-family homes are not as well represented in the standard phase 1 sample as they are when offered $80.
Drawing these people into the sample improves the representativeness of the sample and raises the response rate. This appears to justify the use of the $80 incentive for a small subset of the sample.
Given the costs of conducting the experiment and its effects on response rates, NCHS and study staff asked
permission to end the experiment after three quarters of data collection. NCHS’
Institutional Review Board, or its Research Ethics Review Board, granted this request on August 29, 2007 (Amendment 11, NCHS Protocol Number 2006–01); OMB granted the request on November 21, 2007. The $80 incentive in phase 2 was adopted in all subsequent quarters of the 2006–2010 NSFG.
Table II. Comparison of sample characteristics between the $10/$40 experimental groups in quarters 2, 3, and 4 in token of appreciation Survey of Family Growth, 2006–2010
and $40/$40