Robustness

In document Essays on natural resources and labor economics (Page 35-38)

CHAPTER 2. IS OUTDOOR RECREATION RECESSION PROOF? AN

6 Robustness

We conduct three robustness checks. First, we use a placebo recession year to check if our general assumption of no differential trend for treatment and control groups for DID matching estimator is valid in our setting. Second, we change the specification for propensity score estimation including a subset of covariates previously used: we exclude recreation preference variables. Third, we match each treatment observation with controls from the same geographic region to control for unobservable factors that are time-variant in a spatial manner.24

6.1 Placebo exercise

The objective of the placebo exercise is to check whether it is unemployment during the recession or some pre-existing unobservable factors working differently across the treatment and control group are driving our results. If the treatment and control group exhibit differentiated trend in the pre-recession years, and recession truly has no impact on recreation, the DID matching estimator picks up this difference in trend as impact of the change in employment during recession.

For the placebo exercise, we assume year 2003 as placebo recession year.25 Table B1-B3 in the appendix A report that balancing of covariate is satisfied in all cases except for only one covariate

in one matching process for treatment group one. We report the estimates from the Placebo exercise in Table 7 and 8. In all matching processes, neither participation nor frequencies of trips

24 Rural and urban areas may be affected differently during a recession.

25Although Iowa lake project survey was conducted in 2003 and 2004 as well, we have a matched non-missing sample for all of our treatment and control group observations in year 2003. In the survey year 2004 , we have missing information for 8 treatments and 52 controls from the sample of 971 observations that we are using for our base estimation

in lake recreation turned out to be statistically different across the treatment and control groups in 2003. This finding gives us confidence in saying that our analysis based on matching exercises as reported in the previous section are not contaminated due to differential group trend.

6.2 Specification without recreation preference variable

We estimate the propensity score excluding the recreation preference variables and including only demographics and type of region for residence in the pre-recession year. Table 9 reports the estimates for participation.26 For treatment group 1 and 3, the treatment effect estimates on participation in lake recreation follows the same pattern that we observe previously in Table 5. For these two treatment groups, the difference-in-difference matching estimates are also robust to this different set of observables covariates. Panel b in Table 9 reveals that unemployed and part-time employed households (treatment group 2) participate more in lake recreation during the recession compared to what they would have done had they been employed. Note that, for the DID matching, previously in Table 5, radius matching estimators did not show any statistical significance but under the new setting, all five matching algorithms exhibit statistical significance. For matching on the level of participation, only radius matching estimators show statistical significance whereas previously in Table 5 all five matching estimators turned out to be statistically significant.

For the frequency of lake trips, the results are reported in Table 10. With the new set of covariates, none of the matching estimators across the three treatment groups exhibit statistically any significant effect of a change in employment status during recession on frequencies of trips.

Our previous finding that frequencies of lake trips do not change due to unemployment or retirement during the recession is robust to the choice of covariates.

26 Covariates balancing results, as reported in table C1-C3 in the appendix, reveal that quality of the match is good.

Except for one variable for treatment group one in one estimation process, all other covariates balance well.

6.3 Matching within RUCA cell

Although we have accounted for the effects of mean time-invariant unobservable through matching on the differences, we recognize that we still might end up finding estimates confounded by unobservable fixed factors that vary across regions with time. For example, rural and metropolitan areas may be affected differently during a recession year and exhibit different economic environment. Employment statistics in a rural agricultural county may not change during the recession while employment in the metropolitan area usually drops sharply during the economic crisis. Although we incorporate information on one’s residence location while estimating the propensity score, we still may end up matching a rural treatment with an urban control. Our DID matching estimators cannot control for such region-specific time-variant unobservable confounding factors. So matching individuals within region can control for such time-variant confounding effects.

To control for such possible regionally time-variant confounders, we match each treatment observation with controls from the same RUCA region. An individual from a small town experiencing employment shock during the recession is matched with counterfactuals from a small town area rather than from a metropolitan or rural area. Since we will match exactly within RUCA cell, in the first step, we estimate propensity score excluding variables on geographic regions. The results are reported in Table 11 and Table 12. Table D1-D3 report covariate balance for the cell matching. In contrast to the previous exercises, quality of the matches is not satisfactory here for the treatment group one and three since some covariates do not balance after matching. However, covariates balance well for the treatment group two- the unemployed group.

The estimates in Table11 reveals that when matching is done within the RUCA cell, only one out of five matching estimators for treatment group 1, and none for treatment group 3 show

statistically significant effect of change in employment status on lake trips at the extensive margin.

For treatment group two, two out of five matching estimators turn out to be statistically significant.

However, once we apply the difference-in-difference matching, only for treatment group 1 the estimates display statistical significance. For treatment group 2 and 3, all five matching estimators are statistically insignificant. It suggests that for the unemployed and part-time employed group, positive impact of the recession on participation in lake recreation is not robust once we control for spatially time-variant confounders. In the case of frequencies of lake trips, as can be seen in Table 12, matching within RUCA cell generates similar estimates as before. However, since matching within rural-urban region causes quality of matching to be low, we are cautious in interpreting the estimates for treatment group one and three.

In document Essays on natural resources and labor economics (Page 35-38)