Implementing Propensity Score Matching and Related Estimators This dissertation takes advantage of advances in statistical matching techniques to

estimate the causal effect of SBIR on firm outcomes. It also combines matching with regression-based methods. Following Ho, Imai, King, and Stuart (2007), I also use standard regression models (e.g. linear and logistic regressions) to estimate the effect of SBIR after balancing the data. This is to ensure comparison between comparable groups (following the Neyman-Rubin-Holland counterfactual approach) when performing regression analysis.

The central idea of matching is to control for observable heterogeneity by finding in the untreated group “look-alikes” of treated participants. When implemented

manually, matching is a tedious exercise. In practice, matching directly on observable attributes becomes more difficult the larger the set of covariates to match. This is called the “Curse of Attribute Dimensionality”. For illustration purposes, let us assume that we are looking for a “look-alike” or a match of a small business start-up that has 10

employees, is located in California, is currently competing in the computer equipment industry, performs in-house R&D, recorded a profit in 2005, and is managed by its owner-founder who has a postgraduate degree. Finding a close match (much less an exact match) of this SBIR-recipient small firm is very difficult if not impossible. This dimensionality problem can be significantly reduced by matching on the propensity score, i.e., the conditional probability of treatment or program participation. Thus,

instead of an empirical strategy of constructing a comparison group with identical covariates X, the alternative strategy, which this dissertation adopts, requires a

comparison group that has a similar distribution of covariates X with that of the treatment group by matching on the propensity score. The propensity score is formally expressed as:

Propensity Score = P(T=1׀ X) [16]

Thus, propensity score matching (PSM), which originated from Rosenbaum and Rubin (1983), is a statistical method to match treated and untreated cases on the basis of the propensity score, which is a scalar variable, instead of manually matching on a vector of variables. If the strong ignorability of assignment assumption26 holds, the use of the matched comparison group to construct the counterfactual outcome of treated cases is sufficient to remove selection bias, yielding a valid and consistent estimate of the mean impact of treatment (Heckman et. al, 1998; Rosenbaum and Rubin, 1983).

To summarize, the aim of matching is to balance the covariate distribution

between the treated sample and the matched comparison sample. An important statistical result from Rosenbaum and Rubin (1983) is that it is enough to match on the conditional probability of treatment or the propensity score. On average, observations with the same distribution of propensity scores will have the same distribution of observed covariates X. Thus, matching on propensity scores, the ATT estimator in [14] can be reexpressed as:

ATT EstimatorPSM = EP(Xi ׀T=1) [E(Yi ׀ Ti=1, Xi) - E(Yi ׀ Ti=0, Xi)] [17]

26 _{Strong ignorability of assignment assumes both (1) presence of overlap and (2) mean independence. See}

The PSM estimator of ATT in [17] implies that untreated observations whose propensity scores are outside the support of the propensity scores of the treated observations will be discarded.

To implement PSM, this dissertation followed the following matching protocol (Caliendo and Kopeinig, 2008, 2005) to construct the comparison group for treated firms. First, I divided small business start-ups into (1) those that receive SBIR financing (the treatment group) and (2) those that did not (potential control group). Second, I ran a logistic regression to model the participation of small business start-ups in the SBIR program and obtain estimates of their propensity scores. PSM predicted the probabilities of participation (propensity scores) of both treated and untreated small business start-ups using relevant covariates to be discussed in section 4.3. The propensity score model included variables that affect both treatment assignment and outcomes (Rosenbaum & Rubin, 1983; Gelman &Hill, 2007).

Third, I excluded from the sample non-recipient small business start-ups whose propensity scores are either (1) lower than the minimum propensity score of the recipient small firms or (2) higher than the maximum propensity score of the recipient firms to satisfy the key identifying assumption of the PSM estimator, which is the presence of a “common support” between the two groups.

Fourth, I paired each participant i with some group of comparable non-

participants on the basis of the estimated propensity scores. I used the nearest neighbor matching algorithm i.e., search for non-participant j with the closest propensity score. I followed Abadie and Imbens (2002) who suggested using four matches for each treated participant.

Fifth, I assessed matching quality. The matching procedure should balance the distribution of the relevant independent variables in both the treatment and the

comparison group. After the matching, the covariates should be balanced in both groups and hence no significant difference should be found. If there are significant differences, covariate balancing is not completely successful and remedial measures are necessary. For instance, Caliendo and Kopeinig (2005) recommended including high-order polynomial terms and/or cross-product interaction terms in the estimation of the propensity score to improve the match between the treatment and comparison groups.

Sixth, I computed the treatment effect as the difference between the mean outcome of the treatment group and the mean outcome of the comparison group. Specifically, the input additionality effect is the difference in the mean R&D

expenditures of SBIR recipients and the mean R&D expenditures of observationally similar non-recipient small business start-ups and the certification effect is the difference in mean external financing. Estimating the treatment effect on other firm-level outcomes (e.g. employment and innovation propensities) followed the same approach. For

statistical inference, the standard error of the treatment effect was estimated using Abadie and Imbens’ (2006) bias-corrected variance estimator.

In the treatment effects analyses, the size of the comparison and treated

subsamples varies from one model to another. PSM balances the covariate distribution of the recipient and non-recipient groups by dropping untreated observations that are not observationally similar to the treated cases. Recipient small firms that are off common support and those with missing values for a particular outcome variable will also be dropped from the treatment effects analyses.

Finally, following Ho, Imai, King, and Stuart (2007) and Gelman and Hill (2007), I also estimated the treatment effect by using regression-based methods after the

observable characteristics of the treated and matched comparison subsamples.

Regression analysis was only applied within the common support of X between the two groups. For example, a linear regression as in [10] can be estimated. In the case of a dichotomous outcome variable, the following logistic regression where  is the key parameter of interest can be fitted by maximum likelihood estimation.

log (Yi/1-Yi) = τ+ Ti + Zi’γ + εi [18]

In a regression framework, the estimate of the treatment effect is the coefficient () of the binary treatment variable T. The regression coefficient  is interpreted as the difference in mean outcomes between SBIR recipients and non-recipients, holding constant a set of confounding variables Z in the model. For statistical inference, the OLS variance estimate V(α^) is:

V(α^) = s2 (X’X)-1 [19]

where the estimate of the error variance (s2) = SSE/(n-k) = e’e/(n-k).27