10.1177/0891242404265399ARTICLE ECONOMIC DEVELOPMENT QUARTERLY / August 2004 Rogers, Tao / QUASI-EXPERIMENTAL ANALYSIS
Quasi-Experimental Analysis of Targeted
Economic Development Programs:
Lessons From Florida
Cynthia L. Rogers
Jill L. Tao
University of Oklahoma
The authors highlight three sources of endogeneity bias that typically haunt analyses of local government policies, and offer an empirical methodology for estimating pro-gram impacts given such concerns. They investigate Florida’s experience with im-plementing two common targeted economic development policies, community redevelopment areas (CRAs) and enterprise zones (EZs). Developing a simple appli-cation decision model as a guide, they find significant differences in policy implemen-tation for small cities compared with larger cities. Florida’s small-city program implementation offers a unique opportunity to compare areas that received state-level approval for the programs with all areas that qualified for but did not receive designa-tion using a quasi-experimental framework. In so doing, the authors explicitly address the potential for endogeneity bias caused by program rationing on the part of adminis-trators and by nonrandom targeting of distressed areas. Consistent with existing research, they do not find evidence validating the efficacy of targeted development programs for small cities.
Keywords: enterprise zones; community redevelopment; quasi-experimental research design; endogeneity bias; targeted development
State-level economic development policies that target distressed communities have become increasingly common over the past 20 years. Two of the most ubiquitous policies at the state level are community redevelopment areas (CRAs) and enterprise zones (EZs). Both policies attempt to target assistance to businesses in economically distressed neighborhoods. Such programs are politically viable, particularly in southern states, because they do not entail a direct tax loss. State and local policy makers continue to pursue such policies despite the apparent lack of conclusive supportive research.
Methodological problems inherent in previous empirical investigations contribute to the uncer-tainty regarding the efficacy of targeted economic development programs (Bartik, 1991; Blank, 1997; Lambert & Coomes, 2001; Mossberger, 2000). Recent research stresses the importance of using a unit of analysis consistent with substate, geographically targeted programs (Boarnet, 2001;
Cynthia L. Rogers is an associate professor of economics at the University of Oklahoma. Her work has appeared in Journal of Labor Economics, Journal of Urban Economics, Regional Science and Urban Economics, and Review of International Economics.
Jill L. Tao is an assistant professor of political science at the University of
Oklahoma. Her work has appeared in Economic Development Quarterly, International Journal of Economic Development, and in the forthcoming Critical Evaluations of Economic Development Policies.
AUTHORS’ NOTE: We would like to thank Arnold Fleischmann, Timothy Bartik, Peter Eisinger, Larry Bennett, and a number of anonymous reviewers for comments on previous drafts. This research was supported by a grant through the Tennessee Valley Authority Rural Studies Program. We each contributed equally to the work and accept all responsibility for errors or omissions.
ECONOMIC DEVELOPMENT QUARTERLY, Vol. 18 No. 3, August 2004 269-285 DOI: 10.1177/0891242404265399
Engberg & Greenbaum, 1999; Tao & Feiock, 1999). Endogeneity bias is also blamed for the lack of measurable success in evaluations of targeted policies (Boarnet & Bogart, 1996; Erickson, Friedman, & McCluskey, 1989; James, 1991; Jones, 1990; McDonald, 1997; Papke & Papke, 1990; U.S. General Accounting Office, 1988). Endogeneity bias arises when communities im-plementing policies are likely to fare better or worse on average with respect to outcome measures relative to communities that do not implement such policies (Mossberger, 2000).
We offer the Florida experience as an ideal case that allows for an innovative substate analysis of program efficacy. As the first state in the nation to enact state-level EZ legislation in 1980, Florida remains a front-runner in setting development policy (Florida Office of the Auditor Gen-eral, 1993). The state’s implementation of targeted development programs facilitates a quasi-experimental analysis, an approach recommended when endogeneity bias is a concern (Bartik, 1991; Boarnet, 2001).1 Reed and Rogers (2003) demonstrated situations in which a quasi-experimental estimator would outperform conventional regression estimators when faced with endogeneity bias. Our analysis differs from other studies using a similar design by including data from all qualifying areas, including those that did and did not implement targeted policies.2This focus allows us to give careful consideration to endogeneity of policy implementation with respect to outcome measures.
We contribute to the literature by investigating three sources of endogeneity bias associ-ated with the implementation of targeted economic development programs. The most widely rec-ognized source arises because of the potential preexisting differences in economic distress for the zone and nonzone areas. The programs are available only to distressed areas, which are likely to experience less economic growth regardless of the program impacts. The qualifying criteria are determined at the state level and differ across states. Florida is the only state to identify all distressed areas within its borders prior to program implementation. Consequently, we elimi-nate this target bias by including only areas meeting the same socioeconomic criteria in the analysis.
Another recognized source of endogeneity bias involves the process by which program admin-istrators determine which applications receive program designation (Boarnet & Bogart, 1996; Mossberger, 2000). Program administrators, for example, may give the most distressed areas higher priority when faced with resource constraints, or they may cream the cities most likely to successfully implement the programs. A striking feature of Florida’s experience is that all applica-tions from small qualifying cities—those with populaapplica-tions between 7,500 and 20,000 in the 1980 census—were approved. Consequently, our analysis of areas in small cities substantially elimi-nates the potential for nonrandom sampling bias caused by program rationing on the part of program administrators.
A third, and less widely acknowledged, source of endogeneity bias involves the decision to apply for program designation (Blank, 1997; Bondonio & Engberg, 2000; Mossberger, 2000). In particular, cities with better administration and strategic planning may be more likely to submit applications and more likely to benefit from the programs.3Florida’s experience offers insights regarding the nature of the endogeneity bias caused by the self-selection of qualifying cities. Given data for all qualifying cities and the knowledge about the approval status for small cities, we are able to identify factors associated with the application decision for small cities. We also develop a simple model to investigate the choice between applying for EZ or CRA designation and find that small cities prefer the EZ programs.
Consistent with the previous literature, we cannot conclude that the targeted economic develop-ment programs have enhanced economic growth in small Florida cities. We reach this conclusion, however, after addressing key criticisms of previous studies that cast doubt on the findings of poor performance for such policies. In particular, our evaluation reveals important insights regarding potential sources of endogeneity bias associated with program implementation. In this way, we narrow the scope of possible interpretations of the conventional wisdom regarding program effi-cacy and help to refocus future assessments.
OVERVIEW OF FLORIDA’S
TARGETED DEVELOPMENT PROGRAMS
EZs and CRAs are two popular programs among the growing number of targeted economic development instruments available to local governments. Targeted development tools attempt to funnel resources from outside a local government’s jurisdiction toward areas within the jurisdic-tion that are lagging. We focus on these specific tools because they require local governments with qualifying areas to apply for state sanction. This offers the opportunity to investigate the applica-tion decision and outcomes of all areas facing a similar level of distress. In addiapplica-tion, although both programs were established at different times, they were implemented almost contemporaneously, thus offering a unique opportunity to conduct a comparative assessment of policy adoption at the local level.
Florida’s CRA Programs
As an outgrowth of the Community Redevelopment Act of 1969 (1980), CRAs are defined as geographically bounded areas that fall within the jurisdiction of either a city or county government. Following a supply-side approach toward economic development, the logic behind CRAs is that private investment will take hold only when local market conditions are stabilized through govern-ment intervention. Establishing a CRA is fairly straightforward. A local governing body, such as a city council or county commission, passes an ordinance establishing a redevelopment board, which then designates a “redevelopment area”and then drafts a redevelopment plan. The plan out-lines how the local government will improve infrastructure in the area to boost private sector investment.
The advantage of establishing a CRA was greatly enhanced by the 1977 amendment to the Community Redevelopment Act (1980). This granted local governments the authority to imple-ment increimple-ment financing within CRAs. Much like the old industrial revenue bonds, tax-increment financing eliminates the need to raise funds through local taxes. Instead, it allows local governments to issue bonds using the redevelopment trust fund as collateral. However, the consti-tutionality of this measure was challenged in 1979. Only after the state’s supreme court resolved the matter in 1980 (Florida Senate Committee on Economic, Community, and Consumer Affairs, 1981) were redevelopment agencies allowed to go forward with tax-increment financing of rede-velopment trust funds. The ability to implement tax-increment financing increased the appeal of CRAs. As evidence of this, only two cities in Florida applied to and received designation for the CRA program prior to 1981.
Florida’s EZ Programs
In 1980, the Florida legislature created the nation’s first state-sponsored EZ program. The Florida EZ program represents an economic development policy initiative based on the assump-tion that localities benefit when private sector actors receive incentives to invest in blighted neigh-borhoods. The program allows local governments the opportunity to apply for state tax relief in the following areas: sales tax on purchases of renovation materials, corporate income tax, sales tax on business property purchased within a zone, sales tax exemption on electrical power provided by a municipally owned utility, and a variety of sales tax credits for businesses that hire zone residents or recipients of public assistance, such as the (former) Aid to Families with Dependent Children, who reside anywhere within city limits. The EZ programs give local governments a variety of policy options, but they are leveraged at the state level.
Unlike later programs in other states, the Florida legislature sought to make EZs business friendly by not requiring local governments to provide job training or community development programs per se. All expected outcomes that might be classified as improvements to the areas were to be achieved through the aggregate effects of individual businesses taking advantage of the tax incentives. Local governments might also promise to improve services to blighted areas, such as
In 1980, the Florida
legislature created the
nation’s first
state-sponsored enterprise
zone program. The
Florida EZ program
represents an economic
development policy
initiative based on the
assumption that localities
benefit when private
sector actors receive
incentives to invest in
blighted neighborhoods.
dedicating fire and police personnel, but were not required to provide evidence that such dedica-tion was in place before programmatic approval. As such, this approach privatized the job of policy implementation by placing policy success in the hands of private firms. Furthermore, in contrast to more traditional economic development programs, the EZ program specifically targets employ-ment rather than job creation as a primary programmatic goal (Pressman & Wildavsky, 1984, pp. 147-162).
Application and Implementation Procedures: CRAs Versus EZs
The application processes for receiving EZ or CRA designation are somewhat different but require similar expenditure of effort on the part of local governments. To receive designation for a CRA, local governments must create a redevelopment board, which identifies the area in question for redevelopment, drafts a redevelopment plan, and submits it to the Florida Department of Com-munity Affairs (FDCA) for final approval.4EZ designation does not require the drafting of a plan, but it does require the submission of an application that identifies the area in question, names a zone coordinator who will oversee program implementation at the local level and serve as a liaison with state personnel, and outlines the local resources that will be dedicated to the area, as well as any additional support the local government may be receiving from other sources, such as urban development action grant eligibility or community development block grants. In practice, local governments applying for CRA approval simply designate themselves (i.e., city council or county commission) as the redevelopment board and draft plans with the help of FDCA personnel. The practice for EZs is similar; the zone coordinator is often someone already on staff, and FDCA works with local governments submitting applications for zone status. Thus, the decision to apply for one designation versus the other is not predicated on an appreciable difference in the effort required to apply.
Although targeted development programs require state sanction and local initiative, there are differences in the extent of local involvement in the implementation of the programs. The intent for establishing a redevelopment area is more redistributive in nature compared with EZ programs. CRAs seek to target the expenditure of local revenues within economically depressed areas of a city. EZs seek to provide business investment within economically depressed areas of a city through tax incentives, with employment incentives offered through tax expenditure at the state level. Thus, whereas CRAs allow local governments to augment budgets through an expanded ability to issue bonds, there is still an expectation that expenditures from the local budget will be focused on the area in question. Small local governments may have a more limited ability to redi-rect spending as compared with larger city governments. In contrast, there is no such expenditure involved at the local level with an EZ. Accordingly, the implementation costs for the EZ programs are likely to be less than for the CRA programs.
IMPLEMENTATION OF PROGRAMS IN FLORIDA
In anticipation of the creation of a federally sponsored initiative under the newly elected Reagan administration, the legislature authorized the FDCA to conduct a statewide study to identify all areas that “meet minimum standards of physical, socioeconomic and fiscal distress” (Florida Sen-ate Committee on Economic, Community, and Consumer Affairs, 1981, p. 10). These measures are outlined as follows:
1. Physical distress is defined as at least 37% of the housing in the area built before 1940, at least 15% of the housing in the area lacking some or all plumbing facilities, or at least 25% of the structures in the area declared dilapidated by the local code enforcement agency. 2. Socioeconomic distress is defined as at least 34% of the households in the area with incomes
below poverty level, per capita income in the area less than $3,750 (50% of the state aver-age), or at least 10% of families in the area receiving public assistance or public welfare income.
3. Fiscal distress describes areas where per capita taxable value of property in the area is less than $9,820, local taxes per capita are more than $46, or local fiscal effort ratio (a measure of tax effort in relation to the taxable base) exceeds 0.015 (Florida Senate Committee on Eco-nomic, Community, and Consumer Affairs, 1981, p. 12).
By 1983, the FDCA had identified and geographically defined at the census tract level all areas within the state that qualified as distressed (Florida Department of Community Affairs, 1983). The 98 qualifying areas were distributed throughout the state across all regions, from the coastal areas to the central interior to the northern panhandle. About half of the qualifying areas were in small cities, which we categorize as cities with 1980 populations of less than 20,000.5Figure 1 shows the distribution of qualifying areas in small cities in relation to the Bureau of Economic Analysis (BEA) economic area definitions. The economic areas are generally larger than census metropoli-tan areas because they group metropolimetropoli-tan areas and economically linked suburban areas together. The distribution is similar for the large qualifying cities. As we explain below, whereas key infor-mation about program implementation process is unknown for the larger cities, the small city program lends itself particularly well to a quasi-experimental analysis.
Designation Status
For a qualifying area to be deemed either an EZ or CRA program, the local government had to apply for designation and be approved by the FDCA.6Not all applications were approved, and not
Figure 1: Small Cities With Qualifying and Designated Areas by Bureau of Economic Analysis (BEA) Economic Areaa
NOTE: + indicates city has at least one area receiving program designation; 䉱 indicates city with qualifying area but no des-ignated area.
all governments with qualifying areas applied for CRA or EZ designation. In addition, cities could have multiple qualifying areas, so there could be more designated areas than qualifying cities.
Table 1 summarizes the designation outcomes for the qualifying cities by city size. Only 9 of the 30 small qualifying cities received an area designation, one of which had two areas designated. In contrast, having multiple qualifying areas was prevalent in the large cities. Of the 24 large qualify-ing cities, slightly more than half had multiple areas designated.7Consequently, analyzing the application decision for small cities is more transparent because the occurrence of multiple quali-fying areas is less problematic.
Application Decision
Two types of bias typically come into play when local governments seek assistance through state-level programs: the decision to apply for program designation (Bondonio & Engberg, 2000; Mossberger, 2000) and selection bias on the part of program administrators at the state level (Boarnet & Bogart, 1996; Mossberger, 2000). As Figure 2 highlights, it is not possible to distin-guish those nondesignated areas in large cities (with populations over 50,000) whose applications were declined from those that did not submit an application.8Although the program designations were equally split between EZs and CRAs, it is not known to what extent this reflects strategic planning by state administrators or the preferences of the large qualifying cities. Consequently, for large cities, we cannot assess the extent of potential selection bias on the part of the program administrators.
In contrast to the large-city case, the application decisions of small qualifying cities are more straightforward. As shown in Figure 3, only 9 of the 30 areas in small cities received program des-ignation. However, according to the program audit, all applications from small qualifying cities
TABLE 1
Number of Qualifying Cities by Program Designation Status
City Designation Status Small Cities (n = 30) Big Cities (n = 29) With 1 or more designated areas
1 area designated 8 13
2 areas designated 1 8
3 areas designated 0 2
7 areas designated 0 1
Total 9 24
With no program designation
1 area qualified 21 4
2 areas qualified 0 1
Total 21 5
Designated (42)* Apply (?)
Qualifying areas (67) Not designated (?) (25) Don’t apply (?) Not designated (?)
Figure 2: Designation Status of Qualifying Areas in Large Cities, With Number of Areas in Parentheses
with populations between 7,500 and 20,000 were approved (Florida Office of the Auditor General, 1993, p. I-2). Thus, there did not appear to be any explicit rationing on the part of the program administrators in the case of these small cities. Another difference between small and large cities was the desirability of EZs versus CRAs. Seven of the nine approved applications from small cities were for EZs and only two for CRAs. One city, Chipley, applied for and received both designations for the same area. Given that the cities decided between the two programs, small cities apparently found the EZ program more desirable.
Based on administrative information, we can infer that selection bias on the part of program administrators is probably minimal for small cities because designation status was mostly deter-mined by a city’s decision to apply. Self-selection on the part of the small-city applicants, however, could still lead to endogeneity bias when estimating program efficacy. For example, if cities with the highest expected net benefits were more likely to place applications, then estimates of program impacts could be biased when comparing designated areas to nondesignated areas in qualifying cities that did not apply. Accordingly, we develop a simple model of the city application decision to investigate potential bias in this regard.
A Simple Model of the Small-City Application Decision
By modeling the city application decision, we are able to reveal important aspects of that choice by small qualifying cities. The following model captures our contention that cities attempt to deter-mine the payoff of program designation before applying. The expected payoff from applying for a targeted development program can be specified as follows:
Vit= Pit(Bit– Cit) – Ait, (1)
where Pitis the probability of city i receiving program t designation, Bitis the discounted present value of the expected benefit of implementing the program, Citis the discounted present value of the cost of implementing the program, and Aitis the cost of completing the application. A city ap-plies for a program if its expected payoff is positive.
Note that cities are allowed to apply for more than one designation for a given qualifying area. In this case, city i first applies for the program that has the highest expected payoff, assuming it is the only program implemented. If Vi, EZ> Vi, CRA, city i will choose to apply for EZ program designa-tion. Then the city evaluates the merit of applying for the second program. It is likely that the mar-ginal benefit of receiving either program designation would be smaller if it is selected second instead of first.9Also, the cost of completing the second application is likely to be less, given that both programs require a similar strategic planning process. Even if the probability of receiving both programs is low, a city may still benefit from applying to a second program if the cost of the second application is sufficiently small or if submitting both applications increases the probability of receiving either designation enough to compensate for the extra application costs.
Based on our knowledge of the implementation process, we can make several assumptions about the parameters of the small-city application decisions. First, it is reasonable to assume that the application costs for the two programs are likely to be about the same for a given city (Ai, EZ= Ai,
Designated (9) Apply (12)
Qualifying areas (31) Not designated (3) Don’t apply (19) Not designated (19)
CRA). Although the choice of policy instruments differs, both program applications require the city to develop an implementation plan for a program application.
In contrast, the implementation costs are likely to be higher for the CRA program than for the EZ program, (Ci, EZ< Ci, CRA). Under the EZ program, benefits to firms come mostly in the form of state tax expenditures (tax breaks) with no out-of-pocket costs to cities. The CRA program allows cities to raise additional revenues by issuing bonds but requires targeted investment in the desig-nated area. This explicit spending requirement increases the implementation costs associated with the CRA programs.
Although little is known about the perceived benefits or the expected probability of receiving designation, we can still make inferences about the choice between the two programs. In particular, unless the benefit of implementing a CRA is sufficiently high to offset the higher implementation costs, and as long as the probabilities of receiving designation are the same, we would expect small cities to select EZs over CRAs in the application procedure. For the most part, our expectations held. Table 2 compares features of small cities by application decision using 1980 census data. Except for Auburndale, which applied for a CRA, and Chipley, which applied for both for the same area, the small qualifying cities primarily applied for EZ program designation.
We also investigated potential differences in the application decision parameters across small cities. Application costs, for instance, may vary across cities depending on idiosyncratic features of government. Although cities submitting applications were larger on average than those that did not, the difference was not statistically significant. In addition, the expected benefits of the pro-grams may be greater for cities in relatively worse economic conditions. Table 2 shows that cities submitting applications had lower median incomes, lower median property values, and higher unemployment levels compared with the qualifying cities that did not submit applications. How-ever, only the difference in the mean unemployment rates was statistically significant.
The application process is overlooked in most studies of the impact of targeted program adop-tion (Blank, 1997; Mossberger, 2000; Wilder & Rubin, 1996). We found that the small cities were more likely to opt for EZs, suggesting, perhaps, that this type of instrument is chosen over CRAs because of the perceived lower cost of implementation or higher expected benefits. Furthermore, we found that the relatively worse-off small cities, at least in terms of unemployment measures, were more likely to target themselves for program designation. This sort of self-selection is desir-able from a policy standpoint if such areas are also likely to benefit more from targeted devel-opment programs (Bartik, 1991, p. 192). The downside, however, is that self-selection can be problematic when evaluating program efficacy.
We found that the small
cities were more likely to
opt for EZs, suggesting,
perhaps, that this type of
instrument is chosen over
CRAs because of the
perceived lower cost of
implementation or
higher expected benefits.
TABLE 2
Difference of Means for Small Qualifying Cities, 1980 (Standard Error) Cities by Application Status
Applied (n = 9) Did Not Apply (n = 21) Difference in Meansa Mean (X1) Mean (X0) X1– X0
Population 8,087 6,544 1,543
(4,932) (3,950) (49,366,757)
Median income $12,044 $12,594 –$550
(2,004) (2,873) (17,906,748)
Median property value $31,478 $38,564 –$7,086
(4,598) (18,332) (629,786,153)
Unemployed 853 639 214
(597) (426) (580,432)
Unemployment rate 0.10 0.09 0.01*
(0.0162) (0.0197) (0.0009)
a. Estimates were calculated using a standard difference in population means test. *Significant at 1% level.
EMPIRICAL SPECIFICATION AND QUASI-EXPERIMENTAL ANALYSIS
To investigate the efficacy of targeted economic development programs and to control for the three sources of endogeneity bias discussed earlier, we employed a quasi-experimental analysis using data for qualifying areas in small cities. Recent research has argued that the evaluation of tar-geted development programs requires analysis at the area level (Boarnet, 2001; Engberg & Greenbaum, 1999; Tao & Feiock, 1999). Although citywide effects are desirable, even a large impact at the area level may not be evident when using a city level of analysis. Our focus was on whether the targeted programs had discernible impacts on the targeted areas.
One issue when using targeted areas as the unit of analysis is that the boundaries of the 98 quali-fying areas do not strictly correspond with city, census place, or other political boundaries.10For the most part, however, the areas did correspond with census block-level divisions. Our analysis used data from the 1980 and 1990 census surveys (U.S. Department of Commerce, 1980, 1990a, 1990b) originally constructed by Tao (2000), who mapped qualifying area definitions with census tracts and blocks. When the areas in question did not comprise complete tracts, block-level units were used. Tao followed the standard approach for geographical information systems when deal-ing with census tract data; tracts themselves were assumed to be internally uniform unless there was information available to contradict this assumption. Thus, portions of census blocks were treated as proportional microcosms of the whole, and the portions that were included were calcu-lated as percentages of the whole based on the relative geographic region covered by a targeted area. For example, if a city had three census tracts designated as a targeted area but only 65% of the third tract was included, the level of analysis dropped to the block-group level, where inclusion was based on figures for block groups. For areas that were not designated as either redevelopment areas or EZs, the 1980 configurations were used again in 1990. For areas that were so designated, the 1990 configurations were used as the basis for comparison to 1980 figures.
For the majority of the qualifying areas in smaller cities, the boundaries corresponded to city borders. The appendix shows the 1980 city and area population for the small qualifying cities. The areas were smaller than their primary city in only seven cases (Auburndale, Cocoa, Dania-Liberia, Goulds, Opa Locka, Perrine, and Washington Park-North Miami). In Opa Locka and Sweetwater, the qualifying areas extended well outside the city’s political jurisdictions.
Quasi-Experimental Analysis
The idea of a quasi-experiment is to simulate a laboratory experiment ex post. The distinguish-ing feature of the procedure is the comparison of “treatment” groups with “control” groups.11The control group serves as the counterfactual for what would have happened to the treatment group in the absence of treatment. It provides a baseline forecast. Typically, the matching is pairwise so that each treatment observation is matched to a twin control observation. It is also possible to form observations by matching groups of treatment and control observations. For instance, each treat-ment can be matched to many controls (one to many), and multiple controls can be matched to mul-tiple treatments (many to many). The treatment impact is measured as the difference in outcome measures between the treatments (average of the treatment groups) and the controls (average of the control groups) in the posttreatment period. Any divergence in outcome measures in the post-treatment period is attributed to the post-treatment impact. In most studies, several matching schemes and outcome measures are evaluated to check for the sensitivity of the results to the matching scheme.12
In our study of small cities, there were 9 areas that received program designation (treatments) and 22 areas that were qualified but did not receive designation (potential controls). We evalu-ated the efficacy of the targeted development program by comparing how treatments performed on average compared with the controls (the many-to-many approach) using several grouping schemes. Estimated program impacts were measured as the divergence in outcome measures
between the treatments and the controls. The outcome measures analyzed included population, median property values, median household income, and the unemployed-to-population ratio.13
Several approaches can be used in a quasi-experimental framework to evaluate the significance of estimated impacts. For example, the impact can be computed using a difference of population means test assuming normality and independence variances across the groups. Table 3 shows the preprogram and postprogram means for the qualifying areas by designation status. Consistent with the city-level analysis, on average, the designated areas had slightly higher population, higher unemployed-to-population ratio, lower median property values, and lower median household incomes compared with the nondesignated areas in 1980. Only the mean differences in the pre-program and postpre-program unemployment-to-population ratios, however, were statistically significant.
In 1990, the gaps between the designated areas and the nondesignated areas had lessened in regard to all of the measures examined. Just looking at the before and after comparisons, one would be tempted to infer that the programs had positive impacts on the designated areas. The direction of the changes in all of the outcome measure means indicated improvements in the designated areas relative to the nondesignated qualifying areas.
To test whether these changes were statistically different from 0, we employed a log change specification for each of the outcome measures:
TABLE 3
Outcome Analysis of Qualifying Areas in Small Cities (Standard Error) Designated Areas Nondesignated Areas Difference in
(n = 9) (n = 22) Meansa
OLS Impact Area Measure Mean (X1) Mean (X0) X1– X0 Estimate Population 1980 6,820 6,206 614 (6,531) (4,038) (59,582,260) 1990 7,988 7,420 567 (6,541) (3,820) (56,531,596) Log change 0.2951 0.2276 0.0676 0.0676 (0.4840) (0.3893) (0.4406) (.1652) Unemployed/population ratio 1980 0.1072 0.0908 0.0164*** (0.0313) (0.0190) (0.0013) 1990 0.0431 0.0408 0.0023** (0.0186) (0.0203) (0.0010) Log change –0.9955 –0.8883 –0.1072 –0.1072 (0.4283) (0.4219) (0.4536) (.1676)
Median property value
1980 28,550 37,509 –8,959 (6,472) (17,258) (574,263,125) 1990 30,618 36,570 –5,952 (5,054) (18,568) (648,735,626) Log change 0.0864 –0.0235 0.1100 0.1100 (0.3343) (0.2017) (0.1524) (.0972)
Median household income
1980 11,464 12,556 –1,093 (3,003) (2,882) (21,486,827) 1990 12,245 13,064 –819 (949) (3,950) (29,186,760) Log change –0.0941 –0.0260 –0.0680 –0.0680 (0.2952) (0.1737) (0.1159) (.0848)
NOTE: OLS = ordinary least squares.
a. Estimates were calculated using a standard difference in population means test. **Significant at 5% level. ***Significant at 1% level.
Y= log(Y1990/ Y1980). (2) The log change specification offers the benefits of being a symmetric and uniform measure of relative change.14It controls for the baseline differences in the levels of the variables. The impact of the targeted program was estimated by taking the average of the log changes for the treatment group (YT) minus the average of the log changes for the control group (YC), where Y was the log change in the outcome variable as specified in Equation 2. None of the mean differences in the log changes was found to be statistically significant. Consequently, we could not conclude, based on these estimates, that the programs had impacts on any of our outcome measures that were statistically different from 0.
We also estimated the program impact using ordinary least squares (OLS) estimation on obser-vations from all the qualifying areas (i) in small cities as
Yi= constant + BTi+ e, e ~ N(0, σ2), (3)
where T was a dummy for designation status (the treatment) and B estimated the treatment impact. This is very similar to a matched-sample design, because our control observations were “selected” according to their similarity with the treatments with respect to the pertinent variables (Rubin, 1979).15
In contrast to the difference in means test above, the OLS estimates of Equation 3 assumed that the treatments and controls came from the same distribution. Although the estimated impact (the difference in the log change) would be the same, the standard errors might differ and thereby affect significance tests. As shown in the last column of Table 3, the standard errors using OLS regres-sion were all larger than those found using the difference in population means specification. Accordingly, all of the estimated coefficients for the outcome measures remained statistically insignificant.
Regression Adjustment Analysis
To test for robustness of our estimates, we included additional explanatory variables in the regression model. The cross-section regression adjusted model was
Yi= constant + BTi+ Xi+ e, e ~ N(0, σ2), (4)
where X was a vector of explanatory variables for area i. We considered both political and eco-nomic explanatory variables. The purpose of including the additional variables was to account for differences in outcome variables that might bias the estimates of the program impacts.
Following the literature on local policy making, the political variables focused on the features of government structure thought to influence decision making by local policy makers. Captured as dummy variables, these included whether the city had a strong mayor or a council manager (coded as 1) as opposed to a weak mayor or a rotating council style of government (coded as 0), and whether the city had district or ward elections (coded as 1) as opposed to at-large elections (coded as 0). The expectation was that cities with district or ward elections offered political incentives for local policy makers to support targeted development. The same expectation was held for cities with council-manager systems, which historically have been more supportive of redistributive policies targeting the economically disadvantaged.
The economic variables included economic area dummies, county-type dummies, and the 1980 values of the outcome measure being estimated (lagged explanatory variables).16The BEA eco-nomic area definitions are shown in Figure 2. Given the small number of cases, ecoeco-nomic areas 33 and 34 were combined, as were economic areas 29, 35, 36, and 81. The purpose of these dummies was to account for influences on outcome measures that were common to qualifying areas in a given economic region. The county-type dummies were based on the 1989 rural-urban (RU) con-tinuum classification scheme developed by the U.S. Department of Agriculture (1989).17The RU codes classify counties by size of urban population and adjacency to metropolitan areas, where adjacency is determined by physical proximity and a finding that at least 2% of the employed labor
force in the nonmetropolitan county commutes to metropolitan central counties. We considered four groupings for the analysis: central and fringe counties of large metro areas of 1 million or more residents (RU = 0 or 1), counties in metro areas of 250,000 to 1 million residents (RU = 2), counties in metro areas of fewer than 250,000 residents (RU = 3), and nonmetropolitan counties with an urban population of 2,500 to 20,000 that are adjacent to a metropolitan area (RU = 6). These variables were included to account for the influence of metropolitan proximity on outcome measures. For example, qualifying areas in more metropolitan counties may grow more quickly than other areas due to agglomeration economies.
To test for robustness, we estimated various empirical specifications of Equation 4. The empiri-cal specifications estimated ranged from one that included only the lagged dependent variable, constant, and program impact dummy to one that included all the political and economic variables discussed earlier. We summarize these results for brevity. All of the signs of the coefficients of all of the outcome measures were consistent with those given in Table 3. The magnitudes of the esti-mates were remarkably robust as well. The estiesti-mates for the measure of the unemployed-to-population ratio had the largest range (–.1033 to –.1778). The estimated coefficients for median property value had the smallest (–.0422 to –.0506). None of the estimated program impact coeffi-cients was found to be statistically significant at conventional levels. These results suggest that estimates presented in Table 3 are robust to the inclusion of common economic and political con-trol variables in the regression analysis.
Alternative Matching Schemes
We also investigated the sensitivity of our impact estimates to alternative matching schemes. Instead of pooling all the designated and nondesignated qualifying areas in a single regression, we estimated program impacts for groups of the qualifying areas with common influences separately. Although this approach may better capture influences that would bias impact estimates if not prop-erly accounted for in the empirical specification, it entails using fewer observations, which weak-ened our hypothesis tests. Another drawback is that using the group analysis increases the chances that some unobserved factor related to the outcome measure but not attributable to the program implementation biases the impact estimates.
To investigate the potential bias caused by common regional influences, we estimated program impacts for each economic area grouping separately, rather than pooling all the areas in a single regression, as was done before. Using the population measure, we found impact coefficient esti-mates that were positive and significant in two regional groupings and negative in another. As shown in Table 4, which reports only the significant impact coefficient estimates, very few of the program impact coefficients were statistically significant using the separate regional analysis. Notably, we found significant impact coefficient estimates using the population measure that were positive and significant in two regional groupings and negative in another. Similarly, the impact coefficient estimate using the median household income outcome measure was positive and signif-icant for one region and negative for another.
We also checked for robustness by performing separate analyses according to the RU category. In this case, the regression adjustment model included an intercept, program dummy, and the lagged dependent variable.18Again, we found that the signs of the estimated impact coefficients switched across county types. As indicated in Table 5, significant impact coefficient estimates were found only for the median household income outcome measure, with a positive coefficient for areas in central counties of large metropolitan areas and a negative coefficient for areas in non-metropolitan counties that are adjacent to a non-metropolitan area.
Summary of Empirical Analysis
Using the pooled analysis, we cannot conclude that the estimated program impacts on any of our four outcome measures were different from 0. Our best guess is that the population and median property values increased faster and the unemployment-to-population rate fell faster in designated areas. This is consistent with the program intent of redistributing resources within qualifying
Using the pooled
analysis, we cannot
conclude that the
estimated program
impacts on any of our
four outcome measures
were different from 0.
cities. The estimated negative impacts on the median household income measure may be an arti-fact, not of negative program impact, but perhaps of the ineffectiveness of the program at address-ing preexistaddress-ing deficiencies in this measure. The differential in program impact estimates across groups of areas suggests that not all small cities benefited to the same extent from implementing the targeted development programs.
Caution is warranted in concluding that the programs did not improve the outcome measures in the designated areas. A major obstacle to our analysis is the small number of observations. Although we investigated the entire population of areas in small cities, this yielded at most 31 areas. Using the group matching techniques reduced the statistical power of our hypothesis test even further. Assumptions of normality may also be inappropriate and result in misleading
TABLE 4
Selected Coefficient Estimates From Separate Regional Estimation (Standard Error)
Estimated Impact
Outcome Measure and BEA Economic Areaa OLS (Equation 3)b Adjusted OLS Regressionb Log change in population
Economic area 30 (n = 9) .5703 .1145 (.2634)* (.1592) Economic areas 33 + 34 (n = 6) –.5045 –.4232 (.3611) (.0211)** Economic areas 29 + 35 + 36 + 81(n = 16) .3515 .2416 (.1758)* (.16148)
Log change in median household income
Economic area 31 (n = 9) .0584 .3611
(.1592) (.0947)*
Economic areas 29 + 35 + 36 + 81 (n = 16) –.1473 –.1671
(.0918) (.0533)***
NOTE: BEA = Bureau of Economic Analysis; OLS = ordinary least squares. a. See text and Figure 1 for a description of the BEA economic areas.
b. The OLS estimation of Equation 3 included the intercept and the program dummy variable. Additional explanatory vari-ables included in the regression adjusted OLS (Equation 4) were the lagged dependent variable, city manager dummy, strong mayor dummy, and a dummy for areas in large metropolitan counties (rural-urban = 0 or 1).
*Significant at 10% level. **Significant at 5% level. ***Significant at 1% level.
TABLE 5
Selected Coefficient Estimates From Separate Estimation by County Type (Standard Error)
Estimated Impact on Change in Median Household Income
RU Classificationa OLS (Equation 3)b Adjusted OLS Regressionb RU = 0: Central counties of large metro areas of 1 million
or more residents (n = 13) .4381 .4107
(.2249)* (.2399)
RU = 6: Nonmetropolitan counties with an urban population
of 2,500-20,000 that are adjacent to a metropolitan area (n = 4) –.3784 –.3940
(.0289)*** (.0469)*
NOTE: RU = rural-urban; OLS = ordinary least squares. a. See text for a description of the code RU classification.
b. The OLS estimation of Equation3 includedthe intercept and the programdummyvariable. The regression adjusted speci-fication (Equation 4) also included the lagged dependent variable.
estimates. A larger population of observations would allow for the application of nonparametric tests that do not rely on assumptions about underlying distributions.
Another valid concern is how well the controls represent the counterfactual for the designated areas. We suspect that much of the potential endogeneity in the selection procedure was accounted for by using only distressed areas as designated by the FDCA. Self-selection among the small cities that submitted applications (and therefore implemented programs) still may be evident. Although the designated areas and cities were in relatively worse condition on average in the pre-program period, these differences were statistically significant only for the unemployment mea-sure, and the differences in this measure were relatively small, about 1%. Furthermore, factors external to the area (and surrounding city) are likely to be the driving forces influencing the unem-ployment and other economic outcome measures. The response to these external factors is likely to be similar for all distressed areas in small Florida cities. Consequently, the quasi-experimental approach is appropriate for analyzing impacts of targeted development programs.
CONCLUSION
Empirical evaluations of targeted economic development programs are fraught with method-ological problems. The endogeneity of programs with outcome measures is an especially endemic problem that can lead to biased estimates of program impacts. We highlight three sources of the endogeneity bias and employ a quasi-experimental design method for evaluating program efficacy in the presence of endogeneity bias.
Investigating the Florida experience with EZs and CRAs allows us to give careful consideration to endogeneity bias in estimating program efficacy. Our analysis differs from other studies using a similar design by including data from all areas that qualified for the programs, whether they did or did not implement targeted programs. Such an approach overcomes the bias associated with pro-grams that target distressed areas. Employing a quasi-experimental analysis using data for qualify-ing areas in small cities, our focus was whether the targeted programs had discernible impacts on the targeted areas.
We build on previous research by developing a simple model of a city’s application decision. By focusing on small cities in Florida, we are able to explicitly account for two potential sources of bias associated with the program designation: (a) selection bias on the part of city governments in the application process and (b) selection bias on the part of program administrators when programs are rationed. There was no rationing of program designation among the small cities. Although we find some differences in the characteristics of the small cities that applied and those that did not, these are statistically significant only for the unemployment measure. Accordingly, our compari-son of all areas determined to be distressed reduces, but does not completely eliminate, the concern about the self-selection bias on the part of cities.
We find that small cities overwhelmingly preferred the EZ programs to the CRA programs. This suggests that the small cities did not view tax-increment financing, the main advantage of the CRA programs, as a viable policy instrument. This may be attributed to a number of factors, including the higher implementation costs associated with CRAs compared with EZs, the lower perceived benefits of CRAs compared with EZ programs, or the infeasibility of redirecting city resources to targeted areas, especially for distressed cities facing budget shortfalls.
The results confirm the findings of previous studies as to the questionable efficacy of adopting a targeted development instrument. Employing the entire set of qualifying areas in small cities and a variety of empirical specifications, we find little evidence that the areas receiving the targeted pro-grams were helped by the propro-grams. Using separate group matching schemes, however, we find some scattered and inconsistent evidence of positive and negative program impacts. Caution should be used when drawing more generalized conclusions about the merit of such programs, however, given the small number of observations in our analysis.
The empirical design employed in our analysis provides a notable advance in the ability to infer local government behavior in a historical context when available data are limited. Even with a lim-ited number of observations, it may be possible to account for potential sources of endogeneity
The results confirm the
findings of previous
studies as to the
questionable efficacy of
adopting a targeted
development
instrument. . . . we find
little evidence that the
areas receiving the
targeted programs were
helped by the programs.
bias inherent in the implementation process. Evaluations of other local government programs fac-ing endogeneity bias may benefit from a similar empirical method.
APPENDIX
Qualifying Areas in Small Cities
Designation = 1 1980 Population
Area Name (N = 31) County EZ CRA Area City
Auburndale Polk 0 1 2,514 6,501
Bagdad/East Milton Santa Rosa 0 0 1,479 1,479
Chipley Washington 1 1 3,330 3,330 Cocoa Brevard 1 0 1,243 16,096 Dania-Liberia Broward 0 0 3,052 11,811 Eatonville Orange 0 0 2,185 2,185 Frostproof Polk 0 0 2,995 2,995 Goulds Dade 0 0 4,936 7,078 Gretna Gadsden 0 0 1,448 1,448 Gulfport Pinellas 0 0 11,180 11,180
Haines City Polk 1 0 10,799 10,799
Hialeah Gardens Dade 0 0 2,700 2,700
Lake Wales Polk 0 0 8,466 8,466
Lake City Columbia 0 0 9,257 9,257
Milton Santa Rosa 1 0 7,206 7,206
Opa Locka 1 Dade 0 0 9,643 14,460
Opa Locka 2 Dade 1 0 21,889 14,460
Pace Santa Rosa 0 0 5,006 5,006
Palatka Putnam 0 0 10,175 10,175
Palm Harbor Pinellas 0 0 5,215 5,215
Palmetto Manatee 1 0 8,637 8,637
Perrine Area Dade 0 0 9,747 16,129
Punta Gorda Charlotte 0 0 6,797 6,797
Safety Harbor Pinellas 0 0 6,461 6,461
Sebring Highlands 0 0 8,736 8,736
South Bay Palm Beach 1 0 3,886 3,886
Sweetwater Area Dade 0 0 17,432 8,251
Umatilla Lake 1 0 1,872 1,872
Wahneta Polk 0 0 3,329 3,329
Washington Park/No. Miami Beach Dade 0 0 4,811 7,240
Wimauma Hillsborough 0 0 1,477 1,477
NOTE: EZ = enterprise zone; CRA = community redevelopment area. NOTES
1 Other methods have been suggested to deal with endogeneity bias. Bondonio and Engberg (2000), for instance, ana-lyzed enterprise zone (EZ) programs using fixed effects and propensity score methods.
2. Lambert and Coomes (2001) investigated the impact of an EZ program in Louisville, Kentucky, by comparing it to the rest of the county. Bondonio and Engberg (2000) used zip code-level data from five states.
3. Bostic (1996) pointed out that the strategic planning associated with the application process could produce benefi-cial results regardless of the program implementation.
4. The Florida Department of Community Affairs (FDCA) was responsible for both programs during the period of this study.
5. The appendix identifies the qualifying areas in small cities.
6. For the period covered in this study, the EZ program was administered by the FDCA. However, when the program was evaluated in 1994, just prior to sunset legislation, the Florida legislature and the governor’s office moved the program to the Department of Commerce. When the Department of Commerce was privatized in 1995, the program was moved to the
governor’s office. After the 1998 gubernatorial race and the installation of the new Bush administration, the program was returned to the FDCA.
7. Miami had seven qualifying areas designated for a targeted development program; five of the original areas were combined to form one countywide EZ, the North Dade County EZ.
8. We do not know whether large qualifying cities with multiple qualifying and designated areas submitted applica-tions for the nondesignated areas.
9. Alternatively, there couldbe synergistic benefits fromimplementingbothprogramsin an area. This possibility, how-ever, does not appear to be salient, given that only one small city chose to apply for and implement both programs for the same area.
10. See Tao (2000) for a description of the methodology used by the FDCA to construct zone definitions. 11. For a basic introduction to quasi-experimental control group analysis, see Isserman and Merrifield (1987). 12. For details of the methodology, see Reed and Rogers (2003).
13. Unemployment rates were not available at the area level. We used the unemployed-to-population ratio as a proxy. It would be informative to investigate welfare measures. However, at this time, we do not have complete enough area-level data to perform a meaningful analysis.
14. Tornqvist, Vartia, and Vartia (1985)showed that the log change measure is the only symmetric, additive, and normed indicator of relative change.
15. Unlike typical studies using a matched-sample design, we did not have to create comparison groups ex post because the program administrators identified all qualifying areas before program implementation.
16. The city government classifications are listed in the Florida County Atlas and Municipal Fact Book (Institute of Sci-ence and Public Affairs, 1994).
17. The rural-urban (RU) codes classify counties by degree of urbanization and nearness (adjacency) to metropolitan areas, where metropolitan status is determined by the Office of Management and Budget. As defined by the U.S. Depart-ment of Agriculture, adjacency is determined by physical boundaryadjacency and a finding that at least 2% of the employed labor force in the nonmetropolitancounty commuted to metropolitan central counties. The 1989 update of the 1983 classifi-cations affected six counties with qualifying areas in Florida. We used the 1989 classificlassifi-cations for this article. Information about the classification can be found at http://www.ers.usda.gov/data/RuralUrbanContinuumCodes/.
18. Attempts were made to include a richer set of control variables. However, there was less variability in these controls within a county type.
REFERENCES
Bartik, T. (1991). Who benefits from state and local economic development policies? Kalamazoo, MI: Upjohn Institute. Blank, R. M. (1997). It takes a nation: A new agenda for fighting poverty. Princeton, NJ: Princeton University Press. Boarnet, M. (2001). Enterprise zones and job creation: Linking evaluation and practice. Economic Development Quarterly,
15(3), 242-254.
Boarnet, M., & Bogart, W. (1996). Enterprise zones and employment: Evidence from New Jersey. Journal of Urban Eco-nomics, 40(2), 198-215.
Bondonio, D., & Engberg, J. (2000). Enterprise zones and local employment: Evidence from the states’programs. Regional Science and Urban Economics, 30, 519-549.
Bostic, R. W. (1996). Enterprise zones and attraction of business investment: The importance of implementation strategies and program incentives. Unpublished paper, Federal Reserve Board of Governors, Division of Research and Statistics, Washington, DC.
Community Redevelopment Act of 1969. (1980). Fla. Stat. § 163-340.
Engberg, J., & Greenbaum, R. (1999). State enterprise zones and urban housing markets. Journal of Housing Research, 10(2), 163-187.
Erickson, R. A., Friedman, S. W., & McCluskey, R. E. (1989).Enterprise zones: An evaluationof state government policies. Washington, DC: U.S. Department of Commerce, Economic Development Administration, Technical Assistance and Research Division.
Florida Department of Community Affairs. (1983). A distress atlas. Tallahassee, FL: Author.
Florida Office of the Auditor General. (1993). Review and evaluation of the enterprise zone program (Report No. 12003). Tallahassee, FL: Florida Office of the Auditor General, Program Audit Division.
Florida Senate Committee on Economic,Community, and ConsumerAffairs. (1981,February).A review of communityrevi-talization efforts. Tallahassee, FL: Author.
Institute of Science and Public Affairs. (1994). Florida county atlas and municipal fact book. Tallahassee: Florida State University.
Isserman, A., & Merrifield, J. (1987). Quasi-experimental control group methods for regional analysis: An application to an energy boomtown and growth pole theory. Economic Geography, 63, 3-19.
James, F. J. (1991). The evaluation of enterprise zone programs. In R. E. Green (Ed.), Enterprise zones: New directions for economic development (pp. 225-257). Newbury Park, CA: Sage.
Jones, B. (1990). Public policies and economic growth in the American states. Journal of Politics, 52(l), 219-233. Lambert, T. E., & Coomes, P. A. (2001).An evaluation of the effectiveness of Louisville’s enterprise zone. Economic
McDonald, J. F. (1997). Comment on Kala Seetharam Sridhar’s (1996) “Tax costs and employment benefits of enterprise zones.” Economic Development Quarterly, 11(3), 222-224.
Mossberger, K. (2000). The politics of ideas and the spread of enterprise zones. Washington, DC: Georgetown University Press.
Papke, J., & Papke, L. (1990). State-local tax concessions as urban redevelopment tools: Incentives, subsidies, or windfalls? Proceedings of the 83rd Conference on Taxation. National Tax Journal, 43(l), 215-225.
Pressman, J., & Wildavsky, A. (1984). Implementation (2nd ed). Berkeley: University of California Press.
Reed, W. R., & Rogers, C. L. (2003). A study of quasi-experimental control group methods for estimating policy impacts. Regional Science and Urban Economics, 33, 3-25.
Rubin, D. B. (1979). Using multivariate matched sampling and regression adjustment to control bias in observational stud-ies. Journal of the American Statistical Association, 74, 318-329.
Tao, J. L. (2000). Local economic development policy instruments and issues of redistribution: Market versus government approaches to a1leviating poverty in the state of Florida. Unpublished doctoral dissertation, The Florida State Univer-sity, Tallahassee.
Tao, J. L., & Feiock, R. C. (1999). Directing benefits to need: Evaluating the distributive consequences of urban economic development. Economic Development Quarterly, 13(l), 55-65.
Tornqvist, L., Vartia, P., & Vartia, Y. 0. (1985). How should relative changes be measured? The American Statistician, 39(l), 43-46.
U.S. Department of Agriculture, Economic Research Service. (1989). Rural-urban continuum codes. Washington, DC: Author. Retrieved from http://www.ers.usda.gov/data/RuralUrbanContinuumCodes/
U.S. Department of Commerce, Bureau of the Census. (1980). Characteristics of the population: General social and eco-nomic characteristics—Florida (PC80-1-C I 1). Washington, DC: Author.
U.S. Department of Commerce, Bureau of the Census. (1990a). General population characteristics: Florida (CP-1-II). Washington, DC: Author.
U.S. Department of Commerce, Bureau of the Census. (1990b). Social and economic characteristics: Florida (CP-2-11). Washington, DC: Author.
U.S. General Accounting Office. (1988, December). Enterprise zones: Lessons from the Maryland experience (GAO/ PEMD Publication No. 89-2). Washington, DC: Government Printing Office.
Wilder, M. G., & Rubin, B. M. (1996). Rhetoric versus reality: A review of studies on state enterprise zone programs. Jour-nal of the American Planning Association, 62, 473-491.
