H
aving discussed the conceptual and methodological frameworks that are applied in the literature on the evaluation of planning policies, this section outlines the research approaches that are used in this thesis to assess the impact of planning policies on the performance of the industrial property market in the Netherlands. Different methodological approaches will be employed to answer the research questions identified in the third section. This is summarised in Table 1.5.1 INTRODUCTION
The purpose of the first research question is to undertake an analysis of the process by which land use plans are implemented. Following Monk and Whitehead (1999), a behavioural approach is adopted to explore how municipalities make decisions to increase the supply of land for industrial development and also as a means of examining how they determine the price of the land they offer for sale. This analysis draws from insights generated through qualitative research that involved in-depth interviews with municipalities. The second research question seeks to examine the factors influencing the designation of regeneration areas so as to establish whether initiatives are being delivered in the right areas. An econometric model is developed for analyzing and estimating which factors influence the likelihood that a site will be subject to regeneration initiatives. Because the dependent variable is dichotomous in nature (i.e. 1 = intervention, 0 = no intervention) a logit analysis is used to estimate this model.
Table 1.5
Research questions and evaluation approach
RESEARCH QUESTIONS EVALUATION APPROACH METHODS
1. How do municipalities determine the Process evaluation Interviews
overall quantity and price of land to be made available for industrial development?
2. Are physical regeneration initiatives Process evaluation Statistical analysis
being delivered in appropriate areas?
3. What is the impact of the planned Impact assessment Statistical analysis
provision for new industrial development on both new construction and
refurbishment activity?
4. Are physical regeneration initiatives Impact assessment / Statistical analysis /
having the desired effects? Programme theory Document analysis
The last two research questions are concerned with assessing the impact of land use planning and regeneration policies respectively. The basic aim of an impact assessment is to produce an estimate of the net effects of an intervention – that is, the changes on outcome measures that can be attributed solely to the intervention, free and clear of the influence of any other confounding factors that may also influence outcomes. As a result, issues of internal validity have been one of the main concerns in the evaluation literature. Classic experimental designs are generally thought to be most powerful in dealing with factors that jeopardize internal validity (see Campbell & Stanley, 1963). As this thesis investigates the effectiveness of ongoing policies and initiatives, which have been in operation for a substantial period of time, it is not possible to use a classic experimental design in which sites are randomly assigned to a ‘treatment’ group, that is subject to regeneration, and a control group, from which the intervention is withheld. The prospects of using such a design to answer the third research question are further constrained because the planning system applies everywhere in the Netherlands. As a consequence, there will be no areas available that are not subject to these policies, which could serve as controls. Instead, statistical methods will be employed for isolating the effects attributable to the policies in question. Statistical
approaches are often considered next to best to randomized designs in dealing with threats to internal validity (Campbell & Stanley, 1963; Rossi et al., 1999).
The third research question will be addressed by exploiting ‘natural variation’ in planning regulations, which arises because the Dutch planning system is not uniform over space or time. The extent of restriction applied by land use plans and regulations on new industrial development varies locally, as Olden (2010) has shown. Thus, while it may not be possible to assess what the net effects of planning regulations are (as compared to no regulations at all), it is possible to assess the effects of differences in restrictiveness. For this purpose, an econometric model of investment activity in industrial buildings is developed and estimated that incorporates several measures of planning restrictiveness. In contrast to existing studies that explore the supply-side impact of land use regulations at the city or regional level, this research conducts an analysis at the level of individual sites. Given the nature of the dependent variable a mixture of discrete and continuous variable modelling is used (as suggested in Bramley & Leishman, 2005).
1 INTRODUCTION
The fourth research question could be addressed by interrogating local authorities about their impressions of the impact of regeneration policies. Although previous evaluations have principally focused on output delivery, some studies have conducted surveys with policy staff to supplement output measures with outcome information (see Table 1.6). For example, drawing on a survey of policy officials, the evaluation study by Berenschot (2004) reported that the TIPP programme would stimulate the creation – and retention – of thousands of jobs. However, it will be extremely difficult for the staff responsible for these initiatives to make judgments about net effects because the necessary information for making such judgments is ordinarily lacking. This was also admitted by the team of evaluators that undertook the TIPP evaluation. Moreover, policy officials clearly have some incentive in making their efforts seem successful. Surveys of firms located on the sites that are subject to regeneration could also be used to get an idea of the success of these initiatives. But yet again, it will be difficult for these actors to make judgments about net effects. Indeed, such surveys are better suited for revealing how the activities undertaken on the sites concerned have affected firm behaviour (Bartik, 2004). Rather, as noted above, this thesis relies on statistical methods to obtain estimates of the net effects of regeneration initiatives.
Notes: The figures represent the percentage of respondents indicating that regeneration initiatives had a
positive impact on these outcome measures. Regeneration activities were undertaken in less than half of all initiatives financed by the StiREA programme. The figures reported for this programme are thus likely to overestimate the effects of regeneration.
Sources: Berenschot (2004), Nicis Institute (2009). Table 1.6
Judgmental assessments by municipal policy officials of the impact of regeneration policies
OUTCOMES STIREA TIPP TOPPER (N=18) (N=41) (N=23)
Job creation 78% 17% 43%
Job retention 50% 37% n/a
Attraction of businesses 33% 7% 39%
Retention of businesses 17% 12% 30%
Environmental quality improvements 50% 68% 57%
Accessibility improvements 44% 61% 57%
The model that is employed to estimate the impact of planning regulations on investment activity also includes regeneration initiatives as one of the key independent variables, which allows for an assessment of the contribution of these policies over and above that associated with the other (control) variables. This thesis also utilizes an alternative, two-stage procedure to assess the impact of regeneration initiatives in which the first step is to explicitly model the selection process as a means of offsetting selection biases. To this end, a model is developed that predicts the probability of regeneration activities being undertaken on a particular site. This model, generally known as a selection model, is similar to the one applied to address the second research question.12 The results of this analysis are used to match regeneration sites
to control sites on the basis of those characteristics that caused sites to be targeted by regeneration. This method is known as propensity score matching (Rosenbaum & Rubin, 1983, 1984). To gain a better understanding of the basic rationale on which regeneration policies are based, the main assumptions and expectations about how they are supposed to work are elicited from policy documents. The evaluation methodology just outlined will be used to test the validity of this underlying programme theory.