How to measure variability: the dependent variable

At its core, this study is about sources of variability in environmental policy arising from the policy- making process. It is not a direct evaluation of environmental policies per se. But, in looking at causes of policy variability we need to assess variability against a measure of policy effectiveness. Policy theory, particularly policy evaluation, offers ideas about policy effectiveness. Environmental theory offers ideas about environmental effectiveness. This study uses intersecting components, drawn from the ideas of policy design and evaluation, and from environmental ideas about what is likely to increase the environmental success of policy.

Policy literature suggests that policy effectiveness depends on the characteristics of the policy, assumptions about the policy goals, rules and tools, resources for implementation, and degree of support.337

Environmental literature tends to come at the subject of effectiveness from a cognitive perspective, stemming from normative views that human welfare depends on environmental quality, or empirical views that environmental quality improves efficiency and coherence in policy-making. There is a large measure of agreement that environmental effectiveness in government policy arises from the priority given to environmental values and whether environmental values feature in belief systems,

4: Analytical framework, methods, and case study rationale

assumptions, knowledge and information. How people involved with decision-making perceive that priority determines whether, and to what degree, those values are integrated into and across policies and institutions.338

An analysis of environmental priority combined with characteristics of policy-effectiveness should predict the likely effectiveness of policy in environmental terms.339 _{However, carrying out analysis on} this basis would require an extensive detailed evaluation of policies, something that a number of commentators say is a demanding and difficult exercise.340 _{As noted above, this is not the primary} focus of this study. Its focus is on intervening causal factors within a government system that influence environmental agencies and environmental policy efforts. Schneider and Ingram suggest it is sufficient to identify those features that seem most critical to understanding the effect of causal factors.341 _{Because this is an environmental study as well as a policy-making one, dependent} variables should include a predictor of likely environmental effectiveness. Because it involves policy forming, it should include a predictor of potential policy effectiveness arising from the policy formation.

1. To assess environmental effectiveness, the case studies will qualitatively assess what priority and weight was given to environmental values in the development of policy, and the extent to which those values were either:

a. Integrated internally in the environmental policy domain. This is based on the idea that, because of likely multiple causes, environmental policies should endeavour to

complement each other and create synergies. 342 _{Assessment would involve looking at} the extent to which an environmental policy is aligned with other environmental policy efforts. There may be circumstances where an environmental policy can be effective independent of other policy efforts, but only where causes can be effectively addressed

338 _{Beginning with Caldwell, "Environment: A New Focus for Public Policy." through, Bartlett, "Rationality and}

the Logic of the National Environmental Policy Act." Dryzek, Rational Ecology: Environment and Political Economy. Jordan, "Integrating the Environment for Sustainable Development.", Bührs, Environmental Integration: Our Common Challenge, Jordan, "Environmental Policy Integration: A State of the Art Review.". Nilsson says priority of environmental values has been advocated from normative perspectives that human welfare depends on environmental goals and from a perspective that environmental values contribute to greater efficiency and coherence of policy-making. M. Nilsson, "Learning, Frames, and Environmental Policy Integration: The Case of Swedish Energy Policy," Environment and Planning C: Government and Policy 23 (2005). Pp.208-209.

339 _{The idea of potential policy effectiveness is distinct from policy outcomes because the latter are also}

contingent on policy implementation, which has its own challenges.

340 _{Schneider, "Policy Design: Elements, Premises, and Strategies." Pp.88-89. Jordan, "Environmental Policy}

Integration: A State of the Art Review."P.154. M.E. Kraft, Vig, N.J., "Environmental Policy over Four Decades," in

Environmental Policy, ed. N.J. Vig, Kraft, M.E. (Los Angeles: Sage, 2013). Pp. 19-20.

341 _{Schneider, "Policy Design: Elements, Premises, and Strategies." P.78 & 84.} 342 _{Bührs, Environmental Integration: Our Common Challenge. P.121.}

4: Analytical framework, methods, and case study rationale

by an isolated course of action without a need to invoke changes in other areas.343 Assessment in this case would look at the clarity of objectives in the policy, and whether the causes can indeed be isolated. Or,

b. Integrated externally across other government policies. This is based on the idea that environmental policy should be a component of traditionally non-environmental policy dimensions in order to achieve comprehensive effectiveness.

2. Assessing policy effectiveness needs indicators of what seems most critical to understanding causal factors (Schneider and Ingram’s criterion). Because this study isn’t a post hoc

assessment of outcomes, it takes us back to basing assessment on structural logic and political architecture, as discussed in Chapter 2.

In terms of structural logic, Grant provides eight possible criteria for evidence of effective environmental policy: clear authority; rules and laws; sufficient resources and incentive structures; behaviour changes; intellectually robust remedies; remedies that advance objectives without disadvantaging excessively those relevant to the policy; alternatives being worse; or else serving symbolic goals.344 _{Of these, behaviour changes and the effect of} remedies require post hoc assessment, but the others seem applicable as indicators of potential policy effectiveness. Schneider and Ingram argue that the most important elements of policy effectiveness are the policy tools and the behavioural assumptions on which they rely, and what they describe as the implementation patterns (meaning whether the policy should have strong directive institutions and clear accountabilities, or rely on maximum leeway to generate locally applicable solutions and/or support and further knowledge).345 The structural logic of the policy (especially rules and tools and implementation patterns) seems on the face of it to provide an indicator of potential policy effectiveness. But care is required because there is a vigorous debate about what constitutes effective policy rules and tools, and effective means of implementation.346 _{This risks diverting attention away from} reasons for policy variability into evaluating the policies themselves, which is not the focus of this research\.

A policy can be logically coherent, however, but still be ineffective because of political architecture in terms of strong or weak opposition or support. In the face of prevailing strong

343 _{Weale, "Governance, Government and the Pursuit of Sustainability." P.71-72.} 344 _{Grant, The Effectiveness of European Environmental Policy. P.4.}

345 _{Schneider, "Policy Design: Elements, Premises, and Strategies." Pp. 96-97 & 99.}

346 _{See S.H. Linders, Peters, B.G., "The Design of Instruments for Public Policy," in Policy Theory and Policy}

Evaluation: Concepts, Knowledge, Causes, and Norms, ed. S. Nagel (New York: Greenwodd press, 1990), J.L. Pressman, A.B. Wildavsky, Implementation: How Great Expectations in Washington Are Dashed in Oakland; or Why Bits Amazing That Federal Programs Work at All (Berkley: Iniversity of California Press, 1973).

4: Analytical framework, methods, and case study rationale

opposition or without support (or the prospect of support), a policy is unlikely to be enduring and thus less effective. A caveat is that policy almost inevitably affects peoples’ preferences and values.347 _{Hence policy, even if it is only symbolic, frequently gives rise to opposition.} From an assessment perspective, it is sufficient, however, to make at least a qualitative assessment of the support basis of a policy when it was adopted.

Thus, indicators of causal variables of policy effectiveness, for this study, will focus on structural logic and political architecture of the policy, in terms of assumptions, rules,and tools, and support.

Formulated as questions, analysis of dependent variables for this research involves asking: Question 1.

What priority and weight was given to environmental values in the outcome of the development of policy, and how integrated were those values, either internally or externally?

Question 2.

Did the policy process result in tools and rules to back up the policy? What was the nature of those tools and rules?

Question 3.

What basis of support was there for the policy?