Published data analysis

The analysis of published data had two parts. The first part used a variety of published data to assess regional council performance and attributes. The second part sought to determine the level of correspondence between environmental conditions and problem, and jurisdictional area of the regional councils. This required a different type of analysis and is explained separately below.

Various reports on different aspects of regional council performance are available. At the national level, for example, the Ministry for the Environment reports two-yearly on resource consents administration under the RMA, while the Department of Internal Affairs reports on voter turnout and candidate composition after every local government elections. Statistics New Zealand collects and reports census data at national, regional and local levels. As well, various specialised reports and databases have been prepared for various reasons, such as informing policy development, supporting research and for administrative purposes. Examples include national water quality and quantity reports, land and waterway environmental typing, and grants allocations. Each data set has its strengths and weaknesses, but collectively, these data were aggregated and analysed to provide a data set to populate the public value framework of regional and national level environmental management performance.

4.1 Correspondence of issues with jurisdiction

A critical component of this research is the validation of the correspondence principle (discussed in chapter 2). This principle requires matching between the geographic span of issues with the jurisdiction of the institution responsible for managing it. If the same issues are found across a number of regions, the principle suggests that the level of devolution is too low, and that it should be addressed primarily at a higher level that singularly encompasses it. A wealth of environmental planning documents exists within New Zealand, and internationally, that identify environmental issues needing to be addressed. The documents differ in scope and implementation methods, but nevertheless provide a means for measuring consistency and convergence within levels of government.

4.1.1 National level issues

Ideally, national level issues and performance would draw on data collected nationally as part of a comprehensive environmental performance monitoring strategy. However,

identifying changes in environmental quality since regional councils were established in 1989 is difficult. Sumits and Morrison, searching for models to reform California‘s environmental management, summarised the difficulty in evaluating the effectiveness of New Zealand‘s environmental management institutions:

baseline environmental quality information was not collected and adequate monitoring programs were not established at the outset, thereby limiting empirical assessments of whether the new system has in fact resulted in improvements (Sumits and Morrison, 2001: ii).

Substantive documentation for the overall quality of New Zealand‘s environment is patchy at best, while at the regional level, different environmental monitoring priorities and methods mean that direct and comprehensive comparisons are difficult. The OECD criticised New Zealand‘s poor environmental monitoring record in 1996 and again in 2007 in its country environmental performance reviews (OECD, 1997, 2007). The Ministry for the Environment prepared a state of the environment report in 1997 (MfE, 1997) that is notable for being a compendium of information, rather than a basis for establishing changes in environmental quality. The 2002 country statement on the environment prepared for the Johannesburg World Summit (MfE and MFAT, 2002) is notable for its generality. The Ministry for the Environment released its 2007 state of the environment report in February 2008 (MfE, 2007). This, too, is largely a compendium of general data, reflecting the earlier failure by the MfE to complete development and implement its national environmental indicators programme. Essentially the data for substantiating overall institutional performance largely do not exist. Instead, most assessments of New Zealand‘s environmental state rely on specific studies that cover a few regions and aggregating individual regional councils‘ monitoring data. The National Institute of Water and Atmospheric Science (NIWA) has been able to identify trends between 1989 and 2005 using its own national monitoring network, (e.g. Scarsbrook, 2006) and these specialist reports are drawn on in the absence of other data.

4.1.2 Regional level issues

Regional councils‘ Regional Policy Statements (RPS) were initially considered as a source of differentiating regional issues as councils are required to identify in them significant resource management issues in their regions. This source was rejected on a variety of grounds. The first generation RPS documents were prepared some thirteen or more years ago, although they may only recently have been adopted by some councils after undergoing legal appeals; hence they are dated. The documents are required to be reviewed at least 10 years after adoption and some councils have already embarked on their review and are at different stages of this exercise. As a result, there is no temporal consistency in plans to allow comparison between documents. In any case, a common criticism levelled at them even at the time they were written was that they were too general in nature. Lindsay Gow, Deputy Secretary for the Ministry for the Environment, expressed the criticism that:

to date, and from MfE‘s perspective, the results evident in notified regional policy statements have been generally disappointing. Many but not all, policy statements do not seem to us to yet be able to fulfil the functions intended for them. Problems that concern us are:

1. there is too much very general material in many statements…

2. a number of natural resource users… consider that the generality of many statements provides them with uncertainty… (Gow, 1994: 2).

Instead, more contemporary planning documents, regional councils‘ 2006-16 LTCCPs were used as a source of information about communities‘ aspirations in relationship to

environmental well-being. These data take the form of text and are qualitative, presenting a different challenge to analyse. Source material is likely to be rich in content but unwieldy and intertwined in context, which can be daunting to researchers and challenging to analyse (Ritchie & Lewis, 2003). There are no clearly agreed rules or procedures for analysing qualitative data, varying by epistemological assumptions and traditions of research, so that a range of analytical processes have developed, including narrative analysis, discourse analysis and grounded theory (Spencer et al., 2003: 200). Some of these processes such as discourse analysis of regional council documents could be very revealing about the regional council policy-making process.

To analyse the documents, a simple framework analysis was undertaken (Ritchie & Spencer, 1994). This is a matrix based method for classifying, ordering and synthesising qualitative data using an analytic hierarchy and is now widely used by qualitative researchers (Ritchie et al, 2003). This requires organising data according to key themes, concepts and emergent categories. Data sets (or respondents) are then ascribed to the identified themes‘ within a matrix structure. The matrix forms thematic charts that allow development of descriptive categories. It does, however, rely on the researcher‘s ability to identify groups and patterns within the text.

4.2 Regional environmental classifications

Secondary data were also used for determining regional similarities. Geographic information system (GIS) interrogation of existing datasets was used to measure similarity within and between regional council jurisdictions using river and terrestrial environmental parameters. The two primary datasets were the Land Environment New Zealand database (LENZ) (MfE, 2002) and the River Ecosystems Classification (REC) (MfE, 2002), which were developed to assist environmental managers. (See Appendix 2 for a detailed description of these databases and interrogation methods used.)13

The LENZ uses numerical data layers describing various aspects of New Zealand‘s climate, landforms and soils, drawing on long-term meteorological data and the New Zealand Land Resource Inventory. It is the most comprehensive synthesis yet undertaken in New Zealand and its data layers are well documented and substantiated (MfE, 2002a). LENZ is scalable, but provides as a default four levels of aggregation. This research used comparisons between jurisdictions at Level I, containing twenty different environments.

The REC is an ecosystem-based spatial framework for river management purposes and provides a context for inventories of river resources, and a spatial framework for effects assessment, policy development, developing monitoring programmes and interpretation of monitoring data and state-of-environment reporting. The REC has been used to classify all the rivers of New Zealand at a 1:50,000 mapping scale. The area classified comprises 267,000 km2 and 426,000 km of river network.

13 _{Rachel Summers, Senior Lecturer GIS, Massey University, undertook the GIS interrogation of the} LENZ and REC database to produce the initial data tables from which comparisons between regions

5. Survey of environmental resource managers and