Data and Information Collection

Case study research does not imply the use of a particular type of evidence. Case studies can be based on either qualitative or quantitative evidence. The evidence may come from a variety of primary and secondary sources such as official statistics, literature, research reports, interviews, observations, or any combination of these. A case study approach does not imply the use of a particular data collection method (Yin, 1981). Data, however, needs to support the research objective. The design of an effective data collection strategy is important as it sits between the research questions and the data analysis (Punch, 2013).

The central research question of this study revolves around important equity issues inherent in an electricity sector reform. Data collection is largely stakeholder driven.

Stakeholders define the problem situation, which subsequently determines the detailed data requirements in terms of the data itself, the source and the collection method.

At the core of the data collection is the elicitation of information from stakeholders in the form of qualitative interviews and close ended questionnaires. Experts in the Chinese energy sector and carbon trading as well as residential electricity users are interviewed in order to obtain their views on a market-led reform of the electricity sector. The problem situation itself presents variables and evidence which need to be obtained from various sources in order to be able to construct a causal description of the energy sector. In addition to stakeholder input these variables are identified through the review of relevant literature.

Quantitative socio-economic and electricity sector data quantifying these variables and the causal links between them is required to test solution alternatives in a systems dynamic model. Primary data is mainly obtained from official government publications. Past research serves as source of data which could not be located in official records.

In light of the participatory element of this study it is important to understand the difference between data and information. Data are facts while information is interpreted facts. Information gives meaning to data (Scheaffer et al., 2011). Information is, therefore, subjective and dependent on the perspectives of the one interpreting the data. The output from the consultation of stakeholders is data from the point of view of the researcher as she has not analysed and processed it. However, as people see the world through their own specific lens, stakeholder observation of the events is considered information.

The combination of various data collection methods is vital in this research as it establishes strong links between what people think (interviews) and what they want (questionnaires) in terms of a solution with a carbon market at its core (written documentation including official statistics and literature; system modelling).

97 4.2.1 Socio-Economic and Environmental Data

Official government statistics at national and provincial level such as Statistical Yearbooks published by National Statistical Bureau of China were consulted to obtain socio-economic and environmental data. When the data analysis brought to light gaps between the retrieved data and the required data, the attempt was made to fill the void with data originating from previous research.

In order to better understand in the context of a carbon market, numerous pieces of formal documents such as policy papers plus hundreds of informal documents (mainly news reports) were collected via different channels such as the internet and print media over the course of the study. The information and ideas conveyed through these documents provide clues as to why certain event patterns have emerged over time. They offer insights into the structure of the system, give evidence for the existence of norms and rules and allow a glimpse of prevalent mental models in a particular social setting. Despite their usefulness, documents require careful interpretation as relevant questions regarding their origin and the author’s intention need to be addressed before incorporating information into the study (Hammersley and Atkinson, 1983).

4.2.2 Stakeholder Information

The consultation of stakeholders forms the central pillar of the empirical study. The views of stakeholders on the electricity sector reform are the basis of the analysis, from which data requirements for systems modelling and Multi-Criteria Decision Analysis are established.

Before explaining the study’s approach to sampling and information elicitation, the next section provides the rationale for employing stakeholder analysis as methodology to support the information and data collection.

4.2.2.1 Stakeholder Analysis

There is growing evidence that the assumption of all stakeholders having similar interests leads to the failure of initiatives and engenders resistance and resentment of those affected.

In the private and public sector stakeholder consultation has gained in importance over the years as there is growing realisation that those affected by a decision could influence the success of its implementation (Brugha and Varvasovsky, 2000). Grimble and Wellard (1997) specifically connect the importance of participatory stakeholder analysis to the trade-offs inherent in sustainable development.³³

33Grimble and Wellard (1997) state that “Clearly there is a link between the three E’s [economic efficiency, equity and environmental objectives] and various social or economic groups or stakeholders with differing spheres of interest, concerns and priorities. Stakeholder analysis (SA) has been developed in response to the

98

A plethora of different approaches and definitions give rise to confusion over what is really meant by stakeholder analysis (Donaldson and Preston, 1995; Stoney and Winstanley, 2001; Weyer, 1996). A review of relevant literature revealed a difference of opinion over who or what exactly stakeholders are. Most definitions seem to be derived from Freeman’s (1984) seminal work on stakeholder theory. According to Freeman stakeholders are those who affect or are affected by a decision or action. He recognises that “stakeholders (are) in a system”. Decision makers or experts, who frame and structure a problem and present implications of interventions, are usually included in the first category.

The second category includes all those who are affected by the decisions made by others.

In the context of environmental pollution, Coase (2013) classifies stakeholders as polluters and victims. Polluters can implement change by altering the level of pollution and the victims are those who are directly or indirectly affected. Checkland (1981) rejects the idea of stakeholders being ‘helpless victims’ relying on someone else, an actor, to resolve the problem. He believes that whoever is affected by a problem should be a part of the process to solve it. The definition of stakeholders as active participants in the process of solution formulation adopted by this study is the one provided by Checkland. The understanding of stakeholders being part of the solution finds direct application in Multi-Criteria Decision Analysis. The requirement to consider the views of different stakeholder groups is commensurate with the ‘inclusiveness’ notion of equity that is the central theme of this research. For the avoidance of doubt, it shall be mentioned that the study is not concerned with the legitimacy of stakeholder groups, their level of decision making authority or their capacity of getting attention. The common understanding of a stakeholder which only considers individuals and groups upon which an organisation relies for its survival (Mitchell et al., 1997; Pajunen, 2006; Neville et al., 2011) appears to be too narrow for the purpose of this study. This definition would exclude marginalised stakeholders with no political clout or limited financial resources. The exclusion of stakeholders could lead to the

implementation of unequitable solutions.

Stakeholder analysis is closely related to the central notion of soft systems methodology (SSM), which regards the direct involvement of stakeholders as the key to successful intervention design, in particular in the presence of complex sustainability related problems with many inherent trade-offs. Soft systems approaches are concerned with creating awareness of the ‘context’ in which stakeholders face a certain problem

(Checkland, 1981). Soft systems methodology seeks to understand the mental models, the concepts and beliefs individuals hold and use to make judgments about reality (Elsawah, 2015). Stakeholder analysis provides a toolset to elicit the, often diverging, perspectives, interests and objectives of those who are selected to be within the system of

consideration.

challenge of multiple interests and objectives and added to the basket of approaches available for the analysis and formulation of development policy and practice.”

99 4.2.2.2 Stakeholder Sampling

Stakeholder identification and selection are a critical part of the study as the precondition of inclusion determines whose views are considered and so define the boundaries of the system of interest. Miles and Huberman (1994) noted that “you cannot study everyone everywhere doing everything.” Even though it is desirable to understand as many views as possible, it is not practical to study everyone who is affected by the residential electricity sector reform in China. Sampling is concerned with the selection of a subset of individuals from within a population (Tranter, 2014).

It is reiterated here that stakeholder input is required at two points during the empirical phase of the research. Firstly, stakeholders are interviewed to support the qualitative part of the study which is concerned with better understanding the problem situation, which presents itself with the introduction of a carbon market for residential electricity users.

Secondly, stakeholders state their preferences for the outcome of the market based reform in a questionnaire, which provides quantitative input for a Multi-Criteria Decision Analysis.

Sampling is as important in qualitative research as it is in quantitative research. However, there are differences in sampling in qualitative and quantitative research (Punch, 2013). In quantitative sampling the focus is on identifying a sample which is representative of the larger population being studied. Qualitative research tends to use a form of deliberative sampling, with a focus on a particular purpose.

A thorough and organised approach of stakeholder analysis is important at the beginning of the participatory process. This is particularly relevant in complex situations to avoid the haphazard identification and selection of stakeholders. There is a great variety of sampling approaches in qualitative research to suit different research purposes and settings (Punch, 2013). Miles, Huberman and Saldana (2013) identify around 20 different sampling

methods. Janesick (1994) and Patton (2002) highlight the existence of even more.

Common approaches for stakeholder sampling fall into top-down and bottom-up methods.

The strength of a top-down approach is that it is a systematic way of discovering stakeholders. Its very broad and generic nature allows it to identify a wide range of stakeholders. Top-down identification minimises the chance of leaving out central stakeholders which could lead to crucial questions not being asked (Bryson, 2004).

Identifying stakeholders ‘bottom-up’ contributes to minimising researcher bias which subconsciously could prioritise certain views and exclude others. Snowball sampling is a commonly used stakeholder identification technique where initially located individuals identify new stakeholder groups or individuals during interviews. Snowball sampling is a convenient way to increase the size of the sample as the process continues (Reed et al., 2009).

The main criterion of choosing a sampling method needs to be based on the objective of the study and the questions it seeks to answer. In the context of this study the combination of theory based sampling, a top down approach, and a snowball approach is considered to be most suitable to fit the purpose of the research. Theory based sampling is used in a

100

purposive way to set the sampling frame. The sampling frame provides a map that identifies individuals or groups to be included in the study (Willmot, 2005). The basic conceptual model of the system, which is being studied, is used as a sampling frame. It guides the identification of the stakeholders affected by the carbon market based electricity sector reform.

Although the important first step in any participatory approach, stakeholders are often identified randomly on an ad-hoc basis. This carries the risk of marginalising certain individuals or groups, not addressing the actual problem and leading to a biased and unequitable outcome (Reed at al., 2009). A carefully thought-through approach to sampling can address this risk. A systemic view allows the identification of different stakeholder groups and their position within the system. In order to be able to answer the main research question which is concerned with the equity aspect of the reform, the process of sample selection needs to ensure that the majority of the system elements are being represented.

Top-down analysis based purely on desk research and published material, however, entails the danger to close down the identification of stakeholders prematurely. Lacking the divergence of perspectives could potentially undermine the idea of systems based

research, which seeks to fully understand an issue. The adoption of a bottom-up approach as the empirical study evolves could mitigate this risk.

It has been noted that there is a clear trade-off between the number of stakeholders and the feasibility of the study in terms of data collection and analysis. In light of the vast number of stakeholders in the reform and the great variety of different groups affected by the introduction of a carbon market, the sampling criteria therefore need to consider the

‘level of affectedness’ to make the task of stakeholder identification manageable within the time and resource constraints of the research. The issues involved in identifying and

selecting individuals as well as obtaining their commitment to take part in the study within the cultural setting of China are discussed in 4.5.

Sampling in the quantitative part of this study is people sampling. The idea behind people sampling is that the researcher uses data collected from the sample in order to infer findings back to the entire population that is being studied. Representativeness is a key concept. The extent to which it is applicable depends on the objective of the study. As data collection supporting the quantitative part of the study involves the completion of

questionnaires by the general public, there is the practical issue of gaining access to a large and representative sample. Convenience sampling (Etikan et al., 2016), where the

researcher takes advantage of a particular situation which provides access to study

participants, is expected to play an important part in obtaining quantitative data for further analysis. Convenience sampling must not be carried out haphazardly in order not to

compromise the sampling frame determined by each of the study cases.

It is anticipated that secondary analysis, which involves the perusal of data collected and analysed by other researchers, supplements the insights gained from evaluation of own

101 survey data. This approach could mitigate the lack of representativeness of the data

collected during the fieldwork stage of this research (Procter, 1996). Despite the potential of secondary data analysis to improve the quality of this study, it needs to be recognised that methodological difficulties could arise caused by different research objectives and incompatible conceptual frameworks (Reeve and Walberg, 1997).

4.2.2.3 Stakeholder Information Elicitation Techniques

Elicitation, in which knowledge is sought directly from human beings, is usually

distinguished from indirect methods such as gathering information from written sources.

In the context of this study the conclusions from the extensive review of literature covering a wide range of issues and topics is underpinned by tacit knowledge elicited from a variety of stakeholders. The success of stakeholder involvement in terms of gaining a deeper understanding of the situation and the requirements for a solution are greatly influenced by the techniques employed during the elicitation stage.

4.2.2.4 Qualitative Data Elicitation: Semi-Structured Interviews

There are many different elicitation techniques ranging from observations to close ended quantitative surveys. The decision to use interviews as the main technique to gather information is based on the study’s requirement for participatory research. Qualitative research interviews typically reveal ‘subject perceptions’ (Kvale, 1996).

The objective of the initial information elicitation phase is to enhance the knowledge gained from the literature review by understanding different perspectives on the electricity reform. In line with stakeholder identification through purposive sampling, interviewing experts in the Chinese energy sector as well as actors in the reform appears to be the most efficient way to discover tacit knowledge on the political factors and technical details which have not appeared in mainstream literature. Stakeholders representing residential

electricity consumers, who are not professionally, politically and academically linked to the Chinese energy system or the reform, are also consulted. Semi-structured interviews are carried out to discover an individual’s personal areas of concern and his view on the wider issues linked to the consumption of electricity.

Semi-structured interviews, in contrast to fully structured or unstructured interviews, appear to be the most suitable interview type for systems based research. Open ended questions are thought to be an appropriate method to identify the key issues associated with energy modelling and sustainability analysis. The pre-set structure allows for a degree of consistency to ensure comparability between stakeholder/expert groups (Bryman, 2008). The open nature of the queries provides a flexible approach well-suited to explore topics or issues as they emerge with a series of follow-up questions or probes.

102

For the great majority of the pre-arranged meetings a list of ‘key areas’ is identified, that the interview intends to cover. This list is sent to the interview partner in preparation for the meeting. Despite the structured approach, the open ended nature of the questions allows for variation in responses. In addition to allowing the possibility for the exploration of novel themes, this flexible type of interviewing technique appears useful as it permits to tailor the communication style to the interviewee. A catalogue of questions serves as reference for the researcher during the interview. This ensures that all relevant areas were covered. In the case of non-English speaking interview partners the questions were

formulated prior to the interview, translated by a research assistant into Chinese and forwarded to the interviewee.

4.2.2.5 Quantitative Data Elicitation: Survey

The choice of survey as a research method for collecting information regarding people’s views on issues linked to residential electricity consumption is driven by the prospect of being able to collect a lot of information over a relatively short period of time. The

versatility of the survey in the form of a standardised questionnaire allows to gather a great variety of information which can be analysed with computer based spreadsheet

applications.

The survey is developed to collect information to answer the research questions regarding people’s preferences for the outcome of the reform and their priorities regarding economic development, environmental quality and regional equity, the key sustainability objectives of the ‘New Normal’. Another requirement for the questionnaire is to capture attitudes towards the adoption of pro-environmental behaviour. These different dimensions need to be operationalised through the adoption of indicators once relevant questions for the questionnaire have been developed.

Questionnaires can be administered in person, by post and electronically. It is anticipated that the study locations identified as part of the stakeholder led case definition (Chapter 5) influences the mode of survey delivery. Self-administered (internet) questionnaires, which do not require the presence of an administrator, can boost the overall number of surveys completed, particularly in regions which cannot be visited due to resource or time constraints (Meadows 2003; Wright, 2005).

A number of potential issues have been identified with the surveying approach adopted by this research. The most common criticism levelled at survey based information collection is

A number of potential issues have been identified with the surveying approach adopted by this research. The most common criticism levelled at survey based information collection is