INDICATOR WEIGHTS BASED ON EXPERT OPINION

In composite indices, indicator weighting reflects the importance given to the variables forming the index. During the calibration process, pilot studies were conducted with equal weightings in order to test the capabilities and accuracy of the model. Moreover, a series of workshops were organised with the team of ARC Linkage project experts, researchers and local government policy-makers to provide their professional opinion about selected indicators. In these workshops, participants were asked to provide their professional opinion about the relevance of selected indicators. They were asked to comment on whether the indicators were: (1) too specific and needed to be merged as new indicators or with another indicators in the list; (2) too general and needed to be defined more specifically; or (3) irrelevant and needed to be removed from the list.

The construction of composite indicators consists of different stages (i.e., analytical approach, weighting criteria, aggregating technique, and sensitivity analysis). Each stage is subjective, which requires selecting an appropriate methodological approach (Maggino and Ruviglioni, 2009). One of the key tasks is to select appropriate weighting criteria. Indicators need to be chosen carefully so that they reflect the environmental issues and measure the environmental performance of the study area effectively. As a result of the subjective nature of indicator selection, expert survey allows experts from various backgrounds to agree on a consensus view of the relative importance of the indicators based on their experience and subjective judgment. For this study, expert opinion weighting was selected due to the spatial scale and scope of the research. First of all, the MUSIX model is developed to measure the local-level environmental performance of an urban area. In this sense, consultation of local expert‟s opinion helps to reflect the implications of the current planning policies, local environmental issues and needs of the study area. Secondly, the MUSIX model is developed as an assessment tool to serve in policy and

95 decision-making processes. In this sense, the model results are highly benefited from the input from developers, planners and policy makers that consist of the expert survey participants. Expert judgment has been used in a number of studies, including Environmental Performance Index (Esty et al., 2006), Environmental Sustainability Index (ESI, 2005), Eco-indicator 99 (Pre Consultants, 2004), E-Business Readiness Index (Pennoni et al., 2006), Urban Sustainability Index (Zhang, 2002), and Index of Environmental Friendliness (Puolamaa et al., 1996).

In the next step, weightings for the indicators were assigned via expert survey.

A total number of 21 experts participated in the survey. The participants comprised academics, planners, engineers and architects who are familiar with policy priorities and theoretical background. Participants were chosen from the industry partners of the ARC Linkage project: Queensland Transport and Main Roads (n=7) and Gold Coast City Council (n=7) and Queensland University of Technology (n=7).

Purposive sampling was used to select the experts for this study, which means that industry partner representatives were asked to suggest appropriate contact persons.

The invitation letters were sent by email (See Appendix 3.3). The interview times and locations were arranged that were most convenient for participants.

The survey comprised of two stages. The first stage consisted of a demonstration survey showing snapshots from various parcels (with their equally weighted indicator scores) in the selected case study areas (See Appendix 3.4). Each participant was asked to assign a sustainability level for each parcel using a five-point Likert scale (1=low, 5=high), by analysing the aerial photos and indicator scores. This survey was designed to make participants more familiar with the calculation and interpretation of the indicators. Therefore, the results were not included in the study.

For the second stage, a ranking survey sheet (which consists of two steps) was prepared (See Appendix 3.5). In the first step, each participant was asked to rate the importance of each indicator in terms of its contribution to environmental sustainability assessment using a five-point Likert scale as follows:

1. Not important: Does not affect the assessment of environmental sustainability.

96 2. Slightly important: Affects the assessment of environmental sustainability in

a minor way.

3. Moderately important: Affects the assessment of environmental sustainability in a moderately way.

4. Important: Essential and affects the assessment of environmental sustainability in a significant way.

5. Very important: Very essential and affect the assessment of environmental sustainability in an extremely significant way.

In the second step, each participant was asked to assign a weight using the budget allocation method by allocating a total of 100 points to each sub-category in terms of their importance in the model and for each indicator in terms of their importance in the sub-category. First, weightings for sub-categories were calculated by dividing the sum scores of each category by the total sum score of all sub-categories and then the result was multiplied by 100 to provide a percentage weighted score. Second, weightings for indicators were calculated by dividing the sum scores of each indicator by the total sum score of all indicators in the same sub-category and then the result was multiplied by the sub-sub-category‟s weighted score.

Lastly, these scores were rescaled between 0 and 1, as illustrated by a chart in Figure 3.11.

Figure 3.11 Rescaled expert weightings

Afterwards, statistical analysis of the participant‟s responses was computed using PASW Statistics 18. A descriptive analysis and Cronbach's alpha reliability test were conducted to identify the central tendencies of data (See Appendix 3.6). The

97 Cronbach's alpha reliability test is used to measure the internal consistency of the data. The Cronbach's alpha result (α = 0.824) was over the acceptable reliability threshold stated by George and Mallery (2003). The descriptive analysis showed the average of all participants‟ level of agreement per indicator (mean, median, and mode) and the distribution of the respondents` score that fell within the scale (frequency distribution). Furthermore, the standard deviation showed the average amount that respondent's ratings varied from the mean and indicated the varied view between respondents.

Table 3.23 Mean relevance rate, rescaled weightings and ranking of indicators

Indicators Relative

Evapotranspiration 3,81 0,071 7

Landscape design 3,81 0,071 7

Stormwater pollution 3,76 0,068 9

Proximity to land use destinations 3,76 0,068 9 Access to public transport stops 3,67 0,064 11

Walkability 3,62 0,062 12

Air pollution 3,52 0,050 13

Noise pollution 3,48 0,048 14

Tables 3.23 present the mean relevance rating, rescaled weightings and ranking of indicators based on their relative importance. The results shown in Table 3.2 indicate that experts assigned „energy conservation‟ as the most important indicator and they assigned „noise pollution‟ as the least important indicator. Moreover, the results show that all indicators met the minimum required relevance rate of 3 and above so that they were confirmed by experts as key components in environmental sustainability assessment.