Sensitivity Analysis of the Model

Each composite index is constructed by several subjective steps, which include the calculation method, selection of indicators, choice of aggregation and weighting procedures that are associated with some uncertainties in the methodology. Therefore, it is necessary to analyse the sensitivity of the index by using alternative methodological assumptions (Manca et al., 2010). A sensitivity analysis helps to assess the robustness of the index, and investigate the potential changes and their impact on the results derived from the index. As stated by Pannell (1997, p. 140), a sensitivity analysis is helpful in model development in order to: (1) test the model for validity or accuracy; (2) search for errors in the model; (3) calibrate the model; (4) cope with poor or missing data, and; (5) prioritise acquisition of information. In this context, a sensitivity analysis was performed to show the impact of the alternative methodological approaches on the overall results of the MUSIX model.

As the first part of the sensitivity analysis of the model, alternative techniques were applied in the weighting and aggregation procedures as follows: (1) Equal Weighting, which provides the measurement of each indicator with the same degree of importance, (2) Factor Analysis (FA), which allows investigating a statistical relationship to determine the importance of each indicator (the Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy (above 0.8 is acceptable) and the Bartlett‟s Test of Sphericity (below 0.05 is acceptable) are used to examine the appropriateness of FA (Hanafizadeh et al., 2009), see Appendix 4.1), and; (3) Geometric aggregation (in which indicators are multiplied and weights appear as exponents), which allows investigating the correlation among the performance of the indicators (Nardo et al., 2005b; Saisana, 2008). The composite index scores were calculated by different combinations of alternative methodological techniques, as illustrated in Appendix 4.2.

As seen from the maps in Appendix 4.2, for all sites, the calculation based on „Expert Opinion Weighting & Geometric Aggregation‟, „FA Weighting & Geometric Aggregation‟ and „Expert Opinion Weighting & Geometric Aggregation‟ yield lower sustainability results compared to the MUSIX model results. Specifically, FA weighting with geometric aggregation performed negative differences in a couple of grid cells compared to other scenarios. The underlying reason for this difference

151 depends on the fact that geometric aggregation uses multiplication to summarise data; hence, it performs lower scores than arithmetic aggregation. Additionally, the FA revealed a slightly different categorisation of the indicator set, which is grouped under four factors. As shown in Table 4.12, the first factor includes indicators referring to Hydrology, Design and Efficiency categories. This correlation can be interpreted as being due to the large amounts of impervious surfaces, which are associated with increased surface runoff, unsustainable design of built environment and higher resource consumption. The second factor includes indicators referring to the Pollution category. The third factor includes indicators referring to the Location category and the fourth factor includes indicators referring to the Ecology category. These factors show the same structure as the MUSIX model categorisation.

Table 4.12 Factor analysis weightings of the indicator set

CATEGORY INDICATORS Weighted Factor Loadings

1 2 3 4

HYDROLOGY Impervious surface ratio (ISR) 0,101 0,000 0,001 0,001 Surface runoff (SR) 0,092 0,001 0,000 0,003 DESIGN Lot design (LOTDSG) 0,073 0,003 0,000 0,005 Landscape design (LNDDSG) 0,092 0,002 0,001 0,004 EFFICIENCY Energy consumption (ENERGY) 0,070 0,000 0,000 0,004 Water consumption (WATER) 0,051 0,005 0,003 0,020 POLLUTION

Air pollution (AIR) 0,001 0,096 0,001 0,000 Stormwater pollution (SW) 0,000 0,094 0,004 0,001 Noise pollution (NOISE) 0,001 0,047 0,014 0,006 LOCATION

Land use destinations (LUD) 0,001 0,000 0,096 0,000 Public transport (PT) 0,001 0,002 0,091 0,002 Walkability (WLK) 0,000 0,000 0,021 0,000 ECOLOGY Green area ratio (GAR) 0,014 0,000 0,000 0,056

Albedo (EA) 0,002 0,000 0,000 0,018

In order to assess the overall impact of these different methodological assumptions on the MUSIX model results, Spearman‟s rank correlation analysis was performed with reference to a number of similar studies (Groh et al., 2008; Groh and Wich, 2009; Saisana, 2010). Due to the large data set, the level of significance was set at 0.05 and a two-tailed test was chosen to identify the level of significant differences between the indicator data set in either direction. The correlation analysis revealed that the impact of any of these assumptions is negligible overall as the correlations between the MUSIX model results and the others is greater than 0.9 (Table 4.13). All correlations are positive, which indicates that they point in the same direction. „FA Weighting & Geometric Aggregation‟ method has the lowest

152 correlation while „Equal Weighting & Linear Aggregation‟ method has the highest correlation with the implemented method.

Table 4.13 Correlation between the MUSIX model results and different methodological assumptions

Alternative calculation methods Correlation with the implemented method

(Expert opinion weighting, Linear aggregation)

Equal Weighting, Linear Aggregation ,995

**

FA Weighting, Linear Aggregation ,988**

Equal Weighting, Geometric Aggregation ,985

**

FA Weighting, Geometric Aggregation ,975**

Expert Opinion Weighting, Geometric Aggregation ,990**

**. Correlation is significant at the 0.05 level (2-tailed)

Complementary to the correlation analysis, the impact of an underlying indicator on overall outcome of the model was assessed by performing „exclusion of one indicator at a time‟. The analysis was conducted by removing one indicator at a time and then recalculating a reduced model score (Table 4.14). A low correlation between the MUSIX model score and reduced model score implies that the model is highly sensitive to the exclusion of that indicator. The analysis revealed that the removal of the noise pollution indicator had the highest effect while removal of air pollution and walkability indicators have the lowest effect on the overall model score. In general, the correlation between the MUSIX model score and the reduced model scores are greater than 0.5, which is considered to be acceptable (Katz, 1999; Lehman et al., 2005; Morien, 2006; Christmann and Badgett, 2009). This means that the removal of indicators does not significantly change the overall MUSIX model score.

Table 4.14 Correlation between the MUSIX model score and reduced model scores

Reduced Model Spearman’s Correlation

Evapotranspiration removed ,727**

Surface Runoff removed ,657**

Urban Habitat removed ,607**

Microclimate removed ,630**

Stormwater Pollution removed ,674**

Air Pollution removed ,808**

Noise Pollution removed ,563**

Proximity to Land Use Destinations removed ,696**

Access to Public Transport Stops removed ,709**

Walkability removed ,861**

153

Landscape Design removed ,759**

Energy Conservation removed ,661**

Water Conservation removed ,641**

The MUSIX model 1,000

**. Correlation is significant at the 0.05 level (2-tailed)

4.5 SUMMARY OF THE CHAPTER

In this chapter, the findings of the MUSIX model from the case study sites were presented at both parcel and grid-based levels. While the indicator set of the model provided specific information about the environmental impacts in the area at the parcel scale, the composite index score provided general information about the sustainability of the area at the neighbourhood scale. Following model implementation, sensitivity analysis was undertaken to assess the robustness of the model. The results of the sensitivity analysis showed that the MUSIX model scores are reliable and not highly sensitive to changes in the weighting or aggregation methods. Furthermore, none of the indicators have a dominant effect on the overall result. According to the model findings, the sites located in the canal development performed lower sustainable behaviour than the other case study sites. Environmental impacts associated with canal-estate development include: increased stormwater and runoff, loss of natural vegetation, inadequate public transportation, automobile dependency, and irregular shaped lots covered by large impervious surfaces and lack of energy efficient design (e.g., lot shape, siting of the house, building orientation, use of rain water tanks or solar panels). Furthermore, the sites that are close to the local amenities and services performed better than canal-estate developments. The parcels located in the sites provide a high percentage of green spaces, which also promotes microclimate and outdoor thermal comfort. Additionally, the sites provide a good picture of environmental quality in terms of air and noise pollution. However, these sites also confront the same environmental impacts, such as increased surface runoff, auto-dependent pattern of development as well as dependence on non- renewable resources. Briefly, analysis of the findings clearly shows that there are major environmental impacts in the study area arising from increased impervious surfaces due to urban development and population growth. In light of the model findings, the following chapter provides a more in-depth discussion of the environmental impacts arising from development pressure on urban ecosystems in the case of GCC by highlighting the recommended environmental policy actions.

154