7.1 Sensitivity analysis
7.1.1 Method of sensitivity analysis
Sensitivity analysis plays an important role in understanding the relationship between model input parameters and important outputs that have some kind of theoretical or practical relevance. It helps assess how the uncertainty in model inputs impacts model outputs and helps identify the most significant parameters in the model. Trucano et al. (2006) have suggested that ‘sensitivity analysis is required for understanding the extent to which a model is complicated enough to be credible but not too complicated’. A considerable number of
studies have conducted sensitivity analysis to evaluate the robustness of agent model results and to examine how the uncertainty in independent input variables can influence dependent output variables. The exercise of sensitivity analysis also aims to ensure that the model outputs comply with the theoretical assumptions and expectation underlying the model. In this case, it is of interest what parameters have the greatest influence on variations in the degree of social risk amplification as an outcome. Representing social risk amplification as a relatively complex mechanism means that the link between independent variables and the difference between objective and perceived risk becomes difficult to predict. Sensitivity analysis helps us understand how strong this link is, as an emergent property of the model.
Sensitivity analysis is commonly performed in a qualitative way by varying the value of testing parameters while keeping other parameters constant (Anderson et al., 2007; Grow et al., 2015; Jiang et al., 2016; Kimbrough and Murphy, 2013; Lee et al., 2013; Liu and Wu, 2016; Malik et al., 2015; Millington et al., 2014; Nagarajan et al., 2012; Okada, 2011; Stummer et al., 2015). For example, in an agent-based model of humanitarian assistance policies (Anderson et al., 2007), the authors studied the effect of all input parameters (levels of food and water, levels of security, medical personnel, medical resources, and sanitation) on the sickness rate of refugees. Each testing parameter was varied around its prior value while all other parameters were kept constant at their midpoints. Results show that the sickness rate decreases with the increase of levels of sanitation, security, medical resources, and medical personnel.
Occasional studies have made sensitivity explorations in a quantitative way. Zhang and Li (2014) considered an agent model of the search behaviour in China’s resale housing market. They selected four parameters (matching efficiency, unit search cost, market tightness ratio, and broker commission rate) and one output (search intensity of both buyers and sellers) for the sensitivity analysis, and used the simple random sampling to estimate the correlation of each single input parameter with the output. The results show that the increase in the matching efficiency can reduce the search time of buyers and sellers significantly, but there is no evidence that the unit search cost exerts strong impact on the search time. In a more complex approach, Fonoberova et al. (2013) proposed a global sensitivity approach that evaluates the effect of a parameter while all other parameters are varied simultaneously. The measure is based on support-vector regression and thus takes account of the interactions between model parameters. The authors tested variance-based and derivative-based global sensitivity measure through an agent-based model of civil violence, and global sensitivity analysis was found to be capable of identifying the most significant and non-significant parameters in the model. Similarly, Kucherenko et al. (2009) also presented a derivative-based global sensitivity measures and provided evidence that their approach could be more efficient and more
Sensitivity analysis in this study evaluates the main effect linking each of the model input parameters with defined outcome variables. More specifically, it examines the effect of one parameter with other parameters held constant at their default values. This simple approach was selected in order to make the sensitivity analysis straightforward to interpret. In principle, interaction effects between model parameters could be significant, and global sensitivity analysis could uncover variations in sensitivity over the parameter space. But these more complex analyses are demanding in the software for model evaluations and the amount of time to complete the analyses.
The analysis was conducted based on a calibrated small-world network model. Eight parameters (see Table 7.1) are considered significant for defining the global uncertainty in model outputs. Recall voluntariness v t
is not included as it is a binary variable, but the sensitivity analysis was performed in the case of both voluntary recall and involuntary recall. Parameters used in the sensitivity analysis are maximum initial condition I , number of neighbours K , rewiring probability P, low contamination level Clow
t , high contaminationlevel Chigh
t , contamination end period Tend (contamination start period Tstart is fixed at2000), maximum perception threshold S, and maximum recreancy variation H . In particular, certain starting conditions such as initial risk and recreancy belief are taken into account as difference was observed between results associated with a higher level of initial conditions and those in Chapter 5. According to the traces of risk perception in a single run in Section 5.2.2 and Section 5.3, contamination appears to have an impact on the occurrence and degree of risk amplification, so parameters pertaining to contamination are all included. Simulation results in 5.2.2 and Section 5.3 have proven recreancy to be influential in shaping risk responses, thus parameters related to recreancy belief are selected for sensitivity analysis.
Table 7.1 Input parameters used in sensitivity analysis
Input parameter Description
Maximum initial condition I Defines initial risk and recreancy belief Number of neighbours K Number of neighbours in initial lattice Rewiring probability P Probability of reconnecting a lattice edge Low contamination level Clow
t Level before and after crisisHigh contamination level Chigh
t Level during the crisisContamination end period Tend Time when the crisis ends
Maximum perception threshold S Defines when a recall increases recreancy Maximum recreancy variation H Maximum by which recreancy can change
Four outcome variables provided in Table 7.2 are considered for the sensitivity analysis. They are mean risk amplification over crisis m, peak risk amplification
p, peak delay fromcrisis start c, and peak delay from recall start r. Mean risk amplification over crisis m is
associated with the mean degree of risk amplification during the contamination incident, and peak risk amplification
p involves the maximum discrepancy between public risk perceptionand the objective risk. The purpose of considering these two variables is to gain an insight into the relationship between input parameters and the degree of risk amplification. Peak delay from crisis start c and peak delay from recall start r are employed to explore how changes
in model inputs affect the timing of risk amplification.
Table 7.2 Outcome variables used in sensitivity analysis
Outcome variable Description
Mean risk amplification over crisis m
Mean ratio of public risk perception to the objective risk during the crisis
Peak risk amplification p
Ratio of peak risk perception to the objective risk
Peak delay from crisis start c Time delay between peak risk amplification and crisis start
Peak delay from recall start r Time delay between peak risk amplification and recall start
Mean risk amplification over crisis m is the mean of ratio of public risk perception to the
objective risk level during the crisis:
1 1 1 1 1 / end start Z T N m i high end start j t T i b t C t Z T T N
(7.1) where N, Tstart, Tend, Z, and Chigh
t denote the number of agents, crisis start period, crisisend period, the number of runs, and the contamination level during the crisis. Peak risk amplification
p is defined as the ratio between peak risk perception and the objective risklevel:
1 1 1 1 max / Z N i j p hig i h N b t t Z C
(7.2) where N and Z signify the number of agents and number of runs, and Chigh
t is thecontamination level during the crisis (fixed during the crisis). Peak delay from crisis start c
refers to the time delay from the start of the crisis to the occurrence of peak risk amplification:
1 1 1 1 max Z N c peak s i start j i T b t T Z N
(7.3) where Tpeak represents the time when peak risk perception in a single run s arises, Tstart ismodel runs. Similarly, peak delay from recall start r is defined as the delay between the start
of recall and peak risk amplification:
1 1 1 1 1 max Z N r peak s i a t j i T b t T Z N
(7.4) where Tpeak denotes the time when peak risk perception in a single run s emerges, Ta t 1indicates the time when a recall announcement is made, and N is the number of agents, and
Z represents the number of times the model is replicated.
The approach used for sensitivity analysis in this study is one-at-a-time (OAT), which is the simplest and most widely used approach seen in the literature. It evaluates the effect of one parameter at a time with all other parameters left at base values shown in Table 6.11. Varying one factor at a time means that the effect observed on the output is due solely to that factor so makes interpretation simpler. Another important consideration is that the computational cost (i.e. the number of times the model has to be evaluated) is relatively low when dealing with thousands of simulations, which is actually the case in this study.
Each parameter is sampled 200 times uniformly from a specified range provided in Table 7.3. The maximum and minimum selected for each parameter are a subjective choice, but they are believed to encompass the reasonably likely range of each parameter. Parameters are sampled uniformly, first, because no particular assumption is then made about the distribution of input parameters. Second, this study concentrates on exploring how sensitive the model outputs are to the variations of model inputs rather than performing an uncertainty analysis to describe the distribution of possible outcomes given uncertainty about a set of inputs with known distributions. Some scholars have used a uniform distribution to draw samples of input parameters within specific spaces and obtained sensible sensitivity analysis results (Fonoberova et al., 2013; Nagarajan et al., 2012).
Table 7.3 Sample range of input parameters
Input parameter Range Base value
Maximum initial condition I [0, 1] 10-4
Number of neighbours K [2, 50] 4
Rewiring probability P [0, 1] 0.5
Low contamination level Clow
t [10-6, 10-3] 10-4High contamination level Chigh
t [2×10-4, 1] 0.2Contamination end period Tend [3×103, 16×103] 5999
Maximum perception threshold S [0, 1] 1
For each sample the model is run 100 times for the duration of 20,000 periods with 1,000 agents in a small-world network. Pearson product-moment correlation coefficient is calculated to analyse the impact of each considered parameters. In this way the model outcome variables are numerically compared. However, OAT ‘is predicated on assumptions of model linearity’ (Saltelli and Annoni, 2010), and Pearson’s correlation coefficient measures the fitness of a linear correlation between two variables. As a consequence, scatter plots, together with associated least-squares approximation, of outcome variables against individual input variables are plotted to visually depict the correlations.