Simulation Model Development

Reference Modes and Stylized Facts

The simulating model is an etiological, or theoretical, model to understand basic

57

to reproduce prototypical patterns of behavior, rather than the patterns of specific individuals, and are qualitatively validated (Gilbert, 2008; Hoffer, 2013).

In system dynamics, these patterns over time are called reference modes. Reference modes are represented in abstracted graphs over time, rather than time series data, that highlight shifts in behavior modes over time (Sterman, 2000). Behavior modes are common patterns of behavior found in social systems such as exponential growth and decay, goal-seeking behavior, and oscillatory behavior (Sterman, 2000). Identifying these behavior modes is important because it can point to possible underlying feedback loops (Sato, 2016).

In system dynamics, reference modes are framed not just in terms of how they have changed over time, but also what they are expected to look like in the future without any changes to the system (“business as usual” or “feared”), and what their desired pattern would be in the future (“desired”). The focus is not just on the nature of the problem currently, but in

understanding how it has evolved and what it would take to create desirable change. Reference modes can represent just one variable over time, or a set of variables. When a set of variables is the focus, that means not only that each variable should change over time as expected based on prototypical patterns, but also that the relationships between variables should change in expected ways.

A similar concept is ‘stylized facts,’ which are “broad, but not necessarily universal generalizations of empirical observations and describe essential characteristics of a phenomenon that call for an explanation” (Railsback & Grimm, 2012, p. 228). For example, we would expect that as a person who is addicted to drugs increases their drug use, functioning decreases. Use and functioning may not change at the same rate, and declining functioning may differ in severity

58

across individuals, but in no circumstances would we expect improvement in functioning the more that an addicted person uses drugs.

This study used a set of reference modes, all related to the operationalized definition of recovery. The goal was for the model to reproduce patterns in which the variables of social role functioning, expectations for same, support received, and propensity to use drugs changed over time, and together over time, in expected ways based on generalizations of empirical

observations, i.e., stylized facts. These reference modes were defined based on a combination of extant literature and the interviews, and thus will be discussed in the Results.

Interview Excerpts to Codes to Causal Links to Equations

To build a parsimonious model able to replicate the reference modes, it was critical to gain clarity about how variables are connected. Most of the conceptual work lay in determining how recovery concepts were connected in participants’ minds, i.e., their mental models.

Decisions about causal links were made based on codes from the qualitative data and prior research, mostly qualitative, that had been conducted to highlight the perspective of PWUD and loved ones.

Figure 3.4 depicts the iterative process of moving from interview excerpts to a code, a causal link, and finally an equation, before returning to the data or collecting additional data. Simulating the model is not shown as a step, but it also occurs continuously throughout the model building process. Simulating the model informs coding by clarifying the logic that

underlies participants’ experiences; it informs the causal links by identifying redundant links and creating a more parsimonious model; it informs the equations by highlighting what changes in

59

formulation could better approximate the reference modes; it informs returning to the data by clarifying what data could be helpful to produce a more useful simulation.

Figure 3.4. Iterative Translation between Interviews, Coding, and Model

However, the figure also highlights the inevitable loss of nuance and complexity when translating text into a code, and a code into an equation that can be used in a simulation. Initially, there were many more concepts connecting propensity to use and social role functioning, but they all showed the same patterns when simulated. Moreover, many different construct names were used to capture what is now called “social role functioning.” These variations were either in vivo language from interviews or attempts to capture concepts directly from coding. However, in the end it became clear that what people were describing was their functioning – a ubiquitous concept in addiction literature – and, more specifically, their functioning in social roles (compared to, for instance, their health or mental health functioning).

60

Model Calibration Based on Reference Modes and Stylized Facts

Each parameter in the model (the variables and causal links between them) requires a corresponding value or set of values for the model to simulate. Identifying the “correct” value is different for an individual-level model because of variation across individuals, such as how long it takes to change perceptions. A population-level model might take the average across people for a given variable, but at an individual-level it is more accurate to think of it as the most typical value for a given individual. Thus, any combination of parameters could theoretically represent a single individual or, more likely, a typology representing PWUD with similar underlying

motivations, experiences, and behaviors. However, some combinations produce more plausible model results than others.

Calibration was used to identify values that produced plausible results in the simulation consistent with the reference modes. When these values are changed independently, the model is highly sensitive to those changes. However, when one or more parameters are changed together so that they keep their relative ratios, the model is less sensitive. Thus, the relative difference between parameters, especially delays, were the focus of calibration. Often, values were chosen based on the qualitative data. For instance, words like “eventually” suggest delays of months or years, while in other instances – especially when talking about changing patterns of use – participants would reference changes that typically last a few days to a few months at most.

Causal links also affect the model results. To test the sensitivity of these links, modular testing was used. In modular testing, parts of the model are systematically turned off to test their effects. If turning off parts of the model does not change the model results, this suggests that part of the model could be eliminated to achieve greater parsimony; however, the decision is also

61

dependent on its theoretical value. The goal is to identify the simplest structure with the least uncertainty that can still produce realistic results.

The final model was one whose structure (i.e., the causal links) could reproduce the reference modes and whose behavior responded as expected to changes in parameters. All models built for this study were built in Stella Architect version 1.8.1.

Model Experimentation

After the baseline model was able to reproduce the reference modes, experimentation with the model was used to identify potential ways to intervene to create sustainable and positive change. The changes, or “experiments,” that were tested on the model were based on participant interviews and extant research, treatment interventions, and policy.

The goal in an etiological model is to understand the basic mechanisms that could be contributing to a problematic or interesting behavior pattern. The goal in experimentation with this type of model is to identify a “difference that makes a difference,” meaning a qualitative and sustainable shift of the overall trends in the model in a different direction (M. Agar, 2003; Yang & Gilbert, 2008). This contrasts with numeric differences that do not change the overall

trajectory or shape of the curve (Figure 3.5). However, there may be exceptions where a slight numeric improvement could translate to a meaningful difference for a PWUD.

62

Figure 3.5. Difference Between a Qualitative Shift and a Numeric Difference in Trends

Mechanisms that qualitatively and sustainably shift trends are called leverage points, or small changes that have a disproportionate impact on outcomes (Epstein, 2008; Lich, Ginexi, Osgood, & Mabry, 2013; D. A. Marshall, Burgos-Liz, IJzerman, Osgood, et al., 2015). The impact of leverage points is ideally positive, though it could be negative. “Small changes” and “disproportionate impact” are emphasized because sometimes the effort that would be required to make large positive changes is proportionately large. Whether the effort is worth it would need to be considered carefully. Furthermore, a small change in the model might not translate into a small change in people’s lives or policy.

The goal was to identify what combination of experiments could produce recovery, operationalized as sustained low propensity to use, high social role functioning and expectations for functioning, and high support received. For it to be recovery, these patterns must be sustained for an extended period of time in the model. Once leverage points were identified, the next step was to “work backwards” to understand what specific types of policy interventions, if any, could

0 10 Years 0 10 Numeric difference Status quo Qualitative shift

63

produce those kinds of changes (Hovmand, 2014; Moss, 2008). Implications for interventions are addressed in the Discussion.

Results for the qualitative, theoretical causal loop diagrams and the simulating model will be presented sequentially. The first chapter will discuss the Results based on the theory arising from the analysis of qualitative data. The theory was developed while using causal loop diagrams as heuristic tools to clarify possible causal links and variables. However, many of these were complex causal loop diagrams that are not part of the simulation and so are not shown here. The most important loops are those that were carried forward into the simulation.

64