Missing data

The near inevitability of missing data in intensive longitudinal studies has been well documented (Black et al., 2012; Kimhy et al., 2012). Three kinds of naturally arising missing data can be differentiated in studies of this nature. First, item non-response occurs when participants answer only a subset of items at any particular measurement occasion, and do not respond to certain individual items (Schafer & Graham, 2002). This type of non-response is not such a problem when questionnaires are administered via smartphone, since participants are typically required to complete the entire questionnaire. Second, wave non-response occurs when participants do not complete any items for a particular measurement occasion (Jelicić, Phelps, & Lerner, 2009). This typically occurs when participants miss or dismiss the alarm. This is the most common type of missing data in ESM studies. Finally, missing data can occur due to attrition (a special case of wave non-response); where a participant drops out of the study and does not return.

Further to these causes, the decision is often made in ESM studies to exclude measurements that are not completed within a requested time-frame (e.g. within 15 minutes of the prompt); these measurements are typically considered ‘invalid’, as they may no longer represent ambulant monitoring of experience (Palmier-Claus et al., 2011).

In addition, it is often recommended to exclude participants who provide a limited number of valid reports (typically those completing less than a third of assessments over the sampling procedure), since these measurements can no longer be considered a random sample of momentary experiences (Hartley, Varese, et al., 2014; Palmier-Claus et al., 2011).

Analyses of data included in Chapters Four, Five and Seven (N=31) indicated that wave non-response accounted for the vast majority of missing data in this study (939 cases), with invalid responses due to delayed questionnaires being the second most common cause (135 cases). There were no cases of item non-response. Thus, overall, a total of 1071 waves were missing, out of a possible 2790 (i.e. 90 per participant), indicating that 38.4% of data was missing overall (i.e. a compliance rate of 61.6%). This is similar to compliance rates demonstrated in previous ESM studies within schizophrenia populations (Hartley, Varese, et al., 2014).

3.4.1 Determining the mechanism of missingness

Missing data are not inherently problematic; however, threats to the validity of statistical inferences arise when missing data are handled inappropriately (Black et al., 2012). The processes by which missing data occur (known as missing data mechanisms) have important implications for choosing analytic techniques that provide valid statistical inferences (Black et al., 2012).

Three mechanisms of missingness have been described, with respect to the relationship between the probability of missingness and variables in the dataset (Rubin, 1976). In brief, data are considered ‘missing at random’ (MAR), when the probability of missingness depends on the observed data, but not on the values of missing data. Data are considered ‘missing not at random’ (MNAR) when missingness is a function of the unobserved values themselves, even after controlling for observed variables. Finally, an important special case of MAR, called missing completely at random (MCAR), occurs when the distribution does not depend on either the observed or unobserved data (Schafer & Graham, 2002). Missingness is considered ‘ignorable’ (i.e. the processes accounting for missingness do not need to be modelled within the substantive analysis) if the mechanism that created the missing data is either random or it is related to information that is known (i.e. MCAR or MAR; McKnight, McKnight, Sidani, & Figueredo, 2007). Whilst it is not possible to affirm statistically that data are MAR or MNAR, because the unobserved values are not available for such testing, the analyst can test the assumption of MCAR, and consider the plausibility of ignorable missingness (Black et al., 2012).

With respect to determining mechanisms of missingness in ESM data, the assumptions of MCAR can be tested by assessing model-relevant predictors of item- and wave-non-response (Granholm et al., 2008; Hartley, Varese, et al., 2014; Jelicić et al., 2009). In the present study, our exploration of the potential causes of missingness focuses on wave non-response (since there was no evidence of item non-response). Due to our small sample size (N=31), we used a series of simple linear regression analyses to assess the associations between the number of missing waves (i.e. measurement occasions) as the dependent variable, and i) within-person ESM item means; ii) sociodemographic variables (age and gender); ii) clinical variables (PSYRATS-AH total; an indicator of overall voice severity; see Section 2.2.7) as predictors. Significant effects of these predictors on wave non-response would indicate that the pattern of non-response departs significantly from the MCAR assumption (Black et al., 2012). The results of these

Table 3.2. Results of simple regression analyses with number of missing measurement occasions as dependent variable (N=31). Unstandardized betas are reported.

Predictor B SE p

Age 0.15 0.23 0.53

Gender (0 = Male, 1 = Female) 0.85 5.62 0.88 (0 = Male, 1 = ‘Other’) -4.59 11.29 0.69 Diagnosis (0 = Psychosis, 1 =

Other) -6.24 5.10 0.23

PSYRATS-AH Total 0.25 0.62 0.69

Mean ESM Voice intensity 1.41 1.40 0.32 Mean ESM Depersonalisation -0.16 1.32 0.91 Mean ESM Momentary stress -1.26 2.07 0.55 Mean ESM Negative voice content 0.90 1.93 0.65 Mean ESM Voice dominance 0.42 1.38 0.76 Mean ESM Voice uncontrollability 0.43 1.39 0.76 Mean ESM Voice intrusiveness 1.73 1.98 0.39 Mean ESM Voice resistance -1.03 1.50 0.50 Mean ESM Voice compliance 1.33 1.90 0.49 Mean ESM Voice-related distress 1.50 1.73 0.39

These results indicate that, similar to the findings of Hartley et al. (2014), there were neither large nor significant differences in the degree of wave non-response according to demographic or clinical characteristics of the sample. Furthermore, missing data were not correlated with any of the ESM variables.

Given indications of fatigue effects in past ESM studies (Broderick et al., 2003) we further explored whether these effects were partially responsible for wave non-response (i.e.

whether wave non-response was more likely as the study progressed). A multi-level logistic regression model was estimated using the MELOGIT command, with the dichotomous variable ‘missing wave’ [1 = wave missing; 0 = wave present] as dependent variable and measurement occasion (1-90) as the independent variable. This analysis demonstrated a significant increase in the likelihood of missing data over the course of the nine days (OR = 1.01, z = 6.64, p < .001, 95% CI [1.00, 1.01]), suggesting the presence of fatigue effects, and thus divergence from the MCAR assumption.

3.4.2 Applying appropriate techniques

Whilst there is no diagnostic procedure that validly differentiates between MAR and MNAR (McKnight et al., 2007), the plausibility of MAR can be increased by including

nonresponse-relevant auxiliary variables in the analytic model (i.e. variables that predict missingness; Collins, Schafer, & Kam, 2001; Little & Rubin, 2002). This increases the likelihood that covariates of missingness are controlled for (such that any remaining variance in missingness is nonsystematic), and reduces the probability of bias in parameter estimation (Black et al., 2012; Graham, 2003). Therefore, all subsequent analyses proceeded under the assumption that missing data, including data that are missing due to attrition, were ignorable, whilst increasing the plausibility of MAR by controlling for linear effects of time (i.e. measurement occasion).

When there is evidence that missing data is statistically ignorable (under the MAR assumption), statistical and empirical evidence has established that principled missing data techniques, including maximum likelihood (ML) estimation algorithms and multiple imputation (MI), provide more accurate and efficient estimates than older ad hoc approaches such as complete case analysis or single imputation (Schafer & Graham, 2002). Furthermore, these principled techniques can be applied under less restrictive missing data assumptions than ad hoc approaches; even when MAR is not precisely satisfied, such departures are rarely large enough to effectively invalidate the results (Collins et al., 2001).

Maximum likelihood is the default estimation procedure for multilevel data models (the approach typically employed with ESM data; see section 2.3.2) in many commonly used statistical packages. With these estimation algorithms, the parameters that have the greatest likelihood of producing the observed data, given the specified model, are identified. MLE does not require observations to be balanced; individuals may have differing numbers of observations spaced at different intervals, which makes MLE well suited for intensive longitudinal designs (Black et al., 2012; Schafer & Graham, 2002).

All complete and partially observed cases contribute to the MLE of model parameters, and the missing data values are treated as random variables to be averaged across (Collins et al., 2001). Given a properly specified model, ML parameter estimates from incomplete longitudinal data will be unbiased and efficient when missingness is ignorable. As such, all models within this thesis will be estimated using ML estimation methods.