Missing Data Analysis

In longitudinal studies, researchers usually have to deal with unplanned missing values, “in fact it is difficult to imagine a longitudinal study without at least some un- planned missing data” (Collins, 2006, p. 521). According to Schafer and Graham (2002), there are two main reasons for missing data: individuals that leave the questioning for one or more waves of questioning (unit nonresponse), which is a specific missing data problem within longitudinal data; and items that were not answered (item nonresponse).

Types of missing data. To handle missing data, it is necessary to discover whether

missingness is related to the data or not. In 1976, Rubin created a typology that described three different missing data classifications. Schafer and Graham (2002) described these three types as follows: missing completely at random (MCAR) “means that the probability that Y is missing for a participant does not depend on his or her own values of X and Y” (p. 151); missing at random (MAR) indicates that missing data may be related to X but not to Y; missing not at random (MNAR) shows that missing data are related to Y. Most up-to- date missing data techniques (e.g., maximum likelihood estimation, multiple imputation) require MCAR or MAR to analyze data including missing values (e.g., Collins, Schafer, & Kam, 2001; Feng, Cong, & Silverstein, 2012; Newman, 2003; Schafer & Graham, 2002). In order to test whether data are MCAR, Little’s MCAR test can be used (Little, 1988) and a non-significant outcome indicates that data are MCAR. There is no further option to test whether data are MAR or MNAR. In case MCAR is not reached, Graham (2009) stated

the best way to think of all missing data is as a continuum between MAR and MNAR. Because all missingness is MNAR (i.e., not purely MAR), then whether it is MNAR or not should never be the issue (…), we should answer the question of whether the violation is big enough to matter to any practical extent. (p. 567)

Thus, it has to be ruled out that data are overall MNAR, meaning that missingness is related to Y (Schafer & Graham, 2002). Collins et al. (2001) also declared that when “the cor- relation between the cause of missingness and the variable subject to missingness was .4,

37 _{A group of researchers, namely Ulrich Wagner, Mario Gollwitzer, Stefan Thörner, Lisa Gutenbrunner, and} the author of this thesis, was involved in identifying the solution (i.e., coding of the level-2 control variables) for this fixed effects approach including only a small number of level-2-units.

omitting the cause of missingness had a negligible effect” (p. 333). Under these circum- stances, even a combined assumption of MAR and MNAR can hold as prerequisite for the respective missing data analysis.

Missing data techniques. After exploring the type of missing data, a technique to

handle missing data has to be chosen. In the past, missing values were handled using ad hoc methods which tried to fix the data ahead of the calculations (Peugh & Enders, 2004). These ad hoc measures usually applied listwise as well as pairwise deletion or the missing values were substituted with the variable mean. However, as Wilkinson and the Task Force on Statistical Inference of the American Psychological Association stated already in 1999 “the two popular methods for dealing with missing data that are found in basic statistics packages–listwise and pairwise deletion of missing values–are among the worst methods available for practical applications“ (p. 598). Accordingly, Schafer and Graham (2002) recommended using all available data to recover missing information because especially in the case of longitudinal data, repeated measures of individuals are probably related.

Currently two other missing-data techniques are regarded as state of the art: maximum likelihood estimation (ML; sometimes also referred to as full-information-maximum- likelihood [FIML]) and multiple imputation (MIM38_{; Peugh & Enders, 2004; Schafer &}

Graham, 2002). Although the two missing data techniques ML and MIM are recommended by several authors (e.g., Baraldi & Enders, 2010; Brown, 2006; Feng et al., 2012; Peugh & Enders, 2004; Schafer & Graham, 2002), they are barely used in case of longitudinal exper- imental data in contact research. Differences between the two recommended missing data techniques are considered as small (Peugh & Enders, 2004) and according to Enders (2013), the choice for either one of the missing data techniques depends on personal

preferences and the utilized software. Nevertheless, there are some differences between the two missing data techniques, which are outlined in Table 3 (Graham, 2003; Peugh & Enders, 2004; Schafer & Graham, 2002).

38 _{Usually the abbreviation of multiple imputation is MI; however, due to the fact that measurement} invariance is also shortened to MI in research literature and measurement invariance was already introduced in this text, we abbreviate multiple imputation with the letters MIM.

Table 3: Characteristics of Maximum Likelihood Estimation (ML) and Multiple Imputation Analysis (MIM)

Issues Maximum Likelihood _Estimation Multiple Imputation

Estimator Maximum likelihood Bayesian

Data Basis Data is estimated directly in the model, no complete data sets exist

Data is used to compute several complete data sets

Number of Steps One step: analysis Two steps: imputation and _analysis Auxiliary variables Relatively difficult to implement _{in analysis} Relatively easy to implement in _{imputation process} Standard errors In general slightly smaller In general slightly larger

Within this thesis, we decided to calculate missing data using ML when calculating latent variables via the computer software Mplus (Version 7.2; Muthén & Muthén, 1998- 2014). In Mplus, “standard errors for the parameter estimates are computed using the ob- served information matrix” (Muthén & Muthén, 1998-2012, p. 387; see also Kenward & Molenberghs, 1998). We used MIM when data was analyzed with manifest variables via IBM SPSS Statistics (Version 21.0; IBM Corp., 2012). In general, in case MIM is con- ducted, predominantly between m = 5 to m = 10 new imputations are generated (Collins et al., 2001). Within this thesis, we generated m = 10 data sets.39