Data collection and data management

The sample size used for the DEMoBinc project was based on the sample needed to have sufficient power to identify QuIRC domains that independently predicted better service user experiences of care. Multilevel models were used to analyse the associations between service user ratings (level 1, dependent variables) and facility QuIRC domain scores (level 2, independent variables). A minimum of 203 facilities were required to test for 10 predictors of a medium effect size (R2 = 0.35) with 90% power at a 1.25% significance level (Dunlap et al 2004).

Data were collected between January and November 2009 across the ten participating countries. Researchers in all countries entered data into separate but common databases - one for QuIRC assessment data and one for service user data. Double data entry on 10% of

158 the data from each country was completed by a second researcher. Double entry of the entire dataset was required if initial double entry exceeded an error rate of 5%. Completed databases were then merged into facility (QuIRC) and service user master databases and cleaned prior to analysis. Any missing QuIRC data was assumed to be missing at random. Data on national mental health expenditure, levels of deinstitutionalisation, stigma and the number of years since the introduction of mental health policies were added to the master databases. Data were analysed using STATA release 12.

6.3.1 Data analysis

Multilevel modelling

Multilevel models were used to examine the relationships between dependent and independent variables as they allow for effects of data clustering to be taken into consideration when examining the variation between outcomes (Luke 2004). The multilevel equation, which is an extension of a regression equation, evaluates the relationship between a dependent variable and one or more independent variables. The simplest regression equation is:

Y = β0 + β1X1+e1

This equation maps a line of best fit through the approximate centre of a plot of values. The independent and dependent variables are represented by X1 and Y, respectively. The intercept is represented by β0. The slope of the line is represented by β1and e1 denotes the variance between the mapped value and the real data. Ordinary regression models are limited, however, in that they are unable to detect differences which are attributed to the clustering of data. However, in health research much of the data collected are inherently

159 clustered. For example, patient outcomes may be related to the hospital in which they receive treatment. In order to account for potential differences across clusters, multilevel equations include a second variable to explain additional variance. The basic multilevel equation is nearly identical to the simple regression equation:

Y = β0 + β1Xij + u0j + e0ij

Again the dependent variable is represented by Y, β0 is the intercept and β1 denotes the line’s slope. Xij provides the value of the independent variable (e.g. medication dosage) for patient i in hospital j. The two variables representing variance denote the departure of the average outcome of patients in hospital j from the average of all hospitals (u0j) and the departure of the outcome of patient i in hospital j from the average outcome of all patients (e0ij).

In this study, clustering may occur at three levels: the service user, facility and country. It is likely that observations are interdependent across levels (i.e. facilities may be more similar to other facilities within the same country than those in other countries and service user ratings may be more similar within a facility than across facilities), making multilevel modelling an appropriate method for analysis. Country- and facility-level variables were modelled using fixed effects due to the low number of these highest level groups, countries, and the fact that the countries and facilities were not randomly chosen.

160

Figure 6.2 Multilevel models of the association between mental health expenditure and quality of longer term care

Investigating the association between mental health care

expenditure and quality of care

In order to investigate the association between quality of mental health care and expenditure (hypothesis 1), four two-level models were developed (see Figure 6.2). In Model A, the seven QuIRC domains (human rights; living environment; recovery-based practice; self-management & autonomy; social interface; therapeutic environment; treatments and interventions) were included separately as dependent variables at the facility level (level 1). National mental health expenditure, measured as the percentage of the health budget spent on mental health (percentage expenditure) or per capita total mental health expenditure (per capita expenditure), was included as an independent variable at the

1

Quality Indicator for Rehabilitative Care

2

161 country level (level 2). In Model B, the independent variables facility type (hospital = 0; community = 1) and FTE staff to service user ratio (below sample mean = 0; above sample mean = 1) were added to the model as level 1 fixed effects. In Model C, the degree of national stigma associated with schizophrenia and the number of years since the introduction of mental health policies were added as fixed effect, independent variables to level 2 in Model A. In Model D, both facility and country independent variables were added to Model A as fixed effects. All models were subjected to a visual inspection to ensure that the variables were normally distributed and the variance of the error terms (e0ij) was constant (homoscedasticity). As no mental health budget data were available for Greece and stigma scores were unavailable for the Czech Republic, both countries were excluded from all models.

The results of each set of models were then evaluated to determine the model of best fit. It is widely accepted that the best model “provides an adequate account of the data while using a minimum number of parameters [independent variables]” (Wagenmakers & Farrell 2004, p. 192). Models of best fit for each dependent variable were determined by its Akaike’s information criterion (AIC; Akaike 1987) value. The AIC is a popular method of determining which of the available models best approximates the true model. This method assumes the existence of a true model which is not included in the available models. The AIC value represents the difference between the most parsimonious model and the model which has been developed. The greater the difference between the models, the worse the fit. Therefore, the model with the lowest AIC value is deemed the best fitting model.

162

Figure 6.3 Multilevel models of the association between deinstitutionalisation and quality of longer term care

Investigating the association between deinstitutionalisation and

quality of care

In order to examine the association between quality of mental health care, and deinstitutionalisation (hypothesis 2), four two-level models were developed (see Figure 6.3). In Model A, QuIRC domain scores (human rights; living environment; recovery-based practice; self-management & autonomy; social interface; therapeutic environment; treatments and interventions) were included separately as dependent variables at the facility level (level 1). Deinstitutionalisation score was included as an independent variable at the country level (level 2). In Model B, the independent variables facility type, FTE staff to service user ratio and presence of a maximum length of stay (no = 0; yes = 1) were added to the model as level 1 fixed effects. In Model C, the degree of national stigma and the

1

Quality Indicator for Rehabilitative Care

2

163 number of years since the introduction of mental health policies were added as fixed effect, independent variables to level 2 in Model A. In Model D, both facility and country independent variables were added to Model A as fixed effects. As stigma data were unavailable for the Czech Republic, it was excluded from all models. Models of best fit were selected using AIC values.

Investigating the association between mental health care

expenditure and service user ratings

In order to examine the association between mental health expenditure and service user ratings (hypothesis 3), four, three-level models were developed (see Figure 6.4). In Model E, service user ratings of autonomy (RCS), experience of care (YTC), life satisfaction (MANSA) and therapeutic milieu (GMI) were included as dependent variables at the service user level (level 1). Mental health expenditure was included as a fixed effect at the country level (level 3). In Model F, the independent variables facility type and staff-to- service user ratio were added to the model as facility level (level 2) fixed effects. In Model G, the degree of national stigma and the number of years since the introduction of mental health policies were added to Model E as level 3 fixed effect, independent variables. In Model H, both facility and country independent variables were added to Model E as fixed effects. Data from Greece and the Czech Republic were excluded from all models due to missing data. Models of best fit were determined using AIC values.

164

Figure 6.4 Multilevel models of the association between mental health expenditure and service user ratings of care

Investigating the association between deinstitutionalisation and

service user ratings

Four, three-level models were developed to examine the association between deinstitutionalisation and service user ratings (hypothesis 4; see Figure 6.5). In Model E, the service user ratings of autonomy (RCS), experience of care (YTC), life satisfaction (MANSA) and therapeutic milieu (GMI) were included as dependent variables at the service user level (level 1). Deinstitutionalisation score was included as a fixed effect at the country level (level 3). In Model F, the independent variables facility type, FTE staff to service user ratio and presence of a maximum length of stay were added to the model as

1

165

Figure 6.5 Multilevel models of the association between deinstitutionalisation and service user ratings of care

facility level (level 2) fixed effects. In Model G, the degree of national stigma and years since development of mental health policy were added to Model E as level 3 fixed effect, independent variables. In Model H, both facility and country independent variables were added to Model F as fixed effects. As stigma scores were unavailable for the Czech Republic, it was excluded from all models. Models of best fit were then selected using AIC values.

l

1

166

Chapter 7

Results