Cross-Validation Analyses

While there is confidence in the reported pattern of relationships between the different safety climate scales, the extremely low associations between safety

0.328

170 Individual and Aggregate Comparisons

climate and safety outcomes at the individual level require further cross-validation, particularly given that the findings contrast the previous cross-sectional criterion validity analyses. Hence, individual level analyses were performed using MlwiN v2.17 (Rasbash et al., 2009), with self-reported near misses as the dependent variable, and all three safety climates scales as independent variables. Like the path analyses, this analysis will determine which safety climate scales are significantly associated with self-reported safety outcomes, even after controlling for the effects of one another. As per the cross-sectional criterion validity analyses, logistic

regression with a logit link function was utilised, with data from the first year.

Models were first computed using first order MQL approximation, followed by second-order PQL approximation. Though corrections due to the non-independence of data were not possible in the path analyses, in these analyses random intercepts were added to the workgroup and facility level. The presence of outliers was ascertained by the inspection of standardised residuals and influence diagnostics, with one outlier removed at the facility level, and one outlier removed at the group level. Since maximum likelihood estimation techniques were used, statistical significance was ascertained via the Wald test. The results of these analyses supported the original cross-sectional criterion validity analyses as the co-worker scale was significantly associated with self-reported near misses, with a regression coefficient of -0.494 (0.238), χ² (1) = 4.307, p < 0.05. In contrast, the supervisor scale failed to reach significance, with a coefficient of -0.240 (0.181), χ² (1) = 1.758, p >

0.05. Similarly, the manager scale was not significantly associated with self-reported safety outcomes and had a mildly positive relationship, with a regression coefficient of 0.222 (0.192), χ² (1) = 1.338, p > 0.05. Therefore, the analyses suggest that the previous path analyses may not be correct in terms of the relationship that safety climate has with self-reported safety outcomes. While the path analyses suggested that none of the safety climate scales at the individual level were associated with safety outcomes, these results suggest that perceptions of co-workers are the strongest individual influence on safety outcomes – even after controlling for higher-level variance.

Individual and Aggregate Comparisons 171 9.3.4 Exploratory Multilevel Mediation Analyses

The analyses in the thesis so far have found that perceptions of co-worker

commitment to safety at the group level are not predictive of safety outcomes. This was clearly shown in the cross-sectional criterion validity analyses, the predictive validity analyses, and the path analysis comparisons. However, at the individual level, a different pattern of results emerge, with co-worker safety climate being a stronger indicator of safety outcomes compared to supervisor and manager safety climate. The significant association between co-worker safety climate and safety outcomes remained even after controlling for group and facility level variance, indicating that perceptions of co-workers at the individual level have a role in predicting safety outcomes that is separate from aggregated supervisor and manager safety climate. Given that this thesis has supported the notion that supervisor safety climate mediates the relationship between manager safety climate and safety outcomes, the strong predictive power of individual level co-worker safety climate suggests that it may mediate the relationship between supervisor safety climate and safety outcomes.

In order to test this tentative hypothesis, the Baron and Kenny (1986) 3-step method for testing mediation was used. Hence, supervisor safety climate should predict safety outcomes, supervisor safety climate should predict co-worker safety climate, and when co-worker safety climate is added to the supervisor safety climate  safety outcomes regression equation, the relationship between supervisor safety climate and safety outcomes should diminish or become non-existent.

These analyses were performed using MLwiN v2.2 (Rasbash et. al., 2009). In the analysis, supervisor safety climate was assessed at the group level, while co-worker safety climate and safety outcomes were assessed at the individual level. Safety outcomes were operationalized as self-reported near misses, assessed as a dichotomous variable (no near misses, one or more near misses). Hence, logistic regression was used in the associations between safety climate and safety outcomes, with a logit link function selected and random intercepts included.

172 Individual and Aggregate Comparisons

Models were first estimated with 1st order MQL approximation, followed by 2nd order PQL approximation in order to achieve the most accurate estimates. Since the association between supervisor safety climate and co-worker safety climate does not involve dichotomous outcome variables, standard multilevel regression techniques were used. Given that normality is assumed in such analyses, a square root transformation with a reflection was carried out on both supervisor and co-worker safety climate to correct the slight negative skew. Standardised residuals and influence diagnostics were inspected to determine the presence of any extreme values, with one outlier being removed at the group level for the safety climate  safety outcomes analyses, and 6 outliers being removed at the group level for the supervisor safety climate  co-worker safety climate analysis. Significance was calculated via the Wald test, which employs a chi-square distribution with one degree of freedom.

The results supported a mediation process taking place. Firstly, supervisor safety climate predicted self-reported near misses, χ² (1) = 5.787, p < 0.05. Secondly, supervisor safety climate predicted individual level co-worker safety climate, χ² = 7.530, p < 0.05. Lastly, when both supervisor and co-worker safety climate were included in the prediction of self-reported near misses, supervisor safety climate fell slightly short of significance, χ² (1) = 3.161 , p > 0.05, while co-worker safety climate remained significant, χ² (1) = 5.664, p < 0.05. Hence, the relationship between group level supervisor safety climate and safety outcomes was mediated by individual level co-worker safety climate.

9.4 Discussion

This chapter uncovered a number of interesting and important findings, with the overarching finding being that safety climate at the individual level (i.e.

perceived/psychological safety climate) displays a different pattern of results compared to aggregated safety climate. While there were similarities between individual and aggregate analyses in terms of the lack of relationship with self-reported injuries, the pattern of relationships each scale had with safety outcomes

Individual and Aggregate Comparisons 173 differed. Co-worker safety climate had the weakest relationship with safety

outcomes when aggregated to the group level, though when it was analysed at the individual level it displayed the strongest relationship, even after controlling for group and facility level variance. The analyses further demonstrated the potential problems faced by researchers who operationalize safety climate at the individual level and who do not take into account the dependency of perceptions at higher levels. These analyses were the first to compare different analysis methods on the same dataset in the safety climate literature, and found that regression coefficients tended to be smaller and standard errors tended to be larger in individual level analyses which adequately accounted for higher level variance, compared to naïve analyses in which no such correction was made. Possibly the biggest contribution to the literature is the finding that the relationship between supervisor safety climate and safety outcomes is mediated by individual level co-worker safety climate. This indicates that in the prediction of individual safety outcomes, it is an employee’s perception of the commitment to safety of those around them (i.e. a co-worker psychological/perceived safety climate) which is the most proximal. This finding therefore suggests an extension to Zohar’s Multilevel Model of Safety Climate to include individual perceptions of co-workers, and additionally sheds light on the processes in which climate perceptions may affect behaviour and ultimately safety outcomes.

Researchers in the safety climate literature have commonly operationalized safety climate at the individual level with little regard to the possible consequences, both in terms of the effects on the analytical findings and on the conceptual

development of the construct. While the arbitrary operationalization of safety climate clearly fosters ambiguity and impedes conceptual development, it was not known before this analysis how results may be affected by the level of analysis and analytical technique. Cross-sectional associations at the individual level found that perceptions of co-workers had stronger associations with self-reported near misses compared to supervisor or manager safety climate. These findings contrasted with group level analyses, where the opposite pattern of results emerged. Before this analysis, the only comparison between individual and aggregate level analyses was

174 Individual and Aggregate Comparisons

in the form of meta-analysis, with Christian and colleagues (2008) finding that individual level analyses tended to have weaker associations with safety outcomes.

Though authors such as Zohar argued for the aggregation of safety climate data on conceptual and methodological grounds, there was little direct evidence to suggest that findings at the individual level were not generalizable to the wider safety climate literature. While the analyses upheld the findings of Christian, with regression coefficients weaker at the individual level, the dichotomy in findings based on level of aggregation suggests that level of analysis does affect the manner in which certain safety climate scales/subscales interact with safety outcomes. This therefore calls into question the generalizability of findings at the individual level.

While co-worker commitment to safety is a relatively rare dimension in the safety climate literature, researchers that include the dimension (e.g. Seo et al., 2004;

Turner et al., 2010) may find that their findings will not generalise when scores are aggregated to the group level. In addition, while supervisor safety climate at the group level will have a direct association with group level safety outcomes, at the individual level this relationship is mediation by co-worker safety climate.

Therefore, the level of safety outcomes needs to be taken into account in addition to the level of safety climate when ascertaining the generalizability of findings.

This analysis was the first to operationalize climate at the individual level and control for higher level variance, with comparisons between naïve and corrected analyses producing compelling results. Compared to the standard ‘naïve’ analyses in which no correction is made, the individual multilevel analyses tended to be more conservative, with lower regression coefficients and higher standard error.

Hence, the results reflect that of Twisk (2006) who achieved similar results in his comparisons in the epidemiological context. While the correction did not change the overall pattern of results for the co-worker and supervisor scales, the regression coefficients for the manager scale declined significantly. The standard error

remained relatively unchanged, possibly because the small sample size at the facility level (N = 11) was insufficient for the detection of the relatively small differences in standard error as per the group level analyses. However, the correction to regression coefficients alone was enough for previously significant

Individual and Aggregate Comparisons 175 associations between subscales and self-reported near misses to become non-significant. Given that manager safety climate is the most commonly used dimension in the safety climate literature, this has concerning implications for research at the individual level which does not correct for the dependency in climate perceptions. The results suggest that research utilising naïve analyses (e.g.

Seo et al., 2004; Turner et al., 2010; Vinodkumar & Bhasi, 2008; Wills, Watson, &

Biggs, 2006) may be overestimating the relationship that safety climate has with safety outcomes. In the case of manager commitment to safety, the results suggest that once facility/organisational variance is accounted for, individual level

perceptions of management commitment to safety become far less important in the prediction of safety outcomes. In other words, once differences between facilities are accounted for, differences between individuals in terms of their perception of manager safety climate become redundant. Additionally, in datasets where there is high within-group homogeneity and high between-group

heterogeneity in climate perceptions, the effect of not accounting for the dependency in perceptions would be heightened. The intraclass correlation coefficients in the current analysis, particularly at the facility level, were relatively low, and therefore the potential for confounded results when corrective steps are not taken by the researcher is made clear by these findings.

The finding that the relationship between supervisor safety climate and individual outcomes is mediated by individual level co-worker safety climate is an important and unexpected result that sheds new light on the processes in which climate perceptions affect safety outcomes. Before this analysis, the dominant perspective was that individual employees reach a consensus on the priority of safety through their shared experiences and interactions with the supervisor (i.e. observing reward and punishment), and these perceptions inform desired behaviours, which in turn affect individual behaviour and the subsequent possibility of injury (see Zohar &

Luria, 2005). The current analysis has demonstrated that there is an additional link between supervisor safety climate and individual safety outcomes. It appears that based on perceptions of commitment to safety at higher levels of the organisation, individuals develop their own perception of safety’s priority among fellow frontline

176 Individual and Aggregate Comparisons

employees in their area, and it is these more proximal perceptions that most strongly predicts safety outcomes for that individual. In other words, employees synthesise higher level climate perceptions, which when combined with their own characteristics and experiences (i.e. individual variability), result in the formation of a perception of “the way things are done around here” among fellow frontline employees, which informs behaviour and therefore safety outcomes.

Though managers and supervisors are predictive of an individual’s co-worker safety climate, the significant associations with safety outcomes after controlling for group and facility level variance demonstrates that it is a distinctively individual level phenomena. Hence, while there are similarities between individuals when it comes to their perceptions of co-workers (as shown by the non-zero ICC’s), the results demonstrate that individual variability remains important in the prediction of safety outcomes and that a psychological/perceived safety climate construct can coexist with the more theoretically and empirically supported aggregated safety climate construct. This is an important finding, given that research generally describes safety climate as consisting of shared perceptions (Cooper & Phillips, 2004; Oliver et al., 2006; Zohar, 2000; Zohar & Luria, 2005), thereby requiring aggregation and multilevel analysis to avoid confounding results. While safety climate perceptions still need to be aggregated to the appropriate level, an individually operationalized psychological/perceived safety climate can be seen to exist as a distinct, important, and non-confounded predictor of safety outcomes, provided that adequate

multilevel corrections are implemented.

The fact that co-worker perceptions were more important than supervisor and manager perceptions at the individual level makes it easier to differentiate among psychological/perceived safety climate and safety climate and therefore reduce the probability of future conceptual ambiguity. While perceptions of supervisor and manager commitment to safety remain the domain of safety climate and are always aggregated to their respective level, these individually relevant perceptions of co-workers can remain at the individual level in the assessment of

psychological/perceived safety climate. This distinction can form the basis for the development of conceptual models which explain the role psychological/perceived

Individual and Aggregate Comparisons 177 safety climate has in the prediction of safety outcomes and its relationship with other variables such as safety climate.

Overall, the current analysis has uncovered a number of findings which should promote the conceptual development of safety climate. Firstly, it was demonstrated that individually operationalized safety climate displays a different pattern of results compared to aggregated safety climate, with perceptions of co-workers having a much stronger relationship with safety outcomes at the individual level. These analyses also demonstrated the potential inaccuracy of research findings which operationalize safety climate at the individual level without correcting for the

dependency in observations, with non-corrected analyses underestimating standard error and overestimating regression coefficients. Lastly, it was found that the

relationship between group-level supervisor safety climate and individual safety outcomes was mediated by individual level co-worker safety climate. These findings suggest an extension to Zohar’s Multilevel Model of Safety Climate, offer further insight into the processes in which safety climate affects individuals, and provide a platform from which safety climate and psychological/perceived safety climate research can proceed in a conceptually and methodologically distinct manner

178 General Discussion

Chapter 10