Preliminary Analyses

In order to determine the leading effects of safety climate on safety outcomes, it was initially planned to utilise the personal code present on each survey to link individuals’ data from Year 1 to Year 2. This individual data would then have been

120 Predictive Validity of Safety Climate

able to be aggregated to the workgroup/facility level to allow longitudinal multilevel analyses to take place. However, the number of respondents with personal codes which could be matched to the same workgroup was extremely low, with only 36 participants having useable data. Since these 36 participants still needed their data aggregated for the multilevel analyses, the resulting analyses would be too

underpowered and the dataset would likely be unrepresentative of the population.

The lack of personal code data was due to the majority of individuals not completing this optional item in the survey. It is difficult to ascertain why

respondents did not provide a personal code. Since the item clearly stated that it was optional and for external research purposes, it is possible that respondents did not feel as compelled to provide the code. It is also possible that respondents were fearful that the code would lead to their identification. A minority of respondents went as far as to obscure or remove the workgroup code found on the front of the survey, likely due to this fear of identification, and so there is some evidence for this interpretation.

Since individual data linking was not possible, it was necessary to use aggregated data from both years sampled, whereby aggregated safety climate scores in Year 1 would be tested for association with aggregated safety outcomes in Year 2. Though this approach does not have the statistical rigour of multilevel analysis, it is far superior to using individual level data since it acknowledges the lack of

independence among individual respondents.

Table 11

Zero Order Correlations between Variables (Year 2)

Variable 1 1a 1b 1c 1d 2 2a 2b 2c 2d 3 3a 3b 3c 3d 4 5

1. Co-worker Climate

1a. Standards .85*

1b. Communication .91* .73*

1c. Risk Management .91* .72* .81*

1d. Involvement .86* .57* .68* .69*

2. Supervisor Climate .68* .57* .62* .59* .61*

2a. Standards .65* .55* .59* .55* .58* .93*

2b. Communication .64* .52* .59* .54* .60* .94* .85*

2c. Risk Management .65* .54* .60* .58* .56* .94* .83* .84*

2d. Involvement .60* .518 .54* .51* .56* .93* .79* .82* .84*

3. Management Climate .63* .47* .58* .55* .59* .58* .56* .56* .55* .53*

3a. Standards .62* .47* .59* .54* .56* .57* .55* .55* .54* .51* .93*

3b. Communication .56* .41* .52* .48* .54* .51* .49* .49* .48* .47* .95* .86*

3c. Risk Management .59* .45* .55* .51* .55* .55* .52* .52* .53* .50* .95* .84* .87*

3d. Involvement .60* .46* .54* .53* .56* .57* .54* .54* .52* .52* .94* .82* .85* .88*

4. Near Miss -.043 -.018 -.03 -.04 -.06 -.04 -.04 -.02 -.02 -.06 -.03 -.03 -.03 -.02 -.02

5. Minor Injury -.039 -.026 -.04 .01 -.07 -.07 -.05 -.04 -.06 -.09* -.04 -.05 -.03 -.04 -.04 .35*

6. Injury -.022 -.022 -.02 .00 -.07 -.04 -.02 -.03 -.04 -.05 -.01 -.02 -.02 -.01 .01 .37* .55*

Predictive Validity of Safety Climate 121

Table 12

Descriptive Statistics of all Variables (Year 2)

*minimum = 1 maximum = 6

Variable Mean* Standard Deviation Minimum Maximum Skewness Kurtosis

1. Co-worker Climate _{4.68 0.68 2.35 6.00 -0.42 0.16}

1a. Standards 4.69 0.73 2.00 6.00 -0.39 0.24

1b. Communication _{4.89 0.72 2.00 6.00 -0.74 0.66}

1c. Risk Management _{4.75 0.75 2.00 6.00 -0.42 0.04}

1d. Involvement _{4.38 0.90 1.40 6.00 -0.39 -0.12}

2. Supervisor Climate 4.71 0.88 1.61 6.00 -0.78 0.58

2a. Standards _{4.74 0.91 1.00 6.00 -0.79 0.75}

2b. Communication _{4.64 0.96 1.20 6.00 -0.71 0.43}

2c. Risk Management 4.74 0.89 1.60 6.00 -0.76 0.56

2d. Involvement 4.73 1.01 1.00 6.00 -0.92 0.67

3. Management Climate _{4.56 0.89 1.45 6.00 -0.59 0.06}

3a. Standards _{4.67 0.89 1.17 6.00 -0.70 0.34}

3b. Communication 4.52 0.98 1.40 6.00 -0.63 0.00

3c. Risk Management _{4.61 0.91 1.17 6.00 -0.56 0.10}

3d. Involvement _{4.42 0.99 1.00 6.00 -0.58 0.15}

4. Near Miss _{0.55 1.82 0.00 30.00 9.05 122.49}

5. Minor Injury 0.67 1.86 0.00 31.00 8.57 117.86

6. Injury _{0.11 0.45 0.00 6.00 6.49 60.93}

122Predictive Validity of Safety Climate

Predictive Validity of Safety Climate 123 In order to be included in the analysis, each workgroup needed to contain at least 3 members. Data from participants with no workgroup code were also excluded. This resulted in 91 participants nested in 20 workgroups being removed in Year 1, and 216 participants nested in 31 workgroups being removed in Year 2. Overall, there were 95 workgroups in Year 1, and 75 workgroups in Year 2. Before data

aggregation took place, ICC’s were calculated to assess the homogeneity of climate perceptions. In Year 1, the ICC for the co-worker scale was 0.125 (12.5%), 0.136 (13.6%) for the supervisor scale, and 0.088 (8.8%) for the manager scale. In Year 2, the ICC was 11.09% for the co-worker scale, 0.109 (10.9%) for the supervisor scale, and 0.044 (4.4%) for the manager scale. Though the scores for the co-worker and supervisor scales indicated that there was sufficient homogeneity of perceptions, the manager scale ICC was lower, particularly in the second year. However, such a small ICC is not uncommon in the literature (e.g. Neal & Griffin, 2006), with an ICC as small as 0.01% being shown to increase the Type 1 error rate as high as .17%

(Barcikowski, 1981). Hence, it was decided that an ICC of this size was not sufficient grounds against data aggregation. The data does suggest however that facility membership exerts a smaller effect on individual perceptions of management commitment to safety in comparison to the medium effect size of group membership on supervisor/co-worker perceptions.

A one-way analysis of variance using unaggregated data was performed to determine whether there was sufficient between group variance to justify

aggregation. A respondent’s workgroup was the independent variable when testing the co-worker and supervisor scales, while facility was the independent variable when testing the manager Scale. The dependent variable was the total safety climate score. Results indicated that all safety climate scales exhibited significant between group variance in Year 1, Co-worker Scale: F (95, 751) = 2.03, p < .001 (ηp2

= .227), Supervisor Scale: F (95, 738) = 2.18, p < .001 (ηp2

= .244), Manager Scale: F (10, 736) = 7.83, p < .001 (ηp2

= .097). The results were similarly positive in Year 2, Co-worker Scale: F (75, 602) = 1.956, p < .001 (ηp2

= .218), Supervisor Scale: F (75, 598) = 1.918, p < .001 (ηp2

= .216), Manager Scale: F (10, 585) = 4.418, p < .001 (ηp2

=

124 Predictive Validity of Safety Climate

.071).These results coupled with the ICC’s suggest a need to consider the safety climate perceptions as “shared”, and therefore requiring aggregation.

Since each workgroup and facility needed data in both years to be included in the analysis of predictive validity, those workgroups and facilities with data in only a single year were also removed from the dataset. This process resulted in 49 workgroups being included for the analysis of co-worker and supervisor safety climate predictive validity, and 9 facilities being involved in the analysis of manager safety climate predictive validity. Due to the rarity of injuries in the organisation, Poisson regression was performed using SPSS 18.0. Group size was controlled for by utilising an offset, which was the log of the workgroup’s size.

Before any analysis took place, histograms were inspected to ensure safety outcomes had an approximately Poisson shaped distribution. While safety

outcomes at the group level had an approximately Poisson shaped distribution (see Figures 14 and 15 in Appendix D), at the facility level the small sample size made interpretation difficult, with inspection of histograms, skewness and kurtosis statistics in fact suggesting that self-reported safety outcomes adhered to an approximately normal distribution (see Figures 16 and 17 in Appendix D). The Shapiro-Wilk test for both near misses and injuries at the facility level were non-significant, suggesting normality. However, given the potential inaccuracy of normality tests with a small sample size, coupled with the difficulty in determining the distribution of the data, it was cautiously decided to utilise a non-parametric test devoid of distributional assumptions for facility level analyses. Therefore, the relationship between manager safety climate in Year 1 and safety outcomes in Year 2 was assessed using Spearman’s Rho correlation. Since it is not possible to have an offset controlling for group size with a correlation analysis, mean scores were used rather than total injury counts.

Inspection of possible outliers took place for the group level data, with examination of standardised residuals and influence statistics not detecting any outliers. A final test for the group-level analyses was determining whether equidispersion was present. Not accounting for inequality between the variance and mean can lead to

Predictive Validity of Safety Climate 125 an increased risk of Type 1 errors (Coxe, West, & Aiken, 2009). Determining the presence of equidispersion was achieved by inspecting the Pearson Chi-Square statistic divided by degrees of freedom. The statistic indicated that overdispersion was present and standard Poisson regression may not be appropriate. Hence, a number of different models in the Poisson family were tested and compared to determine best model fit. The models tested included the standard Poisson model, the overdispersed Poisson model and the negative binomial model. Nested model comparisons were conducted using the likelihood ratio test, while the Akaike’s Information Criterion was used to compare the overdispersed Poisson model and the negative binomial model given they are not nested and thus cannot be

compared using the likelihood ratio test. The negative binomial and overdispersed Poisson models provided more conservative estimates of standard error compared to standard Poisson regression, and provided a better fit to the data given they brought dispersion scores closer to one and were a better fit to the data as shown by the likelihood ratio test. Overall, the overdispersed Poisson model was the more accurate and parsimonious representation of the data given it accounted for the substantial overdispersion and achieved superior fit indices compared to the negative binomial model.