Results of the case study XI

RATINGS

In addition to the author, four members of the company’s personnel made their own independent assessment. The results of the assessments are presented in Appendix D. The ratings obtained by each of the evaluators are presented in Table 21.

Table 21. Activity ratings obtained by the evaluators in Case XI.

Activity area Evaluator

AU MD MC S SR x

_

A. Organization and administration 36 30 39 29 28 32.4

B. Participation, communication, and motivation

33 39 58 30 12 34.4

C. Work environment 36 47 71 50 58 52.4

D. Follow-up 61 50 72 11 11 41.0

Total activity score 38 39 55 33 31 39.2

AU = author

MD = company’s managing director MC = company’s maintenance chief

S = supervisor

SR = employees’ safety representative

x _

= arithmetic mean

In this case, the ratings obtained were more dispersed than in Case X. The company’s maintenance chief was clearly most satisfied with the current safety activities. However, since there was no safety policy prepared, also he obtained low rating for the organization and administration of the safety activities. Company’s supervisor and the safety representative were somewhat more strict in their assessment than the author. The safety representative was most critical, as was the situation also in Case X. However, the safety representative found the work environment to be quite good. The author was most strict compared to the others when assessing the quality of the work environment.

RELIABILITY

The strength of the overall agreement between the author and the other evaluators is presented in Table 22. Weighted Kappa coefficient of agreement was used in the calculations. For the strength of agreement see Section 5.3.

Table 22. Overall agreement between the author and the company’s evaluators in Case XI.

Author by Value of weighted Kappa (κκκκ_w)

Strength of agreement

company’s managing director 0.58 Moderate

company’s maintenance chief 0.50 Moderate

supervisor 0.41 Moderate

employees’ safety representative 0.38 Fair

The strength of agreement between the author and the other evaluators was considerably better in this case than in Case X. This implies that the further development of the MISHA method led to increased reliability. In this case, the conformance was best between the author and the company’s managing director. Since the managing director was in charge of the everyday management activities, it is probable that his result also reflects the real situation quite well. VALIDITY

The method for the validity evaluation is here the same as in Case X. Accidents from the past 29 months were studied. The accident types, and their number and frequency are presented in Table 23.

Table 23. Accident types in Case XI, (n = number of accidents, f = frequency).

Accident type n f (%)

Slip of a handtool 4 26

Slippery or uneven walkway inside factory area 4 26

Overexertion injury (excluding back) 2 13

Falling or collapsing object 1 7

Pinching of hand or fingers, caused by a machine 1 7

Acute low back injury due to lifting 1 7

Slippery walkway outdoors in factory area 1 7

Falling down from an unstable platform 1 7

Total 15 100

Due to the small size of the company, also the number of accidents was small. However, it can be identified that the typical injuries were related to handtools, and the housekeeping of the plant. Besides this, manual material handling caused some problems, mainly overexertion of the back and muscular system. The lack of written safety rules, as well as inadequate supervision and daily safety communication can explain most of the housekeeping problems. The musculoskeletal overexertion can be due to the inadequate training for correct lifting techniques, or the lack of suitable lifting device. Interesting was that the author and the managing director found muscular work to be a problem while the others did not find any improvements necessary. This proves that an external auditor or a person not working in the area can identify some problems better than the persons staying most of the time in the area.

Hazard analyses or risk assessments were not carried out regularly in this company. However, three auditors did not find any problems with these activities. Only the author and the managing director identified this problem. Regular and systematic accident hazard assessments would probably have revealed that the housekeeping is at low level.

An interesting result was also the auditors’ understanding of the follow-up procedures. The author, the managing director, and the maintenance chief found

the follow-up procedures to be at acceptable level. The two other auditors found them to be at very low level. This shows that the auditor’s see the acceptable level in different ways.

8.5 Summary

When the MISHA was designed, the aim was to create a comprehensive, reliable, and valid safety audit method. Whether the MISHA method can improve the reliability of auditing was tested in two companies. The first test indicated that the inter-observer reliability was not very good. Also, the test showed that some of the method’s activities over-lap each other. Thus, the MISHA was modified and re-tested. In the second test, the agreement between the author and the company’s evaluators was fair or moderate. This reliability level can be considered sufficient when we bear in mind that the company’s evaluators were not trained for using the MISHA method.

The validity of the MISHA method was considered against the accident types and frequencies in the two companies. Some need for improvements can be found, also in the second version of MISHA. Firstly, the method should emphasize more the importance of risk assessments. Some auditors did not see the obvious weaknesses in the risk assessment activities in Case XI. In this case, these weaknesses clearly contributed to the poor housekeeping and slipping accidents. Secondly, the follow-up procedures is an activity that should be reconsidered in the MISHA method. It seems that the inexperienced auditors in Case XI did not have the knowledge on acceptable accident investigation and work ability assessment activities.

In Case XI, it was possible to see that the MISHA probably emphasizes too much the safety policy and safety documentation. In a small company, it is not always necessary to prepare and implement a separate safety policy. It might be enough if the company’s management has defined a written safety program, and follows that it is effectively implemented. This kind of a program should cover, as a minimum, the definition of responsibilities, the risk assessment procedures, and the follow-up of safety results.