Elements of the variable-base analysis include frequency data, factor analysis (confirmatory analysis and principle components analysis), scale reliability (Cronbach α), scale scores, chi-squared and Fisher’s exact test. Software used for variable-based analyses was SPSS (versions 20 and 21) (IBM Corp, released 2011, released 2012) and Monte Carlo (Watson, 2000) .Each of the specific analyses will now be discussed further.
Choice of chi-squared and Fisher’s exact test were based upon the data being categorical in nature (Allen & Bennett, 2012; Howell, 2007). Although considered a scale, the factor scores for safety climate and teamwork were determined to be
positive or not positive and therefore also categorical data. Similarly, in determining views of reporting and disclosure as Always and Not Always again resulted in
categorical data.
Factor analysis is considered an important element of safety climate questionnaires (Flin et al., 2006). It was determined that responses to the SAQ items were not normally distributed (refer to Appendix 6) however it is considered that principal
107 components analysis is robust enough to be used with such data (Pallant, 2013; Tabachnick & Fidell, 2007).
Use of Cronbach’s α is recommended and this was used to check the internal consistency of the scales from factors determined by a previous study undertaken using the same questionnaire (Hutchinson et al., 2006). Due to these factor scales being determined unreliable, an exploratory principal components analysis was then undertaken. More details of this process will be presented later in this and the following chapter (Section 4.3.3 and Section 5.6)
More detail of the approach taken with regard to frequency data will now be presented including handling of missing data and assessment of sample
representativeness. The process of the exploratory principal components analysis is then described, followed by details of the inferential statistical used.
Frequency data
The initial analysis included examination of the frequency data. Missing data were considered first, with questionnaire responses which had no data being excluded, as were those with post-codes outside the geographical area of interest. Responses with missing data relating to postcode were included in the analysis. It had been disclosed to participants that the student researcher had previously been employed by one of the unions assisting recruitment. Respondents may therefore have known the student researcher, leading to reluctance to divulge postcode information in case they could be personally identified. For example, some worksites were the sole site for health care in a particular postcode area with only one or two management staff.
It was therefore considered that whilst there was a risk that responses with no postcode information could be from outside the geographical area of interest, there was also a possibility they may not have been. It was also considered that
respondents were more likely to be concerned regarding identification than they were about not providing postcode data as their worksite was outside the ASGC-RA
108 area of interest. If a respondent who was outside the area of interest had an
awareness of the ASGC-RA classification, then it would be possible for them to provide an appropriate postcode rather than not provide any information.
The decision to include these responses was incorporated with the use of sensitivity tests to determine the impact of this decision relating to statistical significance from any results. Where inferential statistics results with borderline statistical
significance were obtained the tests were repeated with these responses omitted. The results tables of these tests are located in Appendix 7. A low number of
responses also resulted in the combining of some categories of responses reported in the results.
Data were also missing for other items of the questionnaire. The approach to analysis determined how the missing data were treated with an emphasis on including the maximum possible respondents in any analysis. The nature of PCA determined that the missing data from the SAQ items were excluded pairwise (Pallant, 2013). For other items missing data listwise exclusion was applied.
Only those cases with complete data (that is responses to all the SAQ items and the views of reporting and disclosure) were included in the case-based analysis. This ensured that no assumptions were made regarding missing data. Although this reduced the number of cases included the calculation of a factor score from SAQ data relied on complete data so some reduction was already required.
The representativeness of the sample obtained was also considered. Where possible the frequency of demographic responses were compared to data sources such as the Nursing and Midwifery Labour Force 2012 data(Australian Institute of Health and Welfare, 2013).
Frequency data for demographic information and views of reporting and disclosure is presented in graph format with the frequency of responses for the SAQ items presented in tables. For some analyses, responses to demographic categories were
109 combined (Howell, 2007; Pallant, 2013) as this enhanced the analysis as well as reduced the possibility of respondents being identified.
Frequency data were also produced regarding positive scores for the factors of teamwork and safety climate. However, these could not be calculated until the number of factors and items were identified through analysis of the SAQ including principal components analysis. This will now be outlined.
Analysis of conditions data
As mentioned earlier in this chapter (Section 4.2.1) the first section of the survey contained the SAQ items. A recent review has outlined several ways in which a questionnaire such as the SAQ may be assessed in relation to reliability and validity.
Reliability is defined as the consistency to which a data collection instrument measures something across a variety of conditions whereas validity refers to whether or not the questionnaire measures what it is supposed to measure (Valentine et al., 2015).
The review highlights four areas that address these elements. These areas are
internal consistency, interrater agreementand reliability, structural validity and
content validity.
The previous use of the SAQ in the UK addressed three of these four areas with only interrater agreement and reliability not reported by the study (Valentine et al., 2015). The work of Sexton and colleagues is noted to have addressed these same three areas (Valentine et al., 2015).
Interrater agreement was not assessed in this research. This process is often not reported for this type of questionnaire as many of the analyses rely upon a test- retest approach which was not part of the research design for this study (Valentine et al., 2015). In addition, neither of the questionnaires informing this research had reported interrater agreemen.t leaving no capacity to benchmark this (Hutchinson et al., 2006; Sexton et al., 2006; Valentine et al., 2015).
110
Internal consistency refers to the degree to which items in a scale correlate and is therefore a form of determining reliability (Valentine et al., 2015). Use of
Cronbach’s α is one approach to assessing internal consistency. For this research the internal consistency of the SAQ items was checked with the use of Crohnbach’s α (Pallant, 2013) (Valentine et al., 2015). A value of 0.70 from this test is considered to indicate reliability (Matsunaga, 2010).
Structural validity considers the degree to which scale items have a high covariance in structure and may be assessed through both confirmatory and exploratory factor analysis (Valentine et al., 2015). The previous UK study had conducted both
exploratory and confirmatory analyses (Hutchinson et al., 2006). It is recommended that a different data set be used for confirmatory analysis (Matsunaga, 2010) and these authors met this through randomly dividing their sample in two and
conducting each analysis on a separate data set.
Assessment of structural validity of this research was limited. The assessment using Cronbach’s α indicated scale data were not reliable when applied to the scale determined from the UK study (Hutchinson et al., 2006). Therefore an exploratory principal components analysis was necessary.
It is recommended that confirmatory analysis be undertaken following this process using a separate set of data (Matsunaga, 2010). This was not possible as the entire data set was used for the exploratory analysis. Although this is a limitation of this research of the four elements highlighted earlier (Valentine et al., 2015) as many as possible have been undertaken from the data that were obtained.
Principal components analysis
Principal components analysis (PCA) (often also referred to as factor analysis) is a statistical technique used to determine a set of independent subsets from within a set of variables (Tabachnick & Fidell, 2007). This process is used in developing scales and may be used in an exploratory or confirmatory capacity (Pallant, 2013). Here it has been used in an exploratory capacity.
111 The key element of PCA is its use for item-screening (Matsunaga, 2010). In this thesis PCA has been used for the analysis of the first section of the survey which used the safety attitudes questionnaire (SAQ) (Sexton et al., 2006). This approach was also chosen due to its previous use by other applications of the same survey tool (Hutchinson et al., 2006).
Several steps are outlined for the process of PCA (Pallant, 2013). Data must be considered suitable, the number of factors to extract needs to be determined and finally the rotation of data and interpretation is finalised. Steps to encompass this include determining the suitability of data and sample size, factor extraction and rotation of factors. These are outlined in the following sections and were followed in order to determine the number of factors to be extracted. It is generally regarded that a single factor is made up of at least four items (Matsunaga, 2010).
There needs to be sufficient data for this analysis (Pallant, 2013). It has been previously been noted that sample size needs to be as large as possible so as to minimise the risk of incorrect results through bias or misspecification (Matsunaga, 2010). Opinions of the sufficiency of data vary from 5:1 to 10:1 ratios of responses to items (Pallant, 2013) with one author citing a minimum of 100 respondents (Matsunaga, 2010).
When considering the sample size (Section 4.2.5) it was determined that between 125—150 responses would be adequate. This assessment also took into
consideration that from a previous use of the SAQ analysis had been undertaken in two separate analyses, one for teamwork items and another for safety climate items (Hutchinson et al., 2006).
Factor extraction
The extraction of factors involves a variety of different steps (Pallant, 2013). These include the use of Bartlett’s Test of Sphericity, Kaiser-Meyer-Olkin (KMO) and Eigenvalues, Kaiser’s criterion, scree test and parallel analysis. Some of these are
112 considered controversial and capable of producing variety in the results (Hubbard & Allen, 1987).
Both Bartlett’s Test of Sphericity and the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (Pallant, 2013; Tabachnick & Fidell, 2007) were used to determine the suitability of the data for further analysis. A KMO value of 0.6 is recommended along with significance of Bartlett’s Test of p<0.05 (Pallant, 2013; Tabachnick & Fidell, 2007).
Some of the SAQ results obtained for this research impacted upon this process. Further details of how this was dealt with appear in the following chapter (Sections 4.3.1 and 4.3.3).
Analysis then proceeded to consider Kaiser’s criterion and Catell’s scree test
(Pallant, 2013; Tabachnick & Fidell, 2007). These processes assist in determining the appropriate number of factors to extract in each of the two sample sets (Tabachnick & Fidell, 2007).
Kaiser’s criterion recommends an eigenvalue score of ≥1.0 (Matsunaga, 2010; Pallant, 2013; Tabachnick & Fidell, 2007). Caution is urged however as it has been established that this process often leads to over extraction of factors (Hutchinson et al., 2006; Matsunaga, 2010). Catell’s scree test may also be inconclusive in similar circumstances as often it is difficult to determine the number of possible factors to extract (Pallant, 2013).
An alternative to the two aforementioned tests is parallel analysis and this approach was adopted to assist addressing the above mentioned concerns. This complex approach is becoming more popular in social science literature (Pallant, 2013) and is considered often to be the most accurate means of extraction (Hubbard & Allen, 1987; Matsunaga, 2010). Parallel analysis was applied along with use of Horn’s test (Hubbard & Allen, 1987; Zwick & Velicer, 1986) and the freely available Monte Carlo software (Watson, 2000).
113 Rotation of factors
The final element of PCA that needs to be discussed is the use of rotation. The two main approaches are oblique and orthogonal (Child, 2006; Matsunaga, 2010; Pallant, 2013). The main difference between the two is that orthogonal rotation assumes there is no underlying relationship or correlations in the data (Tabachnick & Fidell, 2007). It is therefore considered by many that oblique rotation methods are more appropriate as most data collection is likely to be correlated (Tabachnick & Fidell, 2007) especially in the social sciences (Child, 2006; Matsunaga, 2010). Component and pattern matrix tables are elements that also need to be considered when undertaking PCA (Pallant, 2013; Tabachnick & Fidell, 2007). The component matrix reflects the unrotated loadings of the Kaiser criterion whereas the pattern matrix presents the rotated solution (Pallant, 2013). There is debate in the literature about which of these should be interpreted (Matsunaga, 2010).
Following factor extraction the unrotated solution is often difficult to interpret and the goal of rotation is therefore to improve this (Tabachnick & Fidell, 2007).
Numerous approaches to rotation are available but unfortunately there is no consistent approach with recognition that different approaches may be taken by different researchers (Tabachnick & Fidell, 2007).
Oblique rotation using Direct Oblimen (Pallant, 2013; Tabachnick & Fidell, 2007) was found to be the approach that yielded factors that could be interpreted with a simple structure. This particular rotation method is useful as it allows for a wide range of factor correlations (Tabachnick & Fidell, 2007).
The experience of use of the SAQ in the UK study was also drawn upon in
determining which items to retain (Hutchinson et al., 2006). It is worth noting here that in a previous study the PCA for the teamwork factors was difficult and this also occurred with this present research (Hutchinson et al., 2006)
114 Following completion of the above steps the extracted factors were finalised. Having determined the items to retain, the internal consistency was assessed through the use of Cronbach’s α (Pallant, 2013). A score of 0.7 or higher is considered to reflect reliability (Allen & Bennett, 2012; Pallant, 2013). Following determination of the reliability of the identified scales, scores for each respondent were calculated. The formula used for this purpose was outlined previously when set calibration was discussed in Section 4.2.2.
This approach to handling of the data for the conditions of this research allowed for transparency in terms of the manner in which the conditions were determined. It also provides an approach consistent with other studies that provided for a comparison of the results obtained from this research with those of other studies.