Reliability and validity of the HSOPSC

The term ‘reliability’ refers to whether or not research results can be repeated (Oppenheim, 2001 & Bryma, 2012). The measurement of reliability comprises of three aspects, equivalence, stability and internal consistency (homogeneity) in its results. Equivalence refers to the level of agreement between two or more instruments when administered around the same time. The second aspect, stability, refers to whether similar or identical scores are obtained if tests are repeated on the same group of respondents, i.e. whether scores recorded are consistent from one particular time to the next.

Internal consistency (homogeneity) is in reference to the degree to which items on an instrument or test are providing a measurement of the same particular thing, and the degree

94

to which the questionnaire is free from random errors (Bowling, 2002 & Miller, 2014). Internal consistency (homogeneity) can be estimated through use of the Kuder-Richardson or the split-half reliability index, coefficient alpha index.Cronbach’s alpha was used in the measurement for testing questionnaire reliability of this study. It measures the average of all questionnaire items and their correlation with their scales (Bowling, 2002). Quantitative researchers such as Sekaran (2003), and Crano & Brewer (2008) noted that the closer a reliability coefficient is to 1.0 the more reliable are the findings of the study. They also, noted that if the reliabilities had a value of below 0.6, the research findings could be considered to be unreliable (Crano & Brewer, 2008).

Validity is the degree to which an instrument succeeds in measuring what it is supposed to measure (Alston & Bowles, 2003). There are two measurements for research validity. External validity refers to confidence in the applicability of the findings of a research study to other settings or people. While, internal validity is about how the research was done rightly and consider whether it avoided the influence of confounding factors on the research outcome (Roberts et al., 2006). Internal validity can be measured mainly with three approaches namely: content validity, construct validity and criterion-related validity (Punch, 1998).

Miller (2014) explained these three approaches as follows: content validity relates to the level to which an instrument is able to fully measure or assess the construct of interest. Typically, an instrument is developed through the achievement of a rational instrument analysis by the respondents that are familiar with the construct of interest (ideally from 3 to 5 respondents). Respondents will, in particular, review all items for their clarity, comprehensiveness and readability and arrive at a degree of agreement as to which of the items ought to be included within the final instrument. Construct validity is considered to

95

be a process that is ongoing in that a theory is refined, if need be, so that predictions can be made about test scores in different situations and settings. Assessment of criterion-related validity is done if there is interest in determining the relationship of specific criterion to scores from a test. Construct validity is the extent to which measurement done by an instrument has measured the theoretical construct or trait that a researcher has intended to measure (Miller,2014).

The HSOPSC survey used in this study was considered to be valid as it had been piloted on 1,419 hospitals employees from 20 hospitals across the USA. The results showed that all 12 dimensions had high levels of reliability (Cronbach's alpha ranging from 0.63 to 0.84) (Sorra& Nieva, 2004). Singla et al. (2006) conducted a systematic review of 13 instruments to compare between different patient safety surveys in terms of the emphasis of their content and their length, as well as noting their common usage in assessing patient safety culture. They argued that, whilst no instruments are perfect, the HSOPSC is a good instrument as it contains extensive information for assessing the patient safety culture.

Indeed, many studies have shown that the HSOPSC has good psychometric properties; for instance, Sorra and Dyer (2010) analysed survey data from 2,267 hospital units and 50,513 respondents to examine the psychometric properties of the items and composites of the HSOPSC whilst studying 331 U.S. hospitals. Their results provided overall supporting evidence that the 12 dimensions and 42 items of the survey had acceptable psychometric properties at all levels of analysis.

Moreover, Najjar et al. (2013) investigated the psychometric properties of the HSOPSC and its appropriateness for Arabic hospitals with a panel of experts to evaluate the

96

reliability and validity of a version translated into Arabic. Data were collected from 13 Palestinian hospitals including 2,022 healthcare professionals. The researcher acknowledged that the Arabic version had low internal consistency in some of its scales compared to the original survey or to other translated versions, such as those used in Turkey, England, Norway and Belgium. However, their study concluded that the Arabic version had good validity and acceptable reliability, with Cronbach's alpha ranging from 0.41 to 0.87, and that it was a suitable instrument to assess safety culture in hospitals in the Arabic speaking world. Table 4.2 displays also details of the reliability of the original survey and the Arabic version.

Table 2.4 Reliability of safety culture dimensions

Dimension No of items Cronbach’s alphas of (a) which developed by HSOPSC (AHRQ)

Cronbach’s alpha (α) HSOPSC(Arabic version)

Frequency of event reporting 3 0.84 0.87

Overall perceptions of safety 4 0.74 0.43

Manager expectations 4 0.75 0.75

Organizational learning 3 0.76 0.79

Teamwork within departments 4 0.83 0.77

Communication and openness 3 0.72 0.41

Feedback about error 3 0.78 0.69

No punitive response to error 3 0.79 0.59

Staffing 4 0.63 0.65

Hospital management support for patient safety

4 0.83 0.66

Teamwork across hospital departments 4 0.80 0.61

Hospital handover and transitions 4 0.80 0.73

97

The HSOPSC contains a number of 12 safety culture dimensions, which assess patient safety culture amongst health care workers. The survey measured 7 patient safety culture dimensions at the departmental levels:

no Dimension item

1 Supervisor/manager expectations & actions promoting patient safety. 4

2 Organisational learning & continuous improvement. 3

3 Teamwork within units. 4

4 Openness of communication. 3

5 Feedback & communication about errors. 3

6 Non-punitive response to error (blame-free environment). 3

7 Staffing. 4

Furthermore, the survey measures 3 patient safety dimensions at the hospitals levels:

no Dimension item

1 Hospital management support for patient safety 3

2 Teamwork across hospital units. 3

3 Hospital handover. 4

Moreover, 2 patient safety culture outcomes were included:

no Dimension item

1 Overall perceptions of patient safety. 4

2 Frequency of events reporting. 3

The survey items are measured on a 5-point Likert scale and ranged from (1) "Strongly Disagree" to (5) "Strongly Agree and take on average about 15 minutes to complete. Beside the 12 listed dimensions the survey includes an item that asks about the number of events reported the past 12 months. Participants are also asked to grade the patient safety in their work area on a five point Likert scale ranging from “Excellent” to” Failing”(See Appendix (1).

98

The questionnaire can be used to assess the safety culture of a hospital as a whole, and for specific units within hospitals, as well as to track changes in patient safety culture over time and evaluate the impact of patient safety interventions (Sorra & Nieva, 2003). Smits and colleagues (2008) conducted a study using HSOPSC to measure patient safety culture in some Dutch hospitals. They confirmed that the survey was effective in assessing a group safety culture just as well as individual attitude.

The justification for the survey method adopted during Phase 1 is as follows. Firstly, a questionnaire survey is a widely used method for the collection of primary data within previous research related to the assessment of patient safety (Singer, 2003; Sorra & Nieva, 2004 & Colla et al., 2005). Secondly, the approach taken in this survey was believed to be appropriate because descriptive information and an overall picture of the study population was provided by the identifying of relationships between research variables and the conducting of the required statistical tests to analyse the data (Collis & Hussey, 2003 ; Saunders et al.,2009). Thirdly, for the achievement of the research objectives, it is necessary to have a large sample from the targeted population from different geographic locations and from different professional backgrounds. The use of survey ensures that these requirements are achieved.