Sequential models work well for losses caused by failures of physical components or human errors in
R EVIEW AND I NITIAL D EVELOPMENT
5.3.17 Summary of findings
The candidate set of Railway Performance Shaping Factors (R-PSFs) has been selected on the basis of several sources of information. In general, the contextual factors throughout the assessed taxonomies show strong similarities, where contextual factors are considered as conditions or factors that influence human performance. The process followed to select the relevant R-PSFs started with an initial selection based on the review of contextual factors identified in five railway related taxonomies (Table 5-1). As a result, six PSFs categories were determined as relevant: personal, task, team, organisational, system and environmental factors, which embrace all the generic categories of factors as identified by the existing taxonomies. In addition, more than 75 individual PSFs relevant to railway operations were identified.
The initial findings were augmented with a review of non-railway related taxonomies. This phase involved the review of eleven well-documented and widely implemented taxonomies designed primarily to analyse human errors in the nuclear, aviation and process industries (Kirwan, 1994, Forester et al., 2006, Bell and Holroyd, 2009). The literature shows that taxonomies have a tendency to be repetitive, i.e. include the same PSFs (Wilson et al., 2007a). Therefore, although the selected taxonomies may not be exhaustive, it can
REVIEW AND INITIAL DEVELOPMENT
_________________________________________________________________________ reasonably be argued that they contain the majority of the factors that influence human performance in every working environment.
The assessed taxonomies and their related factors are presented in tabular form in Appendices II and III. Due to the fact that there is no agreed taxonomy for the railway industry, taxonomies from other domains were analysed and compared with those within the railway domain. The comparisons were conducted on two levels, firstly between the categories of PSFs and secondly, amongst the individual PSFs. The comparisons were mostly straightforward as particular factors were identified in almost all taxonomies, e.g. the factors “procedures” and “training”. The comparison on the first level did not reveal any categories that should be added to the six categories as obtained from the railway taxonomies. However, this was not the case for the comparison on the second level, as six R-PSFs elements could not be directly identified as belonging to any of the railway related taxonomies. These factors are: routine, expectation, vigilance, interpretation, decision- making skills and Safety Management Systems (SMS). However, routine is indicated as an underlying factor within the RARA taxonomy. In addition, the factors vigilance, interpretation, decision-making skills and expectation are defined within the TRACEr-Rail approach as operational error types. Finally, although not identified in any railway relevant taxonomy the factor SMS is clearly included within the CREAM taxonomy. Therefore, they are all retained within the set of candidate R-PSFs due to their relevance. The final list of candidate R-PSFs contains more than 250 elements, as shown in the Appendix III.
5.4
Identification of dynamic R-PSFs
In Chapter 4 it was stated that the relationship between PSFs and time has not been addressed adequately. This is confirmed from the results in Section 5.3 where no taxonomy refers directly to the dynamic perspective of PSFs. However, many factors may gradually or rapidly change while a task is executed, e.g. distraction, stress or fatigue. These factors can be claimed to have a direct impact on human performance just before an error is made and subsequently an accident or incident occurs. To address this issue, this thesis introduces an additional R-PSFs category, referred to as dynamic personal factors. It describes all the factors that characterise and affect the individuals and are strongly related to the precise moment of operation/occurrence, e.g. distraction. In addition, environmental R-PSFs are considered as dynamic factors.
The remaining categories are considered as static factors, i.e. the PSFs that have less association with the time of the occurrence. However, under certain conditions, e.g. an
REVIEW AND INITIAL DEVELOPMENT
_________________________________________________________________________ accident on the network, factors such as the workload and time pressure may be affected significantly by the precise moment of the operation/occurrence. For these conditions the factors are considered dynamic.
5.5
Augmentation with operators hierarchical task analysis
After the identification of the R-PSFs, as derived from the relevant literature, the task analysis of operators was again reviewed. The purpose of this review is twofold. First, to map each of the factors with the duties of operators so as to confirm that the extracted R- PSFs may indeed potentially affect operators’ performance. Second, to identify whether any factors have been either ignored, neglected or do not match to the operational concept of railways. The review focused particularly on the factors that affect the signallers and controllers performance as the majority of the railway approaches are built on train drivers’ performance.
The mapping showed that all identified R-PSFs are relevant to the railway operational concept. Furthermore, no factors were conspicuously identified as missing or ignored. For train drivers this outcome was rather expected since the majority of relevant taxonomies were developed on their tasks. On the other hand, with respect to signallers, two are the most likely reasons for such observation. The first is that the number of PSFs is limited regardless of the task or operational procedures (Kirwan, 1994).The second is that within a generic operational concept it can be claimed that the duties of operators in control rooms of industries other than railways, e.g. aviation, have many similarities to the duties of train signallers and controllers. For instance, both air traffic controllers and railway signallers are responsible to: maintain the scheduling of services, contact with either pilots or train drivers when necessary, and prevent any type of conflicts. Subsequently, it is assumed that the majority of the identified factors from the review of relevant non-railway taxonomies cover sufficiently the factors that affect the performance of railway signallers and controllers.
5.6
Presentation of the R-PSFs taxonomy
Having reviewed sixteen PSFs taxonomies and the tasks of railway operators, the R-PSFs taxonomy was finally created. The review revealed seven R-PSFs categories and more than 250 R-PSFs elements. However, many of the individual elements are either identical or overlap. Therefore, they have been combined based on their common characteristics and
REVIEW AND INITIAL DEVELOPMENT
_________________________________________________________________________ when necessary renamed and redefined, e.g. the R-PSFs “Railway communication means”, instead of “Communication equipment” as indicated in the HERA taxonomy.
Finally, 43 R-PSFs were identified and allocated to each one of the seven R-PSFs categories as displayed in the Table 5-3. Furthermore, detailed definitions (Appendix IV) to both categories of factors and the elements are provided in order not only for the taxonomy to be implemented by researchers with different backgrounds and perspectives, but also for the derived results to be comparable. Definitions aim to be short and transparent, so as to be easily applied from both academic researchers and practitioners. Unless otherwise stated, definitions were derived from the Collins English Dictionary (2009). Finally, constructive examples are also given (Appendix IV), which aim at supporting researchers to better understand the context within each of the R-PSFs should be used.
Table 5-3 The complete R-PSFs taxonomy
Category of R- PSFs Railway Performance Shaping Factors
Personal Factors
! Experience ! Familiarity ! Fit to work (health) ! Individual characteristics ! Motivation
! Training - competence
Dynamic Personal Factors
! Decision making skills
! Distraction-loss of concentration ! Expectation ! Fatigue ! Interpretation ! Perception ! Situational awareness ! Stress ! Vigilance Task Factors ! Monotony ! Routine ! Task complexity ! Task instructions
! Time pressure - time to respond ! Workload
Team Factors
! Communication between employees ! Relations within team
! Teamwork
REVIEW AND INITIAL DEVELOPMENT
_________________________________________________________________________
Category of R- PSFs Railway Performance Shaping Factors
Organisational Factors
! Communication within organisation ! Fit to work aspect
! Incentives for employees ! Leadership
! Quality of procedures, standards and regulations ! Relations within organisation
! Safety culture (disregard procedures) ! Safety Management Systems
! Shift pattern (working hours, breaks, manning) ! Supervision
! Training / training methods
System Factors
! Human Machine Interface (HMI) ! Railway Communication Means (RCMs) ! System design
! Trust in equipment ! Working environment
Environmental Factors ! ! Visibility Weather conditions
In addition, Figure 5-3 illustrates the structure of the taxonomy by showing the interactions between operators and R-PSFs.
Figure 5-3 The structure of R-PSFs taxonomy
The structure displays the engineering and psychologists perspectives of the system and has been established on the generic model introduced by Hammerl and Vanderhaegen (2009) (Section 4.10.1.3). The structure clearly indicates how the performance of operators
!"#$%&%'()*+,'-.$)% /0$-"1/(% 20$-",/-+%3$-#/-4"(5$%% 67-"*(%8-*9$-:%!*;("<<$-:%=/(,-/<<$->% !?+,$4%#"5,/-+% 8?("4*5%3$-+/("<% #"5,/-+% @(9*-/(4$(,"<% #"5,/-+% 2-;"(*+"1/("<%#"5,/-+% 7$"4%#"5,/-+% 3$-+/("<%#"5,/-+% 7"+A% #"5,/-+% B%C%3!D+%,"E/(/4?%+,-'5,'-$%% @E0$-*$(5$:% F/19"1/(% G$",H$-%5/()*1/(+:% I*+*.*<*,?% !,-$++:%D"1;'$% JFK:%G/-A*(;% $(9*-/(4$(,%% G/-A</"):% =/40<$E*,?%% =/44'(*5"1/(:% =//-)*("1/(% 7-"*(*(;%4$,H/)+:%%!"#$,?%5'<,'-$%
REVIEW AND INITIAL DEVELOPMENT
_________________________________________________________________________ is influenced by the different R-PSFs as well as how the R-PSFs interact with each other. As shown in Figure 5-3, blue factors reflect the dynamic R-PSFs, while orange represent the static. Although the structure was developed for the railway industry, it can be claimed generic, as it stems from the review of taxonomies from several domains. Therefore, it can be argued that both the structure model of the R-PSF taxonomy as well as the included R- PSFs categories are transferable to any other transport mode or industry. However, the individual R-PSFs might change according to the attributes and features of the domain or/and industry in which they will be implemented.
Finally, it is clear from the taxonomy’s structure and the provided definitions that R-PSFs not only directly but also indirectly affect human performance. That is when a factor A may influence a factor B and subsequently affects operators level of competence. For instance, unclear procedures may increase operators’ levels of workload and stress and consequently indirectly influence their performance. The interdependencies among R-PSFs are in detail described in Section 6.3.3.
Having identified the suggested R-PSFs taxonomy and its constituents, a validation process based on real data and Subject Matter Experts (SMEs) consultation was than conducted as described in the remainder of this chapter and Chapter 6.
5.7
Consolidation of taxonomy with real data
To consolidate the taxonomy, that is, crosscheck, verify, and confirm literature findings with real data, worldwide human error related accident and incident investigation reports were analysed. The literature shows that up to 80-90% of all serious railway accidents are attributed to human error due to degraded human performance (Rail Safety and Standards Board, 2004b). Thus, this thesis focuses on the study of factors that affect the performance of operators resulting in major accidents or high-risk incidents. Therefore the selected reports describe events that by legislation should be investigated. In Chapter 3 it was shown that different definitions are used to describe occurrences internationally. The European (European Commission, 2004) definitions are deemed to be more comprehensive, hence are adopted for the purposes of this thesis. Subsequently, all reports are classified into serious accidents, accidents and incidents. The reports describe occurrences relevant to operational errors caused by train drivers, signallers or controllers while trains move from origin to destination (including shunting operations). Railway occurrences caused by maintenance personnel are not included in the analysis.
REVIEW AND INITIAL DEVELOPMENT
_________________________________________________________________________
5.8
Data characteristics
In total 2278 publicly available and 418 confidential reports were gathered from several worldwide stakeholders. Reports were studied and only those describing human error related operational occurrences were finally retained. Subsequently, 479 reports were selected and classified according to the EU Directive 2004/49/EC (European Commission, 2009), with 70 serious accidents, 189 accidents and 220 incidents. The reports describe events that occurred in the period 1997-2011 in 24 different countries includings 21 European countries such as U.K., Norway, Germany and Switzerland; U.S.A; Canada and Australia. The timeframe was decided upon consulting summaries of railway activities issued by institutions of national and international reach, e.g. the International Union of Railways (International Union of Railways, 2007), the Federal Railroad Administration (Federal Railroad Administration, 2012) and the European Railway Agency (European Railway Agency, 2008). Data represent occurrences only from countries with highly developed railway systems. Undoubtedly, it would be beneficial if data from developing countries could also be gathered; however, this was not possible due to access and resources limitations.
5.9
Data requirements
In Chapter 4 was highlighted the importance of examining human error by accounting for the overall context within which an error occurs, including any underlying or hidden factors that may affect operators performance. Hence, the selected reports should contain a limited amount of information that allows researchers to analyse in detail the causes of a human error. Such information contains the:
! description of the event;
! involved personnel (responsibility), e.g. individual characteristics;
! type of railway operation, e.g. passenger (intercity or regional), freight or metro; ! features of system, e.g. type of train, on-board and infrastructure equipment; ! severity of event, e.g. accident or incident;
! type of occurrence, ,e.g. train derailment or trains collision; ! location and time of event, e.g. open line, station;
! immediate cause, e.g. train unable to stop; ! causal factors, e.g. train driver falls asleep;
! contributing factors (R-PSFs) that affect operators performance, e.g. adverse weather conditions, inadequate training, communication errors, and
REVIEW AND INITIAL DEVELOPMENT
_________________________________________________________________________ ! number of casualties or material damages, which defines the severity of events. To capture such level of information, the analysis presented in this thesis is based on detailed official railway investigation reports, as descriptive reports contain limited amounts of information (particularly with respect to the human factors aspect). The importance of analysing detailed reports also arose from the author’s personal communication with two railway organisations, i.e. the Swiss Federal Railways (SBB) and the UK Network Rail. Both the SBB (Swiss Federal Railways, 2010) and Network Rail (Gibson et al., 2013b), have acknowledged the need to collect specific information with respect to human factors as significant contributors to railway occurrences.
Although the selected reports are official documents completed by authorised and eligible personnel, their quality was verified to ensure the reliability and robustness of the analyses. Therefore certain quality criteria were considered and examined, as presented in the following section.
5.10
Data quality criteria
The literature (Wang and Strong, 1996, Brackstone, 1999, Dupuy, 2011) identifies up to eight criteria (dimensions) that can be used to test the quality of collected data. These are: accessibility, accuracy, completeness, consistency, credentials, interpretability, relevance, and timeliness. All eight criteria are described in more detail in this section, while their summary is shown in Table 5-4.
Table 5-4 Data quality dimensions review (Dupuy, 2011)
Quality
dimension Characteristics Related notions
Accessibility Availability of data to the user Cost, availability
Accuracy Extent to which data values are correct, close to reality and free from error Correctness
Completeness Quantity of reported data for each unique record Missing values
Consistency Extent to which data values are coherent and compatible with one another within their corresponding framework and consistent with the framework definition
Coherence, regularity, definitions, validity conformance
Credential Extent to which data are obtained from reliable sources Reliability
Interpretability Extent to which data are completed or perceived in an unambiguous manner Clarity
REVIEW AND INITIAL DEVELOPMENT
_________________________________________________________________________ Quality
dimension Characteristics Related notions
Timeliness Currentness of the data in terms of the delay in providing and processing the information Up-to date
5.10.1
Accessibility
Accessibility refers to the ease with which data are obtained by any user. This encompasses the extent of ease to which the existence of this data can be ascertained and the suitability of mediums through which the data can be retrieved (Dupuy, 2011). The cost of the information may also be an aspect of accessibility for a number of users (Brackstone, 1999). Regarding railway investigation reports, accessibility is associated with confidentiality considerations. That is the willingness of railway organisations to share and allow access to their data (Dupuy, 2011). For this research, all collected investigation reports refer to occurrences that railway organisations are obliged to report according to the relevant safety regulations. Subsequently the majority of reports are publicly available and railway organisations, i.e. train operators, infrastructure managers, national investigation bodies, were not involved in the collection process. In addition, two of the databases were collected directly from an infrastructure manager and an integrated company (train operator also being
an infrastructure manager) on a confidentiality agreement. Thus, accessibility is not relevant
as quality dimension for the reports collected for this research. However, accessibility should be considered as quality criterion if access to data of developing countries was tested.
5.10.2
Accuracy
Data accuracy accounts for the degree to which data correctly describe the elements they are aimed to measure (Dupuy, 2011). Accuracy is often expressed in terms of error in statistical estimates and is classified into bias (systematic error) and variance (random error) components (Brackstone, 1999). It may also be defined in terms of the main sources of error that could potentially lead to inaccuracy (e.g., coverage, sampling). For this research, due to the format of data, accuracy is not considered as quality dimension, since all investigation reports correctly describe the elements they are aimed to measure.
REVIEW AND INITIAL DEVELOPMENT
_________________________________________________________________________
5.10.3
Completeness
Completeness is the degree to which a database provides information for all data fields that are supposed to contain information (Dupuy, 2011). This dimension is directly related to the predefined minimum requirements. Therefore, for this thesis, completeness is measured by the degree to which investigation reports satisfy the expectations of the predefined minimum requirements. Any report that did not include the minimum requirements was no analysed, therefore completeness, for this study, does not account for quality criterion.
5.10.4
Consistency
Consistency of data indicates the degree to which data could be considered coherent and compatible with one another within a broad analytic framework and over time (Brackstone, 1999, Dupuy, 2011). Coherence is enhanced by using standard concepts, classifications, methodology and practices, target populations and common procedures across surveys. Within the context of this research, all investigation reports included demonstrate high levels of consistency, as their format and detail of information are identical. However, depending on the severity of events the quantity of included information may vary. Therefore, reports were classified and analysed according to their classification. Subsequently, consistency is not considered a quality dimension for this research.
5.10.5
Interpretability
Interpretability reflects to which extent data are presented in a clear manner and indicates the availability of a common and shared definition for the collected data (Dupuy, 2011). For investigation reports, this typically implies any underlying concepts, classification and collection methodologies (Brackstone, 1999), where similar responses demonstrate well- interpreted collected data. Interpretability was selected as quality dimension, since many reports do not clearly indicate the contributing factors that influence the performance of operators. Subsequently, the author, while reviewing the selected reports, was extracting the implied information. Therefore, a level of bias, subjectivity and uncertainty may arise concerning the extracted information and the final findings could be questioned. To account for this, three final year PhD students in transport safety, familiar with analysis of safety reports, reviewed a random sample of the 479 reports to identify the contributing factors included in them based on the given definitions (Appendix IV). The findings of the PhD students confirm those of the author, and therefore the author concludes that whilst bias or
REVIEW AND INITIAL DEVELOPMENT
_________________________________________________________________________ subjectivity issues cannot be eliminated, these do not compromise the quality of the findings. Secondly, SMEs verified the final findings, as described later in the Section 6.3.2.
5.10.6
Relevance
The relevance dimension represents the degree to which the reported data are suitable for a specific domain or study (Dupuy, 2011). For this thesis, relevant investigation reports are considered those that refer to railway accidents and incidents due to human errors. In particular, as stated earlier, this thesis accounts only for occurrences caused by train drivers,