Implications for OP3

The feedback collected for the third category of the emerging framework—OP3 Control over implementation quality at the service provider-software manufacturer boundary— essentially confirmed the role of this precursor in explaining the quality of human machine interaction as found in the operational automated system. In particular, three corroboratory categories have emerged. Categories 1 and 2 indicated that the problems involved in the trust based control observed in the Deltasky case were not limited to this organisation, but were also observed in other service providers—both small and large ones. Finally, the last category of feedback, in addition to further confirming the accuracy OP3, also expanded this category, for it highlighted the fact that the actual control over implementation depends on the prior knowledge of the service provider. Without such a knowledge, it is important to frame the relationship with the software manufacturer as a long term partnership. Overall, this feedback seems to confirm the influence of OP3 in the handling of HAI issues within safety critical service provider organisations.

Table 23. Evolution of OP1 as a result of Study 3: this category has remained unchanged.

OP3: CONTROL OVER IMPLEMENTATION QUALITY AT THE SERVICE PROVIDER MANUFACTURER BOUNDARY (Study 2)

OP3: CONTROL OVER IMPLEMENTATION QUALITY AT THE SERVICE PROVIDER MANUFACTURER BOUNDARY (Study 3)

Chapter 7. Study 3 results

135

SUMMARY AND DISCUSSION

The aim of this study was to explore the plausibility of the organisational patterns of the initial version of the OPHAII framework resulting from Studies 1 and 2. By means of a focus group discussion combined with a qualitative questionnaire, this study obtained corroboratory feedback from a group of 11 safety net experts. These experts were invited to comment on the definitions of the identified organisational precursors of the framework, based on knowledge coming from their direct and indirect professional experience.

The evidence collected in this study took the form of a total of 11 corroboratory categories: four categories identified for OP1; four, for OP2; and three, for OP3. It can be noted that some differences can be observed regarding the relative depth of these categories. The corroboratory categories identified for OP1 and OP3 are organised on one hierarchical level; those identified for OP2, on the other hand, are organized on two hierarchical levels. In other words, the corroboratory categories identified for OP2 enjoy a deeper structure than OP1 and OP3. This difference reflects the fact the experts had different types of reactions to OP1 and OP3 on the one hand, and OP2 on the other. OP1 and OP3 received essentially supporting feedback, where the experts’ response to the definitions of these categories of organisational precursors mainly suggests that they were witnessed by the experts participating in the study (as concluded under § 7.2.1.1 and § 7.2.3.1). So, this expert’s response added confidence without proposing a refinement. Somehow, this suggests that the earlier definition of OP1 and OP3 was perceived as trustworthy.

The earlier definition of OP2, on the contrary, was perceived as being too narrow in scope. The definition of organisational capability for handling HAI issues, in fact, was perceived as underrepresenting important components, namely the role of senior management, the existence of a safety net team, the existence of a dedicated safety net performance framework (as concluded under § 7.2.2.1 ). According to the experts, these elements needed to be more visible in the framework, and therefore an update and more trustworthy version of OP2 had to include them.

Having elaborated on how the collected evidence evolved the OPHAII framework, it becomes important to elaborate on the implications of the same evidence on:

 OPHAII’s interpretive validity7_{, i.e., the extent to which the identified categories}

of organisational precursors capture organisational dynamics that appeared plausible according to the viewpoints and experiences of the interviewed experts;

 OPHAII’s generalisability, the extent to which the framework can be applied to other organisational contexts besides those of Alphasky and Deltasky.

Regarding interpretive validity, it can be said that the study has allowed to collect corroboratory evidence which appeared to improve the initial OPHAII’s version (as delivered by Studies 1 and 2). In particular, the study has identified a total of 11 corroboratory categories, which capture relevant experts’ opinions, experiences, and suggestions for improvement about the framework. This evidence suggests that the OPHAII’s categories of organisational precursors have been witnessed by the participating experts in real life situations, i.e., situations related to the implementation and improvement of the MSAW and other safety nets occurred in European ANSPs. The fact that these patterns resonate with the knowledge of the selected experts is an important indicator of the study’s interpretive validity. Two reasons support this position. In qualitative research member checking is usually accepted as one of the most important validity check (Morgan, 1997) (§ 4.3.4). Furthermore, the importance of member checking is especially important in the case of social and organisational phenomena that, as for the phenomenon under study in this research, cannot be directly observed, or measured by objective and independent quantitative data collection means. In this case recurring to the opinions of experts become a necessary means to draw conclusions.

Regarding generalisability of the framework—i.e., the extent to which the results can be applied, at least, to other service providers—it can be noted that the expert professional experience was also based on contexts other than Alphasky and Deltasky. This invites some scepticism about the idea that the identified categories of precursors are the products of the unique organisational contexts of Alphasky and Deltasky. Rather, it seems reasonable to expect that the same precursors may occur also in other ANSPs, at least European ones. In addition to the collected experts’ view, one further consideration can support this point. In particular, European ANSPs operate in a common competitive and institutional environment, i.e., the European air traffic management system, which dictates common European-wide modernization roadmaps, and common rules, standards and policies for the conduct of commercial practices and safety. Arguably, these conditions expose European ANSPs to similar common pressures for modernization, cost- reduction, and productivity.

Of course these considerations do not address the question of whether the framework can be generalised to other contexts other than European ANSPs. This important question will be addressed in the next chapter, together with a reflection on the broader theoretical implications of the framework for the understanding of HAI issues in in safety- critical domains.

Chapter 7. Study 3 results

137

CHAPTER CONCLUSIONS

This study reported on the third and final empirical study of the thesis. The study aimed to corroborate and refine the initial version of OPHAII delivered by Studies 1 and 2. The study consisted of an SME study based on a sample of 11 safety net experts. The evidence collected further confirmed the first and the third organisational precursors of the framework, while the second precursor had to be refined in order to better reflect the experiences of the experts. This evidence led to a refined version of the framework, which is reported below. At this point, what remains to be discussed at this point are the broader implications of the framework for the understanding of HAI issues in complex, high consequence organisations, and what are the practical implications of the framework. These questions will be addressed in the following chapter.

Table 24. The revised and expanded version of OPHAII, as resulting from Study 3. While OP1 and OP3 have been confirmed, OP2’s subcategories have been refined and expanded.

OP1: ORGANISATIONAL ASSUMPTIONS DRIVING AUTOMATION

IMPLEMENTATION AND IMPROVEMENT - Assumptions about the system’s role - Assumptions about the HAI issue

OP2: ORGANISATIONAL CAPABILITY FOR HANDLING HAI ISSUES - Parameterisation process

- Positive attitude towards air traffic controllers’ involvement - Senior management support

- Supporting tools - Safety net team

- Safety net specific performance framework

OP3: CONTROL OVER IMPLEMENTATION QUALITY AT THE BOUNDARY BETWEEN THE SERVICE PROVIDER AND THE SOFTWARE MANUFACTURER