4.5 Assessment
4.5.1 Objective Attributes
In this section we explain the cells in Table 4.4 that are annotated with a superscript number. The numbers reference comments which provide a justification for the annotated
Name Errors Faults Phase
Qua[L] /
Qua[N]t Math Detail
[T]op-[D]own / [B]ottom-[U]p
FMEA P1 P2 PD-DD L/N Y High BU
EMV2 FMEA P1 P2 DD3 L/N N4 High BU
FTA P F PD-DD L/N Y Med/High5 TD
STPA F6 P7 CD8-PD-DD-Oper9 L N Med/High5 TD
M-SAFE F10 F11 PD12-DD L N Med/High5 TD & BU13
T-SAFE F10 F11 DD14 L N Med/High5 TD & BU13
Table 4.4: Summary of objective attributes of major hazard analyses discussed in this dissertation, with references to explanations in the text of Section 4.5.1. Abbreviations used: P – Partial, F – Full, PD – Preliminary Design, DD – Detailed Design, CD – Conceptual Design, Oper – In Operation. Adapted from Table 3.4 of [7, pg. 47]
STPA’s Hazardous Control Action Aviˇzienis et al.’s Service Failure Mode
Not Providing Causes Hazard Halt Failures
Providing Causes Hazard Erratic Service
Wrong Timing/Order Causes Hazard Early/Late Timing Failures Stopped Too Soon or Applied Too Long Early/Late Timing Failures Table 4.5: A mapping from STPA’s ways control actions can be hazardous to Aviˇzienis et al.’s service failure modes. Hazardous Control Action terms are from Figure 8.4 in [30, pg. 219], Service Failure Modes are from Fig. 8 in [3, pg. 9]
entry; table entries without numbers are considered to be self-explanatory.
1. FMEA – Full Detection of Errors: FMEA would not typically catch, for example, component interaction errors.
2. FMEA – Full Detection of Faults: FMEA would not typically catch, for example, faults in software.
3. EMV2 FMEA – Detailed Design Only: Because EMV2 annotations are added to AADL system descriptions, which cannot easily be created for preliminary designs, an EMV2-supported FMEA cannot be done on a preliminary design. Note that this is
STPA’s Fault-Related Guideword Aviˇzienis et al.’s Fault Class Flaws in creation (Controller) 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 Component failures (Controlled Process) 11, 12, 13, 14, 15 Process changes (Controller)/Incorrect modification or
adaptation (Controller)/Changes over time (Controlled Process)
19, 20, 21, 29, 30, 31
Table 4.6: A mapping from STPA’s fault-related guidewords to Aviˇzienis et al.’s fault classes. The STPA guidewords are from Figure 8.6 in [30, pg. 223], Aviˇzienis et al.’s terms are from Figure 5 in [3, pg. 6]
mitigated somewhat by the ease with which system models that are built in AADL can be updated. Additionally, it may be possible to capture a preliminary design in AADL by using appropriate abstractions.
4. EMV2 FMEA – No Math Required: As EMV2 is tool-supported, it can produce the estimates of a hazard’s likelihood (and impact, if the necessary annotations for a FMECA are added) without requiring the analyst to perform the underlying calcula- tion himself.
5. Several – Medium/High Level of Detail: These hazard analyses can be performed at either a moderate or high level of detail, depending on which the analyst chooses. 6. STPA – Full Detection of Errors: We believe that the classes of unsafe control ac-
tions identified in STPA’s first step can be argued as equivalent to Aviˇzienis et al.’s service failure modes using the mapping in Table4.5. There is an important caveat to this mapping, though, in that Leveson’s terms are specific to binary control actions. Extensions to ranged control actions (e.g., value too high or too low) are natural, how- ever, and guide-phrases for analyzing complex (i.e., aggregate or parametric) control actions have been documented by Thomas and Leveson [99].
space of potential errors, her guidewords for faults are quite abstract and do not approach the level of coverage provided by Aviˇzienis et al. STPA’s set of causality guidewords are not fault specific: i.e., the terms for errors, like “Delayed operation” are mixed in with terms for faults, like “Flaws in creation.” While STPA’s fault terms arguably include 22 of Aviˇzienis et al.’s fault classes (see Table 4.6 for the full mapping), they do so at such a high level of abstraction that actually achieving full coverage is unlikely. For example, the STPA term “Component failures” might be argued to cover classes 11-15 of Aviˇzienis et al.’s faults, but it would require consid- erable analyst skill to look at that term and derive faults from all five fault classes. See Section 4.5.2for a full discussion of the difference in difficulty between SAFE and STPA.
8. STPA – Applicability in Conceptual Design: As it is described in [30], STPA is most applicable to preliminary and detailed designs, but recent work by Fleming extended the analysis to conceptual designs [100].
9. STPA – Applicability in Operation: Though we primarily discuss STPA in this disser- tation, Leveson has also developed Causal Analysis based on STAMP (CAST) which can be used for post-hoc reasoning [30].
10. SAFE – Full Error Coverage: We use Aviˇzienis et al.’s service failure modes for our analysis of errors that propagate into an element. The authors argue that there are two failure domains, content and timing, and that failures can be classified into one or both of those domains [3]. Since a service failure in a preceding element is equivalent to an error, we claim complete, though admittedly abstract, coverage of failures. 11. SAFE – Full Fault Coverage: We use a reduced set of Aviˇzienis et al.’s fault classes
for our analysis of root causes [3]. We further supplement these classes with three more designed to detect faults arising from element interactions, and we claim that we achieve the best possible coverage of faults using this combined set.
12. M-SAFE – Applicability in Preliminary Design: Since the manual version of our pro- cess only requires names and box-and-line diagrams for system descriptions, it can be applied to a preliminary design which exists at a more abstract level of specification than, say, a model built using our subset of AADL.
13. SAFE – Top-Down and Bottom-Up Nature: The new process described in this disserta- tion contains both top-down and bottom-up steps. Step 0 involves working backwards from the accidents and hazards that need to be avoided, while step 2 involves taking faults and considering how they would impact the rest of the system. Step 1 exists between the two notions, as it involves marrying the top-down notion of successor dangers with the bottom-up concept of manifestations.
14. T-SAFE – Exclusive Applicability to Detailed Design: As the tool-assisted version of our process relies on a system model built in AADL, it necessarily requires a detailed design, since preliminary designs cannot easily be built in AADL. Like the EMV2- supported FMEA, though (see comment 3), we note that these designs are more easily updated than manually-specified detailed designs.