In this section we explain specifically how the compositional aspects of our hazard analysis process from Chapter4work. Table5.1has examples of the results of these steps having been applied to the PCA Interlock scenario. Note, however, that we only discuss the SpO2 sensor
(leaving out other possible physiological parameters like Respiratory Rate and ETCO2).
Concrete States Abstract States Abstrac.on Func.on Top Undesirability Interac.on Points Sensor Actuator
Controller System Boundary
Controlled Process
Figure 5.1: Semantic objects in our formalism, labels used in our definitions are shown in Figure 5.2
we arbitrarily chose a single component of each role and traced them though the full process. 1. For the purpose of these formalizations, we term the current element under analysis m. The important aspect of this step, for the formalisms presented here, is to find a delineation between sub-elements inside an element’s boundary and those outside of it, i.e., those of its environment. As a result of this step, an analyst should be able to identify using Definition 2 the component’s name (m), set of concrete states (S), role (r), interaction points (I), and if applicable subcomponents (Sub).
2. When given a state of component m and a state of its environment, an analyst will determine whether or not the pairing is both wanted and observable according to
ˆ
S
urmS
rmu
Abs
i ∈ I
out mr
Sub
ˆ
S
umS
mn
x
Figure 5.2: Labels used by our formalism, descriptive labels are shown in Figure 5.1
some notion of undesirability u using Definition 3: Concrete Undesirability. This corresponds to Definition 1: Undesirabilityfrom Section 5.1.1. Note that u is a not a state or set of states, but rather a concept which we use to index our later definitions. There are two potential sources for this notion of undesirability, depending on if an analyst is working with a component in isolation (i.e., without a surrounding system context) or one that is part of a full system.
(a) For isolated components, our analyst would anticipate the use of her device and identify a corresponding harm, i.e., an analgesic pump manufacturer would antic- ipate that overadministration would be a potential source of harm. This is similar to how device manufacturers currently have in mind an intended use when devel-
Name Sensor Controller Actuator Process Definition2: Component
m Pulse Oximeter Interlock App PCA Pump Patient
S {. . . } {. . . } {. . . } {. . . }
r Sensor Controller Actuator Process
I {(Blood, in), (SpO2, out)} {(SpO2, in), (P umpCmd, out)} {(P umpCmd, in), (Analgesic, out)} {(Analgesic, in), (Blood, out)}
Sub {BloodM onitor,
SpO2Calc, SpO2Send} {SpO2M onitor, P umpCmdLogic, CmdSend} {CmdM onitor, P umpLogic, P umpM otor} N/A
Definition3: Concrete Undesirability
u Overestimating
patient health
Enabling the pump erroneously
Pumping when unsafe Overadmin. of
Analgesic Definition5: Abstraction ˆ Sum {1%, 2%, . . . } {CmdP umpEnable, CmdP umpDisable} {GiveDrug, N oDrug} {Healthy, Risk, Overdose} ˆ Surm {1%, 2%, . . . } {ShldEnabP ump, ShldntEnabP ump} {ShldGiveDrug, ShldntGiveDrug} N/A Definition6: Abstract Undesirability
n 95% P umpEnable GiveDrug N/A
Su,nm {94%, 93%, . . .} {ShldntEnabP ump} {ShldntGiveDrug}
Smu,n {95%, 96%, . . .} {ShldEnabP ump} {ShldGiveDrug}
Definition8: Avoidance
Imu {Blood} {SpO2} {P umpCmd} N/A
Table 5.1: Examples of the formalisms applied to components of the PCA Interlock scenario oping a particular device, see the discussion of ISO 14971 in Section2.2.5. Thus, for isolated components, u is a violation of one of the component’s safety-related functional goals and in the language of SAFE is a successor danger.
(b) For a full system, our analyst would identify a notion of harm, which is equivalent to an Accident in SAFE or STPA, or an Unintended Consequence in IEC 80001 [30, 32]. Then, they would “push down” that top-level notion into individual
elements. In the manual version of SAFE, this is done when the analyst imports the Successor Dangers from a successor component, see Section4.3.1. For exam- ple, in the PCA interlock scenario, the system is designed to avoid an overdose. Thus, the PCA pump running when the patient is at risk of an overdose would be the actuator-specific version of that harm (see Table 5.1 for more examples). Unlike in (a), here u is a system-level notion, and each subcomponent must take into account the behavioral aspects of the rest of the system—this is similar to how device manufacturers judge whether or not a particular software or hardware subcomponent is fit for use in a given device.
3. Using the relation established in Definition3, we now establish an equivalence relation using Definition 4: Distinguishability that allows us to group the states of either an element or its environment. These groups can then be abstracted into a set of equivalence classes using Definition 5: Abstraction. This is simply a formalization of the implicit, intuitive step that analysts perform now in many hazard analyses when they create a mental model of how their system interacts with a notion of harm. STPA uses the term “Process Model” for this concept, and we re-use the term, requiring the creation of a process model for controller components in SAFE (see Section 4.3.1). Consider again Leveson’s moving train example from [30]. Rather than differentiate between cases where the train’s doors are 24 or 25 inches apart, the doors are said to be “open” and the states are equivalent to the notion of a passenger falling out. Similarly, regardless of if the train is moving at 60 or 61 miles per hour, we simply say the train is “moving” (and these names are what we mean by representatives of the equivalence classes identified in Definition 5). Even in more traditional hazard analyses (e.g., FTA or FMEA), huge numbers of concrete states are grouped together by their observable effect on the fault, failure, or safety problem being considered (see, e.g., [7]). We believe that analysts will benefit from formal guidelines that speak to the appropriateness of their mental models, and that this formalization is necessary
before suitable tooling can be developed.
4. For each abstract state of the component, use Definition 6: Abstract Undesirability to consider which states of the component’s environment (as viewed through the com- ponent’s role) are undesirable. That is, given our definition of undesirability from Section 5.1.1, what are the “worst case conditions” of our component’s environment associated with this system state and notion of unwanted, observable behavior? If our analyst is working with a full system, this step is fairly straightforward: it is undesir- able for the pump to have a nonzero ticket duration when the patient cannot tolerate more analgesic (i.e., the environment is in the abstract state “the pump should not run”). Similarly, because higher SpO2 means that our patient’s respiratory status
is healthier, it is undesirable for the pulse oximeter to overstate the patient’s actual blood-oxygenation level—understating the value may be unwanted for other reasons (e.g., avoiding underinfusion), but in this example we focus on overinfusion. If our analyst is working with a component in isolation, though, this step requires anticipat- ing a class of harm and considering how the component’s observable behavior could contribute to that harm. Pairs of the form (abstract component state, undesirable ab- stract environment state) that exist at the penultimate level of the system hierarchy are equivalent to the system’s hazards.
5. Finally, determine which interaction points (typically an element’s ports) carry in- formation about the component’s environment by using Definition 7: Environmental Awareness, and verify that it is possible for the component to “know” when it is in any potentially undesirable state of its environment using Definition 8: Avoidance. In order for a component to meet its safety-related goals, it is necessary2 for it to be informed of the state of its environment so it can avoid the specified notion of unde- sirability; here we ask the analyst to identify which incoming interaction points carry
2Assuming correct behavior and freedom from side-channel interference; Section5.5discusses a calculus
environmental information relevant to u. This is equivalent to identifying what the component requires from the environment, a common step in requirements-gathering methodologies, e.g., “2.2.4 ‘Choose Monitored Variables’ ” in [104] or determining the members of the vector
˜i
t in [94].
Having completed this process, our analyst will know the subset of her component’s interaction points that need to be connected in a fully instantiated system in order to avoid u. That information can then be used to either argue that a composed system (in which every component has been similarly analyzed) is safe (Section 5.4), or to consider what happens when the required information is not provided (Section 5.5). After this process has been completed once, the resulting requirements for avoidance of u (i.e., Im
u) can be used to
argue that a component’s safety-related goals are met when it is reused in other systems.