National Programme for IT

Safer Design – Key Clinical Safety Activities

National Programme for IT

Safer Design

FOREWORD 3

1.0 INTRODUCTION 4

2.0 OvERvIEW 5

3.0 DESIGN ACTIvITIES 7

3.1 Patient Safety Risk Assessments and Risk Register (PSRA) (RR) 7

3.2 Clinical Systems Safety Case 8

3.3 Closure Report 8

3.4 Key Activities for Design Phase Checklist 8

4.0 BUILD ACTIvITIES 9

4.1 Change Management (Review and Revise PSRA & RR) 10

4.2 Clinical Safety in Building the IT Aspects of the Solution 10

4.3 Preparing Non-IT Aspects (manuals, training documents, etc.) 11

4.4 Key Activities for Build Phase Checklist 11

5.0 TEST ACTIvITIES 12

5.1 Verification of Safety Features and Functions 12

5.2 Safety Closure Notice and Clinical Authority To Release 13

5.3 Handover 13

5.4 Key Activities for Test Phase Checklists 13

6.0 QUALITY ASSURANCE 14

APPENDIX I Design Phase Activities Checklist 16 APPENDIX II Build Phase Activities Checklist 17 APPENDIX III Test Phase Activities Checklist 18

Foreword

NHS Connecting for Health (NHS CFH) champions patient safety through the application of well designed software and resilient IT systems, and through its support for safe design initiatives.

As NHS CFH’s National Clinical Director for Safety, I have seen the success we have had in improving the design and implementations of our systems. To date, there have been approximately 600 clinical incidents NHS CFH has handled. Although none of these have caused any significant harm to patients, they resulted in delays in treatment and, if some had developed differently, the potential for harm was present. We must, therefore, maintain our focus and attention on clinical safety now and in the future.

To aid those designing and implementing software solutions for use in healthcare organisations, we are creating a set of guidance documents, and delivering training. The aim is to provide those involved with the solutions with the knowledge and skills to enable them to ensure clinical safety.

This guidance document focuses on the key activities which need to be undertaken for the ‘safer design’ of software solutions. It draws on the experience and learning from previous design initiatives. As a short and simple document for all those leading software design activities in their trusts and other healthcare organisations, I recommend it to you.

Dr Maureen Baker CBE DM FRCGP NHS Connecting for Health

Clinical Director for Safety [email protected]

1.0 Introduction

This document outlines the activities that need to be considered when designing and developing NHS National Programme for Information Technology (NPfIT) systems to be applied in a clinical environment.

It provides an overview and ‘checklists’ of key activities NHS CFH recommend manufacturers of healthcare software include in the design and development of their IT solutions.

The activities described are part of a typical manufacturer’s Clinical Safety Management Approach and will help ensure that all NHS CFH solutions delivered, deployed and operated in hospitals, clinics, GP practices, pharmacies, nursing homes and other health and social care environments, to support the provision of safe care for patients.

Good clinical safety management is essential to providing safer care for patients. The aim of clinical safety management, during the design and development of a NPfIT solution is to ‘ensure the safer manufacture of systems’.

Who is it for?

This guidance is for any member of the team, clinical or non-clinical who may be involved in any aspect of the design and development of a NPfIT solution. It is written on the basis that manufacturers have their own quality and safety management systems, and that there will be a person with ‘clinical/patient safety’ responsibility in the organisation and in the design project’s senior management team.

It is expected that their systems will include formal documentation and quality assurance, design and peer review, configuration control, as well as risk assessment requirements.

Note: Throughout this document the NHS CFH term ‘clinical safety’ is used, this is equivalent to the term ‘patient safety’ which is commonly used by many manufacturers and healthcare organisations.

2.0 Overview

Figure 1 depicts a simple waterfall model used for software development process.

Testing

Test

Build

Design

The key clinical safety activities are presented in Figure 2 are for the design, build and test phases. Each activity is covered in this guide. Not all the activities are mandated but they are seen as best practice for safer design. It should be noted that this document is not a methodology for running a project, its objective is to highlight key controls a manufacturer can action to assist in delivering ‘safer’ solutions.

Figure 2 Key safer design activities and deliverables

Safer Design

Design

Activities

_Activities

Build

_Activities

Test

Patient Safe by Assessment and

Risk Register

Safety Closure Notice and Clinical Authority to Release (CATR) Handover verification of Safety Features and Functions Change Management (Review/Revise Patient Safety Risk Assessment

(PSRA) and Risk register (RR) Clinical Systems Safefty Case Design Phase Closure Report Preparing Non-IT Aspects (Manuals, Training Docs., etc.) Preparing IT Aspects

Design is the phase of work where manufacturers:

Model clinical processes and use cases to form the high level design. Map out information flows through the clinical process and produce data

flow diagrams and database designs.

Identify all aspects of interoperability with other systems, and model these through the whole system.

Evolve technical design and the technical architecture based on the above activities.

The design phase’s three key clinical safety activities are:

Undertaking the Patient Safety Risk Assessment(s) and setting up the Risk Register (also called Hazard Analysis and Hazard Log by some

manufacturers).

Producing the Clinical Systems Safety Case for the solution. Producing the Design Phase Closure Report.

Note: As well as supporting the safe design of solutions, these deliverables are required by the NHS CFH’s Clinical Safety Group.

3.0 Design activities

3.1 Patient Safety Risk Assessments and Risk Register

Identify any patient safety risks in the clinical process (whether IT or technical risks, organisational risks, human error).

Determine those risks which, due to their potential consequences and the likelihood of these consequences, pose an unacceptable level of risk to patients1_.

Design mitigations which will be effective in mitigating these risks to an acceptable level.

Identify which of these mitigations are to be actioned, either in the design and development by the manufacturer or in the implementation phase by the healthcare organisation (HO).

Produce a framework to assist in managing and verifying that risk controls are included and are effective in the solution as deployed.

1

Note: There needs to be careful consideration of what is an acceptable level of risk that the population can be exposed to (i.e. those following the clinical pathways supported by the IT solution) and this needs to be justified in the Clinical Systems Safety Case.

3.2 Clinical Systems Safety Case

The objectives of the Clinical Systems Safety Case (CSSC) are to: State the controls that mitigate the identified risks.

Present systematic arguments on the safety of the system.

For example, a typical safety argument for an electronic prescribing system could be that messages are important as a message failure or corruption could result in a patient:

Not receiving their medication. Receiving the wrong medication. A delay in receiving their medication.

The system is therefore designed to identify whether messages are not received and / or not verified.

Explain how the safety of the system will be demonstrated. For example - a safety case may propose that:

System reliability will be evidenced by hours of operation during testing, to show that the mean time between failures achieves its target with, say, 95% confidence.

Messages are not lost, and transfer is robust. This will be achieved via a series of tests that shows there is a control that places

rejected messages and corrupted messages into a ‘dead letter’ box. The system is fault tolerant (i.e. corrupt messages cannot bring the system down) and this will be achieved via a series of tests during the test phase where corrupted messages are sent.

3.3 Design Phase Closure Report

The objectives of the Design Phase Closure Report are to:

Systematically start collecting design stage evidence for the key clinical safety risk controls as set-out in the CSSC.

Capture any emerging issues and ensure they are handled properly such that the solution remains in line with the safety arguments in CSSC.

3.4 Key Activities for Design Phase Checklist

The build phase is where manufacturers:

Construct and baseline the system release, packaging it so that it will be fit for external release. This includes implementing the controls in CSSC into the designed application.

Construct the non-software deliverables, for example, user guides, training manuals, release notes (with defects), etc.

The implementation phase’s three key clinical safety activities: Change management (review / revise PSRA and RR). Clinical safety in building the IT aspects of the solution.

Clinical safety in preparing non-IT aspects of the solution (manuals, training documents, etc.).

All these tasks will interact with each other. Change management is listed first, but could be seen as being embedded within the other activities. It is important to ensure within the build phase that all changes are identified and reviewed for their clinical safety implications, and that clinical safety features and functions are modified or added to maintain an acceptably safe solution. In building the software and preparing the non-IT aspects, the clinical safety activities are focused on:

Building and preparing clinical safety features and functions into the solution.

Verifying and documenting that the clinical safety features and functions are in place.

The build and preparation activities are typically undertaken by the core team, whereas the verification and collection of the evidence may be completed with or by someone from the manufacturer’s clinical safety or risk team.

4.1 Change Management (Review and Revise PSRA

& RR)

Changes that need to be managed can be changes to either IT aspects or non-IT aspects of the solution. The objectives for the change management (review and revise PSRA & RR) are to:

Ensure that there is a controlled document baseline in place against which changes can be identified.

Identify any changes or requests for changes against the baseline.

Determine if the changes result in defects in, or non-compliance with, the safety requirements.

Ensure that the risk with the changes is still as low as reasonably practicable (ALARP).

4.2 Clinical Safety in Building the IT Aspects of

the Solution

This section covers key clinical safety activities in building of the IT aspects of the solution. The objectives for these activities are to:

Develop the IT elements so that they include all the clinical safety features and functions required, and ensure that they have been correctly embodied in the solution. This ensures that they function as intended and deliver their clinical safety requirements.

Produce the verification evidence that the IT clinical safety features and functions have been included in the solution.

Produce the means for verifying the effectiveness of the IT clinical safety features and functions, that is, the test plans for these features and functions.

Ensure that the RR and the CSSC remain current, relevant and reflect any changes.

Ensure that the IT aspects progress through the governance gates in the required manner and in accordance with the acceptance criteria (as detailed on the project management / recognised project management framework product sheets) and collate the verification evidence.

Provide NHS CFH’s Clinical Safety group with relevant documentation in a timely manner.

4.3 Preparing Non-IT Aspects (Manuals, Training

Documents, etc.)

This section covers key clinical safety activities in preparation of the non-IT aspects of the solution. The objectives parallel those for the IT aspects. The objectives for the non-IT activities are to:

Develop the non-IT elements so that they include all the clinical safety features and functions required, and ensure that they have been correctly embodied in the solution.

Produce the verification evidence that the non-IT clinical safety features and functions have been included in the solution.

Ensure that the RR and the CSSC remain current, relevant, and reflect any changes.

Ensure that the non-IT aspects progress through the governance gates in the required manner and in accordance with the acceptance criteria (as detailed on the project management / recognised project management framework product sheets) and collate the verification evidence.

Provide NHS CFH’s Clinical Safety Group with relevant documentation in a timely manner.

4.4 Key Activities for Build Phase Checklist

5.1 Verification of Safety Features and Functions

Demonstrate, where verifiable through testing, that the solution’s clinical safety requirements from the CSSC are met.

Identify any issues of concern arising out of the test phase. Summarise all the test phase verifications.

Compile all design and test phase verifications as input to the Safety The test phase is where the manufacturers:

Test the solution to ensure that it delivers the functionality that is intended. This includes testing the safety features and functions.

Test the solution to demonstrate that it is reliable and resilient. Test and demonstrate messages are not lost and transfer is robust. Test and demonstrate the system is fault tolerant.

As can be seen from the tasks above, the test phase is where much of the evidence to demonstrate the safety arguments is produced, confirming that the solution is fit for purpose.

When defects are detected and taken forward with work arounds - or changes, these should be selected and reviewed for their clinical safety risk, and further changes made to ensure they are clinically safe.

The test phase’s three key clinical safety activities: Verification of Safety Features and Functions.

Production of the Safety Closure Notice and gaining Clinical Authority to Release (CATR).

Handover to those responsible during the implementation and use of the solution.

5.2 Safety Closure Notice and CATR

The objectives in producing the Safety Closure Notice (SCN) are to:

Report and present to NHS CFH all clinical safety information to enable their decision to allow the solutions to be released (from the clinical safety view) thereby gaining the Clinical Authority to Release (CATR) with details of any caveats which NHS CFH wish to include.

Incorporate caveats into the solution and its implementation.

CATR may require caveats which are conditions which NHS CFH’s Clinical Safety Group require to be part of a solution when giving a CATR. For example, where a solution still has a defect, the caveat may require that all implementing healthcare organisations are made aware of the defect and are trained in the work around.

5.3 Handover

The objective for the handover clinical safety activities are to ensure that the NHS CFH service management organisation and the implementing healthcare organisation that will use the solution understand:

The clinical safety risks for clinical and business process which use or are affected by the introduction of the solution.

The clinical safety controls and measures for the processes and the solution once in operation and use.

The manufacturer design organisation needs to be proactive in handing over the clinical safety aspects of their solution. They need to help the NHS CFH service management organisation, the manufacturer’s service management and the healthcare organisation(s) to understand:

The proposed new operational clinical and business processes, supporting systems, etc. This will allow them to verify that the new processes can be supported nationally and by the planned local configuration of the solution. The risks and controls with the new solution, so that they can manage these

risks through their own Risk Management Systems.

That operating outside the solution’s design envelope means there is the potential they could be operating at a level of risk which is greater than that in the manufacturer’s CSSC, and as approved via the NHS CFH’s CATR.

5.4 Key Activities for Test Phase Checklists

6.0 Quality assurance

Quality assurance refers to a program for the systematic monitoring and evaluation of the various aspects of a project, service, or system to ensure that standards of quality are being met. It is a process for assuring the development and delivery of high quality and clinically-safe IT systems.

Below are examples of safety incidents with potential for errors and harm to patients..

An important part of clinical safety is assuring the quality of clinical safety documents to prevent errors and potential harm to patients, a checklist can be found as Appendix IV.

Clinical Safety Design Event Failure to Preserve Values and Units

Formula for Dosage Calculation

On three occasions the Clinical Authority to Release (CATR) has been revoked by NHS CFH because, on release, it was found that volumes and units of measure were not preserved when downloading prescriptions.

The design and development of the dispensing systems had paid inadequate attention to rules for the interface specifications to the ‘Spine’ Electronic Transmission of Prescriptions (ETP) database and protocols for storing and handling qualities.

This led to problems, for example:

A failure to preserve values due to a failure to include a zero before a decimal point (i.e. to format as ‘0.x’ not ‘.x’) or for there to be issues with strengths (e.g. mg vs. μm).

One pack of 20 tablets being transcribed into 20 packs of tablets. Such problems have the potential to lead to errors and harm to patients.

A formula for calculating dosage instructions for local formulary drugs had been implemented by a primary care trust in a number of GP practices resulting in an unpredictable corruption of the dosage string.

The impact for the GP was that they were unable to: Print a prescription.

Send an initial GP summary to Summary Care Records. Unable to update Summary Care Records.

GPs have to manually correct the dosage instructions. Scope:

Formula for Dosage Calculation

Clinical Safety Design Event Delimiter Problem with Prescribing Software Issue resolution:

‘Made safe’ - the initial mitigation was for all affected users to manually enter the dosage instructions in free text. ‘Close’ - further recommendations saw the reload of the formula to fully correct the issue, only when fully

reloaded was the incident made safe. Preventative actions:

Clinical Safety Group made additional recommendations to the supplier to provide guidance on the use of local formulas by users.

Lessons learned:

Supplier engagement, making the supplier aware of intended changes to software. Risk of local changes to supplier tested systems.

A pharmacy system was piloted by a group of practices when it was found that the electronic prescription form, ‘FP10’, showed a ‘!’ prior to the quantities in dosage instructions on some prescriptions. The ‘!’ was not evident on the paper copy of the form.

Such a design error was recognised as a potential cause for an overdose drug error where, for example, a prescription 50ml drug (which would be presented as ‘!50ml’) could be read as 150ml.

NHS CFH’s Clinical Safety Group help desk were contacted. It was quickly discovered that the ‘!’ was in fact a

delimiter used to separate fields in tables and NHS CFH’s Clinical Safety Group were able to download all prescriptions from the Spine which exhibited this problem and highlight the ones which could potentially misguide a user.

Whilst the paper copy should be the legal entity for dispensing, it was recognised that the ‘!’ being present in the quantity field could constitute a clear risk. The pilot project was suspended immediately with full supplier cooperation.

Drug Text Qty Value Translation Display Name Dosage Instructions

Alendronic acid 70mg tablets 8 tablet AS DIRECTED ONCE A

WEEK

Alendronic acid 70mg tablets 4 tablet ONCE A WEEK

Insulin isophane biphasic human 20/80 AS DIRECTED 100 units/ml suspension for injection 45 cartridge !34uam/32upm NovoMix 30 Penfill 100 units/ml !20 UNITS BD/30 UNITS suspension for injection 3ml cartridges 30 cartridge ONCE

APPENDIX I - Design Phase Activities Checklist

Check

or X

Check Item

1 All preparatory information for the Patient Safety Risk Assessments (PSRA) has been completed:

a. Information to describe the solution collected and shared with the PSRA participants, (e.g. clinical process models, use cases, data flow models, etc.).

b. List of changes and specifically all changes in risk controls (between the current situation and the future with the solution) prepared.

c. List of all interactions between the solution and other (clinical and technical) systems and where these interactions are changed either in content or form prepared.

d. Patient safety incident reports for the clinical pathways relevant to the solution collected and analysis with key learning extracted.

e. Risk technique to be used determined.

f. Materials required for the assessment (e.g. record sheets, a risk matrix, etc.) prepared. g. Team for PSRA workshop identified and invited.

h. Team includes:

i. Subject matter experts (people who know the business / clinical processes, including users) ii. Technical personnel.

iii. An independent facilitator (i.e. someone from outside the solution’s project team). iv. A person to capture and record the workshop discussions.

2 PSRA completed and record produced. 3 Risk Register is fully updated and signed-off:

a. Output of the PSRA is put into the Risk Register (also called a Hazard Log) b. PSRA and Risk Register approved and signed-off internally

c. PSRA and Risk Register shared with NHS CFH’s Clinical Safety group

d. All comments received from NHS CFH have been addressed and the PSRA and Risk Register have been updated accordingly.

4 Recommendations and existing risk controls have been systematically reviewed and developed to a state where they can be designed into the solution and are verifiable.

5 Clinical Systems Safety Case (CSSC) fully drafted: a. Contains safety arguments.

b. Presents the overall risk of the solutions and demonstrates that it has been reduced to being as low as reasonably practicable.

c. Details the inherent safety features of the solution

d. Details how clinical safety will be assured through the lifecycle (the build, test, implementation and use phases) of the solutions

e. Explains how clinical safety will be demonstrated. 6 CSSC reviewed, approved and signed off:

a. CSSC is formally reviewed, approved and signed off by the manufacturer. b. CSSC is submitted to the NHS CFH’s Clinical Safety Group for review and sign-off.

c. All comments received from NHS CFH have been addressed and incorporated in the CSSC and the planned clinical safety activities.

d. CSSC is signed off by the NHS CFH’s Clinical Safety Group. 7 Design verification plan produced.

8 Formal handover of clinical safety at each step in the design process (all manufacturers will have their own design and development processes, it is, therefore, recommended that you create your own handover check list based on

APPENDIX II - Build Phase Activities Checklist

Check

or X

Check Item

1 Clinical Safety personnel involved in the change management process. 2 Configuration index audited.

3 Changes identified, their clinical safety impact determined, the change risk assessed, and the risk register updated. 4 Record of the clinical safety impact and control changes resulting from changes being maintained.

5 Code inspection / reviews completed and verification record produced.

6 Integration and interoperability activities reviewed to ensure that the external interfaces meet the clinical safety requirements.

7 Audit and log-in, validation, error handling, and other control aspects of the solution examined to ensure that they meet the clinical safety requirements.

8 Test strategy and scripts produced.

9 Test plans (including tests for safety features and functionality) produced.

10 Human factors / usability assessment to ensure that the interface, as designed, has not introduced any unintended risks.

11 Clinical safety have signed-off the IT aspects of the solution at the governance gates.

12 Training materials reviewed and verification record produced. 13 User manuals reviewed and verification record produced. 14 Help text reviewed and verification record produced. 15 Release note reviewed and verification record produced.

APPENDIX III - Test Phase Activities Checklist

Check

or X

Check Item

1 Test implemented in accordance with plans: a. Clinical safety tests completed.

b. Witnessing of required tests by clinical safety. c. Hands on testing by clinicians.

d. Corrective actions where test results do not meet requirements (corrective actions taken and closed out). 2 Test results collated.

3 Clinical safety have signed-off the test phase governance gates. 4 Formal clinical safety closure review with NHS CFH.

5 CSSC updated:

a. Documentation updated.

b. Safety justifications proved valid (with justified changes where required).

c. Risk is shown to be as low as reasonably practicable (with justified changes where required). 6 Formal review with NHS CFH Clinical Safety Officer.

7 Safety Closure Notice (with supporting materials) produced. 8 Submission to NHS CFH’s Clinical Safety Group.

9 CATR received.

10 CATR ‘caveats’ incorporated into implementation deliverables and plans.

11 Programme Delivery Board clinical safety presence.

12 Assurance delivered to the National Change Advisory Board (NCAB). 13 Formal handover requirements defined for handovers to:

a. NHS CFH service management organisation. b. Manufacturer’s service management organisation. c. Healthcare organisation(s).

14 Formal handover to:

a. NHS CFH service management organisation. b. Manufacturer’s service management organisation. c. Healthcare organisation(s)1_.

15 Final review and sign-off (closure) of the design risk register. 16 Implementation mobilisation.

1 _{Note that handover to healthcare organisations is covered in the NHS CFH guidance document ‘National Programme for IT Safer Implementation – Key Clinical}

APPENDIX Iv - Assessing Quality of Design and Development Safety

Documentation Checklist

Check

or X

Check Item

1 What is the Hazard Assessment technique used? Is it clearly defined with statements to show that there was appropriate clinical and technical engagement during the development (e.g. clinical personnel knowledgeable and experienced in the clinical process were involved and put in an appropriate amount of effort)?

2 Does the Risk Register (commonly called a ‘hazard log’ within the NHS CFH community) clearly relate clinical risks to the clinical / business process within the programme of work?

3 Wherever risks are deemed out of scope or unable to be mitigated, are there clear statements on reasons leading to the decision given.

4 Has an Accredited Clinical Lead with experience of NHS CFH Clinical Safety Management and risk management contributed to the development of the Patient Safety Assessment?

5 Has an Accredited Clinical Safety Lead, NHS CFH Technical Architect and the solution’s design and development Project Manager (or equivalent personnel) approved / signed-off the Patient Safety Assessment?

6 Does the Safety Case clearly present:

a. How risks were identified and assessed and provide a justification for using the methodology followed? b. Key clinical risks and mitigations (that are sound both clinically and technically) and there is a demonstration

that the risks have been reduced to being ‘as low as is reasonably practical’? c. Statements and evidence is presented on how this has been achieved and managed.

d. Details of how the supplier will continue clinical safety management of their solution, including how / when they will review the risk assessment, e.g. in the light of new information, incidents or changes in the solution or how it is to be used?

7 Has any transfer of risk been explicitly stated and agreed by recipient / end user representatives? 8 All clinical risks should have an appropriate level of mitigating evidence:

a. Are mitigations presented, e.g. design changes, technical assurance, etc.? b. Is there full traceability to original requirements for mitigations?

c. Are the mitigations dependent on the magnitude of the risk?

9 Have the suppliers Clinical Safety Lead and the relevant NHS SHA (or equivalent personnel) signed off the Safety Case? 10 Is there a clear statement which is based on the clinical risk management activities on the safety of the system,

and does this include references to detailed evidence based conclusions?

11 Is the transfer of any residual risk suitably documented with respect to the impact on clinical processes existing or introduced, limitation on functionality delivered, etc?

12 Are there any additional impacts on clinical processes and details on how it is proposed these are to be managed? 13 Has the System Architect and Test Manager reviewed the Safety Closure Report?

14 Has the Supplier’s Clinical Safety Lead and the relevant NHS SHA (or equivalent personnel) signed off the Safety Closure Report?

NHS Connecting for Health is the national organisation which is

Appendices are available for download from the NHS Connecting for Health website:

http://www.connectingforhealth.nhs.uk/safety