Modified Delphi study to determine optimal data elements for inclusion in an emergency management database system

(1)

Modiﬁed Delphi study to determine optimal data elements

for inclusion in an emergency management database system

Enqueˆte Delphi modifieé afin de de´terminer les e´le´ments de

donne´es optimales en vue de leur inte´gration a` un syste`me de

base de donne´es de gestion des urgences

A. Jabar

a,

*

_{, LA. Wallis}

b

_{, A. Ruter}

c,d

_{, WP. Smith}

b

a

School of Child and Adolescent Health, University of Cape Town, South Africa

b

Division of Emergency Medicine, University of Cape Town, South Africa

c

Centre for Teaching and Research in Disaster Medicine, Faculty of Health Science, Department of Clinical and Experimental Medicine, University of Linko¨ping

d_{Sophiahemmet University College, Stockholm, Sweden}

Available online 24 October 2011

KEYWORDS Database; Emergency centre; Hospital beds; Delphi; Management

Abstract Objectives: Information and communication technologies (ICT) are introduced into organisations with the goals of managing resources, increasing efﬁciency and work productivity and reducing workload. The aim of this study was to identify hospital institutional capacity indica-tors to provide recommendations to an existing emergency management database system operating in Cape Town, the Western Cape Province of South Africa. Using these indicators, this study seeks to augment and update the existing emergency database system.

* Corresponding author. Address: 8 Cavendish Place, Carbrook Avenue, Claremont 7708, Western Cape, South Africa. Tel.: +27 855579612.

E-mail address:[email protected](A. Jabar).

doi:10.1016/j.afjem.2011.10.009

Production and hosting by Elsevier

African Federation for Emergency Medicine

African Journal of Emergency Medicine

www.afjem.com

(2)

Methods: A modiﬁed Expert Delphi study consisting of two rounds was conducted by email. A panel of 16 experts drawn from the ﬁelds of emergency medicine, critical care, trauma surgery and disaster medicine were consulted. Participants were initially asked to propose hospital institu-tional capacity indicators that warranted inclusion in the emergency management database system currently operating in Cape Town, South Africa. In the second round these proposals were collated and scored using a 7 point Likert scale.

Results: Round 1 comprised 237 statements. Consensus was deﬁned a priori to be >80%. A total of 59 of 237 statements had reached consensus upon completion of the Delphi study. This repre-sented 24.5% of the total number of statements. Of these 19 reached consensus at >90% and 40 reached consensus at >80%. Subheadings for proposed indicators included stafﬁng speciality cat-egories, hospital equipment and services and special hazard/circumstance services Examples of accepted indicators include theatre availability, ICU surge and ventilator capacity and the availabil-ity of Chemical Biological Radiological Nuclear (CBRN) Decontamination services.

Conclusion: The use of a modiﬁed Expert Delphi study achieved consensus in aspects of hospital institutional capacity that can be translated into practical recommendations for implementation by the local emergency management database system. Additionally, areas of non-consensus have been identiﬁed where further work is required. This purpose of this study is to contribute to and aid in the development of this new system.

Abstract Objectifs: Les technologies de l’information et de la communication (TIC) sont introdu-ites dans des organisations afin de ge´rer les ressources, d’augmenter l’efficacite´ et la productivite´ du travail et de re´duire la charge de travail. L’objectif de cette e´tude consistait a` identifier des indica-teurs de capacite´ institutionnelle des structures hospitalie`res afin de fournir des recommandations a` un syste`me de base de donneés de gestion des urgences ope´rant au Cap, dans la province du Cap occidental en Afrique du Sud. En utilisant ces indicateurs, cette e´tude vise a` de´velopper et mettre a` jour le syste`me de base de donneés de gestion des urgences existant.

Me´thodes: Une enqueˆte Delphi modifieé consistant en deux phases a e´te´ conduite par email. Un panel de 16 experts issus du domaine de la me´decine d’urgence, des soins intensifs, de la chirurgie traumatologique et de la me´decine des catastrophes a e´te´ consulte´. Il a d’abord e´te´ demande´ aux participants de proposer des indicateurs de capacite´ institutionnelle des structures hospitalie`res qui garantissent l’inclusion du syste`me de base de donneés de gestion des urgences ope´rant au Cap, en Afrique du Sud. Au cours de la seconde phase, ces propositions ont e´te´ rassembleés et not-eés selon une ećhelle de Likert en 7 points.

Re´sultats: La phase 1 comprenait 237 dećlarations. Un consensus a e´te´ de´fini a priori comme e´tant >80%. Au total, 59 dećlarations sur 237 ont atteint un consensus apre`s ache`vement de l’e´tude Del-phi. Cela repre´sentait 24,5% du nombre total de dećlarations. Sur ces dećlarations, 19 ont atteint un consensus >90% et 40 un consensus >80%. Les sous-titres des indicateurs propose´s incluaient des cate´gories de spećialite´ de personnel, des services et e´quipements hospitaliers et des services spećia-lise´s dans les situations de danger/les circonstances particulie`res. Ces indicateurs accepte´s incluent par exemple la disponibilite´ d’une salle d’ope´ration, la capacite´ d’intervention et de respirateurs mobilisables de l’USI et la disponibilite´ de services de dećontamination chimique, biologique, radio-logique et nucleáire (CBRN).

Conclusion: L’utilisation d’une enqueˆte Delphi modifieé a permis d’atteindre un consensus sur cer-tains aspects de la capacite´ institutionnelle des structures hospitalie`res, pouvant eˆtre traduit en recommandations pratiques en vue d’une mise en œuvre par le syste`me de base de donneés de ges-tion des urgences local. De plus, des domaines de non consensus ont e´te´ identifie´s pour lesquels un travail supple´mentaire est requis. L’objectif de cette e´tude consiste a` contribuer au de´veloppement de ce nouveau syste`me.

(3)

What’s new

The study has identiﬁed 59 more variables for the further development and augmentation of the current hospital bed bureau database system.

The study has identiﬁed several limitations of the current system, as well as advocating further research into the com-pliance of this technology with the existing system. The study advocates more research into Disaster ICT

emer-gency systems in South Africa with the intention of building a system that operates contiguously through the nine provinces.

African relevance

One of the development objectives this study concentrates on is leveraging health care resources and promoting social capital and services through better health care informatics. In the African context it is vitally important to promote the use of and access to ICT for improved equity and social and economic development.

The new system is a step towards introducing a universal ICT emergency management database in South Africa and there-after to replicate the model in other African countries. Introduction

The ability of health care staff to view available hospital re-sources on a computer screen as a live time tool and thereby make better informed decisions with regard to patient care is a necessity for a health system that is understaffed with limited resources. Communication is essential for effective major inci-dent management without which the command and control and safety structure would ultimately collapse[1]. In the South African context of a limited resource setting, the development of an emergency management database system will be a signif-icant technological achievement in the new millennium. Mass casualty incident – A deﬁnition

‘An incident in which emergency medical services personnel and equipment at the scene are overwhelmed by the number and severity of casualties at that incident[2]’. If casualties in major incidents are to receive the best possible care then qual-ity planning and preparations is essential[3]. Information and communication technology (ICT) employed by the Bed Bureau allows one the opportunity to quantify, survey, and control these incidents. It does this by creating real time updating and data collection, as well as a data presentation interface shared by the stakeholders in order to deﬁne and better allo-cate available resources. This requires up to the minute data generation and input for it to be useful and accurate. It is imperative that this holistic approach be taken when dealing with disaster management and improving efﬁciencies. The development of an emergency management database system may be used to address the stresses placed on a daily basis on the health system so that the system enjoys daily employ-ment and service whilst being readily available for use follow-ing a mass casualty incident (MCI). This will ensure a resource

informed Emergency Medical Service (EMS) that delivers quality health care to the citizen.

The development of the Hospital Bed Bureau

The current and future capacity of South Africa to generate and sustain access to ICT for its citizens is a development pri-ority. The Bed Bureau was developed for the FIFA 2010 World Cup so that the designated hospitals could upload their bed status online. This information was then accessible at the EMS Control Centre. Thus at a glance the EMS controllers could direct ambulance to those hospital that have the re-quired capacity. The project was piloted during the World Cup event and subsequent roll out to facilities is being ear-marked for later this year. The initial pilot only included the 12 designated FIFA hospitals situated in the metropolitan area of Cape Town – This included both state and private facilities – as well as the military hospital. Operation of the database system was limited to the Western Cape for the World Cup.

The hospital Bed Bureau is an electronic emergency man-agement database system that was developed to ensure optimal use of hospital beds during the 2010 World Cup[4]. Currently the system shares a single variable within the network: that of bed count, which looks at the number of beds available in the following departments: ICU Maternity Paediatrics Emergency Centre Medicine Orthopaedics Surgery

The individual department bed counts are further divided into:

Male/Female beds High dependency beds

With the exception of the Emergency Centre which is di-vided into:

Resuscitation beds available Trolleys available

Patients waiting

The system uses SQL (Structured Query Language), which is a database computer language[5] as the database support structure. The system also uses Microsoft Silverlight as the development platform.

It is a very simple system involving updating of bed avail-ability to as close to real time as possible. This is made achiev-able on account of it being web-based. The author cannot comment on functional speciﬁcations and complete system overview as the intellectual property belongs to the parent company.

The system is currently online and functional and login ac-cess is limited to hospital and technical support staff involved in the management and updating of the system. The results of this study have been submitted to the Department of

(4)

Emergency Medicine in the Western Cape. No changes have been made to the existing system in accordance with the results of this study at this time.

Methods

A two round modiﬁed expert Delphi study was conducted be-tween November 2010 and March 2011 using a panel of 16 ex-perts from specialties involved in the management of patients in major incidents. This included two Provincial heads of Emergency Medicine, two University Heads of Department of Emergency Medicine, one Hospital Head of Department of Emergency Medicine and one Paediatric Head of depart-ment of Paediatric Emergency Medicine. This was to ensure that decisions were made by persons in senior posts, so that subsequent implementation and recommendations would be eased. The views of all participating experts were given equal weight. Other members of the Delphi panel included represen-tatives from critical care medicine and trauma surgery.

Delphi is a structured process that uses a panel of experts to investigate a complex or imprecise issue using a series of struc-tured statements.

The process occurs in three stages:

Stage 1. A panel of experts formulate a series of ideas pertain-ing to the subject in question. This is done individually and anonymously.

Stage 2. The statements from stage 1 (Table 1) are collated and sent to all members of the expert group. They indicate their level of agreement with each statement using a Likert scale.

Stage 3. Each statement is fed back to the panel with their own and the rest of the panel’s previous opinions. All feed-back is anonymous. Numerous iterations may be necessary.

The ﬁrst round of the study asked the panellists to consider the aspects of hospital institutional capacity that should be represented on the emergency management database system. They were asked to propose indicators under the broad sub-headings shown in the box, in order to reduce the chance of indicators being overlooked.

Their replies were collated into a series of statements. In the second round these statements were returned to the panel members in the form of a series of statements about which they were required to express their level of agreement with the use of the proposed measure as a performance indicator. This was done using a 7 point Likert scale[6].

Positive consensus was defined priori as 80% or more of respondents scoring 6 and above, with this value being used to produce final recommendations. Negative consensus was defined as 80% or more of respondents scoring 3 and below.

The second and ﬁnal round comprised 237 statements. Each of the statements was presented with a summary explain-ing the reason for the panellists’ choices. This allowed group members to adjust their response in light of group opinion.

Comments or concerns that had been expressed by panel-lists in the ﬁrst round were also added to the round two ques-tionnaires. Areas that had not reached consensus in the Delphi study, i.e. a score of 4 and 5, were not considered further in analysis. (see appendix for all statements.)

Results

Sixty seven experts met the inclusion criteria and were invited to participate in the Delphi study; 22 responses (33%) were re-ceived. Six of these respondents did not complete the survey, meaning a total of 16 experts (24%) completed all rounds and comprised the Delphi panel.

Round 1 produced a series of 237 statements that were then returned to the panellists. Consensus was deﬁned a priori to be >80%, a standard commonly used for a modiﬁed expert Del-phi study limited to twenty people. After round two 59 indica-tors had reached positive consensus. This represented 24.5% of the total number of statements. Of these 19 reached consensus at >90% and 40 reached consensus at >80%. 178 statements had reached no consensus. No statements reached negative consensus. The 59 indicators reaching consensus as having good potential for inclusion in an emergency management database system are shown in Table 2. The remaining state-ments are not presented here.

In the equipment and services section, critical care services were deemed important by the panel and included respiratory supplies, ICU ventilator capacity and operating theatre capacity. Radiology in this section reached close to 60% negative consensus. Though no statements in the study reached negative consensus, the radiology statements produced one of the highest negative consensus percentages. This suggests the choices were made in consideration of an MCI. Following an MCI, critical life saving equipment such as ventilators and services such as the operating theatre would be essential in reducing patient mortality. This is clearly recognised in the panel’s choice. Choices by the panel in previous sections indicated the sole interest in radiology was the availability of CT scan facilities. Following a MCI with multiple head injury victims, CT scan services are required to determine the presence of intra-cerebral haemorrhaging which will determine type of interventional therapy and possible transfer to a higher care centre.

The implications of these patient mortality rates driven choices, shape the database system into a tool that is struc-tured towards lowering patient mortality. While the system also seeks to bring more efﬁciency to the rescue progress, the overall beneﬁts of augmenting the existing system become clear. The Hospital Bed Bureau currently is a tool used to re-port live status bed capacity and co-ordinates the rescue sys-tem response. The potential for the further development of this system is expansive. By providing further data elements to record and make more informed decisions, the system can use its own recorded data to analyse and regulate its own development.

Table 1 Subheadings from proposed indicators in round one.

Hospital departments Hospital departmental features Hospital staging areas Operational levels of hospital Stafﬁng factors

Stafﬁng speciality categories Hospital equipment and services Special hazard/circumstance services

(5)

Table 2 Proposals reaching consensus for inclusion in an emergency management database system. Hospital departments: 1. Surgery 2. Emergency Centre 3. Medicine 4. Paediatrics 5. Theatre 6. Burns unit 7. ICU/HCU 8. Paediatric ICU Departmental features:

1. Size of Emergency centre and percentage occupancy 2. Number of beds/trolleys available and capacity 3. Number of resuscitation beds

4. Number of patients unseen and triage colour 5. Number of minor and major beds

6. Current Operating Theatres available and in use 7. Medical beds available

8. Total number of high care/ ICU beds and availability 9. Stafﬁng of ICU (including surge staff)

Hospital staging areas:

1. Transportation staging area – Emergency services know where transport needs to go and how route/entry/exit changes in a major incident 2. Alternate care site – Capacity building determines needs for long-term planning. Personnel know where to refer patients

3. CT Scan – Head injury imaging, patient will need to be referred if no CT facility available 4. Wards – Bed status and stafﬁng capacity

5. Operating theatre – Capacity and capability, particularly Emergency Theatres 6. ICU – Number of available beds

7. Surgery – Capacity and capability including bed availability 8. Medicine – Capacity and capability including bed availability 9. Command centre – Is there command control awareness 10. Resuscitation room – Need to know capacity/stafﬁng/resources 11. Isolation room – Location, capacity

12. Decontamination – Essential following a chemical, radiological, biological or nuclear event Operational level of hospital:

Full service – fully operational hospital

Partial service – emergency services operating, speciﬁed elective services suspended Emergency service only – Only emergency services operating, all elective services suspended Staﬃng Factors:

1. Number of staff on active duty

2. Number of emergency and/or trauma staff 3. Number of critical care staff

4. Surgical capacity

5. Ability for expansion – standby staff, disaster call out staff etc Hospital equipment and services:

1. Pharmacy: Emergency access facility

2. Respiratory supplies: Mechanical ventilation available 3. Respiratory supplies: Portable oxygen available 4. Communications – radios on site, tracking, VHF 5. Theatre – availability/emergency list

6. ICU – Surge capacity/ventilator capacity Special circumstances (availability of services):

1. Chemical Biological Radiological Nuclear(CBRN) – Decontamination service 2. Infective agents/biological decontamination service

3. Staff PPE availability

4. Power generation – Adequate backup if required 5. Disaster Plan – internal and external present 6. Burn beds availability – ICU and high Care 7. Management – Activation and plan management

8. Mass casualty area for large incoming patients, area for triage, area for arriving family 9. Helicopter landing response and helipad facilities

(6)

In the staffing factor section, emergency and trauma staff attained 100% consensus. Only five staffing factor choices at-tained consensus in comparison to the sixteen chosen for the departmental features and eleven chosen for Hospital staging areas. This may be interpreted as a reflecting the panellists’ placement of importance on infrastructure and hospital ser-vices over human factors. Emergency physicians are used to working in understaffed conditions. It was possibly felt that staff shortages could be handled, however the omission of hos-pital services would result in critical loss of the hoshos-pitals’ abil-ity to fully function and maintain qualabil-ity patient care.

Notable non-consensus statements in staffing factors were non-emergency related factors which included level of training, handover times, locum staff details and staff rosters. This sug-gests that factors that do not hinder staffing practical functions were not of importance to the panel. This is consistent with the emergency themed choices made by the panel which placed va-lue on options that were likely to influence patient care.

Although there was no consensus from the staffing specialty category, the addition of slightly agree (5) to the moderately agree (6) and strongly agree (7) categories produced >80% consensus with regard to four options. These options were medical officer, staff nurse, emergency physician and surgeon. This suggests that although the panel could not reach consen-sus on specific staff specialty definitions, they recognised the need for the basic staff specialties to be recognised and imple-mented in the system.

Discussion

The panel’s overall choices are in keeping with an emergency medicine based principles. As an emergency physician relies

on being fed rapid data by first responders bringing in a pa-tient to the emergency centre, the physician has to filter the rel-evant data in their mind, extracting the most essential data in order to make the best informed decision in their choice of medical therapy. The results of this study replicate this thought process in that the choices reflect the most important clinical and situational data elements relevant to improving decision making by command and control, thereby reducing patient mortality rates and increasing efficiency of transfer of patients to appropriate referral centres.

Currently, no speciﬁc standards exist for disaster communi-cations systems [7]. This is true for the developed as well as developing world. One reason for this is that individual coun-tries have each varied resource limits and funding towards disaster management services. Consequently emergency man-agement systems are developed to meet the resources of the country it is used in.

Research into Disaster ICT development ensures an evi-dence based approach into the understanding and provision of optimal data for inclusion in these systems. It does so by providing expert inﬂuence in choosing the data as well as encouraging the regulation of disaster terms thereby attempt-ing to brattempt-ing conformity to the language of disaster medicine.

The panellists were chosen to represent a varied number of viewpoints and the use of the modiﬁed expert Delphi technique allowed the panel to express their views anonymously. Consen-sus could thus be sought without prejudice and interpersonal relationships introducing bias. Panellists were also allowed the opportunity to change their minds once they had seen the opinions of the rest of the group.

The Delphi process does however have several limitations

[8]. First, the deﬁnition of expertise is subjective and relies

(7)

upon the leading researcher and advisors knowing who the po-tential experts in the ﬁeld are. Second, although 237 indicators were proposed in round one, it is possible that important issues might have been overlooked. To reduce this, the panel were asked to propose indicators under broad subheadings shown in the box with every statement being reproduced in the second round for the entire panel to vote on. The panel also included experts from ﬁve specialities to provide a broad knowledge and expertise base. Third, although the group appears to have achieved consensus on many statements, this does not neces-sarily mean agreement. Delphi group members who are tired or bored with the process might shift towards consensus to stop the process.

The proposed indicators for the ﬁrst round reﬂected the four Ss of surge capacity i.e. structure, staff, stuff (equipment) and systems [9]. Special circumstances encompass all four 4 groups and are included in established emergency management databases systems in the developed world, including the UK, US and Australia, who make provisions for this in their sys-tems emergency protocols. Special circumstances were consid-ered an important inclusion by the author, as they demand the availability of specialist services without which would result in higher patient mortality and morbidity.

The results of this study can be used as a basis for further work. A widespread accurate system for data collection needs to be in place. Currently the data collection is limited to bed capacity and sub headings within this category. Signiﬁcant investment in resources and time is required for this to occur. More practically, many of these measures can be applied to the development of emergency management database systems in other provinces. The modiﬁed expert Delphi study provides a starting point for the development of indicators within the speciality of Disaster ICT. The study provides some of the fol-lowing recommendations:

Further areas of research, including a pilot study into the use of wireless PDA technology for ﬁrst responders have been identiﬁed, as well as further research into the compli-ance of this technology with the existing system.

A National Committee should be formed to review existing emergency management database systems development, and to implement the use of emergency database systems in other provinces.

EMS should encourage and reward computer literacy amongst staff. This will ensure familiarity with the system amongst all staff, should primary users not be available, and encourage acceptance of system by users.

Finally the author recommends the renaming of the Bed Bureau emergency management database system to better reﬂect the different data elements the system provides beside bed capacity.

The results of this study are currently being used to develop new emergency management database software with the inten-tion of a pilot study later this year (Fig. 1).

Data analysis

A Microsoft Excel database was created for this study, with simple summary and descriptive statistics being used.

Ethics

Ethical clearance for this study was approved by the Health Sciences Research Ethics Committee of the University of Cape Town.HREC REF: 021/2011.

Appendix A. Short Answer Questions

Test your understanding of the contents of this original paper (answers can be found at the end of the regular features section)

1. In which South African city is the hospital bed bureau employed? a) Johannesburg b) Durban c) Pretoria d) Port Elizabeth e) Cape Town

2. What single variable does the current emergency manage-ment database system focus on?

a) Hospital departments b) Stafﬁng factors c) Equipment and services d) Operational level of hospital e) Bed count

3. In which section of the Delphi study was there no consensus reached?

a) Hospital staging areas

b) Hospital departmental features c) Special circumstance services d) Equipment and services e) Stafﬁng specialty category

References

1. Smith W, Wallis L. Disaster Medicine, Cape Town, Juta 2011;46. 2. Hick JL, Barbera JA, Kelen GD. Reﬁning surge capacity:

conven-tional, contingency and crisis capacity. Disaster Medicine and Public Health Preparedness2009;3(1):59–67.

3. Service NH. Emergency planning in the NHS: health services arrangements for dealing with major incidents, HMSO, Editor. Lon-don: NHS Health council; 1990.

4. S. Writer (2010) Electronic bed bureau system for WC. Cape Argus,

http://www.project2010.co.za/2010_World_Cup_technology.asp. 5. D.D. Chamberlin, R.F. Boyce, SEQUEL: A Structured English

Query Language, in Proceedings of the 1974 ACM SIGFIDET Workshop on Data Description, Access and Control (Association for Computing Machinery). 1974.

6. Likert R. A technique for the measurement of attitudes. Methods Psychology1932;22:25.

7. Rothman RE et al.. Research Priorities for Surge Capacity. Academic Emergency Medicine2006;13:1160–8.

8. CM G. The Delphi technique: a critique. Journal of Advanced Nursing1987;12:729–34.

9. Barbisch DF, Koenig KL. Understanding surge capacity: essential elements. Academic Emergency Medicine 2006;13:1098–102.