Bridging the Gap Between Public and Private
Healthcare: Influenza-like Illness Surveillance in a
Practice-based Research Network
Zsolt Nagykaldi, James W. Mold, Kristy K. Bradley, and John E. Bos
r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r
T
his article describes the development, testing, and implementation of the OKAlert-ILI System, a bidirectional, dual-use influenza-like illness surveillance and messaging system, during the influenza seasons of 2003–2004 and 2004–2005 in the Oklahoma Physicians Resource/Research Network, a primary care practice-based research network. We describe how the Oklahoma Physicians Resource/Research Network connected 30 primary care providers to the Oklahoma State Department of Health and how surveillance results were analyzed and fed back to the clinicians on a weekly basis. We demonstrate the timeliness, sensitivity, specificity, acceptability, validity, flexibility, and cost of the system. Finally, we describe upgrades and enhancements to the system based on user evaluation and feedback.KEY WORDS:influenza-like illness, practice-based research, public health, surveillance
Practice-based research networks (PBRNs) play a vi-tal role in translating research findings into practice by interfacing research and quality improvement in a
learning community.1 At the time of this publication,
there were more than 110 PBRNs in the United States.2
In addition, PBRNs are proving grounds for develop-ment, testing, and implementation of new ideas and
technologies that enhance primary care.3Most PBRNs
are affiliated with academic institutions (eg, medical schools and family medicine departments), and some also collaborate with local or state public health en-tities (eg, health departments and public health clin-ics). PBRNs offer unique opportunities for bridging the gap between the public and private healthcare sectors.
J Public Health Management Practice, 2006, 12(4), 356–364
C
2006 Lippincott Williams & Wilkins, Inc.
Enhancing the sensitivity and timeliness of infec-tious disease surveillance has become a priority for public health to rapidly identify illness patterns or trends that may signify a naturally occurring disease outbreak or a bioterrorism event. Traditional public health surveillance relies primarily on passive reports from hospitals, private healthcare providers, and labo-ratories. By tapping into an electronic sentinel physi-cian reporting system, the public health sector has the potential to increase the timeliness, sensitivity, and granularity of conventional disease surveillance while participating healthcare providers benefit from timely aggregated information about infectious diseases af-fecting their communities.
The Oklahoma Physicians Resource/Research Net-work (OKPRN) includes 235 primary care physicians scattered throughout Oklahoma. It is affiliated with the Department of Family and Preventive Medicine at the University of Oklahoma Health Sciences Center and
This study was partially supported by the Centers for Disease Control and vention Cooperative Agreement Number U90/CCU616982 for Public Health Pre-paredness and Response for Bioterrorism. The authors thank Dr Jim Cacy for his support at the University of Oklahoma Health Sciences Center, Department of Family and Preventive Medicine, Dan Hollacher and Wendy Zhou at Medical Data Solutions for assisting us with their expertise in system programming, and Dr Mike Crutcher, Commissioner of Health, Oklahoma State Department of Health, for providing support to this project.
Corresponding author: Zsolt Nagykaldi, PhD, Department of Family and Pre-ventive Medicine, University of Oklahoma Health Sciences Center, 900 NE 10th St, Oklahoma City, OK 73104 (e-mail: [email protected]).
q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Zsolt Nagykaldi, PhD,is Assistant Professor at the University of Oklahoma Health Sciences Center, Department of Family & Prevention Medicine, Oklahoma City. James W. Mold, MD, MPH,is Professor and Director of Research Division at the University of Oklahoma Health Sciences Center, Department of Family & Prevention Medicine, Oklahoma City.
Kristy K. Bradley, DVM, MPH,is Deputy State Epidemiologist, Oklahoma State Department of Health, Oklahoma City.
John E. Bos, MPH,is an Epidemiologist, Oklahoma State Department of Health, Oklahoma City.
FIGURE 1. OKAlert-ILI surveillance system. ILI indicates influenza-like illness; PDA, personal digital assistant; PC, personal computer.
with the Oklahoma Academy of Family Physicians. The network has been involved in a number of projects focused on improving chronic disease management and delivery of preventive services. Some have been done in collaboration with the Oklahoma State De-partment of Health (OSDH), including initiatives to increase the delivery of primary and secondary pre-ventive services, diagnosis and treatment of the dys-metabolic syndrome in primary care, and development of a cancer-reporting system.
In the spring of 2003, the OSDH partnered with re-searchers in the Department of Family and Preven-tive Medicine and OKPRN to develop a bidirectional influenza-like illness (ILI) surveillance and messaging system. During the summer of 2003, the OKPRN Health Information Technology team in cooperation with pro-grammers in a private company (Medical Data Solu-tions) developed the OKAlert-ILI System.
●
System Description
The OKAlert-ILI System has been developed in an open-source environment utilizing Apache Tomcat server 4.1.3 and Java Struts technology on SuSE Linux Professional 9.1. ILI data are captured in a PostgreSQL 7.3.4 database via a secure Web interface using OpenSSL 0.9.7b. The secure Web interface is uti-lized by the OSDH epidemiology staff to send OKAlert messages directly to OKPRN providers, to review and analyze ILI report data, and to provide feedback to participating sites on local and statewide influenza activity.
Alternatively, the system is accessible via Palm OS– based handheld devices. The OKPRN Health Informa-tion Technology team designed a Palm client applica-tion in NSBasic for Palm, a Visual Basic—like rapid de-velopment environment. The personal digital assistants (PDAs) are connected to an intermediate Microsoft SQL server 2000 database supporting our general research data collection system. This database is then ported to the open source PostgeSQL database to capture and store ILI messages synced from the handheld devices.
The PDA client includes a basic error-checking algo-rithm to ensure ILI data integrity. The client is able to track PDA users’ actions within the ILI system to de-termine daily handheld usage, options users selected to share health alerts with others, and possible errors users encountered. The PDA and the Web client prompt providers to answer three simple questions each day: (1) the number of patients with ILI symptoms seen that day, (2) the number of patients with ILI symptoms hos-pitalized that day, and (3) total number of patients seen that day.
●
Information Flow and System Operation
The information flow in the OKAlert-ILI System is shown in Figure 1. The OSDH epidemiology team sends a brief text message each week to the central database containing the surveillance summary for influenza and other infectious diseases. The server then pushes out these messages to the OKAlert-ILI Web site and also to handheld devices located in OKPRN practices. When
358
❘
Journal of Public Health Management and PracticeFIGURE 2. Sample personal digital assistant screen showing influenza-like illness criteria.
handheld users sync their PDAs, they receive the last three messages and are prompted to review and disseminate them in their practices. At a time set by the user, the PDA prompts the user to enter and sync ILI reports back to the central server. Consequently, incom-ing ILI data are populated and stored in the PostgreSQL database. ILI criteria is given according to the Centers for Disease Control and Prevention definition: a fever of
100oF or above PLUS cough and/or sore throat PLUS
myalgia in the absence of a known cause other than influenza (Figure 2).
Researchers at the Department of Family Medicine at the University of Oklahoma Health Sciences Cen-ter and OSDH epidemiologists continuously monitor the database and generate a report for further anal-ysis, feedback, and dissemination. The OKAlert-ILI’s Web site provides various extraction filters to populate ILI data, including clinic/user name and date range of reports. Data extraction and analysis is quick and convenient, because the flat file format allows a wide range of data analysis tools to receive and present ILI data. Analyzed system data are used by the OSDH to enhance its existing influenza surveillance system, which is based on weekly sentinel physician and labo-ratory reports. Surveillance results are then propagated to OKPRN practices via both OKAlert messages and e-mail listserv messages containing additional informa-tion, links, Centers for Disease Control and Prevention health alerts, and circulating respiratory virus activity reports.
●
2003–2004 Influenza Season Results
(October 2003–May 2004)
During the 2003–2004 influenza season, the OKAlert-ILI System received 15,428 individual patient encounter
reports from 30 volunteer OKPRN primary care clini-cians located in 15 counties. The distribution of par-ticipating clinicians is shown in Figure 3. Six percent of patient encounters (927 cases) resulted in ILI re-porting during the entire season. In 20 cases (0.1%), patients required hospitalization. The temporal distri-bution of ILI sentinel reporting during the peak in-fluenza activity of this season is plotted in Figure 4. ILI reports were submitted daily from Monday through Friday, except for some days during the Christmas holidays.
OKPRN sentinel ILI reports correlated with labo-ratory influenza test results reported to the OSDH. The OKAlert-ILI System contributed significantly to the timely detection of ILI cases and enhanced spa-tial resolution of the existing syndromic surveillance. Early analysis of the 2003–2004 season data indicated ILI cases appearing first in the southern part of the state with a subsequent northward shift. This finding cor-related with information from other sources. Timely recognition of this pattern made it possible for the OKAlert-ILI system manager and OSDH epidemiolo-gists to alert providers whose practices had not yet been significantly affected, who were then able to prepare for a substantial increase in the number of acute patient vis-its. Primary care providers particularly appreciated this information.
Figure 4 demonstrates the pattern of ILI activity during the 2003–2004 season. Approximately 4 weeks after a significant increase in ILI reports, ILI activity had already peaked. The 2003–2004 influenza season started significantly earlier than typical historical pat-terns. The first culture-confirmed case was reported on October 18, 2003.4Testing of referred isolates confirmed
influenza type A (H3N2) (both Fujian-like and Panama-like). No influenza type B was confirmed in Oklahoma in this season.
The OKAlert-ILI System detected an unusually high percentage of patients presenting with ILI symptoms at the peak of the season (average of more than 12% of visits attributable to ILI) and a rapid decline in ILI activity after approximately 4 weeks. A streaming video demonstrating the development and decline of the GIS-mapped ILI activity during the 2003–2004 sea-son is available at www.okprn.org. Screen shots from the video are shown in Figure 5.
Timeliness and enhanced granularity were espe-cially demonstrated in one practice location where a nurse practitioner in a residency clinic, who had pri-marily been seeing pediatric patients, began consis-tently reporting ILI cases. However, other providers in the same clinic who were seeing mostly adults were not reporting a significant increase in ILI cases in the early stage of the epidemic (first 2 weeks of Novem-ber 2003). The OSDH contacted the provider via the OKAlert system and inquired about the discrepancy.
FIGURE 3. Distribution of participating Oklahoma Physicians Resource/Research Network practices in Oklahoma.
They were able to determine that the significant report-ing difference was due to distinct practice patterns of the providers with regard to seeing separate age groups at the same location. This finding suggested that in the early stage of the 2003–2004 epidemic, circulating in-fluenza strains were mostly affecting young children. It also underscored the importance of enrolling sepa-rate providers at a larger practice site without pooling their ILI report data.
●
2004–2005 Influenza Season Results
(October 2004–April 2005)
During the 2004–2005 season, the OSDH received 33,437 individual patient encounter reports from 31 OKPRN primary care providers located in 15 counties. Three percent of patient encounters met the ILI criteria (1,114 cases) during the entire season, and 19 patients (0.06%) were hospitalized because of ILI. Sentinel reporting data for the 2004–2005 season is plotted in Figure 6.
Figure 6 demonstrates the distribution of sentinel ILI reporting. In contrast to the 2003–2004 season data, the curve is somewhat multimodal and ILI activity peaks in mid-February, much later than during the 2003–2004 season. Additional fluctuation before and after the peak is likely attributable to a changing mix of circulating res-piratory viruses. Retrospectively, these included ade-novirus, parainfluenza virus, influenza types A and B, and respiratory syncytial virus based on sentinel laboratory reports to the OSDH during the 2004–2005 season.
●
Timeliness of ILI Reporting
Analysis of ILI data demonstrated the timeliness of ILI reporting. The average lag time between seeing and re-porting ILI cases to the OSDH by providers was only 1.8 days (approximately 44 hours). Timeliness was due to a rigorous practice of daily reporting, offering mul-tiple ways to report (via the PDA, a personal computer
360
❘
Journal of Public Health Management and PracticeFIGURE 4. Influenza-like illness sen-tinel reporting in the Oklahoma Physi-cians Resource/Research Network (2003–2004 influenza season).
FIGURE 5. Screen shots from the 2003–2004 influenza-like illness movie. The size of dots is proportional to the percentage of influenza-like illness cases reported from that location. Each screen shot represents a specific day from the 46th to the 51st week in 2003.
FIGURE 6. Influenza-like illness sen-tinel reporting in the Oklahoma Physicians Resource/Research Net-work (2004–2005 influenza season).
[PC] application, or a Web browser) and instantaneous registering of cases in the central database. Daily re-porting of ILI cases took an average of 30 seconds to 1 minute depending on whether the PC or the PDA was used to enter and send data. ILI data were recorded and became available for analysis in the database immedi-ately. The OSDH was able to run surveillance reports with a click of a button. Reports could be easily pop-ulated by multiple parameters that included sentinel sites, ILI report time frame, ILI cases seen in the office, hospitalizations, and total number of patients seen by the particular provider from which a ratio of ILI visits was calculated. Reports generated from real-time data could be downloaded into a flat file for further analy-sis, if it was necessary. Report results could be instanta-neously fed back to users via short text messages from the same Web site that all sentinel providers received within an hour. Interestingly, providers could also run some of these reports and see the distribution of cases
FIGURE 7. Percentage of culture-positive influenza isolates from sen-tinel laboratories by specimen col-lection date, Oklahoma 2004–2005 (Oklahoma State Department of Health data). ILI indicates influenza-like illness.
in the state based on real-time data. This feature en-hanced specificity of the information feedback to the area where the provider was located.
●
Sensitivity and Specificity of ILI Reporting
There has been a very strong correlation between OKPRN surveillance reports via the OKAlert-ILI Sys-tem and reports from independent laboratories incor-porated into the OSDH conventional influenza surveil-lance system. We plotted the percentage of culture-positive influenza isolates interlaced with OKPRN sen-tinel network ILI reports in Figure 7 to demonstrate the sensitivity and specificity of the system. During the 2004–2005 season, 75.5 percent of influenza isolates were type A H3N2 (Fujian-like) viruses in Oklahoma.
Analysis of the two datasets in Figure 7 showed a 7-day lag time between OKPRN and OSDH reports in
362
❘
Journal of Public Health Management and Practicefavor of the sentinel surveillance. When respective data pairs were analyzed in the part of the curve correspond-ing to the peak season, the Pearson correlation analysis yielded a coefficient of 0.827, indicating a strong corre-lation between the two datasets representing laboratory and sentinel reports (Statistix for Windows, Analytical Software, Tallahassee, Fla).
●
Acceptability and Simplicity of the
OKAlert-ILI System
Regular feedback received from OKPRN providers throughout the two influenza seasons indicated a high level of satisfaction with the system. Users found all versions of the system (PDA, PC, and Web-based) easy to use and efficient. The Web-based solution especially required little time and effort. The system was self-explanatory and no additional tutorial was necessary. Analysis of reporting patterns of the 2004–2005 season showed that 85 percent of providers reported consis-tently (at least three times every 5 days) during the peak season. Only three providers dropped out before or during peak ILI activity.
●
Representativeness, Usefulness, and
Importance of the OKAlert-ILI System
Participating primary care providers were located in 15 counties representing all regions in Oklahoma, with the exception of the Panhandle of the state. The OSDH received regular ILI reports from both private and pub-lic healthcare providers, which included solo practi-tioners, small (2–5 physician) groups, large practices and medical centers, three residency clinics, two com-munity health centers, and several Native American tribal healthcare providers. The OKAlert-ILI System is the first electronic sentinel ILI surveillance system in Oklahoma. Data generated from the OKAlert-ILI Sys-tem have proven to be a beneficial complement to the statewide influenza surveillance program in Oklahoma that collates reports from multiple sources (eg, laborato-ries and metropolitan syndromic surveillance systems). Although it has not yet been tested in a public health emergency, the alert component of the system has var-ious potential applications. For example, in the case of a bioterrorist attack, public health authorities could access the system to rapidly notify and disseminate medical guidance to frontline healthcare providers. The dual-use technology feature was a particularly impor-tant incentive for the development of the system. Ev-eryday use ensures that end users, data monitoring and analyzing personnel, authorities, and technical staff are prepared to use the system effectively in emergency
sit-uations as well as for everyday purposes. In the case of emergency, prompt text and e-mail messages can be sent both from the OSDH and from the Department of Family and Preventive Medicine to 80 providers throughout the state. Immediate feedback via the same channels can be requested, and incoming messages can be accessed and analyzed in a timely manner.
Usefulness of OKAlert messages has been evalu-ated during the two described influenza seasons via frequent personal feedback from users. Providers indi-cated repeatedly that OKAlert messages helped them provide a more timely and accurate response to ILI cases. Detailed knowledge of circulating virus strains and area-specific influenza activity reports made the se-lection of appropriate therapy significantly easier while a physician’s confidence also increased. A listserv ver-sion of a representative OKAlert message is shown in Figure 8. A more concise version of this message was pushed out to PDAs and PCs each week. During the second (2004–2005) influenza season, OKAlert mes-sages significantly improved by focusing more on in-formation that was immediately relevant for practicing clinicians.
●
Completeness and Validation of ILI
Report Data
The OKAlert-ILI System incorporated client-side and server-side validation algorithms for ILI surveillance data entered by the providers. Validation included warnings or denial of entry for unusual or unrealistic values, incomplete dataset, wrong dates, and mathe-matical nonsense entries (eg, inputting number for ILI patients seen that exceeds total number of patients or more patients hospitalized with ILI than all patients seen with ILI ). When possible, data entry was assisted with automatic user controls (eg, JavaScript calendars) to ensure correct data format and accelerate data entry. In addition, human review of the raw incoming data was also performed frequently. Less than 0.7 percent of the reports had to be excluded from the final analysis because of confirmed data entry errors, and 0.08 per-cent of reports were corrected or deleted by providers. These changes were logged in the SQL database.
●
Flexibility of the OKAlert-ILI System
Although the OKAlert-ILI System has completed its specific mission, due to technical limitations and the variety of supported communication channels, tracked parameters could not be easily modified or added to the system. Developers are currently working on a more
FIGURE 8. Representative OKAlert listserv message. ILI indicates influenza-like activity.
flexible system that is able to switch to collecting alter-native disease surveillance data during the summer.
The system was designed to be able to shift from outbreak detection to management via a bidirectional messaging solution that provides multiple channels for communication. Information flow from the OSDH to the providers (feedback) is just as important as submis-sion of ILI data by the providers. The dual-use tech-nology makes the system capable of communicating information rapidly in both directions.
System scalability has also been an important fea-ture. The current setup requires no technical alterations to significantly increase the number of providers in the system. Geographical location of providers has not been an issue either, provided that Internet connection is available at participating sites.
●
System Costs
Development of the OKAlert-ILI System was funded by a $50,000 OSDH contract, and the system has been implemented in OKPRN, a nonprofit PBRN using open-source standards and technologies. As a result, the system has been made available to participating clin-icians at no cost. Additional system upgrade and main-tenance costs during the 2004–2005 season, including programming, salaries, technical support, and infras-tructural costs, have also been covered by the OSDH contract. When PCs were used by practitioners, no ad-ditional investment in hardware was necessary. All par-ticipants had computers and adequate Internet connec-tion in their offices prior to starting the project. Some clinicians utilized handheld devices as a personal pref-erence. These devices were distributed before the ILI project and were funded by grants targeted at
collect-ing research data and enhanccollect-ing the quality of care. We were able to use the existing handheld network for the OKAlert-ILI surveillance project.
●
Limitations
During the 2003–2004 and 2004–2005 seasons, ILI data were not classified by patient age groups. Therefore, we have not been able to analyze how different age groups were impacted by influenza. Furthermore, it has been difficult to maintain the same level of sentinel par-ticipation year round. Although some clinicians have been reporting baseline ILI activity during the summer, most of them stopped reporting temporarily due to lack of interest (very low or no ILI activity). However, the OKAlert-ILI System has been used to provide informa-tion on animal or human West Nile virus surveillance findings and other West Nile virus–related topics dur-ing summer time.
●
Conclusion
The success of the OKAlert-ILI System demonstrates that PBRNs have a significant potential for bridging the communication gap between the public and private healthcare sectors. Their unique position and affiliation with an array of healthcare entities empowers them to develop and implement viable disease surveillance so-lutions that are accepted and utilized by all parties. They are able to understand and approach all stake-holders and bring together professionals with a variety of expertise to develop and implement a complex pub-lic health solution cost effectively.
364
❘
Journal of Public Health Management and PracticeREFERENCES
1. Mold JW, Peterson KA. Primary care practice-based research networks: working at the interface between research and qual-ity improvement.Ann Fam Med. 2005;3(suppl 1):S12–S20. 2. Green LA, White LL, Barry HC, Nease DE Jr, Hudson BL.
Infrastructure requirements for practice-based research net-works.Ann Fam Med. 2005;3(suppl 1):S5–S11.
3. Nagykaldi Z, Mold JW. Diabetes patient tracker, a per-sonal digital assistant-based diabetes management system for primary care practices in Oklahoma.Diab Technol Ther. 2003;5:997–1001.
4. Smithee L, Bos J. Influenza season 2003–2004.Epidemiol Bull. 2004;36:1.
