Automated Syndromic Surveillance for the 2002 Winter Olympics

Abstract

The 2002 Olympic Winter Games were held in Utah from February 8 to March 16, 2002. Following the terrorist attacks on September 11, 2001, and the anthrax release in October 2001, the need for bioterrorism surveillance during the Games was paramount. A team of informaticists and public health specialists from Utah and Pittsburgh implemented the Real-time Outbreak and Disease Surveillance (RODS) system in Utah for the Games in just seven weeks. The strategies and challenges of implementing such a system in such a short time are discussed. The motivation and cooperation inspired by the 2002 Olympic Winter Games were a powerful driver in overcoming the organizational issues. Over 114,000 acute care encounters were monitored between February 8 and March 31, 2002. No outbreaks of public health significance were detected. The system was implemented successfully and operational for the 2002 Olympic Winter Games and remains operational today.

The terrorism acts of September 11, 2001, and the bioterrorism in October 2001 brought the realities of a biological attack¹ into our communities and drew attention to the emerging science of early detection of disease outbreaks.² Developing early warning systems for the detection of bioterrorism has since become a national priority. If the next bioterrorism attack were to come in the form of a large-scale contamination of air, food, or water, its impact would be devastating unless mitigated by very early detection, characterization, and response. Early detection of such threats requires surveillance systems that can acquire appropriate data from multiple sources for a large portion of the population under surveillance and analyze the data in real or near-real time.

The spirit of collaboration and unity prompted by the events of September 11, 2001, and the approaching 2002 Olympic Winter Games in Salt Lake City provided the opportunity to deploy an automated syndromic surveillance system on a highly accelerated schedule. In this report, we discuss the choices we faced, the decisions we made, and the challenges we encountered during the course of this implementation. We describe our experience and suggest that the methods used form a practical framework for deploying a syndromic surveillance system at a regional level. We highlight the advantages of regional biosurveillance coalitions involving public health, academia, health care systems, and industry partners.

Background

Public Health Surveillance

Traditional disease surveillance relies on the routine manual filing by clinicians of reportable diseases (and also on astute clinicians who notice and report suspicious clusters of cases to health departments).³ Disease outbreaks are detected when the case counts are higher than expected.⁴ The utility of this traditional disease surveillance approach for the early detection of outbreaks is severely limited by delays in obtaining and analyzing the data, by the reliability of the antiquated manual disease reporting system, and by delays that result from clinicians waiting until the diagnosis is confirmed by definitive testing before reporting. These limitations currently are being addressed by the development of automated surveillance systems designed to rapidly detect epidemics from routinely collected and often prediagnostic data.^5,6,7,8

New approaches to surveillance are exploring a variety of data sources as possible signals for public health surveillance.^{5,6,7,9,10,11} The focus of attention is on monitoring data that already are available in electronic format for early evidence of outbreaks.^10,11,12,13 Examples of preclinical signals include an increase in retail sales of cough medicine or an increase in health-related Web queries.^2,14,15 Examples of clinical but prediagnostic signals include a rise in “chief complaints” of respiratory illness (also referred to as triage diagnosis or reason for visit) recorded by triage nurses during telephone consultation or patient registration.^{16,17,18,19,20,21} Other examples of prediagnostic signals include rises in coded admitting diagnoses; computer-coded signs, symptoms, and vital signs; and orders for blood cultures, stool cultures, and chest x-rays.²²

Syndromic Surveillance

The term syndromic surveillance refers generally to public health surveillance methods that collect and analyze data about clinical case features such as symptoms that are appreciable by physicians (and available for analysis) prior to definitive diagnosis.¹⁰ Currently syndromic surveillance also is used by public health departments to refer to any approach that uses unorthodox types of surveillance data including sales of over-the-counter health care products. Typically, syndromic surveillance uses a patient's presenting signs and symptoms (or other elements of a diagnostic workup) as the basis for classifying the patient (manually or automatically) into a syndrome of interest, for example, “respiratory infection with fever.” The underlying assumption in monitoring such syndromes is that an unusual temporal (and/or spatial) clustering of patients with similar syndromes might indicate the earliest sign of a large-scale bioterrorist attack, thereby providing precious hours of improved response time relative to detection strategies based on more definitive diagnostic testing.

The technique is not new and, in fact, has existed in highly manual forms for several years. For example, New York City has collected information about sales of antidiarrheal products since 1999.²³ The Centers for Disease Control and Prevention (CDC) has deployed drop-in syndromic surveillance in several high-visibility settings: the 1999 meeting of the World Trade Organization in Seattle; within New York City in the weeks following the September 11th terrorist attacks^24,25; and at the 2002 World Series in Phoenix, Arizona. The authors define “drop-in” surveillance as a method that uses manual data collection by clinicians and public health officials working in the major hospitals in the area under surveillance. These health officials typically use a special-purpose paper form to record the patient's signs and symptoms, demographics, and contact information. The forms may then be scanned into a computer, which then aggregates the data for analysis by epidemiologists.

To date, the CDC's “drop-in” and other form-based methods of syndromic surveillance have been deployed only for short durations (weeks), partially because operating the system requires a substantial commitment of resources and personnel that is too expensive to sustain beyond the lifetime of a specific event. The “drop-in” method is also inherently intrusive and disruptive to care delivery, requiring public health workers to be present at the point of care and in some cases requiring care providers to complete additional forms. The level of intrusiveness was such that the Utah Department of Health (UDOH) decided against using the drop-in approach for surveillance during the 2002 Olympic Winter Games.

Opportunity for Automation

With recent advances in health care information technology, including electronic medical records, messaging standards, and controlled terminologies, many of the data elements appearing on a typical drop-in form can now be captured directly and automatically from clinical information systems.

The Real-time Outbreak and Disease Surveillance (RODS) system is an example of such automation. The RODS system is an electronic biosurveillance system that has been deployed since 1999 in Western Pennsylvania.^5,9,26 RODS takes advantage of existing Health Level 7 (HL7) message routers in health care systems to obtain data in real time from clinical information systems. Then, like “drop-in” and other types of surveillance, RODS analyzes the data for anomalous densities of cases compared with historical patterns.

Design Objectives

The objectives of the project were to implement the RODS system to (1) adequately cover the large region in Utah involved in the 2002 Olympic Winter Games, which meant obtaining data from multiple hospitals and clinics; (2) obtain chief complaint data in a timely fashion; (3) analyze the data for aberrant patterns; (4) notify appropriate individuals in the event of an anomalous pattern (i.e., a sudden, statistically significant increase in the count of a particular syndrome); and (5) facilitate the investigation of the anomaly.

Surveillance Objective

A clear surveillance objective must be articulated if a deployment is to succeed. A surveillance system may have several objectives that range from the capture of the first, or one of the early cases of disease caused by bioterrorism pathogens, to detecting disease outbreaks through identification and investigation of aberrant patterns of disease in the context of widespread exposure.

In anticipation of the 2002 Olympic Winter Games, UDOH and Salt Lake Olympics Committees had spent years planning and preparing. Administrative rules providing the legal framework for the manual surveillance methods were established months before. Activities planned included daily polling of selected physicians, pharmacies, veterinarians, the Poison Control Center staffs, and the forensic pathologists (sentinel surveillance); 24-hour batch mode syndromic surveillance for Intermountain Health Care's urgent care facilities (an urgent care facility is a clinic that sees patients without appointments); and a manual emergency department syndromic surveillance system employing daily review of encounter logs and the medical records of syndromic patients. The proposal to use RODS during the 2002 Olympic Winter Games was made following the November 2001 American Medical Informatics Association (AMIA) meeting.

In mid-November 2001, discussions with the state epidemiologist and other senior public health officials overseeing the Olympic surveillance efforts identified a niche role for RODS in the overall Olympic surveillance effort as an early warning system, which could be used to identify aberrant patterns of syndromic activity. RODS, with its ability for real-time collection and analysis of data, had the potential to provide a 12- to 24-hour earlier indication of clinical anomaly than the manual and batch-mode systems that were planned.

Geographic Coverage

The geographic region selected for surveillance was the “Wasatch Front,” the area to be affected by the Olympic Games. The region is composed of seven counties, the combined population of which is roughly 2 million (∼80% of the total state population). The investigators recognized that even this large geographic scope had limitations. For example, if an attack occurred near the end of the games, the resulting infections might not produce symptoms until athletes and spectators had departed to their homes. However, there was no practical way to track visitors after the games, so the scope of surveillance was limited to the region in which the games were held.

Stakeholders

Public health surveillance projects typically cut across jurisdictional boundaries and require the permission and cooperation of those entities, which are referred to as stakeholders.²⁷ Stakeholders in this project included the UDOH and representative local health departments and the regional health systems that provided the needed data. Engaging the public health and health system stakeholders required meetings with decision makers from health systems, the state health department, and the academic community. In Utah, the individual stakeholders included the state epidemiologist, an Olympic surveillance coordinator, a vice president for medical research, an associate chief information officer (CIO), a medical informatics department chair, and the director of the RODS laboratory.

Public Health Authority

The public health entities, which had responsibilities for the surveillance region, included the UDOH and six local health districts. Effective collaboration with these entities was enhanced because of the planning and preparation for the 2002 Olympic Winter Games. In addition to the specific response structure described below, the Utah deployment of RODS sought to integrate and coordinate the RODS deployment with existing public health practice. For example, it requested customizations of the RODS application programs to analyze and display data by individual local health districts, both for the graphical user interface (syndromic run charts, custom query, and geographic information system) and for the statistical detection algorithms.

Health Systems

In Utah, our RODS implementation efforts were directed at the two dominant health care delivery systems: the University of Utah Health Sciences Center, a state-affiliated academic medical center with approximately 10% market share, and Intermountain Health Care (IHC), a large, regional, not-for-profit health care delivery organization with approximately 60% market share with academic ties to the University of Utah. Both health systems possessed advanced information systems and integrated care delivery networks and were technically capable of providing the required data in an electronic format and in real time. The emergency departments and urgent care facilities from just these two health systems provided an estimated 70% of acute care for the surveillance region, including the Olympic Village and athletes.

System Operation

Overview

A brief overview of the RODS system, as implemented in Utah, is provided. A detailed technical description of the RODS system has been provided by Tsui et al.²⁶ All of the clinical data used by RODS are produced in the course of normal clinical workflow. Of the types of data that RODS can monitor, free-text chief complaints were widely available in Utah. Therefore, the project focused on surveillance from that type of data, specifically monitoring the counts of patients presenting for acute care with chief complaints consistent with the seven RODS syndromes: constitutional, respiratory, encephalitic, rash, hemorrhagic, gastrointestinal, and botulinic.

Data Sources

During the Olympics, encounter data were collected from 19 urgent care centers and nine emergency departments owned and operated by IHC and from the University of Utah Hospital's emergency department and the Polyclinic located in the Olympic Village.

Data Collection, Transmission, and Transformation

Free-text chief complaints and the demographic data used to drive RODS were collected from the HL7 messages coming from each patient registrations system. A custom HL7 interface was constructed by each health system to send the data to RODS. The interface filtered ADT messages for patient registrations originating from acute care facilities and created a patient deidentified HL7 message that was an abstract of the encounter. Each encounter's abstract included the sending facility ID, a unique record identifier, gender, age (years), the free-text chief complaint, the visit date/time, and the patient's residential zip code. The HL7 messages then were transmitted across a secure telecommunications network—VPN and T1 lines—to the RODS servers at the University of Pittsburgh. Upon receipt, the HL7 messages were processed and the free-text complaint was forwarded to a complaint-coding application for transformation into a coded syndrome. The natural language processor used for the Olympics was the Naïve Bayes Complaint Coder (CoCo) developed at the University of Pittsburgh.²⁸

Monitoring

During the Olympics, the data were analyzed and reviewed by the automated detection algorithm (described below), which ran every four hours, and by frequent visual inspection through the RODS user interface. The RODS system provides three different data views via an encrypted, password-protected Web interface. The interface was organized into three screens—Main, Epiplot, and Mapplot.

The Main screen simultaneously showed eight time-series plots corresponding to the daily total visits and the seven syndromes for the past week. The user could also view these graphs by county or for the whole state.

The Epiplot screen allowed the user to generate custom time-series plots based on syndrome, region, start dates, and end dates specified by the user. A “get cases” button allows users to view case-level detail for encounters making up the specific time-series.

The Mapplot screen provided an interface to ArcIMS, an Internet GIS server developed by Environmental Systems Research Institute, Inc. (ESRI). Mapplot can display the proportion of a particular syndrome of all the acute health care encounters for patients reporting that zip code as their place of residence.

Data Analysis and Reporting

Detection Algorithms

The primary statistical tool used by RODS during the Olympics for automated pattern recognition was the recursive least square (RLS) adaptive filter. RLS, a dynamic autoregressive linear model, computes an expected count of each syndrome within a region from historical data, adjusting its model coefficients based on prediction errors.^26,29 The investigators selected the RLS algorithm over other potential algorithms because it required only a few days of historical data to generate model coefficients. There was concern that the influx of Olympic visitors into the region would render algorithms that depended on previous seasons, years, or even recent weeks, unreliable. The fact that RLS places more emphasis on recent historical data (i.e., the last few days) in deriving predicted counts, also made it well suited for detecting sudden positive deflections with steep slopes (i.e., “spikes”) in a signal. Such sudden positive deflections would be expected in a sudden outbreak due to contamination of air, food, or water, which was the niche surveillance role of the RODS implementation.

During the Olympics, RODS also was running the What's Strange About Recent Events (WSARE) algorithm developed at Carnegie Mellon University.^30,31 WSARE performs a heuristic search over combinations of temporal and spatial features to detect anomalous densities of cases in space and time. The case features analyzed by WSARE include syndrome category, age, gender, and geographical information. The current count of patients with specific features was compared with the counts on the same day of the week during recent weeks. WSARE required several weeks of historical data (see above) to generate its comparisons and therefore was not included in the formal response structure for the Olympics.

Signal Alarm Thresholds and Alerting

During the Olympics, RLS was set to trigger a signal alarm when the current actual count exceeded the 95% confidence interval of its predicted count. The 95% confidence interval threshold was chosen to minimize the number of false alarms generated by the RODS system—a great concern to public health planners. WSARE was set to trigger an alarm when the system detected a statistically significant increase in counts (p < 0.05). The signal alarm notifications for the primary detection algorithm—RLS—were broadcast to system administrators, investigators, and public health officials through e-mail, alphanumeric paging, and short messaging service. Upon receipt of such a notification, the response team (described below) logged into the RODS system and began reviewing the data relevant to the signal alarm. The alarm model required that every individual on the response team be able to receive messages (i.e., have a pager, cell phone, or other text messaging device) and could connect to the Internet. A teleconferencing service set up by the UDOH proved to be very useful for ad hoc conference calls during the response to signal alarms.

Response

A response structure mimicking that developed for the Olympics for other unfamiliar surveillance signals (e.g., biosensors) was chosen to mitigate concerns over the potential repercussions from false alarms. A two-tiered response structure was used. A technical advisory group (TAG) with representatives from the RODS laboratory, the health systems, UDOH, and the relevant local health districts performed a preliminary review of any signal alarms generated by the RODS system. The TAG analyzed the alarm and other data and, if appropriate, reported its findings to a policy advisory group. The policy advisory group reviewed the TAG's interpretations and recommended responses and then made the final determination of how to proceed with handling the signal alarm. Web-based access to the electronic medical record was available to the policy advisory group through a health system intermediary for reviewing individual patient records if that level of detail was needed.

Privacy and Confidentiality

Syndromic surveillance using clinical data requires that data from health systems be made available to those conducting the surveillance. Such data sharing with the UDOH raised a number of concerns with the health systems in Utah and resulted in three major issues:

• (A) The health systems in Utah already were responding to a legal mandate to share data and allow intrusive chart reviews during the 2002 Winter Olympic Games. The operational disruption and risks posed to patient privacy by the data sharing required for the numerous surveillance projects had to be balanced against the potential benefits. Had the RODS system been accepted as an alternative to having public health officials in the health system's emergency departments, the benefits would have been clear. Because the RODS system was a relatively late and “untested alternative” to the long planned Olympic Surveillance effort, the UDOH was unwilling to abandon their earlier plans. Also, the fact that the syndromes being monitored by RODS and UDOH were different (described below) made this substitution unacceptable to public health.

• (B) One health system classified RODS as a “research project” since it was not the planned-on and agreed-to surveillance methodology. As a consequence, before the implementation of RODS was allowed, compliance with federal privacy law (HIPAA) and the institutional review board (IRB) approval were required.

• (C) Both health systems required data-use agreements that outlined the management and permitted uses of their data. The specific goals of the surveillance project, existing state and federal law, and the relationships between health systems and health departments had direct impacts on these negotiations.

Utah Law

To facilitate enhanced public health surveillance for the Salt Lake Olympics, UDOH put into effect an administrative reporting rule designed to establish legal authority for syndromic surveillance (be it manual, automated, or semiautomated). Administrative rules are statements written by state agencies, that have the effect of law. Under this rule, designated emergency centers were to report syndromic information for patients from the preceding day's encounters, either by reporting it themselves or by allowing UDOH representatives to gather the data. Encounters required a report if and when diagnostic information indicated the presence of one of 11 syndromes defined by the UDOH (i.e., respiratory tract infection with fever, gastroenteritis without blood, bloody diarrhea, febrile illness with rash, suspected viral hepatitis, meningitis/encephalitis or unexplained acute encephalopathy/delirium, sepsis or unexplained shock, unexplained death with history of fever, botulismlike syndrome, lymphadenitis with fever, illicit drug-related episode). The report included the following protected health information: the reason for visit, chief complaint, presenting diagnosis, final diagnosis (when available), facility, date/time of visit, patient demographics (age, gender, residential zip code), the syndrome detected, admission status, and a record identifier for follow-up investigation.

The RODS approach did not quite fit this rule because the seven syndromes used in RODS were more general than the 11 stated in the UDOH rule. Although RODS, which works by natural language processing of chief complaints, was built to support any syndrome set (through machine learning from a training set), it was felt that some of the fine-grained distinctions in the 11 existing syndromes (e.g., fever and cough) could not be determined accurately from chief complaint data, even by manual inspection. Had we as investigators had sufficient time (i.e., well in advance of the Olympics) to empirically determine how RODS performed compared with the UDOH rule, we might have addressed the concerns of the health systems and public health.

Federal Law

On April 14, 2003, the use and disclosure of protected health information became subject to regulation under the Health Insurance Portability and Accountability Act (HIPAA). The participating health care systems in Utah had chosen to adopt the regulations late in 2001; hence, HIPAA was a strong influence on the planning and decision making. One health data provider viewed the RODS project as research and, at the time of implementation, the proposed HIPAA regulations permitted disclosure for public health surveillance activities but did not specifically address the sharing of data for public health surveillance research. The fact that the trusted broker (described below) existed for no other purpose than public health surveillance satisfied the HIPAA exclusion for uses and disclosures for public health activities, although one of the parties did not accept this point.

In August 2002, in an effort to avoid curtailing important public health and health care operations research, the United States Department of Health and Human Services added a new standard and implementation specification for a limited data set to the HIPAA rule. A minimal set of data elements (identical to what is currently being sent to RODS) can be shared for specified public health activities and research without the added constraints of HIPAA accounting and disclosure.³² As was the practice during the RODS Olympic surveillance and afterward, direct patient identifiers (e.g., name, street address, and phone number) must be removed, and data sharing agreements, specifying the data elements of the limited data set and their permitted uses, are required.

Trusted Broker

To address legal and data provider concerns, we chose to use a trusted broker model with detailed data sharing agreements specifying permitted uses of the data and the methods for protecting patient privacy.^15,33 A “Trusted Broker” acts as the repository of the surveillance data being collected for the project. A Trusted Broker is an entity to which health systems agree to send data and, as such, is typically removed from the driving political and economic forces in the region under surveillance.

The Trusted Broker maintains and manages the surveillance databases under the auspices of a governing board. The governing board was comprised of representatives from the UDOH, the academic institutions conducting the research, and the health system data providers. In implementing RODS in Utah, we took advantage of the existing Trusted Broker architecture created by the RODS laboratory.

Memoranda of Understanding

With the decision to use a Trusted Broker model, the next task was to execute data-sharing agreements. The data sharing agreements defined the conditions under which data would be sent to the surveillance system, how the data could be used, and how data would be protected and managed by the Trusted Broker (Fig. 1—Sample Memoranda of Understanding, available as an online data supplement at <www.jamia.org>).

The approach used in the Pennsylvania RODS implementation involved trilateral agreements between the Pennsylvania Department of Health, the University of Pittsburgh (the University of Pittsburgh is the parent organization of the RODS laboratory and the actual signatory), and each data provider.

For the Utah deployment, we started with a goal of executing two trilateral agreements (among the University of Pittsburgh, the UDOH, and each of the two health care systems) and at the end of seven weeks could only achieve a bilateral agreement and a trilateral agreement. This outcome reflects the different views held by the health systems on sharing data with the UDOH. Since the University of Utah receives partial funding from the state, the working relationship between the two entities (at least from a data provision perspective) has always been strong, making data sharing for collaborative public health projects relatively commonplace and painless. Thus, the University of Utah had no trouble executing a trilateral agreement.

Intermountain Health Care, however, is a large not-for-profit health care delivery organization that directly competes with the University of Utah and other regional providers and has maintained a majority of the health care business in the region for decades. According to a Senior Vice President, IHC had, on occasion, borne the brunt of state mandates to provide clinical data for public health purposes because of its dominant market share and advanced data systems. IHC felt such an action was unfair and had exacted both economic and political tolls. This history, coupled with the fact that data received by the UDOH often became property of the state, led IHC to favor a bilateral agreement—excluding UDOH. The final agreement specified that UDOH could only view aggregate time-trended and spatially displayed data (i.e., absent the free-text chief complaint and with age reported in five-year ranges) and could only receive more detailed data in the event of a public health emergency.

Privacy Protection

The potential repercussions (e.g., monetary penalties, litigation, and imprisonment) of violating HIPAA regulations (and misusing data in general) required that strict attention be paid to protecting privacy. In the Utah RODS implementation, individual health system data are blended with other health system data and made available in aggregated, summary format for public health surveillance purposes through an encrypted, password-protected Web interface. Access to patient level details, including chief complaints and protected health information, is restricted.

Under the data-sharing agreements, all health system data sent to the system remain the property of the sending health system. However, in the event of a public health emergency, the public health authorities have the right to obtain more detailed information about possible cases, and any data recorded during the course of managing such an emergency would become the property of the state.

Experience

Deployment

Deploying the surveillance system required substantial technical resources. Fortunately, these were available at the health systems in Utah and the RODS laboratory and from their collaborators at the University of Pittsburgh and Carnegie Mellon. The core technical expertise required for the Utah deployment included software engineers to create the HL-7 interfaces; network architects to create the connections to the surveillance system; informaticians with the programming and database administration expertise to modify the database and redesign the applications to handle data from a new region; and programmers to generate the output of the mapping system. Data architects and analysts familiar with the health system's information systems were extremely valuable in making determinations about data availability and data quality. Clinicians and analysts with access to the electronic medical record are required to obtain additional clinical data and patient contact information in the event of emergency.

Generalizability

If a health system uses HL-7 messages to transmit data from the registration system and captures free-text or coded chief complaints, the Utah RODS approach will be applicable. In the simplest case, the message streams can simply be duplicated and routed securely to the RODS system in an application services provider model. The RODS laboratory has been in contact with several state health departments interested in this approach. We think that the project management task structure we developed is generic and would be extremely valuable in planning and managing a third deployment (Fig. 2—Sample Gantt chart, available as an online data supplement at <www.jamia.org>).

Acceptability and Usefulness

Because data acquisition for RODS is imperceptible to the site of care delivery, acceptability to the health care provider is excellent. The UDOH and local health department officials readily accepted the presence of RODS during the Olympics. The system provided an additional data source of corroborating information to support information coming from other surveillance systems and provided a high degree of reassurance that sudden, large aberrations in acute care visit counts would not go unnoticed for hours. The system currently is the only source of syndromic surveillance data for the state of Utah.

Cost

Fortunately, securing funding to support the Utah implementation was not an issue because the RODS laboratory had funding, the Utah project lead had fellowship funding from the National Library of Medicine, and the regional hospitals were willing to absorb the minimal cost of HL-7 message router configuration. The authors have not estimated the actual costs of hundreds of hours of donated time from the RODS laboratory, the health systems, and the health department. The project also benefited from donated industry support in the form of leased data lines, but the costs of the current Utah RODS network are minimal because of the use of the free Internet (the cost of two virtual private network routers).

Status Report

Between February 8, 2002, and March 31, 2002, RODS analyzed 43,895 emergency visits (average of 861 per day) and 70,684 (average of 1,386 per day) urgent care registrations. ▶ shows the number of encounters broken down by facility type and by syndrome.

Table 1.

Utah RODS Acute Care Visit Counts by Syndrome and Facility Type for the Period of February 8, 2002, through March 31, 2002

Syndrome	Urgent Care (%)	Emergency (%)	All Acute Care (%)
Constitutional	19,245 (27.23)	6,078 (13.85)	25,323 (22.10)
Respiratory	4,440 (6.28)	4,967 (11.32)	9,407 (8.21)
Encephalitic	2,895 (4.10)	4,107 (9.36)	7,002 (6.11)
Gastrointestinal	906 (1.28)	2,173 (4.95)	3,079 (2.69)
Rash	1,535 (2.17)	561 (1.28)	2,096 (1.83)
Hemorrhagic	285 (0.40)	762 (1.74)	1,047 (0.91)
Botulinic	18 (0.03)	30 (0.07)	48 (0.04)
None	41,360 (58.51)	25,217 (57.45)	66,577 (58.11)
Total	70,684	43,895	114,579

During the Olympics, a total of 31 users logged in 233 times to the user interface. They viewed the graphs on the Main screen 1,244 times, generated 702 custom graphs with Epiplot, requested 618 sets of detailed case data, and accessed Mapplot 511 times.

During the Olympic time period, the RLS alarm detection algorithm “fired” twice. The first signal alarm was generated when the number of patients from Morgan County presenting within a 24-hour period complaining of constitutional symptoms reached seven, just slightly above the alarm threshold of 6.69 expected cases for that day. The second signal alarm was generated when the number of patients with hemorrhagic complaints reached 33 in the seven counties within a 24-hour period, just above the expected threshold of 29.34. On both occasions the TAG was able to convene via teleconference, review the data relevant to the alarm, and reach consensus that the signal was a false alarm. The low cost in terms of the time and investigation effort required to address these two alarms is in contrast to the concerns that were present during planning and also that appear in the literature.³⁴

Discussion

Our experience shows that an automated syndromic surveillance system can be deployed quickly (< 7 weeks) when high-profile events such as the Olympics drive the situation. Also instrumental was the fact that the technical hurdles for health system data providers were low because the technical capacity and expertise of the health data providers and the experts of the RODS team were available. The majority of the time and work of the project was organizational and political.

The speed of installing would be possible in another region only if leadership were provided by the mayor of a city or the governor of the state and also by the health department, because success depends on the active participation of the executives and lawyers of many entities finding time in their busy schedules to meet to resolve sometimes longstanding issues. Unless there is a call to action, a clear articulation of purpose, and direct personal action—in short, leadership—a similar system involving real-time data collection at the 70% level for most of a state may take years to accomplish.

In our RODS deployment, the organizational issues outweighed the technical issues. Institutional data sharing concerns overshadowed the ease with which the data interfaces were created. The time and effort required for establishing the Trusted Broker (including data sharing negotiations and ethical oversight) and integrating the system with public health practice exceeded that required for adapting the system to accommodate Utah data. In addition, the depth and range of technical skills needed also were beyond that of any of the individual participating groups and included communications networking, hospital-side HL7 interfacing, and the surveillance system itself.

The following factors contributed to the successful rapid deployment of RODS. First, the decisions to use a minimal data set and an application-services-provider model with the RODS laboratory serving as both the Trusted Broker of the data and the surveillance service provider were important. Second, the past experience of deploying RODS in Western Pennsylvania proved to be invaluable in deploying Utah RODS. In addition to experience with technical issues, the RODS laboratory had available personnel including an attorney with substantial experience with many of the legal and organizational decisions that we faced. Third, the maturity of HL7 in clinical computing meant that data were available in HL7 format, and interface engineers and engines existed at the health systems. The health systems had already integrated the information processing of their diverse facilities so that the number of HL7 interfaces required was only two. Fourth, we did not need to obtain funding because the resources required were not large, and none of the entities participating had to shoulder the entire cost. We also had donated industrial support for setting up the network and map servers.

If the RODS system were to be deployed in a new region, the technical approach and resources required would require few, if any, changes, but the organizational approach could vary considerably based on local factors. Our experience may be unique in that many of the participants were faculty colleagues from academic Medical Informatics Departments of the University of Utah and the University of Pittsburgh, and, therefore, the project benefited from existing professional relationships that spanned the participating hospitals and the RODS team in Pittsburgh as well as a common understanding of the approach being taken. The project also benefited from professional relationships among the public health departments and regional health systems as a result of their preparations for the Olympics that had been underway for several months. Our experience may not be representative because we only had to negotiate two data-sharing agreements. In the absence of some of these factors, a similar project may require more time and effort to elicit participation from hospitals and other sources of surveillance data. If a project had to rely more heavily on information technology vendors and expert consultants, the time and effort would also be expected to increase.

Conclusions

We found that an automated syndromic surveillance system, driven by chief complaints obtained from existing HL7 messaging systems, could be implemented in Utah in less than seven weeks. Key factors contributing to such a successful rapid deployment were (1) motivation provided by fear of a terrorism attack on a high-profile event; (2) the fact that the RODS system already existed; (3) the existence of surveillance expertise of the RODS laboratory; (4) the HL7 expertise and technical capabilities of the participating health systems; and (5) a collaborative coalition driven by medical informaticists and that included public health, health care systems, and industrial partners.