Innovative Surveillance Methods for Rapid Detection of Disease Outbreaks and Bioterrorism: Results of an Interagency Workshop on Health Indicator Surveillance

Abstract

A system designed to rapidly identify an infectious disease outbreak or bioterrorism attack and provide important demographic and geographic information is lacking in most health departments nationwide.

The Department of Defense Global Emerging Infections System sponsored a meeting and workshop in May 2000 in which participants discussed prototype systems and developed recommendations for new surveillance systems. The authors provide a summary of the group’s findings, including expectations and recommendations for new surveillance systems.

The consensus of the group was that a nationally led effort in developing health indicator surveillance methods is needed to promote effective, innovative systems.

IN LIGHT OF RECENT TRAGIC events, including the deliberate use of a biological organism to cause disease and death among unsuspecting victims, the need is paramount to improve public health capabilities in the United States, especially the ability to rapidly detect and respond to unusual disease events. These events have highlighted our lack of preparedness for biological attacks as well as naturally occurring disease outbreaks.1–3

One of the primary goals of public health is to prevent disease in a community. To best prevent disease, knowledge of existing disease rates, risk factors, and the effectiveness of preventive measures is necessary. The first step in gaining this knowledge is a working surveillance system that rapidly allows public health practitioners to know the health status of the community. Indeed, public health surveillance is a core element of public health practice. Unfortunately, most infectious disease surveillance systems are passive and rely on practitioners voluntarily reporting to the public health system,4 and they are often not sufficiently sensitive or timely to be of great value in terms of controlling outbreaks.

In addition to earlier detection of events, surveillance systems are essential for focusing limited response assets and providing evidence-based information to governmental risk communicators. Lessons learned from a May 2000 bioterrorism response exercise conducted in Denver, which involved top government officials, illustrate these needs.2 Officials interviewed after the exercise noted the need for information systems that could “deliver real-time data showing the number and locations of persons with the specific illness in the affected area” and that “allow rapid collection and analysis of patient epidemiological information to determine source(s) of exposure to an agent.”4 In the face of these imperatives, plans to improve public health capabilities to identify and address such disease emergencies must include determining how surveillance systems can be made more timely, flexible, and sensitive without overly compromising other aspects of quality.

To share experiences, avoid costly mistakes, and foster efficient progress toward the objective of creating innovative, responsive surveillance systems, the Department of Defense’s Global Emerging Infections System sponsored a meeting in May 2000 that focused on 3 areas: (1) identifying surveillance system needs, (2) examining existing prototype systems attempting to meet these needs, and (3) identifying the ideal features of a “system of surveillance systems” that would be more timely, sensitive, and flexible in terms of detection and response. This final area was addressed by gaining a consensus among the meeting attendees and through continued dialogue among the participants in the period following the workshop. Here we summarize the proceedings of the meeting, which we believe was the first national workshop focusing on nontraditional approaches to surveillance for emerging infections, including those related to bioterrorism.

CONSENSUS METHODS

Approximately 70 selected individuals attended the 3-day symposium. Most attendees either had significant experience in implementing innovative methods of surveillance for emerging infections or were involved with relevant aspects of policy development or health system management at the federal, state, or local level. Parallel work groups produced consensus recommendations for designing and implementing innovative surveillance methods and addressed weaknesses in current methods.

EXPECTATIONS FOR A HEALTH INDICATOR SURVEILLANCE SYSTEM

The meeting attendees (“the group”) coined the term health indicator surveillance to describe the variety of information being used to monitor community health. Drawing on insights from a range of experiences, the group recommended that the requirements of different users be documented while new surveillance systems are still in development or early operation. The workshop participants identified several criteria that should be met in improving surveillance for emerging diseases. For example, a system should achieve the following goals:

• Facilitate the rapid recognition of a disease outbreak

• Improve data transmission and analysis speed

• Be capable of integration with other surveillance systems at all levels of government

• Provide detailed information to assist with outbreak investigations

• Assist in determining exposure sites through geographic information systems

• Assist in providing efficient delivery of limited medical countermeasures, such as vaccines or antibiotics

• Evaluate success in the areas of containment and mitigation

• Provide historical and trend data to be used in baseline comparisons and long-term monitoring

EXISTING PROTOTYPE SYSTEMS

Over the past several years, many agencies and municipalities have attempted to improve their public health capabilities with novel and innovative approaches to surveillance. The systems used have many similarities and differences. Some use already existing data, whereas others collect new information. Some use more traditional medical data sets, whereas others use nontraditional data sources. Many examples of these systems can be found.5–17 A comparison of some of the systems has also been published.18

The strengths of these systems include large populations that can be placed under surveillance; previously, it was often the case that only those meeting reportable disease conditions were followed. This benefit is increased by the ability to have continuous data streams that are not dependent on the participation of active health care providers. A reliable source of health information that captures all outpatient visits, all over-the-counter sales, all school absentees, and so forth, although possibly not as accurate as a provider-based system, may be a more sensitive indicator than those previously available simply because of the completeness of the data it captures. In contrast, many active systems involve very low compliance rates, so it is difficult to determine whether disease rates are low or whether cases of disease simply have not been reported.

In light of this finding, it is not surprising that systems that require additional data input, while more accurate in regard to diagnoses and involving better symptom clustering, are also extremely difficult to maintain. After the terrorist attacks on September 11, 2001, the Centers for Disease Control and Prevention (CDC) deployed 45 epidemic intelligence service officers to New York City to assist in a syndromic surveillance effort. With 24-hour surveillance suport in place at the 15 emergency departments selected, the compliance rate was 90%; with 18-hour support, it dropped to 82%. After the departure of the service officers, this type of surveillance could not be maintained.19

For these reasons, and because they are hindered by a lack of baseline data with which to make comparisons, the working group’s opinion was that these types of systems should be reserved for emergency situations.

Other issues still to be resolved include the use of extremely nonspecific information, such as absenteeism and total volume of emergency visits. Some of this nonspecific information may be available earlier in the disease spectrum, but it may be difficult to ascertain with certainty that an infectious disease syndrome is causing an aberration. Therefore, the opinion of the group was that a combination of systems was the best approach to verifying and confirming anomalies found in various data sources.

KEY ISSUES IN DEVELOPING A SURVEILLANCE SYSTEM

Data Sources

Measurable alterations in personal behaviors within the first hours or days of illness, including work or school absenteeism and purchase of over-the-counter remedies, can assist in the early detection of an event or epidemic.20 Data relating to medical care delivery have value not only for outbreak detection but also for the ongoing management and tracking of an epidemic.21 These types of data include emergency response calls, required disease reporting, outpatient clinic and emergency room activity, inpatient and intensive care unit records, and laboratory and prescription drug requests.

An infrastructure for data collection is not readily available in the case of early (preclinical) health-related behaviors and many types of clinical data. A system designed to track preclinical personal behaviors could use data that are already being collected for other purposes such as billing, inventory control, or resource management. However, concerns over ownership may block access to existing data that are deemed valuable for surveillance. Resolving such issues will require high-level leadership and a commitment to making these data available. A summary of potential data sources, with their pros and cons, is outlined in Table 1 ▶.

TABLE 1—

Possible Sources and Utility of Health Indicator Surveillance Data

Type of Data	Data Source	Pros	Cons and Confounders
Typical health surveillance	Reportable diseases	Specificity of diagnoses; limited to diseases of interest	Relies on passive reporting; limited to specific diseases—may not detect new emerging infections; not timely
	Laboratory-based surveillance	Specificity of diagnoses	Often relies on passive reporting; may be limited to specific diseases; not timely
	Specific disease surveillance (e.g., influenza)	Follow trends for specific diseases	Limited to the disease in question; not timely
Existing health data not normally used for surveillance	Diagnostic information for inpatients and outpatients	Reflects incidence of disease in general population	Nonspecific—may be difficult to document definitive information; may not be accurate
	Intensive care unit admissions	Best indicator of rare events (e.g., West Nile virus)	Will not capture milder cases
	Prescription and over-the-counter pharmacy sales	Reflects symptomatology most broadly	Subject to promotions/sales; nonspecific
	Clinical laboratory submissions	Ordered by clinicians; reflects illness patterns	May not be ordered for all (or most) patients
	Medicare or Medicaid claims	Ease of data capture	Problems with timeliness and accuracy; not broadly representative
	Acute diagnoses in nursing home populations	Reported by medical personnel; immobile population with limited exposure possibilities	Immobility reduces exposure potential; not broadly representative; may not be automated
	Ambulance call chief complaints	Many communities with timely access to data	Nonspecific
	Radiology test ordering and results	Ordered by clinicians; may reflect working diagnosis	Not ordered for all (or most) patients; multiple reasons for radiological tests
	Poison information calls	Timeliness	May not be related to infectious diseases
	Medical advice call-in	Occurs very early in disease outbreak	May be difficult to categorize
	Emergency room use	Ease of calculation	Does not reflect accurate cause of increased patient visits
	Internet hits for medical information	Large database with relatively easy access to information	May be difficult to determine geographic location
	Medical examiner/mortality surveillance	May capture severe diseases	Not timely
Non–health data sources	Road and transit usage	Captures many segments of the population	Changes may be difficult to interpret
	Entertainment venue usage	May reflect behavior early in illness	May be difficult to collect data; may not reflect expected behavior patterns of ill people
	Weather data	Usually readily available	May not be associated with illness patterns
	Vector data	Allows knowledge of potential for spread of vector-borne diseases	May not be associated with illness patterns
	School and work absenteeism	May occur earlier than visits to clinician	May be absent for nonmedical reasons; data often not automated

Need for a “System of Systems”

Ideally, a surveillance system would be sensitive enough to identify the emergence of an outbreak, categorize its nature, and identify those affected so that the outbreak could be quickly and effectively contained. Bringing together information from various health indicator data sets can allow public health practitioners to (1) evaluate many indicators simultaneously, (2) compare variations and identify common trends, and (3) track confounding factors and decrease false alarms.

Compilation of information provided by independent and complementary data sources allows intersystem comparisons. By comparing the data derived from several indicators, some of which are more sensitive than others in different scenarios, one can assess whether or not a trend observed in any single system is confirmed by the other systems. Simultaneous small and unexpected but concordant variations in multiple data sets may suggest an actual disease outbreak.

The ideal solution would be a single organization in charge of a “virtual” data warehouse where all collected data are compiled, integrated, and analyzed. Moreover, there is a need for these data to be shared effectively and efficiently at different levels of the health systems already in existence. Of utmost importance, the fundamental issue of personal and organizational privacy needs to be addressed before such a system is set up.

WHAT NEEDS TO BE DONE?

Evaluation

Surveillance systems should be periodically evaluated to ensure that they are efficient and effective. Because many new health indicator surveillance systems are still in development, special emphasis should be placed on their evaluation and subsequent improvement. Information on the particular aspects of a particular system that resulted in it being more efficient or useful should be made available to others who are developing similar systems.

The attributes listed in the CDC’s guidelines for evaluating public health surveillance systems should be used in system assessments.22 Some attributes will be more important to some surveillance systems than others. Early warning systems for infectious disease outbreaks will rank timeliness first, with acceptability, flexibility, sensitivity, and representativeness following closely thereafter. Data quality and positive predictive value, while still important, will be less important than in the case of specific reportable disease surveillance.

In addition to traditional attributes that must be considered, new surveillance systems based on automated data sources will involve other concerns, including a standard user interface, data format and coding, compatible hardware and software, quality assurance, and strict adherence to security and confidentiality.22

In the final phase of evaluation of a surveillance system, recommendations should be made to improve the system. However, efforts to improve certain attributes may detract from the effectiveness of others. After elements that cannot be changed (e.g., security restrictions that may impede timeliness but cannot be removed) have been taken into account, those elements that are most important should be identified and given the highest priority for improvement.

There is often pressure to begin using public health data for surveillance before the usefulness of these data is evaluated; however, we must provide proof of the effectiveness and sensitivity of data sources before considering them as providing reliable information. Care must be taken to ensure that the data being used have meaning in the public health community and that the output being generated is derived from appropriate information.

Validation is also required to demonstrate that the system appropriately captures the events it was intended to capture. Surveillance systems can be validated through the use of tools such as preparedness exercises incorporating simulated outbreaks and evaluation of their ability to detect emerging natural epidemics. Perhaps up to several times a year, more realistic assessments of a surveillance system can be made when natural, expected, recurring epidemics occur.

Examples might include assessing influenza cases in patients each winter, especially in epidemic years, and monitoring cases of respiratory syncytial virus or rotavirus in young infants from fall to early spring.

The rare, but perhaps inevitable, outbreak of an emerging infectious disease or epidemic of a known but uncommon disease would also represent an excellent opportunity to validate a system. The outbreak of West Nile virus encephalitis in the New York metropolitan area in 1999 illustrates the type of infrequent incident that could be used to evaluate a system’s capacity for early detection of unexpected epidemics. In this case, intensive care unit surveillance may have best detected this low-frequency, serious illness. Ideally, a “system of systems” with demonstrated sensitivity in terms of detecting events in multiple data sources would also perform well in a genuine disease outbreak. If an outbreak does occur during the system’s operation, its detection ability will have to be evaluated retrospectively.

Limitations

There will always be limitations in the usefulness of even rapid, technologically advanced, and accurate surveillance systems. In the case of health indicator surveillance systems, the consensus group recognized that such systems will not be as sensitive as an astute clinician in terms of detecting small numbers of cases of more severe illness. In addition, even with rapid detection, a system’s ability to mitigate the consequences of a point source outbreak such as a bioterrorist attack may not be significant if the disease has a short and relatively invariable incubation period.

Health indicator surveillance systems should be viewed as an adjunct tool to traditional systems, allowing local public health practitioners to gain knowledge of and track the health status of the community. Despite automated data collection and analysis, the most important person remains the trained public health practitioner, who, in close working relationship with the medical community, will consolidate and interpret multiple sources of information and make appropriate recommendations. Finally, once detected, an outbreak must be investigated, and the public health practitioners who use these systems must be allocated the necessary resources to launch an appropriate response.

DO WE NEED A NATIONAL PLAN?

The consensus of the group is that we need a system that works locally but can share information and be interpreted globally. The “ownership” and day-to-day monitoring of the system need to remain at the local level, which is where the response will be initiated and where the data can best be interpreted. However, all of the workshop participants agreed that system data need to be shared across jurisdictions to provide state and national epidemiologists a composite view of the population’s health status and to monitor the spread of infectious disease patterns.

To alleviate privacy and other jurisdictional concerns, shared data should be accessed only by appropriate state and federal authorities and only for public health purposes. In most instances, only aggregate statistical analyses will be required, and there will be little need to share individual patient data beyond the local level.

Most of the workshop participants believed that the greatest need in a national plan is a way to rapidly share data obtained from large governmental suppliers, such as Medicare and Medicaid. Use of such large, already available data sources would allow rapid assessment of changes in health patterns with few additional requirements placed on health care providers.

Finally, the group debated the question of who should lead this effort to put together a national surveillance plan. Most of the participants believed that the Department of Health and Human Services should take the lead in coordinating many of the independent surveillance system development efforts and facilitate the modeling of a system that provides communication across boundaries. We believe that the creation of a national working group on innovative surveillance strategies may help to harmonize current efforts and encourage further innovation.