Abstract
The United States currently faces several new, concurrent large-scale health crises as a result of terrorist activity. In particular, three major health issues have risen sharply in urgency and public consciousness—bioterrorism, the threat of widespread delivery of agents of illness; mass disasters, local events that produce large numbers of casualties and overwhelm the usual capacity of health care delivery systems; and the delivery of optimal health care to remote military field sites. Each of these health issues carries large demands for the collection, analysis, coordination, and distribution of health information. The authors present overviews of these areas and discuss ongoing work efforts of experts in each.
The United States currently faces several new, concurrent large-scale health crises as a result of terrorist activity. These crises are both real and potential, both known and unknown in their direction and magnitude. Each one carries large demands for the collection, analysis, coordination, and distribution of health information. The need for applied informatics expertise may be more pressing, and more in the public eye, than ever before.
In particular, three major health threats have risen sharply in urgency and public consciousness:
Bioterrorism—the threat of widespread delivery of anthrax, smallpox, and other agents of illness. Optimal response to bioterrorist threats requires continuous surveillance with the collection of multifactorial data; coordination and standards to combine data coming from many hospitals and agencies; analysis and logic to detect unusual, statistically significant patterns from highly specific and less-specific indicators; and reliable, centralized sources of current diagnostic and therapeutic information.
Mass disasters—local events that produce thousands of casualties, often overwhelming the usual capacity of health care delivery systems. In disasters, it is imperative to manage and dispatch resources to avoid bottlenecks, increase capacity through the temporary use of additional services, reduce idle time through precise advance communication, and track both patients and supplies.
Remote military operations—problems of preventing and treating illness and injury among soldiers in remote, inaccessible regions, thousands of miles from appropriate medical expertise. Field health care can be improved by appropriate communication from the field to local medical corps and base hospitals, secure access to patient data from central data banks, and telemedicine techniques that allow a centrally located expert to provide service to multiple field locations.
Without question, there is a critical need for information management and communication in generating coordinated, effective action to prevent large-scale health problems and to respond to them when they arise. In this article, we present overviews of each area and discuss the ongoing efforts of experts who began working on these problems long before the current national crisis.
Biosurveillance and Bio-agent Detection
The threat of bioterrorism is causing fundamental changes in the practice of both medicine and public health. Clinicians now must consider anthrax and smallpox in the differential diagnosis of acute febrile illness, and they must consider the consequences of their decisions to test, treat, or alert public health authorities with respect to the entire community as well as to the patient. More than ever before, clinicians must use, in their decision making, rapidly changing information about epidemiologic events and risk factors (e.g., level of exposure to mail and mail sorting). The effect on public health is arguably greatest in the area of public health surveillance, which is the ongoing systematic collection, analysis, and dissemination of data about disease. Public health surveillance must now occur in real time and utilize data that inherently become available much earlier than confirmed case reports or positive microbiology cultures.
Detection of a Bio-aerosol Threat
Biological warfare agents are often adapted for delivery in aerosol form, because they can simultaneously infect many persons by that route and because the air supply is much harder for military units to protect than the water or food supply. Anthrax, plague, tularemia, glanders, and smallpox are examples of organisms that can be delivered by this route.
Figure 1▶ illustrates the hypothetical effects of a large-scale bio-aerosol release of Bacillus anthracis on a suburban population. This figure is based on an economic analysis conducted by Kaufmann et al.1 In this model, a population of 50,000 persons is exposed. The three curves show the time courses of onset of nonspecific symptoms, presentations to emergency departments with signs and symptoms sufficiently specific to B. anthracis (e.g., radiographic findings in the lung on plain films or computerized tomography, coupled with a compatible clinical picture) that presumptive treatment would be initiated, and deaths. The estimates for onset of symptom presentation and percentage mortality are based on the model of Kaufmann et al. and are also influenced by recent American data.2
Figure 1 .
Epidemic curves and detection strategies for an outbreak of inhalational Bacillus anthracis due to a large bio-aerosol release. A indicates intelligence; B, biosensors; C, 500 prodromal presentations coupled with epidemiologic (e.g., work address) or intelligence data, or both; D, few cases with specific syndrome (e.g., x-rays, gram stains); E, culture-based diagnosis.
The shaded area in the curve indicates the relatively brief window of opportunity for mitigation following anthrax release. Anthrax is most treatable early, ideally before symptoms become evident; this is the rationale for prophylactic antibiotic treatment. After day 8, treatment is unlikely to significantly reduce further morbidity and mortality. Therefore, both detection and response must occur within this narrow time frame. Generating a response (treatment and prophylaxis) for hundreds of thousands of persons requires time, so the window for detection is even narrower.
Figure 1▶ suggests several possible strategies to improve the promptness of detection. There are several opportunities in this model for identification of the anthrax outbreak. Early identification could be based on a small number of patients with findings strongly suggestive or pathognomonic of anthrax infection, if the set of such patients happened to present to a single clinician or if the existence of all such cases could be known in real time by means of a detection and communication strategy.
An even earlier diagnosis could be based on the finding of an abnormally high number of patients with nonspecific symptoms that are consistent with anthrax infection. This detection method has lower specificity, since an unusual spike in presentations could derive from any of a number of agents. The detection of the spike, however, could prompt further investigation, such as probing for an epidemiologic association among the affected patients or providing increased, more specific testing of those patients. Like the previous method, this strategy would also be greatly enhanced by a real-time detection and communication system, especially one that includes epidemiologic data such as work address and home address, along with symptom data. This strategy is currently being pursued by many research groups; those projects are described in a separate paper in this issue.3
Figure 1▶ also shows that physiologic monitors and environmental biosensors that detect bacteria and viruses during the pre-infective and pre-symptomatic periods will eventually accomplish the earliest possible detection. However, such technology is currently expensive and uncommon. In the meantime, early detection of a surreptitious release will depend on monitoring people and animals for early effects of such a release and on detailed analysis of the epidemiologic characteristics of sick persons.
An ideal detector is timely, sensitive, and specific. Each practical strategy involves tradeoffs among these qualities. Therefore, a strategy that we and others are using is to couple a very early nonspecific detector with a second-stage detector that can achieve appropriate specificity. In the first stage, we are especially interested in methods that use brute-force computer searching to reveal anomalies in the data relative to historical trends. A comprehensive framework for other possible strategies is described elsewhere.4
Early Warning Through Integrated Regional Data
The detection strategies described above are much more effective if the necessary data are widely available over a large geographic region and can be integrated into an effective bioterrorism detection system. Several regions are testing such integrated systems; some of these are described by Lober et al.3 in this issue of the Journal. In the RODS (Real-time Outbreak and Disease Surveillance) test bed in western Pennsylvania, involving real-time data feeds from 17 hospitals, we have been able to demonstrate early detection through integrated data.
We found that case detection, even if it is relatively inaccurate, is an element in epidemic detection. We conducted an experiment in detecting an outbreak of respiratory illness using ICD-9-coded chief complaints from emergency department visits as the only data source. We determined that we could detect acute respiratory illness (5 days or less) not accounted for by a nonrespiratory diagnosis with a sensitivity of 44 percent and a specificity of 97 percent.5 A companion experiment to identify an influenza outbreak sounded the alarm 1 week earlier than the pneumonia- and influenza-death–based gold standard. Using the cross-correlation function, we showed that the data obtained from emergency department visits were inherently available 2 weeks earlier than pneumonia and influenza mortality data.6
Although these findings are more relevant to detection of outbreaks of food-borne illness than of inhalational anthrax, we also demonstrated a 6-day reduction in time lag for documenting reportable diseases.7
Architecture for Community Detection
We have also shown that it is feasible to obtain and integrate data of several types from different hospitals and health systems, including patient demographics, test orders, microbiology results, encounter notes, and other data obtained from electronic medical records. To date, every health system that we have approached has an emergency department computerized registration system that captures reasons for visits in real time and can provide these data to our central system through an HL7 outbound interface. Using these data alone, we can track viral symptoms, respiratory symptoms, diarrhea, rash, and encephalitis. The hospitals have been willing to share de-identified data in the interest of mutual assistance. To ensure that data from the regional early warning system can be aggregated as needed to provide statewide or national surveillance, we have committed to adherence to the standards promulgated by the National Electronic Disease Surveillance System (NEDSS).
Clinical Decision Support for Single-case Detection
The problem of detecting a single case of an extremely rare disease, such as anthrax, early enough to help the patient is a challenge that is fundamentally changing medical care. By the time such a case progresses to a pathognomonic or even relatively specific presentation, the case mortality rate will be high. Thus, early treatment, even with an uncertain diagnosis, is a goal. The field of medical informatics has very good computational models and methods for assisting physicians with the diagnosis and management of single cases. These methods have been under continuous development since publication of the seminal paper by Ledley and Lusted in 1959.8
Using a Bayesian model of inhalational anthrax, for example, we can easily model the relationship between the disease, its findings, and risk factors such as postal work. We can even establish parameters for the model using recently available anthrax case data.2 However, the prior probability of inhalational anthrax is so low that no finding or test, other than a definitive culture, has a positive likelihood ratio sufficiently high to move the posterior probability of an anthrax diagnosis above any decision threshold. This suggests that current diagnostic and treatment decision-making approaches must be expanded to include additional information or knowledge. Such information might include information about the health of other individuals in the population or about contaminations in specific locations. Although physicians unassisted by computer systems are capable of making such inferences, their ability to stay abreast of rapidly changing population-based risk data is very limited.
There are, of course, many pathogens besides anthrax, and an almost unlimited number of methods for dissemination, so it is not certain that lessons learned from these examples can be generalized to all situations. In a separate report, we discuss a clustering of known threats into ten categories, which reduces the analytic complexity without loss of generalizability.9 On the basis of our work with that set of threats, we think that the basic strategies for early detection outlined here also hold for the broader set of threats.
Requirements for Event Identification
No one expected the ramifications of the events of September 11. Other episodes of bioterrorism occurred even before that, however, such as the deliberate contamination of salad bars in Oregon with salmonella. There have also been bio-epidemics unrelated to deliberate terrorist acts, including outbreaks of hoof-and-mouth disease and West Nile virus. In all these cases, deliberate or not, we have faced the same problem of how to detect and track the illnesses.
Informatics unquestionably plays a significant role in this effort. However, in the weeks since October 4, when the first case of anthrax was detected in the 2001 episode, we saw that our national capabilities needed to be enhanced to be useful. We were able to handle some of the consequences of this incident, but we did not have pre-emptive surveillance in place to detect and track the incident in its earliest phases. We have now recognized the need for greater collaboration and data integration among hospitals. There are good systems in Pennsylvania, in Denver, and in other areas. But in most parts of the country, such systems do not yet exist.
Our job here is to detect and identify a significant event, specifically, the introduction of a biological agent into a community. Surveillance is a must, and it needs to occur in real time. Several key data elements need to be collected, which can be found in many different parts of the community. The data set includes elements typically found in an electronic medical record, such as symptoms, chief complaints, and laboratory results. It also includes dispensing patterns from pharmacies, epizootic information from veterinarians, postmortem findings from morgues, and school- and work-absence data. All these data have to be temporally correlated, and the reporting must be mandatory; voluntary reporting inevitably leads to diminished data.
As discussed earlier, sensors with high specificity and sensitivity greatly facilitate detection. Pathogen identification is where we need to start using some of the newer techniques that are available, such as polymerase chain reaction and, eventually, biochips, which can detect the DNA sequences of a number of biological agents. These devices need to automatically report to our surveillance networks to be effective. All the armed services sent biological assessment teams with polymerase chain reaction tools to New York City and Washington during the 2001 anthrax incident.
The leaders System
Surveillance is exceptionally complex; one attempt to solve this complexity is a program called leaders (Lightweight Epidemiology Advanced Detection and Emergency Response System). This system includes several modules that provide key information services for biosurveillance—data entry, data distribution, algorithms to analyze the data, knowledge distribution, and command and control.
Leaders is designed as a continuous surveillance system. It can collect symptom complexes, link them to laboratory data and other major information sources, and generate local epidemiology reports as well as provide centralized global information.
Laboratory tests are the key to and, in many cases, the gold standard for, definitive diagnosis. Polymerase chain reaction findings move the diagnosis timeline to the left more effectively than laboratory cultures do. Gram stains may be the fastest way to generate a diagnosis, with polymerase chain reaction slightly slower but more definitive. In all cases, the output of these sensors should be in a standard electronic form that can be integrated with other information sources.
Signal-to-Noise Ratio
We can use the techniques of the signal processing field to tease out a rare but specific bioterrorist incident (signal) from a sea of routine health events (noise). The “noise” is the baseline of other events that will be picked up by the sensors—for example, an influenza epidemic will increase the overall number of respiratory illnesses and may obscure cases of anthrax, which may present with indistinguishable symptoms. The “signal” is the data that indicate an anthrax release. This also points up an additional value of influenza vaccination: If we can reduce the number of patients who get influenza, we can also improve our detection by reducing noise. In other words, an outbreak of respiratory symptoms would be more suggestive of anthrax if we were confident that a flu outbreak was not occurring.
More-specific data produce a greater signal for each observation, so we do not need so many specific observations to get a good signal. Less-specific data have a lower signal or may even be part of the noise. Refinements in biosensors may yield new, specific data that carry a high signal value. Sometimes the best we can do is screen, to detect a possible event sensitively, without much specificity. Then we can analyze the data block in detail, analyzing more detailed data that can improve the signal-to-noise ratio.
Increasing Efficiency in Disaster Response
Coordination of Information from Field to Hospital
The integration of medicine, technology, and human factors is critical to the successful application of informatics to disaster medicine and other emergencies. The Human Factors Research Program (http:// www.hfrp.umm.edu) at the University of Maryland is testing several applications of technology during simulated disasters in the Maryland Emergency Medical System, and in real life at the University of Maryland Shock Trauma Center and Medical System. Such enhanced telecommunication for emergency medical care is important for future military and civilian applications, both in disaster management and in response to bioterrorist attacks with weapons of mass destruction.
Voice communication is currently available through a microwave network covering 97 percent of the surface area of the State of Maryland. This communication network enables pre-hospital field care providers to communicate directly with physicians in trauma centers (known as the Trauma Line) and other referral centers. The Trauma Line information is abstracted and put on a white-board. The recorded data include vital signs of the patient, estimated time of arrival, means of transport (helicopter or ground transportation), mechanism of injury, level of consciousness, and priority status (1=severe). The trauma team obtains the summary information from the white-board after a group-page alerts them to the patient's estimated time of arrival.
This system is relatively inefficient, because the whole trauma team responds to each admission. The voice data may omit useful information because the field care provider is performing under stress in difficult circumstances. Variations in what is reported can lead to gaps in information. There may be a lack of information about a patient, but there may also be a lack of communication of the available information. Reports confirm that observations at the injury scene are communicated only 75 percent of the time, and additional information is helpful in 52 percent of cases.10 A review of voice communications from the field showed that the receiving team rarely asked questions. The pre-hospital person also spent a lot of time waiting to find the right recipient for their information. In a typical example, 205 of 385 seconds of a communication were spent “on hold.” In disasters in which multiple ambulances and field units are reporting to multiple hospitals, this kind of delay and lack of communication of available information is unacceptable, and an improved and automated system of information transfer is needed.
We sought to improve this process by the use of a fax “notepad” linked to a cell phone in an ambulance, to send information before a patient's arrival. We compared such admissions with conventional admissions that lacked this information. We found that, at 15 minutes after admission, the trauma team achieved more of the landmarks in advanced trauma life support when patient information had been faxed ahead.
The next enhancement that we are testing is global positioning transponder system on ambulances for cardiac transport. The fully automated system transmits the current location of an ambulance to an ambulance coordination center, where it is displayed on a map of the state. This facilitates appropriate and efficient dispatch of ambulance units. The system also pages the team in the cardiac catheterization laboratory. As the ambulance approaches the hospital, it passes through three zones of activation, and the team is paged 20 min, 10 min, and 5 min before its predicted arrival (based on location), so that they can coordinate their activities with the arrival. Other parts of the system allow a physician at the hospital to look at electrocardiogram and vital sign waveforms transmitted from the ambulance.
In disaster situations, the information provided by such a system could help emergency teams anticipate the workload for an incoming patient, prepare early intervention before a patient's arrival, and get a better sense of the flow of multiple patient arrivals.
Rapid Acquisition of Multi-patient Information from a Disaster Site
Another project, known as “MobiDoc,” makes use of next-generation wireless technology to create an entirely mobile telecommunication system. This communication kit, which is the size of a briefcase, contains eight cell phones and wireless data-acquisition devices that are connected to the cell phones. A field team can perform multiple charting, vital-signs monitoring, image collection, and other data acquisition tasks for multiple patients; the data are sent securely to the hospital's intranet, where they can be viewed on a Web browser by control personnel. We are currently able to transmit up to five images a second through this system.
All these systems allow more patient data and arrival information to be communicated to a hospital or disaster control center. The systems need to be coupled with intelligent algorithms to make use of the data to balance resources, e.g., to make sure that patients who need CT scans are equally distributed among the hospitals that can provide them. Furthermore, while these systems allow us to make more efficient use of existing resources, we also need to find ways to temporarily increase capacity in a local area when disaster strikes.
Increasing Capacity
In June 2001, Tropical Storm Allison, the “rain of terror,” caused huge flooding in Houston. Emergency departments already working to capacity had to face great increases in patient volumes. In addition, the generator of one large hospital was flooded and the entire hospital lost power, forcing closure of their emergency department and evacuation of all their patients. In a matter of hours, the system went from normal activity, which is already close to capacity, to extreme overload.
The Air Force placed a rapidly deployable mobile surgical hospital in Houston, which was able to handle about 1,000 patients. Although it did not meet the entire need, it took the peak overload off the other emergency rooms so that they could again manage their own patient workloads. These resources are available through the Federal Emergency Management Agency (FEMA) and are dispatched on the basis of established entry and exit criteria.
Information Elements for Incident Management
The leaders system includes several modules—a dashboard for overall viewing and coordination of information; resource allocation modules to track ambulances, intensive care unit beds, burn treatment facilities, and other resources; emergency contact lists and automatic notification software; checklist managers to maintain guidelines for hazardous material and other key procedures; and casualty tracking. These are the critical elements of incident management software.
Such software should be deployable for use by local and regional emergency management agencies as well as by FEMA and other national agencies. The leaders system is built on an application service provider model, so that with appropriate permissions, the system can be accessed for use by a variety of agencies.
Tele-presence
Tele-diagnosis
The third major problem area is the provision of expert assistance to remote medical caregivers in military operations. Through telemedicine technologies, it may be possible to have an expert in one location directing, and using, the eyes and hands of field personnel. A specialist at a single base station could then provide assistance to many frontline units concurrently.
We have been looking at a task model for acute stroke, using transportation time (the time for a patient to be transported to the hospital) as part of the diagnostic time. In stroke, time to diagnosis is critical because the window for thrombolytic treatment is so small.
Using telemedicine technology, a neurologist in the hospital can complete a National Institutes of Health stroke scale examination in 8 minutes, while the patient is transported to a hospital. If the neurologist concludes that the patient is probably having an acute stroke, the patient is not taken to the emergency department at all, but straight to the CT scanner. Our aim is to reduce arrival time at the scanner by 30 minutes, compared with the time it takes when pre-hospital telemedicine assessments are not used. We have seen successful reversal of strokes with this system that may not have been possible otherwise.11 This model for remote diagnosis can be applied to complicated injuries and medical syndromes in military operations at any distance.
Telemedicine inputs can be provided from the home, the clinic, primary hospitals, and ambulances into a single control area in a tertiary trauma center. Once received, the information can be transmitted through an intranet to any desktop in the institution. Physicians inside the hospital can thus coordinate their activities from several locations, such as radiology and operating rooms.
Tele-presence for Procedures
Another side of the telemedicine coin is remote guidance of procedures. We looked at tele-supervision of emergency airway management of a trauma patient. The procedure was guided by a task communication algorithm that specified for field personnel how to look at the chest, when to listen to breath sounds on the left or the right, when to check carbon dioxide monitors, and so on. In an experiment, we used this tool to provide airway management tele-consultation, or tele-mentoring, to a group of trainees who were intubating human beings for the first time. One group used conventional equipment. The other group used the above algorithm with a head camera and a communication system that brought the mentor into the process. The group with the tele-mentor took a little longer to insert the tube properly. This is probably because the mentor needed more time to get oriented, a matter of 1 to 2 seconds.
In another project, funded by the Army Research Institute, we are looking at the remote management of trauma cases, which perhaps bears a more direct relationship to military operations. In this experiment, a trauma surgeon in a tele-control center directs the activities of the trauma team, which is “remotely” situated 50 to 100 feet away but out of direct sight and hearing of the trauma surgeon. Equipment includes a wireless headset for communication with the team leader, a wireless video head camera that transmits images from the wearer, a pan tilt-and-zoom camera, and an overview camera.
We compared the trauma evaluation and management process in this set-up to that of a similar case in which the trauma surgeon was on the scene. When remote, the surgeon asked more questions (72 vs. 60 percent of communications) and gave fewer instructions (28 vs. 40 percent) than when on site. These data suggest that the remote surgeon was not so sure about what was going on and possibly not so confident about giving instructions. The system is, nonetheless, better than having no trauma surgeon present at all, although it needs improvement to be as good as direct presence.
Recommendations
The recent terrorist incidents in the United States should be taken as a “call to arms” to medical informaticians. As discussed in this article, informatics has a prominent role to play in the prevention and management of terrorist attacks of many kinds.
Coordination of information among local, regional, and national agencies needs to be much better. Several policy changes may be needed to accomplish this, including:
Greater deployment of biosurveillance systems in more regions of the country, possibly through application service provider models or other rapid-deployment techniques
Support for research that leads to optimal algorithms for evaluating early symptom and epidemiologic data, to better identify bioterrorist attacks
Consolidated sources of reliable education for medical providers and for the public, about the current status of any biologic incident and the best steps to take in response to it
Coordination of efforts in disaster preparation and acute disaster management
Education in appropriate application of enhanced telecommunication technologies
Fewer barriers to the exchange of de-identified public health information among health systems, with appropriate confidentiality protections
Policies for rapid, urgent deployment of increased resources in disasters
Reimbursable billing for screening diagnostics
Adoption of standards for data exchange, and a standard data collection set such as NEDSS.
Conclusions
There are many critical roles for superior information management in the detection, prevention, and management of disasters. Many of these roles have been explored at a few research centers. We have shown that existing systems can improve early detection of bioterrorist agents, can improve the management of large-scale disasters, and can deliver medical expertise to remote field sites.
For a reliable, efficient, immediate (and sometimes pre-emptive) response, optimal techniques must be established and deployed over all target regions. Centralized services for information and centralized clearinghouses for data must also be established, to maximize the available data for surveillance and to allow rare resources and expensive reserve capacity to be applied over many sites. Government and payer support is needed to promote data exchange standards, to support research, and to make surveillance economically feasible.
Dr. Wagner is supported in part by grants GO8 LM06625-01 and T15 LM/DE07059 from the National Library of Medicine, contract 290-00-0009 from the Agency for Healthcare Research and Quality; a contract from the Air Force under the DARPA (Defense Advanced Research Projects Agency) Biosurveillance Program; and by cooperative agreement U90/CCU318753-01 from the Centers for Disease Control and Prevention.
Dr. Mackenzie is supported in part by grants from the National Science Foundation (IIS 9900406), the National Library of Medicine (GNL M000003), and Army Research Institute (DASW01-99-K-0003) and by a contract from the Telemedicine and Advanced Technology Research Center, Fort Detrick, Maryland (DAMD 17-01-P0827). The projects described in his presentation were performed in collaboration with Dr. Yan Xiao and Peter Fu-Ming Hu.
The opinions expressed in this article are solely those of the authors and do not necessarily represent the official views of granting agencies or the Department of Defense.
Footnotes
The article is based on a panel presentation given at the 2001 AMIA Annual Symposium in Washington, DC, on Nov 5, 2001. The major contributor for each topic was Dr. Wagner, for biosurveillance and bio-agent detection; Brig. Gen. Schafer, requirements for event identification, signal-to-noise, capacity increase in disasters, and the leaders system; Dr. Mackenzie, coordination of field information, and telemedicine; and Dr. Teich, introduction, recommendations, and conclusions.
References
- 1.Kaufmann A, Meltzer M, Schmid G. The economic impact of a bioterrorist attack: Are prevention and post-attack intervention programs justifiable? Emerg Infect Dis. 1997;3:83–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Update: Investigation of bioterrorism-related anthrax and interim guidelines for clinical evaluation of persons with possible anthrax. MMWR Morb Mortal Wkly Rep. 2001;50:582–5. [PubMed] [Google Scholar]
- 3.Lober W, Bryant T, Wagner M, et al. Roundtable on bioterrorism detection: information-system-based surveillance. J Am Med Inform Assoc. 2002;9:105–15 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Wagner M, Tsui FC, Espino J, et al. The emerging science of very early detection of disease outbreaks. J Public Health Manag Pract. 2001;7(6):51–9. [DOI] [PubMed] [Google Scholar]
- 5.Espino J, Wagner M. Accuracy of ICD-9-coded chief complaints and diagnosis for the detection of acute respiratory illness. Proc AMIA Annu Symp. 2001:164–8. [PMC free article] [PubMed]
- 6.Tsui FC, Wagner MM, Dato V, Chang CCH. Value of ICD-9-coded chief complaints for detection of epidemics. Proc AMIA Annu Symp. 2001:711–5. [PMC free article] [PubMed]
- 7.Panackal AA, M'ikanatha NM, Tsui FC, et al. Evaluation of automatic electronic laboratory reporting of notifiable infectious diseases at a large health system. Pittsburgh, Pa.: University of Pittsburgh, 2001. [DOI] [PMC free article] [PubMed]
- 8.Ledley RS, Lusted LB. Reasoning foundations of medical diagnosis. Science. 1959;130:9–21. [DOI] [PubMed] [Google Scholar]
- 9.Dato V, Wagner M, Allswede M, Aryel R, Fapohunda A. The nation's current capacity for the early detection of public health threats including bioterrorism. Washington, DC: Agency for Healthcare Research and Quality, 2001.
- 10.Brown R, Warwick J (). Blue calls: Time for a change? Emerg Med J. 2001; 8(4):289–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.LaMonte MP, Xiao Y, Mackenzie CF, et al. Mobile telemedicine for the Brain Attack Team. J Stroke Cerebrovasc Dis. 2000; 9:128–35. [DOI] [PubMed] [Google Scholar]