Abstract
Objective
This paper presents a model designed to enable rapid detection and assessment of biological threats that may require swift intervention by the international public health community.
Design
We utilized Strauss’ grounded theory to develop an expanded model of social disruption due to biological events based on retrospective and prospective case studies. We then applied this model to the temporal domain and propose a heuristic staging model, the Wilson–Collmann Scale for assessing biological event evolution.
Measurements
We retrospectively and manually examined hard copy archival local media reports in the native vernacular for three biological events associated with substantial social disruption. The model was then tested prospectively through media harvesting based on keywords corresponding to the model parameters.
Results
Our heuristic staging model provides valuable information about the features of a biological event that can be used to determine the level of concern warranted, such as whether the pathogen in question is responding to established public health disease control measures, including the use of antimicrobials or vaccines; whether the public health and medical infrastructure of the country involved is adequate to mount the necessary response; whether the country’s officials are providing an appropriate level of information to international public health authorities; and whether the event poses a international threat. The approach is applicable for monitoring open-source (public-domain) media for indications and warnings of such events, and specifically for markers of the social disruption that commonly occur as these events unfold. These indications and warnings can then be used as the basis for staging the biological threat in the same manner that the United States National Weather Service currently uses storm warning models (such as the Saffir-Simpson Hurricane Scale) to detect and assess threatening weather conditions.
Conclusion
Used as a complement to current epidemiological surveillance methods, our approach could aid global public health officials and national political leaders in responding to biological threats of international public health significance.
Introduction
The current threat of a pandemic and general evolving situation of global emerging disease threats requires increasingly sophisticated cross-cultural approaches to detecting and tracking biological events affecting humans or animals. This paper presents a novel model designed to enable rapid detection and assessment of biological events that may require swift intervention by the international public health community. Used as a complement to current epidemiological surveillance methods, our approach could aid global public health officials and national political leaders in responding to biological threats of international public health significance.
Background
In the wake of severe acute respiratory syndrome (SARS) in 2002–2003, and with the current threat of pandemic influenza, there is a critical need for a rapid means of detecting, assessing, and evaluating biological threats of potential transnational significance. Such a capability is essential to encourage communication between decision makers and the international community at large, a development that can facilitate diagnoses and intervention, among other responses. Unfortunately, the evolving profile of a socially disruptive event later known as SARS in southern China was not noticed by the World Health Organization until many months later, at a time when the disease had already translocated in the air traffic grid to eight countries including the United States. 1,2 (See Appendix 1, available as a JAMIA on-line data supplement at www.jamia.org).
The international public health community has recognized the need to develop this capability. For example, one proposed strategy for rapid response to pandemic influenza is to identify the first focus of human-to-human efficient transmission of the H5N1 influenza virus anywhere in the world, then to deploy antivirals to the area within 48 hours. Proponents claim that this strategy may avert pandemic if it is enacted within the recommended time frame; however, there remains great controversy regarding its feasibility given current constraints in comprehensive global disease surveillance. 3
Currently, official notification of international health threats is provided by the World Health Organization (WHO), in partnership with the Global Outbreak Alert and Response Network (GOARN), and supported legally through WHO’s promotion of the International Health Regulations. WHO and GOARN utilize a service called the Global Public Health Intelligence Network (GPHIN) to scan media articles globally for references to disease outbreaks and epidemics. † GPHIN is limited, however, by the volume of media material that can be processed in multiple languages. Additional limitations may prevent WHO from detecting and assessing a rapidly spreading epidemic, including the lack of a proper public health infrastructure in the country experiencing an outbreak, the involvement of a previously uncharacterized pathogen, or the WHO Member State’s unwillingness to report the event. 1,4
Unofficial mechanisms also exist for reporting international heath threats, such as ProMED, a not-for-profit organization that relies on volunteers throughout the world who submit information about infectious diseases and surrounding issues. Most of these reports are media articles. In an attempt to display only the most relevant information, all submissions are processed by a group of volunteer moderators with substantial field expertise. This approach is limited by the number of staff available to process the volume of reports submitted throughout the world in multiple languages. 5,6
Model Formulation Process
We drew on Strauss’ grounded sociological theory to develop a set of indications and warnings of social disruption illustrating the dynamic properties of each type of social response over time. We chose Strauss’ grounded theory methodology as a means to systematically sample and analyze data to develop theoretical concepts. Grounded theory calls for repeating this process until no further modifications are made to the theoretical model. 7
We began our model formulation with a literature review on social disruption due to epidemics. We then drafted the first version of our indication and warning taxonomy based on the 2002–2003 emergence of SARS in southern China [Appendix 1, available as a JAMIA on-line data supplement at www.jamia.org]. The model was then refined based on two additional retrospective case studies, Rift Valley fever (RVF) in Kenya (1997–1998) and Venezuelan equine encephalitis (VEE) in Venezuela and Columbia (1995) [Appendices 2 and 3, available as a JAMIA on-line data supplement at www.jamia.org]. Finally, the model was tested prospectively through a media harvesting engine based on keywords corresponding to the model parameters.
For our prospective case studies, we obtained public-domain reports—including news and business reports presented on television, on the radio, over the Internet, and in newspapers—that provided comprehensive coverage of the Pacific Rim from May 1, 2005, to July 15, 2005. For our search, we used MiTAP. Originally developed under the Defense Advanced Research Projects Agency’s Translingual Information Detection Extraction and Summarization Program (DARPA-TIDES), MiTAP offered an online news service and information retrieval system that captures and categorizes information according to user-defined, searchable newsgroups. 8–11 Using MiTAP, we searched the above media sources iteratively using multiple keyword combinations. These searches included only articles published in English or translated into English by regional news sources. To extend the range of our data sources, we conducted additional searches with MiTAP using Chinese-, Russian-, and Korean-language local news sources and regional business and financial reports, sometimes with the aid of a bilingual human translator. We did not assess inter-coder variability, nor perform a comparison with other global surveillance systems in this study.
Model Description
Social disruption is defined as “a severe alteration in the normal functioning of a social system” that, under extreme conditions, can lead to the collapse of social infrastructure. 12 Such extreme social disruption exacerbates the effects of the biological event. By containment, we mean the process by which an indigenous public health and medical infrastructure undertakes countermeasures aimed at controlling an outbreak or epidemic.
Both adaptive and maladaptive social responses to disease epidemics have been studied previously. McGrath (1991) examined the historical ethnographies of 229 ethnic groups from North and South America, Europe, Africa, the Middle East, Russia, and Oceania, and recorded six basic social responses to epidemics: mass evacuation or flight from the site of the epidemic, extraordinary therapeutic or preventive measures, scapegoating of individuals or institutions, acceptance of disease, ostracism of the ill, and conflict. Of these, flight was found to be the most common. McGrath noted that the acute, dramatic appearance of disease, particularly unfamiliar disease, with high rates of morbidity and mortality provoked the strongest social response. She observed that if established indigenous countermeasures failed, innovative countermeasures were undertaken, followed by social disruption that began with flight, then proceeded to acceptance of disease, to rejection of authorities, and ultimately to conflict. This sequence, she argued, depicts a society that has degraded from normal functional status to disintegration. 12 We infer social disruption may be characterized as an emergent property triggered by a biological threat that defies countermeasures, control, and containment.
In this paper, we expand upon prior sociological and anthropological descriptions of social disruption by applying the concept to the detection and assessment of biological threats. Indications and warnings (or markers) of social disruption, used in tandem with explicit reports of disease, can identify an emerging biological event and indicate its level of containment. Three general conditions favor the use of this approach: inadequate support for local disease reporting, suppression of information about health crises, and/or the emergence of a previously unrecognized pathogen or of a well-known pathogen that displays dramatically different transmission properties.
Our indications and warning categories included reports of human or animal disease, demand for medical services, local perception of threat, official acknowledgment, official action, and integrity of infrastructure. These indication and warning categories represent important ways in which an emerging epidemic might disrupt the everyday social life of individuals, communities, and institutions; therefore they might appear in local, regional, or national media sources. Our expanded capture of media reports from local and regional sources provided crucial information because such sources reflect local knowledge of typical and atypical phenomena.
We hypothesized the selected indications and warnings would display cross-cultural reliability. For example, we expected to find evidence of local perception of threat wherever and whenever individuals initially become aware of an emerging epidemic. We used the term “parameter” to refer to manifestations of an indication and warning category appearing in a specific cultural or historical context. We expected parameters for most categories to display not only cross-cultural reliability, but also cross-cultural variation depending on the impact of the disease and local conceptions of appropriate response. The parameters for some categories, such as official acknowledgment, may display less cross-cultural variation than others, a point for exploration in future analyses.
We also identified three variable properties of media reports—recurrence, elevation, and diversification—that may serve as indirect measures of the intensification of indications and warnings over time. “Recurrence” refers to additional reports of a phenomenon already described at least once in the media. Media reports may also describe increases in a phenomenon, indicating “elevation” of its manifestation. “Diversification” refers to the emergence of multiple parameters of a single marker. As media reports repeatedly describe elevated levels of increasingly diverse parameters of particular indications and warnings, a picture emerges of an uncontained, intensifying biological threat with increasing potential for international translocation.
Our analyses highlighted indications and warnings in two broad categories—direct and indirect—that were consistent across the cultures, diseases, and situations that we examined. Direct indications and warnings, such as reports of a disease outbreak, derive from medical and veterinary information and refer specifically to a biological event. Indirect indications and warnings encompass qualitative information regarding social disruption and containment status, such as reports suggesting that individuals and organizations have begun changing their daily routine activities in response to a perceived biological event. These indications and warnings are detailed below, followed by a description of our heuristic staging model for characterizing the evolution of a biological event.
Direct Indications and Warnings
Direct indications and warnings are used in traditional international disease reporting by organizations such as GOARN and ProMED that follow the standard epidemiological practice of initial surveillance followed by event verification. Direct indications and warnings included diagnostic impression, which refers to whether the causative pathogen has been reported to be known and well understood or unknown. Our analyses revealed that diagnostic impressions were often ambiguous, particularly when an indigenous society was confronting a pathogen that was either an unrecognized or unexpected agent or was identified initially as well understood, but defied countermeasures thought to have been effective in the past.
Direct indications and warnings also encompassed epidemiological features. Specific parameters for this marker included unique clinical presentation, geography, high morbidity or mortality, unexpected appearance of disease in relation to season, and discreet population involved. Geography referred to whether an outbreak of disease was unifocal (described in patients or animals located at a single facility) or multifocal (described at multiple facilities or present at a regional level). It also included descriptions of expected or unexpected propagation patterns, such as the presence of multiple foci when previous experience with a disease dictated a single-point origin for an outbreak or epidemic. Discreet population referred to a high incidence of disease observed in a specific group, such as children or health care workers. As discussed below, the presence of a disease that was multifocal and was directly incapacitating health care workers pointed to a possible compromised medical infrastructure.
We did not observe substantial differences between direct markers of biological events affecting animals and direct markers of such events affecting humans, except that animals tended to experience disease on farms and were cared for by veterinarians, whereas humans tended to experience disease in hospitals and were cared for by medical workers.
Indirect Indications and Warnings
Indirect indications and warnings included official acknowledgment, official action, demand for medical services, local perception of threat, changes in business practice, and integrity of infrastructure.
Official acknowledgment referred to the behavior of an indigenous government when communicating threat information to its citizens. In our analyses, we observed three basic messages that governments sent: acceptance or declaration of an event, denial, or no response. We noted that public announcements by local officials indicated a unifocal biological event, while those by national officials suggested an epidemic spreading throughout a region. Types of announcements included explicit declarations of the occurrence of an outbreak, a health alert or declaration of a public health emergency, and calls for international assistance. When a biological event has defied all local countermeasures, announcements became more emphatic and culminate in one or more requests for international assistance.
Official action referred to an indigenous government’s response to a biological event. It included official investigation, implementation of countermeasures, activation of biosurveillance or screening, information suppression, and prosecution:
• Local official investigation implied a lower level of official concern regarding a biological event than investigation conducted by national officials or international organizations called in for assistance, such as WHO.
• Implementation of countermeasures covered a wide variety of actions, such as vector control measures (like mosquito spraying campaigns), hygiene campaigns (like official orders to disinfect elementary school classrooms), pharmaceutical deployment (like vaccination campaigns), preemptive infrastructure closure (like closure of sites of public congregation, such as festivals, to prevent disease transmission), official orders to quarantine infected individuals, propaganda or public awareness campaigns, and closure of borders and ports of entry to prevent the international spread of disease.
• Activation of biosurveillance or screening, such as screening passengers at airports for disease, revealed a shift from passive to active surveillance.
• Information suppression was a frequent behavior, irrespective of the country involved. An indigenous government’s attempts to deny the presence of a biological threat stemmed from concerns about inciting public panic, from the threat’s perceived effects on the economy and tourism, and from a lack of official understanding of the threat.
• Prosecution of individuals, organizations, and companies occurred for a variety of reasons, including failure of a government official to handle an event properly, unauthorized dissemination of information, profiteering, and disregard of orders to quarantine.
Demand for medical services encompassed parameters that revealed shifts in supply and demand, such as increased demand for medical care, pharmaceuticals, and supplies and mobilization of those resources:
• Increased demand for medical care was indicated by a rise in patient presentations at hospitals; demand for specialized care, such as intensive care unit support; and calls for military assistance to support civilian care providers.
• Demand for pharmaceuticals and supplies included increased purchase or use of medications, vaccines, face masks, and gloves. As a biological event progressed to fully defy local countermeasures, depletion of pharmaceutical and medical supply stockpiles was seen. If the public was aware of this stockpile depletion, both seeking behaviors (such as travel to other areas to find medicine) and innovative behaviors (such as using bras as face masks) was observed. Stockpile depletion often prompted calls to neighboring regions and countries for assistance and mobilization of resources.
Local perception of threat denoted public awareness of a change in the local status quo of disease incidence. For example, social anxiety was noted when a disease appearred abruptly and the public was aware that it may be defying countermeasures. Anxiety was also indicated by increased telephone and Internet use. Heightened incidence of depression or apocalyptic feelings was observed and prompted special psychiatric intervention, such as the creation of suicide hotlines or psychological support centers. As the media’s references to anxiety turned into references to panic, hoarding and self-preservation behaviors were noted, including changes in purchasing behavior (such as panic buying of staples or home remedies) and avoidance of sites of public congregation (such as subway stations, stores, and festivals). As a biological event became more difficult to contain, mass voluntary evacuations were observed. The public’s loss of confidence in the government’s ability to mount countermeasures against the disease led to expressions of dissent in newspaper editorials, public demonstrations, and eventual rioting.
Changes in business practice were observed when demand outstripped supply and certain items were at a premium. Changes in prices of staples and medicine were observed, as well as profiteering, changes in advertising, and formation of black markets.
Integrity of infrastructure denoted the progressive strain that occurred when a disease defied countermeasures. Compromised infrastructure was observed when hospitals were reported to be overburdened or inundated with patients, often to the point of creating new disease wards or placing patients in hallways because of a lack of space. Infrastructure collapse was observed when hospitals were forced to close or turn patients away because an overwhelming number of patients were presenting, because resources were depleted, or because health care workers were incapacitated by the disease. As an indigenous medical infrastructure progresses from compromise to collapse, a combination of multiple social disruption parameters was observed, indicating a full social crisis: declaration of martial law to preserve basic government functions, open conflict between citizens and government officials, widespread economic damage, and heavy dependence on international support. These parameters indicated maximal social disruption and a loss of ability to control or contain the disease.
Staging Model: The Wilson–Collmann Scale
Other investigators have attempted to define event evolution as a function of media reporting. Cieri and colleagues (2002) proposed that an event be defined as “a specific thing that happens at a specific time and place along with all necessary pre-conditions and unavoidable consequences.” 13 Makkonen (2003) observed that a seminal event can lead to various related events and outcomes, and the initial cause of these events may become less obvious over time. 14 Chin and colleagues proposed that a media-reported event can be considered “a life form with stages of birth, growth, decay, and death,” and that the maintenance of the reported event is dependent on sensationalism. 15 As discussed earlier, our analyses identified variable properties of media reports that are potentially useful as indirect measures of the intensification of indications and warnings over time: recurrence, elevation, and diversification. By these measures, biological events were reported as increasingly complex phenomena over time, whose “nourishment” was dependent on whether the biological event in question was perceived by the local community to remain an active issue of concern.
We propose that a heuristic staging model based on the indications and warnings of social disruption described above is better suited than current methods to detect and assess the dramatic appearance of diseases, particularly unfamiliar diseases, with high rates of morbidity and mortality. Such evolving biological events provoke the strongest social response.
Our heuristic staging model was intended to serve as a guide for analysis and interpretation. In daily real-world analysis, it would be unlikely to observe a logical flow of information representing the systematic progression of a biological event and associated social responses. Gaps in information were likely because of inadequate local disease reporting, information suppression, the emergence of a previously unrecognized pathogen, or the emergence of a well-known pathogen that displayed dramatically different transmission properties.
The basic framework for our model followed a biological event as it evolved from unifocal, to multifocal, to uncontained, and finally to a state that induced social collapse. On the whole, the model’s first two stages (1 and 2) were reminiscent of public health definitions of biological events, whereas the final two stages (3 and 4) were based on sociological definitions. The following descriptions focus first on indication and warning stages for biological events that affect humans, and then present caveats for biological events that affect animals. Obviously, indications and warnings for the former diseases impact medical services and those for the latter diseases affect veterinary services. For zoonotic pathogens (those that affect both animals and humans), indications and warnings exist that affect both types of service.
Stage 0: Environmental Risk Conditions Present
Stage 0, indicative of a potential increased risk for disease, is a pre-event condition that applies to specific mosquito-vectored and waterborne diseases in certain areas of the world, such as RVF in Kenya, VEE in Venezuela, and diarrheal illness in India and Bangladesh. We found that in most of these cases, local public and government officials appeared to respond to a history of disease following both expected and unexpected large-scale seasonal floods. This response included public awareness campaigns, active disease surveillance, and mosquito spraying.
One important feature of the Stage 0 condition is the impact of excessive flooding on medical infrastructure and the ability of the indigenous community to respond to the subsequent emergence of disease. This impact was observed during the 1995 epidemic of VEE in Venezuela and the 1997–1998 epidemic of RVF in East Africa. We observed that if the local infrastructure was already compromised prior to the appearance of a rapidly transmissible disease, the associated social disruption might proceed at a much faster pace.
We propose that Stage 0 is relevant for Kenya, Venezuela, India, and Bangladesh. Stage 0 is likely to be relevant for other areas of the world as well; however, our analyses were limited to our case studies.
Stage 1: Unifocal Biological Event
This stage represents the beginning of an identified biological event, when human cases of a disease have presented to a single medical facility and the event has therefore appeared as a unifocal phenomenon. This condition is analogous to the epidemiological term “outbreak,” which denotes the unusual appearance of a disease, above baseline but limited in scope. Our model simplifies this definition to mean the appearance of cases at a single medical facility, and thus reflects the uncertainty of dealing with media sources rather than information vetted by the public health community.
▶ depicts the indications and warnings seen during Stage 1. Direct indications and warnings include reports of human disease that describe clinical impression and epidemiological features of the event. Indirect indications and warnings, which are not substantial in Stage 1, fall into two categories: official acknowledgment and official action. Official action is limited and typically includes official investigation and low-level countermeasures, such as vector control campaigns (like mosquito spraying).
Figure 1.
Stage 1 indications and warnings of biological events involving humans.
For biological events that affect animals, the basic description is the same except that explicit reporting originates in official and unofficial communities linked to veterinary medicine, agriculture, industry, field biology, and forestry, as expressed to local media. Locations where such events occur include veterinary, research, agricultural, and industrial facilities; park lands; and local communities. The staging of such events is similar to that for human disease, with the event being reported to occur at one such facility. A key difference, however, is the implementation of countermeasures. An official order to quarantine the involved animal or animal herd may be reported. If a pathogen with serious economic repercussions, such as foot and mouth disease in cattle or H5N1 avian influenza, is suspected to be present, mass culling of animals may be mandated for both the involved facility and surrounding areas. For the current situation with H5N1 avian influenza, this stage may include military support for mass culling of animals in developing nations.
Stage 2: Multifocal Biological Event
In Stage 2, disease is present in multiple facilities, and is analogous to the public health definition of an “epidemic.” Stage 2 therefore describes a phenomenon that is multifocal but contained (as perceived by the indigenous society). Early indications and warnings in this stage include reports of increased demand for pharmaceuticals and medical supplies (see ▶).
Figure 2.
Stage 2 indications and warnings of biological events involving humans.
As the biological event progresses, direct indications and warnings may include more epidemiological details. However, it is at this point that concern is expressed publicly about whether the initial clinical impression was correct. This step takes place if the pathogen in question is a well-known one that is now displaying new transmission characteristics, or if the initial clinical impression was wrong, and a newly emergent pathogen is present.
Indirect indications and warnings include elevated official acknowledgment, with national in addition to local officials making statements regarding the event. Official action progresses to include activation of biosurveillance or screening of individuals and attempts to isolate those infected. Under the category of official action, official investigation progresses to involve national and occasionally international organizations. Countermeasures expand to include disinfection campaigns, vaccine deployment, and official treatment recommendations, reflecting a reactionary posture by the indigenous government to exert control over the situation. Increased demand for medical care, pharmaceuticals, and supplies also appears. Stage 2 sees the first documentation of local perception of threat, with the public reported as being “anxious” or “concerned” about the disease.
When animal populations are involved, a “multifocal” event occurs in more than one of the locations described for Stage 1, above. A key epidemiological feature for Stage 2 events involving animals is documentation of which animal species are involved. This information is important for determining the possible identity of the pathogen. If the event involves a suspected pathogen of economic consequence, more aggressive and widespread animal culling is observed, frequently with military support. Additionally, domestic and international trade restrictions may be implemented by both the host nation and other countries. An official order to quarantine an entire affected facility may be issued. Active screening of animals and animal products at farms, festivals, markets, businesses, and ports of entry is also seen. Disinfection of individuals, equipment, and transportation vehicles is observed, along with roadblocks and checkpoints.
Stage 3: Uncontained Multifocal Biological Event
Stage 3 denotes an uncontained multifocal event that affects the medical infrastructure to the point of strain. Key indications and warnings (see ▶) include declaration of the inability to contain or control the disease in question, depletion of vaccine and drug stockpiles and medical supplies, and concerns about whether medical facilities can continue to handle patients. If the disease entity continues unabated, social collapse may ensue.
Figure 3.
Stage 3 indications and warnings of biological events involving humans.
Direct indications and warnings escalate to include health care workers becoming infected, which may occur with such pathogens as influenza or SARS. This marker is an ominous sign, as it indicates a serious potential threat to the continued functioning of medical facilities.
Further diversification of indirect indications and warnings is noted in Stage 3. Official acknowledgment includes declaration of a health emergency and requests for international assistance. Official action is increasingly aggressive, with international organizations featuring more prominently in official investigations. Such action includes preemptive infrastructure closures (of schools, festivals, and public transit, among others), official orders to quarantine, initiation of propaganda campaigns, and border and port-of-entry closings; information suppression and prosecution of citizens and organizations appear as well.
Also in Stage 3, demand for medical services diversifies. There is increased demand for specialized, innovative, or alternative medical care. Demand for pharmaceuticals and medical supplies progresses to the point of stockpile depletion and subsequent mobilization of resources from neighboring regions. Local perception of threat evolves and leads to new behaviors, such as hoarding and self-preservation (e.g., panic buying of staples); avoidance of sites of public congregation; and expressions of dissent toward officials whom the public now deems incompetent in their handling of the event. Changes in business practice reflect market responses to a severe shift in the supply-and-demand ratio, as local medical resources become depleted. Price gouging and formation of black markets are seen. Integrity of infrastructure is observed to be compromised, with medical facilities reporting “strain” or seeing patients at “full capacity,” implying few reserves left to treat the public. Multiple schools and businesses close on account of widespread illness—not because of an official mandate to control its spread, but rather because the disease is directly affecting these institutions. These parameters are signs of severely compromised social functioning.
In Stage 3 events involving animals, the military’s participation in countermeasures is more widespread and common. Preemptive infrastructure closures include individual businesses, farms, festivals, and the stock market. Local perception of threat is manifested in panic selling of farm animals at risk of exposure to the disease, panic buying of animal products in anticipation of a market restriction, and avoidance of public places. Public dissent may be observed as both the disease itself and official countermeasures, such as animal culling, begin to have a severe effect on the livelihood of the agricultural community. A compromise in the integrity of infrastructure is noted with the closure of individual businesses, farms, and festivals; these closures are not preemptive, but rather stem directly the pathogen itself, from attendant countermeasures, and from social anxiety. Severe economic damage for the host nation is imminent.
Stage 4: Maximally Disruptive Biological Event
Stage 4 is the regional- or national-level end point for a socially disruptive biological event, when social collapse results from the disease’s sustained defiance of countermeasures and maximal social disruption occurs. Key indications and warnings of a Stage 4 event (see ▶) include conflict due to public outcry over the handling of the event, mass evacuations, and refusal of the medical infrastructure to see patients.
Figure 4.
Stage 4 indications and warnings of biological events involving humans.
Stage 4 is defined principally by indirect indications and warnings. Official action is maximally aggressive, with preemptive infrastructure closures occurring systematically across multiple sectors (such as schools, festivals, and public transit). Systematic closures and redirection involving the entire local medical infrastructure are noted, with medical facilities becoming wholly devoted to treating those with the disease. Mass quarantines of thousands of people may be seen. Information suppression and prosecution of citizens and organizations escalate to arrests and threats of capital punishment if control measures such as quarantine are violated. Medical facilities may also be punished for refusing to see patients.
Demand for medical care results in a total depletion of pharmaceuticals and medical supplies; military transport of materiel on a national level, with mobilization of internationally acquired supplies, may be observed. Absenteeism is observed among health care workers and first responders due to disease, fear, or both. Consequently, dependence upon military medical support reaches maximum levels.
Local perception of threat appears as an abject lack of public confidence in the government’s ability to handle the situation. This lack of confidence takes such forms as open defiance of official mandates, like quarantine or curfew; rioting; and open conflict between civilians and law enforcement or military personnel. Psychological impact may be observed, with signs of mass depression, anxiety, and apocalyptic and suicidal ideation. Counseling hotlines and support services may be inundated with requests for assistance. People thought to have the disease may be ostracized. Patients may avoid medical facilities for fear of receiving inadequate care, being placed in isolation or quarantine, or becoming infected by the pathogen. Conversely, medical facilities may be at the point of requiring systematic security—both to hold back the panicked public and to prevent health care workers from leaving.
Business practices evolve to the limit of tolerable price gouging. Emergency price controls on pharmaceuticals and basic staples may be broadly implemented.
Integrity of infrastructure reaches the point of collapse at the societal level, with martial law having become the mechanism for maintaining order. Important social functions, such as weddings, are canceled. Incoming and outgoing international air traffic is terminated, as are diplomatic visits by representatives of other countries, and nonessential foreign diplomatic personnel evacuate. Indeed, it is at this level that ostracism of the affected country by the rest of the international community may be observed.
During Stage 4 biological events involving animals, extreme measures may be taken to assist with animal culling, such as having prisoners perform this task. An entire city or region may be officially ordered into quarantine, with military or police enforcement. Prosecution of citizens or businesses for unauthorized dissemination of information, profiteering, or disregarding a quarantine may be observed. Maximal local perception of threat is reflected in public demonstrations, rioting, and refusal to comply with official orders. Broad, sector-wide, systematic collapse of infrastructure occurs, including industry-wide closures, widespread economic collapse with gross stock market changes, loss of access to basic staples, declaration of martial law, declaration of a complete social crisis, and total dependence on international support.
Because a disease that is focused primarily in animals generates less social anxiety than one that is actively incapacitating and killing humans, a Stage 4 biological event involving animals tends to generate less social disruption than one involving humans. For example, mass evacuation and panic are typically not seen in a Stage 4 event that involves animals.
In summary, a Stage 4 biological event entails maximal regional- or national-level social disruption and a complete loss of containment.
Stage P: Preparatory Posture
In the course of our investigations, we noted many instances of countries assuming a preparatory posture in anticipation of a biological event. Sometimes, this posture implied the presence of an event when there was none; other times, this posture anticipated the presence of an event that was confirmed once active surveillance measures were implemented. We found that, particularly in closed societies, a declared preparatory posture for a biological event was a possible mechanism for disinformation, to explain the abrupt appearance of countermeasures and social disruption related to the actual presence of a disease. Therefore, we propose Stage P to represent a preparatory posture that reflects anticipation of a biological event. Stage P indications and warnings fall solely in the indirect category because presumably no biological event is actually present yet. However, if an actual event occurs during the course of preparation and is later reported by the host nation, Stage P may be changed to Stage 1 or 2.
Indirect indications and warnings during Stage P include official acknowledgment, official action, demand for medical services, and local perception of threat. Official acknowledgment typically takes the form of an explicit official announcement that preparatory measures are required because of concern about an imminent biological threat. Official action includes official investigation, implementation of countermeasures, and activation of biosurveillance or screening. Prophylactic countermeasures include vector control measures; hygiene campaigns; activation of pharmaceutical programs, such as vaccine delivery; preemptive infrastructure closures; quarantine of inbound international flights and ships from affected areas; public awareness campaigns; and closings of borders and ports of entry. Demand for medical services reflects anticipation versus actual demand, and includes restructuring of the medical infrastructure to accommodate a potential influx of patients requiring specialized care, as well as mobilization of pharmaceuticals and supplies. Local perception of threat is indicated by public concern or anxiety, with dissent occasionally being reflected in media articles that criticize the lack of an appropriate state of preparedness.
With regard to animal disease, prophylactic countermeasures include animal culling in locations where a pathogen may be introduced from a neighboring country. Commerce and trade restrictions may be implemented to prevent pathogen introduction. Preemptive infrastructure closures may include farms. Quarantine may be implemented for a farm or facility, for freight transport, or for inbound international air flights or ships, to enable screening. Demand for veterinary services is observed as an anticipatory condition similar to the preparatory posture for human disease. Local perception of threat regarding animal disease appears as changes in consumer behavior, specifically avoidance of the purchase of animal products.
▶ summarizes the stages of social disruption induced by a biological event according to our model.
Table 1.
Table 1 The Wilson–Collmann Scale: Summary of Staging for Biological Events
Stage | Condition |
---|---|
0 | Environmental conditions favorable to support the appearance of a biological event (for specific diseases and locations only— see text) |
1 | Uni-focal biological event |
2 | Multi-focal biological event |
3 | Severe infrastructure strain, depletion of local response capacity |
4 | Social collapse at the regional or national level |
P | Preparatory posture |
Validation
Case Studies: SARS, RVF, and VEE
SARS
In 2002, the emergence of SARS in the PRC appeared to be largely unnoticed by the international community. [Appendix 1] Indications and warnings of an “unseasonal bad flu” appeared in September. At this point, the analyst applying our model would have assessed the situation as representative of either a Stage 1 or more likely a Stage 2 biological event. Diagnosis of the pathogen in question would not have been apparent beyond “bad flu”; however, “unseasonal” should have been noted by the analyst, as it indicates a potential departure from local baseline disease.
In October, enough information was available to infer that a Stage 2 event was occurring. Social anxiety was being reported at this point. By November, although staging of the event would have remained the same, official concern was being expressed regarding potential public panic. In December, an abrupt decrease in reporting, indicating possible information suppression, indicated a change in local awareness of this novel threat. Stage 3 was firmly established by January 2003, as supply depletions and mobilization of resources were reported. By April, reports documented martial law and rioting due to SARS-related social disruption, indicating a Stage 4 event.
Note that, although a biological event meriting a staging analysis was reported in September, this event likely was an epidemic involving a variety of respiratory pathogens; to date, there remains uncertainty regarding precisely when SARS emerged within this context. In any case, reports of “unseasonal bad flu” in September and, more important, of social anxiety in October would have helped the analyst determine whether unusual disease was present. ▶ summarizes the staging for SARS in 2002–2003.
Figure 5.
Staging of SARS in the People’s Republic of China, 2002–2003.
RVF
The 1997–1998 epidemic of RVF in East Africa involved both compromised infrastructure induced by flooding and the spread of multiple diseases, posing a challenge to the analyst. [Appendix 2] In September 1997, cholera was already present in Kenya with the advent of El Niño-induced flooding. The local medical infrastructure was strained by the cholera biological event, indicating a Stage 3 situation. By December, the public health infrastructure had collapsed because of several factors, including cholera. At this point, the cholera epidemic had reached Stage 4. Malaria was also reported during this month, at a Stage 2 level. It was in December that the “mysterious disease” later recognized as RVF appeared as a multifocal animal and human disease, thus warranting a designation of Stage 2 animal and Stage 2 human disease. In January, the epidemic of malaria was reported as straining the local infrastructure, representing a Stage 3 condition. At this point, the RVF situation would have been upgraded to Stage 3 as well because of acute pharmaceutical shortages and reports of the disease overwhelming not only local resources, but also those of the international Red Cross.
The key to tracking indications and warnings of several diseases in the context of a complex humanitarian disaster, as in this study, is noting the specific locations of disease reports and clinical descriptions when available. ▶ summarizes the staging of the 1997 RVF epidemic in East Africa.
Figure 6.
Staging of RVF and cholera in East Africa, 1997–1998.
VEE
The 1995 epidemic of VEE presented a complex picture of flood-induced infrastructure collapse; the presence of a zoonotic disease (VEE); and possibly the presence of other diseases as well, such as dengue fever. [Appendix 3] In March and April 1995, flooding was reported, indicating a Stage 0. In April, equine health evaluations were reported, but there was no explicit declaration of a biological event of disease. This scenario could have been interpreted as Stage P. However, the astute analyst would have had to ascertain whether this official action was standard practice for Venezuela; if not, it may have been identified as a Stage 1 event in animals (equines). By June, enough information was available for the analyst to note the presence of a Stage 2 event in equines, co-occurring with at least a Stage 1 event in humans. In July, infrastructure strain was reportedly related to equine disease, warranting a Stage 3 animal event designation. Multifocal human disease was also present, corresponding to Stage 2. In August, strain on the medical infrastructure was reported, which would have upgraded the status for human disease to Stage 3. By September, signs of social collapse were present, implying Stage 4 for human disease. ▶ summarizes the staging for VEE in 1995.
Figure 7.
Staging of VEE in Venezuela, 1995.
This case study of VEE vividly illustrates the complexity of staging a zoonotic biological event whose transmission was influenced by a perturbation in environmental and climatic conditions. Flooding was a key factor in the progression of this epidemic, not only because of its effects on expansion of the vector population, but also because of its direct effect of disrupting multiple sectors of Venezuela’s local infrastructure, such as electricity, roadways, and communication. In this example, although documentation of infrastructure collapse due to flooding appeared as early as June, local reporting of social collapse due specifically to disease did not appear until September. It could easily be argued that the effects of flooding on local infrastructure facilitated the rapid loss of disease containment in this event.
Table 2 (available as a JAMIA online supplement at www.jamia.org) summarizes the indications and warnings of social disruption described above for the VEE, RVF, and SARS case studies. As noted above, while indications and warnings may be common across all events and locations, the specific parameters vary.
Applying this model, we found that several pathogens (plague, SARS, RVF, and VEE) tended to generate high levels of social disruption in certain countries.
In our prospective analyses for May 1–July 15, 2005, 50 biological events involving 13 distinct disease entities were documented; they affected humans in 15 countries and caused varying levels of social disruption (see Table 3, available as a JAMIA online supplement at www.jamia.org).
Influenza-like illness was the most common event detected in our prospective analysis. It generated widespread social disruption, consistently reaching Stage 3 across multiple countries with medical infrastructures of varying sophistication. We found that 14 percent of the events involving influenza-like illness progressed from Stage 1 to 2, and 60 percent from Stage 2 to 3. Of a total of 50 biological events detected for the Pacific Rim (involving 12 diseases in 15 countries), 58 percent were detected at Stage 1, 26 percent at Stage 2, 16 percent at Stage 3, and none at Stage 4.
Our combined retrospective and prospective analyses detected a variety of biological events, several of which had reached an advanced stage (3 or 4), as shown in ▶.
Table 4.
Table 4 Social Disruption Caused by Biological Events, by Stage, as Detected in Retrospective and Prospective Analyses
Disease | Maximum Stage Level Reached |
---|---|
Brucellosis | 1 |
Avian influenza (human disease) | 2 |
Japanese encephalitis | |
Typhoid | 2 |
Dengue fever | 3 |
Diarrheal disease | 3 |
Influenza-like illness | 3 |
Malaria | 3 |
Meningococcal meningitis | 3 |
Polio | 3 |
Plague | 4 |
SARS | 4 |
Rift Valley fever | 4 |
Venezuelan equine encephalitis | 4 |
Certain indications and warnings were cause for particular concern (see ▶). We believe that they should be among the most important targets for surveillance, given that they may indicate the presence of a biological agent that is novel, is highly transmissible, causes high morbidity or mortality, and/or is defying countermeasures. Consequently, we propose that these indications and warnings require rapid verification.
Table 5.
Table 5 Critical Indications and Warnings
Parameter | Implied Stage | Example |
---|---|---|
Illness or death among health care or veterinary workers | 1–4 | SARS, influenza, emerging zoonotic disease |
Military medical or veterinary support | 1–4 | SARS, influenza, emerging zoonotic disease |
Depletion of stockpiles of pharmaceuticals or medical supplies | 3 | SARS, influenza, VEE, RVF |
Regional or national mobilization of medical resources | 3 | SARS, influenza, VEE, RVF |
Mass evacuation and panic | 4 | Plague, Ebola hemorrhagic fever |
Rioting and martial law | 4 | SARS, pandemic influenza |
Discussion
Significance
For several decades, the United States National Weather Service has used various models to describe the disruptive and destructive potential of storm systems. For example, the Saffir-Simpson Hurricane Scale classifies hurricanes on a scale of 1 to 5, which describes progressively severe sustained winds and potential to cause serious damage upon landfall. 16 Similarly, the Fujita Scale rates the damage inflicted by a tornado on a scale from F0 to F5, from light to heavy damage. 17 For both of these scales, the frequency of storm systems is generally inversely proportional to the magnitude of the rating; in other words, milder storms are observed more frequently than severe ones.
The Wilson–Collmann Scale is analogous to these heuristic models. Our heuristic model estimates the impact of a biological event on society and therefore may be used to prompt response decisions and anticipatory consequence management. Such decisions may range from simple requests for more information to quarantine, closure of country borders, and international trade restrictions. Just as most storms are given low stage designations, the majority of biological events we evaluated merited low stage assessments, reflecting the relatively low probability of social collapse induced by biological events.
As of this writing, WHO is actively engaged in following the progress of H5N1 avian influenza as it spreads throughout Asia, Europe, and Africa. There is great concern as to whether the world may be witnessing the early stages of the next influenza pandemic. This concern is exacerbated by the lack of sufficient operational capability for rapid biological event detection, assessment, and diagnostics, which would allow vaccines to be developed quickly enough to ensure adequate coverage for the global population. 18
Our analyses demonstrate the potential value of a sociological model for rapid detection and characterization of biological threats, and specifically one that is not dependent solely on traditional public health disease surveillance mechanisms. Our findings support the observations of McGrath and others regarding the types and the sequence of social responses to biological threats. 12 Our model uses multiple types of indications and warnings to heuristically depict social disruption that may warrant closer scrutiny for a potentially uncontained, rapidly spreading disease of transnational threat potential. This model and process could serve as a valuable adjunct to current biological threat detection and analysis methods used by WHO’s GOARN and its international partners.
Limitations and Future Work
The identity and characteristics of particular indications and warnings may vary with the disease involved and the social routines of the affected culture. Determining what constitutes an indication and warning thus may depend on understanding local disease baselines and social conditions. Expanded capture of media reports from indigenous local sources provides valuable information because these sources reflect timely and qualitative local knowledge of typical and atypical phenomena. This kind of knowledge cannot be obtained by relying solely on regional, national, and international media reports. We observed remarkable similarity in social disruption parameters across biological events that involved different pathogens affecting humans. This similarity was also seen across multiple countries and time periods. Conversely, our initial observations suggest that there is some variation in the social disruption caused by biological events not only across diseases, but also across countries. We hypothesize that this variation is likely due to differences in social tolerance for disruption, the sophistication of public health and medical infrastructures, and the direct effects of the disease in question. We have observed that biological events taking place in countries with limited infrastructure may progress to higher stages more quickly than those occurring in countries with more robust infrastructure. One disease in particular—influenza-like illness—consistently defied countermeasures across multiple countries with a wide range of medical sophistication. This finding is troubling, particularly in light of current concern about pandemic influenza.
We propose, after an appropriate sample size is examined over time, each disease will exhibit a characteristic, locally specific level of social disruption. For example, diseases with limited capability for human-to-human transmission should present as Stage 1 events. If a disease that typically exhibits focal transmission characteristics is found to generate higher levels of social disruption, this information may prompt closer scrutiny of the event for unusual epidemiological features and possible implications regarding attribution. If the local indigenous behavior (e.g., typical geotemporal profile of local countermeasures used) is well understood, then sensitivity for an unusual change in behavior may be observed as an indicator for an emerging disease or other unusual phenomenon. This deserves further evaluation.
We found that Stages 1 to 4 seemed to apply across disease and location, whereas Stage 0 appeared to apply for specific diseases in particular locations. This observation warrants further research due to the potential for developing predictive algorithms to support anticipatory analytic methods.
In this paper, we have described social disruption as it escalates in severity. However, we recognize that the dynamic progression of a biological event may follow a pattern of upgraded and downgraded staging. Such a pattern might reflect successful countermeasures or particular disease transmission characteristics, among other factors. Furthermore, our observations of media behavior suggest that reporting becomes more vigorous and descriptive as an event evolves and generates maximal impact. However, the media tend to report resolution of an event less vigorously, particularly in closed societies. This trend poses a challenge to the analyst with regards to downgrading the staging of an event. Future investigations will be necessary to address the difficult question of how social disruption is resolved and normalcy is reestablished in a society.
It is important to note that the real-world application of our staging model would likely be complicated by uncertainty. We anticipate that media reporting of some biological events at any stage may provide only indirect indications and warnings, just a single report, or social disruption parameters not included in our model. Some biological events may not follow the order of social disruption embodied in our model. Conversely, if the cause of an indication and warning, such as rioting, is not verified, an analyst may be falsely alarmed by it and thus may assign a higher stage level than is necessary. We propose that our categories of recurrence, elevation, and diversification, along with proper contextualization of indications and warnings, will allow for more precise categorization of a biological event. Moreover, the staging model proposed here is likely to be revised in the future as more analyses are conducted across time, cultures, and disease entities. We recognize that ongoing operational use of this methodology may yield longitudinal data that could be used in forecasting models.
Our staging model could be used as an alerting methodology for diseases that affect primarily animals. With regard to zoonotic diseases such as VEE, RVF, or avian influenza, parallel but related events must be analyzed. This situation introduces complexity into the analysis as the pathogen in question directly affects the species involved, with attendant social disruption.
The global online community is growing and now stands at nearly 25 billion indexed web pages. 19 ProMED, a current standard in voluntary online, moderated disease outbreak reporting system, has over 40,000 subscribers in 165 countries. 20 We propose it is unlikely the Internet is being fully exploited for disease surveillance. A key challenge in advancing disease surveillance will be to identify methods that can enable the efficient triage of information to detect and track evolving biological events. These methods will require effective coupling to online harvesting engines for Internet exploitation such as we have used in this study. Evaluation of appropriate configurations of harvesting engines that efficiently capture information based on this model will be an important future research requirement.
The model proposed here explores the parameters of various stages of biological event evolution. Detection may be achieved only at a later stage of evolution, depending on the local community’s awareness of the event, transparency in reporting, and level of social disruption. While we will need to compare the timeliness of this approach to other methodologies, we recognize there will be an inherent limitation when using parameters for Stage 1 through 4 as the base detection parameters. Investigation of the parameters for Stage P and Stage 0 may offer limited pre-event warning, however this needs more evaluation. Understanding long range trends for a locality may also provide data for a forecasting model, which may aid in early detection as well.
We will report our ongoing endeavors to continually evaluate and validate our model in accordance with Strauss’ grounded theory method. It will be necessary to examine larger samples of data for a broad variety of diseases that effect humans and animals in a broad variety of cultures. Such research will likely refine not only the indication and warning categories and parameters but also the Wilson–Collmann Scale. Additional validation studies are in process now to examine whether improvements in geotemporal coverage are achieved with a media harvesting engine that exploits the model described in this paper in comparison to other capabilities currently considered the standard in global disease surveillance such as the WHO Disease Outbreak News and ProMED. Ongoing assessments such as these will be essential for defining the limitations of this approach.
Conclusion
An important future challenge in the use of models for event detection and threat assessment, such as the one proposed here, is to define possible concepts of operation that can link progressive warnings generated in Stages 1 through 4 with prompt, appropriately coordinated response decisions. Difficult response decisions may have to be made using limited indication and warning information when no diagnostic information regarding the disease in question is available. If response must await the receipt of diagnostic information, the resulting delay may mean the difference between mitigation and having to manage consequences after the fact. With regard to the current threat of pandemic influenza, the United States will be prompted into action once WHO has declared that a pandemic is occurring. We have documented indications and warnings that are often present in indigenous local media weeks to months in advance of WHO’s public declaration of an international public health threat. Such indications and warnings, considered within a model such as the one proposed here, may enable earlier threat assessment that can prompt scrutiny for an event of potentially global concern. An integrated biosurveillance strategy for global biological threats should meet these requirements.
Acknowledgments
The authors thank the following individuals and organizations for their generous contributions: Dr. Ronald Walters, Dr. John Philips, and Jacqueline R. Cooper, Intelligence Technology Innovation Center (ITIC); Dr. Seong Ki Mun and Mrs. Sarah Klimenko Riedl, Imaging Science and Information Systems Center, Georgetown University; Mr. Edward Lee Tilton, Mr. Adam Robinson, and Dr. Tom McEntee, The MITRE Corporation; Dr. Jeffrey Roller, Dr. Ken Curley, Dr. Mary Parker, and Mr. Kyle Martin, US Army Medical Research and Materiel Command Telemedicine and Advanced Technology Research Center (TATRC); Mr. Frank Connors, Defense Threat Reduction Agency; Dr. John Davies-Cole, District of Columbia Department of Health (DC DOH); Dr. Ray Arthur and Dr. Ellis McKenzie, US Centers for Disease Control and Prevention; Dr. Tracey McNamara, Bronx Zoo; Dr. Mark Thurmond and Ms. Carla Thomas, University of California-Davis; Dr. Bradley Clark, National Biosurveillance Integration Center, Department of Homeland Security; and all of the anonymous manuscript reviewers.
Footnotes
This work was supported by ITIC contract number 2006-1016 426-000, TATRC contract numbers W81XWH-04-1-0857 and DAMD17-94-V-4015, NLM Contract number N01-LM-3-3506, and DC DOH Contract number PO-HC-2004-P-1545. Dr. Wilson is supported by an NIH Loan Repayment Program Award (L40 AI057616).
The term “articles” as used here encompasses all content in open-source news media.
References
- 1.Heyman DL, Rodier G. Global surveillance, national surveillance, and SARS. Emerg Infect Dis. http://www.cdc.gov/ncidod/EID/vol10no2/03-1038.htmAccessed Feb 2003. [DOI] [PMC free article] [PubMed]
- 2. Avian influenza: perfect storm now gathering? Lancet 2005;365(9462):820http://thelancet.url123.com/26swp 2005. Mar 5–11. [DOI] [PubMed] [Google Scholar]
- 3.Ferguson NM, Cummings DA, Cauchemez S, Fraser C, Riley S, Meeyai A, Iamsirithaworn S, Burke DS. Strategies for containing an emerging influenza pandemic in Southeast Asia Nature 2005;437(7056):209-214Sep 8. [DOI] [PubMed] [Google Scholar]
- 4.World Health Organization Global outbreak alert and response network [online]http://www.who.int/csr/outbreaknetwork/en/ 2005. Accessed May 23, 2005.
- 5.Woodall J, Calisher CH. ProMED-mail: background and purpose Emerg Infect Dis 2001;7(3 Suppl):563. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Madoff LC, Woodall JP. The internet and the global monitoring of emerging diseases: lessons from the first 10 years of ProMED-mail Arch Med Res 2005;36(6):724-730Nov–Dec. [DOI] [PubMed] [Google Scholar]
- 7.Strauss A, Corbin J. Basics of qualitative research: grounded theory procedures and techniquesNewbury Park, CA: Sage; 1990.
- 8.Damianos L, Wohlever S, Kozierok R, Ponte J. MiTAP for real users, real data, real problems Proceedings of the conference on Human Factors of Computing Systems (CHI 2003). Fort Lauderdale, Florida, April 5–10. 2003.
- 9.Damianos L, Wohlever S, Kozierok R, Ponte J. MiTAP: a case study of integrated knowledge discovery tools Proceedings of the Thirty-Sixth Annual Hawaii International Conference on Systems Sciences (HICSS-36). Big Island, Hawaii, January 6–9. 2003.
- 10.Damianos L, Ponte J, Wohlever S, Reeder F, Day D, Wilson G, Hirschman L. MiTAP for bio-security: a case study AI Mag 2002;23(4):13-29. [Google Scholar]
- 11.Damianos L, Ponte J, Wohlever S, Reeder F, Day D, Wilson G, Hirschman L. MiTAP, text and audio processing for bio-security: a case study Proceedings of IAAI-2002: The Fourteenth Innovative Applications of Artificial Intelligence Conference. Edmonton, Alberta, Canada, July 28–August 1. 2002.
- 12.McGrath JW. Biological impact of social disruption resulting from epidemic disease Am J Phys Anthropol 1991;84(4):407-419Apr. [DOI] [PubMed] [Google Scholar]
- 13.Cieri C, Strassel S, Gra D, Martey N, Rennert K, Liberman M. Corpora for topic detection and trackingIn: Allan J, editor. Topic detection and tracking—event-based information organization. Norwell, Massachusetts: Kluwer Academic Publisher; 2002. pp. 33-66.
- 14.Makkonen J. Investigations on event evolution in tdt Proceedings of HLT-NAACL 2003, Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics Student Workshop 2003. Edmonton, Alberta, CA.
- 15.Chien Chin C, Yao-Tsung C, Yeali S, Meng Chang C. Life cycle modeling of news events using aging theory, lecture notes Comp Sci 2003;2837:47-59Jan. [Google Scholar]
- 16.National Hurricane CenterNational Weather ServiceNational Oceanic and Atmospheric Administration The Saffir-Simpson Hurricane Scalehttp://www.nhc.noaa.gov/aboutsshs.shtml 2003. Accessed May 23, 2005.
- 17.National Weather ServiceNational Oceanic and Atmospheric Administration Fujita Tornado Damage Scalehttp://www.spc.noaa.gov/faq/tornado/f-scale.html 2003. Accessed May 23, 2005.
- 18.WHO WHO Global Influenza Preparedness Plan 2005. Geneva, Switzerland.
- 19.WorldWideWebSize.com WorldWideWebSize.com 2007http://worldwidewebsize.com 2005. Accessed Dec 3, 2007.
- 20.ProMED-mail About ProMED. ProMED-mail 2007. http://www.promedmail.org 2005. Accessed Dec 3, 2007.