Developing syndrome definitions based on consensus and current use

Abstract

Objective

Standardized surveillance syndromes do not exist but would facilitate sharing data among surveillance systems and comparing the accuracy of existing systems. The objective of this study was to create reference syndrome definitions from a consensus of investigators who currently have or are building syndromic surveillance systems.

Design

Clinical condition–syndrome pairs were catalogued for 10 surveillance systems across the United States and the representatives of these systems were brought together for a workshop to discuss consensus syndrome definitions.

Results

Consensus syndrome definitions were generated for the four syndromes monitored by the majority of the 10 participating surveillance systems: Respiratory, gastrointestinal, constitutional, and influenza-like illness (ILI). An important element in coming to consensus quickly was the development of a sensitive and specific definition for respiratory and gastrointestinal syndromes. After the workshop, the definitions were refined and supplemented with keywords and regular expressions, the keywords were mapped to standard vocabularies, and a web ontology language (OWL) ontology was created.

Limitations

The consensus definitions have not yet been validated through implementation.

Conclusion

The consensus definitions provide an explicit description of the current state-of-the-art syndromes used in automated surveillance, which can subsequently be systematically evaluated against real data to improve the definitions. The method for creating consensus definitions could be applied to other domains that have diverse existing definitions.

Automated syndromic surveillance systems group information into syndromes (eg, respiratory syndrome) rather than into more specific diagnoses (eg, pneumonia),^{1 2} because many diseases in their early phase both lack diagnostic laboratory results and have overlapping features.^3–8 Syndromic surveillance is used to identify shifts in spatial or temporal trends or to describe the distribution of illness in a population.^9–11 Syndromic surveillance attempts to monitor population health through the timely analysis of indicators based on patients' perceptions of illness, as evidenced by health-seeking behaviors and self-described statements (eg, chief complaints), as well as through indicators based on clinicians' findings and perceptions.

Syndrome groupings of patients in a particular population can also provide a useful characterization of the distribution of illness in the community for situational awareness during a disaster or outbreak. The information from syndromic surveillance can reflect population patterns, such as health-seeking behaviors, can provide a measure of community stress, and can be used to assess the effectiveness of risk communication messages or to identify subpopulations that require essential public health actions. Syndrome groupings are often used as one element in a multisource approach to surveillance.

Our objective was to generate explicit consensus syndrome definitions based on current syndromic surveillance practice. Our approach was to identify the main clinical concepts underlying existing syndromes to make explicit the list of conditions that users and developers believe are important for surveillance, and to provide a starting point for open discussion of the foundations and goals of automated surveillance systems. Unambiguous consensus syndromes could provide an explicit list of the current state-of-the-art syndromes used in automated surveillance, which could then be systematically evaluated against real data to improve the definitions and overall system performance.

Background

The informatics community has responded to the need for enhanced bioterrorism and public health surveillance in a pragmatic but non-standardized fashion. As syndromic surveillance systems have been deployed across the country and the world, the number of syndrome definitions has grown. It is not clear, however, why or how these definitions differ from one another. The absence of standardized syndrome definitions presents several barriers to successful surveillance. First, it is difficult to compare surveillance data collected using different definitions, and therefore to evaluate the relative performance of different systems. Second, linking regional or national data for surveillance is more difficult without standard syndrome definitions. One attempt to share regional and national data is the Distribute project (http://www.isdsdistribute.org) that collects counts of influenza-like illness (ILI) from across the United States without consideration of how ILI is defined by the participating surveillance system. The Distribute project performs a vital service in displaying changes in ILI counts over time across the United States. However, understanding the components of the different ILI definitions—for example, whether Georgia's ILI definition includes sore throat—would enhance interpretation of time series charts from different states. Syndromic surveillance could produce more meaningful results with a common understanding of what observations constitute a syndrome.

To take an initial step towards a controlled vocabulary for syndromic surveillance, a group of public health practitioners, syndromic surveillance system developers, nurses and physicians representing 10 of the most visible syndromic surveillance systems across the United States collaborated on developing consensus syndrome definitions based on existing surveillance. Ontologies and vocabularies representing knowledge of the public health and surveillance domain already exist. However, existing knowledge representations focus on reportable diseases (PHSkb, retired),¹² infectious diseases (The Infectious Disease Ontology),¹³ disaster management and internet surveillance for media reports of existing outbreaks (Biocaster ontology),¹⁴ and are sometimes proprietary (OntoReason Public Health Ontology).¹⁵ These existing ontologies do not specifically address the types of information described in clinical data, such as chief complaints or emergency department reports, and do not group concepts by syndromes. Our goal was to collaboratively develop syndrome definitions that (1) represent the knowledge and experience of the diverse syndromic surveillance systems, (2) represent the knowledge of this discipline in a way that is intuitive to humans and computable by machines, and (3) are practical and usable for syndromic surveillance purposes.

Methods

We catalogued clinical condition–syndrome pairs for 10 surveillance systems across the country (all US surveillance systems with which we were familiar) and brought the representatives of these systems together, where we came to consensus on definitions for the four syndromes monitored by the majority of the 10 participating surveillance systems: respiratory, gastrointestinal, constitutional and ILI. The consensus process centered around an International Society for Disease Surveillance-funded, 2-day workshop in the Department of Biomedical Informatics at the University of Pittsburgh. Below we describe the participating systems and the process we used to come to consensus.

Participating systems

Participants included 20 scientists from academia, private industry, state and city public health departments and federal agencies. The scientists were representatives of 10 functioning US-based syndromic surveillance systems: Aegis (Harvard), BioPortal (University of Arizona), BioSense (Centers for Disease Control and Prevention), Boston Public Health Department, ESSENCE (Department of Defense), NC DETECT (University of North Carolina and the NC Division of Public Health), New York City (NYC DOHMH), New York State, Seattle-King County and RODS (University of Pittsburgh).

Consensus process

The consensus process involved two stages: (1) developing baseline syndrome definitions comprising a union of all the participants' syndrome definitions and (2) coming to consensus on essential syndrome definitions through discussions of the baseline syndrome definitions.

Step 1: Pre-workshop preparation—generating baseline syndrome definitions

To prepare for the workshop, each participant compiled syndrome definitions for their system. For this study, syndrome definitions were not narrative descriptions of the syndrome's meaning. Instead, syndrome definitions comprised a list of all symptoms, findings and diseases (ie, clinical conditions) mapped to each syndrome by the surveillance system. For example, a gastrointestinal syndrome definition could include the clinical conditions diarrhea, vomiting and gastroenteritis. In this paper, a clinical condition is a concept representing a single problem but could take the linguistic form of a single word, such as vomiting, or a phrase, such as abdominal pain.

We realized that non-standard naming conventions for clinical conditions could make merging multiple definitions difficult, so we provided each participant with a standard terminology of clinical conditions as a starting point. We developed a modified version of the coded chief complaints for emergency department systems (CCC–EDS) terminology for chief complaints¹⁶ as our initial terminology by removing terms that were not relevant to biosurveillance, resulting in a subset of 91 of the original set of 228 terms. Participants compiled their syndrome definitions by including relevant CCC–EDS terms and by adding terms that were not present in the CCC–EDS terminology but were mapped by the surveillance system to a relevant syndrome.

Step 2: Collating definitions into baseline definitions

University of Pittsburgh researchers (authors WWC and JND) created baseline syndrome definitions by merging together syndromes that appeared to be identical across any of the 10 participating systems. For example, all systems with the syndrome named ‘respiratory’ were combined into a baseline syndrome definition with the name ‘respiratory’. Each baseline syndrome contained a union of all clinical conditions comprising the syndrome with a count of how many times that condition was mapped to that syndrome by the systems whose definitions were merged. Author JND, a physician board-certified in internal medicine and infectious diseases with 30 years of clinical experience, provided the medical expertise needed to merge definitions. He only included a clinical condition from the original syndrome definition if it met the following inclusion criteria, which were drafted by four participants (WWC, JND, CS, CH):

• Sign, symptom, finding, or diagnosis

• Representing a single problem (eg, cough but not cough/SOB)

• Likely an acute care presentation (eg, chest pain but not CHF)

• Likely to occur in chief complaint or admit complaint (eg, abdominal pain but not diverticulitis).

We did not merge syndromes that appeared related but not identical. For instance, we did not merge upper respiratory syndrome with respiratory syndrome or severe gastrointestinal with gastrointestinal. This process resulted in 18 unique baseline syndrome definitions.

Step 3: Workshop—coming to consensus on syndrome definitions

Fifteen of the 20 participants attended the meeting in Pittsburgh, and three attended by phone conference. Because we were a diverse group with varying backgrounds and experience, we spent the first part of the workshop building a common understanding of why we were generating consensus syndrome definitions, what we meant when we used terms like ‘clinical condition’, ‘syndrome’ and ‘classifier’, and how we would approach the consensus process. We agreed upon the following common definitions and guidelines for the workshop:

1. The purpose was to develop syndrome definitions to assist public health in monitoring, characterizing, detecting and responding to changes in population health based on patients' initial clinical presentation of acute outbreaks and exposures. In our subsequent discussion, we frequently returned to this purpose to decide whether to include a particular clinical condition.

2. We were not sure whether complete consensus was possible and decided to aim for consensus but to settle for majority vote in cases without consensus.

In viewing the baseline syndrome definitions, it was clear that all participants were monitoring for general syndromes like respiratory and gastrointestinal, whereas a minority of participants were monitoring for specific versions of the same syndromes, such as lower respiratory or diarrhea. We therefore decided only to develop consensus syndromes for general syndromes. This decision is further supported by previous research¹⁷ showing that admit data does not contain enough information to detect patients with specific syndromes such as febrile respiratory. We also chose to focus on the general syndromes monitored by the majority of the participants: respiratory, gastrointestinal, constitutional and ILI.

We examined the baseline syndrome definitions for the general syndromes one clinical condition at a time—beginning with conditions that were most frequently included in the baseline definitions. We unanimously agreed to include some conditions, such as cough, that were monitored by everyone. However, other conditions, such as nasal congestion, provoked arguments over inclusion. In order to meet the surveillance needs for both broad and narrow conditions of interest, we decided to develop both a sensitive and a specific definition for each of the syndromes, which allowed us to come to consensus more quickly. We came to consensus on definitions for six syndromes: sensitive respiratory syndrome, specific respiratory syndrome, sensitive gastrointestinal syndrome, specific gastrointestinal syndrome, constitutional syndrome and ILI syndrome.

Results

Generating baseline syndrome definitions

The 10 participants submitted a total of 43 individual syndrome definitions, comprising 15 respiratory-related syndromes, 16 gastrointestinal-related syndromes and 12 related to fever, constitutional or ILI. After merging similar syndromes, there were 18 unique syndromes. Table 1 shows the merged syndrome names and the number of systems that monitor that syndrome. Some systems monitored derivations of the general syndromes such as lower and upper respiratory, abdominal pain, diarrhea, vomiting and sepsis.

Table 1.

Syndromes related to respiratory, gastrointestinal and constitutional illnesses and the number of 10 participating groups who monitor that syndrome

Syndrome	No of groups
Respiratory	10
Gastrointestinal	9
Influenza-like Illness	3
Febrile/fever	5
Constitutional	4
Vomiting	2
Ashma	1
Fever/flu	1
Sepsis	1
Upper respiratory	1
Lower respiratory	1
Abdominal pain	1
Diarrhea	1
Bloody diarrhea	1
Cold	1
Shock/coma	1

Participants included in their clinical definitions 59 of the 91 CCC–EDS conditions¹⁶ used as our initial terminology, and they added 145 unique new conditions. Of the 204 unique clinical conditions submitted by the 10 participants, 127 were included by only a single surveillance system, indicating the diversity of syndrome definitions monitored by the participants' surveillance systems. After reviewing the 145 unique new conditions, the physician excluded 31 synonymous conditions and excluded 36 conditions based on not meeting the inclusion criteria described earlier.

Coming to consensus on syndrome definitions

We examined the 18 unique syndromes and decided to develop six consensus syndromes based on those that the majority of participants monitor. Next, we describe the resulting syndrome definitions, which are summarized in table 2.

Table 2.

Clinical conditions comprising the final consensus syndrome definitions

Respiratory syndrome	Sensitive	Specific
Apnea	X
Bronchiolitis	X	X
Bronchitis	X	X
COPD	X	X
Cough	X	X
Dyspnea	X	X
Earache	X
Epiglottitis	X
Hemoptysis	X	X
Hoarseness	X
Hypoxia	X	X
Influenza	X	X
Lower respiratory infection	X	X
Pleural effusion	X	X
Pleuritic pain	X
Pneumonia	X	X
Rhinitis	X
Sinusitis	X
Sore throat	X
Stridor	X	X
Tachypnea	X	X
Upper respiratory congestion	X
Upper respiratory infection	X
Wheezing	X	X

Gastrointestinal syndrome	Sensitive	Specific
Abdominal pain	X
Dehydration	X
Diarrhea	X	X
Gastroenteritis	X	X
Nausea	X
Vomiting	X	X

Constitutional/ILI syndromes	Constitutional	ILI
Anorexia	X
Body aches	X	X
Bronchiolitis*		X
Chills	X	X
Cough*		X
Diaphoresis	X
Faintness	X
Fever	X	X
Generalized weakness	X
Influenza*		X
Irritable	X
Lethargy	X	X
Lymphadenopathy	X
Malaise	X	X
Pneumonia*		X
Sore throat*		X
Upper respiratory infection		X
Viral symptoms	X	X

The downloadable version expands on the definitions by adding synonyms and related concepts and by providing regular expressions for classifying each concept in free text.

^*Condition also listed in another syndrome.

COPD, chronic obstructive pulmonary disease; ILI, influenza-like illness.

Respiratory syndromes

We reviewed as a group each of the 48 respiratory-related conditions in the baseline syndromes and determined which to include in the consensus sensitive and specific definitions. After much discussion, we included 24 conditions in sensitive respiratory syndrome and 14 in specific (see table 2). Breathing difficulty, cough, hemoptysis and pneumonia are examples of conditions we included in both sensitive and specific definitions, whereas runny nose, pleuritic pain and sore throat were only included in the sensitive definition. Following our guidelines for inclusion, we did not include conditions such as earache, for example, because even though it affects the respiratory system, it is not the primary presentation of a significant illness of interest.

Gastrointestinal syndromes

From 25 gastrointestinal symptoms in the baseline definitions, we included six in the sensitive and three in the specific gastrointestinal syndromes. We excluded weight loss, as generally being caused by diseases not typically associated with a communicable disease outbreak, and intussusception, as too specific for the category. We included diarrhea, vomiting and gastroenteritis in both the sensitive and specific definitions, and included abdominal pain, nausea and dehydration only in the sensitive definition.

Constitutional, fever and influenza-like illness syndromes

Five unique baseline syndromes were related to fever or influenza, and we spent more time discussing which syndrome definitions to create for this category than we did for the other syndrome categories. From 32 conditions in the baseline definitions for constitutional syndrome, we included 12, such as irritable baby, fever, weakness and malaise. Although only two of the 10 participants monitor ILI, we decided to create a consensus definition for ILI because of the recent emphasis on ILI detection and characterization. From the 17 conditions in the baseline definition, we included 12. Except for the concept influenza, every condition in the consensus ILI definition is already represented in either a respiratory syndrome—bronchiolitis, pneumonia, influenza—or a constitutional syndrome—fever, weakness, body aches.

Refining the definitions

Once the meeting concluded, we had regular phone conferences to refine our definitions. First, we named the clinical conditions, applying the following guidelines: (1) select a preferred name for the condition, using medical language rather than lay-person vocabulary (eg, dyspnea rather than shortness of breath); (2) prefer symptom names over disease names (eg, wheezing rather than asthma); and (3) describe the meaning of the condition as broadly as possible (eg, sore throat rather than pharyngitis or laryngitis).

Second, we expanded the clinical conditions with synonymous and related concepts that essentially define each clinical condition. For instance, the concepts spitting up, hematemesis and vomiting all map to the clinical condition with the preferred name vomiting, which maps to sensitive and specific gastrointestinal syndrome. Organizing the syndrome definitions this way permits pre-coordination and post-coordination of concepts for user flexibility, allowing us to (1) explicitly define a syndrome through the list of clinical conditions that map to that syndrome, (2) explicitly define a condition through related and synonymous concepts, (3) map the concepts to standardized vocabularies, such as SNOMED, and (4) provide a transparent and explicit way to modify syndrome definitions by excluding or including specific concepts.

Third, based on their experience with real chief complaints, participants provided keywords they use in their surveillance systems to identify the relevant concepts in chief complaints. We included keywords in our definitions to delineate precisely what we meant at each layer of the definitions. Moreover, we wanted to maximize the utility of the definitions by providing a mechanism to deploy the definitions as a keyword search for chief complaint classification. We mapped the keywords contributed by the authors to regular expressions for easy implementation as an automated chief complaint classifier. Table 3 shows all components of the clinical condition vomiting. Because of the variety of keywords used by the participants and because of differences in regional dialects, determining which keywords to include was probably the most time-consuming aspect of our effort to refine the definitions. We made several decisions about the types of keywords to include, such as only including English keywords (even though some areas see a lot of Spanish chief complaints), allowing the inclusion of keywords that contain concepts other than the concept to which the keyword is mapped (eg, including ‘n/v/d’ in the nausea concept), and not including keywords that may cause a large number of false-positive matches when being applied to chief complaints (eg, match ‘uri’ as an entire word to avoid falsely classifying ‘pruritus’, ‘urinary’ and ‘injuries’ as upper respiratory infection). A concept in the definitions can contain many keywords. The vomiting concept, for instance, contains over 30 keywords including misspellings, variants such as ‘retching’, and abbreviations such as ‘n/v’ (see table 3). A few of the concepts also include keywords that must be paired with exclusions. For instance, faintness includes ‘faint’ or ‘faintness’ but not ‘fainting’ or ‘fainted’, which typically indicate a disorder different than faintness, such as pregnancy or a neurological illness.

Table 3.

The clinical condition ‘vomiting’, which maps to both sensitive and specific gastrointestinal syndrome

Clinical condition: vomiting
Concept (relation to condition)	Keywords, regular expressions representing keywords and UMLS CUI
Vomiting (Condition name)	vmt, n/v, v/n, v/d, d/v, v+d, d+v, n+v, v+n, v + d, d + v, d & v, dry heaves, emsis, f v, fv, n v, n & v, n v, n v d, n&v, n+v, nv, nvd, retching, v d, v f, v&d, v+d, vimiting, viomiting, vmt, vo, voimiting, voiting, vom, vometing, vomi, vomikting, vomintg, vominting, vomit, vomited, vomiti, vomitibg, vomitin, vomitine, vomiting, vomitinig, vomitiong, vomitng, vomitt, vomitting, vomittng, vomitus, vommiting, vomoting, vomtiing, vomtiming, vomting, vomtiting, puke, n v, retching, v d, vomit, vomicking, cant hold any food down, threw up, throw up, throwing up, emesis, emisis
	Regular expressions (3 of 73 shown below) and UMLS CUI:
	\bd\s&\sv\b	C0042963 Vomiting
	\bdry heaves\b	C0232602 Retching
	\bvomitus\b	C0042965 Vomitus
Spitting up (Synonym)	Keywords:
	bringing up, spitting up
	Regular expressions and UMLS CUI:
	\bbringing\sup\b	C0042963 Spitting up
	\bspitting\sup\b	C0042963 Spitting up
Hematemesis (Related concept)	Keywords:
	hematemesis, coffee ground emesis, throw* up blood, vomit* blood
	Regular expressions and UMLS CUI:
	\bhematemesis\b	C00189261 Hematemesis
	\bcoffee\sground\semesis\b	C1510416 Coffee ground vomiting
	\b(throw\w*?\s+up|vomit)\s*\+\s*blood\b	C00189261 Hematemesis

A single clinical condition is composed of concepts that map to the condition as either the condition name, a synonym of the condition or a related condition. Each concept is associated with keywords used by implemented surveillance systems. We also created regular expressions for the keywords and mapped each regular expression to a UMLS concept unique identifier. Some regular expressions map to the same concept unique identifier (CUI).

Fourth, we mapped each concept in the syndrome definitions to several standardized vocabularies, including SNOMED and UMLS concepts. To do this, we processed the keywords with MetaMap,¹⁸ an application developed by the National Library of Medicine for mapping text to UMLS concepts. We manually filtered the output to select the best UMLS concepts for each concept in our definitions.

We also encoded the structure of the syndrome definitions in a syndromic surveillance ontology (SSO).¹⁹ We developed the SSO using OWL (web ontology language), the current W3 consortium standard for ontology development. Representing the syndrome definitions with an ontology allows for machine interpretability and is crucial for using the definitions to drive automated surveillance systems. In constructing the SSO, we built on earlier work by others, including PHSKb¹² and the BioCaster ontology¹⁴ used for media surveillance. The SSO provides a human-readable textual and graphical representation of the syndromes, the clinical concepts and other associated concepts. Researchers and public health practitioners can use this ontology to verify the correctness of our assumptions and to identify potential extensions to the syndrome definitions. The SSO also serves as a repository of explicit, computable definitions for use in automated systems conducting online surveillance and for use by researchers evaluating a range of surveillance methodologies.

The final consensus syndrome definitions can be browsed at http://www.surveillance.mcgill.ca/projects/sso/SyndromeDef.html. You can download the definitions in a tab-delimited file from http://www.dbmi.pitt.edu/blulab/resources.asp and as an ontology from the BioPortal website at http://www.bioportal.bioontology.org/ontologies/40646. We have also provided a string-matching classifier to classify text based on the definitions at http://www.surveillance.mcgill.ca/ onto_classifier. Open source distribution will facilitate not only sharing of the definitions but also an open mechanism for validating and maintaining the definitions through community effort.

Discussion

We developed consensus definitions for the most frequently monitored syndromes from a large sample of syndromic surveillance systems in the United States. Our goal was to generate working definitions of syndromes that represent what state-of-the-art systems currently monitor in real practice, not to characterize the ideal syndromes, which are difficult to define. We were able to leverage the previous research, experimentation and experience of developers and users of syndromic surveillance systems to develop informative syndromes in a relatively short time. We ultimately created six definitions containing from three to 26 clinical conditions. Clinical conditions are composed of related and synonymous concepts, providing transparency and flexibility needed to easily alter and adapt the syndrome definitions based on regional preferences or validation with real data. Respiratory and gastrointestinal definitions have both a sensitive and specific version, which helped us come to consensus more quickly and provides more flexibility for surveillance: sensitive definitions can help detect outbreaks sooner by casting a larger net for patients who may be in the early stages of a serious illness, and specific definitions can be used to decrease false alarms or to refine a search when characterizing a potential outbreak.

Finding consensus within a diverse group is daunting, and there was a great deal of trepidation in bringing 20 independent participants together to reach agreement on common syndrome definitions. In reality, we came to consensus quite easily for several reasons:

• The participants all had the same goal and had a spirit of cooperation rather than of dominance or divisiveness.

• The participants represented stakeholders in syndromic surveillance: public health practitioners, physicians, nurses and informaticists who are involved directly in developing and using real syndrome-based surveillance.

• We carefully and explicitly defined the terminology we used, including terms used to describe our goals and terms comprising definitions of the syndromes.

• All participants contributed to preparation for the meeting by assembling and organizing information in a way that facilitated productive discussion at the meeting.

• We were able to meet in person to discuss the definitions. Refinement occurred over the course of approximately a year through phone calls and was much less efficient than our face-to-face time.

• We built our definitions from the foundation of what already existed, making the task more clear and more relevant to the participants.

There are several alternative methods we could have used to develop reference definitions. For example, we could have defined the syndrome groupings empirically, by examining data from historical outbreaks to determine the most sensitive and specific chief complaints for a syndrome. A lack of chief complaints associated with known outbreaks (or a true ‘gold standard’) with which to assess the accuracy of chief complaint classifications across a broad range of conditions prevents the use of this approach. Alternatively, we could have based our syndrome groupings on expert clinical opinion from the literature. Our goal, however, was to represent classifications the way syndromic surveillance users are applying them in practice, rather than develop ‘ideal’ classifications that might bear limited practical relevance. We therefore chose to develop our working definitions by leveraging the body of work that has already been undertaken by the syndromic surveillance community, and by reaching consensus among current users to characterize common practices in syndrome classification.

Our approach should transfer to other domains with similar characteristics to ours: (1) a small to medium set of relevant concepts (versus the domain of internal medicine, for example, which requires a huge number of concepts); (2) the existence of multiple terminologies for the domain; and (3) a group of people invested in the domain and dedicated to cooperation.

There are several limitations to the consensus definitions. The most obvious is that the definitions have not yet been validated. However, case definitions in public health are not usually developed based on validation from real data. Instead, they are developed based on expert consensus, in a manner analogous to how clinical guidelines are developed. The definitions should not be considered definitive or even preferable to other definitions, but are meant to provide a common reference to facilitate dialogue. With the definitions, we can now support cross-jurisdictional validations, along with better understanding of these syndromes relative to each other and relative to other population health and surveillance indicators. The regular expressions for identifying concepts in text are certainly incomplete and biased by a few geographical locations and do not include patterns to match numeric values, such as for mapping ‘temp 38’ to fever. Moreover, the definitions were assembled from systems performing syndromic surveillance in primary care and emergency care settings from a limited number of geographical locations, and it is unclear how well these definitions will apply to other situations.

In spite of the limitations, the syndrome definitions can be utilized in a number of ways. Like other vocabularies or ontologies for a domain, the syndrome definitions facilitate sharing a common understanding of the structure of information in the domain, enable the re-use of domain knowledge and make domain assumptions explicit.²⁰ We hope the definitions will provide a starting point to perform systematic evaluations of the sensitivity and positive predictive value of concepts included in the definitions. For example, it would be useful to know the proportion of true influenza patients identified by the concept sore throat. The definitions can be used to compare regional differences in chief complaint occurrence and to evaluate the precision of regular expressions. After some validation of the definitions, it would be informative to use the definitions to classify a repository of chief complaints from across the country into syndrome categories for training and testing chief complaint classifiers. Ultimately, a set of validated definitions could be extremely helpful in facilitating cross-jurisdictional communication among different surveillance systems. In fact, there is a pilot project underway (facilitated by the International Society for Disease Surveillance) to implement a standard ILI definition in multiple sites and compare data collected using that approach with data collected using local definitions.

Conclusion

We created reference syndrome definitions from the consensus of 20 developers and users of syndromic surveillance. The investigators met to discuss the conditions that should be included in the three most frequently monitored syndromes: respiratory, gastrointestinal and constitutional. Coming to consensus was not as difficult as was expected, primarily because we built the definitions from lists of clinical conditions monitored by existing surveillance systems and because we developed sensitive and specific versions of the definitions. We believe our approach could be replicated by others attempting to facilitate consensus in a domain in which multiple different definitions already exist. We hope the definitions will provide a rich starting point for further discussion, experimentation, implementation, communication and refinement as we attempt to monitor the population for illnesses of public health importance.