Skip to main content
PMC home page
Public Health Reports logoLink to Public Health Reports
. 2017 Jul 10;132(1 Suppl):31S–39S. doi: 10.1177/0033354917706517

Evaluation of a Novel Syndromic Surveillance Query for Heat-Related Illness Using Hospital Data From Maricopa County, Arizona, 2015

Jessica R White 1,✉, Vjollca Berisha 1, Kathryn Lane 2, Henri Ménager 3, Aaron Gettel 1, Carol R Braun 4, the Council of State and Territorial Epidemiologists Heat Syndrome Workgroup
Editors: Paula W Yoon, Amy I Ising, Julia E Gunn
PMCID: PMC5676503  PMID: 28692392

Abstract

Objectives:

We evaluated a novel syndromic surveillance query, developed by the Council of State and Territorial Epidemiologists (CSTE) Heat Syndrome Workgroup, for identifying heat-related illness cases in near real time, using emergency department and inpatient hospital data from Maricopa County, Arizona, in 2015.

Methods:

The Maricopa County Department of Public Health applied 2 queries for heat-related illness to area hospital data transmitted to the National Syndromic Surveillance Program BioSense Platform: the BioSense “heat, excessive” query and the novel CSTE query. We reviewed the line lists generated by each query and used the diagnosis code and chief complaint text fields to find probable cases of heat-related illness. For each query, we calculated positive predictive values (PPVs) for heat-related illness.

Results:

The CSTE query identified 674 records, of which 591 were categorized as probable heat-related illness, demonstrating a PPV of 88% for heat-related illness. The BioSense query identified 791 patient records, of which 589 were probable heat-related illness, demonstrating a PPV of 74% for heat-related illness. The PPV was substantially higher for the CSTE novel and BioSense queries during the heat season (May 1 to September 30; 92% and 85%, respectively) than during the cooler seasons (55% and 29%, respectively).

Conclusion:

A novel query for heat-related illness that combined diagnosis codes, chief complaint text terms, and exclusion criteria had a high PPV for heat-related illness, particularly during the heat season. Public health departments can use this query to meet local needs; however, use of this novel query to substantially improve public health heat-related illness prevention remains to be seen.

Keywords: syndromic surveillance, heat stress disorders, epidemiology

Exposure to high ambient temperatures is associated with substantial morbidity and mortality, making extreme heat a serious public health threat around the world.1–9 The effects of heat exposure range from mild symptoms, such as muscle cramps and heat syncope, to more life-threatening symptoms, such as heat exhaustion and heat stroke.10 All people are at risk of developing these symptoms, but older adults (aged >65), younger children (aged <4), those with low socioeconomic status, and those with poor access to adequate cooling systems are particularly vulnerable to extreme heat.1,10 High ambient temperatures can also exacerbate chronic health conditions, including cardiovascular, respiratory, and renal diseases.1,10–12 In the United States, the burden of heat-related illness has been substantial: emergency departments treated an average of 65 299 patients for acute heat-related illness each summer from 2006 through 2010.13

Because most US health departments do not mandate heat-related illness reporting, the burden of extreme heat on a community can be difficult to monitor, especially when resources are limited.14 To estimate the number of hospital visits for heat-related illness, epidemiologists have typically relied on relevant International Classification of Diseases, Clinical Modification (ICD-CM) codes in hospital discharge data.13,14 This method provides a reliable retrospective estimate of disease burden. However, because of a lag of several months to a year before hospital discharge data become available for analysis, it does not allow for real-time situational awareness.

As an alternative, epidemiologists can filter through more timely data provided through syndromic surveillance systems. Syndromic surveillance data sets usually include key demographic characteristics and health information obtained from medical records; however, they do not always provide the full context of patient visits, and some of the information may change as updates are made to the record over time. Nevertheless, depending on data use agreements, public health agencies can use heat-related illness syndromic surveillance to receive and collect near real-time data from health care providers, including pharmacies, emergency medical services, emergency departments, and/or inpatient hospitals.15–17 Agencies can then query these data to find cases of heat-related illness.

Although monitoring heat-related illness through syndromic surveillance has shown some promise in settings around the world, no standard query exists to identify patients with heat-related illness in syndromic surveillance data.18–23 Some public health agencies have used predefined syndrome queries from the 2 most popular syndromic surveillance systems: the National Syndromic Surveillance Program BioSense Platform15 and the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE).16 The BioSense query uses diagnosis codes and chief complaint search terms, whereas the ESSENCE query uses only chief complaint search terms.24 Neither the BioSense nor the ESSENCE syndrome query uses exclusion criteria. Many public health agencies have developed their own custom queries for heat-related illness. National standardization of queries for heat-related illness would be beneficial in several ways, including allowing public health agencies to compare data across jurisdictions.14

The Council of State and Territorial Epidemiologists Climate Change Subcommittee convened a multijurisdictional collaborative Heat Syndrome Workgroup in 2014 to develop a novel standardized query for heat-related illness that would successfully identify heat-related illness patients from emergency department and inpatient hospital data, according to chief complaint terms, diagnostic codes, and exclusion criteria.24 Our objective in this study was to evaluate this novel syndromic surveillance query for identifying heat-related illness cases in near real time, using emergency department and inpatient hospital data from Maricopa County, Arizona, a region particularly prone to extreme heat.14 We compared the results through this query with those from the predefined National Syndromic Surveillance Program BioSense query for heat-related illness, hypothesizing that the novel query would have a higher positive predictive value (PPV) for heat-related illness.

Methods

Setting

Heat-related illness surveillance and response are public health priorities for the Maricopa County Department of Public Health (MCDPH). Arizona’s Maricopa County is the fourth-most populous county in the United States. The region has >3.8 million residents, and it hosts approximately 19 million overnight visitors annually.25,26 Each year, the region’s hot desert climate produces an average of 110 days >100°F.27 From 2006 to 2013, 632 people died in the county from excessive heat exposure between June and August.28 Additionally, from 2008 to 2013, 9414 people presented to hospitals in the area with heat-related illnesses.29

Data Sources

For this study, MCDPH obtained hospital data from the National Syndromic Surveillance Program BioSense Platform. Participating health care facilities in Arizona began transmitting data to the BioSense Platform in July 2014. By January 1, 2015, 9 of the 33 Maricopa County nonfederal health care facilities were actively transmitting data to the BioSense Platform, and 2 additional facilities began transmitting data on January 19, 2015. As a result, the BioSense Platform captured data from 49% of emergency department visits and 55% of inpatient hospitalizations in Maricopa County in 2015. Compared with the general population in Maricopa County, the BioSense data set for 2015 overrepresented patients who were female, aged >65, African American, and non-Hispanic/non-Latino.25

We obtained daily maximum and minimum temperatures for Phoenix, Arizona, from the National Weather Service website.30 To calculate incidence rates per 100 000 population for various patient demographic characteristics, we obtained population data for Maricopa County from the 2010 US Census American FactFinder.25

Heat-Related Illness Queries

MCDPH used 2 queries for heat-related illness to generate lists of patient records. The first was the BioSense built-in, predefined query for heat-related illness called “heat, excessive” (hereinafter, BioSense query), which was obtained through the BioSense Front End Application (Table 1).31 The BioSense query included searches for ICD-CM diagnosis and injury codes in the patient emergency department visit or inpatient hospital diagnosis fields, including ICD-CM codes 992 and E900 (ninth revision) and codes T67 and X30 (tenth revision). This query also included searches for heat-related text terms in the patient emergency department visit or inpatient hospital diagnosis and chief complaint fields.

Table 1.

Diagnosis search codes, terms, and chief complaint search terms used in an NSSP BioSense “heat, excessive” query and CSTE novel heat-related illness query to produce initial line lists of patients with heat-related illnesses, Maricopa County, Arizona, January 1 to December 31, 2015a

Syndrome Query Search Codes and Termsb
Developer Name
NSSP, BioSense Platform BioSense predefined “heat, excessive” query Inclusion criteria:
ICD-9-CM and ICD-10-CM diagnosis search codesc 992, E900, T67, and X30
Diagnosis text search terms: demasiado caliente, to hot, too hot, excessive + heat, heat apoplexy, heat collapse, heat cramps, heat edema, heat effects, heat exhaustion, heat fatigue, heat prostration, heat pyrexia, heat stroke, heat syncope, over + heated, sunstroke
Chief complaint text search terms: demasiado caliente, to hot, too hot, enlosacion, heat, hypertermia, hyperthermia, insolacion, over + heated, overheated, sobre calentado, sobre caliente
CSTE Heat Syndrome Workgroup Novel heat-related illness query Inclusion criteria:
ICD-9-CM and ICD-10-CM diagnosis search codesc 992, E900, T67, and X30
Chief complaint text search terms: heat_,d heatcramp, heatex, heatst, heat-exhaust, heat-related, heat-stroke, hypertherm, overheat, “over heat,” “sun stroke,” sunstr, sun-str, “to hot,” “too hot,” ([heet OR hot] AND [excessive OR exhaust OR expos OR fatigue OR cramp OR stress OR “in car” OR outside OR prostration])
Exclusion criteria:
ICD-9-CM and ICD-10-CM diagnosis search codese 1992, 6992, E900.1, E9001, and T50.992A
Chief complaint text search terms: allerg, “feeling hot,” “feels heat,” “feels hot,” “felt hot,” (hot AND sensation), “heat sensation,” inflam, (pain AND (limb OR arm OR shoulder OR elbow OR wrist OR hand OR leg OR hip OR groin OR thigh OR knee OR ankle OR foot OR feet), pain AND red, radiat, redness, swell, swollen, surg, “post op,” ibuprofen, ibuprophen, alieve, motrin, tylenol, injur, trauma, heat AND ice, heat AND (applied OR tried OR used OR using), “heat pack,” “heating pad,” (pain AND [back OR neck OR flank]), lumbargo, relief, resolve, relieve, releive, dental, heat AND cold, hot AND cold, oral AND surg, pain AND (jaw OR mouth OR teeth OR tooth), sensitiv AND (heat OR hot), hot AND coff, “hot dog,” “hot grease,” “hot peppers,” “hot tea,” “heat ache,” heatache, “heat attack,” “heat beat,” heatbeat, “heat burn” heatburn, “heat flutter,” “heat racing,” “heat rate,” heatrate, heatlh, heath, heatth, “hitting heat,” palpitation, cheat, heated, heater, Heather, heating, hotel, lithotr, methotr, photo, psychotic, sheath, sheet, shot, Sunday, theat, wheat, accident, alcohol, burn AND mouth, distress, fever, “gets hot,” “heat flash,” “hot flash,” heat AND rash, hives, hot AND shower, “hot tub,” “no heat,” oven, suicid
Open in a new tab

Abbreviations: CSTE, Council of State and Territorial Epidemiologists; ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; NSSP, National Syndromic Surveillance Program.

aTable based on Appendix 2 of the CSTE guidance document for implementing heat-related illness syndromic surveillance in public health practice.24

bMisspelled words included intentionally.

cCodes to include: 992, effects of heat and light; E900, accident caused by excessive heat; T67, effects of heat and light; X30, exposure to excessive natural heat.

dChief complaint search term “heat_” is equivalent to term “heat” immediately followed by a space.

eCodes to exclude: 1992, multiple codes contain this combination of digits; 6992, multiple codes contain this combination of digits; E900.1/E9001, accidents due to excessive heat of manmade origin; T50.992A, poisoning by other drugs, medicaments, and biological substances, as well as intentional self-harm and initial encounter.

The second was a novel heat-related illness query, which was developed by the Council of State and Territorial Epidemiologists Heat Syndrome Workgroup. Details on query development are available online and in a guidance document published by the Council of State and Territorial Epidemiologists.24 The novel query expanded the scope of the BioSense query by using the same diagnosis codes but adding more chief complaint inclusion terms. In addition, it included exclusion terms that were intended to eliminate the records of patient encounters that were not likely related to environmental heat exposure, thereby potentially improving the specificity and PPV of the query (Table 1). This query was constructed with the BioSense application for building custom queries, phpMyAdmin, which uses MySQL code.

Producing Line Lists

Epidemiologists at MCDPH used these 2 queries to produce 2 line lists of Maricopa County emergency department and inpatient hospital records for January 1, 2015, through December 31, 2015. The BioSense query produced a line list with de-identified patient records that contained 12 variables, including name of facility; visit date, diagnosis, and chief complaint; and patient age, sex, and ZIP code.31 The line list generated through the novel query also contained de-identified records, which included the variables in the BioSense query as well as patient medical record number, race, ethnicity, and discharge disposition. Subsequently, we used SAS Enterprise Guide 5.1 to remove any records that met the novel query’s exclusion criteria.32

Manual Review of Line Lists

Two MCDPH epidemiologists manually reviewed both line lists and examined the chief complaint and diagnosis code fields to categorize each record as representing a “probable,” “ruled out,” or “undetermined” case of heat-related illness. The 2 reviewers resolved discordant categorizations by discussing the discrepancy and mutually agreeing on a final reclassification. The reviewers categorized records as “probable” rather than “confirmed” because the final diagnoses associated with these records were not available at the time of publication. The reviewers automatically categorized records with 1 of the 4 heat-related illness ICD-CM diagnosis or injury codes as “probable” cases of heat-related illness. When there was >1 code in the field, the reviewers gave all codes equal consideration regardless of the order in which they were listed.

When records did not include 1 of the diagnosis codes for heat-related illness, the epidemiologists used the chief complaint text to categorize the record. To be categorized as “probable,” the chief complaint needed to mention symptoms of heat illness AND (patient concern for heat exhaustion; OR report of being “found down”; OR spending time outside/in the heat for work, recreation, or circumstance [eg, broken-down car or homelessness]; OR spending time in a hot vehicle; OR having no air-conditioning). The epidemiologists categorized records as “ruled out” for numerous reasons, including obvious misspellings, larger words that included the letter combinations “heat” or “hot” in them (eg, trichotillomania or gunshot), adverse reactions to a drug, overheating in response to hormone imbalance, and burns from hot asphalt. They categorized records as “undetermined” when they lacked strong evidence for environmental heat exposure but heat-related illness could not be completely ruled out.

Statistical Methods

We calculated descriptive characteristics and incidence rates for each query.33 We calculated incidence rates using 2010 US Census data and did not include nonresidents.25 We plotted the frequencies of heat-related illness cases over time with maximum and minimum daily temperatures. We measured the performance of each query by calculating PPV as the total number of “probable” cases of heat-related illness divided by the total number of possible cases of heat-related illness identified by each query. We determined the performance of each query by calculating PPV for the entire 2015 calendar year, for just the heat season (May 1 through September 30), and for just the cooler seasons (January 1 through April 30 and October 1 through December 31). We used SAS Enterprise Guide 5.1 for all calculations and figure production.32 County employees in the MCDPH Office of Epidemiology conducted all analyses. MCDPH determined this study to be exempt from institutional review board review because it did not involve human subjects.

Results

Of the 1 021 067 total Maricopa County visits captured in the BioSense Platform in 2015, a total of 791 (0.08%) patient records were identified by the BioSense heat-related illness query as possible cases of heat-related illness, whereas 674 (0.06%) records were identified by the novel query as possible cases of heat-related illness. The distribution of demographic characteristics was similar in the records resulting from both queries, with most patients being male and aged 18-49 (Table 2). Among patients identified by the BioSense query, incidence rates of heat-related illness per 100 000 population were highest for adults aged ≥65 (28.3), but the lack of data from this query about race or ethnicity made it impossible to calculate incidence rates for these characteristics. Among those identified by the novel query, incidence rates of heat-related illness per 100 000 population were highest for patients who were aged 50-64 (24.2), African American (27.8), and non-Hispanic/non-Latino (18.4).

Table 2.

Frequency and incidence rates of demographic characteristics of patients who met criteria for NSSP BioSense “heat, excessive” query and CSTE novel heat-related illness query for heat-related illness, Maricopa County, Arizona, January 1 to December 31, 2015

Demographic Characteristics NSSP BioSense Querya (n = 791) CSTE Novel Query (n = 674)
No. (%) Heat-Related Illness Incidence Rate per 100 000 Populationb No. (%) Heat-Related Illness Incidence Rate per 100 000 Populationb
Sex
 Male 491 (62.1) 26.0 448 (66.5) 23.7
 Female 300 (37.9) 15.6 226 (33.5) 11.7
Age group, y
 <5 18 (2.3) 6.4 11 (1.6) 3.9
 5-17 46 (5.8) 6.3 33 (4.9) 4.6
 18-49 426 (53.9) 25.0 377 (55.9) 22.1
 50-64 170 (21.5) 26.5 155 (23.0) 24.2
 ≥65 131 (16.6) 28.3 98 (14.5) 21.2
Race
 White NA NA 505 (74.9) 18.1
 Black or African American NA NA 53 (7.9) 27.8
 American Indian/Alaska Native NA NA 15 (2.2) 19.1
 Asian NA NA 3 (0.5) 2.3
 Other NA NA 98 (14.5) 15.6
Ethnicity
 Non-Hispanic or non-Latino NA NA 494 (73.3) 18.4
 Hispanic or Latino NA NA 180 (26.7) 15.9
Open in a new tab

Abbreviations: CSTE, Council of State and Territorial Epidemiologists; NA, not available; NSSP, National Syndromic Surveillance Program.

aBioSense Front End Application did not provide data on race or ethnicity.

bIncidence rates calculated with 2010 US Census population data for Maricopa County, Arizona.25

For both queries, the temporal trends of cases of heat-related illness tracked Maricopa County’s heat season (May 1 to September 30). Consequently, the frequency of cases of heat-related illness was low from January through April; increased in May; peaked during June, July, and August; decreased in September; and remained low through December (Figure).

Figure.

Figure.

Open in a new tab

Trends in maximum and minimum daily temperatures and in frequency of heat-related illness cases identified by (A) the National Syndromic Surveillance Program BioSense “heat, excessive” query (n = 791) and (B) the Council of State and Territorial Epidemiologists novel heat-related illness query (n = 674), Maricopa County, Arizona, January 1 to December 31, 2015.

Among the 791 records retrieved by the BioSense query, 589 (74.4%) were categorized as “probable” cases of heat-related illness, whereas 186 (23.5%) were categorized as “ruled out” (Table 3). The reviewers ruled out records because they included text with misspellings of “heart” or “head”; about feeling heat and swelling, redness, or pain; about using heat and ice for therapeutic reasons; or about having dental sensitivities to hot and cold. In contrast, of the 674 records retrieved through the novel query, 591 (87.7%) were categorized as “probable” cases of heat-related illness, whereas only 62 (9.2%) were categorized as “ruled out.” The records ruled out by the novel query were considered false positives. When we examined these false-positive records, we observed that they contained no common words that could be used as additional exclusion terms. Therefore, we concluded that the Council of State and Territorial Epidemiologists exclusion criteria were sufficient for making this query as specific as possible. Finally, only 16 (2.0%) records from the BioSense query were categorized as “undetermined,” and only 21 (3.1%) records from the novel query were categorized as “undetermined.”

Table 3.

Distribution of categories of heat-related illness cases, by query type (NSSP BioSense “heat, excessive” and CSTE novel heat-related illness) and period (full year, heat season, and cooler season), Maricopa County, Arizona, January 1 to December 31, 2015

Case Classification by Query Type No. (%)
Full Year (Jan 1 to Dec 31) Heat Season (May 1 to Sep 30) Cooler Season (Jan 1 to Apr 30; Oct 1 to Dec 31)
BioSense “heat, excessive” query 791 (100.0) 647 (100.0) 144 (100.0)
 Probable heat-related illness case 589 (74.4a) 547 (84.5a) 42 (29.2a)
  ICD-9-CM or ICD-10-CM diagnosis code (992, E900,   T67, or X30)b 388 (49.1) 363 (56.1) 25 (17.4)
  Chief complaint text (natural heat exposure) 201 (25.4) 184 (28.4) 17 (11.8)
 Undetermined heat-related illness case 16 (2.0) 11 (1.7) 5 (3.5)
 Ruled-out heat-related illness case 186 (23.5) 89 (13.8) 97 (67.4)
Novel heat-related illness query 674 (100.0) 594 (100.0) 80 (100.0)
 Probable heat-related illness case 591 (87.7a) 547 (92.1a) 44 (55.0a)
  ICD-9-CM or ICD-10-CM diagnosis code (992,   E900, T67, or X30)b 354 (52.2) 328 (55.2) 26 (32.5)
  Chief complaint text (natural heat exposure) 237 (35.2) 219 (36.9) 18 (22.5)
 Undetermined heat-related illness case 21 (3.1) 14 (2.4) 7 (8.8)
 Ruled-out heat-related illness case 62 (9.2) 33 (5.6) 29 (36.3)
Open in a new tab

Abbreviations: CSTE, Council of State and Territorial Epidemiologists; ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; NSSP, National Syndromic Surveillance Program.

aPositive predictive value = total number of probable heat-related illness cases / total number of records identified by the query as possible heat-related illness cases.

bCodes to include: 992, effects of heat and light; E900, accident caused by excessive heat; T67, effects of heat and light; X30, exposure to excessive natural heat.

For all of 2015, the PPV for the BioSense query was 74.4%, whereas the PPV for the novel query was 87.7%. These data took into account that about half of the records produced by each query had a diagnosis code for heat-related illness and therefore were automatically categorized as “probable.” More than 80% of the identified records originated in Arizona’s heat season, during which the PPV for the BioSense query was 84.5% and the PPV for the novel query was 92.1% (Table 3).

We also applied the exclusion terms noted in Table 1 to the line list generated by the initial BioSense query, which eliminated 335 (42.4%) records from the line list. Among the 456 records that remained in the line list, 426 (93.4%) had been categorized as “probable” cases of heat-related illness, 6 (1.3%) as “undetermined” cases, and 24 (5.3%) as “ruled out” cases. Thus, applying these exclusion criteria to the BioSense query increased the overall PPV of this query from 74.4% to 93.4%.

Discussion

To our knowledge, this heat-related illness syndromic surveillance initiative by a national workgroup is the first to create and evaluate a robust novel query for heat-related illness that combines diagnosis codes, chief complaint terms, and exclusion criteria. Both the BioSense query and the novel query identified hundreds of patients with probable heat-related illness from Maricopa County’s emergency department and inpatient hospital records. In addition, both queries had high PPVs. However, the novel query had a higher PPV (88%) than the BioSense query (74%), and it produced a line list with fewer records retrieved in error.

The main difference between the BioSense query and the novel query was the use of exclusion criteria in the latter. The results suggest that the use of exclusion criteria in the novel query may have helped increase the PPV for heat-related illness and eliminate numerous false-positive records. The benefits of these exclusion criteria are further demonstrated by the fact that when exclusion terms were applied to the initial BioSense query, the PPV for heat-related illness improved from 74.4% to 93.4%.

Syndromic surveillance is defined by the Centers for Disease Control and Prevention as public health surveillance that emphasizes the use of near real-time prediagnostic data.34 Indeed, both queries were applied to near real-time prediagnostic data and resulted in high PPVs. These findings are consistent with other studies that showed the value of using real-time health data for syndromic surveillance of heat-related illness.18–21,35

However, with the exception of the New Jersey study,35 all of these studies relied solely on diagnosis codes to identify cases of heat-related illness. Although diagnosis codes used in both queries in our study identified about half of the cases in each line list, 25% to 35% of the cases of heat-related illness in our study were detected with chief complaint text. Therefore, by using chief complaint search terms, we found many additional cases of heat-related illness in both queries that otherwise may have been missed, confirming the importance of including chief complaint search terms in syndromic surveillance of heat-related illness.

As expected, the frequency of cases of heat-related illness found with either query was substantially higher during Arizona’s heat season than during its cooler seasons. In addition, syndromic surveillance of heat-related illness via either query resulted in a higher PPV and a lower false-positive rate during the hotter months than during the cooler months. The proportion of false positives resulting from the BioSense query was almost 5 times lower during the heat season than during the cooler season, and the proportion of false positives resulting from the novel query was >6 times lower during the heat season than during the cooler season. These findings are consistent with findings from a study in New Jersey that demonstrated higher sensitivity and PPV for heat-related illness surveillance during heat waves when compared with other times during the summer.35

Our analysis is the first step in implementing heat-related illness syndromic surveillance in Maricopa County. MCDPH plans to validate the novel query with finalized 2015 hospital discharge data and to modify the query based on sensitivity and specificity calculations. The department will analyze line lists to identify descriptive characteristics of patients with heat-related illness, risk factors for heat-related illness, and clinical outcomes after heat-related illness, which may help target public health interventions toward those most affected by heat-related illness. In addition, current trends will be compared with historical data to identify aberrations and signal critical moments for public health action.36

Limitations

This analysis had several limitations, the understanding of which may help guide future studies of heat-related illness syndromic surveillance. First, MCDPH identified “probable” cases of heat-related illness through an unblinded, 2-person manual review of each line list. Classification decisions may have been subject to bias, although we attempted to mitigate potential bias by establishing detailed case definitions for internal consistency. A potentially more accurate way to calculate sensitivity and PPV would have involved validating the queries with final hospital discharge data. However, these data were still not available to MCDPH as of May 2016, and the objective of our study was actually to evaluate the performance of near real-time syndromic surveillance, based on data that were available months before final discharge data could be obtained.

Second, the findings may not be generalizable to the general or patient populations in Maricopa County. Because the BioSense data came from only 11 of the 33 hospitals in the county, the true burden of heat-related illness in Maricopa County may have been underestimated by syndromic surveillance. Nevertheless, syndromic surveillance captured heat-related illness trend information that was useful for MCDPH. Third, these findings may not be generalizable to other regions of the United States. The novel heat-related illness query performed well for MCDPH during the heat season, but it performed less well during the cooler seasons (although still better than the BioSense query). This finding suggests that heat-related illness syndromic surveillance may be less effective in regions with cooler climates or where heat-related illness is less common, which is particularly relevant given that the incidence of heat-related illness varies throughout the United States by geography and climate.13,37

Fourth, because the novel query was based on terms that focused solely on heat exhaustion and heatstroke, the true burden of extreme heat exposure may have been underestimated. It may be difficult to distinguish between patients presenting with exacerbations of other diseases resulting from heat exposure and patients who would have had those exacerbations regardless of the temperature. Given this uncertainty, it may be useful to monitor other conditions (eg, dehydration) during extreme heat events. This approach may increase sensitivity (ie, detect more cases of heat-related illness) but decrease specificity (ie, increase the number of cases of non–heat-related illness).35

Finally, new bioinformatics technology, such as machine learning programs and natural-language processing, was not available to MCDPH for this analysis. These programs develop algorithms that can be applied to various data sets, allowing records to be accurately categorized. Indeed, research in the field of bioinformatics has demonstrated that the use of machine learning algorithms, natural-language processing, and logistic regression, alone or in combination with free text search, can increase query sensitivity and specificity >90%.38–41 Public health agencies could explore the long-term use of these tools and strategies in syndromic surveillance as a way to save time and conserve labor resources.42,43

Conclusion

In Maricopa County, this novel query demonstrated a high PPV for heat-related illness, particularly during the heat season. The performance of this query confirmed that it can be used successfully and may be adapted by public health departments to meet local needs. If public health agencies standardized heat-related illness syndromic surveillance queries, data could be shared among jurisdictions, and situational awareness could be improved nationally. Whether this novel query will become the standard and whether its use in the future would substantially improve public health heat-related illness prevention outcomes remains to be seen.

Acknowledgments

We thank Matthew Roach, Jessica Wurster, Lorri Cameron, Fatema Mamou, Jessica Helms, and the Council of State and Territorial Epidemiologists for their contributions and support of this work.

Footnotes

Declaration of Conflicting Interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Henri Ménager is funded by Centers for Disease Control and Prevention grant 5NU38EH000618-08-00.

References

  • 1. Basu R. High ambient temperature and mortality: a review of epidemiologic studies from 2001 to 2008. Environ Health. 2009;8:40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Basu R, Samet JM. Relation between elevated ambient temperature and mortality: a review of epidemiologic evidence. Epidemiol Rev. 2002;24(2):190–202. [DOI] [PubMed] [Google Scholar]
  • 3. Gasparrini A, Guo Y, Hashizume M, et al. Mortality risk attributable to high and low ambient temperature: a multicountry observational study. Lancet. 2015;386(9991):369–375. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Matte TD, Lane K, Ito K. Excess mortality attributable to extreme heat in New York City, 1997-2013. Health Secur. 2016;14(2):64–70. [DOI] [PubMed] [Google Scholar]
  • 5. Anderson GB, Bell ML. Heat waves in the United States: mortality risk during heat waves and effect modification by heat wave characteristics in 43 US communities. Environ Health Perspect. 2011;119(2):210–218. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Knowlton K, Rotkin-Ellman M, King G, et al. The 2006 California heat wave: impacts on hospitalizations and emergency department visits. Environ Health Perspect. 2009;117(1):61–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Mastrangelo G, Fedeli U, Visentin C, Milan G, Fadda E, Spolaore P. Pattern and determinants of hospitalization during heat waves: an ecologic study. BMC Public Health. 2007;7:200. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Lippmann SJ, Fuhrmann CM, Waller AE, Richardson DB. Ambient temperature and emergency department visits for heat-related illness in North Carolina, 2007-2008. Environ Res. 2013;124:35–42. [DOI] [PubMed] [Google Scholar]
  • 9. Johnson MG, Brown S, Archer P, Wendelboe A, Magzamen S, Bradley KK. Identifying heat-related deaths by using medical examiner and vital statistics data: surveillance analysis and descriptive epidemiology—Oklahoma, 1990-2011. Environ Res. 2016;150:30–37. [DOI] [PubMed] [Google Scholar]
  • 10. Lugo-Amador NM, Rothenhaus T, Moyer P. Heat-related illness. Emerg Med Clin North Am. 2004;22(2):315–327. [DOI] [PubMed] [Google Scholar]
  • 11. Fletcher BA, Lin S, Fitzgerald EF, Hwang SA. Association of summer temperatures with hospital admissions for renal diseases in New York State: a case-crossover study. Am J Epidemiol. 2012;175(9):907–916. [DOI] [PubMed] [Google Scholar]
  • 12. Lin S, Luo M, Walker RJ, Liu X, Hwang SA, Chinery R. Extreme high temperatures and hospital admissions for respiratory and cardiovascular diseases. Epidemiology. 2009;20(5):738–746. [DOI] [PubMed] [Google Scholar]
  • 13. Hess JJ, Saha S, Luber G. Summertime acute heat illness in US emergency departments from 2006 through 2010: analysis of a nationally representative sample. Environ Health Perspect. 2014;122(11):1209–1215. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Petitti DB, Hondula DM, Yang S, Harlan SL, Chowell G. Multiple trigger points for quantifying heat-health impacts: new evidence from a hot climate. Environ Health Perspect. 2016;124(2):176–183. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Centers for Disease Control and Prevention. National Syndromic Surveillance Program (NSSP): BioSense background. http://www.cdc.gov/nssp/biosense/background.html. Accessed April 28, 2016.
  • 16. Lombardo J, Burkom H, Elbert E, et al. A systems overview of the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE II). J Urban Health. 2003;80(2, suppl 1):i32–i42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Heffernan R, Mostashari F, Das D, Karpati A, Kulldorff M, Weiss D. Syndromic surveillance in public health practice, New York City [published erratum appears in Emerg Infect Dis. 2006;12(9):1472]. Emerg Infect Dis. 2004;10(5):858–864. [DOI] [PubMed] [Google Scholar]
  • 18. Smith S, Elliot AJ, Hajat S, Bone A, Smith GE, Kovats S. Estimating the burden of heat illness in England during the 2013 summer heatwave using syndromic surveillance. J Epidemiol Community Health. 2016;70(5):459–465. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Elliot AJ, Bone A, Morbey R, et al. Using real-time syndromic surveillance to assess the health impact of the 2013 heatwave in England. Environ Res. 2014;135:31–36. [DOI] [PubMed] [Google Scholar]
  • 20. Schaffer A, Muscatello D, Broome R, Corbett S, Smith W. Emergency department visits, ambulance calls, and mortality associated with an exceptional heat wave in Sydney, Australia, 2011: a time-series analysis. Environ Health. 2012;11(1):3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Rhea S, Ising A, Fleischauer AT, Deyneka L, Vaughan-Batten H, Waller A. Using near real-time morbidity data to identify heat-related illness prevention strategies in North Carolina. J Community Health. 2012;37(2):495–500. [DOI] [PubMed] [Google Scholar]
  • 22. Josseran L, Fouillet A, Caillère N, et al. Assessment of a syndromic surveillance system based on morbidity data: results from the Oscour network during a heat wave. PLoS One. 2010;5(8):e11984. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Leonardi GS, Hajat S, Kovats RS, Smith GE, Cooper D, Gerard E. Syndromic surveillance use to detect the early effects of heat-waves: an analysis of NHS direct data in England. Soz Praventivmed. 2006;51(4):194–201. [DOI] [PubMed] [Google Scholar]
  • 24. Council of State and Territorial Epidemiologists. Heat-related illness syndrome query: a guidance document for implementing heat-related illness syndromic surveillance in public health practice. http://www.cste.org/resource/resmgr/pdfs/pdfs2/CSTE_Heat_Syndrome_Case_Defi.pdf. Published 2016. Accessed October 4, 2016.
  • 25. US Census Bureau. Profile of general population and housing characteristics, 2010: Maricopa County, Arizona. http://factfinder.census.gov/faces/nav/jsf/pages/index.xhtml. Accessed May 9, 2016.
  • 26. Longwoods Tourism Economics. 2014 Arizona Domestic Visitation Profile: Arizona vs Phoenix and Central Region. Phoenix, AZ: Arizona Office of Tourism; 2014. [Google Scholar]
  • 27. South West Weather. Facts about 100 degree temperatures for Phoenix. http://www.southwestweather.com/wx/wx100degreeday.php. Accessed July 23, 2015.
  • 28. Maricopa County Department of Public Health. Heat-Associated Deaths in Maricopa County, AZ: Multiyear Report for 2006-2013. Phoenix, AZ: Maricopa County Department of Public Health; 2014. [Google Scholar]
  • 29. Maricopa County Department of Public Health. Heat Morbidity Surveillance Using Hospital Discharge Data: Multi-year Data (2008-2012). Phoenix, AZ: Maricopa County Department of Public Health; 2014. [Google Scholar]
  • 30. National Oceanic and Atmospheric Administration. NOWData—NOAA online weather data. http://w2.weather.gov/climate/xmacis.php?wfo=psr. Accessed May 6, 2016.
  • 31. Tokars JI, English R, McMurray P, Rhodes B. Summary of data reported to CDC’s national automated biosurveillance system, 2008. BMC Med Inform Decis Mak. 2010;10:30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. SAS Institute Inc. SAS Enterprise Guide Version 5.1. Cary, NC: SAS Institute Inc; 2012. [Google Scholar]
  • 33. Harlan SL, Declet-Barreto JH, Stefanov WL, Petitti DB. Neighborhood effects on heat deaths: social and environmental predictors of vulnerability in Maricopa County, Arizona. Environ Health Perspect. 2013;121(2):197–204. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34. Centers for Disease Control and Prevention. National Syndromic Surveillance Program. http://www.cdc.gov/nssp/overview.html. Accessed December 1, 2016.
  • 35. Berry M, Fagliano J, Tsai S, McGreevy K, Walsh A, Hamby T. Evaluation of heat-related illness surveillance based on chief complaint data from New Jersey hospital emergency rooms. Online J Public Health Inform. 2013;5(1):e125. [Google Scholar]
  • 36. Josseran L, Caillère N, Brun-Ney D, et al. Syndromic surveillance and heat wave morbidity: a pilot study based on emergency departments in France. BMC Med Inform Decis Mak. 2009;9:14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Fechter-Leggett ED, Vaidyanathan A, Choudhary E. Heat stress illness emergency department visits in national environmental public health tracking states, 2005-2010. J Community Health. 2016;41(1):57–69. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Liu W, Chung BC, Wang R, Ng J, Morlet N. A genetic algorithm enabled ensemble for unsupervised medical term extraction from clinical letters. Health Inf Sci Syst. 2015;3:5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Xi N, Wallace R, Agarwal G, Chan D, Gershon A, Gupta S. Identifying patients with asthma in primary care electronic medical record systems chart analysis-based electronic algorithm validation study. Can Fam Physician. 2015;61(10):e474–e483. [PMC free article] [PubMed] [Google Scholar]
  • 40. Lee SM, Abbott P, Johantgen M. Logistic regression and Bayesian networks to study outcomes using large data sets. Nurs Res. 2005;54(2):133–138. [DOI] [PubMed] [Google Scholar]
  • 41. Ford E, Carroll JA, Smith HE, Scott D, Cassell JA. Extracting information from the text of electronic medical records to improve case detection: a systematic review. J Am Med Inform Assoc. 2016;23(5):1007–1015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. Haerian K, Varn D, Vaidya S, Ena L, Chase HS, Friedman C. Detection of pharmacovigilance-related adverse events using electronic health records and automated methods. Clin Pharmacol Ther. 2012;92(2):228–234. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Chen R, Sun J, Dittus RS, et al. Patient stratification using electronic health records from a chronic disease management program [published online January 4, 2016]. IEEE J Biomed Health Inform. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Public Health Reports are provided here courtesy of SAGE Publications

RESOURCES