Abstract
Morbidity and mortality from exposure to extreme cold highlight the need for meaningful temperature thresholds to activate public health alerts. We analyzed emergency department (ED) records for cold temperature–related visits collected by the Acute Care Enhanced Surveillance system—a syndromic surveillance system that captures data on ED visits from hospitals in Ontario—for geographic trends related to ambient winter temperature. We used 3 Early Aberration Reporting System algorithms of increasing sensitivity—C1, C2, and C3—to determine the temperature at which anomalous counts of cold temperature–related ED visits occurred in northern and southern Ontario from 2010 to 2016. The C2 algorithm was the most sensitive detection method. Results showed lower threshold temperatures for Acute Care Enhanced Surveillance alerts in northern Ontario than in southern Ontario. Public health alerts for cold temperature warnings that are based on cold temperature–related ED visit counts and ambient temperature may improve the accuracy of public warnings about cold temperature risks.
Keywords: syndromic surveillance, cold alert, cold temperature threshold, Early Aberration Reporting System, extreme cold
The consensus of scientific research is that climate change is associated with a global increased risk of extreme weather and climate events.1 Generally, health research has focused on the effects of extreme heat and global warming trends; however, chaotic weather patterns induced by a warming Arctic may result in atypically cold winters midlatitude.2 In a recent study of the relative impact of cold and hot temperatures on mortality rates for selected cities in temperate and tropical climates, Gasparrini et al determined that cold was responsible for a substantially higher proportion of deaths than heat.3,4 Understanding the relationship between temperature exposure and acute or chronic health risks and death is essential for creating public health protocols for emergency planning and risk communication.
Real-time syndromic surveillance of acute care patient records could help identify cold temperature–related (hereinafter, cold-related) health effects. Myriad cofactors influence the effect of temperature on health—for example, a person’s age and socioeconomic status, length and temperature of exposure, humidity, and volume of precipitation. Therefore, it is difficult to predict temperatures at which alerts should be issued. Real-time surveillance of cold-related health effects, in concert with real-time surveillance of ambient meteorological factors, provides an opportunity to characterize a population according to the temperature (or its related cofactor) at which health effects occur. This information may allow for more prudent determination of temperatures (or other factors or combinations thereof) for public health protective measures, such as alert messaging.
Kingston, Frontenac and Lennox & Addington Public Health operates the Acute Care Enhanced Surveillance (ACES) system, which monitors emergency department (ED) visits in real time across the province of Ontario, Canada. Canada’s universal health care system offers equitable access for essentially all health-seeking behavior, regardless of socioeconomic or insurance status. Currently, 132 hospitals provide data to ACES, representing >80% of hospitals in Ontario that have the technical capacity to share data. Triage records for all ED visits are grouped into syndromes of medical importance via natural-language processing techniques based on a maximum entropy algorithm and keywords or phrases in the chief complaints. In addition to monitoring total visit metrics, ACES uses various statistical process control methods to detect aberrant counts that indicate, for example, a possible disease outbreak.5
The objectives of our study were to explore the relationship between real-time observations of cold-related ED visits and cold temperatures to determine if (1) there is a relationship between incidences of health-seeking behavior and ambient temperature and (2) these relationships vary by geography.
Methods
We analyzed data on ED visit records collected by ACES during each winter season (December 1 through March 31) from December 2010 through March 2016 to characterize cold-related ED visits according to ambient temperature exposure. The cold-related ED visit syndrome in ACES uses a maximum entropy algorithm and the following keywords: “frostbite,” “hypothermia,” “cold injury,” and “cold exposure.”
We used the C1, C2, and C3 adaptive algorithms developed for the Early Aberration Reporting System (EARS) by the Centers for Disease Control and Prevention to detect aberrant ED visit counts and generate public health alerts. The methods for the EARS C-algorithms have been described in detail elsewhere.6,7 Briefly, the algorithms standardize daily ED counts through a 7-day moving average: the C1 method uses counts from the 7 previous days, and the C2 and C3 methods include a 2-day lag to prohibit a decrease in sensitivity because of gradual increases in daily counts.8,9 Alerts are triggered when daily counts exceed a predetermined upper alerting threshold limit (usually 3 standard deviations for C1 and C2). The C3 method uses a modified method to determine alerting thresholds based on a 3-day average rather than the daily count.7
We assessed alerts generated through each C-algorithm at the geographic level of hospital to determine rates of cold-related ED visits per 100 000 ED visits, and we aggregated data for hospitals within local public health agency (LPHA) regions. We then compared alert dates with daily minimum temperatures recorded at nearby Environment and Climate Change Canada meteorological stations.10 We pooled the northern LPHAs to represent a population that was exposed to the lowest temperatures in the province. The northern LPHAs—Northwestern Health Unit, Porcupine Health Unit, and Thunder Bay District Health Unit—compose the entire northern portion of Ontario above Lake Superior. The southern LPHA is Toronto Public Health, which is smaller in land area and larger in population when compared with the northern LPHAs, and it represents a population exposed to less extreme cold than the northern LPHAs. We compared the temperatures of alerts in the northern LPHAs with those observed in Toronto Public Health and qualitatively assessed the relative effectiveness of the C-algorithms for determining cold threshold temperatures for public warning protocols.
The aberration alerts generated by the C1, C2, and C3 methods for the northern LPHAs and Toronto Public Health were collated for only the 2014-2015 and 2015-2016 winters because the first 4 winters of the study period were relatively mild and resulted in very few cold-related ED visits.
The health information used in this analysis was obtained and is reported herein in accordance with data-sharing agreements between each hospital or LPHA and ACES.
Results
The northern LPHAs were exposed to the lowest temperatures and had the highest rates of cold-related ED visits of all Ontario LPHAs: Northwestern Health Unit (39.8 per 100 000 ED visits), Porcupine Health Unit (32.0 per 100 000 ED visits), and Thunder Bay District Health Unit (37.3 per 100 000 ED visits). The Toronto-area LPHA had an ED visit rate of 18.8 per 100 000 ED visits for cold-related health conditions. For cold-related ED visits in the northern LPHAs, the average age of patients was 32 years, and 72.1% of patients were male. For cold-related ED visits in the Toronto Public Health area, the average age of patients was 40 years, and 73.2% of patients were male.
For both LPHA regions, the C2 method generated the most alerts: 31 for the northern LPHAs (Table 1) and 34 for Toronto Public Health (Table 2). The C1 and C3 methods produced a similar number of alerts (C1: 16 alerts; C3: 13 alerts) but on different dates for the northern LPHAs. The C1 and C3 methods produced 14 and 20 alerts, respectively, for Toronto Public Health. The number of ED visits resulting in an alert ranged from 1 to 5 for the northern LPHAs and from 1 to 12 for Toronto Public Health.
Table 1.
Minimum temperaturesa at which cold temperature–related ED visit alertsb were generated, based on 3 aberration detection methods,c December-March 2014-2015 and 2015-2016, northern Ontario, Canada
| Date | No. of Cold Temperature–Related ED Visits | Minimum Temperature,°C | Aberration Detection Methodd | ||
|---|---|---|---|---|---|
| C1 | C2 | C3 | |||
| 12-01-14 | 2 | –25.3 | ✓ | ✓ | |
| 12-03-14 | 1 | –18.4 | ✓ | ||
| 12-13-14 | 1 | –3.1 | ✓ | ||
| 12-21-14 | 2 | –9.4 | ✓ | ✓ | |
| 12-31-14 | 2 | –26.8 | ✓ | ✓ | |
| 01-01-15 | 1 | –30.8 | ✓ | ✓ | |
| 01-02-15 | 4 | –30.3 | ✓ | ✓ | ✓ |
| 01-03-15 | 2 | –28.8 | ✓ | ✓ | |
| 01-04-15 | 2 | –30.1 | ✓ | ✓ | |
| 01-08-15 | 4 | –24.6 | ✓ | ||
| 01-12-15 | 4 | –30.1 | ✓ | ||
| 01-30-15 | 3 | –28.1 | ✓ | ✓ | |
| 02-01-15 | 1 | –36.4 | ✓ | ||
| 02-14-15 | 3 | –31.0 | ✓ | ✓ | |
| 02-15-15 | 2 | –35.5 | ✓ | ✓ | |
| 02-16-15 | 4 | –31.0 | ✓ | ✓ | |
| 02-17-15 | 1 | –26.2 | ✓ | ||
| 03-10-15 | 2 | –3.0 | ✓ | ||
| 03-23-15 | 1 | –19.3 | ✓ | ✓ | |
| 12-15-15 | 1 | –9.4 | ✓ | ✓ | |
| 12-18-15 | 1 | –12.0 | ✓ | ||
| 01-01-16 | 1 | –10.2 | ✓ | ✓ | |
| 01-02-16 | 1 | –2.9 | ✓ | ||
| 01-03-16 | 1 | –18.3 | ✓ | ||
| 01-11-16 | 2 | –27.5 | ✓ | ✓ | |
| 01-13-16 | 3 | –29.8 | ✓ | ✓ | |
| 02-08-16 | 1 | –16.3 | ✓ | ✓ | |
| 02-10-16 | 1 | –28.2 | ✓ | ||
| 02-11-16 | 4 | –28.5 | ✓ | ✓ | |
| 02-12-16 | 5 | –31.0 | ✓ | ✓ | |
| 02-13-16 | 3 | –34.2 | ✓ | ✓ | |
| 02-14-16 | 2 | –32.5 | ✓ | ||
| 02-15-16 | 3 | –13.8 | ✓ | ||
| 03-02-16 | 1 | –28.8 | ✓ | ✓ | |
| 03-03-16 | 4 | –30.9 | ✓ | ✓ | |
| 03-19-16 | 1 | –16.2 | ✓ | ||
| 03-20-16 | 1 | –11.8 | ✓ | ||
| Total alerts generated, No. | 16 | 31 | 13 | ||
| Minimum temperature, °C, mean ± SD | –20.5 (10.3) | –23.7 (8.8) | –30.0 (5.6) | ||
Abbreviation: ED, emergency department.
aMinimum temperatures at which alerts occurred for hospitals in the northern local public health agencies of Ontario (Northwestern Health Unit, Porcupine Health Unit, and Thunder Bay District Health Unit). The minimum temperature was the lowest temperature recorded for all locations of ED visits for the date that the alert was generated. Temperature data were collected from local Environment and Climate Change Canada weather-monitoring stations.10
bAlerts were generated by the Acute Care Enhanced Surveillance system, which provides syndromic surveillance for most of Ontario’s acute-care facilities.
cC1, C2, and C3 are adaptive algorithms of differing sensitivity developed for the Early Aberration Reporting System to detect aberrant ED visit counts standardized to 7-day baselines: C1 uses the previous 7 days; C2 and C3 include a 2-day lag before calculating the 7-day baseline. C3 uses a 3-day average rather than daily counts.6,7
dCheckmarks indicate that an alert was generated for that date, based on the aberration detection method indicated.
Table 2.
Minimum temperaturesa at which cold temperature–related ED visit alertsb were generated, based on 3 aberration detection methods,c December-March 2014-2015 and 2015-2016, Toronto, Ontario, Canada
| Date | No. of Cold Temperature–Related ED Visits | Minimum Temperature,°C | Aberration Detection Methodd | ||
|---|---|---|---|---|---|
| C1 | C2 | C3 | |||
| 12-03-14 | 2 | –0.6 | ✓ | ✓ | |
| 12-18-14 | 2 | –4.2 | ✓ | ✓ | |
| 12-23-14 | 2 | 1.9 | ✓ | ||
| 12-24-14 | 2 | 5.6 | ✓ | ||
| 01-05-15 | 3 | –12.5 | ✓ | ✓ | |
| 01-06-15 | 2 | –12.1 | ✓ | ✓ | |
| 01-07-15 | 1 | –17.6 | ✓ | ||
| 01-08-15 | 5 | –14.8 | ✓ | ✓ | ✓ |
| 01-09-15 | 2 | –12.3 | ✓ | ||
| 01-10-15 | 3 | –14.6 | ✓ | ✓ | |
| 01-19-15 | 3 | –7.7 | ✓ | ✓ | |
| 01-20-15 | 2 | –11.7 | ✓ | ||
| 01-30-15 | 2 | –13.7 | ✓ | ✓ | |
| 01-31-15 | 2 | –13.8 | ✓ | ✓ | |
| 02-02-15 | 3 | –14.7 | ✓ | ✓ | |
| 02-03-15 | 5 | –14.0 | ✓ | ✓ | ✓ |
| 02-04-15 | 4 | –7.9 | ✓ | ✓ | |
| 02-05-15 | 1 | –13.7 | ✓ | ||
| 02-06-15 | 3 | –11.4 | ✓ | ||
| 02-15-15 | 11 | –25.1 | ✓ | ✓ | |
| 02-16-15 | 11 | –22.4 | ✓ | ✓ | |
| 02-17-15 | 5 | –13.3 | ✓ | ✓ | |
| 02-19-15 | 5 | –20.5 | ✓ | ||
| 03-28-15 | 3 | –10.0 | ✓ | ✓ | |
| 03-29-15 | 2 | –6.2 | ✓ | ||
| 12-19-15 | 2 | –2.3 | ✓ | ✓ | |
| 12-27-15 | 1 | –0.6 | ✓ | ||
| 12-31-15 | 2 | 0.6 | ✓ | ✓ | |
| 01-01-16 | 2 | –2.9 | ✓ | ✓ | |
| 01-02-16 | 1 | –2.9 | ✓ | ✓ | |
| 01-16-16 | 2 | –9.1 | ✓ | ✓ | |
| 01-18-16 | 1 | –10.1 | ✓ | ||
| 01-19-16 | 3 | –13.3 | ✓ | ||
| 01-20-16 | 1 | –6.3 | ✓ | ||
| 02-13-16 | 4 | –24.7 | ✓ | ✓ | |
| 02-14-16 | 12 | –22.4 | ✓ | ✓ | ✓ |
| 02-15-16 | 6 | –9.6 | ✓ | ||
| 03-02-16 | 2 | –10.3 | ✓ | ||
| 03-15-16 | 1 | 6.9 | ✓ | ||
| 03-20-16 | 1 | –3.0 | ✓ | ||
| 03-21-16 | 1 | –1.9 | ✓ | ||
| 03-31-16 | 1 | 8.5 | ✓ | ||
| Total alerts generated, No. | 14 | 34 | 20 | ||
| Minimum temperature, °C, mean ± SD | –10.9 (8.9) | –8.8 (8.5) | –12.8 (5.5) | ||
Abbreviation: ED, emergency department.
aMinimum temperature at which alerts occurred for hospitals in the region served by Toronto Public Health. Minimum temperature was determined from the lowest temperature recorded for all locations of ED visits for the date that the alert was generated. Temperature data were collected from local Environment and Climate Change Canada weather-monitoring stations.10
bAlerts were generated by the Acute Care Enhanced Surveillance system, which provides syndromic surveillance for most of Ontario’s acute care facilities.
cC1, C2, and C3 are adaptive algorithms of differing sensitivity developed for the Early Aberration Reporting System to detect aberrant ED visit counts standardized to 7-day baselines: C1 uses the previous 7 days; C2 and C3 include a 2-day lag before calculating the 7-day baseline. C3 uses a 3-day average rather than daily counts.6,7
dCheckmarks indicate that an alert was generated for that date, based on the aberration detection method indicated.
We found lower temperatures for alerts generated in the northern LPHAs compared with alerts generated by Toronto Public Health. The mean temperature for alerts in the northern LPHAs ranged from –30.0°C ± 5.6°C for the C3 method to –20.5°C ± 10.3°C for the C1 method (Table 1). The mean temperature alerts for Toronto Public Health ranged from –12.8°C ± 5.5°C for the C3 method to –8.8°C ± 8.5°C for the C2 method (Table 2).
Discussion
Choosing the appropriate alerting algorithm is key to minimizing false alerts in any syndromic surveillance system, especially when seeking to identify a signal from a very specific and relatively small set of keywords, such as keywords related to cold exposure. In our study, the C2 method was the most effective for identifying cold-related ED visits even when considering that the large number of alerts generated need to be assessed for false alerts. The C1 method inherently missed alerts on subsequent days after an initial increase because of the inflated upper threshold that resulted from using the preceding 7-day moving average. For example, the C1 method failed to identify an alert on January 3, 2015, for the northern LPHAs. The C3 method is generally considered the most sensitive of the 3 methods7; however, the C3 method did not identify alerts on several days with aberrant counts, such as on February 11, 2016, in the northern LPHAs and on February 15, 2015, in Toronto Public Health. Although the C2 method generated the most alerts, decisions on which alerts should be investigated lie with the analyst who is investigating them. The work needed to investigate the possibility of false alerts for a low-volume syndrome such as cold exposure should not be too onerous and therefore is likely the preferred option to missing an alert.
The differences in mean alert temperatures may reflect several factors that potentially differ for the populations served by the LPHAs—for example, population acclimation to low temperatures (including biological factors and social norms in preparedness) and the high incidence of homelessness or poverty in an urban environment. For example, 1 hospital in the Toronto Public Health area, Toronto Western Hospital, had rates of cold-related ED visits that were comparable to those of the northern LPHAs (37.8 per 100 000 ED visits). This rate may have resulted from the presence of higher concentrations of cold-vulnerable populations in its service area (eg, homeless people or an immigrant population that may not be habituated to cold temperatures). It may be more prudent to analyze ED visits by hospital geography rather than LPHA geography for the analyses to control for population differences.
One limitation of this study was the use of chief complaint data and the lack of standardization of this data field across hospitals submitting data to ACES. Given the nature of environmental exposure, however, we feel that misclassification would be minimal. Underrepresentation of the burden of cold temperatures through ED visits was more likely, making it imperative to look at other sources of data, such as telemedicine calls and ambulance dispatch data, to augment data on ED visits.
These preliminary results emphasize the need to test different aberration detection methods and to expand on regional temperature thresholds for alerts. We are examining other aberration detection methods, such as other cumulative sums–based methods, to increase the sensitivity of aberration detection with the low counts of ED visits for cold-related health conditions. Future research will explore the effects of extreme cold temperatures on the social determinants of health that are accessible using metrics from the Canadian census data. These analyses should refine our alerting practices to account for differences in population-level characteristics and provide information on population-level heat and cold resiliency. We will also explore alternate sources of health information to validate these results, including retrospective hospital admission records and telemedical records.
Footnotes
Declaration of Conflicting Interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The authors received no financial support for the research, authorship, and/or publication of this article.
References
- 1. Intergovernmental Panel on Climate Change. Climate change 2013: the physical science basis. Summary for policymakers. https://www.ipcc.ch/pdf/assessment-report/ar5/wg1/WGIAR5_SPM_brochure_en.pdf. Published 2013. Accessed May 20, 2016.
- 2. Cohen J, Screen JA, Furtado JC, et al. Recent Arctic amplification and extreme mid-latitude weather. Nature Geosci. 2014;7(9):627–637. [Google Scholar]
- 3. Gasparrini A, Guo Y, Hashizume M, et al. Mortality risk attributable to high and low ambient temperature: a multicountry observational study. Lancet. 2015;386(9991):369–375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Gasparrini A, Guo Y, Hashizume M, et al. Temporal variation in heat-mortality associations: a multicountry study. Environ Health Perspect. 2015;123(11):1200–1207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. KFL&A Public Health. Acute Care Enhanced Surveillance (ACES) user manual. http://www.kflaphi.ca/wp-content/uploads/DRAFT-User-Manual-June-2015.pdf. Published 2015. Accessed June 17, 2016.
- 6. Hutwagner L, Thompson W, Seeman GM, et al. The bioterrorism preparedness and response Early Aberration Reporting System (EARS). J Urban Health. 2003;80(2, suppl 1):i89–i96. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Hutwagner LC, Thompson WW, Seeman GM, et al. A simulation model for assessing aberration detection methods used in public health surveillance for systems with limited baselines. Stat Med. 2005;24(4):543–550. [DOI] [PubMed] [Google Scholar]
- 8. Fricker RD, Jr, Hegler BL, Dunfee DA. Comparing syndromic surveillance detection methods: EARS’ versus a CUSUM-based methodology. Stat Med. 2008;27(17):3407–3429. [DOI] [PubMed] [Google Scholar]
- 9. Jafarpour N, Izadi M, Precup D, et al. Quantifying the determinants of outbreak detection performance through simulation and machine learning. J Biomed Inform. 2015;53:180–187. [DOI] [PubMed] [Google Scholar]
- 10. Government of Canada. Historical climate data: historical data. http://climate.weather.gc.ca/historical_data/search_historic_data_e.html. Accessed June 9, 2016.