Emergency Medical Services and Syndromic Surveillance: A Comparison With Traditional Surveillance and Effects on Timeliness

Abstract

Objective

Traditional public health surveillance of nonfatal opioid overdose relies on emergency department (ED) billing data, which can be delayed substantially. We compared the timeliness of 2 new data sources for rapid drug overdose surveillance—emergency medical services (EMS) and syndromic surveillance—with ED billing data.

Methods

We used data on nonfatal opioid overdoses in Kentucky captured in EMS, syndromic surveillance, and ED billing systems during 2018-2019. We evaluated the time-series relationships between EMS and ED billing data and syndromic surveillance and ED billing data by calculating cross-correlation functions, controlling for influences of autocorrelations. A case example demonstrates the usefulness of EMS and syndromic surveillance data to monitor rapid changes in opioid overdose encounters in Kentucky during the COVID-19 epidemic.

Results

EMS and syndromic surveillance data showed moderate-to-strong correlation with ED billing data on a lag of 0 (r = 0.694; 95% CI, 0.579-0.782; t = 9.73; df = 101; P < .001; and r = 0.656; 95% CI, 0.530-0.754; t = 8.73; df = 101; P < .001; respectively) at the week-aggregated level. After the COVID-19 emergency declaration, EMS and syndromic surveillance time series had steep increases in April and May 2020, followed by declines from June through September 2020. The ED billing data were available for analysis 3 months after the end of a calendar quarter but closely followed the trends identified by the EMS and syndromic surveillance data.

Conclusion

Data from EMS and syndromic surveillance systems can be reliably used to monitor nonfatal opioid overdose trends in Kentucky in near–real time to inform timely public health response.

The dynamics of the opioid overdose epidemic in the United States have been changing rapidly in recent years, with a shift from predominantly prescription opioids to heroin and synthetic opioids. ^{1 -3} In 2018, the number of fatal opioid overdoses in the United States was 46 802. ⁴ In 2015 and 2016, the Centers for Disease Control and Prevention (CDC) initiated 3 programs to support expanded state capacity to address the opioid overdose epidemic: Prevention for States, ⁵ Enhanced State Opioid Overdose Surveillance (ESOOS), ⁶ and the Data-Driven Prevention Initiative. ⁷ Under ESOOS, CDC funded states to provide timely and comprehensive data on fatal and nonfatal opioid overdoses and use the data to inform prevention and intervention efforts at local, state, and national levels.

Traditional public health surveillance of nonfatal opioid overdose relies heavily on emergency department (ED) billing data. In some jurisdictions, the availability of ED billing data for epidemiologic analysis is delayed by 6 to 24 months, leading to a delay in national reporting of up to 3 years. ⁸ Active monitoring of health outcomes for public health surveillance using ED billing data is retrospective and has limited value to inform timely response to emerging public health problems.

Emergency medical services (EMS) ⁹ and syndromic surveillance ^10,11  data are relatively new public health surveillance data sources that allow timely and broadened surveillance on nonfatal opioid overdoses. Recent opioid overdose methodological research—typically at the hospital or local agency level—has focused on creating and validating various surveillance definitions using EMS and syndromic surveillance data. Data on EMS naloxone administration have been used as a proxy for identification of opioid overdose. ^12,13 Where opioid overdose cases were identified by EMS naloxone administration, EMS trends highly correlated with ED billing data. ⁹ In addition to administration of naloxone, definitions have been developed to include criteria for other coded and free-text elements in the EMS data to improve the accuracy of indications of opioid overdose. ¹⁴ Limited methodological work has been conducted on the validity of these EMS definitions and the alignment between opioid overdose trends captured by EMS data and ED billing data. Thus, unlike with ED billing data, no consensus definition for capturing opioid overdose encounters in EMS data has been established.

Syndromic surveillance was developed for early detection of biologic agents after 9/11. ¹⁵ Syndromic surveillance data are generated from medical provider reporting of treatment encounters and include patient symptoms and complaints, physician notes, discharge diagnosis codes, and other clinical features. ¹⁶ Syndromic surveillance has been used extensively to track public health outbreaks ¹⁵ and has also proven useful for opioid overdose surveillance. ¹⁷ Under ESOOS, CDC worked with participating jurisdictions to develop definitions for capturing opioid overdose encounters in syndromic surveillance data and used these definitions to identify overdose trends. ¹⁷ Syndromic surveillance data are timely: records are typically available within 24-48 hours after an encounter of care. ¹⁷ As with EMS data, limited methodological work has been conducted on the validity for opioid overdose monitoring of current syndromic surveillance definitions.

As an ESOOS recipient, Kentucky supplemented ED billing data–based opioid overdose morbidity surveillance with EMS data for prehospital treatment of suspected opioid overdoses and syndromic surveillance data as a rapid form of ED data for opioid overdose surveillance. This study is part of the Kentucky ESOOS team’s methodological work to determine the 2 rapid systems’ usefulness for monitoring opioid overdose trends when traditional data are not yet available because of lags in reporting. This study aimed to retrospectively assess how well the Kentucky weekly EMS and syndromic surveillance opioid overdose time series aligned with the time series for opioid overdose captured in ED billing data.

Methods

Data Sources

We included data on opioid overdose encounters that occurred from January 1, 2018, through December 31, 2019, captured by 3 Kentucky statewide data sets: ED billing data, EMS data, and syndromic surveillance data. Kentucky hospitals are required to submit quarterly ED billing data to the state no later than 45 days after the close of each quarter. ^18,19 Finalized data are typically available for analysis 90 days after the end of a quarter. ²⁰ EMS agencies licensed in Kentucky are required to submit monthly data to the Kentucky Board of EMS on the 15th of the next month, ²¹ but the Kentucky ESOOS team estimated that the median reporting time between encounter occurrence and EMS record submission to the state EMS system is approximately 60 hours. Hypothetically, a spike in overdoses occurring on a weekend would be detectable in EMS data as early as the middle of the following week. The same spike would not be detected in ED billing data for 3 to 6 months. Syndromic surveillance data in Kentucky are timely, with records typically available within 24–48 hours after an ED encounter—in line with national availability. ¹⁷ The University of Kentucky Institutional Review Board approved this study.

Case Definitions

ED billing data

We classified opioid overdose–related ED visits based on an International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) ²² discharge diagnosis of T40.0X-T40.4X, T40.60, or T40.69, with a sixth character in the range from 1 to 4, and a seventh character “A” indicating an initial visit, in alignment with a national consensus definition. ¹⁴

EMS data

We classified EMS records as suspected opioid overdoses using the Rhode Island ESOOS definition, which includes rule-based criteria for coded and free-text field entries (eg, primary and secondary impression, patient care report narrative) to identify opioid overdose description or naloxone administration with positive response to naloxone. ^23,24 We included only EMS encounters that resulted in transportation of the patient to an ED. Kentucky EMS data conform to national standards (National Emergency Medical Services Information System [NEMSIS] version 3). ²⁵

Syndromic surveillance data

We classified syndromic surveillance records as opioid overdose–related encounters based on a definition developed by CDC’s National Syndromic Surveillance Program and ESOOS participants that uses a combination of chief complaint free-text terms and ICD-10-CM codes. ¹⁷

Time-series analysis

We aggregated EMS, syndromic surveillance, and ED billing opioid overdose–related encounters into weekly counts for time-series analysis. We used a cross-correlation function (CCF) to analyze relationships between the time series. To accurately perform CCF, each time series must be stationary (ie, the basic properties of the time series of the mean, variance, and autocorrelations do not change over time). ²⁶ To establish stationarity in each of the 3 time series (EMS, syndromic surveillance, ED billing), we created a first-order difference time series by replacing the value of each observation (after the first time point) with the difference between its value and the value of the previous time point. We then tested stationarity by using the augmented Dicky–Fuller test ²⁷ and accepted the alternative hypothesis for stationarity when the P value of the test was <.05.

To control for additional influences of autocorrelation, we fit the EMS and syndromic surveillance time series with an autoregressive integrated moving average (ARIMA) model using the auto.arima function in the forecast R package and selecting the model with the lowest Akaike information criterion for each EMS and syndromic surveillance time series. ²⁸ For the comparisons of EMS and ED billing data and syndromic surveillance and ED billing data, we applied the respective best ARIMA models, retaining the residuals to be used in the CCF. In effect, this approach modeled the time-series autocorrelation component (the “white noise”) in EMS and syndromic surveillance data and then removed that autocorrelation component from the respective de-trended rapid surveillance data and the de-trended ED billing opioid overdose–related time-series data, before CCF comparison. This approach is commonly referred to as pre-whitening and, in combination with stationarity, is a necessary component for CCF, as detailed by Dean et al. ²⁶

Thus, we adjusted the EMS, syndromic surveillance, and ED billing time series for the effects of trend (stationarity) and autocorrelation (autoregressive and moving average) and calculated CCF between syndromic surveillance and ED billing time series and between EMS and ED billing time series. The CCF calculates the Pearson correlation coefficient (r) between 2 time series on a sliding lagged time scale. Measuring the Pearson correlation on a sliding lag scale can be conceptualized as holding the ED billing time series in place while the comparison time series is moved left and right with a time lag (eg, ±1 week, ±2 weeks). We plotted all calculated Pearson correlation coefficients against the lag to determine the time point at which the maximum correlation between 2 time series occurred. Of special interest for the analysis was the strength of the correlation between each rapid surveillance time series and the ED billing time series when no shift occurred (ie, lag 0 cross-correlation, also known as simultaneity). If 2 series are significantly and highly correlated only at 0 lag, they reflect simultaneous trend patterns. We reported the estimated Pearson correlation coefficients at 0 lag with their 95% CIs. For sensitivity analysis, we repeated this methodology for daily counts of opioid overdose in EMS and syndromic surveillance compared with ED billing time series.

Application of Rapid Surveillance Systems: Opioid Overdose Monitoring During COVID-19

We analyzed data on opioid overdoses using the 3 data systems to facilitate public health planning and response. We calculated weekly opioid overdose counts by data source from January 1, 2019, through September 30, 2020 (the most recent data available at the time of analysis). We overlaid the 3 time series on the same time scale to indicate the coverage by data set along with the national COVID-19 emergency declaration on March 13, 2020. ²⁹ The national emergency declaration is a crude proxy for COVID-19 social distancing and stay-at-home behavior changes. We reported the estimated average weekly totals before and after the declaration with their estimated SDs.

Results

From January 1, 2018, through December 31, 2019, EMS data captured more opioid overdose–related encounters (n = 17 090) than the number of opioid overdoses identified in ED billing (n = 10 606) or syndromic surveillance (n = 10 893) data. EMS, syndromic surveillance, and ED billing time series of weekly counts showed similar trends (Figure 1). A comparison of EMS and syndromic surveillance data with traditional ED billing data revealed several comparable temporal sinusoidal shifts in weekly aggregate trends during the study period.

Figure 1.

Suspected opioid overdose encounters, by week and surveillance data source (emergency department billing, emergency medical services, and syndromic surveillance), with 4-week rolling average, Kentucky, 2018-2019. The 4-week rolling average provides a general smoothing of each surveillance system for the purpose of visual comparison. Data sources: claims files, Cabinet for Health and Family Services, Office of Health Data and Analytics; Kentucky Board of Emergency Medical Services: Kentucky State Ambulance Reporting System; and Kentucky Syndromic Surveillance Data Cabinet for Health and Family Services, Department for Public Health.

First-order differencing was sufficient to establish stationarity for EMS, syndromic surveillance, and ED billing weekly time series. The augmented Dicky–Fuller test confirmed that all 3 time series met stationarity (respective test statistics: –5.95, –5.82, –6.71; P = .01 for all tests). For the results of the CCF, with the sliding lag correlations of EMS and syndromic surveillance data compared with ED billing data, the most significant Pearson correlation occurred at a lag of zero for EMS–ED billing (r = 0.694 [95% CI, 0.579-0.782]; t = 9.73; df = 101; P < .001) and syndromic surveillance–ED billing (r = 0.656 [95% CI, 0.530-0.754]; t = 8.73; df = 101; P < .001) (Figure 2, Table). Other statistical time-lagged relationships had weak correlations. Results of time-series comparisons indicated that both time series followed similar patterns at the weekly aggregate level.

Figure 2.

Cross-correlation function between rapid surveillance systems (emergency medical services and syndromic surveillance) and a traditional surveillance system (emergency department billing) in detecting suspected opioid overdose cases, by various positive and negative lags, Kentucky, 2018-2019. The graphs visually indicate a moderately strong correlation at lag of 0 (simultaneity). The correlations at lag different from 0 are relatively weak. If there were no real relationship between the systems, much lower correlation would be expected (similar to white noise). Data sources: claims files, Cabinet for Health and Family Services, Office of Health Data and Analytics; Kentucky Board of Emergency Medical Services: Kentucky State Ambulance Reporting System; and Kentucky Syndromic Surveillance Data Cabinet for Health and Family Services, Department for Public Health.

Table.

Cross-correlation function results for zero lag, between rapid surveillance systems and traditional ED billing data, for identified nonfatal suspected opioid overdoses, Kentucky, 2018-2019 ^a

Aggregate	Pearson correlation (r) (95% CI)	t Test statistic	df	P value b
Weekly aggregate
SyS–ED	0.66 (0.53-0.75)	8.73	101	<.001
EMS–ED	0.69 (0.58-0.78)	9.68	101	<.001
Daily aggregate
SyS–ED	0.68 (0.64-0.72)	24.00	728	<.001
EMS–ED	0.54 (0.48-0.59)	17.09	728	<.001

Abbreviations: ED, emergency department; EMS, emergency medical services; SyS, syndromic surveillance.

^aResults indicate the Pearson correlation at weekly and daily aggregate levels. Results are presented for the maximum correlation between each set. This maximum was lag of zero for all comparisons.

^bUsing the t test, with P < .05 considered significant.

A sensitivity analysis using daily aggregate counts of statewide EMS, syndromic surveillance, and ED billing opioid overdoses indicated similar results to the weekly time-series analysis. The CCF indicated a significant relationship between EMS and ED billing opioid overdose–related data at zero lag (r = 0.535 [95% CI, 0.481-0.585]; t = 17.09; df = 728; P < .001) and syndromic surveillance and ED billing opioid overdose–related data at zero lag (r = 0.680 [95% CI, 0.639-0.717]; t = 25.00; df = 728; P < .001) (Table).

Opioid Overdose Monitoring During COVID-19

An examination of weekly counts of opioid overdose–related encounters captured in EMS, syndromic surveillance, and ED billing data with 4-week rolling averages found similar patterns for the 3 time series: relatively flat opioid overdose trends before declaration of a national emergency (March 13, 2020), a steep increase through the end of May 2020, and a sharp decline during June–September 2020 (Figure 3).

Figure 3.

Suspected opioid overdose encounters, by week and surveillance data source (emergency department billing, emergency medical services, and syndromic surveillance), with 4-week rolling average, Kentucky, January 1, 2019, through September 30, 2020. The solid vertical line indicates when a national emergency was declared for COVID-19, on March 13, 2020. At the time of analysis, emergency department billing data were available only through quarter 2 of 2020, confirming the large increase in opioid overdoses several months after initial indications in emergency medical services and syndromic surveillance data. Data sources: claims files, Cabinet for Health and Family Services, Office of Health Data and Analytics; Kentucky Board of Emergency Medical Services: Kentucky State Ambulance Reporting System; and Kentucky Syndromic Surveillance Data Cabinet for Health and Family Services, Department for Public Health.

Discussion

The time series from the 2 rapid surveillance systems, EMS and syndromic surveillance, followed patterns that were similar to ED billing data. We found a significant relationship between syndromic surveillance and EMS data compared with ED billing data, which are traditionally used for public health surveillance of nonfatal overdoses. Increases or decreases in the number of opioid overdoses in Kentucky could be detected in near–real time in EMS and syndromic surveillance systems to alert public health. Omission of these rapid surveillance systems and relying on ED billing data only would have delayed the detection of changes in patterns by several months.

The COVID-19 pandemic provided an opportunity to test the usefulness of syndromic surveillance and EMS data to identify anticipated changes in opioid overdose patterns resulting from possible interruptions of opioid use disorder treatment services, changes in access to naloxone and other harm reduction services and treatments, and additional risk for opioid overdose associated with social isolation and economic uncertainty. The call for timely data to monitor changes in opioid overdose trends and inform rapid response was never so urgent. ^30,31 Using the state capacity for timely opioid overdose surveillance built with support from the ESOOS program, in collaboration with partners from the Kentucky Board of EMS, the Kentucky Department for Public Health, and the Kentucky Health Information Exchange, Kentucky used EMS and syndromic surveillance data to provide weekly updates to state agencies, communities, and researchers. ³² The constant stream of data from the 2 complementary rapid data sources allowed for ongoing monitoring and early detection of an escalation in opioid overdoses that began as early as April 2020, signaling the need for immediate response and harm reduction activities. ³³ Traditional ED billing data were available for analysis with a 6-month delay, but every additional quarter of ED billing data confirmed the trend in opioid overdoses indicated by syndromic surveillance and EMS data.

The results of this study add to the growing body of research that supports EMS ^12,24,34,35 and syndromic surveillance ^11,17 as effective public health surveillance systems for opioid overdose. Previous researchers typically focused on only one surveillance system, limited to one agency or facility, and did not compare the surveillance system with traditional standards. Our study found that both EMS and syndromic surveillance trends for opioid overdose are statistically comparable with trends indicated by ED billing data in Kentucky. Not only do these sources align with traditional standards, but they also expand on the information and context of opioid overdoses. EMS data, for example, may have additional inherent benefits in both coverage and granularity. EMS data can capture cases in which the patient refused transport to the ED and, thus, may expand coverage of nonfatal overdose incidence provided by ED billing data. In addition to increasing coverage and timeliness, EMS data contain rich information such as patient care narratives, incident and resident location, and medications administered. Likewise, syndromic surveillance data provide additional information from the encounter that is not available in traditional ED billing data, including chief complaint, triage notes, and clinical impressions. In short, both EMS and syndromic surveillance data can be used accurately for overdose surveillance and may also provide contextual information in their expansive data fields.

A key focus of this article was to demonstrate the application of CCF to compare statewide rapid surveillance systems with traditional standards. Replication at local levels will incur data quality and statistical considerations. Rapid surveillance systems often have varying data quality at the agency or facility level; as such, an analysis of an individual county primarily served by a facility with incomplete data could magnify data quality issues compared with aggregate views. Compared with ED billing data, low-quality rapid surveillance data may depress time-series correlations; conversely, high-quality data may increase correlations. In addition, the statistical methodology presented here is not robust when events are rare. It is common for rural areas to have only a few nonfatal opioid overdoses in a given year. A weekly aggregate would result in mostly zeros, leading to statistical instability for the CCF. Further research into robust correlations for CCF in these instances is necessary. Researchers and practitioners interested in application of this methodology at the local level should consider data quality and statistical robustness.

Limitations

This study had several limitations. First, we used data from only one state; as such, the results may not be generalizable to other states. Second, the quality and completeness of the data may vary by period and reporting entity. Kentucky EMS data are derived from more than 200 EMS agencies operating in the state. Syndromic surveillance data are based on nearly 100 EDs using various electronic health record systems. Third, the time-series comparison methodology is sensitive to time series with small counts and may not be applicable to evaluation in small local areas, such as a rural county or town.

This study also had several strengths. First, the study is centered in the results that 2 relatively novel rapid surveillance systems are both comparable with traditional standard ED billing data for the purpose of trend analysis of opioid overdose surveillance. The results of our analysis provide evidence that both surveillance systems are similarly accurate in monitoring opioid overdose patterns, yet each has the potential to provide unique contextual information. Second, this time-series analysis approach can be easily adopted by jurisdictions evaluating their own surveillance systems and definitions, within the context of the limitations. Third, monitoring of opioid overdoses during COVID-19 provides a meaningful example of significant change in opioid overdose patterns that can be detected by using rapid surveillance in near–real time.

Conclusions

This study could inform future efforts among state and local jurisdictions to investigate the usefulness of EMS and syndromic surveillance data to inform efforts to use data proactively for public health response to the evolving opioid overdose epidemic, which is intertwined with the COVID-19 pandemic. Sustaining and expanding infrastructure and methodology for syndromic surveillance and EMS data will be increasingly important for proactive use of opioid overdose surveillance data that guide effective public health and safety response.

ORCID iDs

Peter J. Rock, MPH https://orcid.org/0000-0002-9893-4084

Svetla Slavova, PhD https://orcid.org/0000-0002-4541-6574