Abstract
Objective:
We sought to determine the influence of the Levels of Care for Rhode Island Emergency Departments and Hospitals for Treating Overdose and Opioid Use Disorder (Levels of Care) on emergency department (ED) provision of take-home naloxone, behavioral counseling, and referral to treatment.
Methods:
A retrospective analysis of Rhode Island ED visits for opioid overdose from 2017 to 2018 was performed using data from a statewide opioid overdose surveillance system. Changes in provision of take-home naloxone, behavioral counseling, and referral to treatment before and after Levels of Care implementation were assessed using interrupted time series analysis. We compared outcomes by hospital type using multivariable modified Poisson regression models with generalized estimating equation estimation to account for hospital-level variation.
Results:
We analyzed 238 overdose visits prior to Levels of Care implementation (January to March 2017) and 1,299 overdose post-implementation (hospital certification to December 2018). After implementation, the proportion of patients offered naloxone increased on average by 13% (95% CI 5.6%, 20.4%). Prior to implementation, the proportion of patients receiving behavioral counseling and treatment referral was declining. After implementation, this decline slowed, stabilized, and on average 18.6% more patients received behavioral counseling (95% CI 1.3%, 35.9%) and 23.1% more patients received referral to treatment (95% CI 2.7%, 43.5%). Multivariate analysis showed that after implementation, there was a significant increase in the likelihood of being offered naloxone at Level 1 (aRR 1.31 [95% CI 1.06, 1.61]) and Level 3 (aRR 3.13 [95% CI 1.08, 9.06]) hospitals and an increase in referrals for medication for opioid use disorder (2.5% to 17.8%) at Level 1 hospitals (RR 7.73 [95% CI 3.22, 18.55]. Despite these increases, less than half of patients treated for an opioid overdose received behavioral counseling or referral to treatment
Conclusion:
Establishment of ED policies for treatment and services after opioid overdose improved naloxone distribution, behavioral counseling, and referral to treatment at hospitals without previously established opioid overdose services. Future investigations are needed to better characterize implementation barriers and evaluate policy impact on patient outcomes.
Introduction
In response to rising opioid overdose deaths in Rhode Island, the Rhode Island state general assembly passed the 2016 Alexander C. Perry and Brandon Goldner Act which outlines emergency department (ED) and hospital requirements for comprehensive discharge planning after an ED visit for opioid overdose (henceforth the “discharge planning law”).1 This legislation also mandated that the Rhode Island Department of Health (RIDOH) and the Rhode Island Department of Behavioral Healthcare, Developmental Disabilities, and Hospitals (BHDDH) establish regulations for ED and hospital treatment of opioid overdose and opioid use disorder (OUD). In March 2017, RIDOH and BHDDH published treatment standards for ED and hospital care of adult patients treated after opioid overdose and with OUD, the Levels of Care for Rhode Island Emergency Departments and Hospitals for Treating Overdose and Opioid Use Disorder (Levels of Care).2
Rhode Island’s Levels of Care is the first state-based standards of care for overdose and OUD and outlines three tiers of care that hospitals can provide. The most basic requirements, Level 3, entail providing patient education on safe opioid storage and disposal, substance use disorder screening, referral to treatment upon discharge, peer recovery consultation, urine drug testing for fentanyl and fentanyl analogues, naloxone distribution, and mandated reporting of suspected opioid overdoses to RIDOH within 48 hours of their occurrence.3 Level 2 facilities meet all of Level 3 requirements and must have trained staff that can conduct comprehensive substance use assessments. Level 1 facilities meet all of Levels 2 and 3 requirements and can also initiate, stabilize, and link patients to an outpatient treatment program that can maintain individuals on medication for OUD (MOUD), buprenorphine, methadone, or naltrexone. Referral to outpatient treatment with MOUD can occur to an office-based clinic, an Opioid Treatment Program, or an Opioid Treatment Program that is also a BHDDH certified Center of Excellence. Centers of Excellence are specialty treatment centers that provide comprehensive, evidence-based treatment of OUD with MOUD.4 As of 2018, four organizations were running Centers of Excellence at 13 locations throughout Rhode Island.5,6
Levels of Care requirements are a combination of evidence-based interventions and treatments, recommendations by content experts, and actions prescribed by the existing discharge planning legislation. Community naloxone distribution has been shown to be associated with reduced overdose mortality and increased services uptake.7–15 ED behavioral counseling with referral to treatment has also been shown to improve treatment uptake.16–18 Finally, MOUD not only reduces risk of repeat overdose, but also decreases mortality by more than 50%.19 ED initiation of buprenorphine has been shown to improve referral to treatment compared to behavioral intervention alone and to be cost effective.13,17 Some of these interventions had been implemented in Rhode Island EDs prior to passage of the discharge planning law.20,21 For example peer recovery specialists (also known as peer recovery coaches) and take-home naloxone became available in some Rhode Island EDs in 2014.21 By the time the Levels of Care were established, peer recovery specialists were available for consultation in all Rhode Island EDs.20,22 With technical support from RIDOH, from May 2017 to June 2018, all acute care hospitals in Rhode Island built on these previous efforts to fully implement protocols meeting policy standards. No funding was provided to sites for protocol development and implementation; however, RIDOH provided technical assistance with and feedback about protocols as they were being developed at each site. Upon completion of protocol development and local implementation, RIDOH and BHDDH representatives conducted site visits and final protocol reviews. Suggestions for modifications were made if needed. Hospitals were certified by RIDOH and BHDDH as either a Level 3, Level 2, or Level 1 facility following finalization of protocols considered to satisfy the Levels of Care requirements. From May 2017 to June 2018, six hospitals were certified as Level 1 facilities; four were certified as Level 3. One hospital was certified as a Level 2 facility but subsequently closed. Certification entailed a hospital site visit by representatives of RIDOH and BHDDH to review and evaluate whether their OUD and opioid overdose policies and protocols met Level of Care certification requirements.23
Barriers and facilitators to implementation of the Levels of Care have been previously described24, but policy impact on delivery of take-home naloxone, behavioral counseling, and referral to treatment at the time of an ED visit for opioid overdose is currently unknown. Using data from RIDOH’s 48-hour Opioid Overdose Reporting System3, this study evaluates the impact of Levels of Care policy implementation on offering and receipt of take-home naloxone, ED behavioral counseling, and referral to treatment following an ED visit for opioid overdose.
Methods
Study population
ED visits for suspected opioid overdose were analyzed from 2017-2018 using data from the 48-hour Opioid Overdose Reporting System, a web-based, state-mandated surveillance system in which every Rhode Island ED must report all ED visits for suspected opioid overdose within 48 hours of their ED presentation.3 Overdose incidents are reported through an online data collection form by hospital staff. Most reports are done retrospectively through electronic health record review by staff not directly involved in patient care. Information reported includes patient demographics, date of ED visit, hospital site, naloxone administration prior to hospital arrival, care offered and received during the ED visit, and referral to treatment. The data include hospital medical record numbers but no other personal identifiers, making repeat visits identifiable for patients that visited the same hospital or hospitals within the same health care organization, but not between hospitals in different hospital systems.
Data were examined for completeness and duplicate entries were removed. We included all ED overdose visits from January 1st, 2017 to March 31, 2017 (pre-implementation) and from the date of hospital Levels of Care certification (varies by site) to December 31st, 2018 (post-implementation) for patients 18 years of age and older who were discharged from the ED alive. The period of time from the Levels of Care policy guidance release on March 31, 2017 to hospital certification was considered an implementation period and removed from the analysis. We excluded visits from patients who were admitted to the hospital, left without being seen, or died. Data from ten EDs in Rhode Island were included: six certified as Level 1 facilities and four certified as Level 3. We excluded EDs associated with psychiatric hospitals and Women’s and Infant’s Hospital, whom do not generally provide acute general medical services, the Veterans Affairs Medical Center, which is not subject to state regulations, and the Level 2 hospital site which closed during the study period. This study was deemed exempt by the Rhode Island Department of Health Institutional Review Board.
Outcomes
We examined five outcomes reported in the 48-hour ED overdose reporting system: 1) provider offering of take-home naloxone, 2) patient receipt of take-home naloxone, 3) provider offering of behavioral counseling (from a peer recovery coach, counselor, social worker or psychiatrist), 4) patient receipt of behavioral counseling, and 5) referral to outpatient treatment or admission to inpatient treatment. We also conducted a sub-analysis to examine changes in referrals to an outpatient Center of Excellence or office based-MOUD provider pre-/post-policy implementation. There is currently no data demonstrating improved patient outcomes between Centers of Excellence and office based MOUD treatment and these were combined in this analysis. When these outcomes were not reported, we assumed they were not offered or not provided.
Analysis
We followed the Strengthening Reporting of Observational Studies in Epidemiology (STROBE) checklist to guide study reporting.25 The number and percent of overdoses within each demographic category (age, sex, race/ethnicity) were calculated overall and before and after hospital certification (Level 1 and Level 3 combined). We tested for significant differences in the distribution of outcome variables across categories before and after certification using a chi-squared test of independence. To examine changes in the percent of OD visits receiving services each week, we conducted an interrupted time series analysis. The interrupted time series model tests for an immediate change in the percent of visits receiving or offered services after hospitals were certified. The post-implementation period began for each hospital at the date of their certification as a Level 3 or Level 1 facility.
Many Level 1 sites had post-overdose treatment protocols in place prior to release of the Levels of Care policy guidance. Therefore, we hypothesized that the effect of certification might be greater for hospitals certified at Level 3 compared to hospitals certified at Level 1. We estimated the percentage (and 95% confidence interval using the sandwich estimator to adjust for clustering within hospitals) of visits receiving services before and after certification at all hospitals combined, and separately for hospitals certified as Level 1 and Level 3. We tested for a significant change overall and by certification level using a two-sample test of proportions adjusted for clustering within hospitals.
Next, we used a modified Poisson regression with a log link to estimate the unadjusted relative risk of each of the five outcomes at Level 1 vs. Level 3 hospitals after certification compared to before certification. These initial models included an interaction term to allow for a different effect of certification on the outcome at Level 1 and Level 3 hospitals.
Finally, to account for visit- and hospital-level variation, we constructed multivariable, multilevel models with the Poisson distribution and log link and robust variance clustered within hospitals using GEE estimation. The variables included in the multivariable models were selected from data available in the 48-hour reporting system as well as factors identified by the study team that may impact study outcomes, drawing from our collective experience as clinicians, researchers, and in opioid overdose policy, public health programming, and data surveillance. In Rhode Island, there has been observed seasonal changes in overdose frequency. The study team considered factors they thought may differ between hospitals which could impact services provision and hypothesized that hospital service delivery may be different at Level 1 and Level 3 sites. Final variable models included: patient sex, age (in 5 categories pre-specified by RIDOH), whether naloxone was administered prior to ED arrival, whether the patient had more than one ED visit for opioid overdose during the study period, season (categorized as winter, spring, summer, fall), and calendar time (months) as covariates. We did not include patient ethnicity or racial identity due to a high prevalence of missing data. All regression models were estimated in the subpopulation of suspected overdose visits with complete data on naloxone administration prior to ED arrival (missing in 67 or 3% of observations) during the study period.
In order to assess the sensitivity of our results to the assumption we made about missing data, we conducted two sensitivity analyses for offering and receiving naloxone or counseling. The first assumed that the outcome was offered/provided when it was missing, and the second excluded observations where the outcome was not reported. Referrals to treatment were only reported if they were provided (there was no “unknown” response in the reporting form).
Results
Characteristics of study subjects.
In 2017 and 2018, the 48-hour reporting system recorded a total of 3,110 suspected overdoses across all acute care EDs in Rhode Island. Study analysis included visits by people seen and treated before release of the Levels of Care policy (pre-implementation) and those treated after hospital Level of Care certification (post-implementation). Pre-implementation, 238 people were seen and treated for an opioid overdose in an ED. Post-implementation 1,299 people were treated for an opioid overdose. We summarize the characteristics of the 1,585 (Figure 1) opioid overdose patients over age 18 who were discharged from the ED alive after seeing a provider during the study period in Table 1.
Figure 1.
Analytic sample from Rhode Island Opioid Overdose Reporting System data, pre-Levels of Care (January–March 2017) and post-Levels of Care implementation (hospital certification date–December 2018).
Table 1.
Rhode Island adult emergency department patients discharged after an opioid overdose, overall and before and after Level of Care certification, 2017–2018.
| Total (n=1,585) n (%) | Pre-certification (n=245) n (%) | Post-certification (n=1,340) n (%) | ||
|---|---|---|---|---|
| Sex | ||||
| Male | 1098 (69.3) | 159 (64.9) | 939 (70.1) | |
| Female | 484 (30.5) | 85 (34.7) | 399 (29.8) | |
| Transgender* | <5 | <5 | <5 | |
| Age | ||||
| 18-24 | 232 (14.6) | 34 (13.9) | 198 (14.8) | |
| 25-34 | 673 (42.5) | 97 (39.6) | 576 (43.0) | |
| 35-44 | 345 (21.8) | 49 (20.0) | 296 (22.1) | |
| 45-54 | 212 (13.4) | 41 (16.7) | 171 (12.8) | |
| 55+ | 123 (7.8) | 24 (9.8) | 99 (7.4) | |
| Race | ||||
| White | 1144 (72.2) | 203 (82.9) | 941 (70.2) | |
| Black/ African American | 74 (4.7) | 13 (5.3) | 61 (4.6) | |
| Other | 81 (5.1) | 27 (11.0) | 54 (4.0) | |
| Unknown | 286 (18.0) | 2 (0.8) | 284 (21.2) | |
| Ethnicity | ||||
| Hispanic | 85 (5.4) | 24 (9.8) | 61 (4.6) | |
| Non-Hispanic | 945 (59.6) | 208 (84.9) | 737 (55.0) | |
| Unknown | 555 (35.0) | 13 (5.3) | 542 (40.5) | |
| Season | ||||
| Winter (December 21–March 20) | 460 (29.0) | 228 (93.1) | 232 (17.3) | |
| Spring (March 21–June 20) | 323 (20.4) | 17 (6.9) | 306 (22.8) | |
| Summer (June 21–September 20) | 278 (17.5) | 0 (0.0) | 278 (20.8) | |
| Fall (September 21–December 20) | 524 (33.1) | 0 (0.0) | 524 (39.1) | |
| Emergency Departments | ||||
| Level 1 | ||||
| Rhode Island Hospital | 602 (38.0) | 110 (44.9) | 492 (36.7) | |
| Kent | 351 (22.2) | 53 (21.6) | 298 (22.2) | |
| Miriam | 209 (13.2) | 17 (6.9) | 192 (14.3) | |
| Roger Williams | 141 (8.9) | 11 (4.5) | 130 (9.7) | |
| Fatima | 51 (3.2) | 13 (5.3) | 38 (2.8) | |
| Newport | 33 (2.1) | 8 (3.3) | 25 (1.9) | |
| Level 3 | ||||
| Landmark | 128 (8.1) | 16 (6.5) | 112 (8.4) | |
| Westerly | 31 (2.0) | 12 (4.9) | 19 (1.4) | |
| South County | 39 (2.5) | 5 (2.0) | 34 (2.5) | |
| Repeat visits at same hospital during study period | ||||
| 1 visit | 1128 (71.2) | 175 (71.4) | 953 (71.1) | |
| >1 visit | 457 (28.8) | 70 (28.6) | 387 (28.9) | |
| Naloxone administered prior to ED** arrival | ||||
| No | 292 (18.4) | 62 (25.3) | 230 (17.2) | |
| Yes | 1247 (78.7) | 176 (71.8) | 1071 (79.9) | |
| Unknown | 46 (2.9) | 7 (2.9) | 39 (2.9) | |
| Outcomes | OR (95% CI) | |||
| Patient offered naloxone | 3.61 (2.71 - 4.82) | |||
| No | 348 (22.0) | 108 (44.1) | 240 (17.9) | |
| Yes | 1237 (78.0) | 137 (55.9) | 1100 (82.1) | |
| Patient discharged with naloxone | 1.53 (1.16 - 2.02) | |||
| No | 804 (50.7) | 146 (59.6) | 658 (49.1) | |
| Yes | 781 (49.3) | 99 (40.4) | 682 (50.9) | |
| Patient offered counseling | 0.88 (0.59 - 1.30) | |||
| No | 235 (14.8) | 33 (13.5) | 202 (15.1) | |
| Yes | 1350 (85.2) | 212 (86.5) | 1138 (84.9) | |
| Patient received counseling | 0.66 (0.50 - 0.87) | |||
| No | 1044 (65.9) | 141 (57.6) | 903 (67.4) | |
| Yes | 541 (34.1) | 104 (42.5) | 437 (32.6) | |
| Referred to substance abuse treatment, recovery service, inpatient detox, Center of Excellence, or MOUD** | 0.72 (0.55 - 0.95) | |||
| No | 959 (60.5) | 132 (53.9) | 827 (61.7) | |
| Yes | 626 (39.5) | 113 (46.1) | 513 (38.3) | |
| Referred to Center of Excellence or MOUD | 8.60 (3.78 - 19.57) | |||
| No | 1341 (84.6) | 239 (97.6) | 1102 (82.2) | |
| Yes | 244 (15.4) | 6 (2.5) | 238 (17.8) | |
| Patient started MOUD in the ED** | 0.42 (0.11 - 1.65) | |||
| No | 1575 (99.4) | 242 (98.8) | 1333 (99.5) | |
| Yes | 10 (0.6) | 3 (1.2) | 7 (0.5) | |
Results <5 suppressed per Rhode Island Department of Health small numbers policy.
ED = Emergency Department; MOUD = medication for opioid use disorder
Main Results.
Interrupted time series analysis accounted for overall time trends in reported services provision (Figure 2). In the 3 months leading up to release of the Levels of Care policy guidance, there was a significant linear increase in the percentage of visits where the patient was offered take-home naloxone (Figure 2, Panel A). After implementation, the proportion of patients offered naloxone increased on average 13% (95% CI 5.6% to 20.4%). The rate of increase in naloxone receipt slowed and, on average, there was no significant change in the proportion of patients receiving take home naloxone (Figure 2, Panel A) after implementation. The proportion of patients offered behavioral counseling was high prior to release of the Levels of Care policy guidance and did not change after policy implementation (Figure 2, Panel B). However, in the three months before release of the Levels of Care policy guidance, the proportion of patients receiving behavioral counseling was declining by 3.9% each week (95% CI −6.5%, −1.3%). After implementation, this decline slowed and the proportion of patients receiving behavioral counseling stabilized. On average, 18.6% more patients received behavioral counseling after certification (95% CI 1.3%, 35.9%). Similarly, referral to treatment was declining by 4.5% each week (95% CI −7.4%, −1.6%) before the Levels of Care were announced. After certification, the decline ceased and on average 23.1% more patients received referral to treatment (95% CI 2.7%, 43.5%) (Figure 2, Panel C).
Figure 2. Interrupted time series of percentage of opioid overdose ED visits before and after levels of care certification in Rhode Island Opioid Overdose Reporting System, 2017–2018.
Week 0 is the first week the hospital was certified. Weeks < 0 are weeks prior to the announcement of the Levels of Care on March 31st, 2017. The lines represent the linear trend for the time period before or after certification. Sample includes 1,575 suspected opioid overdose ED visits where the patient was discharged after seeing a provider in a Rhode Island ED (sample described in table 1, further limited to visits occurring from January-March 2017 and those occurring for up to 64 weeks after hospital certification).
Figure notes: Behavioral counseling included behavioral counseling with a peer recovery coach, counselor, social worker, or psychiatrist. Treatment referrals included linkage to treatment by referral to outpatient services, admission to an inpatient program, referral to an outpatient Center of Excellence (COE) or medication for OUD (MOUD) provider.
In bivariable analysis, significant changes were noted when stratifying results by whether the overdose presented to a Level 3 or Level 1 hospital. Offering take-home naloxone increased significantly after certification at hospitals overall (56% vs 82%, Table 1, Figure 3), and at Level 1 facilities (61% vs 84%, Figure 3A).There was no statistically significant change in receipt of behavioral counseling after certification, with referral rates ranging from 20-30% over the entire study period (Figure 3, Panel D). Finally, there was a statistically significant increase in referrals to Center of Excellence or MOUD (2.5% to 17.8%, Table 1) at Level 1 hospitals (RR 7.73 [95% CI 3.22, 18.55] Table 2, Figure 3, Panel F).
Figure 3. Percentagea of visits offered and receiving services before and after Levels of Care hospital certification by Level of Certification, 2017-2018.
Behavioral counseling included behavioral counseling with a peer recovery coach, counselor, social worker, or psychiatrist. Treatment referrals included linkage to treatment by referral to outpatient services, admission to an inpatient program, referral to an outpatient Center of Excellence (COE), or medication for OUD (MOUD) provider.
a Percentage of visits where services were offered and received before after levels of care certification, total and by level of certification. Sample is 1,585 ED visits by patients discharged after seeing a provider in a Rhode Island ED for an opioid overdose from January to March, 2017 and the date of hospital certification through December 2018.
Table 2.
Multivariable results: estimated crude (RR) and adjusted (aRR) risk of being offered and receiving services associated with Levels of Care certification by level of certification (Level 3 vs Level 1) among 1,537 suspected opioid overdose ED visits by adult men and women seen by a provider and discharged from the ED alive from the Rhode Island opioid overdose reporting system, 2017–2018.
| Outcome | Level 3 | Level 1 | Interaction p-value* |
|---|---|---|---|
| Offered naloxone | |||
| RR (95% CI) | 3.48 (1.69-7.16) | 1.39 (1.24-1.55) | 0.014 |
| aRR (95% CI) | 3.13 (1.08-9.06) | 1.31 (1.06-1.61) | 0.103 |
| Received naloxone | |||
| RR (95% CI) | 2.38 (1.04-5.41) | 1.23 (1.04-1.46) | 0.125 |
| aRR (95% CI) | 2.03 (0.65-6.36) | 1.16 (0.88-1.54) | 0.347 |
| Offered counseling | |||
| RR (95% CI) | 1.35 (1.02-1.78) | 0.95 (0.90-1.00) | 0.015 |
| aRR (95% CI) | 1.24 (0.79-1.94) | 0.91 (0.68-1.22) | 0.154 |
| Received counseling | |||
| RR (95% CI) | 1.87 (0.81-4.29) | 0.70 (0.59-0.83) | 0.024 |
| aRR (95% CI) | 2.29 (0.88-5.98) | 0.81 (0.69-0.94) | 0.044 |
| Referred to treatment | |||
| RR (95% CI) | 1.83 (0.99-3.38) | 0.77 (0.65-0.90) | 0.007 |
| aRR (95% CI) | 1.48 (0.41-5.28) | 0.72 (0.47-1.10) | 0.278 |
| Center of Excellence or MOUD | |||
| RR (95% CI) | 3.66 (0.51-26.26) | 7.73 (3.22-18.55) | 0.496 |
| aRR (95% CI) | § | § |
RR estimated by multilevel models with GEE estimation using the Poisson distribution and log link and robust variance clustered within hospitals. aRR additionally controls for patient sex, age (in 5 categories), whether naloxone was administered prior to ED arrival, whether the patient had more than one ED visit for opioid overdose during the study period, season (winter, spring, summer, fall), and calendar time (months). Sample is 1,537 ED visits by men and women discharged after seeing a provider in a Rhode Island ED for an opioid overdose from January to March, 2017 and the date of hospital certification through December 2018 with non-missing data on naloxone prior to arrival.
Interaction p-value tests for a significant difference in the association between the outcome and Level of Care certification between Level 3 and Level 1 hospitals.
The small number of events in this category precluded an adjusted estimate in the multivariable model
In multivariable analysis, many of these relationships were attenuated and no longer statistically significant. After adjusting for seasonality, age, naloxone administered prior to ED arrival, repeat visits, linear time trends, and accounting for hospital-level variation with GEE estimation, there was a significant increase in the likelihood of being offered naloxone after certification at Level 1 (aRR 1.31 [95% CI 1.06, 1.61]) and Level 3 (aRR 3.13 [95% CI 1.08, 9.06]) hospitals (Table 2). Receipt of naloxone at Level 1 (RR 1.23 [95% CI 1.04, 1.46]) and Level 3 (RR 2.38 [95% CI 1.04, 5.41]) sites and offering of behavioral counseling at Level 3 sites (RR 1.35 [95% CI 1.02, 1.78]) were significant in the unadjusted model, but no longer significant in the fully adjusted model. (Figure 3, Panels C & D; Table 2). Finally, the fully adjusted models showed no significant change in referrals to treatment at either certification level.
Sensitivity analysis results were very similar to those presented here, despite making the opposite assumption about missing outcome data or dropping observations missing the outcome.
Limitations
There are several limitations to this policy evaluation. The data in the 48-hour opioid overdose reporting system were derived from hospital self-report drawn from retrospective review of electronic health records. While compliance with reporting was high, data was entered retrospectively by individuals with varying clinical background, ranging from administrative staff to nurses to ED physician directors. Therefore, data provided is only as valid as what was recorded in the electronic health record and results may be subject to reporting bias and error. These data also do not capture patients with opioid use disorder who have not overdosed, so it does not assess services delivered to other patients that may benefit from take-home naloxone, behavioral counseling, referral to treatment, and MOUD.
Since the overdose reporting system does not collect patient identifiers, we were unable to account for repeated visits at the person level between different hospital systems, which may result in incorrect estimation of the standard errors in the statistical analyses. We do not think this significantly impacts study results, as internal RIDOH assessments of repeat ED overdose visits have shown that approximately 70% of patients return to the same ED after an opioid overdose. Further, it is reasonable to consider each ED overdose encounter as an independent observation in terms of which services the patient was offered.
Results are also limited by high rates of missing race and ethnicity data and that we did not impute for any missing data. Race information was missing for 17.5% of the study sample and ethnicity was missing for 33.7% (Table 1). Furthermore, reporting worsened overtime, with higher amounts of missing data in the post-implementation period. We inquired with hospitals about these missing data, which they say is a result of this information not being collected at the initial patient encounter and therefore not available to be reported. RIDOH has subsequently engaged hospitals about improving recording and reporting of race and ethnicity data to enable identification of differences or changes in overdose and services provision by race and ethnicity. Due to the degree of missing data, we cannot fully account for the impact of race or ethnicity on services provision, both of which are important factors that could impact whether services were offered or received. Absence of patient identifiers also limits our ability to control for individual factors impacting services uptake. Important among these are concurrent use of other substances, including stimulants, co-occurring psychiatric disorders, social determinants of health (e.g., unstable housing or homelessness), and prior experiences in addiction treatment and with the medical system.
Differences in the number of patients treated at hospital sites may also impact study outcomes, as staff may be more experienced at delivering post-overdose care or one hospital may treat a greater proportion of patients in one study period. Given low counts at some hospital sites, we were unable to stratify by hospital to account for confounding by hospital site. We tried to minimize the effect of site-specific changes by analyzing data by certification type (Level 1 vs Level 3). Because of small sample sizes for some outcomes, some observed effects may have been null because of a type II error. Finally, improvements in ED services provision over time may have also been impacted by other, unrelated quality improvement initiatives and increased local and national attention about the opioid epidemic and not necessarily attributable to Levels of Care certification.
Discussion
The ED is a key point of entry into the health care system for patients with opioid use disorder and individuals presenting after an opioid overdose. As ED visits for opioid overdose have surged,26 they have become recognized as a logical and strategic site for interventions to improve OUD treatment and to reduce opioid overdose deaths. This evaluation of changes associated with implementation of Rhode Island’s Levels of Care policy demonstrates the greatest improvement in provision of take-home naloxone, behavioral counseling, and referral to treatment at Level 3 hospitals. Compared to Level 1 facilities, these were smaller community hospitals that did not have robust addiction medicine services or ED opioid overdose initiatives prior to policy implementation. Behavioral counseling receipt was declining prior to hospital certification but stabilized after policy implementation. We also observed a significant increase in reported offering of behavioral counseling and referral to treatment from Level 3 sites, and referral to a Center of Excellence or MOUD provider from all hospital types in the unadjusted analysis; however, estimates were largely not significant after adjusting for co-variates. While policy implementation may result in an increase of reported offering of services, services receipt is more complicated, and impacted by additional patient, provider, and institutional level factors. Low counts of Center of Excellence or MOUD referrals precluded conduction of an adjusted analysis for that outcome. It is important to note that after adjusting for co-variates and hospital clustering for other outcomes the estimated risk ratios do not substantially change, while the confidence intervals widen. Widening confidence intervals may be because added co-variates are poor predictors of services provision and, at Level 3 sites, due in part to a low number of visits.
Improvements noted at Level 3 sites in addition to or in the absence of improvements at Level 1 sites does not mean that Level 3 sites are superior or equivalent to Level 1 hospitals. This study examines changes within each certification grouping before and after policy implementation and does not directly compare Level 3 and Level 1 hospitals. All of the outcomes measured in this analysis (ED naloxone distribution, behavioral counseling, or treatment referral) are Level 3 certification requirements. This study does not examine initiation of buprenorphine after an opioid overdose, which is the key feature distinguishing Level 1 from Level 3 sites. Future investigations examining patient outcomes, specifically repeat overdose, treatment engagement, and death, are needed to determine whether there are further downstream benefits to certification.
After certification, more than 80% of people treated after an opioid overdose were offered naloxone and behavioral counseling. Time trends demonstrate an overall increase in naloxone distribution throughout the study period, but no significant increase at the time of policy adoption and certification. This analysis does not identify reasons why naloxone was offered but not provided. Reasons may include patients already having naloxone or perceived lack of risk of future overdose. With multiple community sources of naloxone, ED naloxone receipt does is not entirely reflective of naloxone access, although is one important source. Contemporaneous expansion of community naloxone distribution is a likely contributing factor to the slowed rate of ED naloxone receipt after policy implementation and may reflect relative community saturation of naloxone. Behavioral counseling and treatment referral overall did not increase over time; however, at the time of certification, bivariable and unadjusted multivariable analysis demonstrated an increase in the percentage of overdose visits where patients were referred to a Center of Excellence and/or MOUD. Some of this may be due to increased awareness and knowledge about the Centers of Excellence and MOUD gained through the implementation process. Observed absence of changes before and after certification at Level 1 sites may be related to pre-existing overdose care policies and high services provision prior to Level of Care certification, particularly at two hospital sites, which had been offering peer-recovery counseling and take-home naloxone to ED patients since 2014.21,27,28 Institutional, provider, patient, and community factors impacting service uptake need to be more fully evaluated to improve policy implementation and services provision.
The improvements noted at the Level 3 hospitals are significant, but modest. Overall, despite statewide policy implementation, the proportion of patients receiving take-home naloxone, behavioral counseling, and referral to treatment remains low. Reasons for this are not completely understood. Using a de-identified administrative dataset limits our ability to identify individual patient, provider, and hospital factors that may contribute to decreased uptake, such as patient and provider attitudes, patient-provider relationships, ED crowding, and prolonged ED wait times. It is also possible that how or what is currently offered is not what patients want and/or need. Withdrawal symptoms after overdose reversal with naloxone may also impact patient interest in engaging with ED providers. Treating these withdrawal symptoms, especially with buprenorphine (when appropriate and with patient consent), may result in improved uptake of services and should be evaluated.29
ED post-overdose care and referral to ongoing treatment of OUD needs to be systematically improved nationally and in Rhode Island. While there is significant room for improvement, Rhode Island did implement many other statewide programs to prevent overdose and increase access to evidence-based treatment during the study period. As a result, implementing a similar a policy similar to the Levels of Care in other states may have a larger impact than we observed. Steps to improve uptake of services and treatments outlined in Rhode Island’s Levels of Care may include enhanced education of ED clinicians and staff, addressing stigma amongst ED clinicians and staff, engagement of ED leadership and professional societies, and/or financial incentivization for improvements in quality of post-overdose care. We also need to increase our understanding of patient engagement in the ED including patient needs at time of an ED visit and perspectives on how to improve ED care.
There are several ongoing national efforts to improve ED-based overdose and OUD treatment. With supportive research evidence and support from regional advocates, over the last five years, attitudes and practice patterns have shifted considerably locally and nationally. The American College of Emergency Physicians has helped ED physician leaders develop and launch an ED-specific Drug Addiction Treatment Act of 2000 (DATA 2000) waiver training series (ED X-Waiver Training Corps)30, produce an ED buprenorphine initiation phone application (BUPE) on ACEP’s emPOC app,31 and launch a national opioid quality improvement initiative through their Emergency Quality Network.32 There is also increasing dedicated research funding focused on ED OUD treatment. Through the National Institute of Health’s HEAL (Helping to End Addiction Long-termSM) Initiative, the NIDA Clinical Trials Network is running a multi-site clinical trial focused on ED buprenorphine.33–35 We anticipate that by leveraging results from this trial along with utilizing existing tools and resources, EDs can improve the care they provide overdose patients and people with OUD.
Our evaluation of Rhode Island’s Levels of Care demonstrates that establishment of hospital and ED policies for treatment after opioid overdose can improve naloxone offering, behavioral counseling, and referral to medication for OUD for ED opioid overdose patients. However, a significant need for improvement remains. To further improve ED care of patients with OUD and/or opioid overdose will require a more comprehensive evaluation of barriers and facilitators to services utilization, development of enhanced implementation strategies, and evaluating policy impact on patient outcomes, specifically engagement in treatment, repeat overdose, and death.
Acknowledgements:
The authors would like to acknowledge the support and collaboration of the Rhode Island Department of Behavioral Health, Developmental Disabilities, and Hospitals, and our emergency department partners at Fatima, Kent, Landmark, Newport, Rhode Island, Roger Williams, South County, Miriam, and Westerly hospitals.
Conflicts of Interest:
This work was supported by a Centers for Disease Control and Prevention SURGE grant and a Centers for Disease Control and Prevention R01 CE003149. Dr. Samuels is supported by an Advance Clinical and Translational Research Mentored Research Award at Brown University, which is supported by a National Institute of General Medical Sciences IDeA-CTR grant (U54GM115677). Dr. Marshall is supported by a COBRE on Opioids and Overdose grant (5P20GM125507-02). The authors have no other conflicts of interest to disclose.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Meetings: Portions of the results from this study were presented at the American Public Health Association Annual Meeting and Expo on November 2, 2019 in Philadelphia, PA and at the American College of Emergency Physician Scientific Assembly on October 28, 2019 in Denver, CO.
References
- 1.RI Gen L § 23-17.26-3 Comprehensive Discharge Planning - The Alexander C. Perry and Brandon Goldner Act. In. S 2356. Providence, RI: State of Rhode Island General Assembly,; 2017. [Google Scholar]
- 2.Rhode Island Department of Health, Rhode Island Department of Behavioral Healthcare DD, and Hospitals. Levels of Care for Rhode Island Emergency Departments and Hospitals for Treating Overdose and Opioid Use Disorder. March 2017.
- 3.McCormick M, Koziol J, Sanchez K. Development and Use of a New Opioid Overdose Surveillance System, 2016. R I Med J (2013). 2017;100(4):37–39. [PubMed] [Google Scholar]
- 4.Rhode Island Governor’s Overdose Prevention and Intervention Task Force. Rhode Island’s Strategic Plan on Addiction and Overdose Four Strategies to Alter the Course of an Epidemic. November 4 2015.
- 5.Rhode Island Department of Behavioral Healthcare Developmental Disabilities and Hospitals. Centers of Excellence for Opioid Use Disorders Locations and Contact Information. http://www.bhddh.ri.gov/sections/opioid_use_disorders.php. Published 2019. Accessed Nov 16, 2019.
- 6.Rhode Island Department of Behavioral Healthcare Developmental Disabilities and Hospitals. Centers of Excellence Information, Application and Standards. http://www.bhddh.ri.gov/quick_links/excellence.php. Published 2019. Updated 2019. Accessed Nov 16, 2019.
- 7.Galea S, Worthington N, Piper T, Nandi V, Curtis M, Rosenthal D. Provision of naloxone to injection drug users as an overdose prevention strategy: early evidence from a pilot study in New York City. Addict Behav. 2006;91:907–912. [DOI] [PubMed] [Google Scholar]
- 8.Maxwell S, Bigg D, Stanczykiewicz K, Carlberg-Racich S. Prescribing naloxone to actively injecting heroin users: a program to reduce heroin overdose deaths. Journal of addictive diseases. 2006;25(3):89–96. [DOI] [PubMed] [Google Scholar]
- 9.Strang J, Manning V, Mayet S, et al. Overdose training and take-home naloxone for opiate users: prospective cohort study of impact on knowledge and attitudes and subsequent management of overdoses. Addiction. 2008;103(10):1648–1657. [DOI] [PubMed] [Google Scholar]
- 10.Seal KH, Thawley R, Gee L, et al. Naloxone distribution and cardiopulmonary resuscitation training for injection drug users to prevent heroin overdose death: a pilot intervention study. J Urban Health. 2005;82(2):303–311. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wagner KD, Valente TW, Casanova M, et al. Evaluation of an overdose prevention and response training programme for injection drug users in the Skid Row area of Los Angeles, CA. Int J Drug Policy. 2010;21(3):186–193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Walley AY, Xuan Z, Hackman HH, et al. Opioid overdose rates and implementation of overdose education and nasal naloxone distribution in Massachusetts: interrupted time series analysis. BMJ. 2013;346:f174. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Coffin PO, Sullivan SD. Cost-Effectiveness of Distributing Naloxone to Heroin Users for Lay Overdose Reversal. Ann Intern Med. 2013;158:1–9. [DOI] [PubMed] [Google Scholar]
- 14.Piper TM, Stancliff S, Rudenstine S, et al. Evaluation of a naloxone distribution and administration program in New York City. Subst Use Misuse. 2008;43(7):858–870. [DOI] [PubMed] [Google Scholar]
- 15.Doe-Simkins M, Walley AY, Epstein A, Moyer P. Saved by the nose: bystander-administered intranasal naloxone hydrochloride for opioid overdose. Am J Public Health. 2009;99(5):788–791. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.The BNI ART Institute. Project ASSERT: SBIRT in Emergency Care. The Boston University School of Public Health. https://www.bu.edu/bniart/sbirt-experience/sbirt-programs/sbirt-project-assert/. Accessed August 15, 2019.
- 17.D'Onofrio G, O'Connor PG, Pantalon MV, et al. Emergency department-initiated buprenorphine/naloxone treatment for opioid dependence: a randomized clinical trial. Jama. 2015;313(16):1636–1644. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Bernstein SL, D'Onofrio G. A promising approach for emergency departments to care for patients with substance use and behavioral disorders. Health Aff (Millwood). 2013;32(12):2122–2128. [DOI] [PubMed] [Google Scholar]
- 19.Sordo L, Barrio G, Bravo MJ, et al. Mortality risk during and after opioid substitution treatment: systematic review and meta-analysis of cohort studies. BMJ. 2017;357:j1550. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Forum AP. AnchorED Rhode Island. February 2017.
- 21.Samuels E. Emergency department naloxone distribution: a Rhode Island department of health, recovery community, and emergency department partnership to reduce opioid overdose deaths. RI Med J (2013). 2014;97(10):38–39. [PubMed] [Google Scholar]
- 22.Waye KM, Goyer J, Dettor D, et al. Implementing peer recovery services for overdose prevention in Rhode Island: An examination of two outreach-based approaches. Addict Behav. 2019;89:85–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Rhode Island Department of Behavioral Healthcare DD, and Hospitals,, Rhode Island Department of Health. Levels of Care for Rhode Island Emergency Departments and Hospitals for Treating Overdose and Opioid Use Disorder. March 2017.
- 24.Samuels EA, McDonald JV, McCormick M, Koziol J, Friedman C, Alexander-Scott N. Emergency Department and Hospital Care for Opioid Use Disorder: Implementation of Statewide Standards in Rhode Island, 2017-2018. Am J Public Health. 2019;109(2):263–266. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Med. 2007;4(10):e296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Vivolo-Kantor AM, Seth P, Gladden RM, et al. Vital Signs: Trends in Emergency Department Visits for Suspected Opioid Overdoses — United States, July 2016–September 2017. MMWR Morbidity and Mortality Weekly Report. 2018;67(9). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Samuels EA, Bernstein SL, Marshall BDL, Krieger M, Baird J, Mello MJ. Peer navigation and take-home naloxone for opioid overdose emergency department patients: Preliminary patient outcomes. Journal of substance abuse treatment. 2018;94:29–34. [DOI] [PubMed] [Google Scholar]
- 28.Samuels EA, Baird J, Yang ES, Mello M. Adoption and Utilization of an Emergency Department Naloxone Distribution and Peer Recovery Coach Consultation Program. Acad Emerg Med. 2018. [DOI] [PubMed] [Google Scholar]
- 29.D'Onofrio G, O'Connor PG, Pantalon MV, et al. Emergency Department-Initiated Buprenorphine/Naloxone Treatment for Opioid Dependence A Randomized Clinical Trial. Jama. 2015;313(16):1636–1644. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.American College of Emergency Physicians. ED X-Waiver Training Corps. https://www.acep.org/education/ed-x-waiver-training-corps/. Published 2020. Accessed November 11, 2020.
- 31.Ketcham E, Ryan R. BUPE: Buprenorphine use in the Emergency Department Tool. American College of Emergency Physicians. https://www.acep.org/patient-care/bupe/. Published 2018. Accessed Aug 1, 2019.
- 32.American College of Emergency Physicians. E-QUAL Network Opioid Initiative. https://www.acep.org/administration/quality/equal/emergency-quality-network-e-qual/e-qual-opioid-initiative/. Published 2019. Accessed June, 2019.
- 33.NIDA Announces New NIH HEAL Initiative Awards to Address the Opioid Crisis. https://www.drugabuse.gov/about-nida/noras-blog/2019/09/nida-announces-new-nih-heal-initiative-awards-to-address-opioid-crisis. Published September 26, 2019. Accessed October 1, 2019.
- 34.NIDA. Emergency Department-INitiated bupreNOrphine and VAlidaTIOn Network Trial (ED-INNOVATION) (NIH HEAL Initiative). https://www.drugabuse.gov/about-nida/organization/cctn/ctn/research-studies/emergency-department-initiated-buprenorphine-validation-network-trial-ed-innovation-nih-heal. Published 2019. Accessed October 1, 2019.
- 35.NIH HEAL Initiative. NIH HEAL Initiative Research Plan. NIH https://heal.nih.gov/about/research-plan. Published Sept 26, 2019. Accessed October 1, 2019.