Evaluation of an emergency department-based opioid overdose survivor intervention: Difference-in-difference analysis of electronic health record data to assess key outcomes

Abstract

Background:

In recent years, a number of emergency department (ED)-based interventions have been developed to provide supports and/or treatment linkage for people who use opioids. However, there is limited research supporting the effectiveness of the majority of these interventions. Project POINT is an ED-based intervention aimed at providing opioid overdose survivors with naloxone and recovery supports and connecting them to evidence-based medications for opioid use disorder (MOUD). An evaluation of POINT was conducted.

Methods:

A difference-in-difference analysis of electronic health record data was completed to understand the difference in outcomes for patients admitted to the ED when a POINT staff member was working versus times when they were not. The observation window was January 1, 2012 to July 6, 2019, which included N = 1462 unique individuals, of which 802 were in the POINT arm. Outcomes of focus include MOUD opioid prescriptions dispensed, active non-MOUD opioid prescriptions dispensed, naloxone access, and drug poisonings.

Results:

The POINT arm had a significant increase in MOUD prescriptions dispensed, non-MOUD prescriptions dispensed, and naloxone access (all p-values < 0.001). There was no significant effect related to subsequent drug poisoning-related hospital admissions.

Conclusions:

The results support the assertion that POINT is meeting its two primary goals related to increasing naloxone access and connecting patients to MOUD. Generalization of these results is limited; however, the evaluation contributes to a nascent area of research and can serve a foundation for future work.

1.1. Introduction

The third wave of the U.S. opioid epidemic that began around 2013 has been characterized by an increase in overdose deaths due to illicit opioids, primarily illicitly manufactured fentanyl (Ciccarone, 2019; Scholl et al., 2019). Emergency medicine plays a key role in the response to this crisis given it is often the only connection many illicit opioid and injection drug users have with the formal healthcare system (Masson et al., 2002; Mor et al., 1992; Samuels, 2019; Sohler et al., 2007). As a result, a number of emergency department (ED)-based interventions have been developed to provide harm reduction and treatment linkage supports for people who use opioids (Bagley et al., 2019; Chen et al., 2020). Despite this spread, there is a dearth of research supporting the effectiveness of such programs for improving patient outcomes to date (Watson et al., 2019a). This paper presents findings from an evaluation of Project Planned Outreach, Intervention, Naloxone, and Treatment (POINT), an ED-based intervention aimed at providing opioid overdose survivors with naloxone and connecting them to evidence-based medications for opioid use disorder (MOUD; e.g., methadone, buprenorphine, or naltrexone).

Prior work has demonstrated the ED can be an effective place for linking patients with opioid use disorder (OUD) to MOUD through an approach known as buprenorphine bridging (Berg et al., 2007; Busch et al., 2017; D’Onofrio et al., 2015; Hawk et al., 2015; Johns et al., 2018). By providing immediate buprenorphine induction within the ED, bridge services address patients’ fear and discomfort of opioid withdrawal that is a noted barrier to long-term treatment intake, which can take days or weeks to initiate (Pergolizzi et al., 2020). Other approaches for intervening with opioid users presenting in the ED include interventions that utilize harm reduction strategies (mainly naloxone training and distribution), supportive services, and/or health navigation (Chen et al., 2020; Watson et al., 2019a), which can all be used alone or in combination with buprenorphine induction. While ED-based linkage interventions without buprenorphine bridging are considered promising, more research is necessary to assess their effectiveness given their recent growth in numbers. Project POINT provided an excellent opportunity of assessing an intervention with several of these features.

Project POINT formally launched in February 2016 as a quality improvement initiative in the ED of Eskenazi Hospital located in Indianapolis, Indiana. Eskenazi is the safety net hospital for Marion County, Indiana, which, at the time POINT began, had a higher non-fatal opioid overdose rate than the national average (131 vs. 93 per 100,000; Vivolo-Kantor et al., 2018; Vivolo-Kantor et al., 2020). POINT had two overarching goals from its start: (1) provide patients revived from a non-fatal overdose with access to naloxone and (2) connect those same patients with long-term substance use disorder treatment—with the ideal being evidence-based MOUD. The program works with all patients following a harm reduction approach. As such, it provides patients with naloxone and supportive services regardless of their initial willingness to begin treatment. Additionally, POINT has a formal relationship with an MOUD provider who implemented walk-in hours for POINT patients that effectively reduced the treatment initiation wait time from more than a week to 0–4 days depending on their ED discharge time.

While all POINT services were originally delivered solely by ED and social work staff, foundation funding supported the addition of two full-time peer recovery coach positions in January of 2017. These positions were added based on the perceived utility of coaches in engaging the target population. This is because peer recovery coaches are paraprofessionals with lived experience in recovery who help clients navigate their personal recovery journeys (White and Evans, 2014). The coaches were added to POINT because it was believed they would have greater success connecting with patients and supporting them through the treatment linkage process due to their experiences as people in recovery from a substance use disorder. This assertion is supported somewhat by prior research demonstrating the acceptability of peer-facilitated services related to OUD and substance use disorder more broadly (see Bassuk et al., 2016; see Eddie et al., 2019; see Reif et al., 2014; Waye et al., 2019). However, the designs of these prior studies have lacked rigor necessary to determine intervention effectiveness. For instance, Samuels et al. (2018) conducted one of the few studies on peer recovery coaches working with OUD patients in an ED setting. While they found patients who saw a peer recovery coach had shorter treatment initiation times and lower mortality, it was underpowered and lacked the ability to make causal inferences related to the intervention.

We conducted an evaluation of the intervention with the primary goal of understanding whether patients who received POINT services have better outcomes than those who did not. Answering this question has implications beyond the POINT program due to the rapid scaling of similar interventions that is occurring in the absence of strong evidence to support them (McGuire et al., 2019; Watson et al., 2019a).

2.1. Material and methods

2.1.1. Intervention

Table 1 outlines 13 identified key components of the POINT intervention. This table also indicates which components were added with the introduction of the aforementioned recovery coach positions in January of 2017. The harm reduction philosophy underpins the intervention, and is largely reflected in its emphasis on patient choice (component 12) and naloxone distribution (component 13). In addition to these components, it is important to point out that staff do attempt to attend intake appointments with patients when able. POINT’s hours varied over the course of the observation window identified below (these variations were accounted for in the analysis described below); however, the intervention’s general hours of operation were between 10:00am-10:30pm weekdays (POINT occasionally covered Saturdays for shifts starting at 2pm that lasted various lengths, and the weekday hours were expanded to 8am-10:30pm in May 2019). During the evaluation’s observation window, 90% of all patients approached by POINT staff agreed to participate in services.

Table 1:

Components of the POINT intervention

Component	Description
Multi-method tracking/alert system	Staff monitor the ED tracking-board, receive automatic alerts from emergency services (ambulance runs), as well as receive direct referrals from ED staff to alert them to an eligible patients’ presence in the ED.
Office space in ED	POINT has a physical presence in the ED so staff are familiar with the team and are more likely to refer patients.
ED-based encounter	Staff meet with patients at the ED bedside.
Peer recovery coachesa	Clinical contact is primarily initiated by certified coach.
Lived experience of peer recovery coachesa	Coaches have lived personal experience in substance use disorder recovery.
Support for peer recovery coachesa	Includes introducing coaches to the ED environment and culture, proper clinical supervision, as well as encouraging coaches to attend to their own recovery and wellness.
Transportation assistance	POINT provides rides from the ED to appointments for treatment and related services.
Designated MOUD provider	Formalized referral relationship MOUD provider that includes staff attending POINT meetings and providing specialized intake procedures for POINT patients.
Walk-in clinic/reduced barriers to MOUD access	Provider allows POINT patients to present for enrollment during without an appointment during 2-hour blocks on Tuesdays, Wednesdays, and Thursdays.
Linkage with treatment providers	The team maintains information about a full range of community recovery services providers and their intake process.
Financial support for non-billable expenses	During the evaluation, grant support covered non-billable expenses.
Patient choice	Although MOUD is the preferred, evidence-based recovery option, staff utilize motivational interviewing to ensure patients’ choice dictates referrals.
Naloxone dispensation and education	Naloxone is provided to the patient before leaving the ED, and education is provided on its use.

^aComponent not present until coaches were added to the intervention in January 2017.

2.1.2. Data source

All data come from the Indiana Network for Patient Care (INPC) databases, from which the evaluators were provided a complete de-identified, patient-level dataset. The INPC includes electronic health record data from all 5 major hospital systems and prescription dispensation information from participating pharmacies in the county where Eskenazi is located. INPC prescribing data come from the third-party vendor Surescripts. Surescripts does not provide detail as to which specific pharmacies in the county report into the database; however, their publically available list of nationally participating pharmacies does capture all major retail chains within Marion County (Surescripts, n.d.). Prescribing data include all opiates and MOUDs dispensed from the participating pharmacies. Data reflecting patients from all available hospitals in Marion County were included in the analysis. Outcomes of focus include change in MOUD prescriptions (measured as the dispensation of a formulation of buprenorphine used for MOUD or injectable naltrexone), change in active non-MOUD opioid prescriptions (measured as dispensation of non-MOUD opioid), naloxone access (measured as naloxone dispensation), and drug poisonings (occurring after the initial recorded ED visit). MOUD treatment, continued non-MOUD opioid prescriptions, and naloxone dispensation were identified using the National Drug Code (NDC) and the NDC-active ingredient crosswalk of all prescription opioids (Centers for Disease Control and Prevention, 2018). Drug poisonings were identified using episode current procedural terminology codes.¹ The observation window for which data were provided was January 1, 2012-July 6, 2019 (this includes approximately 11 months prior to and 18 months after the addition of the recovery coach positions).

2.1.3. Analysis

We conducted a quasi-experimental difference-in-difference analysis of patient-level INPC data to understand the difference in outcomes for patients admitted to the ED when a POINT staff member was working in comparison to patients admitted when no POINT staff were present. The sample includes 1,462 individuals. To be included in the analysis, a patient had to be identified in the electronic health record as being admitted to the Eskenazi ED for an opioid overdose during the observation window defined above. Analysis defines the study arms based on the hours the patients presented to the ED: the POINT arm refers to all eligible patients admitted during POINT’s operating hours (n = 802), and the Control arm refers to all patients admitted to the ED during a time outside of the program’s hours of operation (n = 660). Our implementation of the difference-in-difference design did not directly compare patients admitted to the ED when POINT was available to eligible patients admitted to the ED when POINT was unavailable. This is because individuals in the treatment arm may differ from patients in the control arm in other ways besides participation in POINT. Instead, the analysis examines the differential change in outcomes of individuals in the POINT arm and the control arm before and after POINT launched. In doing so, the analysis simply relies on constant unobserved differences within each individual in these two groups over time (Angrist and Pischke, 2008). We would also expect a simple pre and post comparison (before and after POINT launched in February 2016) to provide a biased estimate. There are three reasons for this. First, outcomes might be impacted by different rates of exposure, as there were 241 more days included in the pre-POINT observation period (1,492 days total) than the post-POINT period (1,251 days total). Second, the sampling frame was retrospectively generated and is likely to be more densely populated in the post-POINT period due to improvements in overdose recording/tracking in the electronic health record that occurred over time. Third, the observation window coincides with the exponential rise of the ‘opioid epidemic’ and may necessarily be capturing that confounding effect. The difference-in-difference analysis is particularly suitable to control for such confounders, as they are exogenous to POINT participation.

The analysis seeks to understand the impact of the POINT from February 2016 through the end of the observation window (this includes approximately 11 months prior to and 18 months after the addition of the recovery coach positions). The level of analysis was the individual with each patient observed over two periods: before (t = 0) and after (t = 1) POINT was in effect (February 1, 2016). The final estimating sample consists of 1462 unique individuals, of which 802 individuals (55% of sample) were in the POINT arm. We test four hypotheses, that in comparison to control, the POINT arm will: (1) have higher rates of MOUD linkage; (2) lower rates of non-MOUD opioid prescribing; (3) higher rates of naloxone dispensation; and (4) lower rates of subsequent poisonings.

As an evaluation utilizing de-identified administrative data conducted at Eskanazi Health’s request, this project was determined to fall under the category of quality improvement and was not required to undergo review by Indiana University’s Institutional Review Board.

3.1. Results

Table 2 compares the demographic characteristics (e.g., age, gender, and race) of the two arms. As shown, the POINT and control patient demographics are statistically indistinguishable based on the demographic characteristics available in the INPC data.

Table 2:

Descriptive comparison of POINT and control patient characteristics.

	POINT Patients (N = 802)	Control Patients (N = 660)
Age (mean)	37.30 (0.29)	37.66 (0.33)
Male (proportion)	0.56 (0.01)	0.56 (0.01)
Race (proportion)
White	0.89 (0.01)	0.90 (0.01)
Black	0.04 (0.01)	0.04 (0.01)
Hispanic	0.004 (0.002)	0.002 (0.001)
Other	0.05 (0.005)	0.03 (0.005)
Unknown	0.02 (0.003)	0.02 (.004)

NOTES: Calculation based on Indiana Network for Patient Care (INPC) data from January 1, 2012- July 6, 2019. Figures shown are mean age and proportions of male and racial composition of the patients in the treated POINT and control arms of the estimating sample, Standard deviations in parentheses.

Table 3 presents summary statistics for the four outcomes considered for the two arms in the pre- and post-POINT periods. From this table, we note drug poisonings are centered close to 1 on average, and, despite a wide range, values greater than 1 are rarely observed. As such, the outcomes are recoded as binary, and the impact of POINT participation on patient outcomes is captured using an individual fixed-effect logistic regression. The remaining three outcomes (e.g., MOUD prescriptions, non-MOUD opioid prescriptions, and naloxone access), are not suitably characterized as binary variables given their wider ranges (see range in square brackets in Table 3). As such, changes in these patient outcomes in response to potential POINT participation is modelled using an individual fixed effect negative binomial regression.

Table 3:

Descriptive comparison of POINT and non-POINT patient outcomes in the period before and after Project POINT was in effect.

	Treatment (N = 802)		Control (N = 660)
	(1) Pre- POINT	(2) Post- POINT	(3) Pre- POINT	(4) Post- POINT
MOUD prescription	0.08 (0.01) [0,35]	0.21 (0.01) [0,57]	0.10 (0.01) [0,42]	0.17 (0.02) [0,237]
Non-MOUD opioid prescription	0.37 (0.02) [0,47]	0.31 (0.02) [0,53]	0.43 (0.02) [0,74]	0.37 (0.02) [0,51]
Naloxone access	0.01 (0.00) [0,3]	0.07 (0.01) [0,19]	0.02 (0.01) [0,8]	0.06 (0.01) [0,29]
Drug poisonings	0.88 (0.04) [0,7]	2.90 (0.10) [0,25]	0.83 (0.04) [0,10]	1.28 (0.05) [0,12]

NOTES: Calculation based on Indiana Network for Patient Care (INPC) data from January 1, 2012- July 6, 2019. Figures shown are means of the four outcomes considered, presented separately for the treated (POINT) and control arms, and during the pre- and post- POINT periods. Stand errors are in parentheses, variable ranges are in square brackets.

As discussed above, the identifying assumption underlying the difference-in-difference analysis is that before POINT was implemented, the four outcomes in the POINT and control arms— MOUD linkage, non-MOUD opioid prescribing, naloxone access and drug poisonings—on average, followed the same time trend. Had it not been for POINT, the outcomes in the treated and controls arms would have continued as parallel. We test for common trends in the outcomes between POINT and controls arms by estimating an event-study. The event-study measures differences in outcomes between the treated (POINT) and the control arm in each year of the observation window, with the year before the implementation of POINT (i.e., 2015) as the origin. Estimates with 95 percent confidence intervals are graphically presented for each of the four outcomes in Figure 1. Equality in outcomes in POINT and controls arm patients prior to POINT (2012–2014) suggest that, generally, there were no systematic differences in the studied outcomes between POINT and control arm patients at baseline; the identifying assumption of a difference-in-difference estimation holds.

Figure 1. Event-study estimates of the effect of project POINT on patient outcomes.

Notes: Calculation based on INPC data January 1, 2012- July 6, 2019; unit of observation is the individual patient; all event-study regression estimates control for individual fixed effects, fixed effect for the period in which POINT was in effect (after February 2016), demographic controls and patient-level exposure to treatment, proxied by duration of the pre- and post- period for each patient. N=5,049 for subfigures (1), (2) and (4); N=691 for subfigure (3); Estimates presented are marginal effects, standard errors are presented in parenthesis; (1), (2), and (3) modeled using individual fixed-effect negative binomial regression; (4) modeled using a fixed-effect logistic regression.

Table 4 presents the average effect of POINT intervention on patients in the treatment arm from the difference-in-difference estimation. We examine whether patients admitted to the ED during a POINT shift saw improvements in the outcomes of focus. As shown, the POINT arm had a significantly greater increase in MOUD prescriptions, non-MOUD prescriptions, and naloxone access (all p-values < 0.001). However, there was no significant effect related to drug poisoning-related hospital admissions. (Table A included as supplementary material displays similar results at shorter six- and one-month follow-up periods.)

Table 4.

Difference-in-difference estimates of the effect of project POINT on patient outcomes.

	MOUD linkagea (N = 1462)	Non-MOUD opioid prescriptionsa (N = 1462)	Naloxone accessa (N = 1462)	Drug poisoningb (N = 1462)
Marginal effect of POINT	1.53***	1.00***	2.35***	0.14
	[0.21]	[0.11]	[0.51]	[0.78]
Incident-rate-ratios	5.29***	2.65***	10.38***
p-value	<0.001	<0.001	<0.001	0.856

Notes: Calculation based on INPC data January 1, 2012- July 6, 2019; unit of observation is the individual patient; all event-study regression estimates control for individual fixed effects, fixed effect for the period in which POINT was in effect (after February 2016), demographic controls and patient-level exposure to treatment, proxied by duration of the pre- and post- period for each patient. Estimates presented are marginal effects, standard errors are presented in parenthesis;

^amodeled using individual fixed-effect negative binomial regression;

^bmodeled using a fixed-effect logistic regression.

4.1. Discussion

The results support the assertion that POINT is meeting its two primary goals related to increasing naloxone access and connecting patients to MOUD, as the POINT arm saw greater positive changes in these outcomes during the observation period than the Control arm. This is a welcome outcome considering rapid scaling of similar programs that has occurred over the past 5 years despite the lack of a strong evidence base (McGuire et al., 2019; Watson et al., 2019a). The results also provide some support for the wide range of ED-based peer services for OUD that have been implemented across the United States in recent years (McGuire et al., 2019; Powell et al., 2019; Samuels et al., 2019, 2018; Wagner et al., 2019; Waye et al., 2019). The bulk of POINT services were delivered by peer recovery coaches. While this does not provide clear indication of the effectiveness of ED-based peer supports, it does strengthen the relatively weak existing evidence-base for peer services until randomized control trials can be completed (Bassuk et al., 2016; Eddie et al., 2019; Goedel et al., 2019; Reif et al., 2014). One such study currently being led by members or our team is a cluster randomized pragmatic trial of a pure peer-delivered POINT program within a different hospital system (Watson et al., 2019b).

The POINT evaluation also demonstrates that an ED-based program focused on the provision of recovery supports can improve MOUD linkage without buprenorphine bridge services. A key element of this success is likely the MOUD clinic’s walk-in hours that are reserved for POINT patients reduced recognized hurdles to treatment initiation (Deering et al., 2011; Stöver, 2011), as such low-barrier approaches to MOUD benefit both treatment initiation and retention (Winograd et al., 2019). However, the relationship leading to POINT patients’ enhanced treatment access is unique and likely difficult to replicate in other hospital settings. Additionally, it is important to note that Eskenazi Health also has strong relationships with providers elsewhere in the community that further improve POINT’s ability to link patients with MOUD and other supportive services. Therefore, full replication of POINT’s components within another ED might not be possible in hospitals lacking such strong and pre-established facilitators, which would make it more difficult to obtain such positive outcomes (see Scanlon et al., 1997).

Though the evaluation results are largely positive, it is likely POINT could be improved with the addition of buprenorphine bridge services. In addition to its strong evidence-base for improving OUD outcomes (D’Onofrio et al., 2015), bridge services would ensure those patients presenting to the ED on days where there are no clinic walk-in hours are able to begin treatment immediately. Recognizing this, POINT did implement bridge services and expanded its scope to include all patients with OUD (not just overdose survivors) after the evaluation was completed in 2019. While the combination bridge and recovery support services in the ED is not novel, noted barriers to bridge services have prevented their spread to many hospitals (D’Onofrio et al., 2019; Im et al., 2020; Lowenstein et al., 2019). As such, there is a need for research aimed at understanding the barriers and facilitators affecting implementation of different types of ED-based interventions targeting people with OUD and the comparative effectiveness of different intervention types, including various combinations of evidence-based and promising components (e.g., buprenorphine induction, naloxone education, recovery supports, recovery coaching). Such research can help establish which interventions or intervention components lead to the strongest outcomes for the least cost within different ED contexts.

Despite the POINT arm’s positive results related to MOUD linkage and naloxone dispensation, there was no significant difference related to drug poisonings in comparison to the Control arm. This is likely because we were underpowered to detect a change in this outcome considering opioid poisonings resulting in a hospital admission are not as frequent among this population as is commonly believed. Indeed, previous research looking at 656 overdose decedents in Marion County (i.e., the setting for the POINT evaluation) identified an average of only 2.3 emergency medical services events per a decedent in the year prior to overdose death, with only 18% of those events being related to overdose (Ray et al., 2020). In addition, our reliance on health record data means we only captured overdose outcomes for hospital patients, thus missing overdoses victims who were not taken to the ED, such as those who died in the community. Furthermore, the increased proliferation of naloxone within the community that has occurred in response to the opioid epidemic has likely increased the number of lay responder-reversed overdoses that do not result in hospitalization (Ambrose et al., 2016; Watson et al., 2018), and these events are almost impossible to track. Recognizing these issues, the POINT pragmatic trial currently underway is collecting state vital records data to capture overdoses occurring within the community, which will also make it possible to measure all-cause mortality (Watson et al., 2019b).

Regarding non-MOUD opioid prescriptions, the POINT arm had a greater increase of these medications prescribed to them. This was the opposite of what we had expected, given higher treatment uptake should ideally result in reduced use of non-MOUD opioids. While this might seem inconsequential given the higher treatment uptake identified, it was a measure of particular concern for stakeholders guiding the evaluation. This is because the harm reduction philosophy underpinning the POINT intervention considers reduced use (and harmful consequences associated with it) to be a successful outcome despite treatment uptake (Marlatt, 1996).

The primary limitation of this study is its reliance on electronic health record data. In addition to preventing us from capturing overdoses that did not result in a hospital admission, this administrative dataset did not provide a complete picture of opioid dispensations given that it did not completely capture every pharmacy in the community. Furthermore, when used as an MOUD (as opposed to a pain killer), methadone is dispensed through opioid treatment programs rather than pharmacies; therefore, our dataset does not capture any methadone treatment patients might have received. Likewise, a considerable amount of naltrexone in Marion County, IN is administered in community settings and are therefore not recorded in pharmacy systems. Use of data from Indiana’s prescription drug monitoring system and methadone treatment registry would have provided a more compressive capture of opioids dispensed. However, this evaluation was only a first step in understanding POINT’s effectiveness, and we are collecting data from these other systems, as well as data that will provide us a better understanding of patient’s baseline characteristics, as part of our current pragmatic trial (Watson et al., 2019b). Regarding the study’s strengths, we have a larger sample size than has typically been seen (e.g., < 500) in research on such interventions (Chen et al., 2020). Second, under the identification assumptions of a difference-in-difference design, we are able to attribute the change in observed outcomes to the POINT intervention. This improved ability to attribute causality to similar ED-based interventions is lacking in prior research (Chen et al., 2020; Lechner, 2011; Wing et al., 2018). Second,

5.1. Conclusion

During the evaluation’s observation window, overdose survivors admitted to the ED when POINT staff were present were significantly more likely to receive naloxone and be connected to MOUD than those who were not, which demonstrates POINT is meeting its two primary goals. While our design limits generalization of the results, there is a need for research on such interventions given their proliferation in recent years (Watson et al., 2019a). Our evaluation contributes to this nascent evidence base and can serve as a foundation for future work in this area.

Highlights.

• ED-based opioid use disorder treatment linkage interventions are scaling.

• There is limited evidence for linkage interventions without buprenorphine induction.

• An evaluation of such an intervention, Project POINT, was conducted.

• The POINT arm had significantly better treatment linkage and naloxone dispensation.

• There was no improvement in subsequent poisoning-based hospital admissions.

Abbreviations:

ED

emergency department

MOUD

Medication for opioid use disorder

NDC

National Drug Code

OUD

opioid use disorder

POINT

Project Planned Outreach, Intervention, Naloxone, and Treatment