Abstract
Background
Treatment engagement is a well-established performance measure for the treatment of substance use disorders. This study examined whether outpatient treatment engagement is associated with a reduced likelihood of subsequent detoxification admissions.
Methods
This study used administrative data on treatment services received by clients in specialty treatment facilities licensed in Massachusetts. The sample consisted of 11,591 adult clients who began an outpatient treatment episode in 2006. Treatment engagement was defined as receipt of at least one treatment service within 14 days of beginning a new outpatient treatment episode and receipt of at least two additional treatment services in the next 30 days. The outcome was a subsequent detoxification admission. Multilevel survival models examined the relationship between engagement and outcomes, with time to detoxification admission as the dependent variable censored at 365 days.
Results
Only 35% of clients met the outpatient engagement criteria and 15% of clients had a detoxification admission within a year after beginning their outpatient treatment episode. Controlling for client demographics, insurance type, and substance use severity, clients who met the engagement criteria had a lower hazard of having a detoxification admission during the year following the index outpatient visit than those who did not engage (Hazard Ratio = 0.87, p < .01).
Conclusions
Treatment engagement is a useful measure for monitoring quality of care. The findings from this study could help inform providers and policy makers on ways to target care and reduce the likelihood of more intensive services.
INTRODUCTION
Treatment engagement, a performance measure for the treatment of substance use disorders (SUDs), provides an assessment of whether individuals receive at least a minimum amount of services during the first weeks of treatment. This measure has been adopted by the National Committee on Quality Assurance and is endorsed by the National Quality Forum as a performance measure for health plans.1,2 Additionally, engagement is included in the Centers for Medicare & Medicaid Services incentive program for the meaningful use of electronic health records and is part of the Core Set of Adult Health Care Quality Measures for Medicaid. The suitability of treatment engagement as a performance measure in outpatient settings of publicly funded specialty treatment has been evaluated.3,4 For treatment episodes beginning with outpatient services, treatment initiation is defined as an individual receiving a treatment visit within two weeks of the initial visit and treatment engagement is defined as an individual receiving two or more additional treatment services within 30 days of the initiation visit. Engagement in outpatient treatment has been associated with improved outcomes, including decreased criminal justice involvement and improved employment outcomes among adult outpatient clients,5,6 improved alcohol and legal outcomes among adult patients in the Veterans Administration,7 and lower substance use among adolescents.8
The purpose of this study was to expand knowledge concerning the association between engagement and substance use. We examined whether treatment engagement predicts a reduction of a subsequent more intensive SUD service, detoxification. Admission to detoxification after outpatient treatment generally is considered to be a negative event because it indicates that an individual has continued their use of substances or has relapsed and requires intervention to minimize the physical harm due to withdrawal symptoms. This study sought to answer the following research question: Is outpatient treatment engagement associated with a reduction in the likelihood of detoxification admissions during the next year?
Given the high rates of relapse and that addiction is considered a chronic disease,9 some in the treatment field view that receiving detoxification after outpatient treatment may be an appropriate step in someone’s path to recovery as it may help a client who has relapsed prepare for further treatment. There is a general agreement, however, that detoxification alone is not enough to promote recovery, but in all instances individuals should be linked with treatment services soon after being discharged from detoxification.10 Thus, as a complementary analysis, we also explored a different outcome: detoxification admissions that are not followed by treatment.
METHODS
Data Sources
We used administrative data from clients who received publicly-funded outpatient substance abuse treatment services in specialty facilities in Massachusetts in Fiscal Year (FY) 2006 (July 1, 2005 – June 30, 2006) from the Bureau of Substance Abuse Services (BSAS) of the Massachusetts Department of Public Health. Data include client demographics, social characteristics, treatment referral source, prior mental health treatment, and substance use history based on the clients’ self-report. Encounter data, type and dates of treatment services, were also used. Treatment data for these clients from FY 2005 and FY 2007 were employed to determine whether clients received treatment services prior to the beginning of their outpatient treatment episode, and to follow clients for up to a year after they began their treatment episode. All study protocols were reviewed and approved by the Institutional Review Board at the corresponding author’s institution.
Analytic Sample
The analytic sample was made up of adult clients who began a new outpatient treatment episode in FY 2006. A new outpatient treatment episode is defined by receipt of outpatient treatment (the index visit), after a period of at least 60 days during which no treatment services were received. Clients receiving intensive outpatient services (N = 2,430) were not included based on recommendations by the Washington Circle Group, a multidisciplinary group that was focused on developing and disseminating performance measures for SUD services, that measures of engagement be computed separately by treatment modality.4 Additionally, because clients in intensive outpatient treatment receive a minimum of 9 hours of SUD services per week,11 the engagement criteria in IOP requires a very low threshold and is unlikely to have the same meaning or predict the same impact as engagement in regular outpatient treatment.
In FY 2006, 12,803 adult clients had at least one new outpatient treatment episode. If a client had more than one treatment episode during the course of the year, only the first episode was used in the analysis. Clients were excluded if their intake information was collected more than 30 days before or after the index date, making their time-varying (e.g., recent substance use) variables inaccurate (839 clients), had been discharged from outpatient treatment within 45 days and their reason for discharge was death or incarceration (71 clients). We also excluded from the sample all clients (N= 222) who had a detoxification admission within 45 days of the index to avoid “immortal time bias”12,13 (see analysis section below). Finally, we excluded clients with missing data in any of the variables of interest (N=79), or were the only client with a new treatment episode in 2006 at their facility after other exclusions (N=1). The final analytic sample was made up of 11,591 clients in 59 treatment facilities.
Variables
Dependent Variables
Our outcomes are defined as time to an event. There are two events of interest: 1) any detoxification admission; and 2) a detoxification admission not followed by treatment. In Massachusetts, detoxification programs are medically monitored programs and serve adult individuals who are at risk of acute withdrawal from alcohol and other drugs. These programs provide 24-hour nursing care to monitor and manage an individual's withdrawal symptoms.
In our complementary analysis, for our second outcome, we defined subsequent treatment as having received any treatment service (e.g., outpatient or residential) within 14 days after being discharged from detoxification. Fourteen days was used as the observation window as it has been endorsed by several public sector state agencies as an appropriate window for continuity of care, and 14-day continuity of care after detoxification has been used in prior research showing a reduction in readmissions.4,14 Time to event was measured as the number of days between time 0 (see definition in Analysis section below) and the client’s first event. We censored the client’s time-to-event at 365 days when no detoxification admission was found before that time. Figure 1 is a graphical representation of our outcomes and these observation periods. The observation period for each outcome begins on Day 45 (Time 0) for all clients, regardless of whether or not they engaged in treatment, and ends 365 days later (on day 410 in Figure 1). In this example, the client had a detoxification admission on Day 95. To determine whether the client had a detoxification admission followed by treatment or not, we examine the 14 days after the client is discharged (here on day 100) from detoxification.
FIGURE 1.
Treatment Engagement, Outcomes, and Observation Periods: An Example
Treatment Engagement
The public sector measure specifications for treatment engagement in specialty settings were used to determine whether a client met the performance measure criteria.4 Treatment initiation is defined as the client returning for an additional substance abuse treatment service within 14-days of the outpatient index visit indicating a new treatment episode. Following initiation, treatment engagement captures whether the client received at least two additional treatment services within 30 days of initiation. Detoxification services or crisis care do not count as an additional treatment service for either measure. In the example in Figure 1, a client who began treatment on Day 0 had an outpatient treatment 8 days later. That visit is the Initiation visit as it is within 14 days after beginning the treatment episode. This client then had two more visits on Day 13 and Days 22, meeting the engagement criteria.
Client Covariates
The following covariates were included: gender, age, race/ethnicity, marital status, prior mental health treatment, source of referral, severity of substance use, receipt of treatment in the prior year, and receipt of detoxification services in the prior year. With the exception of prior treatment and admissions to detoxification in the prior year, all the client-level covariates were obtained or created from the client admission record. Measures of substance use severity were created for alcohol, marijuana, heroin and cocaine, incorporating age of first use, last regular use, and frequency of last use for each substance.15,16 These measures range from 0 to 1, with a higher number representing higher severity. For this study, we created terciles of each measure, with the exception of heroin severity where there was a natural cutoff between the lower 70% not reporting any heroin use and the remaining 30% reporting past or current use. Using encounter data, two dichotomous variables were used to indicate whether the client had received any residential or outpatient treatment services in the 12 months prior to their index visit. We also created indicator variables for whether clients had one, two or more, or no detoxification admissions in that time frame.
Facility engagement rate
Using aggregate client data, we calculated the facility engagement rate, or the proportion of outpatient adult clients admitted in that facility who met the engagement criteria. This variable was included as a covariate because of the importance of considering facility performance rates17 in the relationship between performance measures and outcomes. Proportions were calculated prior to client exclusions.
Analysis
Survival analysis was employed to test the association between engagement and each dependent variable. Multilevel Cox Proportional Hazards Models were used to test the relationship between the survival time or hazard function (the number of days between the index visit and a detoxification admission) and the main independent variable (engagement), while controlling for covariates. Specifically, we used a shared frailty model to control for clustering within facility. The traditional Cox model assumes independence of survival times for all clients and ignores the potential correlation between individuals within the same cluster often leading to incorrect conclusions. In treating facilities as random effects, the shared frailty model accounts for unobserved homogeneity among clients from the same facility.
A key decision in survival analysis is the selection of the “time 0”, or the origin. The origin for this study was selected as 45 days after the index to address “immortal time bias”. Immortal time refers to the time during which “death” or outcome (a detoxification admission in our study) cannot occur for certain clients under study. Using the immortal times of these clients in an analysis may bias the apparent effect of “treatment” (engagement). To avoid immortal time bias and make a time-to-event analysis fair, the treatment and comparison groups, (i.e., our engaged and non-engaged groups), need to be balanced with respect to their time-to-event distributions in the absence of treatment. One way to do this is to start time 0 at a fixed time after index (for example, at 45 days in our study), when the treatment status of all clients is determined. Clients who had the outcome, a detoxification admission, during that window of time (N = 222) were excluded from the analyses to address the possibility of bias. In other words, clients who had a detoxification admission within 45 days would have been less likely to have engaged, thus biasing the results towards finding the predicted negative association between engagement and a detoxification admission.
In cases where a client did not have the event, their data were censored at 365 days with a few exceptions. If a client had a residential treatment stay during the year after the origin, the client’s likelihood of using substances and having a detoxification admission is substantially lower during their residential stay. Thus, for those clients the days to a detoxification admission was reduced by the length of stay (LOS) in residential treatment (N = 58). However, if the client had a residential stay and no detoxification admission during the 365 days after the origin (N = 51), a client’s period of observation was extended by the residential LOS.
RESULTS
Client Characteristics at Treatment Admission and Treatment Engagement Rates
A summary of the characteristics of the sample is in Table 1. About a third of the clients were women and the two largest sources of referral to treatment were self/family and the criminal justice system (approximately 30% each). About 8% of clients had received outpatient treatment services in the year prior to beginning their index outpatient treatment episode. Most clients (80%) had not had detoxification services in the prior year; although some clients (9%) had had two or more detoxification admissions in that time.
Table 1.
Client Characteristics at Treatment Admission and Engagement Rates (N= 11,591)
| Client Characteristic | Sample Characteristic | Engagement Rate of Subgroup |
|
|---|---|---|---|
| N | (%) | % | |
| Overall | 11,591 | 34.5 | |
| Client demographics | |||
| Gender | |||
| Female | 3,740 | (32.3) | 37.8a |
| Male | 7,851 | (67.7) | 33.0a |
| Age | |||
| 18–25 | 3,269 | (28.2) | 32.4a,b |
| 26–35 | 3,033 | (26.2) | 33.1c,d |
| 36–44 | 2,963 | (25.6) | 36.8a,c |
| 45+ | 2,326 | (20.1) | 36.5b,d |
| Race/Ethnicity | |||
| White | 8,556 | (73.8) | 34.3a,b |
| Latino | 1,433 | (12.4) | 31.4a,c |
| Black | 1,196 | (10.3) | 40.2b,c,d |
| Other race/ethnicity | 406 | (3.5) | 33.3d |
| Education | |||
| Less than H.S. | 3,169 | (27.3) | 33.2a |
| H.S. Grad | 5,772 | (49.8) | 34.5 |
| More than H.S. | 2,650 | (22.9) | 36.1a |
| Marital status | |||
| Married | 1,653 | (14.3) | 35.2 |
| Unmarried | 9,938 | (85.7) | 34.4 |
| Housing Status | |||
| Homeless | 1,006 | (8.7) | 39.8a |
| Not Homeless | 10,585 | (91.3) | 34.0a |
| Employment Status | |||
| Employed | 4,329 | (37.4) | 32.3a |
| Not Employed | 7,262 | (62.7) | 35.8a |
| Insurance | |||
| None | 4,853 | (41.9) | 35.0 |
| Public | 4,254 | (36.7) | 34.8 |
| Private/Other | 2,484 | (21.4) | 33.1 |
| Referral Source | |||
| Self/family | 3,450 | (29.8) | 33.5a,b |
| Criminal Justice | 3,810 | (32.9) | 32.2c |
| Health Professional | 1,443 | (12.5) | 33.2d |
| Substance Abuse Tx | 1,839 | (15.9) | 39.8a,c,d |
| Other | 1,049 | (9.1) | 38.9b,c |
| Mental Health Treatment History | |||
| Prior mental health treatment | 6,418 | (55.4) | 34.5 |
| No prior mental health treatment | 5,173 | (44.6) | 34.5 |
| Substance Abuse Treatment History | |||
| Residential treatment services in prior year | 158 | (1.4) | 43.7a |
| No residential treatment services in prior year | 11,433 | (98.6) | 34.4a |
| Outpatient treatment services in prior year | 950 | (8.2) | 40.4a |
| No outpatient treatment services in prior year | 10,641 | (91.8) | 34.0a |
| Detox Admissions in Prior year | |||
| No Detox services in prior year | 9,313 | (80.4) | 33.8a |
| 1 Detox admission in prior year | 1,295 | (11.2) | 38.0a |
| 2+ Detox admissions in prior year | 983 | (8.5) | 36.6 |
| Substance Use Severity | |||
| Alcohol severity | |||
| Lowest tercile | 3,625 | (31.3) | 33.1a |
| Middle tercile | 4,179 | (36.1) | 34.8 |
| Highest tercile | 3,787 | (32.7) | 35.6a |
| Marijuana severity | |||
| Lowest tercile | 3,681 | (31.8) | 34.8 |
| Middle tercile | 4,455 | (38.4) | 34.6 |
| Highest tercile | 3,455 | (29.8) | 34.1 |
| Cocaine/crack severity | |||
| Lowest tercile | 4,600 | (39.7) | 32.1a,b |
| Middle tercile | 3,523 | (30.4) | 34.6a,c |
| Highest tercile | 3,468 | (29.9) | 37.7b,c |
| Heroin severity | |||
| Lower 70% | 8,180 | (70.6) | 34.1 |
| Higher 30% | 3,411 | (29.4) | 35.7 |
Notes: a,b,c,d,e,f,gSuperscripts connote pairs that are significantly different at the overall p < .05 level.
Overall, only 35% of clients had engaged in treatment. However, engagement rates varied by client characteristics with higher engagement rates among women, clients who were older, clients who were unemployed or homeless, and those having received treatment services in the prior year (Table 1). Engagement rates varied substantially among the 59 facilities (data not shown) ranging from 12% to 71%, with both the mean and median being 37%.
Unadjusted Rates of Admissions to Detoxification Services
Table 2 shows the rates of admission to detoxification in the year following the beginning of an outpatient treatment episode.
Table 2.
Rates of detoxification admissions following an outpatient treatment episode
| Had a detox admission | ||
|---|---|---|
| n | % | |
| Overall | 1,723 | 14.9 |
| Engagement Status | ||
| Engaged | 583 | 14.6 |
| Not engaged | 1,140 | 15.0 |
| Gender | ||
| Female | 625 | 16.7a |
| Male | 1,098 | 14.0a |
| Age | ||
| 18–25 | 463 | 14.2a |
| 26–35 | 493 | 16.3a,b |
| 36–44 | 467 | 15.8c |
| 45+ | 300 | 12.9b,c |
| Race/Ethnicity | ||
| White | 1,333 | 15.6a,d |
| Latino | 184 | 12.8a,b |
| Black | 174 | 14.6c |
| Other | 32 | 7.9b,c,d |
| Education | ||
| Less than H.S. | 453 | 14.3a |
| H.S. Grad | 935 | 16.2a,b |
| More than H.S. | 335 | 12.6b |
| Marital Status | ||
| Married | 192 | 11.6a |
| Unmarried | 1,531 | 15.4a |
| Housing Status | ||
| Homeless | 265 | 26.3a |
| Not Homeless | 1,458 | 13.8a |
| Employment Status | ||
| Employed | 414 | 9.6a |
| Not Employed | 1,309 | 18.0a |
| Insurance | ||
| None | 649 | 13.4a,b |
| Public | 860 | 20.2a,c |
| Private/Other | 214 | 8.6b,c |
| Referral Source | ||
| Self/family | 555 | 16.1a,b,c |
| Criminal Justice | 352 | 9.2a,d,g,h |
| Health Professional | 232 | 16.1d,e,f |
| Substance Abuse Tx | 450 | 24.5b,e,g,i |
| Other | 134 | 12.8c,f,h,i |
| Mental Health Treatment History | ||
| Prior mental health treatment | 835 | 16.1a |
| No prior mental health treatment | 888 | 13.8a |
| Substance Abuse Treatment History | ||
| Residential treatment in prior year | 65 | 14.5a |
| No residential treatment in prior year | 1,658 | 41.1a |
| Outpatient treatment in prior year | 279 | 29.4a |
| No outpatient treatment in prior year | 1,444 | 13.6a |
| Detox Admissions—Prior year | ||
| No Detox services | 750 | 8.1a,b |
| 1 Detox admission | 400 | 30.9a,c |
| 2 Detox admissions | 573 | 58.3b,c |
| Substance Use Severity | ||
| Alcohol | ||
| Lowest tercile | 522 | 14.4a |
| Middle tercile | 542 | 13.0b |
| Highest tercile | 659 | 17.4a,b |
| Marijuana | ||
| Lowest tercile | 581 | 15.8a |
| Middle tercile | 606 | 13.6a,b |
| Highest tercile | 536 | 15.5b |
| Cocaine/crack | ||
| Lowest tercile | 469 | 10.2a,b |
| Middle tercile | 492 | 14.0a,c |
| Highest tercile | 762 | 22.0b,c |
| Heroin | ||
| Lower 30% | 736 | 9.0a |
| Higher 70% | 987 | 28.9a |
Notes: a,b,c,d,e,f,g,h,iSuperscripts connote pairs that are significantly different at the p < .05 level
Approximately 15% (N = 1,723) of clients were admitted for detoxification services. Unadjusted rates of any detoxification admission did not vary based on engagement status.
Rates of detoxification admission after outpatient treatment varied sometimes substantially based on client characteristics, substance use severity, and prior experiences with treatment and detoxification services (Table 2). For example, women had higher rates of detoxification admissions than men, and clients who were homeless or unemployed at admission had almost twice the rate of detoxification admissions than those with housing or employment. The largest differences in detoxification admissions were based on prior detoxification experiences. Those who had had two detoxification stays in the year prior to outpatient treatment were more than seven times more likely to have a post-outpatient treatment detoxification admission compared to those who had not had a detoxification stay.
Survival Analyses Results
Table 3 shows the results of the survival analyses predicting any detoxification admission in the year following the outpatient index visit. Engaged outpatient treatment clients had a significant lower hazard of a detoxification admission than non-engaged outpatient treatment clients (Hazard Ratio (H.R.) = 0.87, 95% CI: 0.78, 0.96). At any point in time after beginning outpatient treatment, clients who had not yet been admitted to detox had 13% lower likelihood of a detoxification admission if they had engaged in treatment than if they had not.
Table 3.
Survival analysis predicting admission to detox in year post-index
| Any Detox Admission | ||
|---|---|---|
| Hazard Ratio |
95% CI | |
| Engagement | 0.87** | (0.78, 0.96) |
| Female | 0.98 | (0.88, 1.09) |
| Age (reference: Ages 36–44) | ||
| 18–25 | 1.08 | (0.94, 1.24) |
| 26–35 | 0.98 | (0.86, 1.11) |
| 45+ | 0.88 | (0.76, 1.03) |
| Race/Ethnicity (reference: White) | ||
| Latino | 0.89 | (0.75, 1.05) |
| Black | 1.01 | (0.84, 1.21) |
| Other Race/Ethnicity | 0.76 | (0.54, 1.09) |
| Education (reference: High School Grad) | ||
| Less than H.S. | 0.90 | (0.80, 1.01) |
| More than H.S. | 0.86* | (0.76, 0.98) |
| Married | 1.01 | (0.87, 1.19) |
| Homeless | 1.10 | (0.95, 1.29) |
| Employed | 0.88* | (0.78, 0.99) |
| Insurance(Reference: Private/Other) | ||
| No Insurance | 1.28** | (1.09, 1.51) |
| Public | 1.30** | (1.10, 1.53) |
| Referral Source (reference: Self/Family) | ||
| Criminal Justice | 0.82** | (0.71, 0.94) |
| Health Professional | 1.04 | (0.88, 1.22) |
| Substance Abuse Tx | 1.08 | (0.94, 1.23) |
| Other | 0.84 | (0.69, 1.03) |
| Prior mental health treatment | 0.94 | (0.85, 1.05) |
| Prior year residential substance abuse Tx | 1.03 | (0.80, 1.34) |
| Prior year outpatient substance abuse Tx | 1.38** | (1.21, 1.58) |
| Prior Year Detox (reference: No Detox Prior Year) | ||
| 1 Detox admission in prior year | 2.97** | (2.60, 3.38) |
| 2+ Detox admissions in prior year | 5.43** | (4.77, 6.18) |
| Alcohol Severity (reference: Low) | ||
| Middle Tercile | 0.97 | (0.85, 1.10) |
| Highest Tercile | 1.16* | (1.02, 1.32) |
| Cocaine Severity (reference: Low) | ||
| Middle Tercile | 1.17* | (1.02, 1.35) |
| Highest Tercile | 1.21** | (1.06, 1.39) |
| Heroin Severity (reference: Low) | ||
| Highest Tercile | 1.98** | (1.76, 2.22) |
| Marijuana Severity (reference Low) | ||
| Middle Tercile | 0.80** | (0.70, 0.90) |
| Highest Tercile | 0.77** | (0.67, 0.88) |
| Facility Engagement Rate | 1.32 | (0.71, 2.47) |
p < .05,
p < .01
Other Predictors of a Detoxification Admission
The strongest predictors of a subsequent detoxification admission were detoxification admissions prior to the outpatient treatment episode, particularly having multiple detoxification admissions in the prior year. Other predictors were having public or no insurance (compared to having private insurance), having received outpatient treatment services in the year prior to the outpatient index visit, and higher cocaine and heroin use severity. A higher level of education, being employed at admission, having been referred from the criminal justice system (compared to being self-referred), and higher marijuana severity were all associated with a reduction in the hazard of a subsequent detoxification admission.
Facility engagement rate did not predict reductions in detoxification admissions.
Detoxification Admissions Not Followed by Treatment
In a test of the sensitivity of the results to an alternate definition of detoxification service, twelve percent of clients (N=1,366) were admitted to a detoxification service that was not followed by treatment. In other words, almost 80% (1,366/1,723) of clients who were admitted to detoxification did not receive treatment services promptly after being discharged. In our regression model, controlling for other covariates, clients who engaged had a 21% lower likelihood of a detoxification admission not followed by treatment than clients who had not engaged (H.R. = 0.79, CI: 0.70, 0.88) (results not shown). For the most part, client characteristics predictive of a detoxification admission not followed by treatment were similar to those predictive of any detoxification admission.
DISCUSSION
Clients who engaged in outpatient treatment were less likely to be admitted to detoxification throughout the next year, an outcome that supports the critical importance of engaging clients in treatment. Our results are consistent with previous studies linking outpatient treatment engagement with improved treatment outcomes.5–8,18
Additionally, engagement was associated with an even stronger reduction in detoxification admissions that are not followed by treatment. This is an important finding given that detoxification alone is not expected to maintain recovery, while receipt of treatment services soon after detoxification has been associated with better outcomes.14,19,20
Results from this study provide further evidence of the appropriateness of engagement in outpatient treatment as a tool that state substance abuse agencies or treatment providers might use to monitor quality of treatment. In addition to the potential for improving treatment outcomes at the individual level, improving treatment engagement rates may also benefit the public sector substance abuse treatment system through potential reductions in expensive detoxification admissions. State agencies could include treatment engagement benchmarks in their provider contracts and provide support and trainings to help lower-performing outpatient treatment programs improve their engagement rates. Some states already monitor engagement rates, and some are using public reporting or testing incentives to improve treatment engagement rates.3,21 However, the impact of these strategies has not yet been demonstrated.
Treatment providers could implement process improvement techniques to improve treatment engagement. The Network for the Improvement of Addiction Treatment (NIATx) provides information on “promising practices” to improve early retention in treatment, which is closely related to the engagement measure.22–25 Examples include reminding clients about appointments, following-up with clients who miss appointments, and providing counselors with regular feedback on their no-show and continuation rates.26
It is important to note that there was great variation in both treatment engagement rates and rates of admission to a detoxification program based on client characteristics. Outpatient treatment providers may want to provide additional support or implement targeted interventions to clients who may be at higher risk of not engaging in treatment (e.g., men, younger individuals) or higher risk of having a detoxification admission (e.g., those with higher severity of drug use). Prior detoxification and prior year treatment were both associated with a shorter time to entering a detoxification facility in the subsequent year after beginning outpatient treatment. These utilization patterns may be a normal part of the chronic nature of addiction.27 Still, it is important that for persons with need for repeated detoxification, that these patterns not be an indication of a “revolving door.” For these clients, receiving timely services in outpatient may be particularly beneficial.
The findings from this study should be interpreted with caution, as limitations exist in generalizability and data completeness. The negative association of treatment engagement with detoxification admissions may be specific to Massachusetts’ publicly funded specialty treatment system and it would be important to replicate the study using data from other states or other treatment systems, such as the VA. As more SUD treatment is integrated with primary medical care,28 examining engagement’s association with detoxification in those settings also is critical. Administrative data were used to test the hypotheses, and thus unmeasured factors, such as motivation and participation in self-help groups could not be considered. However, administrative data are readily available without the extra costs and complexities of additional data collection, facilitating the adoption of engagement as a performance measure by state agencies and treatment providers. Finally, some clients in the sample may have obtained detoxification services outside of the Massachusetts public sector or outside of the state completely. Those omissions would bias the relationship between engagement and detoxification downward, and thus the true relationship between engagement and post-treatment detoxification admissions might actually be stronger.
This study provides further evidence of the importance of ensuring that all clients receive prompt services at the beginning of their new treatment outpatient treatment episode. Our findings also support the usefulness of treatment engagement as a performance measure. Although other studies have also shown that engagement is associated with better treatment outcomes, our results suggest that treatment engagement also may provide benefits within the treatment system itself through potential savings if detoxifications admissions can be prevented.
Although engagement represents a minimum of recommended care, for regular outpatient treatment in Massachusetts, the engagement rate is low. Only about 40% of clients met the engagement criteria in FY 2006. Furthermore, as shown in this study, there is great variability in engagement rates across facilities. Thus, there is clear room for improvement.
With an increase in the insured population and improved coverage of SUD treatment services by insurance companies as the result of the ACA, many through the expansion of Medicaid, it is likely that individuals who need treatment will be better able to access services. As access increases, it is crucial that quality of treatment is monitored. Our results show that treatment engagement could be useful way to assess the minimum level of care that all individuals beginning treatment should receive.
Acknowledgements
The authors would like to thank Hermik Babakhanlou-Chase of the Bureau of Substance Abuse Services (BSAS) in Massachusetts and Andrew Hanchet formerly at BSAS for their assistance with the development of the data set. Jennifer Miles and Timothy Creedon of Brandeis University assisted with the preparation of the manuscript. We also thank the National Institute on Alcohol Abuse and Alcoholism for financial support provided and which made the current work possible.
Funding and Degree to which the funding agency was involved in manuscript preparation: This study was supported by Award Numbers F31AA018246 (PI: Acevedo) and T32AA007567 (PI: Horgan) from the National Institute on Alcohol Abuse and Alcoholism. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Alcohol Abuse and Alcoholism or the National Institutes of Health. The funding entity had no further role in study design; in the collection, analysis and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication.
Footnotes
Author contribution: Dr. Acevedo designed the study, conducted the statistical analysis, and wrote the first draft of the manuscript. Dr. Garnick contributed to the design of the study and interpretation of the statistical analyses, and critical revision of the manuscript. Dr. Ritter assisted with the analyses design and interpretation of findings and edited the manuscript. Drs. Lundgren’s and Horgan’s contribution included the interpretation of the data analyses and critical revision of the manuscript. All authors contributed to and have approved the final manuscript.
Contributor Information
Andrea Acevedo, Institute for Behavioral Health The Heller School for Social Policy and Management Brandeis University.
Deborah Garnick, Institute for Behavioral Health The Heller School for Social Policy and Management Brandeis University.
Grant Ritter, Institute for Behavioral Health The Heller School for Social Policy and Management Brandeis University.
Lena Lundgren, Boston University School of Social Work.
Constance Horgan, Institute for Behavioral Health The Heller School for Social Policy and Management Brandeis University.
REFERENCES
- 1.National Quality Forum. NQF Endorsed Standards 2014. [Accessed July 7, 2014]; http://www.qualityforum.org/. [Google Scholar]
- 2.National Committee on Quality Assurance. NCQA HEDIS 2014. [Accessed July 7, 2014];2014 http://www.ncqa.org/HEDISQualityMeasurement/HEDISMeasures/HEDIS2014.aspx. [Google Scholar]
- 3.Garnick DW, Lee MT, Horgan C, et al. Lessons from five states: public sector use of the Washington Circle performance measures. Journal of substance abuse treatment. 2011;40(3):241–254. doi: 10.1016/j.jsat.2010.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Garnick DW, Lee MT, Horgan CM, Acevedo A Washington Circle Public Sector W. Adapting Washington Circle performance measures for public sector substance abuse treatment systems. Journal of substance abuse treatment. 2009;36(3):265–277. doi: 10.1016/j.jsat.2008.06.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Garnick DW, Horgan CM, Acevedo A, et al. Criminal justice outcomes after engagement in outpatient substance abuse treatment. Journal of substance abuse treatment. 2014;46(3):295–305. doi: 10.1016/j.jsat.2013.10.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Dunigan R, Acevedo A, Campbell K, et al. Engagement in outpatient substance abuse treatment and employment outcomes. J Behav Health Serv Res. 2014;41(1):20–36. doi: 10.1007/s11414-013-9334-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Harris AH, Humphreys K, Bowe T, Tiet Q, Finney JW. Does meeting the HEDIS substance abuse treatment engagement criterion predict patient outcomes? J Behav Health Serv Res. 2010;37(1):25–39. doi: 10.1007/s11414-008-9142-2. [DOI] [PubMed] [Google Scholar]
- 8.Garnick DW, Lee MT, O'Brien PL, et al. The Washington circle engagement performance measures' association with adolescent treatment outcomes. Drug Alcohol Depend. 2012;124(3):250–258. doi: 10.1016/j.drugalcdep.2012.01.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.National Institute on Drug Abuse. Understanding Drug Abuse and Addiction. [Accessed September 25, 2014];2012 http://www.drugabuse.gov/publications/drugfacts/understanding-drug-abuse-addiction.
- 10.National Institute on Drug Abuse. Treatment Approaches for Drug Addiction. [Accessed September 25, 2014];2009 http://www.drugabuse.gov/publications/drugfacts/treatment-approaches-drug-addiction.
- 11.Mee-Lee D, Schulman GD, Fishman M, Gastfriend DR, Griffith JH. ASAM Patient Placement Criteria for the Treatment of Substance-Related Disorders, Second Edition-Revised (ASAM PPC-2R) Chevy Chase, Maryland: American Society of Addiction Medicine, Inc; 2001. [Google Scholar]
- 12.Suissa S. Immortal time bias in observational studies of drug effects. Pharmacoepidemiol Drug Saf. 2007;16(3):241–249. doi: 10.1002/pds.1357. [DOI] [PubMed] [Google Scholar]
- 13.Suissa S. Immortal time bias in pharmaco-epidemiology. American journal of epidemiology. 2008;167(4):492–499. doi: 10.1093/aje/kwm324. [DOI] [PubMed] [Google Scholar]
- 14.Lee MT, Horgan CM, Garnick DW, et al. A performance measure for continuity of care after detoxification: relationship with outcomes. Journal of substance abuse treatment. 2014;47(2):130–139. doi: 10.1016/j.jsat.2014.04.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Caspi Y, Turner WM, Panas L, McCarty D, Gastfriend DR, et al. The severity index: an indicator of alcohol and drug dependence using administrative data. Alcohol Treat Quart. 2001;19:49–64. 2001. [Google Scholar]
- 16.McCamant LE, Zani BG, McFarland BH. Prospective validation of substance abuse severity measures from administrative data. Drug and Alcohol Dependence. 2007;86:37–45. doi: 10.1016/j.drugalcdep.2006.04.016. [DOI] [PubMed] [Google Scholar]
- 17.Finney JW, Humphreys K, Kivlahan DR, Harris AH. Why health care process performance measures can have different relationships to outcomes for patients and hospitals: understanding the ecological fallacy. American journal of public health. 2011;101(9):1635–1642. doi: 10.2105/AJPH.2011.300153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Campbell KM. Impact of record-linkage methodology on performance indicators and multivariate relationships. Journal of substance abuse treatment. 2009;36(1):110–117. doi: 10.1016/j.jsat.2008.05.004. [DOI] [PubMed] [Google Scholar]
- 19.Ford LK, Zarate P. Closing the gaps: the impact of inpatient detoxification and continuity of care on client outcomes. J Psychoactive Drugs. 2010;(Suppl 6):303–314. doi: 10.1080/02791072.2010.10400554. [DOI] [PubMed] [Google Scholar]
- 20.Mark TL, Vandivort-Warren R, Montejano LB. Factors affecting detoxification readmission: analysis of public sector data from three states. Journal of substance abuse treatment. 2006;31(4):439–445. doi: 10.1016/j.jsat.2006.05.019. [DOI] [PubMed] [Google Scholar]
- 21.Division of Behavioral Health and Recovery. Incentives and Alerts for Improving Substance Abuse Treatment in Washington State. [Accessed September 25, 2014];2014 http://www.dshs.wa.gov/dbhr/incentives_project.shtml.
- 22.Capoccia VA, Cotter F, Gustafson DH, et al. Making "stone soup": improvements in clinic access and retention in addiction treatment. Jt Comm J Qual Patient Saf. 2007;33(2):95–103. doi: 10.1016/s1553-7250(07)33011-0. [DOI] [PubMed] [Google Scholar]
- 23.Hoffman KA, Ford JH, 2nd, Choi D, Gustafson DH, McCarty D. Replication and sustainability of improved access and retention within the Network for the Improvement of Addiction Treatment. Drug Alcohol Depend. 2008;98(1–2):63–69. doi: 10.1016/j.drugalcdep.2008.04.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.McCarty D, Gustafson DH, Wisdom JP, et al. The Network for the Improvement of Addiction Treatment (NIATx): enhancing access and retention. Drug Alcohol Depend. 2007;88(2–3):138–145. doi: 10.1016/j.drugalcdep.2006.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Roosa M, Scripa JS, Zastowny TR, Ford JH., 2nd Using a NIATx based local learning collaborative for performance improvement. Evaluation and program planning. 2011 doi: 10.1016/j.evalprogplan.2011.02.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.NIATx. NIATx: Removing Barriers to Treatment & Recovery. [Accessed September 25, 2014]; n.d. www.niatx.net. [Google Scholar]
- 27.McLellan AT, Lewis DC, O'Brien CP, Kleber HD. Drug dependence, a chronic medical illness: implications for treatment, insurance, and outcomes evaluation. JAMA : the journal of the American Medical Association. 2000;284(13):1689–1695. doi: 10.1001/jama.284.13.1689. [DOI] [PubMed] [Google Scholar]
- 28.Substance Abuse and Mental Health Administration. 100 Strong: Integration Continues to Grow. [Accessed October 1, 2014];2013 http://www.integration.samhsa.gov/about-us/esolutions-newsletter/september-2013-esolutions.