Abstract
Background
The onset of the COVID-19 pandemic necessitated the rapid transition of many types of substance use disorder (SUD) treatments to telehealth formats, despite limited information about what makes treatment effective in this novel format.
Objective
This study aims to examine the feasibility and effectiveness of virtual intensive outpatient programming (IOP) treatment for SUD in the context of a global pandemic, while considering the unique challenges posed to data collection during an unprecedented public health crisis.
Methods
The study is based on a longitudinal study with a baseline sample of 3642 patients who enrolled in intensive outpatient addiction treatment (in-person, hybrid, or virtual care) from January 2020 to March 2021 at a large substance use treatment center in the United States. The analytical sample consisted of patients who completed the 3-month postdischarge outcome survey as part of routine outcome monitoring (n=1060, 29.1% response rate).
Results
No significant differences were detected by delivery format in continuous abstinence (χ22=0.4, P=.81), overall quality of life (F2,826=2.06, P=.13), financial well-being (F2,767=2.30, P=.10), psychological well-being (F2,918=0.72, P=.49), and confidence in one’s ability to stay sober (F2,941=0.21, P=.81). Individuals in hybrid programming were more likely to report a higher level of general health than those in virtual IOP (F2,917=4.19, P=.01).
Conclusions
Virtual outpatient care for the treatment of SUD is a feasible alternative to in-person-only programming, leading to similar self-reported outcomes at 3 months postdischarge. Given the many obstacles presented throughout data collection during a pandemic, further research is needed to better understand under what conditions telehealth is an acceptable alternative to in-person care.
Keywords: telehealth, substance use disorder, COVID-19, substance use treatment, feasibility study, routine outcome monitoring data, mental health, addiction, digital health, telemedicine, outpatient program, virtual health, addiction treatment, virtual care, patient outcomes
Introduction
Substance use disorder (SUD) is a chronic relapsing disease associated with numerous psychosocial harms and health sequalae. Addiction was a leading global cause of disability and death prior to the COVID-19 pandemic [1], which has since disproportionately impacted individuals suffering from SUD. Recent studies indicate that individuals with SUD may be more susceptible to severe disease and have higher rates of mortality and postvaccination breakthrough infections [2-4]. Isolation, uncertainty, and financial instability have also compounded substance use and the challenges of early recovery [5-7]. These vulnerabilities have reinforced the critical need for ongoing and safe access to treatment throughout the course of the pandemic through virtual services.
Prior to the COVID-19 pandemic, virtual services showed promise but were slow to develop. Early applications have shown promise as a means of preventing premature dropout from SUD treatment [8]. The pandemic rapidly accelerated the implementation of telehealth services for mental health and substance use treatment [9], and both providers and participants have viewed these types of services favorably [10,11]. Unfortunately, little is known about the actual efficacy or effectiveness of individual SUD treatment in telehealth treatment settings [12-14], and even less about traditional group treatment formats [15].
The onset of the pandemic became a catalyst for addiction treatment programs to quickly pivot to provision of services through telehealth formats despite limited data to guide their delivery. Change was facilitated by paradigm shifts in federal, state, and local policies and in organizational and provider practices [16,17]. Although these policies allowed for the continuity of care through available technology, stakeholders within the addiction field are now facing decisions on which elements of policies and programs to sustain, adapt, or discontinue. Continuation of these policies is dependent upon rigorous assessment of clinical data to define the new standard of SUD treatment through virtual platforms. Unfortunately, the pandemic had a devastating impact on research, with ongoing disruptions to recruitment and study progress, as well as a dramatic reduction in survey participation and response rates across many fields of study [18-20].
There is still a significant need for research related to the application and assessment of telehealth for SUD. Unfortunately, best practices for patient outcome collection for SUD treatment in mixed settings have yet to be established. In this paper, we describe how the COVID-19 pandemic presented a novel opportunity to bridge the gap and assess the effectiveness of a virtual intensive outpatient programming (IOP) for substance use treatment through the examination of short-term postprogram outcomes of adults who received IOP services through different delivery formats at the largest SUD treatment provider in the United States.
Methods
Study Design and Population
The Hazelden Betty Ford Foundation (HBFF) is the largest national provider of addiction services in the United States. The HBFF utilizes evidence-based practices through a multidisciplinary and integrated approach to addiction treatment across varying levels of care. In 2019, the HBFF piloted a single virtual intensive outpatient group with planned expansion of virtual services in 2020, as informed by routine outcome monitoring (ROM) data. The HBFF has an established infrastructure and process for collection of ROM data that has been used and refined since 1974. ROM data provide an understanding of real-world conditions, offering applied generalizability to community-based treatment settings where the majority of care is provided [21,22]. These data can be designed as a feedback loop, intended to quickly translate findings into treatment implementation [23,24]. In a rapidly evolving global pandemic, this type of real-world feedback is invaluable to informing the refinement of virtual treatment, despite potentially lower response rates than a formal randomized controlled trial [22].
This study presents 3-month findings (n=1060, 29.1%) from a 12-month longitudinal assessment of patients, 18 years and older, who were discharged from IOP between January 2020 and March 2021 (N=3642). Patients were separated by 3 distinct treatment delivery settings in response to pandemic changes: (1) in-person care only (n=957, 26.3%); (2) hybrid, in-person, and virtual care (n=541, 14.9%); and (3) virtual care only (n=2144, 58.9%).
Ethics Approval
The study was reviewed and approved by Emory University’s Institutional Review Board (STUDY00001822) and was determined to have met the human research exemption since all data were collected within the context of the HBFF’s standard ROM practices.
Data Collection Procedures
Trained research data collection specialists (RDCS) utilized a systematic and manualized process for data collection. Web-based surveys were automatically assigned to IOP patients at 1, 3, 6, 9, and 12 months postdischarge, with survey completion windows open for 30 days. Survey links were emailed to patients, with reminder prompts every 3 days for up to 2 weeks. Patients were contacted by an RDCS every 4-7 days to complete the survey over the phone or to encourage patients to complete it online if not initially completed. Patients were still prompted to provide responses if admitted to a different level of care.
Impact of the Pandemic on Data Collection
In response to a notable decline in response rates at the beginning of the pandemic, RDCS spent over 1200 hours attempting to contact patients to complete the 3-month survey throughout the course of the virtual IOP study. The timing of data collection was also impacted. Virtual IOP began in March 2020, while data collection began in May 2020, largely due to reallocation of resources to facilitate the transition of direct care to virtual services. To retrospectively capture an in-person IOP comparison group, all patients discharged from an in-person IOP on or after January 1, 2020, were opted into receiving IOP outcome surveys. However, due to the 30-day survey windows, the majority of in-person and hybrid patients were excluded from completing the baseline survey at admission and the 1-month postdischarge survey, impacting response rates for those time points (Figure 1).
Figure 1.
Data completeness from baseline to 3-month follow-up. IOP: intensive outpatient program; ROM: routine outcome monitoring.
Measures
Demographic information was collected from patient electronic medical records. The majority of the full sample of patients were White (n=3296, 91.3%) and male (n=2258, 62.0%), with a mean age of 39.1 (SD 13.5) years.
Outcome Measures
A variety of self-reported outcome measures were used to assess for health and well-being at 3-month follow-up. Continuous abstinence from drugs and alcohol (abstinent since discharge vs relapsed) during the follow-up period was assessed using a question from a modified Form 90 Alcohol Questionnaire (Form 90-AQ) [25], adapted to ask about the use of any substances and to include the specific time period for clarity: “Have you used any drugs or alcohol since your last survey on (last survey date)?” Compliance to prescribed anticraving medications (eg, buprenorphine, naltrexone, or acamprosate) was assessed with a single-item, binary question: “Have you taken your anticraving medication, as prescribed?” Peer support group engagement was measured by 1 item from the Alcoholics Anonymous (AA) Involvement Scale [26] and adapted to include a reference to other peer group support beyond AA: “About how often have you been attending 12-step/peer support/mutual aid group meetings since you were discharged?”. Participants rated their frequency of attendance on a 6-point scale: daily, 4 or more times per week, 1-3 times per week, 2-4 times per month, once a month or less, and never.
Quality of life was measured using the 4-item self-reported Centers for Disease Control Healthy Days Survey [27,28]. An additional question assessing overall quality of life was also added: “How would you rate your overall quality of life?” Patients were asked to rate their overall quality of life and quality of general health using a 5-point Likert rating, from 1 (poor) to 5 (excellent), and indicated the number of days out of the previous 30 that they experienced poor mental or physical health. Higher numbers of unhealthy days indicated a lower quality of life. Psychological well-being was assessed by the summed composite of the 8-item Flourishing Scale [29]. For each item, patients rated their level of agreement on a 7-point Likert scale, from 1 (strongly disagree) to 7 (strongly agree). The scale yielded high internal consistency (α=.94). Financial well-being was measured using the 5-item Consumer Financial Protection Bureau (CFPB) Financial Well-Being Scale, with higher scores illustrating greater perceived financial well-being [30]. The CFPB Financial Well-Being Scale showed good internal consistency (α=.85). Patients’ confidence in their ability to stay sober was measured using an adapted form of the Brief Situational Confidence Questionnaire (BSCQ) to create a sobriety self-efficacy scale [31]. The 7-point Likert response categories were reworded to maintain consistency across the different scales, and the original BSCQ question 5, “I could probably go back to social drinking or other moderate drug use if I wanted to,” was removed as initial interitem correlations and α values indicated that this question did not adequately add to the measure of sobriety self-efficacy. After removal of question 5, the adapted scale of sobriety self-efficacy showed high internal consistency (α=.89).
Statistical Analyses
Analyses were performed using IBM SPSS Statistics version 28 [32]. Data were examined using chi-square tests of independence and one-way ANOVAs to ascertain the relationship between IOP delivery setting and patient outcomes. Direct comparisons between the different settings of IOP were not indicated because the virtual IOP study was not prospectively designed and was instead reactively implemented as a result of the pandemic. This pandemic reality resulted in differences in the timing of care, akin to a cohort effect within a single year, where those in-person care reached the 3-month survey earlier in the pandemic (eg, May-July 2020), while those in virtual-only care completed the 3-month survey over a much longer period (eg, June 2020-June 2021).
Results
Sample Characteristics
Sample characteristics are reported by IOP modality in Table 1. Differences between IOP settings emerged in biological sex, age, and length-of-stay distributions. In comparison to in-person and virtual groups, the hybrid group members were more likely to be male (133/192 [69.3%] in the hybrid group vs 161/256 [62.9%] in the in-person group and 363/612 [59.3%] in the virtual group). The virtual group had a greater number of individuals aged from 45 to 64 years (293/612 [47.9%] in the virtual group vs 92/256 [35.9%] in the in-person group and 67/192 [34.9%] in the hybrid group). Individuals in the hybrid group had significantly longer lengths of stay (mean 74.67 [SD 41.78] days) than those who participated in in-person IOP (mean 53.88 [SD 34.79] days) or virtual IOP (mean 54.67 [SD 33.31] days). No significant differences were detected between formats by race, ethnicity, marital status, employment status, education level, whether a patient was discharged against staff/medical advice, use of insurance for services, or type or number of active SUD diagnoses.
Table 1.
Baseline characteristics of 3-month outcomes survey respondents (N=1060).a
| Characteristics | In-person only (N=256) | Hybrid (N=192) | Virtual only (N=612) | Overall (N=1060) | ||||
| Biological sex, n (%); χ22=6.3, P=.04 | ||||||||
|
|
Male | 161 (62.9) | 133 (69.3) | 363 (59.3) | 657 (62.0) | |||
|
|
Nonbinary | N/Ab | N/A | N/A | N/A | |||
|
|
Missing | N/A | N/A | N/A | 0 | |||
| Age (years), n (%);χ26=19.5, P=.003 | ||||||||
|
|
18-25 | 34 (13.3) | 33 (17.2) | 64 (10.5) | 131 (12.4) | |||
|
|
26-44 | 120 (46.9) | 86 (44.8) | 241 (39.4) | 447 (42.2) | |||
|
|
45-64 | 92 (35.9) | 67 (34.9) | 293 (47.9) | 452 (42.6) | |||
|
|
65+ | 10 (3.9) | 6 (3.1) | 14 (2.3) | 30 (2.8) | |||
|
|
Missing | N/A | N/A | N/A | 0 | |||
| Racec, n (%); χ22=11.3, P=.88 | ||||||||
|
|
American Indian or Alaska Native | 2 (0.8) | 1 (0.5) | 6 (1.0) | 9 (0.8) | |||
|
|
Asian or Asian American | 3 (1.2) | 2 (1.0) | 3 (0.5) | 8 (0.8) | |||
|
|
Black or African American | 3 (1.2) | 4 (2.1) | 12 (2.0) | 19 (1.8) | |||
|
|
Native Hawaiian or other Pacific Islander | 1 (0.4) | 0 | 1 (0.2) | 2 (0.2) | |||
|
|
White | 233 (91.0) | 180 (93.8) | 560 (91.5) | 973 (91.8) | |||
|
|
Biracial or multiracial (2+) | 6 (2.3) | 3 (1.6) | 9 (1.5) | 18 (1.7) | |||
|
|
Other | 7 (2.7) | 2 (1.0) | 13 (2.2) | 22 (2.1) | |||
|
|
Missing | 1 (0.4) | 0 | 8 (1.3) | 9 (0.9) | |||
| Ethnicity, n (%); χ22=2.3, P=.31 | ||||||||
|
|
Hispanic or Latinx or Spanish origin | 16 (6.3) | 6 (3.1) | 29 (4.7) | 51 (4.8) | |||
|
|
Not Hispanic or Latinx or Spanish origin | 232 (90.6) | 180 (93.8) | 551 (90.0) | 963 (90.8) | |||
|
|
Missing | 8 (3.1) | 6 (3.1) | 32 (5.2) | 46 (4.3) | |||
| Marital statusc, n (%); χ24=6.3, P=.18 | ||||||||
|
|
Single/never married | 87 (34.0) | 78 (40.6) | 204 (33.3) | 369 (34.8) | |||
|
|
Cohabiting | 4 (1.6) | 3 (1.6) | 11 (1.8) | 18 (1.7) | |||
|
|
Married/life partner | 125 (48.8) | 95 (49.5) | 312 (50.9) | 532 (50.1) | |||
|
|
Married but separated | 14 (5.5) | 6 (3.1) | 19 (3.1) | 39 (3.7) | |||
|
|
Divorced | 21 (8.2) | 9 (4.7) | 57 (9.3) | 87 (8.2) | |||
|
|
Widowed | 3 (1.2) | 1 (0.5) | 1 (0.2) | 5 (0.5) | |||
|
|
Missing | 2 (0.8) | 0 | 8 (1.3) | 10 (0.9) | |||
| Employment statusc, n (%); χ22=1.9, P=.76 | ||||||||
|
|
Full-time employment/Self-employed | 164 (64.1) | 119 (62.0) | 389 (63.6) | 672 (63.4) | |||
|
|
Part-time employment | 12 (4.7) | 4 (2.1) | 21 (3.4) | 38 (3.6) | |||
|
|
Home and family manager | 6 (2.3) | 3 (1.6) | 9 (1.5) | 18 (1.7) | |||
|
|
Student (full- or part-time) or retired | 18 (7.0) | 21 (11.0) | 36 (5.9) | 74 (7.0) | |||
|
|
Unemployment, actively seeking a job | 16 (6.3) | 6 (3.1) | 25 (4.1) | 47 (4.4) | |||
|
|
Unemployment, not seeking a job | 36 (14.1) | 36 (18.8) | 113 (18.5) | 185 (17.5) | |||
|
|
Missing | 4 (1.6) | 3 (1.5) | 19 (2.9) | 26 (2.5) | |||
| Education levelc, n (%); χ24=8.4, P=.08 | ||||||||
|
|
Some high school or less, no diploma | 1 (0.4) | 3 (1.5) | 6 (1.0) | 10 (1.0) | |||
|
|
High school diploma or equivalent (General Educational Development [GED]) | 17 (6.6) | 23 (12.0) | 66 (10.8) | 106 (10.0) | |||
|
|
Some college, no degree | 31 (12.1) | 34 (17.7) | 73 (11.9) | 138 (13.0) | |||
|
|
Associate degree/vocational-technical studies | 15 (5.9) | 14 (7.3) | 36 (5.9) | 65 (6.1) | |||
|
|
College graduate/bachelor’s degree | 85 (33.2) | 56 (29.2) | 166 (27.1) | 307 (29.0) | |||
|
|
Graduate/professional degree | 35 (13.7) | 18 (9.4) | 61 (10.0) | 114 (10.8) | |||
|
|
Missing | 72 (28.1) | 44 (22.9) | 204 (33.4) | 320 (30.2) | |||
| Length of IOPd stay, mean (SD); Welch F2,411.943=19.67, P<.001 | ||||||||
|
|
Average length of stay (in days) | 53.88 (34.79) | 74.67 (41.78) | 54.67 (33.31) | 58.10 (38.156) | |||
|
|
Missing | N/A | N/A | N/A | 0 | |||
| Discharged against staff advice, n (%); χ22=0.8, P=.67 | ||||||||
|
|
Yes | 24 (9.4) | 17 (8.9) | 66 (10.8) | 107 (10.1) | |||
|
|
No | 232 (90.6) | 175 (91.1) | 546 (89.2) | 953 (89.9) | |||
|
|
Missing | N/A | N/A | N/A | 0 | |||
| Used insurance for services, n (%): χ22=2.0, P=.36 | ||||||||
|
|
Yes | 247 (96.5) | 185 (96.4) | 599 (97.9) | 1,031 (97.3) | |||
|
|
Self-pay | 9 (3.5) | 7 (3.6) | 13 (2.1) | 29 (2.7) | |||
|
|
Missing | N/A | N/A | N/A | 0 | |||
| Active SUDe diagnosis, n (%) | ||||||||
|
|
Alcohol use disorder; χ22=0.2, P=.93 | 227 (88.7) | 172 (89.6) | 542 (88.6) | 941 (88.8) | |||
|
|
Cannabis use disorder; χ22=0.3, P=.88 | 51 (19.9) | 42 (21.9) | 128 (20.9) | 221 (20.8) | |||
|
|
Cocaine use disorder; χ22=5.0, P=.08 | 22 (8.6) | 15 (7.8) | 30 (4.9) | 67 (6.3) | |||
|
|
Hallucinogen use disorder; χ22=1.5, P=.48 | 2 (0.8) | 3 (1.6) | 4 (0.7) | 9 (0.8) | |||
|
|
Inhalant use disorder; χ22=4.5, P=.10 | 0 | 1 (0.5) | 0 | 1 (0.1) | |||
|
|
Opioid use disorder; χ22=0.7, P=.71 | 19 (7.4) | 17 (8.9) | 56 (9.2) | 92 (8.7) | |||
|
|
Sedative use disorder; χ22=0.3, P=.85 | 24 (9.4) | 15 (7.8) | 53 (8.7) | 92 (8.7) | |||
|
|
Other stimulant use disorder; χ22=2.4, P=.30 | 31 (12.1) | 15 (7.8) | 59 (9.6) | 105 (9.9) | |||
|
|
Other psychoactive use disorder; χ22=1.0, P=.61 | 4 (1.6) | 4 (2.1) | 7 (1.1) | 15 (1.4) | |||
|
|
Missing | N/A | N/A | N/A | 0 | |||
| Number of co-occurring SUD diagnosesc, n (%); χ26=8.4, P=.21 | ||||||||
|
|
1 | 168 (65.6) | 134 (69.8) | 422 (69.0) | 724 (68.3) | |||
|
|
2 | 61 (23.8) | 31 (16.1) | 135 (22.1) | 227 (21.4) | |||
|
|
3 | 21 (8.2) | 20 (10.4) | 37 (6.0) | 78 (7.4) | |||
|
|
4 | 6 (2.3) | 7 (3.6) | 18 (2.9) | 31 (2.9) | |||
|
|
Missing | N/A | N/A | N/A | 0 | |||
aClinical variables associated with a patient’s treatment measured included the patient’s length of IOP stay (in days), whether the patient was discharged against staff advice (yes/no), whether the patient used insurance or self-pay to finance their treatment (yes/no), the patient’s active SUD diagnoses (eg, alcohol, opioids), and the number of co-occurring SUD diagnoses.
bVariables where categories were collapsed into 2-4 levels in order to test for group differences due to small cell sizes.
cN/A: not applicable.
dIOP: intensive outpatient programming.
eSUD: substance use disorder.
Demographic and clinical characteristics of noncompleters of the 3-month survey were compared with those who completed the survey (Table 2). There were no significant differences in completer status in regard to biological sex, race, identification as Latinx, or the highest level of education attained. However, a few differences emerged in age, marital/relational status, and employment type. Completers of the 3-month survey were slightly older (mean age 42.26 [SD 12.93] years) than noncompleters (mean age 38.26 [SD 13.17] years). In addition, completers were more likely to be employed full-time (672/1060 [63.4%] vs 1400/2503 [55.9%]) and to be married/cohabiting (550/1060 [51.9%] vs 985/2503 [39.8%]).
Table 2.
Sample characteristics of completers of the 3-month follow-up survey and noncompleters.a
| Variables | Completers (N=1060) | Noncompleters (N=2503) | Statistics | ||||||||||
|
|
|
n (%) | Mean (SD) | n (%) | Mean (SD) | F test (df) | χ2 (df) | P value | |||||
| Demographic characteristics | |||||||||||||
|
|
Biological sex: male | 657 (62.0) | N/Ab | 1554 (62.1) | N/A | N/A | 0.01 (1) | .93 | |||||
|
|
Age (years) | 1060 (100.0) | 42.26 (12.93) | 2503 (100.0) | 38.26 (13.17) | 69.35 (1,3561) | N/A | <.001 | |||||
|
|
Race: White | 973 (91.8) | N/A | 2249 (89.8) | N/A | N/A | 3.2 (1) | .07 | |||||
|
|
Ethnicity: Latinx | 51 (4.8) | N/A | 135 (5.4) | N/A | N/A | 0.6 (1) | .45 | |||||
| Employment | 16.1 (2) | <.001 | |||||||||||
|
|
Full-time | 672 (63.4) | N/A | 1400 (55.9) | N/A | N/A | N/A | N/A | |||||
|
|
Unemployed | 232 (21.9) | N/A | 677 (27.0) | N/A | N/A | N/A | N/A | |||||
| Education | 1.8 (2) | .40 | |||||||||||
|
|
General Educational Development (GED) or less | 116 (10.9) | N/A | 306 (12.2) | N/A | N/A | N/A | N/A | |||||
|
|
Some college or bachelor’s degree | 510 (48.1) | N/A | 1180 (47.1) | N/A | N/A | N/A | N/A | |||||
| Marital status | 50.6 (2) | <.001 | |||||||||||
|
|
Married/life partner | 550 (51.9) | N/A | 985 (39.4) | N/A | N/A | N/A | N/A | |||||
|
|
Single | 369 (34.8) | N/A | 1160 (46.3) | N/A | N/A | N/A | N/A | |||||
| Clinical characteristics | |||||||||||||
|
|
Single active SUDc | 724 (68.3) | N/A | 1489 (59.5) | N/A | N/A | 24.6 (1) | <.001 | |||||
|
|
Discharged against staff advice | 107 (10.1) | N/A | 531 (21.2) | N/A | N/A | 62.6 (1) | <.001 | |||||
|
|
Length of stay | 1060 (100.0) | 58.10 (36.16) | 2503 (100.0) | 48.57 (38.73) | 37.60 (1,3561) | N/A | <.001 | |||||
|
|
Step down into IOPd | 559 (52.7) | N/A | 1259 (50.3) | N/A | N/A | 1.8 (1) | .18 | |||||
aMean (SD) reported for continuous variables and proportions (%) of samples reported for categorical variables. Pairwise differences calculated with chi-square tests and ANOVAs, as appropriate.
bN/A: not applicable.
cSUD: substance use disorder.
dIOP: intensive outpatient programming.
A greater number of differences arose in regard to clinical characteristics. Those who completed the 3-month survey were less likely to have multiple active SUDs (336/1060 [31.7%] vs 1014/2503 [40.5%]) and to get discharged against staff/medical advice (107/1060 [10.1%] vs 531/2503 [21.2%]). Completers also showed a longer length of stay in IOP care (mean 58.10 [SD 36.16] days) than noncompleters (mean 49.57 [SD 38.73] days). There was no difference between those who stepped down into IOP from a higher level of programming within the HBFF (eg, residential vs day treatment).
Multivariate Comparisons
A few differences emerged between IOP settings across multiple domains of functioning (Table 3). There was no significant difference by setting in self-reported continuous abstinence, with over two-thirds of the sample (680/960, 70.8%) reporting no drug or alcohol use since discharge. Approximately one-third (332/1060, 31.3%) overall reported still being prescribed an anticraving medication. Of those, no difference in medication compliance emerged between in-person, hybrid, or virtual IOP respondents. Individuals across all settings reported attending peer support meetings, on average, of 1 or 2 times per week. Further, there were no differences across settings in the overall perceived quality of life or in the total number of poor physical and mental health days at 3-month follow-up. Finally, there were no differences detected between IOP setting and the individuals’ confidence in their ability to stay sober, financial well-being, or psychological well-being.
Table 3.
Differences by IOPa setting at 3-month follow-up.b
| Variable | In-person (N=256) | Hybrid (N=192) | Virtual (N=612) | Statistics | ||||||||
|
|
n/N (%) | Mean (SD) | n/N (%) | Mean (SD) | n/N (%) | Mean (SD) | F test (df) | χ2 (df) | P value | |||
| Continuous abstinence | 162/231 (70.1) | N/Ac | 131/180 (72.8) | N/A | 387/549 (70.5) | N/A | N/A | 0.4 (2) | .81 | |||
| Craving medication compliance | 58/75 (77.3) | N/A | 46/55 (83.6) | N/A | 154/202 (76.2) | N/A | N/A | 1.4 (2) | .50 | |||
| Peer support meeting attendance | 235/256 (91.8) | 2.54 (1.51) | 180/192 (93.8) | 2.56 (1.51) | 567/612 (92.6) | 2.60 (1.52) | 0.35 (2,979) | N/A | .86 | |||
| Overall quality of life | 231/256 (90.2) | 3.99 (0.83) | 176/192 (91.7) | 4.09 (0.74) | 422/612 (69.0) | 3.95 (0.81) | 2.06 (2,826) | N/A | .13 | |||
| Overall quality of general health | 225/256 (87.9) | 4.00 (0.84) | 175/192 (91.1) | 4.08 (0.75) | 520/612 (85.0) | 3.89 (0.83) | 4.19 (2,917) | N/A | .01 | |||
| Total number of poor physical and mental health days | 224/256 (87.5) | 3.70 (6.66) | 176/192 (91.7) | 3.59 (6.50) | 531/612 (86.8) | 3.83 (6.43) | 0.10 (2,928) | N/A | .90 | |||
| Self-efficacy for staying sober | 220/256 (85.9) | 5.93 (1.16) | 173/192 (90.1) | 6.00 (1.06) | 551/612 (90.0) | 5.95 (1.27) | 0.21 (2,941) | N/A | .81 | |||
| Psychological well-being | 219/256 (85.5) | 45.09 (9.40) | 170/192 (88.5) | 45.83 (7.51) | 532/612 (86.9) | 44.91 (8.86) | 0.72 (2,918) | N/A | .49 | |||
| Financial well-being | 220/256 (85.9) | 48.40 (8.52) | 170/192 (88.5) | 46.73 (7.96) | 380/612 (62.1) | 47.20 (8.16) | 2.30 (2,767) | N/A | .10 | |||
aIOP: intensive outpatient programming.
bMean (SD) reported for continuous variables and proportions (%) of samples reported for categorical variables. Pairwise differences calculated with chi-square tests and ANOVAs, as appropriate.
cN/A: not applicable.
The only significant difference by IOP setting that emerged was in the overall quality of one’s general health, where those in the hybrid group (mean 4.08 [SD 0.75]) were more likely than those in the virtual group (mean 3.89 [SD 0.83]) to report a higher level of general health.
Discussion
Principal Findings
This study is the first of its kind to assess telehealth for SUD in the IOP setting in a large cohort of patients (N=1000+). No meaningful differences in outcome measures were identified between delivery settings at 3-month follow-up, with individuals reporting similar levels of continuous abstinence, quality of life, and social/emotional well-being. Our findings in regard to continuous abstinence were consistent with previous studies following patients at 3-6 months postdischarge from IOP (eg, 65/103 [63.1%]) [33]. These results are promising and suggest a potential continuing role for virtual IOP as an effective component in addiction treatment settings. Advocacy is needed to maintain these services as a standard offering within the SUD treatment continuum.
Historically, peer-based connections and the therapeutic milieu have been integral parts of addiction treatment. Concern has been expressed by the addiction treatment community regarding the shift to virtual services and its impact on group engagement and patient-centered outcomes [34]. These preliminary results demonstrate the feasibility of offering services virtually. Further research is necessary to obtain feedback on patient experience and measures of group cohesion, such as secure emotional expression, as they apply to virtual addiction treatment [35].
Our findings aid in establishing a platform for future evaluation of data collection processes that inform the effective development of standardized protocols for routine outcomes data practices, including frequency of contact, method of outreach, and training of staff. Standardized protocols must consider the context for accurate interpretation of collected data. For example, differences in response rates emerged in this study based on the timing of data collection in relation to the global pandemic and due to unanticipated staff burden and should be interpreted in this context. At 3-month follow-up, response rates were lowest for those in in-person IOP (256/957, 26.8%). These rates are likely attributable to these being completed by patients between May and August 2020, timing that coincided with major city- and statewide lockdowns and great uncertainty about the unknowns presented by the pandemic. Furthermore, because of the large opt-in of the in-person cohort at the beginning of data collection (May 2020), many participants at 3-month follow-up were given access to the survey well into the 30-day response window, reducing the likelihood that they would have adequate time to complete it prior to survey close. In contrast, response rates for those in virtual IOP (612/1532, 28.5%) were most impacted by a higher-than-projected admission rate of individuals into IOP throughout the study period, resulting in a larger sample size than originally anticipated. As a result, there were proportionally lower staffing levels than would be typically allocated for the final sample size, which may explain the lower response rates for the virtual group.
Strengths and Limitations
A key strength of this study is the breadth of data collected from such a large number of patients receiving SUD treatment during a period of extensive change. Although these results are mainly descriptive, these analyses are necessary to carefully evaluate the impacts of a global shift in treatment approach. A number of limitations should be considered when interpreting results. Similar to existing research used with ROM data [36-39], response rates at 3-month follow-up were low. Nonresponse bias is a risk inherent to survey analysis. However, even in studies with high response rates, research has shown that nonresponse rates are not always directly predictive of nonresponse bias [37,40-42]. Research supports that highly resource-intensive recruitment deployed to capture late responders does not necessarily alter the outcomes found at lower, less resource-intense response rates, and this type of recruitment can be both cost- and time-prohibitive [40,42]. Despite COVID-induced high nonresponse rates, we expect these data to accurately reflect the effectiveness of IOP services during the COVID-19 pandemic. Substance use at follow-up was based on retrospective self-report of use. Utilizing additional methods for verification, such as urine drug analyses, would strengthen the validity of these reports in future studies. As discussed earlier, due to the sudden onset of the pandemic and subsequent data collection for this unique cohort, baseline and 1-month follow-up data were unable to be collected from a substantial portion of the full sample. The missing data influenced our ability to directly compare the effectiveness of IOP across delivery type, given the inherent confounding effect of the timing of patient care in relation to the unfolding public health crisis. We recommend future prospective studies be designed to compare in-person and virtual treatment directly, with inclusion of a formal evaluation of the ideal conditions for patient success (ie, dosage, treatment duration, frequency).
Finally, although our sample was representative of HBFF program participants, it differs from the general population in a few specific ways that are important to acknowledge when considering the generalizability of the findings. First, patients in the sample were primarily White and non-Latinx. As a result, there may have been too few non-White/Latinx participants to detect differences. However, this sample directly compares to findings from the 2020 National Survey on Drug Use and Health (NSDUH) [43] in terms of full- and part-time employment (United States: 67.3% vs sample: 67.0%) and bachelor’s degree attainment (United States: 27.7% vs sample: 29%) among adults 18+ years old with a SUD in the past year. This demonstrates that addiction affects individuals across varying educational and employment statuses. This is in contrast to stereotypes of individuals struggling with addiction, which may characterize this as a disease of the uneducated and unemployed. Even though a higher percentage of patients employed full-time completed the 3-month survey (672/1060 [63.4%] vs 1400/2503 [55.9%]), post hoc analyses revealed no difference in outcomes based on employment status; therefore, we believe this had minimal impact on our findings. Future research should endeavor to improve the representation of racial and ethnic minorities in order to improve generalizability across a wider cross section of demographic variables.
Conclusions
Results from this study suggest that virtual outpatient care for the treatment of SUDs is a feasible alternative to in-person care, leading to similar rates of self-reported continuous abstinence, health, and well-being in patients at 3-month follow-up. This study should serve as a baseline for the assessment and refinement of the role of virtual services in the field of addiction treatment in order to better understand under what circumstances telehealth can function as an effective alternative to the established in-person standard of care.
Acknowledgments
The authors wish to thank Siara Sitar, who assisted in formatting the manuscript, and Clare Schmidt, for her contributions to the Methods section.
Abbreviations
- AA
Alcoholics Anonymous
- CFPB
Consumer Financial Protection Bureau
- HBFF
Hazelden Betty Ford Foundation
- IOP
intensive outpatient programming
- RDCS
research data collection specialists
- ROM
routine outcome monitoring
- SUD
substance use disorder
Footnotes
Conflicts of Interest: JWW received consulting fees from Applied Clinical Intelligence LLC (ACI Clinical). QMN, KG, and JEB are employees of the Hazelden Betty Ford Foundation (HBFF). LAW has no disclosures to report.
References
- 1.Degenhardt L, Whiteford HA, Ferrari AJ, Baxter AJ, Charlson FJ, Hall WD, Freedman G, Burstein R, Johns N, Engell RE, Flaxman A, Murray CJ, Vos T. Global burden of disease attributable to illicit drug use and dependence: findings from the Global Burden of Disease Study 2010. Lancet. 2013 Nov;382(9904):1564–1574. doi: 10.1016/s0140-6736(13)61530-5. [DOI] [PubMed] [Google Scholar]
- 2.Wang L, Wang Q, Davis PB, Volkow ND, Xu R. Increased risk for COVID-19 breakthrough infection in fully vaccinated patients with substance use disorders in the United States between December 2020 and August 2021. World Psychiatry. 2022 Feb 05;21(1):124–132. doi: 10.1002/wps.20921. doi: 10.1002/wps.20921. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Wang QQ, Kaelber DC, Xu R, Volkow ND. COVID-19 risk and outcomes in patients with substance use disorders: analyses from electronic health records in the United States. Mol Psychiatry. 2021 Jan;26(1):30–39. doi: 10.1038/s41380-020-00880-7. http://europepmc.org/abstract/MED/32929211 .10.1038/s41380-020-00880-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Dorjee K, Kim H, Bonomo E, Dolma R. Prevalence and predictors of death and severe disease in patients hospitalized due to COVID-19: a comprehensive systematic review and meta-analysis of 77 studies and 38,000 patients. PLoS One. 2020 Dec 7;15(12):e0243191. doi: 10.1371/journal.pone.0243191. https://dx.plos.org/10.1371/journal.pone.0243191 .PONE-D-20-19976 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hurley EA, Piña K, Cegielski V, Noel-MacDonnell JR, Miller MK. Recovering from substance use disorders during the early months of the COVID-19 pandemic: a mixed-methods longitudinal study of women in Kansas City. J Subst Abuse Treat. 2021 Oct;129:108378. doi: 10.1016/j.jsat.2021.108378. http://europepmc.org/abstract/MED/34080549 .S0740-5472(21)00104-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Marsden J, Darke S, Hall W, Hickman M, Holmes J, Humphreys K, Neale J, Tucker J, West R. Mitigating and learning from the impact of COVID-19 infection on addictive disorders. Addiction. 2020 Jun 28;115(6):1007–1010. doi: 10.1111/add.15080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Sherman AC, Williams ML, Amick BC, Hudson TJ, Messias EL. Mental health outcomes associated with the COVID-19 pandemic: prevalence and risk factors in a southern US state. Psychiatry Res. 2020 Nov;293:113476. doi: 10.1016/j.psychres.2020.113476. http://europepmc.org/abstract/MED/33198047 .S0165-1781(20)33137-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Tarp K, Bojesen AB, Mejldal A, Nielsen AS. Effectiveness of optional videoconferencing-based treatment of alcohol use disorders: randomized controlled trial. JMIR Ment Health. 2017 Sep 29;4(3):e38. doi: 10.2196/mental.6713. https://mental.jmir.org/2017/3/e38/ v4i3e38 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ellis LA, Meulenbroeks I, Churruca K, Pomare C, Hatem S, Harrison R, Zurynski Y, Braithwaite J. The application of e-mental health in response to COVID-19: scoping review and bibliometric analysis. JMIR Ment Health. 2021 Dec 06;8(12):e32948. doi: 10.2196/32948. https://mental.jmir.org/2021/12/e32948/ v8i12e32948 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Molfenter T, Roget N, Chaple M, Behlman S, Cody O, Hartzler B, Johnson E, Nichols M, Stilen P, Becker S. Use of telehealth in substance use disorder services during and after COVID-19: online survey study. JMIR Ment Health. 2021 Feb 08;8(2):e25835. doi: 10.2196/25835. https://mental.jmir.org/2021/2/e25835/ v8i2e25835 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Sugarman DE, Busch AB, McHugh RK, Bogunovic OJ, Trinh CD, Weiss RD, Greenfield SF. Patients' perceptions of telehealth services for outpatient treatment of substance use disorders during the COVID-19 pandemic. Am J Addict. 2021 Sep 17;30(5):445–452. doi: 10.1111/ajad.13207. http://europepmc.org/abstract/MED/34405475 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Huskamp HA, Busch AB, Souza J, Uscher-Pines L, Rose S, Wilcock A, Landon BE, Mehrotra A. How is telemedicine being used in opioid and other substance use disorder treatment? Health Aff (Millwood) 2018 Dec;37(12):1940–1947. doi: 10.1377/hlthaff.2018.05134. http://europepmc.org/abstract/MED/30633671 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Bergman BG, Kelly JF. Online digital recovery support services: an overview of the science and their potential to help individuals with substance use disorder during COVID-19 and beyond. J Subst Abuse Treat. 2021 Jan;120:108152. doi: 10.1016/j.jsat.2020.108152. http://europepmc.org/abstract/MED/33129636 .S0740-5472(20)30409-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Ashford RD, Bergman BG, Kelly JF, Curtis B. Systematic review: digital recovery support services used to support substance use disorder recovery. Human Behav Emerg Tech. 2019 Apr 30;2(1):18–32. doi: 10.1002/hbe2.148. [DOI] [Google Scholar]
- 15.King VL, Stoller KB, Kidorf M, Kindbom K, Hursh S, Brady T, Brooner RK. Assessing the effectiveness of an internet-based videoconferencing platform for delivering intensified substance abuse counseling. J Subst Abuse Treat. 2009 Apr;36(3):331–8. doi: 10.1016/j.jsat.2008.06.011.S0740-5472(08)00121-9 [DOI] [PubMed] [Google Scholar]
- 16.Drug Enforcement Administration DEA Qualifying Practitioners and DEA Qualifying Other Practitioners Announcement. [2022-03-08]. https://tinyurl.com/yckm2ea5 .
- 17.Substance Abuse and Mental Health Services Administration Opioid Treatment Program (OTP) Guidance. [2021-12-20]. https://www.samhsa.gov/sites/default/files/otp-guidance-20200316.pdf .
- 18.Tolaney S, Lydon C, Li T, Dai J, Standring A, Legor KA, Caparrotta CM, Schenker MP, Glazer DI, Tayob N, DuBois SG, Meyerhardt JA, Taplin M-E, Johnson BE. The impact of COVID-19 on clinical trial execution at the Dana-Farber Cancer Institute. J Natl Cancer Inst. 2021 Nov 02;113(11):1453–1459. doi: 10.1093/jnci/djaa144. http://europepmc.org/abstract/MED/32959883 .5906529 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Horrell LN, Hayes S, Herbert LB, MacTurk K, Lawhon L, Valle CG, Bhowmick A. Telemedicine use and health-related concerns of patients with chronic conditions during COVID-19: survey of members of online health communities. J Med Internet Res. 2021 Feb 18;23(2):e23795. doi: 10.2196/23795. https://www.jmir.org/2021/2/e23795/ v23i2e23795 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Krueger T, Gogolewski K, Bodych M. Assessing the risk of COVID-19 epidemic resurgence in relation to the Delta variant and to vaccination passes. medRxiv. 2021:1–34. doi: 10.1101/2021.05.07.21256847. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Kelly PJ, Deane FP, Davis EL, Hudson S, Robinson LD, Keane CA, Hatton EL, Larance B. Routine outcome measurement in specialist non-government alcohol and other drug treatment services: establishing effectiveness indicators for the NADAbase. Drug Alcohol Rev. 2021 May 05;40(4):540–552. doi: 10.1111/dar.13191. [DOI] [PubMed] [Google Scholar]
- 22.van der Lem R, de Wever WW, van der Wee NJ, van Veen T, Cuijpers P, Zitman FG. The generalizability of psychotherapy efficacy trials in major depressive disorder: an analysis of the influence of patient selection in efficacy trials on symptom outcome in daily practice. BMC Psychiatry. 2012 Nov 8;12(1):192. doi: 10.1186/1471-244x-12-192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Boswell JF, Kraus DR, Miller SD, Lambert MJ. Implementing routine outcome monitoring in clinical practice: benefits, challenges, and solutions. Psychother Res. 2015 Jul 26;25(1):6–19. doi: 10.1080/10503307.2013.817696. [DOI] [PubMed] [Google Scholar]
- 24.Lambert MJ, Whipple JL, Kleinstäuber M. Collecting and delivering progress feedback: a meta-analysis of routine outcome monitoring. Psychotherapy (Chic) 2018 Dec;55(4):520–537. doi: 10.1037/pst0000167.2018-51673-016 [DOI] [PubMed] [Google Scholar]
- 25.Miller W. Form 90: A Structured Assessment Interview for Drinking and Related Behaviors. 1996. [2022-03-08]. https://pubs.niaaa.nih.gov/publications/projectmatch/match05.pdf .
- 26.Tonigan JS, Connors GJ, Miller WR. Alcoholics Anonymous Involvement (AAI) scale: reliability and norms. Psychol Addict Behav. 1996 Jun;10(2):75–80. doi: 10.1037/0893-164x.10.2.75. [DOI] [Google Scholar]
- 27.Centers for Disease Control and Prevention Health-Related Quality of Life (HRQOL) [2022-03-08]. https://www.cdc.gov/hrqol/methods.htm .
- 28.Moriarty DG, Zack MM, Kobau R. The Centers for Disease Control and Prevention's Healthy Days Measures: population tracking of perceived physical and mental health over time. Health Qual Life Outcomes. 2003 Sep 02;1(1):37. doi: 10.1186/1477-7525-1-37. https://hqlo.biomedcentral.com/articles/10.1186/1477-7525-1-37 .1477-7525-1-37 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Diener E, Wirtz D, Tov W, Kim-Prieto C, Choi D, Oishi S, Biswas-Diener R. New well-being measures: short scales to assess flourishing and positive and negative feelings. Soc Indic Res. 2009 May 28;97(2):143–156. doi: 10.1007/s11205-009-9493-y. [DOI] [Google Scholar]
- 30.Consumer Finance Protection Bureau Measuring Financial Well-Being. A Guide to Using the CFPB Financial Well-Being Scale. [2021-12-14]. https://www.consumerfinance.gov/data-research/research-reports/financial-well-being-scale/
- 31.Delaney DJ, Bernstein MH, Harlow LL, Farrow M, Martin RA, Stein LAR. The Brief Situational Confidence Questionnaire for alcohol: a psychometric assessment with incarcerated youth. Psychol Assess. 2020 Mar;32(3):254–264. doi: 10.1037/pas0000780.2019-66268-001 [DOI] [PubMed] [Google Scholar]
- 32.IBM Corp IBM SPSS Statistics 24. [2021-11-30]. https://www.ibm.com/support/pages/release-notes-ibm-spss-statistics-240#relnotes__description .
- 33.Bottlender M, Soyka M. Efficacy of an intensive outpatient rehabilitation program in alcoholism: predictors of outcome 6 months after treatment. Eur Addict Res. 2005 Jul 9;11(3):132–7. doi: 10.1159/000085548.EAR2005011003132 [DOI] [PubMed] [Google Scholar]
- 34.Mark TL, Treiman K, Padwa H, Henretty K, Tzeng J, Gilbert M. Addiction treatment and telehealth: review of efficacy and provider insights during the COVID-19 pandemic. Psychiatr Serv. 2021 Oct 13;:appips202100088. doi: 10.1176/appi.ps.202100088. [DOI] [PubMed] [Google Scholar]
- 35.Joyce AS, MacNair-Semands R, Tasca GA, Ogrodniczuk JS. Factor structure and validity of the Therapeutic Factors Inventory–Short Form. Group Dyn Theory Res Pract. 2011 Sep;15(3):201–219. doi: 10.1037/a0024677. [DOI] [Google Scholar]
- 36.de Beurs E, den Hollander-Gijsman ME, van Rood YR, van der Wee NJA, Giltay EJ, van Noorden MS, van der Lem R, van Fenema E, Zitman FG. Routine outcome monitoring in the Netherlands: practical experiences with a web-based strategy for the assessment of treatment outcome in clinical practice. Clin Psychol Psychother. 2011 Jan 12;18(1):1–12. doi: 10.1002/cpp.696. [DOI] [PubMed] [Google Scholar]
- 37.Hendra R, Hill A. Rethinking response rates: new evidence of little relationship between survey response rates and nonresponse bias. Eval Rev. 2019 Oct;43(5):307–330. doi: 10.1177/0193841X18807719. [DOI] [PubMed] [Google Scholar]
- 38.Hoenders RHJ, Bos EH, Bartels-Velthuis AA, Vollbehr NK, van der Ploeg K, de Jonge P, de Jong JTVM. Pitfalls in the assessment, analysis, and interpretation of routine outcome monitoring (ROM) Data: results from an outpatient clinic for integrative mental health. Adm Policy Ment Health. 2014 Sep 25;41(5):647–59. doi: 10.1007/s10488-013-0511-7. [DOI] [PubMed] [Google Scholar]
- 39.Stiles WB, Barkham M, Twigg E, Mellor-Clark J, Cooper M. Effectiveness of cognitive-behavioural, person-centred and psychodynamic therapies as practised in UK National Health Service settings. Psychol Med. 2006 Feb 14;36(4):555–566. doi: 10.1017/s0033291706007136. [DOI] [PubMed] [Google Scholar]
- 40.Amico KR. A poor metric for study rigor in hosted intervention designs. Am J Public Health. 2009 Sep;99(9):1567–1575. doi: 10.2105/ajph.2008.134767. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Groves RM, Peytcheva E. The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opin Q. 2008 May 02;72(2):167–189. doi: 10.1093/poq/nfn011. [DOI] [Google Scholar]
- 42.Davern M, McAlpine D, Beebe TJ, Ziegenfuss J, Rockwood T, Call KT. Are lower response rates hazardous to your health survey? An analysis of three state telephone health surveys. Health Serv Res. 2010 Oct;45(5 Pt 1):1324–44. doi: 10.1111/j.1475-6773.2010.01128.x. http://europepmc.org/abstract/MED/20579127 .HESR1128 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Substance Abuse and Mental Health Services Administration 2020 NSDUH Detailed Tables. [2022-03-07]. https://www.samhsa.gov/data/report/2020-nsduh-detailed-tables .