Skip to main content
Journal of General Internal Medicine logoLink to Journal of General Internal Medicine
. 2019 Aug 29;34(11):2520–2529. doi: 10.1007/s11606-019-05257-3

Implementing Motivational Interviewing for Substance Misuse on Medical Inpatient Units: a Randomized Controlled Trial

Steve Martino 1,2,, Paula Zimbrean 1, Ariadna Forray 1, Joy S Kaufman 1, Paul H Desan 1, Todd A Olmstead 3, Kathryn Gilstad-Hayden 1, Ralitza Gueorguieva 4, Kimberly A Yonkers 1
PMCID: PMC6848470  PMID: 31468342

Abstract

Background

General medical hospitals provide care for a disproportionate share of patients who misuse substances. Hospitalization provides a unique opportunity to identify and motivate patients to address their substance misuse.

Objective

To determine the effectiveness of three strategies for implementing motivational interviewing for substance misuse with general medical inpatients.

Design

Type 3 hybrid effectiveness-implementation randomized controlled trial (Clinical Trials.gov: NCT01825057).

Participants

Thirty-eight providers (physicians, physician assistants, nurses) from 13 general medical inpatient services, and 1173 of their patients admitted to an academically affiliated acute care hospital.

Interventions

Implementation strategies included (1) a continuing medical education workshop on detection of substance misuse and provision of a motivational interview; (2) workshop plus bedside supervision (apprenticeship condition); and (3) a workshop plus ability to place a medical order for an interview from a consultation-liaison service (consult condition).

Main Measures

Primary outcomes were the percentage of study-eligible patients who received an interview for substance misuse and the integrity (adherence, competence) of the interviews. The secondary outcome was the percent of patient statements within the interviews that indicated motivation for reducing substance misuse.

Key Results

20.5% of patients in the consult condition received an interview, compared to 0.8% (Hedge’s g = 1.49) and 3.0% (Hedge’s g = 1.26) in the respective workshop only and apprenticeship conditions (p < 0.001). Motivational interviews in the consult condition were performed with more fundamental motivational interviewing adherence and competence than the other conditions. Most statements made by patients during the interviews favored reducing substance misuse, with no differences between conditions.

Conclusions

Providers’ ability to place an order to have experts from the consultation-liaison service deliver a motivational interview was a more effective implementation strategy than a workshop or apprenticeship method for ensuring motivational interviewing is available to medical inpatients who misuse substances.

Trial Registry

NCT01825057

KEY WORDS: behavioral science, consultation, implementation research, substance abuse

INTRODUCTION

Acute medical hospitalizations involve a disproportionate share of patients who misuse substances19 and can serve as a “reachable” moment to assist these patients.10, 11 Motivational interviewing is an evidence-based behavioral intervention that can capitalize on this reachable moment.1214 Providers blend fundamental patient-centered communication skills (e.g., reflective listening) with motivational methods (e.g., exploring reasons to not misuse substances) to elicit and support patients’ arguments for change (called change talk) and to resolve arguments against change. When providers use such practices with sufficient frequency (adherence) and skill (competence), their patients are more likely to express interest in changing problematic health habits; this in turn predicts better treatment outcomes.15, 16 Motivational interviewing can be taught to a broad range of healthcare providers,17 particularly when the implementation strategies used to train them are compatible with existing practices and settings.1822 However, implementation strategies for training providers in motivational interviewing have different pros and cons, leaving it unclear which are optimal methods.

One common implementation strategy for enhancing medical practice is a continuing medical education (CME) workshop. This strategy may temporarily boost motivational interviewing skills,17, 23, 24 but proficiency attenuates without subsequent supervision.2527 Another familiar strategy is the apprenticeship model2830 wherein an instructor explains the practice and demonstrates it with simulated or actual patients. Trainees then perform it under expert supervision and receive feedback and coaching to improve performance. Such supervision promotes proficient use of motivational interviewing26, 27 and other behavioral practices,29, 31 particularly when the supervision is ongoing and intervention-specific.26, 27, 32, 33 However, the apprenticeship model can be cumbersome since it relies on the availability of appropriate patients and experts.

A third common implementation strategy is to order (via the electronic health record) an expert consultation or procedure. In inpatient settings, this involves a consultation-liaison (CL) service. Typically, psychiatrists or other addiction experts and affiliated staff (e.g., advanced practice nurses) provide specialty guidance for providers’ management of patients.34 About 20% of CL consultations involve patients’ who misuse substances.3539 The use of CL as a vehicle for implementing counseling approaches such as motivational interviewing has not been tested against other approaches. Nonetheless, “ordering” behavioral health services was used successfully to prompt nurse-led inpatient tobacco cessation interventions40 and mental healthcare management and therapy within Veterans Health Affairs facilities.41 Providers might find using CL a simple, minimally burdensome process and compatible with how they secure other specialist services. However, it requires patients to accede to interventions delivered by unfamiliar professionals, and consults must be completed before patients are discharged.

The primary aim of this study was to determine which of the above three strategies would best promote use of motivational interviewing and its delivery with integrity (i.e., adherence and competence). We hypothesized that, compared to workshop only, (1) the percentage of interviews delivered by providers to study-eligible patients would be higher in the apprenticeship and consult approaches, and (2) providers in these conditions would administer motivational interviews with more integrity. Secondarily, as a proxy of motivational interviewing’s effectiveness, we hypothesized that, compared to workshop only, other conditions would lead patients to use a greater percentage of “change talk.”

METHODS

Study Design

This study is a type 3 hybrid effectiveness-implementation randomized controlled trial42 that primarily aims to compare three implementation conditions (workshop, apprenticeship, or consult), while also collecting data about patients’ statements during interviews (change talk as a proxy for treatment outcome).15, 16 We randomized providers to condition, with patients following their providers’ assignment. Data were collected February 21, 2013, through August 16, 2017. The Yale University Human Investigation Committee approved the study (ClinicalTrials.gov: NCT01825057).

Study Setting

The study took place on general medical units of a university-affiliated teaching hospital in which physician-physician assistant (PA) teams from the hospitalist service shared care of 10–18 patients daily, seeing them 1–2 times per day. The teams covered about 160 beds/day on 13 different units, excluding intensive care, and were responsible for approximately 10,000 discharges annually.

Participants

Provider participants included physicians, PAs, and nurses who were (1) assigned to cover a general medical hospitalist unit and (2) predominantly worked daytime shifts when CL was most available. Exclusion criteria were (1) previous supervision in motivational interviewing or (2) anticipated leave or employment termination.

Patient eligibility included (1) 18 years of age or older; (2) acknowledged problematic use of nicotine, alcohol, illicit drugs, or prescription medications (e.g., benzodiazepines, opioids) within the previous 28 days; and (3) an expected length of stay of at least 2–3 days, thus providing sufficient time to deliver an interview or place a consult. Exclusion criteria were (1) acute altered mental status (assessed with The Short Confusion Assessment Scale), encephalopathy, dementia, or a Mini Mental Status Exam that was < 21; (2) inability to speak English; and (3) any medical condition that made it difficult to complete an assessment and interview (e.g., stroke, deafness, tracheostomy).

Screening and Randomization

We screened interested providers and obtained written informed consent alongside baseline assessments. After attending a workshop, we randomized providers to conditions (workshop, apprenticeship, or consult). The statistician used Research Randomizer43 software to develop a block randomization sequence that equally allocated participants to fixed block sizes of three. Research assistants (RAs) screened and obtained written informed consent from eligible and interested patients. A hospital-based psychiatrist/co-author (PD) assisted RAs with eligibility determination when patient appropriateness for participation was uncertain (e.g., encephalopathy or other altered mental status).

Study Procedures

For the consult condition, three psychiatrists and one social worker were trained in motivational interviewing with: (1) a 2-day skill-building workshop; (2) three post-workshop supervised practice cases; and (3) follow-up monthly group supervision to maintain proficiency.44 They also learned supervisory practices45 to perform bedside supervision in the apprenticeship condition.

Patients completed a brief demographic questionnaire and were screened for (1) neurocognitive deficits (exclusionary); (2) past 28-day substance misuse; and (3) sufficient substance use disorder symptoms for provider detection, namely, > two symptoms endorsed on the modified Mini-International Neuropsychiatric Inventory (MINI).46 They completed baseline measures and were paid $30. They were told that a provider might approach them to discuss their substance use and audio record the conversation for the study.

We did not disclose to providers which of their patients had enrolled in the study. All providers received audio recorders for interviews, but only providers randomized to the consult condition could place an order for a motivational interview. We followed providers until they each cared for 40 study-enrolled patients, regardless of whether they performed an interview with their patients. Providers received $350 and eight CME credits for workshop training and $50 for post-trial assessments.

Conditions

Providers participated in a 1-day workshop conducted by the first author (SM), a member of the Motivational Interviewing Network of Trainers (MINT), according to MINT recommendations.12 Providers also learned screening methods using the modified CAGE (cut down, annoyed, guilty, eye opener) for alcohol and drugs47 and Heaviness of Smoking Index.48 Providers in the workshop only and consult conditions had no further motivational interviewing training, but those in the apprenticeship condition met with a CL motivational interviewing expert for two post-workshop coaching sessions. They were offered a refresher session after they had treated 20 eligible patients. Nurses contacted a physician or PA to place the order.

Outcomes and Measurements

The primary outcomes were the percentage of study-eligible patients who received an interview and the integrity of the interviews. The secondary outcome was amount of (percent) change talk by patients during sessions. We considered an interview completed if it was recorded by the provider. We used the Independent Tape Rater Scale (ITRS)49 to assess the integrity of interviews on five fundamental and five advanced motivational interviewing skills. Three external, blinded raters evaluated audio recorded interviews (i.e., extent/frequency of using motivational skills) and competence (i.e., quality of skill implementation) along 7-point Likert scales. They also evaluated the percent change talk using the Motivational Interviewing Skills Code 2.1 Client Language Coding System.50

We collected provider and patient descriptive data to characterize the samples. Providers completed the Clinician Survey51 for demographics and a Motivation for MI scale52 to measure initial interest in learning and confidence in and likelihood of using motivational interviewing. Patients completed the (1) MINI Neuropsychiatric Interview46 to generate patient DSM 5 drug, alcohol, and nicotine use symptoms and diagnoses; (2) Heaviness of Smoking Index48 and sections of the Addiction Severity Index53 to evaluate the severity of nicotine and alcohol/drug problems, respectively; (3) Patient Health Questionnaire-954 to assess depression; (4) Short Form Health Survey-1255 to measure patients’ functional health; and (6) Motivation for Change Scale56 to gauge importance, ability, and commitment to not misuse substances. The study team conducted a medical chart review for length of stay and for Healthcare Cost and Utilization Project (HCUP) disease category.

Statistical Methods

We originally estimated 80% of patients across conditions would receive an interview. Based on prior meta-analyses of the motivational interviewing training literature,26, 27 we assumed medium effect sizes for between-group comparisons of integrity. Intra-class correlations accounting for expected provider variability were expected to be small (0.05 to 0.10 range).57 Feasibility constraints (i.e., 10 providers per condition) limited the number of provider clusters we considered. With a 2-sided alpha level of 0.05, and power of 80%, we targeted a sample size of 30 providers and 1200 patients (i.e., 40 per provider).

We compared descriptors for provider and patient participants by training condition. We calculated the proportion of patients who received an interview for each provider (# interviews by provider or CL/# study-enrolled patients assigned to provider) and mean proportion who received an interview across conditions using the Kruskal-Wallis test, the non-parametric version of ANOVA. We used the Dwass, Steel, Critchlow-Fligner multiple comparison procedure to test pairwise differences. We used the same analytic strategy for outcomes of adherence, competence, and percent change talk. We calculated Hedge’s g to estimate effect size due to small and unequal sample sizes.

RESULTS

We screened 42 providers, but four were ineligible due to inability to attend training (n = 2) and changing jobs (n = 2), resulting in 38 randomized to conditions (workshop = 13, apprenticeship = 12, consult = 13) (Fig. 1). The groups were similar demographically (Table 1). The sample included 13.2% physicians, 36.8% PAs, and 50% registered nurses. Providers estimated that half of their patients had current substance misuse. Almost 40% (15/38) of providers did not have 40 opportunities to conduct a motivational interview with study-eligible patients (no condition differences) because they unexpectedly terminated employment, changed administrative roles, or moved to another campus.

Figure 1.

Figure 1

Provider participant CONSORT flow chart. CONSORT = Consolidated Standards of Reporting Trial.

Table 1.

Baseline Characteristics of Provider Participants

Overall (N = 38) Workshop (N = 13) Apprenticeship (N = 12) Consult (N = 13)
N % N % N % N % p value
Sex 0.77
  Female 33 86.8 11 84.6 10 83.3 12 92.3
  Male 5 13.2 2 15.4 2 16.7 1 7.7
Race/ethnicity 0.4
  White, non-Latino 29 76.3 10 76.9 11 91.7 8 61.5
  Black, non-Latino 3 7.9 1 7.7 1 8.3 1 7.7
  Asian, non-Latino 3 7.9 2 15.4 0 0.0 1 7.7
  White, Latino 2 5.3 0 0.0 0 0.0 2 15.4
  Other, non-Latino 1 2.6 0 0.0 0 0.0 1 7.7
Type of clinician 0.96
  Physician’s assistant 14 36.8 5 38.5 4 33.3 5 38.5
  Nurse 19 50.0 6 46.2 6 50.0 7 53.8
  Doctor 5 13.2 2 15.4 2 16.7 1 7.7
Highest degree earned 0.69
  Associate’s 4 10.5 1 7.7 2 16.7 1 7.7
  Bachelor’s 14 36.8 6 46.2 4 33.3 4 30.8
  Master’s 14 36.8 4 30.8 3 25.0 7 53.8
  Doctorate 6 15.8 2 15.4 3 25.0 1 7.7
Mean SD Mean SD Mean SD Mean SD
Age in years 35.3 11.0 39.2 13.2 34.4 8.9 32.2 10.1 0.27
Motivation for MI*
  Interested in learning MI* 8.1 1.7 8.2 1.6 7.8 2.2 8.2 1.2 0.97
  Confident in ability to use MI* 4.6 3.1 4.1 3.5 4.3 3.1 5.4 2.8 0.50
  Likely to use MI* 5.3 3.5 5.1 3.9 5.0 3.7 5.8 3.1 0.87
  Mean motivation score* 6.0 2.3 5.8 2.5 5.7 2.6 6.5 2.0 0.66
Estimated % of patients currently misusing substances 49.6 19.9 49.2 19.3 44.2 19.8 55.0 20.8 0.41

Column percentages may not add up to 100% due to rounding

*Motivation items were measured on a scale from 0 (not at all) to 10 (extremely)

p values are from Fisher’s exact test for categorical variables and the Kruskal-Wallis test for continuous variables

We approached 9475 patients for screening and enrolled 1173 (Fig. 2). Main reasons for ineligibility included neurocognitive disorder/deficit (24.5%), no recent substance misuse (16.5%), overly medically complicated (15.3%), or disinterest in study participation (11.9%). Patient participants fell into the following conditions: (1) workshop (n = 336), (2) apprenticeship (n = 379), and (3) consult (n = 458). This did not differ significantly by condition (mean (SD) = 25.8 (16.9) for workshop; 31.6 (12.9) for apprenticeship; and 35.2 (7.7) for consult, ANOVA F value = 1.71, p = 0.196). Patients were demographically similar across conditions (Table 2). Most patients indicated alcohol (40.5%) or nicotine (39.5%) as their primary substance problem, which they used on average 18.4 (10.1) days/month. Criterion for disordered use was highest for these substances (alcohol = 46.5% and nicotine = 67.7%); they indicated low to moderate addiction severity and moderate interest in changing substance use. Patients had a variety of medical conditions; as a group, they were in poor physical and mental health. On average, patients stayed in the hospital 1 week and were treated by their assigned study-enrolled provider for ~ 2 days.

Figure 2.

Figure 2

Patient participant CONSORT flow chart. CONSORT = Consolidated Standards of Reporting Trial.

Table 2.

Baseline Characteristics of Patient Participants

Overall (N = 1173) Workshop (N = 336) Apprenticeship (N = 379) Consult (N = 458)
N % N % N % N % p value#
Race/ethnicity 0.56
  White, non-Latino 665 56.7 192 57.1 224 59.1 249 54.4
  Black, non-Latino 352 30.0 98 29.2 105 27.7 149 32.5
  Latino 144 12.3 44 13.1 47 12.4 53 11.6
  Other, non-Latino 12 1.0 2 0.6 3 0.8 7 1.5
Gender 0.45
  Male 639 54.5 184 54.8 197 52.0 258 56.3
  Female 534 45.5 152 45.2 182 48.0 200 43.7
Primary substance* 0.65
  Alcohol 475 40.5 137 40.8 159 42.0 179 39.1
  Nicotine 463 39.5 137 40.8 148 39.1 178 38.9
  Opioids 86 7.3 24 7.1 29 7.7 33 7.2
  Cannabis 64 5.5 14 4.2 20 5.3 30 6.6
  Cocaine 50 4.3 17 5.1 10 2.6 23 5.0
  Sedatives 31 2.6 7 2.1 12 3.2 12 2.6
  Other 4 0.3 0 0.0 1 0.3 3 0.7
Substance use disorders
  Alcohol 545 46.5 157 46.7 174 45.9 214 46.7 0.97
  Nicotine 794 67.7 234 69.6 247 65.2 313 68.3 0.41
  Opioids 142 12.1 41 12.2 49 12.9 52 11.4 0.78
  Cannabis 164 14.0 40 11.9 56 14.8 68 14.8 0.43
  Cocaine 131 11.2 44 13.1 33 8.7 54 11.8 0.15
  Sedatives 72 6.1 21 6.3 26 6.9 25 5.5 0.70
  Other 22 1.9 5 1.5 9 2.4 8 1.7 0.66
HCUP disease category
  Mental illness 363 30.9 101 30.1 115 30.3 147 32.1 0.79
  Undefined symptoms 337 28.7 91 27.1 116 30.6 130 28.4 0.57
  Circulatory system 311 26.5 80 23.8 94 24.8 137 29.9 0.10
  Respiratory system 281 24.0 83 24.7 91 24.0 107 23.4 0.91
  Digestive system 240 20.5 58 17.3 76 20.1 106 23.1 0.12
  Endocrine, nutritional, metabolic diseases/immunity disorders 226 19.3 57 17.0 73 19.3 96 21.0 0.37
  Infectious/parasitic 157 13.4 35 10.4 53 14.0 69 15.1 0.15
  Genitourinary system 148 12.6 39 11.6 44 11.6 65 14.2 0.43
  Musculoskeletal system 122 10.4 30 8.9 38 10.0 54 11.8 0.41
  Skin/subcutaneous tissue 119 10.1 31 9.2 34 9.0 54 11.8 0.33
  Nervous system 117 10.0 31 9.2 37 9.8 49 10.7 0.78
  Injury and poisoning 98 8.4 30 8.9 32 8.4 36 7.9 0.86
  Blood/blood forming organs 92 7.8 23 6.8 31 8.2 38 8.3 0.72
  Neoplasms 24 2.0 7 2.1 8 2.1 9 2.0 1.00
  Residual codes 15 1.3 3 0.9 4 1.1 8 1.7 0.51
  Pregnancy complications 8 0.7 2 0.6 3 0.8 3 0.7 0.95
  Congenital anomalies 2 0.2 0 0.0 0 0.0 2 0.4 0.21
Mean SD Mean SD Mean SD Mean SD
Age in years 46.7 14.3 46.8 14.0 46.3 14.5 46.9 14.5 0.86
Days with provider 1.9 1.4 1.8 1.5 1.8 1.0 2.1 1.6 < 0.01
Length of stay in days 6.9 7.2 6.7 8.1 6.8 6.7 6.9 6.9 0.90
Addiction severity
  Cigarettes 2.1 1.6 2.0 1.6 2.1 1.5 2.3 1.6 0.40
  Alcohol§ 0.4 0.3 0.4 0.3 0.4 0.3 0.4 0.3 0.24
  Other drugs§ 0.2 0.1 0.2 0.2 0.2 0.2 0.2 0.1 0.19
SF-12 scoresǁǁ
  Physical component 36.2 11.3 36.9 11.2 36.4 11.3 35.7 11.4 0.34
  Mental component 42.4 12.6 41.8 12.6 42.4 12.9 42.8 12.4 0.54
Motivation for change score
  Importance 7.3 3.5 7.6 3.3 7.1 3.6 7.3 3.4 0.16
  Ability 6.8 3.2 6.7 3.1 7.0 3.2 6.8 3.3 0.43
  Commitment 6.9 3.6 7.1 3.5 6.9 3.7 6.9 3.6 0.58

HCUP Healthcare Cost and Utilization Project database used to characterize patients’ medical diagnoses

*The substance the patient identified as most problematic

Substance use disorder was determined using DSM 5 criteria from the Mini-International Neuropsychiatric Inventory

Addiction severity for cigarettes was measured with the Heaviness of Smoking Index, with scores interpreted as follows: 1–2 = low addiction; 3–4 = moderate addiction; 5–6 = high addiction. HSI scores represent average among patients identifying nicotine as primary drug

§Addiction severity for alcohol and other drugs (illicit and prescription medications) was measured with the Addiction Severity Index composite scores, ranging from 0 (no problems) to 1 (severe problems). Alcohol and other drug scores represent average among patients identifying alcohol and other drug, respectively, as primary drug

ǁǁThe 12-item Short Form Survey (SF-12) provides component scores for physical and mental health ranging from 0 (lowest level of health) to 100 (highest level of health)

Motivation for change items were rated from 0 to 10, with higher ratings indicative of higher motivation

#p values are from chi-squared test for categorical variables and ANOVA F test for continuous variables

Providers used motivational interviewing with an average of 8.2% of their patients. The mean percentage of interviews was highest in the consult group with 20.5% (SD = 17.9) of eligible patients, compared to 0.8% (SD = 2.3) in workshop and 3.0% (SD = 5.4) in apprenticeship groups. The Kruskal-Wallis test showed these percentages differed significantly by condition (chi-square with 2 degrees of freedom = 23.71, p < 0.001), and post hoc pairwise comparison tests reveal that consult was significantly higher than apprenticeship (p < 0.001, Hedge’s g = 1.26) and workshop (p < 0.001, Hedge’s g = 1.49), while workshop and apprenticeship did not differ significantly (p = 0.50, Hedge’s g = 0.52). There were no significant differences in the mean percentage of completed motivational interviews by provider type (i.e., physician, PA, or nurse) across (p = 0.38) or within (p values > 0.28) conditions.

All interviews in the consult condition were conducted by CL. Of the 116 orders placed, CL interviewed 100 patients (86.2%) before discharge. Every provider in the consult condition placed at least one order, but the percentage of patients who received an interview varied from < 3 to 65% (Fig. 3). Only 2/13 providers in workshop (15.4%) and 4/12 providers in apprenticeship (33.3%) conditions conducted a motivational interview with study-enrolled patients.

Figure 3.

Figure 3

Number of completed motivational interviews out of total eligible sessions for each provider by condition.

Motivational interviews conducted by CL clinicians had significantly higher mean fundamental adherence and competence scores than those conducted by providers in the other two conditions (Table 3), and their interviews were longer on average (24.2 versus 16.4 and 10.5 min for apprenticeship and workshop, respectively). We did not find significant pairwise differences in mean advanced adherence or competence scores between groups. There were no significant group differences in patient change talk (i.e., 69–81%) for patients who received a motivational interview.

Table 3.

Motivational Interview Length, Adherence and Competence Scores, and Percent Change Talk

Workshop (N = 3) Apprentice (N = 11) Consult (N = 100) Consult versus workshop Consult versus apprenticeship
Mean SD Mean SD Mean SD Hedge’s g Hedge’s g
Interview length (in min) 10.5* 6.2 16.4 8.7 24.2*,† 8.3 1.65 0.93
Fundamental adherence 4.0* 0.4 4.6 0.8 5.2*,† 0.6 2.00 0.96
Fundamental competence§ 3.7* 0.6 4.3 0.4 4.7*,† 0.5 1.98 0.81
Advanced adherence 3.9 0.6 4.4 1.0 4.8 0.7 1.28 0.54
Advanced competence§ 3.9 0.9 4.3 0.5 4.5 0.5 1.17 0.40
Percent change talkǁǁ 68.9 18.3 80.9 9.8 76.6 16.4 0.47 − 0.26

Inter-rater reliability based on intra-class correlation coefficients (ICC) was good to very good for fundamental and advanced adherence and competence scores (ICCs ranged from 0.71 to 0.85) and excellent for percent change talk (0.96)58

Means that share a superscript (*,†) are significantly different at p < 0.05 based upon results of the Kruskal-Wallis test with the Dwass, Steel, Critchlow-Fligner method for post hoc pairwise comparisons

Adherence rated from 1 (not at all) to 7 (extremely)

§Competence rated from 1 (very poor) to 7 (outstanding)

ǁǁPercent change talk > 50% indicates most patient statements in sessions favored reducing primary substance use

DISCUSSION

Reliance on CL for providers to offer motivational interviewing to patients who misuse substances was more effective than training providers to conduct the interviews themselves. Providers in the consult condition placed an interview order for a quarter of their patients with substance misuse, and CL clinicians reached 86% of these consult requests. In contrast, providers receiving workshop training, with or without subsequent bedside supervision, rarely used motivational interviewing with their patients (on average 0.8% and 3%, respectively, for workshop and apprenticeship conditions). When conducted, CL clinicians held longer sessions and deployed fundamental skills of motivational interviewing more often and with greater competence than other medical providers, though not with significantly more advanced adherence and competence. Across conditions, an in-session patient language predictor of treatment outcome (percent change talk)15, 16 indicated most patient statements expressed during interviews favored reducing substance misuse, suggesting the motivational intervention, when conducted by providers or CL clinicians, likely had a positive treatment effect.

The superiority of using CL to increase delivery of motivational interviews is notable. Medically hospitalized patients who misuse substances are frequently nonadherent with discharge plans59 and at heightened risk for readmission.60 Moreover, the opioid epidemic is driving an increase in medical hospitalizations.6163 In response, hospitals are establishing CL services to address patients’ substance misuse and provide a full spectrum of services, including motivational counseling and medication assisted therapy.6466 A medical team member might simply state that the reason for the consult is a “motivational interview.”

While our study supports the superiority of this approach over individual training, the number of patients overall who did not receive a motivation interview was surprising. Several reasons for the low use of motivational interviewing may exist. First, providers may not have detected patients who misused substances. Substance misuse is commonly undetected among medical inpatients,67 especially when the severity of symptoms is low as it was with our patients. Given the success of enhancing motivation, as shown through the change talk findings, this is a missed opportunity. Second, providers may have identified their patients with substance misuse but not used motivational interviewing because they felt it was too time-consuming to perform or outside of their scope of practice.68 Third, substance misuse is a sensitive issue, and some providers may have felt that a referral or even discussion with a patient about their use of substances would adversely affected their treatment alliance with the patient. Fourth, providers were very interested in learning motivational interviewing, but they typically had only ~ 2 days to care for a patient, suggesting lack of contact with the patient might have been a contributing factor. Nurses typically spend more time with a patient on any given day and this may help with time, but they typically are only assigned a given patient for a day. We did not address the issue of care continuity and other potent organizational barriers (e.g., scheduling, caseloads) or the implementation climate (e.g., hospital or unit-based administrative support and reward for using MI) that could have affected the providers’ capacity to deliver or order an interview.6971

There were group differences in the length of the interviews and delivery of fundamental skills in the motivational interviews, although there were no differences between groups in advanced motivational interviewing skills. These findings should be interpreted cautiously given the low number of interviews conducted by providers in the workshop and apprenticeship conditions. However, it is interesting that interviews across conditions led to the same proportion of change talk. While research has yet to establish how long a well-conducted motivational interview must be (e.g., 24 versus 16 and 10 min for consult, apprenticeship, and workshop, respectively) to achieve positive treatment outcomes,15, 16 medical providers trained in motivational interviewing may have as great an impact as expert consultants when they elect to conduct interviews with their patients who misuse substances.

This study has several limitations. It was conducted across medical units within one academically affiliated large urban hospital and only with English-speaking and hearing-able patients, thereby reducing the generalizability of the findings. Logistical constraints prohibited blind collection of recorded interviews, though that would not have affected the number of interviews completed. Additionally, blind independent raters determined interview integrity and percent change talk. Finally, we did not follow patients’ substance use or its effect on related health outcomes following discharge from the hospital, so determination of the actual effectiveness of motivational interviewing delivered in this study is unknown.

Despite these limitations, this study shows the potential of using CL to provide motivational interviewing for patients who present with substance misuse. A CL platform can be used for behavioral management, as well as medication management, in acute hospitalized settings. Future research should determine optimal ways of detecting substance misuse, the cost-effectiveness of CL, organizational factors that influence the use of CL, and the effectiveness of CL for a broader range of behavioral interventions for substance misuse in this patient population.

Acknowledgments

The authors wish to thank the medical providers at the medical hospital where the study was conducted. We also would like to thank Victor Morris, Will Cushing, Jose Salvana, and Scott Sussman for the work they have done in support of the project. Finally, we want to acknowledge our research staff (Rebecca Aldi, Monica Canning-Ball, Heather Howell, Jessica Katon, Ashley McCaherty, and Jonathan Ryan) for their efforts to implement this trial with integrity.

Funding Information

The National Institute on Drug Abuse funded the study (R01 DA034243). The content of this manuscript solely reflects the authors’ views and not those of the funding agency or the authors’ institutional affiliates.

Data Availability

The study protocol, statistical code, and data set are available to approved persons through agreement with the authors (e-mail Drs. Martino and Yonkers, steve.martino@yale.edu; kimberly.yonkers@yale.edu).

Compliance with Ethical Standards

The Yale University Human Investigation Committee approved the study (ClinicalTrials.gov (NCT01825057). We screened interested providers and obtained written informed consent alongside baseline assessments. Research assistants (RAs) screened and obtained written informed consent from eligible and interested patients.

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Footnotes

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Stein MD, O’Sullivan PS, Ellis P, Perrin H, Wartenberg A. Utilization of medical services by drug users in detoxification. J Subst Abus. 1993;10:187–93. doi: 10.1016/0899-3289(93)90062-g. [DOI] [PubMed] [Google Scholar]
  • 2.Richmond RL, Bell AP, Rollnick S, Heather BBN. Screening for smokers in four Sydney teaching hospitals. J Cardiovasc Risk. 1995;2:199–203. [PubMed] [Google Scholar]
  • 3.Katz A, Goldberg D, Smith J, Trick WE. Tobacco, alcohol, and drug use among hospital patients: concurrent use and willingness to change. J Hosp Med. 2008;3:370–5. doi: 10.1002/jhm.358. [DOI] [PubMed] [Google Scholar]
  • 4.Orford J, Somers M, Daniels V, Kirby B. Drinking amongst medical patients: levels of risk and models of change. Br J Addict. 1992;87:1691–1702. doi: 10.1111/j.1360-0443.1992.tb02682.x. [DOI] [PubMed] [Google Scholar]
  • 5.Santora PB, Hutton HE. Longitudinal trends in hospital admissions with co-occurring alcohol/drug diagnoses, 1994–2002. J Subst Abus Treat. 2008;35:1–12. doi: 10.1016/j.jsat.2007.08.002. [DOI] [PubMed] [Google Scholar]
  • 6.Smothers BA, Yahr HT. Alcohol use disorder and illicit drug use in admissions to general hospitals in the United States. Am J Addict. 2005;14:256–67. doi: 10.1080/10550490590949433. [DOI] [PubMed] [Google Scholar]
  • 7.Smothers BA, Yahr HT, Sinclair MD. Prevalence of current DSM-IV alcohol use disorders in short-stay, general hospital admissions, United States. Arch Intern Med. 1994;163:713–9. doi: 10.1001/archinte.163.6.713. [DOI] [PubMed] [Google Scholar]
  • 8.Saitz R, Freedner N, Palfai TP, Horton NJ, Samet JH. The severity of unhealthy alcohol use in hospitalized medical patients. The spectrum is narrow. J Gen Intern Med. 2006;21:381–5. doi: 10.1111/j.1525-1497.2006.00405.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Stinson FS, Grant BF, Dawson DA, Ruan WJ, Huang B, Saha T. Comorbidity between DSM–IV alcohol and specific drug use disorders in the United States: results from the National Epidemiologic Survey on alcohol and related conditions. Alcohol Drug Dep. 2005;80:105–16. doi: 10.1016/j.drugalcdep.2005.03.009. [DOI] [PubMed] [Google Scholar]
  • 10.Mitka M. “Teachable moments” provide a means for physicians to lower alcohol abuse. JAMA. 1998;279:1767–8. doi: 10.1001/jama.279.22.1767. [DOI] [PubMed] [Google Scholar]
  • 11.Saitz R, Palfai T, Cheng DM, et al. Brief intervention for medical inpatients with unhealthy alcohol use. Ann Intern Med. 2007;146:167–76. doi: 10.7326/0003-4819-146-3-200702060-00005. [DOI] [PubMed] [Google Scholar]
  • 12.Miller WR, Rollnick S. Motivational Interviewing: Helping People Change Third Edition. New York: Guilford Press; 2013. [Google Scholar]
  • 13.Hettema JE, Hendricks PS. Motivational interviewing for smoking cessation: a meta-analytic review. J Consult Clin Psychol. 2010;78:868–84. doi: 10.1037/a0021498. [DOI] [PubMed] [Google Scholar]
  • 14.Lundahl BW, Kunz C, Brownell C, Tollefson D, Burke B. Meta-analysis of motivational interviewing: twenty five years of empirical studies. Res Soc Work Pract. 2010;20:137–60. [Google Scholar]
  • 15.Magill M, Gaume J, Apodaca TR, Walthers J, Mastroleo NR, Borsari B, et al. The technical hypothesis of motivational interviewing: A meta-analysis of MI’s key causal model. J Consult Clin Psychol. 2014;82:873–83. doi: 10.1037/a0036833. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Magill M, Apodaca TR, Borsari B, Gaume J, Hoadley A, Gordon REF, Tonigan JS, Moyers T. A meta-analysis of motivational interviewing process: Technical, relational, and conditional process models of change. J Consult Clin Psychol. 2018;86:140–57. doi: 10.1037/ccp0000250. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Soderlund LL, Madson MB, Rubak S, Nilsen P. A systematic review of motivational interviewing training for general health care practititoners. Pat Educ Counsel. 2011;84:16–26. doi: 10.1016/j.pec.2010.06.025. [DOI] [PubMed] [Google Scholar]
  • 18.Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82:581–629. doi: 10.1111/j.0887-378X.2004.00325.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Grol RP, Bosch MC, Hulscher ME, Eccles MP, Wensing M. Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q. 2007;85:93–138. doi: 10.1111/j.1468-0009.2007.00478.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Rogers E. Diffusion of Innovations. 5. New York: Free Press; 2003. [Google Scholar]
  • 22.Seale JP, Shellenberger SB, Tillery, et al. Implementing alcohol screening and intervention in a family medicine residency clinic. Subst Abus. 2005;26:23–31. doi: 10.1300/j465v26n01_03. [DOI] [PubMed] [Google Scholar]
  • 23.Davis DA, Thomson MA, Oxman AD, Haynes B. Evidence for the effectiveness of CME. JAMA. 1992;268:1111–7. [PubMed] [Google Scholar]
  • 24.Davis D. Does CME work? An analysis of the effect of educational activities on physician performance or health care outcomes. Int J Psychiatric Med. 1998;28:21–39. doi: 10.2190/UA3R-JX9W-MHR5-RC81. [DOI] [PubMed] [Google Scholar]
  • 25.Walters ST, Matson SA, Baer JS, Ziedonis DM. Effectiveness of workshop training for psychosocial addiction treatments: A systematic review. J Subst Abus Treat. 2005;29:283–93. doi: 10.1016/j.jsat.2005.08.006. [DOI] [PubMed] [Google Scholar]
  • 26.de Roten Y, Zimmermann G, Ortega D, Delpland J. Meta-analysis of the effects of MI training on clinician behavior. J Subst Abus Treat. 2013;45:155–62. doi: 10.1016/j.jsat.2013.02.006. [DOI] [PubMed] [Google Scholar]
  • 27.Schwalbe CS, Oh HY, Zweben A. Sustaining motivational interviewing: A meta-analysis of training studies. Addiction. 2014;109:1287–94. doi: 10.1111/add.12558. [DOI] [PubMed] [Google Scholar]
  • 28.Edwards KS, Woolf PK, Hetzler TH. Pediatric residents as learners and teachers in evidence-based medicine. Acad Med. 2002;77:748. doi: 10.1097/00001888-200207000-00037. [DOI] [PubMed] [Google Scholar]
  • 29.Greif R, Eggers L, Basciani RM, Lockey A, Vogt A. Emergency skill training – a randomized controlled study on the effectiveness of the 4-stage approach to traditional clinical teaching. Resuscitation. 2010;81:1692–97. doi: 10.1016/j.resuscitation.2010.09.478. [DOI] [PubMed] [Google Scholar]
  • 30.Rodriguez-Paz JM, Kennedy M, Salas E, et al. Beyond “see one, do one, teach one”: toward a different training paradigm. Qual Saf Health Care. 2009;18:63–8. doi: 10.1136/qshc.2007.023903. [DOI] [PubMed] [Google Scholar]
  • 31.Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clin Psychol Rev. 2010;30:448–66. doi: 10.1016/j.cpr.2010.02.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Beidas RS, Edmunds JM, Marcus SC, Kendall PC. Training and consultation to promote implementation of an empirically supported treatment: a randomized trial. Psychiatr Serv. 2012;63:660–665. doi: 10.1176/appi.ps.201100401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Dorsey S, Pullman MD, Deblinger E, Berliner L, Kerns SE, Thompson K, et al. Improving practice in community-based settings: a randomized trial of supervision – study protocol. Implement Sci. 2013;8:89. doi: 10.1186/1748-5908-8-89. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Huyse FJ, Herzog T, Lobo A, et al. Consultation-liaison psychiatry service delivery: results from a European study. Gen Hosp Psychiatry. 2001;23:124–31. doi: 10.1016/s0163-8343(01)00139-6. [DOI] [PubMed] [Google Scholar]
  • 35.Bourgeois JA, Wegelin JA, Servis ME, Hales RE. Psychiatric diagnoses of 901 inpatients seen by consultation-liaison psychiatrists at an academic medical center in a managed care environment. Psychosomatics. 2005;46:47–57. doi: 10.1176/appi.psy.46.1.47. [DOI] [PubMed] [Google Scholar]
  • 36.Clarke DM, Smith GC. Consultation-liaison psychiatry in general medical units. Aust N Z J Psychiatry. 1995;29:424–32. doi: 10.3109/00048679509064950. [DOI] [PubMed] [Google Scholar]
  • 37.Loewenstein RJ, Sharfstein SS. Psychiatric consultations at the NIMH. Gen Hosp Psychiatry. 1983;5:83–7. doi: 10.1016/0163-8343(83)90104-4. [DOI] [PubMed] [Google Scholar]
  • 38.McKegney FP, McMahon T, King J. The use of DSM-III in a general hospital consultation-liaison service. Gen Hosp Psychiatry. 1983;5:115–21. doi: 10.1016/0163-8343(83)90110-x. [DOI] [PubMed] [Google Scholar]
  • 39.Ormont MA, Weisman HW, Heller SS, Najara JE, Shindledecker RD. The timing of psychiatric consultation requests: utilization, liaison, and diagnostic considerations. Psychosomatics. 1997;38:38–44. doi: 10.1016/S0033-3182(97)71502-0. [DOI] [PubMed] [Google Scholar]
  • 40.Duffy SA, Ronis DL, Karvonen-Gutierrez CA, Ewing LA, Hall SV, Yang JJ, et al. Effectiveness of the Tobacco Tactics Program in the Trinity Health System. Am J Prev Med. 2016;51:551–65. doi: 10.1016/j.amepre.2016.03.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Tew J, Klaus J, Oslin DW. The behavioral health laboratory: building a stronger foundation for the patient-centered medical home. Fam Syst Health. 2010;28:130–45. doi: 10.1037/a0020249. [DOI] [PubMed] [Google Scholar]
  • 42.Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50:217–26. doi: 10.1097/MLR.0b013e3182408812. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Urbaniak GC, Plous S. Research Randomizer (Version 4.0) [Computer software]. 2013. Retrieved on June 22, 2013, from http://www.randomizer.org/
  • 44.Baer JS, Ball SA, Campbell BK, Miele GM, Schoener EP, Tracy K. Training and fidelity monitoring of behavioral interventions in multi-site addictions research: a review. Drug Alcohol Depend. 2007;87:107–18. doi: 10.1016/j.drugalcdep.2006.08.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Martino S, Ball SA, Nich C, Canning-Ball M, Rounsaville BJ, Carroll KM. Teaching community program clinicians motivational interviewing using expert and train-the-trainer strategies. Addiction. 2011;106:428–41. doi: 10.1111/j.1360-0443.2010.03135.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Sheehan D, Lecrubier Y, Janavs J, Knapp E, Weiller E. Mini International Neuropsychiatric Interview (MINI) Tampa: University of South Florida; 1996. [Google Scholar]
  • 47.Brown RL, Rounds LA. Conjoint screening questionnaires for alcohol and other drug abuse: criterion validity in a primary care practice. WMJ. 1995;94:135–40. [PubMed] [Google Scholar]
  • 48.Heatherton TF, Kozlowski LT, Frecker RC, Rickert W, Robinson J. Measuring the Heaviness of Smoking: Using self-reported time to the first cigarette of the day and number of cigarettes smoked per day. Br J Addict. 1989;84:791–799. doi: 10.1111/j.1360-0443.1989.tb03059.x. [DOI] [PubMed] [Google Scholar]
  • 49.Martino S, Ball SA, Nich C, Frankforter TL, Carroll KM. Community program therapist adherence and competence in motivational enhancement therapy. Drug Alcohol Depend. 2008;96:37–48. doi: 10.1016/j.drugalcdep.2008.01.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Miller WR, Moyers TB, Ernst D, Amrhein P: Manual for the Motivational Interviewing Skills Code (MISC) v. 2.1. 2008. http://casaa.unm.edu/download/misc.pdf. Accessed 26 June 2019.
  • 51.Ball SA, Bachrach K, DeCarlo J, et al. Characteristics, beliefs, and practices of community clinicians trained to provide manual-guided therapy for substance abusers. J Subst Abus Treat. 2002;23:309–18. doi: 10.1016/s0740-5472(02)00281-7. [DOI] [PubMed] [Google Scholar]
  • 52.Decker SE, Martino S. Unintended effects of training on clinicians’ interest, confidence, and commitment in using motivational interviewing. Drug Alcohol Depend. 2013;132:681–7. doi: 10.1016/j.drugalcdep.2013.04.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.McLellan AT, Kushner H, Metzger D, et al. The fifth edition of the Addiction Severity Index. J Subst Abus Treat. 1992;9:199–213. doi: 10.1016/0740-5472(92)90062-s. [DOI] [PubMed] [Google Scholar]
  • 54.Kroenke K, Spitzer R, William W. The PHQ-9: validity of a brief depression severity measure. JGIM. 2001;16:606–16. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Ware J, Kosinski M, Keller S. A 12-Item Short-Form Health Survey: Construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34:220–33. doi: 10.1097/00005650-199603000-00003. [DOI] [PubMed] [Google Scholar]
  • 56.Miller WR, Johnson WR. A natural language screening measure for motivation for change. Addict Behav. 2008;33:1177–82. doi: 10.1016/j.addbeh.2008.04.018. [DOI] [PubMed] [Google Scholar]
  • 57.Imel ZE, Baer JS, Martino S, Ball SA, Carroll KM. Mutual influence in therapist competence and adherence to motivational enhancement therapy. Drug Alcohol Depend. 2011;15:229–36. doi: 10.1016/j.drugalcdep.2010.11.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6:284–90. [Google Scholar]
  • 59.Gonzalez A, Barinas J, O’Cleirigh C. Substance use: impact on adherence and HIV medical treatment. Curr HIV/AIDS Rep. 2011;8:223. doi: 10.1007/s11904-011-0093-5. [DOI] [PubMed] [Google Scholar]
  • 60.Ahmedani BK, Solberg LI, Copeland LA, et al. Psychiatric comorbidity and 30-day readmissions after hospitalization for heart failure AMI, and pneumonia. Psychiatr Serv. 2015;66:134–40. doi: 10.1176/appi.ps.201300518. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Hsu DJ, McCarthy EP, Stevens JP, Mukamai KJ. Hospitalizations, costs and outcomes associated with heroin and prescription opioid overdoses in the United States 2001-12. Addiction. 2017;112:1558–64. doi: 10.1111/add.13795. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Ronan MV, Herzig SJ. Hospitalizations related to opioid abuse/dependence and associated serious infections increased sharply, 2002-12. Health Aff. 2016;35:832–7. doi: 10.1377/hlthaff.2015.1424. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Wurcel Alysse G., Anderson Jordan E., Chui Kenneth K. H., Skinner Sally, Knox Tamsin A., Snydman David R., Stopka Thomas J. Increasing Infectious Endocarditis Admissions Among Young People Who Inject Drugs. Open Forum Infectious Diseases. 2016;3(3):ofw157. doi: 10.1093/ofid/ofw157. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Trowbridge P, Weinstein ZM, Kerensky T, et al. Addiction consultation services-linking hospitalized patients to outpatient addiction treatment. J Subst Abus Treat. 2017;79:1–5. doi: 10.1016/j.jsat.2017.05.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Wakeman SE, Metlay JP, Chang Y, Herman GE, Rigotti NA. Inpatient addiction consultation for hospitalized patients increases post-discharge abstinence and reduces addiction severity. J Gen Intern Med. 2017;32:909–16. doi: 10.1007/s11606-017-4077-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Nordeck CD, Welsh C, Schwartz RP, et al. Rehospitalization and substance use disorder (SUD) treatment entry among patients seen by a hospital SUD consultation-liaison service. Drug Alcohol Depend. 2018;186:23–8. doi: 10.1016/j.drugalcdep.2017.12.043. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Smothers BA, Yahr HT, Ruhl CE. Detection of alcohol use disorders in general hospital admissions in the United States. Arch Intern Med. 2004;164:749–56. doi: 10.1001/archinte.164.7.749. [DOI] [PubMed] [Google Scholar]
  • 68.Rollnick S, Miller W, Butler CC. Motivational Interviewing in Health Care: Helping Patients Change Behavior. New York: Guilford Press; 2008. [Google Scholar]
  • 69.Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop strategic climate to evidence-based practice implementation. Annu Rev Public Health. 2014;35:255–74. doi: 10.1146/annurev-publhealth-032013-182447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Beidas RS, Marcus S, Aarons GA, Hoagwood KE, Schoenwald S, Evans AC, et al. Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatr. 2015;169:374–82. doi: 10.1001/jamapediatrics.2014.3736. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Dorsey S, Pullmann MD, Kerns SEU, Jungbluth N, Meza R, Thompson K, et al. The juggling act of supervision in community mental health: Implications for supporting evidence-based treatment. Adm Policy Ment Health Ment Health Serv Res. 2017;44:838–52. doi: 10.1007/s10488-017-0796-z. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The study protocol, statistical code, and data set are available to approved persons through agreement with the authors (e-mail Drs. Martino and Yonkers, steve.martino@yale.edu; kimberly.yonkers@yale.edu).


Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine

RESOURCES