Skip to main content
Health Services Research logoLink to Health Services Research
. 2017 May 31;53(3):1702–1726. doi: 10.1111/1475-6773.12716

Quality Metrics and Systems Transformation: Are We Advancing Alcohol and Drug Screening in Primary Care?

Traci Rieckmann 1,, Stephanie Renfro 2, Dennis McCarty 1, Robin Baker 1, K John McConnell 2
PMCID: PMC5980159  PMID: 28568245

Abstract

Objective

To examine the influence of Oregon's coordinated care organizations (CCOs) and pay‐for‐performance incentive model on completion of screening and brief intervention (SBI) and utilization of substance use disorder (SUD) treatment services.

Data Sources/Study Setting

Secondary analysis of Medicaid encounter data from 2012 to 2015 and semiannual qualitative interviews with stakeholders in CCOs.

Study Design

Longitudinal mixed‐methods design with simultaneous data collection with equal importance.

Data Collection/Extraction Methods

Qualitative interviews were recorded, transcribed, and coded in ATLAS.ti. Quantitative data included Medicaid encounters 30 months prior to CCO implementation, a 6‐month transition period, and 30 months following CCO implementation. Data were aggregated by half‐year with analyses restricted to Medicaid recipients 18–64 years of age enrolled in a CCO, not eligible for Medicare coverage or Medicaid expansion.

Principal Findings

Quantitative analysis documented a significant increase in SBI rates coinciding with CCO implementation (0.1 to 4.6 percent). Completed SBI was not associated with increased initiation in treatment for SUD diagnoses. Qualitative analysis highlighted importance of aligning incentives, workflow redesign, and leadership to facilitate statewide SBI.

Conclusions

Results provide modest support for use of a performance metric to expand SBI in primary care. Future research should examine health reform efforts that increase initiation and engagement in SUD treatment.

Keywords: Evidence‐based practice, primary care, state health policies, substance abuse


The Patient Protection and Affordable Care Act (ACA) encourages transformation of behavioral health services within the health care system at large (Mechanic 2012). Integrating such services within primary care supports an ideological shift toward treating substance use disorders (SUDs) as legitimate health care concerns. It normalizes behavioral health as an essential element of health care (McCance‐Katz and Satterfield 2012; Shim and Rust 2013) and promotes earlier recognition of problems rather than waiting for them to manifest at a more severe and recalcitrant point in the course of the disorder.

Oregon, supported by funding from the Centers for Medicare & Medicaid Services (CMS), offers one model for consideration that prioritizes and incentivizes integration of SUD services into primary care. Promising to reduce annual health care spending growth from 5.4 to 3.4 percent, the state built on the Oregon Health Plan (the state's Medicaid system), authorizing and approving coordinated care organizations (CCOs)—a version of accountable care organizations. Sixteen CCOs integrate financing and service delivery for medical, behavioral, and dental health. Savings are projected to total $11 billion over 10 years, and initial results are encouraging; enrollment in the Oregon Health Plan reached 1.1 million people in 2015, while primary care visits increased and emergency room visits decreased (Chang et al. 2014; Howard et al. 2014; OHA 2015c).

Screening, Brief Intervention, and Referral to Treatment

Within the context of reform and integration of SUD care, policymakers and health care leaders identified Screening, Brief Intervention, and Referral to Treatment (SBIRT) as an efficacious intervention (Agerwala and McCance‐Katz 2012). The U.S. Preventive Services Taskforce recommends routine screening for alcohol misuse for adults aged 18 and older (Jonas et al. 2012; Moyer 2013). However, an analysis of screening for illicit drug use found insufficient evidence to support recommendation (Polen et al. 2008). Despite the need for additional comparative effectiveness evidence to support the use of SBIRT for drug use, several states are implementing or have already implemented it as an intervention to integrate SUD services into primary care.

People with SUDs often suffer from comorbid medical problems, generate expensive readmissions, and misuse hospital services (Brick 2012; Neighbors et al. 2013). An estimated 24.5 million Americans aged 12 and older were current illicit drug users in 2013 and 60.1 million individuals were past‐month binge drinkers (SAMHSA 2014). They create a tremendous cost burden on the health care system and the nation as a whole (Bouchery et al. 2011; NDIC, 2011; Sacks et al. 2015). Health care costs for individuals with SUDs are nearly twice as high in comparison with individuals without SUDs (Clark, Samnaliev, and McGovern 2009; Boyd et al. 2010). Many patients use the emergency department for the treatment of conditions related to SUDs; however, primary care may be a more suitable point of intervention because it could “maximize teachable moments.” Patients are more likely to see their primary care physicians at varying points while symptomatic for an SUD than at the emergency department, which becomes a point of screening and intervention only during active crisis (Krupski et al. 2012). Thus, the opportunity to identify and engage patients earlier has the potential to shift care to a lower cost, more effective point in the course of a patient's struggle with an SUD. The average cost to provide SBIRT per positive screen, for one year, is estimated to be about $400 (Barbosa et al. 2016), but some studies suggest early intervention and treatment can bend the cost curve (Estee et al. 2010; Barbosa et al. 2015; Pringle, Aldridge, and Kearney 2015).

Research and experience confirm that overall uptake of new interventions is delayed and providers rarely rush to adopt new practices, even with sufficient evidence of efficacy (Lewis et al. 2014). The Consolidated Framework of Implementation Research (CFIR), which acknowledges that the process through which interventions are adopted is complex and impacted by multiple factors (Greenhalgh et al. 2004; Damschroder et al. 2009; Damschroder and Hagedorn 2011), helps us understand these delays. In addition, research has identified that core barriers for SBIRT uptake include difficulty of use in real‐world clinical settings, failure of education and dissemination efforts, and inadequate systems transformation and implementation strategies (Babor et al. 2007; Désy and Perhats 2008; Tetrault et al. 2012). As more health care systems adopt SBIRT, research is necessary to monitor and assess the efficacy of SBIRT as a mechanism for health care transformation.

CCO and Incentive Metrics

In approving Oregon's current 1115 Medicaid waiver, CMS held Oregon accountable for 33 quality and access metrics to ensure that cost savings are not achieved by downgrading quality or withholding needed care. In addition, 17 metrics were identified as CCO incentive metrics (16 of the 17 were included in the 33 quality and access metrics). Following a pay‐for‐performance model, OHA established a quality pool as required by the special terms and conditions of Oregon's Section 1115 demonstration (OHA, 2016). CCOs have the opportunity to receive pay‐for‐performance awards based on performance across all incentive metrics. Quality pool funds increase over time as a percentage of aggregate payments made to all CCOs (2 percent in 2013, 3 percent in 2014, 4 percent in 2015, and 4.25 percent in 2016) (OHA, 2013, 2014, 2015a, 2016).

The metric for screening and brief intervention (specifically, adults 18 years of age and older seen in primary care complete a standardized screen for at risk substance use) was selected as one of the incentive metrics (OHA 2015b). To facilitate implementation, interim improvement targets were specified rather than an expectation that the benchmark (12 percent) would be met immediately. In 2015, OHA reported a statewide performance of 12.6 percent—a 73 percent increase over the 2014 rate (7.3 percent). In addition, 14 of 16 CCOs met the interim improvement target (OHA 2016).

Driven by CMS innovation funding and a unique pay‐for‐performance redesign, this study assesses step one of the statewide integration of SUD services—identification and brief intervention within primary care. Employing a multilevel, mixed‐methods longitudinal analysis of administrative claims data and stakeholder and provider interviews, our findings may assist policymakers, providers and administrators with systems redesign, selection of performance measures, and behavioral health integration.

Methods

This study used a QUANT + QUAL design balanced with simultaneous data collection and emphasis of each method (Cresswell 2011; Palinkas et al. 2011). Our outcome variables allowed us to assess and monitor the influence of CCOs on access to and utilization of services related to the prevention and treatment of SUDs. Throughout the remainder of the paper, we use the term SBI rather than SBIRT because the data available and analysis do not include the full details of Referral to Treatment (RT); we hope to clarify that additional research specific to the referral process is necessary to understand the full SBIRT intervention. The Oregon Health & Science University IRB reviewed and approved the study.

Quantitative Measures and Analysis

Data

Medicaid enrollment, claims, and encounter data were obtained from Oregon's Health Systems Division for all Medicaid encounters 30 months prior to CCO implementation (January 1, 2010, to June 30, 2012), a 6‐month transition period (July 1, 2012, to December 31, 2012), and 30 months following CCO implementation (January 1, 2013, to June 30, 2015). Data were aggregated by the half‐year (January–June and July–December).

Participants

The analytic data set was restricted to individuals aged 18–64 years. Individuals dually enrolled in Medicare and Medicaid were excluded because Medicare FFS claims were not available. Individuals eligible for coverage under Medicaid expansion criteria were also excluded as this population differs in critical ways from “existing” Medicaid clients prior to CCO implementation. In addition, individuals not assigned to a CCO were removed from analysis; these include American Indian tribal enrollees who were allowed to opt out of the program as well as individuals with special health needs, including medically fragile children, Breast and Cervical Cancer Treatment Program members, individuals receiving CareAssist assistance due to human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS), and those receiving services for end‐stage renal disease. This left a total of 516,708 members in our study population. Figure 1 illustrates how the inclusion/exclusion criteria affected the study population size. The number of eligible members during the first half of each calendar year was as follows: 180,641 (2010), 247,134 (2011), 256,502 (2012), 255,107 (2013), 191,017 (2014), and 198,677 (2015). The decrease in enrollment beginning in 2014 reflects improving economic conditions and a shift of members originally enrolled under “traditional” Medicaid eligibility criteria into ACA expansion‐eligible populations; Oregon's elimination of Temporary Assistance for Needy Families (TANF) accounts for the largest shift. See Table 1 for detailed study population characteristics for SBI‐screened and SBI‐unscreened members.

Figure 1.

Figure 1

Inclusion/Exclusion Criteria Applied to Generate Study Population [Color figure can be viewed at http://wileyonlinelibrary.com]

Table 1.

Study Population Characteristics during Final Measurement Period (January–June 2015)

Characteristic (%) Received SBI Screening, N = 4,005 Did Not Receive SBI Screening, N = 82,808 p‐Value[Link]
Age (mean) 34.8 35.5 .002
Female 69.6 70.3 .359
Pregnant 14.1 11.0 <.001
White 72.9 70.0 <.001
Black 3.7 4.5 .013
Hispanic 10.5 11.1 .214
Other race or ethnicity 5.3 6.6 .001
Unknown race or ethnicity 7.7 7.7 .958
Rural 39.4 42.4 <.001
History of alcohol or drug Dx 15.9 10.3 <.001
Health comorbidities 83.7 83.1 .326
PCPs per capitaa 0.1 0.1 .478
Specialists per capitaa 0.1 0.1 .048
Institutionalizeda 1.0 1.0 .014
Below high school educationa 4.6 4.5 <.001
Linguistically isolated householda 13.9 13.8 .221
Income below poverty levela 14.9 14.6 <.001
Median household incomea 48,971 49,564 <.001

Note: Tests of statistical significance were performed using Welch two‐sample t‐tests (for continuous variables) and two‐sample tests for equality of proportions with continuity correction (for indicator variables).

a

Denotes area‐level indicator.

Analysis

We examined three sets of outcome measures: (1) screening, and brief intervention for alcohol or drug use (see Table S1 for diagnostic and procedure codes used to track this measure; Medicare codes were excluded because the analysis did not include Medicare recipients), (2) ICD‐9 diagnostic codes for alcohol and drug use disorders, and (3) HEDIS (Healthcare Effectiveness Data and Information Set) measures for treatment initiation (percent initiating treatment within 14 days of an alcohol or drug use diagnosis) and treatment engagement (percent who initiated care and received two or more services within 30 days of treatment initiation). Measurement periods for all of the measures were defined to be 6 months in duration (first/last half of a calendar year).

For inclusion in the SBI measure, members were required to have had at least one outpatient visit during the measurement period. For inclusion in the alcohol and drug use disorder diagnosis measures, members must have been continuously enrolled during the entire (6 months) measurement period. For inclusion in the initiation and engagement measures, members must have been enrolled during the 6 months prior to the new episode of alcohol or drug dependence, through 44 days after (14 days to initiate treatment + 30 days for continued engagement). Continuous enrollment requirements for the latter two measure types ensure that evidence of diagnosis and treatment would be represented in the data. The SBI measure does not require continuous enrollment as the outpatient visit presents an opportunity for screening. Enrollment was tracked across CCOs. Across the study period, 360,606 members met the inclusion criteria for the SBI measure, 423,829 for the alcohol and drug use disorder diagnosis measures, and 46,300 for the initiation and engagement measures.

Note that Oregon's CCO SBIRT metric requires 12 months of continuous enrollment within a single CCO and includes individuals with dual Medicaid and Medicare coverage. We opted to shorten our measurement periods and continuous enrollment criteria to 6 months to achieve a more granular picture of changing practice patterns. We calculated continuous enrollment across CCOs to include members who may have moved but remained within the CCO system, and we excluded dually eligible members for whom claims data were not complete. As a result, the percentages in this study are similar to, but do not replicate, the CCO metrics.

Logistic regression assessed the association between SBI screening and rates of treatment initiation and engagement and controlled for gender, age, race and ethnicity, rural residence, Primary Care Service Area socioeconomic variables (Dartmouth Atlas), and Chronic Illness and Disability Payment System (CDPS) risk indicators (Wilson, Wilson, and Canales 1981; Kronick et al. 2000). A subsequent regression using measurement periods of length 12 months—consistent with Oregon's CCO SBIRT specifications—instead of 6 months was conducted to assess the sensitivity of the relationship between SBI screening and treatment initiation and engagement with respect to timeframe.

Claims data management and analysis were completed using R version 3.3.1 software (R Foundation for Statistical Computing, Vienna, Austria). Claims and encounter data document the pace of implementation of screening for alcohol and drug use, while qualitative interviews help explain and interpret the quantitative findings and the process of scaling up.

Qualitative Data and Analysis

Data

Semi‐structured interview guides were used to elicit participants’ experiences and knowledge about SBI implementation and perspectives on development of the metric and processes for monitoring performance. Developed using CFIR, the guides probed characteristics of SBI (e.g., quality, advantage, adaptability), outer setting (e.g., patient needs, external policies, incentives), inner setting (e.g., practice settings, networks and communication, practice culture, implementation climate), practitioner characteristics (e.g., knowledge and beliefs, self‐efficacy), and the implementation process (e.g., engaging, executing, evaluating) (Damschroder et al. 2009; Damschroder and Hagedorn 2011). The principal investigators conducted 60‐minute interviews in person or by telephone.

Participants

Qualitative interviews with CCO leadership, providers, and other stakeholders were conducted between May 2013 and December 2015. We completed 114 interviews with 76 informants: CCO leaders (n = 37), state and county employees (n = 17), behavioral health providers (n = 12), primary care providers (n = 6), and other stakeholders (n = 4). Other stakeholders included policy experts with knowledge about key regulations that impact SUD services. Participants were selected through both purposeful (Creswell & Plano Clark, 2011) and snowball sampling techniques (Goodman 1961). Participants were re‐interviewed to monitor change over time.

Analysis

Interviews were recorded, transcribed, and entered into the qualitative analysis software program ATLAS.ti (Friese 2013) for coding and analysis. Following an adapted and iterative narrative content analysis approach, the research team developed potential codes, organized the codebook collaboratively, and independently coded transcripts. The team then compared and reviewed coding decisions and revised the coding scheme. At the conclusion of document coding, 35 percent of the documents were selected for “check‐coding.” The discrepancy between coder and check‐coder code choice was calculated using percent agreement to assess intercoder reliability. A .87 index demonstrated a strong coder to check‐coder consistency (Neuendorf 2002).

Results

Quantitative Results

The Oregon Health Authority promulgated the SBI mandate to encourage primary care settings to systematically screen and provide appropriate intervention when indicated. During the 30‐month baseline period beginning in 2010, the procedure and diagnostic codes for SBI screening were used infrequently; screening rates were consistently less than 0.1 percent among eligible members with a primary care visit. This increased to 0.1 percent during the 6‐month transition period (July 1, 2012–December 31, 2012). Evidence of screens increased slowly and significantly in the 2.5 years following CCO implementation: 0.2 percent first half of calendar year 2013 (235/138,175), 2.2 percent second half of 2013 (2,987/133,089), 2.4 percent first half of 2014 (2,448/100,294), 3.8 percent last half of 2014 (3,252/86,016), and 4.6 percent first half of 2015 (4,005/86,813) (see Figure 2).

Figure 2.

Figure 2

Screening and Brief Intervention Screening versus Alcohol or Drug Use Treatment Patterns
  • Notes: SBIRT rates include adults 18–64 with an outpatient visit. Initiation/Engagement rates include adults 18–64 with a new diagnosis of alcohol/drug disorder. Gray‐shaded bar indicates CCO implementation (“transition”) period.

Changes in SBI screening are evident across all of the individual 16 CCOs, with some efforts appearing soon after CCO implementation and others developing more gradually. Several CCOs demonstrate consistently improving screening rates as of the end of the measurement period (see Figure 3: CCOs A, D, G, H, J, P), and others appear to stagnate somewhat (see Figure 3: CCOs B, F, I).

Figure 3.

Figure 3

Screening and Brief Intervention Screenings by CCO
  • Notes: SBIRT rates include adults 18–64 with an outpatient visit. Gray‐shaded bar indicates CCO implementation (“transition”) period.

Compared to the unscreened population, SBI‐screened members were characterized by statistically greater proportions of pregnant women (14.1 percent vs. 11.0 percent, p < .001) and white individuals (72.9 percent vs. 70.0 percent, p < .001), and smaller proportions of black (3.7 percent vs. 4.5 percent, p = .013), non‐white/black/Hispanic (5.3 percent vs. 6.6 percent, p = .001), and rural‐living (39.4 percent vs. 42.4 percent, p < .001) individuals. Unsurprisingly, alcohol or drug diagnoses were significantly more prevalent among the screened population versus the unscreened population (15.9 percent vs. 10.3 percent, p < .001) (see Table 1).

Despite the increase in screens, our analysis found no clear evidence of change in the rate of alcohol and drug use disorder diagnoses in the Medicaid encounter data. Rates of tobacco, alcohol, and opioid use diagnoses demonstrated only small fluctuations across the baseline and implementation periods. Tobacco use was coded for 10 percent to 15 percent of Medicaid recipients 18–64 years of age. Diagnoses of alcohol use disorder decreased slightly from approximately 4 percent (2010, 2011, 2012) to 3.5 percent (2013), 2.7 percent (2014), and 2.5 percent (2015). Conversely, the rate of opioid use disorder diagnoses increased initially from 2.6 percent (2010) to 2.9 percent (2011 and 2012) and 3.1 percent (2013), before declining to 2.2 percent (2014, 2015). See Figure 4. Diagnostic rates were lower for cannabis and amphetamine (about 1.6–2.1 percent), and minimal for cocaine (0.2–0.3 percent) use disorders across the study period.

Figure 4.

Figure 4

Medicaid Recipients Diagnosed with Alcohol or Drug Use Disorders
  • Notes: Includes adults 18–64, continuously enrolled for 6 months. Gray‐shaded bar indicates CCO implementation (“transition”) period.

The HEDIS measures of treatment initiation and engagement suggest that among members with a new episode of alcohol or other drug dependence, about 4 in 10 initiated care within 14 days of the diagnosis. Specifically, rates of initiation in the first and last halves of each calendar year were as follows: 41.7 percent and 42.6 percent (2010), 41.1 percent and 41.3 percent (2011), 40.1 percent and 41.0 percent (2012), 38.6 percent and 37.5 percent (2013), 38.1 percent and 37.7 percent (2014), and 39.1 percent (first half of 2015). Rates of engagement (treatment initiation plus two or more services within 30 days) were lower: 22.9 percent and 23.2 percent (2010), 23.1 percent and 23.4 percent (2011), 21.1 percent and 23.4 percent (2012), 20.2 percent and 20.1 percent (2013), 17.7 percent and 18.4 percent (2014), and 18.1 percent (first half of 2015).

Whereas statewide use of SBI increased following CCO implementation, rates of initiation and engagement of alcohol and other drug treatment decreased steadily. Furthermore, logistic regression revealed no association between SBI screening and likelihood of initiating treatment among members with a new episode of alcohol or other drug dependence (Table S2). This result was robust to a sensitivity analysis using 12‐month measurement periods—in line with the state's measurement scheme—instead of 6‐month periods (Table S3).

Qualitative Results

In the context of progressive and innovative statewide payment and service delivery reform, our narrative content analysis identified 13 salient and recurrent themes that can be framed within three core domains: (1) Accelerators of implementation, (2) Roadblocks, and (3) Health system transformation. See Table 2 to view the 13 core themes.

Table 2.

Qualitative Themes Related to SBIRT Implementation

Themes Description Supporting Quotation
Accelerators
Aligned incentivizes Measuring, tracking, and paying CCOs based on performance on the SBIRT metric has advanced implementation “I think it's the metric and the focus. We do what we're measured for; it's the right thing to do but let's be clear, this is about the money.”
Internal champions Support from internal champions, particularly providers, generated buy‐in and helped address concerns and resistance. “The CCO is blessed with a physician champion who works closely with the chief behavioral health officer to help think about how physicians can screen.”
Workforce development Initial and continuous training for SBIRT and addictions more broadly has resulted in increased capacity to implement screening and brief intervention. “What they're training around SBIRT is getting PCPs and other types of providers comfortable with SBIRT. And that has been very helpful in helping those providers understand SBIRT and answering questions when there's concerns about a particular response.”
Workflow redesign Mapping clinical workflow for integration and efficiency was essential to accommodate SBI and the required documentation. “What I've heard from my primary care colleagues is that implementing … is a change of workflows, a different approach to practicing, figuring out how to best use it, who does it … but I think trainings have helped”
Roadblocks
Time requirements Providers struggled to carve out the additional time necessary to complete SBIRT in their already busy schedules. “ … we're talking about way too many people than what we can handle … primary care has been feeling overwhelmed for some time. So nothing new there. But it was adding a lot of stress to that process, SBIRT was.”
Confusion about coding The complexity of the metric has resulted in some clinics entering codes that are not counted toward the metrics, leading to their CCOs ultimately losing credit for the work. “It is tied to billing codes and that is a mistake … We note that the SBIRT codes should not be charged but the coders just delete them because they are billing codes. So we are at zero percent on screening and brief intervention. The CCO gets no credit for our work.”
Questionable efficacy data The evidence for SBIRT is equivocal for drug use; some respondents question its identification as a metric to push integration and system change. “The A&D metrics don't go far in terms of driving change in the system. SBIRT is not very helpful.”
Addressing positive screens SBIRT implies that the complete package, Screening, Brief Intervention, and Referral to Treatment, is present. Our respondents indicate that while screening has increased, the necessary integration and competencies for brief intervention and referral to treatment are not present. “The continuum of care is not clearly defined for patients and physicians. We need to help primary care providers to know when to refer and where to refer and the expectations they should have from these referrals.”
Systems transformation: positive outcomes
Increased awareness of impact and need for services There has been an increased awareness of the prevalence and the importance of addressing SUD in primary care. SBIRT has highlighted the need for integration. “SBIRT is great because it allows us to standardize our discussion of alcohol and drug use. It has become a uniform conversation with every patient—bank president to adolescent.”
Attitudinal shifts There have attitudinal shifts from why to how; these shifts are essential for health care transformation. “When I first came on there was a lot of questions  … do we really have to do it … is it really doable? Is it really necessary …  throughout my year here it has changed. People have been asking more, well, how can we do it? I get much less whys, occasionally. But really, it's more like how's.”
Systems transformation: concerns
Interpretation of available data Participants reported disconnect between the data from the metric and actual clinical practice. “Just in how they're collected and that sort of thing. So it took me awhile to come to terms with the fact that they are counted by going through claims, because we know that method is so imperfect. [R agreeing] We know the inaccuracies of it.”
Confusion over metric OHA has changed the way that completed SBIRTs are counted several times. “But yeah, we've struggled hard on that committee with looking at the denominator. And also trusting why does the denominator constantly change?”
Metric formula Complicated and confusing requirements for properly documenting and coding for SBIRT have hampered implementation. “CPT codes are cumbersome measures.”

Accelerators

Interviews with key informants revealed four salient themes across CCOs that facilitated implementation of SBI: (1) aligning incentives, (2) internal champions, (3) workforce development, and (4) workflow redesign.

Coordinated care organization participants reported that linking the SBI metric to incentive payments accelerated implementation and turned attention to increasing screening in primary care settings (see Table 2 for themes and quotes). While some CCOs reported physician resistance, the presence of internal champions increased acceptance and willingness to use SBI. Providers who felt comfortable with SBI and documentation seemed to shift the dial, create energy, and anchor stability for increasing uptake. Thus, Oregon's experience with the SBI metric suggested that aligning attention with an internal champion contributed to implementation and broader acceptance across the system.

Workforce development and the closely linked workflow redesign were recurrent themes in our analysis. Findings indicate that trainings and collaborative learning sessions were essential for increasing provider knowledge and confidence with SBI. Trained providers reported a better understanding of integration, addiction, validated screening tools with which licensed providers may conduct screening, and appropriate codes for billing and documentation. CCOs and clinics used a variety of methods to increase access to this information including hiring trainers and consultants from the Addiction Technology and Transfer Center Network (ATTC), employing technology to share resources (e.g., webinars, consultation/conference sessions), encouraging participation in state sponsored trainings, and co‐sponsoring learning community workshops.

While workforce development increased knowledge about SBI, it was not sufficient for sustained change and participants commented repeatedly about the need for changes in workflow, inclusive of coding, and documentation. Implementing SBI in a busy primary care setting can be incredibly complex, requiring shifts in patient flow and new staff responsibilities and roles. In early interviews, one of the most common themes was concern about the time needed for SBI implementation and the desire for creation of efficient workflows with providers, administrators, and coding and documentation specialists. At follow‐up interviews, participants reported that the most successful adopters had examined and modified their care models collaboratively to accommodate screening and documentation and had trained providers and support staff.

Roadblocks

While SBI rates have increased over the past 3 years, significant challenges remain: (1) time required for screening and a potential brief intervention, (2) confusion about which procedure and diagnostic codes to use, (3) unclear efficacy data for SBI, and (4) uncertainty of how to respond to positive screens including referral challenges. In fact, as a consequence of policy and performance metric confusion, some CCOs did not receive credit for screening efforts and may not have reached the benchmark or interim improvement target. Expressing hesitation about the evidence base for SBI, particularly for problematic drug use, informants questioned using SBI as the metric for integrating SUD services in primary care.

Concerns about how to respond existed even though the state and CCOs provided training to increase capacity of primary care to address SUDs. Despite progressive steps toward integration of screening, none of the respondents indicated that full implementation of SBIRT was in place. Thus, the system appears to be working toward the metric of screening without sufficient attention to referral. Qualitative findings also indicate that while screening was relatively easy to train to proficiency, providers needed more intensive, continual training to effectively conduct brief interventions. These barriers were not unique to Oregon's CCO model; previous research into adoption and integration of SBIRT reported similar challenges (Agerwala and McCance‐Katz 2012; Finnell 2012; Pilowsky and Wu 2012; Parker et al. 2013; Jones et al. 2014).

Health System Transformation

To better understand the absence of an increase in initiation and engagement in treatment per our quantitative findings, we examined the qualitative themes related to follow‐up for positive screens. Interestingly, participants described several positive outcomes related to the SBI incentive metric suggesting that it increased awareness of the prevalence and importance of addressing SUDs in primary care and created an attitudinal shift essential for health care transformation. They noted an elevation of conversation about SUDs and integration as a result of both the metric and research related to high costs associated with patients with SUDs. At the same time, participants reported that naming/defining and implementation of the SBI metric failed to address the system redesign necessary to fully integrate the needed behavioral health services. Three broad themes reflected concerns about the implementation process for SBI: (1) interpretation of available data; (2) confusion about the metric; and (3) complex formula used to calculate, measure, and track SBI.

Participants reported conflict between how the metric was calculated and actual clinical practice, in particular, emphasizing that rates of SBI completed in clinics are likely higher than what is documented through the metric. Part of this discrepancy between actual clinical practice and documented rate was due to widespread confusion about criteria that had to be met for a screening to be counted toward the incentive metric. For example, under Medicaid only licensed providers are allowed to facilitate SBIRT and bill for those services in medical settings; however, in behavioral health settings, auxiliary providers under supervision of licensed providers may facilitate and bill for SBIRT. In addition, many participants falsely believed that only positive full screens counted toward the metric. Participants indicated that the coding was so complex that it often got “let go” on the billing and claims side. Widespread confusion was a pervasive theme as indicated by the perceived instability of the metric itself. Despite the fact that the formula used to calculate the rate of SBI remained stable, participants believed that it was altered on several occasions, changing the denominator of the total population eligible for SBI.

Participants also reported different approaches to implementing SBI. Some providers conducted full screenings in the absence of risk factors or a positive brief annual screen, while others only conducted the full screening on targeted subgroups of patients with risk factors or a positive brief annual screen. Thus, lack of understanding about the metric, its meaning and intent, and the current state of screening all surfaced as salient concerns during our discussions with CCO leadership and clinicians.

Discussion

This study was framed around CFIR, which acknowledges the complexities of health care systems; it provides a model for examining the implementation of SBI and generates data for other states and health care systems as they expand access and integrate behavioral health and primary care. In the context of the current ACA, Oregon developed a unique pay‐for‐performance model in which each CCO is accountable for quality and access metrics while achieving cost savings. To encourage screening for alcohol and drug misuse, the metric for SBI was identified as one of 17 incentive metrics associated with bonus payments.

Analysis of Medicaid claims and encounter data provided evidence of significant increase in the use of procedure and diagnostic codes for SBI over the 2.5 years (2013 through mid‐2015) postimplementation of CCOs and incentive metrics. Interestingly, our analysis found no evidence of corresponding change in the rate of alcohol and drug use disorder diagnoses in the Medicaid encounter data. Similarly, documented SBI was not associated with increased rates of initiation of treatment. Thus, it is possible providers and clinics are able to increase SBI but are not yet prepared for expanded clinical assessment, diagnosis, and treatment. It is also possible that increases seen in SBI rates may be due to increases in documentation of SBI rather than actual increases in the practice of SBI. With training in how to appropriately code and pressure to document to receive credit, providers may not only be more proficient at coding but also incentivized to document SBI. Our findings demonstrate that further research is needed to understand the barriers to referral, initiation, and engagement in services in primary care settings.

To better understand the change in use of SBI, with limited corresponding increases in rates of diagnosis of SUDs and decreasing initiation and engagement in treatment, qualitative interviews provided rich contextual information. According to key informants, the SBI metric and related incentive metrics enabled systems to establish baselines, set performance targets, and make complex, abstract goals more tangible. In addition, tying metrics to performance not only resulted in increased evidence of SBI but also greater awareness of the impact of SUDs on overall health. This type of environmental or outer setting expectation and pressure for accountability corresponds with CFIR, which notes that implementation of an intervention can be accelerated if there is proper alignment of external policy and incentives (Damschroder et al. 2009; Damschroder and Hagedorn 2011). Thus, in Oregon, properly aligned incentives appear to have accelerated the attitudinal shift necessary for health care transformation and served as a precursor to integrating SUD services in primary care settings.

The SBI metric developed and monitored by the state is actually a screening and brief intervention metric and includes no mechanism for tracking, measuring, or rewarding referral to treatment. This challenge is particular to the system in Oregon and suggests that future incentive programs should consider how to incorporate a measure specific for referral to treatment. In particular, it suggests that while incentive metrics can facilitate change, mechanisms and processes for enabling desired outcomes must be developed. Respondents consistently reported that no system‐wide procedures had been put in place to facilitate this process, limiting the full uptake of screening and leaving a bit of the “then what” experience.

In order for Oregon's CCOs to achieve the 16.5 percent performance benchmark set for 2017, several factors must be considered. CFIR indicates that careful attention to the inner setting is critical and appropriate resources to support the intervention should be made available (Damschroder et al. 2009; Damschroder and Hagedorn 2011). First, improved documentation and coding procedures are essential. Clinical documentation improvement specialists could enhance documentation and coding by providing on‐site assistance to ensure that SBIs are coded appropriately so that the metric is accurately calculated and CCOs receive credit for the work done. Second, ambulatory sites need access to leaders with expertise in brief intervention as well as an updated and current list of referral sites. Third, primary care practitioners need more training. While initial training has been an important strategy in encouraging adoption of SBI in clinics, continued training and support are necessary to increase the capacity of primary care to provide consistent and efficacious SUD services. Finally, investments in information technology could play an essential role in improving resource access via web‐based trainings as well as enhancing the electronic health record data available for clinical decision making.

Adopting brief validated screening tools into the electronic health record systems is one strategy that may assist in completion of clinical quality/performance measures and creating holistic treatment plans for patients (Ghitza and Tai 2014). It must be acknowledged, however, that information sharing is complicated by provider and organizational concerns about the potential to violate 42 CFR part 2, the federal regulation that adds additional protection for information related to SUDs.

The potential repeal or replacement of the ACA may rollback Medicaid expansion, resulting in the removal of SUD treatment as an essential health benefit and a reduction in access to SUD treatment. Thus, addressing implementation challenges remains a national health care imperative. The care model redesign necessary to provide behavioral health services across primary care settings on a consistent basis appears to be challenging and we are in the early stages of determining exactly what is required to drive this transformation. Our findings indicate that shared local risk models and reorganization of services through the CCOs impacted SUD services associated with bonus payments (i.e., SBI). Additional research is needed to increase SBI and further develop mechanisms and processes to increase initiation and engagement in care.

Supporting information

Appendix SA1: Author Matrix.

Table S1. Oregon Health Authority Metric for Screening and Brief Intervention for Risky Alcohol or Drug Use.

Table S2. Multilevel Multivariable Logit Model: Odds Ratios for Initiation of Alcohol or Drug Use Treatment, Six‐Month Measurement Period.

Table S3. Multilevel Multivariable Logit Model: Odds Ratios for Initiation of Alcohol or Drug Use Treatment, Twelve‐Month Measurement Period (Sensitivity Analysis).

Acknowledgments

Joint Acknowledgment/Disclosure Statement: A National Institute on Drug Abuse award supports the study: R33 DA035640.

Disclosures: None.

Disclaimer: None.

References

  1. Agerwala, S. M. , and McCance‐Katz E. F.. 2012. “Integrating Screening, Brief Intervention, and Referral to Treatment (SBIRT) into Clinical Practice Settings: A Brief Review.” Journal of Psychoactive Drugs 44 (4): 307–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Babor, T. F. , McRee B. G., Kassebaum P. A., Grimaldi P. L., Ahmed K., and Bray J.. 2007. “Screening, Brief Intervention, and Referral to Treatment (SBIRT).” Substance Use 28 (3): 7–30. [DOI] [PubMed] [Google Scholar]
  3. Barbosa, C. , Cowell A. J., Bray J., and Aldridge A.. 2015. “The Cost Effectiveness of Alcohol Screening, Brief Intervention, and Referral to Treatment (SBIRT) in Emergency and Outpatient Medical Settings.” Journal of Substance Abuse Treatment 53: 1–8. [DOI] [PubMed] [Google Scholar]
  4. Barbosa, C. , Cowell A. J., Landwehr J., Dowd W., and Bray J. W.. 2016. “Cost of Screening, Brief Intervention, and Referral to Treatment in Health Care Settings.” Journal of Substance Abuse Treatment 60: 54–61. [DOI] [PubMed] [Google Scholar]
  5. Bouchery, E. E. , Harwood H. J., Sacks J. J., Simon C. J., and Brewer R. D.. 2011. “Economic Costs of Excessive Alcohol Consumption in the US, 2006.” American Journal of Preventive Medicine 41 (5): 516–24. [DOI] [PubMed] [Google Scholar]
  6. Boyd, C. , Leff B., Weiss C., Wolff J., Clark R., and Richards T.. 2010. Full Report: Clarifying Multimorbidity to Improve Targeting and Delivery of Clinical Services for Medicaid Populations. Hamilton, NJ: Center for Health Care Strategies, Inc. [Google Scholar]
  7. Brick, J ., editor. 2012. Handbook of the Medical Consequences of Alcohol and Drug Abuse. New York: Routledge. [Google Scholar]
  8. Chang, A. M. , Cohen D. J., McCarty D., Rieckmann T., and McConnell K. J.. 2014. “Oregon's Medicaid Transformation—Observations on Organizational Structure and Strategy.” Journal of Health Politics Policy and Law 40 (1): 257–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Clark, R. E. , Samnaliev M., and McGovern M. P.. 2009. “Impact of Substance Use Disorders on Medical Expenditures for Medicaid Beneficiaries with Behavioral Health Disorders.” Psychiatric Services 60 (1): 35–42. [DOI] [PubMed] [Google Scholar]
  10. Creswell, J. W. , and Plano Clark V. L.. 2011. Designing and Conducting Mixed Methods Research, 2d Edition. Thousand Oaks, CA: SAGE Publications. [Google Scholar]
  11. Damschroder, L. J. , and Hagedorn H. J.. 2011. “A Guiding Framework and Approach for Implementation Research in Substance Use Disorders Treatment.” Psychology of Addictive Behaviors 25 (2): 194–205. [DOI] [PubMed] [Google Scholar]
  12. Damschroder, L. J. , Aron D. C., Keith R. E., Kirsh S. R., Alexander J. A., and Lowery J. C.. 2009. “Fostering Implementation of Health Services Research Findings into Practice: A Consolidated Framework for Advancing Implementation Science.” Implementation Science 4: 50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Dartmouth Atlas . 2016. “PSCA Data Download – 2010 (Census Tract Basis).” http://www.dartmouthatlas.org/tools/downloads.aspx?tab=42
  14. Désy, P. M. , and Perhats C.. 2008. “Alcohol Screening, Brief Intervention, and Referral in the Emergency Department: An Implementation Study.” Journal of Emergency Nursing 34 (1): 11–9. [DOI] [PubMed] [Google Scholar]
  15. Estee, S. , Wickizer T., He L., Shah M. F., and Mancuso D.. 2010. “Evaluation of the Washington State Screening, Brief Intervention, and Referral to Treatment Project: Cost Outcomes for Medicaid Patients Screened in Hospital Emergency Departments.” Medical Care 48 (1): 18–24. [DOI] [PubMed] [Google Scholar]
  16. Finnell, D. S. 2012. “A Clarion Call for Nurse‐Led SBIRT across the Continuum of Care.” Alcoholism, Clinical and Experimental Research 36 (7): 1134–8. [DOI] [PubMed] [Google Scholar]
  17. Friese, S . 2013. ATLAS.ti 7: User Manual: User Guide and Reference. [Computer Software]. Berlin: Scientific Software Development. [Google Scholar]
  18. Ghitza, U. E. , and Tai B.. 2014. “Challenges and Opportunities for Integrating Preventive Substance‐Use‐Care Services in Primary Care through the Affordable Care Act.” Journal of Health Care for the Poor and Underserved 25 (1 Suppl): 36–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Goodman, L. A. 1961. “Snowball Sampling.” Annals of Mathematical Statistics 32 (1): 148–70. [Google Scholar]
  20. Greenhalgh, T. , Robert G., Macfarlane F., Bate P., and Kyriakidou O.. 2004. “Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations.” Milbank Quarterly 82 (4): 581–629. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Howard, S. W. , Bernell S. L., Yoon J., and Luck J.. 2014. “Oregon's Coordinated Care Organizations: A Promising and Practical Reform Model.” Journal of Health Politics Policy and Law 39 (4): 933–40. [DOI] [PubMed] [Google Scholar]
  22. Jonas, D. E. , Garbutt J. C., Amick H. R., Brown J. M., Brownley K. A., Council C. L., Viera A. J., Wilkins T. M., Schwatz C. J., Richmond E. M., Yeatts J., Evans T. S., Wood S. D., and Harris R. P.. 2012. “Behavioral Counseling after Screening for Alcohol Misuse in Primary Care: A Systematic Review and Meta‐Analysis for the US Preventive Services Task Force.” Annals of Internal Medicine 157 (9): 645–54. [DOI] [PubMed] [Google Scholar]
  23. Jones, E. , Ku L., Smith S., and Lardiere M.. 2014. “County Workforce, Reimbursement, and Organizational Factors Associated with Behavioral Health Capacity in Health Centers.” Journal of Behavioral Health Services & Research 41 (2): 125–39. [DOI] [PubMed] [Google Scholar]
  24. Kronick, R. , Gilmer T., Dreyfus T., and Lee L.. 2000. “Improving Health‐Based Payment for Medicaid Beneficiaries: CDPS.” Health Care Financing Review 21 (3): 29–64. [PMC free article] [PubMed] [Google Scholar]
  25. Krupski, A. , Joesch J. M., Dunn C., Donovan D., Bumgardner K., Lord S. P., Ries R., and Roy‐Byrne P.. 2012. “Testing the Effects of Brief Intervention in Primary Care for Problem Drug Use in a Randomized Controlled Trial: Rationale, Design, and Methods.” Addiction Science & Clinical Practice 7: 27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lewis, V. A. , Colla C. H., Tierney K., Van Citter A. D., Fisher E. S., and Meara E.. 2014. “Few ACOs Pursue Innovative Models That Integrate Care for Mental Illness and Substance Abuse with Primary Care.” Health Affairs 33 (10): 1808–16. [DOI] [PubMed] [Google Scholar]
  27. McCance‐Katz, E. F. , and Satterfield J.. 2012. “SBIRT: A Key to Integrate Prevention and Treatment of Substance Abuse in Primary Care.” American Journal on Addictions 21 (2): 176–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Mechanic, D. 2012. “Seizing Opportunities under the Affordable Care Act for Transforming the Mental and Behavioral Health System.” Health Affairs 31 (2): 376–82. [DOI] [PubMed] [Google Scholar]
  29. Moyer, V.A. , and Preventive Services Task Force . 2013. “Screening and Behavioral Counseling Interventions in Primary Care to Reduce Alcohol Misuse: U.S. Preventive Services Task Force Recommendation Statement.” Annals of Internal Medicine 159(3): 210–8. [DOI] [PubMed] [Google Scholar]
  30. National Drug Intelligence Center (NDIC) . 2011. National Drug Threat Assessment. Washington, D.C.: U.S. Department of Justice. Product No.: 2011‐Q0317‐001. [Google Scholar]
  31. Neighbors, C. J. , Sun Y., Yerneni R., Tesiny E., Burke C., Bardsley L., McDonald R., and Morgenstern J.. 2013. “Medicaid Care Management: Description of High‐Cost Addictions Treatment Clients.” Journal of Substance Abuse Treatment 45 (3): 280–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Neuendorf, K. A. 2002. The Content Analysis Guidebook. Thousand Oaks, CA: SAGE Publications. [Google Scholar]
  33. Oregon Health Authority . 2013. “2013 Quality Pool Reference Instructions” [accessed on March 3, 2016]. Available at http://www.oregon.gov/oha/analytics/CCOData/Reference%20Instructions.pdf
  34. Oregon Health Authority . 2014. “2014 Quality Pool Reference Instructions” [accessed on March 3, 2016] http://www.oregon.gov/oha/analytics/CCOData/2014%20Reference%20Instructions.pdf
  35. Oregon Health Authority . 2015a. “Oregon's Health System Transformation: CCO Metrics 2015 Final Report” [accessed on November 3, 2016]. Available at http://www.oregon.gov/oha/Metrics/Documents/2015_performance_report.pdf
  36. Oregon Health Authority . 2015b. “Alcohol and Drug Misuse (SBIRT): Measure Basic Information” [accessed on December 23, 2015]. Available at http://www.oregon.gov/oha/analytics/CCOData/Alcohol%20and%20Drug%20Misuse%20(SBIRT)%20-%202016.pdf
  37. Oregon Health Authority . 2015c. “Oregon's Health System Transformation: 2014 Final Report” [accessed on December 29, 2015]. Available at http://www.oregon.gov/oha/Metrics/Documents/2014%20Final%20Report%20-%20June%202015.pdf
  38. Oregon Health Authority . 2016. “2016 Quality Pool Reference Instructions” [accessed on November 2, 2016]. Available at http://www.oregon.gov/oha/analytics/CCOData/2016%20Reference%20Instructions.pdf
  39. Palinkas, L. A. , Aarons G. A., Horwitz S., Chamberlain P., Hurlburt M., and Landsverk J.. 2011. “Mixed Method Designs in Implementation Research.” Administration and Policy in Mental Health 38 (1): 44–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Parker, G. D. , Libart D., Higgs T., Schrader S., McCollom B., Fanning L., and Dixon J.. 2013. “SBIRT in Primary Care: The Struggles and Rewards.” Journal of Addictive Behaviors, Therapy & Rehabilitation 2: 1. [Google Scholar]
  41. Pilowsky, D. , and Wu L. T.. 2012. “Screening for Alcohol and Drug Use Disorders Among Adults in Primary Care: A Review.” Substance Abuse and Rehabilitation 3 (1): 25–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Polen, M. R. , Whitlock E. P., Wisdom J. P., Nygren P., and Bougatsos C.. 2008. Screening in Primary Care Settings Illicit Drug use: Staged Systematic Review for the United States Preventive Services Taskforce. Rockville, MD: Agency for Healthcare Research and Quality (US). Report No.: 08‐05108‐EF‐1. [PubMed] [Google Scholar]
  43. Pringle, J. , Aldridge A., and Kearney S.. 2015. “A New Methodology for Examining the Efficacy of SBIRT Protocols on Reducing Healthcare Utilization and Costs.” Addiction Science & Clinical Practice 10 (Suppl 2): O44. [Google Scholar]
  44. Sacks, J. J. , Gonzales K. R., Bouchery E. E., Tomedi L. E., and Brewer R. D.. 2015. “2010 National and State Costs of Excessive Alcohol Consumption.” American Journal of Preventive Medicine 49 (5): e73–9. [DOI] [PubMed] [Google Scholar]
  45. Shim, R. , and Rust G.. 2013. “Primary Care, Behavioral Health, and Public Health: Partners in Reducing Mental Health Stigma.” American Journal of Public Health 103 (5): 774–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Substance Abuse and Mental Health Services Administration . 2014. “The NSDUH Report: Substance Use and Mental Health Estimates from the 2013 National Survey on Drug Use and Health: Overview and Findings” [accessed on August 8, 2016]. Available at https://store.samhsa.gov/shin/content/NSDUH14-0904/NSDUH14-0904.pdf [PubMed]
  47. Tetrault, J. M. , Green M. L., Martino S., Thung S. F., Degutis L. C., Ryan S. A., Martel S., Pantalon M. V., Bernstein S. L., O'Connor P. G., Fiellin D. A., and G. D'Onofrio . 2012. “Developing and Implementing a Multispecialty Graduate Medical Education Curriculum on Screening, Brief Intervention, and Referral to Treatment (SBIRT).” Substance Abuse 33 (2): 2012. [DOI] [PubMed] [Google Scholar]
  48. Wilson, L. O. , Wilson F. P., and Canales L.. 1981. “Algorithm‐Directed Triage in a Pediatric Acute Care Facility: A Retrospective Study.” Annals Emergency Medicine 10 (8): 427–31. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix SA1: Author Matrix.

Table S1. Oregon Health Authority Metric for Screening and Brief Intervention for Risky Alcohol or Drug Use.

Table S2. Multilevel Multivariable Logit Model: Odds Ratios for Initiation of Alcohol or Drug Use Treatment, Six‐Month Measurement Period.

Table S3. Multilevel Multivariable Logit Model: Odds Ratios for Initiation of Alcohol or Drug Use Treatment, Twelve‐Month Measurement Period (Sensitivity Analysis).


Articles from Health Services Research are provided here courtesy of Health Research & Educational Trust

RESOURCES