Early implementation of screening for substance use in rural primary care: A rapid analytic qualitative study

Abstract

Background:

Few primary care patients are screened for substance use. As part of a phased feasibility study examining the implementation of electronic health record-integrated screening with the Tobacco, Alcohol, and Prescription Medication Screening (TAPS) Tool and clinical decision support (CDS) in rural primary care clinics, focus groups were conducted to identify early indicators of success and challenges to screening implementation.

Method:

Focus groups (n = 6) were conducted with medical assistants (MAs: n = 3: 19 participants) and primary care providers (PCPs: n = 3: 13 participants) approximately one month following screening implementation in three Federally Qualified Health Centers in Maine. Rapid analysis and matrix analysis using Proctor’s Taxonomy of Implementation Outcomes were used to explore implementation outcomes.

Results:

There was consensus that screening is being used, but use of the CDS was lower, in part due to limited positive screens. Fidelity was high among MAs, though discomfort with the CDS surfaced among PCPs, impacting adoption and fidelity. The TAPS Tool’s content, credibility and ease of workflow integration were favorably assessed. Challenges include screening solely at annual visits and self-administered screening for certain patients.

Conclusions:

Results reveal indicators of implementation success and strategies to address challenges to screening for substance use in primary care.

Introduction

Primary care is a critical setting in which to identify and treat substance use.^1,2 Substance use disorders are highly prevalent among adult primary care populations,³ and can contribute to negative health outcomes (e.g. chronic infectious diseases, mental health comorbidities including mood and anxiety disorders, chronic pain, and difficulties with sleep).⁴ The United States Preventive Services Task Force (USPSTF) has recommended screening for unhealthy alcohol use in adult primary care patients since 2004,⁵ and a recent USPSTF draft recommendation also supports screening for illicit drug use.⁶ However, few patients are screened for either.⁷ Only an estimated 25–28% of adult patients report being screened annually for substance use in primary care settings,^7–10 and even fewer with a validated tool.¹¹

Early efforts to introduce substance use screening into primary care include the demonstration that systematic screening delivered in primary care settings combined with brief interventions were capable of reaching and reducing alcohol consumption among large numbers of at-risk drinkers.¹² Additionally, efforts to develop a screening test and brief interventions for illicit drugs as well as alcohol and tobacco were initiated in the late 1990s.¹³ However, the Substance Abuse Mental Health Services Administration’s (SAMHSA) Screening, Brief Intervention, and Referral to Treatment (SBIRT) initiative¹⁴ is the most ambitious example of current evidence-based efforts to test and disseminate screening and intervention in US primary care and other medical settings.

Previous research has examined barriers to implementing screening for substance use in primary care settings.^10,15–18 Commonly reported barriers include lack of provider training and comfort treating substance use,^15,16,19 treatment resources,^17,18 and time.^15,16 Though research has extensively explored barriers and facilitators to the implementation of screeners pre-implementation,²⁰ fewer studies have conducted post-implementation interviews to understand the process and outcomes of implementing systematic screening for substance use, particularly in rural primary care settings where patients are less likely to be screened than in urban settings.²¹

Screening for substance use is of heightened importance in rural areas which have been disproportionately impacted by the opioid epidemic. Most notably, there are significantly more opioids prescribed to rural residents than their urban counterparts.^22,23 This fact is associated with the lack of availability in rural locales for alternative treatment options for pain (e.g. physical therapy, occupational therapy, acupuncture) which may lead to greater reliance on prescription opioids.²³ Compounding these inequities, the rural setting exacerbates barriers to implementing screening in primary care, e.g. rural patients have less access to primary care,²⁴ limited behavioral health services,^25–28 and less access to treatment than urban counterparts.²⁹

The overall aims of this multi-phase study, funded by the National Institute on Drug Abuse (NIDA) Clinical Trials Network (CTN), were to examine the feasibility of integrating screening for substance use into rural primary care settings and of collecting screening information in electronic health records (EHRs). This study represents an expansion of a parent study that focused on implementation of substance use screening in urban health systems,³⁰ and differs from the parent study in its screening tool (the Tobacco, Alcohol, and Prescription Medication and other Substance use Screening (TAPS) Tool),^31–34 and setting (rural Federally Qualified Health Center (FQHC) clinics in Maine).

To translate the TAPS Tool from research to practice, the ongoing rural study involves four phases. In Phase 1, we gathered information from primary care providers (PCPs), medical assistants (MAs), and patients to inform the programming of the TAPS Tool and clinical decision support (CDS) tool into the EHR, identify current screening practices, and explore barriers and facilitators to screening.³⁵ In Phase 2, the EHR-integrated screening (i.e. TAPS and CDS) tools were tailored through an iterative process of usability testing. Simultaneously, implementation preparation occurred in one clinic by identifying clinical champions, educating clinic leadership, training staff, and conducting a workflow analysis to identify barriers and facilitators of implementation. In Phase 3, screening was implemented in the first of three clinical sites, along with monitoring of adoption of the screening tools into routine clinical practice. During Phases 3 and 4, focus groups were conducted at each clinic following the introduction of screening; findings presented here are from these early implementation focus groups.

The TAPS Tool was developed by the National Drug Abuse Treatment Clinical Trials Network. This 2-stage screening and brief assessment tool was designed specifically to screen primary care patients for tobacco, alcohol, nonmedical prescription medication, and other drug use.^32–34,36 In an earlier Clinical Trials Network study, a tablet-delivered version of the TAPS Tool was tested with 2,000 adult primary care patients across five clinics and found to be easy to use and well-accepted.³¹ In addition to the TAPS Tool, a clinical decision support tool was integrated within the EHR to provide PCPs with support in conducting brief motivational counseling, based on the brief negotiated interview, to provide feedback on positive screening results, enhance motivation to change and set goals, and negotiate a plan. The plan may include facilitating a referral to treatment when appropriate. The primary case site partnered with software developer PatientLink to create the substance use screening forms based on the TAPS Tool and clinical decision support, as well as create and maintain bridging software that migrates patient responses from a tablet into the patient’s EHR at the point of care. Within the EHR, TAPS scores are presented for each substance. To alert medical staff to positive scores, any scores indicating high risk substance use are red, while those indicating moderate risk use are yellow. (See: Figure 1 for workflows and Supplementary Appendix for screenshots of TAPS Tool, clinical decision support tool).

Figure 1.

C1–C3 workflows.

The screening strategy includes: use of the TAPS Tool solely at “annual physical” visit types (i.e. in response to feedback received during Phase 1); patient self-administration via tablet; reviewed by MA; and delivered to PCP with an alert and clinical decision support. Training for providers entailed presentations led by the clinical champion at staff meetings with step-by-step screenshots of how and when to use the TAPS Tool and clinical decision support tool at least two weeks prior to the launch date, as well as provision of paper copies of the presentations to all providers. Training for MAs and front desk staff entailed brief presentations and individual interactions with the tablet, facilitated by the site’s project manager. Real-time support for medical staff was available after the launch of screening via email or in person as needed. The project manager supported the front desk staff who handed the tablet to patients during patient check-in, and the MAs to help them calculate the TAPS score and adjust their workflow as needed. The trainer also supported providers through the process of reviewing the patient answers, preplanning their conversation with the patient, and correctly documenting screening results and referrals to treatment, when warranted prior to entering the exam room.

In this manuscript, we report on findings of focus groups conducted at each clinic during early implementation of screening (Phases 3 and 4), to explore implementation outcomes and identify stakeholder experiences. The primary aim of this part of the study was to systematically examine the implementation process of substance use screening and intervention using EHR-integrated screening and clinical decision support in three rural FQHCs clinic sites to identify early indicators of success and potential challenges to feasibility.

Methods

Design

Medical staff (PCPs and MAs) perspectives regarding provider-level implementation outcomes were gathered in focus groups. Focus groups were conducted across three clinic sites at one to two months after implementation of the tablet-based TAPS Tool screening and clinical decision support tool, using a moderator guide adapted from the parent study and following themes from the Action Cycle component of the Knowledge To Action (KTA) conceptual framework (i.e. monitoring knowledge use and evaluating outcomes).³⁷ Developed by implementation researchers based on a synthesis of thirty-one theories for planned action, the aim in using the Knowledge To Action conceptual framework, a process-oriented implementation framework, is to guide and inform the implementation process of the TAPS Tool in rural primary care.³⁷ Subsequently used to explore five of the most relevant implementation outcomes resulting from the actions of medical staff across sites, Proctor’s Taxonomy of Implementation Outcomes served as an evaluating framework enabling assessment of successful implementation of the screening program.^38,39

The Proctor definitions of the five implementation outcomes follow: (1) adoption – action to try or employ an innovation; (2) acceptability – perception among stakeholders that innovation is agreeable, palatable, or satisfactory based on stakeholder’s knowledge of or direct experience with various dimensions of the innovation to be implemented such as its content, complexity, comfort, delivery and credibility; (3) feasibility – extent to which an innovation can be successfully used or carried out within a given setting; (4) fidelity – degree to which an innovation was implemented as it was prescribed in protocol; and (5) appropriateness – perceived fit, relevance, or compatibility of innovation for a given setting, provider, or consumer. For detail on how the implementation outcomes are applied in this study, see Table 1. Patient screening data and attitudes toward screening are still being collected and will be published separately. The study was approved by the Institutional Review Boards (IRBs) of New York University School of Medicine and the Dartmouth Committee for the Protection of Human Subjects.

Table 1.

Implementation outcome detail and study-specific modifications/applications.

Implementation outcome definitions	Level of analysis	Study-specific definition modifications	Interview guide domains/corresponding proctor outcomes*	How implementation outcomes are applied in this study
Adoption “Action to try or employ an innovation or evidence-based practice”	Provider, organization/Setting	Awareness; value neutral description of actions that indicate experience with the screening and CDS intervention Usage/Non-usage = Adoption	• Adoption • Screening results • CDS tool usage	Adoption of screening, the CDS tool, and making referrals to specialty treatment
Acceptability “Perception among stakeholders that innovation is agreeable, palatable, or satisfactory. Acceptability should be assessed based on stakeholder’s knowledge of or direct experience with various dimensions of the treatment to be implemented, such as its content, complexity, comfort, delivery, credibility.” [Measured from perspective of various stakeholders] Feasibility	Provider, consumer	Evaluations (e.g. “overall good”); Evaluations of accuracy as proxy for satisfaction with credibility; Evaluations of simplicity as proxy for complexity; Evaluations of challenges (literacy or tech) as proxy for complexity; Descriptions of discomfort with acting on positive screening results.	• Adoption • Workflow logistics • Perceived Accuracy of results • Patient characterizations • Screening results • Needs	Acceptability to medical staff of the screening and CDS tools and screening process.
“Extent to which an innovation can be successfully used or carried out within a given agency.”	Provider, Organization/Setting	More technical side to descriptions of acceptability: Practical instead of evaluative Include workflow issues exemplifying difficulties carrying out innovation. Also, include descriptions of successful use.	• Workflow logistics • Use of centricity • CDS tool usage	Feasibility of implementation of screening into current clinic workflows
Fidelity “Degree to which an innovation was implemented as it was prescribed in protocol or as intended by developers.”	Level of analysis: Provider	Descriptions of people “forgetting” to do screen/review/etc., or simply not screening/reviewing/etc. (e.g. using CDS tools). Also, include descriptions of the use of Centricity. Usage/Non-usage AND aware of existence = Fidelity	• Utilization of TAPS tool instead of previous screening in EHR • Consistency of screening	Fidelity to both consistent conduct of screening during physical visits as well as use of CDS as necessitated by positive screen
Appropriateness “Perceived fit, relevance, or compatibility of innovation for a given setting, provider or consumer.”	Provider, consumer, organization/Setting	Descriptions of patient populations for whom this might not be a good fit; Also useful for descriptions of where providers express frustration that not capturing all people in need due to use at physicals.	• Patient characterizations • Reaction of patients • Screening as a help vs. hindrance to medical care	Appropriateness of screening for patient populations seen across the three clinics; compatibility and value of screening in this clinical setting

^*Text relevant to implementation outcomes often surfaced across more than one interview guide domain.

Setting

Study sites are three primary care clinics (e.g. C1, C2 and C3) affiliated with Penobscot Community Health Care (PCHC), a rural FQHC based in the Bangor, Maine region. With a regional network of nine primary care practices, PCHC is among the largest FQHCs in New England. PCHC provides a wide range of integrated mental health services to residents of Penobscot, Somerset, and Waldo Counties – a region that is predominantly rural, poor, and medically underserved. Much of greater Bangor can be characterized as low-income, plagued by chronic job loss and a lack of high-paying employment opportunities. Many uninsured and underinsured individuals present at PCHC’s primary care practices, walk-in clinics, and homeless shelter/clinic. Also, between 2006 and 2012, more pain pills were prescribed on average per person in Penobscot County than in any other New England county making Bangor an epicenter of the opioid addiction crisis.⁴⁰

Participating clinics were selected by health system leadership based on patient census, use of a common EHR (Centricity), and presence of multiple providers serving adult primary care patients. Clinic 1 (C1) was the first practice in the PCHC network and the first clinic to implement the TAPS screening process. C1 employs more than 20 primary care and behavioral health providers. Clinic 2 (C2), the largest of the three clinics with more than 30 providers, was the second clinic to implement the TAPS Tool screening process. Neither C1 nor C2 had previous experience administering any screening via tablet. Clinic 3 (C3) is the smallest and most rural of the clinics involved in this study. Unlike the other sites, C3 had one previous attempt at implementing tablet-based screening for substance use, which was discontinued due to technical challenges. At the time of the TAPS and clinical decision support tool’s implementation, C2 and C3 had experienced some turnover among their PCPs and therefore had several new PCPs involved with screening patients for substance use.

Participants

Six focus groups (n = 32 participants) were conducted between February and June 2019 with PCPs (3 groups, n = 13) and MAs (3 groups, n = 19) currently working in each of the clinics. All participants were given a study information sheet and provided verbal consent to participate in audiotaped focus groups (See Table 2 for more detail).

Table 2.

Demographic characteristics of primary care providers.

Characteristics	Primary care providers (n = 13)	Medical assistants (n = 19)
Age, m (SD) years	41.5 (14.2)	37.6 (11.5)
Gender, n (%)
Male	2 (15.4%)	1 (5.0%)
Female	11 (84.6%)	18 (95.0%)
Race, n (%)
Asian	3 (23.1%)	0 (0.0%)
White	10 (76.9 %)	19 (100%)
Ethnicity, n (%)
Hispanic or Latino	1 (7.7%)	0 (0.0%)
Not Hispanic or Latino	12 (92.3%)	19 (100%)
Clinic, n (%)
Clinic 1	6 (46.2%)	7 (36.8%)
Clinic 2	5 (38.5%)	6 (31.6%)
Clinic 3	2 (15.4%)	6 (31.6%)
Years practicing medicine, m (SD)	9.8 (10.0)	7.4 (6.4)

Focus groups

All PCPs and MAs at each clinic were invited by email and word-of-mouth to participate in focus groups and offer feedback on their experiences screening patients for substance use. Focus groups were led by a research team member (BM or JM), neither of whom were directly involved with screening implementation at the clinics. Each focus group was approximately 45 minutes in length and participation was incentivized with a $50 gift card. Participants were asked about the screening processes, their perceptions of acceptability and adoption, the clinical utility of the TAPS Tool, the clinical utility and acceptability of the clinical decision support tool, and for suggestions to improve the screening process (See Supplementary Appendix for Moderator Guide).

Analysis

Focus groups were audio recorded and transcribed verbatim. Analytic team members (EH, ES, SM) each read all six transcripts, and listened to the focus group audiotapes from one clinical site each to compare and correct transcripts for accuracy. Given the service context and exigencies of applied health research, rapid analysis (RA)^41,42 was used to deliver timely, valid findings to further inform the process while implementation was ongoing. The steps in RA follow.⁴¹ Initially, after reviews of the transcripts, the analytic team identified a neutral domain name that corresponded with each interview question (e.g. accuracy of results, patient characterizations) and created a summary template (See Supplementary Appendix for Summary Template). Each analyst applied the template to summarize the key content of one randomly selected transcript and assessed its usability and relevance. The analytic team collectively refined the template, establishing consistency across the team and divided up the remaining transcripts to create three-page summary templates for the total dataset.

Next, we used Proctor’s implementation outcomes (acceptability – operationalized in terms of complexity/delivery, comfort, content, credibility – adoption, appropriateness, feasibility, and fidelity) to code the summary templates based on Proctor’s outcome definitions. We modified the implementation outcomes to fit the context of our study in two ways. First, we removed three outcomes (e.g. penetration, sustainability, and costs) that were not expected to be referenced by stakeholders based on our interview questions. Second, we made minor modifications to implementation outcome definitions based on language we found in the data but did not change the meaning of any implementation outcome (see Table 1 for Proctor implementation outcome definitions and study-specific modifications). With new summaries based on Proctor’s outcomes, we created three clinical site-by-role-by-outcome matrices. The site-specific matrices preceded and enabled the creation of a summary matrix (see Table 3) organizing findings across sites by Proctor’s implementation outcomes framework.

Table 3.

Implementation outcome ratings by roles and sites.

Role Site	MA			PCP
	C1	C2	C3	C1	C2	C3
Implementation
Outcome
Acceptability
Complexity/Delivery	O	+	+	O	O	+
Comfort	+	+	+	−	−	−
Content	+	+	+	+	+	+
Credibility	+	+	O	+	+	−
Adoption	+	+	+	O	O	O
Fidelity	+	+	+	−	−	−
Appropriateness	O	O	O	O	O	O
Feasibility	O	+	+	−	O	O

Consensus ratings (i.e. +, −, O):

Established through discussion and deliberation among three analytic team members. We did not quantify discrepancies. However, we can report that it was not difficult to come to consensus on ratings due to the substantial relevant text evidence coded by the different implementation outcomes and multiple levels of analysis through which analysis occurred.

(+)=Implementation outcome surfaced as an indicator of success in the early implementation stages of the TAPS Tool and/or CDS.

(−)=mplementation outcome surfaced as an indicator of the lack of success/or challenge in the early implementation stages of the TAPS Tool and/or CDS.

(O)=Implementation outcome surfaced as a neutral indicator (i.e. credible/reliable interviewees contradict each other, expressing mixed opinions in terms of the indicator’s success and/or insufficient exposure to/experience with the TAPS Tool and/or CDS).

“Because a practice will not be effective if it is not implemented well, implementation outcomes serve as necessary preconditions for attaining subsequent desired changes in clinical or service outcomes.”³⁹ The summary matrix of sites, stakeholders and implementation outcomes was designed to enable evaluation of whether stakeholders perceived their experiences with implementation relevant to the outcomes as positive, negative or neutral indicators of the success of early implementation.⁴³ Analytic team members worked together through a process of discussion and deliberation⁴⁴ to come to consensus on ratings. Ratings were discussed by the analytic team members until everyone came to agreement. The summary matrix illuminates simultaneously and systematically both similarities and differences between stakeholder groups in terms of the implementation outcomes, facilitating identification of indicators of success and potential challenges to implementation. The guidance the team developed and used to evaluate the experiences relative to outcomes is included in Table 3. In results below, participant’ quotes are sourced in parentheses following the quote, e.g. C1 (clinic number): PCP (medical staff stakeholder group), 3 (participant identification number).

Results

Acceptability

Medical staff across sites generally felt that the implementation of the TAPS Tool screening process has been, “overall good” (C1: PCP, 3), and colleagues have been “pretty open” to it (C3: PCP, 2). One exchange among MAs at C2, however, reveals a few less satisfied attitudes among PCPs:

C2: MA, 5: “I think if you asked anyone directly, if it’s important, they’ll say, ‘yes.’ But we don’t also use it as [if] it’s that way.”

C2, MA, 7: “I did hear one provider state that it feels redundant, because they already have to ask those kinds of questions. Yours are more in depth [questions], which is good, but I’m not sure they’re taking that as it’s needed. Because we already know that they do this, we don’t need to know how much they take each day.”

C2: MA, 5: “I don’t need to be told what to say to my patient.”

C2, MA, 7: “That too.”

Complexity/delivery

For MAs at C2 and C3, screening was perceived to be easy to fit into their workflow. MAs at C1 qualified their assessment: “when it works [emphasis added], I don’t have to do anything but just pretty much go click, click” (C1: MA, 4). Compared with MAs, PCPs have experienced more challenges. At C2, PCPs felt the TAPS Tool was “not very user friendly” (C2: PCP, 2). PCPs at C1 similarly suggested that when the technology works, its “fine,” (C1: PCP, 6) but “when it doesn’t work, it drives [MAs] crazy cause they can’t get the ‘note’ (i.e. encounter form in the EHR) going and they can’t get the patient admitted” (C1: PCP, 5). PCPs at C3 found the results easy to read and understand, and unless patients are slow or late to their appointments (i.e. necessitating MA-administration of screening questions), the TAPS Tool can be administered without impeding workflows.

Content

There was near unanimity among PCPs regarding the TAPS Tool’s utility as a “stepping-stone to the conversation” (C1: PCP, 2), particularly around alcohol use. Medical staff at C2 described the TAPS Tool’s results as helpful, explaining that the screening process itself primes patients for discussion about substance use, e.g. “having the iPad before they come in, they kind of expect us to go through it” (C2: PCP, 4). PCPs expressed appreciation for the patient education handouts, and mentioned that patients also appreciated receiving information, particularly on upper limits for alcohol use to compare to their own behavior. At C3, medical staff highlighted the relative advantage of the in-depth TAPS Tool questions compared to standard social history questions previously used.

Credibility

Evidence supporting the perceived credibility of the TAPS Tool screening process emerged in discussions of the accuracy of screening results. Despite acknowledgements that patients may not always be honest about their substance use, the majority of medical staff thought that patients answered more honestly with the TAPS Tool than with prior informal screening efforts. One PCP felt, “you have to believe whatever the patient says. If somebody comes in with chest pain, we might think otherwise, but if they are saying they’re in chest pain, then that’s what we have to believe” (C2: PCP, 2). MAs at C3 expressed concerns about the under-reporting of alcohol use and cannabis use.

There was near consensus that patient self-administered screening promotes greater honesty compared to staff-administered questions; e.g. “when I wasn’t the one asking the questions, ‘it all came out,’ (C1: MA, 3); ‘[patients] are more honest on that thing [tablet] than they are with a human being asking them’” (C1: MA, 4). Though a few PCPs expressed divergent beliefs: “[when patients] are answering more concretely or to a permanent record, electronic device, instead of talking to a person, I find that they lie more often. I’ve had patients who come into the room and say ‘yeah, I just lied to the iPad but I actually do smoke marijuana’” (C1: PCP, 4). At C1 and C2, there was agreement that medical staff sometimes encountered patients who misunderstood questions, and whose results were therefore not accurate. Both the MAs and PCPs suggested changing the language on the TAPS Tool to allow greater specificity with respect to timeframe and frequency of patient’s substance use, e.g. “like, was it a one-time thing?” (C2: MA,7) due to patient discomfort reporting infrequent drug use.

At C3, patient discomfort when screening occurs at initial meetings with new providers surfaced:

It’s been kind of hard for me because I am a new provider and a lot of these patients have had two other providers leave before me, so this may be their first time meeting me …. They don’t necessarily want to talk to me about that (C3: PCP, 2).

However, despite various perspectives on why medical staff may have reason to doubt the accuracy of screening results, there was widespread agreement that the larger point behind use of screening is not to achieve total accuracy/honesty but instead “reach[ing] the objective, which is to open up the conversation” (C1: PCP, 5). “Even if people aren’t being honest, every now and then you are going to save one. And that’s where I think it really counts because if I was in a situation like that, I would want somebody to reach out to me” (C3: MA, 3).

Comfort

Comfort with the use of the screener and clinical decision support tool surfaced in response to questions about the potential need for additional resources or training. MAs were either non-committal (C1) or indicated that they did not have the need for either (C2). However, MAs at C2 did feel that PCPs at their site could benefit from additional training, noting “there seems to be a little bit of confusion” (C2: MA, 5). PCPs across sites did indeed express discomfort. At C1, the following exchange among PCPs reveals anticipatory anxiety about encountering a positive screen;

C1: PCP, 2: “I haven’t been in that situation where a red flag has kind of taken over [need to refocus the visit exam on addressing substance use].”

C1: PCP, 4: “That’s what I worry about… hang on, I gotta find rehab for somebody.”

C1: PCP, 2: “Yeah, exactly.”

C1: PCP, 5: “How are we going to do that?”

At C2, PCPs unanimously endorsed a need for more training, both on using the TAPS Tool, e.g. “It’s probably my own fault for not taking the time to go through it, but I would love to have been able to do a run through of using the TAPS Tool … to totally explore it” (C2: PCP, 4), and on treatment for substance use. And at C3, PCPs expressed feeling unprepared for conversations with patients who are uncomfortable discussing substance use, e.g. “more guidance in terms of when you hit those road blocks or when someone doesn’t want to hear what you are having to say would be helpful” (C3: PCP, 3), as well as a lack of confidence as a new provider making informed referrals.

Adoption

Medical staff descriptions of their respective roles in the screening process for annual physical exam patients offers abundant evidence that the TAPS Tool is being used. Observations that there have been relatively few positive screenings for alcohol and other drugs are another oft-referenced indicator of adoption, e.g. “We haven’t really had any red flags. There was, I think, one yellow that they had drunken alcohol, or something” (C2: MA, 5). One PCP noted, “[screening] is just part of the routine” (C3: PCP, 3) (See Figure 1 for clinic workflows). Beyond extensive evidence of the adoption of the TAPS Tool to screen patients, awareness of and experience with other aspects (e.g. clinical decision support, distribution of handouts, and use of the referral system) varied across individuals and sites. PCPs reported mixed utilization of the clinical decision support tool. At C2, one PCP reported using the clinical decision support tool every time, while another expressed confusion: “the one that drops down? The numbers?” One PCP at C1 said they liked the clinical decision support tool: “I’ve definitely had patients who I would ask ‘how ready are you to change?’ and they surprised me by saying they’re actually very ready to change and I feel like I’ve talked about it to them in the past, and it was not that way” (C1: PCP, 6). Also, the referral tool had reportedly been utilized only by a few PCPs at C1.

Fidelity

Descriptions of MA roles related to the screening process convey overall adherence to clinical workflow and protocols (see workflow descriptions under Feasibility). A few exceptions were noted. Sometimes physical exams (a screening-eligible visit type) were scheduled in the calendar in appointment slots reserved for other appointment types. At C2, MAs mentioned that patients are not routinely screened when the physical exam is scheduled in a nonphysical slot in the appointment calendar: “You only are allowed two physicals a day … to go around it, they’ll book an [outpatient visit] appointment and in comments put ‘physical’”(C2: MA, 6). Because the screening workflow was initiated by the staff at the front desk, misclassifying the visit type in the appointment calendar led to missed opportunities for screening. To counteract this, at C3, MAs said that they verbally-administered the screening questions to patients when more than two physicals per day were scheduled as these patients were not consistently presented with the iPad during the check-in process.

PCPs reported more variability in their role in the screening process. PCPs at C1 were aware of the screening yet, “sometimes still forget to look at it and address it” (3). Reasons for forgetting across various staff roles were due to: familiarity with a patient (e.g., know patient well, so believe no need to review results); having yet to establish use of TAPS Tool as part of annual visit routine; and MAs who, “don’t say anything if it’s negative” (C1: PCP, 3). Unstable fidelity is also acknowledged by several PCPs at C1 who “do not necessarily follow” (C1: PCP, 4) the clinical decision support tool questions. One provider postulated that, “I think I’m just used to talking to [patients] about [substance use] because I did a lot of that before, especially for tobacco and alcohol. So, I just kind of have a thing I do when I talk about it” (C1: PCP, 3). Similarly, one PCP said,

Depending on how the person is responding, [clinical decision support] questions can be kind of awkward … So, when I try to talk to them, I’ll look at those questions but sometimes just say, “So, how often do you do this?” Trying to get them to talk and then try to circle back to those questions if possible, but a lot of the time I just do what I would normally do when I talk to them (C3: PCP, 2).

At C2, implementation fidelity regarding the clinical decision support tool came into question due to unstable or variable use across patient populations. One PCP reported using the readiness/motivation to change tool (see supplemental online material for TAPS Tool Screenshots) within the clinical decision support tool with every positive screen to “grab all the information I can and to clarify,” (C2: PCP, 3) while two others used those questions mostly with positive screens for alcohol, but not for other substances. Some PCPs across sites had distributed informational handouts, while others had not yet used them despite awareness of their existence; and due to limited positive screens across sites, PCPs would appear to be demonstrating fidelity with respect to referrals evidenced by their relative absence (i.e. no referrals were indicated because no patients had screened positive for high-risk alcohol or drug use).

Appropriateness

The overall perceived fit of the TAPS Tool screening process was evidenced in part by what was left unsaid. No stakeholders suggested that implementation of screening was ill-conceived or inappropriate for the primary care clinic setting. At C2, MAs suggested that screening was being implemented at time when a growing number of patients have become aware of substance use and consequently are more willing to discuss use due to the “bad drug epidemic right now … I’ve noticed that the patients are, actually, a lot more willing to talk about things now” (C2). Screening implementation seemed especially beneficial to the MAs because more patients are establishing care within the clinic specifically to obtain treatment for opioid use disorder.

Though the timing of screening implementation is perceived to be opportune, stakeholders across sites identified concerns that screening patients solely at annual physical exams limits the potential reach of interventions: “patients at higher risk aren’t the patients who you’re gonna get in for an annual physical” (C1: PCP, 2). Suggestions for phased expansion of the screening include: “Medicare and Wellness, there’s plenty of people over 65 using substances” (C1: MA, 3), people on Medicare under 65, people who have not been seen in over a year, new and/or transfer patients, those whose insurance no longer covers physicals, and younger patients, e.g. “it may catch things a whole lot sooner” (C3: MA, 4).

While patient self-administered screening on the tablet was deemed highly feasible overall, specific patient subpopulations were identified as potentially ill-suited to self-administered use of the screening via tablet (e.g. intellectually challenged, illiterate, and elderly who struggle with new technology). MAs at C2 discussed one case where there were concerns about the appropriateness of screening:

We have a patient and she can’t hold a conversation with you … Her caretaker answered the questions for her … I told [provider] about it, and she said, “Yeah, she’s not fit to answer her own questions.” And I said, “So it’s okay that her caretaker speaks for her?” And she goes, “Eeh, not necessarily, either” (C2: MA, 5).

One PCP at C3 wondered whether patients who change their answers when their results are discussed fully understand what they are being asked: “I feel like sometimes the patients don’t understand why we’re doing it … it’s a question of did you not understand? Or are you changing because you don’t want the answers to be what they are?” (C3: PCP, 2). The other related concerns raised by medical staff at C1 and C3 regarding patients’ lack of understanding either how to use the technology and/or illiteracy is poignantly underscored here:

[Front desk staff] give it to these little elderly people and they’re embarrassed to say they’d have no idea how to work an iPad. They just sit there dumbfounded with it in their lap … One guy was almost in tears because he was illiterate and embarrassed to say anything (C1: MA, 4).

Feasibility

Medical staff discussed differing degrees of difficulty with integrating screening into their workflows. MAs at C2 and C3 reported that despite initial worries, integration had, “turned out pretty flawless” (C3: MA, 3). Front desk staff at these sites facilitate the process by waiting until patients complete the screening and return the iPad before indicating in scheduling software that patients are ready to be escorted to an exam room. Unlike these sites, several MAs at C1 noted that they had not yet smoothly integrated use of the TAPS Tool into their workflow at the time of the focus group (e.g. C1 was the first clinic to use the TAPS, and technical problems were still being discovered and addressed). Challenges include not knowing how long a patient has had the iPad or if they have been given one at all, precluding MAs from knowing when a patient is ready to be escorted to an exam room. MAs estimated, “it works great … 80% of the time …” (C1: MA, 4).

Staff across sites identified delays in the transfer of results into the EHR. At C1 and C3, the delay seemed to arise when the note was opened before patients completed the screener. At C2, some MAs thought opening the note prior to completion might expedite the process. Despite this issue, MAs at C3 downplayed the delay, “[it’s] like, three seconds. It [just] feels like an eternity because you are ready” (C3: MA, 4). Interviewer-administration of the TAPS Tool (e.g. when patients are late or scheduled for a physical in a nonphysical appointment slot) also reportedly, “slows the process down” but one MA explained, “It definitely saves time having them do it themselves” (C3: MA, 5). The relative advantage of the efficiency of self- over interviewer-administration of the TAPS Tool was expressed well by one MA:

[Patients] struggle more when we’ve had to ask them those questions because some … can’t just give you a clear-cut answer. They have to add a story in there. With answering it themselves, we don’t have that, so it does make it easier (C3: MA, 6).

Compared to the limited role of MAs, PCPs are expected to review, interpret and respond to the screening results. PCPs questioned the use of the TAPS Tool during the annual visit because of time limitations: e.g. “instead of trying to squeeze it all into the physical, it would be helpful to grab all the information using the TAPS, but set it aside perhaps for a follow-up appointment” (C2: PCP, 3). Some PCPs suggested discussing the TAPS results at a separate appointment, because PCPs would “like to spend a bit more time talking to [patients] about this” (C3: PCP, 3). Lastly, another PCP suggested that in her experience, patients don’t want referrals “on the spot” (C2: PCP, 3) and may be more receptive at a later appointment.

Discussion

This study systematically examined the process of implementing the TAPS Tool, clinical decision support tool and referral to treatment in rural primary care to identify indicators of success and potential challenges to screening implementation. The unique study contributions are the focus on the process of implementing screening and exploring early implementation outcomes (versus service or patient outcomes) often not reported in clinical research.³⁹ The study also contributes to the broader effort in the implementation science community to advance understanding of implementation processes. Five noteworthy indicators of success and challenges to the implementation of the screening emerged.

First, although there was broad consensus that the integrated TAPS screening was being used, multiple participants indicated that the clinical decision support and referral to treatment components of this EHR-integrated tool were not as well utilized. The consistent adoption of use of the screening is a necessary precondition for the examination of the implementation process. Due to the few positive screens reported by staff, the finding that the clinical decision support tool was not consistently deployed limits examination of the implementation of these components but is not necessarily an indicator of lack of adoption.

Second, implementation fidelity was stable and high among MAs, yet unstable among PCPs. Fidelity among MAs is an indicator of implementation success. However, unstable fidelity among PCPs across sites (paralleling uneven adoption of the clinical decision support tool) emerged as a notable challenge. PCPs cited lapses in review of the screening results due to familiarity with patients and confidence in their judgment that based on previous patient behavior, no new substance use is likely. Although it is the case that rates of new alcohol or drug use onset significantly decline as people age,⁴⁵ providers who do not screen patients they have treated for some time may miss important diagnoses. Indeed, a systematic review of cognitive biases or flaws in reasoning processes affecting medical decisions⁴⁶ found the presence of cognitive biases (e.g. overconfidence) was associated with diagnostic inaccuracies in 36.5–77% of case-scenarios. Adopting reflective reasoning (e.g. careful, effortful, analytical considerations of each case) has been shown to counteract cognitive biases in medical settings.⁴⁷ Additionally, screening at any visit type would likely increase the number of patients identified as having unhealthy substance use, although concerns related to burden associated with this approach are noted below.

PCPs also cited lapses in review of screening results due to not yet having established the TAPS Tool as part of annual visit routines. Evidence suggesting that professional norms and routines are typically highly stable and not easily influenced by outsiders is abundant,⁴⁸ however, changing these routines is an important intervention target. Underscored by Mittman (2012):

Traditional norms of professionalism favor individual judgment and patient-by-patient decisions over standardized, codified policies and procedures, leading physicians to rely more heavily on their own individual judgment rather than clinical practice guidelines, evidence-based practices, and other summaries of research and guidance.⁴⁹

Addressing PCPs’ unstable fidelity to the process of reviewing the screening results or using the clinical decision support tool may be of critical importance to the long-term sustainability of validated screening given the potential for PCPs to revert to established verbal routines that may or may not accurately screen those at risk. A final reason for unstable fidelity among PCPs was the altered workflow in which MAs intermittently alerted PCPs to a positive screen as the PCP prepared to meet with a patient. This adaptation to the workflow seemed to create an assumption on the part of some PCPs that if not forewarned of a positive screen by the MA, there was no need to review the data. This introduced a risk of missing positive substance use screening results that would likely not be acceptable with other important clinical data such as x-ray results or lab findings. While PCPs were alerted to positive screens if they looked at the TAPS section of the EHR, additional reminders may be needed to facilitate integration of the results review into PCPs’ workflow.⁵⁰

Third, although the TAPS Tool screener was well accepted, lack of comfort and familiarity with the clinical decision support tool limited its acceptability among PCPs. The resoundingly endorsed acceptability of the content and credibility of the TAPS Tool are key indicators of implementation success. When assessing the credibility of the tool, medical staff generally believed that self-administration via tablet promotes greater honesty compared to a medical staff-administered approach. This belief, consistent with findings from the parent study and Phase 1 of this ancillary study,³⁰ is supported by a growing literature documenting a tendency for respondents to be more self-disclosing in a self-administered format.^51–53 However, participants underscored that the primary function of the TAPS Tool was to start a conversation, potentially normalizing the topic as part of routine primary clinic care. Consistent with our findings from the earlier Phase 1 interviews, new providers were the exception and felt patients did not want to discuss alcohol or substance use upon meeting a provider for the first time - weak rapport with PCPs was viewed as a barrier to honest disclosure.³⁵

Aside from some sporadic workflow issues related to screening results transferring into the EHR mainly at C1 (i.e. a known timing issue, identified early and has since been addressed), the TAPS Tool was deemed easy to fit into workflows. In general, technological integration between technology-based interventions and existing EHR systems has been posited to be a critical limiting factor in realizing the public health impact promised by such interventions.⁵⁴ Assessed through the lens of feasibility, the integration of the TAPS Tool into the EHR is a success of this implementation effort.

The lack of comfort among PCPs (e.g. feeling unprepared for conversations with patients about substance use, uncertainty about how to address positive screens), potentially poses the most significant challenge to the implementation of the TAPS Tool.^19,55–58 While many PCPs do not perceive substance use screening or counseling as a desirable or necessary aspect of their role,^59–61 PCPs in these FQHC clinics generally embraced it, but endorsed a need for more training on both the TAPS Tool and in some instances, general treatment for substance use.

Fourth, implementation timing was perceived to be opportune in the context of the opioid epidemic, though all stakeholders expressed concern that screening patients solely at annual physical exam visits limits the reach of interventions, likely missing persons most at risk. Based on this latter feedback, one clinical site now plans to screen all adult patients annually. Studies of barriers and facilitators to screening and brief interventions in primary care and FQHCs have identified the expectation for PCPs to administer screeners to every patient as too burdensome,^62–64 suggesting close monitoring of this adaptation. Another consideration for expanded screening is youth, as suggested by participants. A study assessing the feasibility of self-administered substance use screening to encourage behavior change among young people in primary care (15–24 years) underscored the potential for broad impact of expanded screening to this subgroup for whom substance use prevalence is high.⁶⁴ However, adolescents are not currently included in the USPSTF screening recommendation.⁶

Fifth, certain patient sub-populations were deemed potentially ill-suited to self-administered screening via tablet. A secondary analysis of data from a study validating the TAPS Tool examined the feasibility and acceptability of the electronic self-administered format to directly addresses these concerns.³¹ Almost all participants reported that the TAPS Tool was easy to use with a few caveats: 25% of patients requested some assistance, and patients over 65 required more time and more frequently requested assistance in using the tablet-based TAPS Tool, in comparison to younger patients. The authors recommend that clinics adopting electronic screening be prepared to assist some patients and have the capacity to use an interviewer-administered approach as needed, as is being done by MAs in the study clinics. This issue may be exacerbated in rural regions, where a larger proportion of the population is 65 years of age or older.⁶⁵

Feasible ways to improve screening and implications relevant to future research are suggested by the study findings. We postulate that if more people had screened positive during the early implementation study period, PCPs would have had more experience with the clinical decision support tool likely increasing reports of comfort with this component. In addition to increased experience, research invested in altering professional routines either involving the use of a reflective reasoning intervention,⁴⁷ the creation of opportunities for busy PCPs to explore new tools, and possibly ongoing training following the implementation period are warranted. These strategies may increase comfort with screening, potentially thereby increasing both fidelity and its corollary, accuracy of results. Additionally, research into the optimization of reminders in the EHR (patient-specific information vs. generic reminders; point-of-care vs. follow up appointment in the case of new providers)⁵⁰ is suggested to increase PCP experience with the clinical decision support tool, the numbers of patients whose data are interpreted in a timely manner, and also to improve screening accuracy. Finally, PCPs access the current iteration of the clinical decision support tool by clicking a separate tab. Eliminating this extra step may increase its use, including referral to treatment when indicated.

Screening for substance use is of heightened importance in rural areas which have been disproportionately impacted by the opioid epidemic. These areas also have the greatest shortage of addiction treatment providers, and this could pose a barrier to implementation of a screening program in rural primary care clinics. However, the majority of primary care patients who screen positive for unhealthy alcohol or drug use do not have moderate or severe SUDs,⁶⁶ and thus do not require referral to specialized addiction treatment. Even in more treatment rich environments, there is a lack of evidence demonstrating that patients identified through screening can be effectively linked to addiction treatment programs.⁶⁷ What may be a better approach, particularly in rural areas, is to provide primary care-integrated behavioral treatment for patients and providers who require more support, including for patients who need medication for OUD (MOUD).⁶⁸ Within the rural FQHCs participating in this study, there is access to a collaborative care program for addiction treatment, which is located in one of the three primary care practices and accessible to all patients in the FQHC system. Delivery of integrated care behavioral health and general medical care is influenced by health plan policies. For rural primary care practices without a collaborative care program, systemic changes by both providers and payers will be required.⁶⁹

Telemedicine offers another approach to increasing access to treatment that is particularly well suited to rural settings. The use of telemedicine to provide medication for OUD (MOUD) has been shown to be effective in helping rural consumers lacking local medication treatment and/or behavioral health supports connect with providers whom they would otherwise be unable to access.^70,71 A study of 2012–2017 Medicaid claims data⁷² found low overall telemedicine utilization for behavioral health treatment among rural beneficiaries, with utilization for substance use disorder services being particularly low. However, the greatest observed difference between rural and nonrural beneficiaries was in amount of telemedicine for opioid use disorder (versus telemedicine for alcohol use disorder), with rural beneficiaries having more telemedicine visits on average. Telemedicine is a promising tool for addressing unmet behavioral health treatment needs, especially among Medicaid beneficiaries, and may increase adoption of screening among PCPs who would have greater confidence in their ability to access treatment if such supports were in place.

Reliable and valid measurement is fundamental to advancing knowledge in any field of inquiry and the new field of implementation science with its many measurement gaps is no different.^39,73,74 By using focus groups to further conceptualize implementation outcomes, our study contributes site-specific data by analyzing descriptive language used by stakeholders as they think and talk about implementing the TAPS Tool in a rural, primary care setting. Implementation outcomes are themselves interrelated in complex ways³⁹ and are likely to change throughout a site’s implementation process to adopt empirically-supported innovations, like the TAPS Tool. The matrix analysis revealed relationships among the implementation outcomes of adoption, fidelity and comfort, and complexity and feasibility that may provide insights for implementation science researchers to model early implementation success in healthcare settings or more generally.

Limitations

The salience of implementation outcomes at one to two months following implementation is a caveat framing the discussion. Though predicting the success of implementation may be challenging in the early implementation period, these early outcomes suggest areas for targeted improvement to support adoption and sustainability. This study was conducted in small, rural FQHC-settings. Context is critically important for implementation research,⁷⁵ so the pattern of relationships between outcomes found in this study may be different in larger or more urban settings. The present study examined exclusively data from focus groups with PCPs and MAs. Focus groups are particularly useful for exploring individual’s experiences and eliciting group norms,^76,77 but social desirability bias can sometimes prevent participants from presenting opposing viewpoints. Finally, not all PCPs and MAs participated in the focus groups. Thus, the attitudes expressed may not reflect the full range that may exist in the clinics. Additionally, other staff (e.g. front desk staff, behavioral health providers) and patients were not included in the focus groups, so views of all stakeholders are not represented.

Conclusions

Conceptually, implementation outcomes precede both service (e.g. efficacy) and clinical (e.g. symptomology) outcomes in treatment effectiveness and quality of care research, and thus are important early indicators of both the success and challenges to implementation efforts.³⁹ Heeding these signals is the crucial next step. Addressing PCP comfort and preparedness through increased resources for ongoing training, re-tooling workflows to eliminate potentially erroneous provider’ assumptions, and expanding screening to meet the needs of local communities are all prescribed. As workflows are established and reified, and increasing numbers of positive screens are navigated, it is likely that adoption will continue to expand, fidelity will stabilize, and perceptions of acceptability related to comfort will strengthen to support the sustainment of screening. Early indicators suggest that implementing substance use screening into rural primary care is feasible, allowing for some growing pains (e.g. EHR-integration), and that it is relevant for providers and patients in these settings. Our findings also contribute to a growing body of research informing implementation science through matrix analysis revealing relationships among implementation outcomes.