Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Aug 12.
Published in final edited form as: BMJ Qual Saf. 2018 Oct 31;28(5):397–404. doi: 10.1136/bmjqs-2018-008160

Systems Consultation for Opioid Prescribing in Primary Care: A Qualitative Study of Adaptation

Nora Jacobson 1, Roberta Johnson 2, Bri Deyo 2, Esra Alagoz 3, Andrew Quanbeck 4,*
PMCID: PMC8359873  NIHMSID: NIHMS1724972  PMID: 30381332

Abstract

Background

In order to promote guideline-concordant opioid prescribing practices, a blended implementation strategy called systems consultation was pilot tested in four primary care clinics in one U.S. health system.

Objectives

To describe (1) how systems consultation worked during the pilot test and (2) the modifications necessary to adapt this implementation strategy to primary care.

Methods

A team of investigators conducted observations (n=24), focus groups (n=4), and interviews (n=2); kept contact logs documenting all interactions with the intervention clinics; and preserved all work products resulting from the intervention. Initial analysis was concurrent with data collection and findings were used to modify the intervention in real time. At the conclusion of the pilot test, a pragmatic descriptive analysis of all data was performed to explore key modifications.

Results

Time constraints, entrenched hierarchical structures, and a lack of quality improvement skills among clinical staff were the main barriers to implementing systems consultation. Modifications made to address these conditions included creating a consulting team, giving change teams more direction, revising process improvement tools, supporting the use of electronic health record functionalities, and providing opportunities for shared learning among clinics.

Discussion and conclusion

With the lessons of this research in mind, our goal in future iterations of systems consultation is to give clinics a combination of clinical, organizational change, and electronic health record expertise optimized according to their needs. We believe a streamlined process for assessing the key characteristics identified in this study can be used to develop a plan for this kind of optimization, or tailoring, and we will be developing such a process as part of an upcoming clinical trial.

Keywords: Implementation science, clinical practice guidelines, primary care, evidence-based medicine, qualitative research

INTRODUCTION

Clinical practices lag behind evidence in healthcare. [1] This gap produces an especially urgent problem for opioid prescribing. In the United States since 1999, the number of opioid overdose deaths and the amount of prescription opioids dispensed have both quadrupled.[2] Prescription opioids accounted for more than half of overdose deaths,[3] and about half of opioid prescriptions were written in primary care.[4] Clinical guidelines for opioid prescribing[5] include such evidence-based practices as requiring treatment agreements, urine drug screening, and mental health screening, but the uptake of these practices varies widely.[6] Implementation strategies “are the ‘how to’ component of changing healthcare practice”; they are “the methods and techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice.”[7] The literature has not identified which implementation strategies are effective for getting complex evidence-based practices into use in primary care.[8,9] Strategies targeted at the organizational (rather than individual provider) level are particularly needed.[8]

As its name suggests, systems consultation has its roots in systems engineering. It is blend of implementation strategies derived from the Network for the Improvement of Addiction Treatment (NIATx) model, an evidence-based approach to promoting organizational change, designed for and widely used in addiction treatment agencies. [1113] Systems consultation retains some strategies used in NIATx and modifies others. It retains the use of coaches (outside experts in process improvement who help organizations make changes) as well as specific tools, such as the use of change teams (small groups of clinic staff members who work with the coach to implement changes), walk-throughs and flowcharts (in which staff members experience a clinic process as a patient does and record the results in a diagram), Nominal Group Technique (a process for group decision making), the collection and use of data in rapid-cycle tests (or Plan-Do-Study-Act [PDSA] cycles), and on-site training and regular support of staff. [14] Systems consultation modifies NIATx by using a physician as coach and adding audit and feedback (providing clinic performance data to clinics). Additional modifications—those found during the pilot test to be necessary to adapt systems consultation to the primary care setting—are the subject of this paper.

Although the blend of strategies that constitute systems consultation has not been tested before, the component strategies in the systems consultation model have established roots in implementation science. (See online supplementary Appendix D, which maps component strategies to those identified in the Expert Recommendations for Implementing Change [ERIC] project.[21]) Some of these component strategies, such as audit and feedback, educational meetings, and educational outreach, have demonstrated a modest effect at the level of individual providers. [8] Yet most implementation strategies—like systems consultation—are multifaceted and multilevel,[22] and such interventions have shown mixed results, with little evidence for interventions that target organizations or a wider environment. [8]

We conducted a mixed methods pilot test of systems consultation designed to assess the feasibility, acceptability, and preliminary effectiveness of this implementation strategy in promoting the adoption of guideline-concordant opioid prescribing practices in primary care clinics. The main quantitative results, reported separately, found a 19% reduction in morphine-milligram equivalents in intervention vs. control clinics.[10] The purpose of this paper, which focuses on the qualitative results, is to describe (1) how systems consultation worked in primary care and (2) the modifications necessary to adapt systems consultation to this setting.

METHODS

The intervention

The clinical content of the opioid prescribing implementation strategy was developed by a group of pain medicine, family medicine, and systems engineering experts who engaged in an integrated group process, a systematic method for facilitating consensus among groups of experts[16] to translate opioid prescribing guidelines[5] into a simplified checklist of recommended practices to be implemented using the tools from the NIATx model (described above) that are part of systems consultation (e.g., a walk-through).[10]

In the pilot test, each of four intervention clinics was randomly assigned one of two coaches. The coaches, who also served as co-investigators on the research team, were university faculty physicians certified in both family medicine and addiction medicine. Medical directors at the intervention clinics were asked to form change teams composed of prescribers and other staff, such as nurses, medical assistants, laboratory technicians, and receptionists. Each change team selected a team leader to organize the work of the team and act as the point of contact between the clinic and the researchers.

Change teams began the intervention by conducting a walk-through exercise [17] in which a researcher helped the team experience the clinic’s workflow processes for refilling an opioid prescription, which were then diagrammed in a flowchart intended to help change teams identify obstacles to implementing the checklist. The coach and other researchers later traveled to each clinic for an initial site visit. During this visit, the coach spoke to the change team about the latest research on balancing the benefits and risks of long-term opioid use. The coach also presented the clinic’s opioid prescribing performance data, gathered from the electronic health record (EHR), relative to other clinics in the same health system. The coach reviewed the information gathered during the walkthrough and flowcharting exercise and facilitated a brainstorming session using nominal group technique (NGT),[18] in which the team identified and prioritized the opioid prescribing issues they wanted to address. Finally, the coach introduced the concept of PDSA change cycles,[19] small-scale, incremental changes intended to enable the team to implement their ideas.

In the months that followed, the change teams adjusted their workflows to make their opioid prescribing practices consistent with guideline care and improve their performance on key indicators. Monthly meetings with the coach and other researchers gave the change team opportunities to get expert advice on implementing workflow changes, assessing their impact, and planning for additional changes.

The setting

The four participating family medicine clinics were affiliated with a university health system in the Midwestern United States. Characteristics of the four clinics and the communities they serve (as of February 2015, during the planning phase of the study) are shown in Table 1. Systems consultation was introduced to the intervention clinics on staggered start dates between February and May 2016, ending in each clinic six months later. Just before systems consultation was introduced, the health system announced a new policy related to opioid prescribing but provided little guidance to clinics about how to implement it.

Table 1.

Characteristics of intervention clinics

Characteristics Clinic 1 Clinic 2 Clinic 3 Clinic 4
Clinic
Providers* 3 2 12 7
Total provider FTE+ 2.6 1.25 8.63 6.36
Active panel± 5,320 3,263 16,819 13,361
Providers who prescribe opioids 2 1 12 5
Patients with any opioid orders§ 537 561 1,945 1,341
Patients with 3–9 opioid orders§ 134 166 417 294
Patients with 10+ opioid orders§ 67 106 250 120
Community
Population 9,970 14,301 28,487 26,294
Median age 41.1 39.0 30.9 34.6
Race (% non-white) 1.7 6.2 26.7 11.7
Median household income (US$) 67,250 80,965 87,892 70,127
Percentage of population below federal poverty level 5.3 7.2 12.4 9.1
*

Provider, anyone who can prescribe (medical doctor, nurse practitioner, physician assistant, Doctor of Osteopathic Medicine)

+

FTE, Full-Time Equivalent (1.00 = 40 hours per week)

±

Active panel, number of adult patients with a primary care provider in the specified clinic who have had any type of health care visit in the health system in the past three years.

§

Prescriptions within the previous 12 months.

Study design

This qualitative analysis was conducted as part of the pilot study, which used a randomized matched-pair design with eight primary care clinics, four intervention and four control, to assess the preliminary effectiveness of systems consultation. The qualitative component focused on the feasibility and acceptability of systems consultation, and also explored the modifications necessary to adapt it to the primary care setting, given that systems consultation was derived from a model used in the specialty addiction treatment system. The intervention clinics were randomly selected from a pool of 13 family medicine clinics. The two physician coaches approached the medical directors of seven clinics to recruit four clinics. Details about recruitment, study procedures, quantitative data collection, analysis, and findings are detailed elsewhere. [10,20] The study was approved by the [specific name to come] Institutional Review Board (IRB) at [institution to come]. All change team members at the intervention clinics provided written informed consent.

Qualitative data collection and analysis

Because this study was the first use of systems consultation in primary care, extensive resources were devoted to qualitative data collection using a number of methods, including observation, focus groups, interviews, and document review. A project ethnographer (EA—a doctoral-level qualitative researcher) attended all six-monthly coaching sessions in all four intervention clinics (n =24, a total observation time of approximately 24 hours) and wrote field notes. Two other researchers (BD—a masters-prepared study coordinator; and AQ—a systems engineer and the study multiple-PI) also attended all meetings and provided additional observations.

After the intervention period in each of the four clinics, all change team members were invited to participate a site-specific focus group (n = XX across a total of n = 4 focus groups, one in each intervention clinic). Online supplementary Appendix B shows the size of the teams and the demographic characteristics of members, including focus group participants.) The 60-minute focus groups were conducted by a researcher (RJ—a masters-level researcher and scientific editor) who was unknown to participants. The focus groups explored the change team’s opinions about the implementation strategy components and their perceptions of what supported or hindered their team’s ability to make changes. The same researcher (RJ) also conducted semi-structured interviews (n = 2) with the two coaches. These 60-minute interviews, held after all work with the clinics had concluded, invited the coaches to reflect on their coaching experiences and on each component of the implementation strategy. (Question guides for the focus groups and interviews appear in online supplementary appendixes C and D.) All focus groups and interviews were audio-recorded and transcribed verbatim. In addition to these data, the study coordinator (BD) kept detailed logs of all contacts with the clinics and preserved all products resulting from the change teams’ work.

While the intervention was ongoing, the qualitative working group (NJ—a doctoral-level qualitative research consultant; BD; EA; and AQ; joined by RJ toward the end of the project) met regularly to review field notes and other observations of the coaching meetings. The group inductively identified and wrote memos about patterns in the data. These emergent patterns were then used to shape ongoing data collection. Because the intervention was unfolding concurrently, the qualitative component also functioned as a formative evaluation: discussions during these meetings led to real-time modifications to systems consultation.

For this pragmatic descriptive analysis, the lead author (NJ) constructed case files for each of the intervention clinics, including field notes, focus group and interview transcripts, and the clinic’s work products and contact logs. Using the memos resulting from the qualitative working group’s collaborative analysis sessions, and with the aim of exploring modifications necessary to adapt systems consultation to the primary care setting, she coded all data in the case files for further insights about patterns identified in the earlier stage of the analysis, employing both within-case and across-case comparison to better describe these patterns. The coded material was summarized, and these summaries shared with other members of the qualitative working group, whose comments were integrated into the final analysis.

Importantly, some investigators played multiple roles in this project. The physician coaches simultaneously functioned as components of the intervention, research participants, and members of the research team. BD and AQ were part of the qualitative working group, but also active participants in designing and delivering the implementation strategy, including interacting with the coaches and change team members and providing hands-on assistance to them. These multiple roles risked raising conflicts of interest for the researchers (who were tasked with honestly reflecting on faulty assumptions and shortcomings in their own work) and confusing clinic participants, one of whom expressed uncertainty about whether the researchers were there to “study us or help us.” The doubling and tripling of perspectives led to important insights, however, and the concentration of responsibilities speeded the use of feedback, allowing modifications of systems consultation to be made and assessed quickly.

RESULTS

Table 2 summarizes elements of systems consultation as originally planned and modifications made to the implementation strategy as a result of the pilot test.

Table 2.

Summary of systems consultation elements and modifications

Original systems consultation model Modifications made during pilot test
Physician coach provides both expert advice on opioid prescribing and guidance on organizational change. The term coach was changed to consultant. The physician consultant was joined by a facilitator to form a consulting team, with the physician offering academic detailing (expert advice on opioid prescribing) and the facilitator guiding organizational change and handling logistics.
Clinic change teams lead changes in opioid prescribing practices and processes. Change teams required more direction than planned and made slower progress than when changes were led by individual prescribers. More intra-clinic communication was needed to spread improvements to other staff members in the medium and large clinics.
Change teams use tools: walk-throughs and flowcharts, Nominal Group Technique, performance data, Plan Do Study Act (PDSA) change cycles, a checklist for safe opioid prescribing. Flowcharts were not referred to after the initial meeting. PDSA change cycles were introduced but not used, and informal feedback rather than data was used to evaluate changes. The checklist was superseded by a new health-system opioid prescribing policy. Two tools were added: a script to help prescribers initiate conversations with patients about opioid use and electronic health record training.
Provision of on-site training Half of the 6 monthly consulting sessions were on-site, as planned. The rest were held using distance technology.
Initial two-hour site visit All site visits and monthly meetings were limited to one hour at lunchtime with lunch provided.
Use PDSA cycles to make workflow changes Change was made in response to emerging trends and circumstances, not through PDSA cycles.
No provision for clinics to learn from one another Added a teleconference for the four intervention clinics, at their request

Recruitment

Although clinic recruitment was not conceived as an element of systems consultation, an important lesson from the pilot test of systems consultation was that despite the timeliness of the topic, recruitment required not only multiple email and in-person contacts, but also collegial relationships between physician consultants and clinic medical directors to recruit four intervention clinics. While full or half day kick-off sessions are common in the NIATx model, it soon became clear that in primary care 60 minutes was a hard time limit for the kick-off and all meetings. The best way to boost attendance at the monthly sessions was to schedule them over the noon hour and provide lunch.

Coaching

The first modification to the implementation strategy was a semantic one: the terms coach and coaching were changed to consultant and consulting because primary care physicians explained to researchers that “doctors don’t like to be coached.”

The two physician consultants had very different styles. Consultant 1 was directive, providing not just advice, but also explicit instructions. Consultant 2 offered options and resources, but rarely told clinics what to do. Before the pilot study began, the assumption was the less directive approach would be more effective because it had been in addiction treatment organizations. In fact, clinic change teams had neither the time nor the skills to do the open-ended work required by systems consultation, and often expressed a desire for more direction. As one change team member said, “[the consultant’s] role was not to give us guidance [in identifying a problem], we were supposed to come up with that alone…[I]f they had come in…with an idea, instead of us doing it, that would have been more satisfying.”

The two consultants brought different backgrounds to the project. Consultant 1, although certified in addiction medicine, maintained a practice at another university-affiliated family medicine clinic. She was familiar with the types of patients seen in the clinics she was consulting with, with clinic workflows, and, very importantly, with clinics’ EHR (Epic Systems). She presented information in ways that made sense to the teams—likening the way they should approach chronic pain, for example, to the way they approached other chronic conditions such as diabetes. One change team member described this consultant as “Boots on the ground. Practices what she preaches.” Consultant 2 practiced addiction medicine exclusively, but in hospitals and other settings as well as primary care, and thus lacked such intimate familiarity with the clinics. Although the teams respected both consultants’ expertise in addiction medicine, they seemed also to resist its relevance. Of Consultant 1, a team member opined that she appreciated her “family med side,” but her “addiction med side” was not as useful, because “our patients are not addicts.”

Systems consultation assumed that the physician consultants would provide expertise in opioid prescribing and guide organizational change. The two consultants received several hours of training from experienced NIATx coaches, but in the field, it became clear that the consultants lacked many of the skills needed to help clinic teams analyze their workflows, identify process improvements, and run test cycles. As one consultant noted, this role “was really unfamiliar. It was a steep learning curve.” (They also did not have time to handle the many logistical details involved in the implementation strategy.) Another researcher stepped in as a facilitator to take over these tasks. When later clinics started the intervention, it had been reconceived as a team consulting model, with the physician consultants doing something akin to academic detailing and the facilitator handling process improvement and logistics.

This team consulting approach addressed another challenge. From recruitment onwards, the researchers noted that the primary care setting was more hierarchical than the addiction treatment setting: physicians possessed much more authority (and responsibility) than other staff members, while in addiction treatment staff tend to be on more equal footing. The physician consultants established good rapport with the clinic medical directors and physician change team members, but the non-physician change team members often were not active participants in their conversations. Physicians were empowered to make some changes in their own workflows, and to direct changes in the work of others, but non-physician members were not. When physician change team members were tardy or absent at consulting sessions, non-physician members often seemed reluctant or unable to discuss the status of changes the team was making. Once the team consulting model was in place, the facilitator, who was not a physician, was able to encourage and support non-physician change team members, who became more active.

The pilot test revealed the importance of conveying to clinics clear expectations about the consultant role. Consultant 2 described a “tension” around being asked by teams to serve as a clinical consultant for questions about “difficult cases and challenging patients.” Consultant 1 often provided technical support in using the EHR (e.g., demonstrating which button to click to get to a specific screen). Clinical consultation and EHR support were related to the clinics’ management of opioid prescribing, but they were time consuming and tangential to the organizational change aims of the consulting sessions. Clearly, however, clinics needed these services, suggesting enhancements to future iterations of the implementation strategy.

Change teams

Clinics were asked to form multidisciplinary change teams that included at least one prescriber. Some clinics solicited volunteers; in others, managers assigned people to the team. Teams chose their own leaders. The composition of each team is shown in online supplementary Appendix B.

Multidisciplinary representation on the team was important. In the focus groups, team members noted that changing a workflow requires understanding the tasks performed by staff members in all occupations involved and securing their cooperation to make the change. The change process wouldn’t have worked without a multidisciplinary team, one team member explained, because “we all see healthcare from a different perspective.” Assigning leadership of the teams to non-prescribers was not as effective. While non-prescribers successfully handled logistics, they were not able to demand accountability from other team members. Given the hierarchy of the setting—reinforced by structural constraints such as scope-of-practice regulations—only prescribers could make final decisions about changes, and implementation depended on their actions. Clinics that progressed quickly and easily toward improving their opioid prescribing practices relied less on change team-based processes and more on individual prescribers deciding to change their practices, thus eliciting shifts in the workflows of their support staff.

Researchers assumed that clinic change teams would have a basic proficiency in the theory and practice of quality improvement. At the initial consulting sessions, change teams completed the NGT exercise and voted to prioritize the issues raised. The researchers then issued hurried instructions to begin a PDSA cycle on the issue of highest priority. However, teams lacked the skills to do this work. For the first clinics in the pilot test, teams floundered for several months. By the fourth clinic, the team was given more direction; this clinic had a much smoother path from identifying problems to solving them.

The planned intervention did not address communication between the change team and other clinic staff. This did not cause any difficulty in the small clinic, where the change team constituted nearly the entire staff, but it did pose a problem in the one medium and two larger clinics, when staff members affected by workflow changes were not informed about the reason for the changes and quickly became resentful. From this, the researchers learned the importance of encouraging intra-clinic communication about the intervention from the beginning. Relatedly, the researchers assumed that in the larger clinics workflow changes implemented by change teams would at some point be rolled out to the entire clinic, although teams were not given specific instructions for how this should be done. Post-intervention, teams were contacted informally to find out how things were going. Some change teams were still meeting regularly, but it did not appear that most had taken steps toward expanding changes clinic wide. Building frequent intra-clinic communication into the intervention might be important to sustainability.

Change tools

The tools included in the systems consultation intervention had varying degrees of utility. The walk-through exercise was described as “eye-opening” and “beneficial…for understand[ing] people’s roles” by change team members who participated in it, and the flowchart that resulted from it prompted useful conversations at the initial consulting sessions. However, only one member of one team referred to her clinic’s flowchart after the initial meeting. Participants were enthusiastic about the NGT exercise, which they found effective for focusing on clinic practices and building change team cohesion. The checklist was presented to the change teams early in the intervention, but at the focus groups few participants remembered seeing it and none of the teams used it. Instead, the guidance offered by the checklist came from the new system-wide opioid prescribing policy. (Because both were based on extant clinical guidelines, the checklist and the policy were consistent.) Change teams took great interest in the performance data and were curious about how they compared to other clinics in the sample. However, the data proved problematic in several ways: first, because the teams were not presented with benchmarks, they could not discern how they were doing in an absolute (rather than relative) sense; second, the data always lagged a month or two behind and were subject to some imprecision. As one change team member noted, “[the data] had a lot of flaws. And so it was difficult to tell if we were making progress or not.”; and third, the teams did not use the PDSA approach, which involves using data to assess changes being made. Teams lacked the analytic skills to conceptualize their workflows as a series of steps and to identify and implement small-bore solutions. The new prescribing policy created anxiety, and teams felt pressured to implement all aspects of the policy at once, rather than, as one change team member said, “piecemealing it.” Although the teams did make workflow changes, they almost never collected data to test the changes, instead relying on informal feedback from colleagues and patients. The PDSA approach included a form for recording teams’ test cycles. A change team member described this form as “confusing” and “overwhelming.” Change teams never independently completed the forms, relying instead on the facilitator to do it.

During the intervention, the researchers and the change teams worked together to develop new tools to respond to clinics’ needs: a script for initiating conversations with patients about opioid use; training related to a new EHR workbench, created independently of the project, designed to monitor and prompt guideline-concordant opioid prescribing; and a teleconference for all four clinics’ change teams, held in response to change teams expressing an interest in other clinics’ challenges and solutions. (A similar learning collaborative structure is part of the NIATx model but was not originally included in systems consultation.)

DISCUSSION

The systems consultation approach assumes organizational change is made incrementally, by an empowered team, supported by outside experts, who use data to identify problems and plan and test workflow modifications. This pilot test of systems consultation applied to opioid prescribing in primary care challenged some of these assumptions and suggests lessons for others seeking to implement evidence-based practices in primary care using systems consultation or a similar suite of implementation strategies. While the strong external emphasis on opioid prescribing from the health system and the news probably helped motivate clinics to participate in the pilot study, the pressure of the new system-wide opioid prescribing policy also made it difficult for clinic staff to conceptualize change as incremental, and, as a result, to see the PDSA approach as a way to accomplish their goals. The hierarchy in primary care constrained the ability of non-physicians to make workflow changes. Decisions about changes were made not by multidisciplinary teams of equals, but rather by individual prescribers, then implemented with the help of other staff members. The hierarchical environment also limited the ability of physician consultants to motivate non-physician members of the change team. Rogers’ concept of homophily [15] —the degree to which external change agents resemble the internal actors whose behavior they are trying to influence—helps explain this observation. It also suggests why the physician consultant whose professional experience most closely matched that of the teams with whom she was working appears to have been more effective at providing guidance.

As conceived, systems consultation assumed that the physician consultants would be skilled in promoting organizational change. In fact, while the physician consultants possessed clinical expertise about opioid prescribing—which clinicians were very eager to access, especially when grappling with challenging patients—they were not as adept at guiding process improvement. Similarly, the change teams lacked quality improvement skills and appreciated being given explicit direction. The researchers initially underestimated how central the EHR is in the primary care setting. It became clear that clinical staff depended on the EHR not just for patient history, but also to monitor and structure their work. Building EHR functionalities to support change and supporting clinic team members as they learned to use these functionalities, proved essential to organizational improvement. With all these lessons in mind, our goal in future iterations of systems consultation is to give clinics a combination of clinical, organizational change, and EHR expertise optimized according to their needs.

This study has notable limitations and strengths. Investigators evaluated an intervention of their own design; they are not disinterested assessors. Clinics were self-selected; barriers to implementation might be higher or different in clinics that were not motivated to participate in the pilot test. Patient populations were predominantly white and fairly high income; clinic staff were entirely white. We cannot speculate how systems consultation might work in clinics with greater ethnocultural and economic diversity. The coincident introduction of the health system’s opioid prescribing policy was very motivating to prescribers and other clinic staff. We do not know what might have supported or hindered systems consultation if the policy had not been in place. On the other hand, as noted earlier, the dual roles played by investigators was a strength as well as a limitation. The phased design of the study was a strength, since it enabled the researchers to observe and analyze what was working and not working and make modifications that could be tested in another clinic. The pilot study was small enough that investigators could observe all intervention interactions and conduct a focus group or interview with all participants, without sampling bias. Finally, the multidisciplinary qualitative working group allowed for multiple perspectives in interpreting the data.

CONCLUSION

The qualitative component of our pilot study taught us important lessons about adapting systems consultation to primary care. Although the resource intensiveness of qualitative data collection and analysis makes it infeasible to replicate this qualitative study in larger scale implementation efforts, we believe that a streamlined process for assessing the key contextual characteristics identified in this study (such as a clinic’s experience with quality improvement efforts) as well as other factors reported in the literature [23, 24, 25] can be used to guide tailoring, and we will be developing such a process as part of an upcoming clinical trial of systems consultation focused on optimization and cost effectiveness.

Supplementary Material

Supplementary Appendix B
Supplementary Appendix A
Supplementary Appendix C
Supplementary Appendix D

Funding

The National Institute on Drug Abuse (NIDA) is the primary funder of the study (R34-DA-036720-01 and 1K01-DA039336-01). The funder had no role in study design, the collection or interpretation of data or the publication of results.

Footnotes

Competing interests AQ has a shareholder interest in CHESS Health, a public benefit corporation that disseminates software to the specialty addiction treatment system. The relationship between the author and CHESS Health is managed by the University of Wisconsin–Madison’s Conflict of Interest Committee.

Ethics approval The study protocol was reviewed and approved by the [institution’s] [specific] Institutional Review Board, submission 2015-0280-CR002.

References

  • 1.Berwick D Disseminating innovations in health care. JAMA 2003;289:1969–75.doi: 10.1001/jama.289.15.1969. [DOI] [PubMed] [Google Scholar]
  • 2.Centers for Disease Control and Prevention (CDC). Wide-ranging online data for epidemiological research (WONDER). https://wonder.cdc.gov (accessed September 2017).
  • 3.Rudd RA, Seth P, David F, et al. Increases in drug and opioid involved overdose deaths – United States, 2010–2015. MMWR Morb Mortal Wkly Rep 2016;65:1445–54 doi: 10.15585/mmwr.mm655051e1. [DOI] [PubMed] [Google Scholar]
  • 4.Daubresse M, Chang HY, Yu Y, et al. Ambulatory diagnosis and treatment of nonmalignant pain in the United States, 2000–2010. Med Care 2013;51:870–8 doi: 10.1097/MLR.0b013e3182a95d86. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Chou R, Fanciullo GJ, Fine PG, et al. Clinical guidelines for the use of chronic opioid therapy in chronic noncancer pain. J Pain 2009;10:113–30 doi: 10.1016/j.jpain.2008.10.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Centers for Disease Control and Prevention (CDC). Vital signs: opioid prescribing. https://www.cdc.gov/vitalsigns/opioids/index.html (accessed March 2018).
  • 7.Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci 2013;8:139 doi: 10.1186/1748-5908-8-139. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Lau R, Stevenson F, Ong BN, et al. Achieving change in primary care – effectiveness of strategies for improving implementation of complex interventions: systematic review of reviews. BMJ Open 2015;5:e00999 doi: 10.1136/bmjopen-2015-009993. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Wensing M, Bal R, Friele R. Knowledge implementation in healthcare practice: a review from The Netherlands. BMJ Qual Saf 2011;21:139–42 doi: 10.1136/bmjqs-2011-000540. [DOI] [PubMed] [Google Scholar]
  • 10.Quanbeck A, Brown RT, Zgierska AE, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for opioid prescribing in primary care. Implement Sci 2018;13:21 doi: 10.1186/s13012-018-0713-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Gustafson DH, Quanbeck AR, Robinson JM, et al. Which elements of improvement collaborative are most effective? A cluster-randomized trial. Addiction 2013;108:1145–57 doi: 10.1111/add.12117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.McCarty D, Gustafson DH, Wisdom JP, et al. The Network for the Improvement of Addiction Treatment (NIATx): enhancing access and retention. Drug Alcohol Depend 2007;88:138–45 doi: 10.1016/j.drugalcdep.2006.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Hoffman KA, Ford JH 2nd, Choi D, et al. Replication and sustainability of improved access and retention within the Network for the Improvement of Addiction Treatment. Drug Alcohol Depend 2008;98:63–9 doi: 10.1016/j.drugalcdep.2008.04.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Gustafson DH, Johnson KA. The NIATx Model: Process Improvement in Behavioral Health. Madison, WI: University of Wisconsin – Madison; 2011. [Google Scholar]
  • 15.Rogers EM. Diffusion of Innovations. New York, NY: Free Press; 1995. [Google Scholar]
  • 16.Gustafson DH, Hundt AS. Findings of innovation research applied to quality management principles for health care. Health Care Manage Rev 1995;20:16–33. [PubMed] [Google Scholar]
  • 17.Ford JH 2nd, Green CA, Hoffman KA, et al. Process improvement needs in substance abuse treatment: admissions walk-through results. J Subst Abuse Treat 2007;33:179–89 doi: 10.1016/j.jsat.2007.02.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Delbecq AL, Van de Ven AH, Gustafson DH. Group techniques for Program Planning: A Guide to Nominal Group and Delphi Processes. Glenview, IL: Scott, Foresman; 1975. [Google Scholar]
  • 19.Deming EW. Out of Crisis. Cambridge, MA: Massachusetts Institute of Technology; 1986. [Google Scholar]
  • 20.Quanbeck A, Brown RT, Zgierska AE, et al. Systems consultation: protocol for a novel implementation strategy designed to promote evidence-based practice in primary care. Health Res Policy Syst 2016;14:8 doi: 10.1186/s12961-016-0079-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci 2018;10:21 doi 10.1186/s13012-015-0209-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Mittman BS. Implementation science in health care. In: Brownson RC, Colditz GA, Proctor EK, eds. Dissemination and Implementation Research in Health: Translating Science to Practice. New York: Oxford University Press; 2012:400–419. [Google Scholar]
  • 23.Powell BJ, Beidas RS, Lewis CC, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res 2017;44:177–194 doi 10.1007/s11414-015-9475-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Robertson N, et al. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev. 2015;4 10.1002/14651858.CD005470.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Lewis CC, Scott K, & Marriott BR (2018). A methodology for generating a tailored implementation blueprint: an exemplar from a youth residential setting. Implementation Science, 13(1), 68. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Appendix B
Supplementary Appendix A
Supplementary Appendix C
Supplementary Appendix D

RESOURCES