Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Sep 25.
Published in final edited form as: Prog Community Health Partnersh. 2024;18(3):309–321.

Patient perspectives adapting collaborative care for opioid use disorder with depression and/or posttraumatic stress disorder

Isabel M Leamon 1, Vanessa Jacobsohn 2, Alex R Dopp 1, Grace Hindmarch 1, Venice Ceballos 2, Miriam Komaromy 3, Katherine E Watkins 1, Karen Chan Osilla 4
PMCID: PMC11424021  NIHMSID: NIHMS1999191  PMID: 39308375

Abstract

Background:

Individuals experiencing opioid use disorder (OUD) and co-occurring mental health concerns experience heightened consequences and lower rates of treatment access. Engaging patients as research partners alongside health systems is critical for tailoring care for this population.

Objectives:

Collaborative care is promising for the treatment of co-occurring disorders, but there is little research incorporating patient perspectives into its design.

Methods:

We utilized the Community-Participatory Partnered Research (CPPR)1 approach to partner with patients, providers, and clinic administrators to adapt and implement a collaborative care intervention for co-occurring disorders in primary care. We conducted qualitative interviews with patients to assess their feedback on the proposed collaborative care model prior to intervention implementation. A first round of interviews was conducted to obtain patient feedback on our adaptation ideas (n=11). The team then incorporated these suggestions and beta-tested the intervention with additional participants (n=9) and assessed their feedback. Data were analyzed using rapid content analysis and then implemented by health systems.

Results:

Patient feedback underscored the need for the care coordinator (CC) to be trained in patient engagement and stigma reduction and to provide assistance around socioeconomic barriers and relapse. Patients shared that it was helpful to have the CC address co-occurring disorders, emphasized the need for the CC to be flexible, and expressed that telehealth was acceptable. Patient feedback was integrated in subsequent CC training with health systems.

Conclusions:

The present research demonstrates the feasibility and utility of incorporating patient perspectives into treatment design and implementation in health systems using CPPR.

Keywords: Community health research, Community-Based Participatory Research, Opioid use disorder, Patient stakeholders, Collaborative care, Primary care

Introduction

Individuals who experience co-occurring opioid use disorder (OUD) and mental health conditions experience high disease burden that require treatment for multiple complex conditions and they often experience poorer treatment outcomes in traditional healthcare settings.214 Fewer than 25 percent of individuals with co-occurring OUD and mental health conditions receive treatment for both conditions.15 Low rates of treatment utilization is particularly problematic because of the severity of consequences this population experiences. For example, individuals with co-occurring OUD and severe depression or PTSD are significantly more likely to experience a recent overdose and suicidal ideation or attempted suicide compared to individuals with OUD who did not have a co-occurring mental health condition.16,17

Increasing the availability of treatment for OUD and co-occurring disorders in primary care settings may be one mechanism for improving treatment utilization.18,19 Collaborative care is an effective service delivery intervention that uses a team-based approach to improve primary care treatment access, quality, and outcomes for individuals with mental health conditions or substance use disorders. The team typically consists of a care coordinator (CC), who helps link patients to care and communicate with the patient’s care team; a primary care physician (PCP), who performs traditional primary care functions; and a behavioral health consultant (BHC), who acts in a consultative capacity to provide input on behavioral health treatment decisions.20

There is substantial evidence that collaborative care can help alleviate barriers to treatment utilization and care integration compared to usual care services. Specifically, collaborative care improves outcomes such as medication adherence, utilization of behavioral healthcare, remission, and quality of life for patients with behavioral health disorders21 such as depression2225 or posttraumatic stress disorder (PTSD).26 Collaborative care has also been effective in treating opioid and alcohol use disorders2731 and in treating depressive disorders comorbid with substance misuse.32,33 In these populations, collaborative care increased receipt of treatment including psychotherapy, medications, and faith-based services, and decreased alcohol and opioid use.31,32 Collaborative care is also highly effective for disadvantaged populations (e.g., low income, racial/ethnic minority) and can reduce disparities in treatment utilization.3439 Despite the evidence of collaborative care’s effectiveness in treating substance use disorders or mental illness, there is insufficient research on the impact of collaborative care on individuals who have co-occurring disorders.

The current study is part of a randomized clinical trial called Collaboration Leading to Addiction Treatment and Recovery from Other Stresses (CLARO) and examines the effectiveness of collaborative care adapted for OUD with co-occurring PTSD and/or depression.40,41 We utilized the Community-Participatory Partnered Research (CPPR)1 approach to partner with patients and health systems (e.g., medical and behavioral health providers, clinic administrators, clinic CCs) to adapt and implement a collaborative care intervention in primary care. The CPPR approach emphasizes the joint leadership between community members and the research team, where the community becomes part of the research team and the research team becomes part of the community. In this two-way knowledge exchange, a health issue that is prioritized by patient, health system, and academic partners is identified (i.e., co-occurring disorders), a coalition of diverse partner representatives is engaged (i.e., patients, clinic champions, clinic CCs), meetings are convened to obtain input, and work groups execute recommendations from these meetings.

CPPR partners are involved in three phases of a research study: planning the study, executing the study, and dissemination of findings to health system and community audiences.42 Patients are critical members of the partnership. Incorporating patient perspectives into the design of treatment models is a crucial element of high quality, patient-centered care.43,44 Comparisons of patient and provider perceptions of health services demonstrate that patients often identify different barriers to care and priorities for treatment compared to providers, suggesting that the omission of patient perspectives can decrease the accessibility and acceptability of treatment interventions and cause health systems to overlook critical factors during intervention implementation and delivery.45,46 Patient perspectives have provided important insights into the feasibility and acceptability of collaborative care interventions for mental health in primary care, including the overall model and specific components such as measurement-based care.47,48 Data on patient experiences have informed primary care clinics on how to implement integrated care models in ways that optimize patient preferences, engagement, and experience.49,50 The current study aimed to solicit patient perspectives on the delivery of collaborative care for co-occurring disorders within a CPPR approach that enabled health system partners to engage with and learn from patient partners.

Obtaining input from individuals experiencing co-occurring disorders is especially important because these individuals are frequently marginalized within traditional treatment settings, due to institutional (e.g., siloed systems of treatment) and sociocultural (e.g., stigma) factors.51,52 Concomitantly, prior studies suggest that patients with co-occurring disorders have fewer opportunities to engage in shared decision making or provide feedback in their day-to-day interactions with providers51,53 and have high levels of mistrust toward providers.52

We utilized an iterative approach to design collaborative care for co-occurring disorders (specifically OUD with PTSD and/or depression) in primary care based on feedback obtained through patient interviews before and after drafting an intervention prototype. A previous publication described our overall adaptation process for creating CLARO.40,41 In the current article, we provide a more in-depth description of how patient perspectives informed the adaptation of the collaborative care model for co-occurring disorders and how these perspectives were incorporated into the final model and implemented by health systems. Our study sought to answer the following questions:

  • What features should the proposed collaborative care model for co-occurring disorders include in order to address existing barriers to care? (Interview 1)

  • How acceptable and feasible is the proposed collaborative care for co-occurring disorders prototype? (Interview 2)

  • How can patient feedback be incorporated into collaborative care implementation and delivery activities by the health systems?

Methods

Setting

We initiated our CPPR partnerships with three health systems in New Mexico (NCT04559893).40,41 The clinics within these health systems were chosen because they predominantly serve low-income and Hispanic patient populations, and they are located in the regions of New Mexico with the highest rates of opioid overdose (Northeast and Central) and primarily rural regions (Southwest).54 All participating clinics were located in Health Professional Shortage Areas, indicating that there were insufficient behavioral health and/or medical care providers to meet community needs.55

Patient Participants

Of the 11 patients recruited for the first interview (June-July 2020), 7 were recruited from the Northeast region, 3 from the Central region, and 1 from the Southwest region. Of the 9 participants recruited for the second interview (September 2020), 5 were recruited from the Northeast region, 3 from the Central region, and 1 from the Southwest region. Clinics were not conducting collaborative care prior to our research study, but provided outpatient pharmacotherapy and psychosocial therapy for OUD and co-occurring mental health disorders. All participants were taking buprenorphine at the time of their interviews, but few were receiving psychosocial treatment.

Procedures

We worked with clinic staff for about a year prior to implementation to promote the health system partnership, learn about clinic workflows and climate, guide study implementation, and identify clinic champions (one to three per health system).56,57 Clinic champions and other health system representatives (e.g., providers, administrators, and CCs) were actively involved in planning for, executing, and interpreting data. They drafted and reviewed recruitment materials, created workflows, interpreted patient feedback, worked with CCs and other clinic staff to implement the intervention, and provided continuous guidance on how best to implement the trial in their clinics. We also engaged in additional implementation strategies with health systems during the clinical trial, which are elaborated on in another report.58

Prior to adapting our collaborative care intervention, we engaged medical provider partners to recruit patients currently receiving treatment for OUD with depression/PTSD at one of the participating clinics. Provider partners described the study to the patients, informed them that their care would not be affected by their decision to participate or not, and asked interested patients to complete a consent-to-contact form. The consent-to-contact form was sent to the research team via secure fax; a research team member then contacted the patient to describe the study and invite the patient to participate.

Patients were eligible to participate in two phone interviews. The first was conducted prior to adapting the collaborative care intervention to solicit patients’ input on the acceptability of having a CC work directly with patients to address co-occurring disorders in primary care. Participants were told that a CC is someone who helps with tasks such as making appointments, providing appointment reminders, assessing symptoms, describing treatment options, linking patients to resources, and being the “glue” between the patient and their care team. In the interview, participants were asked about prior experiences with substance use and mental health treatment (e.g., What suggestions do you have for how your health care team, including primary care providers and therapists, could improve your experience getting treatment for opioid use? How about for depression and/or PTSD?), facilitators and barriers to treatment access and retention, and initial thoughts on the proposed collaborative care model (e.g., How frequently do you think it would be helpful to connect with a care coordinator? What things would be important to check-in on that might influence whether someone drops out of treatment and goes back to using?). The interviews lasted about 60 minutes and participants received a $30 gift card for their time.

We used feedback from the first interview to draft our intervention and then beta-tested an intervention session with patients in the second interview. For the beta-testing intervention session, a member of the research team role-played as the CC and the participant role-played as a patient seeking treatment for OUD and depression/PTSD. The CC asked the patient about current depression and PTSD symptoms, opioid use, and treatment goals as outlined in the draft intervention protocol for a typical first visit; patients were encouraged to draw on their lived experiences but could invent a character or specific details rather than sharing personal information if they preferred. The CC and patient role-played discussing treatment options and determined next steps. After the session, the CC disconnected from the call, and a second researcher led the participant in a debriefing interview. During the debrief, participants were asked for their thoughts on the session’s helpfulness, acceptability, relevance to patients at different stages of treatment, and areas for improvement (e.g., What did you think about the session? How helpful do you think this session would be in helping people who are beginning treatment?). The beta-testing session lasted about 30 minutes and the interview that followed lasted about 60 minutes. Participants received a $40 gift card for their time.

Two research staff participated in the first interview (one led the call and the other typed notes) and three in the second (one led the mock intervention session, one led the debrief, and one typed notes). A total of four research staff rotated to administer the interviews. Before each interview, research staff explained the study and obtained verbal consent from all participants. Calls were recorded and the research team reviewed the recordings to supplement any gaps in the notes. Training/supervision, implementation monitoring, and quality improvement activities with our partnering health systems were conducted after adapting the collaborative care intervention (see Results). The study procedures were approved by RAND’s Institutional Review Board.

Data Analysis

We utilized rapid content analyses to synthesize common themes across interviews. Rapid content analysis allows for teams to efficiently glean insights that can be incorporated while intervention design and implementation are ongoing. Our approach started with having a first coder write detailed notes during each interview and then categorizing those notes into themes after the interview; then, a second coder validated the themes by listening to the interview recordings and providing feedback on themes. Coders (Interview 1: IL, KO, VJ; Interview 2: IL, KO, AD, GH) extracted key quotations or concepts that contained recommendations for the intervention (Interview 1) or pertained to the intervention’s feasibility and acceptability (Interview 2).59,60 The researchers discussed the quotations and concepts and generated themes from the data in weekly meetings. Interrater agreement was high. Reaching sufficient sample size was ultimately determined by thematic saturation, which was defined as the point in which data collection was redundant and themes began to repeat with no additional insights identified.61

Results

Participant Characteristics

Medical providers at participating clinics nominated 13 patients who were currently on buprenorphine for OUD and had a history of co-occurring PTSD or depression. A total of 11 participated in the first round of interviews and 9 of the 11 participated in the second. The individuals who did not participate in the interviews did not respond to the research team’s contact attempts: none of the nominees were excluded from participation and none explicitly refused to participate. Table 1 provides a summary of participant demographics.

Table 1.

Participant Characteristics

Participant Characteristics Interview 1: Recommendations
(n = 11)
Interview 2: Beta-Testing
(n = 9)
Age, m (SD) years 52.36 (9.46) 55.56 (9.90)
Gender, n (%)
 Male 3 (27.27%) 3 (33.33%)
 Female 8 (72.73%) 6 (66.67%)
Race/ethnicity, n (%) *
 Hispanic 6 (54.55%) 4 (44.44%)
 White 5 (45.45%) 5 (55.56%)
*

Participants provided an open-ended response to the question, “What is your race or ethnicity?”

Note: SD = standard deviation.

Interview 1: Recommendations for Collaborative Care for Co-Occurring Disorders

Participant feedback fit into the following themes: Ensure the CC is trained in patient engagement and stigma reduction, address social determinants of health, provide support for managing prescriptions, taper the frequency of CC visits over time, and check in with patients regarding possible relapse triggers. Table 2 lists the themes along with representative quotes for each theme.

Table 2.

Interview 1 Themes

Recommendation n Representative Quotes
Ensure CC is trained in patient engagement 10 My doctor is so great that I can’t wait to talk to him and hear his voice. We have almost like a friendship. A care manager should be (like that). (P8)

(My doctors) relate, understand, and provide guidance when I have problems. My doctor tells me, “Do this or do that, and if anything happens, please call us”… That gives me a positive reinforcement that if I have a problem, my doctor will help me. (P11)

My main problem with therapy is feeling like it’s okay to talk to somebody without feeling judged. Am I going to be able to walk out of this appointment and feel like it’s not going to be all over town or they’re going to tell somebody? Are they going to get it? Because there are people out there that aren’t going to get it. It’s the trust that’s the main thing. And being judged. (P1)

It seemed like the people there, especially at AA, the people thought that if you took any medications you weren’t in true recovery. I didn’t like that. That’s the way they feel. At NA, I just never could feel like I was important enough to talk. (P2)
Address socioeconomic barriers 5 Suboxone can be very expensive. I pay 150 dollars a month, so it is cheaper to go with pain pills. (P6)

I had to quit counseling because my insurance got cut off. (P6)
Provide support for managing prescriptions 7 I do worry when I’m running low on my Suboxone. My prescription has to go into [nearby town], and I worry they’re not going to have it at that point or I’m going to be running late with filling it. How am I going to deal with being late if they don’t have it ready in time? (P1)
Taper frequency of CC visits over time or as needed 6 Yeah I think that once a week would be ok for the first couple months if they’re showing progress…then go longer. (P2)

Whenever we need them – maybe keep in touch every 2 weeks? (P5)
Check-in with patients about possible relapse triggers 7 People who do drugs, they are doing it for a reason, they want to get away from reality. The care manager should explore those underlying reasons and the root of the problems… Missing the high and craving is a thing, also the lifestyle, including friends. (P11)

They should check on how are we’re doing emotionally; emotions have to do a lot with opioid use. Also, keeping patients busy is important. [You] want to know that someone genuinely cares about you, and sometimes strangers can do a better job than family members. (P7)

Note: CC = care coordinator; P = patient.

Ensure CC is Trained in Patient Engagement and Stigma Reduction

Nearly all patients (n = 10) expressed that it would be paramount for the CC to establish a trusting relationship with patients. Participants described that trustworthy providers are a source of support and encouragement to remain in treatment or to seek additional help. One patient shared that feeling comfortable talking with their PCP allowed them to seek treatment for depression: “I got over this one hump being ashamed of being addicted to medication. And that let me talk a little bit more about the other problems I was having, which led Dr. [Name] to recommend I try (an anti-depressant)… He always helps me solve my problems. He helps me figure it out and makes it get better” (P1). Other participants mentioned the converse: that having a negative relationship or not trusting a provider makes it difficult to continue in treatment.

Just over half of patients (n = 6) reported that stigma was a barrier to receiving treatment. Participants expressed concern over being judged for their mental health issues or being labeled an addict by themselves, loved ones, or providers. Patients also discussed stigma toward taking medications for OUD, such as the participant who expressed the opinion that “I feel that all they do (with MOUD) is switch one narcotic to another one. You may not be doing pain pills or heroin, but you’re still getting an opiate in your system” (P4).

Address Socioeconomic Barriers

Socioeconomic barriers (e.g., money, transportation) additionally posed significant obstacles to treatment for some participants (n = 5). Patients reported that issues with money, insurance, and transportation can cause delayed starts, gaps, and unwanted cessation of treatment. One participant shared that they relapsed until Medicaid began covering Suboxone: “In the beginning, it was hard to get treatment. Medicaid wasn’t paying for it… and I didn’t have money. So of course I ended up relapsing over and over again until Medicaid started paying for it” (P2).

Provide Support for Managing Prescriptions

Most participants (n = 7) mentioned that it would be beneficial for the CC to provide support for managing prescriptions, such as contacting the doctor for refills or assisting with communication around dose changes. Participants noted the utility of this role for both MOUD and psychotropic medications for treating depression/PTSD. One participant stated, “I need my depression medications to be increased and I cannot wait, so a person that helps me reach out my doctor is great, someone that doesn’t make us wait until our next appointment.” (P8)

Taper the Frequency of CC Visits Over Time or as Needed

Several participants (n = 6) supported the proposed model in which CC visits should be more frequent at the start of treatment and then taper off once the patient has stabilized. Participants indicated that more frequent visits at the beginning of treatment would be helpful for treatment retention: “When you first get on [Suboxone], it’s nerve wracking. Am I going to go through withdrawals? You were addicted to something – you don’t know how the medication is going to help you. Just knowing that you have someone there to make sure you don’t get sick or have that temptation to get back to the pills would help make you feel comfortable about it” (P1). One participant added that the visit frequency should be flexible to individual patient needs, saying, “I wonder if in the transition to having less communication people might need more attention… but I could also understand if you are doing this program for a while, you don’t want to be bothered like that either. This is a very individual decision, and people have different needs” (P3).

Check-in with Patients About Possible Relapse Triggers

Participants recommended that the CCs check in with patients about topics that could predict relapse or discontinuation of treatment such as cravings, stress, spending time with friends who still use drugs, and general emotional wellbeing. One participant cautioned that, for check-ins to be helpful, the CC must have a nonjudgmental attitude and frame questions: “When they ask those questions (about cravings, withdrawals, etc.), I find myself embarrassed to answer, I don’t want them to think ‘oh she is about to relapse’ if I share about cravings and triggers. The CC should talk more about ways to avoid those things.” (P3).

Interview 2: Feasibility and Acceptability of Collaborative Care for Co-Occurring Disorders

The second round of interviews assessed the feasibility and acceptability of the proposed intervention. There was high agreement among participants that the intervention form and content were feasible and acceptable: participants shared that the session made them feel supported, it was helpful that the session addressed both mental health and OUD, and it was acceptable (or even beneficial) to conduct the session by telephone. Participants also unanimously indicated that the CC would need to maintain a flexible approach for the intervention to be feasible for patients who are reluctant or ambivalent about starting treatment. Patients agreed with the proposed intervention structure meeting 13 times over the course of six months with the option to schedule additional visits if clinically indicated (i.e.., weekly in the first two months, biweekly in the third month, and monthly in months four through six). Table 3 summarizes the themes and representative quotes from these interviews.

Table 3.

Interview 2 Themes

Theme n Representative Quotes
Session made participants feel supported 8 It was very helpful knowing there are people out there who want to help me. (P6)

I highly recommend this program for anybody. It makes us feel good because you guys are concerned. No matter what the questions are and no matter what we tell you, you tell us it’s confidential. But still it makes me feel good that you care. (P10)

It sounds like what you’re doing with the sessions now is doing good… I don’t know for sure how others would feel, but it made me feel good. It boosted my confidence level. (P11)
Beneficial to address mental health and OUD together 7 Mental health is the reason I started using. (After a death in my family) I did not want to feel… So everybody’s journey to the drug is different and everybody’s journey after the drug is different (P7).

When I was in treatment (for OUD), it seemed like there wasn’t one person in there that wasn’t depressed or had PTSD. I think it’s all together. I think people that have an addiction problem have mental problems too. (P2)

I think pain pills got me more depressed. (P8)
Conducting session by telephone was acceptable 9 It was fine (doing the session over the phone). It would actually be easier. Especially with addicts, you don’t have to walk into a place. There is some comfort in that. (P7)

You’re not actually getting seen, but you’ll still feel like you’re able to talk… They aren’t seeing you, so they are not judging you. But you’re still getting help. (P1)
Flexible CC approach based on patient readiness 9 It all comes down to, are they forced to go to treatment or is it something they want to do themselves? That’s a big difference right there. If somebody’s making you do something, you’re not going to want to do it. But if you’re sick and tired of the situation and you want help, there’s a different motivational factor. (P4)

I think [people at] different stages [of treatment] are more accepting… If they’re just starting out – I would be very wary of people. I don’t know who to trust. I don’t know if they’re going to run off and tell the cops on me about what I’ve done. People aren’t trusting at first. At least I wasn’t. (P2)

Asking for help is a huge step. And ask (the patient) more than once. Don’t give up on them. Go back and talk to them a week later or the next time they have their appointment. That next time they might need you. (P7)

Note: CC = care coordinator; OUD = opioid use disorder; P = patient; PTSD = posttraumatic stress disorder.

Session Made Participants Feel Supported

Nearly all participants (n = 8) stated that the demonstrated intervention session was helpful to them and would likely be helpful to other patients with co-occurring disorders. For instance, one participant stated that “Just talking to (the CC) helps with the worry and judgmental feelings. Just talking to her made me feel like I moved up a step in my life” (P1).

Beneficial to Address Mental Health and OUD Together

Participants (n = 7) additionally stated that it was helpful that the session addressed both mental health and opioid use together. These participants felt that their mental health and opioid use were interconnected, and it was beneficial to discuss their treatment holistically with the CC. One participant expressed the view that mental health treatment was necessary to treat their OUD: “It helped me that I got to tell (the CC) that I thought I was doing heroin because of my PTSD and depression, and he really heard that and said okay let’s get you into see a therapist or take medication” (P9).

Conducting Session by Telephone was Acceptable

Every participant (n = 9) said that conducting the session by telephone was acceptable. Some even mentioned that they preferred participating by phone due to convenience or reduced stigma. One participant explained that doing the session by phone can reduce barriers to attending appointments, but it can also create an environment where the patient may be engaging in risky behaviors that the provider is unaware of: “The whole lot of going, registering, waiting to be called up, is kind of stressful right off the bat. The call made it a lot easier to say what I felt because I’m here at my house versus sitting in a sterile clinical situation. It has its benefits, but it also has its downsides too because I could be sitting here fixing up a shot of heroin as we talk. I could be lying my ass off” (P4).

Flexible CC Approach Based on Patient Readiness

All interviewees (n = 9) shared that for the intervention to be appropriate for people with different levels of motivation, the CC will need to tailor their approach to the patient’s level of readiness for change. They expressed that treatment is less likely to be successful if a patient feels pressured to enter treatment, but the CC should not give up on patients who are initially reticent. A respondent shared that for patients who are ambivalent about treatment, “(The CC) could probably say, ‘I know you don’t want treatment right now, but why don’t we try to get you to talk to somebody or maybe go to a meeting once a month or once a week. Try slowly to get into recovery if you’re not completely ready… I’m here to help you with whatever you need. I’m not going to force you. Let’s just try little baby steps’” (P9).

Incorporating Patient Recommendations into Implementation Activities with the Health Systems

Patient recommendations from each of the interviews were integrated into the final intervention and implemented by CCs employed at our partnering health systems.

Training and supervision

Based on patient feedback, we added content to CC training on cultural humility, stigma reduction, social determinants of health, relapse triggers, and ways to monitor pharmacotherapy. Cultural humility and stigma reduction material was added in the context of Motivational Interviewing,62 where we shared about how language should be chosen intentionally with consideration for its impacts on the patient (e.g., common labels to avoid, using person-first language, avoiding dichotomies). We also encouraged CCs to examine the values/beliefs important to them and how that might impact their services to patients, and discussed how understanding their culture and values could help navigate and address power imbalances with the patient. Training also focused on patient engagement and working with patients at varying levels of readiness for change. CCs were trained on ways to engage the patient, discuss prior treatment experiences and barriers to care, understand the patient’s goals and motivation, and explore any ambivalence around treatment.41

We also worked with CCs to sequence multiple measurement-based care assessments across multiple visits to reduce patient burden, while still monitoring important factors such as social determinants of health and relapse triggers. For example, we trained CCs on assessing social determinants of health in the first visit and PTSD/depression symptoms at the second visit. To help address socioeconomic barriers to care, the CC administers the WellRx screener, which assesses social determinants of health (e.g., food insecurity, housing, employment),63 and the CC works with the patient to prioritize what to work on throughout treatment. To assess for PTSD and depression symptom severity, the CC asked the PCL-564 and PHQ-965 monthly. We trained CCs on discussing measurement-based care with patients including sharing the rationale for the questionnaires, discussing patient responses and how they could inform treatment decisions, and how to help patients cope if questions were stressful to them.66

Discussion

The study solicited patient perspectives on developing collaborative care for co-occurring disorders in primary care and used CPPR with the participating health systems to sustain the feedback obtained. Overall, patients felt a CC was an acceptable member of their care team and the collaborative care for co-occurring disorders intervention was a feasible and acceptable treatment model. Patients underscored the need for the CC to be trained in patient engagement and stigma reduction; provide assistance with socioeconomic barriers to care, prescription management, and relapse triggers; and follow a tapered frequency for visits. Patients felt supported by the CC, agreed it was helpful to have the CC address both mental health and substance use problems, emphasized the need for the CC to have a flexible approach based on the patient’s level of readiness, and expressed that telehealth was an acceptable modality for the intervention. These findings are consistent with existing collaborative care studies focused on mental health disorders where patients and healthcare providers have emphasized the value of specialized attention from the care team, as well as increased accessibility and decreased stigma to mental health care for patients.47,48 Our results extend the existing literature by examining patient perspectives on collaborative care for co-occurring disorders and by documenting how our CPPR approach facilitated patient feedback between patients and health systems during intervention implementation and delivery.

These interview findings also provide important insights because they demonstrate the feasibility and utility of incorporating patient perspectives into the development of treatment models. Obtaining patient perspectives is a core element of patient-centered care, both in treatment and treatment development.43,44 The present study illustrates a straightforward approach for learning about patient experiences and engaging patients throughout the process of program design. The iterative design of this study is particularly important because it allowed patients to provide feedback to the research team as to whether their initial recommendations were adequately incorporated into the design of the intervention, and then for us to facilitate patient feedback to the CCs and clinic teams at our partnering health systems. Our clinical trial is currently underway and this work has fostered ongoing implementation and quality assurance activities with our health systems to enhance the intervention’s feasibility and implementation. It is also worth noting that many of the needs patients articulated for co-occurring disorders treatment in primary care (e.g., help coordinating care, mutual respect and trust, focus on social determinants of health) match the values patients have articulated elsewhere as being necessary for primary care more broadly (e.g., personalizing the intervention to the patient’s readiness to change, increasing patient engagement by creating safe spaces with low stigma, emphasizing collaboration with patients; facilitating team-based care).45,49,50

The consensus among participants that telephone-based care is as acceptable or even more acceptable than in-person visits is particularly relevant in the context of the COVID-19 pandemic, during which the use of telemedicine increased for many healthcare services.67 For areas with limited healthcare resources, such as the regions of New Mexico where this study took place, the acceptability of telephone-based care suggests a strategy to partially address local staff shortages by allowing remote staff to provide care via telehealth.68,69 Despite the perceived acceptability of telephone-based care, it is important to note that accessibility of technology still serves as a barrier. Across the four months that interviews were conducted, multiple participants changed phone numbers or had periods without phone service (e.g., broken device, change in carriers). Maintaining virtual engagement with patients in this population or other under-resourced populations may therefore require a greater expenditure of time and effort than is typical.

Finally, it is worth noting that when asked about desired characteristics of a CC, participants focused on interpersonal skills, communication with other providers, and ability to assist with linkage to social services. Participants did not indicate a need for the CC to possess knowledge or skills that would be conferred by formal clinical training. This is significant because many collaborative care models use registered nurses or other licensed providers as CCs,20 which may present an obstacle to implementing collaborative care in under-resourced or understaffed clinics. The needs articulated by patients in the present study suggest that this level of licensure may not always be needed when the CC is working in concert with other providers.

Limitations

The primary limitation of the present study is that the participants were already engaged in buprenorphine treatment and had a positive relationship with their provider. Patients who do not meet this profile (i.e., those who never successfully linked to care or did not have a longstanding relationship with their PCP) may have different treatment barriers or needs. Additionally, our sample had limited racial diversity. With a different sample, we may have received additional feedback about ways to make the intervention culturally relevant. Finally, we have not yet been able to evaluate the adaptations made to the intervention for effectiveness, but such evaluation is ongoing.40

Future Directions

Future studies should consider other ways to engage patients in the development of treatment models. For instance, Siriwardena and Gillam44 note that supplementing qualitative feedback with quantitative data can be an effective technique for understanding both the depth and breadth of patient feedback around program design.44 While the current study did not have the capacity to include a quantitative portion, it may be a useful approach for circumventing some of the limitations here (e.g., relying on provider referrals for recruitment, limited diversity in the sample). Examples of future quantitative studies may include asking patients to rate the usability, acceptability, and satisfaction of the intervention to guide future implementation.70 Understanding these outcomes during the formative research phase and before a clinical trial starts can help further optimize the quality of the intervention. In addition, as we experienced during the clinical trial, formative research can be used to prioritize foci for quality assurance and improvement activities once the clinical trial is underway. Assessing patient perceptions of the intervention during a clinical trial can also help offer context to challenges affecting the intervention’s implementation. Future studies may also stratify patient participants to incorporate more sample heterogeneity by treatment status, symptom severity, location/clinic type, social needs, insurance, and potentially other characteristics to increase sample generalizability. Ultimately, gaining input from community and health system partners early in the process and continuing to incorporate feedback throughout intervention implementation and delivery are critical for ensuring the interventions are responsive to the patients and those serving them.

Acknowledgements:

The authors would like to acknowledge our community health system and patient partners, without whom this work would not have been possible. We also thank the CLARO study team for all their study contributions, the CLARO care coordinators for their feedback, and the consultants who informed the behavioral health and medical treatments used in this study (Dr. Patrick Raue, Dr. Denise Sloan, Dr. Johanna Thompson-Hollands, Dr. Jasen Christensen, and Dr. Hannah Roggenkamp). We also specifically thank Tiffany Hruby, Natalia Giraldo-Santiago, and Carol Moss for their assistance with project administration and patient interviews, and Dr. Michael Schoenbaum (National Institute of Mental Health) and the CLARO Scientific Advisory Board for their advisement on the science and practice of collaborative care interventions.

Funding:

This research was supported by a grant to Drs. Watkins and Komaromy (Multiple PIs) from the National Institute of Mental Health (NIMH); grant number: U01MH121954.

Footnotes

This paper was presented at the following meetings:

Dopp, A. R., Watkins, K., Meredith, L., Leamon, I., Becker, K., Murray-Krezan, C., Cefalu, M., Tarhuni, L., Page, K., & Komaromy, M. (2020, November). Collaborative care for co-occurring opioid use and mental health disorders. In A. R. Dopp (Chair), Novel intervention and implementation approaches to improve behavioral health care access and outcomes in primary care settings. Symposium conducted at meeting of the Association for Behavioral and Cognitive Therapies (virtual conference).

Dopp, A. R., Osilla, K. C., Ceballos, V., Leamon, I., Meredith, L., Komaromy, M., Jacobsohn, V., & Watkins, K. (2020, November). Dynamic learning in a stakeholder-engaged adaptation process: Examples from collaborative care for co-occurring disorders. In C. Johnson (Chair), Adaptations to improve access and quality of evidence-based treatments: Processes for selecting, reporting, and evaluating. Symposium conducted at meeting of the Association for Behavioral and Cognitive Therapies (virtual conference).

Disclosure of Interest: The authors report there are no competing interests to declare.

References

  • 1.Jones L, Wells K. Strategies for academic and clinician engagement in community-participatory partnered research. JAMA. 2007;297(4):407–410. [DOI] [PubMed] [Google Scholar]
  • 2.Brown PJ, Recupero PR, Stout R. PTSD substance abuse comorbidity and treatment utilization. Addict Behav. 1995;20(2):251–254. [DOI] [PubMed] [Google Scholar]
  • 3.Dore G, Mills K, Murray R, Teesson M, Farrugia P. Post-traumatic stress disorder, depression and suicidality in inpatients with substance use disorders. Drug Alcohol Rev. 2012;31(3):294–302. [DOI] [PubMed] [Google Scholar]
  • 4.Harnish A, Corrigan P, Byrne T, et al. Substance use and mental health stigma in veterans with co-occurring disorders. J Dual Diagn. 2016;12(3-4):238–243. [DOI] [PubMed] [Google Scholar]
  • 5.Meier A, Lambert-Harris C, McGovern MP, et al. Co-occurring prescription opioid use problems and posttraumatic stress disorder symptom severity. Am J Drug Alcohol Abuse. 2014;40(4):304–311. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Mills KL, Teesson M, Ross J, Darke S, Shanahan M. The costs and outcomes of treatment for opioid dependence associated with posttraumatic stress disorder. Psychiatr Serv. 2005;56(8):940–945. [DOI] [PubMed] [Google Scholar]
  • 7.Nunes EV, Sullivan MA, Levin FR. Treatment of depression in patients with opiate dependence. Biol Psychiatry. 2004;56(10):793–802. [DOI] [PubMed] [Google Scholar]
  • 8.Ouimette PC, Ahrens C, Moos RH, Finney JW. Posttraumatic stress disorder in substance abuse patients: Relationship to 1-year posttreatment outcomes. Psychol Addict Behav. 1997;11(1):34–47. [Google Scholar]
  • 9.Ouimette PC, Brown PJ, Najavits LM. Course and treatment of patients with both substance use and posttraumatic stress disorders. Addict Behav. 1998;23(6):785–795. [DOI] [PubMed] [Google Scholar]
  • 10.Seal KH, Shi Y, Cohen G, et al. Association of mental health disorders with prescription opioids and high-risk opioid use in US veterans of Iraq and Afghanistan. JAMA. 2012;307(9):940–947. [DOI] [PubMed] [Google Scholar]
  • 11.Shei A, Rice JB, Kirson NY, et al. Characteristics of high-cost patients diagnosed with opioid abuse. J Manag Care Spec Pharm. 2015;21(10):902–912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Shiner B, Leonard Westgate C, Bernardy NC, Schnurr PP, Watts BV. Trends in opioid use disorder diagnoses and medication treatment among veterans with posttraumatic stress disorder. J Dual Diagn. 2017;13(3):201–212. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Tormohlen KN, Mojtabai R, Seiwell A, et al. Co-occurring opioid use and depressive disorders: Patient characteristics and co-occurring health conditions. J Dual Diagn. 2021;17(4):296–303. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Williams R, Farquharson L, Rhodes E, et al. Impact of substance use disorder on quality of inpatient mental health services for people with anxiety and depression. J Dual Diagn. 2021;17(1):80–93. [DOI] [PubMed] [Google Scholar]
  • 15.Jones CM, McCance-Katz EF. Co-occurring substance use and mental disorders among adults with opioid use disorder. Drug Alcohol Depend. 2019;197:78–82. [DOI] [PubMed] [Google Scholar]
  • 16.Darke S, Ross J, Marel C, et al. Patterns and correlates of attempted suicide amongst heroin users: 11-year follow-up of the Australian treatment outcome study cohort. Psychiatry Res. 2015;227(2-3):166–170. [DOI] [PubMed] [Google Scholar]
  • 17.Fendrich M, Becker J, Hernandez-Meier J. Psychiatric symptoms and recent overdose among people who use heroin or other opioids: Results from a secondary analysis of an intervention study. Addictive Behaviors Reports. 2019;10:100212. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Alexander CL, Arnkoff DB, Glass CR. Bringing psychotherapy to primary care: Innovations and challenges. Clin Psychol (New York). 2010;17(3):191–214. [Google Scholar]
  • 19.Donroe JH, Bhatraju EP, Tsui JI, Edelman EJ. Identification and management of opioid use disorder in primary care: An update. Curr Psychiatry Rep. 2020;22(5):23. [DOI] [PubMed] [Google Scholar]
  • 20.AIMS Center. Principles of Collaborative Care. [Internet]. https://aims.uw.edu/collaborative-care/principles-collaborative-care. Accessed May 1, 2021.
  • 21.Archer J, Bower P, Gilbody S, et al. Collaborative care for depression and anxiety problems. Cochrane Database of Systematic Reviews. 2012;10:Cd006525. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Meredith LS, Jackson-Triche M, Duan N, et al. Quality improvement for depression enhances long-term treatment knowledge for primary care clinicians. J Gen Intern Med. 2000;15(12):868–877. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Rost KM, Duan N, Rubenstein LV, et al. The Quality Improvement for Depression collaboration: general analytic strategies for a coordinated study of quality improvement in depression care. Gen Hosp Psychiatry. 2001;23(5):239–253. [DOI] [PubMed] [Google Scholar]
  • 24.Rubenstein LV, Meredith LS, Parker LE, et al. Impacts of evidence-based quality improvement on depression in primary care: A randomized experiment. J Gen Intern Med. 2006;21(10):1027–1035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Wells KB, Sherbourne C, Schoenbaum M, et al. Impact of disseminating quality improvement programs for depression in managed primary care: A randomized controlled trial. JAMA. 2000;283(2):212–220. [DOI] [PubMed] [Google Scholar]
  • 26.Meredith LS, Eisenman DP, Han B, et al. Impact of collaborative care for underserved patients with PTSD in primary care: A randomized controlled trial. J Gen Intern Med. 2016;31:509–517. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Hunt P, Ober AJ, Watkins KE. The costs of implementing collaborative care for opioid and alcohol use disorders in primary care (RR-2049-NIDA). Santa Monica, CA: RAND Corporation; 2017. Available at: https://www.rand.org/pubs/research_reports/RR2049.html. [Google Scholar]
  • 28.Hunter SB, Ober AJ, McCullough CM, et al. Sustaining alcohol and opioid use disorder treatment in primary care: A mixed methods study. Implementation Science. 2018;13:83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Ober AJ, Watkins KE, Hunter SB, et al. Assessing and improving organizational readiness to implement substance use disorder treatment in primary care: Findings from the SUMMIT study. BMC Fam Pract. 2017;18(1):107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Watkins KE, Ober AJ, Lamp K, et al. Implementing the chronic care model for opioid and alcohol use disorders in primary care. Prog Community Health Partnersh: Res Educ Action. 2017;11(4):397–407. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Watkins KE, Ober AJ, Lamp K, et al. Collaborative care for opioid and alcohol use disorders in primary care: The SUMMIT randomized clinical trial. JAMA internal medicine. 2017;177(10):1480–1488. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Morton I, Hurley B, Castillo E, et al. Outcomes of two quality improvement depression care interventions in individuals with substance use problems. The American Journal of Drug and Alcohol Abuse. 2020;46(2):251–261. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Watkins KE, Paddock SM, Zhang L, Wells KB. Improving care for depression in patients with comorbid substance misuse. Am J Psychiatry. 2006;163(1):125–132. [DOI] [PubMed] [Google Scholar]
  • 34.Arean PA, Ayalon L, Hunkeler E, et al. Improving depression care for older, minority patients in primary care. Med Care. 2005;43(4):381–390. [DOI] [PubMed] [Google Scholar]
  • 35.Ell K, Aranda MP, Xie B, Lee PJ, Chou CP. Collaborative depression treatment in older and younger adults with physical illness: Pooled comparative analysis of three randomized clinical trials. Am J Geriatr Psychiatry. 2010;18(6):520–530. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Ell K, Katon W, Cabassa LJ, et al. Depression and diabetes among low-income Hispanics: Design elements of a socioculturally adapted collaborative care model randomized controlled trial. Int J Psychiatry Med. 2009;39(2):113–132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Ell K, Katon W, Xie B, et al. Collaborative care management of major depression among low-income, predominantly Hispanic subjects with diabetes: A randomized controlled trial. Diabetes Care. 2010;33(4):706–713. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Miranda J, Duan N, Sherbourne C, et al. Improving care for minorities: Can quality improvement interventions improve care and outcomes for depressed minorities? Results of a randomized, controlled trial. Health Serv Res. 2003;38(2):613–630. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Miranda J, Schoenbaum M, Sherbourne C, Duan N, Wells K. Effects of primary care depression treatment on minority patients’ clinical status and employment. Arch Gen Psychiatry. 2004;61(8):827–834. [DOI] [PubMed] [Google Scholar]
  • 40.Meredith LS, Komaromy MS, Cefalu M, et al. Design of CLARO (Collaboration Leading to Addiction Treatment and Recovery from other Stresses): A randomized trial of collaborative care for opioid use disorder and co-occurring depression and/or posttraumatic stress disorder. Contemp Clin Trials. 2021;104:106354. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Osilla KC, Dopp AR, Watkins KE, Ceballos V, Hurley B, Meredith LS, Leamon IM, Jacobsohn V & Komaromy M Collaboration Leading to Addiction Treatment and Recovery from Other Stresses (CLARO): Process of adapting collaborative care for co-occurring opioid use and mental disorders. Addict Sci Clin Pract. 2022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Chung B, Jones L, Dixon EL, et al. Using a community partnered participatory research approach to implement a randomized controlled trial: Planning the design of community partners in care. J Health Care Poor Underserved. 2010;21(3):780. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Bate P, Robert G. Experience-based design: From redesigning the system around the patient to co-designing services with the patient. Quality and Safety in Health Care. 2006;15(5):307. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Siriwardena AN, Gillam S. Patient perspectives on quality. Qual Prim Care. 2014;22(1):11–15. [PubMed] [Google Scholar]
  • 45.Bechtel C, Ness D. If you build it, will they come? Designing truly patient-centered health care. Health Aff (Millwood). 2010;29(5):914–920. [DOI] [PubMed] [Google Scholar]
  • 46.Sorsa M, Greacen T, Lehto J, Åstedt-Kurki P. A qualitative study of barriers to care for people with co-occurring disorders. Arch Psychiatr Nurs. 2017;31(4):399–406. [DOI] [PubMed] [Google Scholar]
  • 47.Fuller SM, Koester KA, Erguera XA, et al. The collaborative care model for HIV and depression: Patient perspectives and experiences from a safety-net clinic in the United States. SAGE Open Med. 2019;7:2050312119842249. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Rugkåsa J, Tveit OG, Berteig J, Hussain A, Ruud T. Collaborative care for mental health: A qualitative study of the experiences of patients and health professionals. BMC Health Serv Res. 2020;20:1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Youssef A, Chaudhary ZK, Wiljer D, Mylopoulos M, Sockalingam S. Mapping evidence of patients’ experiences in integrated care: A scoping review. Gen Hosp Psychiatry. 2019;61:1–9. [DOI] [PubMed] [Google Scholar]
  • 50.Drainoni M-L, Farrell C, Sorensen-Alawad A, et al. Patient perspectives of an integrated program of medical care and substance use treatment. AIDS Patient Care STDS. 2014;28(2):71–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.McNeil R, Kerr T, Pauly B, Wood E, Small W. Advancing patient-centered care for structurally vulnerable drug-using populations: A qualitative study of the perspectives of people who use drugs regarding the potential integration of harm reduction interventions into hospitals. Addiction. 2016;111(4):685–694. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Merrill JO, Rhodes LA, Deyo RA, Marlatt GA, Bradley KA. Mutual mistrust in the medical care of drug users. J Gen Intern Med. 2002;17(5):327–333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.McNeil R, Small W, Wood E, Kerr T. Hospitals as a ‘risk environment’: An ethno-epidemiological study of voluntary and involuntary discharge from hospital against medical advice among people who inject drugs. Soc Sci Med. 2014;105:59–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.New Mexico Department of Health. Health status of New Mexico 2018, slides 2, 3, and 249. https://nmhealth.org/data/view/general/2230/ Accessed March 25, 2019.
  • 55.Rural Health Information Hub. New Mexico Chart Gallery. [Internet]. https://www.ruralhealthinfo.org/charts/5?state=NM. Accessed March 21, 2019.
  • 56.Ritchie MJ, Dollar KM, Miller CJ, et al. Using implementation facilitation to improve healthcare (version 3). Veterans Health Administration, Behavioral Health Quality Enhancement Research Initiative (QUERI). https://www.queri.research.va.gov/tools/implementation.cfm. Accessed May 21, 2021.
  • 57.Ritchie MJ, Parker LE, Edlund CN, Kirchner JE. Using implementation facilitation to foster clinical practice quality and adherence to evidence in challenged settings: A qualitative study. BMC Health Serv Res. 2017;17(1):1–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Dopp A, Komaromy M, Brown L, et al. CLARO implementation manual: Strategies and supports for successful implementation of the CLARO intervention. Santa Monica, CA: RAND Corporation; in press. [Google Scholar]
  • 59.Krippendorff K. Content analysis: An introduction to its methodology. Beverly Hills, CA: Sage; 1980. [Google Scholar]
  • 60.Weber RP. Basic content analysis. Beverly Hills, CA: Sage; 1990. [Google Scholar]
  • 61.Hennink M, Kaiser BN. Sample sizes for saturation in qualitative research: A systematic review of empirical tests. Soc Sci Med. 2022;292:114523. [DOI] [PubMed] [Google Scholar]
  • 62.Miller WR, Rollnick S. Motivational interviewing: Helping people change. Third ed. New York, NY: Guilford Press; 2012. [Google Scholar]
  • 63.University of Washington, Psychiatry & Behavioral Sciences, AIMS Center, Behavioral Health Integration and Collaborative Care. Registry strategies in medical settings. [Internet]. https://aims.uw.edu/sites/default/files/Collaborative%20Care%20Registry%20Requirements%20Guide_2019.pdf. Accessed December 7, 2020.
  • 64.Blevins CA, Weathers FW, Davis MT, Witte TK, Domino JL. The Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5): Development and initial psychometric evaluation. J Trauma Stress. 2015;28(6):489–498. [DOI] [PubMed] [Google Scholar]
  • 65.Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: Validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–613. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Brooks Holliday S, Hepner KA, Farmer CM, et al. Discussing measurement-based care with patients: An analysis of clinician-patient dyads. Psychother Res. 2021;31(2):211–223. [DOI] [PubMed] [Google Scholar]
  • 67.Hoffman DA. Increasing access to care: telehealth during COVID-19. Journal of Law and the Biosciences. 2020;7(1). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Dorsey ER, Topol EJ. State of telehealth. N Engl J Med. 2016;375(2):154–161. [DOI] [PubMed] [Google Scholar]
  • 69.Myers CR. Using telehealth to remediate rural mental health and healthcare disparities. Issues Ment Health Nurs. 2019;40(3):233–239. [DOI] [PubMed] [Google Scholar]
  • 70.Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12:1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES