Abstract
Background
Digital interventions, such as websites and smartphone apps, can be effective in treating drug use disorders (DUDs). However, their implementation in primary care is hindered, in part, by a lack of knowledge on how patients might like these treatments delivered to them.
Objective
This study aims to increase the understanding of how patients with DUDs prefer to receive app-based treatments to inform the implementation of these treatments in primary care.
Methods
The methods of user-centered design were combined with qualitative research methods to inform the design of workflows for offering app-based treatments in primary care. Adult patients (n=14) with past-year cannabis, stimulant, or opioid use disorder from 5 primary care clinics of Kaiser Permanente Washington in the Seattle area participated in this study. Semistructured interviews were recorded, transcribed, and analyzed using qualitative template analysis. The coding scheme included deductive codes based on interview topics, which primarily focused on workflow design. Inductive codes emerged from the data.
Results
Participants wanted to learn about apps during visits where drug use was discussed and felt that app-related conversations should be incorporated into the existing care whenever possible, as opposed to creating new health care visits to facilitate the use of the app. Nearly all participants preferred receiving clinician support for using apps over using them without support. They desired a trusting, supportive relationship with a clinician who could guide them as they used the app. Participants wanted follow-up support via phone calls or secure messaging because these modes of communication were perceived as a convenient and low burden (eg, no copays or appointment travel).
Conclusions
A user-centered implementation of treatment apps for DUDs in primary care will require health systems to design workflows that account for patients’ needs for structure, support in and outside of visits, and desire for convenience.
Keywords: user-centered design, qualitative, drug use disorders, stimulants, cannabis, opioids, primary care, mHealth, mobile phone
Introduction
Background
Drug use disorders (DUDs) are prevalent and deadly worldwide [1,2]. Addiction epidemics are worsening—in 2018, 184 people a day died from drug overdose in the US, and the number of fatal-and nonfatal overdoses increased in 2020 [3,4]. However, most people with DUDs do not receive treatment [5,6]. Experts recommend that treatments for DUD should be implemented in primary care to reduce this treatment gap [7-11].
Digital treatments, such as smartphone or tablet app or websites, have been touted as a means to reach more people with effective DUD treatments [12,13]. One way to classify digital treatments is to place them on a spectrum ranging from apps that are used as self-care without the help of a clinician to apps that are fully incorporated into patients’ health care and guided by a clinician [14]. Several clinician-guided treatments for alcohol or drugs are supported by the evidence of efficacy or effectiveness [15-17]. For instance, two platforms—the Therapeutic Educational System and CBT4CBT—deliver cognitive behavioral treatment through web-based modules [15,18]. Both were initially designed by researchers to augment standard care for substance use disorder and have since been marketed to health care systems in the form of apps or websites with patient-facing and clinician-facing features [19,20].
Despite the promise of digital treatments, there is a lack of knowledge about how to optimally integrate them into routine primary care. In trials of digital treatments, including those for DUD and other health conditions, patients often fail to engage with the software, leading to null results [21,22] and failed real-world clinical trials [23]. Without adequate support to enhance motivation to engage with apps, patients prematurely stop using apps or use them rarely, thereby decreasing efficacy [21,22,24-26]. This poses design and implementation problems in primary care. Evidence suggests that patients can benefit from digital treatments if they receive extensive coaching and help to use them. In contrast, the time of patients and clinical teams is limited and filled with competing demands [27-29]. Therefore, the integration of apps into health care will need to balance patients’ desires and needs with these constraints.
Objectives
To incorporate patient voices into the design of approaches for offering apps for DUDs in primary care, this study combined user-centered design methods [30] with qualitative research methods. Drawing from the medical informatics literature, work systems models are often used to guide the design of clinical workflows [31-33]. Central to the concept of work systems’ models are actions, people, and tools that can assist patients and health care teams in embracing health care technologies. We applied these concepts to understand patient preferences on how to introduce DUD treatment apps to patients, assist them with the app setup, and offer appropriate follow-up over time. Consistent with the principles of user-centered design [30], this study served as the first within a series of implementation science studies that will iteratively design and test the effectiveness of approaches for implementing digital treatments for DUDs in real-world health care settings.
Methods
Study Setting
Study participants were recruited from 5 primary care clinics of Kaiser Permanente Washington, a regional integrated health care system, in Seattle area. All clinics employed licensed independent clinical social workers with some training in DUD interventions, and several clinics had primary care providers (PCPs) who prescribed buprenorphine. Consultative addiction psychiatry was available in the health system. Specialist addiction treatment programs were available through an external care network that was contracted by the health system [34].
Eligibility Criteria
The main eligibility criteria, assessed by phone screening, were smartphone use to ensure a basic familiarity with these devices [35] along with the presence of a past-year cannabis, stimulant, or opioid use disorder based on the Mini International Neuropsychiatric Interview for the Diagnostic and Statistical Manual of Mental Disorders–Fifth Edition [36,37]. During our accrual period, we revised the eligibility criteria to also include patients without a past-year opioid use disorder based on Diagnostic and Statistical Manual of Mental Disorders–Fifth Edition if they had been prescribed buprenorphine formulations used to treat opioid use disorder [38].
Participant Identification and Recruitment
Between July 2018 and December 2018, we queried electronic health records to identify patients aged 18 years or more with a recent primary care visit, who had screened positive for cannabis or other drugs [39-42] or had a documented DUD diagnosis. The study staff mailed invitations with a preincentive of US $2 to 101 potentially eligible patients [43,44]. We phoned 77 of these patients for eligibility screening while providing a US $10 incentive for those who completed phone screening, regardless of eligibility. Multimedia Appendix 1 presents a participant recruitment flow diagram noting the reasons for exclusion. We used purposeful sampling [45] to promote sample diversity by race, gender, age, substance type (opioid, cannabis, stimulant), and prior DUD treatment or mutual support program attendance [46].
Sample
The study’s initial recruitment goals were based on pragmatic and empirical criteria. We sought to capture a sufficient diversity of patient perspectives to inform the initial implementation of digital treatments in a primary care practice that could then be iteratively adjusted during implementation within a health care system. We targeted a minimum of 12 participants [47] and stopped recruitment at 14 while monitoring saturation during the analysis [48]. Among the 14 participants, 6 had opioid use disorder or were taking buprenorphine, 9 had cannabis use disorder, and 4 had stimulant use disorder (participant details in Multimedia Appendix 1).
Semistructured Interviews
The interview guide was designed to elicit preferences regarding workflow design among candidate approaches for offering and supporting the delivery of app-based DUD treatments. The interviews utilized user-centered design tools, including personas and storyboards. Personas depicted a visual user profile to show participants an example of a person who might be a typical user of an app [49]. We developed three personas—one each for stimulants, opioids, and cannabis—that depicted a hypothetical patient with several DUD symptoms who was interested in learning about options for help (Multimedia Appendix 1). We presented one persona per participant. We also used storyboards [50], which used illustrated panels to show core features of candidate workflow designs.
The storyboards showed alternative scenarios to support patients’ app use. Scenarios fit into three workflow phases: (1) introducing the app to patients and helping them learn about it (Introduction); (2) setting up patients interested in using the app with the treatment (Setup); and (3) following up with patients who agree to use the app to promote engagement and execute a care plan that includes the app (Follow-up). There were 3 to 4 candidate scenarios as options for each workflow phase (Figure 1; complete storyboards are presented in Multimedia Appendix 1, which includes detailed information about the storyboard design process and rationale behind each scenario). Scenarios varied in whether interactions were done virtually or in person, how much help the patient received, which team members were involved, and other aspects of communication and workflow. Interviewers read aloud all scenarios within a phase, asked participants to rank scenarios according to their preferences, and probed for additional information.
Study interview materials were informed by domains of work systems models (eg, workflow actions or tasks, individuals or people, and tools that facilitate work) [33], implementation science frameworks (eg, the extent of support and how to guide patients) [14], and prior digital treatment studies (eg, privacy issues and concerns regarding the use of electronic self-reported data) [22,51-53]. The guide also drew from theory and literature on patient treatment engagement and health services design in primary care [21,22,52-63]. Clinical leaders and clinicians from Kaiser Permanente and a safety net health clinic provided input to the materials. Although not analyzed in this study, the interviews asked about background experiences (eg, app use; Multimedia Appendix 1 contains an interview guide summary). Interviews were conducted in person and digitally recorded and professionally transcribed. Interviews lasted for an average of 87 minutes, including informed consent.
The Kaiser Permanente Washington Institutional Review Board granted ethical approvals. Written informed consent was obtained from all the participants enrolled in this study. A waiver of written informed consent was obtained to conduct telephone screenings. A waiver of informed consent was obtained for the identification of potential participants using electronic health records.
Data Analysis
For qualitative analysis, we imported transcripts into Dedoose v7.0.23 (SocioCultural Research Consultants) [64]. We applied template analyses to qualitatively code excerpts of text [65], which used a combination of inductive and deductive approaches. A priori codes were based on the interview guide topics. Codebook definitions were informed posthoc by the Workflow Elements Model [31,32] constructs to help in classifying participant preferences regarding the physical or virtual actions that need to be performed, the people (eg, clinicians) who need to perform the actions, and the physical and virtual tools used to promote app delivery and use. For the first 5 transcripts, 2 study team members coded them, a third study team member reconciled them, and the team discussed achieving consensus in the form of a codebook. To further increase rigor, we kept memos and an audit trail of code revisions [66,67]. The remaining transcripts were split among team members for coding, and the principal investigator (PI; JEG) reconciled them. The lead analyst (TEM) examined the fully coded data for patterns within the coding hierarchy and preliminary themes that occurred across codes, regularly meeting with the PI to review results. In several meetings, the PI and lead analyst used affinity mapping, a user-centered design activity that helps in clustering similar ideas or concepts together [68] to further refine the themes. The themes were discussed with other team members before they were finalized.
Availability of Data and Materials
The interview materials are provided in Multimedia Appendix 1. Additional interview materials are available from the first author upon reasonable request.
Results
Preferences Regarding Workflow Actions
This section presents patient preferences regarding the actions performed while facilitating a DUD treatment app, by each workflow stage.
Introduction Stage: Learning About the Treatment App
Nearly all participants preferred to learn about app-based treatment options during an in-person primary care visit where drug use was already being discussed. Participants generally did not want to learn about an app during an unexpected cold call with a clinician they had not met. Cold calls in relation to drug use would feel intrusive, potentially awkward, and embarrassing. Participants who were open to receiving a phone call to learn about an app said they would at least need to know about the phone call in advance. Only 2 participants were interested in a self-guided treatment where they would learn about an app via a pamphlet without any interaction with a health care professional, and only 1 participant said that they would respond to a waiting room advertisement. A few noted that the convenience of accessing the app without help from a clinician was appealing, but others said they would not be motivated to use an app without support. As one participant explained:
I just don’t see that I would pick [the flyer] up ... It would be more effective if, after meeting with my primary care person about my addiction issues, they got me in direct contact with [a mental health clinician].
P3, who was receiving medication treatment for opioid use disorder
Setup Stage: Getting Started With the App
Overall, participants thought that it was critical to avoid getting “stuck” while setting up and starting to use an app. Having a seamless experience would be critical for maintaining motivation. One participant said the following:
Sit next to me and walk me through it. Hold my hand.
P13, participant with cannabis use disorder
Most participants preferred in-person assistance to help them get started with the app once they decided to use it. However, several others wanted to receive this help over the phone to reduce the amount of time spent at their visit or to avoid returning for a visit. For these participants, the phone setup was “the best of both worlds.” One participant said as follows:
number one is like, we agree that [persona name] will schedule an in-person appointment. In my case I’m very reluctant to do stuff like that, so okay, extra work for me–awesome, thanks. This one [solo setup scenario] is like all on my own, no support. So this one [phone setup scenario] I feel like is the best of both worlds.
P3, participant with cannabis use disorder
Still, some thought that scheduling phone appointments would be a chore. Few preferred to set up the app without help; these participants noted that setting up the app on their own would save time.
Follow-up Stage: Getting Support Over Time for Engagement and Care With an App
Half of the participants preferred follow-up to occur over the phone. This would provide personalized communication and a relationship that would help them stay accountable without the inconvenience and monetary costs of an in-person visit. Even those who favored an initial visit in person generally preferred a phone follow-up. Few preferred a follow-up strategy that only involved secure messages, but they liked the idea of receiving messages in addition to phone calls. One participant noted:
Secure messages and phone for me ... Because that way you have somebody on the phone to tell you if you’ve interpreted the message right ... I’ve had that happen many times. A doctor will say something and I don’t necessarily interpret it correctly or the way that it’s meant.
P14, participant with opioid and stimulant use disorder
Most participants thought that reading and responding to a message at a later time would be convenient, and some were more receptive to more frequent contacts if done over messages. However, others noted that they do not check messages or thought they would be “too easy to ignore.” One participant added the following:
If you’re having that reluctance, that’s so much easier to just be like well, no, I’m not going to do that.
P10, participant with cannabis use disorder
Preferences Regarding the People Who Could Deliver Digital Treatments
Preferences regarding the people who could facilitate a DUD treatment app were often applicable across multiple stages of the workflow (Multimedia Appendix 1). Thus, the preferences are presented here for the overall workflow.
Most participants preferred a clinician with mental health expertise to introduce and guide them in app use over time. Several participants said that mental health clinicians might have more knowledge about treatment options than a PCP, more experience treating drug use, or more time to describe features of an app and walk them through the setup process. However, some preferred to only talk to PCPs and said that mental health clinicians would make it feel like their “issue was serious.” Other benefits of working with a PCP ranged from wanting to work with someone with whom they had an established rapport, wanting fewer clinicians involved in their care, and wanting to work with someone who can offer medical advice. Many acknowledged logistical constraints of working with a PCP (eg, limited time and challenge of scheduling appointments). One participant said the following:
I don’t think the primary care [provider] needs to be concerned with that. They need to have some knowledge of it, but I think the major, the main focus of it would be with the social worker, plus the social worker would be able to follow up with the patient as far as once they get started using the app. It would be easier follow up for the social worker than the primary care [provider].
P5, participant with stimulant use disorder
A few participants said it would be helpful to hear from someone with lived experience who had “been through it before” to help them decide whether to use the app over other treatment options. They did not feel that it was important to have such an individual remain involved after they had started using the app.
Finally, participants described the value of having access to technical support for help with downloading, setup, and use of the app. Participants pointed out that they would not want technical issues to consume valuable visit time. Participants clearly differentiated technical or setup assistance from treatment-related assistance and drew boundaries around the type of help they would want from each person.
Preferences Regarding the Tools That Could Help Facilitate the Use of the Digital Treatment
Participants described a range of ideas about virtual or physical tools that could facilitate the delivery of an app (Textbox 1). For instance, pamphlets, user ratings and reviews, and trial versions of the app were suggested as tools for the introduction stage. Written instructions and video tutorials could help with the setup stage. Several tools were suggested for the follow-up stage to facilitate communication (eg, a “Get Support” button to help contact the care team).
Tools that could help facilitate the use of a drug use disorder treatment app in primary care, as suggested by patients. Tools are presented by the workflow stage.
Introduction Stage: Learning About the Treatment App
Pamphlet explaining how the treatment app will benefit the patient
User ratings and reviews that could lend credibility to the treatment app
Trial version or demonstration of the treatment app allowing patients to test it out before committing
Advertisements or information about treatment app on a health plan or agency website
Setup Stage: Getting Started With the App
Written instructions for getting started with the treatment app or a user guide
Video tutorial for getting started with the treatment app
Smartphone requirements for downloading and running the treatment app (memory storage, operating system version, and needing to know their app store password)
A webpage or button in the app that provides answers to frequently asked questions
Follow-up Stage: Getting Support Over Time for Engagement and to Execute a Care Plan While Using the App
Telephone caller identification (caller ID) so that the patient knows if their clinician is calling (patients may not answer their phone if the incoming call looks like a generic or toll-free number)
Contact or Get Support button
Technical assistance (for app and device-related questions)
Clinician contact information or direct messaging feature (for treatment questions)
Screen sharing functionality so the patient’s smartphone screen can be viewed by a support professional
Reminders and notifications, sent in accordance with the level of engagement with the treatment app (ie, more frequent reminders or notifications if patient is not using the app)
Having a way to check in with your doctor about it would be useful ... In [the phone setup scenario] if there could be a way to demonstrate it like in [the in-person scenario], with a screen sharing thing – I think that would be super useful, or maybe like a video, like an instructional video showing how to navigate it, that would be useful, since you wouldn’t be able to do it in person.
P12, participant with cannabis and stimulant use disorder
Cross-Cutting Themes
Four cross-cutting themes emerged across codes in the codebook. Themes are described below; Textbox 2 provides representative quotes for each theme (Multimedia Appendix 1).
Themes derived from analyses across the codebook: cross-cutting recommendations for designing a patient-centered approach for offering treatment apps for drug use disorders in primary care.
Established Relationships and Trust Would Facilitate a Better Patient Experience
“It’s a person who’s already talking to her about her drug addiction, like supplying Suboxone or whatever, that is saying hey, this might help. And I feel like people would be a little bit more receptive if it’s someone that they already trust with their treatment.” (P1, participant with cannabis and opioid use disorder. Had prior drug use disorder [DUD] treatment.)
“I mean the main thing is support...I would say the best people that will help you is if they understand what you’re going through, especially during that period, and they need to understand what you went through. It doesn’t necessarily have to be that way, but that’s how I was, and primary care provider helped me in the professional way, and then the emotional issues of that time...But I will say if someone is truly trying to help this person making them know that there’s still someone that cares about them, whatever, in any way it’s a good thing. It is. Some people have no one, and even small interactions can make a difference in positive ways.” (P2, participant taking buprenorphine whose opioid use disorder is in remission. Had prior DUD treatment.)
“I know for me personally that if I’m talking with somebody–I mean, if you can send me something, I can more likely put it on the back burner. Whereas if I have an actual conversation with them – I don’t know, it’s just more personal. I think it’s more of a personal check in, that there’s a real human being right there that’s interested in my care.” (P3, participant taking buprenorphine whose opioid use disorder is in remission. Had prior DUD treatment.)
Patients Were Open to Team-Based Approach When Getting Support From Their Primary Care Team in the Use of Apps
“I mean if there were actually social workers doing this, then I suppose that would be who I would be contacting, and I wouldn’t really have any issue with that because that’s just sort of how it would be. It would be another person I’m meeting to satisfy a different medical need that I have. I don’t have a lot of hang-ups about meeting different providers for different issues. I’ve done a lot of that in the past.” (P8, participant with cannabis and stimulant use disorder. Had prior DUD treatment.)
“I see the benefit of having somebody else who’s maybe more focused on either the treatment or the app itself...I don’t think the primary care [provider] needs to be concerned with that. They need to have some knowledge of it, but I think the major, the main focus of it would be with the social worker, plus the social worker would be able to follow up with the patient as far as once they get started using the app. It would be easier follow up for the social worker than the primary care.” (P5, participant with stimulant use disorder. Had prior DUD treatment.)
Patients Felt a Tension Between Effectiveness and Convenience in Aspects of Workflow Design
“Frankly I don’t have a lot of time, so if I could do it over the phone I would...But I think this one [an in-person visit] would be more effective for a lot of people.” (P2, participant taking buprenorphine whose opioid use disorder is in remission. Had prior DUD treatment.)
“Probably I have a slight preference for an in-person visit, just on the basis that I definitely have an easier time talking with someone in-person than over the phone. But yeah, in-person would work a little better in that sense, but on the phone is also very convenient.” (P10, participant with cannabis use disorder. No prior DUD treatment).
The Workflow Needs to Meet Patients Where They Are At
“The first [phone call] I’d even say like within three days. Because if I go to the doctor and you gave me a screening and realize I’m an addict, you give me this thing, I actively want to make a difference, and the next day ...hang out with some friends and do coke, I’m not going to remember that. So two or three days later I would be totally–I wouldn’t be annoyed by that...And then like every week after that, just as a – hey, I’m here. I’m that app, remember? But the likelihood of me actually going home and doing it right away, probably very little. I would either have to get worse in whatever I was doing, or maybe the day after, on that terrible hangover, you’re like God, I need help – that’s when I’d probably look at it, or like look for it and try and find it.” (P4, participant with cannabis and stimulant use disorder. No prior DUD treatment.)
“if it seems like I’m on track and using the app more often, then I wouldn’t need as many reminders, but if I’m off track, then it would be more helpful for someone is keeping up with me more constantly to hold me to it.” (P10, participant with cannabis use disorder. No prior DUD treatment.)
Established Relationships and Trust
Participants told us that they placed value in having a connection with the health care professional who worked with them on using a digital treatment. It would be ideal if this would be someone who had already established a relationship with them, regardless of their clinical role. Even if there was no pre-existing relationship, it was critical for this person to be compassionate and caring because conversations about substance use can be stigmatizing or embarrassing.
Openness to a Team-Based Approach
Although participants had a desire to work with someone with whom they had an existing relationship, the trust between a PCP and a patient appeared to extend to the broader primary care team. That is, a PCP would not necessarily be the one holding [their] hand throughout using an app. Some noted that PCPs are busy and often delegate care to other team members, including DUD care. However, there were some important bounds around this division of responsibility. For instance, having a “third party” from outside the health system support patients was seen as a bad idea.
An important exception to this theme is that a participant firmly wanted to work with one clinician. Needing to talk to an additional person about an app could be an “extra step” and could “add like the risk that they won’t do it at all.”
Tension Between Effectiveness and Convenience
Participants described the pros and cons of the intensity of hypothetical workflow interactions. Regarding the general workflow design, they noted that convenience and effectiveness were in opposition. In-person visits were more “personable” and would hold them more “accountable” to using an app; however, an in-person visit would require more time, travel, and potentially copays. Secure messages, or receiving reminders through the app, were considered convenient because participants could read and respond whenever they wanted; however, they acknowledged that these messages might not be as powerful as a phone call or face-to-face visit in keeping them engaged.
The Workflow Needs to Meet the Patient Where They Are At
Participants suggested that workflows should be tailored to individuals depending on their level of motivation, how often they use substances, and how long or how successfully they have been managing their substance use. For instance, this desire for tailoring reflected that someone with less motivation may need more hands-on help, someone who is using substances daily might want to be contacted every day, and someone who is just beginning to address their use and/or who is actively struggling may need more frequent follow-ups.
Synthesis
Upon synthesis of the qualitative data, and considering the number of participants who preferred each storyboard scenario (Multimedia Appendix 1), we suggest a general approach for facilitating the use of DUD treatment apps, which is described using the story of a fictitious patient (Table 1). Briefly, once the patient expresses interest in using an app for DUD, the PCP confirms their willingness to talk with a clinician with mental health expertise that will help them get started and keep them accountable while initiating treatment. The clinician would teach the patient about the app, help them plan for using the app, agree on a structure for follow-up, and then provide regular follow-up by telephone and messaging (bold boxes, Figure 1). The clinician would further build rapport and ask the patient their preferred cadence, method of follow-up, and modality of contact (eg, in person vs phone). Importantly, the specific approach would remain flexible and tailored to the patient.
Table 1.
Workflow stage | Hypothetical experience for a fictitious patient Cory | Why participants liked this experience | Other experiences preferred by participants |
Introduction stage: learning about the treatment app | Cory completes an annual health screen that asks about alcohol and drug use. Cory’s PCPa expresses concern that her regular substance use could affect her health. Cory is interested in learning about options that could help her change. She agrees to talk to a mental health clinician on the primary care team. Privately, Cory and the mental health clinician discuss Cory’s goals for change, and review a few different options, including a treatment app for drug use. | Participants wanted to discuss substance use with a provider they already knew—but they also recognized that their PCP might not have the time or right expertise. Being seamlessly connected to a mental health clinician would feel “more personal” and provide support beyond what their PCP could offer. | Participants who wanted to talk to only one person—usually their PCP—said they would try to set up the treatment app on their own after their PCP ordered it. |
Setup stage: getting started with the app | A mental health clinician describes features and content of the DUDb treatment app that might be helpful to Cory. They give Cory instructions for how to get started, and they agree to check in after a couple weeks. | Most participants wanted to learn from someone on their care team how an app would benefit them and how to use it. Chances of using the app would be higher if these were discussed when motivation was high. | Several participants felt comfortable getting started with a treatment app on their own. Some said that technical support would be necessary. |
Follow-up stage: getting support over time for engagement and executing a care plan while using the app | Cory gives the app a try. She eventually stops using the app after a couple weeks. However, she re-engages with the app after exchanging secure messages with her mental health clinician that covers a status update, tailored recommendations for using the app, and plans for a follow-up phone call. | Participants said that phone follow-up offered more support than secure messages and placed fewer demands on their time or finances than an in-person visit. Many wanted follow-up spaced out over time to help hold them accountable to using the app. | Benefits of follow-up via secure message include the choice in when and how often to responded to messages (unlike visits or phone calls). In-person appointments would be reserved for additional support and accountability. |
aPCP: primary care provider.
bDUD: drug use disorder.
Discussion
Principal Findings
This study conveyed patient perspectives on the use of apps as part of treatment for DUD in primary care. Overall, participants desired to receive support from their health care teams in using apps and voiced little interest in using them without clinician guidance. There was a consensus among participants that they preferred to work with a trusted, competent clinician who could guide them in using the app over time. They stressed the importance of follow-up and felt that in most cases, this could generally be done through telephone with the addition of asynchronous secure messaging whenever needed.
These findings contribute novel information about patient preferences, laying the groundwork for research on the implementation of apps for DUDs in health care. Patients desire low-barrier, nonstigmatizing interventions for DUD in primary care [69,70]. Apps could potentially help address this gap in care. Although prior clinical trials have relied on research staff to “train” patients in using apps or facilitate their ongoing use [15,21,71-74], future studies can use these findings to inform the involvement of health care teams, instead of researchers, in various aspects of app delivery and implementation.
Overall, participants expressed that the most ideal way to offer apps for DUDs was during routine clinical interactions regarding drug use. This approach enables several aspects of care that participants desired, including the ability to build trust with a clinician, obtain medical and mental health advice, and go home with an appropriate app and clear expectations about treatment. The literature has described an alternative approach, where apps are offered as a self-guided option in the absence of clinician guidance [14,75], potentially reducing health care costs and burden. Such an approach was generally not preferred by participants in this study; some wanted access to a self-care option, but clinician-guided care was viewed as more effective and engaging. The findings are consistent with the literature that describes the need for supportive accountability while delivering app-based treatments, where patients establish a relationship with a helper that, among other things, sets expectations for app use and provides assistance over time [76]. We note that this study excluded participants whose drug use did not lead to a DUD; perhaps, self-care options should be further studied among patients with lower drug use severity.
Participants felt that treatment with an app should be seamless, free of technical glitches, and other barriers that could decrease their motivation or lead to “getting stuck.” Apps and associated clinical workflows need to impose few barriers. Indeed, many people with DUD report logistical barriers to treatment [77-82]. Some participants were also concerned about copays, which contributed to their support preferences. Future studies should design app implementation approaches that address socioeconomic and other barriers that could lead to the inequitable provision of these treatments [83,84].
The literature has also highlighted potential logistical challenges from a health system perspective. For instance, PCPs are heavily burdened and busy [27-29]. Indeed, a prior implementation study of a DUD app had to modify its initial plans by not involving PCPs [51]. Fortunately, participants in this study acknowledged these difficulties and were open to working with team members other than their PCP. Some studies in primary care failed to adequately engage patients in app use over time [21,22], and/or have been halted because workflows were not adequately developed to reach and communicate with patients [23]. This study advances the literature by adding patient voices that can inform future research on app delivery and implementation in routine care.
Limitations
This study has several limitations. The sample size was small, which limits generalizability; however, we followed the standard of saturation in qualitative research. Furthermore, our use of purposeful sampling promoted a rich accounting of a diversity of patient perspectives and preferences across important demographic and substance use characteristics to inform the implementation of apps. Nonetheless, the small sample size limited our ability to identify the ideas and preferences of all possible patients. Although there was saturation in the analysis of workflow design principles and cross-cutting themes (eg, new data analyzed were repetitious of prior data and there was no indication of new emerging themes), the ideas regarding tools remained diverse. In this case, we opted to present a table that captured all participants’ ideas. Future study iterations are driven by additional data collection. Participants were from 5 clinics of a single integrated health care system in the United States from a single geographic region. Participants may have been more accustomed to working with multiple care providers than patients who received care in health systems with little care coordination. Thus, the results might not translate to patients from other groups. Although the goal of this study was to understand patient perspectives, clinicians’ voices were not assessed, which is also a limitation. This study’s focus on patients was driven by knowledge that variations in patient engagement impacted the results of trials of digital treatments [21,22,24-26,73]. Future research should elicit clinician perspectives on the mechanics of delivering apps for DUDs in primary care.
Conclusions
The perspectives of primary care patients with cannabis, opioid, and stimulant use disorders suggest that a user-centered implementation of DUD treatment apps in primary care will require health systems to guide and support patients. Research is needed to evaluate clinician perspectives on workflows for delivering apps and to test the feasibility of the design considerations suggested by participants in this study. One such study is our ongoing implementation research trial funded by the National Institute on Drug Abuse (award number R01DA047954), which uses a randomized factorial design to evaluate different approaches for implementing a digital treatment for substance use disorders, while concurrently collecting qualitative data on implementation and workflow design. As more emphasis is placed on the use of digital treatments for DUDs for primary care patients, convenient methods for engaging patients and supporting them before, during, and after treatment will be paramount.
Acknowledgments
The authors are grateful to the patients of Kaiser Permanente Washington for volunteering for this study. The authors would also like to thank Dr Manu Thakral at the University of Massachusetts Boston for providing input on workflow appropriateness and feasibility and Kristen Delaney and Kay Theis at Kaiser Permanente Washington Health Research Institute for their expert database and electronic medical records programming that made this study possible. Finally, the authors thank the faculty and fellows of the National Institutes of Health Implementation Research Institute and the Mixed Methods Research Training Programs for their feedback on this work. The primary funding for this study was obtained through a pilot award to JEG from the Kaiser Permanente Washington Research Institute Development Fund. JEG and KAB were awarded funding by the National Institute on Alcohol Abuse and Alcoholism under K01AA023859 and K24AA022128. This was also supported by the National Institute on Drug Abuse under award R01DA047954. GMC is supported by the Translational Research Institute UL1TR003107 through the National Center for Advancing Translational Sciences of the National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Abbreviations
- DUD
drug use disorder
- PCP
primary care provider
- PI
principal investigator
Appendix
Interview materials and additional results for the study. Approaches for implementing app-based digital treatments for drug use disorders into primary care: A qualitative, user-centered design study.
Footnotes
Authors' Contributions: Funding acquisition and the first manuscript draft was done by JEG. Study conceptualization and design was conducted by JEG, TEM, CL, ALH, RP, RMC, AGM, GMC, and KAB. Data acquisition was carried out by JEG, TEM, CL, KK, AKL, TB, RP, and RMC. Data analysis was done by TEM, JEG, CL, AKL, and GMC. All authors were involved in editing and manuscript approval.
Conflicts of Interest: Several authors (JEG, RP, RMC, KAB) are investigators on a trial funded by the National Institute on Drug Abuse (R01DA047954; PI: Glass), which evaluates the implementation of reSET and reSET-O that are digital therapeutics for substance use disorder marketed by Pear Therapeutics, Inc. Pear Therapeutics Inc provided digital therapeutic prescriptions at no cost to Kaiser Permanente Washington during a quality improvement pilot study that precedes the implementation trial. JEG is a coinvestigator on a Small Business Innovation Award funded awarded to Pear Therapeutics, Inc by the National Institute on Drug Abuse, which evaluates potential improvements to reSET-O (R44DA042652). This study is informing our research on the implementation of digital therapeutics, but Pear Therapeutics, Inc was not involved in this study and does not provide funding to the authors.
References
- 1.Dwyer-Lindgren L, Bertozzi-Villa A, Stubbs RW, Morozoff C, Shirude S, Unützer J, Naghavi M, Mokdad AH, Murray CJ. Trends and patterns of geographic variation in mortality from substance use disorders and intentional injuries among US counties, 1980-2014. J Am Med Assoc. 2018 Mar 13;319(10):1013–23. doi: 10.1001/jama.2018.0900. http://europepmc.org/abstract/MED/29536097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.US Burden of Disease Collaborators. Mokdad AH, Ballestros K, Echko M, Glenn S, Olsen HE, Mullany E, Lee A, Khan AR, Ahmadi A, Ferrari AJ, Kasaeian A, Werdecker A, Carter A, Zipkin B, Sartorius B, Serdar B, Sykes BL, Troeger C, Fitzmaurice C, Rehm CD, Santomauro D, Kim D, Colombara D, Schwebel DC, Tsoi D, Kolte D, Nsoesie E, Nichols E, Oren E, Charlson FJ, Patton GC, Roth GA, Hosgood HD, Whiteford HA, Kyu H, Erskine HE, Huang H, Martopullo I, Singh JA, Nachega JB, Sanabria JR, Abbas K, Ong K, Tabb K, Krohn KJ, Cornaby L, Degenhardt L, Moses M, Farvid M, Griswold M, Criqui M, Bell M, Nguyen M, Wallin M, Mirarefin M, Qorbani M, Younis M, Fullman N, Liu P, Briant P, Gona P, Havmoller R, Leung R, Kimokoti R, Bazargan-Hejazi S, Hay SI, Yadgir S, Biryukov S, Vollset SE, Alam T, Frank T, Farid T, Miller T, Vos T, Bärnighausen T, Gebrehiwot TT, Yano Y, Al-Aly Z, Mehari A, Handal A, Kandel A, Anderson B, Biroscak B, Mozaffarian D, Dorsey ER, Ding EL, Park E, Wagner G, Hu G, Chen H, Sunshine JE, Khubchandani J, Leasher J, Leung J, Salomon J, Unutzer J, Cahill L, Cooper L, Horino M, Brauer M, Breitborde N, Hotez P, Topor-Madry R, Soneji S, Stranges S, James S, Amrock S, Jayaraman S, Patel T, Akinyemiju T, Skirbekk V, Kinfu Y, Bhutta Z, Jonas JB, Murray CJ. The state of US Health, 1990-2016: burden of diseases, injuries, and risk factors among US states. J Am Med Assoc. 2018 Dec 10;319(14):1444–72. doi: 10.1001/jama.2018.0158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hedegaard H, Miniño AM, Warner M. Center for Disease Control and Prevention. Atlanta, GA: Center for Disease Control and Prevention; 2020. [2020-09-01]. Drug overdose deaths in the United States, 1999–2018. https://www.cdc.gov/nchs/products/databriefs/db356.htm. [Google Scholar]
- 4.The consequences of COVID-19 on the overdose epidemic: overdoses are increasing. Overdose Detection Mapping Application Program (ODMAP) 2020. [2021-05-24]. http://odmap.org/Content/docs/news/2020/ODMAP-Report-May-2020.pdf.
- 5.Grant BF, Goldstein RB, Saha TD, Chou SP, Jung J, Zhang H, Pickering RP, Ruan WJ, Smith SM, Huang B, Hasin DS. Epidemiology of DSM-5 alcohol use disorder: results from the National Epidemiologic Survey on alcohol and related conditions III. JAMA Psychiatry. 2015 Aug;72(8):757–66. doi: 10.1001/jamapsychiatry.2015.0584. http://europepmc.org/abstract/MED/26039070. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Grant BF, Saha TD, Ruan WJ, Goldstein RB, Chou SP, Jung J, Zhang H, Smith SM, Pickering RP, Huang B, Hasin DS. Epidemiology of DSM-5 drug use disorder: results from the National Epidemiologic Survey on alcohol and related conditions-III. JAMA Psychiatry. 2016 Jan;73(1):39–47. doi: 10.1001/jamapsychiatry.2015.2132. http://europepmc.org/abstract/MED/26580136. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ghitza UE, Tai B. Challenges and opportunities for integrating preventive substance-use-care services in primary care through the Affordable Care Act. J Health Care Poor Underserved. 2014 Feb;25(1 Suppl):36–45. doi: 10.1353/hpu.2014.0067. http://europepmc.org/abstract/MED/24583486. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Saitz R, Larson MJ, Labelle C, Richardson J, Samet JH. The case for chronic disease management for addiction. J Addict Med. 2008 Jun;2(2):55–65. doi: 10.1097/ADM.0b013e318166af74. http://europepmc.org/abstract/MED/19809579. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Crowley RA, Kirschner N, Health and Public Policy Committee of the American College of Physicians The integration of care for mental health, substance abuse, and other behavioral health conditions into primary care: executive summary of an American College of Physicians position paper. Ann Intern Med. 2015 Aug 18;163(4):298–9. doi: 10.7326/M15-0510. [DOI] [PubMed] [Google Scholar]
- 10.Institute of Medicine . Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, D.C., United States: National Academy Press; 2006. pp. 1–204. [Google Scholar]
- 11.National Council for Behavioral Health. [2021-05-24]. https://healthinsight.org/tools-and-resources/download/123-educational-resources/1535-implementing-care-for-alcohol-other-drug-use-in-medical-settings-an-extension-of-sbirt.
- 12.Quanbeck AR, Gustafson DH, Marsch LA, McTavish F, Brown RT, Mares M, Johnson R, Glass JE, Atwood AK, McDowell H. Integrating addiction treatment into primary care using mobile health technology: protocol for an implementation research study. Implement Sci. 2014 May 29;9(1):65. doi: 10.1186/1748-5908-9-65. https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-9-65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Marsch LA, Campbell A, Campbell C, Chen C, Ertin E, Ghitza U, Lambert-Harris C, Hassanpour S, Holtyn AF, Hser Y, Jacobs P, Klausner JD, Lemley S, Kotz D, Meier A, McLeman B, McNeely J, Mishra V, Mooney L, Nunes E, Stafylis C, Stanger C, Saunders E, Subramaniam G, Young S. The application of digital health to the assessment and treatment of substance use disorders: the past, current, and future role of the National Drug Abuse Treatment Clinical Trials Network. J Subst Abuse Treat. 2020 Mar;112S:4–11. doi: 10.1016/j.jsat.2020.02.005. https://linkinghub.elsevier.com/retrieve/pii/S0740-5472(19)30582-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hermes ED, Lyon AR, Schueller SM, Glass JE. Measuring the implementation of behavioral intervention technologies: recharacterization of established outcomes. J Med Internet Res. 2019 Jan 25;21(1):e11752. doi: 10.2196/11752. https://www.jmir.org/2019/1/e11752/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Campbell AN, Nunes EV, Matthews AG, Stitzer M, Miele GM, Polsky D, Turrigiano E, Walters S, McClure EA, Kyle TL, Wahle A, Van VP, Goldman B, Babcock D, Stabile PQ, Winhusen T, Ghitza UE. Internet-delivered treatment for substance abuse: a multisite randomized controlled trial. Am J Psychiatry. 2014 Jun;171(6):683–90. doi: 10.1176/appi.ajp.2014.13081055. http://europepmc.org/abstract/MED/24700332. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Gustafson DH, McTavish FM, Chih M, Atwood AK, Johnson RA, Boyle MG, Levy MS, Driscoll H, Chisholm SM, Dillenburg L, Isham A, Shah D. A smartphone application to support recovery from alcoholism: a randomized clinical trial. JAMA Psychiatry. 2014 May;71(5):566–72. doi: 10.1001/jamapsychiatry.2013.4642. http://europepmc.org/abstract/MED/24671165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kaner EF, Beyer FR, Garnett C, Crane D, Brown J, Muirhead C, Redmore J, O'Donnell A, Newham JJ, de Vocht F, Hickman M, Brown H, Maniatopoulos G, Michie S. Personalised digital interventions for reducing hazardous and harmful alcohol consumption in community-dwelling populations. Cochrane Database Syst Rev. 2017 Dec 25;9:CD011479. doi: 10.1002/14651858.CD011479.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kiluk BD, Nich C, Buck MB, Devore KA, Frankforter TL, LaPaglia DM, Muvvala SB, Carroll KM. Randomized clinical trial of computerized and clinician-delivered CBT in comparison with standard outpatient treatment for substance use disorders: primary within-treatment and follow-up outcomes. Am J Psychiatry. 2018 Sep 01;175(9):853–63. doi: 10.1176/appi.ajp.2018.17090978. http://europepmc.org/abstract/MED/29792052. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Pear Therapeutics, Inc., Boston. 2019. Mar 2, [2020-03-02]. https://peartherapeutics.com/products/reset-reset-o/
- 20.Scientific evidence is at the heart of chess. Chess Health. 2021. [2021-02-17]. https://www.chess.health/evidence/
- 21.Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, Barkham M, Bower P, Cooper C, Gask L, Kessler D, Lester H, Lovell K, Parry G, Richards DA, Andersen P, Brabyn S, Knowles S, Shepherd C, Tallon D, White D. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. Br Med J. 2015;351:h5627. doi: 10.1136/bmj.h5627. http://www.bmj.com/cgi/pmidlookup?view=long&pmid=26559241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Knowles SE, Lovell K, Bower P, Gilbody S, Littlewood E, Lester H. Patient experience of computerised therapy for depression in primary care. BMJ Open. 2015 Nov 30;5(11):e008581. doi: 10.1136/bmjopen-2015-008581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Thies K, Anderson D, Cramer B. Lack of adoption of a mobile app to support patient self-management of diabetes and hypertension in a federally qualified health center: interview analysis of staff and patients in a failed randomized trial. JMIR Hum Factors. 2017 Oct 03;4(4):e24. doi: 10.2196/humanfactors.7709. http://humanfactors.jmir.org/2017/4/e24/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Johansson R, Andersson G. Internet-based psychological treatments for depression. Expert Rev Neurother. 2012 Jul;12(7):861–70. doi: 10.1586/ern.12.63. [DOI] [PubMed] [Google Scholar]
- 25.Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther. 2009;38(4):196–205. doi: 10.1080/16506070903318960. [DOI] [PubMed] [Google Scholar]
- 26.Dedert EA, McDuffie JR, Stein R, McNiel JM, Kosinski AS, Freiermuth CE, Hemminger A, Williams JW. Electronic interventions for alcohol misuse and alcohol use disorders: a systematic review. Ann Intern Med. 2015 Aug 4;163(3):205–14. doi: 10.7326/M15-0285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Chernof BA, Sherman SE, Lanto AB, Lee ML, Yano EM, Rubenstein LV. Health habit counseling amidst competing demands: effects of patient health habits and visit characteristics. Med Care. 1999 Aug;37(8):738–47. doi: 10.1097/00005650-199908000-00004. [DOI] [PubMed] [Google Scholar]
- 28.Jaén CR, Stange K, Nutting P. Competing demands of primary care: a model for the delivery of clinical preventive services. J Fam Pract. 1994 Feb;38(2):166–71. [PubMed] [Google Scholar]
- 29.Rost K, Nutting P, Smith J, Coyne J, Cooper-Patrick L, Rubenstein L. The role of competing demands in the treatment provided primary care patients with major depression. Arch Fam Med. 2000 Feb;9(2):150–4. doi: 10.1001/archfami.9.2.150. [DOI] [PubMed] [Google Scholar]
- 30.Ergonomics of Human-System Interaction - Part 210: Human-centred design for interactive systems. Industry Standards Organization. 2019. [2021-05-24]. https://landingpage.bsigroup.com/LandingPage/Undated?UPI=000000000030388991.
- 31.Mickelson RS, Unertl KM, Holden RJ. Medication management: the macrocognitive workflow of older adults with heart failure. JMIR Hum Factors. 2016 Oct 12;3(2):e27. doi: 10.2196/humanfactors.6338. https://humanfactors.jmir.org/2016/2/e27/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Unertl KM, Novak LL, Johnson KB, Lorenzi NM. Traversing the many paths of workflow research: developing a conceptual framework of workflow terminology through a systematic literature review. J Am Med Inform Assoc. 2010;17(3):265–73. doi: 10.1136/jamia.2010.004333. http://europepmc.org/abstract/MED/20442143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Carayon P. The balance theory and the work system model… twenty years later. Int J Hum-Comput Int. 2009 Jun 08;25(5):313–27. doi: 10.1080/10447310902864928. [DOI] [Google Scholar]
- 34.Glass JE, Bobb JF, Lee AK, Richards JE, Lapham GT, Ludman E, Achtmeyer C, Caldeiro RM, Parrish R, Williams EC, Lozano P, Bradley KA. Study protocol: a cluster-randomized trial implementing Sustained Patient-centered Alcohol-related Care (SPARC trial) Implement Sci. 2018 Aug 06;13(1):108. doi: 10.1186/s13012-018-0795-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.McClure JB, Hartzler AL, Catz SL. Design considerations for smoking cessation apps: feedback from nicotine dependence treatment providers and smokers. JMIR Mhealth Uhealth. 2016;4(1):e17. doi: 10.2196/mhealth.5181. http://mhealth.jmir.org/2016/1/e17/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Sheehan D, Lecrubier Y, Sheehan KH, Janavs J, Weiller E, Keskiner A, Schinka J, Knapp E, Sheehan M, Dunbar G. The validity of the Mini International Neuropsychiatric Interview (MINI) according to the SCID-P and its reliability. Eur Psychiatry. 1997;12(5):232–41. doi: 10.1016/s0924-9338(97)83297-x. [DOI] [Google Scholar]
- 37.Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Hergueta T, Baker R, Dunbar GC. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59 Suppl 20:22–34. [PubMed] [Google Scholar]
- 38.Boudreau DM, Lapham G, Johnson EA, Bobb JF, Matthews AG, McCormack J, Liu D, Campbell CI, Rossom RC, Binswanger IA, Yarborough BJ, Arnsten JH, Cunningham CO, Glass JE, Murphy MT, Zare M, Hechter RC, Ahmedani B, Braciszewski JM, Horigian VE, Szapocznik J, Samet JH, Saxon AJ, Schwartz RP, Bradley KA. Documented opioid use disorder and its treatment in primary care patients across six U.S. health systems. J Subst Abuse Treat. 2020 Mar;112S:41–8. doi: 10.1016/j.jsat.2020.02.001. https://linkinghub.elsevier.com/retrieve/pii/S0740-5472(19)30590-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Lapham GT, Lee AK, Caldeiro RM, Glass JE, Carrell DS, Richards JE, Bradley KA. Prevalence of behavioral health conditions across frequency of cannabis use among adult primary care patients in Washington State. J Gen Intern Med. 2018 Nov;33(11):1833–5. doi: 10.1007/s11606-018-4558-8. http://europepmc.org/abstract/MED/29992423. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Lapham GT, Lee AK, Caldeiro RM, McCarty D, Browne KC, Walker DD, Kivlahan DR, Bradley KA. Frequency of cannabis use among primary care patients in Washington State. J Am Board Fam Med. 2017;30(6):795–805. doi: 10.3122/jabfm.2017.06.170062. http://www.jabfm.org/cgi/pmidlookup?view=long&pmid=29180554. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Richards JE, Bobb JF, Lee AK, Lapham GT, Williams EC, Glass JE, Ludman EJ, Achtmeyer C, Caldeiro RM, Oliver M, Bradley KA. Integration of screening, assessment, and treatment for cannabis and other drug use disorders in primary care: an evaluation in three pilot sites. Drug Alcohol Depend. 2019 Aug 01;201:134–41. doi: 10.1016/j.drugalcdep.2019.04.015. http://europepmc.org/abstract/MED/31212213. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Sayre M, Lapham GT, Lee AK, Oliver M, Bobb JF, Caldeiro RM, Bradley KA. Routine assessment of symptoms of substance use disorders in primary care: prevalence and severity of reported symptoms. J Gen Intern Med. 2020 Apr;35(4):1111–9. doi: 10.1007/s11606-020-05650-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Edwards P, Cooper R, Roberts I, Frost C. Meta-analysis of randomised trials of monetary incentives and response to mailed questionnaires. J Epidemiol Community Health. 2005 Nov;59(11):987–99. doi: 10.1136/jech.2005.034397. http://jech.bmj.com/cgi/pmidlookup?view=long&pmid=16234429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Doody MM, Sigurdson AS, Kampa D, Chimes K, Alexander BH, Ron E, Tarone RE, Linet MS. Randomized trial of financial incentives and delivery methods for improving response to a mailed questionnaire. Am J Epidemiol. 2003 Apr 01;157(7):643–51. doi: 10.1093/aje/kwg033. [DOI] [PubMed] [Google Scholar]
- 45.Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. 2015 Sep;42(5):533–44. doi: 10.1007/s10488-013-0528-y. http://europepmc.org/abstract/MED/24193818. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Hasin DS, Greenstein E, Aivadyan C, Stohl M, Aharonovich E, Saha T, Goldstein R, Nunes EV, Jung J, Zhang H, Grant BF. The Alcohol Use Disorder and Associated Disabilities Interview Schedule-5 (AUDADIS-5): procedural validity of substance use disorders modules through clinical re-appraisal in a general population sample. Drug Alcohol Depend. 2015 Mar 01;148:40–6. doi: 10.1016/j.drugalcdep.2014.12.011. http://europepmc.org/abstract/MED/25604321. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Virzi RA. Refining the test phase of usability evaluation: how many subjects is enough? Hum Factors. 2016 Nov 23;34(4):457–68. doi: 10.1177/001872089203400407. [DOI] [Google Scholar]
- 48.Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, Burroughs H, Jinks C. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. 2018;52(4):1893–907. doi: 10.1007/s11135-017-0574-8. http://europepmc.org/abstract/MED/29937585. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Adlin T, Pruitt J. The Essential Persona Lifecycle: Your Guide to Building and Using Personas. Burlington, Massachusetts, United States: Morgan Kaufmann; 2010. pp. 1–10. [Google Scholar]
- 50.Truong K, Hayes G, Abowd G. Storyboarding: an empirical determination of best practices and effective guidelines. Proceedings of the 6th conference on Designing Interactive systems; DIS06: Designing Interactive Systems 2006; June 26-28, 2006; University Park PA USA. 2006. pp. 12–21. [DOI] [Google Scholar]
- 51.Mares M, Gustafson DH, Glass JE, Quanbeck A, McDowell H, McTavish F, Atwood AK, Marsch LA, Thomas C, Shah D, Brown R, Isham A, Nealon MJ, Ward V. Implementing an mHealth system for substance use disorders in primary care: a mixed methods study of clinicians' initial expectations and first year experiences. BMC Med Inform Decis Mak. 2016 Sep 29;16(1):126. doi: 10.1186/s12911-016-0365-5. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-016-0365-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Lapham GT, Hawkins EJ, Chavez LJ, Achtmeyer CE, Williams EC, Thomas RM, Ludman EJ, Kypri K, Hunt SC, Bradley KA. Feedback from recently returned veterans on an anonymous web-based brief alcohol intervention. Addict Sci Clin Pract. 2012 Aug 28;7:17. doi: 10.1186/1940-0640-7-17. https://ascpjournal.biomedcentral.com/articles/10.1186/1940-0640-7-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.O'Connor S, Hanlon P, O'Donnell CA, Garcia S, Glanville J, Mair FS. Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Med Inform Decis Mak. 2016 Sep 15;16(1):120. doi: 10.1186/s12911-016-0359-3. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-016-0359-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Jochems EC, Mulder CL, Duivenvoorden HJ, van der Feltz-Cornelis CM, van Dam A. Measures of motivation for psychiatric treatment based on self-determination theory: psychometric properties in Dutch psychiatric outpatients. Assessment. 2014 Aug 02;21(4):494–510. doi: 10.1177/1073191113517928. [DOI] [PubMed] [Google Scholar]
- 55.Ryan RM, Plant RW, O'Malley S. Initial motivations for alcohol treatment: relations with patient characteristics, treatment involvement, and dropout. Addict Behav. 1995 May;20(3):279–97. doi: 10.1016/0306-4603(94)00072-7. [DOI] [PubMed] [Google Scholar]
- 56.Urbanoski KA, Wild TC. Assessing self-determined motivation for addiction treatment: validity of the Treatment Entry Questionnaire. J Subst Abuse Treat. 2012 Jul;43(1):70–9. doi: 10.1016/j.jsat.2011.10.025. [DOI] [PubMed] [Google Scholar]
- 57.Cloud R, Kingree J. Concerns about dose and underutilization of twelve-step programs: models, scales, and theory that inform treatment planning. Recent Dev Alcohol. 2008;18:283–301. doi: 10.1007/978-0-387-77725-2_16. [DOI] [PubMed] [Google Scholar]
- 58.Wild TC, Cunningham JA, Ryan RM. Social pressure, coercion, and client engagement at treatment entry: a self-determination theory perspective. Addict Behav. 2006 Oct;31(10):1858–72. doi: 10.1016/j.addbeh.2006.01.002. [DOI] [PubMed] [Google Scholar]
- 59.Wild T, Newton-Taylor B, Alletto R. Perceived coercion among clients entering substance abuse treatment: structural and psychological determinants. Addict Behav. 1998 Jan;23(1):81–95. doi: 10.1016/s0306-4603(97)00034-8. [DOI] [PubMed] [Google Scholar]
- 60.Bauer AM, Rue T, Keppel GA, Cole AM, Baldwin L, Katon W. Use of mobile health (mHealth) tools by primary care patients in the WWAMI region Practice and Research Network (WPRN) J Am Board Fam Med. 2014;27(6):780–8. doi: 10.3122/jabfm.2014.06.140108. http://www.jabfm.org/cgi/pmidlookup?view=long&pmid=25381075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Hartzler AL, BlueSpruce J, Catz SL, McClure JB. Prioritizing the mHealth design space: a mixed-methods analysis of smokers' perspectives. JMIR Mhealth Uhealth. 2016 Aug 05;4(3):e95. doi: 10.2196/mhealth.5742. http://mhealth.jmir.org/2016/3/e95/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.McClure JB, Anderson ML, Bradley K, An LC, Catz SL. Evaluating an adaptive and interactive mhealth smoking cessation and medication adherence program: a randomized pilot feasibility study. JMIR Mhealth Uhealth. 2016 Aug 03;4(3):e94. doi: 10.2196/mhealth.6002. http://mhealth.jmir.org/2016/3/e94/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Andréasson S, Danielsson A, Wallhed-Finn S. Preferences regarding treatment for alcohol problems. Alcohol Alcohol. 2013;48(6):694–9. doi: 10.1093/alcalc/agt067. [DOI] [PubMed] [Google Scholar]
- 64.Dedoose Version 8.0.35, web application for managing, analyzing, and presenting qualitative and mixed method research data. SocioCultural Research Consultants, Los Angeles, CA2018. [2021-05-24]. https://www.dedoose.com/
- 65.King N. Using templates in the thematic analysis of text. In: Cassell C, Symon G, editors. Essential Guide to Qualitative Methods in Organizational Research. Thousand Oaks, California, United States: SAGE Publications Ltd; 2004. pp. 256–70. [Google Scholar]
- 66.Creswell J. Qualitative Inquiry and Research Design: Choosing Among Five Approaches, 3rd Ed. Thousand Oaks, CA: SAGE Publications; 2013. pp. 1–448. [Google Scholar]
- 67.Iversen RR. Book Review: Padgett, D. K. (2008). Qualitative Methods in Social Work Research (2nd ed.). Thousand Oaks, CA: Sage. 304 pp., $49.95 (paperback), ISBN 978-1-4129-5193-7; $84.95 (hardbound), ISBN 978-1-4129-5192-0. Res Soc Work Pract. 2009 Dec 03;20(1):125–6. doi: 10.1177/1049731509347859. [DOI] [Google Scholar]
- 68.Beyer H, Holtzblatt K. Contextual Design. San Francisco, CA: Morgan Kaufman; 1998. pp. 1–496. [Google Scholar]
- 69.Wallhed FS, Bakshi A, Andréasson S. Alcohol consumption, dependence, and treatment barriers: perceptions among nontreatment seekers with alcohol dependence. Subst Use Misuse. 2014 May;49(6):762–9. doi: 10.3109/10826084.2014.891616. [DOI] [PubMed] [Google Scholar]
- 70.Barry CL, Epstein AJ, Fiellin DA, Fraenkel L, Busch SH. Estimating demand for primary care-based treatment for substance and alcohol use disorders. Addiction. 2016 Aug;111(8):1376–84. doi: 10.1111/add.13364. http://europepmc.org/abstract/MED/26899802. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Christensen DR, Landes RD, Jackson L, Marsch LA, Mancino MJ, Chopra MP, Bickel WK. Adding an internet-delivered treatment to an efficacious treatment package for opioid dependence. J Consult Clin Psychol. 2014 Dec;82(6):964–72. doi: 10.1037/a0037496. http://europepmc.org/abstract/MED/25090043. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Bennett GG, Steinberg D, Askew S, Levine E, Foley P, Batch BC, Svetkey LP, Bosworth HB, Puleo EM, Brewer A, DeVries A, Miranda H. Effectiveness of an app and provider counseling for obesity treatment in primary care. Am J Prev Med. 2018 Dec;55(6):777–86. doi: 10.1016/j.amepre.2018.07.005. http://europepmc.org/abstract/MED/30361140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.Gilbody S, Brabyn S, Lovell K, Kessler D, Devlin T, Smith L, Araya R, Barkham M, Bower P, Cooper C, Knowles S, Littlewood E, Richards DA, Tallon D, White D, Worthy G, REEACT collaborative Telephone-supported computerised cognitive-behavioural therapy: REEACT-2 large-scale pragmatic randomised controlled trial. Br J Psychiatry. 2017 May;210(5):362–7. doi: 10.1192/bjp.bp.116.192435. [DOI] [PubMed] [Google Scholar]
- 74.Graham AK, Greene CJ, Kwasny MJ, Kaiser SM, Lieponis P, Powell T, Mohr DC. Coached mobile app platform for the treatment of depression and anxiety among primary care patients: a randomized clinical trial. JAMA Psychiatry. 2020 Sep 01;77(9):906–14. doi: 10.1001/jamapsychiatry.2020.1011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.Muñoz RF. The efficiency model of support and the creation of digital apothecaries. Clin Psychol: Sci Pract. 2017 Mar;24(1):46–9. doi: 10.1111/cpsp.12174. [DOI] [Google Scholar]
- 76.Mohr DC, Duffecy J, Jin L, Ludman EJ, Lewis A, Begale M, McCarthy M. Multimodal e-mental health treatment for depression: a feasibility trial. J Med Internet Res. 2010 Dec 19;12(5):e48. doi: 10.2196/jmir.1370. http://www.jmir.org/2010/5/e48/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Perron BE, Mowbray OP, Glass JE, Delva J, Vaughn MG, Howard MO. Differences in service utilization and barriers among Blacks, Hispanics, and Whites with drug use disorders. Subst Abuse Treat Prev Policy. 2009 Mar 13;4(1):-. doi: 10.1186/1747-597x-4-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78.Copeland J. A qualitative study of barriers to formal treatment among women who self-managed change in addictive behaviours. J Subst Abuse Treat. 1997;14(2):183–90. doi: 10.1016/s0740-5472(96)00108-0. [DOI] [PubMed] [Google Scholar]
- 79.Cunningham JA, Sobell LC, Sobell MB, Agrawal S, Toneatto T. Barriers to treatment: why alcohol and drug abusers delay or never seek treatment. Addict Behav. 1993 May;18(3):347–53. doi: 10.1016/0306-4603(93)90036-9. [DOI] [PubMed] [Google Scholar]
- 80.Rapp RC, Xu J, Carr CA, Lane DT, Wang J, Carlson R. Treatment barriers identified by substance abusers assessed at a centralized intake unit. J Subst Abuse Treat. 2006 Apr;30(3):227–35. doi: 10.1016/j.jsat.2006.01.002. http://europepmc.org/abstract/MED/16616167. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Sexton RL, Carlson RG, Leukefeld CG, Booth BM. Barriers to formal drug abuse treatment in the rural south: a preliminary ethnographic assessment. J Psychoactive Drugs. 2008 Jun;40(2):121–9. doi: 10.1080/02791072.2008.10400621. [DOI] [PubMed] [Google Scholar]
- 82.van der Pol P, Liebregts N, de Graaf R, Korf DJ, van den Brink W, van Laar M. Facilitators and barriers in treatment seeking for cannabis dependence. Drug Alcohol Depend. 2013 Dec 01;133(2):776–80. doi: 10.1016/j.drugalcdep.2013.08.011. [DOI] [PubMed] [Google Scholar]
- 83.Cheng C, Beauchamp A, Elsworth GR, Osborne RH. Applying the electronic health literacy lens: systematic review of electronic health interventions targeted at socially disadvantaged groups. J Med Internet Res. 2020 Aug 13;22(8):e18476. doi: 10.2196/18476. https://www.jmir.org/2020/8/e18476/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Fang ML, Canham SL, Battersby L, Sixsmith J, Wada M, Sixsmith A. Exploring privilege in the digital divide: implications for theory, policy, and practice. Gerontologist. 2019 Jan 09;59(1):1–15. doi: 10.1093/geront/gny037. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Interview materials and additional results for the study. Approaches for implementing app-based digital treatments for drug use disorders into primary care: A qualitative, user-centered design study.
Data Availability Statement
The interview materials are provided in Multimedia Appendix 1. Additional interview materials are available from the first author upon reasonable request.