Experience Starting an Addiction Medicine Practice-based Research Network

Abstract

Objectives:

This paper outlines the experience developing AMNet (Addiction Medicine Practice-Based Research Network), which promotes the adoption of patient-reported outcome measures (PROMs) and measurement-based care in outpatient addiction treatment practices and creates a platform for quality improvement and research.

Methods:

From August 2019 to July 2023, the AMNet team selected patient-reported outcome measures for implementation in the American Psychiatric Association’s clinical data registry (PsychPRO), recruited addiction medicine providers, and collected PROMs data.

Results:

AMNet selected 12 PROMs for implementation in PsychPRO. Through July 2023, 1,565 providers expressed interest, of whom 216 of the 929 eligible providers (23%) attended an onboarding call/webinar. Two hundred and six providers (95%) from 54 practices returned Participation Agreements. Subsequently, 65 providers (32%) from 39 practices withdrew, resulting in 141 (68%) providers from 15 practices. From November 2020 to July 2023, 38 providers submitted PROMs data using one of three PsychPRO patient portals. Sixteen of the 53 providers (30%) who signed up for the initial portal collected data from 468 patients. As of July 2023, 83 of the 141 providers (59%) opted to submit PROMs data from their own portal or electronic health record.

Conclusions:

Next steps will include continued recruitment of providers, addressing barriers to data transfer, and integrating data from providers’ portals into the registry to create a platform for future research.

Introduction

Patient-Reported Outcome Measures (PROMs) can be used to gather information directly from patients on their symptoms and outcomes.¹ Their use to assess patient health status and drive improvements in patient outcomes through shared decision making is a critical component of measurement-based care (MBC).^2,3,4 Despite being considered useful for improving quality of medical care, PROMs are not routinely implemented as standard practice.^5,6 When they are collected, they are often not standardized across practices to facilitate data interpretation.⁷ Additionally, many practices lack access to technology, or the reporting tools needed to aggregate data to track and benchmark measures for quality improvement (QI).⁸

Data collected for MBC and practice-level quality measurement have been used to create disease registries for diabetes and cardiovascular conditions.^9,10 Such registries can be used for QI to evaluate outcomes, offer performance feedback to clinicians, and study real-world practice.¹¹ Efforts are needed to expand the implementation of MBC and the use of patient registries as platforms for QI and research in addiction medicine.¹²

The Addiction Medicine Practice-based Research Network (AMNet) was developed as a strategy for increasing the use of PROMS to facilitate MBC. Previous reports on AMNet focused on the partnerships involved in the collaboration, selection of its PROMs, and AMNet providers’ views on implementing an electronic system to collect PROMs data.^{13,14, 15} The present paper describes AMNet’s experience in recruiting addiction treatment providers and implementing PROMs in electronic systems for provider use in MBC.

Methods

AMNet was funded through a cooperative agreement from the National Institute on Drug Abuse (NIDA) with funds from the US Department of Health and Human Services, Assistant Secretary for Planning and Evaluation, Office of the Secretary Patient-Centered Outcomes Research Trust Fund. Partners in the research included the American Psychiatric Association (APA), American Society of Addiction Medicine (ASAM), and Friends Research Institute (FRI), in collaboration with the American Academy of Addiction Psychiatry (AAAP).¹³ AMNet utilized the APA’s qualified clinical data registry (PsychPRO) as its foundation, adding substance use disorder (SUD)-specific outcomes and quality measures to its array of mental health outcome and quality measures. The APA implemented PsychPRO in 2017 with a single-source health information technology (HIT) vendor to develop the technological infrastructure to support quality data collection and CMS Merit-Based Incentive Payment System (MIPS) reporting. Since then, PsychPRO has provided quality data collection and reporting processes for more than 70 practices and over 1,000 clinicians. During this time, PsychPRO has grown and established collaborations with HIT vendors to build out an infrastructure that supports data collection including patient-reported information and electronic health record (EHR) clinical data, that ultimately comprise a de-identified data repository that can facilitate mental health research.

AMNet activities described herein were conducted from August 2019 to July 2023. Activities included selecting PROMS and implementing them in PsychPRO, recruiting providers, collecting PROMs data, and creating a registry for future clinical research.

The APA’s Institutional Review Board approved AMNet as an exempted QI activity under the umbrella of PsychPRO.

Selection and Implementation of AMNet PROMs in PsychPRO Portal

As described elsewhere, the AMNet team identified and selected addiction medicine-specific PROMs following review and approval by consensus of its Steering Committee consisting of the project’s investigators, AMNet’s NIDA Science Officer, three external clinician researchers, one patient representative, and two scientists from federal agencies.¹⁴ Preferred instruments were: (1) brief; (2) clinically useful; (3) in the public domain at no cost; and (4) psychometrically sound.

AMNet PROMs were integrated into PsychPRO’s original web-based patient and clinician portal developed by PsychPRO’s HIT vendor as an add on to the registry functionality. Beta testing assessed the clinical utility and feasibility of their use. Providers manually entered or electronically transferred patient demographic data into the system and assigned PROMS. Twenty-four hours prior to their appointment, patients received provider emails with a link to complete the assigned PROMs. Subsequently, providers could view a report with PROMs results and summary scores to help them track patients’ clinical progress.

Provider Recruitment

To create a robust registry, the AMNet team determined a priori to recruit 120 providers in office-based practice who treated patients with opioid use disorder (OUD) in order to have 6,000 or more patients. Recruitment occurred over two years. The emphasis on providers treating OUD was due to efforts to address the opioid epidemic and to learn more about the characteristics and outcomes of patients with OUD treated in outpatient physician practices. Eligible providers included physicians, physician assistants, and nurse practitioners providing OUD treatment. The initial inclusion criteria were: (1) United States (US) location; (2) having at least 100 patients of whom at least 26% were being treated for OUD; (3) prescribing buprenorphine or extended-release naltrexone; (4) utilizing EHRs; and (5) practice willing to participate in AMNet. Universities or Veterans Affairs (VA) settings were initially excluded because they represent large systems with pre-existing MBC/QI activities. Opioid Treatment Programs (OTPs) were excluded because much of the extant OUD treatment research has been carried out in these settings. Due to recruitment challenges in the first year, related mainly to COVID-19, practice eligibility criteria were expanded to include: (1) university affiliation; (2) having at least five patients being treated for OUD or alcohol use disorder (AUD); (3) with or without EHRs; and (4) non-OTP, state-certified outpatient treatment programs. Membership in APA, ASAM, or AAAP was not required.

We used a passive recruitment strategy based on APA’s prior recruitment efforts for PsychPRO. Communication materials directed interested providers to the APA’s AMNet webpage and sign-up portal which outlined the activities, eligibility criteria, and the offer of joining AMNet. Recruitment communications were posted on the APA and ASAM membership LISTSERVs, as well as other online communities for addiction medicine providers, and via US mail. Providers who joined could receive free access to PsychPRO to obtain real-time data to track patients’ progress; access to webinars with free continuing medical education credit; the option to participate in MIPS; an honorarium of $1,000 based on level of participation; potential credit towards American Board of Psychiatry and Neurology Maintenance of Certification Part Four, and the opportunity to participate in future research.

Through the recruitment survey and sign-up portal, interested providers were asked to provide information on their patient and practice characteristics, use of medications for OUD, barriers to pharmacotherapy, existing outcome measurement practices, and interest in conducting research. The sign-up portal also included an option to respond to items on COVID-related practice adaptations. The team, consisting of two staff members from FRI, three from APA, and one from ASAM, assessed responses for eligibility.

Eligible practices were required to sign the PsychPRO Registry Participation Agreement and the Business Associate and Data Use Agreement affirming their willingness to comply with the registry’s data, privacy, and security requirements. AMNet staff provided onboarding webinars to introduce PsychPRO’s patient and clinician portals and the scope of data collection. Providers were able to speak with an AMNet team member as needed for more information.

PROMs Data Collection

Practices had the option to use a PsychPRO portal or their own patient portals to collect and monitor patient PROMs data. Transfer of PROMs data to the registry from sites using their own portals are ongoing and hence are not included in the present report.

Results

Selection and Implementation of AMNet PROMs in PsychPRO Portal

The Steering Committee approved 12 PROMs and categorized them as either Tier 1/core or Tier 2/optional by consensus as the MBC framework for AMNet (Table 1). The AMNet MBC framework and PROMs were implemented in PsychPRO’s original portal. However, as PsychPRO continued to grow during the AMNet implementation a change in HIT vendor was needed to accommodate expanding registry functionality. The APA adopted a collaborative approach with MBC specialized vendors to take advantage of state of art technology to meet the needs of the participating clinicians, including AMNet participants. Following a change in PsychPRO HIT vendors, AMNet PROM implementation was staggered since clinicians indicated a desire to prioritize a limited set of PROMs (Figure 1). PsychPRO’s original portal was discontinued on the project on December 31, 2021, stemming in part from providers’ dissatisfaction with the portal’s user interface and software limitations, such as PROM access challenges for patients without email addresses or computers¹⁵. In March 2021, the APA hired TryCycle Data Systems to implement TetherAll¹⁶, its mobile app-based tool to provide a portal that would address patients’ challenges with logging onto the original web-based portal. In February 2022, the APA contracted with mdlogix to adopt its bhworks platform to have a more user-friendly web-based portal as a complement to the app-based TetherAll.¹⁷

Table 1.

Assessment Measures Included in AMNet

Tier	Tool Name	Description
I	Tobacco, Alcohol, Prescription Medications, and other Substances (TAPS 1) Tool30	4-item screen for substance use followed by substance-specific brief assessment for screen positives
	Brief Addiction Monitor (BAM)31	17-item screen to assess risk factors associated with relapse or worsening addiction severity and protective factors
	PhenX Cigarette Smoking Status32	4-item self-rated tool to assess smoking status
	PhenX Injection Drug Use measure33	7-item self-rated tool to assess injection drug use
	Treatment Effectiveness Assessment (TEA)34	4-item self-rated tool to assess progress in treatment and recovery based on substance use, health, lifestyle, and community engagement
	Short Opiate Withdrawal Scale (SOWS- Gossop)35	10-item self-rated screening tool to evaluate opioid withdrawal symptoms
	Single-item Visual Analog Scale (VAS)36	Subjective rating opioid craving scale
	Patient Health Questionnaire (PHQ)-2+137	3-item self-report tool to screen for depression and suicidal ideation
II	Clinical Opiate Withdrawal Scale (COWS)38	11-item clinician-rated scale of opiate withdrawal
	PHQ-937	9-item self-report tool to screen for and monitor severity of depression
	Columbia-Suicide Severity Rating Scale (C-SSRS+)39	Suicidal ideation and behavior rating scale
	PROMIS Pain Interference Instrument-Adult Short40	Self-report tool for consequences of pain’s impact on activities of daily living

Figure 1. Timeline of Changes in PsychPRO Registry and Portal Providers.

Note: APA=American Psychiatric Association; HIT= Health Information Technology; AMNet=Addiction Medicine Practice-based Research Network; PROMs=Patient-reported Outcome Measures

Initial PsychPRO Portal:

The AMNet team and the APA’s initial HIT vendor began implementing selected PROMs in the PsychPRO portal in January 2020. Providers were instructed to assign baseline assessments at the initial patient visit. Brief follow-ups could be completed at subsequent visits, as appropriate. The PROMs were tested in June 2020 and patient data entry began in September 2020.

TetherAll:

TetherAll began by implementing the three most frequently assigned PROMs among AMNet providers who used the initial portal between June and December 2021. This included the Brief Addiction Monitor [BAM], the Patient Health Questionnaire-2+1 [PHQ-2+1], and the Visual Analog Scale [VAS]. Subsequently, the Treatment Effectiveness Assessment (TEA), the Tobacco, Alcohol, Prescription Medications, and other Substances Screening Tool (TAPS Tool), and the PhenX Cigarette Smoking Status assessment were implemented between August 2022 and April 2023.

Bhworks:

Similar to implementation experiences with TetherAll, the BAM, the PHQ-9, and the TAPs Tool were implemented in May 2022 followed by the TEA and VAS in November 2022.

Provider Recruitment

The AMNet webpage was launched in June 2020 after declaration of the COVID-19 public health emergency. Initial recruitment efforts were slowed due to the burden the COVID-19 pandemic caused on practices and providers who had to adjust clinical workflows and, in many cases, to shift to remote treatment to continue to minimize the risk for COVID-19 transmission.¹⁸ As a result, practice recruitment was slow in the first year of the initiative and began to pick up in the second year as providers and practices adjusted to adaptations to care delivery. From June 2020 to July 2023, 1,565 providers expressed interest in joining AMNet through the sign-up portal of whom 1,565 providers, 929 (59%) were eligible. 216 of the 929 eligible providers (23%) attended an onboarding call/webinar. Two hundred and six providers (95%) from 54 practices returned Participation Agreements. Subsequently, 65 providers (32%) from 39 practices withdrew resulting in 141 providers (68% of those 206 who returned Participation Agreements) from 15 practices (see Figure 2). Providers cited time constraints and lack of support staff to assist in participation as the main factors contributing to withdrawal from the initiative. The challenges faced by participating providers is described elsewhere.¹⁵ Practices were distributed across 15 states and consisted of eight (53%) solo practices, four (27%) small group practices (two to nine providers), and three (20%) large group practices (19–62 providers). Two practices were non-profit (13%) and the remainder were for-profit (87%).

Figure 2. AMNet Participant Recruitment Flowchart.

Note: AMNet=Addiction Medicine Practice-based Research Network

As the project launch coincided with the start of the COVID 19 pandemic, the team included items related to COVID-19 practice adaptations to its sign-up survey as mentioned above. Items included potential adaptations to medication for opioid use disorder practices, urine drug screening practices, medication visit type (i.e. in person versus virtual), and patient impact. The results of these item responses have been reported elsewhere.¹⁸

The AMNet webpage also contained a link for participating providers (https://www.psychiatry.org/psychiatrists/research/amnet/amnet-participant-resources) to view resources developed for the initiative, including a copy of the PROMs and their accompanying guide, practical advice on discussing MBC with patients and using AMNet PROMS to assess OUD and AUD. There were also webinars for CME credit, newsletters, and tutorials on using the PROMs portals. The webinar topics were selected based on provider suggestions and Steering Committee members’ recommendations for salient topics to the field. They were recorded and made available on APA and ASAM’s Learning Management Systems. Of the 3,558 providers who enrolled in the courses, 2,241 providers completed a webinar course and obtained CME credit. These AMNet courses remain available on APA and ASAM’s platforms. Providers who used the PROMs portals found the tutorials and one-on-one training with AMNet team members helpful as they learned to navigate the systems. Although providers expressed interest in resources designed to assist with the implementation of MBC in their practice, several expressed not having the time or staff support to aid in implementing the processes recommended in these resources.

PROMs Data Collection

As of July 2023, 83 providers from four practices planned to use their own portals in place of one of the PsychPRO portals. The remaining providers planned to use the following PsychPRO portals.

Initial PsychPRO Portal:

A total of 53 providers from 26 practices signed up to use the initial portal of whom 16 providers (30%) collected data from 468 patients (see Table 2). The majority of patient PROMs data were gathered by three of the providers. The low uptake of the portal was associated with provider feedback that the portal was not user-friendly for both providers and patients, as well as patient difficulty accessing the portal due to lack of computer access or an email account.¹⁵

Table 2.

Data Collected from Original web-based AMNet Portal, latest web-based portal (BH Works), and app-based portal (TetherAll) through July 2023

Measures	Original Portal*	BHWorks	TetherAll	Total
Brief Addiction Monitor (BAM)	785	437	2,338	3,560
Patient Health Questionnaire (PHQ)-2+1	595	116	2,348	3,059
Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS Tool 1)	65	28	194	287
The Treatment Effectiveness Assessment (TEA)	154	3	178	335
Craving Visual Analogue Scale (VAS)	100	3	604	707
PROMIS Pain	11	6	N/A	17
Columbia-Suicide Severity Rating Scale (C-SSRS)	0	2	N/A	2
PhenX Cigarette Smoking Status (Baseline)	66	N/A	12	78
PhenX Injection Drug Use (Baseline)	26	N/A	N/A	26
Short Opiate Withdrawal Scale (SOWS)	2	N/A	N/A	2

Note:

^*Original portal data shown represent PROMs that were at least 75% completed.

The number of providers and practices that submitted PROMs data in each portal were: 16 providers from 16 practices in Original Portal, although the majority of patient PROMs data were gathered from three providers; 17 providers from seven practices in BHworks; eight providers from eight practices in TetherAll. Some of the providers that entered data subsequently withdrew and some used multiple portals, switching between them as new portals became available.

TetherAll:

Forty-four providers from 12 practices signed up to use TetherAll. Of these 44 providers, eight providers (18.2%) from eight practices entered patient data. It should be noted that one large practice with 31 of the 44 providers who joined TetherAll had not yet begun to use it as of this writing, leaving eight of 13 providers (excluding the large practice) using TetherAll (62%). The data collected from December 2021 to July 2023 are presented in Table 2.

Bhworks:

AMNet data collection via bhworks began in June 2022. Seventeen of the twenty-five providers (68%) who signed up to use bhworks entered patient data. The data collected from June 2022 to July 2023 are presented in Table 2. As can be seen in Table 2, across all three portals, a total of 3,560 Brief Addiction Monitor assessments and 3,059 Patient Health Questionnaires were gathered.

Discussion

Although the use of PROMs for MBC has been reported in the medical and mental health literature^{19, 20, 21} PROMs adoption in addiction treatment settings has been rare, outside of the Veterans Administration.^{19, 22, 23} While it is possible that the benefits of MBC observed in mental health treatment settings could be applicable to addiction treatment settings, there has been limited research investigating such benefits.²⁴ The limited adoption of PROMS in addiction treatment may stem from the absence of a well-established MBC approach, due to patient, provider, and system-level challenges including workflow disruption, limited clinical time, and poor integration of PROMs into EHRs.^24,25 This paper describes the experience starting AMNet, recruiting providers, and adopting PROMs through PsychPRO portals in outpatient addiction treatment practices.

AMNet was able to exceed its recruitment membership target of 120 medical providers, despite the challenges wrought by COVID-19 on ambulatory addiction treatment practices, and the loss of 32% of eligible providers after signing the Participation Agreement. Approximately one-quarter of those who signed up to use the original web-based portal began to collect PROMs data.

AMNet providers did not consider the initial portal to be user-friendly and patients without access to computers or email accounts to access PROMs assignments were not able to use it. The lack of patient-friendly PROMs collection software was also found to be a hindrance to implementing MBC in a study of large hospital systems.²⁶ To improve utilization, an app-based portal was made available in PsychPRO for patients to complete select PROMs on their smartphones. Excluding a large practice that committed to, but had not yet used the app, nearly two-thirds of the providers who signed up for this app-based portal were able to gather PROMs data. In addition, a new more user-friendly web-based portal was engaged; approximately two-thirds of the providers who signed up to use it were able to collect PROMs data.

A growing number of PROMs have been collected as a result of changing the portals. In contrast, the extent to which PROMs were collected in the providers’ own portals is unknown due to the wide variety of EHRs and EHR vendors which posed a barrier to data transfer. Going forward, it will be necessary to incorporate these PROMS data with those from the PsychPRO portals. In the future, to facilitate MBC implementation in addiction treatment practices, more research is needed for practices to understand how to overcome issues associated with limited staffing, clinician motivation and EHR interoperability.

The challenges of gathering PROMS data are not unique to AMNet and smaller practices. While there is limited literature on MBC adoption in addiction treatment settings, specifically in smaller clinics and practices, some challenges have been identified such as the lack of an EHR or insufficient staff support.^27,28 Insufficient support staff was also a challenge encountered during AMNet, and it, along with a lack of time, resulted in provider drop-out. In contrast, the majority of data were submitted by solo clinicians who were engaged and took advantage of support from registry staff. This may point to motivational characteristics that vary across clinicians that need further study.

Hospitals, healthcare organizations and mental health professionals have encountered varying degrees of success in implementing PROMs as standard care in clinical settings.^29,6 Leaders of large hospital systems identified a lack of clinician awareness and experience with PROMs as barriers to their implementation.²⁶ Some leaders offered increased clinician training on the use of PROMs. AMNet provided training and resources to providers on PROMs as well.

Although the integration of PROMs into a practice’s EHR could simplify data collection and eliminate the need for challenging data transfer processes, small practices in AMNet had neither IT staff, nor additional funds to support integration of PROMs with their EHRs. As a workaround, smaller AMNet practices had the option to upload a PDF of assessment results to their EHRs. Making PROMs available to small practices without an EHR or to practices with an EHR willing to use a separate PROMs portal can help to support these practices in monitoring patients, obtaining outcome data, and conducting QI activities. It is hoped that AMNet will continue to provide this service to interested providers, while expanding its reach, organizing the data it has collected, and serving as a resource to the field.

Conclusions

Notwithstanding the recruitment and technological challenges experienced, AMNet was able to recruit addiction medicine treatment providers and some practices were able to use PsychPRO’s standalone portal to implement MBC. We anticipate that AMNet will continue to recruit addiction medicine treatment providers, address barriers to data transfer, and incorporate data from providers’ portals into the registry to create a platform for future research.