Operationalizing and selecting outcome measures for the HEALing Communities Study

Abstract

Background

The Helping to End Addiction Long-term^SM (HEALing) Communities Study (HCS) is a multisite, parallel-group, cluster randomized wait-list controlled trial evaluating the impact of the Communities That HEAL intervention to reduce opioid overdose deaths and associated adverse outcomes. This paper presents the approach used to define and align administrative data across the four research sites to measure key study outcomes.

Methods

Priority was given to using administrative data and established data collection infrastructure to ensure reliable, timely, and sustainable measures and to harmonize study outcomes across the HCS sites.

Results

The research teams established multiple data use agreements and developed technical specifications for more than 80 study measures. The primary outcome, number of opioid overdose deaths, will be measured from death certificate data. Three secondary outcome measures will support hypothesis testing for specific evidence-based practices known to decrease opioid overdose deaths: (1) number of naloxone units distributed in HCS communities; (2) number of unique HCS residents receiving Food and Drug Administration-approved buprenorphine products for treatment of opioid use disorder; and (3) number of HCS residents with new incidents of high-risk opioid prescribing.

Conclusions

The HCS has already made an impact on existing data capacity in the four states. In addition to providing data needed to measure study outcomes, the HCS will provide methodology and tools to facilitate data-driven responses to the opioid epidemic, and establish a central repository for community-level longitudinal data to help researchers and public health practitioners study and understand different aspects of the Communities That HEAL framework.

1. Introduction

The Helping to End Addiction Long-term (HEALing) Communities Study (HCS) is a multisite, parallel-group, cluster randomized wait-list controlled trial evaluating the impact of the Communities That HEAL intervention to reduce opioid overdose deaths and other associated adverse outcomes (The HEALing Communities Study Consortium, 2020). The intervention includes three components:

• (1)a community-engaged and data-driven process to assist communities in selecting and implementing evidence-based practices to address opioid misuse and opioid use disorder (OUD), and reduce opioid overdose deaths (Sprague Martinez et al., 2020);
• (2)the Opioid Reduction Continuum of Care Approach which contains a compendium of evidence-based practices and strategies to expand opioid overdose education and naloxone distribution, medications for opioid use disorder (MOUD), and safe opioid prescribing (Winhusen et al., 2020); and
• (3)community-based health communication campaigns to increase awareness and demand for the evidence-based practices and reduce their stigma (Lefebvre et al., 2020).

A total of 67 communities across four highly affected states (Kentucky, Massachusetts, New York, Ohio) were recruited to participate in the HCS and randomized to one of two waves in a wait-list, controlled design. The communities were randomized to receive either the intervention (referred to as Wave 1 communities) or a waitlist control (referred to as Wave 2 communities). The HCS has one primary hypothesis (H1) and three secondary hypotheses (H2, H3, H4) (The HEALing Communities Study Consortium, 2020). It is hypothesized that during the evaluation period (January 1, 2021 to December 31, 2021), Wave 1 communities compared with Wave 2 communities, will:

H1

reduce opioid overdose deaths (primary outcome);

H2

increase naloxone distribution;

H3

expand utilization of MOUD; and

H4

reduce high-risk opioid prescribing.

Quality data are needed to measure the study outcomes and assess the impact of the integrated intervention and the specific evidence-based practices. Data are also an important component of the intervention because communities can use data on opioid overdose mortality and morbidity supplemented with data on community resources and needs to develop a data-driven action plan to expand the utilization of evidence-based practices. Communities also need timely and accurate data for visualization in data dashboards designed to monitor the uptake and success of the selected evidence-based practices and strategies, and respond to emerging challenges and community needs (Wu et al., 2020).

This article describes the process for using administrative data to develop the HCS outcome measures aligned with the primary and three secondary hypotheses of the study.

2. Methods

Each research site developed collaborations and partnerships with state agencies and other data owners to understand the regulations and policies governing the use of administrative data for research. An HCS Data Capture Work Group was formed and included representatives from the four research study sites, the data coordinating center at the RTI International, and the sponsors (the National Institute on Drug Abuse and the Substance Abuse and Mental Health Services Administration [SAMHSA]). A structured consensus decision-making strategy was used to:

• Aidentify data sources to measure the primary, secondary, and other study outcomes;
• Breview the literature on existing measures relevant to this study (e.g., measures developed by the Centers for Disease Control and Prevention [CDC], the Medicaid Outcomes Distributed Research Network, the National Committee on Quality Assurance, Centers for Medicare & Medicaid Services [CMS], Adult Care Quality Measures, the CMS 1115 SUD Waiver Evaluation Metrics, the Pharmacy Quality Alliance, or state systems, including state prescription drug monitoring programs);
• Cdevelop data governance strategy and data use agreements; and
• Ddevelop study measure definitions, technical specifications, programming code, procedures for data quality control, common data model, and schedule for data transfer to the data coordinating center.

During development, priority was given to use of existing state-level administrative data sources with regulated and sustained data collections and established infrastructures for quality assurance and control. This is an efficient and cost-effective way to study community-level changes, capitalizing on the federal and state investments for collecting standardized surveillance data, and adopting, when possible, validated surveillance definitions. In addition, using multiple administrative data sources allowed for the construction of measures at the community/population level (i.e., unit of analysis being HCS community) by aggregating individual-level data (e.g., unit of measurement being a community resident or a provider practicing within an HCS community) that best matched HCS outcomes.

Priority also was given to data sources with timely reporting, preferably with less than a 6-month lag between the occurrence of events and data availability. Timeliness and near-real-time access to data were critical for three reasons:

• (1)the community engagement component of the intervention is data-driven and dependent on providing ongoing data feedback to community partners throughout the process (The HEALing Communities Study Consortium, 2020);
• (2)it is imperative that the study results are made publicly available quickly because of the magnitude and impact of the opioid crisis on US communities; and
• (3)the HCS was designed as a four-year study.

This study protocol (Pro00038088) was approved by Advarra Inc., the HEALing Communities Study single Institutional Review Board. The study is registered with ClinicalTrials.gov [NCT04111939].

3. Results

This section presents the results from the selection and operationalization of administrative data measures for study hypotheses testing (Table 1 ), as well as study measures for secondary analyses and monitoring the progress in implementing evidence-based practices (Table 2 ).

Table 1.

HEALing Communities Study primary and secondary outcome measures for hypothesis testing.

Hypothesis	Outcome measure name, description, and data source
H1	Opioid overdose deaths
	Number of opioid overdose deaths among HCS residents during the evaluation period as measured by deaths with an underlying cause-of-death being drug overdose (i.e. an underlying cause-of-death ICD-10 code in the range X40-X44, X60-X64, X85, Y10-Y14) where opioids, alone or in combination with other drugs (i.e. a multiple cause-of-death ICD-10 code in the range T40.0-T40.4, or T40.6), were determined to be contributing to the drug overdose death.
	Data source: Drug overdose deaths are captured by death certificate records; additional medicolegal death investigation records can be used (per established protocol) to determine opioid involvement when specific drugs contributing to the overdose deaths are not listed on the death certificate.
H2	Naloxone distribution
	Number of naloxone units distributed in an HCS community during the evaluation period as measured by the sum of (1) the naloxone units distributed to community residents by overdose education and naloxone distribution programs with support from state and federal funding, including dedicated HCS funding, and (2) the naloxone units dispensed by retail pharmacies located within HCS communities.
	Data source: Data are captured from state administrative records and supplemented by study records to include naloxone funded through HCS, as well as IQVIA Xponent® database.
H3	Individuals on medication for opioid use disorder (MOUD)
	Number of HCS residents receiving buprenorphine products approved by the Food and Drug Administration for treatment of OUD as measured by the number of unique individuals residing in an HCS community who had at least one dispensed prescription for these products during the evaluation period.
	Data source: State prescription drug monitoring program data.
H4	Measure for incidence of high-risk opioid prescribing
	Number of HCS residents with new incidents of high-risk opioid prescribing during the evaluation period as measured by the number of residents in an HCS community who met at least one of the following four criteria for a new high-risk opioid prescribing episode after a washout period of at least 45 days: (1) incident opioid prescribing episode greater than 30 days duration (continuous opioid receipt with no more than a 7-day gap); (2) starting an incident opioid prescribing episode with extended-release or long-acting opioid formulation; (3) incident high-dose opioid prescribing, defined as ≥90 mg morphine equivalent dose over 3 calendar months; or (4) incident overlapping opioid and benzodiazepine prescriptions greater than 30 days over 3 calendar months.
	Data source: State prescription drug monitoring program data.

Table 2.

List of selected measures developed for the HEALing Communities Study.

Name	Description	Data Source	Submeasures
Drug Overdose Measures
Opioid overdose deaths (primary outcome measure)	Number of opioid overdose deaths among community residents.	Death Certificate Data	•Heroin •Synthetic opioids other than methadone •Cocaine •Non-cocaine psychostimulants •Benzodiazepine co-involvement
Drug overdose deaths	Number of drug overdose deaths among community residents.	Death Certificate Data	•Cocaine •Non-cocaine psychostimulants •Benzodiazepine
	Note: captures drug overdose deaths caused by any drug, including opioids.
Nonfatal opioid overdose events	Number of emergency department or inpatient hospital encounters for nonfatal opioid overdose among community residents.	State Emergency Department and Inpatient Discharge Data	•Heroin •Synthetic opioids other than methadone •Amphetamine •Cocaine •Any psychostimulant •Benzodiazepine co-involvement
Nonfatal drug overdose events	Number of emergency department or inpatient hospital encounters for nonfatal drug overdose among community residents.	State Emergency Department and Inpatient Discharge Data	•Amphetamine •Cocaine •Any psychostimulant •Benzodiazepine
	Note: captures drug overdose encounters caused by any drug, including opioids.
Emergency Medical Services (EMS) events that involve naloxone administration	Number of EMS events that involve naloxone administration in the community.	State EMS data	N/A
Emergency department (ED) visits for opioid overdose	Number of ED visits for opioid overdose among community residents.	Syndromic Surveillance	N/A
Overdose Education and Naloxone Distribution (OEND) Measures
Community naloxone distribution	Number of naloxone units (1 unit = 2 doses) distributed to community residents by state OEND programs or the HCS.	State OEND data and HCS logs	N/A
Pharmacy dispensed naloxone	Number of naloxone units (1 unit = 2 doses) dispensed by retail pharmacies in the community.	IQVIA XPONENT	N/A
Jail overdose education	Number of jails serving the community that provide overdose education.	De novo survey	N/A
Jail naloxone distribution	Number of jails serving the community that provide naloxone upon release.	De novo survey	N/A
Medication for Opioid Use Disorder (MOUD) Measures
Buprenorphine for treatment of opioid use disorder (OUD)	Number of community residents who received buprenorphine products approved by the Food and Drug Administration (FDA) for treatment of OUD.	Prescription Drug Monitoring Program	•Individuals retained on treatment ≥6 months
Any medications for opioid use disorder (MOUD) for OUD	Number of community residents receiving buprenorphine, methadone, or naltrexone as medications for treatment of OUD.	Medicaid claims	•Buprenorphine •Methadone •Naltrexone •Individuals retained on treatment ≥6 months
MOUD after opioid overdose	Number of community residents receiving MOUD after an emergency department or hospital inpatient encounter for opioid overdose.	Medicaid claims	N/A
MOUD after opioid-related emergency department (ED) visit	Number of community residents receiving MOUD after an opioid-related ED encounter.	Medicaid claims	•Proportion receiving MOUD a second time within 30 days of initial MOUD receipt
MOUD following release from prison	Number of community residents receiving MOUD within 28 days of release from prison.	State Department of Corrections data linked to Medicaid claims
Practitioners who have received waiver to prescribe or dispense buprenorphine under the Drug Addiction Treatment Act of 2000 (DATA 2000)	Number of community providers with a waiver to prescribe or dispense buprenorphine for OUD.	U.S. Drug Enforcement Administration (DEA) Controlled Substances Act (CSA) Active Registrants database	•30 patient limit •100 patient limit •275 patient limit
DATA 2000-waivered providers who actively prescribe buprenorphine for treatment of OUD	Proportion of community providers with a waiver to prescribe buprenorphine for OUD who prescribed buprenorphine for OUD.	DEA CSA Active Registrants data linked with Prescription Drug Monitoring Program data	•30 patient limit •100 patient limit •275 patient limit
Jails providing MOUD during incarceration	Number of jails serving community residents that provide MOUD during incarceration.	De novo survey	N/A
Jails starting MOUD prior to release	Number of jails serving community residents that initiate MOUD prior to release.	De novo survey	•Buprenorphine •Methadone •Naltrexone
Jails linking to MOUD prior to release	Number of jails linking community residents to MOUD in the community prior to release.	De novo survey	N/A
High-Risk Opioid Prescribing Measures
New high-risk opioid prescribing	Number of community residents receiving new (after 45-day washout period) “high-risk” opioid prescriptions.	Prescription Drug Monitoring Program	•New opioid episode lasting at least 31 days •Initiating opioid treatment with extended-release or long-acting opioid •Incident high dosage (average ≥90 mg morphine equivalent per day) •Incident overlapping opioid and benzodiazepine for at least 31 days
New opioid prescription with less than 7 day supply	Number and proportion of community residents with first prescription of new opioid episode with less than 7 day supply.	Prescription Drug Monitoring Program	N/A
Opioid prescriptions from multiple prescribers or pharmacies	Number of individuals receiving opioid prescriptions from four or more prescribers or four or more pharmacies in a quarter.	Prescription Drug Monitoring Program	N/A
Drug take-back drop boxes	Number of take-back drop boxes in communities.	DEA data	N/A
Other Measures
Prevalence of opioid use disorder (OUD)	Number of community residents with an OUD diagnosis.	Medicaid claims	N/A
Behavioral health treatment for OUD	Number of community residents with OUD receiving behavioral health treatment.	Medicaid claims	•Inpatient •Intensive outpatient •Outpatient •Case management •Peer support
Screening for hepatitis C among those with OUD	Number of community residents with OUD receiving hepatitis C testing.	Medicaid claims	N/A
Hepatitis C diagnoses among those with OUD	Number of community residents with OUD receiving hepatitis C diagnosis.	Medicaid claims	N/A
Hepatitis C treatment among those with OUD	Number of community residents with OUD receiving hepatitis C treatment.	Medicaid claims	N/A
New HIV diagnoses	Number of community residents with new diagnosis of HIV.	State registry	N/A
Number of individuals screened for substance use (alcohol or other drug use)	Number of community residents with a claim for screening of alcohol or other drug use.	Medicaid claims	N/A

3.1. Measure for opioid overdose deaths (H1)

Number of opioid overdose deaths among HCS residents as measured by deaths with an underlying cause-of-death being drug overdose where opioids, alone or in combination with other drugs, were determined to be contributing to the drug overdose death. Drug overdose deaths are captured by death certificate records; additional medicolegal death investigation records can be used (per established protocol) to determine opioid involvement when specific drugs contributing to the overdose death are not listed on the death certificate.

3.1.1. Scientific rationale for inclusion and methodological considerations

The primary HCS outcome is the number of opioid overdose deaths among residents in HCS communities. The traditional data source for capturing drug overdose deaths is death certificate records (Injury Surveillance Workgroup 7 (ISW7), 2012; Hedegaard et al., 2020; Warner et al., 2013). Suspected drug overdose deaths are considered unnatural deaths and are subject to medicolegal death investigation before the death is certified by a coroner or a medical examiner (Hanzlick, 2014; Hanzlick and Combs, 1998) and a completed death certificate is filed with the office of vital statistics in the state where the death occurred (National Center for Health Statistics (NCHS), 2003a,b). Selected fields from the death certificate record are then sent to the National Center for Health Statistics where the cause-of-death information is coded with one underlying and up to 20 multiple (i.e., supplementary) cause-of-death codes using the International Classification of Diseases, Tenth Revision (ICD10) coding system (World Health Organization (WHO), 2016). The CDC definition for identifying drug overdose deaths with opioid involvement in ICD-10-coded death certificate records is commonly accepted by researchers and public health agencies. Using ICD-10-coded death certificate data, drug overdose deaths are identified as deaths with an underlying ICD-10 cause-of-death code X40–X44 (unintentional), X60–X64 (suicide), X85 (homicide), or Y10–Y14 (undetermined intent) (Centers for Disease Control and Prevention (CDC), 2019a; Hedegaard et al., 2020). Among the selected drug overdose death records, opioid involvement is identified by one or more ICD-10 multiple cause-of-death code(s) for poisoning by opium (T40.0), heroin (T40.1), other natural and semisynthetic opioids (T40.2), methadone (T40.3), synthetic opioids other than methadone (T40.4), or other and unspecified narcotics (T40.6) (Centers for Disease Control and Prevention (CDC), 2019a; Scholl et al., 2018).

Previous research has identified several methodological challenges for identification of opioid involvement in drug overdose deaths (e.g., lack of routinely performed postmortem toxicology testing, especially for fentanyl and designer opioids; challenges to detection and quantification of new designer opioids; variation in jurisdictional office policy in completion of drug overdose death certificates and differences in the proportion of drug overdose death certificates completed by different jurisdictions that do not list the specific contributing drugs) (Buchanich et al., 2018; Ruhm, 2018; Slavova et al., 2015, 2019; Warner and Hedegaard, 2018; Warner et al., 2013). Prior to the evaluation period, the research sites are administering surveys among the coroners, medical examiners, and toxicology labs serving both Wave 1 and Wave 2 communities to collect information related to death investigations of suspected drug overdose deaths (including postmortem toxicology testing, timelines for death certificate completion, and possible COVID-19-related changes in these processes that could have lasting effects during the HCS evaluation period) in order to understand possible limitations and changes in the completeness and accuracy of the primary outcome measure.

3.1.2. Data sources for capturing opioid overdose deaths

Each HCS research site will use death certificate records from their state office of vital statistics to identify HCS resident deaths with opioid contribution. One challenge in using death certificate data for the primary study outcome is the lag between the death date and the date when death certificate records are available for analysis (Rossen et al., 2017). Sites have been working with local coroners, medical examiners, and state vital statistics offices to improve the timeliness of data availability across all HCS communities. In 2019, almost all the death certificate records in Kentucky, Massachusetts, New York, and Ohio were available for analysis within 6 months after the overdose death (Centers for Disease Control and Prevention (CDC), 2020).

3.1.3. Operational definition for capturing opioid overdose deaths

The following steps describe the HCS operational definition for capturing opioid overdose deaths for testing the primary study hypothesis:

• •Step 1: All sites will use state death certificate files captured 6 months after the end of the evaluation period to identify the death certificate records for residents of HCS communities with a date of death within the evaluation period, an underlying cause-of-death of drug overdose (ICD-10 code of X40-X44, X60-X64, X85, or Y10-Y14), and a multiple cause-of-death indicating opioid involvement (ICD-10 code in the range T40.0–T40.4 or T40.6);
• •Step 2 (site-specific): Because of the historically large proportion of Kentucky drug overdose death certificate records that did not list any involved drugs and the continuous improvements in the last few years (e.g., from >20 % in 2016, to <10 % in 2019) (Centers for Disease Control and Prevention (CDC), 2020), the Kentucky Chief Medical Examiner will review all drug overdose death certificates not listing any contributing drugs, from January 2018 until the end of the study period and will use additional medicolegal death investigation records to determine opioid involvement. The identified additional opioid overdose deaths will be added to the Kentucky cases identified based solely on the death certificate records in Step 1. This step is not required for other sites because they have a small and stable percentage of death certificates not listing any contributing drugs.

This process will ensure a quality harmonized measure that is captured consistently across the four research sites.

3.2. Measure for naloxone distribution (H2)

Number of naloxone units distributed in an HCS community as measured by the sum of the naloxone units (1) distributed in HCS community by overdose education and naloxone distribution programs with support from state and federal funding, including dedicated HCS funding, and (2) dispensed by retail pharmacies located within HCS communities. Data are captured from state administrative records and supplemented by HCS study records to include naloxone funded through HCS, as well as IQVIA Xponent® database.

3.2.1. Scientific rationale for inclusion and methodological considerations

The US Surgeon General’s advisory on naloxone emphasized that expanding naloxone availability in communities is a key public health response to the opioid crisis (U.S. Department of Health and Human Services (HHS), 2018). Research has shown that opioid overdose death rates were reduced both in communities that implemented overdose education and naloxone distribution programs (Walley et al., 2013) and in jurisdictions enacting laws allowing direct pharmacist dispensing of naloxone (Abouk et al., 2019).

3.2.2. Data sources for HCS naloxone distribution measure

Naloxone distribution sponsored by state and federal funding is tracked by offices within the state departments of health in Massachusetts, New York, and Ohio, and by the Kentucky Pharmacists Association in Kentucky. The four states have administrative databases allowing quantification of distributed naloxone units by the location of the distributing entity (NY, OH) or by residence of the naloxone recipient (KY, MA). Naloxone units purchased with HCS funding and distributed to HCS residents are also tracked and will be added to the counts provided through state administrative records. While the attribution of dispensed naloxone units to an HCS community varies by research site, the measure will be captured consistently within each research site over time and it is the only way to harmonize across research sites.

The data source for naloxone dispensed by pharmacies is the IQVIA prescription database Xponent®. IQVIA is a well-established data source with a high level of pharmacy participation, and it provides a consistent, common source across the sites. The IQVIA data provide counts of dispensed naloxone units from retail pharmacies, and units are assigned to communities based on the location of the pharmacy outlet. The IQVIA data capture 92 % of all prescription transactions nationally, and IQVIA utilizes a proprietary algorithm to account for the non-sampled portion when providing estimates. Because of confidentiality requirements, data are provided for a community only if it has four or more pharmacies within its geographic area.

There are three limitations of this data source: (1) no information is provided about the number of pharmacies dispensing naloxone prescriptions; (2) suppression rules preclude reporting of data for geographic areas with fewer than four pharmacies; and (3) prescriptions are assigned to communities based on the location of the pharmacy rather than the customer’s residence. Suppression rules impacted three communities in Massachusetts; this was resolved by requesting the total for the three communities and dividing it relative to the community populations. The assignment of a pharmacy to a community based on pharmacy address may result in an overcount of naloxone in a community with pharmacies that serve residents of non-HCS communities or an undercount if a pharmacy is just outside an HCS community but serves HCS residents.

A limitation of the measure is that it may not capture naloxone distributed in hospitals, correctional facilities, or other venues when the naloxone is purchased with support from private donations, foundations, or locally awarded federal funding.

3.3. Measure for individuals receiving MOUD (H3)

Number of HCS residents receiving buprenorphine products approved by the Food and Drug Administration (FDA) for treatment of OUD as measured by the number of unique individuals residing in an HCS community who had at least one dispensed prescription for these products during the measurement period, captured from the state prescription drug monitoring program (PDMP) data.

Given timeliness of the PDMP data in the four states (provided by state agencies to the research sites monthly for controlled substance prescriptions dispensed in the previous month) and the fact that buprenorphine is a scalable part of the intervention across research sites, PDMP data were selected to develop a measure for utilization of buprenorphine products approved by the FDA for treatment of OUD as the measure for testing the secondary hypothesis on expanding the utilization of MOUD. This measure captures dispensed prescriptions with national drug codes for buprenorphine products FDA-approved for treatment of OUD. The list of drug codes is updated quarterly by the HCS team using the MEDI-SPAN ELECTRONIC DRUG FILE (MED-File) V2 for active and inactive products (Wolters Kluwer, 2020). Buprenorphine products directly purchased for administration in practitioner offices (i.e., that are not first dispensed by pharmacies) are not captured in PDMP data. Transdermal, parenteral, and buccal formulations of buprenorphine approved for treatment of pain were excluded. The measure counts unique individuals who reside in an HCS community, with at least one dispensed prescription for buprenorphine during the measurement period.

3.3.1. Scientific rationale for inclusion and methodological considerations

There are three FDA-approved MOUD products: buprenorphine, methadone, and naltrexone (SAMHSA, 2018). Multiple randomized controlled trials (Krupitsky et al., 2011; Mattick et al., 2009, 2014) have demonstrated that MOUD can reduce cravings and illicit opioid use. Observational studies have identified that buprenorphine and methadone are associated with reduced mortality (Larochelle et al., 2018; Sordo et al., 2017). Thus, as part of the Opioid Reduction Continuum of Care Approach, communities are required to expand MOUD with buprenorphine and/or methadone (Winhusen et al., 2020). Access to MOUD is geographically heterogeneous and differs by patient population (Haffajee et al., 2019; Pashmineh Azar et al., 2020). For example, Opioid Treatment Programs providing methadone are less common in rural than urban areas (Joudrey et al., 2019). Criminal justice-involved populations, where there has been a historical preference toward naltrexone (Krawczyk et al., 2017), are less likely to receive buprenorphine and methadone. There also is a great deal of variation in billing and documentation of the type of MOUD, administration modality (e.g., office-based administration as compared with prescriptions filled at pharmacy by patient), provider type, state policies, and insurance coverage.

Accurate estimation of the prevalence of OUD in HCS communities is important for planning and scaling of the MOUD uptake. However, estimating the population at need for MOUD is a challenge for the HCS. The HCS team is working on developing improved estimations for OUD prevalence in each HCS community using a capture-recapture statistical methodology previously applied by Barocas et al. (2018).

3.3.2. Data sources for an HCS measure

Five potential sources for measurement of MOUD were identified: Medicaid claims, all-payer claims databases, PDMPs, Opioid Treatment Program Central Registries, and pharmacy dispensed prescriptions (IQVIA). The disparate data sources vary in completeness and timeliness.

3.3.2.1. Medicaid claims data

Medicaid claims are common across all four states and include comprehensive health service utilization data for individuals enrolled in Medicaid. Because it is collected primarily for reimbursement purposes, there is a great deal of motivation for providers to include accurate information about services rendered. The key components required for identifying individuals with an OUD diagnosis and their engagement in treatment (person-level identifier, date of service, type of service rendered) use mostly common coding, terminology, and definitions. Medicaid enrollment data contain residence and demographic information.

A significant body of work exists around developing consistent, cross-state, opioid-related measures with Medicaid data done by the Medicaid Outcomes Distributed Research Network (MODRN) (AcademyHealth, 2020; Adams et al., 2019). Much of this work leverages state partnerships to strike a balance between consistency and comparability between states, and accurate adjustments to the handling of variables required to identify MOUD treatment in accordance with evolving state policy. For this reason, we primarily relied on MODRN measures to develop MOUD measures for Medicaid data.

Medicaid data has some limitations: 1) it can require more than 6 months to be sufficiently complete for analysis; 2) it only captures information for individuals enrolled in Medicaid and receiving MOUD covered by Medicaid; 3) MOUD coverage varies from state to state; and 4) billing processes differ, requiring consolidation of customized code lists for each research site.

3.3.2.2. All-payer claims databases

All-payer claims databases are large state databases that typically include medical claims across multiple settings (e.g., hospitalizations, emergency departments visits, outpatient visits), pharmacy claims, and eligibility and provider files. Data are collected from both public and private payers and reported directly by insurers to a state repository. All-payer claims databases are structured similarly to Medicaid claims data and allow for linking of individuals across claims to identify individuals with OUD and their treatments. The key advantage is the inclusion of private insurance, allowing more accurate estimation of prevalence of individuals with diagnosed OUD and treatment with MOUD in a state.

All-payer claims have been used previously in OUD-related research (Burke et al., 2020; Freedman et al., 2016; LeBaron et al., 2019; Saloner et al., 2017). Seventeen states have all-payer claims system that mandate payer submission of claims and 11 states have data systems that are voluntary (All-Payer Claims Database Council, 2009). Of the HCS states, New York and Massachusetts have mandated all-payer claims databases; Kentucky and Ohio do not, which prevented their use for this study.

3.3.2.3. PDMPs

State PDMPs are electronic databases that track prescriptions of controlled substances and are regulated by state laws (PDMP TTAC, 2018; PDMP, 2020). All HCS sites have state PDMPs collecting data on all dispensed controlled substances within 24 h or one business day of dispensing, with some exceptions (Commonwealth of Kentucky, 2018; Mass.gov, 2020; New York Department of Health, 2019; Ohio Automated RX Reporting System, 2017). Compared with claims-based data, PDMP data are timelier and include cash-paid prescriptions.

PDMP records capture buprenorphine products that are FDA-approved for the treatment of OUD (as opposed to pain), ensuring well-defined case selection criteria. PDMP data cannot identify if buprenorphine is being prescribed off-label for pain. Buprenorphine administered in office settings (e.g., Sublocade) would not be reported to PDMP. Methadone for treatment of OUD is dispensed by Opioid Treatment Programs and not reported to state PDMPs per a federal confidentiality law that protects addiction treatment records (42 CFR Part 2) (e-CFR, 2020b). Similarly, Opioid Treatment Programs do not report data on buprenorphine for OUD to PDMPs. Naltrexone is not a controlled substance and is not captured in PDMP data. PDMPs are state-specific and do not capture prescriptions filled in another state. States that participate in data-sharing networks enable clinicians and pharmacists to search patient’s prescription history in other states to make clinical decisions. However, there is no routine sharing of PDMP datasets between neighboring states. This could lead to underestimation of buprenorphine uptake in HCS communities close to state borders, where residents might seek care and utilize health resources in neighboring states (Grecu et al., 2018).

3.3.2.4. Opioid Treatment Program Central Registries

Opioid Treatment Programs are the only facilities allowed to deliver methadone for OUD but may also offer buprenorphine and naltrexone along with behavioral therapy. They must be certified by SAMHSA and an independent, SAMHSA-approved accrediting body to dispense MOUD (SAMHSA, 2020). They also must be licensed by the state in which they operate and must be registered with the Drug Enforcement Administration. The registries are established to prevent patient’s simultaneous enrollment in multiple locations (e-CFR, 2020a). While the number of enrolled patients, aggregated at the HCS community level, as permitted by section §2.52 Research, 42 CFR Part 2 (e-CFR, 2020a) can be used as a measure for methadone treatment uptake, central registries were not available in all four research sites.

3.3.2.5. Pharmacy dispensed prescriptions data

IQVIA data capture pharmacy dispensed naltrexone. However, naltrexone is indicated for treatment of OUD and for alcohol use disorder. Because pharmacy records do not include diagnose-related information for making this distinction, this data source may overestimate the uptake of naltrexone for OUD.

3.4. Measure for incidence of high-risk opioid prescribing (H4)

Number of HCS residents with new incidents of high-risk opioid prescribing as measured by the number of residents in an HCS community who met at least one of the following four criteria for a new high-risk opioid prescribing episode after a washout period of at least 45 days: (1) incident opioid prescribing episode greater than 30 days duration (continuous opioid receipt with no more than a 7-day gap); (2) starting an incident opioid prescribing episode with extended-release or long-acting opioid formulation; (3) incident high-dose opioid prescribing, defined as ≥90 mg MME over 3 calendar months; or (4) incident overlapping opioid and benzodiazepine prescriptions greater than 30 days over 3 calendar months.

3.4.1. Scientific rationale for inclusion and methodological considerations

High opioid dosages, co-prescribing opioids with benzodiazepines or other sedative hypnotics, and receipt of opioid prescriptions from multiple providers or pharmacies are associated with opioid-related harms (Bohnert et al., 2011; Cochran et al., 2017; Dunn et al., 2010; Rose et al., 2018). Characteristics of opioid initiation are also important. For example, initiating opioid treatment with extended-release/long-acting opioids (Miller et al., 2015) is associated with increased risk of overdose, and longer prescription duration is associated with transition to long-term opioid use (Shah et al., 2017).

Based on available evidence, the CDC published guidelines (Dowell et al., 2016) for prescribing opioids for chronic pain. Numerous quality measures have been developed to encourage and measure progress toward improving the safety of opioid prescribing. After decades of increases, rates of opioid prescribing peaked and are now declining, although they remain historically high (Guy et al., 2017; Schieber et al., 2020).

Developing safe and patient-centered approaches for individuals receiving long-term opioid therapy has been a challenge to address in underlying evidence or guidelines. Increasing reports of potential harms to individuals undergoing opioid tapers have led the authors of the CDC guidelines, the FDA, and the U.S. Department of Health and Human Services to issue warnings and guidance against rapid involuntary opioid tapers (Dowell et al., 2019; U.S. Department of Health and Human Services (HHS), 2019; U.S. Food and Drug Administration (FDA), 2019).

Two constructs with the best available evidence to support decreases in opioid-related harms were targeted with the intention of reducing the number of individuals initiating high-risk opioid prescribing and the likelihood that new opioid prescribing episodes develop into long-term episodes (Shah et al., 2017).

3.4.2. Data sources for the HCS measure

State PDMPs were identified as the best available data source for these measures across all four research sites. A limitation of these data is the lack of clinical context (e.g., diagnostic codes for disease or condition) associated with the prescribed medication. As a result, at the patient level, it is difficult to assess the appropriateness of a high-dose opioid prescribing episode, such as that needed for management of severe pain for patients with cancer or end-of-life care. Another limitation of this measure is the lack of automated data sharing among state PDMPs on prescriptions filled across state boundaries. A benefit of the PDMP data source is that it is timely and captures dispensed prescriptions for controlled substances paid for by both insurance and cash.

Medicaid claims and all-payer claims databases are alternative data sources. The main advantage of claims data compared with PDMP data is the clinical context. However, claims data lack information on prescriptions paid by cash or alternative insurance coverage, which is associated with increased risk of opioid-related harms (Becker et al., 2017). Medicaid claims are common across the sites, whereas all-payer claims databases exist in only two of the four states. Claims data lag by at least 6 months, making them less useful for timely monitoring of progress.

3.4.3. Operational definition for capturing incident high-risk opioid prescribing

Existing measures were identified through a review of the literature, including existing measures from CDC, National Quality Forum, and National Committee for Quality Alliance, which were subsequently adapted to the constructs identified above. All opioid agonist medications, including tramadol, were included, with the exception of antitussive codeine formulations and buprenorphine formulated for pain. The reasons for their exclusion are a lack of clear guidance for conversion to morphine milligram equivalents and a lack of evidence that buprenorphine, a partial opioid agonist, conveys the same risk as full opioid agonists.

To maintain consistency across sites, the team developed a standardized list of national drug codes for opioids, benzodiazepines, and MOUD using the MEDI-SPAN ELECTRONIC DRUG FILE (MED-File) V2 and the Drug Inactive Data File (Wolters Kluwer, 2020). The standardized study drug list is updated quarterly. The MED-File includes product names, dosage forms, strength, the NDC, and generic product identifier (GPI). The GPI is a 14-digit number that allows identification of drug products by primary and secondary classifications and simplifies identification of similar drug products from different manufacturers or different packaging. Because our study requires baseline data on opioid utilization, the inactive date file is used to include drugs that may be currently inactive but were used during the baseline period.

All GPIs beginning with the classification “65”—which identifies any drug product containing an opioid or combination—are included in the opioid list. Next, opioid products that are not likely to be used in the outpatient/ambulatory pharmacy setting—such as bulk powder, bulk chemicals, and dosage forms typically used in hospitals or hospice settings (e.g., epidurals, IVs)—are excluded. Products classified as cough/cold/allergy combinations, cough medications, antidiarrheal/probiotic agents, buprenorphine products used for OUD and pain, and methadone products used for OUD were also excluded. The CDC file that identifies oral MMEs (Centers for Disease Control and Prevention (CDC), 2019b) was used to add MMEs to each opioid product and to identify products as long-acting or short-acting. To ensure the HCS list includes all current and inactive products, the CDC list was cross-referenced with the list of all GPI products.

The benzodiazepine products are identified using the GPI classification “57”, which identifies any drug product containing a benzodiazepine or combination. Products that are not likely to be used in the outpatient/ambulatory pharmacy setting—such as bulk powder, bulk chemicals, and dosage forms typically used in hospitals or hospice settings—were excluded.

3.5. Other study measures

The success of the intervention relies on the community’s ability to assess the complexity and specifics of the local opioid epidemic and identify the best ways to implement and promote evidence-based practices locally. A set of additional measures was developed, to be shared with the intervention communities as counts and/or rates over time and visualized as trends on community-tailored dashboards (Wu et al., 2020). These measures monitor the complexity of the opioid-related harms as well as the progress in the three main evidence-based practices from the Opioid Reduction Continuum of Care Approach (Winhusen et al., 2020). A list of selected study measures is provided in Table 2 and includes for example, opioid overdose-related emergency department visits and hospitalizations, Medicaid beneficiaries with OUD receiving MOUD, retention in MOUD, and providers of buprenorphine treatment for OUD. Communities use these data to estimate local capacity to support the selection and implementation of evidence-based practices and to track study progress at the local level. Many of the measures are based on existing consensus measures developed by other groups and organizations. For example, the measures on opioid overdose events captured by emergency department or inpatient hospital discharge data, or syndromic surveillance data are using definitions developed by the Council of State and Territorial Epidemiologists in collaboration with the CDC (CSTE, 2019). The definitions for drug overdose deaths with involvement of different drugs/drug classes are using the methodology utilized by the CDC (CDC, 2019a). For other measures (e.g., number of emergency medical services events involving naloxone administration, Drug Addiction Treatment Act of 2000 (DATA 2000)-waivered providers who actively prescribe buprenorphine for treatment of OUD), the HCS Data Capture Work Group could not identify definitions that could be adopted or modified and the workgroup developed study definitions and specification with input from subject matter experts and stakeholders.

Working closely with state stakeholders, the research sites also developed standard operating procedures for data quality assurance and control, and improved data collection (e.g., improved timelines of an existing data sources or development of new administrative data collections).

3.6. Common data model

The HCS data coordinating center created a common data model to match the complexity and scale of the clinical trial design and measures and the conditions of the data use agreements. The common data model consisted of

• (1)an internal identification number for each HCS measure outcome;
• (2)frequency of reporting (i.e., daily, monthly, quarterly, semi-annually, or annually);
• (3)display features for dashboards and visualization (i.e., display date, display value, research cite/research community identification number, label); and
• (4)internal usage information (i.e., is estimate, is suppressed [per data use agreement suppression requirements], notes, stratification, and version number).

The common data model allows coordinated presentation of data to communities to aid with decision making and monitoring of progress and allows the HCS consortium and trial sponsors to routinely monitor progress.

4. Discussion

During the first year of the HCS, the Data Capture Work Group evaluated more than 15 administrative data sources across the four states for their ability to support study measures in multiple relevant domains. The research site teams established multiple data use agreements with data owners to support the calculation for more than 80 study measures based on administrative data collections, such as death certificates, emergency medical services data, inpatient and emergency department discharge billing records, Medicaid claims, syndromic surveillance data, PDMP data, Drug Enforcement Administration data on drug take back collection sites and events, DATA 2000 waivered prescriber data, HIV registry, naloxone distribution, and dispensed prescription data.

There were many challenges related to state variations in data timeliness and content that needed to be addressed, and compromises were made to achieve harmonization across research sites. The harmonization on Medicaid measure specifications required participation from the state partners because individual states have some unique codes or code bundles for capturing specific services. Collaborative workgroups with participation from state partners were formed with specific focus on Medicaid data, PDMP data, and emergency medical services data.

Another challenge is the lack of quality validation studies for many of the measures, so the degree of possible misclassification of diagnosis or service codes used in some specifications is unknown. One example is attempting to identify OUD prevalence using diagnosis codes in medical claims or other administrative data sources knowing that OUD is often underdiagnosed. Furthermore, the ICD-10 diagnosis codes of opioid abuse and opioid dependence do not map onto DSM-5 (Diagnostic and Statistical Manual of Mental Disorders) criteria for OUD and may capture individuals who have physiologic dependence on prescribed opioid analgesics but not an actual OUD. The sensitivity, specificity, and positive predictive value of these codes to identify OUD are unknown. While the impact of measurement bias may be minimized as it should be nondifferential across Wave 1 and Wave 2 communities, this study provides the opportunity to highlight, study, and improve these measurement issues.

Starting in December 2019, the research sites began to provide monthly study measure updates to intervention communities and the data coordinating center. The HCS Steering Committee, the research site teams, and the data and safety monitoring board evaluate some measures as part of a safety monitoring protocol to identify changes in trends that require rapid response. For example, in April–May 2020, the HCS research site teams identified statistically significant increases in opioid overdose-related events captured with emergency medical services data and emergency department visit data, which triggered “fast-track” distribution of naloxone in the intervention communities and other appropriate local responses.

The HCS has already made an impact on existing data capacity in the four states. For example, in response to the need of the Kentucky HCS team to capture methadone treatment and retention data, in January 2020, the Kentucky Cabinet for Health and Family Services established a statewide Opioid Treatment Program Central Registry supporting near real-time methadone enrollment reporting. Massachusetts is seeking to establish a similar central registry to improve timeliness and accuracy in identifying individuals enrolled in methadone treatment. Massachusetts also is seeking to partner with emergency medical services agencies to improve timeliness of data reporting and completeness of race/ethnicity data. New York developed a cloud-based application to facilitate data aggregation and sharing both for HCS and future research projects. In Ohio, the HCS team partnered with the InnovateOhio Platform, which was established by executive order a few weeks prior to the HCS project start date. The HCS has been a highly successful “test case” for how a single technology platform could be leveraged to provide necessary data quickly and efficiently for a large study involving multiple state agencies. The platform facilitated a multi-agency data use agreements, and curates, cleans, and links data sets across multiple Ohio state agencies monthly. This allowed the Ohio HCS team to sign one data use agreement to cover all project data activities.

The HCS will provide methodology and tools to facilitate data-driven responses to the opioid epidemic at the local, state, and national levels. The study measures rely on data sources that are available in most states, allowing the use of the measure specifications and programming code in other states, outside of the four HCS states . Moreover, the HCS established a central repository for the community-level longitudinal data that can help researchers and public health practitioners to study and understand different factors of the Communities That HEAL framework. Moving forward, the HCS research teams are developing informatics infrastructure and sustainability plans to expand some of the analytical work, such as community dashboards, to benefit all communities/counties in their states after completion of the study.

Trial registration

ClinicalTrials.gov NCT04111939.

Contributors

All authors contributed to the development of the HCS measures, the development of the framework for the manuscript, and the editing of the manuscript. S. Slavova, J. Villani, and S.L. Walsh drafted the introduction, S. Slavova drafted the methods, M.R. LaRochelle developed the tables, and each author participated in drafting parts of the results or discussion sections.

Role of the funding source

This research was supported by the National Institutes of Healththrough the NIH HEAL Initiative under award numbers UM1DA049394, UM1DA049406, UM1DA049412, UM1DA049415, UM1DA049417. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or its NIH HEAL Initiative.

Declaration of Competing Interest

The authors report no declarations of interest.