Abstract
Background and Aim
There is no gold‐standard and considerable heterogeneity in outcome measures used to evaluate treatments for opioid use disorder (OUD) along the opioid treatment cascade. The aim of this study was to develop the US National Institute on Drug Abuse (NIDA) National Drug Abuse Treatment Clinical Trials Network (CTN) opioid use disorder core outcomes set (OUD–COS).
Design
Four‐round, e‐Delphi expert panel consensus study and plenary research group discussion and targeted consultation.
Setting
United States.
Participants
A panel of 25 members including clinical practitioners, clinical researchers and administrative staff from the CTN, the network's affiliated clinical and community sites and the NIDA Centre for the CTN.
Measurements
From a pool of 24 candidate items in four domains (biomedical/disease status; behaviors, symptoms and functioning; opioid treatment cascade; and morbidity and mortality), the panel completed an on‐line questionnaire to rank items with defined specification on a 9‐point scale for importance, with a standard 70% consensus criterion.
Findings
After the fourth round of the questionnaire and subsequent discussion, consensus was reached for five outcomes: two patient‐reported (global impression of improvement and incident non‐fatal overdose); one clinician‐reported (illicit/non‐medical drug toxicology); and two from administrative records (duration of treatment and fatal opioid poisoning).
Conclusions
An e‐Delphi consensus study has produced the US National Institute on Drug Abuse (NIDA) National Drug Abuse Treatment Clinical Trials Network opioid use disorder core outcomes set (version 1) for opioid use disorder treatment efficacy and effectiveness research.
Keywords: Core outcomes set, Delphi consensus, efficacy and effectiveness research, opioid treatment cascade, opioid use disorder, US National Institute on Drug Abuse
INTRODUCTION
Opioid use disorder (OUD) is a debilitating condition that is associated with a substantial global burden of disease [1, 2]. In many countries, a two‐decade increase in opioid poisoning fatalities has focused attention upon the need for more effective treatment interventions [3, 4]. In most health‐care systems around the world, opioid agonist, partial agonist and antagonists are the front‐line medications for OUD (MOUD). These are offered to patients along sequential stages of treatment initiation, retention and re‐admission in what has been termed the OUD treatment cascade [5]. These MOUD are combined with psychosocial interventions (PSI), including general counseling and motivational and cognitive behavioral therapies [6, 7].
Over many years, randomized controlled trials and observational cohort studies have secured an international research evidence base for the efficacy of MOUD interventions. A very broad array of outcome measures has been used in the scientific literature. Outcomes can be grouped as patient‐reported (PRO; for example, subjective evaluations of treatment response and quality of life) [8, 9, 10]; clinician‐reported (ClinRO; for example, structured interviews to record drug use cognitions and behaviors); observer‐reported (ObsRO; for example, clinical evaluation of signs OUD withdrawal) [9]; and performance outcomes (PerfO; for example, administrative records to estimate rates of treatment retention and re‐admission) [8, 11]. According to the specific aims of each study, one of these measures is usually selected as the primary outcome. It is the norm for an OUD study to also include several secondary outcomes.
There are many questions asked by studies along the OUD treatment cascade, so it is to be expected that there is no gold‐standard single outcome measure for OUD research. Different research groups may select the same measure to estimate the efficacy of an intervention at different time‐points; the same intervention can also be evaluated by different studies at the same point but with a different outcome. However, the field has reached a point where findings from different outcome measures make it hard to interpret a study's findings and combine data for subsequent meta‐analysis. Uncertainty about effectiveness makes research less relevant and limits the ability of patients, health‐care professionals and policymakers to make informed decisions. Resolving this problem would enable OUD studies to increase knowledge and have a greater impact.
Established in 1999 by the US National Institute on Drug Abuse (NIDA), the National Drug Abuse Treatment Clinical Trials Network (CTN) is a national infrastructure of 16 nodes that brings together treatment providers and research institutions to collaborate on intervention‐related research on OUD and other substance use disorders. The CTN has had extensive national reach, with approximately 100 studies completed. These have generated more than 650 scientific publications on effectiveness and efficacy as well as in prevention science, knowledge dissemination and work‐force training [12].
In an effort to promote consensus in the use of outcome measures for OUD research, the CTN published a Common Data Elements compendium [13, 14, 15, 16, 17] and advanced procedures to link electronic case report forms and health‐care records [18]. However, consensus on which measures should be used for OUD research proved elusive. There were probably many reasons for this but, arguably, administrative burden has been a key factor. For example, in 2012 a group of 32 instruments/assessments were recommended which would require clinicians and patients to spend approximately 5 hours completing them [13]. Given this burden, it is likely that many researchers looked elsewhere.
A ‘core outcome set’ (COS) is an alternative method of securing consensus. A COS is the minimum set of outcomes that are judged to be most important for a particular health condition and should be included in all clinical efficacy trials and studies of routine care [19]. COS are increasingly popular in many areas of clinical science and health‐care research. The Core Outcomes Measures in Effectiveness Trials (COMET) initiative provides resources for research and manages a database of pre‐registered development studies (http://www.comet-initiative.org). Guidelines for the development of randomized controlled trials protocols [Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT)] and their reporting [Consolidated Standards of Reporting Trials (CONSORT)] recognize the importance of using a COS [20, 21, 22, 23, 24]. To align with CONSORT, a study might select one measure from a COS to be the primary outcome, with the remaining COS measures specified as some or all of the secondary outcomes.
Typically, COS development studies use a Delphi methodology to determine if consensus can be reached. Developed by the RAND corporation in the 1950–60s, Delphi is a stepwise method to elicit and refine the opinions of a panel of people with knowledge and expertise who seek consensus [25, 26, 27]. Determining an optimal panel size in Delphi studies is a pragmatic rather than statistical question; a panel too small may not reach consensus, whereas a panel too large may be unwieldy. Delphi studies have the following principles: anonymity of panel members’ responses to mitigate against group pressure and the biasing influence of dominant viewpoints; statistical aggregation of responses and controlled feedback; and iteration and freedom to revise opinion. Typically, after several rounds of data collection, consensus is judged to have been reached by a pre‐defined threshold for percentage agreement; 70% has been the standard [28]. A plenary discussion to review the findings and addresses practical matters marks the end of the Delphi study process.
There has been no COS developed by the CTN or elsewhere for OUD research. Accordingly, the aim of the present study was to determine the first OUD–COS. To minimize administrative burden, the objective was to identify single items rather than a multi‐item measurement scale.
While the study was initiated in the NIDA CTN context, we anticipated that its findings would be of value for OUD research conducted elsewhere.
METHODS
This was an on‐line (e‐Delphi) consensus study. It was pre‐registered with COMET (http://www.comet-initiative.org/Studies/Details/1579). Initially, the study protocol was screened by the Institutional Review Board (IRB) at the Rush University Medical Center. The IRB judged it exempt from formal review because panel members did not meet the definition of research participants. The protocol was then reviewed by the CTN's Research Development Committee and published before commencement of the consensus process [29]. This report has been prepared to align with guidelines for reporting COS research [30].
Study research group
A study research group was established in January 2020 and coordinated by Rush University Medical Center (CTN Great Lakes Node). A research group (the authors of this article) took forward items for the final COS and agreed definitions and other specifications for end‐use. A subset of this group (U.G., N.S.K., J.M., L.M.) served as the steering committee.
Panel participants
On 24 July 2020, clinicians, researchers and staff from all CTN nodes and their collaborating clinical partners—approximately 200 people listed in the CTN directory at that time—were invited by open call to participate as members of the panel. The role of the Delphi panel was to provide individual and anonymous ranking of the items to determine evidence of consensus. A panel with no more than 40 members was sought to support a range of perspectives and allow for attrition.
The final membership of the panel was determined on 6 August 2020, with all members providing their informed consent to take part and share their opinions. We used an open approach to questionnaire completion and allowed members to miss rounds if needed. This was partially necessitated by the disruptions caused by the COVID‐19 pandemic, which impacted upon work schedules and the availability of the panel to devote time to the study.
Candidate domains and items
Through a comprehensive review of CTN studies that had OUD‐related measures, search of registered OUD trials (www.clinicaltrials.gov) and our knowledge of the research literature, we first created a list of 24 items in three domains, as follows:
Biomedical/disease status: DSM‐5 OUD severity; DSM‐5 OUD remission; clinician‐perceived OUD improvement/deterioration; disease complications.
Behaviors, symptoms and functioning: days used illicit/non‐medical opioids; opioid‐using occasions per day; days to first use of opioids; craving for opioids; perception of OUD problems; patient‐perceived OUD improvement/deterioration; quality of life; emotionally salient life events.
MOUD treatment cascade: illicit/non‐medical drug toxicology; toxicology and medication adherence monitoring; stage of OUD treatment cascade; proportion of days covered in first year after initiation; clinician‐reported patient engagement; total number of days enrolled in MOUD.
Morbidity and mortality: serious adverse events requiring hospitalization; hospitalizations for injury or infection; non‐fatal opioid‐related poisoning/overdose; emergency reversals of overdose with naloxone; opioid poisoning/overdose mortality.
Ratings and consensus threshold
A modified, 9‐point, Grading of Recommendations Assessment, Development and Evaluations rating scale (https://www.gradeworkinggroup.org) was used to rank each candidate item to indicate that it was: of limited importance, and should not be included in the COS (rank: 1–3); important but not critical, and should not be included in the COS (rank: 4–6); or critical and should be included in the COS (rank: 7–9).
A consensus threshold was set, so that if at least 70% of the panel scored an outcome between 7 and 9, and fewer than 15% of the panel scored it 1–3, the outcome was taken forward towards inclusion in the COS. Items that did not attain consensus were removed.
Procedure
A secure, on‐line questionnaire was constructed in REDCap (version 6.18.1). Each questionnaire was designed to take no more than 30 minutes to complete. Panel members were encouraged to provide comments and suggestions on the included items during each round. De‐identified results comprising overall scores for each item (analyzed as a percentage), together with a narrative summary of findings, comments and suggestions, were sent to each panel member after the first round. In the second round, ambiguous items or proposals collected from the first round were compiled (with input from the steering committee as required) before being included into the new round. Rounds 1 and 2 focused upon broad conceptual domains. Rounds 3 and 4 ranked options for measurement. Domains in rounds 1 and 2 could be dropped if they failed to secure consensus—but no domain was included in the final COS without additional consensus in either rounds 3 or 4 when the specific item was defined.
The research group reviewed the final set of items and discussed these at an on‐line consensus meeting held on 13 April 2021 and via subsequent e‐mails with various research group members. Feedback was incorporated into the final operational description for each item. After completion of the Delphi process, the research group deliberated on the best wording for the items selected and drew upon the available literature to guide operationalization of the items.
Finally, the research group recommended that the proposed COS items should be distributed to the various chairs and co‐chairs of the CTN special interest groups (SIGs) for review and feedback. These SIGs include American Indian/Alaska Native, comorbidities, gender, minority, practice‐based research networks, rural, telemedicine translation and implementation and youth.
Analysis
Rankings by item and round were tabulated to indicate if the consensus threshold was attained.
RESULTS
Panel members
Twenty‐five members of the CTN served as members of the e‐Delphi panel. Table 1 shows the panel's demographic and professional characteristics. Several individuals who identified their primary role in the CTN as researchers are clinician‐researchers. There was some attrition in the panel numbers during the course of the e‐Delphi rounds with a range of 25–22 participants. Due to concerns about COVID‐19‐related fatigue for panel members participating in successive rounds of surveys, we judged that there was a need to limit the process and the research group was empowered to conduct further deliberations after the fourth round of the e‐Delphi questionnaire.
TABLE 1.
Demographics of the e‐Delphi panel (n = 25) at round 1
| Characteristic | n | % |
|---|---|---|
| Gender | ||
| Female | 17 | 68.0 |
| Male | 8 | 32.0 |
| Race/ethnicity | ||
| African American | 0 | 0 |
| Asian | 7 | 28.0 |
| Caucasian | 22 | 88.0 |
| Latinx | 1 | 4.0 |
| Highest qualification | ||
| Physician | 10 | 40.0 |
| PhD/ScD | 14 | 56.0 |
| PsyD | 0 | 0 |
| DNP | 0 | 0 |
| PharmD | 0 | 0 |
| Masters | 2 | 8.0 |
| Bachelors | 0 | 0 |
| Primary CTN role | ||
| Clinician | 2 | 8.0 |
| Researcher and clinician‐researcher | 21 | 84.0 |
| Staff | 1 | 4.0 |
| Administration | 1 | 4.0 |
Note: Participants could select multiple race/ethnicity or highest qualification categories and thus cells do not necessarily total to sample size. CTN = clinical trials network; DNP = Doctor of Nursing Practice.
Ranking of measures
Table 2 shows the domains and measures considered by the panel. In rounds 1 and 2, the panel considered broader domains of measurement for OUD. After narrowing these down, rounds 3 and 4 focused upon specific items within the selected domains. At the end of round 3, two items attained consensus—total number of days enrolled in MOUD (count) (consensus 75%); and opioid‐related overdose mortality (yes/no) (consensus 79%). In this same round, five items failed to reach consensus, and these were taken forward into a final round 4 for further evaluation. At the end of this round, an additional item achieved the threshold for inclusion—opioid‐related non‐fatal overdose (number of events) (consensus 73%).
TABLE 2.
Rating of OUD–COS outcome concepts and specifications by domain and round
| Outcome concept | Outcome specification | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Round 1 | Round 2 | Round 3 | Round 4 | |||||||||||||
| Domain | n | Rank 1–3 (%) | Rank 4–6 (%) | Rank 7–9 (%) | n | Rank 1–3 (%) | Rank 4–6 (%) | Rank 7–9 (%) | n | Rank 1–3 (%) | Rank 4–6 (%) | Rank 7–9 (%) | n | Rank 1–3 (%) | Rank 4–6 (%) | Rank 7–9 (%) |
| A. Biomedical/disease status | ||||||||||||||||
| DSM‐5 OUD severity | 25 | 4 | 8 | 88 | 25 | 4 | 36 | 60 | 24 | 17 | 46 | 38 | – | – | – | – |
| DSM‐5 OUD remission | 25 | 0 | 24 | 76 | 25 | 4 | 40 | 56 | 24 | 8 | 42 | 50 | 22 | 5 | 50 | 46 |
| Clinician‐perceived OUD improvement/deterioration | 25 | 4 | 40 | 56 | 25 | 8 | 56 | 36 | 24 | 25 | 67 | 8 | – | – | – | – |
| Disease complications | 25 | 12 | 44 | 44 | – | – | – | – | – | – | – | – | – | – | – |
– |
| B. Behaviors, symptoms and functioning | ||||||||||||||||
| Days used illicit/non‐medical opioids | 25 | 8 | 20 | 72 | 25 | 12 | 40 | 48 | 24 | 8 | 33 | 58 | – | – | – | – |
| Opioid‐using occasions per day | 25 | 36 | 44 | 20 | – | – | – | – | – | – | – | – | – | – | – | – |
| Days to first use of opioids | 25 | 16 | 48 | 36 | – | – | – | – | – | – | – | – | – | – | – | – |
| Craving for opioids | 25 | 12 | 44 | 44 | 25 | 0 | 24 | 76 | 24 | 13 | 58 | 29 | – | – | – | – |
| Perception of OUD problems | 25 | 4 | 32 | 64 | 25 | 4 | 60 | 36 | 24 | 13 | 25 | 63 | 22 | 0 | 36 | 64 |
| Patient‐perceived OUD improvement/deterioration | 25 | 4 | 24 | 72 | 25 | 0 | 36 | 64 | 24 | 17 | 38 | 46 | – | – | – | – |
| Quality of life | 25 | 4 | 24 | 72 | 25 | 8 | 44 | 48 | 24 | 8 | 43 | 50 | – | – | – | – |
| Emotionally salient life events | 25 | 8 | 56 | 36 | – | – | – | – | – | – | – | – | – | – | – |
– |
| C. MOUD treatment cascade | ||||||||||||||||
| Illicit/non‐medical drug toxicology | 25 | 4 | 12 | 84 | 25 | 4 | 12 | 84 | 24 | 8 | 33 | 58 | 22 | 0 | 59 | 41 |
| Toxicology and medication adherence monitoring | 25 | 24 | 52 | 24 | 25 | 12 | 48 | 40 | 24 | 17 | 58 | 25 | – | – | – | – |
| Stage of OUD treatment cascade | 25 | 8 | 24 | 68 | – | – | – | – | – | – | – | – | – | – | – | – |
| Proportion of days covered in treatment in first year after initiation | 25 | 0 | 20 | 80 | 25 | 0 | 36 | 64 | 24 | 4 | 63 | 33 | – | – | – | – |
| Days enrolled in MOUD | 25 | 0 | 16 | 84 | 25 | 0 | 28 | 72 | 24 | 0 | 25 | 75 | – | – | – | – |
| Clinician‐reported patient engagement | 25 | 20 | 60 | 20 | – | – | – | – | – | – | – | – | – | – | – | – |
| D. Morbidity and mortality | ||||||||||||||||
| Serious adverse events requiring hospitalization | 25 | 8 | 32 | 60 | – | – | – | – | – | – | – | – | – | – | – | – |
| Hospitalizations for injury/infection | 25 | 12 | 32 | 56 | – | – | – | – | – | – | – | – | – | – | – | – |
| Non‐fatal opioid‐related poisoning/overdose | 25 | 0 | 16 | 84 | 25 | 8 | 20 | 72 | 24 | 0 | 33 | 67 | 22 | 0 | 27 | 73 |
| Emergency reversals of overdose with naloxone | 25 | 8 | 16 | 76 | 25 | 4 | 28 | 68 | 24 | 0 | 42 | 58 | 22 | 0 | 64 | 36 |
| Opioid poisoning/overdose mortality | 25 | 8 | 12 | 80 | 25 | 0 | 16 | 84 | 24 | 0 | 21 | 79 | – | – | – | – |
OUD = opioid use disorder; OUD–COS = opioid use disorder–core outcome set; MOUD = medications for OUD; n = number of Delphi panel members completing round. Percentages shown in bold type indicate consensus achieved.
These items and the overall results of round 4 were brought forward to the research group for review. The research group accepted the panel's recommendations for the aforementioned three items. At the concept stage, one of the two PRO items (perceived OUD improvement/deterioration) attained consensus (72%). In round 2 consensus dropped to 64%, and then fell to 46% in round 3. Given early consensus the PRO measure was reviewed with the research group, where it was judged important to include, and the simple Patient Global Impression of Improvement (PGI‐I) formulation was determined to be appropriate. A quality‐of‐life outcome attained consensus at the concept stage. However, in rounds 2 and 3 the consensus shifted downwards to 48 and 50%, respectively. After discussion with the research group, it was decided to remove it. A urine toxicology measure had a similar rating profile, scoring 84% in round 1, but falling to 50 and 41% in rounds 3 and 4, respectively, so the group determined that the initial high rankings merited inclusion of an ObsRO measure in the COS.
OUD–COS
The OUD–COS items were finalized with feedback and input from a number of members of the research group (Table 3). The PGI‐I was developed for use in National Institute of Mental Health sponsored clinical trials [31] after SIGs did not offer substantive changes or adjustments to the proposed items. One SIG responded with feedback to consider a Social Determinants of Health (SDoH) measure as part of the OUD COS. It was noted that several domains considered in round 1 were broadly related to SDoH, but that many of the domains fell outside the purpose of the OUD–COS to be used by studies of the OUD treatment cascade.
TABLE 3.
Items and definitions for the OUD–COS (version 1.0)
| Type | Measure | Definition/wording |
|---|---|---|
| PRO | Patient global impression of improvement (PGI‐I) [31] | After a specified start date to end‐point or specified period of enrollment in MOUD or MOUD‐PSI: ‘compared to how you were before you started treatment, how are your opioid problems now?’ Response options (scoring): very much better (7); much better (6); a little better (5); no change (4); a little worse (3); much worse (2); very much worse (1) |
| PRO | Non‐fatal opioid overdose [32, 33, 34] | After a specified start date to end‐point or specified period of enrollment in MOUD or MOUD‐PSI: record the number of times the patient reports experiencing (or is witnessed by others to experience) of accidental, intentional or undetermined overdose after ingestion of drugs known or believed to contain opioids. This measure should specify the appropriate ICD‐10 (F10–19 requiring F11) and T‐codes |
| ObsRO | Illicit/non‐medical drug toxicology | After a specified start date to end‐point or specified period of enrollment in MOUD or MOUD‐PSI: the patient is defined as a ‘treatment responder’ if, in the past 21 days, they provide at least two urine drug screening tests in different weeks that are negative for opioids |
| PerfO | Duration of treatment | After a specified start date to end‐point or specified period of enrollment in MOUD or MOUD‐PSI: the number of days of continuous treatment from the first date of a specified type of MOUD (or all types) received by the patient to the last day of the patient's prescription. NB: treatment is continuous if there is no more than 30 days last expected day of dosing to account for medication re‐starting or transfers between providers |
| PerfO | Fatal opioid poisoning [32, 33, 34] | At any time during enrollment in MOUD or MOUD‐PSI: this is death from accidental or intentional ingestion of drugs, specifying the ICD‐10 F11 and X‐codes, as follows: one or more opioids are recorded, were determined or mentioned on the death certificate; ‘mental and behavioral disorders due to drug use’ (F11–F16, F18, F19); F11 must be present; and one or more of the following: ‘accidental poisoning by drugs, medicaments and biological substances’ (X40–X44); ‘intentional self‐poisoning by drugs, medicaments and biological substances’ (X60–X64); ‘assault by drugs, medicaments and biological substances’ (X85); or ‘poisoning by drugs, medicaments and biological substances, undetermined intent’ (Y10–Y14). ‘All‐cause’ mortality should also be reported if feasible |
OUD = opioid use disorder; OUD–COS = opioid use disorder–core outcome set; MOUD = medications for OUD; PRO = patient‐reported; ObsRo = observer‐reported; PerfRO = performance outcomes; PSI = psychosocial interventions.
DISCUSSION
For the OUD–COS (version 1), five measures were selected—two PRO measures (global impression of improvement and incident non‐fatal overdose); one ObsRO (illicit/non‐medical drug toxicology); and two PerfO (duration of treatment and fatal opioid poisoning). Three of these measures are pragmatically defined and are likely to be extractable from many current and prior studies, given their relative simplicity and direct clinical relevance. Two additional items were added by the research group that oversaw the process based on the e‐Delphi results and the research group's deliberations. One of these is a PRO which has been recommended for best practice by the US Food and Drug Administration [35]; the other is a PerfO relating to the recording of mortality, which is in widespread use and should be straightforwardly extractable from records and study data sets. There is probably significant potential to utilize a subset of this COS as a means of data harmonization for studies that are in progress or that have been completed. Mortality during a clinical trial is a reported event for safety and monitoring and is therefore gathered in adverse event records, and on occasion by linkage to mortality registries.
Additionally, most studies of OUD probably conduct some type of baseline assessment of non‐fatal overdose, and if such an event occurred during the trial then this would also be captured in adverse event reports. Similarly, urine toxicology and duration of treatment are captured during many clinical trials. While variations in data capture might make harmonization challenging, the use of cross‐walks could help to create a means to align studies retrospectively. For example, a study that reports outcomes in weeks could be cross‐walked to the days’ outcome outlined here. Additionally, toxicology results could be normalizing based on class of positive finding rather than specific drug screened. Despite these possibilities. the best results will probably occur by using the COS prospectively.
The OUD–COS is expected to align with other hospital‐required reporting standards. In the United States, the National Committee for Quality Assurance (NCQA) oversees the Health Effectiveness Data and Information Set (HEDIS) measures that are widely used by hospitals for quality reporting. In 2020, NCQA introduced a measure for HEDIS on ‘Pharmacotherapy for Opioid Use Disorder’. They defined this measure as the percentage of new OUD pharmacotherapy episodes that resulted in 180 days or more of treatment for individual age 16 years and older [36]. This measure, while worthwhile, is too long to be used practically by clinical trialists and its intention is more for use in analyzing claims data than in understanding clinical change for individuals.
While we anticipate that this OUD–COS will be deliberated and debated by various professional and stakeholder groups, there is an urgent need for its early adoption in the United States—not least so that key outcomes from the health‐care system are monitored about the delivery of treatment. A study of commercially insured patients has highlighted the need for substantial improvements in MOUD initiation and retention with the proportion of patient‐months on MOUD falling from 25 to 16% during 2010–14, while discontinuation rates of 30 days or less after initiation ranged from 31% (oral buprenorphine) to 52 and 58% (injectable naltrexone and injectable buprenorphine, respectively) to 70% (oral naltrexone) [37].
Limitations
Several study limitations are acknowledged. First, the impetus for the COS was to link trials conducted within the CTN, and while version 1.0 should be suitable for researchers in other groups and countries, this cannot be assured. Secondly, the COS was developed by clinical academic and allied professions and a Delphi panel with other viewpoints, including people who use drugs, and policymakers might well have selected other measures. Thirdly, the panel and research group largely reflected the demographics of the CTN leadership with little African American or Latinx representation. Attempts were made to mitigate this weakness by solicitation of input from various SIGs connected to the CTN. Better representation of ethnic and other minorities is needed for future COS studies for other substance use disorders within the CTN and the field more broadly.
CONCLUSION
Our Delphi panel was able to define a COS for OUD treatment studies that is brief and pragmatic. The measures are largely extractable from many current studies, and it is our belief that time cross‐walks can be created to engender consistency and standardization in reporting. We stress that the COS–OUD in no way rules out another primary or secondary outcome that would more effectively capture the expected benefits of a treatment intervention. We anticipate that the product of this study will advance the ability of the NIDA CTN to make studies more comparable and impactful and facilitate linkage among its studies retrospectively and prospectively. It will also be valuable to consider how the OUD–COS can contribute to data harmonization efforts under way globally.
DECLARATION OF INTERESTS
N.S.K. has no conflicts of interest to report. He is supported, in part, by the National Center for Advancing Translational Science (UL1‐TR002389, KL2‐TR002387), National Institute on Mental Health (R01‐MH117168), National Institute on Alcohol Abuse and Alcoholism (R24‐AA026801), National Institute on Drug Abuse (R01‐DA041071, UG1‐DA049467, R01‐DA051464) and Substance Abuse Mental Health Services Administration (H79‐SM082299). J.M. is a clinical academic consultant for the National Institute on Drug Abuse Center for Clinical Trials Network. He declares grant support from Indivior to King's College London for a study of extended‐release buprenorphine for opioid use disorder in England and Scotland. C.I.C. is supported by grants and contracts from the National Institute on Drug Abuse (UG1‐DA040314, R01‐DA047405) and the Food and Drug Administration. C.I.C. has received support managed through her institution from the Industry PMR Consortium, a consortium of companies working together to conduct post‐marketing studies required by the Food and Drug Administration that assess risks related to opioid analgesic use. S.M.Mc. has no conflicts of interest to report. He is supported, in part, by the National Cancer Institute (R01‐CA252185), National Institute on Alcohol Abuse and Alcoholism (R01‐AA028796, R21‐AA027045, P60‐AA026112, R01‐AA020248), National Institute of General Medical Sciences (P20‐GM121341), Veteran's Health Administration (ORH15531, I01HX002518), National Institute on Drug Abuse (UG1‐DA013714, R44‐DA977631, S06‐GM142130, N44‐DA171210), and the Substance Abuse Mental Health Services Administration (H79‐TI0822557). L.J.M. reports having served on an advisory board for Alkermes, Inc. R.K.M. has no conflicts of interest to report. She is supported, in part by NIDA grant U10‐DA015831. R.P.S. reports having served as a consultant to Verily Life Sciences. He is supported, in part, by NIDA UG1‐DA013034. L‐T.W. has no conflicts of interest to report. She is supported, in part, by NIDA grants UG1‐DA040317 and UG1‐DA040316. R.D.W. reports serving as a consultant to Analgesic Solutions. All other authors have no conflicts of interest to report.
AUTHOR CONTRIBUTIONS
Niranjan Karnik & John Marsden: Conceptualization; formal analysis; investigation; methodology; project administration; supervision; validation. Connor McCluskey: Data curation; formal analysis; resources; software. Randy Boley: Data curation; project administration; resources; supervision; validation. Katharine Bradley: Investigation. Cynthia Campbell: Investigation. Megan Curtis: Investigation. David Fiellin: Investigation. Udi Ghitza: Conceptualization; investigation; project administration; supervision. Kathryn Hefner: Investigation. Yih‐Ing Hser: Investigation. R. Kathryn McHugh: Investigation. Sterling McPherson: Investigation. Larissa Mooney: Investigation. Landhing Moran: Conceptualization; investigation; project administration; supervision. Sean Murphy: Investigation. Robert Schwartz: Investigation. Dikla Shmueli‐Blumberg: Investigation. Matisyahu Shulman: Investigation. Kari Stephens: Investigation. Katherine Watkins: Investigation. Roger Weiss: Investigation. Li‐Tzy Wu: Investigation.
ACKNOWLEDGEMENTS
Resource support for this study was provided by CTN Great Lakes Node at Rush University Medical Center (NIH Grant UG1‐DA049467). The protocol was implemented with no changes. We would like to thank the members of the NIDA CTN and the NIDA Center for Clinical Trials Network (CCTN) for their input to this consensus development study. CCTN scientific staff participated as members of the research group and were involved in the study implementation, review and writing of the manuscript. The funder has no direct role in the study design, implementation and manuscript preparation. The content is solely the responsibility of the authors and does not necessarily represent the official view of NIDA.
Karnik NS, Marsden J, McCluskey C, Boley RA, Bradley KA, Campbell CI, et al. The opioid use disorder core outcomes set (OUD–COS) for treatment research: findings from a Delphi consensus study. Addiction. 2022;117:2438–2447. 10.1111/add.15875
COMET registration: http://www.comet-initiative.org/Studies/Details/1579.
For affiliations refer to page 2445.
Funding information National Institute on Drug Abuse
REFERENCES
- 1. Koob GF, Volkow ND. Neurocircuitry of addiction. Neuropsychopharmacology. 2010;35:217–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Ciccarone D. The triple wave epidemic: supply and demand drivers of the US opioid overdose crisis. Int J Drug Policy. 2019;71:183–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.European Centre for Drugs and Drug Addiction. European Drug Report 2017: Trends and Developments [internet]. Report no.: 978‐92‐9497‐095–4. Luxembourg: Office of the European Union; 2017. Available at: http://www.emcdda.europa.eu/publications/edr/trendsdevelopments/2017. Accessed 21 April 2020.
- 4. Hedegaard H, Warner M, Minino AM. Drug Overdose Deaths in the United States, 1999–2015. NCHS Data Brief 2017;273:1–8. [PubMed] [Google Scholar]
- 5. Williams AR, Nunes EV, Bisaga A, Levin FR, Olfson M. Development of a cascade of care for responding to the opioid epidemic. Am J Drug Alcohol Abuse. 2019;45:1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Blanco C, Volkow ND. Management of opioid use disorder in the USA: present status and future directions. Lancet. 2019;393:1760–72. [DOI] [PubMed] [Google Scholar]
- 7. Wild TC, Hammal F, Hancock M, Bartlett NT, Gladwin KK, Adams D, et al. Forty‐eight years of research on psychosocial interventions in the treatment of opioid use disorder: a scoping review. Drug Alcohol Depend. 2021;218:108434. [DOI] [PubMed] [Google Scholar]
- 8. Clarke DE, Ibrahim A, Doty B, Patel S, Gibson D, Pagano A, et al. Addiction Medicine practice‐based research Network (AMNet): assessment tools and quality measures. Subst Abuse Rehabil. 2021;12:27–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Marsden J, Tai B, Ali R, Hu L, Rush AJ, Volkow N. Measurement‐based care using DSM‐5 for opioid use disorder: can we make opioid medication treatment more effective? Addiction. 2019;114:1346–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Sanger N, Panesar B, Rosic T, Dennis B, D'Elia A, Hillmer A, et al. The future of precision medicine in opioid use disorder: inclusion of patient‐important outcomes in clinical trials. Rev Bras Psiquiatr São Paulo Braz. 2021;43:138–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Williams AR, Nunes EV, Bisaga A, Pincus HA, Johnson KA, Campbell AN, et al. Developing an opioid use disorder treatment cascade: a review of quality measures. J Subst Abuse Treat. 2018;91:57–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Tai B, Dobbins R, Blackeney Q, Liu D, Moran L. The NIDA clinical trials network: evolving, expanding, and addressing the opioid epidemic. Addict Sci Clin Pract. 2021;16:28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Rosa C, Ghitza U, Tai B. Selection and utilization of assessment instruments in substance abuse treatment trials: the National Drug Abuse Treatment Clinical Trials Network experience. Subst Abuse Rehabil. 2012;3(1):81–89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Ghitza UE, Gore‐Langton RE, Lindblad R, Shide D, Subramaniam G, Tai B. Common data elements for substance use disorders in electronic health records: the NIDA clinical trials network experience: integrating standardized data on substance use disorders into EHRs. Addiction. 2012;108:3–8. [DOI] [PubMed] [Google Scholar]
- 15.NIDA CTN. Common Data Elements [internet]. 2014. Available at: http://cde.drugabuse.gov/. Accessed 5 February 2022.
- 16. Ghitza UE, Gore‐Langton RE, Lindblad R, Tai B. NIDA clinical trials network common data elements initiative: advancing big‐data addictive‐disorders research. Front Psychiatry. 2015;6:33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. CTN Assessment Standardization. CTN Investigator Toolbox Website. [Internet]. 2020. Available at: https://ctndsc2.com/system/files/ToolboxStandardAssessments_V1.0_03JAN2020_0.pdf. Accessed 5 February 2022.
- 18. Tai B, Boyle M, Ghitza U, Kaplan RM, Clark HW, Gersing K. Meaningful use of electronic behavioral health data in primary health care. Sci Transl Med. 2012;4(119):119mr3. 10.1126/scitranslmed.3003324 [DOI] [PubMed] [Google Scholar]
- 19. Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET Handbook: version 1.0. Trials. 2017;18(S3):280. 10.1186/s13063-017-1978-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Chan A‐W, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža‐Jerić K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158:200–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Chan A‐W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586. 10.1136/bmj.e7586 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Calvert M, Kyte D, Mercieca‐Bebber R, Slade A, Chan A‐W, King MT, et al. Guidelines for inclusion of patient‐reported outcomes in clinical trial protocols: the SPIRIT‐PRO extension. JAMA. 2018;319:483–94. [DOI] [PubMed] [Google Scholar]
- 23. Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel‐group randomised trials. Lancet. 2001;357:1191–4. [PubMed] [Google Scholar]
- 24. Schulz KF, Altman DG, Moher D, CONSORT Group . CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. PLOS Med. 2010;7(3):e1000251. 10.1371/journal.pmed.1000251 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Dalkey NC. Delphi [internet]. 1967. Available at: https://www.rand.org/pubs/papers/P3704.html. Accessed 5 February 2022.
- 26. Dalkey N, Helmer O. An experimental application of the DELPHI method to the use of experts. Manag Sci. 1963;9:458–67. [Google Scholar]
- 27. Linstone HA, Turoff M. The Delphi method. Techniques and applications [internet]. 2002. Available at: https://web.njit.edu/~turoff/pubs/delphibook/delphibook.pdf. Accessed 5 February 2022.
- 28. Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67:401–9. [DOI] [PubMed] [Google Scholar]
- 29. Karnik NS, Campbell CI, Curtis ME, Fiellin DA, Ghitza U, Hefner K, et al. Core outcomes set for research on the treatment of opioid use disorder (COS‐OUD): the National Institute on Drug Abuse clinical trials network protocol for an e‐Delphi consensus study. Trials. 2021;22(1):102. 10.1186/s13063-021-05051-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Kirkham JJ, Gorst S, Altman DG, Blazeby JM, Clarke M, Tunis S, et al. Core Outcome Set–STAndardised Protocol items: the COS‐STAP statement. Trials. 2019;20(1):116. 10.1186/s13063-019-3230-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Guy W. ECDEU Assessment Manual for Psychopharmacology Revised, 1976. Report no.: DHEW Publication no. (ADM) 76–338. Rockville, MD: US Department of Health, Education, and Welfare; 1976.
- 32. Brady JE, Giglio R, Keyes KM, DiMaggio C, Li G. Risk markers for fatal and non‐fatal prescription drug overdose: a meta‐analysis. Inj Epidemiol. 2017;4(1):24. 10.1186/s40621-017-0118-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Karmali RN, Ray GT, Rubinstein AL, Sterling SA, Weisner CM, Campbell CI. The role of substance use disorders in experiencing a repeat opioid overdose, and substance use treatment patterns among patients with a non‐fatal opioid overdose. Drug Alcohol Depend. 2020;209:107923. 10.1016/j.drugalcdep.2020.107923 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Gomes T, Campbell T, Tadrous M, Mamdani MM, Paterson JM, Juurlink DN. Initial opioid prescription patterns and the risk of ongoing use and adverse outcomes. Pharmacoepidemiol Drug Saf. 2021;30:379–89. [DOI] [PubMed] [Google Scholar]
- 35. Center for Drug Evaluation and Research (CDER) . The Voice of the Patient [internet]. Silver Spring, MD: US Food and Drug Administration (FDA); 2018. Available at: https://www.fda.gov/media/124391/download
- 36. Pharmacotherapy for Opioid Use Disorder [internet] . NCQA [cited 2022 Feb 5]. Available at: https://www.ncqa.org/hedis/measures/pharmacotherapy-for-opioid-use-disorder/
- 37. Morgan JR, Schackman BR, Leff JA, Linas BP, Walley AY. Injectable naltrexone, oral naltrexone, and buprenorphine utilization and discontinuation among individuals treated for opioid use disorder in a United States commercially insured population. J Subst Abuse Treat. 2018;85:90–6. [DOI] [PMC free article] [PubMed] [Google Scholar]