A clinical trial loses access to some of its experimental treatment conditions: What can be done?

Abstract

This commentary examines methodological and ethical problems encountered when a multi-arm clinical trial loses access to one or more of its arms, using the Retention Phase of the NIDA Clinical Trials Network CTN-0100 study, Optimizing Retention, Duration and Discontinuation Strategies for Opioid Use Disorder Pharmacotherapy (RDD) as an example. RDD is a community-based, multi-site trial testing strategies to reduce dropout from medication treatment for opioid use disorder. Among patients with opioid use disorder initiating buprenorphine treatment, the original design was a 3 by 2 factorial comprising 3 pharmacological conditions (Standard-Dose sublingual buprenorphine--16mg/day target [SL-BUP 16], High-Dose sublingual buprenorphine--32 mg/day target [SL-BUP 32], or extended-release injectable buprenorphine [XR-BUP]), crossed with 2 behavioral conditions: medical management with vs. without a technology-based digital therapeutic app providing cognitive behavioral therapy lessons and contingency management. The trial experienced two major disruptions to study interventions: 1) The supply of XR-BUP became temporarily unavailable due to manufacturing problems; and 2) The company supplying the digital therapeutic app went bankrupt, rendering the original app permanently unavailable. Questions considered by the study lead team included: 1) Whether to pause recruitment into the trial altogether or continue recruitment into truncated designs omitting the unavailable interventions; 2) How to account for participants who did not experience full exposure to the halted interventions; 3) Whether to substitute a similar intervention; and 4) The problem of concurrent randomizations, namely that a truncated design does not contain all the concurrent randomizations of the full design, introducing risk of confounding or bias. This experience from the RDD trial demonstrates how multi-arm clinical trials that lose access to an intervention arm can continue with a truncated design, allowing continued progress on study aims, while balancing methodological purity with the pragmatic imperative to keep the trial running and respect subjects’ participation.

The Retention Phase of the Optimizing Retention, Duration and Discontinuation Strategies for Opioid Use Disorder (OUD) Pharmacotherapy (RDD) study (NIDA Clinical Trials Network CTN-0100) is a multi-site, multi-arm trial to test strategies to increase 6-month adherence to medication treatment for OUD (MOUD). At two different times over the course of the trial, study interventions became unavailable. This Commentary discusses the problem of inadvertent loss of study arms in a clinical trial using RDD as an example. We discuss the methodological, clinical and ethical problems posed by the disruptions in study arms, and how the design was adjusted accordingly.

A multi-arm clinical trial loses access to study arms

The hypotheses of RDD are that, compared to standard dose sublingual buprenorphine with a 16 mg/day target (SL-BUP 16), high dose sublingual buprenorphine 32 mg/day target (SL-BUP 32) and/or extended-release injectable buprenorphine (XR-BUP; CAM2038 formulation, (now marketed as Brixadi) (Lofwall et al., 2018) will be more effective in retaining individuals on MOUD, and that the addition of the digital therapeutic behavioral intervention Pear-002a—an experimental version of the reSET app based on the Therapeutic Education System (Christensen et al., 2014) reformulated as a smartphone app—to standard Medical Management (MMR) will increase retention compared to Medical Management (MM) alone, in a 3 (SL-BUP 16; SL-BUP 32; XR-BUP) by 2 (MMR; MM) factorial design. For patients with OUD starting treatment on extended-release injectable naltrexone (XR-NTX), a separate branch tests the hypothesis that the digital therapeutic will increase retention compared to MM alone. The full design and rationale have been previously published (Shulman et al., 2025). Buprenorphine is a highly effective treatment for OUD, reducing opioid use and risk of overdose, but clinical trials and claims-based data show that less than 50% of patients continue buprenorphine for at least 6 months (Hser et al., 2014; Lee et al., 2018), and dropout is associated with relapse and risk of overdose (Greiner et al., 2021; Nunes et al., 2018). XR-NTX is also effective while taken, but discontinuation has been an even greater problem (Hser et al., 2016; Hser et al., 2014; Larochelle et al., 2018; Lee et al., 2018). Thus, strategies to improve adherence to these MOUD are an important public health goal.

After the trial began recruitment in June 2021, two disruptions occurred at different points: 1) XR-BUP (CAM2038) became temporarily unavailable due to a manufacturing pause required by the U.S. Food and Drug Administration (FDA) (Camurus, December 15, 2021); 2) Later, Pear-002a, the digital therapeutic app reSET, became permanently unavailable due to the bankruptcy of the manufacturer, Pear Therapeutics (Jennings, April 7th, 2023). Figure 1 visualizes these disruptions to the 3 by 2 design over the study timeline.

Figure 1. Disruptions to the RDD study resulting from the loss of availability of the extended-release injectable buprenorphine (XR-BUP), and digital therapeutic app.

BUP is buprenorphine. Standard dose is a target of 16mg/day sublingual buprenorphine (SL-BUP 16); High dose is a 32mg/day sublingual buprenorphine target (SL-BUP 32); Extended release is the CAM2038 injectable (XR-BUP), now marketed as Brixadi; MM is Medical Management delivered by a medical clinician; MMR is Medical Management plus a digital therapeutic app (originally Pear-002a/experimental version of reSET, manufactured by Pear Therapeutics).

Clinical management of current participants

During the XR-BUP disruption in RDD, the first imperative, clinical and ethical, was to ensure that research participants who had been randomized into the XR-BUP arms be offered an alternative form of MOUD to continue to protect them from relapse and overdose. Thus, SL-BUP was made available to all XR-BUP-randomized participants to be managed according to clinical judgement of the study medical clinicians. For participants who preferred to remain on a long-acting buprenorphine formulation, sites could attempt to obtain the commercially available version of XR-buprenorphine (Sublocade^®). Operational processes were laid out for site staff in guidance documents and discussed during weekly calls. An informed consent addendum for study participants randomized to XR-BUP explained the lack of availability of study medication, highlighted the importance of continuing effective medication for opioid use disorder, and presented options for continuing medication treatment.

Upon the subsequent behavioral app disruption, where the digital therapeutic app Pear-002a was lost, there was not the same imperative to immediately substitute an alternative, since the participants were already receiving a standard of care (Medical Management) and there was still equipoise around the effectiveness of the app. When it became clear that the loss of Pear-002a or reSET, the commercially available version at the time, would be permanent, the study lead team focused on finding an alternative digital therapeutic with similar features to substitute into the design.

Whether to halt recruitment or continue with a truncated design

In a trial with only two arms, such as a typical placebo-controlled trial or a two-treatment comparative effectiveness trial, a missing arm leaves the study lead team little choice but to halt recruitment until the missing condition can be restored. This was the case with the XR-NTX branch of the RDD trial, as there were only two arms, MM and MMR. When the MMR/digital therapeutic arm was temporarily lost, recruitment had to be halted until an alternative digital therapeutic could be substituted.

Halting recruitment wastes resources, since study site teams are funded but are not being productive. By contrast, in a multi-arm trial, there is the option to continue recruitment, pausing randomization into unavailable arm(s) while continuing randomization into a truncated design with the arms that remain available. This approach allows the study to proceed and collect data relevant to at least some of the study aims, thus continuing to maximally use study staff and resources at the clinical sites, ensuring efficient use of study funds.

During the XR-BUP disruption in RDD, participants could still be randomized to the two SL-BUP conditions—High-Dose (SL-BUP 32) and Standard-Dose (SL-BUP 16). At that time, the randomization to the two behavioral conditions, digital therapeutic app (MMR) vs. no app (MM), was unaffected. Therefore, recruitment was continued into a truncated design (Figure 2) with a 2×2 randomization scheme. The study lead team was able to rapidly develop new consent forms and obtain IRB approval for this design change. Within a few months an alternative supply of XR-BUP (CAM2038) was secured from the European manufacturer, and the original 3 by 2 design was reinstated.

Figure 2. Modified design during the extended-release injectable buprenorphine (XR-BUP) disruption.

XR-BUP arms are suspended, randomization to the other arms continues. SL-BUP 16 is sublingual buprenorphine 16 mg target; SL-BUP 32 is sublingual buprenorphine 32 mg target; XR-BUP is extended-release injectable buprenorphine; XR-NTX is extended-release injectable naltrexone; MM is Medical Management; MMR is Medical Management plus the digital therapeutic app.

During the behavioral app disruption, participants could still be randomized to the three medication conditions, the XR-BUP condition having been restored at that point. Once again, a truncated design was implemented as shown in Figure 3. The study lead team was able to rapidly develop new consent forms, obtain IRB approval for this design change, and resume recruitment. Recruitment into the XR-NTX branch had to be suspended, as noted above.

Figure 3. Modified design during the behavioral app disruption.

Arms involving the digital therapeutic app are suspended. SL-BUP 16 is sublingual buprenorphine 16 mg target; SL-BUP 32 is sublingual buprenorphine 32 mg target; XR-BUP is extended-release injectable buprenorphine; XR-NTX is extended-release injectable naltrexone; MM is Medical Management; MMR is Medical Management plus the digital therapeutic app.

Substituting an alternative intervention

Unlike the XR-BUP disruption, in which another source of the same XR-BUP was known to be available and would only require several months to arrange, the loss of Pear-002a app was permanent. Thus, the study lead team had to decide how to proceed to achieve the study Specific Aim of determining the effectiveness of a technology-based behavioral application to increase retention in treatment. At this point in the trial, it was determined that the sample size available from randomizations-to-date did not afford adequate power to assess the effectiveness of the Pear-002a app. For this reason, it was decided to replace the original app with another commercially available app, Connections (CHESS Health).

Like Pear-002a, Connections delivers the two elements that evidence suggested were effective at improving outcomes of buprenorphine treatment: cognitive behavioral therapy (CBT) and contingency management (Carroll et al., 2008; Carroll & Weiss, 2017; Christensen et al., 2014). Table 1 provides a side-by-side comparison of the apps.

Table 1.

Comparison Between the Pear-002a (Pear Therapeutics) and Connections (CHESS Health) Apps

	Pear-002a (Pear Therapeutics)	Connections (CHESS Health)
Commercial availability	Available by prescription before Pear Therapeutics bankruptcy Insurance coverage often declined, which led to the bankruptcy	Available through contracts to State Agencies and Healthcare Organizations
Access via app on smart phone or tablet	Yes	Yes
CBT	40+ “therapy lessons”, 10 to 15 minutes each, based on the Community Reinforcement Approach, which began with relapse prevention skills, then covered various social skills Quizzes within each therapy lesson tested and reinforced the skills	8 “CBT4CBT” modules, 30 to 40 minutes each, with focus on relapse prevention skills Quizzes within each module test and reinforce the skills Suite of links to community-based resources supportive of recovery
Contingency management	Rewards for completing lessons and providing urines negative for opioids Reinforcement schedule based on Petry’s “Fishbowl” method with a random, game-like element Approximately $200 average potential reward earnings Rewards delivered as gift cards to vendors chosen by the patient	Rewards for completing CBT modules and providing urines negative for opioids Fixed reinforcement schedule Approximately $200 potential earnings Rewards delivered via reloadable debit cards

With the substitution of the Connections app, the study aims were amended to indicate testing the impact of the essential therapeutic elements of the apps, CBT and contingency management, over different app platforms. This prevents losing the ability to test one of the major aims of the study. In a sense, this broadens the aim to cover a type of therapeutic app, rather than any one specific app. It does introduce complexity to the analysis and interpretation of the findings if the two apps differ in effectiveness.

Participants who do not experience full exposure to arms that are halted

When a study intervention abruptly becomes unavailable, some participants will not experience full exposure to interventions to which they were randomized. In RDD, the intended length of exposure to the randomized interventions was at least 26 weeks, to correspond with the primary outcome of continuous retention on MOUD for 26 weeks after randomization. Because of the disruption, participants randomized fewer than 26 weeks before the break in availability of XR-BUP would receive less exposure to study XR-BUP.

As noted above, these participants were transferred to other buprenorphine formulations for clinical safety. According to the Intent-to-Treat principle, such participants should be included in the primary outcome analysis, and this was the plan adopted for RDD. The effect that these participants might have on the estimates of treatment effects could be explored with subsequent “per protocol” sensitivity analyses. These analyses would focus on subsamples that received full exposure to the intended buprenorphine conditions.

During the behavioral app disruption, the contingency management feature of the Pear-002a app ceased immediately, while the learning modules feature of Pear-002a, providing counseling content based on CBT and Community Reinforcement Approach, remained available for a period before ceasing. Thus, participants randomized to the app might have experienced partial exposure to the app. Again, according to the Intent-to-Treat principle, these participants would be included in the primary outcome analysis with secondary sensitivity analyses possible to explore the impact of less than full intervention exposure.

The problem of concurrent randomizations

In a truncated design, the arms that remain lack concurrent randomizations with the arm(s) that have been halted. This raises the question whether participants randomized during the halt in availability of one or more arms may differ from those randomized into the full, originally intended design in ways that affect their prognosis. If this is deemed to be likely, proceeding with a truncated design threatens to introduce bias into the estimates of treatment effects. The participants randomized during the unavailability of one of the randomized conditions could differ because of time trends. It is also possible that the truncated design attracts different participants than the full, original design. For example, in RDD, study sites had reported that some eligible patients with opioid use disorder declined to enter the original 3 by 2-arm trial because they did not want the possibility of an injected medication. Such patients might be more likely to enter the trial under the truncated design with no injection medication arms, and they might differ on illness severity or motivation.

All participants could be included under the original 3 by 2 design and analytic plan, yielding in effect an unbalanced design with periods missing cells and with attempts at statistical adjustment, but bias cannot be ruled out. There is precedent for such an approach in the STAR*D trial, in which patients with depression who had failed to respond to a standard antidepressant medication were randomly assigned to one of four next steps, but participants were allowed to eliminate one of those options from their randomization (Huynh & McIntyre, 2008). This may be appropriate if the risk of bias is believed to be low (not the case in RDD), and using statistical methods such as propensity score matching to adjust for potential confounds, although bias still cannot be ruled out.

As a result of these concerns, the Data and Safety Monitoring Board (DSMB) for RDD initially recommended that the participants randomized into the truncated design during the XR-BUP pause be excluded from the primary outcome analysis, and that they be included only in confirmatory sensitivity analyses and other secondary analyses. This preserved the integrity of the original 3×2 factorial design.

The study lead team agreed that comparisons involving non-concurrent randomizations should indeed not be included in the primary outcome analysis because of the risk of bias. However, a total of 152 participants were randomized into the truncated design during the XR-BUP pause, and 119 during the behavioral app pause, for a total of 271 participants constituting a substantial fraction of the original overall target sample of 1200. Excluding them in the primary outcome analysis seemed wasteful, given a fixed study budget, and would jeopardize achieving the target sample size and planned statistical power. Moreover, this plan seemed to violate the ethical principle of respect for persons--participants had put their time and effort into research participation, expecting that their data would contribute to answering the main study questions. Discarding their data in the primary outcome analyses seemed potentially to disrespect their participation.

To reconcile these issues, the study lead team reasoned that when there were concurrent randomizations, for example randomization to SL-BUP 16 vs SL-BUP 32 during the XR-BUP disruption, there would not be concern about bias in comparison of the two sublingual arms. The study lead team therefore proposed that the original primary outcome analysis plan, based on the full 3×2 factorial design and corresponding 2-way ANOVA analytic model, be abandoned in favor of three separate pair-wise comparisons of treatment conditions with corresponding 1-way ANOVA comparisons: 1) High (32 mg target) vs. Standard-Dose (16 mg target) SL-BUP; 2) XR-BUP vs. Standard-Dose SL-BUP; and 3) Digital therapeutic app vs. no app, in which each comparison is based only on data from concurrent randomizations. This plan would allow for inclusion of primary outcome data from all participants randomized during the disruptions, so their data was not wasted, while ensuring that comparisons of medication or behavioral treatment conditions were only based on concurrent randomizations and therefore not confounded. Simulation studies, based on the original simulation approach for power and sample size from the study protocol, suggested that pairwise comparisons would have power equivalent to that expected from the original factorial design. This plan sacrifices the ability to test interactions between the medication and behavioral app arms in the primary outcome analysis, but interactions were not expected and not part of the stated aims. Exploratory analyses will examine potential interactions and compare effectiveness of the separate apps. The DSMB agreed with this plan.

Concluding thoughts

In multi-arm clinical trials, there is the potential that one or more arms become inadvertently unavailable over the course of the trial, either temporarily or permanently. Existing literature focuses mainly on planned dropping of study arms for futility or efficacy, such as in multi-arm adaptive trials (Granholm et al., 2023). As noted, the STAR*D trial allowed participants to choose to drop one of four arms prior to being randomized, creating in effect an unbalanced design (Huynh & McIntyre, 2008).

The experience in the RDD trial of two episodes of inadvertent loss of study interventions forced the study lead team to analyze the resultant problems and create solutions. The team sought to keep the trial running, making best use of the sites and resources supporting the trial, protect the treatment integrity and safety of participants whose study treatments became unavailable, and amend the design and statistical analysis plans in order to preserve rigorous tests of the main aims, while using the data from all participants randomized during the disruptions to maximize sample size and power and fully respect their participation. Challenges and decision points encountered included: 1) Whether to stop recruitment into the trial altogether or continue recruitment into a truncated design omitting the unavailable interventions; 2) How to account for participants who do not experience full exposure to the halted interventions because they are randomized in the period leading up to the halt; 3) Whether to substitute a similar intervention if the halted intervention will be unavailable indefinitely; and 4) The problem of concurrent randomizations, namely that a truncated design does not contain all the concurrent randomizations of the full design, introducing risk of confounding or bias.

The study was amended accordingly by 1) providing access to sublingual buprenorphine or an alternative XR formulation for those randomized to XR-BUP to ensure continuity of treatment and participant safety; 2) continuing recruitment into modified truncated designs until study interventions could be restored or substituted; 3) including participants who did not experience full intervention exposure in the primary outcome analysis, according to the intent to treat principle, with secondary sensitivity analyses planned to examine the impact of including these participants on the estimated treatment effect; 4) substituting one behavioral treatment for another with similar therapeutic elements in order to preserve a modified aim on the impact of a digital therapeutic behavioral intervention; and 5) abandoning a 2-way ANOVA analytic model for three separate pair-wise comparisons of subsamples who were concurrently randomized, allowing all participants’ data to contribute to at least some of the main aims, while avoiding the risk of bias inherent in non-concurrent randomizations. The lessons learned are presented here in order that this experience may be useful to future multi-arm trials when an intervention arm becomes unavailable mid-study.

Highlights:

• Losing an arm in a multi-arm trial presents methodological and ethical challenges

• Multi-arm studies can continue with a truncated or modified design

• Key issues: Distinguishing affected study aims, whether to pause or substitute arms

• Considerations for adjusting the data analytic plan are described