A Qualitative Scoping Review of Early-Terminated Clinical Trials Sponsored by the Department of Veterans Affairs Cooperative Studies Program From 2010 to 2020

Abstract

Increasing attention has been paid to the risks and benefits of terminating large clinical trials before reaching prespecified targets, because such decisions can greatly affect the implementation of findings. The Department of Veterans Affairs (VA) Cooperative Studies Program (CSP) is a research infrastructure dedicated to conducting high-quality clinical research. A scoping review was performed to characterize barriers preventing the attainment of prespecified recruitment, statistical power, or sample-size targets in VA CSP trials. A trial was eligible for inclusion if the trial was sponsored by the VA CSP, primary findings were published within the last 10 years, and a decision was made to terminate enrollment or follow-up before meeting a priori recruitment or endpoint targets. In 11 of 29 included trials (37.9%), a decision was made to terminate the trial early. The most common reason for early termination was related to under-recruitment (n = 5). Other reasons included early detection of safety signals (n = 2), futility (n = 1), and benefit (n = 1). This review highlights recruitment as a critical facet of trial conduct that may hinder the production of high-quality data and thus warrant additional attention. Solutions to enhance recruitment now implemented by the VA CSP, including dedicated enrollment infrastructure and screening facilitated by informatics approaches, show promise in reducing this cause for early termination.

Abbreviations

CSP

Cooperative Studies Program

DSMB

Data and Safety Monitoring Board

RCT

randomized clinical trial

VA

Department of Veterans Affairs

INTRODUCTION

The US Department of Veterans Affairs (VA) Cooperative Studies Program (CSP) is a research infrastructure that focuses on enhancing the health and health care of veterans and the nation through evidence-based medicine. CSP funds and manages clinical investigations and epidemiologic research using multiple biostatistical data-coordinating and epidemiologic research centers located across the continental United States, as well as a dedicated pharmacy coordinating center. CSP has implemented a collaborative study development model that is designed to connect research-oriented clinicians with dedicated clinical trialist teams (1). This model is both an efficient and effective method of conducting impactful clinical research in the veteran population. Although the CSP has a robust project management infrastructure, continuous process improvement remains critical for certain aspects of clinical trial planning and execution. One of these aspects is the decision to terminate a clinical trial prior to achieving prespecified recruitment or event targets.

Although infrequent, there are multiple published CSP sponsored trials that ended before reaching prespecified endpoints. In the past 10 years especially, there has been greater focus placed on the importance of clinical trials reaching prespecified endpoints. Any research organization that regularly conducts clinical trials is aware of the possibility of ending the trial before prespecified time points and the potential risk of not achieving the intended outcomes that early termination introduces. Increasing attention has been paid to the risks and benefits of early termination, from investigators, independent review boards, as well as funding agencies. Multiple factors play into the decision to terminate a project early, such as inability to recruit, unforeseen changes in clinical practice, or discontinuation of funding. The early termination of a trial can be opportunistic, as when a strong efficacy signal is detected during prespecified interim data analysis and a conclusive answer to the core question is found prior to the prespecified time point, or if safety signals arise that prompt the discontinuation of enrollment. In these instances, early termination can be advantageous from financial, scientific, and ethical standpoints.

A balance must exist between the ethical considerations for early trial discontinuation and the integrity and validity of generated trial data. A trial may be stopped if there is no justification for exposing human participants to additional potential risk by continuing the trial. This decision to terminate a trial typically is at the recommendation of an independent Data and Safety Monitoring Board (DSMB), whose primary function is to review accumulating data regularly and provide recommendations to sponsors on whether a trial should continue as planned, be modified, or be terminated (2). Even with the ethical considerations at the forefront, there are several disadvantages associated with the decision to terminate a trial early. First, the ability to ascertain reproducible data and definitively answer questions for the primary efficacy endpoints may be hindered, particularly when the concern that necessitates the early termination of a trial arises from a secondary safety endpoint analysis. This decision is made due to concern for the current human participants; however, in making the decision, the benefit may not be considered of what a complete dataset could provide the medical community in the future, especially when studying highly prevalent disease states.

Success or failure of a research endeavor may also be determined in part by considering return on investment. Important evaluations of the timing and magnitude of expected impact from the trial data already generated compared with the investment costs may factor into a sponsor’s decision to terminate a potentially futile ongoing clinical trial (3). In some instances, early termination is a risk to the substantial time, effort, and financial investment spent in an effort to answer a clinically meaningful question.

Stakeholders often desire to know if a clinical trial has influenced clinical practice, because this knowledge can be helpful in demonstrating accountability, assessing return on investment from research endeavors, and identifying and planning future research avenues. In the past, this goal has been partly assessed by CSP through systematic review of the impact of its published trials on clinical practice guidelines (4). A review of early-terminated clinical trials can further the ability of a sponsor to meaningfully gauge return on investment. Specifically, reflecting on how the act of terminating early may have hindered the trial’s ability to influence clinical care can be of value to a sponsor. A detailed aggregate of early-terminated trials can potentially improve the accountability of clinical trial teams and foster increased engagement in the planning process.

Our primary objective was to conduct a scoping review and to characterize barriers to the completion of CSP-sponsored randomized clinical trials (RCTs), according to prespecified recruitment, power, and sample-size calculations. Specifically, we focused on CSP-sponsored RCTs published within the past 10 years in order to identify trends in early termination. All RCTs reviewed were conducted to answer clinical questions about various disease states of particular relevance in the veteran population. By analyzing these early-terminated studies in a scoping review, it could be possible to establish trends that can be generalized and considered at the planning stage of future trials, not only for the CSP but by any clinical trial sponsor. We also explore the key decisions made during a CSP RCT that ultimately led to the early termination of the trial, and we identify who the key decision makers are. Finally, our exploratory aim was to determine the degree to which early termination limited the study’s ultimate ability to generate conclusive findings and thus influence clinical care.

METHODS

Inclusion criteria

Articles were included in this scoping review if the following conditions were met: 1) published between January 1, 2010, and October 1, 2020; 2) sponsored or funded primarily by the VA CSP, either through primary sponsorship or significant collaboration; and 3) terminated participant enrollment or follow-up before meeting a priori recruitment or endpoint targets (Figure 1).

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram for the process of identifying the clinical trials involving the Department of Veterans Affairs Cooperative Studies Program (CSP) that were terminated early. The identification process yielded a total of 108 publications. Screening based on title and abstract excluded 36 publications, with an additional 18 publications were excluded for not meeting the criteria for this scoping review. A total of 29 publications were eligible, with 11 of those reviewed in detail for early-termination justification. HMIC, Healthcare Management Information Consortium.

Literature search and study selection

Eligible publications were identified from the following sources: 1) a public database maintained by the VA CSP Central Office containing bibliographic information for all the clinical trials involving the VA CSP in the last 10 years, filtered by publication date (5); and 2) search results from the PubMed/MEDLINE database using the terms “(Veterans[All Fields] AND Cooperative[All Fields] AND Clinical Trial[All Fields])” filtered by article type: “Clinical Trial” and “Randomized Controlled Trial” and publication date “10 years.” These sources were selected and reviewed individually prior to consolidating in order to provide a robust and reliable account of all eligible trials. The data from these 2 sources were combined, and duplicate references were removed. Abstracts and, when needed, full-text publications were screened and reviewed to identify eligible clinical trials.

Studies were determined to satisfy early-termination criteria if participant recruitment or follow-up were formally concluded before attaining desired sample size based on a priori power and sample-size calculations, according to anticipated effect size. This early termination was determined by the following: 1) review of the statistical analysis section of the primary publication; 2) description of sample-size and effect-size calculations in a referenced methods article, if applicable; or 3) review of study record detail provided to ClinicalTrials.gov. Studies were considered to have not met sample-size requirements if the actual number of randomized participants was less than the number needed based on the trial-specific requirements, such as anticipated effect size or primary-outcome event rates. Those studies in which predetermined sample size was not reported were excluded from this review. See Figure 1 for full details.

Trials were grouped into categories according to the most likely reason for early termination. The categories include enrollment, funding, safety, futility, benefit, change in clinical practice, and other. The definitions of each of these categories are described in detail in Table 1. In classifying each study into the appropriate category, an appraisal of the primary termination reasons was made. If a study was ended on the basis of a single reason or category, this was documented as the primary termination reason. For those studies that had multiple reasons for early termination, the primary category was defined as the reason that most significantly contributed to the early trial termination, based on statistical a priori stopping criteria. If an alternate reason was noted in the primary publication as a contributing factor that related to meeting the definition of 1 of the listed outcome categories but was not credited as the ultimate reason for termination, it was attributed as a secondary termination reason. Termination based on outcomes of statistical analysis of interim data was given more weight in the categorization of early-termination decision-making. Each CSP-sponsored clinical trial is assigned a unique number at the initiation of planning, and many studies are referenced using this number in the primary publications. Each study analyzed in this review is referred to by this assigned CSP number.

Table 1.

Description of Early-Termination Categories

Reason for Early Termination: Category	Description
Enrollment	Study terminated because of inability to feasibly recruit target sample size or number of events in a reasonable time.
Funding	Study terminated by the sponsor because the study was not meeting its goals within the approved budget, with no additional funding granted at the end of approval period.
Safety	Study terminated upon recommendation from independent monitoring committee because of excess safety risk to participants, based on interim data analysis or newly published evidence or guidance with direct impact on the study.
Futility	Study terminated because of results of an interim futility analysis indicating that continuing the trial would not yield actionable results.
Benefit	Study terminated on the basis of meeting a priori criteria for successful intervention.
Change in clinical practice	Study terminated because of lack of clinical equipoise of intervention that affected care delivery and successful completion of the trial according to the protocol
Other	Study terminated for unforeseen reasons, including political pressure, data-quality issues, or withdrawal of support from drug product manufacturer.

RESULTS

Search results yielded a total of 65 unique publications sponsored by CSP and published within the timeline of interest. Of those publications, 29 (44.6%) were multicenter RCTs and were assessed for eligibility. The remainder of the publications were excluded from the analysis because they were epidemiologic studies (n = 16), secondary analysis of secondary endpoints (n = 10), review articles (n = 3), or for other miscellaneous reasons (n = 7). A total of 11 trials (37.9%) met the early-termination definition as described (Table 2). Of those 11 trials, 10 were primarily sponsored by the VA CSP, and 1 trial was sponsored via interagency agreements and used VA CSP collaboration. Two trials also received support, funding, or grants from a pharmaceutical company (n = 1) or other research infrastructure (n = 1).

Table 2.

Summary of Early-Terminated Publications

First Author, Year (Reference No.)	CSP Study No.	Publication Title	Clinicaltrials.Gov National Clinical Trial No.	Condition or Disease	Intervention	Primary Outcome	Actual No.	Expected No.	Primary Reason for Early Termination	Secondary Reason For Early Termination	Sponsor
Lin, 2020 (6)	553	Chemotherapy after prostatectomy for high-risk prostate carcinoma: a phase III randomized study	NCT00132301	Prostate cancer	Docetaxel plus prednisone vs. standard of care	Progression-free survival	298	636	Enrollment		CSP; Sanofi
Kamalesh, 2013 (7)	557	Percutaneous coronary intervention versus coronary bypass surgery in United States veterans with diabetes	NCT00326196	Coronary artery disease	PCI vs. CABG	Death or nonfatal MI	198	790	Enrollment		CSP
Brilakis, 2018 (8)	571	Drug-eluting stents versus bare-metal stents in saphenous vein grafts: a double-blind, randomized trial	NCT01121224	Saphenous vein graft atherosclerosis	BMS vs. DES	Death, MI, or revascularization	597	762	Enrollment	Funding	CSP
Rosenheck, 2011 (9)	555	Long-acting risperidone and oral antipsychotics in unstable schizophrenia	NCT00132314	Schizophrenia/schizoaffective disorder	IM risperidone vs. oral antipsychotic	Time to hospitalization	369	450	Enrollment	Clinical practice	CSP
Lo, 2010 (10)	558	Robot-assisted therapy for long-term upper-limb impairment after stroke	NCT00372411	Stroke	Robot-assisted therapy vs. intensive therapy vs. standard of care	Fugl-Meyer Assessment for Motor Recovery scale	127	158	Enrollment	Futility	CSP; Burke Medical Research Institute
Bauman, 2013 (11)	535	The effect of oxandrolone on the healing of chronic pressure ulcers in persons with spinal cord injury a randomized trial	NCT00101361	Pressure ulcer	Oxandrolone vs. placebo	Time to full healing	212	400	Futility		CSP
McFall, 2010 (12)	519	Integrating tobacco cessation into mental health care for posttraumatic stress disorder: a randomized controlled trial	NCT00118534	Mental health, PTSD, tobacco use disorder	Integrated care for smoking cessation vs. standard of care	Prolonged abstinence	943	1,400	Benefit		CSP
O’Dell, 2013 (13)	551	Therapies for active rheumatoid arthritis after methotrexate failure	NCT00405275	Rheumatoid arthritis	Etanercept vs. placebo	Change in disease activity score for 28 joints	353	450	Funding		CSP
Fried, 2013 (14)	565	Combined angiotensin inhibition for the treatment of diabetic nephropathy	NCT00555217	Kidney disease, type 2 diabetes	Losartan monotherapy vs. losartan plus lisinopril	Rate of glomerular filtration rate decline, development of end-stage renal disease, and death	1,448	1,850	Safety		CSP
Fan, 2012 (15)	560	A comprehensive care management program to prevent chronic obstructive pulmonary disease hospitalizations: a randomized, controlled trial	NCT02036294	COPD	COPD self-management education vs. standard of care	Time to first hospitalization for COPD	426	960	Safety		CSP
Holodniy, 2011 (16)	512	Results of antiretroviral treatment interruption and intensification in advanced multi-drug resistant HIV infection from the OPTIMA Trial	NCT00050089	HIV infection	No ARDFP vs. ARDFP; standard ART vs. mega-ART (5 or more anti-HIV drugs)	New or recurrent AIDS or death	368	390	Clinical practice		CSP; MRC; CIHR

Abbreviations: AIDS, acquired immunodeficiency syndrome; ARDFP, antiretroviral drug–free period; ART, antiretroviral therapy; BMS, bare-metal stent; CABG, coronary artery bypass grafting; CIHR, Canadian Institutes of Health Research; COPD, chronic obstructive pulmonary disease; CSP, Cooperative Studies Program Study; DES, drug-eluting stent; HIV, human immunodeficiency virus; IM, intramuscular; MI, myocardial infarction; MRC, Medical Research Council; PCI, percutaneous coronary infusion; PTSD, post-traumatic stress disorder; VA, Veterans Affairs.

Overall, the most common reason for early study termination was enrollment, which accounted for 5 (45.4%) of terminated studies (Table 3). The next most likely primary reason for termination was safety (n = 2). Futility, funding, benefit, and change in clinical practice were equally likely to be the primary reason for termination, with each accounting for 1 study. In terms of secondary reasons for early termination, funding (n = 1), change in clinical practice (n = 1), and futility (n = 1) were all noted to be a secondary reason in at least 1 study.

Table 3.

Primary Reasons for Early Termination in the 11 Studies That Met Early-Termination Criteria^a

Primary Reason for Early Termination, by Category	No. of Studies   (n = 11)	%
Enrollment	5	45.4
Safety	2	18.2
Funding	1	9.1
Futility	1	9.1
Benefit	1	9.1
Change in clinical practice	1	9.1
Other	0	0.0

^a Studies included in this table were those that met all inclusion criteria and additionally met early-termination criteria.

Decision-making bodies most likely to recommend early termination of a study were the DSMB (n = 6) and CSP or the clinical trial sponsor (n = 2). For 3 of the studies reviewed, no description of the responsible decision makers could be ascertained from the primary publication.

Enrollment

CSP 553.

CSP 553 was designed to evaluate the efficacy of adjuvant docetaxel for preventing relapse when added to the standard of care for high-risk patients after prostatectomy (6). The primary outcome was the proportion of participants with progression-free survival, for which 636 patients would produce enough events to provide 90% power to detect a 15% absolute difference in the hazard ratio. The trial was planned to enroll eligible participants over 4 years with a minimum patient follow-up of 12 months.

After 5 years of ongoing recruitment and random assignment of 298 participants, this study was terminated. The primary reason cited was lower than anticipated recruitment due to a limited number of sites with a large-enough eligible population. The lower than expected sample size reduced the power of this study to detect a benefit of therapy and resulted in lack of statistically significant results, with 74% power in the intent-to-treat population (6).

CSP 557.

CSP 557 was a surgical intervention trial that compared coronary artery bypass grafting and percutaneous coronary intervention with drug-eluting stents in patients with diabetes with severe coronary disease to determine the optimal coronary revascularization (7). The primary endpoint was a composite endpoint of all-cause mortality and myocardial infarction. A sample size of 790 was projected to have 90% power to detect a 40% reduction in the primary endpoint, allowing for 48 months of active recruitment and minimum participant follow-up of 24 months. After approximately 44 months of active recruitment, the study had enrolled 198 participants, which was 25% of target sample size. The study was stopped owing to slow enrollment, leaving it underpowered. At that time, the average follow-up time was 24 months, and no differences between treatment groups were evident in either the primary composite endpoint or all-cause mortality (7).

CSP 571.

CSP 571 examined the efficacy of drug-eluting stents versus bare-metal stents at reducing aortocoronary saphenous vein–bypass graft failure (8). The primary outcome of this study was a composite event outcome comprising cardiac death, target vessel myocardial infarction, or target vessel revascularization at 12 months. An a priori sample size of 519 participants at 25 sites was estimated to produce 123 primary outcome events and thus result in a 90% power to detect a difference. The power calculation was later revised during the trial and increased the sample size to require 762 participants with 122 events to provide 86% power (8).

The first planned interim analysis was done 3 years after the initiation of enrollment, at which time there were 35 adjudicated primary outcome events. The DSMB recommended that the study continue; however, the CSP made the decision to terminate enrollment primarily due to slower than expected enrollment. Funding issues, although not the primary reason, were cited in the publication as a contributing factor and were thus classified as a secondary reason for early termination. The final number of randomizations was 599 participants (with data used for a total of 597) with 102 primary outcomes events, such that the resulting power was 86% (8).

CSP 555.

CSP 555 was conducted to determine if long-acting injectable risperidone would be superior in reducing the risk of hospitalization for up to 2 years as compared with oral antipsychotics in patients with a diagnosis of schizophrenia or schizoaffective disorder (9). The primary outcome was a time-to-event analysis of time to first rehospitalization from randomization though 24 months. The original sample size was 600; however, due to recruitment difficulties, this was decreased such that 450 participants would provide 90% power for analyses of the primary outcome. Enrollment occurred over 3 years, during which 369 participants were randomized. Of those, 362 participants (94.8%) were followed for a mean of 11.3 months (those receiving injectable medication) and 10.8 months (those receiving oral medication). Long-acting injectable risperidone was not superior to oral treatment with respect to the time to hospitalization (P = 0.39) (9).

The authors did not comment in the primary article about systematic early termination of the study by CSP (9); however, it should be noted that this study did not meet its prespecified recruitment target for primary endpoint and also did not show statistically significant difference in treatment arms. The authors also noted that 12% of participants randomized to standard-of-care oral therapy ultimately received long-acting injectable risperidone treatment after an average of 5 months into the trial (9). Due to the approval of competing second-generation, antipsychotic, long-acting injectables (olanzapine pamoate and paliperidone palmitate) by the US Food and Drug Administration in 2009 for the treatment of schizophrenia in adults, the exclusion criteria of the study prohibited the recruitment of eligible patients previously exposed to these agents. This represented a change in clinical practice that was not anticipated at the initiation and planning of this study and could be a contributing reason for lack of recruitment; therefore, this was categorized as the secondary reason for early termination.

CSP 558.

CSP 558 was conducted to evaluate options for effective rehabilitative therapies needed for patients with long-term deficits after stroke (10). Eligible patients were those who had moderate to severe upper-limb impairment as result of a stroke that occurred 6 months or longer before enrollment. Enrolled participants were randomly assigned to either receive robot-assisted therapy, intensive comparison therapy, or usual care. The primary outcome was change in motor function, measured using the Fugl-Meyer Assessment of Sensorimotor Recovery after Stroke, at 12 weeks. The trial was designed to test the superiority of robot-assisted therapy, based on a 3-point change in the Fugl-Meyer score with the use of a 1-sided type I error rate 0.025. The target sample size was 158 total participants to provide 90% power (10). Also unique to this study was the plan for a differential treatment allocation: 26 participants were planned to be assigned to usual care and 66 each to robot-assisted therapy and intensive comparison therapy. Another unique feature was that interim monitoring was preformed using sample variance to determine the maximum information for each study group for possible sample-size readjustment.

Enrollment in the usual-care group stopped after 15 months, when the target information had been attained per protocol. Recruitment to the robot-assisted and intensive-comparison groups continued until the prespecified enrollment period at 24 months. At this time, the total number of participants randomized was 127: 49 to robot-assisted therapy, 50 to intensive comparison therapy, and 28 to usual care (10). The DSMB did not recommend continuing recruitment for these 2 treatment groups, because conditional power to detect a treatment difference for the observed trend was low (2%). The authors also noted that the sample size that would be necessary to achieve maximum information (a total of 262 participants) was also not feasible, and thus futility was determined to be a secondary reason for early termination (10). Ultimately, no statistically significant difference between groups was found; however, mean the Fugl-Meyer score for patients receiving robot-assisted therapy was better than that for participants receiving usual care and worse than that for those receiving intensive comparison therapy (10).

Futility

CSP 535.

The objective of CSP 535 was to determine whether oxandrolone 20 mg/day, compared with placebo, was effective at healing chronic subdermal pressure ulcers in the pelvic region of participants whose wounds did not respond to standard or other optimal treatment (11). The primary outcome was the healing rate of target pressure ulcers. A priori power calculations indicated that 400 participants would be needed to provide 85% power to detect a 15% increase in healing in the intervention (oxandrolone) group.

After 3 years of enrollment, 164 participants had been randomized to a clinical arm (11). The DSMB requested that a futility analysis be performed. Results indicated that the likelihood of achieving a statistically significant result was extremely low (conditional probability of occurrence, P < 0.001), and a recommendation was made for the early termination based on futility. The trial ultimately discontinued enrollment 2 months later, enrolling a total of 212 participants. Oxandrolone was not found to have a benefit over placebo, based on an absolute risk difference in the primary outcome of 1.3% (11).

Benefit

CSP 519.

CSP 519 was conducted to determine whether the integration of mental health care and smoking cessation treatment in veterans with post-traumatic stress disorder improves long-term smoking abstinence rates (12). Eligible participants were randomized to receive 3 months of smoking cessation treatment via integrated care or by referral to VA smoking cessation clinics. The primary outcome measure was 12 month of prolonged abstinence from tobacco between 6 and 18 months after randomization. This outcome was defined as smoking for 7 consecutive days or at least once a week for 2 consecutive weeks or using noncigarette tobacco for 7 consecutive days or at least once a week for 2 consecutive weeks. The predetermined target sample size was 1400, which was designed to have 90% power to detect a 5% difference in smoking abstinence (12).

A lower than expected recruitment rate resulted in 943 randomized participants. The DSMB recommended suspending recruitment because the achieved sample size provided 78% power to detect the hypothesized prolonged abstinence rates, which was felt to be adequate. A verified primary outcome occurred in a total of 42 participants (8.9%) in the integrated care group and 21 participants (4.5%) in the smoking cessation clinic group. This was a statistically significant result that showed that integrated care achieved a better abstinence rate than the smoking cessation clinic, with prolonged confidence interval (P = 0.004) (12).

Funding

CSP 551.

CSP 551 was a noninferiority trial that examined the effect of a triple regimen of disease-modifying antirheumatic drugs (methotrexate, sulfasalazine, and hydroxychloroquine) or etanercept plus methotrexate in participants with rheumatoid arthritis who had active disease despite methotrexate therapy (13). The study planned to enroll 450 participants, which would result in 90% power to detect a between-group difference in the primary outcome. Originally, the primary outcome was the difference in the proportion of patients with a Disease Activity Score for a 28-joint count of 3.2 or less at week 48. Approximately 15 months after enrollment was initiated, the protocol was amended to change the primary outcome from a binary outcome to a continuous outcome. This amendment was made as a result of low enrollment. Ultimately, funding constraints mandated the CSP end participant enrollment before the revised sample-size target of 450 was reached. The final number of participants enrolled in the trial was 353, and a total of 308 participants reached the 48-week time point. However, triple therapy was found to be noninferior to etanercept plus methotrexate (P = 0.002) (13).

Safety

CSP 565.

CSP 565 was conducted to ascertain if combination therapy with angiotensin-converting enzyme inhibitors and angiotensin-receptor blockers was superior to angiotensin-receptor blocker monotherapy in preventing progression of diabetic nephropathy in US veterans (14). Participants were all prescribed an angiotensin-receptor blocker (losartan 100 mg) and then randomly assigned to receive additional angiotensin-converting enzyme inhibitor (lisinopril 10 to 40 mg) daily. The primary outcome was a composite outcome of a 50% decrease in kidney function, end-stage renal disease, or death. A time-to-event analysis was planned such that 1850 participants would need to be enrolled, with a minimum follow-up of 2 years, for the study to have 85% power to detect an 18% relative reduction in the primary endpoint. Two interim analyses were planned at 50% and 75% of total events occurred (14).

In October 2012, the data and safety monitoring committee recommended to the sponsor that the study treatment be stopped, due to safety concerns arising from higher rates of both hyperkalemia and acute kidney injury in the combination-therapy group, along with low conditional power (<5% for the observed trend) to detect a treatment effect on the primary end point.. This safety concern was also highlighted by low conditional power to detect a treatment effect on the primary endpoint if the trial continued. Thus, there was consensus that the absolute risk of serious adverse events appeared to be greater than the potential benefit of reducing primary endpoint events. The CSP accepted the recommendation and terminated the trial after a total of 1,448 participants were randomized, with 152 primary endpoints observed (14).

CSP 560.

CSP 560 was conducted to determine the efficacy of the educational intervention of a comprehensive care management program in reducing the risk for chronic obstructive pulmonary disease hospitalization when added to standard medical care (15). The primary outcome for the study was time to event for a first chronic obstructive pulmonary disease hospitalization. The study authors based sample size on the test of the primary hypothesis with 90% power to reject the null hypothesis for the log-rank test, based on enrolling 960 participants with 1 year of active follow-up and assuming a hazard ratio of 1.37 (or 0.73).

Two years after the initiation of enrollment, the DSMB found a notable imbalance in the mortality rate (n = 28 in the intervention group and 10 in the usual care group) during routine data monitoring. After detailed review of the data, it was recommended that study intervention be stopped immediately. A total of 426 participants, which represented 44% of the planned total, were ultimately enrolled. A total of 70 participants met the criteria for the primary outcome, with 34 in the intervention group and 36 in the usual care group. A post hoc futility analysis was also conducted, and it was determined that the probability was 0.0002 that future data acquired up to the planned termination would have led to rejection of the null hypothesis (15). Although this was not the official reason stated in the publication, it would be reasonable to assume that this study could have also been terminated on the basis of futility.

Change in clinical practice

CSP 512.

CSP 512 was a 2 × 2 factorial, randomized, open-label trial planned to evaluate options for best medical management for advanced human immunodeficiency virus disease with multidrug resistance (16). Enrolled participants were randomized to retreatment with either standard of care or intensive antiretroviral therapy, with treatment either starting immediately or after a 12-week monitored antiretroviral therapy interruption. The primary endpoint in this study was time to first new or recurrent acquired immunodeficiency syndrome–defining illness or death from any cause. The authors noted that a lack of equipoise existed in the United Kingdom, so the protocol at those sites was later amended to permit a choice of only 1 of the 2 randomizations, excluding the antiretroviral therapy interruption randomization.

The study researchers initially planned to enroll 1,700 participants in order to observe 652 disease progression events, resulting in a 22% relative reduction in the hazard of progression, with 93% power (17). During the study, difficulties with recruitment as well as observed outcome and crossover rates prompted investigators to extended accrual and follow-up periods. No change in other assumptions were reported, and sample size was recalculated such that 261 outcomes in 390 participants would provide a study power of 75%. A total of 368 participants underwent at least 1 randomization, and 165 participants met the primary outcome. There was no significant difference in the time to primary outcome across the 4 treatment options (P  =  0.87) (16).

The funding and support for this trial were provided by the VA CSP in collaboration with UK Medical Research Council, and the Canadian Institutes of Health Research. Variation in clinical practice among the countries was noted to be a significant barrier to the successful completion of the trial. The authors also noted that this study was challenged by slow accrual, clinical equipoise, and availability of competing drug development trials (16).

DISCUSSION

The decision to terminate a clinical investigation before meeting the prespecified event rate or recruitment targets is an ever-present issue in clinical research that has been of increasing interest in recent years. In this review, we highlight the finding that clinical trials may be terminated for a variety of reasons that are not necessarily indicative of failure in the planning or execution of a clinical trial or of the inability to achieve the intended goals pursuant to a program’s mission statement. The scientific, clinical, ethical, and financial decisions that go into the decision of early termination are abundant.

The results of this scoping review highlight the trends in VA CSP–sponsored clinical trials that were terminated early within the last 10 years and provide insight on the decision points in a clinical trial that most often result in a formal early-termination decision. Failure to meet recruitment targets consistently stands out as the most common reason for early termination and is an important factor to consider in the planning phase of any clinical trial. First characterized in 1979, “Lasagna’s Law” is the phenomenon in which investigators tend to overestimate of the pool of eligible patients during study planning (18). This results in an apparent decrease in available participants once recruitment is underway. This sharp decrease persists as 1 of the most common challenges facing any clinical trial.

Likewise, the decision to terminate a trial most often came from a recommendation from the DSMB. This committee is vital for monitoring the safety of a trial’s implementation and intervention, and the pivotal role a DSMB plays cannot be overstated. Because of the substantial responsibilities assumed by a DSMB, more information should be provided regarding the unique knowledge each member must possess and, consequently, adequate resources should be provided to each member. Overall, this scoping review highlights the complex decision-making process that must occur in an ongoing fashion during the lifecycle of any RCT.

Additionally, the relative return on investment of any endeavor undertaken by a sponsor may be determined in part by the ability to yield clinically meaningful results. A trial that was terminated early is not necessarily incapable of yielding impactful findings. Trials that end on the basis of safety, overwhelming benefit, or futility can still serve to meet such goals. In this review, there was at least 1 study in which the early termination did not result in a lack of impactful findings but rather contributed significantly to available data and made it clear that combination angiotensin-converting enzyme inhibitor and angiotensin-receptor blocker therapy for the treatment of patients with diabetes is a potential safety issue and should not be recommended (19). Furthermore, this review highlights instances where results of a futility analysis resulting in early termination ultimately still have potential to yield clinically important information for health care providers. In comparison, 1 study reported discontinued RCTs were more likely to remain unpublished, accounting for approximately 55% of all trials that are prematurely discontinued (20). More research into the impact of data produced from terminated trials would help ensure maximal programmatic benefit resulting from the investments of trial participants and sponsors of clinical research.

To our knowledge, this is the first scoping review that was designed to look at the key determining factors that led to the early discontinuation of clinical trials sponsored by the VA CSP. There are, however, some limitations to consider when interpreting the findings. First, this review does not account for those trials that were not systematically early terminated but did not reach prespecified targets at the end of the recruitment period. For these studies, it remains unclear the degree of decision-making made by the sponsor (CSP) to extend recruitment past the prespecified period. This is an area for potential review. Potential mitigation strategies such as broadening inclusion criteria, increasing the number of actively enrolling sites, or increasing awareness among providers are all strategies commonly used in CSP clinical trials. The rates of success of those interventions are not reported here.

In this review, we also focused exclusively on clinical investigations that were published, not accounting for those concluded prior to prespecified recruitment targets and that remained unpublished. Although this aspect has the possibility of introducing publication bias, this review highlights that CSP is dedicated to disseminating any meaningful data collected through all research activities, not only those that were successfully completed. Furthermore, when conducting meta-analyses, another area of scientific research in which publication bias is an issue, researchers are also limited to using published study data to aggregate and from which to make conclusions (21). The data generated from all published trials included in this review will undoubtably be important for future systematic reviews and meta-analyses and underscore the ways in which the CSP infrastructure is equipped to disseminate and implement clinically meaningful study results.

Another limitation of this review is that we did not evaluate trials in which interim data analysis prompted adjustment of sample size, effect size, or primary outcome event rates that ultimately led to the successful completion of the trial. Although not the goal of this review, having information on the factors that lead to this decision point could potentially add to the factors to consider when planning and initiating a clinical trial. Finally, the focus of this scoping review was solely on clinical trials conducted though the sponsorship and collaboration with the CSP. As such, it is unclear how seamlessly these results could translate to other non-federally funded trials. However, the reasons for early termination seem consistent with overall trends observed in studies reported on ClinicalTrials.gov as early terminated, although actual rates can vary (22).

Recommendations

We have identified in this review the various reasons for early termination of a CSP-sponsored RCT, yet we did not assess how the CSP is currently improving its multicenter clinical trial model to mitigate some of the more substantial risks. The ability to effectively overcome challenges may require different approaches that more broadly focus on addressing obstacles at sites that cannot be overcome by individual studies. This realization prompted the establishment of a network of dedicated enrollment sites as a core group of clinical hospitals with full-time research support. The involvement of this network can be helpful to generate systematic site-level solutions to recruit for ongoing CSP studies more efficiently (23). Network efforts produced several new strategies and best practices for common problems in CSP research, which suggest that team-based approaches may help better identify targets and increase efficiencies for clinical trial recruitment.

Another area of potential further investigation identified by this review revolves around the use of health records data to efficiently design and implement clinical trials. In 2006, the Veterans Health Administration began to build a Corporate Data Warehouse, which is a repository for patient-level data aggregated from across the Veterans Health Administration’s national health system (24). Recent literature continues to identify health informatics as a method to improve study efficiency throughout the life cycle of a clinical trial. Aggregated patient-level data have been highlighted as a mechanism to facilitate clinical trial research throughout all phases of a trial. Especially innovative is the application of data to create lists of potentially eligible patients for site study teams to enhance recruitment in real time (25). Despite the potential benefit of data-mining techniques to improve patient care and services, the Corporate Data Warehouse and alternative analytical approaches are largely underused by researchers and clinicians. The reason for the underuse of these data in the reviewed studies could be attributed to the relative newness of this approach at the time of the trials’ conduct and the inability to involve data scientists during the planning stages of these clinical trials. The use of Corporate Data Warehouse data for determining sites with large numbers of potentially eligible participants is currently being undertaken by the CSP. Although trends are not yet available, it is an important step forward to improving the success of CSP clinical trials.

CONCLUSION

By carefully reviewing the findings presented here, sponsors and investigators alike can be made aware of common barriers to successful completion of any clinical research study. Further development of best approaches for analyzing and describing the experience from terminated trials would help ensure maximal benefit from the investments of trial participants and funding agencies in future RCTs.