Early termination of cardiovascular trials as a consequence of poor accrual: analysis of ClinicalTrials.gov 2006–2015

Abstract

Objectives

To present a snapshot of experimental cardiovascular research with a focus on geographical and temporal patterns of early termination due to poor accrual.

Setting

The Aggregate Analysis of ClinicalTrials.gov (AACT) database, reflecting ClinicalTrials.gov as of 27 March 2016.

Design

The AACT database was searched for all cardiovascular clinical trials that started from January 2006 up to December 2015.

Results

Thirteen thousand and seven hundred twenty-nine cardiovascular trials were identified. Of these, 8900 (65%) were classified as closed studies. Globally, 11% of closed trials were terminated. This proportion varied from 9.6% to 14% for trials recruiting from Europe and Americas, respectively, with a slightly decreasing trend (p=0.02) over the study period. The most common reason for trials failing to complete was poor accrual (41%). Intercontinental trials exhibited lower figures of poor accrual as the reason for their early stopping, as compared with trials recruiting in a single continent (28% vs 44%, p=0.002).

Conclusions

Poor accrual significantly challenges the successful completion of cardiovascular clinical trials. Findings are suggestive of a positive effect of globalisation of cardiovascular clinical research on the achievement of enrolment goals within a reasonable time frame.

Strengths and limitations of this study.

• To identify cardiovascular clinical trials terminated early because of poor accrual, this study relies on the most updated release of the Aggregate Analysis of ClinicalTrials.gov database and on automated and replicable text mining techniques.

• By analysing early termination due to poor accrual by continent of recruitment over time, the study shows that poor accrual exhibits geographical heterogeneity with lower figures for intercontinental trials.

• In interpreting the results it must be acknowledged that ClinicalTrials.gov is representative of registered trials recruiting from Europe and Americas. On the contrary, it cannot be ruled out that some selection bias occurred for trials recruiting from Asia and Oceania since they are increasingly registered in other regional registries.

Introduction

Clinical trials may terminate for a variety of reasons, some of which correctly envisaged in the study protocol^{1 2} and others unforeseen and attributable to failures in the trial conduct. Examples of appropriate reasons for terminating a trial prior to completion include unequivocal evidence of futility or harm.³ In this case early stopping may prevent additional patient exposure to ineffective or harmful treatments and limit further expenditure of resources on unsuccessful approaches. Conversely, premature termination due to poor accrual clearly reflects a failure of the trial process since trials completed with less than expected enrolment are usually delayed and unable to meet their intended objectives meaningfully.

Clinical trials that fail to recruit successfully or in a timely manner their target number of participants pose ethical, financial and statistical issues.⁴ They are a major concern in research areas where the acuity of the disease and the quickly evolving treatment paradigm render original research questions obsolete if not implemented in a timely manner. This partially explains why poor accrual and its consequences have been extensively investigated in some areas, mostly in oncology^5–7 and less in others.

Unsuccessful recruitment has been recently acknowledged^{8 9} as the leading factor in cardiovascular trial failure, generating resource-consuming underpowered trials that provide only inconclusive data with no or little return on investments.

The past decade has seen a fast growth in cardiovascular research.¹⁰ Documenting impact of poor accrual on early termination of cardiovascular trials, also in terms of geographical and temporal trends, is important as it can provide insights to inform future trials.

The development of clinical trial registries, such as ClinicalTrials.gov, affords a remarkable opportunity to better understand the extent of early termination in cardiovascular trials. In the beginning, ClinicalTrials.gov was set to increase public awareness of clinical trials, and its systematic evaluation was hindered by a lack of access to the complete, annotated data. This registry now serves as a mandatory repository for information on most clinical studies run under US regulations, and registration with ClinicalTrials.gov is mandatory for publishing study results in many peer-reviewed journals. In addition, the database for Aggregate Analysis of ClinicalTrials.gov (AACT),¹¹ a high-quality, searchable database of the information contained in ClinicalTrials.gov, is now publicly available.

By leveraging information available through the AACT database, reflecting ClinicalTrials.gov as of 27 March 2016, we present a snapshot of the last 10 years of cardiovascular research, with a focus on geographical and temporal patterns of early termination due to poor accrual.

Materials and methods

The AACT database, reflecting ClinicalTrials.gov as of 27 March 2016, is the data source. A comprehensive dictionary for the data elements in AACT is available online.¹²

The Medical Subject Headings (MeSH) code field in the AACT database was queried for at least one of the following MeSH codes: A07 cardiovascular system; C14 cardiovascular diseases; D27.505.954.411 cardiovascular agents (including antiarrhythmia/antihypertensive/cardiotonic/fibrinolytic/natriuretic/vasoconstrictor/vasodilator agents, calcium/potassium/sodium channel blockers, cardioplegic/sclerosing solutions and nitric oxide donors); E01.370.370 diagnostic techniques, cardiovascular (including angiography, angioscopy, blood circulation time, blood flow velocity, blood pressure/volume determination, capillary fragility, carotid intima-media thickness, heart function tests, laser Doppler flowmetry, microscopic angioscopy, plethysmography, pulse wave analysis and tilt-table test); E04.100 cardiovascular surgical procedures (E04.100.376 cardiac surgical procedures, E04.100.700 reperfusion and E04.100.814 vascular surgical procedures); G09.330 cardiovascular physiological phenomena (including ventricular function, myocardial contraction, blood circulation, neovascularisation, haemodynamics and cardiovascular deconditioning); and H02.403.429.163 cardiology, with the aim to identify all studies dealing with a cardiovascular condition. The final sample was limited to all such studies, started in the past decade — from January 2006 to December 2015 — and classified as ‘Interventional’, therein referred to as cardiovascular trials.

Study characteristics that have been considered are year of start date, phase (available only for drug and biologics trials), type of intervention, number of arms, primary purpose, endpoint classification, intervention model, masking, number and location of the facilities involved, and number of patients enrolled.

The rate of studies that have stopped recruiting or enrolling participants early and will not start again, defined as ‘terminated’ studies, was estimated on studies that are no longer recruiting participants because they have enough participants already, have ended or have been stopped for some reason, and referred to as ‘closed’ studies. The latter definition embraces also withdrawn studies, intended as studies closed with no patients enrolled.

Termination was used as the dichotomous outcome variable of a conditional inference tree built to detect associations with the study characteristics listed above.

Study duration was calculated as the difference between start date and completion date, only for closed studies and ‘actual’ completion date type.

To establish the reason for stopping, text mining was applied to the narrative field of reason for study termination. The following text preprocessing procedures were applied in the following order: conversion to lower case, removing numbers, removing punctuation, removing stop-words, stemming words, removing stemmed uninformative terms (‘studi’, ‘clinical’, ‘trial’, ‘patient’, ‘subject’, ‘approach’, ‘termin’, ‘stop’), treating the stemmed terms ‘accrual’ and ‘recruit’ as synonyms of ‘enrol’, stripping white space and building a sequence of two adjacent words from the text (bigrams).

A randomly selected sample of about 10% of terminated trials was extracted to assess the accuracy of automated text mining in identifying poor accrual as compared with manual review (gold standard).

AACT was downloaded in pipe delimited text format and managed in R V.3.3.0 software. The location of each facility, expressed as country in the AACT database, was ascribed to a region and to one of the five continents through R packages ‘countrycode’ and ‘rworldmap’. Text mining was performed with ‘tm’ R package. Tree-based models were fit with ‘party’ R package.

Results

In a 10-year time span, from 2006 to 2015, 13 729 cardiovascular trials were identified.

As shown in figure 1, cardiovascular trials were prevalently run in Northern America (41%), followed by Eastern and Western Europe (25%) and Eastern Asia (8%). The regions that together contributed to less than 0.5% were Middle, Eastern and Western Africa and Central Asia.

Figure 1.

Geographical distribution of the facilities involved in cardiovascular trials, 2006–2015.

Single-centre studies accounted for the majority (60%) of cardiovascular trials. Among intercontinental trials (n=1089), the most likely collaborations were bilateral between Americas and Europe (32%), trilateral between Americas, Asia and Europe (14%), quadrilateral between Americas, Asia, Europe and Oceania (11%), and worldwide (8%).

Overall, the most common interventional model was parallel design (60%). Of studies reporting number of arms (n=13 419), 27% were single-armed and 57% had two arms. Multiarm studies (n=9735) were typically randomised (89%) and blinded (38% double-blinded and 20% single-blinded). Phase II and III trials together accounted for 4500 studies and 5052 recorded the phase as ‘not applicable’. The most frequently observed intervention was drug (n=6334), followed by device (n=2736) and procedure (n=1405).

Eight thousand and nine hundred cardiovascular trials (65%) were classified as closed studies. Of these, 11% were terminated. The reason for termination was missing for 141 studies.

According to the classification tree shown in figure 2, termination was significantly associated with facilities’ location (p<0.001). In addition, the type of intervention had an effect on the rate of early termination (lowest for behavioural and dietary supplement interventions as compared with all other types, p<0.001) for trials run in Americas and multinational trials run in Americas and Europe or in Americas, Asia and Europe. Among trials run in Europe, parallel trials were more likely to terminate early than trials with other intervention models (7.3% vs 11.7%, p=0.012).

Figure 2.

Conditional inference tree predicting early termination. Retained variables: facility location (Americas (AM), Americas Europe (AM-EU), Americas Asia Europe (AM-ASIA-EU), Asia, Europe (EU), Other), year of start date, intervention type (behavioural (behav), biological (bio), device, dietary supplement (diet), drug, genetic (gene), procedure (proc), radiation (rad), other), and intervention model (crossover (Cr), factorial (Fac), parallel (paral), single group (sing)).

Low figures of early termination for trials run in Asia and for trials recently started should be interpreted with caution since they may suffer from a selection bias.

The single stemmed term ‘enrol’ occurred in 404 reasons for study termination and it was most associated with the adjectives ‘poor’, ‘low’ and ‘slow’, making poor accrual the most common reason for trials failing to complete. The second most common substantive was ‘fund’, with 85 occurrences and it was associated with the term ‘lack’, followed by ‘sponsor’, with 70 occurrences, and associated with the term ‘decis’. Table 1 reports the most frequently used bigrams in the reason for termination field. A network graph in figure 3 shows the most frequent relations (ie, correlation >0.15) between the (stemmed) terms ‘enrol’, ‘fund’, ‘sponsor’, ‘safety’ and ‘interim’, and the other words in the corpus of reason for termination text data.

Table 1.

Stemmed bigrams occurring in the reason for stopping field at least 10 times: terminated cardiovascular trials, 2006–2015

Bigrams	N
slow enrol	78
low enrol	69
lack enrol	41
poor enrol	36
difficult enrol	25
enrol rate	25
interim analysi	23
lack fund	23
safeti concern	23
insuffici enrol	22
sponsor decis	20
left institut	15
busi decis	14
inabl enrol	14
unabl enrol	14
detail descript	11
safeti issu	10

Furthermore, the term ‘enrol’ occurred as the reason for stopping of 90 withdrawn trials and 18 suspended trials. Table 2 reports the proportion of trials terminated for poor accrual over terminated trials by study characteristic.

Figure 3.

Network graph showing the most frequent relations (ie, correlation >0.15) between the (stemmed) terms ‘enrol’, ‘fund’, ‘sponsor’, ‘safety’ and ‘interim’, and the other words in the corpus of reason for termination text data. Solid lines identify associations with a correlation >0.15 and dotted lines identify tautological associations.

On a random sample of 85 terminated trials, where the reason for termination was manually reviewed by one of the authors (IB), the prevalent reason was poor accrual (47%). Stopping occurred for lack of funding and sponsor decision in 8% and 6% of the sampled trials, respectively. Appropriate reasons for termination based on internal or external evidence of futility or lack of safety accounted for 26% of all reasons. The sensitivity and specificity of the stemmed term ‘enrol’ in identifying poor accrual were 97.5% and 100%, respectively.

Terminated trials actually involved a total of 114 609 enrolled subjects (plus 44 513 scheduled) and more than 10 500 facilities worldwide. Eighteen thousand and eight hundred eight patients and 2319 facilities actually took part in trials terminated because of poor accrual.

The patterns of premature termination shown in figure 4 varied from 9.6% to 14% (p<0.001) among the two most contributing continents, Europe and Americas, and were slightly decreasing over the study period (p=0.02).

Figure 4.

Proportion of early terminations by continents of recruitment over time. Cardiovascular trials, 2006–2015.

Intercontinental trials exhibited comparable figures of termination and lower figures of unsuccessful accrual as the reason for their early stopping, as compared with intracontinental trials (13% vs 11%, p=0.24, termination, of whom 28% vs 44%, p=0.002, were due to poor accrual, respectively). Figure 5 shows the time trend of poor accrual for intercontinental trials (p value for trend=0.02, p value for the difference=0.001).

Figure 5.

Proportion of early terminations due to poor accrual for trials recruiting from a single continent or from different continents over time. Cardiovascular trials, 2006–2015.

A median actual duration of 24 months was observed both on closed trials and on the subset of terminated trials. Only a slight difference emerged in the third quartile, ranging from 38 to 39 months for closed and terminate trials, respectively. For this reason, time trends were truncated at the last biennium under study.

Poor accrual accounted for more than 40% of reasons for early stopping in trials recruiting from Americas and Europe. This proportion was by far inferior for all other continents of recruitment, although caution must be taken in interpreting these figures since they rely on small absolute numbers (table 2).

Table 2.

Characteristics of closed cardiovascular interventional trials – 2005–2015: proportion of terminated trials (% terminated) and proportion of trials terminated because of lack of accrual (% poor) by study characteristics

		N	% Terminated	p Value‡	% Poor	p Value‡
Year	2006–2007	1078	7.0	<0.001	42.3	<0.001
	2008–2009	1893	6.6		43.0
	2010–2011	1955	6.0		43.3
	2012–2013	1596	4.9		32.8
	2014–2015	624	4.2		44.2
Facility location*	Africa	32	3.2	<0.001	100.0	<0.001
	Americas	3769	14.0		46.3
	Americas Asia Europe	129	13.2		17.6
	Americas Asia Europe Oceania	89	7.9		14.3
	Americas Europe	256	18.0		34.7
	Asia	1079	5.6		33.3
	Europe	2454	9.6		40.4
	Oceania	71	18.3		30.8
	Other	374	4.2		50.8
	Missing	647	5.9		23.7
Phase	Phase 0	81	9.9	<0.001	32.9	0.012
	Phase 1	736	11.3		29.8
	Phase 1/Phase 2	415	15.1		36.5
	Phase 2	1542	11.7		44.4
	Phase 2/Phase 3	231	13.2		36.9
	Phase 3	1419	11.0		51.5
	Phase 4	2997	8.2		46.6
	NA	1479	8.6		14.3
Intervention type	Behavioural	603	4.5	<0.001	1.8	<0.001
	Biological	294	10.2		33.3
	Device	1679	11.9		42.7
	Dietary supplement	266	5.3		50.0
	Drug	4309	13.4		39.8
	Genetic	26	7.7		0.0
	Procedure	44	13.6		45.1
	Radiation	784	10.5		83.3
	Other	895	5.4		47.9
No of arms	1	2474	11.2	0.004	42.0	<0.001
	2	4725	11.7		43.5
	>2	1466	8.5		29.8
	Missing	235	13.6		34.4
Primary purpose	Basic science	250	4.8	<0.001	25.0	0.002
	Diagnostic	491	11.0		50.0
	Health services research	197	4.1		37.5
	Prevention	1008	9.5		37.5
	Screening	46	4.3		0.0
	Supportive care	280	10.3		75.0
	Treatment	6264	12.1		39.7
	Missing	364	8.0		48.3
Allocation	Non-randomised	1052	11.0	0.986	31.0	0.155
	Randomised	6039	11.0		41.9
	Missing	1809	11.2		44.1
Endpoint classification	Bioavailability	18	5.6	<0.001	0.0	0.155
	Bioequivalence	38	7.9		66.7
	Efficacy	2668	10.7		47.2
	Pharmacodynamics	121	11.6		35.7
	Pharmacokinetics	124	4.0		0.0
	Pharmacokinetics/Dynamics	111	12.6		28.6
	Safety	625	9.1		36.8
	Safety/Efficacy	3811	12.8		37.2
	Missing	1384	8.5		47.5
Masking	Double-blind	2775	12.6	<0.001	36.7	0.119
	Open-label	4798	11.2		43.1
	Single-blind	1287	7.1		46.2
	Missing	40	15.0		33.3
Intervention model	Crossover	713	7.4	0.009	39.6	0.148
	Factorial	156	8.3		38.5
	Parallel	5246	11.4		41.5
	Single group	2727	11.4		40.8
	Missing	58	15.5		33.3
No of enrolled	(0, 100]	5521	14.0	<0.001	45.3	<0.001
	(100, 1000]	2832	6.3		27.4
	(1000, 20 000]	467	5.6		10.7
	>20 000	53	3.8		0.0
	Missing	27	19.2		40.0
No of facilities (per study)	1	4934	10.3	<0.001	44.4	<0.001
	(1, 10]	1876	14.4		44.1
	(10, 50]	1047	11.5		35.0
	(50, 100]	204	14.2		17.2
	>100	191	9.4		16.7
	Missing	648	5.9		23.7
No of conditions (per study)	1	6703	11.8	0.068	41.3	0.228
	2	1584	8.2		44.0
	>2	613	13.8		32.9
Conditions†	Acute coronary syndrome	205	12.2	-	48.3	-
	Atrial fibrillation	380	13.4		37.2
	Cardiovascular diseases	481	5.4		34.6
	Coronary artery disease	609	8.7		50.9
	Coronary disease	341	10.0		32.3
	Heart failure	644	13.8		53.9
	Hypertension	901	10.2		29.3
	Multiple myeloma	620	15.8		42.9
	Myocardial infarction	216	9.7		38.2
	Myocardial ischaemia	414	10.9		20.0
	Stroke	477	10.9		34.6
Overall		8900	11.1		41.1

*Facility location describes the study-level location of enrolling sites. It results in a single continent for single-centre trials and multicentre multinational trials and in a combination of continents for intercontinental trials.

†The number of MeSH conditions sums to 13 501 across the 8900 closed trials. Only the conditions exceeding an absolute frequency of 200 are listed.

‡χ² or Fisher test p values.

Conclusion

Clinical trials are a key step in advancing new therapeutic concepts for the management of chronic cardiovascular diseases from the research setting to the clinical practice.¹³ Despite this achievement, the successful recruitment of the targeted number of participants in a given time frame remains a significant challenge to clinical trials.¹⁴ Delayed and abandoned trials represent a waste of scarce human and economic resources, which may slow the advancement of medical progress or reduce its timely impact on patient health and well-being.

Therefore, a thorough understanding of the nature of trial enrolment patterns, from an overview on aggregate data through to the working of individual trials, is of paramount importance.

This study contributes to a growing body of research on early termination due to poor accrual in cardiovascular trials,^{2 3 9} providing further insights into geographical and temporal patterns.

Among study findings, about 11% of all closed trials were terminated, which is consistent with other cross-sectional studies using the ClinicalTrials.gov registry.^{9 15} When restricted to phase II and phase III trials together, the proportion of early termination (14%) was slightly inferior to that reported on a recent study⁴ based on the National Library of Medicine clinical trial registry (19%).

Consistent with existing literature,^{8 9} the most common reason for cardiovascular trials failing to complete was poor accrual (41%). Common logistical reasons were related to lack of funds and sponsor decision. Since reason for termination was automatically characterised by word patterns through text mining, we may just speculate that the concepts underlying ‘safety concerns’, ‘safety issue’ and ‘interim analysis’ pertain to appropriate termination, possibly on data safety and monitoring board advice, because of excessive toxicity or in accord with early stopping rules.

Almost all trials are dependent on the willingness of patients and professionals to give their time and effort to participate, and we estimated that the human dimension of poor accrual involved more than 8000 patients and the staff of 2000 sites.

The overall patterns of early termination and those concerned with poor accrual exhibited geographical variations. Cardiovascular trials recruiting in the two most contributing continents exhibited the highest and lowest proportion of early termination due to unsuccessful accrual in Americas and Europe, respectively, across all the study period.

Nevertheless, time trends should be interpreted with caution, since the decrease seen in the last biennium of the evaluation may be an artefact, reflecting both a delay in updating trial status in ClinicalTrials.gov and a selection of trials with a short duration and less likely to suffer from low accrual. However, comparisons between continents of recruitment per year appear to be appropriate and reliable.

It is interesting to notice that intercontinental trials, particularly those in bilateral collaboration with Americas and Europe, suffer more from early termination and less from termination due to lack of accrual, as compared with studies completely run in only one of the participating continents. Clearly, intercontinental trials are often resource-intensive, large-scale randomised controlled clinical trials, recruiting thousands of patients from large numbers of trial sites (ie, megatrials).

On the one hand, this probably allows to recognise the achievement of enrolment goals within a reasonable time frame as one of the numerous advantages of globalisation of cardiovascular clinical trials, often evolving into megatrials,¹⁶ but on the other hand it highlights that other organisational and financial issues challenge this process. Intercontinental trials represent a valuable attempt to harmonise the generation of clinical evidence across continents. Their implementation is particularly promising in device trials,¹⁷ where the comparison of procedural outcomes, using similar devices in different environments, makes it possible to disentangle the effects of practice patterns and procedural technique on clinical results.

Some study limitations must be acknowledged. First is the generalisability of study results. A recent investigation¹⁸ on trends in global clinical trial registration from 2005 to 2013, based on International Clinical Trials Registry Platform data, reports that  trials conducted in Northern America and Latin America and Caribbean were almost exclusively registered in ClinicalTrials.gov. Also European trials were predominantly registered in ClinicalTrials.gov rather than in the EU Clinical Trials Register. Conversely, trials conducted in Oceania and Asia were increasingly registered in other regional registries such as the Australian New Zealand Clinical Trials Registry and the Japan Primary Registries Network, respectively.

As a consequence, study results concerning Europe and Americas can be regarded as representative of all registered trials recruiting from these continents. Moreover, differences in the extent of early termination and of poor accrual can be interpreted as genuine. On the contrary, it cannot be ruled out that some selection bias occurred for trials recruiting from Asia and Oceania. Therefore, the extent of early termination for trials run in these two continents may be underestimated.

Furthermore, the identification of trials terminated for poor accrual relied on an automated process with an estimated sensitivity of 97.5%; thus, the extent of poor accrual may be underestimated.

Poor accrual is a significant barrier to the successful and timely completion of clinical trials and to the advancement of medical knowledge. Although this issue may be addressed in a variety of ways, a global collaborative perspective on the planning and conduct of some cardiovascular clinical trials should be further encouraged.¹⁹