Extent and Reporting of Patient Non-Enrollment in Influential Randomized Clinical Trials, 2002–2010

Because they assign patients to treatment conditions, randomized clinical trials (RCTs) offer unparalleled internal validity for drawing inferences about the efficacy of a medical treatment. Whether such inferences can be generalized is not always clear because many RCTs enroll a low and unrepresentative proportion of all patients.^1–6 The challenges of judging the clinical utility of clinical trial results are increased by poor reporting: Gross and colleagues’ study of trials published in leading medical journals from 1999–2000 found that only 28% reported the proportion of screened patients who were enrolled.⁷ These deficiencies may have been ameliorated in the past decade because the CONSORT statement was revised in 2001 to require more complete information on the enrollment process in reports of clinical trials⁸, and because many treatment research fields have been showing greater concern about generating knowledge that better informs clinical practice. Accordingly, the present study assessed the extent to which low enrollment rates are still characteristic of widely-cited clinical trials, and whether reporting of enrollment information has improved.

Methods

A Web of Science search was employed to identify the 20 most influential English-language RCTs for each of 14 prevalent chronic disorders (alcohol dependence, Alzheimer’s, breast cancer, colorectal cancer, COPD, depression, diabetes, drug dependence, HIV/AIDS, hypertension, ischemic heart disease, lung cancer, nicotine dependence and schizophrenia) published from 2002–2010 (see supplementary material for search terms and citations returned). We sorted the results on citations per year rather than total citations so that recently published trials would still have the chance to rank as influential. Top-cited articles that were not RCTs (e.g., major literature reviews) were excluded.

The final dataset comprised 280 studies (20 studies for each of 14 conditions). Raters double-coded the studies on the number of patients with the disorder of interest who were screened for trial eligibility, the number who were eligible, and the number of eligible patients who agreed to enroll. When available, the reason for non-enrollment also was recorded. For these studies, we recorded the number of non-enrollments that were due to participants not meeting study eligibility criteria, the number excluded for other reasons (e.g., administrative errors), and the number of eligible participants who refused to participate (this included individuals who initially refused to participate and those who initially agreed, but then did not return for the start of the study).

Results

Only 51.8% (n=145) studies provided sufficient information to allow calculation of the non-enrollment rate. These RCTs had a mean non-enrollment rate of 40.1% (SD = 23.7%). For 6 of the 14 diseases, the influential trials included at least one study with a non-enrollment rate over 90%.

No association emerged between year of publication and the proportion of patients not enrolled (r = −.08, p = .372). However, year of publication was positively associated with adequate reporting of enrollment information (OR = 1.19, p = .003). In 2002, only 45% of the trials reported enrollment information, but this proportion rose to 75% by 2010.

Only 35.0% (n =98) of studies provided sufficient information to categorize reasons for non-enrollment. In these studies, an average of 27.3% of participants did not meet eligibility criteria, 11.2% refused participation, and 3.7% were not enrolled due to other reasons.

Comment

Highly-cited clinical trials do not enroll an average of 40.1% of identified patients with the disorder being studied, primarily due to eligibility criteria. Low enrollment rates can lower external validity because, by definition, eligibility criteria create trial research samples that differ from real-world patient samples. The larger the proportion of patients not-enrolled, the more likely it is that the results of the study will not reflect what the intervention would produce in front-line clinical practice. Although exclusion criteria are sometimes essential in trials, including to protect patient safety, we add our voices to those of others who have suggested that treatment researchers employ them as minimally as possible and only with good justification.

On a more positive note, between 2002 and 2010, the proportion of clinical trials reporting complete enrollment information increased from 45% to 75%. Improved reporting may reflect the accrued influence of the CONSORT guidelines, as more authors and editors become aware of them, as well as the impact of numerous studies and editorials raising concerns about unrepresentative research samples.

We close with an important caution. Gandhi and colleagues⁹ found that publications of trial results tend to underreport the number of exclusion criteria that were in the approved protocol. Further, in some trials insufficient effort is put into tracking data on non-enrollment.⁷ Therefore, even though we have identified high rates of non-enrollment, our results may nonetheless understate the degree to which this is a reality of current clinical trial research.