Adequacy and reporting of allocation concealment: review of recent trials published in four general medical journals

In randomised controlled trials, allocation concealment (separating the process of randomisation from the recruitment of participants) is important for rigorously designed trials.^1-4 In 1996 many major medical journals adopted the CONSORT statement (whereby researchers have to include a short checklist of essential items and a flow diagram when reporting trials),⁵ and this move encouraged the reporting of allocation concealment. We reviewed the prevalence of adequate allocation concealment and its association with the statistical significance of trial results.

Methods and results

We searched by hand four general medical journals (the BMJ, JAMA, the Lancet, and the New England Journal of Medicine) to identify randomised controlled trials published from January 2002 to December 2002. We included articles if the authors reported that participants were randomised and if the trial was published as a full report with the results of the main analyses. We categorised articles according to whether allocation concealment was adequate (the person who executed the allocation sequence was different from the person who recruited participants), inadequate (the person who recruited participants also executed the allocation sequence), or unclear (the article failed to describe how the researchers concealed the allocation). We considered the widely used “sealed envelope” method to be inadequate unless performed by an independent third party. We used a kernel density plot to compare the P values of trials that used adequate concealment methods with those that used inadequate methods; we used P values because these were readily available across most of the trials, which used different statistical methods and outcome measures. Our statistical analyses adjusted for clustering effects by journal.

Among the 234 trials that met the inclusion criteria, allocation concealment was adequate in 132 (56%) and inadequate in 41 (18%); in 61 (26%) the concealment method was unclear. Of the trials whose allocation concealment was considered adequate, 118 used independent allocation (which included using a telephone, fax machine, or pager to a randomisation service); five used sealed envelopes opened by a third party; eight used a computer; and one used a combination of adequate methods. Of the 41 trials whose allocation concealment was inadequate, 39 used sealed envelopes, one selected a card from a pile, and one added the name of the next participant to the randomisation list.

What is already known on this topic

The effect of adequacy of allocation concealment in randomised controlled trials may influence the degree of effect

What this study adds

Despite researchers' acceptance that adequate allocation concealment is important, almost a fifth of trials recently published in major medical journals used inadequate concealment and a quarter failed to describe how the allocation was concealed

For 28 (17%) of the 166 trials published in the CONSORT journals (the BMJ, the Lancet, and JAMA) and 33 (49%) of the 68 trials published in the non-CONSORT journal (the New England Journal of Medicine) we could not discern whether allocation was adequately concealed. We excluded 97 of the original 234 trials from the P value analysis because a single P value was not extractable (for example, unclear primary outcome). The figure shows a trend towards lower P values for inadequately concealed trials (mean = 0.022); P values from trials that used adequate methods were more widely spread around a higher mean value (0.052). The difference between these two mean values was significant (P = 0.045). In a logistic regression, adjusted for sample size and journal, we found that the odds of a trial with inadequate concealment yielding a significant result (P ≤ 0.05) compared with a trial with adequate concealment was 1.8 (95% confidence interval 0.8 to 3.7), thus suggesting that trials using inadequate concealment tend to show significant differences between the groups in the primary outcome more often than trials using adequate concealment.

Figure 1.

Distribution of P values by adequacy of allocation concealment. As the P values were highly positively skewed, the data were transformed using the logit function. The vertical lines represent mean P values for trials using adequate or inadequate concealment

Comments

We found that despite the CONSORT statement more than 40% of trials published in major medical journals either did not use adequate methods or failed to describe how they concealed the allocation. Our results confirm the association found in previous studies that trials using inadequate allocation concealment are more likely to report significant findings than those using adequate concealment. Readers should critically assess the reported methods of allocation concealment.