Confidence intervals should be used in reporting trials

Editor—The dispute between Barber and Thompson,¹ who are advocating a t test, and Williams, Cohen, and Russell, who are advocating a Mann-Whitney U test,² has its roots in the use of P values rather than confidence intervals. If Williams et al had reported their results as a confidence interval for the difference in mean cost, as recommended for the results of clinical trials published in the BMJ, the question would not have arisen. The sample size is surely large enough for the large sample Normal comparison, which does not require data to follow a normal approximation and to which the t method approximates, to be valid, even with such highly skewed data. This gives a confidence interval for the difference in cost of secondary treatment (routine minus open access) equal to –£180 to +£238, the point estimate being £29, and the same P value as the t test.

Barber and Thompson are correct in that the Mann-Whitney U test makes an overall comparison of distributions in the two groups, in terms of both shape and location, and does not specifically test for a difference in means. Although it is often described as a test of the difference between medians, this is only the case if we can assume that the two distributions being compared have exactly the same shape. Under these circumstances, it would be a test for the difference between two means also. This is not the case here, as the standard deviations are different. Thus Barber and Thompson are correct in arguing that a significant Mann-Whitney U test implies only a difference in distribution, not mean. Cost data typically have very uneven distributions, with many observations having the same low value and a few observations being very high. Distributions can differ considerably and yet have the same or similar means. I agree with Barber and Thompson that Williams et al have misinterpreted the Mann-Whitney result.

Williams et al say that the Mann-Whitney U test was only an interim analysis, but I could find no mention of this in their original paper. The actual observed difference is only 5% of the mean for the standard treatment, so the statement by Williams et al that analysis to be published elsewhere confirms that open access greatly reduces secondary care costs is very surprising. I look forward to seeing it.