Editor—Connolly and Cowling and Hedley raise several technical issues relating to our article. Connolly says that we have not shown that the number of outliers we identified is above that expected by chance alone. When considering the annual analyses, by chance alone one would expect 5% of consultants to be outliers. As table 2 in our paper shows, in any year, we found three times as many outliers as predicted by chance alone.
Cowling and Hedley indicate that our method may be difficult for clinical audit teams to apply routinely. As indicated in our paper, we believe that it is premature to consider routine implementation of our approach without further rigorous prospective testing. Nevertheless, the issue of implementation is important; as we indicate in our discussion, national bodies need to be engaged in this type of work.
Cowling and Hedley also point out there is a 3% chance of an outlier consultant appearing as an outlier in three or more years. Although this statement serves as a useful reminder about the role of chance in determining outliers, the underlying assumptions of the binomial model they used—constant probability and independent events—make the usefulness of this calculation unclear. In general, such assumptions apply only really in simple artificial demonstrations and as such are questionable in complex situations such as the one we describe. Furthermore, in our data set not all consultants appear in all five years, thus providing a further basis for questioning the underlying assumptions.
We thank Cowling and Hedley for pointing out that the inclusion of a 1/2 in our computation of the mean of the √χ2, would provide a better approximation of the mean. So by using (κ - 1/2) (where κ is the degrees of freedom) and not κ, the mean reduces slightly and the revised number of statistical outliers for table 2 is now 17, 18, 21, 16, and 11 for 1991-2 through to 1995-6, respectively. Ledward is now an outlier in four of the five years analysed.
Competing interests: None declared.