Skip to main content
Journal of Epidemiology and Community Health logoLink to Journal of Epidemiology and Community Health
. 1998 May;52(5):318–323. doi: 10.1136/jech.52.5.318

Clinical significance not statistical significance: a simple Bayesian alternative to p values

P R Burton, L C Gurrin, M J Campbell
PMCID: PMC1756718  PMID: 9764283

Abstract

OBJECTIVES: To take the common "Bayesian" interpretation of conventional confidence intervals to its logical conclusion, and hence to derive a simple, intuitive way to interpret the results of public health and clinical studies. DESIGN AND SETTING: The theoretical basis and practicalities of the approach advocated is at first explained and then its use is illustrated by referring to the interpretation of a real historical cohort study. The study considered compared survival on haemodialysis (HD) with that on continuous ambulatory peritoneal dialysis (CAPD) in 389 patients dialysed for end stage renal disease in Leicestershire between 1974 and 1985. Careful interpretation of the study was essential. This was because although it had relatively low statistical power, it represented all of the data that were available at the time and it had to inform a critical clinical policy decision: whether or not to continue putting the majority of new patients onto CAPD. MEASUREMENTS AND ANALYSIS: Conventional confidence intervals are often interpreted using subjective probability. For example, 95% confidence intervals are commonly understood to represent a range of values within which one may be 95% certain that the true value of whatever one is estimating really lies. Such an interpretation is fundamentally incorrect within the framework of conventional, frequency- based, statistics. However, it is valid as a statement of Bayesian posterior probability, provided that the prior distribution that represents pre-existing beliefs is uniform, which means flat, on the scale of the main outcome variable. This means that there is a limited equivalence between conventional and Bayesian statistics, which can be used to draw simple Bayesian style statistical inferences from a standard analysis. The advantage of such an approach is that it permits intuitive inferential statements to be made that cannot be made within a conventional framework and this can help to ensure that logical decisions are taken on the basis of study results. In the particular practical example described, this approach is applied in the context of an analysis based upon proportional hazards (Cox) regression. MAIN RESULTS AND CONCLUSIONS: The approach proposed expresses conclusions in a manner that is believed to be a helpful adjunct to more conventional inferential statements. It is of greatest value in those situations in which statistical significance may bear little relation to clinical significance and a conventional analysis using p values is liable to be misleading. Perhaps most importantly, this includes circumstances in which an important public health or clinical decision must be based upon a study that has unavoidably low statistical power. However, it is also useful in situations in which a decision must be based upon a large study that indicates that an effect that is highly statistically significant seems too small to be of practical relevance. In the illustrative example described, the approach helped in making a decision regarding the use of CAPD in Leicestershire during the latter half of the 1980s.

 

Full Text

The Full Text of this article is available as a PDF (112.7 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Burton P. R. Helping doctors to draw appropriate inferences from the analysis of medical studies. Stat Med. 1994 Sep 15;13(17):1699–1713. doi: 10.1002/sim.4780131702. [DOI] [PubMed] [Google Scholar]
  2. Burton P. R., Walls J. Selection-adjusted comparison of life-expectancy of patients on continuous ambulatory peritoneal dialysis, haemodialysis, and renal transplantation. Lancet. 1987 May 16;1(8542):1115–1119. doi: 10.1016/s0140-6736(87)91674-6. [DOI] [PubMed] [Google Scholar]
  3. Gokal R., Jakubowski C., King J., Hunt L., Bogle S., Baillod R., Marsh F., Ogg C., Oliver D., Ward M. Outcome in patients on continuous ambulatory peritoneal dialysis and haemodialysis: 4-year analysis of a prospective multicentre study. Lancet. 1987 Nov 14;2(8568):1105–1109. doi: 10.1016/s0140-6736(87)91544-3. [DOI] [PubMed] [Google Scholar]
  4. Greenland S. Re: "P values, hypothesis tests, and likelihood: implications for epidemiology of a neglected historical debate". Am J Epidemiol. 1994 Jan 1;139(1):116–118. doi: 10.1093/oxfordjournals.aje.a116929. [DOI] [PubMed] [Google Scholar]
  5. Hughes M. D. Reporting Bayesian analyses of clinical trials. Stat Med. 1993 Sep 30;12(18):1651–1663. doi: 10.1002/sim.4780121802. [DOI] [PubMed] [Google Scholar]
  6. Lilford R. J., Braunholtz D. The statistical basis of public policy: a paradigm shift is overdue. BMJ. 1996 Sep 7;313(7057):603–607. doi: 10.1136/bmj.313.7057.603. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Lilford R. J., Thornton J. G., Braunholtz D. Clinical trials and rare diseases: a way out of a conundrum. BMJ. 1995 Dec 16;311(7020):1621–1625. doi: 10.1136/bmj.311.7020.1621. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Pocock S. J., Hughes M. D. Estimation issues in clinical trials and overviews. Stat Med. 1990 Jun;9(6):657–671. doi: 10.1002/sim.4780090612. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Epidemiology and Community Health are provided here courtesy of BMJ Publishing Group

RESOURCES