Skip to main content
NCBI home page
Postgraduate Medical Journal logoLink to Postgraduate Medical Journal
. 2005 Oct;81(960):647–652. doi: 10.1136/pgmj.2004.030619

Monitoring surgical and medical outcomes: the Bernoulli cumulative SUM chart. A novel application to assess clinical interventions

G Leandro 1, N Rolando 1, G Gallus 1, K Rolles 1, A Burroughs 1
PMCID: PMC1743380  PMID: 16210461

Abstract

Background: Monitoring clinical interventions is an increasing requirement in current clinical practice. The standard CUSUM (cumulative sum) charts are used for this purpose. However, they are difficult to use in terms of identifying the point at which outcomes begin to be outside recommended limits.

Objective: To assess the Bernoulli CUSUM chart that permits not only a 100% inspection rate, but also the setting of average expected outcomes, maximum deviations from these, and false positive rates for the alarm signal to trigger.

Methods: As a working example this study used 674 consecutive first liver transplant recipients. The expected one year mortality set at 24% from the European Liver Transplant Registry average. A standard CUSUM was compared with Bernoulli CUSUM: the control value mortality was therefore 24%, maximum accepted mortality 30%, and average number of observations to signal was 500—that is, likelihood of false positive alarm was 1:500.

Results: The standard CUSUM showed an initial descending curve (nadir at patient 215) then progressively ascended indicating better performance. The Bernoulli CUSUM gave three alarm signals initially, with easily recognised breaks in the curve. There were no alarms signals after patient 143 indicating satisfactory performance within the criteria set.

Conclusions: The Bernoulli CUSUM is more easily interpretable graphically and is more suitable for monitoring outcomes than the standard CUSUM chart. It only requires three parameters to be set to monitor any clinical intervention: the average expected outcome, the maximum deviation from this, and the rate of false positive alarm triggers.

Full Text

The Full Text of this article is available as a PDF (150.3 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Adam R., Cailliez V., Majno P., Karam V., McMaster P., Caine R. Y., O'Grady J., Pichlmayr R., Neuhaus P., Otte J. B. Normalised intrinsic mortality risk in liver transplantation: European Liver Transplant Registry study. Lancet. 2000 Aug 19;356(9230):621–627. doi: 10.1016/s0140-6736(00)02603-9. [DOI] [PubMed] [Google Scholar]
  2. Altman D. G., Royston J. P. The hidden effect of time. Stat Med. 1988 Jun;7(6):629–637. doi: 10.1002/sim.4780070602. [DOI] [PubMed] [Google Scholar]
  3. Aylin Paul, Best Nicky, Bottle Alex, Marshall Clare. Following Shipman: a pilot system for monitoring mortality rates in primary care. Lancet. 2003 Aug 9;362(9382):485–491. doi: 10.1016/S0140-6736(03)14077-9. [DOI] [PubMed] [Google Scholar]
  4. Belle S. H., Detre K. M., Beringer K. C. The relationship between outcome of liver transplantation and experience in new centers. Liver Transpl Surg. 1995 Nov;1(6):347–353. doi: 10.1002/lt.500010602. [DOI] [PubMed] [Google Scholar]
  5. Bridgewater Ben, Grayson Anthony D., Jackson Mark, Brooks Nicholas, Grotte Geir J., Keenan Daniel J. M., Millner Russell, Fabri Brian M., Jones Mark, North West Quality Improvement Programme in Cardiac Interventions Surgeon specific mortality in adult cardiac surgery: comparison between crude and risk stratified data. BMJ. 2003 Jul 5;327(7405):13–17. doi: 10.1136/bmj.327.7405.13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Dyer Owen. Heart surgeons are to be rated according to bypass surgery success. BMJ. 2003 May 17;326(7398):1053–1053. doi: 10.1136/bmj.326.7398.1053-a. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Fine Leon G., Keogh Bruce E., Cretin Shan, Orlando Maria, Gould Mairi M., UK Cardiac Surgery Experience How to evaluate and improve the quality and credibility of an outcomes database: validation and feedback study on the UK Cardiac Surgery Experience. BMJ. 2003 Jan 4;326(7379):25–28. doi: 10.1136/bmj.326.7379.25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Gallus G., Mandelli C., Marchi M., Radaelli G. On surveillance methods for congenital malformations. Stat Med. 1986 Nov-Dec;5(6):565–571. doi: 10.1002/sim.4780050603. [DOI] [PubMed] [Google Scholar]
  9. Grigg O. A., Farewell V. T., Spiegelhalter D. J. Use of risk-adjusted CUSUM and RSPRT charts for monitoring in medical contexts. Stat Methods Med Res. 2003 Mar;12(2):147–170. doi: 10.1177/096228020301200205. [DOI] [PubMed] [Google Scholar]
  10. Jacobson Bobbie, Mindell Jenny, McKee Martin. Hospital mortality league tables. BMJ. 2003 Apr 12;326(7393):777–778. doi: 10.1136/bmj.326.7393.777. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Karam Vincent, Gunson Bridget, Roggen Francine, Grande Luis, Wannoff Wolfgang, Janssen Magda, Guckelberger Olaf, Delvart Valérie, Bismuth Henri, Höckerstedt Krister. Quality control of the European Liver Transplant Registry: results of audit visits to the contributing centers. Transplantation. 2003 Jun 27;75(12):2167–2173. doi: 10.1097/01.TP.0000080271.20145.07. [DOI] [PubMed] [Google Scholar]
  12. Lovegrove J., Valencia O., Treasure T., Sherlaw-Johnson C., Gallivan S. Monitoring the results of cardiac surgery by variable life-adjusted display. Lancet. 1997 Oct 18;350(9085):1128–1130. doi: 10.1016/S0140-6736(97)06507-0. [DOI] [PubMed] [Google Scholar]
  13. Nashef S. A., Roques F., Michel P., Gauducheau E., Lemeshow S., Salamon R. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg. 1999 Jul;16(1):9–13. doi: 10.1016/s1010-7940(99)00134-7. [DOI] [PubMed] [Google Scholar]
  14. Poloniecki Jan, Sismanidis Charalambos, Bland Martin, Jones Paul. Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups. BMJ. 2004 Jan 29;328(7436):375–375. doi: 10.1136/bmj.37956.520567.44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Roques F., Michel P., Goldstone A. R., Nashef S. A. M. The logistic EuroSCORE. Eur Heart J. 2003 May;24(9):881–882. doi: 10.1016/s0195-668x(02)00799-6. [DOI] [PubMed] [Google Scholar]
  16. Sergeant P., Blackstone E., Meyns B. Can the outcome of coronary bypass grafting be predicted reliably? Eur J Cardiothorac Surg. 1997 Jan;11(1):2–9. doi: 10.1016/s1010-7940(96)01032-9. [DOI] [PubMed] [Google Scholar]
  17. Sergeant P., Meyns B. La critique est aisée mais l'art est difficile. [Criticism is easy but the method is hard; comment]. Lancet. 1997 Oct 18;350(9085):1114–1115. doi: 10.1016/s0140-6736(97)22042-8. [DOI] [PubMed] [Google Scholar]
  18. Spiegelhalter David, Grigg Olivia, Kinsman Robin, Treasure Tom. Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. Int J Qual Health Care. 2003 Feb;15(1):7–13. doi: 10.1093/intqhc/15.1.7. [DOI] [PubMed] [Google Scholar]
  19. Steiner S. H., Cook R. J., Farewell V. T., Treasure T. Monitoring surgical performance using risk-adjusted cumulative sum charts. Biostatistics. 2000 Dec;1(4):441–452. doi: 10.1093/biostatistics/1.4.441. [DOI] [PubMed] [Google Scholar]
  20. Tekkis Paris P., McCulloch Peter, Steger Adrian C., Benjamin Irving S., Poloniecki Jan D. Mortality control charts for comparing performance of surgical units: validation study using hospital mortality data. BMJ. 2003 Apr 12;326(7393):786–788. doi: 10.1136/bmj.326.7393.786. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Thompson William, Hutwagner Lori, Kolczak Margarette. Monitoring mortality events associated with individual physicians and practices. Lancet. 2003 Aug 9;362(9382):417–418. doi: 10.1016/S0140-6736(03)14098-6. [DOI] [PubMed] [Google Scholar]
  22. Treasure Tom, Utley Martin, Bailey Alan. Assessment of whether in-hospital mortality for lobectomy is a useful standard for the quality of lung cancer surgery: retrospective study. BMJ. 2003 Jul 12;327(7406):73–73. doi: 10.1136/bmj.327.7406.73. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Williams S. M., Parry B. R., Schlup M. M. Quality control: an application of the cusum. BMJ. 1992 May 23;304(6838):1359–1361. doi: 10.1136/bmj.304.6838.1359. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Wynne-Jones K., Jackson M., Grotte G., Bridgewater B. Limitations of the Parsonnet score for measuring risk stratified mortality in the north west of England. The North West Regional Cardiac Surgery Audit Steering Group. Heart. 2000 Jul;84(1):71–78. doi: 10.1136/heart.84.1.71. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Postgraduate Medical Journal are provided here courtesy of BMJ Publishing Group

RESOURCES