Editor—The analysis by Dr Foster Ltd of death rates in hospital trusts is so flawed that the NHS should ignore it.1-1 Standardised hospital mortality ratios are inappropriate for this exercise and difficult to interpret. They were originally public health measures intended to apply to whole area populations that are relatively static.
Patients admitted to a hospital do not constitute a predefined population; this population is arbitrary and depends heavily on admissions policy and the availability of support and other community services locally. Furthermore, standardised mortality ratios cannot be used to compare different areal units.1-2
The report does not give managers and clinical leaders any clue about how to improve quality. Should they look for rogue surgeons or killer nurses or shortcomings in clinical care? If the latter, then what's new? We were doing that anyway.1-3,1-4 The study has served only to divert managers in “bad” hospitals into answering hysterical queries from the press; to induce self righteous complacency in “good” hospitals; and to encourage lawyers to chase after every death, expected or otherwise.
Patients don't benefit either. How does knowing a hospital's mortality index help? This index is a crude estimate of the a priori average risk of dying while in hospital. Who does it apply to? It applies only to statistically “average” patients—an esoteric concept for risk modelling enthusiasts, but of no help to individual patients, who need an estimate of their individual chances of a successful outcome.
Nor can the analysis be improved. Clever statistical manipulation of the dataset cannot get us out of the mess resulting from the inversion of the logical process of rational epidemiological analysis. The study started with data that happened to be there; then the researchers did some sophisticated (and therefore seductively persuasive) analysis, suggested a few answers (if you torture a dataset enough it will confess to whatever you want), and then asked “What possible question is this the answer to?” It certainly does not answer the question “Which hospitals have poor quality care as judged by mortality?”
Ideally we should start with the question, refine it as far as possible, determine what data we need to answer it with an acceptable degree of validity, collect the relevant data, and then analyse them. There are other and better approaches to quality measurement.1-5 Blunderbuss analysis of a dataset collected for administrative purposes is unhelpful.
References
-
1-1.Kmietowicz Z. Hospital tables “should prompt authorities to investigate.”. BMJ. 2001;322:127. . (20 January.) [PMC free article] [PubMed] [Google Scholar]
-
1-2.Court BV, Cheng KK. Pros and cons of standardised mortality ratios. Lancet. 1995;346:1432. doi: 10.1016/s0140-6736(95)92449-3. [DOI] [PubMed] [Google Scholar]
-
1-3.Department of Health. A first class service. Quality in the new NHS. London: DoH; 1999. [Google Scholar]
-
1-4.Department of Health. An organisation with a memory: report of an expert group on learning from adverse events in the NHS. London: DoH; 2000. [Google Scholar]
-
1-5.Mohammed MA, Cheng KK, Rouse A, Marshall T. Bristol, Shipman and clinical governance: Shewhart's forgotten lessons. Lancet. 2001;357:463–466. doi: 10.1016/s0140-6736(00)04019-8. [DOI] [PubMed] [Google Scholar]
