Mortality variations as a measure of general practitioner performance: implications of the Shipman case

Changes in the organisation of the NHS or in the management of health professionals are often made within the context of inquiries into specific medical catastrophes rather than through a measured process of policy formulation. Examples include changes in the scrutiny of psychiatric services after the Ely report into malpractice at a Cardiff hospital¹ and, more recently, changes in doctors' self regulation after the high mortality in Bristol paediatric cardiac surgery.² Harold Shipman's conviction for 15 murders committed during the course of his work as a general practitioner will result in several changes in the management of UK general practice. Some external monitoring of primary care is already routine. In the wake of the Shipman case there is pressure to introduce formal monitoring of mortality in general practitioners' patients.³ Here we consider the potential value of such an approach to identifying aberrant practice.

Methods and results

The average list size per general practitioner in England was 1866 in 1998.⁴ The crude mortality in Britain is currently 1.1% a year. The table shows the number of deaths expected according to practice size, together with the minimum number of deaths for which the lower limit of a 99% confidence interval from the Poisson distribution would exclude the national rate. As there are some 9000 practices in England, the use of 99% confidence intervals would, by definition, necessitate the intense investigation of the 0.5% (approximately 45) practices annually above this limit. In larger practices 30-40 excess deaths a year would not be detected as statistically exceptional. Shipman's practice list of 3600 would allow an excess of 18 deaths a year above the average to pass as unremarkable, which is more than the 15 murders over three years he is currently convicted of (a murder rate of five per year) and also more than the high estimate of 175 murders over more than a decade, although repetitively elevated mortality, even if below the threshold, could, with extra effort, receive additional monitoring. Even if such monitoring were restricted to deaths that occurred outside hospital, random variation would mask considerable illegitimate mortality.

Comment

These findings highlight the difficulty of providing quantitative measures of quality, or of malpractice, within relatively small populations. Our estimates for the expected ranges of deaths within general practices are highly conservative, since different compositions of age, social class, and ethnic group in general practitioners' lists will strongly influence the numbers of deaths: rates could vary several fold between areas according to levels of deprivation,⁵ and local factors such as communicable disease and the presence of hospices or old peoples' homes will also markedly influence mortality. Therefore, either more than 0.5% of practices would require intense scrutiny each year or the mortality level at which investigation is introduced would need to be raised, allowing the potential concealment of an even greater number of illegitimate deaths. Major changes in mortality in the smallest practices may reach the upper threshold more easily, but the trend towards discouraging singlehanded practice would make this less relevant. In contrast, aberrations in larger practices would be concealed by the wide confidence intervals and by the dilution of individual malpractice within grouped mortality statistics.

Whatever the appeal of seeking hard end points, such as mortality, in judging the quality of care, in the case of the performance of individual general practitioners these may only be a distraction from the more difficult task of measuring the more mundane attributes of care. Routine monitoring of mortality in general practice lists would have had limited benefit in highlighting the Shipman case, and therefore may have little value in preventing future occurrences. Such monitoring would create both false security and a statistical cacophony of false positive suspicion. Strengthened avenues for informal intelligence of aberrant practice from patients, relatives, other doctors, practice staff, pharmacists, coroners, undertakers, and others are the key protection against lethal doctors.

Table.

Expected and threshold^* numbers of deaths each year in general practice patients according to practice list size

List size (No of patients)	No of deaths among patients
	Expected	Threshold*
1 000	11	21
2 000	22	36
3 600	40	58
5 000	55	76
10 000	111	140
15 000	166	201
20 000	222	263

^*Number of deaths for which the lower limit of 99% confidence interval excludes the national rate.