Quality gaps identified through mortality review

Abstract

Background

Hospital mortality rate is a common measure of healthcare quality. Morbidity and mortality meetings are common but there are few reports of hospital-wide mortality-review processes to provide understanding of quality-of-care problems associated with patient deaths.

Objective

To describe the implementation and results from an institution-wide mortality-review process.

Design

A nurse and a physician independently reviewed every death that occurred at our multisite teaching institution over a 3-month period. Deaths judged by either reviewer to be unanticipated or to have any opportunity for improvement were reviewed by a multidisciplinary committee. We report characteristics of patients with unanticipated death or opportunity for improved care and summarise the opportunities for improved care.

Results

Over a 3-month period, we reviewed all 427 deaths in our hospital in detail; 33 deaths (7.7%) were deemed unanticipated and 100 (23.4%) were deemed to be associated with an opportunity for improvement. We identified 97 opportunities to improve care. The most common gap in care was: ‘goals of care not discussed or the discussion was inadequate’ (n=25 (25.8%)) and ‘delay or failure to achieve a timely diagnosis’ (n=8 (8.3%)). Patients who had opportunities for improvement had longer length of stay and a lower baseline predicted risk of death in hospital. Nurse and physician reviewers spent approximately 142 h reviewing cases outside of committee meetings.

Conclusions

Our institution-wide mortality review found many quality gaps among decedents, in particular inadequate discussion of goals of care.

Introduction

Hospital mortality has been a key quality measure since Nightingale created league tables comparing mortality rates for London hospitals in the mid-19th century,^{1 2} Mortality rates are reported publicly in many jurisdictions as indicators of hospital quality but there continues to be legitimate criticisms of using mortality rates as a measure of quality.^{3 4} These criticisms relate to the effectiveness of risk adjustment to adequately control for differences between hospital and patient characteristics⁵; and the failure of the death rate to inform providers about the nature of quality issues contributing to increased mortality. Despite these concerns, mortality rates will remain an important quality indicator for several reasons: death is a highly visible and usually undesirable outcome, most deaths occur in hospitals^{6 7} and increased mortality rate can be caused by poor-quality care.^8–13

A natural step for hospitals tracking mortality rates is to create processes to investigate deaths and determine if care could be improved. While departmental mortality-review programmes are widespread and have been previously evaluated,^14–19 there are few reports of institution-wide mortality-review programmes.²⁰ An institution-wide process may facilitate understanding of system-wide challenges less visible at the departmental level. However, there are significant obstacles to implementing such a programme including case identification, review methodology, stakeholder engagement and opportunity costs. In order for hospital mortality rates to be a useful metric for improvement, these obstacles must be overcome.

We recently implemented a hospital-wide mortality-review process. In this paper, we report the findings of our review and our experience implementing the review process. This information will be helpful to hospitals that are tracking mortality since quality gaps identified at our hospital likely exist elsewhere. Also, information about how we implemented our review process can guide others interested in building a mortality-review programme.

Methods

Setting and sample selection

The study site was a tertiary-care academic teaching hospital with two campuses. We included every death that occurred at our hospital between 5 September 2013 and 16 December 2013. Patients admitted for day surgery, elective and emergency admissions were all included if they died during the study period. We excluded stillbirths and patients who were admitted as cadaveric donors.

Detection of deaths and screening for quality issues

We built a computer program that used administrative data to detect deaths as they occurred and then created a record in our hospital's Patient Safety Learning System (Datix). Each record included data extracted from our electronic health data repository including patient demographic information, encounter details and patient treatment history at our institution.

The record was then sent to a physician and a nurse from the same admitting service as the patient. For example, if the patient was admitted under general surgery then the nurse and the physician were from general surgery. They independently reviewed each case and created a case vignette describing the patient's care leading up to death. In cases where there was uncertainty about whether care could have been improved, the nurse interviewed the physicians and nurses who were caring for the patient prior to their death to get more details. The nurse and physician reviewers each implicitly judged whether or not:

1. There were any opportunities for quality improvement? We defined a quality-improvement opportunity as any situation where the patient's care was adversely influenced by gaps in standard care processes. This definition was intentionally broad and reviewers were encouraged to include cases if they were uncertain so that all cases with opportunities for improvement would be brought to the next step of the review. The gap did not have to be the cause of death.

2. The death was anticipated? We instructed reviewers to label a death as unanticipated if it was not foreseeable at the time of admission or throughout the hospital stay. Evidence that a death was unanticipated could include active discharge planning, lack of documentation of end-of-life wishes or scheduling elective surgery prior to the death. Unanticipated deaths were all classified as having an opportunity for improvement. Although there are instances of unanticipated death with no opportunity for improvement, we wanted to capture cases where death was a reasonably probable outcome but the medical team did not acknowledge it. When death was judged to be a likely outcome, we considered failure to prepare for it as an opportunity for improvement.

A registered nurse performed oversight of the reviews ensuring that reviews were properly assigned to the correct service. In addition, the oversight nurse conducted some of the reviews for services with a large numbers of deaths (eg, the intensive care unit). If one reviewer thought that the death was unanticipated or the case had an opportunity for quality improvement then the case was taken to the corporate quality review committee (figure 1).

Figure 1.

Review process workflow.

Corporate quality review committee

The committee's mandate was to perform a final classification of whether the death was unanticipated and/or associated with a quality-improvement opportunity and describe the opportunity for improvement. Each decision was reached by group consensus after a presentation of the case summary that had been created by the reviewers. Discussion of each case continued until consensus was reached.

The mortality-review committee met monthly and consisted of nurses and physicians from internal medicine, critical care, psychiatry, emergency medicine and surgery. The institution's Chief Quality and Performance Officer chaired the meetings and all nurses and physicians who work at our institution were invited to attend. We defined the committee as subordinate to our institutional quality-of-care committee, which ensured that all discussions reflecting provider opinions would be protected from freedom of information requests as per local law.

Data collection

We recorded demographic and admission characteristics for all patients in the decedent cohort including age, gender, admission type, number of admissions in the last 6 months, inpatient length of stay and admitting service. We also recorded the risk of death in hospital using a validated risk score described by Escobar et al.²¹

Analysis

We reported the proportion of deaths that were unanticipated, the proportion with quality-improvement opportunities and the frequency of the various types of quality-improvement opportunities. Opportunities for quality improvement were grouped by theme to determine the most common ones. We tested for associations between patient and hospitalisation characteristics and the presence of an opportunity for improvement or unanticipated death. We report p values from t tests for normally distributed variables, Kruskal–Wallis test for variables with highly skewed distributions and χ² tests for categorical variables.

Measures of process success

To assess the implementation of our corporate mortality-review process, we tracked time from case identification to completion of the review by the nurse and the physician, time from case identification to final classification and attendance at review committee meetings. We also documented whether there was consensus in describing the quality-improvement opportunity for each case. Lastly, we estimated time spent by study personnel to implement and maintain our mortality-review system. Our hospital research ethics board approved this study.

Results

Our mortality review included 427 consecutive deaths over a 3-month period. During this time, the hospital admitted 12 819 patients (overall mortality risk=3.3%). The patients who died during the study period were older, more likely to be male, had more admissions in the previous 6 months and had a longer median length of stay than those who did not die (table 1).

Table 1.

Characteristics of patients admitted to hospital during the study period by dead and live status on discharge

	Dead	Alive	Total
	N=427	N=12 392	N=12 819
Gender
Female	206 (48.2%)	7360 (59.4%)	7566 (59.0%)
Male	221 (51.8%)	5032 (40.6%)	5253 (41.0%)
Age at admission (years)
Mean (SD)	74.0 (16.3)	46.1 (27.6)	47.0 (27.8)
Admission type
Elective	13 (3.0%)	3132 (25.3%)	3145 (24.5%)
Emergency	341 (79.9%)	5050 (40.8%)	5391 (42.1%)
Newborn admission	7 (1.6%)	1868 (15.1%)	1875 (14.6%)
Same day admits	4 (0.9%)	1295 (10.5%)	1299 (10.1%)
Urgent	62 (14.5%)	1047 (8.4%)	1109 (8.7%)
Number of admissions per patient in the last 6 months
0	266 (62.3%)	10 140 (81.8%)	10 406 (81.2%)
1	92 (21.5%)	1567 (12.6%)	1659 (12.9%)
2	36 (8.4%)	410 (3.3%)	446 (3.5%)
>3	33 (7.7%)	275 (2.2%)	308 (2.4%)
Length of stay (days)
Median (IQR)	7.0 (3.0–16.0)	3.0 (2.0–6.0)	3.0 (2.0–7.0)
Probability of death in hospital*
Mean (SD)	0.31 (0.20)	0.05 (0.10)	0.06 (0.12)

*Calculated using a validated risk score.²¹

Results of the mortality-review process

The nurse and physician reviewers rated 66 and 89 cases as having an opportunity for quality improvement, respectively while 23 and 24 cases were rated as unanticipated deaths, respectively. Thirty-three deaths (7.7%, 95% CI 5.2 to 10.3%) were rated as unanticipated by at least one reviewer. Figure 1 shows the number of cases rated as having an opportunity for quality improvement or being unanticipated while figure 2 shows the overlap in the reviewer's ratings. In total, 100 (23.4%, 95% CI 19.2 to 27.6%) deaths were judged as having an opportunity for improvement by at least one reviewer. The corporate mortality-review committee reached consensus for all 100 deaths and concluded that 80 cases had opportunities to improve care while 20 were expected deaths with no opportunity to improve care. We held seven committee meetings over 6 months to review these cases. Attendance at the meetings declined over time with 51 in attendance at the first meeting and 14 in attendance at the seventh meeting.

Figure 2.

Proportional Venn diagrams of deaths categorised as having an opportunity for improvement and as unanticipated by the nurse and physician reviewers.

Opportunities for improvement and unanticipated deaths

Of the 100 deaths reviewed at the corporate review committee, 80 (80/427=18.7%, 95% CI 15.3 to 22.7%) were judged to have one or more opportunities for improvement. Deaths with a quality problem had a lower baseline probability of death during their hospitalisation (median 0.24 IQR 0.08–0.40 vs 0.31 IQR 0.15–0.48 p=0.01), a lower probability of being classified as an urgent or emergent case (90% vs 95%, p<0.01) and a longer total length of stay (median 15 days IQR 5–31.5 days vs 6 days IQR 2–15 days p<0.001) compared with deaths with no quality problem (table 2).

Table 2.

Characteristics of decedents with and without opportunities for quality improvement

	Opportunity for improvement	No opportunity for improvement
	N=80	N=347	p Value
Gender
Female	32 (40.0%)	174 (50.1%)	0.11
Age at admission
Mean (SD)	71.4 (13.3)	74.6 (16.9)	0.12
Admission type
Elective	6 (7.5%)	7 (2.0%)	0.03
Emergency	60 (75.0%)	281 (81.0%)
Urgent	12 (15.0%)	50 (14.4%)
Same day admits	2 (2.5%)	2 (0.6%)
Newborn admission	0 (0.0%)	7 (2.0%)
Probability of death in hospital
Median (IQR)	0.24 (0.08–0.40)	0.31 (0.15–0.48)	0.01
Top 10 admitting services
General medicine	24 (30.0%)	137 (39.5%)
Intensive care	6 (7.5%)	65 (18.7%)
Oncology	18 (22.5%)	21 (6.1%)
General surgery	6 (7.5%)	15 (4.3%)
Neurology	1 (1.3%)	15 (4.3%)
Radiotherapy	4 (5.0%)	11 (3.2%)
Family medicine	0 (0.0%)	14 (4.0%)
Malignant Haematology	2 (2.5%)	12 (3.5%)
Orthopaedics	3 (3.8%)	9 (2.6%)
Respirology	2 (2.5%)	9 (2.6%)
Others	14 (17.5%)	39 (11.2%)
Number of admissions in the last 6 months
0	46 (57.5%)	220 (63.4%)	0.55
1	20 (25.0%)	72 (20.7%)
2	9 (11.3%)	27 (7.8%)
>3	5 (6.3%)	28 (8.1%)
Total length of stay (days)
Median (IQR)	15.0 (5.0–31.50)	6.0 (2.0–15.0)	<0.01

Within the 80 cases, there were 97 opportunities for improvement. The most frequent opportunity was: goals of care were not discussed or the discussion was deemed inadequate (26/97, 26.8%, 95% CI 19.0 to 36.4%) followed by delay or failure to achieve a timely diagnosis (8/97, 8.3%, 95% CI 4.2 to 15.4%) and then delay in transfer to long-term care/hospice and uncontrolled pain both with equal frequency (7/97, 7.2%, 95% CI 3.5 to 14.2%) (table 3). Table 4 contains pseudonymised case examples of opportunities for improvement along with processes that were put in place to prevent such errors in the future.

Table 3.

Opportunities for improvement as classified by the corporate mortality-review committee

Opportunity for improvement	Number of occurrences N=97
Goals of care were not discussed or the discussion was inadequate	25
Delay in diagnosis or failure to achieve a diagnosis	8
Uncontrolled pain	7
Inappropriate delay in transfer to hospice or long-term care	7
Developed a pressure ulcer in hospital	5
Did not receive a treatment that was indicated	5
Appropriate specialists were not involved in the patient's care	5
Fall in hospital	4
Delay in surgery that affected patient's outcome and contributed to death	4
Hospital-acquired infection	3
Had multiple ER visits leading to admission and did not receive appropriate treatment	3
Complications of a procedure	2
Admission to hospital was unnecessary. There was no care given in hospital that the patient was not already receiving at their place of residence	2
Inadequate assessment and consideration of preoperative risk	2
Inadequate monitoring of an unstable patient	2
Error made during surgery	2
Other	11

Table 4.

Cases illustrating system issues and processes implemented to mitigate recurrence of the issue

System issue	Case example	Processes implemented to improve quality
Goals of care were not discussed or the discussion was inadequate	70-year-old man with metastatic cancer of unknown primary who was receiving chemotherapy was admitted for febrile neutropenia. He was treated and then discharged to a continuing care hospital because of generalised weakness caused by cancer, chemotherapy and his infection. A week later, he was seen in clinic by the oncologist who noted the patient was declining. The following day the patient returned to hospital with progressive generalised weakness. He said that end-of-life care had never been discussed with him and he did not know his cancer was terminal. There was no record of discussions about end-of-life care in any documentation. He died the following day.	Palliative-care physicians have been incorporated into the cancer clinic. They are available to meet with patients and discuss goals of care and prognosis. Also, a standardised serious illness conversation guide is being implemented on medicine wards so that patients have more opportunities to discuss their prognosis and their wishes for care.
Delay in diagnosis or failure to achieve a diagnosis	75-year-old woman with a history of severe COPD requiring 2 L of home oxygen and severe aortic stenosis presented to ER with 3 days of diffuse abdominal pain, rapid atrial fibrillation and hypotension. An abdominal XR on admission showed right mural thickening of the colon concerning for ischaemic bowel. No treatment was initiated. 24 h later, a CT abdomen was performed showing pancolitis. 48 h into the admission, a urine culture came back positive and the patient was started on antibiotics for the first time. On day 3 of admission, the patient had worsening hypotension and tachycardia. The patient was transferred to intensive care unit and general surgery was consulted for possible ischaemic colitis. The patient died from refractory shock.	A sepsis protocol has been implemented in ER so that patients with signs of septic shock receive broad-spectrum antibiotics early.
Inappropriate delay in transfer to hospice or long-term care and developed pressure ulcer in hospital	74-year-old female with metastatic pancreatic cancer that was progressing on third line chemotherapy presented to ER with constipation and abdominal pain. The constipation was treated and symptoms were controlled. Further chemotherapy was forgone because of progression of disease. The patient remained in hospital for 2 weeks, developed a pressure sore and died in hospital. No application was put in for hospice care.	We have increased collaboration with palliative-care physicians for discharge planning. To prevent pressure sores, we have implemented hourly rounding by all ward nurses.

COPD, chronic obstructive pulmonary disease; ER, emergency room; XR, X-ray.

Of the 427 deaths reviewed, 20 (20/427=4.7%, 95% CI 3.1 to 7.1%) were classified as unanticipated. These cases also had a lower median predicted probability of death in hospital (median 0.08 IQR 0.02–0.18 vs 0.30 IQR 0.15–0.47 p<0.001), a lower probability of being classified as an urgent or emergent case (80% vs 95%, p<0.01) and had a longer median total length of stay (19 days IQR 3.5–37 days vs 7 days IQR 7–13 days p=0.01) (table 5).

Table 5.

Characteristics of decedents with unanticipated and anticipated deaths

		Unanticipated death N=20	Anticipated death N=407	p Value
Gender	Female	8 (40.0%)	198 (48.6%)	0.45
Age at admission	Mean (SD)	70.25 (18.13)	74.15 (16.23)	0.30
Admission type	Elective	2 (10.0%)	11 (2.7%)	<0.01
	Emergency	11 (55.0%)	330 (81.1%)
	Urgent	5 (25.0%)	57 (14.0%)
	Same day admits	2 (10.0%)	2 (0.5%)
	Newborn admission	0 (0.0%)	7 (1.7%)
*Probability of death in hospital	Median (IQR)	0.08 (0.02–018)	0.30 (0.15–0.47)	<0.001
Top10 admitting services	General medicine	4 (20.0%)	157 (38.6%)
	Intensive care	0 (0.0%)	71 (17.4%)
	Oncology	3 (15.0%)	36 (8.8%)
	General surgery	4 (20.0%)	17 (4.2%)
	Neurology	0 (0.0%)	16 (3.9%)
	Radiotherapy	0 (0.0%)	15 (3.7%)
	Family medicine	0 (0.0%)	14 (3.4%)
	Malignant haematology	0 (0.0%)	14 (3.4%)
	Orthopaedics	2 (10.0%)	10 (2.5%)
	Respirology	0 (0.0%)	11 (2.7%)
	Others	7 (35.0%)	46 (11.3%)
Number of admission in the past 6 months	0	15 (75.0%)	251 (61.7%)	0.62
	1	3 (15.0%)	89 (21.9%)
	2	0 (0.0%)	36 (8.8%)
	≥3	2 (10.0%)	31 (7.6%)
Total length of stay (days)	Median (IQR)	19.00 (3.50–37.00)	7.00 (3.00–16.00)	0.01

*Calculated using a validated risk score.²¹

Timeliness of review process

The median time between death and completion of the review by the nurse and the physician was 55 days (IQR 26–80 days). The median time from death to completion of review by the committee was 98 days (IQR 75–118 days).

Resource use

Coordinating the review took a registered nurse 0.2 full-time equivalents during the 4-month study period. Approximately two-thirds of the cases took 5 min for each reviewer to complete while the other one-third of cases took 20 min. The estimated mean time per case per reviewer was 10 min. In total, reviewers spent 142 h reviewing cases, outside of committee meetings.

Discussion

Our mortality-review process found that death was unanticipated at the beginning of the hospitalisation for 1 in 20 deaths and important quality issues were present for 1 in 5 deaths. The most common opportunity for improvement during the study period was goals of care were not being discussed or the discussion was inadequate. We also found that patients with unanticipated deaths or opportunities for quality improvement had lower baseline risk of mortality and had longer total lengths of stay and acute length of stay.

Our finding that the most common opportunity for improvement is absent or inadequate discussion about goals of care implies that medical errors leading to death are relatively less common. This is important, as detecting ‘preventable’ deaths is one of the strong motivating factors for monitoring death rates. On the other hand, it is somewhat predictable given that only 13% of the population in Canada has completed some form of advance care planning.²² In addition, recent research has shown that physicians are not adequately discussing or documenting goals of care in the hospital.^23–26 Our data highlight the need for action on this important issue. Aside from discussing goals of care, our review found numerous other common opportunities for improvement. We have used this data to motivate several quality-improvement projects (table 4).

We found that patients classified as having an unanticipated death or opportunity for quality improvement had a lower baseline risk of dying in hospital and longer length of stay. These findings increase the face validity of our work. Patients classified by clinicians as having ‘unanticipated’ deaths would be expected to have a lower baseline risk by objective measures. Also, the longer a patient is in hospital, the greater the opportunity for errors, which in turn could be amenable to quality improvement. Given that dying patients commonly lack adequately documented goals of care discussions and that these patients tend to remain in the hospital longer suggests that there may be significant opportunity to reduce patient suffering during this difficult time. This has been shown previously.^{23 27} Our results validate this prior work and suggest that continued efforts to improve this aspect of care are required.

Several aspects of our review process helped shift the focus of discussion around mortality review from individual provider behaviour to system issues. A review method, which focuses on assigning responsibility for an unanticipated outcome to an individual, promotes a culture of blame and secrecy that is counterproductive.^{28 29} Our review process helped identify system issues by involving both nurses and physicians from all medical specialties in detecting and evaluating opportunities for quality improvement. Our open door, multidisciplinary approach to mortality review emphasised patient care as a team activity instead of a single individual's responsibility. We argue that the identification of improvement opportunities is enhanced when performed by front-line staff directly involved in service delivery. Furthermore, from a staff engagement perspective, we observed, as others have, that the review process itself contributed to desired behaviours.^{30 31} Using a team approach to quality improvement has been shown to positively affect patient-safety culture, which in turn is linked to patient-safety outcomes.^32–34

Our consensus-driven, multidisciplinary review process required time from clinicians. The estimated mean time per case reviewed was 10 min meaning that 71 h of physician time and 71 h of nurse time were spent reviewing cases outside of committee meetings. This is a significant time commitment from clinicians who have competing demands. The investment must be balanced against the information obtained. Resource usage may be reduced if predictive algorithms can be used to exclude cases with low likelihood of quality problems.

Declining attendance at committee meetings and the long time between death and completion of the review was concerning. Consensus on a particular opportunity for improvement has less weight when only a small number of staff is present. The low attendance may have been the result of poor feedback to the committee about initiatives motivated their review or perhaps the frequent meetings became onerous. The relatively long time between death and review completion was due to inadequate recruitment of clinical reviewers early on in the review process. The future iterations of our review must maintain engagement by giving timely feedback to the committee on actions motivated by their work, and reducing the time lag from death to review completion.

Our study is unique because we examined care gaps that may have contributed to death and care gaps that decreased the quality of death. In this way, our study is unique and important. However, there are some limitations of our work. Our review was conducted in a single teaching hospital. This limits the generalisability of our findings. There were several possible sources of bias. First, it is possible that a reviewer was involved in a case they were reviewing. Considering the size of our hospital, this was an unlikely occurrence and using two reviewers to screen each case ensured at least one objective review in the event that one was involved in the case. Hindsight bias may have occurred because reviews often occurred weeks after the death and further information such as biopsy results may point to a missed diagnosis that was not evident at the time of death. Finally, bias may have been introduced by the subjective nature of assessments. We addressed this concern by having a multistage review. While imperfect, subjective assessment by multiple reviewers is often used in research evaluating quality of care.^{8 9 35–37}

In conclusion, our mortality-review process engaged front-line clinical staff in reviewing consecutive deaths. The process shed light on systemic issues, giving us a broad overview of trends in quality-improvement opportunities and providing an evidence base to guide quality improvement. Our findings related to discussions of ‘goals of care’ have been documented elsewhere. Efforts should be made across the health system to improve this aspect of care. Future iterations of our mortality-review process need to focus on keeping clinical staff engaged by ensuring that discussions about cases occur soon after each death and by taking explicit and visible action to address the quality gaps that are uncovered.