Table 1.
Literature on Physician Assessments of the Preventability of Hospital Deaths
| Study Patient Population (Reference) | Interrater Reliability (κ or ICC of a Single Review) for Preventability or Negligence | Preventability Ratings | Sources Used for Review | Distribution of Reviewer Ratings |
|---|---|---|---|---|
| Rand mortality rate study Dubois and Brook (1988), Dubois and Brook (1987) 182 inpatient deaths for patients admitted for stroke, myocardial infarction, or pneumonia | Average κ 0.2–0.3 | 546 reviews (3 reviews for each of the 182 deaths) 33 percent of cases were rated as “possibly preventable” based on a single review | Medical records and dictated hospital summaries | Results were dichotomized and original data are no longer in the possession of the authors (personal communication, March 1, 2004) |
| Harvard medical practice study Brennan et al. (1991), Leape et al. (1991)∼175 hospital deaths reviewed, with 154 being rated as adverse event deaths | κ∼0.14* (Estimated based upon a reported κ in the study of 0.24 for two reviews) | ∼9 percent of deaths were rated as due to negligence (rating of >“3” by 2 reviewers on the 6-point “due to negligence” scale) | Medical records | Negligence determined based upon a score of >“3” on a 6-point scale of self-reported confidence that medical management caused the AE's: 1, little or no evidence; 2, slight evidence; 3, not quite likely; 4, more likely than not; 5, strong evidence; 6, virtually certain Results were dichotomized and original data are no longer retrievable (personal communication with the authors [December 2003]) |
| Utah/Colorado studyThomas et al. (2000, 2002) ∼55 hospital deaths reviewed with 38 being rated as adverse event deaths | κ = 0.19–0.23* 3 reviewers per case for a subgroup of 500 cases. | ∼6 percent of deaths were rated as due to negligence (rating of >“3” by a review on the 6-point “due to negligence” scale) | Medical records | Same 6-point scale as the HMPS. Required a confidence score of >3 for determination of negligence. Almost all deaths had only a single review. Only 2 percent of ratings were “6”* |
| VA preventable death study Hayward and Hofer (2001) 179 hospital deaths. After excluding 68 deaths that were receiving comfort care only, 111 hospital deaths underwent full review | ICC = 0.22 (ICC was 0.34 for 2 reviewers) | 23 percent of active care deaths were rated as at least “possibly” preventability & 6 percent were rated as at least “probably” preventable (based upon a single review) | Medical records | Probability that the death was preventable by optimal care was rated in two ways: On a 5-point scale (1 = definitely; 2 = probably; 3 = uncertain/possibly; 4 = probably not; 5 = definitely not) And on a 0–100 percent scale |
ICC, Intraclass correlation coefficient.
Intra-class correlation not cited in the article (see Glossary).