Twenty Years of Evidence on the Outcomes of Malpractice Claims

Abstract

Two decades of social science research on the outcomes of medical malpractice claims show malpractice outcomes bear a surprisingly good correlation with the quality of care provided to the patient as judged by other physicians. Physicians win 80% to 90% of the jury trials with weak evidence of medical negligence, approximately 70% of the borderline cases, and even 50% of the trials in cases with strong evidence of medical negligence. With only one exception, all of the studies of malpractice settlements also find a correlation between the odds of a settlement payment and the quality of care provided to the plaintiff. Between 80% and 90% of the claims rated as defensible are dropped or dismissed without payment. In addition, the amount paid in settlement drops as the strength of the patient’s evidence weakens.

Introduction

Physicians regularly hear that juries make irrational decisions and that the settlements made in the shadow of these verdicts bear no relationship to the evidence [1]. As a result, physicians understandably fear the mere filing of a claim against them means they are destined to suffer an unjust punishment. However, two decades of social science research on the outcomes of medical malpractice claims show the current level of dread is unwarranted. Malpractice outcomes bear a surprisingly good correlation with the quality of care as judged by other physicians.

Over the past two decades, investigators have collected data on the fairness of the outcomes of medical malpractice litigation by comparing the outcomes of medical malpractice cases with the outcomes recommended by physician reviewers. This article reviews that body of research, looking first at the data on jury verdicts and then at the findings of studies that have examined both settlements rates and settlement size. It also compares the rate of agreement between juries and physician reviewers with the rate of agreement found when multiple physicians are asked to evaluate the appropriateness of a patient’s care. It concludes with a discussion of the research remaining to be done.

Jury Verdict Outcomes

Over the past three decades, several investigations have evaluated the accuracy and fairness of jury verdicts in medical malpractice lawsuits (Fig. 1). The findings have been remarkably consistent. Physicians win 80% to 90% of the jury trials with weak evidence of medical negligence, approximately 70% of the toss-up cases, and 50% of the cases with strong evidence of medical negligence [18].

Fig. 1.

The odds of a plaintiff’s verdict are correlated with reviewer ratings of the strength of the claim.

Taragin et al. published the largest of these studies in 1992 [24], reviewing 976 malpractice jury verdicts between 1978 and 1992 from the files of a large insurance company that insured approximately 60% of the physicians in New Jersey. As part of its normal procedure, the insurance company asked one or more physicians to evaluate each claim shortly after its receipt. The reviewers rated each claim as “defensible,” “indefensible,” or “unclear.” The investigators found physicians won 79% of the cases that had been rated “defensible.” They also won 70% of the cases rated as “unclear” and 58% of the cases considered “indefensible.”

In 2006, Studdert et al. at the Harvard School of Public Health reported their observations concerning jury verdicts [23]. Physicians in the relevant specialties were hired and trained by the investigators to examine each record and to determine whether the claimants’ injuries had in fact been caused by medical error. No separate category was used for unclear cases. The study found physicians won favorable verdicts in 91% of the trials in which the medical care had been deemed proper and 57% of the cases in which the reviewer believed the physician had committed an error. Interestingly, winning plaintiffs received lower damage awards for similarly severe injuries when the evidence of negligence was rated as weak by the reviewers. Plaintiffs had been awarded an average of $326,009 in no-error cases and $765,486 in the cases believed to involve medical error.

In addition to these two benchmark studies, several smaller studies have been reported. Farber and White examined the files of 252 lawsuits against a single large hospital filed between 1977 and 1989, 13 of which were tried to a verdict [8]. Physicians won all 13 of the jury trials, although one lawsuit involved care that had been rated as poor and three cases were considered borderline. In a later study, the same authors found physicians had won all of the trials involving care that had been rated as “good” by the hospital’s reviewers, two of four trials involving medical care rated as “ambiguous” (50%), and two of the four trials in which the defendant’s care had been rated as “bad” (50%) [9].

Peeples et al. examined 18 jury verdicts from North Carolina and found physicians won 90% of the trials in which the defendant’s care had been rated as good, 83% of the trials involving care rated as uncertain, and 50% of the trials involving care rated as poor [16]. The remaining three studies reached similar results (Fig. 1) [5, 12, 14].

Analysis of Jury Verdicts

Without exception, existing data have shown the weaker the evidence of medical negligence, the greater the likelihood of a verdict in favor of the physician in a malpractice trial. Physicians win approximately 80% to 90% of the cases reviewers believe they should win, approximately 70% to 80% of the cases rated as toss-ups, and roughly 50% of the cases deemed by peer reviewers to have strong evidence of negligence [18]. Only one study found a materially higher plaintiff win rate in cases involving strong evidence of negligence, but the ratings in that study were made by nonphysicians (Fig. 1) [5].

The data also suggest jurors are even more skeptical of malpractice claims than peer reviewers. Doctors consistently win approximately 50% of the cases that physician reviewers have concluded they should lose and 70% to 80% of the cases with unclear or ambiguous evidence of negligence [18]. This success rate suggests the presence of variables that systematically favor medical defendants in the courtroom [18]. Juries may be skeptical of patients who sue their doctors. This is consistent with the social science research finding that jurors have been listening to the unrelenting complaints of physicians and politicians over the past 20 years and sympathize with them [10, 25]. Evidence suggests defendants are much more likely than plaintiffs to hire experienced attorneys [11, 16]. The evidence shows that in medical malpractice trials, attorney experience is important [11]. Juries may take the burden of proof very seriously in medical malpractice cases, giving physicians the “benefit of the doubt” when the experts for both sides are credible [18]. These factors may make it difficult for malpractice plaintiffs to win even strong cases and could explain why malpractice claimants are nearly twice as likely to win in bench trials (trials before judges) than in jury trials [3, 4].

Despite these findings, physicians may continue to be troubled by the finding that patients win some lawsuits that medical experts believe they should lose. In the largest study, plaintiffs won 21% of the cases rated as defensible [24]. However, most of the verdicts in that study occurred before the media awareness campaign that has been promoted by organized medicine during the past decade. The more recent 2006 study by Studdert et al. found the plaintiff win rate in cases deemed by reviewers to lack merit had decreased to 9%, although this study did not have a separate category for the borderline cases [23]. If borderline cases had been included, the rate of disagreement between jury verdicts and expert opinions in the Studdert study would have been even lower.

Professionals themselves disagree on the quality of peer performance; a disagreement rate of 25% to 30% has been reported among professionals who make complex decisions in various fields [6]. The disagreement rate for scientists engaged in peer review was 25%, the rate for employment interviewers was 30%, for psychiatrists diagnosing psychiatric illness it was also 30%, and for physicians diagnosing physical illness it was 23% to 34% [6]. Judges participating in sentencing councils also have a disagreement rate of over 30% [7]. This rate is nearly identical to that of physicians who evaluate the quality of care provided by other physicians [13, 16]. From this perspective, the consistency of 80% to 95% found between jury verdicts and peer review of medical negligence cases is remarkably high.

Because interobserver disagreement is inevitable, and because the discrepancy rate observed in malpractice cases is much lower than the rate found in other settings, virtually all the disagreement between juries and reviewers occurring in cases with relatively weak evidence of negligence may be the product of routine interobserver variability. Further improvement may be possible if courts experiment with innovative efforts to make scientific proof more comprehensible to lay jurors such as early jury instructions, jury note-taking, ongoing jury deliberations, and jury submission of questions. Courts should continue their search for better ways to distinguish legitimate experts from illegitimate ones and experiment with the use of court-appointed experts. However, because the agreement rate between juries and physician reviewers in low-odds medical malpractice cases is already high, such efforts can result in only modest improvements.

Settlement Outcomes

The fairness of settlements in medical malpractice has also been investigated. In all but one of the 12 published studies, the likelihood of a settlement payment and the size of any payment have been correlated with the strength of the evidence alleging negligence. Many studies have categorized the quality of the care provided to the plaintiff as defensible, unclear, and not defensible [8, 9, 15, 16, 20–22, 24]. Three studies used only used two categories, defensible and not defensible [1, 2, 19]. The three-category studies reveal a sharper correlation between the evidence of liability and settlement outcome than the two-category studies. That is because the two-category studies probably force the borderline cases into one of the two available categories, thereby affecting the results (Figs. 2, 3). Taken together, the studies show medical malpractice settlement rates go up stepwise with the strength of the patient’s evidence of substandard care (Fig. 4).

Fig. 2.

Three-category studies show a strong correlation between the likelihood of settlement and the quality of care.

Fig. 3.

Two-category studies show a weaker correlation between the likelihood of settlement and quality of care.

Fig. 4.

This graph shows the correlation between likelihood of settlement and quality of care when results of the reliable 2-category and 3-category studies are combined. The two middle categories of the 6-category Studdert et al. [23] data were combined.

In addition, studies have consistently found a positive correlation between the strength of the patient’s evidence and the size of settlement payments [1, 2, 8, 9, 22, 23]; in all but one study [24], the correlation is statistically significant. Writing in 1991, Farber and White, for example, found a mean settlement of $14,109 for a case with “good” care, $146,160 for “ambiguous” care, and $203,209 for “bad” care [8]. In a later study, their econometric analysis found a predicted settlement of $7112 for good-quality care, $91,008 for ambiguous care, and $177,320 for poor-quality care [9].

Many authors have acknowledged their findings reflect favorably on the civil justice system [8, 21, 23, 24]. When Taragin and his colleagues found plaintiffs received a settlement payment in 91% of the cases in which medical care was judged negligent, in 59% of the cases in which liability was unclear, and in 21% of the cases in which the medical care was defensible, they concluded “the defensibility of the case and not the severity of patient injury predominantly influences whether any payment is made. Our findings suggest that unjustified payments are probably uncommon” [24]. Although payment had been made in 21% of the “defensible” cases, the authors attributed this finding to flaws in their rating process, stating, “First, the determination about physician care was made very early after a claim was generated and may have been inaccurate as more information became available. Second, a physician-based review process may be biased toward assessing physician performance in the physician’s favor. Third, the insurance company may err toward an initial determination of physician care as defensible to avoid unnecessary [settlement] payments” [24].

Farber and White also found “a strong relationship between care quality and disposition” [8]. Settlements were least likely when the care received by the claimant was judged “good” (24.2%), more likely when the quality of care provided to the patient was rated as “uncertain” (68.9%), and most likely when the care was rated “bad” (89%). The authors concluded, “the negligence system provides a substantial incentive for high quality medical care” [8]. Sloan and Hsieh also found a relationship between claim strength and the probability of payment and concluded their findings were inconsistent with the view that the tort system is a “lottery” [21].

The most recent study to collect data on the fairness of malpractice settlements analyzed 1452 claims files randomly selected from the archives of five major malpractice insurers and sampled these archives for four types of clinical mishaps: obstetric, surgical, misdiagnosis, and medication [23]. Because the study used independent reviewers and gave them access to the entire closed-claims file, its findings are especially credible. The mean amount paid to nonerror claimants was considerably lower than the amount paid to claimants with meritorious claims. Claims were divided into six categories using a one to six scale to measure the reviewer’s level of confidence for a determination of fault, ranging from “little or no evidence” to “virtually certain evidence” [23]. The authors noted the probability of a payment after either a voluntary settlement or a plaintiff’s verdict increased as the evidence of negligence became more persuasive. Physicians made settlement payments in 19% of the claims with “little or no evidence” of error, 32% of the claims with “slight to modest evidence,” 52% of claims deemed a “close call but less than 50–50,” 61% of those rated as “close call but greater than 50–50,” 72% of claims with “moderate-to-strong evidence,” and 84% of the claims with “virtually certain evidence” [23]. The authors concluded, “the malpractice system performs reasonably well in its function of separating claims without merit from those with merit and compensating the latter” [23].

Only one study failed to find a major relationship between the merits of a claim and the likelihood of settlement [1]. Just slightly more of the cases involving medical negligence resulted in a settlement payment than did the cases lacking evidence of negligence. Because the difference was not statistically significant, the authors famously concluded, “the determination of negligence may be an expensive sideshow” [1]. An additional study deserves mention; Ogburn et al. [15] examined 220 claims of obstetric negligence filed between 1980 and 1982 against physicians insured by the St Paul Company, which had not been dropped or dismissed before incurring at least $1000 in defense legal fees. Although the likelihood of a settlement was considerably related to the quality of care rendered to the claimant, the relationship was not as strong as other, larger studies. Plaintiffs received a settlement payment in 90% of the cases involving negligent medical care and 55% of the cases involving proper medical care. The high settlement rate in low-odds cases may have resulted from the exclusion of all claims that had been dropped or dismissed without payment before expending $1000 in legal fees; as such, the authors acknowledged their results “cannot be extrapolated to all closed claims” [15]. However, their findings justify further research to determine whether weak neonatal injury cases settle at a much higher rate than other low-odds malpractice claims. If so, reforms targeted to this category of cases may warrant consideration.

Nearly every study of malpractice settlements has found a relationship between settlement rate and case quality with the exception of one investigation that concluded the determination of medical negligence may be irrelevant to the outcome [1]. This conclusion has been frequently quoted by advocates of medical malpractice reform, but three aspects of this study merit scrutiny. First, the sample size was only 52 lawsuits. Second, when study reviewers disagreed on the issue of negligence, the case was arbitrarily classified as a “good care” case, thereby artificially inflating the settlement rate among the “good care” case group by assigning all uncertain cases to this category. Third, plaintiffs with strong claims of negligence had much less success obtaining a settlement than similarly situated claimants had in other studies, suggesting the sample size may have been not representative of the underlying population.

Of note, the relationship between evidence and outcome is not perfect. Approximately 10% to 20% of the claimants with low-odds claims receive a settlement of some kind and roughly the same fraction of patients with strong evidence of negligence recovers nothing. This disagreement rate may be perceived by many as an estimate of the odds that a case will be resolved unfairly. However, the settlement disagreement rate is actually materially lower than the disagreement rates normally found when independent expert observers rate performance [7, 8]. From this perspective, the agreement rate between reviewer ratings and settlement outcomes is better than would be expected, especially when the discounting of settlement payments is taken into account.

Furthermore, the disagreement rate in low-odds cases may be the product of pro-physician bias in the rating system. With rare exception, the studies reported here have relied on physicians to rate other physicians. Physicians are reluctant to label the conduct of another physician as negligent [26]. Together, interrater variability and rater bias could account for much, maybe all, of the 10% to 20% payment rate in low-odds cases and the similar nonpayment rate in meritorious cases.

Settlement size, like the odds of a settlement payment, is related to the merits of the underlying medical malpractice lawsuit [17]. Claimants whose evidence is weak receive smaller payments than claimants who have borderline cases and similar injuries. Plaintiffs with strong cases receive most, although not usually the full amount, of the damages they suffered. Because this reduction in settlement size supplements, rather than substitutes for, the lower settlement rate in the close cases and low-odds cases, the data reveal malpractice claims as a group are discounted in two distinct ways, once in the insurer’s decision whether to make any settlement offer at all and again in the size of the offer to make. When the two are combined, the total discount is larger than would otherwise be justified by the evidence. The double effect is most visible in the “toss-up” cases, ie, those cases in which the evidence of negligence is ambiguous and a defensible verdict at trial could go either way. Negotiation theory predicts nearly all of these 50–50 cases will settle for approximately half of the plaintiffs’ damages. The toss-up cases that settle appear appropriately discounted. However, 40% of the toss-up cases are resolved without any payment whatsoever. This 100% discount is far more than what is justified by the evidence. The ability of malpractice defendants to avoid any payment in 40% of the toss-up cases suggests they have a major advantage in bargaining power. This conclusion is also supported by other studies, which have found the amounts paid to settle malpractice cases fall short of the expected value of the cases (ie, anticipated damages multiplied by the probability of liability) [8, 17].

The most likely sources of the defendants’ advantage lie in asymmetric stakes that give defendants an incentive to fight low-odds claims fiercely, asymmetric risk tolerance that prompts plaintiffs to settle at a discount, shared knowledge that plaintiffs actually win very few jury trials and that cases resulting in plaintiffs’ verdicts routinely are settled for considerably less than the jury award, and the defendants’ superior access to useful resources of several kinds [17]. Together, these factors appear to push the amounts actually paid in settlement below the fair expected value of the claims based on their underlying merits.

In one respect, the evidence that settlements are closely tied to the merits should come as no surprise. Insurers, like claimants, have an economic incentive to evaluate their cases accurately and to shape their settlement strategies accordingly. Insurers accomplish their objectives by undertaking a form of peer review in which they obtain multiple expert evaluations and rely on them heavily. Ironically, physicians see the absence of peer review as the major flaw in the current system of adjudicating malpractice cases [16]. Instead, peer review is precisely what the settlement process currently provides.

Discussion

The widespread assumption that the civil justice routinely produces irrational or unfair outcomes is not supported by the evidence. Although the civil justice system has many drawbacks, including its limited ability to screen out meritless cases quickly and its high cost, bias against physicians is not one of them. To the extent that litigation outcomes and peer assessments diverge, litigation outcomes are more likely to favor physicians than patients.

Despite this body of research, a number of important questions remain. For example, no study has attempted to determine whether some specialties fare better before juries or in the settlement process than others, although the data do raise a question about the vulnerability of physicians in neonatal injury cases. In addition, researchers have yet to pursue the preliminary clues suggesting “unappealing” physicians and patients face a heightened risk of losing their cases. Nor have they explored whether that bias would prejudice some specialties more than others. Even more importantly, investigators have offered no clues, other than witness appeal, to identify those cases in which a jury’s verdict is most likely to differ from the views of the medical reviewers hired by the defendant’s insurance company. That information would be priceless because it would aid physicians and their claims adjusters in their settlement decisions. At present, physicians have only the odds to guide them. Right now, it makes sense to negotiate hard and go to trial, if necessary, whenever the physicians who confidentially evaluated the claim on behalf of the liability insurer (not the ones hired by the physician’s attorneys to testify at trial) believe it can be defended. Defendants should similarly agree to settle the cases in which the insurer’s outside reviewers are concerned about errors in the patient’s care.

There are other limitations in the research. Although a number of studies have been done, only a few have used a large sample and some were limited to a single state. Thus, they may not be representative of the outcomes in other areas. However, the consistency of the findings across the range of studies is quite strong. In addition, the findings of the studies have been relatively consistent over time, although the findings in the 2006 Studdert [23] study suggest physician outcomes in cases with weak evidence of negligence have improved.

Another limitation is subjectivity is inherent in this kind of analysis. That introduces a risk of pro-physician bias on the part of the physician reviewers and adds the confounding effect of interrater disagreement. As a result, the relationship between evidence of error and outcome of cases cannot be evaluated perfectly. Once again, however, the consistency of data accumulated over many years of study alleviates some of this concern.

Given the limits of human capacity to reconstruct past events and the inevitable subjectivity of judgments about the quality of past performance, it is probably not possible to design a fault-based adjudication system that will have a substantially higher agreement rate in cases with weak evidence of negligence. However, modest improvements may be possible through careful refinements. The data on settlement outcomes is similarly reassuring. To the extent that juries and settlements err, the error is more likely to favor the defendant physician than the plaintiff patient.