Disclosure of Individual Surgeon's Performance Rates During Informed Consent

Abstract

Objective:

The purpose of the paper is to examine the ethical arguments for and against disclosing surgeon-specific performance rates to patients during informed consent, and to examine the challenges that generating and using performance rates entail.

Methods:

Ethical, legal, and statistical theory is explored to approach the question of whether, when, and how surgeons should disclosure their personal performance rates to patients. The main ethical question addressed is what type of information surgeons owe their patients during informed consent. This question comprises 3 related, ethically relevant considerations that are explored in detail: 1) Does surgeon-specific performance information enhance patient decision-making? 2) Do patients want this type of information? 3) How do the potential benefits of disclosure balance against the risks?

Results:

Calculating individual performance measures requires tradeoffs and involves inherent uncertainty. There is a lack of evidence regarding whether patients want this information, whether it facilitates their decision-making for surgery, and how it is best communicated to them. Disclosure of personal performance rates during informed consent has the potential benefits of enhancing patient autonomy, improving patient decision-making, and improving quality of care. The major risks of disclosure include inaccurate and misleading performance rates, avoidance of high-risk cases, unjust damage to surgeon's reputations, and jeopardized patient trust.

Conclusion:

At this time, we think that, for most conditions, surgical procedures, and outcomes, the accuracy of surgeon- and patient-specific performance rates is illusory, obviating the ethical obligation to communicate them as part of the informed consent process. Nonetheless, the surgical profession has the duty to develop information systems that allow for performance to be evaluated to a high degree of accuracy. In the meantime, patients should be informed of the quantity of procedures their surgeons have performed, providing an idea of the surgeon's experience and qualitative idea of potential risk.

Public disclosure of surgeon-specific performance ratings has grown over the past decades as part of the trend to improve the quality of health care and empower patients to make better-informed medical decisions. This paper examines the ethical arguments for and against disclosure during informed consent as well as the challenges that generating and using performance information entail.

Public disclosure of hospital and physician performance ratings has grown over the past decades as part of the trend to improve the quality of health care and empower patients to make better-informed medical decisions. Particularly in the field of surgery, information on risks and benefits of a procedure may be relevant to patients’ decision-making about from whom to seek care. While the issue of public disclosure of performance ratings has attracted substantial attention, less attention has been paid to the ethical issues concerning whether or not individual surgeons, as part of the informed consent process, should be ethically and/or legally required to provide patients with measures of their individual success rates. The purpose of this paper is to examine the ethical arguments for such disclosure as well as the challenges of generating and using such information.

Since the early 1990s, several efforts have emerged at the state and national levels to measure and report the quality of care provided by different health plans, hospitals, and individual providers. Most performance measures were originally intended for internal use only, as part of continuous quality improvement efforts. However, pressure to disclose this information to the public, beginning with coronary artery bypass grafting (CABG) mortality rates, has resulted in a proliferation of public report cards on quality. There are 2 anticipated benefits that primarily drive public disclosure.^1–3 First, disclosure is thought to allow patients to make better decisions about whether and from whom to receive care. Second, disclosure is thought to be instrumental in improving the quality of care, by encouraging patients to vote with their feet and promoting competition and improvement among providers. Research has yet to confirm whether public disclosure has achieved either of these potential benefits.³

Surgery is an area where the measurement and public disclosure of provider-specific performance information has increased rapidly over the past 15 years. Beginning with New York State's effort to measure hospital and surgeon-specific mortality rates for CABG in 1989, there are now major efforts by the Society for Thoracic Surgeons and Veteran's Administration in the United States, and as well as more recent efforts in the United Kingdom to measure surgeon-specific performance for a variety of procedures.^4–7 While it is difficult to quantify the extent to which surgical skill or hospital processes contribute to surgical risk, there is little doubt that skill is an independent predictor. Evidence shows that the outcomes of individual surgeons vary even after adjusting for patient case mix and hospital characteristics.^8,9 For example, within high-volume hospitals, the mortality rates for carotid endarterectomy, esophagectomy, and pancreatectomy are lower when performed by more experienced surgeons.⁹ Likewise, cardiac surgeons who perform more CABG procedures have statistically lower risk-adjusted mortality rates than those who perform fewer.¹⁰ Importantly, the mortality rates among different surgeons can vary to an extent that is meaningful to patients who are considering surgery.

While there has been much discussion around providing patients with public report cards containing surgeon performance information, there has been much less discussion in the medical literature about whether such information should be disclosed to patients in the setting of informed consent. In addressing the question of whether or not surgeons ought to disclose their performance rates to patients during informed consent, we begin by examining the legal standards regarding such disclosure.

Legal Standards for Surgical Informed Consent

Legally, the requirement for physicians to obtain consent from patients for a medical procedure was established in the United States as early as 1914, and the more specific requirement that this consent be informed was added by a court ruling in 1957.^11,12 Since that time, legal requirements concerning the nature and quantity of information given to patients during informed consent have evolved. Currently, the law requires that surgeons disclose information that would be material to the consent decision of patients.^13–15 This includes information about the nature of the surgery, its risks and benefits, potential alternatives, and expected postoperative course. However, determining what counts as “relevant information” and how specific this information should be remains open to interpretation. An early proposed informed consent standard was the “professional practice” standard, requiring physicians to disclose information that is customarily provided by other professionals. More recently, the “reasonable person” standard requiring disclosure of information that a hypothetical reasonable patient would want to know, and the “subjective standard” requiring disclosure of information shaped by the preferences of an individual have been adopted in most states.^13,15

Two State Supreme Court cases have confronted the specific question of whether or not information about the performance of individual surgeons is required during informed consent. The courts reached opposite conclusions. In the first case in 1996, Johnson v. Kokemoor, the Supreme Court of Wisconsin ruled that a surgeon's experience and risk statistics did properly fall under the purview of the informed consent doctrine.¹⁶ Specifically, the court's verdict stated, “In this case, information regarding a physician's experience in performing a particular procedure, a physician's risk statistics as compared with those of other physicians who perform that procedure and the availability of other centers and physicians better able to perform that procedure would have facilitated the plaintiff's awareness of ‘all the viable alternatives’ available to her, and thereby aided her exercise of informed consent.”¹⁶

In a similar case in 2001, Duttry V. Patterson, the Supreme Court of Pennsylvania reached a different conclusion.¹⁷ The court held that personal information about the physician, including the physician's experience and qualifications, is outside of the scope of the legal doctrine of informed consent; such information is irrelevant to the doctrine of informed consent, irrespective of whether or not the information is solicited by the patient. In reaching this decision, the court relied upon the strict historical definition of informed consent, under which a physician is required to disclose only information about the surgical procedure itself, implicitly assuming that this is separable from the particular surgeon. It remains unclear whether the federal courts will expand the legal standards for informed consent to include surgeon-specific information, as they were in Kokemoor, or whether the doctrine will remain narrowly focused on the risks and benefits of the surgical procedure itself, following the logic in Duttry.

Ethical Principles Underlying Informed Consent

The legal question of whether or not surgeons are required to disclose personal performance is unsettled. This legal ambiguity reflects the unsettled ethical question of what information surgeons owe their patients. The primary ethical principle underlying informed consent requirements is respect for patient autonomy.^15,18 Therefore, one goal of the informed consent process is to enable patients to make medical care decisions that reflect their values and desires. The history of informed consent reflects an evolution of our understanding of how to best serve the interests of patients without harming the physician-patient relationship. The initial movement toward more explicit informed consent during the mid-20th century raised concerns that more information and control over their care would make patients unduly anxious or confused, allow them to make decisions that were not in their best medical interests, or weaken trust in their physician.¹⁵ While these tensions still exist, the informed consent process has come to be accepted as an appropriate and necessary expression of respect for autonomy that provides overall benefit to patients.

The question of whether information related to a surgeon's skill should be added to the current list of disclosed items has recently been explored by Clarke and Oakley.¹⁹ They argue that because a surgeon's skill, as reflected in a personal performance record, is relevant to patients’ decision-making, disclosure is required during informed consent. These authors state that there is widespread agreement that adequate informed consent for a procedure includes disclosure of “reasonably foreseeable risks of an operation,” and argue that because an individual surgeons's skill in performing an operation is a component of foreseeable risk, it should be disclosed. They conclude that, “disclosures that do not include at least some relevant, material information about the performance ability of available surgeons are an inadequate basis for the provision of effective informed consent.”¹⁹

The argument that respect for patient autonomy requires divulging surgeon-specific performance rates presupposes that the information is accurate enough to enhance patient decision-making, that patients want this type of information, and that the benefits of disclosure outweigh the possible harms. We will explore each of these questions in turn.

Is Surgeon-Specific Performance Information Accurate Enough to Enhance Patient Decision-Making?

Empirical data support the belief that individual surgeon performance directly influences the risks and benefits of a procedure, and that the likelihood of serious harms and benefits can vary widely among surgeons. Birkmeyer et al found that, after controlling for hospital volume, the difference in operative mortality for esophagectomy ranges from 9.2% to 18.8%, for pancreatectomy 4.6% to 14.7%, and for CABG 4% to 5.4%, among surgeons with high and low annual volume, respectively.⁹ When obtaining informed consent from patients, it is currently common practice for a surgeon to cite general morbidity and mortality rates, based upon regional, national, or published averages, rather than the risk and benefit of being treated by a particular surgeon. It may seem obvious that providing patients with the risks of surgery associated with their surgeon enables them to make a more-informed decision about their surgery. However, there are major challenges for generating highly accurate, surgeon-specific performance information and deciding how to use this information to counsel individual patients.

Individual surgeon performance rates are ideally calculated using statistical risk-adjustment modeling or stratification by risk group. These methods are intended to allow for fair comparisons among surgeons with variable case mixes, minimizing concerns that surgeon who operate on high-risk patients will be penalized. However, construction of such models is difficult and may be contentious. From the statistical perspective, the uncertainty in our knowledge of these risks can be decomposed into 2 components: chance and bias. The contribution of chance can be easily quantified, and its primary determinant is sample size. Table 1 shows the precision of an estimated risk as a function of sample size. For infrequent adverse events, large sample sizes are required to “know” these rates even within a factor of 2. For more common outcomes, 100 cases are needed to estimate an individual surgeon's rate within a maximum of ±10% (absolute difference), and 1000 cases to know it within 3%.

TABLE 1. The Precision (95% CIs) of Various Observed Rates Based on the Reported Sample Sizes: Relationship Between Sample Size, Observed Risk, and 95% Confidence Intervals

The ranges in Table 1 represent the minimum uncertainty around an individual surgeon's risk estimate. Additional imprecision is created by uncertainty about which are the important risk factors, which are used to define risk groups. The selection of risk factors to include in risk adjustment models has a significant influence on performance measure accuracy and can make the difference between misleading and accurate estimates.²⁰ If prognostic variables are stratified to create risk groups, with only a few variables the number of such subgroups becomes quite large, often larger than the total number of patients, making the number in each risk group too small to be useful. A stratified risk estimate can be wrong either because a risk group is too large, making it precise but biased, or too small, making it unbiased but imprecise. Statistical modeling addresses this problem by “borrowing” information across groups and estimating the individual contribution of each risk factor, thus permitting adjustment for many more potential risk factors.

Risk-adjustment models typically take into account only those parameters that can be reliably recorded in large populations, and may not include factors that are either difficult to quantify in a standardized fashion or that are specific to a particular setting. Error is produced both by the choice of adjustment factors (the terms in the model), the estimated effect of those factors (the coefficients of the terms), and the mathematical form of the model (which specifies how to calculate the joint effect of several risk factors from their individual effects). The more terms that are in a model, making them more applicable to individual surgeons and patients, the more imprecise those estimates become.

It is notable that there is a modicum of consensus about these models only for the highest volume surgical procedures. Adjustment models to compare surgeon-specific mortality from CABG have taken years to develop,^21,22 and doubts still linger among physicians about whether the risk-adjustment methods are adequate to compare surgeons fairly.^23,24 The CABG debate has been continuing for 2 decades and involves a well-established surgery, well-known risk factors, large numbers, and the unambiguous outcome of mortality. For other major surgical procedures, where outcomes more subjective, like pain, complications or quality of life, and where caseloads are smaller, the development of such models will be far more difficult.

For many of the preceding reasons, surgical risk-prediction models are primarily recommended for application at the institutional or departmental level, not the level of the individual patient or practitioner. One reason for the higher-level analysis is that, for many procedures, surgical risk can be affected as much or more by institutional factors (eg, quality of postsurgical care) as individual physician's technique. A second reason is that, like actuarial (ie, insurance) risk predictions, models that predict well for large, heterogeneous populations do not necessarily tell us what the “right” risk estimate is for single members of that population, ie, the individual patient or surgeon.

Bayesian approaches to institutional or physician profiling teach us that the best estimate of a surgeon's performance may, in fact, not be based solely on data from that surgeon's patients.^25,26 These approaches show that the best estimates for an individual are generated via a weighted average of data from that individual and from a larger group of related individuals, eg, from national rates, rates from comparable centers, or other surgeons within the same institution. The higher quality and more precise the individual surgeon-specific data are, the less heavily the external data are weighted. Conversely, if the data from on an individual surgeon are of poor quality or based on relatively few patients, rates from larger related groups would get greater statistical emphasis. So, the best estimate of an individual surgeon's outcome rates in many situations may, in fact, be numbers at an institutional or even national level.

This can be illustrated with a simple example. Consider a surgeon who has performed 100 operations in which 5 patients died. While the point estimate of the mortality rate is 5%, the 95% confidence interval (CI) is 1.6% to 11%, a range so wide as to be practically useless. Suppose that national or regional averages for the same procedure are 3% (CI, 2.5%–3.5%), and the surgeon believes that s/he uses state-of-the-art methods. The 3% number with its narrow confidence interval may indeed be closer to the truth for that surgeon than the surgeon's own average. But national numbers do not allow between-surgeon comparisons, the primary motivation for divulging surgeon-specific rates. It is interesting that this scientific uncertainty is reflected in the legal decisions, one of which focused on collective risk estimates alone and the other that held that individual-level risk estimates were more relevant.

Do Patients Want This Information?

Physicians are required to provide certain types of information to patients, including the nature, risks, benefits, and alternatives of a procedure, regardless of whether or not patients want or request it. Personal information that is considered irrelevant to the quality of a patient's care and decision-making is part of the sphere of protected private information that a physician may choose to disclose or not. There is a third category of information that physicians may not provide routinely, but disclose in response to patient request.

There seems little doubt that surgeon-specific performance information does not fall in the category of protected personal information, as it is relevant to the quality of patient care and does not intrude on a physician's personal privacy, as information about sleeping or drinking habits could. The question is whether it falls in the category of information that must always be provided, or provided only upon request.

To this end, it is relevant to understand how patients have used existing surgeon-specific information. Studies of patients’ views of surgeon report cards for CABG mortality rates suggest that few patients are aware that such information exists, few access report cards at the right time for making decisions, and few change their choice of surgeon based upon such information.22,23,27–30 Measuring the value of disclosure by whether or not patients actually change surgeons fails to recognize that patients may value such information even if they do not leave their surgeon, or if they have no choice of surgeon, in the same way that they value information about possible surgical complications even if surgery cannot be avoided. Furthermore, if awareness and timely access to such information are perceived barriers, disclosure during informed consent would address those problems.

There is also research on how surgeon-specific performance information affects patients making decisions about surgery. Schwartz et al asked Medicare patients how they decided where to undergo major surgery.³¹ Patients reported that the factor that would most influence their advice to friend about where to go for major surgery was surgeon reputation, and that while performance data could be relevant to their decision-making, they wanted to hear about it from their referring physician rather than reading it on a report card. This underscores the point that patients want direct communication and interpretation of such information from a health provider. More work is needed to clarify the empirical question of which specific types of risk/benefit information patients find most helpful how the uncertainty in performance rates should be communicated to patients, and who is best equipped to do so.

Risks and Potential Benefits From Disclosure

The major potential advantages of disclosure are enhanced patient autonomy, better patient decision-making, and potentially improved quality of surgical care for that patient. In addition, such disclosure may promote a culture of openness and accountability, which may serve to promote physician-patient trust, for the same reasons as does disclosing medical mistakes to patients.^3,32

One major potential disadvantage of disclosure stems from concerns about inaccurate performance measures. Inaccurate performance measures can encourage surgeons to avoid high-risk cases (for which they are unfairly penalized), misinform patients, and unjustly damage surgeons’ reputations. Disclosure of inaccurate rates poses the risk of making patients unduly anxious about their upcoming surgery, of jeopardizing trust between the surgeon and patient, and unfairly inducing patients to switch surgeons. There is also concern that individual reporting may not be accompanied by adequate time for surgeons to improve a relatively low performance score. Further, there is the concern that young surgeons would be severely penalized because of their low surgical volume, hindering their ability to gain patient confidence.

Disclosure generates 2 potential justice concerns about decreased or disparate access to surgery. One concern is that surgery will be withheld disproportionately from patients belonging to particular classes, such as racial or socioeconomic classes, due to perceptions that these patients will have poorer outcomes, dragging down performance statistics. A recent study found a significant difference in the usage of CAGB in white versus black and Hispanic patients in New York following the release of public reporting requirements.³³ The authors hypothesized that surgeons avoided CABG for black and Hispanic patients due to beliefs that these groups were at higher risk for poor outcomes than white patients. The second justice concern raised by disclosure is an exacerbation of unequal access to high-performing surgeons. High-performing surgeons are a limited resource to which all patients, theoretically, should have an equal chance of access. Undoubtedly, however, patients who have better resources will pursue treatment from high-performing surgeons more successfully than those with fewer resources. That patients will vote with their feet is in fact one intended consequence of public disclosure, aimed at increasing competition and improving quality. However, this may inevitably come at the expense of patients who for various reasons cannot change providers even in light of performance information. And as discussed below, it also poses a problem for the surgical profession. This built-in tension between individual patient autonomy and distributive justice is unavoidable but must be balanced carefully to assure that personal and societal interests are not unduly compromised.

It seems that many of the disadvantages from disclosure are diminished when the performance measures generated are highly accurate, ensuring fair comparison among surgeons with variable case mixes. But for patients who are undergoing urgent or emergent surgery, or who have no effective choice of surgeon for other reasons, it is less clear what purpose mandatory provision of even accurate performance information serves. In such situations, it is patient's expressed desire for such information that should determine whether it is shared.

How Should Performance Information Be Conveyed?

Any requirement for mandatory disclosure during informed consent will pose some serious practical dilemmas. Fifty percent of surgeons will, by definition, lie below the median of whatever score is used, whether it be because of chance or skill. Is it reasonable to expect that surgeons will disclose such information to patients? While the unwillingness of a physician to impart this information would not obviate the ethical obligation to disclose, it does suggest the need to find ways to discharge this obligation other than by direct disclosure by a treating physician at the time of consent. This may be done by referring physicians who discuss the performance of potential surgeons with patients, as suggested from recent data on patient preferences for disclosure,³¹ or by actively making patients aware of how to obtain publicly available information (eg, report cards), coupled with efforts to provide such information and make it easier to understand.

Implications and Conclusions

Disclosing low performance data to patients also has important implications for a surgeon's ability to improve. Society has a collective interest in maximizing the number of high-quality surgeons by allowing adequate opportunity and time for individual improvement following poor performance needs. Pitted against this public interest is the individual patient's right to choose a high performing surgeon and the additional incentive to improve that public disclosure might provide. Inevitably, some patients will bear the extra risk burden from some lower performing surgeons as they improve. This already occurs today, but in the age of performance measures, patients will have a number to attach to this risk and be less likely to tolerate it. This leads to the related question of under which conditions a low-performing surgeon may have the duty to refer a patient to another surgeon. The public interest in allowing room for improvement, versus the individual patient's desire to minimize their risk will need to be considered and weighed.

The goal of the perfectly informed patient empowered to make good decisions by clear communication and understanding of their surgical risk is a laudable one. But it should be clear from this discussion that this goal is not necessarily achieved by the calculation and reporting by individual surgeons of their performance rates to each patient. First, for most procedures, performance rates are too imprecise or inaccurate to be useful. Valid, risk-adjusted rates require data systems and analysis that do not currently exist for most procedures and outcomes. Numbers based on informal tallies or poor-quality data systems do not rise to a quality level that incur an ethical obligation to communicate to the patient, although they may incur an obligation on the part of the profession or institutions to rectify that situation. Second, there are little data telling us that this type of information provided directly to patients necessarily enhances their decision-making. Finally, there are many ancillary harms that could occur if the information becomes too finely grained, inhibiting surgeon's ability to improve and inducing anxiety among patients who have limited choice of provider.

At this time, we think that, for most of surgical procedures and outcomes, the accuracy of surgeon-specific performance rates is illusory. Given these current limitations, we do not think that surgeons are ethically obligated to communicate them to patients as part of the informed consent process. In the meantime, a better approach may be to inform patients of the quantity of procedures their surgeons have performed, and providing relevant institutional numbers when possible.

In the future, as precise and accurate performance rates emerge on more procedures, and data on patient preferences and effective risk communication strategies become more robust, the case for disclosure may become more compelling. However, even then important issues will remain. One substantial challenge will be to determine which numbers are most relevant for individual patient decision-making. There are many legitimate choices, including a risk-adjusted average over all patients, averages over the subgroup most “like” the patient being consented, averages over the most recent patients, reflecting current practice, or a weighted average of physician-specific numbers and other patients. This uncertainty highlights the ambiguity of the risk concept when applied to individual patients.³⁴ The goal of such disclosure must be clear: is it intended to inform the patient generally about their surgeon's skill or give the patient an accurate picture of their own risk?

A second question is how new surgeons just beginning independent practice may gain patient trust and establish a practice without their own performance rates yet to show. While new surgeons will not have accurate performance measures to disclose, they may still gain patient trust by disclosing the number of cases that they have performed as well as the institution's/hospital's performance rates, which may serve as a temporary proxy for their individual performance.

A third question is who is best suited to provide performance information to patients. Recent data suggest that patients may prefer to learn about the performance of specific surgeons from their primary care or referring physicians. This could be an effective strategy because referring physicians may be able to present a less biased account of the performance of a number of available surgeons and patients may feel more comfortable asking and receiving this information from these physicians. On the other hand, surgeons have the expertise to explain more thoroughly the meaning of performance rates. These questions deserve further examination, perhaps exploring ways in which the information could be conveyed by persons within a surgeon's institution, but not the surgeon themselves. Restricting reporting to institutional numbers would partially obviate this problem, particularly if accompanied by statements that there were no statistical outliers among specific providers. If there were such outliers (on the poor quality side), there is an institutional or professional obligation to rectify the situation.

A final question arises regarding who bears the responsibility of collecting and calculating such data. The generation of surgeon-specific performance rates is an endeavor best handled at the national or institutional levels rather than by individual surgeons, since the infrastructure required to gather these data reliably involves considerable time, expertise, and expense. It is critical to separate the ethical obligation to disclose during the consent process from the ethical obligation of the individual surgeon, their institution, and the profession to ensure that there are monitoring and training systems to assess and improve surgical skills and quality. But the data needed for continuous quality improvement efforts and even public reporting are not necessarily the same as that sought by an individual patient. Even if it is nominally an individual physician's obligation to maintain their skills, the fact that this goal requires initiatives at the institutional and national level suggests a similar way of addressing the need for optimally informing patients.

Providing patients with surgeon-specific performance measures during the informed consent process has the potential to enhance patient-decision making and autonomy. However, it also has the potential to damage the physician-patient relationship and harm efforts at quality improvement. Much work needs to be done to better understand patients’ informational needs, to generate performance measures of high quality, to improve risk communication strategies, and to develop institutional standards for how such information is to be presented to ensure that the advantages of disclosure outweigh the disadvantages.