The Variability of Patient Preferences

Abstract

Background

Wide variation in procedure utilization suggests that surgical indications might not be rigorously defined. An alternative explanation is that surgical outcomes are valued differently across groups. When a patient, using the information provided by the surgeon, places high value on successful results or is indifferent to the costs of ineffective treatment, the treatment threshold is lower and more surgery will be chosen.

Questions/purposes

Is there a high variation in patients’ preferences and, therefore, high variation in treatment thresholds? Do people poorly estimate their own treatment thresholds?

Methods

I presented a hypothetical scenario describing a diagnostically uncertain meniscus injury to 100 college students, asking them to rate the value of the four end points based on treatment choice (arthroscopy chosen/declined) and post hoc knowledge of the true diagnosis (tear present/absent). From those data, I calculated treatment thresholds. Subjects also estimated their treatment threshold directly.

Results

The calculated treatment thresholds ranged from 4% to 88%. A discrepancy of at least 20% between the calculated and subject-estimated thresholds was present in 61 subjects.

Conclusions

There is great variance in the treatment threshold reported; additionally, many subjects poorly predicted their own calculated treatment thresholds.

Clinical Relevance

Variability in patient preferences for outcome is an important, but perhaps underestimated, clinical parameter. Meaningful assessment of patient preferences when recommending treatment or creating clinical practice guidelines will lead to better shared decision making.

Introduction

The variability in the rates of lumbar fusion surgery in the United States is substantially greater than the variability in disease prevalence [9]. Some have inferred from this that surgical indications might be based on “physician practice style” [29]. Similarly, there were disparities in utilization of joint replacement on the basis of ethnicity [26] and here, too, the “views of referring practitioners on the characteristics of patients likely to benefit” was cited as a determinant of which patients may be offered surgery.

However, Pauker and Kassirer [22, 23] reported that variability in the values attributed to treatment outcomes affects the demanded degree of certainty that a treatment will attain its desired aims. They termed this requisite degree of certainty the “treatment threshold”. Valuable treatments need fewer guarantees and have a lower threshold for invocation. In turn, differences in treatment thresholds, based on differences in the perceived costs and benefits of surgery, might produce differing rates of surgery, independent of disease prevalence.

The importance of patient preference has been investigated in other fields such as low-risk, clinically localized prostate cancer [11]. This condition can be treated with brachytherapy, radiation therapy, or radical prostatectomy It also can be addressed with a regimen of active surveillance (ie, serial prostate-specific antigen measurements and digital rectal examinations). Based on the values attributed to the substantial morbidity associated with active treatment, active surveillance was seen in one decision-analysis model to be reasonable approach to low-risk prostate cancer [11]. Nonetheless, the authors noted that “individual preferences play a central role in the decision”; that is, different values attributed to the outcomes may have favored the active treatment options, and therefore, more surgery was performed [11]. The role of the variability of treatment thresholds in orthopaedic surgery has been explored, for instance, in the matter of choosing between operative versus nonoperative management of acute Achilles tendon ruptures [16], but studies of this type are not widespread in orthopaedic literature.

Based on these studies and others, I suspected there would be high variation among patients in preferences for the possible outcome states after orthopaedic surgery, and that there would be highly variable treatment thresholds and highly variable avidity for surgery, and that such variability in preferences would be found even if the clinical problem was defined without variability [10]. I also suspected that patients do not reliably estimate their own treatment thresholds.

Therefore, my two questions were: (1) are treatment thresholds variable even for a fixed scenario presented to a homogenous group, and (2) are asserted thresholds consistent with those derived from a an explicit calculation of the individual’s stated values?

Materials and Methods

A trained investigator (TC) presented a study population of 100 healthy university student volunteers with a clinical scenario describing an acute meniscal tear said to be possibly small enough to mend on its own, but likewise perhaps big enough to require surgical repair. The students were recruited via a written announcement at the school. All subjects were aware that the purpose of the survey was to determine how much they valued the various possible contingencies related to the diagnosis and treatment of an athletic knee injury. The mean age of the subjects was 22 years (range, 18–37 years).

The single investigator presented the subjects with a written scenario (Appendix 1) and asked them to imagine that they were the patient described. They were told that the diagnosis under consideration was a torn medial meniscus that may or may not need repair. They then contemplated two options, surgery versus observation, and the four possible outcome states (Fig. 1) that might ensue: (1) arthroscopy was chosen and the tear needing repair was present (denoted as Rx+ Dx+), (2) arthroscopy was chosen and no tear needing repair was present (denoted as Rx+ Dx−), (3) nonoperative treatment was chosen but there was a tear needing repair (denoted as Rx− Dx+), (4) nonoperative treatment was chosen and there was no tear needing repair (denoted as Rx− Dx−).

Fig. 1.

The decision tree shows the options offered to the subject and the possible outcome states. The four terminal nodes are as follows: arthroscopy was chosen and a tear was present (denoted as Rx+ Dx+); arthroscopy was chosen and no tear was present (denoted as Rx+ Dx); nonoperative treatment was chosen but there was a tear (denoted as Rx− Dx+); and nonoperative treatment was chosen and there was no tear (denoted as Rx− Dx−).

The subjects were asked to use a 25-cm VAS to mark their preference measures for each of these four possible outcome states. The midpoint of this VAS, halfway between the margins, represented how the subject rated how they felt at the moment of presentation to the physician. The part of the line heading toward the left margin of the scale represented “feeling worse”, and the part of the line heading toward the right margin of the scale represented “feeling better”. The subject then rated each outcome state relative to the starting point by placing a mark on the line at a distance commensurate with the magnitude of improvement/decline (relative to the state at presentation) associated with that outcome. The distance between the mark and the origin represented the value of the given outcome state. From those four values, I calculated treatment thresholds according to the method of Pauker and Kassirer [22, 23], as described below.

Subjects also provided a percentage, 0% to 100%, reflecting how certain they must be that the meniscus was truly in need of repair before consenting to the operation. I compared this latter value (termed the asserted treatment threshold) with the treatment threshold derived from the outcome values ratings. I then used these values in an expected value calculation [2]. In an expected value calculation, the total value of any decision was the sum of the values of all possible states that could have resulted from the decision, multiplied by the chance of attaining that state. This was designated the expected utility. (The utility of a given state is a term of art connoting the value to the individual of that state; the word “expected” reflects that the probabilities of the outcomes also were considered.)

For the specific decision under consideration in the clinical scenario, the expected utility of surgery was the sum of the utility of surgery when a tear was present multiplied by the probability that a tear was present and the utility of surgery when a tear was absent multiplied by the probability that a tear was absent.

Using the notation described above and the term “Prob(X)” to signify the probability of X being true, the expected utility values (EUV) of surgery and nonoperative treatment were given using following two equations, respectively:

1

2

With those equations, I derived the treatment threshold as defined by Pauker and Kassirer [22, 23]. The treatment threshold represented a measure of clinical likelihood above which the surgical option had a greater value, and below it, nonoperative treatment was better. Exactly at the threshold, the two options were equal. Therefore, to calculate the treatment threshold I set EUV(Rx+) to be equal to EUV(Rx−) and solve for Prob [Dx+]:

This ratio, with the subject’s values entered, produced a number ranging from 0 to 1, which corresponded to a derived (or calculated) treatment threshold.

Subjects also provided a percentage, from 0% to 100%, reflecting how preoperatively certain they must be that surgery was needed before they would consent to the operation. I then could compare this asserted treatment threshold with the calculated treatment threshold for each individual. A comparison of these two values produces a measure of consistency of the responses. Means of these two values were compared using a two-tailed t-test after confirming that the data were normally distributed [Excel; Microsoft, Redmond WA, USA].

Results

The data confirmed a high variation in preferences. The mean derived treatment threshold was 39% with a standard deviation of 18.8%. The derived thresholds ranged from 4% to 88%, with a median of 42% (Fig. 2). The asserted treatment thresholds ranged from 9% to 100%, with a median of 58%.

Fig. 2.

A histogram shows distribution of derived (calculated) treatment thresholds. Most subjects had a treatment threshold near 50%. However, the responses ranged from 4% to 88%.

Subjects poorly estimated their own treatment thresholds. In contrast to the derived value, the mean asserted treatment threshold was greater (60%; p < 0.01). For 81 of the 100 subjects, the derived treatment threshold was lower than the asserted treatment threshold. The mean difference was 29%. Among the 19 subjects whose derived treatment threshold was higher than the asserted treatment threshold, the average magnitude of the difference was 17%. There was a discrepancy of at least 20% between the derived and asserted treatment thresholds in 61 subjects.

Discussion

In its report, Crossing the Quality Chasm [8], the Institute of Medicine urged physicians to respond to “patients’ wants, needs and preferences, so that [patients] can make choices in their care that best fit their individual circumstances.” Understanding preferences for outcomes is a necessary condition for a shared decision model [5] and patient-centered medical care. In the current study, a hypothetical case scenario is presented, and treatment thresholds are assessed. Two research questions were asked, (1) are treatment thresholds variable for a homogenous group, and (2) are asserted thresholds consistent with those derived from a utility calculation.

Some may criticize this study on several grounds. First, the preferences were collected as part of a thought experiment divorced from real-world considerations. This experiment was based on an idealized model of medical decision making, not the methods which frequently are encountered in routine, daily practice. Specifically, in routine encounters, most of what a patient knows is derived from the surgeon; and as the patient asks questions to help make his or her decision, the surgeon’s replies might be skewed by what he (and not the patient) values or believes. In other words, there is ordinarily an iterative process that may be strongly guided by the surgeon’s belief, and that process was not replicated here. Another variance from real-world considerations is that the subjects were not injured patients distracted by pain, but rather young, healthy students asked only to imagine they were injured and predict how they would value hypothetical outcome states in the future. Such affective forecasting often has been inaccurate [12], and emotional valence has affected judgment [1, 19]. (Also, owing to the subjects’ age, and possibly socioeconomic biases, the subjects here probably had an atypical concept of risk and long-term consequences of an injury.) Second, the scenario was contrived, and even if the parameters might have reasonably estimated current medical knowledge [3], any deviations from what was medically correct might have affected the thresholds. A related notion was that subjects would have produced different results had the scenarios been phrased differently [25]. As Tversky and Kahneman [28] showed, the choice of words framing alternatives influences responses. Third, a different method of assessing values (conjoint analysis [7] or standard gamble) might have yielded different responses and different thresholds. These criticisms would indeed nullify any inference about the correct treatment threshold for suspected meniscal tears. Yet this study did not purport to assert the correct value for this clinical problem, only that preferences were distributed widely and estimated imprecisely.

The derived treatment thresholds ranged from 4% to 88%, with a mean of 39% and a standard deviation 18.8%. The spread in values found here echoed a similar discovery made in a study of patients with angina, in which researchers found that patients with comparable symptoms nonetheless varied considerably in their tolerance for these symptoms [21]. Similarly, Hayes et al. [14] found that the best choice between active surveillance versus initial treatment for men with localized prostate cancer was highly dependent on patient preferences. Subjects’ stated threshold exceeded the calculated threshold on average by 27%. Yet, because a sizable minority (nearly 20% of subjects) understated their derived treatment threshold, a simple rule to estimate the threshold, such as subtract 30% from the stated number, would have been ineffective.

The finding of wide variability in preferences has particular implications for the creation of clinical guidelines. For example, a guideline for the diagnosis and management of possibly infected joint prostheses might start from the reasonable assumption that retaining the original prosthesis, if possible, is better since a revised arthroplasty may not function as well as the first one and its longevity may be less. Accordingly, one published guideline [18] for possibly infected knee replacements suggested that patients with acute symptoms should have undergo debridement with retention of the prosthesis and administration of antibiotics if the device is well fixed at the time of surgery (Fig. 3). However, some patients may value definitive treatment so highly that immediate revision arthroplasty (without a trial of débridement and reimplantation) is best. For them, the perceived cost of a second operation outweighs the putative benefits of trying to save the original device. Additionally, some patients may discount future benefits to the point that added longevity for the device had little value. (A patient expecting to live 6 years or less will not pay for gain realized seven years hence.) As such, the guidelines for management of a possibly infected arthroplasty might improve with the inclusion of an assessment of preferences (Fig. 4).

Fig. 3.

A sample guideline shows management of a possibly infected knee arthroplasty. (Modified from Figure 6 and published with permission from Leone JM, Hanssen AD. Management of infection at the site of a total knee arthroplasty. J Bone Joint Surg Am. 2005;87:2336–2348.)

Fig. 4.

A modification of the guideline shows consideration for patient preferences. Patients who highly discount future gains or who place high cost on the possibility of a second surgery (broadly grouped as type Y patients) should have immediate revision surgery. Patients with low discount rates or who ascribe relatively low costs to the possibility of debridement failure should retain their prosthesis if it is well fixed and try a course of antibiotics. (Modified from Figure 6 and published with permission from Leone JM, Hanssen AD. Management of infection at the site of a total knee arthroplasty. J Bone Joint Surg Am. 2005;87:2336–2348.)

The findings reported here also are meaningful for the process of shared decision making. Shared decision making is defined as the process “in which the physician attempts to provide the patient and family with the full range of information about the clinical problem so that they can assess potential risks and benefits and make an informed decision about how to proceed” [13]. Bozic and Chiu [5] reasonably predicted that such an approach could increase patient satisfaction for total joint arthroplasty. Yet, although it is reasonable to assume a general desire among patients for ideal outcome states, in some orthopaedic surgeries there is an inherent trade-off between two desirable outcome states. In these instances, shared decision making must go beyond providing information to patients (evidence-based data on what could happen) to also receiving it from them (their preferences regarding the perhaps mutually exclusive outcome states). There are numerous orthopaedic clinical cases with an inherent trade-off between desirable outcome states, including immediate versus delayed reconstruction for acute ruptures of the anterior cruciate ligament [11], arthroplasty versus internal fixation for treatment of displaced femoral neck fractures [4], arthroscopic versus open surgical repairs for management of anterior glenohumeral instability [17], and open reduction and internal fixation versus cast immobilization for nondisplaced or minimally displaced scaphoid fractures [6] (Table 1). In all these cases, the decision selected must be based on the relative values assigned to the possible outcomes; evidence-based medicine cannot, in the abstract, inform us of a universally correct (dominant) decision.

Table 1.

Orthopaedic surgery scenarios with off-setting opportunity costs

Scenario and clinical decision	Evidence based trade-off	Study
Imminent versus delayed reconstruction for acute ruptures of the anterior cruciate ligament	Imminent reconstruction subjects patients to surgery that may be unnecessary, as 2/3 of patients who defer surgery end up with equivalent outcome without ever submitting to reconstruction. Delayed reconstruction is associated with a higher risk of meniscal tears.	Frobell et al. [11]
Arthroplasty versus internal fixation for the treatment of displaced femoral neck fractures	Arthroplasty significantly reduces the risk of revision surgery. Internal fixation has a lower infection rate, less blood loss and possibly lower early mortality rates.	Bhandari et al. [4]
Arthroscopic versus open surgical repairs for the management of anterior glenohumeral instability	Open approaches are more effective in preventing recurrent instability and enabling patients to return to work. Arthroscopic approaches result in better functional scores.	Lenters et al. [17]
Open reduction and internal fixation versus cast immobilization for nondisplaced or minimally displaced scaphoid fractures	Surgery leads to a better functional outcome and less time off work. Cast immobilization leads to fewer complications and a possibly lower rate of scaphotrapezial osteoarthritis.	Buijze et al. [6]

This study suggests people differ in their value assessments for different outcome states [27], perhaps more so than currently assumed [15]. Therefore, to plan treatments appropriately, orthopaedic surgeons must assess the values patients assign to the various treatment outcome states. Such patient-centered communication [20] must go beyond asking the patient and listening passively [24]. Orthopaedic surgeons must develop new tools for helping patients understand and express their preferences, because accurate determination of patient preferences is the gateway to patient-centered medicine and shared decision making.

Appendix 1: Text of clinical scenario presented to subjects

Introductory statement:

The purpose of this survey is to determine how much you, as a potential patient, value various possible outcomes in the diagnosis and treatment of knee pain after an athletic injury. You will be presented with a scenario, and asked to imagine that you are the patient being described. There will be some uncertainty about your diagnosis: you may have a torn medial meniscus (cartilage) in the knee that needs repair. I will offer you the options that would be given to a patient in the situation we describe. I will tell you what can be done, and what outcomes you might expect from each option. You will be asked to mark a number line to denote how you value a specific result. There are no “wrong answers” as this survey is merely a device to find out what you think.”

The standardized scenario presented by the same researcher to all subjects:

Three days ago, you twisted your right knee while running, causing immediate pain. You were able to walk after the injury, but could not continue running. Over the next day, the knee has swollen. You tried to run again, but it was very uncomfortable. You feel a catching sensation in your knee but it does not lock up. You have no significant pain while at rest. When you were examined, your knee was swollen. It moves painlessly from 0° to 90°, but bending it more hurt. When I push on the point where the thigh bone and the shin bone meet at the knee, however, you feel sharp pain. At this point, 3 days after injury, you are still unable to run because of pain. Xrays were taken and were normal, except for some fluid seen in the knee joint. This fluid was aspirated and 20 cc of joint fluid with blood was removed. MRI was obtained, and this looked normal with one exception: there was a small area where the medial meniscus meets the capsule of the knee that was separated. The significance of this finding is not certain: if the area of separation is small, it is apt to heal on its own. On the other hand, if it is either large enough or unstable, it will not heal on its own. Yet the precise size of the tear and whether the tear would heal cannot be known at the initial examination. This information can be obtained only by examining the knee surgically (ie, with an arthroscopy) or by allowing enough time to pass, and noting (after the fact) whether or not the pain goes away. Because of the uncertainty of diagnosis, you face a choice: whether to have an immediate arthroscopy (perhaps undergoing what turns out to be an unnecessary surgery) or to watch and wait (perhaps wasting 6 weeks, and subjecting your knee to more damage while waiting). Of course, it is not known at the time when you make the decision whether the meniscus tear really requires the surgery. Therefore there is an aspect of gambling when you make your choice.