Challenges in Outcome Measurement: Discrepancies Between Patient and Provider Definitions of Success

Abstract

Background

Some orthopaedic procedures, including TKA, enjoy high survivorship but leave many patients dissatisfied because of residual pain and functional limitations. An important cause of patient dissatisfaction is unfulfilled preoperative expectations. This arises, in part, from differences between provider and patient in their definition of a successful outcome.

Where Are We Now?

Patients generally are less satisfied with their outcomes than surgeons. While patients are initially concerned with symptom relief, their long-term expectations include return of symptom-free function, especially in terms of activities that are personally important. While surgeons share their patients’ desire to achieve their goals, they are aware this will not always occur. Conversely, patients do not always realize some of their expectations cannot be met by current orthopaedic procedures, and this gap in understanding is an important source of discrepancies in expectations and patient dissatisfaction.

Where Do We Need to Go?

An essential prerequisite for mutual understanding is information that is accurate, objective, and relevant to the patient’s condition and lifestyle. This critical information must also be understandable within the educational and cultural background of each patient to enable informed participation in a shared decision making process. Once this is achieved, it will become easier to formulate similar expectations regarding the likely level of function and symptom relief and the risk of adverse events, including persistent pain, complications, and revision surgery.

How Do We Get There?

Predictive models of patient outcomes, based on objective data, are needed to inform decision making on the individual level. This can be achieved once comprehensive data become available capturing the lifestyles of patients of diverse ages and backgrounds, including data documenting the frequency and intensity of participation in sporting and recreational activities. There is also a need for greater attention to the process of informing patients of the outcome of orthopaedic procedures, not simply for gaining more meaningful consent, but so that patients and providers may achieve greater alignment of expectations and increased acceptance of both the benefits and limitations of alternative treatments.

Introduction

Alignment of the goals of the surgeon and the patient before an orthopaedic procedure makes it more likely the patient will be satisfied with surgery [14, 38]. In addition, for patients’ consent to be truly informed, there must be a meaningful appreciation of the likely impact of a treatment in terms of symptoms, function, and residual deficits, as well as of the likelihood of an adverse outcome [72, 74]. The question of perspective also forms the basis of patient-reported outcome measures. Historically, assessments of outcome after orthopedic procedures have been derived from the surgeon’s input and expectations [37]. However, over the last three decades, there has been a shift to the perceptions of patients in the evaluation of outcomes of orthopaedic procedures [5]. This change has been fueled by the growing realization that patients, not surgeons, define whether an orthopaedic procedure is successful and whether to seek additional treatment, including revision [6, 9–11, 25].

Nonetheless, the surgeon has a professional and legal responsibility to justify his or her actions in recommending a course of treatment to the patient and in adequately informing each patient of the expected result and possible complications associated with that treatment. This process can only work effectively if patient and provider share an understanding of what defines “success” and if they agree on the relative values, both positive and negative, of deviations from the ideal result [1, 14, 38, 74].

Unfortunately, the definition of success has proved elusive, and the literature offers little specific guidance, in particular from the patient’s point of view. Numerous studies have previously reported the incidence of satisfaction of patients after THAs and TKAs, with values ranging from 77% to 95%, depending on the procedure, the geographic location of the study, and the way in which the questions were asked (Fig. 1) [2, 4, 27, 28, 42, 54, 68]. Large variations have also been noted in terms of the ability of joint arthroplasty to restore patients to the functional status they experienced before the onset of joint disease [43, 44, 49, 54]. This is due, in large part, to differences in the lifestyle and expectations of each individual, as well as the willingness of patients to tolerate residual knee symptoms.

Fig. 1.

This histogram of data derived from the validation trials of the New Knee Society Score depicts the distribution of satisfaction scores at a minimum of 1 year post-TKA (maximum score = 100 points). The range of scores is approximately the same postoperatively as preoperatively, with an average improvement of more than 60 points. Preop = preoperative; postop = postoperative.

Many patients expect joint arthroplasty not only to relieve symptoms but also to restore the ability to perform sporting and recreational activities without pain, stiffness, or swelling [13, 19]. This discrepancy tends to be greater in patients younger than 55 years [25, 52, 65, 74]. Some of these individuals, in fact, developed arthritis as a direct result of injuries incurred in sporting or fitness activities. For this patient group and for an increasing proportion of older patients with joint arthroplasty, the restoration of normal joint function involves returning to a predisease regimen of activities. Despite the desire to participate in recreation and exercise, biomechanical studies using instrumented prostheses of the hip and knee confirm that these physically demanding activities greatly elevate the loads and torques acting on the implanted components [15, 33, 60, 67]. This has led to the formulation of advice to patients from surgical societies concerning activities that are recommended and strongly discouraged after joint arthroplasty [12, 23, 48, 65].

Given the inherent difference in the perspectives of surgeons and patients, combined with variations between individuals in terms of how short-term risk and long-term reward are processed, we performed a review of the available evidence comparing the expectations of patients and surgeons with regard to the outcome of orthopaedic procedures, using hip and knee arthroplasty as the focus of our inquiry, with particular attention to the presence or absence of data supporting these expectations of outcome.

Where Are We Now?

Patients’ expectations after operative procedure are based on preoperative judgment of potential benefits and hopes for achieving the desired goals [24, 31, 61]. Numerous studies have documented that the immediate priority of patients undergoing joint arthroplasty is relief of pain, stiffness, and swelling. Almost all patients expect to perform the activities of daily life (ADLs) without limitation after surgery [19, 29, 39, 41, 45, 50, 53] and that their ability to perform these activities will improve postoperatively [29, 41, 53]. In these studies, the majority of patients also assumed that they would have vast improvements in their ability to complete activities requiring a high range of joint mobility, such as putting on shoes and socks (94%) and clipping toenails (88% and 93% in two studies looking at this end point) [29, 41]. Interestingly, fewer patients expected improvement in sexual ability after surgery (33% and 65% in two studies evaluating this end point) [29, 41].

Patients appear to distinguish little between improvements in ADLs after surgery and improvements in exercise/sports activities (72%–95%) and recreational activities (88%–93%). The only individual sports specifically addressed in published studies have been golfing and dancing, which 41% of patients with TKA expected they would be capable of performing after recovery [53]. However, over a period of up to 5 years after TKA, only 24% of these patients actually experienced an improvement in their sporting or recreational abilities. By contrast, the vast majority of patients with hip arthroplasty reported an improved ability to participate in recreational or social activities at 4 years after surgery [41].

Failure to meet presurgical expectations of activity level after full recovery has been identified as a factor influencing patient satisfaction with surgery (p < 0.001 [54]) [8, 39]. Patients’ dissatisfaction stems from residual pain, swelling, and/or joint stiffness (Fig. 2) [11, 54] and/or failure of the clinical result to match the patient’s preoperative expectations [8, 35, 41, 46, 54, 55, 62]. In addition, a recent study demonstrated a connection between these two factors, as patient dissatisfaction after TKA was found to be most prevalent in patients who experienced knee symptoms when performing activities that they considered personally important [55].

Fig. 2.

Data derived from the validation trials of the New Knee Society Score show the correlation between values of the New Knee Society satisfaction score and pain score (sum of pain with walking and pain on stairs and inclines). All patients were enrolled at a minimum of 1 year post-TKA. Increasing levels of knee pain with activity are reflected in a larger value of the pain score.

A major limitation of the orthopaedic literature is that few studies have reported the expectations and satisfaction of surgeons regarding the outcome of joint arthroplasty. Historically, the clinical results reported by surgeons have been assumed to be accurate and reproducible, despite accounts demonstrating otherwise [22, 41, 47, 50]. The few articles exploring this topic support the conclusion that surgeons’ expectations of patients’ outcomes vary with the surgeons’ operative volume and years of experience, the patient’s overall medical condition, and the complexity of the procedure [71]. In a study by Wright et al. [71], surgeons who performed larger numbers of knee arthroplasty procedures estimated that higher proportions of patients would have a reduction in pain, improvement in walking ability, and improved quality of life after this operation. The high-volume surgeons also judged, on average, that the incidence of complications would be lower than surgeons who performed fewer procedures.

In other studies, surgeons and their patients differed in their assessments of the patient’s current condition and in their predictions of the outcome of an operative procedure, with the surgeons’ expectations tending to be more optimistic (Fig. 3) [13, 31, 37, 50]. In a cross-sectional study of 100 surgeons and 370 patients undergoing THA or TKA, Moran et al. [50] found that the surgeons expected better results and predicted higher postoperative functional scores than the patients. Patients and physicians also display different priorities in evaluating treatments for osteoarthritis, with patients giving priority to elimination of symptoms and resumption of physical activities and physicians prioritizing the absence of long-term adverse events [13, 31]. In a study of 147 THAs, Lieberman et al. [37] reported a marked disparity in the evaluations of patients and physicians, with patients placing greater weight on the presence of residual symptoms in assessing the outcome of their procedure. Differences between patients and their surgeons were greatest in cases where the patient reported significant hip pain at followup and overall dissatisfaction with the outcome.

Fig. 3.

Results of a preoperative survey of 132 patients and 16 joint surgeons concerning expected outcomes after THA are shown. Differences in the average responses of surgeons and patients are plotted for each item, with corresponding 95% CIs. Reprinted from Jourdan C, Poiraudeau S, Descamps S, Nizard R, Hamadouche M, Anract P, Boisgard S, Galvin M, Ravaud P. Comparison of patient and surgeon expectations of total hip arthroplasty. Plos One. 2012;7:e30195–e30195.

Age and preoperative functional status have been identified as potential causes for the discrepancy between patient and surgeon expectations [51, 73]. Patients with lower preoperative functional status express higher expectations from total joint arthroplasty than the operating surgeon [18, 41]. In addition, the discrepancy between patients and surgeons was shown to be greater in patients who had less successful outcomes [11, 47]. In a comparative analysis of patients’ and surgeons’ response to the same clinical questionnaire in 1273 THAs, McGee et al. [47] showed that a larger discrepancy between the reports existed in patients with comorbidities or substantial residual symptoms post-recovery. This may be due to unrealistic expectations (especially with physical activity after joint arthroplasty), persistent pain in the operative limb, or comorbidities, including pain and stiffness in other joints. Several studies have also reported concordance between patients’ and physicians’ evaluations in patients reporting little or no pain at the time of evaluation [37, 47, 53, 63]. Also, patients who were 45 years or younger had greater agreement with their surgeon in their assessment of expectation and outcomes [47]. However, when patients are dissatisfied with their outcome, the patient/surgeon discrepancy is often greatest [11, 47].

The discrepancy between patients’ and surgeons’ assessment of the outcome of orthopaedic treatments arises from a host of factors, including the inherent variability of many procedures and differences in the expectations of patients and surgeons regarding the likely outcome. Surgeons and patients may also differ in their assessment of the significance of different degrees of residual symptoms and the importance of full recovery of function. Surgeons may also fail to adequately communicate the limitations of proposed procedures in terms that the patients can understand. Within the academic literature, there has been increased attention to each of these factors in an attempt to improve the process of providing the right amount and kind of information to facilitate informed consent of patients before treatment. The goal of informed consent is to develop within each patient an understanding of the risks and complications of a course of treatment so that they are able to make informed decisions about their care. However, the method used to educate the patient and obtain consent may have an adverse effect on the accuracy of recall of the information provided [34, 70]. Langdon et al. [34] observed that patients who received written information before admission for THA scored significantly higher on recall questionnaires than the patients who received verbal information.

In addition, poor recollection of information provided by the medical staff can lead patients to poorly judge the outcome of their surgery and the quality of their interactions with their surgeon [59]. Robinson and Merav [59] tested patients for recall at 4 to 6 months postoperatively and identified poor retention in all categories of information provided during informed consent, with the poorest score being achieved in the single category of potential complications. Of the 20 patients tested, 16 expressed no doubts regarding their recollections and denied that major items were discussed at all. In an effort to align patient and provider expectations, it is wise to document consent as part of the clinical record, because the patients’ memory of the event has been proven untrustworthy [26, 34, 59]. The staff responsible for obtaining consent from the patient can also contribute to skewed patient and provider expectations. The most common cause of medical litigation is inadequate doctor-patient communication, often resulting from inaccurate or misleading information provided by junior staff [57, 58]. A study by Soin et al. [64] identified a large discrepancy between the responses of senior and junior members of a surgical team when asked to estimate the mortality rates of elective procedures. It may be inferred that, if senior surgeons were to inform patients of the relevant risks of these procedures, there would be greater alignment of expectations regarding outcome and improved patient satisfaction.

Mancuso et al. [40] showed that patient expectations before total joint arthroplasty influence the postoperative outcomes. Other reports have demonstrated varying expectations among surgeons themselves. In a survey of 357 total joint surgeons, Dy et al. [17] showed significant disagreement between surgeons in predicting the outcome of total joint arthroplasty in patients with higher demands and personal expectations. This lack of uniform opinion mirrors the views expressed in published studies and may be partly due to variations in indications, timing, type of approach, implant and fixation used, postoperative recovery, ethnic and racial factors, and uncertainty concerning the durability of modern implants and materials [7, 17, 20, 36, 39]. Nonetheless, agreement of the surgeon and patient regarding clear and realistic outcomes of orthopaedic procedures is essential to prevent discrepancies in expectations, avoid disappointment, and improve satisfaction.

Where Do We Need to Go?

The recent focus on joint registries and the documentation of revision rates from joint arthroplasties has brought increased awareness to the differences in the ultimate success of these operations due to variations in patients, surgeons, and implants. One of the great strengths of joint registries is the sheer volume of data available for analysis and the relative frequency and similarity of the procedures under study. In fields other than joint arthroplasty and in the treatment of more complex and unusual conditions than osteoarthritis of the hip or knee, smaller case numbers and greater diversity make analysis of outcome and predictions less reliable, especially on a case-by-case basis. In addition, even in common procedures such as hip and knee arthroplasty, much more attention has been paid to the survivorship of the prosthesis than the quality of the outcome experienced by the patient. The data that have been collected reveal that there are large differences between failure rates of procedures and the outcome in terms of patient satisfaction, relief of symptoms, and restoration of abilities significant to the patient [3, 52–55, 69].

Before patients and surgeons can compare their expectations of orthopaedic treatments and procedures, there must be a common factual basis for assessment of the expected results. This includes the likelihood of achieving a specific level of outcome and the risk of adverse events, including loss of function and the possibility of revision surgery. An essential prerequisite for shared decision making is shared information that both surgeon and patient believe is accurate, objective, and relevant to the patient’s condition and lifestyle. As this information does not exist at present, it is not possible to make accurate prognostic predictions that are meaningful to each patient. However, if data were available to form the basis for predictive models, it would be possible to quantify risk factors on the basis of factors presented by the patient (eg, age, sex, BMI, type of prosthesis, etc). This information could then lead to improved outcomes through improvements in decision making and the surgical procedures themselves and in changes in variables under the patient’s control (eg, high-demand activities, participation in physical therapy, moderation of weight, consumption of alcohol, etc).

For shared decision making to be authentic, patients must be committed to developing an informed understanding of the likely outcome and possible complications of alternative treatments. This requires effort in listening to the surgeon and associated staff and in contemplating different scenarios that may arise should the outcome of a procedure be as successful as, or less successful than expected. Most importantly, patients must accept responsibility for their own health and be prepared to play an active role in selecting a course of treatment. This also means being prepared to accept the outcome of that decision, which both surgeon and patient must recognize is not always assured. On the other side of the partnership, surgeons must be willing to explore the goals and aspirations of each individual patient, and discuss the likelihood that these will be achieved. This dialogue must be undertaken using language and terminology appropriate to each patient, given their level of education, native language, and cultural background. Surgeons must also appreciate that because of their commitment to surgical treatment of many orthopaedic conditions, they may unconsciously fail to adequately convey the known limitations of a surgical procedure.

How Do We Get There?

The goal of informed decision making is to provide patients and providers with specific information relating to the expected outcome of an orthopaedic procedure, in terms of the quality of outcome. For this to be achieved, each dimension of outcome (ie, function, relief of symptoms, patient satisfaction) must be measured using validated instruments. These may be in the form of psychometric measurements (eg, patient-generated questionnaires), objective measurements of physical function (ie, joint kinematics or kinematics), or radiographic measurements of skeletal health or stability. Once these outcome data are available, predictive models may be developed, provided that a set of relevant descriptors can be collected and correlated with each of the major outcome variable. For example, to provide a 55-year-old male patient with a prediction of his likely outcome if he resumes downhill skiing or playing tennis, significant factors would probably include the outcome variables predicted (eg, implant failure, recurrent symptoms that resolve with rest), the patient’s BMI, intensity of activity, comorbid conditions, and previous history of injuries. Given the diversity of activities that patients regard as personally important and the array of variables that may affect outcome, a very large data set would be required to make predictions sufficiently specific to be meaningful at the individual level. An additional complication is that some outcomes, such as implant failure, will not be common enough to allow accurate risk assessment until an extended postoperative period, typically more than 10 years. This means that some parameters defining outcome (eg, long-term survivorship) cannot be addressed prospectively in the short term. Nonetheless, data collected as part of the ongoing efforts to expand joint registries may serve as a suitable source of data relating to the risk of revision in the years ahead.

We have also identified the necessity for surgeons and patients to reach an understanding of the true risks and consequences of different lifestyles on the outcome of joint arthroplasty. This will only be achieved once we have comprehensive data documenting the sporting and recreational activities of patients, both in terms of the activities performed and the frequency and intensity of participation, in addition to the outcome experienced by each patient. Although young and active patients will often assume that surgeons are inherently conservative as part of the medical establishment and therefore will give advice that restricts active pursuits, there is published evidence that demanding activities are detrimental to the longevity of joint arthroplasties [21, 30, 56, 60]. In one retrospective study, practicing a high-impact sport was a risk factor for failure of a joint replacement procedure, with an odds ratio of 3.64 (95% CI, 1.49–8.9) [56]. Patients in the high-impact sport group also had more mechanical failures (p = 0.001) than those who did not participate in high-impact sports. At 15 years’ followup, Kaplan-Meier analysis yielded an 80% survivorship for the high-impact sport group versus 93.5% in the low-activity group. Whether the same penalty will be seen with improved bearing materials is presently unknown.

We have also noted that the documented differences in surgeons’ and patients’ assessment of the outcome of orthopaedic procedures arise to a large degree because of differences in priorities. For this situation to change, surgical training will need to place greater emphasis on patient expectations and satisfaction. In assessing their success in treating a patient, trainees must be taught to respect the inherent right of patients to assert their own values and priorities and to recognize that few outcomes will be satisfactory if the outcomes do not meet the needs of the patient.

Discussion

The orthopaedic literature lacks a substantive analysis of factors contributing to the belief of some patients that their postoperative outcome fails to meet their preoperative expectations.

Although both providers and patients share a common desire for restoration of musculoskeletal function after injury or degeneration, the outcome of many orthopaedic procedures remains variable. As hope and aspiration are essential components of surgical recovery, many patients have overly sanguine expectations and may not choose to give their full attention to all the possible outcomes of their procedure. In reality, providers care about the same things patients do; however, a perfect result is never assured, and the definition of success varies from one individual to the next.

Studies show that patient and surgeon expectations after joint arthroplasty vary greatly [19, 29]. While similar expectations were observed for improvement in ADL function, the greatest disparities occurred in items assessing improved ability to perform sporting and high-ROM activities [12, 16, 48, 66]. As many patients report significantly higher expectations than their surgeons when considering the likelihood of actually returning to higher-level activities, improved communication between surgeons and patients appears critical to align the goals aspirations of both parties with realistic goals in terms of functional outcomes [32, 74]. Unless this level of consensus can be achieved as part of the preoperative decision making process, it is likely that the prevalence of patient dissatisfaction with the outcome of joint arthroplasty will continue to increase as patients become younger and more demanding of these procedures.

As the practice of modern medicine becomes increasingly personalized, outcomes tools that consider individual risk factors will become more important. Conversely, patients will discount advice that is not evidence-based and is derived from generalized conclusions that ignore age, sex, experience, activity preferences, and lifestyle. Clearly work and investment are required to harness the promise of modern outcome instruments to facilitate shared decision making as a key component of musculoskeletal medicine in the decades to come.