Severity of Postoperative Complications From the Perspective of the Patient

Abstract

Background:

Although provider-derived surgical complication severity grading systems exist, little is known about the patient perspective.

Objective:

To assess patient-rated complication severity and determine concordance with existing grading systems.

Methods:

A survey asked general surgery patients to rate the severity of 21 hypothetical postoperative events representing grades 1 to 5 complications from the Accordion Severity Grading System. Concordance with the Accordion scale was examined. Separately, descriptive ratings of 18 brief postoperative events were ranked.

Results:

One hundred sixty-eight patients returned a mailed survey following their discharge from a general surgery service. Patients rated grade 4 complications highest. Grade 1 complications were rated similarly to grade 5 and higher than grades 2 and 3 (P ≤ .01). Patients rated one event not considered an Accordion scale complication higher than all but grade 4 complications (P < .001). The brief events also did not follow the Accordion scale, other than the grade 6 complication ranking highest.

Conclusion:

Patient-rated complication severity is discordant with provider-derived grading systems, suggesting the need to explore important differences between patient and provider perspectives.

Introduction

In general surgery patients, reported rates of postoperative complications range from 6% to 44% (1,2). Complications are significant contributors to increased cost and poor patient outcomes (3 –10). As important targets for surgical improvement, postoperative complications are frequently studied (2,11,12). The main complication severity grading systems used to study surgical complications are the Clavien-Dindo classification and the Accordion Severity Grading System (13). Additionally, summative quantitative measures of complication severity—the postoperative morbidity index (PMI) and the comprehensive complication index (CCI)—have begun to be used in the recent literature (14 –19). Both CCI and PMI translated the categorical Clavien-Dindo and Accordion grades, respectively, to linear numerical scales to enable the relative comparison of the severity of various complications within and across the different grades (14,16).

These complication grading systems have been derived from surgeons, rating the severity of hypothetical postoperative scenarios (16 –18,20). For all systems, severity progression across grades has been thought of by surgeons as a surrogate for risk of complication-associated mortality. However, there is currently limited information on the patient perspectives of surgical complication severity.

For a variety of pathologies, from psoriasis and rheumatoid arthritis to urinary symptoms, endometriosis, and depression, numerous studies have identified significant discordance in patient and physician perceptions of disease severity (21 –27). In surgical patients, the literature is more limited. The most robust data on differences in patient and surgeon perceptions of surgical outcomes come from Mannion’s group in Switzerland focusing on spine surgery. They performed prospective assessments of patient and surgeon rating of spine surgery outcomes in over 1000 patients (28 –30). Those orthopedic studies in aggregate suggest patients and surgeons often differ in their assessment of the occurrence and impact of postoperative complications (28 –31). For about 25% of cases, spine surgeons gave better outcomes ratings than their patients, and in 25%, surgeons gave worse ratings than their patients, indicating significant discordance (29).

However, little is known about patient rating of general surgical complication severity and concordance with widely used severity grading scales, and the few studies that exist for general surgery patients are conflicting. Consistent with the spinal surgery literature, a study of patients who had undergone a low anterior resection operation assessed patient, surgeon, and radiation oncologist perspectives on the relative severity of bowel dysfunction and found significant differences in patient-derived and specialist-derived scores (32). Although another study of 227 general abdominal surgery patients by Slankamenac et al who elicited patient ratings of surgical complication severity found that patient and physician perceptions of the severity of complications differ significantly, they actually found high concordance between patient and physician ratings along the Clavien-Dindo classification (33). Patient and provider concordance of complication severity ratings has never been examined using the Accordion complication grading scale.

The primary objective of this study is to determine whether provider-derived postoperative complication grading systems are concordant with patient-rated severity, particularly for lower grade complications where more discordance seems likely to be found.

Materials and Methods

Survey Development

We modified the survey used to obtain expert surgeon input for the PMI to design our patient survey (17). A preliminary survey was drafted and tested by cognitive interviewing of patients admitted to the general surgery service at the University of Wisconsin Hospitals and Clinics (UWHC), general surgery ward nursing staff, and the University of Wisconsin (UW) Department of Surgery research staff. A hybrid method of previously described think-aloud and probing cognitive interview techniques was used to obtain recommendations about survey content and administration (34). Briefly, cognitive interview participants were asked to “think out loud” as they read survey questions and selected responses. After completing the survey, specific open-ended questions were asked to gauge participant comprehension of the questions and survey instructions and to determine how participants approached their responses. Information gathered from these techniques was used to modify the preliminary survey to its final version.

Survey Components

The final survey presented 9 surgical scenarios followed by 21 hypothetical individual postoperative events inclusive of grades 1 to 5 complications in the Accordion system. Our survey methodology was similar to the survey methodology used by Porembka et al to derive the PMI, which used multiple short case vignettes of hypothetical surgical cases, with each vignette representing a different Accordion grade of severity (17). Our survey used the same severity rating instructions and rating scale as Porembka et al; respondents were instructed to rate the severity of each event on an anchored 100-point scale chosen to increase discrimination of responses (see Supplements 1 and 2). Since our survey was specifically designed to capture more information about lower grade complications and was presented to patients rather than surgeons, our survey used additional and different vignettes than those presented by Porembka et al, and the scenario vocabulary and structure was designed to be understandable to patients.

Patients were not asked to incorporate their surgical experiences in any specific way into their assessments, and all had received standard of care informed consent as conducted per the routines of each clinic. In order to minimize the time burden for survey completion, the scenarios were divided between 2 survey versions and ordered randomly so as not to imply increasing or decreasing severity. In total, 8, 5, 2, 4, and 1 event(s) represented grade 1, 2, 3, 4, and 5 complications, respectively, which were not disclosed to patients. One event did not meet the Accordion definition of a complication. This distribution of grades was chosen to reflect the breadth of potential complications represented within each grade.

Both survey versions also asked patients to qualitatively evaluate 18 brief postoperative events according to a 5-point scale in response to the question: “In general, when thinking about problems that occur after surgery, which of the following would make you think of the problem as severe?” The items included 9 postoperative events that do not meet criteria for complications in the Accordion system and 9 that did (2 grade 2, 3 grade 3, 2 grade 4, 1 grade 4-5, and 1 grade 6).

Finally, survey respondents were asked for demographic information, including age and sex, and if they had an operation, whether they had an operation within the last year, and whether they had any problems or complications after any operation. They were asked to comment whether they ever had any of the specific problems described on the survey and were given free space for other comments. No identifying information was collected. Both versions of the full survey are available in the Supplemental Material.

Survey Distribution

The UW-Madison institutional review board granted approval for the study. Either version 1 or version 2 was randomly assigned and sent to all English-speaking, adult (age ≥ 18), nonprisoner inpatients discharged from the UWHC general surgery service between November 2013 and February 2014. Surveys were sent by mail within 3 days of discharge and were returned by mail. Initially, a second wave of surveys was mailed if no response was received; however, this practice was eliminated after a very low number of patients returned the second wave survey.

Statistical Analysis

Survey results were entered into Microsoft Excel (Microsoft Corp, Redmond, Washington), and comments were transcribed. Severity score ratings by respondents were compared between the Accordion grades for the hypothetical postoperative events using a repeated-measures analysis of variance model. This analysis method was chosen because individual respondents graded multiple events within each Accordion grade. The same method was used to compare severity scores for individual postoperative events within a single grade to each other.

Since version 2 contained the only Accordion grade 5 complication and the only ungraded postoperative event, when all grades were compared, only version 2 responses were included in the model. Grades 1 to 4 complications were represented on both survey versions, and therefore, the severity scores were also separately compared between these grades, including both survey versions. The impact of age, sex, and operative history on severity scores for individual hypothetical postoperative events and overall by grade was also assessed. The percentage of patient responses to the brief postoperative events at each level of the 5-point response scale was tabulated. Events were ranked based on the percentage of respondents selecting the highest level of severity option (“A great deal”).

In order to categorize important themes from the patient comments, conventional content analysis was performed (35). Two reviewers coded each comment independently and used an iterative process to refine codes. These were grouped into major categories, and agreement was reached by discussion.

Statistical significance was set at α = .05. Missing values constituting >10% of overall responses were excluded from analyses. Statistical analyses were performed using Stata/SE version 14.2 (College Station, Texas).

Results

Survey Response and Demographics

The survey was mailed to 800 patients, and 168 (21%) responded. The response rate was similar for both survey versions. Missing values for responses ranged from 0% to 3% for the scenario events and 5% to 7% for the brief postoperative events, and none were therefore excluded from analysis. Survey respondent demographics are displayed in Table 1.

Table 1.

Survey Respondent Demographics.

Demographic Questions	Number of Responses	Mean ± SD or n (%)
Age (years)	158	55.1 ± 15.1
<45		34 (20%)
46-55		34 (20%)
56-65		52 (31%)
>65		48 (29%)
Sex: Male	158	64 (40%)
Operative history
Ever had an operation	160	149 (93%)
Within the last year?	153	119 (78%)
Any problems or complications after?	145	57 (39%)

Abbreviation: SD, standard deviation.

Although returned surveys were completely anonymized, the basic demographics of patients who did not respond to the survey could be determined. As is common with survey returns in the general population (36), a slightly larger proportion of males were nonrespondent (47% vs 40% responders, P = .05), and nonrespondents were younger than survey respondents (mean age 51 ± 19 years vs 55 ± 15 years, P < .01).

Patient-Rated Complication Severity by Accordion Grade

Overall, patients scored Accordion grade 4 complications highest in severity, followed by grades 1 and 5 (Figure 1). The severity grades as reported by patients had some areas of congruence with Accordion grades (eg, grade 2 was rated lower than grade 3, which was rated lower than grade 4). However, there were also important areas of discordance, notably with grade 1 complications being rated similarly to grade 5 complications and higher than grades 2 and 3 (Figure 1).

Figure 1.

Patient-rated severity scores for hypothetical postoperative events by Accordion grades. Box-and-whisker plots display the distribution of raw severity scores pooled from patient responses to both survey versions and categorized by grade. Mean severity scores are represented by the central black dots connected by a black line. See “Methods” section for analysis methods. All grades and one postoperative event that did not meet criteria for grading as a complication (“no grade”) were rated significantly different from each (P < .05) other unless noted as “NS” (not significant). The ungraded postoperative event described a hypothetical patient who was weakened by an operation for colon cancer, needed significant help from his family and friends after returning home after surgery, and was unable to stay at home alone. It was specified that the patient required 10 weeks to regain his normal amount of strength. This “no grade” complication was scored significantly higher in severity than all other grades (P < .001), with the exception of grade 4 (P = .08). Since version 2 contained the only grade 5 and ungraded complications, it was used alone to compare all grades. Versions 1 and 2 of the surveys were analyzed for grades 1 to 4 both independently and pooled, demonstrating a pattern and significance consistent with the results for version 2 alone.

Within each grade, there was also significant variability in patient-rated severity scores (Figure 2A). Of the 5 grade 2 individual complications compared, only 2 of the 10 pairs of complications were not rated significantly different. The 2 grade 3 complications were rated significantly different from each other as were 5 of the 6 pairs of grade 4 complications. Additionally, the median severity scores for several individual grades 2, 3, and 4 complications overlapped in the mild to moderately severe range. A comparison of grade 1 individual complications found that complication duration affected patient-rated severity. When time was specified, patients scored complications, spanning a longer time period significantly higher than those spanning a shorter time period (Figure 2B).

Figure 2.

Variation in patient-rated severity scores within individual Accordion grades. A, Box-and-whisker plots of the distributions of patient-rated severity scores for each individual complication representing Accordion grades 2, 3, and 4 are shown. Patients scored complications using a qualitative scale of severity from 0 to 100, represented on the x-axis. Each box plot represents the range of severity score values for a single complication. The colored boxes represent the first to the third quartiles, with a central line representing the median. Outliers were determined as being >1.5 × interquartile range (IQR) above the third quartile and are represented by dots. See “Methods” section for analysis methods. B, Grade 1 complication scenarios in the survey featured the same complication with a different duration to assess the importance of time to patients. The complications of an NG tube placement, wound care requiring dressing changes, and Foley catheter placement were presented with varying times of duration. Box-and-whisker plots of the distribution of scores for each complication are shown, with means indicated by the central dots connected by a line. *P ≤ .05, **P ≤ .01, ***P ≤ .001. NS indicates not significant.

Patient-Rated Severity of Ungraded Complications

One scenario that did not meet the traditional definition for a complication involved an operation for colon cancer after which the surgical patient required significant help from family and friends following discharge due to weakness and only regained his preoperative level of strength after 10 weeks. When scores were compared, patients rated this event the same or worse than all graded complications (Figure 1).

Similarly, when the 18 brief postoperative events were ranked according to impact severity, patients included an event not formally identified as a complication (ie, an iatrogenic event) as one of the most severe (Figure 3). Although the highest grade complication, grade 6 (death), was the most severely ranked complication, a grade 2 complication ranked within the top 5 most severe complications. Additionally, the bottom 5 ranked events included grades 2, 3, and 4 complications.

Figure 3.

The highest and lowest patient-rated severity of brief postoperative events. Patients were asked to score 18 brief events according to a 5-point scale, rating each complication by how likely it would make them think of the problem as “severe.” Responses were ranked according to percentage rating the events as the most severe (ie, “A Great Deal”). The top and bottom 5 ranked events are presented above. The Accordion grades of each event are provided here but were not specified to the survey respondents.

Impact of Sex and Operative History on Patient-Rated Severity

The patient-rated severity scores of individual complications were analyzed to determine any effect of age, sex, and operation history, and only sex had a significant effect on 7 individual postoperative complications (ranging from grades 1-5), with female respondents rating the severity higher than male respondents. When severity scores were pooled by grade, statistically significant difference remained by sex for grade 1, 2, and 4 complications, with female respondents rating the severity of complications higher (P < .01 for all). However, the overall pattern of severity between grades remained the same for males and female respondents and mirrored the pattern of all respondents demonstrated in Figure 1 (see Supplementary Figure 1).

Patient Comments

Ninety-three (55%) patients provided survey comments. Of these, 55 (59%) wrote they had experienced problems described on the survey. Three major themes emerged from comment analysis: (1) referencing pain and discomfort, (2) noting risks to safety, and (3) desiring better communication regarding expectations for surgery. One patient noted 3 “categories of things that scare me the most about complications: (1) pain: physical & emotional, (2) lack of normalcy: lack of ability to perform normal activities; disturbance in body image […], and (3) threat to physical health and safety: immediately and in the future.” Multiple patients referred to the level of pain or discomfort as being important in their consideration of the severity of an experience, with one patient writing, “We don’t want to suffer. How much risk is required to keep us comfortable? Most of us prefer to be ‘asleep’ (a riskier procedure) than to have pain or inconvenience.” Others noted minimal pain as key to their positive experience with surgery, such as the patient who stated “Post surgery was great. I felt healed within 2 days and was absent pain almost immediately.” Several patients mentioned risks associated with surgery. “The risk and complication of blood transfusions was too great,” one patient commented. Another patient wrote, “Everything is a big deal when you’re 70. Very risky, everything.” Many patients wanted better expectations for what would happen to them after an operation. One patient wrote, “I wish someone had warned me of this before the surgery as it is quite common I now know.” Another wrote, “Make sure patients know all possible complications before their surgery.” Another patient requested “more information on what is the normal healing period.” Agreeing with this, another patient wrote, “Advance disclosure of risks is very important and the patient should be educated about what to expect down the road in terms of return to the full function.”

Discussion

The results of this study indicate that although there are areas of agreement, there are important areas of discordance between traditional provider-derived complication severity grades and patient perceptions of complication severity. Specifically, patients rated some low-grade complications, and even events not considered to be complications, as more severe than some high-grade complications. Additionally, we showed significant variability in patient-rated severity scores for complications within a single grade that was due to both the nature of the intervention and duration of the complication.

Consistent with prior studies that demonstrated patient and provider discordance in assessing nonsurgical disease severity and complications after spinal surgery (21 –30), our findings demonstrated patient and physician perceptions of surgical complication severity are discordant. Our findings also agreed with the limited general surgery literature, demonstrating significant differences between provider and patient ratings of complication severity (33). However, although Slankamenac et al found high concordance with general abdominal surgery patient and surgeon ratings along the Clavien-Dindo classification, our study found discordance of patient and provider ratings along the Accordion scale (32,33). Differences in location of the study (Switzerland vs the United States), the patient population (preoperative patients vs postoperative patients), and survey methodology likely contributed in part to differences in our results. Since both the Clavien-Dindo and the Accordion grading systems share common roots in the original 1992 standard reporting system for surgical complications from the Toronto group (the T92 system) and are quite similar, it is unlikely that the different grading scales used in our study would explain this discordance (37).

An interesting consideration from our study is the high severity rating patients gave a postoperative scenario that did not meet the Accordion definition of a complication. The idea that patients perceive certain events to be complications that are not considered to be so by health-care providers has been demonstrated previously in the literature. Patients and their surgeons disagreed 18% and 30% of the time on whether a complication had occurred following spinal surgery and hernia repair, respectively (28,31). This suggests complications defined by medical professionals may not include all relevant critical factors to the postoperative experience from the patient perspective. The argument certainly can be made that postoperative scenarios, such as a prolonged postoperative recovery, which do not meet the Accordion definition of a complication are instead “sequelae” as defined by Clavien and Strasberg in their original complication grading system paper (ie, results that are inherent to the procedure itself rather than deviations from the ideal course) (37). Nevertheless, the strikingly high severity rating patients gave this scenario—rating it overall higher in severity than an intensive care unit stay involving ventilatory and vasopressor support—highlights important differences in the patient’s perspective of the postoperative course compared to the surgeon’s perspective.

The root of patient–provider discordance in complication severity grading may lie in assigning different relative importance to various dimensions of a postoperative event. For example, patients may focus more on the pain involved or the duration of the experience than providers, who use complication severity as a surrogate for significant physiologic derangements and for risk for mortality. The level of pain being important for determination of complication severity was a theme revealed in patient comments in our survey, and patients valued sedation and general anesthesia for pain reduction although they are associated with higher risk. This finding is consistent with differences demonstrated in the literature for nonsurgical disease severity, where patients are noted to give a heavier weight to psychological factors and pain than do providers (21,26,27). Additionally, although the duration of a complication’s effect is not included in current complication severity grading systems, patients in our study did consider time to be a significant factor. Further studies aimed at examining what specific factors surgical patients consider most important for determining complication severity are needed.

Although the investigation of the patient perspective of surgical complication severity remains in its early stages, our results and the literature in this area suggest efforts to target preoperative patient communication would help patients better anticipate their postoperative course (38,39). Achieving better alignment of expectations with the realities of the postoperative course has been shown in the spinal literature to improve patient satisfaction (28,39). Further work in this area is needed to guide these efforts. Ultimately, the development of a quantitative complication grading scale similar to the PMI but from the patient perspective as a patient-reported outcome measure would be a useful tool to enhance patient-centered surgical decision-making. In our study, we found that patients assign a higher severity score than we would have expected to lower grade complications. In order to develop a quantitative patient-derived grading scale, it would first be important to elicit what specific dimensions associated with a complication are important to patients and increase the patient perception of the complication’s severity. The value of a qualitative understanding of a patient’s postoperative experience cannot be overstated in this step. Once the dimensions associated with the patient experience of surgical complications are better understood, ordered quantitative ratings of the relative importance patients assign to each dimension would enable the creation of a patient-derived complication grading scale. The perspectives of both patients who had not yet experienced a surgical complication and those who had are likely both important and would need to be compared and incorporated in the derivation of a quantitative grading scale.

Limitations

Our study had several limitations. First, given the survey response rate of 20%, the impact of selection bias on results should be considered. This response rate is, however, similar to our own institution’s response rate (and better than the national response rate) for the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey and was in line with our expectations for a survey of this length and complexity administered so close to the time of hospital discharge. Indeed, it has previously been demonstrated that there are much lower response rates to HCAHPS surveys in sick patients at the time of discharge (36). Importantly, Elliott et al demonstrated in a large analysis of national HCAHPS responses that the impact of incorporation of nonrespondents on HCAHPS survey responses was negligible and as a result, response rate is not incorporated when hospital survey results are compared and published by the Center for Medicare and Medicaid Services (CMS) (36). Second, the ideal population to survey is unknown in this relatively unexplored topic. We selected to survey patients with familiarity with the surgical process as demonstrated by a recent inpatient surgical admission, reasoning these patients would represent an important group of patients to consider initially as we attempt to contribute to the limited literature on this subject. Recall bias related to differences in proximity of survey completion to the surgical admission is certainly a relevant limitation in this study. Perspectives of other patients and differences between perspectives in the acute phase of the complication versus more delayed recall remain to be explored (40,41). Finally, we did not collect extensive demographic information on patient respondents in this anonymous survey. There are likely patient-related factors, other than sex as we demonstrated, that influence complication severity ratings. Further studies in this area would help elicit these factors.

Conclusion

Only by understanding patient perspectives on complication severity will surgeons be able to address the full spectrum of the relevant negative postoperative events in their preoperative discussions with patients. Our results suggest there is a need for interventions that improve patient understanding of what to expect following surgery, including what surgeons traditionally consider to be “minor” complications as well as the spectrum of “normal” postoperative sequelae.

Author Biographies

Victoria R Rendell is a graduate of Duke University School of Medicine, and is now completing her general surgery training at the University of Wisconsin. Her areas of interest include interdisciplinary clinical research, quality improvement and surgical outcomes.

Alexander B Siy completed his undergraduate training at the University of Wisconsin, during which time he developed interest in clinical research. He is currently working as an Interventional Business Manager for General Electric Health Care in the cardiology, radiology, and hybrid OR space.

Linda M Cherney Stafford earned her Masters in Public Health with a focus in nutrition from the University of Minnesota, and has worked in clinical and health services research for over 15 years. She is currently an Associate Researcher for the Wisconsin Surgical Outcomes Research Program (WiSOR), focusing on topics in pediatric surgery and surgical oncology.

Ryan K Schmocker is a surgical oncology fellow at Johns Hopkins University, who is completing his final year of clinical training. He completed his general surgery residency at the University of Wisconsin. His research interests include value in cancer care delivery, cancer clinical trials, and institutional quality and safety.

Glen E Leverson earned his doctoral degree in statistics at North Dakota State University. He has been employed as a biostatistician in the Department of Surgery at the University of Wisconsin School of Medicine and Public Health for the past 23 years. His areas of interest include survival analysis and ethical interpretation of data.

Emily R Winslow is MedStar Health’s Regional Chief of Hepatopancreaticobiliary Surgery. She earned her medical degree at Harvard Medical School, completed residency in general surgery at Washington University School of Medicine and then completed a fellowship in hepatopancreaticobiliary surgery at Washington University School of Medicine and a fellowship in transplant surgery at Northwestern Memorial Hospital. Her areas of interest include the patient experience, multidisciplinary care coordination, and healthcare communication.