Evaluating the Validity of the Clavien-Dindo Classification in Colectomy Studies: A 90-Day Cost of Care Analysis

Abstract

BACKGROUND:

The Clavien-Dindo classification is widely used to report postoperative morbidity but may underestimate the severity of colectomy complications.

OBJECTIVE:

To assess how well the Clavien-Dindo classification represents the severity of all grades of complications after colectomy using cost of care modeling.

DESIGN:

Retrospective cohort study.

SETTING:

Comprehensive cancer center.

PATIENTS:

Consecutive patients (n = 1807) undergoing elective colon or rectal resections without a stoma performed at Memorial Sloan Kettering Cancer Center between 2009 and 2014 who were followed up for ≥ 90 days, were not transferred to other hospitals, and did not receive intraperitoneal chemotherapy.

MAIN OUTCOME MEASURES:

Complication severity measured by the highest-grade complication per patient and attributable outpatient and inpatient costs. Associations were evaluated between patient complication grade and cost during 3 time periods: the 90 days after surgery, index admission, and post-discharge (up to 90 days).

RESULTS:

Of the 1807 patients (median age 62 years), 779 (43%) had a complication; 80% of these patients had only grade 1 or 2 complications. Increasing patient complication grade correlated with 90-day cost, driven by inpatient cost differences (p < 0.001). For grade 1 and 2 patients, most costs were incurred after discharge and were the same between these grade categories. Among patients with a single complication (52%), there was no difference in index hospitalization, post-discharge or total 90-day costs between grade 1 and 2 categories.

LIMITATIONS:

Retrospective design, generalizability.

CONCLUSIONS:

The Clavien-Dindo classification correlates well with 90-day costs, driven largely by inpatient resource utilization. Clavien-Dindo does not discriminate well among low-grade complication patients in terms of their substantial post-discharge costs. These patients represent 80% of patients with a complication after colectomy. Examining the long-term burden associated with complications can help refine the Clavien-Dindo classification for use in colectomy studies.

INTRODUCTION

Postoperative morbidity is an important outcome measure in colorectal comparative effectiveness research.^1,2 Along with oncologic control, the complication rate is a key metric by which we compare and assess the benefits of new surgical approaches and technologies. Decreasing complications has been a strong driver for innovations such as minimally invasive surgery, natural orifice procedures, and multimodality treatment. However, definitions of various complications and approaches to quantifying their severity have varied widely from study to study, limiting our ability to integrate and compare their findings.^3–5

In the past 15 years, the Clavien-Dindo (CD) classification has been increasingly used to compare postoperative morbidity in surgical studies.^6,7 First developed as a reporting tool for Morbidity and Mortality conference, it is now commonly requested by journals and peer reviewers, and has been an outcome measure in randomized trials.^8–10 The CD classification defines a complication as “any deviation from the expected postoperative course” and assigns it a severity according to the level of resource utilization required to treat it (Table 1). It has been validated with respect to length of stay and 30-day direct hospital cost in the adult surgical population, and is reliably scored by attending surgeons and residents alike.^11,12

Table 1. The Clavien-Dindo classification of surgical complication severity.

From Dindo et al.⁷

Grade	Definition
I	Any deviation from the normal postoperative course without the need for pharmacological treatment, or surgical, endoscopic, and radiological interventions. Allowed therapeutic regimens are: drugs as antiemetics, antipyretics, analgesics, diuretics and electrolytes, and physiotherapy. This grade also includes wound infections opened at the bedside
II	Requiring pharmacological treatment with drugs other than such allowed for grade I complications, or requiring blood transfusion or total parenteral nutrition
III	Requiring surgical, endoscopic, or radiological intervention
IIIa	Intervention not under general anesthesia
IIIb	Intervention under general anesthesia
IV	Life-threatening complication (including central nervous system complications) requiring intensive care unit (ICU) management
IVa	Single organ dysfunction (including dialysis)
IVb	Multiorgan dysfunction
V	Death of a patient

Despite the intuitiveness of the CD classification, clinical experience highlights potential obstacles to its validity as an outcome measure in colorectal surgery. First, the classification lends itself to focusing on “high-grade” complications since these require inpatient management, which can be identified and quantified reliably. However, most complications after colorectal surgery fall into “low-grade” categories. Surgical site infections (SSIs), for instance, are often classified by this system as “low-grade” because they require few in-hospital resources and are instead mostly managed post-discharge, on an outpatient basis. These are graded in a similar manner as hypokalemia, atelectasis on chest x-ray, nausea treated with antiemetics and other low-grade complications which require no long-term management. The current literature has not demonstrated the construct validity of the CD classification for the low-grade complications commonly seen after colorectal surgery e.g., whether SSI complications within a given grade are of similar severity, and whether increasing grade indicates increasing SSI severity. The question is critical if the CD classification is to be used as an outcome measure in colorectal surgery. For example, if a new surgical technique decreases grade 2 pulmonary complications but increases grade 1 wound complications, it would be difficult to assess the superiority of one technique over the other given the heterogeneity in complication severity encompassed in these two categories.

To assess how well the Clavien-Dindo classification represents the severity of complications after colon and rectal resection, we quantified the severity of complications using 90-day direct medical costs including inpatient, outpatient, rehabilitation and home care utilization. We then examined associations between increasing complication grades and increasing costs during the overall 90-day period, the index admission and post-discharge period to see how well the classification correlated with severity in these time-periods.

MATERIALS AND METHODS

We used Current Procedural Terminology (CPT) codes to identify all patients who had an elective colon or rectal resection at our institution between 2009 and 2014. To focus on the costs associated with complications, we excluded new ostomy patients who were likely to require routine post-discharge utilization regardless of complications. We also excluded patients who were transferred to another hospital or were lost to follow-up within 90-days of surgery. Lastly, we excluded patients who received intraperitoneal chemotherapy as inpatients. This study was approved by the Institutional Review Boards of Memorial Sloan Kettering Cancer Center and the Visiting Nurse Service of New York.

Clinical, operative, and postoperative complications within 90 days of surgery were collected by extensive review of the electronic medical records by two reviewers (MW, MK). An additional reviewer (PS) confirmed key outcomes. Comorbidities documented within 6 months of the date of surgery were gathered from an administrative database. Complications were described using National Surgical Quality Improvement Program descriptions when possible.

All complications were graded according to the Clavien-Dindo (CD) classification,⁷ and patients were classified according the highest CD grade complication they developed (PCD 1, 2, 3a, 3b, 4a, 4b, 5), allowing patients with multiple complications to be represented by a single severity grade.^7,12,13 The “disability” suffix was not used because its relationship with severity categories (i.e. 1d vs. 2d) has not been specified, nor is it commonly used in the literature, including in the CD validation studies.^8,11,14

We quantified the severity of complications using direct medical costs incurred during the first 90 days after surgery. Direct cost data was obtained from our accounting system and from the most frequently used homecare provider for our patients, the Visiting Nurse Service of New York (VNSNY). Costs were normalized to reflect Medicare reimbursement levels using the ratio of hospital costs to Medicare reimbursement based on diagnosis-related group (DRG) and CPT codes, as previously described.¹⁵ All hospital costs were estimated based on 2014 US dollars using 2014 costs per unit for all services and were adjusted for inflation to reflect 2014 costs. Final Medicare-proportional dollar (MP$) cost estimates were rounded to the nearest $10.

To focus on the costs associated with complications, day-of-surgery costs were subtracted from all calculations. All costs and complications within 30 days of surgery were included, as this is the commonly used window of analysis for surgical studies and for the CD classification validation.^1,12,16 After the 30-day period, attributable costs and complications included those that were directly related to surgery, such as an admission to a surgical service/ward, homecare services, and any emergency room consult managed by a surgical team.

Missing costs were imputed using multiple imputation methods. Missing homecare costs for patients who received homecare from a different provider were imputed based on known VNS homecare costs using a two-step multiple imputation method in which 5 imputed datasets were created. First, the duration of care was imputed for patients with missing start or end dates. Next, missing costs were imputed using predictive mean matching with patient characteristics and total post-discharge cost (sans homecare costs) as covariates.^17,18 For outside hospital readmissions and emergency room visits, missing costs were imputed by taking the average of MSK costs with similar indications. Outside hospital radiology costs were imputed by using the mean cost of these services at MSK. Finally, rehabilitation center costs were calculated on a per diem basis using a cost of $1301 per day, based on the 2013 Centers for Medicare and Medicaid Services report, which lists an average inpatient rehabilitation facility stay cost as $16785 (in 2014 dollars) and an average length of stay (LOS) of 12.9 days. Due to the use of multiple imputation, all summary statistics are pooled estimates, such that summary statistics should be interpreted as estimates of the true population means.

A linear mixed effects regression model was used to analyze the relationship between cost estimates and PCD grade during the 90 days after surgery, as well as costs within that period incurred during the index admission or post-discharge. The PCD 3a and 3b categories were grouped together due to paucity of these outcomes, as were PCD 4 and 5 categories. Additional exploratory analyses comparing costs between complications and costs based on setting (inpatient vs outpatient) used parametric and nonparametric tests for normally and non-normally distributed data, respectively. In all models, surgeon was treated as a clustering variable. Statistical significance was set at p < 0.05.

RESULTS

Between 2009 and 2014, 1807 patients underwent elective colon or rectal resection. Demographic, preoperative, and operative characteristics are shown in Table 2. Most patients underwent colectomies for cancer. Approximately 62% of the operations were performed using a minimally invasive approach and 6 surgeons from the colorectal division performed all operations. The most frequently imputed variable was homecare cost, which was missing for 15% of patients.

Table 2. Demographics and clinical characteristics of patients undergoing colon or rectal resection between 2009 and 2014 at Memorial Sloan Kettering Cancer Center (n = 1807).

Categorical data are presented as n (%) and continuous data as median (IQR).

Characteristic
Age (years)	61.6 (50.8–72.1)
Female sex	927 (51%)
Race/ethnicity
Black	111 (6%)
Caucasian	1479 (82%)
Other	215 (12%)
BMI	27.8 (24.2–31.7)
Insurance
Medicare or Medicaid	841 (47%)
Private	966 (53%)
7th AJCC stage
0/Benign	247 (14%)
I	356 (20%)
II	472 (26%)
III	42 (24%)
IV	300 (17%)
Smoking history
Current	188 (10%)
Former	615 (34%)
Never	1004 (56%)
Preoperative chemotherapy	246 (14%)
Preoperative homecare	19 (1%)
Surgical approach
Open	692 (38%)
Laparoscopic	846 (47%)
Robotic	268 (15%)

Complications

There were 1465 complications recorded in 779 patients, for an overall 90-day morbidity rate of 43% and mortality rate of 0.7%. The majority (52%) of patients who suffered a complication had a single complication. A list of the most common complications is provided in Figure 1.

Figure 1.

Frequency of 10 most common complications. SSI, surgical site infection; SBO, small bowel obstruction; VTE, venous thromboembolism; UTI, urinary tract infection.

When classified by CD grade, grade 2 complications were the most frequent (47%), followed by grade 1 (40%) (Supplementary Table 1). Together, these low-grade complications accounted for 87% of all complications.

The SSI rate was 22%, including superficial, deep, and organ space infections. Most of these (75%) were superficial or deep incisional infections, and the majority of these were grade 1 (57%) or grade 2 (41%).

Per-Patient Highest CD Grade Complications

The number and proportion of patients with complications stratified by their highest graded complication are presented in Table 3. Patients whose highest CD complication grade was 2 (PCD 2) or 1 (PCD 1) accounted for 80% of the 779 patients with a postoperative complication. Patients whose highest complication was grade 3 (PCD 3) were most likely to have an attributable readmission (75 of 131 patients, 57%) (Table 3). Fourteen patients (1%) with no complications had a readmission in 90 days, most commonly for chemotherapy-related adverse events.

Table 3. Number and proportion of patients with any complication stratified by the highest grade complication experienced.

As some patients had more than one complication, those listed are not necessarily the highest grade.

PCD Category	Complication (3 most common)	n (%)a	Requiring readmissionb	Requiring homecareb
PCD 1		230 (30%)	21 (9%)	138 (60%)
	Superficial or deep incisional SSI	103 (13%)
	Ileus or small bowel obstruction	38 (5%)
	Atelectasis	21 (3%)
PCD 2		389 (50%)	89 (23%)	202 (52%)
	Superficial or deep incisional SSI	164 (21%)
	Ileus or small bowel obstruction	91 (12%)
	Hemorrhage or anemia	53 (7%)
PCD 3		131 (17%)	75 (57%)	92 (70%)
	Organ space SSI	81 (10%)
	Superficial or deep surgical site infection	58 (7%)
	Ileus or small bowel obstruction	31 (4%)
PCD 4+		29 (4%)	10 (35%)	14 (48%)
	Organ space SSI	9 (1%)
	Hemorrhage or anemia	8 (1%)
	Ileus or small bowel obstruction	8 (1%)

^aPercent of all patients with at least one complication (n = 779). Some patients experienced more than one complication within one or more grades.

^bPercent of patients with most severe complication of that CD grade.

PCD, highest graded complication per patient; SSI, surgical site infection

In sum, 535 patients required homecare services after discharge and 36 were discharged to rehab. Most homecare referrals were for attributable complications (501, 94%), with the most common being grade 1 and 2 superficial and deep incisional SSIs. Patients requiring homecare received services for an average of 51 days (95% CI, 42.8–53.4). A subset of patients (93/535, 16%) had multiple indications for homecare, usually because of a secondary referral to physical therapy. Among the PCD 3 patients, 50 (31%) ultimately required homecare for the management of lesser grade 1 or 2 superficial or deep SSI. Finally, 89 patients with no complications (9%) had a referral for home physical therapy. Indications for readmissions and homecare services are listed in Supplementary Tables 2 and 3.

Per-Patient Costs by Highest CD Grade Complication

Figure 2a represents the per-patient estimated index hospitalization and post-discharge (up to 90 days post-surgery) costs, categorized by PCD grade. Increasing PCD grade was associated with greater total 90-day cost (p < 0.001) in both univariate and multivariable analyses. Readmissions accounted for most of the cost differences in post-discharge costs between patients with low-grade complications (PCD 1 and PCD 2) and those with PCD 3+ patients. On average, readmission costs accounted for only 13% of total post-discharge costs for PCD 1 and PCD 2 patients, compared with 46% for PCD 3+ patients. Patients with no complications had significantly lower costs compared to all PCD categories. Costs are further elaborated in Supplementary Table 4.

Figure 2a and 2b.

Estimated cost of care according to a) highest graded complication per patient (PCD) for all patients with complications, and b) complication grade in patients with a single complication, in Medicare proportional dollars (MP$). Excludes day of surgery costs, rounded to the nearest $10. Data are presented as median ± SE.

Overall 90-day costs differed significantly between PCD 1 and PCD 2 patients on both univariate and multivariable analyses. The largest driver of this difference was inpatient utilization during the index admission, as length of stay was significantly longer among PCD 2 patients (8 days vs. 7 days for PCD 1; p < 0.001). In contrast, there was no difference in post-discharge costs between PCD 1 and PCD 2 patients on either univariate or multivariable analyses (p = 0.75 unadjusted, 0.67 adjusted). In these low-grade complication categories, post-discharge costs contributed more to the 90-day total than did the index admission.

Figure 2b shows the per-patient estimated index admission and post-discharge costs in the 405 patients (52%) who had only a single complication. Among patients with grade 1 and 2 complications, there was no association between increasing complication grade and cost in the index admission, post-discharge or total 90-day periods. Figure 3 shows these costs according to the complication suffered. Again, costs were similar between within the low-grade categories, and for complications such as SSI and ileus, grade 1 complications appeared to incur more costs than grade 2.

Figure 3.

Estimated cost of individual complications. Data are derived from patients with only 1 complication (n = 405) and represented as 2014 Medicare proportional dollars (MP$) rounded to the nearest $10, ± SE, excluding surgery costs. Only complications affecting ≥ 5 patients are shown.

Superficial and deep incisional SSIs were the most common indications for homecare in PCD 1 and PCD 2 patients. There was no difference in mean homecare costs (p = 0.31 unadjusted, p = 0.07 adjusted) or length of homecare services (p = 0.45 unadjusted, p = 0.14) between these categories on unadjusted and adjusted analyses. Other common low-grade complications including ileus and hemorrhage or anemia requiring transfusion, did not require referral to homecare services. Venous thromboembolism, including deep vein thrombosis and pulmonary embolism, commonly required homecare referral for anticoagulation.

When we analyzed costs by setting (inpatient vs outpatient) instead of time-period (index admission vs post-discharge), we saw similar associations (Supplementary Table 5). There was a significant difference in inpatient costs between PCD 1 and PCD 2 patients, but their outpatient costs were the same. Outpatient costs accounted for the majority of costs in PCD 1 patients (56%). Among PCD 1 and PCD 2 patients with an SSI, there was no difference in outpatient or total 90-day costs.

DISCUSSION

This study confirmed the strong correlation between Clavien-Dindo complication grade and severity, as quantified by attributable inpatient and outpatient direct medical costs in the first 90 days after surgery. Among PCD 1 and PCD 2 patients, the association between cost and grade was driven by a 1-day length of stay difference during the index admission, with no difference seen in the substantial post-discharge costs between these groups. When the analysis was limited to patients with a single complication, we found no difference in the 90-day, index admission or post-discharge costs between patients with grade 1 versus grade 2 complications.

Our findings suggest that the CD classification underestimates the burden of some complications after colectomy. This is especially true for patients whose complications were low-grade and substantially managed in the outpatient setting, which account for 80% of patients with a complication after colectomy. The dominating effect of PCD grade on overall 90-day costs is due to the high cost of inpatient care relative to outpatient costs; for example, the average cost of 1 day of inpatient care was nearly 6 times the costs associated with a single day of homecare.

Healthcare utilization beyond the discharge period is rarely quantified or included in surgical outcomes studies.^19–21 In our cohort, most costs were incurred in the post-discharge period for PCD 1–3 patients, and nearly 1/3 of all patients received homecare for an attributable indication. Among PCD 1 and PCD 2 patients, 55% required homecare services for average duration was 51 days, consistent with the few reports from the colorectal literature. In the CLASICC trial, post-discharge outpatient utilization was also common (38.3–77.4%, depending on the service).²⁰ In a study of colorectal cancer patients from a Canadian registry, nearly 2/3 of patients received homecare services.²²

The severity of superficial and deep incisional SSIs was potentially the most underestimated complication by the CD classification. Nearly all SSIs (98%) were grade 1 or 2, yet they represented 49% of referrals to homecare. In addition, 31% of PCD 3 patients required homecare specifically for the treatment of lower-graded SSIs. These data suggest that for some patients, care for their high-grade complications ended with discharge, while care for their low-grade complications continued for a significant time after. In patients with only one complication, SSIs were the most expensive low-grade complications, and grade 1 SSIs were indistinguishable from grade 2 SSIs in terms of cost. At a minimum, we suggest that deep and superficial incisional SSIs should be classified as grade 2 complications, regardless of whether antibiotics are used.

The limitations of the CD classification likely apply to other indices. Both the Accordion Severity Grading System and the Comprehensive Complication Index (CCI)^1,9 were created by surveying experts regarding the inpatient care required to manage complications.^23,24 While the framework is intuitive, our study exposes a pitfall of this approach, which tends to focus on easy-to-quantify, hospital-based treatments. By considering the long-term burden associated with complications, many difficult-to-quantify “treatments”—time, dressing changes, nursing care, absenteeism— can be accounted for. Our findings suggest that optimization of the CD classification for colectomy studies requires a better understanding of the true burden associated with complications. Additional modifications including item selection to focus on common and impactful complications could help improve responsiveness and discriminative ability. Ultimately, a colorectal-specific outcome measure based on the Clavien-Dindo framework may be ideal.

This was a retrospective study conducted at a specialized cancer center. While this allowed us to minimize practice pattern variations, it may also limit the generalizability of our findings.²⁵ Compared to nationwide cohorts, our patients were younger, but had similar AJCC stages at diagnosis.²⁶ The age discrepancy permitted us to analyze healthcare utilization and complications among a growing cohort of younger colorectal cancer patients who are not Medicare beneficiaries.²⁶ Our 43% complication rate might appear high, but we believe this is reflective of a known underreporting of low-grade complications in the literature.^27–29 At least one other study that aimed to captured all low-grade complications reported a similar 41% overall morbidity rate.²⁸

An additional limitation of this study is missing data, especially in the outpatient setting. While costs were imputed using validated methods, imputation can lead to less precise estimates. The inclusion of direct non-medical costs, such as those related to absenteeism from work in patients and their caregivers, was not within the scope of the present study, but their impacts cannot be underestimated.^30–32 Inclusion of these costs in our analysis might alter some of our findings, as patients who require significant prolonged outpatient care and can’t work could incur more “costs” than those whose treatment ended at discharge.

Conclusion

In conclusion, the Clavien-Dindo classification may underestimate the true severity of many low-grade complications after colectomy, which are the most common complications after colectomy. A better understanding of the burden associated with surgical complications could guide modifications of the well-known CD classification or development of a more discriminative and sensitive outcome measure.