Accuracy of administrative and clinical registry data in reporting postoperative complications after surgery for oral cavity squamous cell carcinoma

Abstract

Background

The purpose of this study was to describe and compare how postoperative complications after oral cavity squamous cell carcinoma (SCC) surgery are reported in medical records, institutional billing claims, and national clinical registries.

Methods

The medical records of 355 previously untreated patients who underwent surgery for oral cavity SCC at our institution were retrospectively reviewed for postoperative complications. Information was compared with claims and National Surgical Quality Improvement Program (NSQIP) data.

Results

We identified 219 patients (62%) experiencing 544 complications (10% major). Billing claims identified 29% of these patients, 36% of overall complications, and 98% of major complications. Of overlapping patients, NSQIP identified 27% of patients, 33% of overall complications, and 100% of major complications noted on chart abstraction.

Conclusion

The incidence of minor postoperative complications after oral cavity SCC surgery is relatively high. Both claims data and NSQIP accurately recorded major complications, but were suboptimal compared to chart abstraction in capturing minor complications.

INTRODUCTION

There is a large variance in outcome data among hospitals for many operative procedures, including head and neck oncologic surgery.^1,2 Recognizing the importance of standardizing the quality of care delivery, the Quality of Care Committee of the American Head and Neck Society (AHNS) identified specific quality measures for the management of oral cavity and laryngeal cancer.^3,4 These are identified in Donabedian’s conceptual model, which evaluates quality of care according to the 3 dimensions of structure (the health care environment), process (what the provider does), and outcomes (what happens to the patient).⁵ When analyzed according to this model, it is evident that the AHNS recommends measures that emphasize processes of care. The National Quality Forum (mandated by the recently enacted Patient Protection and Affordable Care Act to develop quality metrics in health care) has endorsed general standards that are similarly process-oriented. However, there is mounting evidence to show that focusing solely on process may not directly correlate with measurable, clinically relevant outcomes.^6,7 The current practice of emphasizing process when defining quality is primarily based on convenience, as it is significantly easier and cheaper to measure.⁸ Particularly in head and neck cancer, comparisons based upon quality are vexing, given the heterogeneity of the disease and its management, compounded by the significant medical comorbidities and social habits of the patient population.⁹

Implementing value-based competition in the health care system can improve performance and contain costs.¹⁰ Unlike previous quality initiatives, value emphasizes outcomes, rather than simply volume or processes of care. A tiered hierarchy has been described to evaluate outcome, categorized into survival and degree of recovery, the process of recovery and disutility of care, and the sustainability of health and long-term consequences of therapy.¹¹ Because they represent the culmination of the health care services an individual patient receives, outcome measures offer the greatest opportunity to accurately quantify quality of care.¹²

At present, however, systematic and widely accepted outcome measures in head and neck cancer care simply do not exist. Despite this, measurable outcomes after surgery, such as perioperative mortality, complications, and readmission rates, are becoming increasingly more important, with public reporting and value-based purchasing growing in popularity. Realizing the benefit of implementing value in health care, the Affordable Care Act has mandated that cancer hospitals begin publicly reporting outcomes by 2014.¹³ Because the integrity of public reporting and value-based purchasing is reliant on the accurate recording of surgical complications, standardizing the definitions and diagnosis of complications and their documentation is essential.

In many respects, quality improvement is limited by the fidelity of the data available. In order for outcome measures to take center stage in benchmarking the surgical management of head and neck cancer, a uniform data repository must be agreed upon. Standardization of data collection efforts will mitigate concerns regarding the accuracy and reliability of performance data. Furthermore, in order for providers to improve, the data on which their performance feedback is based must be creditable and actionable.

We hypothesize that complications after surgery for oral cavity squamous cell carcinoma (SCC) remain under-reported, and better understanding of measurable outcomes may improve care delivery and comparison of outcomes across institutions. The purpose of this study was to describe and compare postoperative complications documented in the medical records with institutional billing/insurance claims data and a national clinical registry in a cohort of previously untreated patients undergoing surgery for oral cavity SCC.

PATIENTS AND METHODS

Patients

The medical records of patients with previously untreated oral cavity SCC undergoing surgery at Memorial Sloan–Kettering Cancer Center between January 2009 and December 2012 were accessed and reviewed. These cases were identified from an institutional database that excludes patients with a history of or treatment for head and neck cancer. Three hundred fifty-five patients were eligible for inclusion in the study. Patient demographics, clinical characteristics, health behaviors, oncologic characteristics, and surgical details were extracted from the medical record. Staging was recorded according to the American Joint Committee on Cancer Cancer Staging Manual 7th Edition.¹⁴ This study was assessed by the Memorial Sloan–Kettering Cancer Center Institutional Review Board and was approved after being deemed exempt from formal review.

Primary outcome

The postoperative period was defined as the time interval from the date of surgery to either the date of discharge from the hospital or up to 45 days, whichever occurred later. In the instance that a patient received a staged neck dissection within this period, only complications from date of index surgery to date of the second procedure were recorded. Operative and anesthesia reports, physician progress notes, nursing notes, laboratory reports, radiology findings, outpatient clinic visits, and nursing telephone conversations were reviewed from each patient record.

A “wide net” approach was utilized to mitigate underreporting and the potential for recall bias. Rather than define complications a priori, all postoperative events were recorded. A postoperative event was defined, based upon prior research methodology, as any unexpected deviation from the normal postoperative course not inherent to the procedure itself (or its sequelae), and not comprising a failure to cure.¹⁵ Asymptomatic, self-resolved metabolic derangements were also excluded.

After 2 rounds of independent review of the compiled list of postoperative events, the authors developed a systematic definition of a postoperative complication. A postoperative complication was defined as any deviation from the normal postoperative course not better explained by a previous medical condition, not inherent to the procedure or hospital course, and not reflective of the underlying pathophysiology of the primary diagnosis.

Complications were categorized as local or systemic and defined using specific criteria designed to ensure homogeneity. Health care associated infections were defined according to the criteria used by Centers for Disease Control and Prevention.¹⁶ Wound dehiscence was defined as a spontaneous disruption, partial or complete, after closure. Pressure ulcers graded stage II or higher based upon the National Pressure Ulcer Advisory Panel classification were considered complications.¹⁷ Postoperative fever was defined as a body temperature ≥38.5°C not attributed to any other condition. Hypoxia was defined as oxygen saturation below 90% not attributed to any other condition. Only bleeding events requiring intervention were considered complications; oozing at the surgical site or bleeding that spontaneously resolved were classified as postoperative events only.

In order to ensure accuracy and completeness of the chart review, and in order to verify our observations, International Classification of Diseases, 9th Edition (ICD-9) diagnosis codes newly assigned to the study cohort during the postoperative period were retrieved from an institutional billing database. Medical records were re-reviewed when coding was nebulous. If multiple ICD-9 diagnosis codes were given to describe the same complication, duplicates were removed.

Complication severity was graded based upon the revised Clavien–Dindo classification (Table 1).¹⁸ The grading of “borderline” cases was extrapolated from this study group’s international survey testing the acceptability and reproducibility of difficult cases.¹⁹ When this was not sufficient to grade a complication, the case was discussed among the authors to achieve a consensus. We defined minor complications as grades I to II and major complications as grades III to V.

TABLE 1.

Clavien–Dindo classification of surgical complications.

Grade	Definition
Grade I	Any deviation from the normal postoperative course without the need for pharmacological treatment or surgical, endoscopic, and radiological interventions
	Allowed therapeutic regimens are: drugs as antiemetics, antipyretics, analgesics, diuretics, electrolytes, and physiotherapy. This grade also includes wound infections opened at the bedside.
Grade II	Requiring pharmacological treatment with drugs other than those allowed for grade I complications. Blood transfusions and total parenteral nutrition are also included.
Grade III	Requiring surgical, endoscopic, or radiological intervention.
IIIa	Intervention not under general anesthesia.
IIIb	Intervention under general anesthesia.
Grade IV	Life-threatening complication (including CNS complications)* requiring IC/ICU management.
IVa	Single organ dysfunction (including dialysis).
IVb	Multiorgan dysfunction.
Grade V	Death of a patient.

Abbreviations: CNS, central nervous system; IC, intermediate care; ICU, intensive care unit.

^*Brain hemorrhage, ischemic stroke, and subarachnoid bleeding, but excluding transient ischemic attacks.

Clinical variables from the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) institutional database were retrieved. Because the program utilizes a case sampling system, NSQIP data are only available for a percentage of patients, and NSQIP clinical abstractors are trained to identify a specific list of complications for a 30-day postoperative course. There was agreement in the definitions of the specific complications that NSQIP abstractors are trained to identify.²⁰ Furthermore, there was no overlap in the postoperative events that we excluded and the postoperative complications that NSQIP reports. To achieve a fair and unbiased comparison, only those complications that NSQIP was designed to report within a 30-day postoperative period were included in the comparison against the rates reported by chart review and ICD-9 claims data.

Data analysis

All data were compiled using Caisis (v6.0, BioDigital, New York, NY), an open source, web-based cancer data management system. The dataset was transferred into Excel (v12, Microsoft, Redmond, WA) to calculate frequencies. All percentages were rounded to the closest integer.

RESULTS

Patient characteristics

Patient demographics, clinical descriptors, tumor characteristics, and surgical details are summarized in Table 2. The median age of the patients was 61 years (range, 26–93 years) and 59% were men. Regarding procedures, 70% underwent a neck dissection, 36% had bone resection, and 40% received a reconstructive procedure after primary resection of the tumor, with free flap transfer being the most common (21%). Additionally, 26% had a tracheostomy and 11% had gastrostomy or jejunostomy feeding tube placement.

TABLE 2.

Patient characteristics (n = 355).

Demographics	No. of patients (%)
Age, y, median (range)	61 (26–93)
Sex (male)	209 (59)
BMI, median (range)	27 (15–61)
Smoking
Current	100 (28)
Former	123 (35)
Never	132 (37)
Alcohol, ≥5 drinks per wk	86 (24)
Tumor site
Oral tongue	200 (56)
Mandibular alveolus	48 (14)
Buccal mucosa	36 (10)
Floor of mouth	34 (10)
Maxillary alveolus	24 (7)
Retromolar trigone	9 (3)
Hard palate	4 (1)
Neck dissection
None	108 (30)
Ipsilateral	210 (59)
Bilateral	37 (10)
Ancillary procedures
Tracheostomy	93 (26)
Postoperative PEG/PEJ	40 (11)
Bone resection
None	223 (63)
Marginal mandibulectomy	58 (16)
Segmental mandibulectomy	31 (9)
Mandibulotomy	7 (2)
Maxillectomy	24 (7)
Mandibular and maxillary resection	12 (3)
Reconstruction
None*	216 (61)
Locoregional flap	19 (5)
Microvascular free tissue transfer	74 (21)
Skin graft†	46 (13)
Clinical T classification
T1	152 (43)
T2	122 (34)
T3	26 (7)
T4a	55 (15)
Clinical N classification
N0	238 (67)
N1	43 (12)
N2	72 (20)
N3	2 (1)
Overall disease stage
I	132 (37)
II	75 (21)
III	47 (13)
IV	101 (28)

Abbreviations: BMI, body mass index; PEG, percutaneous endoscopic gastrostomy; PEJ, percutaneous endoscopic jejunostomy.

^*Includes primary closure, healing by secondary intention, and obturator placement.

^†Includes split thickness skin grafts, full thickness skin grafts, and AlloDerm.

Postoperative complications

We identified 1623 postoperative events. After 2 rounds of review, a total of 544 (34%) complications were compiled. Of the excluded events, 691 were postoperative events (local inflammatory changes to the surgical field, such as redness and edema) and tracheostomy-related secretions, as well as isolated abnormal vital signs because of difficulty in determining the normal range from retrospective chart abstraction. The final 388 excluded postoperative events were those that are inherent to surgery and hospitalization (eg, alteration of bowel habits, nausea and vomiting, stitch abscesses, numbness/tingling at the surgical site, oozing, postoperative anemia, transient hematuria related to Foley catheterization, clogged or dislodged feeding tubes necessitating replacement, skin rashes, oral thrush, generalized edema, and dry or irritated eyes) as well as conditions better attributed to underlying medical history (eg, pulmonary hypertension, tinea cruris, paroxysmal atrial fibrillation, and vasovagal episodes).

Perioperative morbidity and mortality

Overall, 219 of the 355 patients developed 544 complications, resulting in a total complication rate of 62% with 36 patients (10%) experiencing 55 major (>grade II) complications. Of patients developing complications, there was an average rate of 2.5 complications experienced per patient. In addition, there were 20 hospital readmissions in 16 patients during the study period, for a hospital readmission rate of 5%.

The frequencies of complications according to severity and type (local vs systemic) are reported in Figure 1. The majority of complications was minor (90%) and was mostly local (56%), whereas major complications were more likely to be systemic (67%). There were 3 postoperative deaths, for an overall mortality rate within the cohort of 0.8%. The 3 deaths were due to congestive heart failure, myocardial infarction, and pneumonia, respectively.

FIGURE 1.

Frequency (%) of complications by severity and type as recorded in the medical record.

Local complications

Table 3 summarizes the local complications of the study cohort. There were 100 patients (28%) who experienced 118 complications related to the surgical resection of the primary tumor in the oral cavity. Of these, 7 were grade IIIb (2%), including 3 bleeding events, 2 fistulas, 1 ectropion, and 1 temporomandibular joint (TMJ) dislocation. Of the 247 patients undergoing neck dissections, 85 (34%) experienced 112 complications. There was a major complication rate of 2% (5 of 247), all of which were grade IIIb. Four patients developed hematomas requiring surgical evacuation, and 1 patient developed wound breakdown requiring surgical debridement. A total of 57 patients (23%) experienced 67 nerve pareses; 43 (64%) were spinal accessory nerves, 22 (33%) facial nerves, 1 (2%) was of the brachial plexus, and 1 (2%) hypoglossal nerve. None of the nerves were transected during the procedure. Nerves that were sacrificed for oncologic reasons were not considered complications. All nerve injuries were either observed or treated with physical therapy (grade I), with all patients regaining nerve function by 1 year after surgery.

TABLE 3.

Local complications.

	No. of complications by severity
	Minor		Major
Type of complication	Grade I	Grade II	Grade III*	Grade IV†	Total no. (%)
Neck
Cranial nerve paresis	67	0	0	0	67 (60)
Infection	0	19	0	0	19 (17)
Hematoma	4	0	4	0	8 (7)
Wound breakdown/dehiscence	6	0	1	0	7 (6)
Chyle leak	5	1	0	0	6 (5)
Seroma	5	0	0	0	5 (4)
Total	87	20	5	0	112
Oral cavity
Trismus	36	0	0	0	36 (31)
Wound breakdown/dehiscence	25	0	0	0	25 (21)
Orocutaneous fistula	2	11	2	0	15 (13)
Hemorrhage	6	4	2	0	12 (10)
Necrosis	9	0	0	0	9 (8)
Infection	0	6	0	0	6 (5)
Orbital complication	2	1	1	0	4 (3)
Burn/trauma	3	0	0	0	3 (3)
Hematoma	1	0	1	0	2 (2)
Hoarseness/stridor	2	0	0	0	2 (2)
Salivary gland infection	0	2	0	0	2 (2)
Epistaxis	1	0	0	0	1 (1)
TMJ dislocation	0	0	1	0	1 (1)
Total	87	24	7	0	118
Flap
Flap dehiscence	17	0	0	0	17 (65)
Partial flap failure	5	0	2	0	7 (27)
Total flap failure	0	0	1	0	1 (4)
Hematoma	0	0	1	0	1 (4)
Total	22	0	4	0	26
Flap donor
Wound breakdown/dehiscence	18	0	0	0	18 (72)
Infection	2	3	0	0	5 (20)
Limb compartment syndrome	1	0	0	0	1 (4)
Nerve injury	1	0	0	0	1 (4)
Total	22	3	0	0	25
PEG/PEJ
Cellulitis	0	3	0	0	3 (50)
Bowel necrosis	0	0	0	1	1 (17)
Upper gastrointestinal bleeding	0	0	1	0	1 (17)
Wound breakdown/dehiscence	1	0	0	0	1 (17)
Total	1	3	1	1	6
Tracheostomy
Fistula	1	0	0	0	1 (33)
Hemorrhage	0	1	0	0	1 (33)
Subcutaneous emphysema	1	0	0	0	1 (33)
Total	2	1	0	0	3

Abbreviations: TMJ, temporomandibular joint; PEG, percutaneous endoscopic gastrostomy; PEJ, percutaneous endoscopic jejunostomy.

^*All grade IIIb.

^†All grade IVb.

Of 141 patients undergoing reconstruction, 43 (30%) experienced 51 complications, of which 4 were major (3%). Of these, 35 involved wound dehiscence (17 in the oral cavity and the remaining 18 at the donor site), all of which were grade I. There were 8 (6%) flap failures, 3 of which required operative intervention, and 1 of which resulted in total flap loss (0.7%). There were no deaths (grade V) because of local complications.

Systemic complications

Of the 544 complications, 254 (47%) were systemic, affecting 123 patients (35%; Table 4). Of these, 25 patients (7%) experienced 37 major complications. Fifteen of the patients with pulmonary complications required mechanical ventilation (grade IVa or higher). Three additional patients were on ventilatory support for >48 hours for congestive heart failure, stroke, and delirium tremens. Therefore, there was a respiratory failure rate of 5% (18 of 355). There were 23 cardiac arrhythmias, including 12 episodes of supraventricular tachycardia (including atrial fibrillation), and 7 paroxysmal ventricular tachycardias (all asymptomatic and discovered on telemetry). There were 7 cases of acute coronary syndrome, 2 of which were myocardial infarctions (0.6%). Two patients (0.6%) had a cerebrovascular accident. Seven patients (2%) met clinical criteria for sepsis, 5 because of pneumonia, 1 catheter-related infection, and 1 intra-abdominal infection. There was a venous thromboembolism rate of 2% (7 of 355), including isolated deep venous thrombosis (4), and pulmonary embolism (3).

TABLE 4.

Systemic complications.

	No. of complications by severity
	Minor		Major
Type of complication	Grade I	Grade II	Grade III*	Grade IVa	Grade IVb	Grade V	Total no. (%)
Pulmonary
Pneumonia	0	14	0	3	7	1	25 (38)
Atelectasis	18	0	0	0	0	0	18 (27)
Hypoxia	10	0	0	0	0	0	10 (15)
Pulmonary edema	5	0	0	3	0	0	8 (12)
Aspiration	3	0	0	0	0	0	3 (5)
Foreign body	0	0	1	0	0	0	1 (2)
Respiratory failure NOS	0	0	0	0	1	0	1 (2)
Total	36	14	1	6	8	1	66
Nervous
Delirium	8	18	0	3	4	0	33 (75)
Altered mental status	3	1	0	0	0	0	4 (9)
Cerebrovascular accident	0	0	0	1	1	0	2 (5)
Extrapyramidal symptoms	1	1	0	0	0	0	2 (5)
Neuropathy	2	0	0	0	0	0	2 (5)
Quadriplegia†	0	1	0	0	0	0	1 (2)
Total	14	21	0	4	5	0	44
Cardiac
Arrhythmia	10	12	0	1	0	0	23 (66)
Acute coronary syndrome	1	4	0	0	1	1	7 (20)
Congestive heart failure	0	1	0	1	2	1	5 (14)
Total	11	17	0	2	3	2	35
Infection
Postoperative fever	9	4	0	0	0	0	13 (43)
Urinary tract infection	0	10	0	0	0	0	10 (33)
Catheter-related infection	0	3	0	0	1	0	4 (13)
Other‡	0	3	0	0	0	0	3 (10)
Total	9	20	0	0	1	0	30
Hematological
Catheter-related complications	7	1	0	0	0	0	8 (38)
Venous thromboembolism	0	4	2	1	0	0	7 (33)
Coagulopathy	1	0	0	1	0	0	2 (10)
Transfusion reaction	1	1	0	0	0	0	2 (10)
Peripheral arterial ischemia	1	0	0	0	0	0	1 (5)
Thrombocytopenia	1	0	0	0	0	0	1 (5)
Total	11	6	2	2	0	0	21
Gastrointestinal
Clostridium difficile colitis	0	3	0	0	0	0	3 (38)
Cholestasis/cholecystitis	2	1	0	0	0	0	3 (38)
Lower gastrointestinal bleeding	0	2	0	0	0	0	2 (25)
Total	2	6	0	0	0	0	8
Metabolic
Dehydration/malnutrition	3	2	0	0	0	0	5 (83)
Refeeding syndrome	1	0	0	0	0	0	1 (17)
Total	4	2	0	0	0	0	6
Other
Urinary retention	21	0	0	0	0	0	21 (48)
Pressure ulcer	18	0	0	0	0	0	18 (41)
Allergic reaction	0	2	0	0	0	0	2 (5)
Intramuscular hematoma/rhabdomyolysis	1	1	0	0	0	0	2 (5)
Syndrome of inappropriate ADH	0	1	0	0	0	0	1 (2)
Total	40	4	0	0	0	0	44

Abbreviations: NOS, not otherwise specified; ADH, antidiuretic hormone.

^*All grade IIIb.

^†Transient, of unknown etiology.

^‡Includes cellulitis, sinusitis, and herpes zoster.

Comparison with International Classification of Disease 9th revision codes

According to institutional billing claims data, 103 patients (29%) experienced a total of 195 complications, 114 of which were systemic (59%). This accounts for 36% of complications identified during chart review (Tables 5 and 6). Of these, 28% of the local complications and 45% of the systemic complications abstracted from patient records had corroborating ICD-9 diagnosis codes. Of the 55 major complications recorded during chart review, the ICD-9 diagnosis codes accounted for 54 (98%), as shown in Table 7.

TABLE 5.

Comparison of local complications with International Classification of Disease 9th revision codes.

	No. of complications identified by:
Type of complication	ICD-9 code	Chart review	Concordance (%)
Neck
Cranial nerve paresis	5	67	7
Infection	10	19	53
Hematoma	4	8	50
Wound breakdown/ dehiscence	1	7	14
Lymphatic leak	0	6	0
Seroma	3	5	60
Total	23	112	21
Oral cavity
Trismus	0	36	0
Wound breakdown/ dehiscence	4	25	16
Fistula	12	15	80
Hemorrhage	9	12	75
Necrosis	1	9	11
Infection	3	6	50
Orbital complication	1	4	25
Burn/trauma	0	3	0
Hematoma	2	2	100
Hoarseness/stridor	0	2	0
Salivary gland infection	1	2	50
Epistaxis	1	1	100
TMJ dislocation	1	1	100
Total	35	118	30
Flap
Flap dehiscence	8	17	47
Partial flap failure	3	7	43
Total flap failure	1	1	100
Hematoma	1	1	100
Total	13	26	50
Flap donor
Wound breakdown/ dehiscence	2	18	11
Infection	1	5	20
Limb compartment syndrome	1	1	100
Nerve injury	0	1	0
Total	4	25	16
PEG/PEJ
Cellulitis	2	3	67
Bowel necrosis	1	1	100
Upper gastrointestinal bleeding	1	1	100
Wound breakdown/ dehiscence	0	1	0
Total	4	6	67
Tracheostomy
Fistula	0	1	0
Hemorrhage	1	1	100
Subcutaneous emphysema	1	1	100
Total	2	3	67

Abbreviations: ICD-9, International Classification of Disease 9th revision; TMJ, temporoman-dibular joint; PEG, percutaneous endoscopic gastrostomy; PEJ, percutaneous endoscopic jejunostomy.

TABLE 6.

Comparison of systemic complications with International Classification of Disease 9th revision codes.

	No. of complications identified by:
Type of complication	ICD-9 codes	Chart review	Concordance (%)
Pulmonary
Pneumonia	18	25	72
Atelectasis	3	18	17
Hypoxia	2	10	20
Pulmonary edema	7	8	88
Aspiration	0	3	0
Foreign body	1	1	100
Respiratory failure NOS	1	1	100
Total	32	66	48
Nervous
Delirium	16	33	48
Altered mental status	2	4	50
Cerebrovascular accident	1	2	50
Extrapyramidal symptoms	1	2	50
Neuropathy	0	2	0
Quadriplegia	1	1	100
Total	21	44	48
Cardiac
Arrhythmia	16	23	70
Acute coronary syndrome	4	7	57
Congestive heart failure	5	5	100
Total	25	35	71
Infection
Postoperative fever	0	13	0
Urinary tract infection	4	10	40
Catheter-related infection	1	4	25
Other	2	3	67
Total	7	30	23
Hematological
Catheter-related complications	1	8	13
Venous thromboembolism	6	7	86
Coagulopathy	2	2	100
Transfusion reaction	0	2	0
Peripheral arterial ischemia	0	1	0
Thrombocytopenia	1	1	100
Total	10	21	48
GI
Clostridium difficile colitis	2	3	67
Cholestasis/cholecystitis	1	3	33
Lower GI bleeding	1	2	50
Total	4	8	50
Metabolic
Dehydration/malnutrition	2	5	40
Refeeding syndrome	0	1	0
Total	2	6	33
Other
Urinary retention	2	21	10
Pressure ulcer	9	18	50
Allergic reaction	0	2	0
Intramuscular hematoma/ rhabdomyolysis	1	2	50
Syndrome of inappropriate ADH	1	1	100
Total	13	44	30

Abbreviations: ICD-9, International Classification of Disease 9th revision; NOS, not otherwise specified; GI, gastrointestinal; ADH, antidiuretic hormone.

TABLE 7.

Summary and comparison of major complications with International Classification of Disease 9th revision codes.

	No. of complications identified by:
	Chart review
Type of complication	Grade III*	Grade IV†	Grade V	ICD-9	Concordance (%)
Neck hematoma	4	0	0	4	100
Neck dehiscence	1	0	0	1	100
Oral cavity hemorrhage	2	0	0	2	100
Oral cavity hematoma	1	0	0	1	100
Fistula	2	0	0	2	100
Ectropion	1	0	0	1	100
TMJ dislocation	1	0	0	1	100
Partial flap failure	2	0	0	2	100
Total flap failure	1	0	0	1	100
Flap hematoma	1	0	0	1	100
PEJ bowel necrosis	0	1	0	1	100
PEG upper GI bleeding	1	0	0	1	100
Pneumonia	0	10	1	11	100
Pulmonary edema	0	3	0	3	100
Foreign body	1	0	0	1	100
Respiratory failure NOS	0	1	0	1	100
Delirium	0	7	0	7	100
Cerebrovascular accident	0	2	0	1	50
Congestive heart failure	0	3	1	4	100
Myocardial infarction	0	1	1	2	100
Atrial fibrillation	0	1	0	1	100
Venous thromboembolism	2	1	0	3	100
Coagulopathy	0	1	0	1	100
Catheter-related infection	0	1	0	1	100
Total	20	32	3	54	98

Abbreviations: ICD-9, International Classification of Disease 9th revision; TMJ, temporoman-dibular joint; PEJ, percutaneous endoscopic jejunostomy; PEG, percutaneous endoscopic gas-trostomy; GI, gastrointestinal; NOS, not otherwise specified.

^*All grade IIIb.

^†Includes grades IVa and IVb.

Three complications were assigned ICD-9 diagnosis codes that were not identified during initial chart review. These were for flap dehiscence, spinal accessory nerve injury, and paroxysmal ventricular tachycardia, which were verified on re-review of medical records.

Comparison with National Surgical Quality Improvement Program data

NSQIP data were available for 27 of the 355 patients in our study cohort (8%), 3 (11%) of whom developed 5 complications. Of these patients, chart review identified 15 complications experienced by 11 patients, according to NSQIP nomenclature and within a postoperative period of 30 days. Therefore, NSQIP identified 27% of patients and 33% of complications. Meanwhile, ICD-9 codes identified 4 patients (36%) who developed 7 complications (47%), as shown in Table 8. There were 2 major complications among the 27 patients, which were correctly identified by all 3 methods. Table 9 summarizes the proportion of complications accurately identified by institutional billing data and NSQIP, stratified by complication severity. Both administrative and clinical registry data achieve greater accuracy in reporting with increasing severity of complications.

TABLE 8.

Comparison of complications with National Surgical Quality Improvement Program and International Classification of Disease 9th revision codes (n = 27).

Type of complication	No. of complications identified by:
	NSQIP	ICD-9	Chart review
Peripheral nerve injury	0	1	6
Superficial surgical site infection	2	3	4
Wound disruption	0	0	2
Bleeding/transfusions	1	1	1
Return to operating room*	1	1	1
On ventilator >48 h†	1	1	1
Total	5	7	15

Abbreviations: NSQIP, National Surgical Quality Improvement Program; ICD-9, International Classification of Disease 9th revision.

^*For neck hematoma.

^†For delirium tremens.

TABLE 9.

Comparison of complications with National Surgical Quality Improvement Program and International Classification of Disease 9th revision codes by severity.

Complication severity	% of complications identified by:
	ICD-9	NSQIP
Minor	29	23
Grade I	18	0
Grade II	55	60
Major	98	100
Grade III	100	100
Grade IV	97	100
Grade V	100	100

Abbreviations: ICD-9, International Classification of Disease 9th revision; NSQIP, National Surgical Quality Improvement Program.

DISCUSSION

In this study, we recorded and graded postoperative complications for a cohort of surgically treated patients with oral cavity SCC using data abstracted from patient medical records and analyzed its concordance with administrative ICD-9 codes data and NSQIP clinical data. To our knowledge, this is the first comparison study of its kind in the head and neck oncologic surgery literature.

Our findings demonstrated a relatively high incidence of postoperative complications, as noted in the medical records after oral cavity SCC surgery. We reported a total and major complication rate of 62% and 10%, respectively. These rates are similar to another study looking exclusively at 182 patients undergoing surgery for oral cancer, reporting 47% and 15.6%, respectively.²¹ Patel et al²² also reported a relatively high major complication rate of 30% in their experience in free flap reconstruction of the head and neck. Meanwhile, a University Health Consortium database study reported overall complication rates ranging from 19.42% to 20.30%, depending on hospital volume.² This discrepancy reinforces the fact that the variability in reported postoperative complication rates is dependent on the data source used.

The relatively higher rates of complications that we report can also be explained by the level of scrutiny and rigor of our chart review and the use of a standardized grading scheme. Depending on the definition of a complication, a single institution can significantly modify its reported complication rates. Veen et al²³ were able to demonstrate a dramatic increase in the total number of registered complications simply by varying small aspects of their definition midway through the same study. There is fear in the surgical community that a high complication rate equates to substandard care. As a result, incentives to properly document complications remain ambiguous. Without clear incentives, the health care system as a whole may be denied much needed opportunities for improvement.²⁴

In our study, severity of complications was graded according to the scaled Clavien–Dindo classification, as opposed to qualitative modifiers, such as “major,” “intermediate,” or “minor.” In this grading scheme, the therapy required to treat a complication acts as a surrogate marker of severity, preserving objectivity.¹⁹ Since its publication, however, the scale has been utilized marginally in the head and neck literature.^25,26

Our findings are consistent with prior reports, albeit in other surgical subspecialties, comparing administrative datasets to chart review.^27,28 Heisler et al²⁷ demonstrated that claims data accurately identifies life-threatening complications, but that other complications after vaginal hysterectomy were underreported. This is consistent with our study, with ICD-9 codes accounting for 98% of all major complications but failing to report many minor complications.

Public reporting of hospital quality and pay-for-performance policies rely heavily on administrative data, because they are routinely collected for billing purposes and therefore readily available. In contrast, abstracting data directly from patient medical records, despite being arduous, time-consuming, and expensive, is more valid and reliable.^28,29

There are several limitations to utilizing ICD-9 codes as a proxy of quality. For one, it is often difficult to discern whether or not an ICD-9 diagnosis code describes a comorbidity or a complication. Furthermore, ICD-9 codes include conditions that are listed in the patient’s medical record as differential diagnoses, only to be systematically ruled out subsequent to assignment. To control for this, the reported postoperative complication rate using claims data was retrospectively modified and edited based upon review of the medical records, whenever coding was nebulous. Simply cross-walking postoperative complications from chart review to administratively reported ICD-9 diagnosis codes would have demonstrated an even greater discordance. Therefore, these drawbacks do not adequately explain the discrepancy in observed postoperative complication rates between chart review and administrative claims data. Instead, these drawbacks argue against the practicality of utilizing administrative data as a data source for effectively and efficiently capturing postoperative complications.

Other potential explanations for the discrepancy between chart review and administrative claims data include the coding process failing to capture diagnoses generated from telephone conversations with patients, claims data having a limited number of fields available for coding, incomplete or unclear medical documentation, and the variability in operating definitions and their application.^29,30 Despite these shortcomings, it is important to note that the ICD-9 codes did accurately report 98% of all major complications we identified in our review. In addition, the claims data identified 3 minor complications not discovered on the original chart review. On the other hand, administrative data failed to report the majority of minor complications.

It could be argued that considering the reliability of claims data in capturing major complications data, efforts at standardizing methodology for reporting and capturing minor complications may not improve public reporting of hospital quality and pay-for-performance policies. However, the majority of patients undergoing surgery for oral cancer develop minor complications, and a percentage of these patients could go on to develop major complications and even die if they are not managed appropriately. The ability to salvage patients successfully from this cycle of complications is a hallmark of an effective multidisciplinary surgical team, and is yet another yardstick by which performance may be measured and compared.¹ Additionally, management of minor complications incurs cost and morbidity, and is not always captured in surrogate markers, such as hospital length of stay. These seemingly “minor” complications, which are not currently being accurately reported with administrative databases, negatively impact quality of life and patient satisfaction. This becomes increasingly important when considering that patient satisfaction will be used by payers as a quality of care indicator to compare institutions and will have a direct impact on hospital reimbursement.³¹ Therefore, accurate recording and reporting of minor complications is as important as reporting major complications.

On October 1, 2014, the Department of Health and Human Services will implement the update to the current ICD-9 codes, or the International Classification of Diseases, 10th Edition, Clinical Modification/Procedure Coding System (ICD-10-CM/PCS).³² Unlike the usual updates to billing codes that occur annually, the ICD-10-CM is touted to be more robust, descriptive, and in line with current medical practices. The new system does not have the same restrictions on the number of codes that can be created (expanded now to allow up to 25 diagnoses). Because ICD-9 is over 30 years old, a new system was needed to accommodate new diagnoses, in line with advancements in technology and information, allowing for greater specificity and accuracy in assigning patients (ICD-10-CM includes approximately 80,000 diagnosis codes compared to ICD-9-CM’s 14,000). Despite such a massive expansion of potential diagnoses, the implications of these changes are not immediately clear. Certainly, the transition itself from ICD-9 to ICD-10 will be a barrier to the adoption of administrative data for surgical outcomes research.³³ This temporary loss of productivity during the transition period, while coders and health care providers alike are being educated and trained and the upfront costs are addressed, could have significant returns in improved data integrity, quality of care, and ultimately improved tracking of meaningful outcomes. However, the expansion of codes still does not address the inconsistencies in physician documentation (off of which the codes are based). Furthermore, whether or not coders will take advantage of the full spectrum of codes offered by ICD-10 is unclear, particularly in the absence of any incentives. Finally, there may be issues unbeknownst to us until the update is implemented before its impact on surgical research can be truly appreciated.³⁴ Not only do we not anticipate significant improvements in the reporting of postoperative complications after ICD-10 implementation, we believe that its introduction does not change the need within our specialty for a system that addresses the dearth of standards, guidelines, and definitions.

The ACS NSQIP represents the best attempt at systematically abstracting clinical data at an institutional level and reporting risk-adjusted surgical outcomes. NSQIP employs specially trained nurses to abstract details from the in-patient and out-patient records of a sample of patients throughout the 30-day postoperative period, with audits randomly performed by the ACS to improve the accuracy of data collection.³⁵ The data are reported to identify outliers among participating hospitals, who can then use the information to improve their own clinical outcomes,³⁶ with application in head and neck surgery as well.²⁴

In our study, 8% of our patients were sampled by NSQIP but there was only 27% concurrence in complication rates compared to chart review, despite only comparing complications that NSQIP abstractors are trained to identify. NSQIP sampling was less complete than was ICD-9 billing/claims data, which identified 47% of complications in the same patient cohort. NSQIP failed to identify any grade I complications, which could be explained by inconsistency and ambiguity in definitions of complications, such as peripheral nerve injury and wound disruption. However, there was significant incongruence even for a relatively well-defined complication, such as surgical site infection between NSQIP data and the ICD-9 codes in our study. It is important to note that these conclusions are drawn on a very small sample size – only 27 patients had NSQIP data available for comparison.

Contrary to our findings, NSQIP has consistently been shown to be superior to administrative claims data in reporting surgical complications.^29,37 These studies, however, use a crosswalk methodology to compare postoperative complications with ICD-9 diagnosis codes. The universal adoption of programs, such as NSQIP, has been limited by the labor-intensive, time-consuming, and expensive nature of data abstraction. Furthermore, the program does not record all complications unique to sub-specialties (such as head and neck surgery) and does not quantitatively grade the severity of complications. Although chart abstraction can be tailored to the surgical subspecialty being studied, this methodology fails to circumvent the expenses and time shared with a national clinical registry effort, such as NSQIP.

The limited number of complications that NSQIP was created to identify limits the head and neck surgical community from using this system to accurately measure the specialty-specific quality of care delivered. Many of complications identified in our chart review that are neglected in NSQIP, such as cardiac arrhythmias and postoperative delirium, have significant negative consequences, such as increased hospital costs, length of stay, and patient anxiety. Such complications, although not entirely specific to the field, are relatively more prevalent because of the medical comorbidities and social habits of the patients with head and neck cancer population. Specific head and neck complications, such as trismus and chyle leaks, are also important to accurately report as they represent a proportion of patients who are substantial health care consumers. This example also illustrates that severity of complications alone is not an adequate surrogate marker of quality; although trismus and chyle leaks are considered minor complications in the Clavien–Dindo classification, patients suffering of these complications require more medical attention, including longer hospital stays and referrals for specialized treatment. For these reasons, an expansion on the complications identified by NSQIP is necessary for there to be advancements in quality of care assessment in oral cavity SCC surgery.

There are limitations to our study. First, this was a retrospective analysis, and the variability in reporting complication practices meant many definitions were open to interpretation. This was minimized by looking at all data sources available, including laboratory values and radiological findings, as well as conducting meetings among the authors to discuss difficult cases. Second, a study of this magnitude is particularly time-consuming and tedious, and was only possible because of reliance upon institutional electronic medical records. Third, although we have focused primarily on postoperative complications, other patient-centered clinical (and oncologic) outcomes need to be measured to assess quality.¹¹ Finally, although this was not the focus of the current study, it is important to highlight that our results do not consider the influence of preexisting patient comorbidities and procedure acuity on complication rates. To avoid unfairly penalizing providers taking care of higher risk patients, outcome measures need to be risk-adjusted before they are compared across institutions.

Furthermore, the incidence of postoperative complications alone does not adequately explain the variability in postoperative mortality among hospitals. There is little doubt that timely recognition and effective management of complications once they have occurred is paramount to successful outcome.¹ Utilizing mortality or complications data as a proxy of quality has its limitations, and this “failure to rescue” phenomenon clearly demonstrates that other factors contribute to the overall quality of care providers deliver to their patients. Therefore, other metrics in addition to postoperative complication rates and mortality need proper documentation to facilitate measuring outcomes longitudinally throughout the treatment cycle.⁸

Our study demonstrates a need to agree on definitions of complications and to standardize methods to report complications in head and neck surgery, which can then be used prospectively to thwart the inaccuracies and costs associated with retrospective data abstraction. With data being collected prospectively, outcomes can be measured throughout the patients’ entire treatment cycle, and this would be conducive to a value-based health care system. If a certain standard of reporting is not set and met, the overall quality of data collection and reporting can be expected to suffer. We believe that for credible and actionable performance data to be accurately reported, we, as a specialty, need to define our own performance indicators based on data that we collect. The Society of Thoracic Surgeons, for example, has created their own National Database to develop benchmarks, facilitate public reporting, and devise quality improvement initiatives.³⁸ The AHNS is also acutely aware of these issues, and there are ongoing efforts toward the goal of improving quality of care. Until this ideal is achieved, public reporting and pay-for-performance policies using billing claims data or national registry data will have overarching ramifications for all involved parties. Better systems therefore need to be put in place expeditiously to adequately benchmark the outcomes of surgical management of patients with head and neck cancer.