Key Points
Question
Can a modified Delphi process identify postoperative readmissions associated with surgical quality?
Findings
In this modified Delphi consensus process study, 14 experts in medical and surgical readmissions built consensus on the association between postoperative readmissions and quality of surgical procedures using International Classification of Diseases, Ninth Revision codes. The panel’s consensus was that one-third of postoperative readmissions are unlikely to reflect problems with surgical quality.
Meaning
Restricting the definition of postoperative readmission to consensus-defined diagnosis codes might yield a more useful quality measure.
Abstract
Importance
Postoperative readmission data are used to measure hospital performance, yet the extent to which these readmissions reflect surgical quality is unknown.
Objective
To establish expert consensus on whether reasons for postoperative readmission are associated with the quality of surgery in the index admission.
Design, Setting, and Participants
In a modified Delphi process, a panel of 14 experts in medical and surgical readmissions comprising physicians and nonphysicians from Veterans Affairs (VA) and private-sector institutions reviewed 30-day postoperative readmissions from fiscal years 2008 through 2014 associated with inpatient surgical procedures performed at a VA medical center between October 1, 2007, and September 30, 2014. The consensus process was conducted from January through May 2017. Reasons for readmission were grouped into categories based on International Classification of Diseases, Ninth Revision (ICD-9) diagnosis codes. Panelists were given the proportion of readmissions coded by each reason and median (interquartile range) days to readmission. They answered the question, “Does the readmission reason reflect possible surgical quality of care problems in the index admission?” on a scale of 1 (never related) to 5 (directly related) in 3 rounds of consensus building. The consensus process was completed in May 2017 and data were analyzed in June 2017.
Main Outcomes and Measures
Consensus on proportion of ICD-9–coded readmission reasons that reflected quality of surgical procedure.
Results
In 3 Delphi rounds, the 14 panelists achieved consensus on 50 reasons for readmission; 12 panelists also completed group telephone calls between rounds 1 and 2. Readmissions with diagnoses of infection, sepsis, pneumonia, hemorrhage/hematoma, anemia, ostomy complications, acute renal failure, fluid/electrolyte disorders, or venous thromboembolism were considered associated with surgical quality and accounted for 25 521 of 39 664 readmissions (64% of readmissions; 7.5% of 340 858 index surgical procedures). The proportion of readmissions considered to be not associated with surgical quality varied by procedure, ranging from to 21% (613 of 2331) of readmissions after lower-extremity amputations to 47% (745 of 1598) of readmissions after cholecystectomy.
Conclusions and Relevance
One-third of postoperative readmissions are unlikely to reflect problems with surgical quality. Future studies should test whether restricting readmissions to those with specific ICD-9 codes might yield a more useful quality measure.
In this study, a Delphi panel assesses the likelihood that certain International Classification of Diseases, Ninth Revision diagnosis codes can indicate whether such readmissions are associated with surgical quality in the index admission.
Introduction
Hospital readmissions are increasingly used as outcome measures associated with quality of care, and rates are publicly reported for private-sector and Veterans Health Administration (VA) facilities.1,2 Furthermore, the Centers for Medicare & Medicaid Services and many state governments impose payment penalties on hospitals with readmission rates greater than the national average.3 Recently, attention has focused on identification of the proportion of unplanned hospital readmissions that truly reflect quality of care as opposed to factors that cannot be controlled. Readmissions after surgical procedures may be more associated with quality issues in the index admission than readmissions after medical hospitalizations.4 Hospitals trying to reduce overall readmission rates may find value in focusing on surgical care. However, there is little agreement about how to identify which postoperative readmissions truly reflect surgical quality–related problems in the index admission; these cases likely require different readmission reduction efforts than cases not associated with surgical quality.5
Early research on VA readmissions suggests that surgical cases contribute to 20% of all hospital readmissions.4 Rosen et al4 used diagnosis-related groups to determine that 40% of all unplanned surgical readmissions were clinically associated with the index diagnosis; 70% of these had a complication (eg, postoperative infection). Diagnosis-related groups are used for case-mix adjustment and to group patients by International Classification of Diseases, Ninth Revision (ICD-9) diagnosis codes, comorbidities, age, and sex.6 In contrast, Morris et al7 used data from the medical record review–based VA Surgical Quality Improvement Program (VASQIP) and found 56% of postoperative readmissions had a surgical complication. Hechenbleikner et al5 found the proportion of postoperative readmission rates associated with quality of surgical procedure (identified through the National Surgical Quality Improvement Program, billing data, and physician review) varying from 51% to 86%. Surgical complications are often used to measure quality of care8; however, surgical readmissions are not necessarily the result of complications, nor do they always reflect quality deficits. Identifying which readmissions are associated with surgical quality is difficult.
We approached this challenge by convening a Delphi panel of experts in medical and surgical readmission to evaluate reasons for readmission among a cohort of veterans who underwent selected surgical procedures at VA hospitals and to identify which of these readmissions might signify a surgical quality problem. Delphi consensus processes aggregate expert input to identify areas of agreement and are commonly used to develop quality measures.9,10 Findings from such expert consensus proceedings have been used to establish valid and reliable criteria for explicit medical record review to identify potentially preventable readmissions.11,12,13 We also assessed differences in readmission rates and patient and procedure characteristics between all readmissions in our sample of VA inpatient operations and those with consensus-defined surgical quality–related readmissions. We further examined procedure-level differences in readmission rates between the overall and surgical quality–related cohorts. To our knowledge, this is the first study to use a Delphi consensus process to determine the proportion of surgical readmissions that may be associated with problems in surgical quality. These findings can inform efforts to develop validated quality metrics for surgical care.
Methods
We conducted a structured Delphi-based consensus process with clinical experts and a quantitative analysis of postoperative readmissions using VA administrative and clinical databases. We began broadly by assigning reasons for postoperative readmissions using codes from the World Health Organization’s ICD-9. We presented these reasons to our Delphi panel to rate the surgical quality–relatedness of each readmission reason. We used the ICD-9–coded readmission reasons that met our standards for consensus and surgical quality–relatedness to define 2 groups of readmissions; those with potential surgical quality–related problems in the index hospitalization and those without. We then performed statistical analyses to address our study objectives. The VA Central institutional review board approved this research. Informed consent was waived and data were deidentified for the purposes of this analysis.
Data Sources
Veterans Affairs Surgical Quality Improvement Program data were used to identify patient demographics, preoperative comorbidities, operative characteristics, and postoperative complications.14 Preoperative health care use, postoperative discharge destination, mental health diagnoses, laboratory test results, and vital signs data in the preoperative and postoperative period were obtained from the VA Corporate Data Warehouse, accessed through the VA Informatics and Computing Infrastructure. The Corporate Data Warehouse was also queried for all inpatient diagnoses occurring in the 30-day postsurgical discharge period to categorize reasons for readmission.
Study Population and Sampling Process
The study population included all VASQIP-assessed patients with inpatient surgical procedures performed at a VA Medical Center between October 1, 2007, and September 30, 2014. To accurately assess postdischarge outcomes of readmission, patients who died during hospitalization were excluded from analyses. Patients with a length of stay less than 2 days or greater than 30 days were excluded from the final analytic data set to obtain a study sample representative of a surgical inpatient. The unit of analysis was the surgical procedure, and some patients underwent multiple operations during the study period. In these cases, only the first surgical procedure within a 30-day period was included in the analysis.
Categorization of Reasons for Readmission
Categories of reasons for readmission were developed using a multistep process examining the ICD-9 diagnosis codes from all inpatient readmissions occurring in the 30 days after the index surgery. Diagnosis codes were identified from the VA Corporate Data Warehouse.
For the first step, the Agency for Healthcare Research and Quality Clinical Classifications Software (CCS) multilevel diagnosis categories were applied to the primary, secondary, and length-of-stay diagnosis codes for all readmissions. Three of us (R. H. H., T. W., and C. J.) independently reviewed the combinations of CCS categories, focusing on the most prevalent categories first, and determined an initial list of potential readmission reasons and definitions based on the CCS categories. The clinicians then convened to reconcile their lists of potential categories, incorporating each clinician’s assessment of possible reasons for readmission. Specific ICD-9 diagnoses in addition to the CCS categories were provided to each clinician to help further specify reasons and confirm the appropriateness of reasons defined during the first review. Finally, combinations of primary and secondary diagnoses for the remaining uncoded readmissions were ranked from most to least prevalent, allowing clinicians to review and categorize less frequent reasons.
Delphi Process
We followed Delphi process guidelines established by the RAND-UCLA appropriateness model.15 This approach uses 3 rounds of consensus building and a phone call between the first and second rounds to discuss responses. We recruited a convenience sample of 14 experts: general, vascular, and orthopedic surgeons; internal medicine physicians familiar with treatment of readmitted surgical patients; surgical quality nurses; and surgical social workers from VA and private-sector academic surgical facilities. The identity of panel members was kept confidential to ensure that each member felt free to agree or disagree with other members’ responses. The questionnaire included instructions, readmissions reasons, broad ICD-9 disease categories (eg, bleeding or gastrointestinal tract problems), and the proportion of readmissions by reason with the median (interquartile range [IQR]) number of days to readmission for each reason category.
In the first round, we provided a generic questionnaire; in the second and third rounds, the participants were given individualized questionnaires that presented them with their own previous responses as well as the mean, median, and range of the group’s previous responses. The questions were the same in all 3 rounds.
Participants answered the following question for each readmission reason on a scale of 1 to 5 (1 = never related, 2 = unlikely to be related, 3 = either related or unrelated, 4 = likely related, and 5 = directly related to index quality): “Does the readmission reason reflect possible surgical quality of care problems in the index admission?” We established consensus as an IQR of 1 or less; we retained readmission reasons as related to surgical quality if consensus was reached and the median rating was 4 or greater.
Quantitative Analysis
For the final quantitative analyses, we applied our definition of surgical quality–related readmission to all readmissions in our study sample. Bivariate analyses assessed differences between readmission cohorts by patient demographic characteristics; preoperative, intraoperative, and postoperative procedure characteristics; and facility characteristics. An α of .05 was defined a priori, and χ2 and unpaired t tests determined significant differences. Finally, we grouped surgical procedures into single-level CCS procedure categories and compared the procedures most frequently associated with any readmission with those considered surgical quality–related by the Delphi panel. All analyses were conducted using SAS version 9.2 (SAS Institute, Inc).
Results
Study Sample
Our sample comprised 340 858 inpatient surgical procedures and 39 664 readmissions (11.6%). Of the 25 521 patients with surgical quality–related readmissions, 1204 (4.7%) were female; 19 769 (77.5%) were of white race/ethnicity; and 4417 (17.3%) were of black race/ethnicity. The mean (SD) age among this subgroup was 65.5 (11.2) years. Of the 14 143 patients whose readmission was not associated with surgical quality, 740 (5.2%) were female; 10 770 (76.2%) were white; 2672 (18.9%) were black; and the mean (SD) age was 63.3 (11.5) years. The most prevalent types of surgical procedures were orthopedic, accounting for 29.8%; general, accounting for 27.3%; and vascular, accounting for 12.3% of all operations. These specialties had readmission rates of 8% (orthopedic), 13% (general), and 17% (vascular).
Readmission Reasons
The group of clinicians developed a list of categories, which identified reasons for 90% of the readmissions in our sample. This list included 50 distinct ICD-9 code–based categories (Table 1). The most prevalent reasons for readmission included other fluid and electrolyte codes (19.7%), other mental health codes (16.5%), anemia (16.9%), pain-related codes (15.8%), and postoperative abscess/septicemia (15.3%). In the first round of the Delphi process, panelists disagreed about the categorization of 2 reasons: amputation neuroma or infection and injury/poisoning; in subsequent rounds, we divided these categories to create amputation neuroma and amputation infection and 6 categories of injury/poisoning. In total, 50 readmission reasons were rated by Delphi panelists.
Table 1. ICD-9–Based Consensus Items and Results for Rounds 1 Through 3 of the Modified Delphi Process Code Descriptionsa.
| Reason for Readmission per ICD-9 Code Description | ICD-9 Coded Cases, % | Days to Readmission, Median (Q1-Q3) | Round 1 | Round 2 | Round 3 | |||
|---|---|---|---|---|---|---|---|---|
| Median (Range) | Consensus | Median (Range) | Consensus | Median (Range) | Consensus | |||
| Wound-Related Code Description | ||||||||
| Postoperative abscess/septicemiab | 15.3 | 9 (5-14) | 5 (3-5) | Yes | ||||
| Acute abscess/diffuse cellulitis/lymphangitisb | 10.0 | 9 (4-14) | 4 (3-5) | No | 4 (3-5) | Yes | ||
| Wound disruption/dehiscence/ruptureb | 7.6 | 10 (6-15) | 4.5 (3-5) | No | 5 (3-5) | Yes | ||
| Amputation neuroma or infection | 2.8 | 12 (7-18) | 3 (1-5) | Yes | ||||
| Amputation infectionb | 2.0 | 12 (7-17) | PR1 | PR1 | 4.5 (3-5) | Yes | ||
| Amputation complication/neuroma | 0.9 | 12 (7-19) | PR1 | PR1 | 2 (1-5) | Yes | ||
| Infected postoperative seromab | 2.4 | 7 (4-11) | 4 (2-5) | No | 5 (3-5) | Yes | ||
| Peritoneal abscessb | 0.8 | 10.5 (6-16) | 5 (1-5) | No | 4 (3-5) | Yes | ||
| Injury—posttraumatic wound infection | 0.3 | 10 (7-18.5) | PR1 | PR1 | 3 (2-5) | Yes | ||
| Bleeding Code Description | ||||||||
| Hemorrhage or hematomab | 9.3 | 7 (3-12) | 4 (3-5) | Yes | ||||
| Hematuria | 1.1 | 8 (3-14) | 3 (1-3) | Yes | ||||
| GI-Related Code Description | ||||||||
| Intestinal obstruction | 8.9 | 4 (2-9) | 3.5 (2-5) | Yes | ||||
| Constipation | 4.2 | 7 (3-14) | 3 (1-4) | Yes | ||||
| Nausea | 4.0 | 6 (3-12) | 3 (2-4) | Yes | ||||
| Vomiting | 3.6 | 6 (3-12) | 3 (2-5) | Yes | ||||
| Ostomy care/complicationsb | 1.3 | 6 (3-12) | 4 (3-5) | Yes | ||||
| Acute pancreatitis | 1.2 | 7 (4-13) | 3 (2-5) | No | 3 (1-3) | Yes | ||
| Ulcers or gastritis | 0.9 | 8 (4-13) | 3 (2-5) | No | 3 (1-4) | Yes | ||
| Gastroenteritis | 0.2 | 8.5 (5-16) | 2 (1-5) | No | 2 (1-4) | Yes | ||
| Diverticulosis | 0.1 | 8 (3-16) | 1 (1-4) | Yes | ||||
| Diverticulitis | 0 | 15 (5-21) | 2 (1-4) | No | 1 (1-3) | Yes | ||
| Device Complication Code Description | ||||||||
| Other uncoded device complications | 9.8 | 11 (5-17) | 3 (3-5) | Yes | ||||
| Catheter-related bloodstream infectionb | 0.3 | 7 (4-14) | 4 (3-5) | Yes | ||||
| Renal Failure and Electrolyte Code Description | ||||||||
| Other fluid and electrolyte codes | 19.7 | 8 (4-14) | 4 (2-5) | Yes | ||||
| Acute renal failureb | 12.0 | 9 (4-15) | 4 (2-5) | No | 4 (3-5) | Yes | ||
| Dehydration | 8.9 | 8 (4-14) | 3.5 (2-5) | Yes | ||||
| Mental Health Code Description | ||||||||
| Other mental health | 16.5 | 8 (4-14) | 3 (1-4) | Yes | ||||
| Delirium and other psychoses diagnosis codes | 9.3 | 8 (4-14) | 3 (2-5) | Yes | ||||
| Alteration of consciousness | 6.4 | 7 (3-13) | 3 (2-5) | Yes | ||||
| Alcoholism | 4.2 | 9 (5-15) | 2 (1-3) | No | 1.5 (1-3) | Yes | ||
| Substance abuse | 2.3 | 9 (4-16) | 2.5 (1-4) | No | 2 (1-3) | No | 2 (1-3) | Yes |
| Other Code Description | ||||||||
| Anemiab | 16.9 | 8 (4-14) | 4 (3-5) | Yes | ||||
| Pain-related | 15.8 | 7 (3-14) | 3.5 (2-5) | Yes | ||||
| Cardiac arrhythmia | 11.0 | 8 (4-14) | 3 (1-4) | No | 3 (2-5) | Yes | ||
| Peripheral vascular disease | 10.0 | 10 (5-15) | 3 (1-4) | Yes | ||||
| Congestive heart failure | 8.9 | 8 (4-15) | 3 (2-5) | Yes | ||||
| Urinary tract infectionb | 8.3 | 8 (4-14) | 4 (1-5) | No | 4 (2-5) | Yes | ||
| Bacterial pneumoniab | 6.7 | 7 (3-13) | 4 (1-5) | Yes | ||||
| Postoperative sepsis (PSI 13)b | 6.0 | 8 (4-14) | 4.5 (1-5) | Yes | ||||
| Cardiac complications and chest pain | 4.8 | 8 (4-14) | 3.5 (2-5) | Yes | ||||
| Postoperative pulmonary embolism or deep vein thrombosis (PSI 12)b | 4.7 | 7 (4-13) | 5 (3-5) | Yes | ||||
| Clostridium difficile infection | 2.8 | 8 (4-14) | 3.5 (2-5) | Yes | ||||
| Seizures | 1.9 | 8 (4-15) | 3 (1-4) | Yes | ||||
| Strokes | 1.4 | 7 (3-13) | 3 (2-5) | Yes | ||||
| Injury and Poisoning Code Descriptions | ||||||||
| Fractures, dislocations, intracranial injuries, internal injuries, open wounds | 3.3 | 9 (4-15) | 3 (2-5) | No | ||||
| Injury – fracture | 1.5 | 10 (4-15) | PR1 | PR1 | 2.5 (1-4) | Yes | ||
| Injury – open wound | 0.8 | 10 (4.5-14) | PR1 | PR1 | 3 (2-5) | Yes | ||
| Injury – poisoning/overdose | 0.5 | 10.5 (4.5-19) | PR1 | PR1 | 3 (1-4) | No | 3 (1-4) | Yes |
| Injury – dislocation | 0.4 | 11 (6-16) | PR1 | PR1 | 3 (1-5) | Yes | ||
| Injury – other nonsurgical injury | 0.4 | 7 (3-13) | PR1 | PR1 | 2 (1-5) | No | 2 (1-3) | Yes |
Abbreviations: GI, gastrointestinal tract; ICD-9, International Classification of Diseases, Ninth Revision; PR1, code added after first round; PSI, Agency for Healthcare Research and Quality patient safety indicator.
The Delphi panel comprised 14 individuals. Each panelist rated readmission reasons on a scale of 1 to 5, with 1 indicating never related and 5, directly related to index surgery quality. The question asked was “Does the readmission reason reflect possible surgical quality of care problems in the index admission?” The blank cells indicate that consensus was achieved at an earlier round.
Acceptability was defined as consensus and a median rating of 4 or higher. Consensus was defined as interquartile range of 1 or lower.
Delphi Results
The Delphi panel included 14 experts: surgeons from general (n = 3), orthopedic (n = 1), vascular (n = 2), and colorectal (n = 2) specialties from within and outside the VA; 2 internal medicine physicians, 1 VA and 1 non-VA, both familiar with readmissions research; 2 VA surgical nurses; and 2 VA social workers familiar with postoperative patients. Panelists rated readmission reasons across 3 rounds and participated in 1 of 2 telephone calls between rounds 1 and 2 of the Delphi process.
Table 1 presents the consensus results for each round of the Delphi process. In round 1, consensus was reached on 28 of the 42 readmission reasons (67%). Seven new readmission reasons were introduced in round 2 and consensus was reached on 18 of the 21 readmission reasons (86%). Thus, in round 3, 3 items were presented and there was 100% consensus. The eTable in the Supplement shows the final list of surgical quality–related readmission reasons and their ICD-9 codes.
Panelists were consistent in how frequently they changed ratings between rounds. Overall, ratings changed after the panelists discussed the readmission reasons on which consensus was not achieved during the phone call between rounds 1 and 2.
Comparison of Surgical Quality–Related and Any Readmission Cohorts
Of 39 664 postoperative readmissions, 25 521 (64%) had an ICD-9–coded reason for readmission that our Delphi panel considered to be associated with surgical quality. Among 340 858 inpatient operations performed during the study period, the surgical quality–related readmission rate was 7.5%. In comparing the surgical quality–related and unrelated groups, operations on older, male, and functionally dependent patients were more likely to have problems with quality. Surgical procedures with clean wound sites, shorter operative times, and shorter lengths of hospital stay were more common in the cohort with readmission reasons not associated with surgery (Table 2).
Table 2. Association of Patient and Operative Characteristics With Surgical Quality–Related Readmissions vs Any Readmissionsa.
| Characteristic | Readmission | P Valueb | |
|---|---|---|---|
| Surgery Related (n = 25 521) | Not Surgery Related (n = 14 143) |
||
| Surgical specialty, No. (%) | <.001 | ||
| General | 8156 (32.0) | 4222 (29.9) | |
| Orthopedic | 5285 (20.7) | 2785 (19.7) | |
| Peripheral vascular | 4959 (19.4) | 2366 (16.7) | |
| Urology | 2724 (10.7) | 1431 (10.1) | |
| Neurosurgery | 1390 (5.4) | 1142 (8.1) | |
| Thoracic surgery | 1373 (5.4) | 1102 (7.8) | |
| Otolaryngology | 607 (2.4) | 547 (3.9) | |
| Other | 1024 (4.0) | 546 (3.9) | |
| Patient demographics | |||
| Age, mean (SD), y | 65.5 (11.2) | 63.3 (11.5) | <.001 |
| Female sex, No. (%) | 1204 (4.7) | 740 (5.2) | .02 |
| Race/ethnicity, No. (%) | .002 | ||
| White | 19 769 (77.5) | 10 770 (76.2) | |
| Black/African American | 4417 (17.3) | 2672 (18.9) | |
| Native Hawaiian/Pacific Islander | 190 (0.7) | 104 (0.7) | |
| American Indian/Alaskan Native | 164 (0.6) | 89 (0.6) | |
| Asian | 68 (0.3) | 47 (0.3) | |
| Unknown | 913 (3.6) | 461 (3.3) | |
| Body mass index, mean (SD)c | 28.0 (6.7) | 27.9 (6.4) | .13 |
| Marital status, No. (%) | <.001 | ||
| Married | 11 610 (45.5) | 6327 (44.7) | |
| Divorced | 7638 (29.9) | 4488 (31.7) | |
| Single | 2617 (10.3) | 1565 (11.1) | |
| Widowed | 2586 (10.1) | 1152 (8.1) | |
| Separated | 1056 (4.1) | 601 (4.2) | |
| Region of facility, No. (%) | <.001 | ||
| Continental | 3578 (14.0) | 2189 (15.5) | |
| Midwest | 5929 (23.2) | 3260 (23.1) | |
| North Atlantic | 5617 (22.0) | 3539 (25.0) | |
| Pacific | 4437 (17.4) | 2394 (16.9) | |
| Southeast | 5869 (23.0) | 2709 (19.2) | |
| Preoperative health and comorbidities | |||
| Functional health status, No. (%) | <.001 | ||
| Independent | 20 891 (81.9) | 12 275 (86.8) | |
| Partially dependent | 3715 (14.6) | 1570 (11.1) | |
| Totally dependent | 901 (3.5) | 289 (2.0) | |
| No. of ED visits within previous 6 mo, mean (SD) | 1.5 (2.7) | 1.5 (2.7) | .71 |
| No. of previous inpatient admissions within 6 mo, mean (SD) | 1.6 (3.4) | 1.5 (3.3) | .002 |
| Depression diagnosis, No. (%) | 4813 (18.9) | 2843 (20.1) | .003 |
| Bipolar disorder diagnosis, No. (%) | 599 (2.3) | 427 (3.0) | <.001 |
| Psychosis diagnosis, No. (%) | 710 (2.8) | 398 (2.8) | .85 |
| ASA classification, No. (%) | <.001 | ||
| 1-2 | 2307 (9.0) | 1914 (13.5) | |
| 3 | 17 431 (68.3) | 9652 (68.2) | |
| 4-5 | 5768 (22.6) | 2571 (18.2) | |
| Chemotherapy for malignant neoplasm in past 30 d, No. (%) | 395 (1.5) | 183 (1.3) | .04 |
| Current pneumonia, No. (%) | 269 (1.1) | 83 (0.6) | <.001 |
| Diabetes with oral agents or insulin, No. (%) | <.001 | ||
| None | 17 011 (66.7) | 10 041 (71.0) | |
| Oral | 3702 (14.5) | 1919 (13.6) | |
| Insulin | 4808 (18.8) | 2183 (15.4) | |
| Dyspnea, No. (%) | <.001 | ||
| No dyspnea | 20 768 (81.4) | 11 830 (83.6) | |
| Minimal exertion | 4194 (16.4) | 2029 (14.3) | |
| At rest | 558 (2.2) | 284 (2.0) | |
| Currently receiving dialysis, No. (%) | 977 (3.8) | 503 (3.6) | .17 |
| Disseminated cancer, No. (%) | 937 (3.7) | 548 (3.9) | .31 |
| More than 2 alcoholic drinks/d in 2 wk before admission, No. (%) | 2050 (8.0) | 1208 (8.5) | .08 |
| Congestive heart failure in month before surgery, No. (%) | 522 (2.0) | 223 (1.6) | <.001 |
| History of severe COPD, No. (%) | 5490 (21.5) | 2641 (18.7) | <.001 |
| History of transient ischemic attacks, No. (%) | 1151 (4.5) | 603 (4.3) | .25 |
| Impaired sensorium, No. (%) | 415 (1.6) | 162 (1.1) | <.001 |
| Preoperative sepsis in 48 h before surgery, No. (%) | 955 (3.7) | 375 (2.7) | <.001 |
| Current (within 1 y) smoker, No. (%) | 8897 (34.9) | 5465 (38.6) | <.001 |
| Steroid use for chronic condition, No. (%) | 992 (3.9) | 480 (3.4) | .01 |
| History of angina in month before surgery, No. (%) | 513 (2.0) | 264 (1.9) | .32 |
| Hypertension requiring medication, No. (%) | 19 214 (75.3) | 9928 (70.2) | <.001 |
| History of myocardial infarction 6 mo before surgery, No. (%) | 286 (1.1) | 147 (1.0) | .46 |
| History of PVD, No. (%) | 3719 (14.6) | 1576 (11.1) | <.001 |
| Rest pain/gangrene, No. (%) | 2718 (10.7) | 1133 (8.0) | <.001 |
| Operative characteristics | |||
| Emergency case, No. (%) | 2823 (11.1) | 1360 (9.6) | <.001 |
| Wound classification, No. (%) | <.001 | ||
| Clean | 11 573 (45.3) | 6937 (49.0) | |
| Clean/contaminated | 9399 (36.8) | 5268 (37.2) | |
| Contaminated | 1939 (7.6) | 799 (5.6) | |
| Infected | 2609 (10.2) | 1138 (8.0) | |
| Work RVU | 20.3 (9.6) | 19.4 (9.5) | <.001 |
| Total operation time, hours, mean (SD) | 2.9 (2.1) | 2.7 (2.0) | <.001 |
| Length of total hospital stay, mean (SD), d | 8.9 (6.4) | 7.9 (6.1) | <.001 |
Abbreviations: ASA, American Society of Anesthesiologists; COPD, chronic obstructive pulmonary disease; ED, emergency department; RVU, relative value units.
Total of 39 664 thirty-day readmissions in 340 858 Veterans Affairs inpatient surgical procedures.
P values are unpaired and 2-tailed.
Calculated as weight in kilograms divided by height in meters squared.
Comparison of Procedures Associated With Surgical Quality–Related and Any Readmission
The 25 most common surgical procedures in our sample accounted for 268 082 of all 340858 (78.6%) operations performed. We observed variation in readmission rates by procedure and in the proportion of the readmissions that could be attributed to surgical quality–related problems (Table 3). For example, cholecystectomy and common duct surgical exploration accounted for 1598 of 15 514 surgical procedures, with a 10.3% readmission rate; however, 853 (53.4%) of these readmissions were coded with a surgical quality–related reason by our Delphi panel. In contrast, there were 3538 readmissions after 24 338 colorectal resections, a 14.5% rate, and 2584 (73.0%) were coded with a surgical quality–related readmission reason. The surgical quality–related readmission rate differed from the overall readmission rate for every procedure commonly performed in our sample. Of procedures with the highest readmission rates, amputation of lower extremity and peripheral vascular bypass had readmission rates greater than 20% and surgical quality–related readmission rates of approximately 15%; however, transurethral excision, drainage, or removal of urinary obstruction had 793 readmissions from 3873 surgical procedures, a 20.5% overall readmission rate but 457 (11.8%) of the readmissions were related to surgical quality.
Table 3. Variation in Surgical Quality-Related Readmission by Procedure Typea.
| Procedureb | No. Performed | Readmitted, No. (%) | Surgical Quality–Related Readmission, No. (%) |
Readmissions Related to Surgical Quality, % |
|---|---|---|---|---|
| Arthroplasty knee | 45 830 | 2808 (6.1) | 1787 (3.9) | 63.6 |
| Colorectal resection | 24 338 | 3538 (14.5) | 2584 (10.6) | 73.0 |
| Hip replacement, total and partial | 24 243 | 1643 (6.8) | 1041 (4.3) | 63.4 |
| Cholecystectomy and common duct exploration | 15 514 | 1598 (10.3) | 853 (5.5) | 53.4 |
| Laminectomy, excision intervertebral disc | 14 109 | 1024 (7.3) | 599 (4.2) | 58.5 |
| Open prostatectomy | 12 356 | 805 (6.5) | 543 (4.4) | 67.5 |
| Artery and vein procedures (eg, endovascular repair of AAA) | 11 760 | 1914 (16.3) | 1201 (10.2) | 62.7 |
| Lobectomy or pneumonectomy | 11 417 | 1237 (10.8) | 688 (6.0) | 55.6 |
| Amputation of lower extremity | 11 133 | 2331 (20.9) | 1718 (15.4) | 73.7 |
| Peripheral vascular bypass | 10 140 | 2097 (20.7) | 15331 (15.1) | 73.0 |
| Incisional, ventral, or umbilical hernia repair | 10 100 | 1091 (10.8) | 744 (7.4) | 68.2 |
| Treatment, fracture or dislocation of hip and femur | 8842 | 1298 (14.7) | 977 (11.0) | 75.3 |
| Nephrectomy, partial or complete | 8676 | 802 (9.2) | 540 (6.2) | 67.3 |
| Spinal fusion | 7935 | 605 (7.6) | 354 (4.5) | 58.5 |
| Endarterectomy, vessel of head and neck | 7223 | 606 (8.4) | 326 (4.5) | 53.8 |
| Lower GI therapeutic procedures (intestine repair, laparoscopy) | 6625 | 844 (12.7) | 536 (8.1) | 63.5 |
| Appendectomy | 5311 | 501 (9.4) | 304 (5.7) | 60.7 |
| Transurethral resection of prostate | 5046 | 440 (8.7) | 286 (5.7) | 65.0 |
| Therapeutic procedures on respiratory system (laryngectomy, thoracoscopy, pleurectomy) | 4293 | 705 (16.4) | 388 (9.0) | 55.0 |
| Treatment, fracture or dislocation of lower extremity | 4001 | 367 (9.2) | 212 (5.3) | 57.8 |
| Arthroplasty other than hip or knee | 3972 | 173 (4.4) | 92 (2.3) | 53.2 |
| Transurethral excision, drainage, or removal of urinary obstruction | 3873 | 793 (20.5) | 457 (11.8) | 57.6 |
| Inguinal and femoral hernia repair | 3868 | 401 (10.4) | 241 (6.2) | 60.1 |
| Therapeutic procedures on muscles and tendons (knee incision, shoulder repair) | 3807 | 471 (12.4) | 310 (8.1) | 65.8 |
| Upper GI therapeutic procedures (laparoscopy, esophagectomy) | 3670 | 536 (14.6) | 330 (9.0) | 61.6 |
Abbreviations: AAA, abdominal aortic aneurysm; CCS, Agency for Healthcare Research and Quality Clinical Classifications Software; GI, gastrointestinal tract.
Table shows 78% of procedures among 340 858 surgical procedures in the sample.
Procedures are grouped into single-level CCS procedure categories.
Ranking the procedures by readmission rate revealed consistently higher ranks for amputation of lower extremity, peripheral vascular bypass, and transurethral excisions and consistently lower ranks for arthroplasty procedures in overall as well as surgical quality-related readmissions. Within these bounds, however, there were shifts in rank when readmission rates were restricted to those deemed to be associated with surgical quality problems.
Discussion
The goal of this study was to use a consensus-based list of readmission reasons to compare the rate of overall readmission with the rate of surgical quality–related readmission. We looked at differences in patient and procedure factors as well as differences in the readmission rate for various surgical specialties and common procedures. In our sample, 64% of postoperative readmissions could be attributed to possible surgical quality–related problems and there were differences in patient and procedure characteristics between quality-related and unrelated readmissions. Although the procedures with the highest and lowest readmission rates were the same in both cohorts, many of the surgical procedures changed rank when readmissions were limited to those deemed surgical quality–related.
Delphi processes for establishing expert consensus have been used to develop clinical quality measures and identify important processes of care associated with medical readmissions.9,10,13 We extended these methods to the realm of surgical readmissions. Strengths of the Delphi process include the anonymous and iterative nature of the data collection, which allows for reasoned expert feedback with less bias from more forthright participants.16 Selection of participants is critical to obtaining useful results, and we identified a variety of clinical experts familiar with surgical care, quality problems, and readmissions; however, it is possible that our Delphi panel was not sufficiently representative and that other clinical experts in surgery would have arrived at different conclusions. Therefore, Delphi findings should be considered an early step in establishing quality measures that needs further reliability testing.17
In addition to conducting a Delphi process, we examined the differences in readmission groups if only the surgical quality–related readmissions were analyzed instead of all readmissions. We found key differences in intraoperative and postoperative characteristics between the 2 groups that reinforce the reliability of the Delphi panel’s definition of surgical quality–related readmissions. We expected that in cases of quality-related postoperative readmission, potential problems would be evident before discharge, and we found higher rates of wound contamination and longer operative times in this group. These factors have been shown to predict surgical complications.18,19
Our results have implications for quality measurement. The primary concern with using overall readmission rates to measure surgical quality is the likelihood that a fraction of readmissions reflect true quality-related problems. As we demonstrated by comparing the procedures associated with any readmissions with those associated with surgical quality–related readmissions, the types of procedures accounting for potential quality problems varies depending on the definition of readmission. A more precise approach to identifying quality-related readmissions may benefit health care systems and clinicians in their quality improvement efforts.
Limitations
Research has largely focused on medical readmissions, and this study is an important contribution to the surgical readmissions literature. However, our study has some limitations. Although the Delphi panel’s definition of surgical quality–related readmission detected a different cohort of patients and identified different surgical procedures associated with these readmissions, the definition relied on ICD-9 coding. Previous studies indicate that ICD-9 coding–based detection of postoperative outcomes has limitations with respect to the reliability of the diagnosis code and the reason for admission.20,21,22 Our readmission results were limited to care recorded in the VASQIP data and did not include all VA inpatient surgical procedures, per the VASQIP sampling methodology; however, VASQIP captured 83% of all VA inpatient operations during the study period. In addition, we did not include readmissions to non-VA hospitals. Although this omission likely reduced the number of readmissions in our sample, previous VA readmissions research found that the number of patients admitted to non-VA facilities after a VA inpatient stay was small.23 Finally, explicit medical record review is considered the criterion standard detection method for surgical quality–related readmissions.5 Although our study lays the groundwork for developing explicit record review guidelines, further work is necessary to confirm the reliability of the definition of surgical quality–related readmissions developed by the Delphi process and to determine whether this administrative coding–based methodology can be used for quality measurement.
Conclusions
Although the use of readmissions as a quality measure is controverisal,24 they are a readily available outcome that can be used to compare performance across hospitals or to target quality improvement work.3,7 However, our analysis suggests that one-third of all postoperative readmissions are unlikely to be associated with surgical quality. Future research should examine whether focusing on readmissions for only those admission codes thought to identify surgical quality–related cases would yield a more useful measure of quality. This is the first step toward the development of a surgical readmission quality metric.
eTable. ICD-9 and ICD-10 Codes for Readmission Reasons Associated by Consensus With Surgical Quality–Related Problems
References
- 1.Centers for Medicare & Medicaid Services CMS Hospital Compare website. https://www.cms.gov/HospitalQualityInits/11_HospitalCompare.asp. Modified April 10, 2013. Accessed April 10, 2013.
- 2.Veterans Health Administration VA Hospital Compare website. https://www.medicare.gov/hospitalcompare/VAData/main.html. Accessed March 9, 2018.
- 3.Centers for Medicare & Medicaid Services Readmissions reduction program. https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. 2017. Accessed March 9, 2018.
- 4.Rosen AK, Chen Q, Shin MH, et al. Medical and surgical readmissions in the Veterans Health Administration: what proportion are related to the index hospitalization? Med Care. 2014;52(3):243-249. [DOI] [PubMed] [Google Scholar]
- 5.Hechenbleikner EM, Makary MA, Samarov DV, et al. Hospital readmission by method of data collection. J Am Coll Surg. 2013;216(6):1150-1158. [DOI] [PubMed] [Google Scholar]
- 6.Centers for Medicare & Medicaid Services Design and development of the diagnosis related group (DRG). https://www.cms.gov/ICD10Manual/version34-fullcode-cms/fullcode_cms/Design_and_development_of_the_Diagnosis_Related_Group_(DRGs)_PBL-038.pdf. Published October 1, 2016. Accessed March 9, 2018.
- 7.Morris MS, Deierhoi RJ, Richman JS, Altom LK, Hawn MT. The relationship between timing of surgical complications and hospital readmission. JAMA Surg. 2014;149(4):348-354. [DOI] [PubMed] [Google Scholar]
- 8.Khuri SF, Daley J, Henderson WG. The comparative assessment and improvement of quality of surgical care in the Department of Veterans Affairs. Arch Surg. 2002;137(1):20-27. [DOI] [PubMed] [Google Scholar]
- 9.Black N, Murphy M, Lamping D, et al. Consensus development methods: a review of best practice in creating clinical guidelines. J Health Serv Res Policy. 1999;4(4):236-248. [DOI] [PubMed] [Google Scholar]
- 10.Ashton CM, Kuykendall DH, Johnson ML, et al. A method of developing and weighting explicit process of care criteria for quality assessment. Med Care. 1994;32(8):755-770. [DOI] [PubMed] [Google Scholar]
- 11.Ashton CM, Kuykendall DH, Johnson ML, Wray NP. An empirical assessment of the validity of explicit and implicit process-of-care criteria for quality assessment. Med Care. 1999;37(8):798-808. [DOI] [PubMed] [Google Scholar]
- 12.Borzecki AM, Chen Q, Mull HJ, et al. Do acute myocardial infarction and heart failure readmissions flagged as potentially preventable by the 3M potentially preventable readmissions software have more process-of-care Problems? Circ Cardiovasc Qual Outcomes. 2016;9(5):532-541. [DOI] [PubMed] [Google Scholar]
- 13.Borzecki AM, Chen Q, Restuccia J, et al. Do pneumonia readmissions flagged as potentially preventable by the 3M PPR software have more process of care problems? a cross-sectional observational study. BMJ Qual Saf. 2015;24(12):753-763. [DOI] [PubMed] [Google Scholar]
- 14.Morris MS, Graham LA, Richman JS, et al. Postoperative 30-day readmission: time to focus on what happens outside the hospital. Ann Surg. 2016;264(4):621-631. [DOI] [PubMed] [Google Scholar]
- 15.Fitch K, Bernstein SJ, Aguilar MS, et al. The RAND/UCLA Appropriateness Method User's Manual Santa Monica, CA: RAND;2001. Monograph Reports. [Google Scholar]
- 16.von der Gracht HA. Consensus measurement in Delphi studies: review and implications for future quality assurance. Technol Forecast Soc Change. 2012;79(8):1525-1536. [Google Scholar]
- 17.Jones J, Hunter D. Consensus methods for medical and health services research. BMJ . 1995;311(7001):376-380. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Mioton LM, Jordan SW, Hanwright PJ, Bilimoria KY, Kim JY. The relationship between preoperative wound classification and postoperative infection: a multi-institutional analysis of 15,289 patients. Arch Plast Surg. 2013;40(5):522-529. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Daley BJ, Cecil W, Clarke PC, Cofer JB, Guillamondegui OD. How slow is too slow? correlation of operative time to complications: an analysis from the Tennessee Surgical Quality Collaborative. J Am Coll Surg. 2015;220(4):550-558. [DOI] [PubMed] [Google Scholar]
- 20.Rosen AK, Itani KM, Cevasco M, et al. Validating the patient safety indicators in the Veterans Health Administration: do they accurately identify true safety events? Med Care. 2012;50(1):74-85. [DOI] [PubMed] [Google Scholar]
- 21.Chen Q, Mull HJ, Rosen AK, Borzecki AM, Pilver C, Itani KM. Measuring readmissions after surgery: do different methods tell the same story? Am J Surg. 2016;212(1):24-33. [DOI] [PubMed] [Google Scholar]
- 22.Romano PS, Chan BK, Schembri ME, Rainwater JA. Can administrative data be used to compare postoperative complication rates across hospitals? Med Care. 2002;40(10):856-867. [DOI] [PubMed] [Google Scholar]
- 23.O’Brien WJ, Chen Q, Mull HJ, et al. What is the value of adding Medicare data in estimating VA hospital readmission rates? Health Serv Res. 2015;50(1):40-57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Girotti ME, Shih T, Dimick JB. Health policy update: rethinking hospital readmission as a surgical quality measure. JAMA Surg. 2014;149(8):757-758. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eTable. ICD-9 and ICD-10 Codes for Readmission Reasons Associated by Consensus With Surgical Quality–Related Problems