Abstract
BACKGROUND:
Present at the time of surgery (PATOS) is an important measure to collect in postoperative complication surveillance systems because it may affect a patient’s risk of a subsequent complication and the estimation of postoperative complication rates attributed to the healthcare system. The American College of Surgeons (ACS) NSQIP started collecting PATOS data for 8 postoperative complications in 2011, but no one has used these data to quantify how this may affect unadjusted and risk-adjusted postoperative complication rates.
STUDY DESIGN:
This study was a retrospective observational study of the ACS NSQIP database from 2012 to 2018. PATOS data were analyzed for the 8 postoperative complications of superficial, deep, and organ space surgical site infection; pneumonia; urinary tract infection; ventilator dependence; sepsis; and septic shock. Unadjusted postoperative complication rates were compared ignoring PATOS vs taking PATOS into account. Observed to expected ratios over time were also compared by calculating expected values using multiple logistic regression analyses with complication as the dependent variable and the 28 nonlaboratory preoperative variables in the ACS NSQIP database as the independent variables.
RESULTS:
In 5,777,108 patients, observed event rates for each outcome were reduced by between 6.1% (superficial surgical site infection) and 52.5% (sepsis) when PATOS was taken into account. The observed to expected ratios were similar each year for all outcomes, except for sepsis and septic shock in the early years.
CONCLUSIONS:
Taking PATOS into account is important for reporting unadjusted event rates. The effect varied by type of complication—lowest for superficial surgical site infection and highest for sepsis and septic shock. Taking PATOS into account was less important for risk-adjusted outcomes (observed to expected ratios), except for sepsis and septic shock.
Present at the time of surgery (PATOS) is an important measure to collect in postoperative complication surveillance systems because it may affect a patient’s risk of a subsequent complication and the estimation of postoperative complication rates attributed to the healthcare system. The American College of Surgeons (ACS) NSQIP started collecting PATOS data in 2011 for superficial, deep, and organ space surgical site infection (SSI); pneumonia; urinary tract infection; ventilator dependence; sepsis; and septic shock. They define PATOS as “yes” if (1) the condition was recorded as a postoperative complication, and (2) if the same condition was present at the start of or during the index surgical procedure. Per communication with ACS NSQIP staff, for specific outcomes (eg pneumonia, urinary tract infection, etc) cases with PATOS are not included in the analysis of risk-adjusted outcomes across the participating hospitals (Cohen ME, personal communication, May 26, 2022). For the morbidity composite model, cases are not dropped, but for outcomes with PATOS, those outcomes are not permitted to trigger the morbidity event; morbidity would need to be triggered by an outcome that does not have PATOS (Cohen ME, personal communication, May 26, 2022).
To our knowledge, little is known about the occurrence rates of PATOS in the ACS NSQIP data, and the effects of taking PATOS into account when estimating unadjusted and risk-adjusted postoperative complication rates. This is important information not only for the ACS NSQIP, but also for other quality improvement initiatives that might or might not take PATOS into account, including the Veterans Affairs Surgical Quality Improvement Program,1 the Centers for Disease Control National Healthcare Safety Network,2 Vizient,3 and emerging programs that use machine learning and natural language processing for preoperative risk assessment and surveillance of postoperative complications using electronic health record data.4–10
The purpose of this study was to use PATOS data from the ACS NSQIP to examine the frequency of the different PATOS complications and to assess the effect of taking PATOS into account when estimating unadjusted and risk-adjusted postoperative complication rates. We hypothesized that taking PATOS into account may reduce unadjusted observed postoperative complication rates, and that these reductions may differ across outcomes, but that taking PATOS into account might not affect risk-adjusted results (ie observed to expected [O/E] ratios). The rationale for the latter hypothesis was that, although taking PATOS into account may reduce unadjusted postoperative complication rates, removing patients with PATOS might also remove higher-risk patients from the analysis and would therefore lower the expected event rate, so that the O/E ratio might have minimal change.
METHODS
This was a retrospective analysis of the prospectively collected ACS NSQIP Participant Use File from 2012 to 2018. The ACS NSQIP is a quality improvement program that collects risk and outcomes data on patients undergoing surgery, including patient demographics, preoperative risk factors, operative data, and 30-day postoperative outcomes.11–15 The ACS NSQIP provides risk-adjusted outcomes to hospitals to facilitate comparison among participating hospitals. Trained surgical clinical reviewers at each participating center collect the data. Thirty-day postoperative outcomes are determined through chart review and by patient and family contact after the index operation. Data are audited to ensure quality and standardization of collection.15 We included the original 9 surgical specialties in the ACS NSQIP database: general, gynecology, neurosurgery, orthopaedics, otolaryngology, plastics, thoracic, urology, and vascular. We excluded patients outside of these specialty areas and patients having missing values for any of the preoperative variables. The Colorado Multiple Institutional Review Board determined this study exempt from review as it uses de-identified data from a publicly available database.
Statistical analyses
We first examined the associations between patient preoperative characteristics and having 1 or more PATOS vs no PATOS outcomes. Here, we combined data across all 8 postoperative complications to simplify the presentation of data. For categorical preoperative characteristics, we compared the percentage distributions for the categories between patients having 1 or more PATOS vs no PATOS using chi-square tests. For continuous preoperative characteristics, we compared mean values of the preoperative variable between patients with and without a PATOS using 2-sample independent t-tests.
In a second analysis, we calculated the percentage of total postoperative complications that were PATOS for each of the 8 postoperative complications having PATOS data. We further calculated the event rates and 95% CIs for each of the 8 postoperative complications, first ignoring the PATOS data (ie including them in the numerator and denominator) and then taking the PATOS data into account (ie removing them from the numerator and denominator). We also calculated the percent change in the event rates as a result of taking PATOS into account.
Hospital identifiers are not available in the ACS NSQIP Participant Use File; therefore, we could not compare risk-adjusted rates (O/E ratios) between hospitals when ignoring PATOS vs taking PATOS into account. Instead, we compared O/E ratios over time for each of the 8 postoperative complications ignoring PATOS vs taking PATOS into account. For each of the 8 postoperative complications, we developed 2 multiple logistic regression models (1 when ignoring PATOS and 1 when taking PATOS into account) with the complication as the dependent variable and the 28 nonlaboratory preoperative ACS NSQIP variables as the independent variables. We then calculated O/E ratios and 95% CIs for each of the 8 complications and 7 years of the study (2012 to 2018) and plotted these results for estimates that ignored PATOS vs taking PATOS into account.
All analyses were performed using SAS software version 9.4 (SAS, Inc). For all tests of hypotheses, we defined statistical significance as a 2-sided p ≤ 0.05. There was no adjustment for multiple testing.
RESULTS
There were 5,881,881 patients in the ACS NSQIP Participant Use File from 2012 to 2018. A total of 28,253 (0.5%) patients were excluded for having operations outside of the 9 specialties mentioned previously, and 76,520 (1.3%) patients were excluded for having missing values for some of the preoperative variables. Therefore, a total of 5,777,108 patients (98.2% of the original cohort) were included for analysis.
Table 1 presents the associations between preoperative and operative variables and patients with 1 or more PATOS vs no PATOS. Of 5,777,108 patients, 120,590 (2.1%) had 1 or more PATOS. Patients with PATOS were generally older (59.8 vs 56.5 years), from minority racial/ethnic groups (Black 12.9% vs 9.8%; Hispanic 6.3% vs 5.8%), had fewer females (48.4% vs 57.3%), and underwent more complicated surgeries (mean work relative value unit 17.9 vs 16.2) compared with patients without PATOS. They also were more functionally dependent (13.0% vs 2.5%); were more likely transferred in from acute and chronic care (22.3% vs 3.6%); were more from general surgery (71.6% vs 45.5%); were less from outpatient surgery (5.9% vs 42.2%); had more emergency surgery (48.7% vs 8.1%); and had higher American Society of Anesthesiologists class (IV 29.0% vs 5.0%; V 3.6% vs 0.1%). Patients with PATOS also had higher rates of 17 preoperative comorbidities collected by the ACS NSQIP. All associations were statistically significant at p < 0.0001. Therefore, removing patients with PATOS from analysis would tend to reduce the expected rates of postoperative complications.
Table 1.
Association Between Patient Preoperative Characteristics and Having 1 or More Conditions Present at Time of Surgery
| Preoperative variable | No PATOS*(n = 5,656,518) | PATOS*(n = 120,590) |
|---|---|---|
| Age, y, mean ± SD | 56.5 (16.8) | 59.8 (17.4) |
| Sex, f, n (%) | 3,240,079 (57.3) | 58,414 (48.4) |
| Race/ethnicity, n (%) | ||
| American Indian or Alaska Native | 31,655 (0.6) | 1,008 (0.8) |
| Asian or Pacific Islander | 182,763 (3.2) | 4,054 (3.4) |
| Black, not of Hispanic origin | 554,330 (9.8) | 15,569 (12.9) |
| Hispanic origin | 325,874 (5.8) | 7,608 (6.3) |
| White, not of Hispanic origin | 3,734,387 (66.0) | 77,256 (64.1) |
| Null/unknown | 827,509 (14.6) | 15,095 (12.5) |
| BMI, n (%) | ||
| Null/unknown | 110,035 (2.0) | 6,276 (5.2) |
| Underweight (<18.5kg/m2) | 83,447 (1.5) | 4,586 (3.8) |
| Normal weight (18.5—24.9 kg/m2) | 1,283,985 (22.7) | 31,543 (26.2) |
| Overweight (25.0–29.9kg/m2) | 1,738,535 (30.7) | 31,931 (26.5) |
| Obese class I (30.0—34.9 kg/m2) | 1,219,039 (21.6) | 21,917 (18.2) |
| Obese class II (35.0—39.9 kg/m2) | 643,912 (11.4) | 11,995 (10.0) |
| Obese class III (>40 kg/m2) | 577,565 (10.2) | 12,342 (10.2) |
| Diabetes mellitus, n (%) | ||
| None | 4,799,969 (5.5) | 90,569 (75.1) |
| Oral | 544,898 (9.6) | 13,067 (10.8) |
| Insulin | 311,651 (94.8) | 16,954 (14.1) |
| Dyspnea, n (%) | ||
| None | 5,354,011 (94.7) | 108,778 (90.2) |
| Moderate exertion | 281,063 (5.0) | 8,926 (7.4) |
| At rest | 21,444 (0.4) | 2,886 (2.4) |
| Functional health status before operation, n (%) | ||
| Independent | 5,518,640 (97.6) | 104,890 (87.0) |
| Partially dependent | 115,978 (2.1) | 10,969 (9.1) |
| Totally dependent | 21,900 (0.4) | 4,731 (3.9) |
| Cigarette smoker within 1 year, n (%) | 978,911 (17.3) | 30,736 (25.5) |
| Ventilator-dependent (within 48 h), n (%) | 5,616 (0.1) | 11,987 (9.9) |
| COPD, n (%) | 239,878 (4.2) | 12,895 (10.7) |
| Ascites (within 30 d), n (%) | 16,348 (0.3) | 3,135 (2.6) |
| Congestive heart failure (within 30 d), n (%) | 39,924 (0.7) | 5,516 (4.6) |
| Hypertension, n (%) | 2,518,843 (44.5) | 64,009 (53.1) |
| Acute renal failure, n (%) | 14,453 (0.3) | 4,988 (4.1) |
| Dialysis or hemofiltration (within 2 wk), n (%) | 68,312 (1.2) | 6,926 (5.7) |
| Disseminated cancer, n (%) | 125,874 (2.2) | 5,726 (4.8) |
| Open wound with or without infection, n (%) | 142,207 (2.5) | 22,558 (18.7) |
| Steroid use for chronic condition, n (%) | 199,340 (3.5) | 10,558 (8.8) |
| >10% loss of body weight (within 6 mo), n (%) | 64,787 (1.2) | 6,039 (5.0) |
| Bleeding disorder requiring hospitalization, n (%) | 219,647 (3.9) | 16,067 (13.3) |
| Transfusion of >1 unit pack red blood cell, n (%) | 42,552 (0.8) | 8,750 (7.3) |
| Systemic sepsis (within 48 h), n (%) | ||
| None | 5,420,759 (95.8) | 46,198 (38.3) |
| Systemic inflammatory response syndrome | 160,285 (2.8) | 9,854 (8.2) |
| Sepsis | 69,562 (1.2) | 48,300 (40.1) |
| Septic shock | 5,912 (0.1) | 16,238 (13.5) |
| Transferred status, n (%) | ||
| Admitted directly from home | 5,452,497 (96.4) | 93,649 (77.7) |
| Acute care hospital | 156,773 (2.8) | 20,872 (17.3) |
| Chronic care facility | 47,248 (0.8) | 6,069 (5.0) |
| Primary surgeon specialty, n (%) | ||
| General surgery | 2,570,228 (45.4) | 86,316 (71.6) |
| Gynecology | 467,566 (8.3) | 1,759 (1.5) |
| Neurosurgery | 294,262 (5.2) | 4,179 (3.5) |
| Orthopaedic | 1,268,662 (22.4) | 11,955 (9.9) |
| Otolaryngology | 156,194 (2.8) | 1,088 (0.9) |
| Plastics | 166,438 (2.9) | 1,572 (1.3) |
| Thoracic | 67,739 (1.2) | 2,671 (2.2) |
| Urology | 324,088 (5.7) | 3,226 (2.7) |
| Vascular | 341,341 (6.0) | 7,824 (6.5) |
| Outpatient, n (%) | 2,389,817 (42.3) | 7,099 (5.9) |
| Emergency status, n (%) | 456,256 (8.1) | 58,680 (48.7) |
| ASA classification, n (%) | ||
| I | 505,168 (8.9) | 4,241 (3.5) |
| ii | 2,584,616 (45.7) | 24,396 (20.2) |
| iii | 2,276,257 (40.2) | 52,667 (43.7) |
| IV | 285,141 (5.0) | 34,932 (29.0) |
| V | 5,336 (0.1) | 4,354 (3.6) |
| Work relative value unit, mean ± SD | 16.2 ± 9.0 | 17.9 ± 10.7 |
n = 5,777,108. p < 0.0001 for all associations.
Superficial SSI, deep SSI, organ/space SSI; pneumonia; urinary tract infection; ventilator-dependent; sepsis; septic shock.
ASA, American Society of Anesthesiologists class (I, normal health patient; II, patient with mild systemic disease; III, patient with severe systemic disease; IV, patient with severe systemic disease that is a constant threat to life; V, a moribund patient who is not expected to survive without the operation); PATOS, present at time of surgery; SSI, surgical site infection.
Effects on unadjusted event rates of ignoring vs taking PATOS into account for the 8 outcomes are presented in Table 2. For each of the 8 outcomes, the percentages of patients with the events that were PATOS ranged considerably, from 6.52% for superficial SSI to 52.92% for sepsis (fourth column from left in Table 2). Observed event rates and 95% CIs are shown when ignoring PATOS (second column) and taking PATOS into account (seventh column) for the 8 outcomes. The event rates after taking PATOS into account declined for all outcomes, and none of the 95% CIs when ignoring PATOS vs taking PATOS into account overlapped. Observed event rates were reduced between 6.1% for SSI to 52.5% for sepsis taking PATOS into account (last column in Table 2).
Table 2.
Effect of Ignoring vs Taking Present at Time of Surgery into Account on Unadjusted Event Rate for the 8 NSQIP Outcomes Having Associated Present at Time of Surgery Data
| Postoperative outcome | Ignoring PATOS | Taking PATOS into account | ||||||
|---|---|---|---|---|---|---|---|---|
| No. of events | Event rate*(95% CI) | PATOS | n for analysis | No. of events remaining | Event rate†(95% CI) | % change in event rate | ||
| No. of events | % of all events | |||||||
| Superficial SSI | 85,588 | 1.48 (1.47–1.49) | 5,579 | 6.52 | 5,771,529 | 80,009 | 1.39 (1.38–1.40) | −6.1 |
| Deep SSI | 28,517 | 0.49 (0.49–0.50) | 7,205 | 25.27 | 5,769,903 | 21,312 | 0.37 (0.36–0.37) | −24.5 |
| Organ space SSI | 75,755 | 1.31 (1.31–1.33) | 24,324 | 32.11 | 5,752,784 | 51,431 | 0.89 (0.89–0.90) | −32.1 |
| Pneumonia | 65,953 | 1.14 (1.14–1.15) | 12,537 | 19.01 | 5,764,571 | 53,416 | 0.93 (0.92–0.93) | −18.4 |
| UTI | 74,169 | 1.28 (1.28–1.30) | 11,971 | 16.14 | 5,765,137 | 62,198 | 1.08 (1.07–1.09) | −15.6 |
| Ventilator | 52,622 | 0.91 (0.90–0.92) | 10,813 | 20.55 | 5,766,295 | 41,809 | 0.73 (0.72–0.73) | −19.8 |
| Sepsis | 93,483 | 1.62 (1.62–1.64) | 49,468 | 52.92 | 5,727,640 | 44,015 | 0.77 (0.76–0.78) | −52.5 |
| Septic shock | 45,050 | 0.78 (0.78–0.79) | 21,645 | 48.05 | 5,755,463 | 23,405 | 0.41 (0.40–0.41) | −47.4 |
n = 5,777,108.
Event rate = no. of events/5,777,108.
Event rate = no. of events remaining/(5,777,108 − no. of PATOS events).
PATOS, present at time of surgery; SSI, surgical site infection; UTI, urinary tract infection
Table 3 and Figure 1 present the O/E ratios and 95% CIs when ignoring PATOS vs taking PATOS into account stratified by operative year for the 8 outcomes. The O/E ratio plots were similar when ignoring PATOS vs taking PATOS into account for all of the outcomes except sepsis and septic shock (Fig. 1). For most of the outcomes there was a general trend of lower O/E ratios over time, except for organ space SSI, which had the opposite time trend. For pneumonia, the O/E ratios initially trended up and then came down in the later years. Most of the CIs for ignoring PATOS vs taking PATOS into account overlapped at each year except for organ space SSI for year 2012 and sepsis and septic shock for several of the years.
Table 3.
Risk-Adjusted Observed to Expected Ratios of Postoperative Outcomes When Ignoring Present at Time of Surgery vs Taking Present at Time of Surgery into Account by Operation Year
| Postoperative outcome | Ignoring PATOS | Accounting for PATOS | ||||
|---|---|---|---|---|---|---|
| Total, n | O/E | 95% CI | Total, n | O/E | 95% CI | |
| Superficial SSI | 5,777,108 | 5,771,529 | ||||
| 2012 | 534,552 | 1.153 | 1.131–1.176 | 533,988 | 1.163 | 1.140–1.187 |
| 2013 | 638,872 | 1.087 | 1.067–1.107 | 638,232 | 1.093 | 1.072–1.114 |
| 2014 | 737,839 | 1.073 | 1.054–1.092 | 737,067 | 1.074 | 1.054–1.094 |
| 2015 | 870,897 | 0.996 | 0.979–1.014 | 869,954 | 0.988 | 0.971–1.006 |
| 2016 | 980,222 | 0.968 | 0.952–0.984 | 979,264 | 0.964 | 0.947–0.981 |
| 2017 | 1,011,488 | 0.923 | 0.907–0.939 | 1,010,586 | 0.921 | 0.905–0.937 |
| 2018 | 1,003,238 | 0.905 | 0.889–0.920 | 1,002,438 | 0.906 | 0.890–0.922 |
| Deep SSI | 5,777,108 | 5,769,903 | ||||
| 2012 | 534,552 | 1.107 | 1.068–1.146 | 533,951 | 1.181 | 1.136–1.229 |
| 2013 | 638,872 | 1.227 | 1.190–1.266 | 637,932 | 1.245 | 1.202–1.290 |
| 2014 | 737,839 | 1.280 | 1.244–1.317 | 736,627 | 1.269 | 1.228–1.312 |
| 2015 | 870,897 | 1.154 | 1.123–1.187 | 869,551 | 1.134 | 1.098–1.172 |
| 2016 | 980,222 | 0.866 | 0.840–0.892 | 979,030 | 0.831 | 0.801–0.862 |
| 2017 | 1,011,488 | 0.792 | 0.767–0.817 | 1,010,475 | 0.787 | 0.759–0.817 |
| 2018 | 1,003,238 | 0.771 | 0.746–0.796 | 1,002,337 | 0.776 | 0.748–0.806 |
| Organ space SSI | 5,777,108 | 5,752,784 | ||||
| 2012 | 534,552 | 0.748 | 0.729–0.768 | 533,153 | 0.820 | 0.796–0.845 |
| 2013 | 638,872 | 0.834 | 0.815–0.853 | 636,704 | 0.860 | 0.837–0.884 |
| 2014 | 737,839 | 0.916 | 0.989–0.935 | 734,905 | 0.915 | 0.893–0.938 |
| 2015 | 870,897 | 0.964 | 0.946–0.982 | 867,272 | 0.954 | 0.933–0.976 |
| 2016 | 980,222 | 1.073 | 1.055–1.091 | 975,734 | 1.056 | 1.034–1.078 |
| 2017 | 1,011,488 | 1.133 | 1.115–1.152 | 1,006,712 | 1.123 | 1.101–1.145 |
| 2018 | 1,003,238 | 1.160 | 1.142–1.179 | 998,304 | 1.139 | 1.117–1.162 |
| Pneumonia | 5,777,108 | 5,764,571 | ||||
| 2012 | 534,552 | 0.889 | 0.866–0.912 | 533,631 | 0.925 | 0.900–0.951 |
| 2013 | 638,872 | 0.987 | 0.965–1.009 | 637,460 | 0.992 | 0.968–1.017 |
| 2014 | 737,839 | 1.087 | 1.065–1.109 | 736,042 | 1.086 | 1.061–1.111 |
| 2015 | 870,897 | 1.088 | 1.068–1.108 | 868,756 | 1.080 | 1.058–1.103 |
| 2016 | 980,222 | 1.031 | 1.012–1.050 | 978,038 | 1.027 | 1.006–1.048 |
| 2017 | 1,011,488 | 0.986 | 0.968–1.005 | 1,009,296 | 0.974 | 0.954–0.995 |
| 2018 | 1,003,238 | 0.913 | 0.895–0.931 | 1,001,348 | 0.912 | 0.893–0.932 |
| UTI | 5,777,108 | 5,765,137 | ||||
| 2012 | 534,552 | 1.194 | 1.168–1.220 | 533,147 | 1.188 | 1.160–1.216 |
| 2013 | 638,872 | 1.086 | 1.063–1.108 | 637,249 | 1.063 | 1.039–1.087 |
| 2014 | 737,839 | 1.053 | 1.033–1.074 | 736,002 | 1.022 | 1.000–1.044 |
| 2015 | 870,897 | 1.028 | 1.009–1.047 | 868,989 | 1.024 | 1.003–1.044 |
| 2016 | 980,222 | 0.946 | 0.929–0.963 | 978,426 | 0.957 | 0.938–0.976 |
| 2017 | 1,011,488 | 0.917 | 0.901–0.933 | 1,009,724 | 0.929 | 0.911–0.947 |
| 2018 | 1,003,238 | 0.918 | 0.901–0.934 | 1,001,600 | 0.940 | 0.922–0.958 |
| Ventilator | 5,777,108 | 5,766,295 | ||||
| 2012 | 534,552 | 1.131 | 1.104–1.160 | 533,277 | 1.164 | 1.132–1.196 |
| 2013 | 638,872 | 1.110 | 1.084–1.136 | 637,456 | 1.124 | 1.095–1.154 |
| 2014 | 737,839 | 1.047 | 1.023–1.071 | 736,373 | 1.059 | 1.034–1.086 |
| 2015 | 870,897 | 1.033 | 1.010–1.055 | 869,177 | 1.034 | 1.009–1.059 |
| Sepsis | 5,777,108 | 5,727,640 | ||||
| 2012 | 534,552 | 0.822 | 0.803–0.840 | 532,299 | 1.168 | 1.136–1.199 |
| 2013 | 638,872 | 0.925 | 0.907–0.944 | 634,299 | 1.019 | 0.992–1.047 |
| 2014 | 737,839 | 1.068 | 1.050–1.086 | 730,977 | 1.061 | 1.034–1.087 |
| 2015 | 870,897 | 1.013 | 0.997–1.030 | 863,207 | 0.997 | 0.973–1.021 |
| 2016 | 980,222 | 1.024 | 1.009–1.040 | 971,158 | 0.958 | 0.936–0.981 |
| 2017 | 1,011,488 | 1.052 | 1.036–1.068 | 1,001,702 | 0.958 | 0.936–0.980 |
| 2018 | 1,003,238 | 1.014 | 0.998–1.029 | 993,998 | 0.921 | 0.900–0.944 |
| Septic shock | 5,777,108 | 5,755,463 | ||||
| 2012 | 534,552 | 0.793 | 0.768–0.819 | 533,442 | 1.012 | 0.973–1.053 |
| 2013 | 638,872 | 0.986 | 0.960–1.013 | 636,607 | 1.088 | 1.050–1.127 |
| 2014 | 737,839 | 1.121 | 1.094–1.148 | 734,674 | 1.159 | 1.121–1.197 |
| 2015 | 870,897 | 1.038 | 1.014–1.062 | 867,364 | 1.008 | 0.975–1.042 |
| 2016 | 980,222 | 1.015 | 0.992–1.038 | 976,454 | 0.974 | 0.943–1.006 |
| 2017 | 1,011,488 | 1.002 | 0.979–1.025 | 1,007,625 | 0.926 | 0.896–0.956 |
| 2018 | 1,003,238 | 0.988 | 0.966–1.011 | 999,297 | 0.895 | 0.865–0.925 |
Expected numbers were calculated by summing the patient probabilities of the events from a multiple logistic regression analysis with the complication as the dependent variable and the 28 nonlaboratory preoperative American College of Surgeons NSQIP variables as the independent variables.
O/E, observed to expected ratio; PATOS, present at time of surgery; SSI, surgical site infection; UTI, urinary tract infection.
Figure 1.
Observed to expected (O/E) ratios of ignoring vs accounting for present at time of surgery (PATOS) on risk-adjusted outcomes by operation year. (A) Superficial surgical site infection (SSI), (B) deep SSI, (C) organ space SSI, (D) pneumonia, (E) urinary tract infection, (F) ventilator, (G) sepsis, (H) septic shock.
DISCUSSION
In the ACS NSQIP, the percentage of the 8 postoperative complications that were PATOS varied considerably by type of complication, lowest for superficial SSI, highest for sepsis/septic shock, and moderate for the other outcomes. Taking PATOS into account reduced unadjusted event rates for all complications. However, risk-adjusted O/E ratios over time were similar ignoring PATOS vs taking PATOS into account for most outcomes, except for sepsis and septic shock. This was probably due to the fact that patients having PATOS were sicker, so that when PATOS patients were removed, both observed and expected event rates decreased. Approximately half of patients with postoperative sepsis or septic shock had PATOS, so the largest effects of accounting for PATOS when estimating unadjusted and risk-adjusted rates were seen in these outcomes.
We found 2 other quality improvement programs that take PATOS into account. Konnor and colleagues16 used National Healthcare Safety Network data to estimate the impact of PATOS on standardized infection ratios for SSIs and found that the standardized infection ratio for colon procedures was reduced by 19% when PATOS was taken into account: 1.26 (95% CI 0.66 to 2.18) to 1.02 (95% CI 0.47 to 1.94). There is a large body of literature on the effect of present on admission (PoA) on rates of the AHRQ patient safety indicators (PSIs) (eg 17 to 24). Bahl and colleagues17 found that 13 of their PSIs had at least 1 PoA; all but 1 of the PSIs were lower after accounting for PoA; and 5 had statistically significant reductions. Borzecki and colleagues,18,19 Cevasco and colleagues,20,21 and Kaafarani and colleagues22 found that PoA was the reason for false positive PSIs 5% to 36% of the time depending on the type of PSI. Glance and colleagues23 found that inclusion of PoA frequently results in changes in the quality ranking of hospitals in California.
Many of the O/E ratios (superficial and deep SSI, urinary tract infection, ventilator dependence, and sepsis and septic shock after taking PATOS into account) trended downward over time. This is the hope of quality improvement programs, ie that risk-adjusted postoperative adverse events will decrease over time as participating institutions employ processes of care to reduce postoperative complications. The pneumonia O/E curve initially increased in the early years and then decreased in the later years, dropping back to the level at the start of the 2012 to 2018 period. Interestingly, there was a monotonically increasing O/E trend for organ space SSI. Dencker and colleagues24 also observed this in their analysis of ACS NSQIP postoperative complications. They speculated that this might be due in part to the increasing proportion of minimally invasive compared with open procedures in abdominal surgery. They reasoned that this would most likely reduce superficial and deep SSIs but not organ space SSIs. These anomalies need to be further explored in future studies.
Strengths of this study included use of the large, multicenter ACS NSQIP database during a 7-year period and the high quality of the data. There were 2 major limitations. First, institution identifiers were not available, so we could not compare the effect of ignoring vs taking PATOS into account in risk-adjusted results and rankings of individual institutions, which is the main purpose of quality improvement programs. The fact that we did not find many differences in O/E ratios ignoring PATOS vs taking PATOS into account across time does not necessarily mean that we would not find differences in O/E ratios across hospitals. Second, from an epidemiologic perspective, the ideal adjustment would omit from the baseline risk sample all patients who had the specific complication condition evident at baseline, to estimate the true incidence rate for the postoperative complication among patients without the condition evident at baseline. However, that information is not available in the ACS NSQIP database; it is available only for patients who did have the specific postoperative complications being studied. Therefore, we do not know who else should have been omitted when deriving rates taking PATOS into account.
In conclusion, accounting for PATOS was important for estimating unadjusted postoperative complication rates. The effect varied by type of complication, lowest for superficial SSI and highest for sepsis and septic shock. Taking PATOS into account was less important for risk-adjusted outcomes (O/E ratios), except for sepsis and septic shock.
Support:
This work was supported by the Agency for Healthcare Research and Quality grant R01HS027417.
Abbreviations and Acronyms
- ACS
American College of Surgeons
- O/E
observed to expected
- PATOS
present at the time of surgery
- PoA
present on admission
- PSI
patient safety indicator
- SSI
surgical site infection
Footnotes
Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. The American College of Surgeons NSQIP and participating hospitals are the source of these data; the American College of Surgeons has not verified and is not responsible for the statistical validity of the data analysis or the conclusions derived by the authors.
Disclosure Information: Nothing to disclose.
Disclosures outside the scope of this work: Dr Meguid is a paid consultant to Medtronic. Other authors have nothing to disclose.
Presented at the American College of Surgeons 108th Annual Clinical Congress, Scientific Forum, San Diego, CA, October 2022.
Contributor Information
Michael R Bronsert, Surgical Outcomes and Applied Research Program; Adult and Child Center for Health Outcomes Research and Delivery Science.
William G Henderson, Surgical Outcomes and Applied Research Program; Adult and Child Center for Health Outcomes Research and Delivery Science; Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO.
Kathryn L Colborn, Surgical Outcomes and Applied Research Program; Department of Surgery, University of Colorado, School of Medicine, Aurora, CO; Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO.
Adam R Dyas, Surgical Outcomes and Applied Research Program; Department of Surgery, University of Colorado, School of Medicine, Aurora, CO.
Helen J Madsen, Surgical Outcomes and Applied Research Program; Department of Surgery, University of Colorado, School of Medicine, Aurora, CO.
Yaxu Zhuang, Surgical Outcomes and Applied Research Program; Department of Surgery, University of Colorado, School of Medicine, Aurora, CO; Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO.
Anne Lambert-Kerzner, Surgical Outcomes and Applied Research Program; Adult and Child Center for Health Outcomes Research and Delivery Science.
Robert A Meguid, Surgical Outcomes and Applied Research Program; Adult and Child Center for Health Outcomes Research and Delivery Science; Department of Surgery, University of Colorado, School of Medicine, Aurora, CO; Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO.
REFERENCES
- 1.Massarweh NN, Kaji AH, Itani KMF. Practical guide to surgical data sets: Veterans Affairs Surgical Quality Improvement Program. JAMA Surg 2018;153:768–769. [DOI] [PubMed] [Google Scholar]
- 2.Horan TC, Andrus M, Dudeck MA. CDC/NHSN surveillance definition of health care-associated infection and criteria for specific types of infections in the acute care setting. Am J Infect Control 2008;36:309–332. [DOI] [PubMed] [Google Scholar]
- 3.Vizient. Vizient Member Networks. 2019. Available at: https://www.vizientinc.com/our-networks/vizient-member-networks. Accessed May 25, 2020.
- 4.Colborn KL, Bronsert M, Amioka E, et al. Identification of surgical site infections using electronic health record data. Am J Infect Control 2018;46:1230–1235. [DOI] [PubMed] [Google Scholar]
- 5.Corey KM, Kashyap S, Lorenzi E, et al. Development and validation of machine learning models to identify high-risk surgical patients using automatically curated electronic health record data (Pythia): a retrospective, single-site study. PLoS Med 2018;15:e1002701. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bihorac A, Ozrazgat-Baslanti T, Ebadi A, et al. MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery. Ann Surg 2019;269:652–662. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Colborn KL, Bronsert M, Hammermeister K, et al. Identification of urinary tract infections using electronic health record data. Am J Infect Control 2019;47:371–375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Bronsert M, Singh AB, Henderson WG, et al. Identification of postoperative complications using electronic health record data and machine learning. Am J Surg 2020;220:114–119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Chen W, Lu Z, You L, et al. Artificial intelligence-based multimodal risk assessment model for surgical site infection (AMRAMS): development and validation study. JMIR Med Inform 2020;8:e18186. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Zhu Y, Simon GJ, Wick EC, et al. Applying machine learning across sites: external validation of a surgical site infection detection algorithm. J Am Coll Surg 2021;232:963–971.e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Birkmeyer JD, Shahian DM, Dimick JB, et al. Blueprint for a new American College of Surgeons: National Surgical Quality Improvement Program. J Am Coll Surg 2008;207:777–782. [DOI] [PubMed] [Google Scholar]
- 12.Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY. Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 2009;250:363–376. [DOI] [PubMed] [Google Scholar]
- 13.Ingraham AM, Richards KE, Hall BL, Ko CY. Quality improvement in surgery: the American College of Surgeons National Surgical Quality Improvement Program approach. Adv Surg 2010;44:251–267. [DOI] [PubMed] [Google Scholar]
- 14.Hall BL, Richards K, Ingraham A, Ko CY. New approaches to the National Surgical Quality Improvement Program: the American College of Surgeons experience. Am J Surg 2009;198:S56–S62. [DOI] [PubMed] [Google Scholar]
- 15.Shiloach M, Frencher SK Jr, Steeger JE, et al. Toward robust information: data quality and inter-rater reliability in the American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg 2010;210:6–16. [DOI] [PubMed] [Google Scholar]
- 16.Konnor R, Russo V, Leaptrot D, et al. How infection present at time of surgery (PATOS) data impacts your surgical site infection (SSI) standardized infection ratios (SIR), with focus on the complex 30-day SSI SIR model. Am J Infect Control 2021;49:1423–1426. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bahl V, Thompson MA, Kau T-Y, Hu HM, Campbell DA. Do the AHRQ patient safety indicators flag conditions that are present at time of hospital admission? Med Care 2008;46:516–522. [DOI] [PubMed] [Google Scholar]
- 18.Borzecki AM, Kaafarani H, Cevasco M, et al. How valid is the AHRQ patient safety indicator “postoperative hemorrhage or hematoma”? J Am Coll Surg 2011;212:946–953.e1. [DOI] [PubMed] [Google Scholar]
- 19.Borzecki AM, Cevasco M, Chen Q, et al. How valid is the AHRQ patient safety indicator “postoperative physiologic and metabolic derangement”? J Am Coll Surg 2011;212:968–976.e1. [DOI] [PubMed] [Google Scholar]
- 20.Cevasco M, Borzecki AM, O’Brien WJ, et al. Validity of the AHRQ patient safety indicator “central venous catheter-related bloodstream infections”. J Am Coll Surg 2011;212:984–990. [DOI] [PubMed] [Google Scholar]
- 21.Cevasco M, Borzecki AM, Zrelak PA, et al. Positive predictive value of the AHRQ patient safety indicator “postoperative sepsis”: implications for practice and policy. J Am Coll Surg 2011;212:954–961. [DOI] [PubMed] [Google Scholar]
- 22.Kaafarani HMA, Borzecki AM, Itani KMF, et al. Validity of selected patient safety indicators: opportunities and concerns. J Am Coll Surg 2011;212:924–934. [DOI] [PubMed] [Google Scholar]
- 23.Glance LG, Osler TM, Mukamel DB, Dick AW. Impact of present-on-admission indicator on hospital quality measurement: experience with the Agency for Healthcare Research and Quality (AHRQ) inpatient quality indicators. Med Care 2008;46:112–119. [DOI] [PubMed] [Google Scholar]
- 24.Dencker EE, Bonde A, Troelsen A, et al. Postoperative complications: an observational study of trends in the United States from 2012 to 2018. BMC Surg 2021;21:393. [DOI] [PMC free article] [PubMed] [Google Scholar]