Abstract
Surgical procedures carry the risk of postoperative infectious complications, which can be severe, expensive, and morbid. A growing body of evidence indicates that high-resolution intraoperative data can be predictive of these complications. However, these studies are often contradictory in their findings as well as difficult to replicate, suggesting that these predictive models may be capturing institutional artifacts. In this work, data and models from two independent institutions, Mayo Clinic and University of Minnesota-affiliated Fairview Health Services, were directly compared using a common set of definitions for the variables and outcomes. We built perioperative risk models for seven infectious post-surgical complications at each site to assess the value of intraoperative variables. Models were internally validated. We found that including intraoperative variables significantly improved the models’ predictive performance at both sites for five out of seven complications. We also found that significant intraoperative variables were similar between the two sites for four of the seven complications. Our results suggest that intraoperative variables can be related to the underlying physiology for some infectious complications.
Keywords: Postoperative complications, Machine learning
Introduction
Surgical procedures carry the risk of postoperative infectious complications, which can be severe, expensive, and can put patients’ lives at risk [1–5]. With the ubiquity of decision support tools integrated into electronic health record (EHR) systems, perioperative decision support capabilities represent a promising direction for reducing postoperative risk or tailoring care including interventions aimed at individualized anticipation and management of the complication risk.
Initial surgical planning is primarily based on a preoperative assessment. While clearly essential, this assessment is insufficient to capture dynamic risks or changes in patient status, which may occur in the operating room. Preoperative assessment is, by definition, based on preoperative data and the planned procedure. Surgeries can deviate from the original plan for a variety of reasons or may involve unexpected physiologic changes such as bleeding, rendering the original risk estimates less valid. While the original estimates reflect the general risk of the patient and the planned surgery, they reflect neither new findings, changes in patient status, nor an altered course of surgery.
A growing body of published literature indicates that high-resolution intraoperative data can be predictive of complications [6–9]. However, the findings in these studies often contradict each other, are difficult to replicate at other sites, and are often based on very limited use of intraoperative data. For example, several studies have reported intraoperative hypothermia increases the risk of post-surgical infectious complications [6–8]. On the contrary, other studies found that there were no differences in the minimum, maximum, or ending temperatures between patients with surgical site infection (SSI) and those without – potentially due to an institutional effect: better temperature control at certain institutions. While temperature was unassociated with the surgical site infection, other intraoperative factors, most prominently blood loss, showed a significant association [9]. Besides institutional differences in clinical practice, differences in findings may also arise due to differing analytical approaches. Studies differ in the way they aggregate intraoperative variables: some studies use moments and extrema (minimum and maximum), while others use the ending measurement (the last measurement or the average of the last few measurements). Finally, differences in findings can also stem from discrepancies in definitions of the variables and outcomes.
Several high-quality registries form the backbone of surgical quality improvement research aimed at understanding and reducing postoperative complications. The National Surgical Quality Improvement Project (NSQIP) registry [10, 11] uses high-quality manually curated data [1, 12] and stands out as the gold-standard for surgical quality improvement and surgical outcomes research. NSQIP offers standardized definitions for exposure/risk variables and outcomes, helping to ensure consistency across studies. Unfortunately, it lacks detailed information regarding intraoperative risk factors, including physiologic data, laboratory data, medications, or other treatments, limiting its use for our purpose.
We conducted this study at two independent sites, Mayo Clinic and University of Minnesota-affiliated Fairview Health Services. These are two large Midwestern academic health systems with tertiary centers providing a wide range of surgical services. Both sites are members of the NSQIP registry; so, we were able to use outcome data from NSQIP patients at each site, serving as a high quality gold-standard. We used preoperative and intraoperative EHR data collected from the clinical data repositories of the respective sites and standardized the data elements across the two sites to facilitate direct comparison of the models.
The study aims to answer the following questions: (i) In the context of perioperative decision support, does the use of intraoperative data improve the performance of 30-day postoperative risk models? (ii) Do significant intraoperative variables in the risk models pertain to the same physiological concepts across the two sites? We proceed with the assumption that when significant intraoperative variables differ despite having been defined identically, the models may capture institutional differences; when the significant variables coincide, they are more likely to relate to the physiological processes underlying the postoperative complications.
Methods
Setting and cohort definition.
We consider two independent Midwestern health care systems: Mayo Clinic (MC) and Fairview Health Services (FHS). We include all patients from MC and FHS between 2010 and 2017 who are part of the NSQIP sample. For these patients we collected all available information about their NSQIP index surgery and a 30-day history before the index surgery from the respective institutions’ EHR repositories. The NSQIP registry collects complications within a 30-day postoperative window. If a patient had another surgery in the 30-day postoperative window, we used the index surgery and measured the 30-day postoperative window for the outcome from the index surgery. For each complication, patients with the same pre-existing complication at the time of surgery were excluded.
Independent variables.
We primarily rely on known risk factors of infection [13–15]. Independent variables were divided into three groups: demographic, preoperative, and high-resolution intraoperative. We limit ourselves to basic demographic information, such as age, sex, and body mass index (BMI) that are generally available. Preoperative variables are historic diagnoses including the problem list, procedures, and medications, as well as the preoperative indication for surgery. We use laboratory results and vitals to establish a preoperative baseline. Data from the preoperative assessment were preferentially used; if the data is not available, we use measurements from no more than 30-days before surgery. Diagnosis codes are rolled up into complications using the Clinical Classification Software [16].
Aggregating intraoperative variables.
The intraoperative variables include orders, medications, and high-resolution vitals and labs. The stream of high-resolution variables needs to be divided based on the three stages of anesthesia. During the first (approximately) 15 minutes, called the induction phase, the vitals drop and deviate heavily from the normal. The last 15 minutes is called the emergence phase, where vitals are expected to return to close to normal but also involves significant changes as the patient is transitioned from anesthesia—often with full ventilation—to no anesthesia. The operation takes place between these two phases, in the so-called maintenance phase, where the vitals are expected to remain stable, although different from the preoperative baseline. In this work, we focus on the maintenance phase and use the mean value of labs and vitals during this phase.
Outcomes.
We consider seven infectious outcomes: sepsis, septic shock, urinary tract infection (UTI), pneumonia (PNA) and the three kinds of SSI (superficial, deep tissue, and organ space SSI). We extract the outcome information from the NSQIP registry.
Modeling.
Each outcome was modeled independently using logistic regression. For each outcome, two models are constructed. The “pre” model only uses demographics and preoperative data (30-day history of complications, baseline labs and vitals), while the “pre+intra” model uses aggregated intraoperative measurements on top of the preoperative and demographic data.
All models are logistic regression models. Missing labs and vitals were imputed using the middle of the normal range for the measurement. This assumes that the measurement is missing because it was deemed unnecessary to measure. The initially high number of independent variables was reduced by causal variable screening [19]. We used the PC-Simple algorithm [20] with a maximal condition set size of three [21]. This algorithm discards all variables that are independent of the outcome given at most three other variables. The rationale is that these variables do not affect the outcome directly, they only affect the outcome through other variables. Subsequent backwards elimination was applied to the remaining set of independent variables with a significance level of .05. The R statistical computing environment was used for all modeling. The PC-Simple algorithm is available in the pcalg R package [20].
Evaluation.
Consistent with the intended use in perioperative decision support, we evaluated the models based on their predictive performance using concordance as the metric. Concordance is the probability that between two randomly selected patients, among which one has the complication in question while the other does not, the one with the complication has the higher predicted risk. Concordance is equivalent to the commonly reported area under the receiver-operating-characteristic curve (AUC).
Internal validation and effect of the intraoperative variables.
Bootstrap estimation with 200 replications was used to estimate the concordance of the models. For each complication, two models were built on the same bootstrap replication: one with and one without the intraoperative variables. We measured the effect of the intraoperative variables as the mean difference in concordance between the models with and without intraoperative variables across the 200 bootstrap replications. The statistical significance of the difference was determined through a paired t-test.
External comparison.
If the intraoperative variables capture a valid physiological phenomenon, then the variables selected at the two sites relate to the same quantity. We manually examined the models with the intraoperative data across the two sites to determine whether the selected variables coincide or at least relate to the same physiological concept. For example, oxygen saturation and ventilator settings relate to the same physiological concept (oxygen in the blood). When the selected variables relate to different concepts, there is a risk that the model simply captures an institutional artifact.
Results
Cohort Description
Table 1 contains a description of the analytic cohort at the two health systems. It contains demographic information, outcomes, history of the outcomes, history of relevant complications, and baseline (preoperative or at most 30 days before the index surgery when preoperative measurement is unavailable) lab results and vitals. Due to the high number of lab results and vital signs, we only report those that were significant in at least one of the models. Categorical variables are reported as count and percentage in parenthesis; continuous variables are reported as median and interquartile range in parenthesis.
Table 1 –
Cohort Description.
| Mayo Clinic | Fairview | |
|---|---|---|
| Number of patients | 38,045 | 9,044 |
| Demographics | ||
| Age | 61 (49, 71) | 55 (40, 66) |
| Gender (male) | 18,769 (48.7%) | 4053 (45%) |
| Outcomes | ||
| SSI Superficial | 586 (1.5%) | 107 (1.2%) |
| SSI Deep Tissue | 186 (0.5%) | 45 (0.5%) |
| SSI Organ Space | 417 (1.1%) | 119 (1.3%) |
| UTI | 282 (0.7%) | 147 (1.6%) |
| PNA | 453 (1.2%) | 111 (1.2%) |
| Sepsis | 494 (1.3%) | 88 (0.98%) |
| Septic Shock | 186 (0.5%) | 40 (0.44%) |
| History of Complications | ||
| SSI | 1,232 (3.2%) | 91 (1.01%) |
| UTI | 1,271 (3.3%) | 127 (1.4%) |
| PNA | 777 (2.0%) | 73 (0.81%) |
| Bacteremia | 133 (0.3%) | 23 (0.25%) |
| Infections | 3,953 (10.4%) | 165 (1.8%) |
| Opport. Inf. | 432 (1.1%) | 45 (0.50%) |
| Malnutrition | 792 (2.1%) | 60 (0.66%) |
| Cancer | 13,547 (35.6%) | 1413 (16%) |
| Metastatic Disease | 4,359 (11.5%) | 112 (1.2%) |
| Transplant | 624 (1.6%) | 224 (2.5%) |
| Diabetes (T1&2) | 5,215 (13.7%) | 462 (5.1%) |
| COPD | 2,372 (6.2%) | 94 (1.04%) |
| Baseline labs and vitals | ||
| BMI | 28.4 (24.8, 32.9) | 27.9 (24.6, 33.6) |
| Pulse | 70.0 (62.6, 80.0) | 75.0 (66.3, 85.0) |
| Respiration | 9.5 (8.1, 10.8) | 16 (16, 18) |
| Bilirubin | 0.45 (0.45, 0.45) | 0.67 (0.4, 1) |
| BUN | 26 (18, 35) | 14 (10, 20) |
| RBC | 3.79 (3.13, 4.36) | 4.27 (3.77, 4.70) |
| WBC | 7.85 (4.45, 11.5) | 8.50 (6.23, 12.1) |
| MCV | 88.8 (87.2, 94.1) | 89.4 (86.0, 93.0) |
| RDW | 14.8 (13.5, 16.6) | 13.9 (13.0, 15.3) |
| Hematocrit | 33.1 (27.6, 39.7) | 38.2 (33.8, 41.6) |
| PH | 7.38 (7.32, 7.42) | 7.38 (7.32, 7.42) |
| CO2 | 40.5 (36.5, 43.5) | 25.5 (24.0, 27.5) |
| Ca | 4.71 (4.55, 4.92) | 4.59 (4.30, 4.78) |
| K | 4.15 (3.90, 4.50) | 3.9 (3.7, 4.2) |
| Na | 138 (137.8, 138.4) | 139 (137, 141) |
(1) Abbreviations: UTI: urinary tract infection; PNA: pneumonia; SSI: surgical site infection; COPD: chronic obstructive pulmonary disease; BMI: body mass index; BUN: blood urea nitrogen; RBC: red blood cell [count]; WBC: white blood cell [count]; MCV: mean corpuscular volume; RDW: red blood cell distribution width; Ca: calcium ion concentration; K: potassium ion concentration; Na: sodium ion concentration. (2) Only variables that were significant in at least one model are presented.
Model performance and effect of the intraoperative variables
Table 2 presents the predictive performance of the models for each of the outcomes. For each outcome, the table contains three rows: the first one shows the performance as measured by AUC of the “pre” model (that uses only demographics and preoperative data), the second one shows the performance of the “pre+intra” model, which uses the intraoperative data in addition to the demographics and preoperative data, and the third row shows the difference in performance between the “pre” and “pre+intra” models. For the “pre” and “pre+intra” models, the standard deviation of the performance (obtained from 200 bootstraps) is shown in parenthesis; for the difference, the p-value of the paired t-test is shown in the parenthesis using the scientific notation (e.g. 2e-8 is 2*10−8). All differences are statistically significant at 0.05 confidence level, but superficial and deep tissue SSI are not significant on FHS side after Bonferroni correction.
Table 2 –
AUC performance of the “pre”, the “pre+intra” models, and the difference in performance.
| Outcome | Model | Mayo Clinic | Fairview |
|---|---|---|---|
| PNA | Pre | .877 (±.021) | .693 (±.044) |
| Pre+intra | .881 (±.022) | .734 (±.044) | |
| Difference | .004 (1e-8) | .041 (1e-16) | |
| Sepsis | Pre | .736 (±.024) | .695 (±.045) |
| Pre+intra | .755 (±.021) | .711 (±.046) | |
| Difference | .019 (1e-16) | .016 (8e-13) | |
| Septic Shock | Pre | .827 (+.031) | .715(+.O78) |
| Pre+intra | .834 (+.030) | .755(+.O79) | |
| Difference | .007 (1e-13) | .040 (1e-14) | |
| SSI Superficial | Pre | .667 (+.018) | .560 (+.044) |
| Pre+intra | .688 (+.017) | .563 (+.042) | |
| Difference | .021 1e-16) | .003 (0.022) | |
| SSI Deep Tissue | Pre | .660 (±.041) | .568 (±.069) |
| Pre+intra | .678 (+.035) | .575 (+.064) | |
| Difference | .018 (1e-7) | .007 (0.3) | |
| SSI Organ Space | Pre | .732 (±.026) | .657 (±.037) |
| Pre+intra | .750 (±.022) | .699 (±.039) | |
| Difference | .021 (1e-16) | .042 (1e-16) | |
| UTI | Pre | .665 (±.022) | .717 (±.031) |
| Pre+intra | .672 (±.021) | .727 (±.032) | |
| Difference | .007 (1e-13) | .010 (5e-14) |
External Comparison of the Significant Variables
Table 4 displays the number of bootstrap iterations in which the variable was selected for each complication at each site. We only list variables that were selected in at least 100 of the 200 bootstrap iterations for at least one outcome at one site.
Table 4 –
Frequency at which each intraoperative variable was selected (out of 200 bootstrap iteration)
| PNA | SSI Superf. | SSI Deep T. | SSI Organ | Sepsis | Septic Shock | UTI | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MC | FV | MC | FV | MC | FV | MC | FV | MC | FV | MC | FV | MC | FV | |
| Calcium Ion | 7 | 17 | 31 | 10 | 62 | 9 | 124 | 66 | 93 | 38 | 7 | 41 | 2 | 94 |
| CO2 | 120 | 18 | 1 | 20 | 19 | 3 | 4 | 24 | 80 | 5 | 1 | 2 | 0 | 0 |
| Glucose | 8 | 1 | 16 | 0 | 19 | 11 | 160 | 0 | 105 | 0 | 115 | 36 | 66 | 0 |
| Lactate | 158 | 144 | 5 | 21 | 1 | 1 | 43 | 27 | 29 | 23 | 28 | 163 | 7 | 1 |
| MCV | 5 | 153 | 27 | 6 | 48 | 64 | 41 | 16 | 10 | 41 | 25 | 5 | 0 | 2 |
| PO2 | 42 | 90 | 6 | 17 | 24 | 63 | 11 | 192 | 12 | 95 | 40 | 12 | 4 | 110 |
| CVP | 11 | 8 | 136 | 12 | 43 | 40 | 122 | 34 | 30 | 8 | 46 | 5 | 31 | 137 |
| Isoflurane Exp. | 1 | 0 | 71 | 1 | 4 | 1 | 127 | 61 | 190 | 0 | 0 | 0 | 0 | 1 |
| FIO2 | 115 | 6 | 1 | 13 | 12 | 0 | 53 | 1 | 1 | 5 | 6 | 120 | 9 | 8 |
| PEEP | 16 | 49 | 200 | 107 | 141 | 2 | 136 | 53 | 156 | 4 | 86 | 7 | 75 | 129 |
| Pulse | 31 | 88 | 173 | 22 | 28 | 13 | 167 | 63 | 197 | 142 | 152 | 0 | 4 | 9 |
| Steroid | 22 | 32 | 5 | 36 | 115 | 50 | 3 | 8 | 1 | 8 | 7 | 0 | 106 | 6 |
| Abx | 162 | 6 | 72 | 4 | 136 | 0 | 147 | 8 | 88 | 0 | 190 | 0 | 178 | 13 |
Notes: (1) Only variables selected in at least 100 iterations are displayed. (2) Abbreviations; MC: Mayo Clinic; FV: Fairview Health Services; MCV: mean corpuscular volume; CVP: central venous pressure; FIO2: Fraction of Inspired Oxygen; PEEP: Positive End Expiratory Pressure; PO2: Partial pressure of oxygen; Abx: Antibiotic use.
Table 3 offers a summary of the information in Table 4. It lists the most important intraoperative variables that are common between the two sites for each outcome.
Table 3 –
Selected variables
| Significant intraoperaitive variables | ||
|---|---|---|
| PNA | antibiotic use, lactate, CO2, FIO2 | Lactate, MCV, pulse, SpO2 |
| Sepsis | Glucose, isoflurane expired, PEEP, pulse | PO2, RBC, pulse, hematocrit |
| Septic Shock | Antibiotic use, glucose, WBC, pulse, PEEP | Lactate, FIO2 |
| SSI Superficial | CVP, PEEP, pulse | PEEP |
| SSI Deep Tissue | Antibiotic use, steroid use, PEEP | PO2 |
| SSI Organ Space | Antibiotic use, bicarb, Ca, glucose, CVP, isoflurane expired | Hemoglobin, PH arterial, PO2, pulse |
| UTI | Antibiotic use, steroid use | Ca, PO2, CVP, PEEP |
Abbreviations: FIO2: Fraction of Inspired Oxygen; PEEP: Positive End Expiratory Pressure; PO2: Partial pressure of oxygen; WBC: white blood cell [count]; CVP: central venous pressure; Ca: calcium ion in the blood; MCV: mean corpuscular volume; SpO2: blood oxygen saturation; RBC: red blood cell [count].
Discussion
We view EHR-integrated perioperative clinical decision support as a key avenue towards reducing postoperative complications or towards better anticipating and managing them. Central to such systems are accurate risk models with the ability to produce risk estimates on demand based on contemporaneous data, including intraoperative data. In this work, we sought to answer the following two questions: (i) Do intraoperative data elements improve our ability to predict postoperative complications? and (ii) Are these models specific to institutions or do they describe possible physiological phenomena that are institution-independently predictive of postoperative complications?
Regarding the first question, we found that the inclusion of intraoperative variables led to statistically significant improvements in the risk models’ predictive ability for five of the seven infectious complications at both sites. These complications were pneumonia, sepsis, septic shock, organ space SSI, and urinary tract infection. At the Mayo Clinic site, the inclusion of the intraoperative variables led to statistically significant (albeit not necessarily clinically relevant) improvements for all complications, likely due to the much larger sample size.
The lack of statistically significant improvement for the other two SSI types, superficial and deep tissue, is unsurprising. These diagnoses have more ambiguity and our previous work on detecting SSI retrospectively also suggests that predicting superficial and deep tissue SSI is difficult [17, 18].
Regarding the second question, we found that some of the intraoperative variables overlap across the two sites for all four of the complications in which intraoperative variables improved the predictive performance. In the cases of pneumonia, sepsis, and organ-space SSI, some of the variables (lac-tate, pulse) matched exactly, while others referred to a shared concept. For example, FIO2 and SpO2 are related to the sufficiency of oxygen in the bloodstream: the first one is the ventilator setting (to supply the oxygen into the bloodstream) while the second one is the direct measurement of oxygen saturation. For sepsis, partial pressure from oxygen and PEEP have an analogous relationship; and for organ space SSI, bicarbonate and PH both measure the acidity of blood. The fact that many of the significant variables are common across the two sites suggests that these variables can be related to the physiological process(es) underlying the outcomes.
Conclusion
Intraoperative variables were found to statistically significantly improve the performance of 30-day postoperative risk models for five of the seven infectious complications at both sites. There was considerable overlap in the significant intraoperative variables across the two sites and the overlapping variables were related to lactate, acidity, and blood oxygen.
Acknowledgements
This work was supported in part by NIGMS award R01 GM 120079, AHRQ award R01 HS024532, and the NCATS University of Minnesota CTSA UL1 TR002494. The views expressed in this manuscript are those of the authors and do not necessarily reflect the views of the funding agencies.
References
- [1].McCoy CC, Englum BR, Keenan JE, Vaslef SN, Shapiro ML, Scarborough JE. Impact of specific postoperative complications on the outcomes of emergency generalsurgery patients. J Trauma Acute Care Surg. 2015. May;78(5):912–8. [DOI] [PubMed] [Google Scholar]
- [2].Vonlanthen R, Slankamenac K, Breitenstein S, Puhan MA, Muller MK, Hahnloser D, Hauri D, Graf R, Clavien PA. The impact of complications on costs of major surgical procedures: a cost analysis of 1200 patients. Ann Surg. 2011. December;254(6):907–13. [DOI] [PubMed] [Google Scholar]
- [3].Khan NA 1, Quan H, Bugar JM, Lemaire JB, Brant R, Ghali WA. Association of postoperative complications with hospital costs and length of stay in a tertiary care center. J Gen Intern Med. 2006. February;21(2):177–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [4].Brown SR 1, Mathew R, Keding A, Marshall HC, Brown JM, Jayne DG. The impact of postoperative complications on long-term quality of life after curative colorectal cancer surgery. Ann Surg. 2014. May;259(5):916–23. [DOI] [PubMed] [Google Scholar]
- [5].Artinyan A, Orcutt ST, Anaya DA, Richardson P, Chen GJ, Berger DH. Infectious postoperative complications decrease long-term survival in patients undergoing curative surgery for colorectal cancer: a study of 12,075 patients. Ann Surg. 2015. March;261(3):497–505. [DOI] [PubMed] [Google Scholar]
- [6].Baucom RB, Phillips SE, Ehrenfeld JM, Muldoon RL, Poulose BK, Herline AJ, Wise PE, Gieger TM. Association of Perioperative Hypothermia During Colectomy With Surgical Site Infection. J Am Med Assoc, 2015. [DOI] [PubMed] [Google Scholar]
- [7].Leaper D, Ousey K. Evidence update on prevention of surgical site infection. Curr Opin Infect Dis. 2015. Apr;28(2):158–63. [DOI] [PubMed] [Google Scholar]
- [8].Seamon MJ, Wobb J, Gaughan JP, Kulp H, Kamel I, Dempsey DT. The effects of intraoperative hypothermia on surgical site infection: an analysis of 524 trauma laparotomies. Ann Surg. 2012. April;255(4):789–95. [DOI] [PubMed] [Google Scholar]
- [9].Melton GB, Vogel JD, Swenson BR, Remzi FH, Rothenberger DA, Wick EC. Continuous Intraoperative Temperature Measurement and Surgical Site Infection Risk. Ann Surg. 2013, 258:606–613. [DOI] [PubMed] [Google Scholar]
- [10].Henderson. WG, Daley J. Design and statistical methodology of the National Surgical Quality Improvement Program: why is it what it is? Am J Surg 2009; 198:S19. [DOI] [PubMed] [Google Scholar]
- [11].Shiloach M, Frencher SK Jr., Steeger JE, Rowell KS, Bartzokis K, Tomeh MG, Richards KE, Ko CY, Hall BL. Toward robust information: data quality and inter-rater reliability in the American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg. 2010. January;210(1):6–16. [DOI] [PubMed] [Google Scholar]
- [12].Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY. Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg. 2009. September;250(3):363–76. [DOI] [PubMed] [Google Scholar]
- [13].Phillips J, O’Grady H, Baker E. Prevention of Surgical Site Infections. Surgery 2014; 32(9). [Google Scholar]
- [14].Korol E, Johnston K, Waser N, Sifakis F, Jafri HS, Lo M, Kyaw MH. A Systematic Review of Risk Factors Associated with Surgical Site Infections among Surgical Patients. Plos One 2013; 8(12). [DOI] [PMC free article] [PubMed] [Google Scholar]
- [15].Pessaux P, Msika S, Atalla D, Hay JM, Flamant Y. Risk factors for postoperative infectious complications in noncolorectal abdominal surgery: a multivariate analysis based on a prospective multicenter study of 4718 patients. Arch Surg. 2003. March;138(3):314–24. [DOI] [PubMed] [Google Scholar]
- [16].Elixhauser A, Steiner C, Palmer L. Clinical Classifications Software (CCS), 2015. U.S. Agency for Healthcare Research and Quality. Available: http://www.hcupus.ahrq.gov/toolssoftware/ccs/ccs.jsp. [Google Scholar]
- [17].Hu Z, Melton GB, Moeller ND, Arsoniadis EG, Wang Y, Kwaan MR, Jensen EH, Simon GJ. Accelerating Chart Review Using Automated Methods on Electronic Health Record Data for Postoperative Complications. AMIA Annual Symposium Proceedings 2016, 1822. [PMC free article] [PubMed] [Google Scholar]
- [18].Hu Z, Simon GJ, Arsoniadis EG, Wang Y, Kwaan MR, Melton GB. Automated detection of postoperative surgical site infections using supervised methods with electronic health record data. Studies in health technology and informatics 2015; 216, 706. [PMC free article] [PubMed] [Google Scholar]
- [19].Ma S, Statnikov A. Methods for computational causal discovery in biomedicine. Behaviormetrika. 2017. January 1; 44 (1): 165–91. [Google Scholar]
- [20].Kalisch M, Maechler M, Colombo D, Maathuis MH, Buehlmann P. Causal Inference Using Graphical Models with the R Package pcalg. Journal of Statistical Software, 2012; 47(11), 1–26. URL http://www.jstatsoft.org/v47/i11/. [Google Scholar]
- [21].Aliferis CF, Tsamardinos I, Statnikov A. HITON: a novel Markov Blanket algorithm for optimal variable selection. In AMIA Annual Symposium Proceedings 2003. (Vol. 2003, p. 21). American Medical Informatics Association. [PMC free article] [PubMed] [Google Scholar]