Abstract
Introduction
Invasive fungal infections (IFI) are associated with high morbidity and mortality. A better method of risk stratifying trauma patients for combat-related IFI is needed to improve clinical outcomes while minimizing morbidity related to overtreatment. We sought to develop combat-related IFI clinical decision support (CDS) tools to assist providers to make treatment decisions both near the point of injury and subsequently at definitive treatment centers.
Materials and Methods
We utilized a training dataset containing information from 227 combat-injured military personnel to build a Bayesian belief network (BBN) to predict the likelihood of developing IFI using information available at the point of initial resuscitation (THEATER model) and in the tertiary care setting (MEDCEN model). After selecting BBN models, external validation used a separate test dataset of 350 wounded warriors. Furthermore, the performance of the BBN models was compared with a “two-rule model” alone (based on physician experience) and combinations of the BBN models plus the two-rule model. The two-rule model contains plausible IFI criteria, but it has not been formally evaluated, and they are not currently actual clinical guidelines.
Results
We found receiver operating characteristic areas under the curve (AUC) of 0.70 (95% CI: [0.62, 0.77]) and 0.68 (95% CI: [0.59, 0.76]) for the THEATER and MEDCEN BBN models, respectively, on cross-validation. External validation with the highest AUC BBN models produced THEATER AUC of 0.68 (95% CI: [0.58, 0.78]) and MEDCEN AUC of 0.67 (95% CI: [0.57, 0.78]). With the incorporation of two-rule model in low IFI-prevalence populations, external validation AUC increased to 0.77 (95% CI: [0.69, 0.84]) for the THEATER model and 0.76 (95% CI:[0.68, 0.85]) for the LRMC model. The two-rule model alone has an AUC of 0.72 (95% CI: [0.63, 0.81]).
Conclusions
Overall, the IFI tools produced clinically useful, robust models. However, the clinical utility of these models is highly dependent upon the clinician’s individual risk tolerance. The threshold probability for optimal clinical use of this CDS tool is currently being evaluated in an ongoing clinical utilization study. CDS tools, such as these, may facilitate early diagnosis of patients with or at risk for IFI, permitting early or prophylactic treatment with the aim of improving outcomes.
Keywords: invasive fungal infection, clinical decision support tool, Bayesian belief network
INTRODUCTION
During the recent military conflict in Afghanistan, invasive fungal infection (IFI) emerged as an infectious complication with surprising incidence (7%) and high morbidity and mortality (8%) among severely injured military personnel.1,2 Devastating mortality rates as high as 38–96% have also been reported in civilian trauma and medical populations.3,4 A primary characteristic of this disease is recurrent tissue necrosis within the wound despite serial surgical débridements, straining valuable health care resources while adding continuous physiologic insult to severely injured patients. Indeed, combat casualites who develop IFIs have significantly more surgical amputations and proximal amputation revisions, a greater number of operative visits, higher proportion of bacterial coinfections, and a longer duration to initial surgical wound closure postinjury compared to patients without the disease.5 Sustaining a blast injury on foot patrol, traumatic transfemoral amputation, and/or requiring massive (>20 units) blood product transfusions during the first 24 hours postinjury were identified to be independent IFI risk factors.6 Treatment recommendation center on aggressive and frequent débridements and early initiation of antifungal therapy when there is a high suspicion of IFI.1 However, if we are to prescribe early and aggressive treatment in a reliable and consistent manner and avoid unnecessary systemic complications of overtreatment, a method to estimate the likelihood of IFI using patient- and injury-specific information is required.
In response to the 1999 Institute of Medicine report “To Err is Human”,7 the U.S. health care industry witnessed a steady rise in the use of clinical decision support (CDS) tools, and with it, a corresponding improvement in patient outcomes.8,9 Like other CDS tools, one developed to identify acutely traumatized patients at risk for IFI must be designed to assist providers at the point of care. The purpose of this manuscript is to present our findings with regard to the development, internal validation, and external validation of such a tool designed to estimate the likelihood of IFI using information available shortly after injury. We believe our strategy has robust applicability across those disciplines requiring complex decision-making such as trauma, critical care, and transplant.
METHODS
Following Institutional Review Board approval, we queried two separate databases containing deployment-related traumatic injury records managed by the Infection Disease Clinical Research Program – Trauma Infection Disease Outcomes Study (TIDOS) data, collected during Operations Enduring Freedom and Iraqi Freedom.
We selected 77 records containing a definite or probable diagnosis of IFI (cases) from June 2009 to August 2011 and 150 non-IFI control subjects (controls) from within the same time period using criteria previously described.6,10 These 227 records served as the training set. Table I shows the complete list of 65 variables (or features) contained in each record. The injury data in this table include data acquired in theater, at the Landstuhl Regional Medical Center (MEDCEN), and military hospitals in the United States after evacuation. A limitation of this dataset is that there is no variable that indicates specifically when IFI was diagnosed. Using these data, we created models that could be used to guide treatment in two settings: at point of injury (THEATER model) and at the first military hospital following medevac from Afghanistan to MEDCEN. Data that would have been available to physicians during the initial debridement(s) in theater were used to train the THEATER models, and data available to physicians at MEDCEN were used to train the MEDCEN models. We also experimented with adding a two-rule model – derived from the original 66-variable feature set – to the implementation of these models.
Table I.
Variable Name | Drop/Keep in Theater Model | Drop/Keep MEDCEN Model |
---|---|---|
Age at injury | Keep | Keep |
Dismounted blast | Keep | Keep |
Blast | Drop | Drop |
Branch of service | Keep | Keep |
Cases | Keep | Keep |
Dismounted status | Drop | Drop |
Gender | Drop | Drop |
Genitourinary injury | Keep | Keep |
IFI class | Drop | Drop |
Initial treatment facility | Keep | Keep |
Initial treatment facility unknown | Drop | Drop |
Injury date | Drop | Drop |
Theater colostomy | Keep | Keep |
MEDCEN ALT at admission | Drop | Drop |
MEDCEN AST at admission | Drop | Drop |
MEDCEN BUN at admission | Drop | Drop |
MEDCEN creatinine at admission | Drop | Drop |
MEDCEN temperature at admission | Drop | Drop |
MEDCEN white blood cell count at admission | Drop | Keep |
MEDCEN admission date | Drop | Drop |
MEDCEN ASPER | Drop | Keep |
MEDCEN base deficit | Drop | Drop |
MEDCEN ISS score | Keep | Keep |
MEDCEN mold presence | Drop | Keep |
MEDCEN MUCOR | Drop | Keep |
MEDCEN other mold presence | Drop | Keep |
MEDCEN PH | Drop | Drop |
MEDCEN pulse | Drop | Drop |
MEDCEN SBP | Drop | Drop |
MEDCEN sepsis indicator | Drop | Keep |
MEDCEN shock index | Drop | Keep |
MEDCEN SIRS indicator | Drop | Keep |
MEDCEN SOFA BILIRUBIN | Create SOFA score | Create SOFA score |
MEDCEN SOFA cardiovascular | Create SOFA score | Create SOFA score |
MEDCEN SOFA coagulation | Create SOFA score | Create SOFA score |
MEDCEN SOFA neurological | Create SOFA score | Create SOFA score |
MEDCEN SOFA renal | Create SOFA score | Create SOFA score |
Max MEDCEN temperature | Drop | Keep |
Max MEDCEN white blood cell count | Drop | Keep |
Military operation | Drop | Drop |
No injury | Drop | Drop |
Number of theater facilities | Drop | Drop |
Pelvis injury | Keep | Keep |
Penetrating abdomen injury | Keep | Keep |
Rectum injury | Keep | Keep |
SOFA SCORE | Drop | Keep |
Theater amputation lower left extremity | Drop | Drop |
Theater amputation upper left extremity | Drop | Drop |
Theater amputation lower right extremity | Drop | Drop |
Theater amputation upper right extremity | Drop | Drop |
Theater base deficit | Keep | Keep |
Theater colostomy | Keep | Keep |
Theater PH | Drop | Drop |
Theater plasma | Drop | Drop |
Theater PRBCs | Combine W/THEATER whole blood | Combine W/THEATER whole blood |
Theater pulse | Drop | Drop |
Theater SBP | Drop | Drop |
Theater shock index | Keep | Keep |
Theater total blood | Drop | Drop |
Theater whole blood | Combine W/THEATER PRBC | Combine W/THEATER PRBC |
Theater whole blood and PRBCs | Keep | Keep |
TIDOS injury cause | Drop | Drop |
Transfemoral amputation | Keep | Keep |
UNIQUE ID | Drop | Drop |
Feature Selection and Model Development
Of the 227 training records, 77 (34%) developed IFI. During the feature selection process, we excluded those that served as proxies for other features, due to the fact that they would confound the Bayesian model by including variables known to be highly correlated. We further excluded variables where data were missing in greater than 25% of records. For the THEATER model, we also excluded data that would not be acquired until admission to hospitals in the USA. This left 23 candidate features for inclusion in the THEATER model and 35 candidate features for inclusion in the MEDCEN model (see Table I for variable inclusion lists). This variable set included six Sequential Organ Failure Assessment (SOFA) scores. These scores are used to evaluate a patient during an intensive care unit stay. We summed all individual SOFA11 scores to create one overall variable for each patient, and also summed the units of whole blood and packed red blood cells (PRBCs) into a single variable (“blood requirement during initial resuscitation”). The SOFA scores were combined because they are correlated and make other relationships in the data look weaker by comparison. Table II presents a list of patient demographics and all candidate features.
Table II.
Statistic | N | Mean | Std. Dev. | Min | Max |
---|---|---|---|---|---|
Age | 227 | 24.623 | 4.883 | 19.200 | 47.200 |
Transfemoral amputation (binary, 0 = no, 1 = yes) | 227 | 0.463 | 0.500 | 0 | 1 |
Marine status (binary, 0 = not a marine, 1 = a marine) | 224 | 0.585 | 0.494 | 0 | 1 |
Genitourinary injury (binary, 0 = no, 1 = yes) | 227 | 0.485 | 0.501 | 0 | 1 |
Theater colostomy (binary, 0 = no, 1 = yes) | 225 | 0.169 | 0.375 | 0 | 1 |
White blood cell count (cells/ml3) | 224 | 8.396 | 2.753 | 1.800 | 19.700 |
ISS | 227 | 21.233 | 8.346 | 4 | 50 |
MEDCEN shock index | 226 | 0.811 | 0.190 | 0.061 | 1.558 |
Pelvis injury (binary, 0 = no, 1 = yes) | 227 | 0.308 | 0.463 | 0 | 1 |
Penetrating abdomen injury (binary, 0 = no, 1 = yes) | 227 | 0.141 | 0.349 | 0 | 1 |
Rectal injury (binary, 0 = no, 1 = yes) | 227 | 0.115 | 0.319 | 0 | 1 |
SOFA score | 226 | 6.049 | 3.975 | 0 | 19 |
Theater base deficit | 186 | 6.430 | 5.652 | 0 | 27 |
Theater whole blood and PRBC (Units) | 227 | 18.432 | 18.336 | 0 | 126 |
Theater shock index | 197 | 1.068 | 0.469 | 0.418 | 3.120 |
ISS = Injury severity score.
We used an iterative modeling process to build Bayesian belief networks (BBNs) using FasterAnalytics v7.0. BBNs are directed, acyclic probabilistic models that capture joint probability distributions (JPDs) between variables (how and under what circumstances the value of one feature may be described in relation to other features). FasterAnalytics uses an unsupervised machine-learning algorithm to build BBNs. This is accomplished by using search heuristics that allow relevant models to be found earlier in the modeling process through a scoring method that allows fast and efficient evaluation of putative subject models. The result is a graphical model, a set of nodes and edges, where the nodes represent variables in the dataset and the edges (or lack thereof) represent JPDs.
Feature selection was conducted by identification of first- and second-degree associates using JPDs within data subsamples. Iterative random sampling of the data, 10 iterations of 90% of the observations, was utilized to identify candidate features found to be a first- or second-degree associate of the outcome IFI in any iteration. An evaluation with all training data was also used to identify associates. The subsamples of the data are assumed to be a representative sample of the larger population and used to further reduce the 23 THEATER and 35 MEDCEN features. All continuous variables were transformed into two bins (≤median and >median). We also tested four minimum descriptive lengths (MDLs) for each model using tenfold cross-validation. The MDL is an evaluation metric used to quantify model complexity and balance accuracy or overfitting of the data depending on the number and quality of JPDs; a lower MDL implies a more complex model. These model parameters were empirically shown to yield higher areas under the curves (AUCs) (>0.7) during feature selection and were selected to optimize the model AUCs.
The models for THEATER and MEDCEN were developed using all patients from the training dataset and the reduced feature set and the MDLs (0.7 and 0.8) selected from the tenfold cross-validation exercise. Records were divided into 10 unique training and test sets and the proportion of positive IFI cases were held constant between sets. Ten models were developed and then tested using the corresponding test set. For each of the cross-validation interactions, we then estimated the area under the “receiver operator characteristic” curve to determine mean model accuracy and confidence interval.
External Validation and Decision Analysis
External validation of the selected THEATER AND MEDCEN BBNs was subsequently performed using a separate subset of the TIDOS database of 350 subjects, 53 of whom (15%) developed IFI. This subset is comprised of data collected from June 2009 to December 2013.
The BBNs were also combined with a “two-rule model.” Previous experience5 has suggested IFI risk factors of (A) transfemoral amputation and (B) receive >10 units of whole blood or PRBCs; subsequently these will be referred to as (A) and (B). The potential predictive ability of the two-rule model was assessed with three different approaches: (1) two-rule model satisfied (TRMS): if both components (A) and (B) were met, the patient was assigned a probability of 1 for having IFI, and otherwise, a probability of 0 was assigned, (2) BBN + TRMS: if both components (A) and (B) were met, the patient was assigned a probability of 1 for having IFI, and otherwise, a probability was obtained from the BBN, (3) BBN + two-rule model not satisfied (BBN + TRMNS): if both components (A) and (B) were not met, the patient was assigned a probability of 0 for having IFI, and otherwise, a probability was obtained from the BBN.
For both THEATER and MEDCEN, a bootstrap procedure was used to compare the performance of all four modeling approaches ((1) BBN (2) TRMS (3) BBN + TRMS (4) BBN + TRMNS). Since the external validation dataset consisted of 350 patients with 51 cases of IFI, bootstrap samples were generated by randomly selecting 102 patients (51 cases and 51 controls). For each bootstrap sample, the following performance metrics were evaluated: AUC, sensitivity/specificity (at probability threshold where product of sensitivity and specificity is maximized), sensitivity/specificity (at a probability threshold where sensitivity is maximized while specificity is closest to 0.5, but not less). One hundred and fifty bootstrap iterations were performed because it was determined to be sufficient number of iterations for convergence of the cumulative AUC mean – additional iterations did not change the AUC by greater than 0.01. Confidence intervals for performance metrics were obtained by calculating the 2.5% and 97.5% quantiles.
RESULTS
The THEATER BBN model (Fig. 1A) demonstrates two first-degree associates of IFI cases (probability of IFI), the number of units of blood (whole blood or PRBCs) used during resuscitation (≤10, >10), and whether the patient required a diverting colostomy. First-degree associates are variables most closely related to our outcome variable and are defined graphically as variables that are connected by one arc to our outcome. Similar to the THEATER BBN model, the MEDCEN BBN model (Fig. 1B) shows that the first-degree associates of IFI cases are the number of units of blood used during resuscitation, and whether the patient required a diverting colostomy. While first-degree associates are the most closely related to our outcome, the rest of the model becomes very important when data are missing from the first-degree associates. For example, in our test dataset, we did not know whether a subject had in theater colostomy for 292 out of 350 subjects. In these cases, the rest of the variables in the model predict the outcome.
Model evaluation with the external validation dataset indicates that the highest AUCs (THEATER mean: 0.77, MEDCEN mean: 0.76) are displayed by the model BBN + TRMNS (Tables III and IV). Kruskal–Wallis and Bonferroni-corrected Wilcoxon-rank sum tests were used to determine that the differences in AUC are significant.
Table III.
Models | AUC | Threshold | Sensitivity | Specificity |
---|---|---|---|---|
THEATER BBN | 0.68 [0.58,0.78] | 0.53 [0.49,0.58] | 0.66 [0.43,0.88] | 0.66 [0.45,0.88] |
THEATER TRMS | 0.72 [0.63,0.81] | 1 | 0.74 [0.64,0.85] | 0.69 [0.55,0.82] |
THEATER BBN + TRMS | 0.71 [0.63,0.80] | 0.89 [0.58,1.0] | 0.76 [0.67,0.86] | 0.69 [0.55,0.82] |
THEATER BBN + TRMNS | 0.77 [0.69,0.84] | 0.42 [0.12,0.49] | 0.70 [0.58,0.82] | 0.83 [0.70,0.94] |
MEDCEN BBN | 0.67 [0.57,0.78] | 0.51 [0.48,0.53] | 0.64 [0.47,0.85] | 0.69 [0.49,0.86] |
MEDCEN TRMS | 0.71 [0.63,0.81] | 1 | 0.74 [0.63,0.87] | 0.68 [0.53,0.80] |
MEDCEN BBN + TRMS | 0.70 [0.61,0.80] | 0.90 [0.53,1] | 0.76 [0.65,0.87] | 0.67 [0.53,0.807] |
MEDCEN BBN + TRMNS | 0.76 [0.68,0.85] | 0.42 [0.11,0.50] | 0.70 [0.58,0.82] | 0.82 [0.70,0.92] |
BBN = Bayesian Belief Network (produces a probability for having IFI); TRMS = Two-rule Model Satisfied (assigns 1 if satisfied, and 0 otherwise); BBN + TRMS = Bayesian Belief Network plus Two-rule Model Satisfied (assigns 1 if TRM is satisfied, otherwise BBN produces a probability for having IFI); BBN + TRMNS = Bayesian Belief Network plus Two-rule Model Not Satisfied (assigns 0 if TRM is not satisfied, otherwise BBN produces a probability for having IFI).
Table IV.
Models | AUC | Threshold | Sensitivity | Specificity |
---|---|---|---|---|
THEATER BBN | 0.68 [0.58,0.78] | 0.51 [0.29,0.58] | 0.68 [0.41,0.88] | 0.63 [0.51,0.86] |
THEATER TRMS | 0.72 [0.63,0.81] | 1 | 0.74 [0.64,0.85] | 0.69 [0.55,0.82] |
THEATER BBN + TRMS | 0.71 [0.63,0.80] | 0.60 [0.49,0.74] | 0.80 [0.69,0.90] | 0.59 [0.51,0.69] |
THEATER BBN + TRMNS | 0.77 [0.69,0.84] | 0.12 [0.12,0,12] | 0.74 [0.64,0.85] | 0.69 [0.55,0.82] |
MEDCEN BBN | 0.67 [0.57,0.78] | 0.48 [0.25,0.50] | 0.71 [0.54,0.87] | 0.57 [0.51,0.69] |
MEDCEN TRMS | 0.71 [0.63,0.81] | 1 | 0.74 [0.63,0.87] | 0.68 [0.53,0.81] |
MEDCEN BBN + TRMS | 0.70 [0.61,0.80] | 0.56 [0.49,0.75] | 0.80 [0.69,0.90] | 0.55 [0.51,0.65] |
MEDCEN BBN + TRMNS | 0.76 [0.68,0.85] | 0.12 [0.12,0.12] | 0.74 [0.63,0.87] | 0.68 [0.53,0.81] |
BBN = Bayesian Belief Network (produces a probability for having IFI); TRMS = Two-rule Model Satisfied (assigns 1 if satisfied, and 0 otherwise); BBN + TRMS = Bayesian Belief Network plus Two-rule Model Satisfied (assigns 1 if TRM is satisfied, otherwise BBN produces a probability for having IFI); BBN + TRMNS = Bayesian Belief Network plus Two-rule Model Not Satisfied (assigns 0 if TRM is not satisfied, otherwise BBN produces a probability for having IFI).
Regarding sensitivity and specificity, Table III displays performance metrics when the probability threshold for IFI classification is chosen to maximize the product of sensitivity and specificity. For a probability threshold of 0.42, both THEATER and MEDCEN models have a mean sensitivity of 0.72 and mean specificity of 0.8. Table IV provides performance metrics after choosing a probability threshold where sensitivity is maximized while specificity is closest to 0.5 but not less. For a probability threshold of 0.42, both THEATER and MEDCEN models have a mean sensitivity of 0.74 and mean specificity of 0.68.
DISCUSSION
Trauma-related IFIs are recognized for their devastating impacts on patients in both military1,2,6,10,13–16 and civilian populations.3,4,17–21 In addition to substantial morbidity resulting from recurrent wound necrosis, the disease is also associated with high mortality.1,5,10 Within the civilian literature, mortality ranges from 10% with localized cutaneous infections to 96% mortality with disseminated infections.4 Following the Joplin, Missouri tornado, 13 patients were diagnosed with trauma-related IFIs, of which five died (38% crude mortality rate).3 Among the 77 IFI patients in the military cohort, there were six deaths (8%); however, many of the deaths could not be directly attributable to the IFI due to their complex, severe multisystem injuries.1,10 As part of the effort to improve clinical outcomes within future similar populations, we developed a pair of CDS tools to aid in the prediction of IFI in combat-wounded personnel. Overall, our models demonstrated good performance on internal and external validation. The THEATER model may permit point-of-care CDS, allowing early risk stratification, so patients deemed high risk for IFI can be identified and treated with systemic intravenous and/or local antifungal therapies to mitigate downstream morbidity and mortality, in some cases accelerating formal treatment by more than a week. Importantly, this approach can be scaled for other emerging diseases using existing data sources provided those sources are accurate.
Following development and validation, the models were deployed on the Surgical Critical Care Initiative (SC2i) website and incorporated into the U.S. Army Institute of Surgical Research Clinical Practice Guidelines22 to accelerate distribution within the Military Health System (MHS). Presently, the IFI tool is only available to military providers. The decision to target the release was in response to the incidence of IFI being far greater for combat-related injuries, and the desire by SC2i to have the model tested and validated by MHS “early adopters and innovators” before attempting to test or adapt the tool for use in civilian treatment facilities. These models have not been validated in a civilian patient population and many of the model variables identified are unlikely to be present following civilian injuries.23 To ensure the tool continues to perform optimally throughout its life cycle,24 we are further using the “crowd sourcing” approach to solicit online feedback from military users via an embedded survey on the output page. A face-validity test is currently being performed on the four qualitative and quantitative questions asked of the user: who they are, where the model was used, which platform was utilized, and how they rated the performance of the tool. Ultimately, our intent is to propose standards and/or best practices for the benefit of the medical community.25
As it relates to the deployment and use of the IFI tool in the civilian health care system, the U.S. Food and Drug Administration has provided general guidance, which ultimately delegates the onus of responsibility onto treatment facilities to make their own determination regarding the proper vetting and use of this technology.26 This determination stems from the fact the IFI models are characterized as knowledge-based CDS tools, which rely on clinical and physiologic inputs, and use an inference model to provide the user with an estimate of the likelihood of IFI. As they are relied upon to assist in medical decision-making, these models will be considered a software as a medical device (SaMD), category II, belonging between the National Surgical Quality Improvement Program27 or Acute Physiology and Chronic Health Evaluation II,28 which are both SaMD category I, or the Breast Cancer Risk Assessment Tool Gail models (SaMD III).29 Where the IFI tool differs is in its use of machine-learning techniques to generate the likelihood of a patient developing an IFI, and that it can thus function in the presence of missing or incomplete input data. As such, it can guide clinical intervention, rather than merely predict (generally poor) clinical outcomes, and be vetted not only by measures of accuracy but also by decision analysis to ensure the tool is and remains suitable for clinical use.
CDS tools based on probabilistic theory have proven useful in a variety of clinical settings. For example, Bayesian network models were developed for use in patients with operable skeletal metastases to estimate the probability of survival up to 12 months postsurgery.12,30 Bayesian models have also been used to predict mortality among patients with end-stage heart failure to determine who would benefit from left ventricular assist device therapy. The latter model was employed in the development of web-enabled Cardiac Health Risk Stratification System, which provides patient-specific prediction of mortality at five different time points following device implantation.31 A Bayesian decision support system has also been used to aid in diagnosis of ventilator-associated pneumonia and predict the likelihood of survival and recurrence in relation to high-risk node-negative colon cancer.32,33
Models based on probability have a variety of advantages in the clinical setting. First, they afford the opportunity to account for uncertainty within datasets, such as the presence of missing or incomplete data. The BBN produces a graphical representation of the probabilistic relationship between the factors, allowing for greater understanding of how and under what conditions the features relate to one another. Lastly, the technique lends itself well to interval improvements as new data, evidence, or treatments become available.
The present study possesses limitations. The models were constructed and tested based upon two retrospectively collected datasets. The two datasets differ in IFI incidence: ~34% (77/227) (internal validation data) vs. ~15% (51/350) (external validation data). The difference in incidence could be related to changes in military injury patterns and geography of more intense operations during these two periods. While data verification has been performed on a large portion of this information, no retrospective database contains perfect and complete data. Furthermore, these CDS models were developed based on data gathered from combat-injured military personnel for use in similar circumstances and patient populations. The applicability to other populations (e.g., civilian trauma patients) is unknown and, for these reasons, clinical use in these other populations is actively discouraged at the present time (civilian access to the functioning model is actively restricted). We know that as sample size increases, the gap between reality and the represented data closes. Our validation set, which has a higher number of records than our training set, has a much lower rate of IFI. Therefore, increasing our sample size may help us more accurately represent the full population.
CONCLUSIONS
We developed two robust, clinically useful models for risk stratification of IFI following combat-related injuries, and these CDS tools may expedite treatment and improve outcomes for severely injured patients. Clinicians may vary the risk threshold depending on the need to maximize sensitivity, specificity, or both together. This highlights both the need for utilization of such CDS tools only under appropriate clinical circumstances and potential limitations. These tools have already been deployed within the MHS22 to reduce undesirable variation in care in the combat setting and improve outcomes as a result.34 The optimal threshold is currently being determined within an ongoing MHS deployment. Future research is necessary to confirm that early systemic and/or local interventions can prevent IFI infections or improve outcomes in wounded warriors at high risk for IFI. This approach may be utilized for emerging diseases using existing data sources and, as such, this approach provides a flexible method in which to respond to unmet needs.
ACKNOWLEDGMENTS
The authors wish to acknowledge the wonderful assistance of Leigh Carson (IDCRP; HJF), Laura Pinson (HJF), and Daniel Harrison (HJF) for manuscript preparation, figure generation, and database management, respectively.
FUNDING
Supported by the SC2i and the Infectious Disease Clinical Research Program (IDCRP), Department of Defense programs executed through the Uniformed Services University of the Health Sciences. This project was funded by the Defense Health Agency, the National Institute of Allergy and Infectious Diseases, National Institutes of Health, and the Department of the Navy under the Wounded, Ill, and Injured Program. Research activities leading to the development of this abstract were funded by the Department of Defense’s Defense Health Program – Joint Program Committee 6/Combat Casualty Care (Uniformed Services University of Health Sciences [USUHS] HT9404-13-1-0032 and USUHS HU0001-15-2-0001).
AUTHOR CONTRIBUTION
I certify that all individuals who qualify as authors have been listed; that the document represents valid work; that if we used information derived from another source, we obtained all necessary approvals to use it and made appropriate acknowledgments in the document; and that each takes public responsibility for it. Nothing in the presentation implies any Federal/DOD/DON endorsement.
REFERENCES
- 1. Tribble DR, Rodriguez CJ: Combat-related invasive fungal wound infections. Curr Fungal Infect Rep 2014; 8(4): 277–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Warkentien T, Rodriguez C, Lloyd B, et al. : Invasive mold infections following combat-related injuries. Clin Infect Dis 2012; 55(11): 1441–49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Neblett Fanfair R, Benedict K, Bos J, et al. : Necrotizing cutaneous mucormycosis after a tornado in Joplin, Missouri, in 2011. N Engl J Med 2012; 367(23): 2214–25. [DOI] [PubMed] [Google Scholar]
- 4. Roden MM, Zaoutis TE, Buchanan WL, et al. : Epidemiology and outcome of zygomycosis: a review of 929 reported cases. Clin Infect Dis 2005; 41(5): 634–53. [DOI] [PubMed] [Google Scholar]
- 5. Lewandowski LR, Weintrob AC, Tribble DR, et al. : Early complications and outcomes in combat injury related invasive fungal wound infections: a case-control analysis. J Orthop Trauma 2016; 30(3): e93–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Rodriguez C, Weintrob AC, Shah J, et al. : Risk factors associated with invasive fungal infections in combat trauma. Surg Infect (Larchmt) 2014; 15(5): 521–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Institute of Medicine : To err is human: building a safer health system. The National Academies of Sciences; Washington, DC, 1999. Available at http://www.nap.edu/catalog/9728/to-err-is-human-building-a-safer-health-system; accessed May 5, 2017.
- 8. Bates DW, Kuperman GJ, Wang S, et al. : Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 10(6): 523–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Belard A, Buchman T, Forsberg J, et al. : Precision diagnosis: a view of the clinical decision support systems (CDSS) landscape through the lens of critical care. J Clin Monit Comput 2017; 31(2): 261–71. [DOI] [PubMed] [Google Scholar]
- 10. Weintrob AC, Weisbrod AB, Dunne JR, et al. : Combat trauma-associated invasive fungal wound infections: epidemiology and clinical classification. Epidemiol Infect 2015; 143(1): 214–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Ferreira FL, Bota DP, Bross A, Melot C, Vincent JL: Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA 2001; 286(14): 1754–8. [DOI] [PubMed] [Google Scholar]
- 12. Forsberg JA, Sjoberg D, Chen QR, Vickers A, Healey JH: Treating metastatic disease: which survival model is best suited for the clinic? Clin Orthop Relat Res 2013; 471(3): 843–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Evriviades D, Jeffery S, Cubison T, Lawton G, Gill M, Mortiboy D: Shaping the military wound: issues surrounding the reconstruction of injured servicemen at the Royal Centre for Defence Medicine. Philos Trans R Soc Lond B Biol Sci 2011; 366(1562): 219–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Paolino KM, Henry JA, Hospenthal DR, Wortmann GW, Hartzell JD: Invasive fungal infections following combat-related injury. Mil Med 2012; 177(6): 681–5. [DOI] [PubMed] [Google Scholar]
- 15. Hospenthal DR, Chung KK, Lairet K, et al. : Saksenaea erythrospora infection following combat trauma. J Clin Microbiol 2011; 49(10): 3707–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Warkentien TE, Shaikh F, Weintrob AC, et al. : Impact of Mucorales and other invasive molds on clinical outcomes of polymicrobial traumatic wound infections. J Clin Microbiol 2015; 53(7): 2262–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Hajdu S, Obradovic A, Presterl E, Vecsei V: Invasive mycoses following trauma. Injury 2009; 40(5): 548–54. [DOI] [PubMed] [Google Scholar]
- 18. Vitrat-Hincky V, Lebeau B, Bozonnet E, et al. : Severe filamentous fungal infections after widespread tissue damage due to traumatic injury: six cases and review of the literature. Scand J Infect Dis 2009; 41(6–7): 491–500. [DOI] [PubMed] [Google Scholar]
- 19. Ribes JA, Vanover-Sams CL, Baker DJ: Zygomycetes in human disease. Clin Microbiol Rev 2000; 13(2): 236–301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Patino JF, Castro D, Valencia A, Morales P: Necrotizing soft tissue lesions after a volcanic cataclysm. World J Surg 1991; 15(2): 240–7. [DOI] [PubMed] [Google Scholar]
- 21. Skiada A, Rigopoulos D, Larios G, Petrikkos G, Katsambas A: Global epidemiology of cutaneous zygomycosis. Clin Dermatol 2012; 30(6): 628–32. [DOI] [PubMed] [Google Scholar]
- 22. Rodriguez CJ, Tribble DR, Murray CK, et al. : Invasive fungal infection in war wounds (CPG: 28). Joint Trauma System. U.S. Army Institute of Surgical Research. 2016. Available at http://www.usaisr.amedd.army.mil/cpgs/Invasive_Fungal_Infection_04_Aug_2016.pdf; accessed February 3, 2017.
- 23. Berwick DM: Disseminating innovations in health care. JAMA 2003; 289(15): 1969–75. [DOI] [PubMed] [Google Scholar]
- 24. Greenes RA: Why clinical decision support is hard to do. AMIA Annu Symp Proc 2006; 2006: 1169–70. [PMC free article] [PubMed] [Google Scholar]
- 25. Jenders RA, Osheroff JA, Sittig DF, Pifer EA, Teich JM: Recommendations for clinical decision support deployment: synthesis of a roundtable of medical directors of information systems. AMIA Annu Symp Proc 2007; 2007: 359–63. [PMC free article] [PubMed] [Google Scholar]
- 26. Karnik K: FDA regulation of clinical decision support software. J Law Biosci 2014; 1(2): 202–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. American College of Surgeons : ACS National Surgical Quality Improvement Program® (ACS NSQIP®). Available at https://www.facs.org/quality-programs/acs-nsqip; accessed February 3, 2017.
- 28. ClinCalc LLC : APACHE II Calculator. Acute Physiology and Chronic Health Evaluation (APACHE) II score to predict hospital mortality. Available at http://clincalc.com/icumortality/apacheii.aspx; accessed February 3, 2017.
- 29. National Cancer Institute : Breast cancer risk assessment tool. Available at https://www.cancer.gov/bcrisktool/; accessed February 3, 2017.
- 30. Forsberg JA, Wedin R, Bauer HC, et al. : External validation of the Bayesian Estimated Tools for Survival (BETS) models in patients with surgically treated skeletal metastases. BMC Cancer 2012; 12: 493. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Loghmanpour NA, Druzdzel MJ, Antaki JF: Cardiac Health Risk Stratification System (CHRiSS): a Bayesian-based decision support system for left ventricular assist device (LVAD) therapy. PLoS One 2014; 9: e111264. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Schurink CA, Visscher S, Lucas PJ, et al. : A Bayesian decision-support system for diagnosing ventilator-associated pneumonia. Intensive Care Med 2007; 33(8): 1379–86. [DOI] [PubMed] [Google Scholar]
- 33. Steele SR, Bilchik A, Johnson EK, et al. : Time-dependent estimates of recurrence and survival in colon cancer: clinical decision support system tool development for adjuvant therapy and oncological outcome assessment. Am Surg 2014; 80(5): 441–53. [PubMed] [Google Scholar]
- 34. Scott IA, Denaro CP, Bennett CJ, Mudge AM: Towards more effective use of decision support in clinical practice: what the guidelines for guidelines don’t tell you. Intern Med J 2004; 34(8): 492–500. [DOI] [PubMed] [Google Scholar]