Combat-Related Invasive Fungal Infections: Development of a Clinically Applicable Clinical Decision Support System For Early Risk Stratification

SUMMARY:

Introduction:

Invasive fungal infections (IFI) are associated with high morbidity and mortality. A better method of risk stratifying trauma patients for combat-related IFI is needed to improve clinical outcomes while minimizing morbidity related to overtreatment. We sought to develop combat-related IFI clinical decision support (CDS) tools to assist providers make treatment decisions both near the point of injury and subsequently at definitive treatment centers.

Materials and Methods:

We utilized a training dataset containing information from 227 combat-injured military personnel to build a Bayesian Belief Network (BBN) to predict the likelihood of developing IFI using information available at the point of initial resuscitation (THEATER model) and in the tertiary care setting (MEDCEN model). After selecting BBN models, external validation used a separate test dataset of 350 wounded warriors. Furthermore, the performance of the BBN models was compared to a “two-rule model” alone (based on physician experience), and combinations of the BBN models plus the two-rule model. The two-rule model contains plausible IFI criteria, but it has not been formally evaluated, and they are not currently actual clinical guidelines.

Results:

We found receiver operating characteristic areas under the curve (AUC) of 0.70 (95% CI: [0.62, 0.77]) and 0.68 (95% CI: [0.59, 0.76] ) for the THEATER and MEDCEN BBN models, respectively, on cross-validation. External validation with the highest-AUC BBN models produced THEATER AUC of 0.68 (95% CI:[0.58, 0.78]) and MEDCEN AUC of 0.67 (95% CI: [0.57, 0.78]). With the incorporation of the two-rule model in low IFI-prevalence populations, external validation AUC increased to 0.77 (95% CI:[0.69, 0.84]) for the THEATER model and 0.76 (95% CI:[0.68, 0.85]) for the LRMC model. The two-rule model alone has an AUC of 0.72 (95% CI: [0.63, 0.81])

Conclusions:

Overall, the IFI tools produced clinically useful, robust models. However, the clinical utility of these models is highly dependent upon the clinician’s individual risk tolerance. The threshold probability for optimal clinical use of this CDS tool is currently being evaluated in an ongoing clinical utilization study. CDS tools, such as these, may facilitate early diagnosis of patients with or at risk for IFI, permitting early or prophylactic treatment with the aim of improving outcomes.

INTRODUCTION

During the recent military conflict in Afghanistan, invasive fungal wound infection (IFI) emerged as an infectious complication with surprising incidence (7%) and high morbidity and mortality (8%) among severely injured military personnel.^1,2 Devastating mortality rates as high as 38–96% have also been reported in civilian trauma and medical populations.^3,4 A primary characteristic of this disease is recurrent tissue necrosis within the wound despite serial surgical débridements, straining valuable healthcare resources while adding continuous physiologic insult to severely injured patients. Indeed, combat casualites who develop IFIs have significantly more surgical amputations and proximal amputation revisions, a greater number of operative visits, higher proportion of bacterial co-infections, and a longer duration to initial surgical wound closure post-injury compared to patients without the disease.⁵ Sustaining a blast injury on foot patrol, traumatic transfemoral amputation, and/or requiring massive (>20 units) blood product transfusions during the first 24 hours post-injury were identified to be independent IFI risk factors.⁶ Treatment recommendations center on aggressive and frequent débridements and early initiation of antifungal therapy when there is a high suspicion of IFI.¹ However, if we are to prescribe early and aggressive treatment in a reliable and consistent manner and avoid unnecessary systemic complications of overtreatment, a method to estimate the likelihood of IFI using patient- and injury-specific information is required.

In response to the 1999 Institute of Medicine report ‘To Err is Human’,⁷ the U.S. healthcare industry witnessed a steady rise in use of clinical decision support (CDS) tools, and with it, a corresponding improvement in patient outcomes.^8,9 Like other CDS tools, one developed to identify acutely traumatized patients at risk for IFI must be designed to assist providers at the point of care. The purpose of this manuscript is to present our findings with regard to the development, internal validation, and external validation of such a tool designed to estimate the likelihood of IFI using information available shortly after injury. We believe our strategy has robust applicability across those disciplines requiring complex decision-making such as trauma, critical care, and transplant.

METHODS

Following Institutional Review Board approval, we queried two separate databases containing deployment-related traumatic injury records managed by the Infection Disease Clinical Research Program – Trauma Infection Disease Outcomes Study (TIDOS) data, collected during Operations Enduring Freedom and Iraqi Freedom.

We selected 77 records containing a definite or probable diagnosis of IFI (cases) from June 2009 to August 2011 and 150 non-IFI control subjects (controls) from within the same time period using criteria previously described.^6,10 These 227 records served as the training set. Table 1 shows the complete list of 65 variables (or features) contained in each record. The injury data in this table includes data acquired in theater, at the Landstuhl Regional Medical Center (MEDCEN), and military hospitals in the United States after evacuation. A limitation of this dataset is that there is no variable that indicates specifically when IFI was diagnosed. Using these data, we created models that could be used to guide treatment in two settings: at point of injury (THEATER model), and at the first military hospital following medevac from Afghanistan to MEDCEN. Data that would have been available to physicians during the initial debridement(s) in theater were used to train the THEATER models, and data available to physicians at MEDCEN were used to train the MEDCEN models. We also experimented with adding a two-rule model–derived from the original 66-variable feature set—to the implementation of these models.

Table 1.

Full Variable List

Variable Name	DROP/KEEP IN Theater Model	DROP/KEEP MEDCEN Model
AGE AT INJURY	KEEP	KEEP
DISMOUNTED BLAST	KEEP	KEEP
BLAST	DROP	DROP
BRANCH OF SERVICE	KEEP	KEEP
CASES	KEEP	KEEP
DISMOUNTED STATUS	DROP	DROP
GENDER	DROP	DROP
GENITOURINARY INJURY	KEEP	KEEP
IFI CLASS	DROP	DROP
INITIAL TREATMENT FACILITY	KEEP	KEEP
INITIALTREATMENT FACILITY UNKNOWN	DROP	DROP
INJURY DATE	DROP	DROP
THEATER COLOSTOMY	KEEP	KEEP
MEDCEN ALT AT ADMISSION	DROP	DROP
MEDCEN AST AT ADMISSION	DROP	DROP
MEDCEN BUN AT ADMISSION	DROP	DROP
MEDCEN CREATININE AT ADMISSION	DROP	DROP
MEDCEN TEMPERATURE AT ADMISSION	DROP	DROP
MEDCEN WHITE BLOOD CELL COUNT AT ADMISSION	DROP	KEEP
MEDCEN ADMISSION DATE	DROP	DROP
MEDCEN ASPER	DROP	KEEP
MEDCEN BASE DEFICIT	DROP	DROP
MEDCEN ISS SCORE	KEEP	KEEP
MEDCEN MOLD PRESCENCE	DROP	KEEP
MEDCEN MUCOR	DROP	KEEP
MEDCEN OTHER MOLD PRESCENCE	DROP	KEEP
MEDCEN PH	DROP	DROP
MEDCEN PULSE	DROP	DROP
MEDCEN SBP	DROP	DROP
MEDCEN SEPSIS INDCATOR	DROP	KEEP
MEDCEN SHOCK INDEX	DROP	KEEP
MEDCEN SIRS INDICATOR	DROP	KEEP
MEDCEN SOFA BILIRUBIN	CREATE SOFA SCORE	CREATE SOFA SCORE
MEDCEN SOFA CARDIOVASCULAR	CREATE SOFA SCORE	CREATE SOFA SCORE
MEDCEN SOFA COAGULATION	CREATE SOFA SCORE	CREATE SOFA SCORE
MEDCEN SOFA NEUROLOGICAL	CREATE SOFA SCORE	CREATE SOFA SCORE
MEDCEN SOFA RENAL	CREATE SOFA SCORE	CREATE SOFA SCORE
MAX MEDCEN TEMPERATURE	DROP	KEEP
MAX MEDCEN WHITE BLOOD CELL COUNT	DROP	KEEP
MILITARY OPERATION	DROP	DROP
NO INJURY	DROP	DROP
NUMBER OF THEATER FACILITIES	DROP	DROP
PELVIS INJURY	KEEP	KEEP
PENETRATING ABDOMEN INJURY	KEEP	KEEP
RECTUM INJURY	KEEP	KEEP
SOFA SCORE	DROP	KEEP
THEATER AMPUTATION LOWER LEFT EXTREMITY	DROP	DROP
THEATER AMPUTATION UPPER LEFT EXTREMITY	DROP	DROP
THEATER AMPUTATION LOWER RIGHT EXTREMITY	DROP	DROP
THEATER AMPUTATION UPPER RIGHT EXTREMITY	DROP	DROP
THEATER BASE DEFICIT	KEEP	KEEP
THEATER COLOSTOMY	KEEP	KEEP
THEATER PH	DROP	DROP
THEATER PLASMA	DROP	DROP
THEATER PACKED RED BLOOD CELLS	COMBINE W/THEATER WHOLEBLOOD	COMBINE W/THEATER WHOLEBLOOD
THEATER PULSE	DROP	DROP
THEATER SBP	DROP	DROP
THEATER SHOCK INDEX	KEEP	KEEP
THEATER TOTAL BLOOD	DROP	DROP
THEATER WHOLEBLOOD	COMBINE W/THEATER PRBC	COMBINE W/THEATER PRBC
THEATER WHOLEBLOOD AND PACKED RED BLOOD CELLS	KEEP	KEEP
TIDOS INJURY CAUSE	DROP	DROP
TRANSFEMORAL AMPUTATION	KEEP	KEEP
UNIQUE ID	DROP	DROP

Feature Selection and Model Development

Of the 227 training records, 77 (34%) developed IFI. During the feature selection process, we excluded those that served as proxies for other features, due to the fact that they would confound the Bayesian model by including variables known to be highly correlated. We further excluded variables where data was missing in greater than 25% of records. For the THEATER model, we also excluded data that would not be acquired until admission to hospitals in the United States. This left 23 candidate features for inclusion in the THEATER model and 35 candidate features for inclusion in the MEDCEN model (see Table 1 for variable inclusion lists). This variable set included six Sequential Organ Failure Assessment (SOFA) scores. These scores are used to evaluate a patient during an intensive care unit stay. We summed all individual SOFA¹¹ scores to create one overall variable for each patient, and also summed the units of whole-blood and PRBCs into a single variable (“Blood Requirement during Initial Resuscitation”). The SOFA scores were combined because they are correlated and make other relationships in the data look weaker by comparison. Table 2 presents a list of patient demographics and all candidate features.

Table 2.

Demographics of Initial Patient Dataset for Model Creation of Cross-Validation

Statistic	N	Mean	St.Dev.	Min	Max
Age	227	24.623	4.883	19.200	47.200
Transfemoral Amputation (Binary, 0 = NO, 1 = YES)	227	0.463	0.500	0	1
Marine Status (Binary, 0 = NOT A MARINE, 1 = A MARINE)	224	0.585	0.494	0	1
Genitourinary Injury (Binary, 0 = NO, 1 = YES)	227	0.485	0.501	0	1
THEATER Colostomy (Binary, 0 = NO, 1 = YES)	225	0.169	0.375	0	1
White Blood Cell count (cells/ml3)	224	8.396	2.753	1.800	19.700
ISS	227	21.233	8.346	4	50
MEDCEN Shock Index	226	0.811	0.190	0.061	1.558
Pelvis Injury (Binary, 0 = NO, 1 = YES)	227	0.308	0.463	0	1
Penetrating Abdomen Injury (Binary, 0 = NO, 1 = YES)	227	0.141	0.349	0	1
Rectal Injury (Binary, 0 = NO, 1 = YES)	227	0.115	0.319	0	1
SOFA Score	226	6.049	3.975	0	19
Theater Base Deficit	186	6.430	5.652	0	27
Theater Wholeblood and PRBC (Units)	227	18.432	18.336	0	126
Theater Shock Index	197	1.068	0.469	0.418	3.120

MEDCEN = Landstuhl Regional Medical Center; ISS = Injury Severity Score, PRBC = packed red blood cells; Sequential Organ Failure Assessment = SOFA.

We used an iterative modeling process to build Bayesian Belief Networks (BBNs) using FasterAnalytics™ v7.0. BBNs are directed, acyclic probabilistic models that capture Joint Probability Distributions (JPDs) between variables (how and under what circumstances the value of one feature may be described in relation to other features). FasterAnalytics™ uses an unsupervised machine-learning algorithm to build BBNs. This is accomplished by using search heuristics that allow relevant models to be found earlier in the modeling process through a scoring method that allows fast and efficient evaluation of putative subject models. The result is a graphical model, a set of nodes and edges, where the nodes represent variables in the data set and the edges (or lack thereof) represent JPDs.

Feature selection was conducted by identification of first and second-degree associates using JPDs within data subsamples. Iterative random sampling of the data, ten iterations of 90% of the observations, was utilized to identify candidate features found to be a first or second degree associate of the outcome (IFI) in any iteration. An evaluation with all training data was also used to identify associates. The subsamples of the data are assumed to be a representative sample of the larger population and used to further reduce the 23 THEATER and 35 MEDCEN features. All continuous variables were transformed into two bins (≤median and >median). We also tested four Minimum Descriptive Lengths (MDLs) for each model using ten-fold cross validation. The MDL is an evaluation metric used to quantify model complexity and balance accuracy or over-fitting of the data depending on the number and quality of JPDs; a lower MDL implies a more complex model. These model parameters were empirically shown to yield higher AUCs (>0.7) during feature selection and were selected to optimize the model AUCs.

The models for THEATER and MEDCEN were developed using all patients from the training data set and the reduced feature set and the MDLs (0.7 and 0.8) selected from the ten-fold cross-validation exercise. Records were divided into ten unique training and test sets and the proportion of positive IFI cases were held constant between sets. Ten models were developed and then tested using the corresponding test set. For each of the cross validation interatons we then estimated the area under the Receiver Operator Characteristic curve (AUC) to determine mean model accuracy and confidence interval.

External Validation and Decision Analysis

External validation of the selected THEATER AND MEDCEN BBNs was subsequently performed using a separate subset of the TIDOS database of 350 subjects, 53 of whom (15%) developed IFI. This subset is comprised of data collected from June 2009 to December 2013.

The BBNs were also combined with a “two-rule model”. Previous experience⁵ has suggested IFI risk factors of (A) transfemoral amputation and (B) receive >10 units of whole blood or packed red blood cells (PRBCs); subsequently these will be referred to as (A) and (B). The potential predictive ability of the two-rule model was assessed with 3 different approaches: (1) two-rule model satisfied (TRMS): if both components (A) and (B) were met, the patient was assigned a probability of 1 for having IFI, and otherwise, a probability of 0 was assigned (2) BBN + TRMS: if both components (A) and (B) were met, the patient was assigned a probability of 1 for having IFI, and otherwise, a probability was obtained from the BBN (3) BBN + two-rule model not satisfied (BBN + TRMNS): if both components (A) and (B) were not met, the patient was assigned a probability of 0 for having IFI, and otherwise, a probability was obtained from the BBN.

For both THEATER and MEDCEN, a bootstrap procedure was used to compare the performance of all 4 modeling approaches ((1) BBN (2) TRMS (3) BBN + TRMS (4) BBN + TRMNS. Since the external validation dataset consisted of 350 patients with 51 cases of IFI, bootstrap samples were generated by randomly selecting 102 patients (51 cases and 51 controls). For each bootstrap sample, the following performance metrics were evaluated: AUC, Sensitivity/Specificity (at probability threshold where product of sensitivity and specificity is maximized), Sensitivity/Specificity (at a probability threshold where sensitivity is maximized while specificity is closest to 0.5, but not less). 150 bootstrap iterations were performed because it was determined to be sufficient number of iterations for convergence of the cumulative AUC mean – additional iterations did not change the AUC by greater than 0.01. Confidence intervals for performance metrics were obtained by calculating the 2.5% and 97.5% quantiles.

RESULTS

The THEATER BBN model (Figure 1a), demonstrates two first-degree associates of IFI cases (Probability of IFI), the number of units of blood (whole blood or PRBCs) used during resuscitation (≤10, > 10), and whether the patient required a diverting colostomy. First-degree associates are variables most closely related to our outcome variable and are defined graphically as variables that are connected by one arc to our outcome. Similar to the THEATER BBN model, the MEDCEN BBN model (Figure 1b) shows the first degree associates of IFI cases are the number of units of blood used during resuscitation, and whether the patient required a diverting colostomy. While first degree associates are the most closely related to our outcome, the rest of the model becomes very important when data is missing from the first degree associates. For example, in our test data set we did not know whether a subject had a in theater olostomy for 292 out of the 350 subjects. In these cases, the rest of the variables in the model predict the outcome.

Figure 1.

Graphic representation Bayesian Belief Network models; (A) the THEATER model; (B) the MEDCEN model. Outcomes, first degree associates, and second degree associates are represented by dark, medium, and light shades respectively in both panels.

Model evaluation with the external validation dataset indicates that the highest AUCs (THEATER mean: 0.77, MEDCEN mean: 0.76) are displayed by the model BBN + TRMNS (see Tables 3 and 4). Kruskal-Wallis and Bonferroni-corrected Wilcoxon-rank sum tests were used to determine that the differences in AUC are significant.

Table 3.

Model performance for external validation dataset (product of sensitivity and specificity is maximized).

Models	AUC	Threshold	Sensitivity	Specificity
THEATER BBN	0.68 [0.58,0.78]	0.53 [0.49,0.58]	0.66 [0.43,0.88]	0.66 [0.45,0.88]
THEATER TRMS	0.72 [0.63,0.81]	1	0.74 [0.64,0.85]	0.69 [0.55,0.82]
THEATER BBN + TRMS	0.71 [0.63,0.80]	0.89 [0.58,1.0]	0.76 [0.67,0.86]	0.69 [0.55,0.82]
THEATER BBN + TRMNS	0.77 [0.69,0.84]	0.42 [0.12,0.49]	0.70 [0.58,0.82]	0.83 [0.70,0.94]
MEDCEN BBN	0.67 [0.57,0.78]	0.51 [0.48,0.53]	0.64 [0.47,0.85]	0.69 [0.49,0.86]
MEDCEN TRMS	0.71 [0.63,0.81]	1	0.74 [0.63,0.87]	0.68 [0.53,0.80]
MEDCEN BBN + TRMS	0.70 [0.61,0.80]	0.90 [0.53,1]	0.76 [0.65,0.87]	0.67 [0.53,0.807]
MEDCEN BBN + TRMNS	0.76 [0.68,0.85]	0.42 [0.11,0.50]	0.70 [0.58,0.82]	0.82 [0.70,0.92]

BBN = Bayesian Belief Network (produces a probability for having IFI)

TRMS = Two-rule Model Satisified (assigns 1 if satisfied, and 0 otherwise)

BBN + TRMS = Bayesian Belief Network plus Two-rule Model Satisfied (assigns 1 if TRM is satisfied, otherwise BBN produces a probability for having IFI)

BBN + TRMNS = Bayesian Belief Network plus Two-rule Model Not Satisfied (assigns 0 if TRM is not satisfied, otherwise BBN produces a probability for having IFI)

Table 4.

Model performance for external validation dataset (sensitivity is maximized while specificity is closest to 0.5 but not less)

MODELS	AUC	Threshold	Sensitivity	Specificity
THEATER BBN	0.68 [0.58,0.78]	0.51 [0.29,0.58]	0.68 [0.41,0.88]	0.63 [0.51,0.86]
THEATER TRMS	0.72 [0.63,0.81]	1	0.74 [0.64,0.85]	0.69 [0.55,0.82]
THEATER BBN + TRMS	0.71 [0.63,0.80]	0.60 [0.49,0.74]	0.80 [0.69,0.90]	0.59 [0.51,0.69]
THEATER BBN + TRMNS	0.77 [0.69,0.84]	0.12 [0.12,0,12]	0.74 [0.64,0.85]	0.69 [0.55,0.82]
MEDCEN BBN	0.67 [0.57,0.78]	0.48 [0.25,0.50]	0.71 [0.54,0.87]	0.57 [0.51,0.69]
MEDCEN TRMS	0.71 [0.63,0.81]	1	0.74 [0.63,0.87]	0.68 [0.53,0.81]
MEDCEN BBN + TRMS	0.70 [0.61,0.80]	0.56 [0.49,0.75]	0.80 [0.69,0.90]	0.55 [0.51,0.65]
MEDCEN BBN + TRMNS	0.76 [0.68,0.85]	0.12 [0.12,0.12]	0.74 [0.63,0.87]	0.68 [0.53,0.81]

BBN = Bayesian Belief Network (produces a probability for having IFI)

TRMS = Two-rule Model Satisified (assigns 1 if satisfied, and 0 otherwise)

BBN + TRMS = Bayesian Belief Network plus Two-rule Model Satisfied (assigns 1 if TRM is satisfied, otherwise BBN produces a probability for having IFI)

BBN + TRMNS = Bayesian Belief Network plus Two-rule Model Not Satisfied (assigns 0 if TRM is not satisfied, otherwise BBN produces a probability for having IFI)

Regarding sensitivity and specificity, Table 3 displays performance metrics when the probability threshold for IFI classification is chosen to maximize the product of sensitivity and specificity. For a proability threshold of 0.42, both THEATER and MEDCEN models have a mean sensitivity of 0.72 and mean specificity of 0.8. Table 4 provides performance metrics after choosing a probability threshold where sensitivity is maximized while specificity is closest to 0.5 but not less. For a proability threshold of 0.42, both THEATER and MEDCEN models have a mean sensitivity of 0.74 and mean specificity of 0.68.

DISCUSSION

Trauma-related IFIs are recognized for their devastating impacts on patients in both military^{1,2,6,10,14–17} and civilian populations.^3,4,18–22 In addition to substantial morbidity resulting from recurrent wound necrosis, the disease is also associated with high mortality.^1,5,10 Within the civilian literature, mortality range from 10% with localized cutaneous infections to 96% mortality with disseminated infections.⁴ Following the Joplin, Missouri tornado, 13 patients were diagnosed with trauma-related IFIs, of which five died (38% crude mortality rate).³ Among the 77 IFI patients in the military cohort, there were six deaths (8%); however, many of the deaths could not be directly attributable to the IFI due to their complex, severe multi-system injuries.^1,10 As part of the effort to improve clinical outcomes within future similar populations, we developed a pair of CDS tools to aid in the prediction of IFI in combat-wounded personnel. Overall, our models demonstrated good performance on internal and external validation. The THEATER model may permit point-of-care clinical decision support, allowing early risk stratification so patients deemed high risk for IFI can be identified and treated with systemic intravenous and/or local antifungal therapies to mitigate downstream morbidity and mortality, in some cases accelerating formal treatment by more than a week. Importantly, this approach can be scaled for other emerging diseases using existing data sources provided those sources are accurate.

Following development and validation, the models were deployed on the Surgical Critical Care Initiative (SC2i) website and incorporated into the U.S. Army Institute of Surgical Research Clinical Practice Guidelines²³ to accelerate distribution within the Military Health System (MHS). Presently, the IFI tool is only available to military providers. The decision to target the release was in response to the incidence of IFI being far greater for combat-related injuries, and the desire by SC2i to have the model tested and validated by MHS ‘early adopters and innovators’ before attempting to test or adapt the tool for use in civilian treatment facilities. These models have not been validated in a civilian patient population and many of the model variables identified are unlikely to be present following civilian injuries.²⁴ To ensure the tool continues to perform optimally throughout its life cycle,²⁵ we are further using the ‘crowd sourcing’ approach to solicit online feedback from military users via an embedded survey on the output page. A face-validity test is currently being performed on the four qualitative and quantitative questions asked of the user: who they are, where the model was used, which platform was utilized, and how they rated the performance of the tool. Ultimately, our intent is to propose standards and/or best practices for the benefit of the medical community.²⁶

As it relates to the deployment and use of the IFI tool in the civilian healthcare system, the U.S. Food and Drug Administration has provided general guidance, which ultimately delegates the onus of responsibility onto treatment facilities to make their own determination regarding the proper vetting and use of this technology.²⁷ This determination stems from the fact the IFI models are characterized as knowledge-based CDS tools, which rely on clinical and physiologic inputs, and use an inference model to provide the user with an estimate of the likelihood of IFI. As they are relied upon to assist in medical decision making, these models will be considered a software as a medical device (SaMD), category II, belonging between the National Surgical Quality Improvement Program²⁸ or Acute Physiology and Chronic Health Evaluation II,²⁹ which are both SaMD category I, or the Breast Cancer Risk Assessment Tool Gail models (SaMD III).³⁰ Where the IFI tool differs is in its use of machine learning techniques to generate the likelihood of a patient developing an IFI, and that it can thus function in the presence of missing or incomplete input data. As such, it can guide clinical intervention, rather than merely predict (generally poor) clinical outcomes, and be vetted not only by measures of accuracy, but also by decision analysis to ensure the tool is and remains suitable for clinical use.

Clinical decision support tools based on probabilistic theory have proven useful in a variety of clinical settings. For example, Bayesian network models were developed for use in patients with operable skeletal metastases to estimate the probability of survival up to 12 months post-surgery.^13,31 Bayesian models have also been used to predict mortality among patients with end-stage heart failure to determine who would benefit from left ventricular assist device therapy. The latter model was employed in the development of the web-enabled Cardiac Health Risk Stratification System, which provides patient-specific prediction of mortality at five different time points following device implantation.³² A Bayesian decision-support system has also been used to aid in diagnosis of ventilator-associated pneumonia and predict the likelihood of survival and recurrence in relation to high-risk node-negative colon cancer.^33,34

Models based on probability have a variety of advantages in the clinical setting. First, they afford the opportunity to account for uncertainty within datasets, such as the presence of missing or incomplete data. The BBN produces a graphical representation of the probabilistic relationship between the factors, allowing for greater understanding of how and under what conditions the features relate to one another. Lastly, the technique lends itself well to interval improvements as new data, evidence, or treatments become available.

The present study possesses limitations. The models were constructed and tested based upon 2 retrospectively collected datasets. The 2 datasets differ in IFI incidence: ~34% (77/227) (internal validation data) vs ~15% (51/350) (external validation data). The difference in incidence could be related to changes in military injury patterns and geography of more intense operations during these two periods. While data verification has been performed on a large portion of this information, no retrospective database contains perfect and complete data. Furthermore, these CDS models were developed based on data gathered from combat-injured military personnel for use in similar circumstances and patient populations. The applicability to other populations (e.g., civilian trauma patients) is unknown and, for these reasons, clinical use in these other populations is actively discouraged at the present time (civilian access to the functioning model is actively restricted). We know that as sampe size increases, the gap between reality and the represented data closes. Our validation set, which has a higher number of records than our training set, has a much lower rate of IFI. Therefore, increasing our sample size may help us more accurately represent the full population.

CONCLUSIONS

We developed two robust, clinically useful models for risk stratification of IFI following combat-related injuries, and these CDS tools may expedite treatment and improve outcomes for severely injured patients. Clinicians may vary the risk threshold depending on the need to maximize sensitivity, specificity, or both together. This highlights both the need for utilization of such CDS tools only under appropriate clinical circumstances and potential limitations. These tools have already been deployed within the MHS²³ to reduce undesirable variation in care in the combat setting, and improve outcomes as a result.³⁵ The optimal threshold is currently being determined within an ongoing Military Health System deployment. Future research is necessary to confirm that early systemic and/or local interventions can prevent IFI infections or improve outcomes in wounded warriors at high risk for IFI. This approach may be utilized for emerging diseases using existing data sources and, as such, this approach provides a flexible method in which to respond to unmet needs.