Presymptomatic Prediction of Sepsis in Intensive Care Unit Patients

Abstract

Postoperative or posttraumatic sepsis remains one of the leading causes of morbidity and mortality in hospital populations, especially in populations in intensive care units (ICUs). Central to the successful control of sepsis-associated infections is the ability to rapidly diagnose and treat disease. The ability to identify sepsis patients before they show any symptoms would have major benefits for the health care of ICU patients. For this study, 92 ICU patients who had undergone procedures that increased the risk of developing sepsis were recruited upon admission. Blood samples were taken daily until either a clinical diagnosis of sepsis was made or until the patient was discharged from the ICU. In addition to standard clinical and laboratory parameter testing, the levels of expression of interleukin-1β (IL-1β), IL-6, IL-8, and IL-10, tumor necrosis factor-α, FasL, and CCL2 mRNA were also measured by real-time reverse transcriptase PCR. The results of the analysis of the data using a nonlinear technique (neural network analysis) demonstrated discernible differences prior to the onset of overt sepsis. Neural networks using cytokine and chemokine data were able to correctly predict patient outcomes in an average of 83.09% of patient cases between 4 and 1 days before clinical diagnosis with high sensitivity and selectivity (91.43% and 80.20%, respectively). The neural network also had a predictive accuracy of 94.55% when data from 22 healthy volunteers was analyzed in conjunction with the ICU patient data. Our observations from this pilot study indicate that it may be possible to predict the onset of sepsis in a mixed patient population by using a panel of just seven biomarkers.

Sepsis is defined as a systemic inflammatory response syndrome (SIRS) in response to infection which, when associated with acute organ dysfunction, may ultimately cause severe life-threatening complications (1). Eventually it leads to a complex syndrome characterized by progressive circulatory collapse, resulting in renal and respiratory failure and abnormalities in coagulation, plus profound and unresponsive hypotension. Although such a sudden and overwhelming response to infection is comparatively rare among otherwise healthy adults, there is an increased risk of sepsis in seriously ill patients in intensive care, immunocompromised individuals, burn patients, and young children. A survey of the epidemiology of sepsis in the United States has estimated the annual incidence of sepsis to be about 0.3% of the population per year (about 750,000 cases), with 30% mortality in diagnosed cases (2).

The diagnosis of sepsis relies on overt symptoms of systemic illness (temperature, blood pressure, heart rate, etc.), as well as the indication of the presence of an infectious organism through microbial culture from clinical samples (24). After the onset of sepsis, the effectiveness of intervention with antibiotics or other therapeutics rapidly diminishes. A novel method of diagnosis that could identify presymptomatic individuals could therefore reduce morbidity and mortality in these patients.

One possible diagnostic strategy for sepsis would be to monitor changes in molecules associated with the host response to pathogens. Circulating levels of procalcitonin (7, 27, 29) and C-reactive protein (CRP) (11, 30, 38) have been shown to be useful indicators of sepsis in intensive care unit (ICU) patients. In addition, changes in the individual levels of the host immune system mediators interleukin-1β (IL-1β) (3), IL-6 (13), IL-8 (20), IL-10 (36), tumor necrosis factor alpha (TNF-α) (14), FasL (16), and monocyte chemoattractant protein 1 (MCP-1) (6) in the blood of ICU patients have been highlighted as indicative of sepsis. Furthermore, a recent study has identified significant differences in the levels of a range of proinflammatory cytokines between hospital patients with Burkholderia pseudomallei-induced sepsis and healthy volunteers (42). However, these studies have focused on changes that occur following the onset of sepsis. Few have examined changes in biomarker expression in a patient population prior to the onset of sepsis (23, 37, 39). Since the nature of the host response that leads to sepsis is highly complex (8), it is likely that any presymptomatic diagnostic test will have to utilize a set of parameters rather than one specific biomarker. We therefore designed a pilot study to examine whether it was possible to predict the development of sepsis by measuring the expression of a number of biomarkers in the blood of presymptomatic individuals before the onset of clinical sepsis. This was done by comparing blood samples of ICU patients who went on to develop sepsis with samples from ICU patients that were discharged with no further complications.

MATERIALS AND METHODS

Patients.

During a yearlong period, a total of 92 patients who were determined to be at risk of developing sepsis due to their admission diagnosis were recruited into the study upon entry into the ICU at the Queen Alexandra Hospital, Portsmouth, Hampshire, United Kingdom (Fig. 1). The study was approved by the local ethics committee and prior consent to take part in the study granted either by the individual or a close relative. Twenty-five ICU patients developed sepsis, according to the criteria defined during the 1991 American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference (1). For comparison with the 25 sepsis patients, we selected an age-matched control group of 22 patients from the remaining group of patients that did not develop sepsis, the nonsepsis group. Patients that were on antibiotics prior to the development of sepsis were excluded from the study. Further reverse transcriptase PCR (RT-PCR) analysis of the samples from the remaining 45 nonsepsis patients was not performed.

FIG. 1.

Summary of the study design.

Study.

Blood samples were collected daily between 0600 h and 1200 h, using arterial lines if present, until either the day of diagnosis for sepsis patients or the day of discharge for nonsepsis patients. For nonsepsis patients, the length of stay in the ICU was a maximum of 2 days. This meant that one or two blood samples were taken for analysis. In the cases where two blood samples were taken, the second sample was used for further analysis, since patients were more acclimated to ICU conditions and the second samples were therefore a better match for those from the sepsis patients, whose stay was usually much longer. An amount of 2.5 ml fresh blood was mixed with 25 ml RNA/DNA stabilization reagent for blood/bone marrow (Roche, United Kingdom) and stored at −80°C for later measurement of cytokine mRNA expression. Analysis of samples for clinical purposes was undertaken on site on the same day as sample collection. Further analysis was undertaken retrospectively once all sepsis patients in the study had been selected and a comparable nonsepsis patient group identified. The data presented in this study are only from these two patient groups; the remaining patient samples were not processed. Sepsis patient data from 4 days prior to sepsis diagnosis until the day of diagnosis only were analyzed. Cytokine analysis was conducted away from the hospital, and the results gained were not used to influence patient management.

Real-time RT-PCR analysis of cytokine expression by blood leukocytes.

mRNA was extracted by using a commercial mRNA isolation kit for blood/bone marrow (Roche). PCR assays were designed and optimized for the histone H3 housekeeping gene described by Wells and Kedes (41), using commercial primers (Oswell Scientific, Southampton, United Kingdom) and a hydrolysis probe using a 6-carboxyfluorescein quencher and 6-carboxytetramethylrhodamine reporter dye (Applied Biosystems, Cheshire).

To measure the threshold cycle (ΔC_T) values for the cytokine mRNAs, TaqMan PCR assay kits for human FasL, MCP-1, TNF-α, IL-1β, IL-6, IL-8, and IL-10 were used (Applied Biosystems, United Kingdom). ΔC_T values for cytokine mRNAs were calculated by subtracting the C_T value for the cytokine from the C_T value obtained for the housekeeping gene.

Statistical and neural network analysis.

Neural networks are mathematical techniques for discovering nonlinear relationships between a number of variables and an outcome. For this study, the variables describe the patient data and the outcome is the imminent onset of sepsis. Finding a relationship between the two would allow us to predict sepsis from test results.

The analysis of patient data used multilayer perceptron architecture with a single hidden layer (35). The parameter setting used the back-propagation algorithm (Fig. 2). The standard practice when building neural-network-based classifiers is to use the majority of the available data to set the parameters of the model and to use the remainder to test the performance of the model on data that has not had an effect on the model's parameters. In this way, it is possible to test the model's ability to correctly classify new data. We chose to split the data 70% for model building and 30% for testing. The results reported in this paper refer to the correct classification rate for the test data.

FIG. 2.

Structure of a typical back-propagated multilayered perceptron used in this study. The predictive efficacy of a group of input parameters, such as expression levels of IL-1β, IL-6, IL-8, IL-10, TNF-α, CCL2, and FasL, was tested through the construction of a simple two-output neural network. Input parameters from a random 70% of the patient data were used to train the model to indicate what cytokine levels looked like in those patients that went on to develop sepsis and in those that did not. Each input parameter was assigned an arbitrary weighting during the training phase of model construction, and this was adjusted by an algorithm hidden layer that gave a prediction as to whether the group of input parameters had come from a patient that went on to develop sepsis (S) or had come from a patient that did not develop sepsis (C). The model then compared this prediction with the actual patient classification. It then adjusted the weightings and bias used to generate the initial classification and repeated the classification process (back propagation). This process was repeated until there was no significant improvement in classification output. Once the network was optimally trained, its predictive accuracy was tested using the remaining, unseen 30% of patient data.

With small data sets such as ours, there is still a risk that the portion of data set aside for testing will produce a good classification rate by chance. To mitigate this risk, we built five neural network classifiers, each with a different random test set of 30% of the data. The agreement of their classification scores by the mean and variance around that mean for each set of parameters analyzed (e.g., cytokines or clinical parameters) is indicated in Results. Clinicians with no experience in using neural networks should seek assistance from an expert in the field, since the analysis described is not intended as a resource for learning about neural networks.

The Andersen-Darling normality test and the F test were used to assess the distribution of ages and the acute physiology and chronic health evaluation II (APACHE II) scores within each patient group. No significant variation was observed, so a two-sample t test was used to compare the age and APACHE II scores of the patient groups. A Chi-squared test was performed to ascertain whether the neural-network-derived predictive accuracies were statistically different from chance (1:1 odds of correct prediction). The data were organized as numbers of correct and incorrect predictions (for both sepsis and nonsepsis patient data). In addition, the test was used to examine differences between sets of input parameters (e.g., the predictive accuracies of a network trained on cytokine data versus the predictive accuracies of a network trained on clinical data). Since six outcomes were possible using the test, a Bonferroni correction was applied so that the α rejection rate was a P value of 0.008. All tests were undertaken by using Minitab software (version 14.13).

RESULTS

Patients.

The criteria for sepsis definition were the manifestation of SIRS (two out of the four following parameters: white cell count above 20 [× 10⁹/liter] or below 3 [× 10⁹/liter], temperature above 38°C or below 36°C, heart rate above 100 beats per minute, and ventilation rate above 20 per min, as defined by Levy et al. [24]) in combination with positive culture from a clinical sample. All sepsis patients were selected on these criteria. Six of the 22 ICU control patients were undergoing an SIRS response at the time of sampling. No microbiological organisms were isolated from clinical samples from these control patients. There was no significant difference in the mean ages of the sepsis and nonsepsis groups, which were 60.16 (range 20 to 83) and 60.18 (range 24 to 82) (P = 0.996), respectively. In the sepsis patient group, the male-to-female ratio was 16/9, while in the nonsepsis group, the ratio was 17/5. There was a statistically significant difference in the mean APACHE II scores of the sepsis and nonsepsis patients, which were 18.88 (range 8 to 37) and 14.04 (range 8 to 33), respectively, on the day of admission into the ICU (P = 0.014). A summary of patient admission diagnoses and relevant microbiology results is given in Table 1.

TABLE 1.

Summary of patient admission diagnoses and relevant microbiology results in a nonsepsis ICU population and a sepsis ICU population^a

Group and age (yrs)	Sex	Diagnosis on admission	APACHE II score	Microbiology result(s) (site)
Nonsepsis ICU patients
59	M	Esophagectomy	19
72	M	Esophagogastrectomy	13
66	M	Elective AAA repair	12
58	M	AAA	15
69	M	Ruptured AAA	15
82	M	AAA	33
60	M	Post-Whipple's operation	19
67	M	Tumor embolism	9
76	M	Infrarenal AAA	22
54	M	Epilepsy	23
66	M	Postoperative hemorrhage	15
74	M	AAA	17
67	M	Overdose	18
42	M	Aorta bifemoral graft	8
77	M	Postcystectomy	12
40	M	Whipple's operation	8
48	M	Post-cardiac surgery gastrointestinal bleed	Not recorded
54	F	Overdose	7
42	F	Overdose	8
35	F	Hematoma on thyroid	9
54	F	Asthma	12
62	F	Respiratory arrest	7
Sepsis ICU patients
53	M	Brittle asthma	16	Pseudomonas spp. (NBL)
54	M	Overdose	20	S. pneumoniae (sputum) and Neisseria spp. (blood)
72	M	AAA repair	18	S. aureus (sputum)
50	M	Status epilepticus	8	C. difficile and MRSA (stool and sputum)
66	M	Emergency AAA	21	Klebsiella spp. (NBL)
21	M	Cardiogenic shock due to amphetamines, pulmonary edema, myocardial infection	12	H. influenzae and E. coli (NBL), Enterococcus spp. (CVP tip)
74	M	Ischemic bowel	21	Klebsiella spp. (NBL)
75	M	Left hemicolectomy	24	Pseudomonas spp. (NBL and wound)
74	M	AAA right hemicolectomy	15	S. epidermis (wound)
58	M	Cardiac arrest, cardiogenic shock	20	H. influenzae (NBL)
54	M	Head injury	12	S. pneumoniae and H. influenzae (NBL)
37	M	Injury to femur and knee, massive transfusion, and coagulopathy	10	Pseudomonas spp. and S. epidermis (CVP tip)
70	M	Ruptured AAA	37	Enterococcus (wound)
42	M	Paracetamol overdose	27	S. aureus and H. influenzae (sputum)
47	M	Trans-hiatus esophagectomy	12	S. epidermis (blood)
75	M	Perforated duodenal ulcer and fistula	26	Candida (sputum)
75	F	Elective AAA repair	16	Enterococcus spp. (blood)
67	F	Femoral popliteal bypass, failed extubation	17	S. aureus (NBL) and Enterococcus spp. (urine)
38	F	RTA chest trauma	13	H. influenzae (NBL) and Candida spp. (sputum)
83	F	Guillain-Barré syndrome	20	S. aureus (NBL and CVP tip)
65	F	Subendocardial myocardial infarction	21	MRSA (CVP tip)
44	F	Airway obstruction grade II, surgical trachea	27	S. aureus (sputum)
67	F	Respiratory failure, systemic lupus erythematosus	20	Pseudomonas spp. (ileal conduit) and Candida spp. (urine)
76	F	Myocardial infarction	24	Candida spp. (sputum)
67	F	Aorta bifemoral bypass, basal pneumonia	15	Coliforms (NBL)

^aA blank space under “Microbiology result” indicates no microbes found. M, male; F, female; AAA, abdominal aortic aneurysm; NBL, nonbronchoscopic bronchoalveolar lavage; MRSA, methicillin-resistant S. aureus; CVP tip, central venous pressure tip; RTA, road traffic accident.

Analysis of parameters for ICU patients that went on to develop sepsis.

Binary linear regression analysis of the RT-PCR and clinical data obtained during this study indicated that there was no significant difference between the levels of parameters measured in ICU patients that went on to develop sepsis and those that did not (data not shown). An alternative method of analysis using nonlinear neural networks was undertaken to assess whether any nonlinear patterns in parameter expression between patients that went on to develop sepsis (presymptomatic patients) and those that did not could be discerned. Insufficient patient data were collected on each day prior to sepsis diagnosis for a temporal model of the events that lead to the development of sepsis. To overcome this, all the data for presymptomatic patients during the period when an infection was likely to develop, days −4 to −1, were pooled in order to answer the question of whether an individual was going to develop sepsis rather than when an individual was going to develop sepsis.

Neural network analysis of cytokine mRNA expression in presymptomatic patients.

Five independent neural networks using blood leukocyte IL-1β, IL-6, IL-8, IL-10, MCP-1, TNF-α, and FasL (“cytokine” group) expression from patient samples collected between days 4 and 1 prior to sepsis diagnosis indicated that the levels of expression of proinflammatory cytokines gave a highly significant predictive accuracy of 83.09% for sepsis (P < 0.0001) (Fig. 3). The specificity and sensitivity of the five cytokine models were assessed by calculating the positive predictive value (PPV) and negative predictive value (NPV), respectively (Fig. 4). The neural network built with cytokine input parameters had a PPV of 91.67% and an NPV of 80.2%.

FIG. 3.

Average predictive accuracies for sets of cytokine, clinical, and combined cytokine and clinical parameters for predicting ICU patient outcome. Neural network analysis of each set of input parameters was repeated 5 times, and a mean predictive accuracy calculated. Each analysis used a randomized data set to train the network and then test its ability to differentiate between sepsis and nonsepsis control patients. The significance of each set of input parameters was assessed by using the chi-square test plus Bonferroni's correction (a P value of 0.008). Error bars show 95% confidence limits. *, significant result (P < 0.0001).

FIG. 4.

The specificity and sensitivity of the neural network models constructed using cytokine, clinical, or combined cytokine and clinical input parameters. The PPV, or specificity, of the combined five iterations of the model for each set of input parameters was calculated as described in Materials and Methods. The NPV, or sensitivity, of the combined five iterations of the model for each set of input parameters was calculated as described in Materials and Methods.

Neural network analysis of clinical parameters alone or in combination with cytokine expression.

Five separate networks using a set of clinical parameters that included blood creatinine and CRP levels and total peripheral blood monocyte, lymphocyte, neutrophil, and white cell counts, gave an average predictive accuracy of 69.35% that was not significant (Fig. 3). The neural networks built with clinical parameters had an average PPV of 70.91% and an average NPV of 66.67% (Fig. 4).

Five further networks constructed from both sets of cytokine and clinical parameters had an average predictive accuracy of 79.7% that was also not significant (Fig. 3). The combined parameter networks had a PPV of 70.51% and an NPV of 82.76%.

Differentiation between sepsis patients and healthy volunteers.

To further assess the predictive accuracy of the cytokine and chemokine data, the model using nonsepsis patient data and all sepsis patient cytokine data from between 4 days and 1 day prior to sepsis diagnosis was reinterrogated. The ability of the model to correctly classify a new set of blood samples based purely on the blood cytokine profile from healthy volunteers from within the laboratory was assessed. The expression levels of IL-1β, IL-6, IL-8, IL-10, MCP-1, TNF-α, and FasL in blood samples from 22 healthy male volunteers (aged between 18 and 40) were compared with expression levels in the group of presymptomatic sepsis and nonsepsis ICU patients described above. The cytokine and chemokine model had a predictive accuracy of 95.45% (95% confidence limit of 7.48) following five iterations of the network as described above.

DISCUSSION

Many immune response parameters fluctuate throughout the course of sepsis (12), so analysis using simple linear techniques cannot be used easily. We believe that the use of nonlinear tools, such as neural networks, is well suited for these types of data. Indeed, neural networks have been used to analyze ICU patient data to determine outcomes for critically ill patients (18). Neural networks have been successfully applied in several studies that have measured a number of clinical parameters and then predicted outcomes successfully following the development of sepsis (18, 25). Since we have created models based on patient data gained before the diagnosis of sepsis, our studies extend the use of neural networks from a tool for sepsis prognosis to an aid for the prediction of sepsis. The successful prediction of which patients are likely to develop sepsis should improve outcomes in the ICU, as it will allow more-focused or more-intensive monitoring and earlier therapeutic or prophylactic intervention.

The best-performing neural network models were those that used proinflammatory cytokine and chemokine mRNA expression in peripheral blood cells. This is perhaps not surprising, as these immune mediators have been implicated in the disease process associated with sepsis and high levels are associated with mortality in patients with established sepsis (40). They are also produced during the early stages of microbial infection, when overt clinical symptoms are not apparent (33). The host immune response that follows is highly complex (8, 10), and it is clear that linear analysis models have difficulty providing predictive accuracy for these complex immunological interactions in which cytokine values will fluctuate throughout the course of illness.

The ability of a set of proinflammatory markers to indicate sepsis suggests that they form part of an immune signature that is characteristic of sepsis. Previous studies have noted differences in immune parameters that can be used to characterize the host response to diseases caused by different agents. The levels of IL-1β, IL-6, and IL-18 are higher in the plasma of patients with sepsis induced by gram-positive bacteria than in patients with sepsis induced by gram-negative bacteria (19). Neisseria meningitidis induces more IL-10 but less IFN-γ production than Streptococcus pneumoniae, while Staphylococcus aureus induced minimal secretion of both cytokines (4). The results of analysis of samples from patients with staphylococcal enterotoxin B- and lipopolysaccharide-induced septic shock have identified pathogen-specific genomic markers and indicates that some can also be stage specific (26). Different transcriptional programs can be triggered upon in vitro exposure of immune cells to distinct pathogens (5, 9, 28). Recent data suggest that different classes of infectious agent elicit different host responses, as indicated by distinct gene expression patterns in blood (21, 32, 34). It is now possible to distinguish patterns of gene expression in blood leukocytes from patients with acute infections caused by four common human pathogens (influenza A virus, Staphylococcus aureus, Streptococcus pneumoniae, and Escherichia coli) (31). These analyses have utilized bioinformatics and statistical methods for class comparison or prediction algorithms to identify discriminative molecular signatures corresponding to a particular infection. The results of these studies all suggest that patterns of immune parameters exist that could be used to predict the development of sepsis in patients who have suffered trauma, have infections, or have undergone major surgery (15).

Our neural network model derived from clinical data that included the numbers of leukocytes of various leukocyte subsets and levels of creatinine and CRP, plus patient temperature, had some utility in the prediction of sepsis (although this was not significant). This is perhaps not surprising, since half of the input parameters are related to the quantification of various innate cells whose number will change according to the type of proinflammatory signal received. In addition, CRP levels have been indicated to be a good prognostic indicator (29) in pediatric sepsis patients; however, because CRP values vary from person to person, baseline values are needed to monitor early or small increases in CRP levels.

Our results suggest that we have found a pool of genes whose expression prior to the onset of sepsis gives a generic indicator of disease, in particular, sepsis, rather than a specific indicator of an infectious agent. In other words, we believe we may have created a model of commonalities of sepsis regardless of the specific etiologic agent. This is in contrast to the results of a recent study of the expression of a panel of proinflammatory cytokines and chemokines which have been used to indicate differences between patients with B. pseudomallei-induced sepsis and noninfected control volunteers (42).

While we hypothesize that the models created in our study are identifying patients who are going to develop sepsis, we may also be identifying individuals who are more likely to develop sepsis because of a genetic predisposition. There is a strong genetic influence on the outcome of sepsis, with polymorphisms in genes coding for TNF-α, TNF-β, IL-1Ra, IL-6, IL-10, heat shock protein, Toll-like receptor 2, and Toll-like receptor 4 being linked to increased susceptibility to sepsis (22). In addition, underlying disorders like cancer or major surgery may increase the likelihood of sepsis in individuals (17).

In conclusion, we have successfully created neural network models that could predict which ICU patients were going to go on to develop sepsis. While the pattern that the neural network identifies and uses to predict sepsis is not known (i.e., a generic response to infection and sepsis or a genetic predisposition for sepsis), our future plans are to validate the findings from this pilot study in a substantially larger patient population to address some of the questions raised in this study.