Differentiating infectious and noninfectious ventilator-associated complications: A new challenge

Abstract

Background

The purpose of this study was to develop an electronic search algorithm which reliably differentiates infectious and noninfectious ventilator-associated events (VAEs). This was a retrospective cohort study used to derive a predictive model. It took place at a tertiary care hospital campus.

Methods

Participants included all ventilated patients who met the Centers for Disease Control and Prevention's National Health Safety Network definitions for VAEs between January 1, 2012, and December 31, 2013. There were 164 patients who experienced 185 VAEs in the study period.

Results

The most predictive variables were fever 2 days before VAE onset, oxygenation changes, and appearance of respiratory secretions. No other variable, including laboratory tests, radiologic findings, and vital sign values, reached statistical significance. A multivariate regression model was constructed, with 68% sensitivity and 75% specificity (receiver operator characteristic area under the curve ‘ROC-AUC’, 0.83). This was modestly better than the clinical pulmonary infection score (CPIS), which had sensitivity of 50%, specificity of 59%, and ROC-AUC of 0.60.

Conclusions

Although diagnosis of VAEs remains challenging, our data indicate that clinical signs and symptoms of a VAE may be present up to 2 days before they screen positive. Sputum, fever, and oxygenation requirements all were indicative, but aggregate models failed to create a sensitive and specific model for differentiation of VAEs. The existing clinical tool, the CPIS, is also insufficiently sensitive and specific. Further research is needed to create a clinically viable tool for differentiating VAE types at the bedside.

Defining ventilator-associated pneumonia (VAP) has always been challenging. Clinical diagnosis depends on nonspecific clinical findings and subjective interpretation of radiographic data. Attempts to assess different diagnostic tools have been hampered by the lack of a practical gold standard for VAP diagnosis.^1-3 Nonetheless, VAPs are associated with longer hospital length of stay, higher costs, and prolonged mechanical ventilation,^4-6 and early recognition and treatment for VAP improve outcomes.⁷ Because of the public health implications of VAP, a clear and objective definition was needed.

A novel surveillance definition for ventilator-associated events (VAEs) was proposed in 2012 that removes radiographic data (subject to significant interobserver variability) and classifies different VAEs along a continuum, including noninfectious ventilator-associated complications (VACs), infection-related ventilator-associated complications (IVAC), and possible or probable VAP.⁸ Events identified using this definition were shown to be associated with poorer patient outcomes, including increased mortality. This definition has been adopted by National Healthcare Safety Network for surveillance.

By this definition, patients with changes in oxygenation after a period of stability are broadly categorized as having a VAC. Those who have fever or leukocytosis and receive antibiotics are considered IVACs, and microbiology results are used to further classify patients meeting IVAC criteria as possible and probable VAP. These definitions are intended for surveillance and are not intended to assist clinical recognition of VAP in real time⁸ and correlate poorly with the older definition of VAP.^9,10 The best available tool for clinical recognition of an infectious complication of mechanical ventilation is the clinical pulmonary infection score (CPIS),¹¹ a tool meant to identify which patients are unlikely to have VAP based on culture, radiographic, vital sign, and laboratory data. However, the CPIS takes up to 72 hours to calculate and has not been validated within the framework of the new surveillance definitions.

A number of markers have been identified for VAP, from traditionally identified risk factors and biomarkers incorporated into the CPIS¹¹ and National Healthcare Safety Network definitions,¹² to novel risk factors such as steroid use.¹³ An electronic syndromic surveillance properly applied should be able to calculate the likelihood that a patient has early IVAC or VAP based on variables already stored in the electronic medical record.

Past studies have demonstrated the feasibility of specific algorithms to identify VAP patients, but none have been validated with the current definitions, and all depend on data that are not available in a clinically relevant time frame. The objective of this study is to identify and validate data points in the electronic medical record that may identify IVAC patients in the intensive care unit (ICU) prospectively.

Methods

The study used a retrospective cohort design examining all VAEs identified at Mayo Clinic-Rochester between January 1, 2012, and December 1, 2013, through infection control surveillance. Surveillance is conducted in all ICUs and 1 step-down respiratory care unit. Human subjects research approval was sought and obtained from our institutional review board. Preparatory work indicated that there were approximately 200 VAEs in this time period, of which 20%-25% were IVAC, possible VAP, or probable VAP.

Data sources and abstraction

Data on subjects was collected via a combination of manual chart review and database retrieval. Several variables relevant to the CPIS (radiologic reports and medical history) were not coded in a way, which is amenable to database searching, and required a traditional chart review. Other variables, such as laboratory and vital signs, were obtained using the ICU data mart,¹⁴ a near–real time database that stores the electronic medical record data of all patients in the ICU and institutional Enterprise Data Trust.¹⁵

Once subjects were identified, medical records were reviewed to confirm the classification of VAEs using the criteria described by Klompas et al.⁸ Date of the VAE was confirmed, and abstraction was conducted for the 72 hours before and after that date.

Definitions

Medical history items were defined as the presence of a diagnosis in the patient's ICU admission history and physical examination under either the medical history or problem list to capture both active and remote diagnoses. Prior antibiotic and steroid use (in the preceding 90 days) was determined by review of the outpatient medication list at time of admission and inpatient medication administration records. Current medication use of steroids or antibiotics was defined as any administration of antibiotics or steroids within the 72-hour window surrounding VAE diagnosis. Other active infections were determined by reviewing daily progress notes in the window for diagnoses for urinary tract infections, bloodstream infections, or other unrelated infections.

Intubation factors were abstracted from the airway management note at the time of intubation; all notes require the physician to select if the intubation is emergent or nonemergent, a reason for intubation, and the mode used. If the patient came to the hospital as a transfer and this data were not available, the patient was omitted from this analysis. Respiratory variables of tidal volume, Positive End Expiratory Pressure, and pressures support settings were defined as the highest settings for each calendar day. Blood gas data were analyzed in terms of the PaO₂/FiO₂ ratio, taking the worst (lowest) value for any given calendar day.

Radiographic data were abstracted from radiology reports, counting any report of an infiltrate that had not previously been noted as a new infiltrate or effusion, and characterized as diffuse or localized by report. Sputum characteristics of consistency, thickness, and color are charted by nursing and were abstracted from the medical record.

Sputum laboratory characteristics, including white blood cells on microscopy, gram stain, and culture, were abstracted from laboratory reports.

Model derivation

Models were derived from the abstracted variables, screening each of the candidate variables for association with IVAC, possible VAP, and probable VAP. These variables represented a wide selection of any variable the investigators could find which has been reported to be associated with VAP or VAE in existing literature that could be reliably identified via chart review. This was used to create a multivariate model for predicting the likelihood of IVAC versus noninfectious VAC. CPISs were calculated for the study population and the receiver operator characteristic; sensitivity and specificity of the models were compared. All data analyses were done using JMP 9.0 (SAS institute, Cary, NC).

Results

During the study period, 185 VAEs were identified in 164 patients. Of these, 142 were VAC only, and 46 were IVAC, possible VAP, or probable VAP (IVAC+). The demographics of the 2 groups were very similar (Table 1).

Table 1. Baseline characteristics of VAE patients.

Characteristic	VAE-VAC (n = 142)	VAE-IVAC+ (n = 46)	P value
Age, y	59.1 ± 17.6	62.7 ± 15.1	.21
Sex, % male	69.7 (99)	69.5 (32)	>.99
BMI	30.4 ± 10.1	30.5 ± 7.7	.91
Medical history
Coronary artery disease	29.6 (42)	28.3 (13)	>.99
Neoplastic disease	23.3 (33)	21.7 (10)	>.99
Solid organ transplant	0.70 (1)	4.36 (2)	.14
Antibiotics in preceding 90 d	52.8 (75)	41.3 (19)	.23
Corticosteroids in preceding 90 d	19.7 (28)	26.1 (12)	.41
APACHE III score	82.0 ± 27.5	81.9 ± 25.0	.98
SOFA score day 1	8.6 ± 4.1	8.3 ±3.9	.66
Hospital length of stay, d	38.4 ± 28.5	36.6 ± 27.5	.70
ICU length of stay, d	20.0 ± 18.5	17.1 ± 14.9	.29
Invasive ventilation duration stay, d	15.4 (12.5)	12.6 (9.4)	.11
Tracheostomy present?	0.7 (1)	2.1 (1)	.43
Admitting ICU
Coronary care	1.1 (2)	0 (0)
Cardiovascular surgery	17.6 (33)	7.98 (15)
Medical ICU	17.1 (32)	0.5 (8)
Surgical ICU	21.3 (40)	8.0 (15)
Medical-surgical ICU	6.38 (12)	2.6 (5)
Neurologic ICU	6.91 (13)	1.1 (2)
Respiratory care or step-down unit	5.32 (10)	0.5 (1)

NOTE. Values are mean ± SD, % (n), or as otherwise indicated.

APACHE III, Acute Physiology and Chronic Health Evaluation Score III; BMI, body mass index; ICU, intensive care unit; IVAC+, Infectious ventilator associated complications and ventilator associated pneumonia; SOFA, sequential organ failure assessment; VAC, ventilator-associated complication; VAE, ventilator-associated event.

Univariate analysis is described in Table 2. Looking at variability in temperature, significant differences were seen in maximum temperature from 2 days before a VAE through 3 days after (all P ≤ .01). Oxygenation as determined by PaO₂/FiO₂ ratios were lower in IVAC+ at day -2 (P = .02), but did not vary significantly at any other time. The appearance of respiratory secretions tended to be more copious starting at 3 days before IVAC onset (P < .01) and continuing until 1 day after onset.

Table 2. Candidate variables for inclusion in model.

Characteristic	VAE-VAC	VAE-IVAC	P value
n available for each day
n with data 4 d before VAE	108	21
n with data 3 d before VAE	120	28
n with data 2 d before VAE	139	46
n with data 1 d before VAE	139	46
n with data day of VAE	142	46
n with data 1 d after VAE	140	45
n with data 2 d after VAE	125	44
n with data 3 d after VAE	118	43
WBC count cells × 109/L
4 d before VAE	12.9 (0.7)	13.8 (1.6)	.60
3 d before VAE	13.1 (0.7)	12.8 (1.5)	.85
2 d before VAE	14.6 (0.9)	13.2 (1.6)	.42
1 d before VAE	13.5 (0.6)	13.2 (1.1)	.81
Day of VAE	13.7 (0.7)	13.5 (1.2)	.87
1 d after VAE	13.5 (0.7)	12.9 (1.2)	.67
2 d after VAE	12.6 (0.6)	13.6 (1.1)	.56
3 d after VAE	12.6 (0.7)	13.6 (1.1)	.47
Maximum temperature °C
4 d before VAE	37.7 (0.1)	37.4 (0.2)	.14
3 d before VAE	37.6 (0.1)	37.8 (0.2)	.31
2 d before VAE	37.6 (0.1)	38.2 (0.1)	<.01
1 d before VAE	37.7 (0.1)	38.2 (0.1)	<.01
Day of VAE	37.6 (0.1)	38.1 (0.1)	.002
1 d after VAE	37.6 (0.1)	38.2 (0.1)	<.01
2 d after VAE	37.5 (0.1)	38.0 (0.1)	.01
3 d after VAE	37.5 (0.1)	38.0 (0.1)	<.01
Tracheal secretions
4 d before VAE			.73
No secretions	67.5%	65.1%
Nonpurulent secretions	21.6%	27.0%
Purulent secretions	10.8%	7.94%
3 d before VAE			<.01
No secretions	64.6%	74.4%
Nonpurulent secretions	32.3%	10.9%
Purulent secretions	3.2%	12.8%
2 d before VAE			.07
No secretions	98.0%	90.0%
Nonpurulent secretions	2.0%	6.7%
Purulent secretions	0%	3.3%
1 d before VAE			.14
No secretions	61.0%	45.5%
Nonpurulent secretions	29.1%	36.4%
Purulent secretions	9.9%	18.2%
Day of VAE			<.01
No secretions	58.2%	35.6%
Nonpurulent secretions	32.0%	40.0%
Purulent secretions	9.9%	24.4%
1 d after VAE			.15
No secretions	61.7%	46.7%
Nonpurulent secretions	25.5%	40.0%
Purulent secretions	12.8%	13.3%
2 d after VAE			.10
No secretions	63.9%	44.4%
Nonpurulent secretions	28.6%	37.8%
Purulent secretions	7.5%	17.8%
3 d after VAE			.11
No secretions	65.9%	53.5%
Nonpurulent secretions	24.8%	25.6%
Purulent secretions	9.3%	20.9%
Worst PaO2/FiO2 ratio
4 d before VAE	201 (9.7)	162 (22)	.10
3 d before VAE	206 (9.7)	186 (20.0)	.38
2 d before VAE	184 (8.1)	145 (13.5)	.02
1 d before VAE	177 (7.5)	152 (12.9)	.09
Day of VAE	207 (8.9)	170 (14.6)	.03
1 d after VAE	214 (10.0)	203 (15.9)	.54
2 d after VAE	229 (13.4)	208 (18.6)	.36
3 d after VAE	227 (12.9)	186 (17.3)	.07
Antecedent noninvasive use	48% (68)	36% (16)	.17
Culture methodology			.008
BAL	9.9% (14)	0% (0)
Bronchial washing	7.0% (10)	21.8% (10)
Tracheal aspirate	36.0% (51)	54.4% (25)
WBCs insputum			.013
None	0%	4.3%
Present	7.8%	4.3%
Few	20.4%	15.2%
Many	18.3%	41.3%

NOTE. Results are n, % or mean SD.

BAL, broncheoalveloar lavage; IVAC, infection-related ventilator-associated complication; VAC, ventilator-associated complication; VAE, ventilator-associated event; WBC, white blood cell.

Microbiology

Sputum samples were collected in 111 VAEs. Most (68.5%) were collected as tracheal aspirates, with the remainder being bronchoalveolar lavage (12.6%), bronchial washing (18.0%), and open lung biopsy (0.9%). Gram stain was rarely discriminatory, with mixed flora or no organisms seen predominating in both IVAC+ and VAC. However, IVACs were significantly more likely to report many white blood cells seen on microscopy, consistent with the finding of purulent sputum being predictive of IVAC (18.3% vs 41.3%, P < .01).

Culture data is summarized in Table 3. There were no organisms that were more predictive of IVAC than VAC, with yeast species being the most commonly identified in both groups, followed by Pseudomonas spp and other gram-negative rods. Of patients with positive cultures, 4 of the 23 (17.4%) that were IVAC+ and 8 of the 44 (18.1%) of others were polymicrobial (P =.93).

Table 3. Microbiologic isolates from VAE.

Organism	n in VAE-VAC (n = 49)	n in IVAC+ (n = 24)
Acinetobacter	1	0
Citrobacter	1	0
Klebsiella	3	0
Enterobacter	2	4
Streptococcus pneumoniae	1	1
Escherichia coli	1	2
Stenotrophomonas	7	0
Pseudomonas	7	5
Staphylococcus aureus	6	2
Serratia	1	2
Yeast	17	8
CMV	2	0

CMV, cytomegalovirus; IVAC+, infectious ventilator associated complication and ventilator associated pneumonias; VAC, ventilator-associated complication; VAE, ventilator-associated event.

CPIS model

Although CPISs were significantly higher in the IVAC+ group than the VAC group (5.7 vs 5.0, P = .02), the receiver operator characteristic area under the curve (ROC-AUC) was poor at 0.60. Using traditional cutoff value of 6, sensitivity was 50% and specificity was 59%. Cutpoint analysis determined 5.0 to be optimal; however, this value had a sensitivity of 74% and specificity of 39%.

Multiple regression models

Screening for multivariate regression found fever, antecedent antibiotic use, purulent sputum, and blood gas oxygenation (PaO₂/FiO₂) to be moderately predictive factors for inclusion in a candidate system. There were 182 cases who had sufficient data for screening with this model, which was found to have a ROC-AUC of 0.73 (Fig 1).

Fig 1.

Receiver operator characteristic curve from the model derived from variable screening. Model includes maximum temperature from days 0, -1, and -2; secretion purulence from the same time frame; and medical history of antibiotic use.

Using a more parsimonious model including only fever, time on ventilator, oxygenation, and secretion purulence, the ROC-AUC was improved to 0.83, with 120 cases having all variables needed for calculation. Using cutpoint analysis, this model is optimally 68% sensitive and 75% specific (Fig 2).

Fig 2.

Parsimonious model receiver operator characteristic curve. Variables included oxygenation at day -2, time from admission, maximum temperature on day 2, and secretion purulence on day -1. The logit model was calculated as 29.9 + Match (“Secretions day -1 (none = 0, nonpurulent =-0.69, purulent =-0.063 – 0.83 × (Maximum temperature day -2”) + 0.000003 × (Days since admission) + 0.009 ×(PaO₂/FiO₂ day-2”). The area under the curve for this model was 0.83.

Discussion

This study highlights the difficulty in reliably diagnosing VAP. Commonly used clinical variables, such as those used in the CPIS, suffer from poor specificity and suboptimal sensitivity. The best multivariate models we developed with a robust informatics infrastructure were only marginally better than the CPIS. This is not surprising given the clinical heterogeneity of VAEs because electronic syndromic surveillance systems presently work by detecting aberrancies in particular values, rather than assessing pre- and post-test probability for a patient with a given a priori suspicion of disease.¹⁶

Previous efforts to make clinical decision models have had mixed results. A model developed by Klompas et al in 2008 had significantly better performance than ours or any preceding model, with 95% sensitivity and 96% specificity; however, information from this algorithm was incorporated into the current definitions for VAC and were not designed to discriminate between infections and noninfection complications.¹⁷ A Bayesian decision support model developed by Schurink et al achieved 78% sensitivity and specificity, similar to ours, but they used older definitions for VAP.¹⁸

Our study was restricted to those who already, retrospectively, met the definition for VAE. This was done to ensure that we had a well-matched population at high risk for IVAC, but excluded most ventilated patients. However, even in a subpopulation with >25% prevalence of IVAC, vital signs and laboratory values are poor at differentiating infections from other ventilator complications. Clearly, better diagnostic tools are needed to guide clinical decision-making with VAP.

IVAC is required to have either fever or leukocytosis and initiation of an antibiotic. Because either fever or leukocytosis is commonly followed with antibiotic initiation, the fact that antecedent fever was predictive of meeting the IVAC criteria is not surprising. This sort of confirmation bias may be worrisome because suspected VAP has already been identified as a cause of antibiotic overuse.¹⁹

However, the worsening in oxygenation, seen in both the VAC and IVAC+ populations, was more pronounced in infection-related cases. Sputum characteristics were also fairly discriminatory, with purulence defined as presence of many white blood cells being one of the most predictive factors for IVAC. Gram-stain positivity, contrary to a recent meta-analysis of VAP,¹ was not predictive of IVAC. Sputum characteristics and testing may be a potential target for future diagnostic research.

A drawback to using sputum characteristics in any model is its inherent subjectivity. VAEs were adopted over traditional VAP models because of the high subjectivity and interobserver variability inherent in visual assessments of radiography and sputum characteristics. Future definitions may benefit from objective scoring criteria for purulence, quantity, and characteristics of tracheal secretions to improve interobserver variability and determine if this can be included in a final model.

This study has 3 major limitations. The first is the newness of the VAE definition. Most available data used to design this study were based on the earlier data on VAP because the VAE definition was only adopted a year ago. There is far less research on VAEs, necessitating screening a large number of variables to determine their utility in VAEs. Second, the low rate of VAEs in this institution limits the number of potential cases for use in developing the model. This is why we attempted only to derive a candidate model and examine potentially useful variables for future models, rather than develop and validate a new diagnostic decision rule. A separate, prospective validation would be necessary to assess the utility of this model.

Other limitations include the retrospective nature of the study and the variable time course of VAEs. Abstraction of pertinent covariates, such as comorbidities, outpatient medications and medical history, is dependent on charted values and may not fully capture a patient's diagnoses. Patients who develop a VAE earlier in their hospitalization are also more likely to have missing data (eg, no 3 days before VAE data points), making predictive modeling difficult.

Conclusions

Our data indicate that sputum characteristics, such as purulence and amount, fever, and oxygenation may each indicate IVAC up to 2 days prior to diagnosis, but with poor aggregated sensitivity and specificity. These each present likely candidates for future predictive model inclusion; however, more research, potentially involving novel parameters will be needed to make a clinically useful tool.

VAP continues to be a major problem in ICUs everywhere, and better tools are needed to identify this in the clinical setting. Although the novel VAE definition is more closely tied to pertinent clinical endpoints for epidemiologic purposes, identifying events in a time frame relevant for intervention remains elusive. Further research, especially targeting sputum characteristics, is needed in this area.