Skip to main content
NCBI home page
PLOS One logoLink to PLOS One
. 2021 Jan 14;16(1):e0245552. doi: 10.1371/journal.pone.0245552

Usefulness of Sepsis-3 in diagnosing and predicting mortality of ventilator-associated lower respiratory tract infections

Alexandre Gaudet 1,2, Matthieu Devos 1, Sylvain Keignart 1, Olivier Pouly 1, Sylvain Lecailtel 1, Frédéric Wallet 3, Saad Nseir 1,4,*
Editor: Yutaka Kondo5
PMCID: PMC7808583  PMID: 33444395

Abstract

Background

Early distinguishing ventilator-associated tracheobronchitis (VAT) and ventilator-associated pneumonia (VAP) remains difficult in the daily practice. However, this question appears clinically relevant, as treatments of VAT and VAP currently differ. In this study, we assessed the accuracy of sepsis criteria according to the Sepsis-3 definition in the early distinction between VAT and VAP.

Methods

Retrospective single-center cohort, including all consecutive patients with a diagnosis of VAT (n = 70) or VAP (n = 136), during a 2-year period. Accuracy of sepsis criteria according to Sepsis-3, total SOFA and respiratory SOFA, calculated at time of microbiological sampling were assessed in differentiating VAT from VAP, and in predicting mortality on ICU discharge.

Results

Sensitivity and specificity of sepsis criteria were found respectively at 0.4 and 0.91 to distinguish VAT from VAP, and at 0.38 and 0.75 for the prediction of mortality in VA-LRTI. A total SOFA ≥ 6 and a respiratory SOFA ≥ 3 were identified as the best cut-offs for these criteria in differentiating VAT from VAP, with sensitivity and specificity respectively found at 0.63 and 0.69 for total SOFA, and at 0.49 and 0.7 for respiratory SOFA. Additionally, for prediction of mortality, a total SOFA ≥ 7 and a respiratory SOFA = 4 were identified as the best-cut-offs, respectively yielding sensitivity and specificity at 0.56 and 0.61 for total SOFA, and at 0.22 and 0.95 for respiratory SOFA.

Conclusions

Sepsis criteria according to the Sepsis-3 definition show a high specificity but a low sensitivity for the diagnosis of VAP. Our results do not support the use of these criteria for the early diagnosis of VAP in patients with VA-LRTI.

Background

Despite decades of research, ventilator-associated lower respiratory tract infections (VA-LRTI), including ventilator-associated tracheobronchitis (VAT) and ventilator-associated pneumonia (VAP), remain frequent complications of mechanical ventilation [1]. The distinction between these two conditions currently relies on the interpretation of chest X-ray, and is based on the presence of a new pulmonary infiltrate in VAP, conversely to VAT [2].

The diagnosis of VA-LRTI is associated with poor outcomes, including longer duration of mechanical ventilation and increased length of stay in the intensive care unit (ICU) [1]. Current IDSA/ATS guidelines recommend the early initiation of antimicrobial therapy in patients with VAP, but not in those with VAT [3]. Therefore, making the early distinction between VAT and VAP appears as a clinically relevant question in mechanically ventilated patients with suspected VA-LRTI.

However, numerous studies have highlighted the difficulties of interpretation of chest radiographies in differentiating VAT from VAP, mainly due to other potential etiologies of lung opacities [4, 5] and to a frequent delay in the appearance of pulmonary infiltrates [6]. Several studies suggest that a diagnosis of VAP would be associated with a greater severity of illness, thus explaining higher mortality rates than in VAT [1, 7, 8]. Accordingly, in the TAVeM study, Martin-Loeches et al. reported higher values of the SOFA score on the day of diagnosis of VA-LRTI in patients with VAP, compared to those with VAT.

In 2016, the Sepsis-3 Definition Task Force developed and released new criteria for the diagnosis of sepsis, based on the assessment of the severity of illness. Based on these criteria, sepsis is defined as a change in total SOFA score ≥2 points consequent to the infection over a 48 hours period [9]. Thus, patients with a diagnosis of VAP may be more likely to experience a greater severity of illness, notably characterized by a worsening in organ failures, therefore leading to a higher frequency of sepsis. Accordingly, we aimed to evaluate the accuracy of sepsis criteria according to the Sepsis-3 definition in differentiating VAT from VAP.

Methods

Study design and patients

This study was conducted in a 50-bed mixed ICU (Department of Intensive Care Medicine, Critical Care Centre, CHU of Lille), during a 2-year period (from January 1st, 2016 to December 31, 2017). Continuous surveillance of ICU-acquired infections allowed prospective identification of patients with VA-LRTI. These patients were subsequently included in this retrospective study and other data were extracted from electronic files.

Ethics statement

This research was examined and validated by the Institutional Review Board of the University Hospital of Lille (CPP Nord Ouest IV) under number HP 20/37. Following IRB recommendations, in accordance with the French law, and because of the retrospective observational design, written informed consent was not required.

Data collection

Patient demographic characteristics, severity scores, comorbidities, primary diagnoses, prior antibiotic exposure were recorded at baseline for all patients. Furthermore, data about clinical, biological, and radiological diagnostic criteria for VA-LRTI, microbiological diagnostic procedures, microbiological findings, degree of severity on the onset of infection, antibiotic use and clinical outcomes were obtained.

Definitions

Criteria from the International ERS/ESICM/ESCMID/ALAT guidelines for the management of hospital-acquired pneumonia and ventilator-associated pneumonia were used for the definition of VAT and VAP (S1 Appendix) [2].

Accordingly, diagnosis of VA-LRTI was based on the presence of at least 2 of the following criteria: body temperature of more than 38.5°C or less than 36.5°C, leucocyte count greater than 12 000 cells per μL or less than 4 000 cells per μL, and purulent endotracheal aspirate. Microbiological confirmation was needed for all episodes of infection, with the isolation in the endotracheal aspirate of at least 105 CFU per mL, or in bronchoalveolar lavage of at least 104 CFU per mL.

VAT was defined as the association of the above-mentioned criteria with no radiographical signs of new pneumonia. Conversely, VAP was defined by the presence of new or progressive infiltrates on chest X-ray along with these criteria [2]. CT-scan images were not used to distinguish VAT from VAP. Only first episodes of VAT and VAP were taken into account for this study.

The diagnosis of sepsis was made in accordance with the Sepsis-3 criteria, and was therefore defined as a change in total SOFA score ≥2 points consequent to the infection over a 48 hours period before collection of the respiratory sample used for the microbiological confirmation of VA-LRTI [9]. Accordingly, we calculated the ΔSOFA as the difference between SOFA scores calculated at the time of microbiological sampling, and 48h before microbiological sampling. The diagnosis of sepsis was established in case of ΔSOFA ≥2 points.

Objectives

The primary aim of this study was to evaluate the accuracy of Sepsis-3 criteria in differentiating VAT from VAP in patients with microbiologically confirmed VA-LRTI. The secondary aims of this study were to assess the accuracy of total SOFA score and respiratory SOFA in differentiating VAT from VAP and to evaluate the accuracy of Sepsis-3 criteria, total SOFA score and respiratory SOFA in predicting mortality in patients with microbiologically confirmed VA-LRTI.

Statistical analysis

Categorical variables were expressed as numbers (percentages) and compared using Chi-square test or Fisher’s exact test, as appropriate. Normality of distribution of continuous variables was checked graphically and by using the Shapiro–Wilk test. Skewed continuous variables were presented as median (interquartile range) and compared using Mann-Whitney U test. Normally distributed continuous variables were presented as means (SD), and compared using Student’s t-test.

We assessed the accuracy of ΔSOFA, total SOFA and SOFA respiratory at the time of microbiological sampling in differentiating VAT from VAP and in predicting mortality by calculating their sensitivity (Se), specificity (Sp), positive and negative predictive values (PPV and NPV) as well as positive and negative likelihood ratios.

All statistical tests were two-tailed, and p values <0.05 were considered statistically significant. Statistical analysis was performed using SPSS 22.0 (IBM, New York, NY) software.

Results

Patient characteristics

Seventy patients with VAT and 136 patients with VAP were included in this study. Study patient characteristics are shown in Table 1. Compared to VAT, patients with a diagnosis of VAP had higher SOFA score on ICU admission (mean (SD) 8.9 (4) vs 7.2 (4.3), p = 0.008) and lower percentage of chronic heart disease (12% vs 23%, p = 0.037), while there was no significant difference for clinical outcomes between the two groups (Table 1). Furthermore, we found a higher frequency of Enterobacter spp. (15% vs 6%, p = 0.043) and a lower frequency of Citrobacter freundii (1% vs 6%, p = 0.047) in patients with VAP than in those with VAT (Table 2).

Table 1. Study population characteristics.

VAT
(n = 70)		 VAP
(n = 136)		 p
Sex 0.74
 Male 51 (73%) 102 (75%)
 Female 19 (27%) 34 (25%)
Age (years) 55.3 (16) 55 (16.4) 0.89
Severity score at ICU admission
 SAPS II 54.6 (19.5) 60 (16.7) 0.079
 SOFA 7.2 (4.3) 8.9 (4) 0.008
Admission type 0.39
 Medical 57 (81%) 117 (86%)
 Surgical 13 (19%) 19 (14%)
Preexisting conditions
 COPD 6 (10%) 21 (15%) 0.17
 Diabetes mellitus 14 (20%) 26 (20%) 0.88
 Immunocompromised patients 7 (10%) 23 (17%) 0.18
 Chronic heart disease 16 (23%) 16 (12%) 0.037
 Chronic respiratory failure 3 (4%) 7 (5%) 0.78
 Cirrhosis 3 (4%) 10 (7%) 0.38
Previous antibiotic use 60 (86%) 107 (79%) 0.30
During ICU stay
 Days on mechanical ventilation 18 (13–29) 17 (11–29) 0.23
 Days in the ICU 24 (16–37) 22 (14–35) 0.3
 ICU mortality 18 (26%) 46 (34%) 0.23
Open in a new tab

Data are presented as number (%) for categorical variables, mean (SD) for normally distributed continuous variables and median (interquartile range) for skewed continuous variables. Admission was defined as surgical if consecutive to a surgery, and medical in the opposite case. COPD chronic obstructive pulmonary disease; SAPS simplified acute physiology score; SOFA sequential organ failure assessment; VAP ventilator-associated pneumonia; VAT ventilator-associated tracheobronchitis.

Table 2. Microbiological findings.

VAT
(n = 70)		 VAP
(n = 136)		 p
Streptococcus pneumoniae 2 (3%) 5 (4%) > 0.999
Stenotrophomonas maltophila 4 (6%) 6 (4%) 0.74
MRSA 1 (1%) 1 (1%) > 0.999
MSSA 6 (9%) 21 (15%) 0.17
Serratia marcescens 3 (4%) 5 (4%) > 0.999
Pseudomonas aeruginosa 20 (29%) 29 (21%) 0.25
Proteus mirabilis 3 (4%) 5 (4%) > 0.999
Klebsiella pneumoniae 9 (13%) 30 (22%) 0.11
Haemophilus influenzae 4 (6%) 6 (4%) 0.74
Escherichia coli 5 (7%) 9 (7%) > 0.999
Enterobacter spp. 4 (6%) 21 (15%) 0.043
Citrobacter freundii 4 (6%) 1 (1%) 0.047
Acinetobacter baumannii 0 (0%) 2 (1%) 0.55
Open in a new tab

Data are presented as number (%) for categorical variables. MSSA methicillin sensitive Staphylococcus aureus; MRSA methicillin resistant Staphylococcus aureus; VAP ventilator-associated pneumonia; VAT ventilator-associated tracheobronchitis.

Accuracy of sepsis criteria according to Sepsis-3 in differentiating VAT from VAP, and predicting mortality

A diagnosis of sepsis was found more frequently in VAP than in VAT patients (40% vs 9%, p <0.001). Subsequently, patients with VAP had higher ΔSOFA than those with VAT (mean (SD) 1 (0.2) vs -0.4 (0.2), p <0.001) (Fig 1). Sepsis criteria according to Sepsis-3 yielded a sensitivity at 0.4 and a specificity at 0.91 for the diagnosis of VAP (Table 3).

Fig 1. Dot plots of ΔSOFA in patients with VAT and VAP.

Fig 1

Open in a new tab

Values of ΔSOFA are shown as single dots for each patient. The dash line separates patients with a ΔSOFA ≥ 2 from those with a ΔSOFA < 2. SOFA sequential organ failure assessment; VAP ventilator-associated pneumonia; VAT ventilator-associated tracheobronchitis.

Table 3. Performances of sepsis criteria according to Sepsis-3 for the diagnosis of VAP and the prediction of ICU mortality in patients with VA-LRTI.

Se Sp PPV NPV LR+ LR-
Diagnosis of VAP 0.4 0.91 0.9 0.44 4.44 0.66
Prediction of mortality 0.38 0.75 0.4 0.73 1.48 0.84
Open in a new tab

Se sensitivity; Sp specificity; PPV positive predictive value; NPV negative predictive value; LR+ positive likelihood ratio; LR- negative likelihood ratio; VA-LRTI ventilator-associated lower respiratory tract infection; VAP ventilator-associated pneumonia.

Among patients with VA-LRTI, there was no significant difference between survivors and non-survivors in the frequency of sepsis (38% vs 25%, p = 0.076), yet ΔSOFA was higher in survivors than in non-survivors (median (IQR) 1 (0; 2.5) vs 0 (-1; 2), p = 0.035) (Fig 2). Sepsis criteria according to Sepsis-3 had a sensitivity at 0.38 and a specificity at 0.75 for the prediction of mortality on ICU discharge (Table 3).

Fig 2. Dot plots of ΔSOFA in survivors and non-survivors.

Fig 2

Open in a new tab

Values of ΔSOFA are shown as single dots for each patient. The dash line separates patients with a ΔSOFA ≥ 2 from those with a ΔSOFA < 2. SOFA sequential organ failure assessment; VAP ventilator-associated pneumonia; VAT ventilator-associated tracheobronchitis.

Accuracy of total SOFA, and respiratory SOFA in differentiating VAT from VAP

At the time of VA-LRTI diagnosis, patients with VAP had higher total SOFA than those with VAT (mean (SD) 7.4 (4.4) vs 4.7 (4), p <0.001) (Fig 3). The best performances to differentiate VAT from VAP were observed for a total SOFA ≥ 6. With this cut-off, sensitivity and specificity were found at 0.63 and 0.69, respectively (Table 4).

Fig 3. Dot plots of total SOFA in patients with VAT and VAP.

Fig 3

Open in a new tab

Values of total SOFA, at VA-LRTI diagnosis, are shown as single dots for each patient. The dash line separates patients with a total SOFA ≥ 6 from those with a total SOFA < 6, identified as the best cut-off according to the Youden index. SOFA sequential organ failure assessment; VAP ventilator-associated pneumonia; VAT ventilator-associated tracheobronchitis.

Table 4. Performances of total SOFA and respiratory SOFA for the diagnosis of VAP in patients with VA-LRTI.

Se Sp PPV NPV LR+ LR- Youden index
Total SOFA
≥ 6 0.63 0.69 0.79 0.48 1.99 0.50 0.31
≥ 7 0.55 0.76 0.82 0.46 2.27 0.44 0.3
≥ 8 0.45 0.86 0.86 0.44 3.14 0.32 0.3
Respiratory SOFA
≥ 1 0.94 0.21 0.70 0.65 1.20 0.83 0.16
≥ 2 0.75 0.40 0.71 0.45 1.25 0.80 0.15
≥ 3 0.49 0.70 0.76 0.41 1.62 0.62 0.19
= 4 0.13 0.94 0.81 0.36 2.19 0.46 0.07
Open in a new tab

SOFA sequential organ failure assessment; Se sensitivity; Sp specificity; PPV positive predictive value; NPV negative predictive value; LR+ positive likelihood ratio; LR- negative likelihood ratio; VA-LRTI ventilator-associated lower respiratory tract infection; VAP ventilator-associated pneumonia.

At the time of VA-LRTI diagnosis, respiratory SOFA was higher in patients with VAP compared to those with VAT (mean (SD) 2.3 (1.2) vs 1.7 (1.1), p = 0.001) (Fig 4). A respiratory SOFA ≥ 3 was associated with the highest Youden index, yielding a sensitivity at 0.49 and a specificity at 0.7 for the diagnosis of VAP (Table 4).

Fig 4. Dot plots of respiratory SOFA in patients with VAT and VAP.

Fig 4

Open in a new tab

Values of respiratory SOFA, at VA-LRTI diagnosis, are shown as single dots for each patient. The dash line separates patients with a respiratory SOFA ≥ 3 from those with a respiratory SOFA < 3, identified as the best cut-off according to the Youden index. SOFA sequential organ failure assessment; VAP ventilator-associated pneumonia; VAT ventilator-associated tracheobronchitis.

Accuracy of total SOFA, and respiratory SOFA in predicting mortality

Total SOFA calculated at the time of VA-LRTI diagnosis was found higher in non-survivors than in survivors (mean (SD) 7.8 (5.3) vs 5.9 (3.9), p < 0.01) (Fig 5). The best performances for the prediction of mortality were observed for a total SOFA ≥ 7. With this cut-off, sensitivity and specificity were found at 0.56 and 0.61, respectively (Table 5).

Fig 5. Dot plots of total SOFA in survivors and non-survivors.

Fig 5

Open in a new tab

Values of total SOFA are shown as single dots for each patient. The dash line separates patients with a total SOFA ≥ 7 from those with a total SOFA < 7. SOFA sequential organ failure assessment.

Table 5. Performances of total SOFA and respiratory SOFA for the prediction of mortality in patients with VA-LRTI.

Se Sp PPV NPV LR+ LR- Youden index
Total SOFA
≥ 5 0.7 0.44 0.36 0.76 1.26 0.79 0.15
≥ 6 0.61 0.52 0.36 0.75 1.27 0.79 0.13
≥ 7 0.56 0.61 0.39 0.75 1.43 0.7 0.17
Respiratory SOFA
≥ 1 0.91 0.12 0.32 0.74 1.03 0.97 0.03
≥ 2 0.77 0.33 0.34 0.76 1.14 0.87 0.10
≥ 3 0.52 0.62 0.38 0.74 1.36 0.74 0.14
= 4 0.22 0.95 0.67 0.73 4.44 0.23 0.17
Open in a new tab

SOFA sequential organ failure assessment; Se sensitivity; Sp specificity; PPV positive predictive value; NPV negative predictive value; LR+ positive likelihood ratio; LR- negative likelihood ratio; VA-LRTI ventilator-associated lower respiratory tract infection.

At the time of VA-LRTI diagnosis, respiratory SOFA was higher in non-survivors compared to survivors (mean (SD) 2.4 (1.2) vs 2 (1.1), p = 0.014) (Fig 6). A respiratory SOFA = 4 was associated with the highest Youden index, yielding a sensitivity at 0.22 and a specificity at 0.95 for the diagnosis of VAP (Table 5).

Fig 6. Dot plots of respiratory SOFA in survivors and non-survivors.

Fig 6

Open in a new tab

Values of respiratory SOFA are shown as single dots for each patient. The dash line separates patients with a respiratory SOFA = 4 from those with a respiratory SOFA < 4. SOFA sequential organ failure assessment.

Discussion

The main purpose of this study was to assess whether a diagnosis of sepsis according to the Sepsis-3 definition could be used as a criterion for the early distinction between VAT and VAP and therefore decide whether antibiotic treatment should be started or not. This question may appear of particular significance for the daily clinical practice. First, because early initiation of antibiotic treatment is recommended in VAP, while there is currently no recommendation regarding such a strategy in VAT [2]. Therefore, finding reliable criteria for the early distinction between VAT and VAP seems relevant. Furthermore, physicians may experience difficulties in making the early distinction between VAT and VAP in the daily clinical practice, due to confounding factors on chest X-ray images (pleural effusion, lung edema) [4, 5] or because of the frequent delay in appearance of lung opacities [6]. Additionally, data from the TAVeM study show that patients with VAP experience greater severity of illness than those with VAT, thus suggesting that a diagnosis of sepsis, reflecting a worsening in organ failures, could be found more frequently in VAP compared to VAT.

Our results suggest that criteria for sepsis according to the Sepsis-3 definition had a high specificity and a low sensitivity to distinguish VAT from VAP, but only a moderate specificity and a low sensitivity to predict ICU-mortality in patients with confirmed VA-LRTI. Further, total SOFA ≥ 6, identified as the best cut-off, showed moderate sensitivity and specificity to differentiate VAT from VAP. On the other hand, the best performances of respiratory SOFA to make the distinction between VAT and VAP were reached for a cut-off ≥ 3. With this threshold, respiratory SOFA had a low sensitivity and a moderate specificity for the diagnosis of VAP.

Our study highlights the weak accuracy of the Sepsis-3 criteria for diagnosis of sepsis in distinguishing VAT from VAP and in predicting mortality in patients with VA-LRTI. Importantly, our results underline the poor sensitivity and negative predictive value of these severity criteria to rule out a diagnosis of VAP, and therefore decide that antibiotic treatment should not be started. These latter findings may be analyzed in light of the data previously published in the literature, which suggest a greater severity of illness in VAP than in VAT. This was reported in a worldwide prospective cohort of ventilated patients by the TAVeM study group, which found a higher mean SOFA value in VAP than in VAT, and subsequently identified VAP as an independent risk factor of mortality, conversely to VAT [1]. Moreover, in this study, the authors reported similar SOFA scores on baseline for patients with VAT and VAP, thus conveying the idea that organ failures were worsening in VAP but not in VAT. However, our study highlights the notable overlap in ΔSOFA when comparing patients with VAP to those with VAT, as well as between survivors and non-survivors, explaining its poor accuracy as a diagnostic and prognostic tool in the daily clinical practice.

Additionally, we found low performances of total SOFA and respiratory SOFA in distinguishing VAT from VAP in our study. However, our results show a higher respiratory SOFA in patients with VAP compared to those with VAT, consistently with previous findings from the TAVeM cohort. Indeed, in that study, patients with VAP experienced more frequent episodes of hypoxemia than in VAT. This finding suggests that a higher respiratory SOFA might be used to distinguish VAP from VAT. However, this hypothesis was not supported by our findings, because of the notable overlap in respiratory SOFA values between the two groups. This result may also be analyzed in regards of the criteria issued by the CDC for the diagnosis of ventilator-associated events, based on the presence of gas exchange worsening [10]. Even though developed for a purpose of reproducibility, these criteria seem to show a poor agreement with the classical definition of VAP, as outlined by several studies [7, 11, 12]. Our results, showing that severity of hypoxemia exhibited poor accuracy for the diagnosis of VAP tend to support these data.

Other diagnosis tools have been proposed to make the early distinction between VAT and VAP. The usefulness of CRP and PCT was thus investigated by the TAVeM study group, who reported a marked overlap in CRP and PCT concentrations between patients with VAT and those with VAP [13]. Subsequently, the area under the ROC curve was found at 0.6 (95% confidence interval (CI) 0.54–0.65) for CRP and 0.63 (95% CI 0.57–0.7) for PCT, reflecting the poor accuracy of these criteria in differentiating VAT from VAP. Furthermore, several studies have underlined the potential interest of alternatives to chest X-ray to allow a more accurate diagnosis of VA-LRTI. This was notably highlighted by Self et al. who reported a better sensitivity of computed tomography (CT) compared to chest X-ray for the detection of lung opacities, raising the question of using this technique to improve the detection of VAP [14]. However, the use of CT to differentiate VAT from VAP can hardly be proposed for daily clinical practice, because of greater costs, time consumption, and higher risks associated with intra-hospital transports in critically ill patients [15, 16]. On the other hand, lung ultrasound has been proposed as an alternative for the diagnosis of VAP, and might be a promising technique in a near future [17, 18]. However, this operator-dependent technique might suffer from a lack of reproducibility and still needs to be validated in larger cohorts of patients. In another approach, Martin-Loeches and Pobo suggested that bronchoscopy might be a helpful tool by identifying secretions coming from deep lung regions, which would be found in VAP but not in VAT [19]. However, no study to date has evaluated this strategy and the usefulness of bronchoscopy in distinguishing VAT from VAP remains unclear. Altogether, these data stress the need for further studies to improve the early detection of VAP in patients with microbiologically confirmed VA-LRTI.

Our study has several limitations. First, this study was conducted in a single center, therefore limiting the applicability of our results to the general population of critically ill patients. However, population characteristics were in line with those observed in the TAVeM study, which enrolled patients from 114 ICU worldwide [1]. Further, we performed a retrospective analysis. Nevertheless, consequences on our results were likely very limited, as we had no missing data for the variables included in our analysis. Additionally, continuous surveillance of ICU-acquired infections allowed prospective identification of patients with VA-LRTI. Another limitation of our study lays in the exclusive use of chest X-ray rather than CT-scan to distinguish VAT from VAP. This was probably associated with a lack of sensitivity in detecting lung opacities in some patients with VAP [4, 5], thus raising concerns about the relevance of using this exam for the diagnosis of VAP in our study. However, it should be remembered that diagnosis of VAT and VAP was made with full knowledge of the patient’s medical record. This included the possibility of observing the appearance of prior opacities on chest X-ray, or secondarily unmasking a differential diagnosis of VAP, such as cardiac overload, atelectasis, or pleural effusion. In these different cases, the diagnosis of VAP could reasonably be made ultimately on the basis of the chest X-ray, while still being difficult in daily practice at the patient’s bedside. Therefore, chest X-ray, which was used in the largest international study describing the characteristics of VAT and VAP, seemed relevant in our study [1]. Moreover, our definitions of VAT and VAP are consistent with those in which a difference in mortality rates were reported [1]. Furthermore, it may be reminded that use of CT-scan for the diagnosis of VAP remains limited in the daily clinical practice, due to greater cost, increased time required to obtain images, higher radiation exposure and risks associated with intra-hospital transportation [20]. As a result, CT scan was only used in a minority of cases to diagnose VAP in our patients, and therefore could not be used in our study. This situation reflects the low practical use of this tool to diagnose VAP. Thus, although the potential value of CT scan in a comprehensive and mechanistic study may be acknowledged, its practical use for the diagnosis of VAP is likely to be limited. Higher levels of SOFA and SAPS2 at ICU admission in VAP compared to VAT was another limitation of our study, as these may partly explain further worsening of organ failures in patients with VAP. Therefore, one could argue that this difference between groups was not addressed in our analysis. However, the goal of our study was merely diagnostic. Accordingly, the practical question that our study aimed to answer is whether a worsening in organ failures could be used as a reliable criterion to differentiate VAT from VAP, and could therefore be used in the daily clinical practice to decide whether antibiotic treatments should be started or not. To answer this practical question, it appears that ΔSOFA must be assessed as it would by the physician at the patient’s bedside, without adjustment for potential confounders. Last, our study did not compare timings of diagnosis of VAP when comparing severity criteria including sepsis criteria according to the Sepsis-3 definition vs chest X-ray. However, our study was not designed to answer this question, but merely aimed to evaluate severity criteria at the time of clinical evidence of VA-LRTI for the diagnosis of VAP, and may thus bring relevant information regarding this practical question at the patient’s bedside.

Conclusions

In patients with evidence of VA-LRTI, sepsis criteria following the Sepsis-3 definition exhibit low performances for the diagnosis of VAP and for the prediction of mortality. Total SOFA and respiratory SOFA were also inaccurate in differentiating VAT from VAP. Accordingly, our results do not support the use of these criteria to drive the early initiation of antibiotic treatments in patients with VA-LRTI.

Supporting information

S1 Fig. ROC curves of ΔSOFA (A), total SOFA (B), and respiratory SOFA (C) for the diagnosis of VAP in patients with VA-LRTI.

SOFA sequential organ failure assessment.

(TIF)

Click here for additional data file. (74.5KB, tif)
S2 Fig. ROC curves of ΔSOFA (A), total SOFA (B), and respiratory SOFA (C) for the prediction of mortality on ICU discharge in patients with VA-LRTI.

SOFA sequential organ failure assessment.

(TIF)

Click here for additional data file. (121.8KB, tif)
S1 Appendix. Criteria for diagnosis of VAT and VAP.

(DOCX)

Click here for additional data file. (14.8KB, docx)
S2 Appendix. Sepsis-3 criteria for diagnosis of sepsis and septic shock.

(DOCX)

Click here for additional data file. (15KB, docx)

Abbreviations

CI

confidence interval

CT

computed tomography

ICU

intensive care unit

IQR

interquartile range

NPV

negative predictive value

PPV

positive predictive value

SD

standard deviation

Se

sensitivity

Sp

specificity

SOFA

sequential organ failure assessment

VA-LRTI

ventilator-associated lower respiratory tract infection

VAP

ventilator-associated pneumonia

VAT

ventilator-associated tracheobronchitis

Data Availability

All relevant data are within the manuscript and its Supporting information files.

Funding Statement

The author(s) received no specific funding for this work.

References

  • 1.Martin-Loeches I, Povoa P, Rodríguez A, et al. (2015) Incidence and prognosis of ventilator-associated tracheobronchitis (TAVeM): a multicentre, prospective, observational study. Lancet Respir Med 3:859–868. 10.1016/S2213-2600(15)00326-4 [DOI] [PubMed] [Google Scholar]
  • 2.Torres A, Niederman MS, Chastre J, et al. (2017) International ERS/ESICM/ESCMID/ALAT guidelines for the management of hospital-acquired pneumonia and ventilator-associated pneumonia: Guidelines for the management of hospital-acquired pneumonia (HAP)/ventilator-associated pneumonia (VAP) of the European Respiratory Society (ERS), European Society of Intensive Care Medicine (ESICM), European Society of Clinical Microbiology and Infectious Diseases (ESCMID) and Asociación Latinoamericana del Tórax (ALAT). Eur Respir J 50:. 10.1183/13993003.00582-2017 [DOI] [PubMed] [Google Scholar]
  • 3.Kalil AC, Metersky ML, Klompas M, et al. (2016) Management of Adults With Hospital-acquired and Ventilator-associated Pneumonia: 2016 Clinical Practice Guidelines by the Infectious Diseases Society of America and the American Thoracic Society. Clin Infect Dis Off Publ Infect Dis Soc Am 63:e61–e111. 10.1093/cid/ciw353 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Butler KL, Sinclair KE, Henderson VJ, et al. (1999) The chest radiograph in critically ill surgical patients is inaccurate in predicting ventilator-associated pneumonia. Am Surg 65:805–809; discussion 809–810 [PubMed] [Google Scholar]
  • 5.Graat ME, Choi G, Wolthuis EK, et al. (2006) The clinical value of daily routine chest radiographs in a mixed medical-surgical intensive care unit is low. Crit Care Lond Engl 10:R11 10.1186/cc3955 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Ramirez P, Lopez-Ferraz C, Gordon M, et al. (2016) From starting mechanical ventilation to ventilator-associated pneumonia, choosing the right moment to start antibiotic treatment. Crit Care Lond Engl 20:169 10.1186/s13054-016-1342-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Ramírez-Estrada S, Lagunes L, Peña-López Y, et al. (2018) Assessing predictive accuracy for outcomes of ventilator-associated events in an international cohort: the EUVAE study. Intensive Care Med 44:1212–1220. 10.1007/s00134-018-5269-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.da Silveira F, Nedel WL, Cassol R, et al. (2019) Acinetobacter etiology respiratory tract infections associated with mechanical ventilation: what impacts on the prognosis? A retrospective cohort study. J Crit Care 49:124–128. 10.1016/j.jcrc.2018.10.034 [DOI] [PubMed] [Google Scholar]
  • 9.Singer M, Deutschman CS, Seymour CW, et al. (2016) The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 315:801–810. 10.1001/jama.2016.0287 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Magill SS, Klompas M, Balk R, et al. (2013) Developing a new, national approach to surveillance for ventilator-associated events*. Crit Care Med 41:2467–2475. 10.1097/CCM.0b013e3182a262db [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Pouly O, Lecailtel S, Six S, et al. (2020) Accuracy of ventilator-associated events for the diagnosis of ventilator-associated lower respiratory tract infections. Ann Intensive Care 10:6 10.1186/s13613-020-0624-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Fan Y, Gao F, Wu Y, et al. (2016) Does ventilator-associated event surveillance detect ventilator-associated pneumonia in intensive care units? A systematic review and meta-analysis. Crit Care Lond Engl 20:338 10.1186/s13054-016-1506-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Coelho L, Rabello L, Salluh J, et al. (2018) C-reactive protein and procalcitonin profile in ventilator-associated lower respiratory infections. J Crit Care 48:385–389. 10.1016/j.jcrc.2018.09.036 [DOI] [PubMed] [Google Scholar]
  • 14.Self WH, Courtney DM, McNaughton CD, et al. (2013) High discordance of chest x-ray and computed tomography for detection of pulmonary opacities in ED patients: implications for diagnosing pneumonia. Am J Emerg Med 31:401–405. 10.1016/j.ajem.2012.08.041 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Bercault N, Wolf M, Runge I, et al. (2005) Intrahospital transport of critically ill ventilated patients: a risk factor for ventilator-associated pneumonia--a matched cohort study. Crit Care Med 33:2471–2478. [DOI] [PubMed] [Google Scholar]
  • 16.Parmentier-Decrucq E, Poissy J, Favory R, et al. (2013) Adverse events during intrahospital transport of critically ill patients: incidence and risk factors. Ann Intensive Care 3:10 10.1186/2110-5820-3-10 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Bouhemad B, Dransart-Rayé O, Mojoli F, Mongodi S (2018) Lung ultrasound for diagnosis and monitoring of ventilator-associated pneumonia. Ann Transl Med 6:418 10.21037/atm.2018.10.46 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Pradhan S, Shrestha PS, Shrestha GS, Marhatta MN (2020) Clinical impact of lung ultrasound monitoring for diagnosis of ventilator associated pneumonia: A diagnostic randomized controlled trial. J Crit Care 58:65–71. 10.1016/j.jcrc.2020.03.012 [DOI] [PubMed] [Google Scholar]
  • 19.Martin-Loeches I, Pobo A (2010) What Is New in Ventilator-Associated Tracheobronchitis? Clin Pulm Med 17:117–121. 10.1097/CPM.0b013e3181dae20f [DOI] [Google Scholar]
  • 20.Keane S, Vallecoccia MS, Nseir S, Martin-Loeches I (2018) How Can We Distinguish Ventilator-Associated Tracheobronchitis from Pneumonia? Clin Chest Med 39:785–796. 10.1016/j.ccm.2018.08.003 [DOI] [PubMed] [Google Scholar]

Decision Letter 0

Yutaka Kondo

26 Oct 2020

PONE-D-20-26296

Usefulness of Sepsis-3 in diagnosing and predicting mortality of ventilator-associated lower respiratory tract infections

PLOS ONE

Dear Dr. Gaudet,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

Overall, well written and some aspects are paid our attention. However, several points need to be modified. Please see our reviewers comments below. 

==============================

Please submit your revised manuscript by Dec 10 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Yutaka Kondo

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following in the Competing Interests section:

'I have read the journal's policy and the authors of this manuscript have the following competing interests:SN received fees from MSD, Pfizer, Gilead, BioMérieux, and Bio Rad for lectures.

Other authors declare that they have no competing interest.'

a. Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these.

Please note that we cannot proceed with consideration of your article until this information has been declared.

b. Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

3. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please delete it from any other section.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: No

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This retrospective study evaluated diagnostic ability of sepsis-3 criteria, total SOFA and respiratory SOFA to differentiate VAP from VAT, and concluded that sepsis-3 was not useful. It is worth publishing, since logic looks appropriate and conclusion is reasonable. However, it also includes several problems to be addressed.

P6, L90

“All adults patients with a diagnosis of VA-LRTI were retrospectively included in the study. The diagnosis of VA-LRTI was prospectively performed by ICU physicians.”

If authors regarded diagnosis was made prospectively, they had to review not only patients diagnosed as VA-LRTI, but also all patients admitted to the ICU during study period. If they had reviewed only retrospectively-included VALRTI patients, prospective diagnosis of VALRTI could not be possible. Or did they only diagnose VAT or VAP among VALRTI patients? Please explain and describe clearly in the Methods section.

P7, L120

Regarding Objective of this study, “accuracy of predicting ICU mortality” was evaluated only in sepsis-3 analysis, not in analyses of total SOFA and respiratory SOFA. However, since authors repeatedly compared sepsis-3 to SOFA in the context of this study, they should have evaluated SOFA’s ability in predicting mortality. Otherwise, they should focus only on ability of diagnosis of VAT and VAP in comparison between sepsis-3 and SOFA, and eliminate results of mortality in spsis-3 analysis.

Reviewer #2: The definition was vague. Did the authors look for signs of pulmonary consolidation in admission? no CT scan?

What is category of admission? Please explain details.

The adjusting the severity is important.

From reading the paper many things remain unclear. What is special about this study in relation to future strategy or research? The study lacks clarity and furthermore, it does not become clear what is particulary special and what the readers can learn from it.

Reviewer #3: I greatly appreciate giving me the opportunity to review the manuscript entitled “Usefulness of Sepsis-3 in diagnosing and predicting mortality of ventilator-associated lower respiratory tract infections”. It is true that the more accurate and rapid diagnostic tool to distinguish VAP and VAT is necessary to improve the management of critically ill ventilated patients. However, the aim of the current study is not clear, and the study design seems not optimal. My concerns are as follows.

1. First, the use of the term “sepsis-3 criteria” is not appropriate. Sepsis-3 is just “Sepsis-3, i.e., it is used to define sepsis based on progress of organ dysfunctions, and it is not a general diagnostic tool for other diseases. Therefore, the authors must use the term such as “increase in SOFA score”, not “sepsis-3 criteria”.

2. The aim of the study is unclear. If the authors have hypothesized that the use of SOFA score can diagnose the progress of infection to lung parenchyma (the true definition of VAP) more accurately compared with chest X-ray, the gold-standard diagnosis must be established by more accurate modality such as chest CT, and the accuracy of two diagnostic methods; SOFA vs. chest X-ray, must be evaluated. Or if the authors are willing to demonstrate the use of SOFA score can diagnose gradual VAP more rapidly compared with chest X-ray, they must show the timing of diagnosis of VAP by two diagnostic methods. In the current manuscript, the authors have just compared the SOFA scores between VAP and VAT patients diagnosed by chest X-ray. It is not clear how they want to utilize the SOFA score to distinguish VAP and VAT.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Kentaro Tojo

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2021 Jan 14;16(1):e0245552. doi: 10.1371/journal.pone.0245552.r002

Author response to Decision Letter 0

24 Nov 2020

Dear Editors,

Dear Reviewers,

Thank you for your helpful suggestions regarding our manuscript.

Following your comments, we have provided some additional information which, we believe, should improve the quality of the article.

Please find appended below our point-by-point response to your questions:

Reviewer #1: This retrospective study evaluated diagnostic ability of sepsis-3 criteria, total SOFA and respiratory SOFA to differentiate VAP from VAT, and concluded that sepsis-3 was not useful. It is worth publishing, since logic looks appropriate and conclusion is reasonable. However, it also includes several problems to be addressed.

1. P6, L90 “All adults patients with a diagnosis of VA-LRTI were retrospectively included in the study. The diagnosis of VA-LRTI was prospectively performed by ICU physicians.” If authors regarded diagnosis was made prospectively, they had to review not only patients diagnosed as VA-LRTI, but also all patients admitted to the ICU during study period. If they had reviewed only retrospectively-included VALRTI patients, prospective diagnosis of VALRTI could not be possible. Or did they only diagnose VAT or VAP among VALRTI patients? Please explain and describe clearly in the Methods section.

Thank you for pointing out this lack of clarity in our initial formulation. Physicians in charge of the patients identified VA-LRTI episodes as they occurred (prospective diagnosis), and retrospectively collected data on these patients. From a methodological point of view, this is therefore a retrospective study. Accordingly, and to avoid confusion, we have changed the description of the study design in the methods section into: “Continuous surveillance of ICU-acquired infections allowed prospective identification of patients with VA-LRTI. These patients were subsequently included in this retrospective study and other data were extracted from electronic files”.

2. P7, L120 Regarding Objective of this study, “accuracy of predicting ICU mortality” was evaluated only in sepsis-3 analysis, not in analyses of total SOFA and respiratory SOFA. However, since authors repeatedly compared sepsis-3 to SOFA in the context of this study, they should have evaluated SOFA’s ability in predicting mortality. Otherwise, they should focus only on ability of diagnosis of VAT and VAP in comparison between sepsis-3 and SOFA, and eliminate results of mortality in spsis-3 analysis.

We fully agree with your comment, and have therefore completed the manuscript by reporting the assessment of total SOFA and respiratory SOFA in predicting mortality in main text, Table 5, Figures 5 and 6, and Figure S2.

Reviewer #2:

1. The definition was vague. Did the authors look for signs of pulmonary consolidation in admission? no CT scan?

Thank you for pointing out these important matters. To avoid any confusion for the reader, we now explicitly mention in the manuscript that criteria from the International ERS/ESICM/ESCMID/ALAT guidelines for the management of hospital-acquired pneumonia and ventilator-associated pneumonia were used for the definition of VAT and VAP (1). Furthermore, these criteria are now described in S1 Appendix, in addition to criteria detailed in the methods section.

This definition does not specify whether chest X-ray or CT-scan should be used for the diagnosis of VA-LRTI. In our study, only chest X-ray was used to distinguish VAT from VAP as CT-scan is not routinely performed for the diagnosis of VAP, due to greater cost, increased time required to obtain images, higher radiation exposure and risks associated with intra-hospital transportation (2).

We have therefore completed the methods section of our manuscript, by explicitly stating that CT-scan was not used to differentiate VAT from VAP in our study.

We acknowledge that lung opacities were probably missed in some patients with VAP, given the reported lack of sensitivity of chest X-ray for the detection of pneumonia (3,4). However, the use of chest X-ray allowed us to diagnose VAT and VAP using the same criteria than in the TAVeM study. Accordingly, our definitions of VAP and VAT are consistent with those in which a difference in mortality rates were reported (5).

These points are now mentioned in the limitations section of the manuscript.

2. What is category of admission? Please explain details.

We acknowledge that clarifications were needed regarding definitions of admission categories.

We are actually referring to the type of admission in ICU. Admission were defined as surgical if consecutive to a surgery, and medical in the opposite case.

The term “Category of admission” has been replaced by “Admission type”, and these definitions are now specified in caption of Table 1.

3. The adjusting the severity is important.

We thank you for raising the important question of adjusting on severity in our analysis.

As pointed out in your comment, total SOFA at ICU admission was significantly higher in VAP than in VAT. Assessment of SAPS2 at ICU admission also showed a trend for higher severity in VAP.

We perfectly acknowledge that confounding factors such as higher levels of severity at ICU admission may partly explain further increases in ∆SOFA in patients with VAP. Addressing this issue would be crucial from the prospect of a study aiming to understand the mechanisms associated with the development of VAP.

However, we’d like to remind that the goal of our study was merely diagnostic. Accordingly, the practical question that our study aimed to answer is whether a worsening in organ failures could be used as a reliable criterion to differentiate VAT from VAP, and could therefore be used in the daily clinical practice to decide whether to initiate or not antibiotic treatments. To answer this practical question, it appears that severity criteria, including ∆SOFA, must be assessed as they would by the physician at the patient’s bedside, without adjustment for potential confounders.

Initial version of our manuscript lacked clarity regarding these points, which have now been developed in the discussion section of the revised version. We believe it will help the reader to understand more clearly the purpose of our study.

4. From reading the paper many things remain unclear. What is special about this study in relation to future strategy or research? The study lacks clarity and furthermore, it does not become clear what is particulary special and what the readers can learn from it.

The rationale of our study mainly relies on the 3 following points:

a) Early initiation of antibiotic treatment is recommended in VAP, while there is currently no recommendation regarding such a strategy in VAT (6). Therefore, finding reliable criteria for the early distinction between VAT and VAP seems relevant.

b) Physicians may experience difficulties in making the early distinction between VAT and VAP in the daily clinical practice, due to confounding factors on chest X-ray images (pleural effusion, lung edema) (3,4) or because of the frequent delay in appearance of lung opacities (7).

c) Data from the TAVeM study show that patients with VAP experience greater severity of illness than those with VAT, thus suggesting that a diagnosis of sepsis, reflecting a worsening in organ failures, could be found more frequently in VAP compared to VAT.

Therefore, our study aimed to clarify a practical question for the daily clinical practice, i.e. whether diagnosis of sepsis according to Sepsis-3 criteria, reflecting a greater severity of illness, would be a reliable criterion to diagnose VAP in patients with VA-LRTI, and accordingly, drive the early initiation of antibiotic treatment.

Our results suggest that assessment of severity, especially through Sepsis-3 criteria, exhibit poor performances to early differentiate VAT from VAP. Importantly, we found poor sensitivity and negative predictive value of these criteria to rule out a diagnosis of VAP. As a consequence, these criteria should probably not be used to drive the early initiation of antimicrobial therapy in patients with evidence of VA-LRTI. Especially, the absence of diagnosis of sepsis appears as a bad criterion to decide not to initiate antibiotic treatments.

To the best of our knowledge, our study is the first to address this question, and we believe it could have a practical impact for the physician at the patient’s bedside.

However, we acknowledge that our message was not enough clearly developed in our manuscript. Accordingly, these points are now clearly developed in the first part of our discussion. We hope these changes will help bringing more clearly the aims and results of our study to the reader.

Reviewer #3: I greatly appreciate giving me the opportunity to review the manuscript entitled “Usefulness of Sepsis-3 in diagnosing and predicting mortality of ventilator-associated lower respiratory tract infections”. It is true that the more accurate and rapid diagnostic tool to distinguish VAP and VAT is necessary to improve the management of critically ill ventilated patients. However, the aim of the current study is not clear, and the study design seems not optimal. My concerns are as follows.

1. First, the use of the term “sepsis-3 criteria” is not appropriate. Sepsis-3 is just “Sepsis-3, i.e., it is used to define sepsis based on progress of organ dysfunctions, and it is not a general diagnostic tool for other diseases. Therefore, the authors must use the term such as “increase in SOFA score”, not “sepsis-3 criteria”.

We acknowledge that the term “Sepsis-3 criteria” was not used in an appropriate way in our manuscript. Especially, use of this term may be confusing as it does not indicate whether criteria for sepsis or septic shock were evaluated. However, the main purpose of our study was to assess whether the presence of sepsis would be a good marker of VAP, and conversely, if absence of sepsis would be sufficient to rule out VAP, and therefore decide not to initiate antibiotic treatment. As a consequence, we believe it is important for a matter of clarity to keep reminding to the reader that our study evaluated the diagnostic criteria for sepsis.

Therefore, we have replaced the term “Sepsis-3 criteria” by “sepsis criteria according to the Sepsis-3 definition” throughout the article. Furthermore, description of criteria for sepsis and septic shock according to the Sepsis-3 definition has been added in S2 Appendix.

2. The aim of the study is unclear. If the authors have hypothesized that the use of SOFA score can diagnose the progress of infection to lung parenchyma (the true definition of VAP) more accurately compared with chest X-ray, the gold-standard diagnosis must be established by more accurate modality such as chest CT, and the accuracy of two diagnostic methods; SOFA vs. chest X-ray, must be evaluated. Or if the authors are willing to demonstrate the use of SOFA score can diagnose gradual VAP more rapidly compared with chest X-ray, they must show the timing of diagnosis of VAP by two diagnostic methods. In the current manuscript, the authors have just compared the SOFA scores between VAP and VAT patients diagnosed by chest X-ray. It is not clear how they want to utilize the SOFA score to distinguish VAP and VAT.

We fully agree on the importance of clearly describing the purpose of our study.

Our study aimed to clarify a practical question for the daily clinical practice, i.e. whether diagnosis of sepsis according to Sepsis-3 criteria, reflecting a greater severity of illness, would be a reliable criterion to early detect or rule out VAP, and therefore decide whether to initiate or not antibiotic treatment in VA-LRTI. In the prospect of a practical diagnostic study, this assessment was made by the time of appearance of clinical signs of VA-LRTI, when radiographical signs of pneumonia on chest X-ray are frequently hidden or delayed (2–4,7).

In our study, only chest X-ray was used to distinguish VAT from VAP as CT-scan is not routinely performed for the diagnosis of VAP, due to greater cost, increased time required to obtain images, higher radiation exposure and risks associated with intra-hospital transportation2.

We acknowledge that lung opacities were probably missed in some patients with VAP, given the reported lack of sensitivity of chest X-ray for the detection of pneumonia (3,4). However, the use of chest X-ray allowed us to diagnose VAT and VAP using the same criteria than in the TAVeM study. Accordingly, our definitions of VAP and VAT are consistent with those in which a difference in mortality rates were reported (5).

Our results suggest that assessment of severity, especially through sepsis criteria according to the Sepsis-3 definition, exhibit poor performances to early differentiate VAT from VAP. Importantly, we found poor sensitivity and negative predictive value of these criteria to rule out a diagnosis of VAP. As a consequence, these criteria should probably not be used to drive the early initiation of antimicrobial therapy in patients with evidence of VA-LRTI. Especially, the absence of diagnosis of sepsis appears as a bad criterion to decide not to initiate antibiotic treatments.

Finally, we agree on the interest of comparing timings of diagnosis of VAP when using both methods, i.e. comparing severity criteria vs chest X-ray. However, our study was not designed to answer this question, but merely aimed to evaluate severity criteria at the time of clinical evidence of VA-LRTI for the diagnosis of VAP, which corresponds to a practical question at the patient’s bedside.

All these points have been developed in the discussion and limitations of the revised manuscript. We believe it will help the reader to understand more clearly the purpose of our study, as well as the significance of our results for the daily clinical practice.

References:

1. Torres A, Niederman MS, Chastre J, Ewig S, Fernandez-Vandellos P, Hanberger H, et al. International ERS/ESICM/ESCMID/ALAT guidelines for the management of hospital-acquired pneumonia and ventilator-associated pneumonia: Guidelines for the management of hospital-acquired pneumonia (HAP)/ventilator-associated pneumonia (VAP) of the European Respiratory Society (ERS), European Society of Intensive Care Medicine (ESICM), European Society of Clinical Microbiology and Infectious Diseases (ESCMID) and Asociación Latinoamericana del Tórax (ALAT). Eur Respir J. 2017 Sep;50(3).

2. Keane S, Vallecoccia MS, Nseir S, Martin-Loeches I. How Can We Distinguish Ventilator-Associated Tracheobronchitis from Pneumonia? Clin Chest Med. 2018;39(4):785–96.

3. Butler KL, Sinclair KE, Henderson VJ, McKinney G, Mesidor DA, Katon-Benitez I, et al. The chest radiograph in critically ill surgical patients is inaccurate in predicting ventilator-associated pneumonia. Am Surg. 1999 Sep;65(9):805–9; discussion 809-810.

4. Graat ME, Choi G, Wolthuis EK, Korevaar JC, Spronk PE, Stoker J, et al. The clinical value of daily routine chest radiographs in a mixed medical-surgical intensive care unit is low. Crit Care Lond Engl. 2006 Feb;10(1):R11.

5. Martin-Loeches I, Povoa P, Rodríguez A, Curcio D, Suarez D, Mira J-P, et al. Incidence and prognosis of ventilator-associated tracheobronchitis (TAVeM): a multicentre, prospective, observational study. Lancet Respir Med. 2015;3(11):859–68.

6. Kalil AC, Metersky ML, Klompas M, Muscedere J, Sweeney DA, Palmer LB, et al. Management of Adults With Hospital-acquired and Ventilator-associated Pneumonia: 2016 Clinical Practice Guidelines by the Infectious Diseases Society of America and the American Thoracic Society. Clin Infect Dis Off Publ Infect Dis Soc Am. 2016 Sep 1;63(5):e61–111.

7. Ramirez P, Lopez-Ferraz C, Gordon M, Gimeno A, Villarreal E, Ruiz J, et al. From starting mechanical ventilation to ventilator-associated pneumonia, choosing the right moment to start antibiotic treatment. Crit Care Lond Engl. 2016 Jun 3;20(1):169.

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file. (36.9KB, docx)

Decision Letter 1

Yutaka Kondo

17 Dec 2020

PONE-D-20-26296R1

Usefulness of Sepsis-3 in diagnosing and predicting mortality of ventilator-associated lower respiratory tract infections

PLOS ONE

Dear Dr. Gaudet,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

ACADEMIC EDITOR: Our reviewers have evaluated your revised manuscript. Please see our reviewers comments and revise as much as possible. At least, the authors need to state the points. Thank you very much. 

==============================

Please submit your revised manuscript by Jan 31 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Yutaka Kondo

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: (No Response)

Reviewer #2: Yes

Reviewer #3: No

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: (No Response)

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: (No Response)

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: Excellent work addressing previous concerns. Happy with the responses to my questions and the revisions.

Reviewer #3: Thank you very much for giving me the opportunity to review the revised version of the manuscript entitled “Usefulness of Sepsis-3 in diagnosing and predicting mortality of ventilator-associated lower respiratory tract infections”. I have understood the aim of the study; the authors are willing to evaluate the utility of the Sepsis-3 criteria to diagnosis VAP, because chest X-ray lacks adequate sensitivity or specificity to diagnose pneumonia. However, according to the fact, the VAP diagnosis by chest X-ray in this study might not be accurate, and the diagnostic utility of Sepsis-3 criteria evaluated based on the inaccurate VAP diagnosis has no meaning. To prove the authors’ hypothesis, the gold-standard diagnosis of the VAP must be performed by more accurate modalities such as chest CT. The design and data set in this study cannot solve the question raised by the authors. It is necessary to comprehensively reconsider the study aim and design.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Kentaro Tojo

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2021 Jan 14;16(1):e0245552. doi: 10.1371/journal.pone.0245552.r004

Author response to Decision Letter 1

19 Dec 2020

Dear Editor,

Dear Reviewer,

Thank you for allowing us to reply to your comments.

We have provided some additional clarifications which, we believe, have improved the quality of our manuscript.

Please find appended below our response to your concerns:

Reviewer #3: Thank you very much for giving me the opportunity to review the revised version of the manuscript entitled “Usefulness of Sepsis-3 in diagnosing and predicting mortality of ventilator-associated lower respiratory tract infections”. I have understood the aim of the study; the authors are willing to evaluate the utility of the Sepsis-3 criteria to diagnosis VAP, because chest X-ray lacks adequate sensitivity or specificity to diagnose pneumonia. However, according to the fact, the VAP diagnosis by chest X-ray in this study might not be accurate, and the diagnostic utility of Sepsis-3 criteria evaluated based on the inaccurate VAP diagnosis has no meaning. To prove the authors’ hypothesis, the gold-standard diagnosis of the VAP must be performed by more accurate modalities such as chest CT. The design and data set in this study cannot solve the question raised by the authors. It is necessary to comprehensively reconsider the study aim and design.

We would like to thank you for this comment. As you are pointing out, our study evaluated the value of the Sepsis-3 criteria for the diagnosis of VAP, as the interpretation of the chest X-ray can be difficult in daily practice.

Response: We fully understand your question regarding the relevance of the radiographic criteria in our study. Nevertheless, it should be remembered that the diagnosis of VAP was made with full knowledge of the patient's medical record. This included the possibility of observing the appearance of prior opacities on the chest X-ray, or secondarily unmasking a differential diagnosis of VAP, such as cardiac overload, atelectasis, or pleural effusion. In these different cases, the diagnosis of VAP could reasonably be made ultimately on the basis of the chest x-ray, while still being difficult in daily practice at the patient's bedside. Therefore, it seems to us that chest X-ray, which was used in the largest international study describing the characteristics of VAT and VAP , was relevant in our study.

On the other hand, CT scan was only used in a minority of cases to diagnose VAP in our patients, and therefore could not be used in our study. This situation reflects the low practical use of this tool to diagnose VAP. Thus, although we recognize the potential value of CT scan in a comprehensive and mechanistic study, its practical use for the diagnosis of VAP is likely to be limited.

We are aware of the importance of these issues in the interpretation of our study. Accordingly, these different points have been developed in the limitations section of our manuscript.

We hope that these explanations have helped to remove your concerns about the value of this study, and remain at your disposal to answer any further questions you may have.

Attachment

Submitted filename: Response to Reviewers R2.docx

Click here for additional data file. (22KB, docx)

Decision Letter 2

Yutaka Kondo

2 Jan 2021

Usefulness of Sepsis-3 in diagnosing and predicting mortality of ventilator-associated lower respiratory tract infections

PONE-D-20-26296R2

Dear Dr. Gaudet,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Yutaka Kondo

Academic Editor

PLOS ONE

Additional Editor Comments (optional): Thank you for great effort to revise the manuscript. Current version is much improved and we decided this decision. Congratulations!

Reviewers' comments:

Acceptance letter

Yutaka Kondo

6 Jan 2021

PONE-D-20-26296R2

Usefulness of Sepsis-3 in diagnosing and predicting mortality of ventilator-associated lower respiratory tract infections

Dear Dr. Gaudet:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Yutaka Kondo

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Fig. ROC curves of ΔSOFA (A), total SOFA (B), and respiratory SOFA (C) for the diagnosis of VAP in patients with VA-LRTI.

    SOFA sequential organ failure assessment.

    (TIF)

    Click here for additional data file. (74.5KB, tif)
    S2 Fig. ROC curves of ΔSOFA (A), total SOFA (B), and respiratory SOFA (C) for the prediction of mortality on ICU discharge in patients with VA-LRTI.

    SOFA sequential organ failure assessment.

    (TIF)

    Click here for additional data file. (121.8KB, tif)
    S1 Appendix. Criteria for diagnosis of VAT and VAP.

    (DOCX)

    Click here for additional data file. (14.8KB, docx)
    S2 Appendix. Sepsis-3 criteria for diagnosis of sepsis and septic shock.

    (DOCX)

    Click here for additional data file. (15KB, docx)
    Attachment

    Submitted filename: Response to Reviewers.docx

    Click here for additional data file. (36.9KB, docx)
    Attachment

    Submitted filename: Response to Reviewers R2.docx

    Click here for additional data file. (22KB, docx)

    Data Availability Statement

    All relevant data are within the manuscript and its Supporting information files.

    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES