Abstract
Objective
Various risk scoring models have been developed to predict stroke-associated pneumonia (SAP). We aim to determine whether these risk models could effectively predict SAP in Chinese patients with ischaemic stroke (IS).
Methods
Consecutive patients with IS in West China hospital between January 2011 and September 2013 were included to assess the predictive performance of risk scoring models, including Chumbler's score, A2DS2 and AISAPS. The area under the receiver operating characteristic curve (AUROC) was used to evaluate the performance of each risk model in predicting pneumonia.
Results
A total of 1569 consecutive patients with IS within 30 days of onset in West China hospital were included. The incidence of pneumonia is 15.3%. The AUROC of Chumbler's score, A2DS2 and AISAPS was 0.659, 0.728 and 0.758, respectively, and AISAPS had the highest AUROC.
Conclusions
A2DS2 and AISAPS had acceptable discriminatory abilities to predict SAP in Chinese patients with IS within 30 days of onset.
Keywords: Pneumonia, Risk scoring model, Ischemic Stroke
Introduction
Pneumonia is a common complication in patients with stroke.1 It occurs in about 9% of patients with stroke, and 28% of patients with stroke in the intensive care unit.2 A variety of factors, such as older age, dysphagia and higher stroke severity score, which are assessed by the National Institutes of Health Stroke Scale (NIHSS), are associated with stroke-associated pneumonia (SAP).3 SAP potentially increases the risk of a poor and fatal outcome,4 and increases the economic burden on patients with stroke.5 Although a preventive antibiotic cannot improve the prognosis of patients with stroke, it is proved to be effective to decrease the incidence of SAP.6 Oral care is also proved to be effective to reduce the incidence of SAP.3 Recognising patients with stroke at high risk of pneumonia and providing preventive intervention can be helpful to reduce the incidence of SAP.
Several risk scoring models were reported to improve the recognition of the high risk of pneumonia in patients with stroke, including Kwon's score,7 the PANTHER-IS score,8 Chumbler's score,9 A2DS210 and AISAPS.11 There are many advantages to the wide use of these risk models, such as assisting clinicians to identify patients with stroke with high risk of pneumonia and facilitating the selection of patients in clinic trials. However, these models have not been applied extensively to patients in different cohorts from various races and areas. They need to be externally validated before they can be used in clinical practice.
We conducted this study to determine whether these risk models could effectively predict SAP during hospitalisation in Chinese patients with IS. Since the risk scores of ischaemic stroke-associated pneumonia (like A2DS2 and AISAPS) are different from that of haemorrhagic stroke (like intracerebral haemorrhage-associated pneumonia score (ICH-APSs)),12 we only focused on patients with ischaemic stroke to control potential confounders.
Methods
Study population
We identified all consecutive patients with IS in West China hospital between January 2011 and September 2013. Inclusion criteria: aged≥18; patients with IS who were admitted to hospital within 30 days of onset; ischaemic stroke was diagnosed based on the WHO criteria; IS was confirmed by brain CT or MRI. Exclusion criteria: patients with transient ischaemic attack (TIA); pneumonia happened before admission. Severity of stroke was assessed by NIHSS.13 Consciousness was evaluated by Glasgow Coma Scale.14 The subtype of IS was classified based on the Oxfordshire Community Stroke Project criteria: partial anterior circulation infarct, total anterior circulation infarct, lacunar infarction and posterior circulation infarct.15 Dysphagia was evaluated by a neurological clinician with the water drinking test. Prestroke dependence was defined as modified Rankin Scale ≥3.
Outcome measures
The primary outcome was the presence of SAP in hospitalised patients with IS. SAP was diagnosed by the treating physician according to the modified criteria of US Centers for Disease Control and Prevention criteria for hospital-acquired pneumonia. Pneumonia was diagnosed when at least one of the first and one of the latter criteria were fulfilled: (A) abnormal respiratory examination, pulmonary infiltrates in chest X-rays; (B) productive cough with purulent sputum, microbiological cultures from the lower respiratory tract or blood cultures, leucocytosis and elevation of C reactive protein.16
Risk prediction scores of SAP
We indentified five risk scoring models of SAP in total. Kwon's score was derived from 286 patients with stroke in a single hospital, as was the PANTHER-IS score (derived from 223 patients with IS). Chumbler et al developed their risk model from 925 patients in multicentre hospitals. A2DS2 was derived from 15 335 patients in multicentre communities and hospitals, as was AISAPS (derived from 8820). We excluded Kwon's score because it involved the variable (mechanical ventilation) which we thought would be a medical intervention rather than the original character of the patient. We excluded the PANTHER-IS score because it focused on patients with IS in the middle cerebral artery territory in the neurological intensive care unit, which would be not suitable to common patients with stroke.
Statistical analysis
Given the non-normal distribution of data, median values with 25th and 75th centiles were calculated for continuous variables and comparisons between groups were performed using the Nonparametric test. Categorical variables were described as counts and percentages, while comparisons between two groups were made with the χ2 test.
The model fit was measured with the Hosmer-Lemeshow goodness-of-fit statistic. The discriminatory ability of each risk model was evaluated using the area under the receiver operating characteristic curve (c statistic). The c statistic for each pair of scores was compared with the DeLong method.
We considered a p value <0.05 (2-tailed) to be statistically significant. All analyses were performed using MedCalc.
Results
Between January 2011 and September 2013, 1814 patients with IS were consecutively referred to the hospital. Of them, 140 patients were with TIA, 2 patients younger than 18 years, 104 patients with stroke onset more than 1 month or with pneumonia prior to admission. A total of 1569 patients were finally included in the present study. The median age was 65 years (IQR, 25–75), and 60% were male. Among all participants, 240 (15.3%) patients acquired pneumonia during hospitalisation (table 1).
Table 1.
Baseline characteristics of patients with and without stroke-associated pneumonia
| Pneumonia group | Non-pneumonia group | p Value | |
|---|---|---|---|
| Age, median (IQR), year | 70.0 (61.0, 78.0) | 64.0 (53.0, 73.0) | <0.001* |
| Sex | 0.205 | ||
| Male, n, % | 134 (55.8) | 800 (60.2) | |
| Female, n, % | 106 (44.2) | 529 (39.8) | |
| Hypertension, n, % | 141 (58.8) | 764 (57.5) | 0.715 |
| Diabetes mellitus, n, % | 71 (29.6) | 371 (27.9) | 0.597 |
| Hyperlipaemia, n, % | 16 (38.1) | 142 (10.7) | 0.057 |
| History of stroke or TIA, n, % | 42 (17.5) | 169 (12.7) | 0.046* |
| Valvular heart disease, n, % | 2 (0.8) | 7 (0.5) | 0.563 |
| Coronary heart disease, n,% | 24 (10.0) | 58 (4.4) | <0.001* |
| Atrial fibrillation, n, % | 72 (30.0) | 109 (8.2) | <0.001* |
| Congestive heart failure, n, % | 3 (1.3) | 1 (<0.1) | <0.001* |
| COPD, n, % | 15 (6.3) | 16 (1.2) | <0.001* |
| History of pneumonia, n,% | 17 (7.1) | 50 (3.8) | 0.019* |
| Current smoking, n, % | 33 (13.8) | 148 (11.1) | 0.243 |
| Excess alcohol consumption, n, % | 25 (10.4) | 156 (11.7) | 0.555 |
| Found down at onset, n, % | 42 (17.5) | 80 (6.0) | <0.001* |
| Dyaphasia, n, % | 130 (9.8) | 272 (20.5) | <0.001* |
| NIHSS, median (IQR) score | 9.5 (5, 14.0) | 3.0 (1.0, 7.0) | <0.001* |
| GCS, median (IQR) score | 14.0 (11.0, 15.0) | 15.0 (14.0,15.0) | <0.001* |
| OCSP | <0.001* | ||
| Lacunar infarction, n, % | 7 (2.9) | 66 (4.9) | |
| Total anterior circulation infarct, n, % | 39 (16.3) | 61 (4.6) | |
| Partial anterior circulation infarct, n, % | 165 (68.8) | 939 (70.7) | |
| Posterior circulation infarct, n, % | 29 (12.1) | 263 (19.8) | |
| Glu, median (IQR) mmol/L | 6.73 (5.20, 8.60) | 5.64 (4.87, 7.13) | <0.001* |
| WCC, median (IQR) 1012/L | 8.34 (6.55, 10.71) | 6.69 (5.44, 8.22) | <0.001* |
*p<0.05.
COPD, chronic obstructive pulmonary disease; GCS, Glasgow Coma Scale; NIHSS, National Institutes of Health Stroke Scale; OCSP, Oxfordshire Community Stroke Project; TIA, transient ischaemic attack; WCC, white cell count.
Table 2 demonstrated the items of each risk scoring model included in the study. The abilities of these three risk models of predicting pneumonia are summarised in table 3. The discriminatory abilities of all models were described as the area under the receiver operating characteristic curve (AUROC), which is shown in figure 1. The AUROC of all models was 0.659 (p <0.001, Chumbler's score), 0.728 (p <0.001, A2DS2) and 0.758 (p<0.001, AISAPS), respectively. The differences in AUROC (ΔAUROC) between any two risk models were significant (p<0.05). The calibrations of all models assessed by the Hosmer-Lemeshow test were not significant (p>0.05). Although the incidence of SAP was underpredicted by all of these risk models, the predicted incidence of AISAPS was closer to the observed incidence (table 4).
Table 2.
The content of each risk model
| Number | Chumbler's score (5 items) | A2DS2 (5 items) | AISASP (11 items) |
|---|---|---|---|
| 1 | age | age | age |
| 2 | dysphagia | dysphagia | dysphagia |
| 3 | NIHSS | NIHSS | NIHSS |
| 4 | found down at onset | atrial fibrillation | atrial fibrillation |
| 5 | history of pneumonia | male | GCS |
| 6 | – | – | congestive heart failure |
| 7 | – | – | COPD |
| 8 | – | – | current smoker |
| 9 | – | – | prestroke dependence |
| 10 | – | – | OCSP subtype |
| 11 | – | – | admission glucose |
COPD, chronic obstructive pulmonary disease; GCS, Glasgow Coma Scale; OCSP, Oxfordshire Community Stroke Project; NIHSS, National Institutes of Health Stroke Scale.
Table 3.
Comparison of various risk models for stroke-associated pneumonia
| Risk models | OR | p Value | Hosmer-Lemeshow test | AUROC | p Value |
|---|---|---|---|---|---|
| Chumbler's score | 1.8138 | <0.0001 | p=0.6243 | 0.659 | <0.001 |
| A2DS2 | 1.4248 | <0.0001 | p=0.0953 | 0.728 | <0.001 |
| AISAPS | 1.2365 | <0.0001 | p=0.8927 | 0.758 | <0.001 |
AUROC,area under the receiver operating characteristic curve.
Figure 1.
The AUROC of each risk model. (A). The AUROC of Chumbler's score; (B). The AUROC of A2DS2; (C). The AUROC of AISAPS; (D). The comparison of AUROC of three risk models. AUROC, area under the receiver operating characteristic curve.
Table 4.
Observed and predicted rate of stroke-associated pneumonia
| Incidence of SAP (%) | |
|---|---|
| Observed | 15.3 |
| Predicted by | |
| Chumbler's score | 10.7 |
| A2DS2 | 10.8 |
| AISAPS | 11.7 |
SAP,stroke-associated pneumonia.
The optimal cut-off value of risk models for SAP is displayed in table 5. The optimal cut-off value was 1 for Chumbler's score, whose sensitivity was 71.49%, specificity was 52.93%, positive predictive value (PPV) was 21.66% and negative predictive value (NPV) was 91.07%. The optimal cut-off value was 3 for A2DS2, whose sensitivity was 69.83%, specificity was 65.56%, PPV was 26.96% and NPV was 92.27%. The optimal cut-off value was 6 for AISAPS, whose sensitivity was 76.45%, specificity was 62.56%, PPV was 27.19% and NPV was 93.59%.
Table 5.
Optimal cut-off value of risk models for stroke-associated pneumonia
| Y-index | Cut-off point | Sensitivity (%) | Specificity (%) | PPV | NPV (%) | |
|---|---|---|---|---|---|---|
| Chumbler's score | 0.2442 | 1 | 71.49 | 52.93 | 21.66 | 91.07 |
| A2DS2 | 0.3540 | 3 | 69.83 | 65.56 | 26.96 | 92.27 |
| AISAPS | 0.3900 | 6 | 76.45 | 62.56 | 27.19 | 93.59 |
NPV, negative predictive value; PPV, positive predictive value.
Discussion
We found that the Chumbler score, A2DS2 and AISAPS models could reasonably predict pneumonia after IS in the Chinese population. Among them, the A2DS2 and AISAPS models (whose AUROC was higher than 0.7) had acceptable discriminatory abilities. The discriminatory ability of every risk model was lower in this external validation cohort than in their derivation cohort, which was consistent with the AISAPS study. The AISAPS study developed the AISAPS model to predict pneumonia after IS, as well as compared it with other risk models. They found Chumbler's score, A2DS2 and AISAPS model had acceptable discriminatory abilities in derivation and validation cohort, except Kwon's score only had acceptable discriminatory ability in derivation cohort. Chumbler's score had acceptable discriminatory ability in the validation cohort, which differed from this study. In addition, the AISAPS study did not report the difference in other risk models except AISAPS.
We found that AISAPS had the highest discriminatory ability, while Chumbler's score had the lowest discriminatory ability, which is consistent with the AISAPS study. The AISAPS model may be the most effective one for Chinese patients with IS. Further study is needed to explore the predictive performance of the AISAPS model in other races and regions.
We also analysed the independent risk factors of SAP in our cohort to investigate the reasons why these risks models had different predictive performances in this study. We found that age (OR=1.035, 95% CI 1.021 to 1.049), atrial fibrillation (OR=2.733, 95% CI 1.837 to 4.067), chronic obstructive pulmonary disease history (OR=5.006, 95% CI 2.143 to 11.693), NIHSS score (OR=1.124, 95% CI 1.092 to 1.157), dysphagia (OR=2.908, 95% CI 1.092 to 1.157) and the count of first time white cell count during hospitalisation (OR=1.169, 95% CI 1.116 to 1.224) were independent risk factors for SAP. The independent risk factors in our cohort had the biggest overlap with the items of AISASP, and smallest overlap with Chumbler's score. This may explain the differences of predictive performances between these risk models.
In diverse populations, the predictive performances of risk models could be different. So they need to be externally validated before they are used in clinic. As recommended in the guideline about clinical decision rules, the rules at level 1 must be prospectively validated in different populations and have one impact analysis demonstrating change in the clinician's behaviour with beneficial consequences, so that rules at level 1 can be used in a wide variety of settings.17
There are some limitations in this study. First, our study was a retrospective study with potential confounding factors. Second, we only included patients in a single hospital, which could cause selected bias.
For now, whether the clinical application of these risk models can assist in decreasing the incidence of SAP is still uncertain. Prospective studies with a large sample containing different races and a regional population are needed.
Footnotes
Contributors: SG and ZZ drafted the manuscript. ZL and NC retrieved the data. JG and MZ performed the statistical analysis. LH critically revised the manuscript.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: No additional data are available.
References
- 1.Emsley HCA, Hopkins SJ. Acute ischaemic stroke and infection: recent and emerging concepts. Lancet Neurol 2008;7:341–53. doi:10.1016/S1474-4422(08)70061-9 [DOI] [PubMed] [Google Scholar]
- 2.Westendorp WF, Nederkoorn PJ, Vermeij JD, et al. . Post-stroke infection: a systematic review and meta-analysis. BMC Neurol 2011;11:110 doi:10.1186/1471-2377-11-110 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Teramoto S. Novel preventive and therapuetic strategy for post-stroke pneumonia. Expert Rev Neurother 2009;9:1187–200. doi:10.1586/ern.09.72 [DOI] [PubMed] [Google Scholar]
- 4.Koennecke HC, Belz W, Berfelde D, et al. . Factors influencing in-hospital mortality and morbidity in patients treated on a stroke unit. Neurology 2011;77:965–72. doi:10.1212/WNL.0b013e31822dc795 [DOI] [PubMed] [Google Scholar]
- 5.Katzan IL, Dawson NV, Thomas CL, et al. . The cost of pneumonia after acute stroke. Neurology 2007;68:1938–43. doi:10.1212/01.wnl.0000263187.08969.45 [DOI] [PubMed] [Google Scholar]
- 6.Westendorp WF, Vermeij JD, Zock E, et al. . The Preventive Antibiotics in Stroke Study (PASS): a pragmatic randomised open-label masked endpoint clinical trial. Lancet 2015;385:1519–26. doi:10.1016/S0140-6736(14)62456-9 [DOI] [PubMed] [Google Scholar]
- 7.Kwon HM, Jeong SW, Lee SH, et al. . The pneumonia score: a simple grading scale for prediction of pneumonia after acute stroke. Am J Infect Control 2006;34:64–8. doi:10.1016/j.ajic.2005.06.011 [DOI] [PubMed] [Google Scholar]
- 8.Harms H, Grittner U, Droge H, et al. . Predicting post-stroke pneumonia: the PANTHERIS score. Acta Neurol Scand 2013;128:178–84. doi:10.1111/ane.12095 [DOI] [PubMed] [Google Scholar]
- 9.Chumbler NR, Williams LS, Wells CK, et al. . Derivation and validation of a clinical system for predicting pneumonia in acute stroke. Neuroepidemiology 2010;34:193–9. doi:10.1159/000289350 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hoffmann S, Malzahn U, Harms H, et al. . Development of a clinical score (A2DS2) to predict pneumonia in acute ischemic stroke. Stroke 2012;43:2617–23. doi:10.1161/STROKEAHA.112.653055 [DOI] [PubMed] [Google Scholar]
- 11.Ji R, Shen H, Pan Y, et al. . Novel risk score to predict pneumonia after acute ischemic stroke. Stroke 2013;44:1303–9. doi:10.1161/STROKEAHA.111.000598 [DOI] [PubMed] [Google Scholar]
- 12.Ji R, Shen H, Pan Y, et al. . Risk score to predict hospital-acquired pneumonia after spontaneous intracerebral hemorrhage. Stroke 2014;45:2620–8. doi:10.1161/STROKEAHA.114.005023 [DOI] [PubMed] [Google Scholar]
- 13.Brott T, Adams HP Jr, Olinger CP, et al. . Measurements of acute cerebral infarction: a clinical examination scale. Stroke 1989;20:864–70. [DOI] [PubMed] [Google Scholar]
- 14.Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scal. Lancet 1974;2:81–4. [DOI] [PubMed] [Google Scholar]
- 15.Bamford J, Sandercock P, Dennis M, et al. . Classification and natural history of clinically identifiable subtypes of cerebral infarction. Lancet 1991;337:1521–6. [DOI] [PubMed] [Google Scholar]
- 16.Chamorro A, Meisel C, Prass K, et al. . Preventive antibacterial therapy in acute ischemic stroke: a randomized controlled trial. PLoS ONE 2008;3:e2158 doi:10.1371/journal.pone.0002158 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.McGinn TG, Guyatt GH, Wyer PC, et al. . Users’ guides to the medical literature XXII: how to use articles about clinical decision rules. Evidence-based medicine working group. JAMA 2000;284:79–84. [DOI] [PubMed] [Google Scholar]