Development of the ESEx index: a tool for predicting risk of recurrent severe COPD exacerbations

Abstract

Background:

In chronic obstructive pulmonary disease (COPD), multiple recurrent severe exacerbations that require hospitalization can occur. These events are strongly associated with death and other clinical complications.

Objectives:

We aimed to develop a prognostic model that could identify patients with COPD that are at risk of multiple recurrent severe exacerbations within 3 years.

Design:

Prospective cohort.

Methods:

The derivation cohort comprised patients with stable, moderate-to-severe COPD. Multivariable logistic regression analyses were performed to develop the final model. Based on regression coefficients, a simplified index (ESEx) was established. Both, model and index, were assessed for predictive performance by measuring discrimination and calibration.

Results:

Over 3 years, 16.4% of patients with COPD experienced at least three severe recurrent exacerbations. The prognostic model showed good discrimination of high-risk patients, based on three characteristics: the number of severe exacerbations in the previous year, performance in the five-repetition sit-to-stand test, and in the 6-minute-walk test. The ESEx index provided good level of discrimination [areas under the receiver operating characteristic curve (AUCs): 0.913].

Conclusions:

The ESEx index showed good internal validation for the identification of patients at risk of three recurrent severe COPD exacerbations within 3 years. These tools could be used to identify patients who require early interventions and motivate patients to improve physical performance to prevent recurrent exacerbations.

Introduction

Acute exacerbations that require hospitalization (here, called severe exacerbations) occur frequently in chronic obstructive pulmonary disease (COPD). ¹ Recurrent hospitalizations due to COPD exacerbations have been associated with disease progression, mortality, and consequently, an increase in costs to the health system.^2,3 Therefore, the abilities to predict recurrent hospitalizations in COPD and assess risk factors are important in COPD management.^2,4–6

Two systematic reviews have described the substantial efforts made in developing prognostic models for predicting future COPD exacerbations.^7,8 Among all the potential risk factors, a history of exacerbations in the previous year was the most reliable for predicting future exacerbations.^5,6,9 Consequently, the recommended criterion for identifying high-risk patients is either ⩾1 severe exacerbation (which requires hospitalization) or ⩾2 exacerbations that do not require hospitalization. ² That criterion was systematically included in recent statistical prognostic models for predicting one severe COPD exacerbation.^6,10

A relevant fraction of patients who experience one severe exacerbation will experience additional, recurrent severe COPD exacerbations within a relatively short time frame.^4,5 Nevertheless, specific prognostic factors associated with recurrent severe exacerbations have not been sufficiently explored.^4,5 Moreover, most studies on prognostic factors for severe exacerbations have focused on one future hospitalization, regardless of whether the exacerbation was recurrent.^6–8,10 Our research group was previously interested in prognostic models for predicting the risk of at least one severe COPD exacerbation. ¹⁰ However, in this study, we aimed to identify prognostic factors for recurrent COPD exacerbations.

This study aimed to develop a readily applicable prognostic model that could correctly discriminate stable patients with moderate-to-very severe COPD who are likely to experience multiple recurrent severe exacerbations within 3 years.

Methods

Study design and participants

The prognostic model was developed with data from an existing prospective cohort study which included outpatients with COPD who attended regular medical visits at the Meseguer Hospital, Murcia, Spain during 2014. The baseline data included tests and measurements of potential prognostic factors. Patients were followed for 3 years (from their recruitment in 2014), and severe COPD exacerbations were recorded during this time. All participants provided written informed consent. The study was approved by the Ethics Committee of the hospital (approval number: EST-35/13).

Study inclusion criteria were (1) a diagnosis of moderate-to-very severe COPD, according to the Global Initiative for COPD recommendations (i.e. the ratio of forced expiratory volume to forced vital capacity, post-bronchodilator [FEV/FVC] <0.7 and percentage of FEV₁ post-bronchodilator <80%), ² (2) a stable COPD stage (without exacerbations during the previous 3 months), and (3) age 40 to 80 years. Exclusion criteria were (1) unstable cardiac condition within 4 months of the study start, (2) cognitive deterioration, or (3) inability to walk.

Predictive outcome

The main outcome was at least three recurrent severe COPD exacerbations during the follow-up period. Time (months) from the recruitment to this outcome was also recorded. For both cohorts, a severe COPD exacerbation was defined as an increase in at least two of three specified symptoms (breathlessness, sputum volume, and sputum purulence) that required an urgent visit to the emergency department and hospital admission.^2,3 Hospitalizations were captured from the patients’ electronic files and checked by physicians to ensure the exact date of exacerbation. Hospital readmissions between 0 and 29 days after hospital discharge were not considered recurrent exacerbations, because it can take 4 weeks to recover completely from an exacerbation. ¹¹

Candidate predictors

We selected 13 candidate predictors that had been associated, in previous studies, with future acute exacerbations of COPD. We only included predictors available in outpatient clinical settings. These included sociodemographic, clinical, pulmonary, and functional measurements. Sociodemographic characteristics included age (years) and sex. The clinical and pulmonary measures were current smoking status (yes/no); body mass index (kg/m²); the number of comorbidities (measured with a functional comorbidity index); ¹² history of heart disease (including angina pectoris, myocardial infarction, cardiac pathology, and ischemic or congestive heart disease); history of vascular disease (including stroke or peripheral vascular disease); the number of severe (those that required hospitalization) COPD exacerbations in the previous year; the grade of dyspnoea (measured with the modified British Medical Research Council [mMRC] scale); ¹³ the COPD Assessment Test (CAT™) score; ¹⁴ the use of inhaled corticosteroids (yes/no); the FEV₁, measured with post-bronchodilator spirometry (MasterScope Spirometer, version 4.6; Jaeger, Würzburg, Germany), according to the American Thoracic Society guidelines; ¹⁵ and finally, the index GOLD stage. The functional measures were a five-repetition, sit-to-stand (5-STS) test, which is the time taken to stand five times from a sitting position as rapidly as possible; and the distance covered in a 6-minute-walk test (6MWT). Both latter tests were measured as described previously. ¹⁰

Sample size calculation

The sample size calculation was based on the criteria proposed by Riley et al., ¹⁶ which aims to minimize overfitting and to ensure precise estimation of key parameters in the prediction model. A minimum sample size of 143 participants was required assuming a prevalence of 10% of patients that experienced the event, ⁴ a shrinkage factor of 0.9 (i.e. a relatively small amount of overfitting equal to 10%), a Cox–Snell R² of 0.26 (i.e. a realist Nagelkerke slightly ⩾50%), and a maximum of four parameters. This R² was assumed because previous models about hospitalized recurrence were not found and our predictors could be directly related to the event. The number of parameters was regarded as possible because we performed a demanding selection of the candidate prognostic factors adjusting them by the number of exacerbations in the previous year.

Data analysis

We used the Student t test or Mann–Whitney U test and chi-square tests to compare baseline characteristics between participants with and without a complete follow-up data in order to assess the risk of attrition bias.

We performed an exploration of candidate prognostic factors by multivariable logistic regression analyses in order to preselect predictors for the prognostic model. For that predictor preselection, we examined associations between exacerbation recurrences and each candidate predictor, adjusted for the total number of exacerbations in the previous year. To prevent omission bias, factors with significance level p ⩽ 0.156 were retained and used for the model development.

Model development, internal validation, and predictive performance

A multivariable backward logistic regression analysis was performed. The final model included only variables that showed p values ⩽0.05. Collinearity statistics of tolerance were calculated to prevent spurious relationships. Furthermore, because prognostic models typically show overly optimistic predictive performance in data derivations, ¹⁷ we performed an internal validation with bootstrapping techniques (500 bootstrap samples) to assess the potential impact of overfitting data in the final model and, when necessary, we adjusted for over-optimism. In addition, we examined the shrinkage factor, and a value below 0.85 indicated overfitting. ¹⁸

We assessed the predictive performances of the model by measuring discrimination, overall performance (R²), and calibration. Discrimination was estimated by the area under the receiver operating characteristic curve (AUC). Calibration was evaluated by the mean calibration (comparing the average predicted risk with the overall event rate), with the calibration intercept and calibration slope of the model, and with a calibration loess curve in the calibration plot. ¹⁹

No imputations were performed for missing values from patients unable to complete any measurement or test (e.g. an incomplete 6MWT or zero repetitions of the 5-STS test). However, for patients with one or more repetitions of the 5-STS test, we imputed an estimated time, as recommended, ²⁰ and as described elsewhere. ²¹

Model presentation

We generated an easy-to-use point assignment system, which was generated by applying regression coefficients to categories of each variable, as described previously, ²² and defined as an index. To calibrate the point system, we estimated the risks of frequent severe COPD exacerbations, and compared them with corresponding scores estimated with the regression equation. Moreover, scores with similar probability were grouped in scales exclusively to describe the time-to-event with a Kaplan–Meier survival curve, and these curves were compared with the log-rank test.

All analyses were performed with SPSS statistical software (SPSS version 24.0; IBM SPSS, Chicago, IL, USA).

Results

Participants

The derivation cohort initially included 131 patients. Of these, nine were lost during the 3-year follow-up (eight died and one dropped out). The baseline characteristics were similar between the lost patients and those who completed the follow-up (Table 1). The 122 followed patients performed all tests, except 3, who did not complete the 6MWT (Figure 1). Of the 122 patients, 20 (16.4%) had ⩾3 severe COPD exacerbations, and of these, 18 (90%) were in a higher GOLD stage. Participants with ⩾3 severe COPD exacerbations were older and had worse history of heart disease, pulmonary characteristics, and performances on the two functional tests, compared with patients with <3 severe COPD exacerbations (Table 1).

Table 1.

Baseline characteristics.

Characteristics	All participants N = 131	Followed N = 122	Lost N = 9	p value
Sociodemographic variables
Age, years	66.82 (± 8.40)	66.71 (± 8.33)	68.33 (± 9.80)	0.579
Males	116 (88.50%)	108 (88.52%)	8 (88.89%)	0.974
Clinical and pulmonary variables
Current smoker	40 (30.50%)	35 (28.69%)	5 (55.56%)	0.091
Body mass index, kg/m2	28.94 (± 5.15)	29.03 (± 5.12)	27.70 (± 5.75)	0.458
Comorbidity index	3.00 (3.00)	3.14 (± 1.67)	3.56 (± 1.74)	0.473
Diabetes, yes	31 (23.70%)	31 (25.41%)	0 (0.00%)	0.083
Depression, yes	16 (12.20%)	15 (12.30%)	1 (11.11%)	0.917
History of heart disease, yes	26 (19.80%)	23 (18.85%)	3 (33.33%)	0.293
History of vascular disease, yes	9 (6.90%)	9 (7.38%)	0 (0.00%)	0.398
Number of severe exacerbations in the previous year	1.00 (1.00)	0.69 (± 0.68)	1.11 (± 0.78)	0.068
mMRC; scale dyspnoea
0–1	82 (62.60%)	77 (63.11%)	5 (55.56%)	0.843
2	33 (25.20%)	30 (24.59%)	3 (33.33%)
3–4	16 (12.20%)	15 (12.30%)	1 (11.11%)
CAT score; range: 0–40	13.00 (9.00)	14.30 (7.50)	15.44 (± 4.85)	0.652
Inhaled corticosteroids use, yes	114 (87.00%)	107 (87.70%)	7 (77.78%)	0.392
FEV1, % predicted	48.84 (± 15.09)	48.66 (± 14.96)	51.22 (± 17.46)	0.625
GOLD stage
A	21 (16.00%)	21 (17.21%)	0 (0.00%)	0.372
B	21 (16.00%)	19 (15.57%)	2 (22.22%)
C	11 (8.40%)	11 (9.02%)	0 (0.00%)
D	78 (59.50%)	71 (58.20%)	7 (77.78%)
Functional tests
6MWT, m	362.00 (88.50)	348.61 (± 87.79)	345.78 (± 37.70)	0.924
5-STS test, s	14.63 (4.52)	16.56 (± 9.03)	18.88 (± 11.29)	0.466

5-STS, five-repetition sit-to-stand test; s, seconds; 6MWT, 6-minute walking test; m, meters; CAT, chronic obstructive pulmonary disease assessment test; FEV₁, forced expiratory volume in 1 s; GOLD, Global Initiative for Chronic Obstructive Lung Disease; IQR, interquartile range; kg, kilograms; m², square meter; mMRC, modified British Medical Research Council.

Data are reported as mean (±standard deviation), median (IQR), or n (%).

Figure 1.

Participants flow diagram for inclusion in the prognostic model.

6MWT, six-minute walking test

Model development and predictive performance

Table 2 shows the prognostic factors that were preselected. Age, history of heart disease, and GOLD stage were excluded. It also shows intercept and the regression coefficients of the predictors that were identified in the final multivariable model. No collinearity was identified among the covariates used in these analyses (all statistics of tolerance were ⩽2, below the threshold of 10 used to indicate potential collinearity). According to this model, the risk of three recurrent exacerbations increased with the number of exacerbations in the previous year and with the time required to complete the 5-STS. The risk decreased as the distance walked in the 6MWT. Three other predictors were initially preselected (CAT, FEV₁%, and dyspnoea), but they were finally excluded. The final model significantly fit the data, χ²(df = 3, N = 119) = 41.35, p < 0.001. The Cox–Snell R² was 0.29, the Nagelkerke R² was 0.50, and the AUC was 0.913 [95% confidence interval (CI): 0.86–0.97]. Its shrinkage factor (0.9) and the estimated model with bootstrapping techniques indicated that it was not over-optimistic, and it had the same AUC after bootstrapping. Thus, no adjustment was applied for overfitting. A post hoc sensitivity analysis showed that AUCs were lower with the no inclusion of both 5-STS and 6MWT (0.770; 95% CI: 0.67–0.87) or no inclusion of the 5-STS (0.895; 95% CI: 0.84–0.95), implying that the three predictors added discrimination. With respect to calibration, the average estimated risk given by model was 0.159 (i.e. 15.9%), which indicated no tendency to give underestimated or overestimated risks in general. The calibration intercept was −0.06 (95% CI: −0.89 to 0.77), supporting non-underestimation. The calibration slope was 1.01 (95% CI: 0.57–1.44), which suggests that risk estimates were not systematically too extreme or moderate (Figure 2). The calibration plot showed the observed incidence and predicted risk data were in general agreement, but a weak level of overestimation of risks across the higher range of observed risks.

Table 2.

Summary of logistic regression analyses for identifying sociodemographic, clinical, pulmonary, and functional characteristics that might predict three recurrent hospitalized COPD exacerbations.

	Adjusted models a	Final multivariable model
Characteristics b	β (SE)	β (SE)
Clinical and pulmonary variables
Number of severe exacerbations in the previous year	1.90 (0.49)*	1.49 (0.61)*
mMRC; scale dyspnoea – per increase to next stage (reference 0–1)
2	0.98 (0.64)	–
3–4	1.42 (0.72)*	–
CAT score; range: 0–40	0.13 (0.04)*	–
FEV1, % predicted	0.07 (0.03)*	–
Functional tests
6MWT, m	−0.01 (0.00)*	−0.01 (0.00)*
5-STS test, s	0.11 (0.04)*	0.10 (0.04)*
Intercept	–	−1.86 (1.51)
R 2	–	0.502

5-STS, five-repetition sit-to-stand test; s, seconds; 6MWT, 6-minute walking test; m, meters; CAT, chronic obstructive pulmonary disease assessment test; FEV₁, forced expiratory volume in 1 s; SE, standard error; β, regression coefficient.

^aAdjusted by exacerbations in the previous year.

^bOnly significant variables in any model are shown.

^*p < 0.05.

Figure 2.

Calibration plots of the regression equation of the final model.

The dotted line is the calibration curve, which shows the actual relation between observed and predicted risks and the solid line represents the perfect calibration line.

The point system developed from the final model is shown in Table 3(a). The scores are assigned to categories of each predictor and the total score ranged from 0 to 6. This index represents exercise capacity, strength, and previous exacerbations (thus, called the ESEx index). Table S1 (Appendix) shows how it was developed. Table 3(b) presents probabilities of the outcome that correspond to the points total. For example, an individual with one severe exacerbation in last year (1 point), distance of 250 m in the 6MWT (1 point), and 18 s in the 5-STS (1 point) has a point total of 3; this corresponds to a probability of recurrent severe exacerbation of 0.43 at 3 years.

Table 3.

Point score system (a) and probability (b) of having at least three recurrent severe COPD exacerbations during the following 3 years.

(a)
Predictor categories	Points
Number of severe exacerbations in the previous year
0	0
1	1
2	2
6MWT, m
⩾350	0
225–349.9	1
⩽224.9	2
5-STS test, s
⩽15	0
15.1-24.9	1
⩾25	2
(b)
Point total	Probability
0	0.009
1	0.038
2	0.148
3	0.433
4	0.769
5	0.934
6	0.980

COPD, chronic obstructive pulmonary disease; 5-STS, five-repetition sit-to-stand test; s, seconds; 6MWT, 6-minute walking test.

The Kaplan–Meier plots (Figure 3) show how the different scales [i.e. patients with high (>3), middle,^2,3 and low (<2) scores] were associated with the time-to-event, χ²(2) = 47.30, log rank; p < 0.001.

Figure 3.

Kaplan–Meier survival curves of all patients divided into groups, from the index scales.

Discussion

Importantly, previous prognostic models for COPD exacerbations have often omitted calibration and simplified model presentations (e.g. nomograms, point-systems).^7,8 In this study, we developed a prognostic model that could identify patients at risk of three recurrent severe COPD exacerbations within 3 years. We evaluated this model with internal validation. The risk estimated with the regression equation model was well calibrated and showed reasonable discriminatory ability, which indicated that recurrent exacerbations could be predicted, based on simple clinical and functional measures. Moreover, we generated an associated point-system, called ESEx with predictive value for the risk of recurrence (i.e. the third severe exacerbation).

Previously, a review by Bertens et al. ²³ described a prognostic model for COPD exacerbations with the lowest risk of bias. ⁸ Similarly, to create our prognostic model, we used logistic regression modeling, instead of Cox regression modeling, because we wanted to identify patients with recurrences, regardless of whether they occurred early or late. We used 3 years as the time frame, consistent with the ECLIPSE study, ⁴ and our outcome measure was three hospitalizations due to severe COPD exacerbations. In both studies, the percentage of patients with that level of recurrences strongly increased with the GOLD stage.

Our findings suggested that a history of exacerbations in the previous year and performance on two functional tests (6MWT and 5-STS) were independently associated with recurrent severe COPD exacerbations. As expected, exacerbations in the previous year were the strongest predictors of recurrences in our model, similar to previously reported findings on recurrences of any level of exacerbation.^9,23 Multiple mechanisms have been proposed to explain the relevance of a history of exacerbations. ⁴ Mechanisms have included muscle weakness and reduced exercise capacity, induced by systemic steroids or physical inactivity after hospitalization for an exacerbation.^4,24,25 Although low exercise tolerance and muscle strength were often cited as consequences of a history of exacerbations, our study showed that these impairments were directly associated with recurrent exacerbations, independent of a history of exacerbations. A potential explanation for our finding might be that our participants were recruited during a phase of clinical stability. Thus, the level of residual functional performance in a stable state could be relevant, in addition to functional changes after a previous hospitalization. ²⁶

We found that patients with three recurrent hospitalizations had worse airflow limitation and CAT score and mMRC than those with fewer recurrences. However, these characteristics were not included in the final model. Some authors have suggested that the prognostic value of these characteristics is controversial,^5,27 due to inconsistent results.^4,5,9,28 In our study, these characteristics showed low significance. Thus, a potential explanation for excluding these characteristics from the model might be that they showed lower variability between subjects, because most of our patients were in higher GOLD stage (i.e. poor CAT score and low percent predicted FEV₁).

Implications for practice

The two functional tests were previously associated with at least one severe COPD exacerbation.^10,29,30 Our findings showed that those tests were also relevant to recurrent severe exacerbations. Thus, it is important to consider physical performance in assessing patients with COPD. ³¹ Our prognostic model and its associated point-system may assist clinicians in identifying high-risk patients who could benefit from initiating preventive strategies or treatments for improving functional performance, and thus, provide motivation for coping.

In addition, in response to previous recommendations,^7,8 we generated a point-system to assist in estimating risk without a calculator or computer. Our ESEx index-stratified survival curves indicated that patients with high scores should receive treatment strategies earlier, because they showed shorter times-to-recurrent hospitalizations. Furthermore, the point-system is useful in clinical setting, even when time is insufficient to perform the 6MWT. In these settings, the ESEx index score would only include the 5-STS score and the history of exacerbations. However, a score ⩾2 points (e.g. 5-STS time ⩾15 s and one previous exacerbation) would be sufficiently valid for identifying patients at risk of recurrence.

Strengths and limitations

This study had several strengths. First, we simultaneously developed and validated a prognostic model and an associated point-system. To our knowledge, while model presentations (e.g. point-system, normograms) are frequent in other medical areas, ³² they have not been described previously in prognostic research for recurrent severe COPD exacerbations. Second, we examined the association between scales of the point-system and the time to the third hospitalization. Previous studies on prognostic factors or models for recurrent COPD exacerbations did not explore this issue.^4,23 Furthermore, all preselected candidate predictors were adjusted for the 1-year history of exacerbations, which was previously shown to be a reliable predictor of subsequent exacerbations.^9,33

This study also had some limitations. First, although the model showed good discrimination and calibration, other prognostic factors could be omitted. In future, when additional strong, independent predictors of recurrent severe exacerbations are identified, we could add them to the ESEx index to increase its predictive capability. Second, we developed a new prognostic model. To improve the model’s generalizability, the validation should be performed by other investigators in other settings. Third, our model results were based on an overall incidence of 15%. In other settings, when the model is applied to discriminate patients with COPD at high risk of recurrence, if the overall incidence is very different from 15%, the intercept in Table 2, it should be adjusted with standard techniques. ³⁴ Fourth, our estimation of the minimum required sample size was not achieved in the analyses and we could not estimate (within a small margin of error) the intercept of the model. This problem increased when our overall recurrence proportion was 16% (i.e. 6% higher than estimated). Despite this requirement not being achieved, the final sample size allows us to minimize overfitting. ¹⁶ Finally, because authors had lower sample size to develop a model compared with what estimated, it is also important to note that you need more samples if a number of factor variables included in the model as you might end up with not enough events in a given factor category.

Conclusion

In conclusion, our newly developed prognostic model and ESEx index showed good internal validation for the identification of patients at risk of three recurrent severe COPD exacerbations within 3 years. Moreover, the ESEx index was associated with the time to the third recurrence. Our findings have important implications for the management and monitoring of patients with COPD with any number of exacerbations. These tools can be used to detect the need for early interventions and to motivate patients in maintaining physical performance to prevent recurrent exacerbations. Moreover, these tools provide more specific information on recurrences than previous prognostic models, which were designed to predict only one future exacerbation.

Take-home message.

Patients with ESEx index ⩾2 points have a higher risk of recurrent severe chronic obstructive pulmonary disease (COPD) exacerbations. The index score is based on the number of exacerbations in the previous year, the five-repetition sit-to-stand, and the 6-minute-walk tests.