Machine Learning and Prediction of All-Cause Mortality in COPD

Abstract

Background

COPD is a leading cause of mortality.

Research Question

We hypothesized that applying machine learning to clinical and quantitative CT imaging features would improve mortality prediction in COPD.

Study Design and Methods

We selected 30 clinical, spirometric, and imaging features as inputs for a random survival forest. We used top features in a Cox regression to create a machine learning mortality prediction (MLMP) in COPD model and also assessed the performance of other statistical and machine learning models. We trained the models in subjects with moderate to severe COPD from a subset of subjects in Genetic Epidemiology of COPD (COPDGene) and tested prediction performance in the remainder of individuals with moderate to severe COPD in COPDGene and Evaluation of COPD Longitudinally to Identify Predictive Surrogate Endpoints (ECLIPSE). We compared our model with the BMI, airflow obstruction, dyspnea, exercise capacity (BODE) index; BODE modifications; and the age, dyspnea, and airflow obstruction index.

Results

We included 2,632 participants from COPDGene and 1,268 participants from ECLIPSE. The top predictors of mortality were 6-min walk distance, FEV₁ % predicted, and age. The top imaging predictor was pulmonary artery-to-aorta ratio. The MLMP-COPD model resulted in a C index ≥ 0.7 in both COPDGene and ECLIPSE (6.4- and 7.2-year median follow-ups, respectively), significantly better than all tested mortality indexes (P < .05). The MLMP-COPD model had fewer predictors but similar performance to that of other models. The group with the highest BODE scores (7-10) had 64% mortality, whereas the highest mortality group defined by the MLMP-COPD model had 77% mortality (P = .012).

Interpretation

An MLMP-COPD model outperformed four existing models for predicting all-cause mortality across two COPD cohorts. Performance of machine learning was similar to that of traditional statistical methods. The model is available online at: https://cdnm.shinyapps.io/cgmortalityapp/.

FOR EDITORIAL COMMENT, SEE PAGE 846

COPD is one of the leading causes of mortality worldwide.¹ Improving performance of mortality prediction models can identify patients with COPD who might benefit from earlier or more specific intervention. One widely used mortality prediction tool is the BMI, airflow obstruction, dyspnea, and exercise capacity (BODE) index.² Studies that have modified the BODE index by varying the input^{3, 4, 5} or adding serum biomarkers⁶ have resulted in similar or slightly improved performance for mortality prediction. Other models include the dyspnea, airflow obstruction, smoking status, exacerbation frequency (DOSE) index⁷; the age, dyspnea, and airflow obstruction (ADO) index⁸; and the St. George’s Respiratory Questionnaire score, airflow limitation, and exercise tolerance (SAFE)⁹ index. A survival analysis in more than 3,500 subjects from 11 COPD cohorts compared predictive accuracy of several of these measures, finding that ADO, BODE, and BODE modifications performed best; after adjusting for age, BODE modifications outperformed ADO.¹⁰ A meta-analysis in 16,000 subjects from the COPD Cohorts Collaborative International Assessment (3CIA) initiative found that ADO and updated BODE showed a higher, but not statistically significantly different, area under the curve for predicting 3-year survival vs BODE.¹¹

These prediction models did not include quantitative CT imaging. Several quantitative CT imaging measures of airway,¹² emphysematous,¹³ vascular,^14,15 and interstitial¹⁶ abnormalities have been independently associated with morbidity and mortality in subjects with COPD. Evaluating many potential predictors is fraught with challenges, which may be addressed by machine learning algorithms that can learn rules from one data set that can be used to make predictions in another data set.¹⁷ Machine learning can be broadly defined; some consider traditional statistical methods such as logistic or Cox regression important machine learning tools,^{18, 19, 20, 21, 22} and in certain cases these perform as well as newer more advanced methods.²³ Machine learning algorithms have the potential to improve predictive modeling of health outcomes,²⁴ and digitalization of health records is facilitating incorporation into health care.²⁵ In COPD, machine learning has been used to identify which features are most important for case identification²⁶ and predicting exacerbations.²⁷ Machine learning methods were used to improve prediction of 5-year all-cause mortality in subjects undergoing CT coronary angiography²⁸ and cardiac motion MRI.²⁹

We hypothesized that machine learning methods applied to an expanded set of clinical and quantitative CT imaging features would be useful for identifying the most important predictors of all-cause mortality and improving mortality prediction in moderate to severe COPD compared with BODE, BODE modifications, and ADO. To test this hypothesis, we used data from subjects from the Genetic Epidemiology of COPD (COPDGene) and Evaluation of COPD Longitudinally to Identify Predictive Surrogate Endpoints (ECLIPSE) studies.

Materials and Methods

Study Participants

We included individuals with moderate to severe COPD (postbronchodilator FEV₁ < 80% predicted, FEV₁/FVC < 0.7; ie, Global Initiative for Chronic Obstructive Lung Disease [GOLD] 2-4 spirometry grades) from the COPDGene and ECLIPSE studies. We selected 30 clinical, spirometric, and imaging features as inputs for a random survival forest. To ensure that the same variables could be examined in both data sets, we selected a set of demographic, clinical, spirometric, and imaging features present in at least 80% of subjects in both data sets (e-Table 1). Notable imaging features included pulmonary artery-to-aorta (PA:A) ratio,³⁰ square root of wall area of a hypothetical airway with internal perimeter of 10 mm (Pi10),³¹ mean wall area percent,¹³ percent emphysema determined by the percent low attenuation area of the lungs < −950 Hounsfield units (% LAA < −950 HU),¹³ and 15th percentile of the lung density histogram on inspiratory scans.³² The primary outcome was time to death from any cause.

Study Design

To develop a mortality prediction model that balanced prediction and interpretability, we applied random survival forests (RSFs) for feature selection followed by Cox regression. We used the randomForestSRC R package³³ and obtained variable importance (VIMP) by means of the VIMP method.³⁴ We filtered for multicollinearity of the RSF features, removing collinear variables to ensure all variance inflation factors were < 10^35,36 and then applied Cox regression (survival R package).^37,38 As a reference for comparison, we used the BODE index,² exacerbations and BODE,³ updated BODE, and ADO.⁸ To understand the contribution of RSF to this approach, we also compared a range of feature selection methods and prediction models, including standard statistical methods (see the Methods in e-Appendix 1).

The accuracy of prediction models was assessed by using C indexes (Hmisc R package)^36,39 and receiver operating characteristic curves (ROCR R package).⁴⁰ The C indexes were compared with the compareC R package by using the one-shot method.⁴¹. For our main model, we developed a Web-based tool by using the Shiny R package. Additional details are available in the Methods in e-Appendix 1.

Results

Characteristics of Study Participants

Characteristics of subjects with complete data used in analysis are shown in Table 1. Compared with subjects in COPDGene, those in the ECLIPSE study had a longer median follow-up and lower BMI and FEV₁ %predicted, and ECLIPSE had a higher proportion of individuals who died. In addition, several quantitative CT imaging measures differed in ECLIPSE (e-Table 1, e-Fig 1). The BODE and the exacerbations and BODE indexes were higher in ECLIPSE, but median scores were the same between samples. The ADO index was higher in subjects in COPDGene.

Table 1.

Demographic Characteristics of Subjects in COPDGene and ECLIPSE Included in Analysis

Characteristic	COPDGene	ECLIPSE	P Value
No. of subjects	2,632	1,268	…
Sex, female, No. (%)	1,157 (44.0)	426 (33.6)	< .001
Age, mean (SD), y	63.58 (8.92)	63.51 (7.03)	.8
Race, African American, No. (%)	485 (18.4)	0 (0)	< .001
FEV1 % predicted, median (IQR)	52.00 (36.88-66.60)	45.95 (35-58.73)	< .001
FVC % predicted, mean (SD)	77.17 (16.76)	79.16 (19.82)	.001
GOLD spirometry grade, No. (%)			< .001
2	1,407 (53.5)	518 (40.9)
3	819 (31.1)	568 (44.8)
4	406 (15.4)	182 (14.4)
Pack-years cigarette smoking, median (IQR)	47.50 (36-68)	45 (32-60)	< .001
Current smoking, No. (%)	1,041 (39.6)	431 (34.0)	.001
Dead at 3 y, No. (%)	221 (8.4)	121 (9.5)	.3
Dead at 5 y No. (%)	454 (17.2)	238 (18.8)	.3
Dead at 8 y No. (%)	631 (24.0)	405 (31.9)	< .001
Total dead, No. (%)	631 (24.0)	405 (31.9)	< .001
Days followed up, median (IQR)	2,321 (2,043-2,652)	2,616 (1,110-2,924)	< .001
6-Min walk distance, mean (SD), ft	1,205.97 (392.28)	1,190.36 (389.77)	.2
BODE, median (IQR)	3.00 (1-4)	3.00 (2-5)	< .001
e-BODE, median (IQR)	3.00 (1-5)	3.00 (2-5)	< .001
Updated BODE, median (IQR)	3.00 (1-7)	3.00 (1-7)	.9
ADO, mean (SD)	4.60 (1.8)	3.84 (1.44)	< .001

Percentages do not necessarily total 100% because of rounding. ADO = age, dyspnea, and airflow obstruction; BODE = BMI, airflow obstruction, dyspnea, and exercise capacity; e-BODE = exacerbations and BODE; COPDGene = Genetic Epidemiology of COPD; ECLIPSE = Evaluation of COPD Longitudinally to Identify Predictive Surrogate Endpoints; GOLD = Global Initiative for Chronic Obstructive Lung Disease; IQR = interquartile range.

Development of a Mortality Prediction Model

A schematic of the study design is shown in Figure 1. We randomly divided 2,632 participants from COPDGene into training (n = 1,974 [479 deaths]) and testing (n = 658 [152 deaths]) samples (e-Table 1). We chose features present in at least 80% of the cohort (initial feature list in e-Table 2) and used RSFs to select features, identifying the components of the BODE score (BMI, FEV₁ % predicted, modified Medical Research Council dyspnea score, and exercise as assessed by the 6-min walk distance [6MWD]), as well as additional clinical (eg, age, diabetes), spirometric (FEV₁/FVC ratio, forced expiratory flow 25% to 75%), and imaging (Pi10, mean wall area percent, % LAA < −950 HU, PA:A ratio) features by using the training sample. Features are shown in order of importance in Figure 2. Before Cox regression, features displaying multicollinearity were excluded; the remaining features and associated hazard ratios are shown in Table 2. Using this subset of RSF-selected features, we developed a Cox regression model (further denoted as machine learning mortality prediction [MLMP] in COPD). The MLMP-COPD model had a C index of 0.74 on the testing sample, outperforming BODE, BODE modifications, and ADO (P < .05) (Fig 3A, e-Fig 2A). The regression coefficients for the MLMP-COPD model are shown in e-Table 3, and clinical interpretations of hazard ratios are shown in e-Table 4.

Figure 1.

Schematic of study design. A total of 2,632 participants in COPDGene were randomly split into training (n = 1,974) and testing (n = 658) data sets. A random survival forest algorithm was applied to the training data set, and features chosen by variable importance were used to develop a Cox regression model. Both models were tested in the testing data set of COPDGene and externally in a sample of participants in ECLIPSE (n = 1,268). COPDGene = Genetic Epidemiology of COPD; ECLIPSE = Evaluation of COPD Longitudinally to Identify Predictive Surrogate Endpoints; GOLD = Global Initiative for Chronic Obstructive Lung Disease; RSF = random survival forest.

Figure 2.

Variable importance based on RSF trained in subjects in COPDGene. These features were all used to develop a Cox regression model, and GOLD spirometry grade and FVC % predicted were removed for collinearity. 6MWD = 6-min walk distance; FEF_25%-75% = forced expiratory flow 25% to 75%; mMRC = modified Medical Research Council; PA:A = pulmonary artery to aorta; % LAA < −950 HU = percent emphysema determined by the percent low attenuation area of the lungs < −950 Hounsfield units; Pi10 = square root of wall area of a hypothetical airway with internal perimeter of 10 mm; SaO₂ = arterial oxygen saturation. See Figure 1 legend for expansion of other abbreviations.

Table 2.

Hazard Ratios for Mortality of Random Survival Forest-Selected Variables Used in MLMP-COPD (Cox Regression) Model Trained on Subjects in COPDGene

Feature	Unadjusted Hazard Ratio (95% CI)	P Value	Adjusted Hazard Ratio (95% CI)	P Value
6-Min walk distance, per 100 ft	0.85 (0.83-0.87)	< .0001	0.9 (0.87-0.93)	< .0001
FEV1 % predicted	0.96 (0.957-0.968)	< .0001	0.99 (0.974-0.997)	.01
Age	1.04 (1.03-1.05)	< .0001	1.04 (1.03-1.06)	< .0001
mMRC dyspnea score	1.49 (1.39-1.61)	< .0001	1.15 (1.05-1.25)	.002
FEV1/FVC ratioa	0.009 (0.005-0.02)	< .0001	0.1 (0.02-0.45)	.003
FEF25%-75%, L/min	0.14 (0.1-0.2)	< .0001	1.74 (1.02-2.98)	.04
Resting SaO2	0.9 (0.88-0.92)	< .0001	0.97 (0.948-0.995)	.02
Exacerbations per year	1.2 (1.14-1.28)	< .0001	1.03 (0.96-1.11)	.4
Pack-years cigarette smoking	1.01 (1.006-1.011)	< .0001	1.01 (1.002-1.008)	.0003
BMI, kg/m2	0.97 (0.96-0.99)	.00022	0.97 (0.95-0.99)	.001
Severe exacerbations	1.95 (1.6-2.4)	< .0001	1.33 (1.05-1.7)	.02
PA:A ratio	5.8 (3.01-11.3)	< .0001	2.7 (1.35-5.55)	.005
Pi10	5.5 (3.1-9.7)	< .0001	1.6 (0.8-3.3)	.2
Diabetes	1.4 (1.1-1.8)	.0072	1.3 (1.01-1.73)	.04
% LAA < −950 HU	1.03 (1.027-1.04)	< .0001	0.995 (0.98-1.004)	.3

Adjusted models include all covariates, and FVC and GOLD spirometry grade were removed for multicollinearity. FEF_25%-75% = forced expiratory flow 25% to 75%; MLMP = machine learning mortality prediction; mMRC = modified Medical Research Council; PA:A = pulmonary artery to aorta; % LAA < −950 HU = percent emphysema determined by the percent low attenuation area of the lungs < −950 Hounsfield units; Pi10 = square root of wall area of a hypothetical airway with internal perimeter of 10 mm; SaO₂ = arterial oxygen saturation. See Table 1 legend for expansion of other abbreviations.

^aFEV₁/FVC is reported as a ratio from 0 to 1 and not as a percentage.

Figure 3.

Receiver operating characteristic curve comparing the RSF-derived mortality prediction models with updated BODE and ADO. A, Models were trained in 75% of the COPDGene sample (n = 1,974) and tested in the remaining 25% of the COPDGene sample (n = 658). B, Models were trained in 75% of the COPDGene sample (n = 1,974) and tested in the ECLIPSE sample (n = 1,268). ADO = age, dyspnea, and airflow obstruction; BODE = BMI, airflow obstruction, dyspnea, and exercise capacity. See Figure 1 and 2 legends for expansion of other abbreviations.

To test mortality prediction in an external validation sample, we applied the MLMP-COPD model to 1,268 participants in ECLIPSE (Table 1, e-Table 1). As expected, the performance of the models decreased in external validation (C index, 0.7) but retained improved prediction compared with BODE, BODE modifications, and ADO (P < .05 for all) (Fig 3B, e-Fig 2B). We repeated our analyses after normalizing imaging features to the mean values across matched subsets of individuals from the COPDGene and ECLIPSE studies in an attempt to reconcile differences in mean values and distributions of imaging features (see the Methods in e-Appendix 1). Our results were not substantially different (e-Fig 3) from those in the original model. To examine calibration, we plotted expected vs actual survival as shown in e-Figure 4 and tested for evidence of miscalibration by using the Greenwood-Nam-D'Agostino test.⁴² As shown in e-Table 5, miscalibration evaluated using the Greenwood-Nam-D'Agostino test for the MLMP-COPD model was not statistically significant, which was in contrast to results for the BODE score also shown in e-Table 5.

As all of these measured features may have variable availability in clinical practice, we calculated the performance of prediction models excluding select features. A Cox model built after excluding imaging features still improved prediction compared with updated BODE and ADO on the ECLIPSE validation sample (P < .05) (Fig 4A). The regression coefficients for the Cox model without imaging features are shown in e-Table 6. Mortality prediction models of individual quantitative imaging features added to BODE were not superior to updated BODE or ADO (e-Table 7).

Figure 4.

Receiver operating characteristic curve comparing the RSF-derived mortality prediction model with updated BODE and ADO after removing select features. Models were trained in 75% of the COPDGene sample and tested in the ECLIPSE sample. A, Imaging variables (% LAA < −950 HU, PAA, Pi10) were excluded when building the Cox regression model. B, The 6MWD was excluded when building the Cox regression model. See Figure 1, 2, and 3 legends for expansion of abbreviations.

Although theoretically easy to obtain, 6MWD is often not available during initial clinical encounters. When we removed 6MWD and retrained our Cox model, the resulting model (C index of 0.69 on the testing sample) retained improved performance compared with that of updated BODE (C index, 0.65; P < .05) and ADO (C index, 0.65; P < .05) (Fig 4B).

We developed an online Web application (https://cdnm.shinyapps.io/cgmortalityapp/) to facilitate use of the MLMP-COPD model in exploring the relative contributions of risk factors for all-cause mortality in COPD. In addition, we provide a case study of a hypothetical patient to demonstrate the potential usefulness of this application (see the Results in e-Appendix 1, e-Fig 5).

To assess the relative performance of other methods, we evaluated several combinations of feature selection methods and prediction models (see the Methods in e-Appendix 1, e-Table 8). All methods performed similarly, including stepwise regression, although RSFs selected the most parsimonious set of features (n = 15).

Subgroup Analyses

We tested the performance of the MLMP-COPD model in subgroups of ECLIPSE (see the Results in e-Appendix 1, e-Tables 9, 10). We divided individuals into four groups according to BODE score categories (0-2, 3-4, 5-6, 7-10) from Celli et al² and then used the MLMP-COPD model predictions (log hazard ratio) to stratify participants into four equally sized groups. We compared the strata by using Kaplan-Meier analysis (Fig 5). Characteristics of the sickest group defined by BODE (score 7-10) and our model are shown in Table 3, and the less severe groups in e-Tables 11, 12, and 13. The sickest group had a 19.5% higher relative and 12.5% absolute mortality than did the sickest BODE group (BODE 7-10; 64% vs 77% absolute mortality; P = .012), with shorter median follow-up (1,281 vs 1,193 days; P = .049). The sickest group was also older and had fewer female subjects, higher PA:A ratio, lower % LAA < −950 HU, more comorbid diabetes, more pack-years of cigarette smoking, and lower resting arterial oxygen saturation. However, both groups had similar FEV₁ % predicted and 6MWD, but the sickest group had higher BMI with lower modified Medical Research Council and BODE scores (P < .001).

Figure 5.

Kaplan-Meier analysis of subjects in a pooled COPDGene and ECLIPSE sample. A, Subjects were grouped by BODE score (0-2, 3-4, 5-6, 7-10). B, Subjects were stratified into four groups based on the machine learning mortality prediction (MLMP)-COPD model. Participants were ranked according to their calculated risk by the MLMP-COPD model and divided into four groups of mortality risk that were of equal size as those of the BODE groups. Circles indicate the probability of survival at each time point, and the bars are 95% CIs. See Figure 1 and 3 legends for expansion of other abbreviations.

Table 3.

Characteristics of Subjects With the Highest Predicted Mortality

Characteristic	BODE Group	Model Group	P Value
No. of subjects	184	184	…
Age, mean (SD), y	63.76 (7.47)	69.83 (6.44)	< .001
Race, African American, No. (%)	18 (9.8)	16 (8.7)	.857
Sex, female, No. (%)	82 (44.6)	57 (31.0)	.010
Total dead, No. (%)	118 (64.1)	141 (76.6)	.012
Days followed up, median (IQR)	1,280.50 (940.50-2,243.50)	1,192.50 (791.75-1,833.75)	.049
BODE, median (IQR)	7.00 (7.00-8.00)	6.00 (5.00-7.00)	< .001
6-Min walk distance, mean (SD), ft	608.18 (343.20)	593.19 (263.68)	.639
FEV1 % predicted, median (IQR)	27.65 (21.60-33.28)	28.10 (20.88-34.38)	.942
mMRC dyspnea score, No. (%)			.009
1	0 (0.0)	5 (2.7)
2	19 (10.3)	32 (17.4)
3	58 (31.5)	64 (34.8)
4	107 (58.2)	83 (45.1)
FEV1/FVC ratio, median (IQR)	0.34 (0.28-0.41)	0.33 (0.27-0.37)	.053
FEF25%-75%, median (IQR), L	0.24 (0.19-0.33)	0.23 (0.17-0.30)	.080
Resting SaO2, median (IQR)	94.00 (91.00-96.00)	93.00 (88.00-94.25)	< .001
Exacerbation frequency, mean (SD), No./y	1.26 (1.33)	1.52 (1.57)	.086
Pack-years cigarette smoking, median (IQR)	45.00 (34.60-67.62)	61.40 (40.00-88.92)	< .001
BMI, mean (SD), kg/m2	23.81 (6.30)	25.47 (5.63)	.008
Severe exacerbations, No. (%)	86 (46.7)	95 (51.6)	.404
PA:A ratio, mean (SD)	0.97 (0.16)	1.02 (0.18)	.006
Pi10, mean (SD)	4.12 (0.35)	4.10 (0.37)	.611
Diabetes, No. (%)	20 (10.9)	38 (20.7)	.015
% LAA < −950 HU, median (IQR)	30.80 (18.50-39.40)	24.48 (15.83-36.96)	.021

The BODE group with the highest predicted mortality had BODE score 7 to 10 (see Materials and Methods). See Table 1 and 2 legends for expansion of abbreviations.

Discussion

In this study, we applied a machine learning approach to clinical features, including quantitative CT imaging, to develop an all-cause mortality prediction model in moderate to severe COPD. The resulting MLMP-COPD model outperformed BODE, BODE modifications, and ADO across two COPD cohorts. Our model also identified subjects at high risk of death on the basis of on variables not included in the BODE index. We have included an online tool to allow researchers and clinicians to explore the contributions of our model features to predicted COPD survival. Although the MLMP-COPD model is a prediction tool, it also provides valuable insights into how individual risk factors influence mortality in the context of other known predictors of mortality.

We developed the MLMP-COPD model in COPDGene and used ECLIPSE for external validation. External validation in diverse cohorts is essential, as even the BODE index, which reported a C index of 0.74 in the initial study, demonstrates significantly reduced performance in several external studies.^{5,6,10,43, 44, 45, 46, 47} Compared with prior models of mortality, our model was developed with more subjects with longer follow-ups. The BODE index, which arguably remains the gold standard for mortality prediction, was developed on 207 subjects followed up for 4 years and prospectively validated in 625 subjects.² By comparison, we developed our model by using 1,974 subjects followed up for a median of 6.4 years from the COPDGene study and validated our model by using data from the ECLIPSE study. Overall, our model was highly consistent with the Galaxy COPD model,⁴⁸ which found a similar set of important features and was externally validated, although it underestimated mortality in one of the two cohorts.⁴⁹ However, the current study had five important differences: (1) the use of quantitative imaging features; (2) automated and reproducible feature selection; (3) direct comparison with BODE, BODE modifications, and ADO by using C-statistics; (4) more subjects; and (5) longer follow-up.

Our analysis provides insights into the relative contributions of predictors of mortality. Current smoking was not selected as a predictor, possibly because sicker patients are more likely to quit, confounding the relationship of current smoking and mortality in study populations enriched for subjects with COPD. The 6MWD was ranked substantially higher in VIMP than all other variables. This finding is consistent with those from prior literature, possibly because 6MWD reflects both pulmonary and extrapulmonary (eg, muscle weakness, pulmonary vascular disease) disease manifestations.^{50, 51, 52} Although the 6MWD, in theory, can be implemented easily, it is not always readily available; thus, we developed a model exclusive of 6MWD that demonstrated improved prediction compared with other mortality prediction indexes. However, we advocate for obtaining 6MWD when possible. Although exacerbation history⁵³ and severe exacerbations requiring hospitalization^54,55 are independent predictors of COPD mortality, they have not been evaluated in a single model together. As exacerbation history has been reported to be a predictor of severe exacerbations,⁵⁶ and mortality increases acutely after a severe exacerbation,⁵⁵ severe exacerbations may be adequately capturing the mortality risk conferred by exacerbation history. Diabetes was chosen as an important predictor of mortality, which has been previously reported.^57,58 The fact that diabetes was chosen as an important predictor of mortality likely reflects the high cardiovascular mortality of COPD but may also support the notion that COPD may be an inflammatory multisystem disease.

Of quantitative imaging features, % LAA < −950 HU, Pi10, and PA:A ratio were chosen by the RSF algorithm, yet only the PA:A ratio was significant in the fully adjusted model. These features likely have a complex relationship with survival, and the PA:A ratio may capture vascular disease less directly captured by other features. Our results are consistent with those of a recent study reporting that PA:A is associated with mortality in patients with COPD after adjusting for BODE variables.⁵⁹ However, this and other articles reporting the association of individual risk factors with mortality do not directly address whether combinations of these individual features leads to better prediction. In the current study, we observed that adding the PA:A ratio and other CT imaging-based disease features individually to BODE does not significantly improve predictive performance. Instead, a combination of clinical variables and CT imaging features are required to improve mortality prediction over that of existing models.

To our knowledge, our study is the first to evaluate multiple imaging features simultaneously in the context of mortality prediction. Removal of quantitative CT imaging features from the MLMP-COPD model decreased predictive performance only slightly. This finding is consistent with the RSF VIMP measures, which ranked quantitative imaging features as less important than most of the other features included. In contrast, a study using coronary CT angiography found age followed by imaging features as the most important predictors of all-cause mortality.²⁸ This finding is likely consistent with the ability of coronary imaging to measure directly the specific lesions that lead to the most likely cause of death, in contrast to COPD, for which CT imaging features may be more important for describing COPD heterogeneity and play a more limited role in predicting all-cause mortality. Future work may use imaging to define and predict mortality directly in COPD.^60,61

When stratifying subjects by using the MLMP-COPD model, the sickest group of the same-sized BODE group (score 7-10) had 19.5% higher mortality. Older age, lower resting arterial oxygen saturation, higher PA:A ratio, lower % LAA < −950 HU, increased pack-years of smoking, and diabetes added prognostic information for these subjects. Thus, it is important to think beyond the BODE variables when trying to identify patients with COPD at the highest risk of death.

RSFs have been used for feature selection in COPD case identification²⁶ and for identifying risk factors for COPD exacerbations.²⁷ The current study differs in that we used a survival implementation of RSFs (ie, time-to-event analysis) to identify predictors of all-cause mortality in people with COPD. Combinations of feature selection methods and traditional prediction models performed similarly to more advanced methods, although the combination of using RSFs for variable selection and Cox yielded the most parsimonious model. This result is perhaps not surprising, as simpler machine learning methods or models have been observed to perform similarly to or better than more advanced methods, depending on the data set.²³ In addition, a simpler model that explains the data equally well is more likely to generalize in independent cohorts, presumably because of less overfitting or overparameterization.¹⁸ In a survival analysis of 1,371 patients with head and neck cancer, RSF identified the most important predictors of survival, whereas Cox regression performed slightly better than RSF.⁶² Thus, although the more advanced RSF machine learning method reduced the number of predictors, it provided accuracy similar to that of traditional Cox regression.

Many have attempted, but failed, to develop a mortality prediction model as consistently superior and equally parsimonious as the BODE index.^{3,5,7, 8, 9,11} Most previously published models were not directly compared with BODE in their initial publication, nor were they externally validated. A 2018 study examined the predictive power of 10 COPD mortality prediction models and reported that none performed significantly better than BODE.¹¹ By contrast, our model demonstrated superior predictive performance for long-term mortality across multiple cohorts in patients with moderate to severe COPD. The MLMP-COPD model, although more accurate than BODE, is more complex; however, even relatively simple measures like BODE, in our experience, are not calculated by hand. The use of online clinical calculators as diagnostic, prognostic, and decision aids has increased in clinical practice.^63,64 There is evidence that such point-of-care resources may improve diagnostic accuracy, adherence to guidelines, and accuracy of calculations.⁶⁵ Therefore, we developed an online Web application that allows users to observe how altering input values affects predicted survival. Although our model should undergo further validation, future endeavors may include automated calculations via the electronic medical record to facilitate clinical implementation.

This study had several limitations. First, the COPDGene and ECLIPSE studies were multicenter case-control studies and not representative of the general population. The MLMP-COPD model should ideally be validated in general population samples. Differences in imaging protocols between the COPDGene and ECLIPSE studies may have accounted for the higher proportion of interstitial lung abnormalities and greater amount of emphysema in ECLIPSE and, thus, diminished the predictive power of quantitative imaging features. However, a simple harmonization (z score transformation) of imaging features did not improve mortality prediction, which could reflect the relatively lower effect of imaging compared with clinical features or the need to both standardize imaging protocols and/or develop more advanced image harmonization techniques. Despite these differences, the MLMP-COPD model performed well across these two heterogeneous populations.

Our study used all-cause mortality, which, although arguably more generalizable, given that many deaths in COPD are not due to respiratory disease, is nevertheless a limitation because of the lack of cause-specific mortality. Age was initially not included in the BODE index to maintain respiratory-specific mortality; however, our model performed better than ADO, which also accounts for age. Age, smoking history, and diabetes may be proxies for cardiovascular disease, which was not included in the model because of differences in assessment and missingness between the cohorts. The BODE index is currently used for selection of lung transplant recipients,⁶⁶ and it is possible that our score could be used to improve this process, given that the median survival after lung transplant is approximately 5 years⁶⁷ and BODE predicts out only to 4 years (vs 8 years with the MLMP-COPD model). Additional testing, ideally in a transplant population, is needed before considering its use as a selection tool. Future studies should examine the performance of the MLMP-COPD model in predicting respiratory- and cardiovascular-specific mortality with cause-specific comorbidities. Finally, previous studies have demonstrated the usefulness of adding blood-based biomarkers to predictive models^6,47; however, our study did not include biomarkers because of the missingness in these cohorts.

Conclusions

In conclusion, the MLMP-COPD model demonstrated predictive performance superior to that of four prior mortality prediction indexes in subjects with moderate to severe COPD across two large cohorts. Further investigation across diverse populations and investigation of cause-specific mortality will help support the validity of this model.