Abstract
Rationale: Mortality prediction is well studied in idiopathic pulmonary fibrosis (IPF), but little is known about predictors of premortality disease progression. Identification of patients at risk for disease progression would be useful for clinical decision-making and designing clinical trials.
Objectives: To develop prediction models for disease progression in IPF.
Methods: In a large clinical trial cohort of patients with IPF (n = 1,113), we comprehensively screened multivariate models of candidate baseline and past-change predictors for disease progression defined by 48-week worsening of FVC, dyspnea (University of California, San Diego Shortness of Breath Questionnaire [UCSD SOBQ]), 6-minute-walk distance (6MWD), and occurrence of respiratory hospitalization, or death. Progression outcomes were modeled as appropriate, by slope change using linear regression models and time to binary outcomes using Cox proportional hazards models.
Measurements and Main Results: The overall cohort experienced considerable disease progression. Top-performing prediction models did not meaningfully predict most measures of disease progression. For example, prediction modeling explained less than or equal to 1% of the observed variation in 48-week slope change in FVC, UCSD SOBQ, and 6MWD. Models performed better for binary measures of time to disease progression but were still largely inaccurate (cross-validated C statistic ≤0.63 for ≥10% decline in FVC or death, ≤0.68 for ≥20-U increase in UCSD SOBQ or death, ≤0.70 for ≥100 m decline in 6MWD or death). Models for time to respiratory hospitalization or death (C statistic ≤0.77) or death alone (C statistic ≤0.81) demonstrated acceptable discriminative performance.
Conclusions: Clinical prediction models poorly predicted physiologic and functional disease progression in IPF. This is in contrast to respiratory hospitalization and mortality prediction.
Keywords: interstitial lung disease, forced vital capacity, dyspnea, 6-minute-walk distance, hospitalization
At a Glance Summary
Scientific Knowledge on the Subject
Accurate prediction models have been developed for mortality in idiopathic pulmonary fibrosis (IPF), but little is known about predictors of premortality disease progression.
What This Study Adds to the Field
Clinical information that is commonly used to evaluate and predict mortality in patients with IPF, including indicators of past disease progression, poorly predict risk of future physiologic and functional disease progression. Molecular and genetic biomarkers that enhance prediction of common measures of disease progression in IPF are desperately needed.
Idiopathic pulmonary fibrosis (IPF) is a chronic progressive fibrotic lung disease of adults (1). The prognosis of IPF is poor, with a median survival of only 3–5 years, but the clinical course is believed to vary (2). Although two medications (nintedanib and pirfenidone) have recently been shown to slow disease progression in IPF, it remains a fatal disease with a large unmet medical need (3, 4).
Accurately predicting the risk of death over time in patients with IPF has obvious clinical value. Hence, predictors of mortality risk in IPF have been extensively studied (2) and combined into clinically useful mortality risk prediction models (5, 6). Predicting the risk of disease progression over time in IPF is much less established. Accurately predicting disease progression would allow clinicians to make treatment decisions and counsel patients more appropriately. It would also allow for cohort enrichment in clinical trials designed around primary endpoints of disease progression (7). Disease progression in IPF is most commonly defined by decline in FVC (6, 8–11), but there are other definitions, including worsening symptoms (e.g., worsening dyspnea) (12, 13), worsening physical function (14–16), and the occurrence of acute respiratory worsening requiring hospitalization (6, 9, 17, 18). All of these measures of disease progression have been associated with increased risk of subsequent death in patients with IPF.
The objective of this study was to determine whether commonly measured clinical characteristics of patients with IPF, several of which are proven predictors of mortality in IPF, can be used to accurately predict risk of future disease progression. To accomplish this, we comprehensively evaluated candidate predictor variables in prediction models for outcomes based on the most common disease progression definitions in a large, well-defined clinical trial cohort of patients with IPF.
Methods
Study Population and Design
The source population included all subjects randomized to the placebo groups of two parallel design, phase 3 randomized controlled trials of pirfenidone (i.e., the CAPACITY [Clinical Studies Assessing Pirfenidone in Idiopathic Pulmonary Fibrosis: Research of Efficacy and Safety Outcomes] trials) (19) and all randomized subjects (placebo and intervention) in a phase 3 randomized controlled trial of IFN-γ1b (i.e., the INSPIRE [International Study of Survival Outcomes in Idiopathic Pulmonary Fibrosis with Interferon γ-1b] trial) (20). Subjects treated with IFN-γ1b in the INSPIRE trial were included in the analysis because IFN-γ1b treatment did not affect disease progression outcomes (20). Subjects who completed the 24-week visit of the parent clinical trial (CAPACITY or INSPIRE) were included in the current study population.
Figure 1 illustrates the study design. To have longitudinal variables included in the risk prediction modeling, Week 24 data from the parent clinical trials were considered “baseline.” Thus, study subjects had a 24-week observation or “lead-in” period (i.e., enrollment to Week 24 in the parent clinical trial) with outcomes collected over the following 48 weeks (i.e., from Week 24 to 72 of the parent clinical trials). In total, 29 subjects were excluded for death (n = 27) or lung transplantation (n = 2) before Week 24 (no subjects were lost to follow-up).
Figure 1.
Study design. 6MWD = 6-minute-walk distance; 6MWT = 6-minute-walk test; BMI = body mass index; DlCO = diffusing capacity of the lung for carbon monoxide; RH = respiratory hospitalization; UCSD SOBQ = University of California, San Diego Shortness of Breath Questionnaire.
Predictors
Baseline predictors were age, sex, body mass index, use of supplemental oxygen (yes/no), dyspnea severity as measured by the University of California, San Diego Shortness of Breath Questionnaire (UCSD SOBQ), FVC, FEV1/FVC ratio, diffusing capacity for carbon monoxide percent predicted (DlCO), 6-minute-walk test distance (6MWD) in meters, lowest oxygen desaturation on the 6-minute-walk test (6MWT desaturation), and Gender-Age-Physiology (GAP) stage (a validated mortality risk prediction model that includes baseline sex, age, FVC, and DlCO as predictor variables) (5). Longitudinal predictors were prior 24-week change in FVC, DlCO, UCSD SOBQ, and 6MWD (calculated as change scores: Week 24 value minus Week 0 value), and a history of hospitalization for respiratory reasons in the prior 24 weeks. Subjects were thus excluded for a missing Week 24 value (n = 31 for FVC and n = 30 for 6MWD and UCSD SOBQ) because of the inability to calculate past-change predictors.
Outcomes
Disease progression outcomes were changes in FVC % predicted, UCSD SOBQ, and 6MWD, the occurrence of respiratory hospitalization, and death over 48 weeks.
Statistical Analysis
For more detailed statistical methods, please see the online supplement. The distributions of cohort characteristics at baseline were described by clinical trial and pooled, using the appropriate descriptive measures. The analyses of slope change in FVC % predicted, UCSD SOBQ, and 6MWD involved a two-step procedure. First, estimates of 48-week slope changes were obtained for each measure based on ordinary least squares regression. Second, estimated slopes were used as continuous outcomes in the development of prognostic models. To generate subject-specific slopes, only subjects with two or more outcome measurements between Weeks 24 and 72 for each outcome variable were included in the analysis. For FVC, 6MWD, and UCSD SOBQ outcome analyses, this resulted in exclusion of 165, 217, and 180 subjects, respectively, who had less than two values measured from Week 24 to 72. Of these exclusions, 25 were caused by death, eight by lung transplant, and the remaining by other loss to follow-up; subjects excluded for death were later included in analyses of composite outcomes (see later). Binary outcome measures of disease progression over 48 weeks were absolute decline of greater than or equal to 10% in FVC % predicted (6, 10, 11, 21), increase in UCSD SOBQ greater than or equal to 5 U (22), and decline in 6MWD greater than or equal to 50 m (23). Because thresholds for meaningful decline in UCSD SOBQ and 6MWD are less well-defined for patients with IPF, we also analyzed declines in UCSD SOBQ greater than or equal to 20 U and 6MWD greater than 100 m.
Analysis of variance and pairwise Student’s t testing were used to compare slope changes in FVC % predicted, UCSD SOBQ scores, and 6MWD over 48 weeks by selected baseline (Week 24 of the parent trials) measures and GAP stage. We then used Cox proportional hazards models to compare risk for experiencing composite outcomes of binary disease progression or death (and risk for death alone) by selected baseline (Week 24 of the parent trials) characteristics. In developing multipredictor prognostic models, linear regression models were used to predict 48-week slope change outcomes and Cox proportional hazards models were used to predict time to each composite outcome and death alone. In the Cox models, death was considered equal to disease progression to account for informative censoring by death. Subjects were censored for other loss to follow-up. Exhaustive sequences of candidate models including all possible combinations of candidate predictors (up to a maximum of eight predictors) were examined. Models were then ranked using a measure of predictiveness appropriate to each outcome (the R2 for linear regression models and C statistic for Cox proportional hazards regression models), each estimated using 20 repetitions of 10-fold cross-validation to minimize overfitting. R2 describes the proportion of variability in the outcome that is accounted for by the model for continuous (linear) outcomes. It ranges from zero to 1.0, where values closer to 1.0 indicate a greater proportion of variance explained by the model (i.e., better model fit). For time to binary outcomes, the C statistic describes the ability of a model to discriminate those with an outcome from those without the outcome; values of 0.5 indicate no discrimination, 0.6 to 0.7 poor discrimination, 0.7 to 0.8 good discrimination, and more than 0.8 excellent discrimination.
Results
Baseline Characteristics and Overall Outcomes
A total of 1,113 subjects were included in this study (329 subjects from the CAPACITY trials and 784 subjects from the INSPIRE trial) (Table 1). At baseline, 39.8% of subjects were GAP stage I, 51.4% were GAP stage II, and 8.8% were GAP stage III. During the 24-week lead-in period, the mean change (SD) in FVC % predicted was −2.1 (6.9), in UCSD SOBQ was +3.3 (15.4), and in 6MWD in meters was −22.7 (104.8); there were 81 respiratory hospitalizations (7.3% of the cohort). During the 48-week follow-up period, the mean change (SD) in FVC % predicted was −4.2 (3.1), in UCSD SOBQ was +6.8 (7.7), and in 6MWD in meters was −36 (42). During this period, 200 (18.0%) subjects experienced a greater than or equal to 10% absolute decline in FVC, 607 (54.5%) experienced a greater than or equal to 5 U increase in UCSD SOBQ, 414 (37.2%) experienced a greater than or equal to 50 m decline in 6MWD, 105 (9.4%) experienced a respiratory hospitalization, and 130 (11.7%) died.
Table 1.
Cohort Characteristics at Study Baseline (Week 24 of the Parent Clinical Trials)
CAPACITY (n = 329) | INSPIRE (n = 784) | Pooled (n = 1,113) | |
---|---|---|---|
Age, mean (SD) | 66.6 (7.8) | 66.0 (7.7) | 66.2 (7.8) |
Male sex, n (%) | 237 (72.0) | 557 (71.0) | 794 (71.3) |
BMI, kg/cm2, mean (SD) | 29.9 (4.7) | 29.8 (4.8) | 29.9 (4.8) |
Physiology, mean (SD) | |||
FVC, L | |||
Baseline | 2.81 (0.77) | 2.75 (0.78) | 2.77 (0.78) |
Prior 24-wk change | −0.09 (0.22) | −0.08 (0.29) | −0.08 (0.27) |
FVC, % predicted | |||
Baseline | 72.7 (16.3) | 70.4 (14.7) | 71.1 (15.2) |
Prior 24-wk change | −2.3 (5.8) | −2.0 (7.3) | −2.1 (6.9) |
FEV1/FVC | 0.83 (0.06) | 0.83 (0.06) | 0.83 (0.06) |
DlCO, % predicted | |||
Baseline | 44.7 (11.2) | 44.1 (11.6) | 44.3 (11.5) |
Prior 24-wk change | −2.5 (6.6) | −3.5 (9.2) | −3.2 (8.5) |
6MWD, m | |||
Baseline | 386 (112) | 371 (129) | 375 (124) |
Prior 24-wk change | −19.2 (75.5) | −24.1 (114.6) | −22.7 (104.8) |
6MWT desaturation | 88.5 (4.4) | 88.7 (4.7) | 88.6 (4.6) |
Dyspnea, mean (SD) | |||
UCSD SOBQ | |||
Baseline | 36.7 (23.7) | 37.5 (23.4) | 37.3 (23.5) |
Prior 24-wk change | 3.7 (15.0) | 3.1 (15.6) | 3.3 (15.4) |
Oxygen use, n (%) | 71 (21.6) | 123 (15.7) | 194 (17.4) |
Respiratory hospitalization in the prior 24 wk, n (%) | 7 (2.1) | 74 (9.4) | 81 (7.3) |
GAP stage, n (%) | |||
I | 135 (41.0) | 308 (39.3) | 443 (39.8) |
II | 166 (50.5) | 406 (51.8) | 572 (51.4) |
III | 28 (8.5) | 70 (8.9) | 98 (8.8) |
Definition of abbreviations: 6MWD = 6-minute-walk distance; 6MWT = 6-minute-walk test; BMI = body mass index; CAPACITY = Clinical Studies Assessing Pirfenidone in Idiopathic Pulmonary Fibrosis: Research of Efficacy and Safety Outcomes; DlCO = diffusing capacity of the lung for carbon monoxide; GAP = Gender-Age-Physiology; INSPIRE = International Study of Survival Outcomes in Idiopathic Pulmonary Fibrosis with Interferon γ-1b; UCSD SOBQ = University of California, San Diego Shortness of Breath Questionnaire.
Unadjusted Analyses of Predictors of Disease Progression
There were no statistically significant associations between baseline or longitudinal change predictors and continuous (slope change) measures of disease progression (data not shown). In particular, there was no apparent relationship between risk of mortality, as estimated by GAP stage, and risk of disease progression, whether defined as change in FVC, UCSD SOBQ, or 6MWD (Figure 2). Irrespective of prior 24-week change in FVC, subjects seemed to have similar declines in FVC over the subsequent 48 weeks (Figure 3).
Figure 2.
Mean ± SE disease progression per week by Gender-Age-Physiology stage at study baseline. FVC change is in percent predicted per week, UCSD SOBQ change is in units per week, and 6MWD change is in meters per week. 6MWD = 6-minute-walk distance; UCSD SOBQ = University of California, San Diego Shortness of Breath Questionnaire.
Figure 3.
Mean ± SE change in FVC % predicted per week categorized by past change in FVC % predicted over the previous 24 weeks.
Unadjusted Cox models were used to evaluate the association of individual baseline and past 24-week change predictors for risk of composite disease progression outcomes and death alone (Table 2). Lower baseline FVC % predicted and greater past 24-week decline in FVC % predicted were significantly associated with risk of all subsequent composite disease progression outcomes and death alone. More severe baseline dyspnea (i.e., higher UCSD SOBQ) was significantly associated with increased risk of subsequent greater than or equal to 10% decline in FVC or death, greater than or equal to 50- or 100-m decline in 6MWD or death, respiratory hospitalization or death, and death alone but was not associated with subsequent worsening in UCSD SOBQ or death; whereas greater past 24-week increase in UCSD SOBQ was only significantly associated with increased risk of subsequent greater than or equal to 10% relative decline in FVC or death and decreased risk of subsequent increase in UCSD SOBQ or death. Lower baseline 6MWD was significantly associated with subsequent increased risk of greater than or equal to 10% decline in FVC or death, greater than or equal to 5- or 20-unit increase in UCSD SOBQ or death, respiratory hospitalization or death, and death alone. Greater decline in past 24-week 6MWD was only significantly associated with increased risk of subsequent death and reduced risk of greater than or equal to 50-m decline in 6MWD or death. Higher baseline GAP stage was associated with increased risk of all subsequent composite disease progression outcomes and death alone, but this was primarily caused by increased risk for GAP stage 3 (Table 2, Figure 4; see Figure E1 in the online supplement).
Table 2.
Unadjusted Cox Models for Time to Disease Progression Outcomes by Selected Measures at Study Baseline (Week 24 of the Parent Clinical Trials), Past 24-wk Change, and GAP Stage
Outcome Variable |
||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
FVC % Predicted |
UCSD SOBQ |
6MWD |
Respiratory Hospitalization or Death |
Death |
||||||||||||
>10% Absolute Decline or Death |
>10% Relative Decline or Death |
>5-Unit Increase or Death |
>20-Unit Increase or Death |
>50-m Decline or Death |
>100-m Decline or Death |
|||||||||||
Predictor Variable | HR | P Value | HR | P Value | HR | P Value | HR | P Value | HR | P Value | HR | P Value | HR | P Value | HR | P Value |
Age, yr | 1.006 | 0.497 | 1.003 | 0.710 | 1.009 | 0.085 | 1.004 | 0.640 | 1.006 | 0.324 | 1.003 | 0.645 | 0.988 | 0.327 | 1.011 | 0.376 |
Male (vs. female) | 0.874 | 0.376 | 0.803 | 0.057 | 1.111 | 0.243 | 0.979 | 0.872 | 1.134 | 0.238 | 1.078 | 0.567 | 0.845 | 0.441 | 1.016 | 0.940 |
Baseline FVC (% predicted) | 0.988 | 0.015 | 0.983 | <0.001 | 0.991 | 0.001 | 0.968 | <0.001 | 0.978 | <0.001 | 0.966 | <0.001 | 0.958 | <0.001 | 0.938 | <0.001 |
Baseline UCSD SOBQ | 1.009 | 0.004 | 1.008 | <0.001 | 0.997 | 0.055 | 1.002 | 0.552 | 1.009 | <0.001 | 1.013 | <0.001 | 1.018 | <0.001 | 1.017 | <0.001 |
Baseline 6MWD, m | 0.996 | <0.001 | 0.997 | <0.001 | 0.999 | <0.001 | 0.997 | <0.001 | 1.000 | 0.619 | 0.999 | 0.021 | 0.998 | 0.007 | 0.994 | <0.001 |
Past 24-wk change in FVC | 0.886 | <0.001 | 0.891 | <0.001 | 0.972 | <0.001 | 0.943 | <0.001 | 0.964 | <0.001 | 0.946 | <0.001 | 0.910 | <0.001 | 0.902 | <0.001 |
Past 24-wk change in UCSD SOBQ | 1.006 | 0.428 | 1.015 | 0.012 | 0.987 | 0.007 | 0.979 | 0.008 | 1.005 | 0.329 | 1.007 | 0.293 | 1.012 | 0.246 | 0.993 | 0.673 |
Past 24-wk change in 6MWD | 0.998 | 0.202 | 1.000 | 0.957 | 0.999 | 0.136 | 0.998 | 0.067 | 1.004 | 0.005 | 1.002 | 0.175 | 0.997 | 0.162 | 0.995 | 0.034 |
Baseline GAP stage | ||||||||||||||||
Stage I (n = 417) | ref | — | ref | — | ref | — | ref | — | ref | — | ref | — | ref | — | ref | — |
Stage II (vs. I) (n = 538) | 1.186 | 0.276 | 1.266 | 0.046 | 1.190 | 0.043 | 1.263 | 0.082 | 1.442 | <0.001 | 1.584 | <0.001 | 1.494 | 0.080 | 1.476 | 0.085 |
Stage III (vs. I) (n = 89) | 2.340 | <0.001 | 1.762 | 0.003 | 1.714 | <0.001 | 2.589 | <0.001 | 2.320 | <0.001 | 3.112 | <0.001 | 2.504 | 0.006 | 5.555 | <0.001 |
Definition of abbreviations: 6MWD = 6-minute-walk distance; GAP = Gender-Age-Physiology; HR = hazard ratio; UCSD SOBQ = University of California, San Diego Shortness of Breath Questionnaire.
Figure 4.
Kaplan–Meier curves for 48-week respiratory hospitalization (Resp hosp) or death stratified by baseline Gender-Age-Physiology (GAP) stage.
Multivariable Models for Disease Progression or Death
The top-ranked multivariate linear prediction models were poor predictors of continuous measures of disease progression (cross-validated r2 <0.01), regardless of how disease progression was defined (Table 3; see Table E1). The top performing multivariable Cox prediction models for most composite disease progression outcomes had poor discriminative performance (0.63 for ≥10% absolute decline in FVC or death, 0.68 for ≥20-unit increase in UCSD SOBQ or death, and 0.70 for ≥100-m decline in 6MWD or death) (Table 3; see Table E2). Only models for time to first respiratory hospitalization or death (cross-validated C statistic, 0.77) or death alone (cross-validated C statistic, 0.81) demonstrated acceptable discriminative performance.
Table 3.
Measures of Predictiveness for the Top Performing Prediction Models
Disease Progression Measure | Top Model Components | Linear Models | Cox Models |
---|---|---|---|
FVC, % predicted | CV-R2 | CV-C statistic | |
Linear model | |||
Slope | Age + DlCO + FVC + SOBQ + Δ6MWD + ΔDlCO + male + ΔSOBQ | 0.0076 | |
Cox model | |||
≥10% decline or death | 6MWD + DlCO + RH + ΔSOBQ | 0.625 | |
UCSD SOBQ | |||
Linear model | |||
Slope | Age + BMI + FEV1/FVC + WSpO2 + male + ΔSOBQ | 0.0025 | |
Cox models | |||
≥5-unit increase or death | 6MWD + DlCO + ΔFVC + RH + ΔSOBQ | 0.604 | |
≥20-unit increase or death | Age + 6MWD + DlCO + FVC + ΔFVC + RH + ΔSOBQ | 0.681 | |
6MWD | |||
Linear model | |||
Slope | 6MWD + DlCO + FEV1/FVC + WSpO2 + male + O2 use | 0.0050 | |
Cox models | |||
≥50-m decline or death | DlCO + FVC + WSpO2 + Δ6MWD + RH | 0.640 | |
≥100-m decline or death | DlCO + FVC + SOBQ + WSpO2 + Δ6MWD + ΔDlCO + RH | 0.702 | |
Respiratory hospitalization (Cox models) | |||
Respiratory hospitalization or death | 6MWD + DlCO + ΔDlCO + ΔFVC + RH + ΔSOBQ | 0.767 | |
Respiratory hospitalization, ≥10% FVC | 6MWD + DlCO + RH + ΔSOBQ | 0.629 | |
decline, or death | |||
Death (Cox model) | 6MWD + FVC + ΔFVC + RH | 0.810 |
Definition of abbreviations: Δ = prior 24-week change; 6MWD = 6-minute-walk distance; BMI = body mass index; CV = cross-validated; DlCO = diffusing capacity of the lung for carbon monoxide; RH = respiratory hospitalization in the prior 24 weeks; UCSD SOBQ = University of California, San Diego Shortness of Breath Questionnaire; WSpO2 = lowest saturation on 6-minute-walk test.
Discussion
In a large clinical trials cohort of well-characterized patients with IPF, we found that commonly measured clinical variables do not reliably predict risk of future disease progression as defined by decline in FVC, worsening UCSD SOBQ score, or worsening 6MWD. This is in contrast to the ability of these variables to predict risk of either respiratory hospitalization or death in IPF (5, 6, 24). These data support recent evidence that disease progression in IPF is nonlinear and unrelated to patient’s previous disease behavior (1, 2, 25, 26), and suggest that respiratory hospitalization and death may be more strongly correlated event types with similar predisposing clinical factors.
Change in FVC and other clinical metrics have been hypothesized to be stepwise rather than linear, with periods of progression followed by periods of relative stability (2). This step-wise model of disease progression could reconcile the strong evidence that measures of disease progression, such as decline in FVC, are useful for stratification of mortality risk, but do not predict shorter term risk of disease progression. This could also explain observations in this cohort and previously that changes in FVC, UCSD SOBQ, or 6MWD in the past are often not correlated or even inversely associated with changes in the future (25).
Random variability in the test characteristics of FVC, UCSD SOBQ, and 6MWD could also diminish the predictive value and the clinical relevance of changes in these variables on an individual patient basis, and could explain the poor correlation between previous and subsequent changes in measures of disease progression. To address the issue of clinical relevance of outcome measures, we dichotomized changes in the three continuous disease progression measures at thresholds that represent clinically significant changes (e.g., ≥10% decline in FVC). It did seem that dichotomization of disease progression outcomes improved model performance and predictive ability, perhaps because it improved the signal-to-noise ratio of the measure.
There are several other possible reasons for the poor prediction of physiologic and functional disease progression outcomes we observed related to data quality and the performance characteristics of the variables included in the models. Informative censoring of the variables used to assess disease progression could have been present. An example would be if death or study dropout, and therefore missing longitudinal measurement of disease progression variables, was more common in subjects experiencing a drop in FVC. The absence of these reduced FVC measurements could result in an overly stable estimate of FVC in the general population seemingly discordant with the rate of death (27). To address this possibility, we evaluated models for composite outcomes including death. These models generally performed better than progression models not accounting for death, but model performance remained marginal (C statistic, 0.63–0.70) and may have been driven in large part by the stronger association of predictor variables with death than disease progression measures.
Our findings have implications for clinical practice and clinical research in IPF. Regrettably, prediction models of disease progression using clinical variables do not seem to be useful to clinicians hoping to counsel patients more accurately regarding future disease behavior and timing of therapy. This is a major disappointment, because patients and clinicians currently have little ability to weigh the risks and benefits of therapy and assess therapeutic response. For example, initiating treatment in a patient with recent disease progression as measured by FVC decline may lead to overestimation of the efficacy of therapy, because our data suggest the patient would be expected to decline less rapidly over the short term regardless. Similarly, FVC decline after initiation of therapy in a patient who seemed stable before therapy may be misinterpreted as treatment failure.
There may be value in the use of prediction models of disease progression in clinical trial cohort enrichment. For example, selecting patients for enrollment based on baseline characteristics present in the prediction model for respiratory hospitalization or death (e.g., 6MWD, DlCO, UCSD SOBQ, past respiratory hospitalization, and change in FVC and DlCO) could help increase event rates in an event-driven trial, reducing sample size requirements and cost. The use of these prediction models for cohort enrichment requires further investigation.
Our study cohort included more than 1,100 well-characterized subjects with IPF with high-quality longitudinal data available. This is a central strength of the study, protecting against the bias and lack of precision that plague most retrospective cohort analyses in IPF. Because of the nature of clinical trial cohorts, however, our study included subjects who were, on average, healthier and more motivated than the general IPF population. Whether these biases are relevant to prediction of disease progression is unknown. A second strength of the study is its comprehensive and unbiased modeling methodology. We screened an enormous number of models constructed from clinically relevant variables and used a variety of disease progression outcome measures. This makes it highly unlikely that we missed better-performing prediction models. We were limited, however, to the clinical data collected in the course of the clinical trial, and it is possible that prognostically important clinical variables (e.g., comorbidities) were missed. Finally, the use of a 24-week “delayed baseline” was important to allow for the simultaneous evaluation of cross-sectional and past-change predictor variables; however, this design may limit the generalizability of our results to patients surviving at least 6 months after their initial evaluation.
In conclusion, we demonstrate that clinical variables, including those known to predict mortality risk in patients with IPF, poorly predict the risk of physiologic and functional disease progression over 48 weeks. We believe this is most likely caused by the heterogeneous nature of IPF disease progression and, to a lesser extent, the random variability in the measurement of common clinical variables. More accurate prediction of physiologic and functional disease progression will likely require the incorporation of molecular and genetic biomarkers that more directly reflect underlying disease activity. Clinical prediction models for event-driven disease progression measures may provide value to clinical researchers looking to enrich populations of patients more likely to experience clinically relevant outcomes, and this should be a focus of future research in this area.
Supplementary Material
Footnotes
Supported by Genentech and NHLBI grant F32HL124895.
Author Contributions: B.L. and H.R.C. conceived the study. B.L., W.Z.B., E.V., D.W., R.M.d.B., and H.R.C. contributed to the study design. B.L., E.V., and D.W. performed the statistical analysis. All authors contributed to interpretation of results. B.L. and H.R.C. wrote the manuscript. All authors contributed to review of the manuscript. B.L. prepared revisions. All authors approved of the final manuscript for submission.
This article has an online supplement, which is accessible from this issue's table of contents at www.atsjournals.org
Originally Published in Press as DOI: 10.1164/rccm.201508-1546OC on March 3, 2016
Author disclosures are available with the text of this article at www.atsjournals.org.
References
- 1.Raghu G, Collard HR, Egan JJ, Martinez FJ, Behr J, Brown KK, Colby TV, Cordier JF, Flaherty KR, Lasky JA, et al. ATS/ERS/JRS/ALAT Committee on Idiopathic Pulmonary Fibrosis. An official ATS/ERS/JRS/ALAT statement: idiopathic pulmonary fibrosis: evidence-based guidelines for diagnosis and management. Am J Respir Crit Care Med. 2011;183:788–824. doi: 10.1164/rccm.2009-040GL. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ley B, Collard HR, King TE., Jr Clinical course and prediction of survival in idiopathic pulmonary fibrosis. Am J Respir Crit Care Med. 2011;183:431–440. doi: 10.1164/rccm.201006-0894CI. [DOI] [PubMed] [Google Scholar]
- 3.King TE, Jr, Bradford WZ, Castro-Bernardini S, Fagan EA, Glaspole I, Glassberg MK, Gorina E, Hopkins PM, Kardatzke D, Lancaster L, et al. A phase 3 trial of pirfenidone in patients with idiopathic pulmonary fibrosis. N Engl J Med. 2014;370:2083–2092. doi: 10.1056/NEJMoa1402582. [DOI] [PubMed] [Google Scholar]
- 4.Richeldi L, du Bois RM, Raghu G, Azuma A, Brown KK, Costabel U, Cottin V, Flaherty KR, Hansell DM, Inoue Y, et al. Efficacy and safety of nintedanib in idiopathic pulmonary fibrosis. N Engl J Med. 2014;370:2071–2082. doi: 10.1056/NEJMoa1402584. [DOI] [PubMed] [Google Scholar]
- 5.Ley B, Ryerson CJ, Vittinghoff E, Ryu JH, Tomassetti S, Lee JS, Poletti V, Buccioli M, Elicker BM, Jones KD, et al. A multidimensional index and staging system for idiopathic pulmonary fibrosis. Ann Intern Med. 2012;156:684–691. doi: 10.7326/0003-4819-156-10-201205150-00004. [DOI] [PubMed] [Google Scholar]
- 6.du Bois RM, Weycker D, Albera C, Bradford WZ, Costabel U, Kartashov A, Lancaster L, Noble PW, Raghu G, Sahn SA, et al. Ascertainment of individual risk of mortality for patients with idiopathic pulmonary fibrosis. Am J Respir Crit Care Med. 2011;184:459–466. doi: 10.1164/rccm.201011-1790OC. [DOI] [PubMed] [Google Scholar]
- 7.Collard HR, Bradford WZ, Cottin V, Flaherty KR, King TE, Jr, Koch GG, Kolb M, Martinez FJ, Montgomery B, Raghu G, et al. A new era in idiopathic pulmonary fibrosis: considerations for future clinical trials. Eur Respir J. 2015;46:243–249. doi: 10.1183/09031936.00200614. [DOI] [PubMed] [Google Scholar]
- 8.Raghu G, Collard HR, Anstrom KJ, Flaherty KR, Fleming TR, King TE, Jr, Martinez FJ, Brown KK. Idiopathic pulmonary fibrosis: clinically meaningful primary endpoints in phase 3 clinical trials. Am J Respir Crit Care Med. 2012;185:1044–1048. doi: 10.1164/rccm.201201-0006PP. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Durheim MT, Collard HR, Roberts RS, Brown KK, Flaherty KR, King TE, Jr, Palmer SM, Raghu G, Snyder LD, Anstrom KJ, et al. IPFnet investigators. Association of hospital admission and forced vital capacity endpoints with survival in patients with idiopathic pulmonary fibrosis: analysis of a pooled cohort from three clinical trials. Lancet Respir Med. 2015;3:388–396. doi: 10.1016/S2213-2600(15)00093-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Richeldi L, Ryerson CJ, Lee JS, Wolters PJ, Koth LL, Ley B, Elicker BM, Jones KD, King TE, Jr, Ryu JH, et al. Relative versus absolute change in forced vital capacity in idiopathic pulmonary fibrosis. Thorax. 2012;67:407–411. doi: 10.1136/thoraxjnl-2011-201184. [DOI] [PubMed] [Google Scholar]
- 11.Zappala CJ, Latsi PI, Nicholson AG, Colby TV, Cramer D, Renzoni EA, Hansell DM, du Bois RM, Wells AU. Marginal decline in forced vital capacity is associated with a poor outcome in idiopathic pulmonary fibrosis. Eur Respir J. 2010;35:830–836. doi: 10.1183/09031936.00155108. [DOI] [PubMed] [Google Scholar]
- 12.Swigris JJ, Han M, Vij R, Noth I, Eisenstein EL, Anstrom KJ, Brown KK, Fairclough D. The UCSD shortness of breath questionnaire has longitudinal construct validity in idiopathic pulmonary fibrosis. Respir Med. 2012;106:1447–1455. doi: 10.1016/j.rmed.2012.06.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Manali ED, Stathopoulos GT, Kollintza A, Kalomenidis I, Emili JM, Sotiropoulou C, Daniil Z, Roussos C, Papiris SA. The Medical Research Council chronic dyspnea score predicts the survival of patients with idiopathic pulmonary fibrosis. Respir Med. 2008;102:586–592. doi: 10.1016/j.rmed.2007.11.008. [DOI] [PubMed] [Google Scholar]
- 14.du Bois RM, Albera C, Bradford WZ, Costabel U, Leff JA, Noble PW, Sahn SA, Valeyre D, Weycker D, King TE., Jr 6-Minute walk distance is an independent predictor of mortality in patients with idiopathic pulmonary fibrosis. Eur Respir J. 2014;43:1421–1429. doi: 10.1183/09031936.00131813. [DOI] [PubMed] [Google Scholar]
- 15.Flaherty KR, Andrei AC, Murray S, Fraley C, Colby TV, Travis WD, Lama V, Kazerooni EA, Gross BH, Toews GB, et al. Idiopathic pulmonary fibrosis: prognostic value of changes in physiology and six-minute-walk test. Am J Respir Crit Care Med. 2006;174:803–809. doi: 10.1164/rccm.200604-488OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Lederer DJ, Arcasoy SM, Wilt JS, D’Ovidio F, Sonett JR, Kawut SM. Six-minute-walk distance predicts waiting list survival in idiopathic pulmonary fibrosis. Am J Respir Crit Care Med. 2006;174:659–664. doi: 10.1164/rccm.200604-520OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Brown AW, Fischer CP, Shlobin OA, Buhr RG, Ahmad S, Weir NA, Nathan SD. Outcomes after hospitalization in idiopathic pulmonary fibrosis: a cohort study. Chest. 2015;147:173–179. doi: 10.1378/chest.13-2424. [DOI] [PubMed] [Google Scholar]
- 18.Song JW, Hong SB, Lim CM, Koh Y, Kim DS. Acute exacerbation of idiopathic pulmonary fibrosis: incidence, risk factors and outcome. Eur Respir J. 2011;37:356–363. doi: 10.1183/09031936.00159709. [DOI] [PubMed] [Google Scholar]
- 19.Noble PW, Albera C, Bradford WZ, Costabel U, Glassberg MK, Kardatzke D, King TE, Jr, Lancaster L, Sahn SA, Szwarcberg J, et al. CAPACITY Study Group. Pirfenidone in patients with idiopathic pulmonary fibrosis (CAPACITY): two randomised trials. Lancet. 2011;377:1760–1769. doi: 10.1016/S0140-6736(11)60405-4. [DOI] [PubMed] [Google Scholar]
- 20.King TE, Jr, Albera C, Bradford WZ, Costabel U, Hormel P, Lancaster L, Noble PW, Sahn SA, Szwarcberg J, Thomeer M, et al. INSPIRE Study Group. Effect of interferon gamma-1b on survival in patients with idiopathic pulmonary fibrosis (INSPIRE): a multicentre, randomised, placebo-controlled trial. Lancet. 2009;374:222–228. doi: 10.1016/S0140-6736(09)60551-1. [DOI] [PubMed] [Google Scholar]
- 21.du Bois RM, Weycker D, Albera C, Bradford WZ, Costabel U, Kartashov A, King TE, Jr, Lancaster L, Noble PW, Sahn SA, et al. Forced vital capacity in patients with idiopathic pulmonary fibrosis: test properties and minimal clinically important difference. Am J Respir Crit Care Med. 2011;184:1382–1389. doi: 10.1164/rccm.201105-0840OC. [DOI] [PubMed] [Google Scholar]
- 22.Kupferberg DH, Kaplan RM, Slymen DJ, Ries AL. Minimal clinically important difference for the UCSD Shortness of Breath Questionnaire. J Cardiopulm Rehabil. 2005;25:370–377. doi: 10.1097/00008483-200511000-00011. [DOI] [PubMed] [Google Scholar]
- 23.du Bois RM, Weycker D, Albera C, Bradford WZ, Costabel U, Kartashov A, Lancaster L, Noble PW, Sahn SA, Szwarcberg J, et al. Six-minute-walk test in idiopathic pulmonary fibrosis: test validation and minimal clinically important difference. Am J Respir Crit Care Med. 2011;183:1231–1237. doi: 10.1164/rccm.201007-1179OC. [DOI] [PubMed] [Google Scholar]
- 24.Ley B, Brown KK, Collard HR. Molecular biomarkers in idiopathic pulmonary fibrosis. Am J Physiol Lung Cell Mol Physiol. 2014;307:L681–L691. doi: 10.1152/ajplung.00014.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Schmidt SL, Tayob N, Han MK, Zappala C, Kervitsky D, Murray S, Wells AU, Brown KK, Martinez FJ, Flaherty KR. Predicting pulmonary fibrosis disease course from past trends in pulmonary function. Chest. 2014;145:579–585. doi: 10.1378/chest.13-0844. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Salisbury ML, Xia M, Zhou Y, Murray S, Tayob N, Brown KK, Wells AU, Schmidt SL, Martinez FJ, Flaherty KR. Idiopathic pulmonary fibrosis: Gender-Age-Physiology index stage for predicting future lung function decline. Chest. 2016;149:491–498. doi: 10.1378/chest.15-0530. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Laird NM. Missing data in longitudinal studies. Stat Med. 1988;7:305–315. doi: 10.1002/sim.4780070131. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.