A multidimensional classifier to support lung transplant referral in patients with pulmonary fibrosis

Abstract

BACKGROUND:

Lung transplantation remains the sole curative option for patients with idiopathic pulmonary fibrosis (IPF), but donor organs remain scarce, and many eligible patients die before transplant. Tools to optimize the timing of transplant referrals are urgently needed.

METHODS:

Least absolute shrinkage and selection operator was applied to clinical and proteomic data generated as part of a prospective cohort study of interstitial lung disease (ILD) to derive clinical, proteomic, and multidimensional logit models of near-term death or lung transplant within 18 months of blood draw. Model-fitted values were dichotomized at the point of maximal sensitivity and specificity, and decision curve analysis was used to select the best-performing classifier. We then applied this classifier to independent IPF and non-IPF ILD cohorts to determine test performance characteristics. Cohorts were restricted to patients aged ≤72 years with body mass index 18 to 32 to increase the likelihood of transplant eligibility.

RESULTS:

IPF derivation, IPF validation, and non-IPF ILD validation cohorts consisted of 314, 105, and 295 patients, respectively. A multidimensional model comprising 2 clinical variables and 20 proteins outperformed stand-alone clinical and proteomic models. Following dichotomization, the multidimensional classifier predicted near-term outcome with 70% sensitivity and 92% specificity in the IPF validation cohort and 70% sensitivity and 80% specificity in the non-IPF ILD validation cohort.

CONCLUSIONS:

A multidimensional classifier of near-term outcomes accurately discriminated this end-point with good test performance across independent IPF and non-IPF ILD cohorts. These findings support refinement and prospective validation of this classifier in transplant-eligible individuals.

Progressive pulmonary fibrosis is a common manifestation of interstitial lung disease (ILD) that results in irreversible parenchymal scarring, declining lung function, and poor survival.^1,2 Idiopathic pulmonary fibrosis (IPF) is considered the prototypical progressive pulmonary fibrosis, but a subset of those with non-IPF forms of ILD also develop this condition, leading to similarly poor outcomes.³ While antifibrotic therapy is now approved for the treatment of IPF and progressive non-IPF ILD, currently available drugs slow, rather than stop or reverse fibrosis.^4,5 Lung transplantation is an additional treatment option for a subset of patients with IPF, but donor organs are scarce, and many patients die before a transplant evaluation is completed or after being listed for transplant.^6,7 While expert guidance has been published by the International Society for Heart and Lung Transplantation (ISHLT), the optimal timing of transplant referral for those with progressive pulmonary fibrosis remains unclear.⁸ Moreover, some patients develop and quickly succumb to rapidly progressive disease,⁹ missing the opportunity to undergo transplant evaluation. To optimize the timing of lung transplant referral, tools to accurately identify those most likely to benefit from lung transplant evaluation are urgently needed.

Clinical prediction models effectively discriminate mortality risk in patients with IPF^10–12 and non-IPF ILDs.¹³ However, these models fail to predict near-term progression,¹⁴ suggesting that they are a better measure of disease severity than biological activity. Moreover, clinical models rely heavily on age, leaving it unclear how well they would perform in younger individuals, who are more likely to be considered for lung transplant. A host of blood-based prognostic biomarkers have been proposed for patients with progressive pulmonary fibrosis, including peripheral blood leukocyte telomere length, a composite gene expression-based signature, many stand-alone proteins, and composite proteomic signatures.^15–23 While these measures remain promising, none have been widely adopted for clinical use. This reality stems in part from uncertainly around biomarker test performance and the way biomarker classification would change clinical management.

In this investigation, we leveraged clinical and high-throughput proteomic data to develop an actionable biomarker to support lung transplant referral in patients with IPF. Using machine learning, we derived clinical, proteomic, and multidimensional prediction models of a composite end-point of death or transplant within 18 months of blood draw. We then dichotomized fitted values for the best-performing model and tested this binary classifier in an independent IPF cohort to determine test performance characteristics. To assess generalizability to other ILD subtypes, we then applied this classifier to an independent non-IPF ILD cohort.

Methods

Cohorts

Consecutive patients with IPF and non-IPF ILD who underwent proteomic profiling as part of the PRECISIONS study²⁴ were eligible for inclusion. The PRECISIONS molecular cohort consists of individuals with IPF and non-IPF ILD participating in prospective registries and biorepositories administered by the Pulmonary Fibrosis Foundation (PFF) Patient Registry (March 2016-June 2018), University of Virginia (September 2018-November 2021), University of California-Davis (July 2016-April 2021), and University of Chicago (March 2007-July 2017). To enrich the cohort with patients potentially eligible for lung transplant, those aged > 72 or with body mass index (BMI) < 18 or > 32 were excluded. Those with missing survival and BMI data were also excluded, as were those without forced vital capacity (FVC) and diffusion capacity of the lung for carbon monoxide (DLCO) performed within 12 months of blood draw to ensure each model could be directly compared in the same population.

Study-specific protocols were approved at UC-Davis (protocol #875917), UChicago (protocol #13-1180), and UVA (protocol #20937). Patients with IPF from the PPF Patient Registry²⁵ comprised the IPF discovery cohort, while those with IPF from the Universities of Virginia, California-Davis, and Chicago comprised the IPF validation cohort. A pooled cohort of patients with non-IPF ILD from all 4 registries comprised the non-IPF ILD validation cohort. PFF Patient Registry participants from the University of Chicago and University of Virginia were excluded to ensure no overlap between cohorts. The University of California-Davis did not contribute patients to the PFF Patient Registry.

Proteomic platform and data

Proteomic data were generated using the Olink (Uppsala, Sweden) Explore 3072 platform, which provides semiquantitative measures for 2921 analytes. This and other Olink platforms use proximity extension assays to quantify proteins, which provide excellent sensitivity and specificity for low-abundance proteins.^26,27 Intensity normalization is used to aggregate data across plates. A value is returned for all proteins, with those falling below the lower limit of detection imputed and flagged. Normalized protein data were log₂ transformed and modeled continuously.

Statistical analysis

Continuous measures are presented as means with standard deviations and compared using a student’s t-test. Categorical measures are presented as count and percentage and compared using a chi-square test. Three separate logistic regression models were fit in the IPF derivation cohort with clinical, proteomic, and multidimensional (clinical and proteomic) data to identify variables predictive of a composite dichotomous end-point of death or transplant within 18 months of blood draw. Variable selection was performed using the least absolute shrinkage and selection operator with 10-fold cross validation used for parameter tuning.²⁸ Clinical variables available for selection included age, sex, race, BMI, smoking history, percent predicted FVC, and percent predicted DLCO. Proteins included all 2921 generated using the Olink Explore platform. A multidimensional model was derived using all available clinical and proteomic data.

Fitted values for each model were dichotomized at the point of maximal sensitivity and specificity using Youden’s index²⁹ and the best-performing classifier was identified using receiver operator curve and decision curve analysis.^30,31 We then applied the best-performing classifier to the IPF validation cohort and pooled non-IPF ILD validation cohort to determine test performance characteristics and generalizability.

Eighteen-month transplant-free survival was plotted for classification groups using the Kaplan-Meier estimator. Univariate and multivariate Cox proportional hazards regression models were used to estimate outcome risk for those with a high-risk classification. The proportional hazards assumption was checked and satisfied for all models. All statistical analyses were performed using Stata (StataCorp. 2022. Release 17. College Station, TX). Statistical significance was set a p < 0.05.

Results

Of 937, 368 and 623 eligible patients from the IPF derivation, IPF validation, and non-IPF ILD validation cohorts, respectively, 314, 105, and 295 met inclusion and exclusion criteria (Figure S1). Baseline characteristics at the time of blood draw for each cohort are shown in Table 1. IPF cohorts were similar in age, sex, race, and smoking history. Those in the derivation cohort had lower percent predicted FVC and DLCO and higher antifibrotic use when compared to the IPF validation cohort. As expected, the non-IPF ILD cohort was younger, with fewer males, whites, smokers, and outcomes than the IPF cohorts. Baseline immunosuppressant use was higher in the non-IPF validation cohort. Near-term outcome occurred in 25.5% (n = 80), 28.6% (n = 30), and 18.3% (n = 54) of patients in the IPF derivation, IPF validation, and non-IPF ILD validation cohorts, respectively. Lung transplant accounted for 35% to 49% of these near-term outcomes depending on the cohort, supporting transplant eligibility for a large minority of patients satisfying this outcome. When comparing 18-month transplant-free survival between cohorts (Figure S2), those with non-IPF ILD displayed slightly favorable survival when compared to the IPF cohorts (p_logrank = 0.03), while there was no difference in transplant-free survival between IPF cohorts (p_logrank = 0.59).

Table 1.

Baseline Characteristics and Outcomes for Derivation and Validation Cohorts

Characteristic	IPF derivation cohort (n = 314)	IPF validation cohort (n = 105)	p-valuea	Non-IPF ILD validation cohort (n = 295)
Age, mean (± SD)	65.5 (6.3)	66.2 (4.5)	0.29	59.5 (10.6)
Male sex, n (%)	235 (74.8)	73 (69.5)	0.31	130 (44.1)
White race, n (%)	293 (93.3)	94 (89.5)	0.21	226 (76.6)
Ever smoker, n (%)	194 (61.8)	64 (61.0)	0.91	145 (49.2)
FVC %, mean (± SD)	66.1 (16.1)	71.6 (17.9)	0.003	66.4 (19.5)
DLCO %, mean (± SD)	41.3 (16.4)	51.4 (18.3)	< 0.001	45.4 (18.6)
Antifibrotic treated, n (%)	262 (83.4)	72 (68.6)	0.002	31 (10.5)
Immunosuppressant treated, n (%)	38 (12.1)	2 (1.9)	< 0.001	112 (38.0)
Outcomes
Death or transplant	80 (25.5)	30 (28.6)	0.42	54 (18.3)
Death	41 (13.1)	19 (18.1)		35 (11.9)
Lung transplant	39 (12.4)	11 (10.5)		19 (6.4)

Abbreviations: DLCO, diffusion capacity of the lung for carbon monoxide; FVC, forced vital capacity; ILD, interstitial lung disease; IPF, idiopathic pulmonary fibrosis.

Antifibrotic = nintedanib or pirfenidone.

Immunosuppressant = prednisone, mycophenolate mofetil, azathioprine, cyclophosphamide, or rituximab.

^aComparing IPF derivation and validation cohorts.

When applying least absolute shrinkage and selection operator to the IPF derivation cohort, 5 clinical variables were selected for the clinical prediction model, 16 proteins for the proteomic prediction model, and 2 clinical variables and 20 proteins for the multidimensional prediction model (Table S1). After dichotomization of fitted values for each model according to Youden’s index, 115 (36.6%), 112 (35.7%), and 91 (29.0%) patients were classified as high-risk for near-term outcome. In comparative analysis, the multidimensional model and binary classifier provided the highest net benefit over a wide range of threshold probabilities when compared to the clinical and proteomic models and classifiers (Figure 1). Individual measures of test performance corroborated these findings (Table 2). The multidimensional classifier was advanced for further testing, as this provided the highest specificity (88%) and positive predictive value (68%) without sacrificing sensitivity or negative predictive value relative to the other 2 classifiers. Based on likelihood ratios for this multidimensional classifier, a high-risk classification would increase the pretest probability of a near-term outcome (based on observed prevalence) from 26% to approximately 70% (Figure 2). A low-risk classification would reduce the pretest probability from 26% to approximately 7% (Figure 2).

Figure 1.

Decision curve analysis comparing clinical, proteomic, and multidimensional models (a) and binary classifiers (b) of near-term outcome.

Table 2.

Comparison of Test Performance Characteristics for Clinical, Proteomic, and Multidimensional Classifiers of 18-Month Death or Transplant in the IPF Derivation Cohort (n = 314)

Classification	Clinical classifier		Proteomic classifier		Multidimensional classifier
	Outcome (+)	Outcome (−)	Outcome (+)	Outcome (−)	Outcome (+)	Outcome (−)
High risk	61	54	65	47	62	29
Low risk	19	180	15	187	18	205
Sensitivity	0.76		0.81		0.78
Specificity	0.77		0.80		0.88
(+) Predictive value	0.53		0.58		0.68
(−) Predictive value	0.90		0.93		0.92
(+) Likelihood ratio	3.30		4.05		6.25
(−) Likelihood ratio	0.31		0.23		0.26

Abbreviations: IPF, idiopathic pulmonary fibrosis.

Figure 2.

Nomograms for change in pre- and post-test probability based on likelihood ratio tests for IPF derivation, IPF validation, and non-IPF ILD validation cohorts. Pretest probabilities established using the observed incidence of 18-month death or transplant. ILD, interstitial lung disease; IPF, idiopathic pulmonary fibrosis.

When applied to the IPF validation cohort, the classifier had lower sensitivity (70%), but higher specificity (92%), producing high positive (78%), and negative (88%) predictive values (Table 3). Using the observed 18-month cumulative incidence of death or transplant (29%) to establish pretest probability, likelihood ratios for this cohort suggested that a high-risk classification would increase the pretest probability of 18-month death or transplant from 29% to just over 70% (Figure 2). A low-risk classification would reduce the pretest probability from 29% to just less than 10% (Figure 2). After combining IPF derivation and validation cohorts, test performance was consistent across different thresholds of age (Table S3), baseline lung function (Tables S4 and S5), antifibrotic use (Table S6), and for death and transplant modeled as separate events (Table S7).

Table 3.

Multidimensional Classifier Test Performance Characteristics for Predicting 18-Month Death or Transplant When Applied to Independent IPF and Non-IPF ILD Validation Cohorts

Classification	IPF validation cohort (n = 105)		Non-IPF ILD validation cohort (n = 295)
	Outcome (+)	Outcome (−)	Outcome (+)	Outcome (−)
High risk	21	6	38	47
Low risk	9	69	16	194
Sensitivity	0.70		0.70
Specificity	0.92		0.80
(+) Predictive value	0.78		0.45
(−) Predictive value	0.88		0.92
(+) Likelihood ratio	8.75		3.61
(−) Likelihood ratio	0.33		0.37

Abbreviations: ILD, interstitial lung disease; IPF, idiopathic pulmonary fibrosis.

When applied in the non-IPF ILD validation cohort, sensitivity was 70% and specificity 80%. The negative predictive value remained high (92%), but the positive predictive value dropped significantly (45%), owing to a lower outcome prevalence in this cohort (Table 3). Likelihood ratios for this cohort suggested that a high-risk classification would increase the pretest probability of near-term outcome (based on observed prevalence) from 18% to approximately 40% (Figure 2), again using observed 18-month incidence of death or transplant to establish the pretest probability. A low-risk classification would reduce the pretest probability from 18% to approximately 6% (Figure 2). When exploring test performance measures in key non-IPF ILD subgroups, the negative predictive value was high across cohorts, but the positive predictive value was low for those with connective tissue disease associated ILD and fibrotic hypersensitivity pneumonitis (Table S2). Test performance in those with non-IPF idiopathic interstitial pneumonia was similar to what was observed in the IPF cohorts.

When plotting transplant-free survival by multidimensional classification, those with a high-risk classification showed significantly worse survival than those with a low-risk classification (Figure 3). High-risk classification status was associated with significantly increased outcome risk in the IPF derivation (hazard ratio (HR) 13.39; 95% confidence interval (CI) 6.05–29.65; p < 0.0001), IPF validation (HR 14.02; 95% CI 8.25–23.8; p < 0.0001), and non-IPF ILD validation cohorts (HR 7.56; 95% CI 4.21–13.58; p < 0.0001) in unadjusted analysis. Adjustment for age, sex, race, smoking history, BMI, baseline immunosuppressant exposure, and baseline antifibrotic exposure did not impact these measures of association (Table S8). Lung function was not included to avoid collinearity, as percent predicted FVC and DLCO were both included in the multidimensional classifier.

Figure 3.

Eighteen-month transplant-free survival for idiopathic pulmonary fibrosis (IPF) derivation (a) IPF validation (b) and non-IPF interstitial lung disease validation cohorts (c) stratified by multidimensional biomarker classification.

Discussion

In this investigation, we present a novel multidimensional classifier of near-term outcome in patients with IPF. This classifier provided good test performance across independent IPF and non-IPF ILD cohorts and produced a positive likelihood ratio of greater than 6 across independent IPF cohorts, suggesting clinically actionable results. Test performance was less compelling when this classifier was applied to those with non-IPF ILD. Beyond measures of test performance, this multidimensional classifier strongly discriminated outcomes across cohorts, producing higher hazard ratios than most categorical biomarkers reported to date. Prospective validation of these findings in transplant-eligible individuals with IPF could help optimize the timing of lung transplant referral and inform future ISHLT guidelines based on classifier risk stratification.

These findings build upon a wealth of clinical and molecular biomarker data published in ILD to date. Clinical features, including age, sex, lung function, and several composite indices that include these measures, have been shown to strongly predict outcomes in patients with IPF and other fibrotic ILDs.^10–13 Our findings corroborate the importance of lung function, which was among the variables of highest importance selected by our multidimensional model. Among molecular biomarkers, those most widely validated to date include peripheral blood leukocyte telomere length, a composite 52-gene signature, stand-alone proteins including KL6, matrix metalloproteinase 7, keratin 19 and its breakdown product CYFRA 21–1, neoepitopes, and several recently described composite proteomic signatures.^15–23 However, while the survival association observed for these biomarkers is clearly reproducible, none have proven to be clinically useful. This reality stems from several key features required of a biomarker.

First, a biomarker must provide actionable results.³² Because antifibrotic therapy is recommended for most patients with IPF upon diagnosis,³³ no biomarkers published to date have been shown to alter this management strategy. While one could argue that high-risk molecular classification could be used to enrich clinical trial cohorts, few trials employ death as a primary end-point, leaving it unclear whether those with a high-risk classification are any more likely to experience a change in FVC, the current gold standard for establishing therapeutic efficacy.³⁴ We attempted to overcome this limitation by selecting an IPF population for whom management may differ based on classifier status. The ISHLT consensus currently advises early referral for all patients diagnosed with IPF, citing unpredictable disease trajectory as its rationale.⁸ Multidimensional classification would allow for more nuanced timing based on prognosis. In finding that nearly 70% to 80% of those with a high-risk classification will die or need a transplant within 18 months of blood draw, our results suggest that this classifier could justify immediate lung transplant referral and expedited work-up for high-risk patients, even with baseline lung function higher than ISHLT recommended referral thresholds. Conversely, we found that < 10% of patients with a low-risk classification died or required transplant over the next 18 months, which could help inform the pace of work-up, decision to list, and timing of listing for those with a low-risk classification.

Key to establishing actionable results for a biomarker is intuitive categorization and transparent reporting of test performance characteristics.³² While continuous biomarker measures often explain a larger proportion of outcome risk than categorical measures, the latter is typically needed for interpretation and clinical adoption. Importantly, measures of association (e.g., risk ratios, odds ratios, and hazard ratios) do not necessarily equate to meaningful measures of test performance (e.g., sensitivity, specificity, predictive values, and likelihood ratios). Underscoring this point is our observation that, while a high-risk classification was associated with > 7-fold increase in outcome risk in those with non-IPF ILD, our multidimensional classifier did not change post-test probability in a clinically meaningful manner. Conversely in IPF cohorts, a high-risk classification corresponded with a high positive likelihood ratio and meaningfully increased post-test probability of 18-month death or transplant, highlighting its potential to prompt early referral.

Augmentation or outperforming easily acquired clinical data is another key biomarker attribute. Clinical prediction models have been shown to discriminate IPF and non-IPF ILD survival with high accuracy across diverse cohorts and settings.^10,11,13,14 As such, outperforming these easily estimated models is a challenge. For this reason, we fitted a stand-alone clinical prediction model for comparison. While we found that the stand-alone proteomic model did outperform the clinical model, a multidimensional model that capitalized on both lung function and 20 proteins outperformed both. Decision curve analysis, an efficient approach to comparing models and allows for a comparison of models over a range of outcome probabilities,^30,31 corroborated these findings.

While our findings support refinement and prospective validation of this promising biomarker in patients with IPF, point estimates generated using semiquantitative data have limited external validity due to batch effects and normalization protocols.^35,36 Commercially available, targeted quantitative multiplex assays are emerging and have the potential to advance this and other composite biomarkers toward clinical implementation.³⁷ Our findings do not support the immediate advancement of this tool for prospective validation in those with non-IPF ILD, as the modest positive and negative likelihood ratios were un-likely to alter post-test probability in a clinically meaningful way. Subgroup analysis suggested that the reduction in test performance was isolated to those with connective tissue disease and fibrotic hypersensitivity pneumonitis. Inflammatory pathobiology may have contributed to these findings and prior work by members of our group suggested that inflammatory proteins may play a more prominent role in non-IPF ILD progression when compared to IPF.^22,23 These findings and others showing poor outcomes among those with non-IPF ILD suggest dedicated investigation in this population is urgently needed.

Our study has several important limitations. First, while we attempted to restrict the cohort to individuals that were potentially eligible for lung transplant, it remains unclear what proportion were referred and how many were deemed ineligible due to unmeasured comorbid conditions. Nearly 50% of patients in the derivation cohort who met the primary end-point did so by undergoing lung transplants, suggesting that most were likely eligible. Sensitivity analysis suggested that the multidimensional classifier performed similarly irrespective of the end-point modeled, increasing confidence that those who died were molecularly similar to those who successfully underwent transplant. Next, clinical prediction models have been developed to predict death in patients listed for lung transplant.^38,39 Our data did not allow for direct comparison to these validated models but suggests that a molecular-based approach could further augment current models. Next, our use of a dichotomous outcome fails to account for those who were censored before reaching the end of the follow-up period; however, few patients were censored (n = 26/714 patients) before 18 months. Next, our cohort was pre-dominantly white, leaving it unclear whether these findings would extend to a more racially diverse population. Finally, semiquantitative proteomic data do not always highly correlate with quantitative measures of the same protein,⁴⁰ as we recently found a strong correlation for nearly 70% of quantified proteins, but a weak correlation for over 30%.²³ For this reason, and those outlined above, prospective validation using quantitative methodology is needed.

Conclusion

Recent advances in proteomic technology have allowed for the identification of novel biomarkers of clinically relevant end-points in patients with IPF and non-IPF ILD. Beyond the biological importance of these biomarkers, a number have a high potential to improve prognostication and risk stratification. Here, we present a composite biomarker tool that showed good test performance for discriminating near-term outcomes among patients with IPF across independent cohorts. Prospective validation of these findings in transplant-eligible patients with IPF would help guide clinical decision-making and optimize the timing of lung transplant referral for this population.

Data sharing statement

Individual-level data will be made available on BioLINCC within 12 months of publication.