Quantitative CT Measures of Lung Fibrosis and Outcomes in the National Lung Screening Trial

Abstract

Rationale:

Incidental features of interstitial lung disease (ILD) are commonly observed on chest computed tomography (CT) scans and are independently associated with poor outcomes. While most studies to date have relied on qualitative assessments of ILD, quantitative imaging algorithms have the potential to effectively detect ILD and assist in risk stratification for population-based cohorts.

Objectives:

To determine whether quantitative measures of ILD are associated with clinically relevant outcomes in the National Lung Screening Trial (NLST).

Methods:

Quantitative measures of ILD were generated using low dose CT (LDCT) data collected as part of the NLST and processed with Computer-Aided Lung Informatics for Pathology Evaluation and Ratings (CALIPER) and deep learning-based usual interstitial pneumonia (DL-UIP) algorithms (Imbio Inc., Minneapolis, MN). A multivariable Cox proportional hazard regression model was used to test the association between ILD measures (percent ground glass opacity, reticular opacity and honeycombing of total lung volume and binary DL-UIP classification) and all-cause mortality. Secondary outcomes of incident lung cancer and lung cancer mortality were also explored.

Results:

Quantitative CT data were generated in 11,518 individuals. Mean age was 61.5 years and 58.7% were male. An increased risk of all-cause mortality was observed for each percent increase in CALIPER-derived ground glass opacity (hazard ratio (HR) 1.02, 95% confidence interval (CI) 1.01 - 1.02), reticular opacity (HR 1.18, 95% CI 1.12 - 1.24), and honeycombing (HR 6.23, 95% CI 4.23 - 9.16). Individuals with a positive DL-UIP classification pattern had a 4.8-fold increased risk of all-cause mortality (HR 4.75, 95% CI 2.50 – 9.04). CALIPER derived reticular opacity was also associated with increased lung cancer specific mortality. No quantitative measures of ILD were associated with incident lung cancer.

Conclusions:

Quantitative measures of ILD on LDCT are associated with clinically relevant endpoints in a large at-risk population of individuals with tobacco use history.

Introduction

The National Lung Screening Trial (NLST) showed that screening of tobacco exposed individuals with low dose computed tomography (LDCT) of the chest increased lung cancer detected and reduced lung cancer-specific mortality (1). Brown et al. demonstrated that asymptomatic interstitial lung abnormalities (ILAs) identified on LDCT review by expert radiologists was independently associated with increased risk of lung cancer development and lung cancer-specific mortality (2). ILAs were subsequently shown to be associated with adverse outcomes in other population-based studies of community dwelling adults (3-5).

While early studies relied on LDCT interpretation by expert radiologists, newer studies suggest that quantitative algorithms can effectively identify ILAs and early features of pulmonary fibrosis. One such algorithm is Computer-Aided Lung Informatics for Pathology Evaluation and Ratings (CALIPER), which quantifies several key features of fibrotic lung disease, including ground glass opacity, reticular opacity, and honeycombing (6). Recent studies have demonstrated that CALIPER-based measures of lung fibrosis are associated with mortality in patients with interstitial lung disease (ILD) (7) and do so with greater accuracy than traditional visual CT scoring (8). Whether CALIPER-based measures of fibrosis are similarly associated with outcomes in those undergoing LDCT for lung cancer screening remains unclear. In contrast to quantitative lung texture mapping and quantification algorithms such as CALIPER, multiple algorithms that detect the probability of UIP pattern on CT have been developed (9-11), all of which have been associated with mortality within ILD cohorts.

In this study, we leveraged the NLST cohort to determine whether multiple quantitative measures of lung fibrosis were associated with all-cause mortality in this large population-based lung cancer screening cohort. We then evaluated secondary outcomes, including lung cancer incidence and lung cancer-specific mortality. We assessed two types of quantitative measures of fibrosis, CALIPER and a recently published deep learning-based classifier for usual interstitial pneumonia (DL-UIP) (10).

Methods

Participants

The NLST enrolled individuals who currently or used to smoke between 55 and 74 years old with a minimum 30-pack-year smoking history at 33 sites across the United States. The previously published study protocol (12), was approved by the institutional review boards at each of these sites to study the primary endpoint of lung cancer mortality. Additional approval to use data for these secondary analyses was obtained as part of Project ID NLST-125 within the National Cancer Institute Cancer Data Access System. Data from 11,518 individuals enrolled in the NLST with appropriate baseline LDCT imaging for analysis and clinical data for modeling were analyzed in this study (STROBE diagram (13) can be found in Figure 1).

Figure 1:

Strengthening the Reporting of Observational studies in Epidemiology (STROBE) flow diagram outlining the patient selection criteria of appropriate NLST LDCT scans processed with Imbio Inc’s CALIPER and DL-UIP algorithms

Variable Definitions

Clinical characteristics (demographics, smoking history, personal history of cancer and family history of lung cancer) were collected at enrollment. CT acquisition parameters can be found in Table 6 of the previously published NLST study design (12). Imbio software was used to generate 1) automated, quantitative CALIPER-based measures of lung disease (percent of total lung volume affected by ground glass opacity, reticular opacity, and honeycombing), 2) an automated densitometric measure of emphysema (percent of total lung volume comprised of low attenuation area (LAA) with voxels less than −950 Hounsfield units)(14), and 3) DL-UIP classification using previously published methods (10).

CALIPER software automatically performs anatomic segmentation and lung parenchymal characterization of each LDCT scan and classifies the pattern in each lung pixel to calculate the absolute volume of each pattern. The DL-UIP output is a UIP pattern confidence score, a continuous variable ranging from 0 to 1, where a score of 1 indicates high confidence a UIP pattern is present. Those with DL-UIP model scores of ≥ 0.2 were classified as DL-UIP positive while those with scores <0.2 were classified as DL-UIP negative (10). The 0.2 threshold was selected as the maximal difference between true and false positives in the validation cohort of our previously published work (10). Test performance characteristics of this DL-UIP classifier can be found in Table 2 of this publication (10).

Table 2:

Adjusted Cox regression models for the relationship between quantitative imaging variables (CALIPER and DL-UIP) and all-cause mortality

Imaging Variable (% total lung volume)	Hazard ratio (HR)	Confidence interval (CI)	p-value
Reticular opacity (RO)	1.18	1.12 – 1.24	p < 0.001*
Ground glass opacity (GGO)	1.02	1.01 – 1.02	p < 0.001 *
Honeycombing (HC)	6.23	4.23 – 9.16	p < 0.001 *
DL-UIP positive	4.75	2.50 – 9.04	p < 0.001 *

^*=p < 0.05

Outcomes

Participant outcomes were provided in the NLST dataset. The primary outcome of interest was overall survival, defined as time in years from study entry to death from any cause. Time to incident lung cancer and lung cancer-specific mortality were assessed as secondary outcomes. Lung cancer diagnoses were confirmed through pathology reports. Death from lung cancer used information from the Endpoint Verification Process (EVP) and death certificates if EVP was not available (15). To determine whether death was from a respiratory or non-respiratory related cause, we manually reviewed the available death certificate information.

Statistical Analysis

Continuous variables are reported using the mean and standard deviation, while categorical variables are reported as count and percentage. The association between continuous CALIPER measures and the outcomes of interest were assessed using multivariable Cox proportional hazards regression models adjusted for potential confounders of the exposure-outcome relationship, including age, sex, race, personal and family cancer history, smoking history (smoking status and pack-years) and %LAA (Figure E1). The same model (Figure E2) was used when assessing the association between these outcomes and binary DL-UIP classification, with the addition of CALIPER-based reticular opacity as a covariate, as the extent of fibrosis on CT likely increases the likelihood of a radiologic UIP pattern (16) and adverse outcomes.

Death was treated as a competing event when modeling time to incident lung cancer and lung cancer-specific mortality and an inverse probability weighting approach was used to account for dependent censoring from competing cause of death in these models. Outcome-free survival was censored at 6 years. The proportional hazard assumption was checked for all Cox regression models, with results omitted from further analysis when this assumption was violated. CALIPER derived fibrosis measures were then categorized by tertile and survival displayed using the Kaplan-Meier estimator.

All analyses were performed in R software version 4.2.2 (R Foundation for Statistical Computing). The following R packages were used: “survival”, “ipw”, “tidyverse”, “survminer” and “ggplot2”. P-values < 0.05 were considered statistically significant.

Visual CT Assessment

To evaluate concordance between the DL-UIP classifier and visual assessment of CT scans, an expert thoracic radiologist rated 96 scans in the NLST (all scans with DL-UIP score ≥ 0.1 and a random subset of scans with DL-UIP score < 0.1) as either UIP positive (cases of probable or definite UIP) or UIP negative (indeterminate or alternative diagnosis).

Results

Participant Characteristics

All individuals enrolled in the NLST (n = 11,518) with appropriate imaging and outcome data were included (Table 1) in multivariable Cox regression models for all the CALIPER variables and DL-UIP classification. The same subset of participants in the NLST was used for both analyses. In this cohort, individuals were on average 61.5 ± 5.0 years old and predominantly (>90%) non-Hispanic white. 58.7% of individuals were male. Nearly half of all individuals currently smoked with an average 56.1 pack-year smoking history. Of all the individuals in this analysis, 19 (0.2%) were classified as DL-UIP positive and 11,499 (99.8%) as DL-UIP negative. A small subset of NLST patients reported a personal history of any cancer type (470 individuals, 4.1%) or a family history of lung cancer (2,555 individuals, 22.2%). In the entire cohort, median CALIPER metrics (interquartile range) for ground glass opacity, reticular opacity and honeycombing were 0.15% (0.04-0.81), 0.26% (0.12-0.65), and 0.01% (0.00-0.02) respectively.

Table 1:

Demographics in the NLST cohort. Data presented as mean (standard deviation) for continuous variables and counts (percentages) for categorical variables

Demographics		Full cohort
		N = 11518
Age, years		61.5 (5.0)
Race	Non-Hispanic White	10480 (91.0)
	Non-Hispanic Black	624 (5.4)
	Hispanic or Other	4141 (3.6)
Male		6760 (58.7)
Smoking pack-years		56.1 (23.7)
Current tobacco use		5639 (49.0)
Personal history of any cancer		470 (4.1)
Family history of lung cancer		2555 (22.2)

Quantitative Fibrosis Measures and All-cause Mortality

725 individuals died of any cause during 6 years of follow-up, with 125 (17%) from a non-lung cancer related respiratory etiology. An increased risk of death was observed for each percent increase in all CALIPER-based measures, including ground glass opacity (hazard ratio (HR) = 1.02, 95% confidence interval (CI) 1.01 - 1.02, p < 0.001), reticular opacity (HR = 1.18, 95% CI 1.12 - 1.24, p < 0.001), and honeycombing (HR = 6.23, 95% CI 4.23 – 9.16, p < 0.001) (Table 2). In Figure 2, Kaplan Meier curves display overall 6-year survival for groups divided by tertile of all the CALIPER derived variables.

Figure 2:

Kaplan Meier curves estimating 6-year survival by tertile of all CALIPER derived fibrosis variables, (a) reticular opacity (RO), (b) ground glass opacity (GGO), and (c) honeycombing (HC)

In the 725 NLST individuals who died during the 6-year follow-up period, 8 out of 19 (42%) were in the DL-UIP positive group and 717 out of 11,499 (6.2%) were in the DL-UIP negative group (HR = 4.75, 95% CI 2.50 – 9.04, p < 0.001) (Table 2). In Figure 3, Kaplan Meier curves show the overall 6-year survival by DL-UIP classification group.

Figure 3:

Kaplan Meier curves estimating 6-year survival by DL-UIP classification

Quantitative Fibrosis Measures and Lung Cancer-related Outcomes

471 individuals developed incident lung cancer during the follow-up period and 163 of these individuals died from lung cancer. None of the CALIPER based quantitative measures of lung fibrosis were associated with incident lung cancer and these models were not included in further analyses as they violated the proportional hazard assumption. An increased risk of lung cancer-specific mortality was observed for each percent increase in the CALIPER-based measure of reticular opacity (HR = 1.12, 95% CI 1.02 - 1.23, p = 0.02) (Table 3). Kaplan Meier curves displaying lung cancer-specific mortality for groups divided by tertile of CALIPER derived reticular opacity are shown in Figure E3. Given the small number of DL-UIP positive participants who developed lung cancer in this study (n = 1), we could not conclude whether DL-UIP classification was associated with incident lung cancer or lung cancer-specific mortality.

Table 3:

Adjusted Cox regression models for the relationship between CALIPER fibrotic variables and lung cancer mortality

Imaging Variable (% total lung volume)	Hazard ratio (HR)	Confidence Interval (CI)	p-value
Reticular opacity (RO)	1.12	1.02 - 1.23	0.02 *
Ground glass opacity (GGO)	1.01	0.98 - 1.03	0.59
Honeycombing (HC)^	3.29	0.79 - 13.8	0.10

^*=p < 0.05

^{^}Variable did not meet the proportional hazards assumption in this model and was omitted from subsequent analyses

Comparing DL-UIP and Visual UIP Classification

On review of 96 CT scans in the NLST by an expert thoracic radiologist, 10 out of 23 (43%) DL-UIP positive individuals were identified as visual UIP positive (Table E4). 70 out of 73 (96%) DL-UIP negative individuals were identified as visual UIP negative (Table E4). Imaging from the 13 false positive individuals were reviewed and revealed primarily paraseptal emphysema and subpleural bronchiolectasis, nearly all of which occurred in the setting of underlying fibrosis.

Discussion

In this investigation, we demonstrated that quantitative imaging algorithms identified evidence of fibrotic lung disease in a large national lung cancer screening cohort that was associated with poor clinical outcomes, namely decreased overall survival. Both traditional machine learning based algorithms such as CALIPER and a fully automated, AI-based classification tool (DL-UIP) identified cases of possible ILD among tobacco exposed individuals in the NLST.

One of the greatest opportunities for the use of these findings is the increasingly widespread adoption of LDCT for lung cancer screening in tobacco exposed individuals annually. Flagging CT scans as suspicious for fibrotic lung disease using algorithms like CALIPER allows for additional review and diagnostic confirmation. An increased suspicion for fibrosis then allows for early pulmonary referral to diagnose ILD, monitor pulmonary function and identify disease progression through repeat imaging from annual LDCT lung cancer screening and ultimately, discussion of eligibility for anti-fibrotic therapy (17, 18). While CALIPER was not originally trained on LDCT images, the findings in this work highlight that survival differences are still apparent in large cohorts despite differences in image quality and resolution. This has been validated in recent work showing that LDCT can accurately detect evidence of ILD even with reduced dose settings of 20 mA and 3-5 mm slice thickness (19).

It is now becoming increasingly important to identify evidence of fibrosis prior to symptom onset. As the NLST was primarily a lung cancer screening cohort, no symptom data was collected as part of study enrollment. Obtaining additional corroborating clinical information would allow us in future studies to assess whether these imaging findings of ILD were in individuals who may have had asymptomatic ILAs or in patients with unexplained dyspnea. We are unable to determine whether these individuals could have carried a prior diagnosis of ILD, although an exclusion criterion in the NLST was having had a chest CT in the 18 months prior to eligibility assessment.

Many UK studies have focused on individuals with ILAs and early ILD in their lung cancer screening cohorts. In one UK single center lung cancer screening pilot of over 1,800 individuals with similar characteristics to the NLST, there were 28 cases (1.5%) of incident ILD, for which many patients were started on antifibrotic and/or immunomodulatory treatment after diagnosis (20). A separate pilot in socially disadvantaged areas of Manchester, UK showed that the identification of fibrotic ILAs identified in their lung cancer screening cohort was an independent risk factor for the development of ILD and mortality (21). Finally, the Yorkshire Lung Screening Trial had similar findings (2.5% of screening CTs reported as having an ILA) and is the largest screening cohort of over 6,600 participants to have reported ILD diagnoses and outcomes (22). Prior studies have also found an increase in ILAs on LDCT in older individuals without a smoking history in China (23). With these possibilities, there may be advantages to extending eligibility of LDCT beyond tobacco exposed individuals by incorporating ILD as a risk factor. Further research is needed to specifically define the role of LDCT in this screening process by understanding the independent associations of fibrotic changes in individuals below current lung cancer screening eligibility thresholds.

DL-UIP is a separate algorithm with its own unique advantages in this study. While the number of cases identified as DL-UIP positive was small, individuals classified DL-UIP positive had a much greater risk of dying during the follow-up period. Even in the cases of false positive classifications, survival outcomes were worse in individuals with false positive UIP classifications compared to outcomes in the true negative reference group (10). Because underreporting of ILD remains common (24), AI-based imaging software has potential to reduce diagnostic delays in patients with early ILD who may also be symptomatic given these patients are likely UIP positive rather than simply have some features of pulmonary fibrosis. The findings here extend those previously published by our group showing that this automated DL-UIP algorithm discriminated survival in a multi-center ILD cohort with similar test performance as visually determined UIP (10). These findings also build on those recently published by Humphries et al. (9), which showed that a DL-UIP classifier trained on histopathological UIP effectively discriminated survival in a large multi-center ILD cohort. These studies suggest that DL-based UIP classification can effectively risk stratify patients with known ILD and those at risk for developing ILD.

While deep learning algorithms have even been used to identify ILD using chest radiographs (25), our study has extended these algorithms for use in LDCT, which has not been previously published. As DL-based algorithms become widely available, deployment of these tools has high potential to impact patient care, especially when deployed in healthcare settings without thoracic radiologists. Given the rare nature of ILD, the proportion of individuals anticipated to be DL-UIP positive is expected to be small. Our results are similar to findings from an Italian lung cancer screening cohort where 0.3% of participants had a UIP pattern on CT (26), and the American College of Radiology Lung Cancer Screening Registry which reported evidence of pulmonary fibrosis in 0.5% of nearly 1.7 million lung cancer screening CTs (27). The poor survival observed among DL-UIP positive cases support the importance of implementation. Further studies in populations with greater DL-UIP positive cases would help determine whether DL-UIP classification is associated with lung cancer-specific clinical outcomes.

This study had several limitations. First, the automated CALIPER algorithm in some cases incorrectly segments the lungs from the rest of the body on HRCT. In some of these cases, the intestines or the liver may be included in lung segmentation, often leading to incorrectly elevated levels of emphysema, ground glass, or reticulation. We attempted to minimize this effect by removing several hundred CT scans flagged for poor segmentation quality using a separate, deep learning-based segmentation algorithm (TotalSegmentator (28)), and removing cases that had large differences in Dice coefficient (< 0.9). This was only a small portion of the total scans available but there could be some remaining scans with residual segmentation issues that were not accounted for in this process.

While our CALIPER fibrosis measures had strong associations with mortality outcomes, the small sample size of those classified as DL-UIP positive, which reduced the precision of the estimates observed. This is expected given the deployment of this technology in a screening population without known ILD and additional validation in other at-risk cohorts should be pursued. We also acknowledge one of the challenges many automated quantitative algorithms, including our DL-UIP classifier, face is the risk of false negatives when they are deployed for screening purposes.

Next, while we attempted to account for potential confounders of the exposure-outcome relationship, residual confounding remains possible. One potential source of residual confounding stems from the inability to include pulmonary function data, which was not available for a large portion of NLST participants in this analysis. However, the consistency of our findings which were replicated in both this large population-based screening cohorts and in our prior study in a multi-center ILD cohort (10) which did adjust for lung function, increases confidence these findings represent true associations.

Beyond segmentation issues, CALIPER has known limitations relevant to this work. CALIPER is unable to detect evidence of traction bronchiectasis or easily differentiate between honeycombing and cystic emphysema, which can be more common in this high-risk population with extensive tobacco use history. With the exciting development of newer iterations of deep learning based quantitative imaging algorithms, some of these questions may be answered in the near future.

In conclusion, automated, quantitative imaging-based algorithms are identifying fibrotic changes earlier, and with additional verification, show promise to improve ILD detection and risk stratification across diverse care settings, in particular in resource scarce settings without thoracic radiologists. Further studies with newer iterations of these algorithms are needed to better delineate their specific role in the screening and diagnostic process in at-risk tobacco exposed individuals as newer therapies for ILD become increasingly available.

Conclusions

Quantitative imaging markers of fibrosis obtained from LDCT chest imaging can provide a wealth of potentially clinically actionable information for tobacco exposed patients undergoing annual lung cancer screening. Traditional machine learning algorithms such as CALIPER and newer iterations of automated AI-based methods, including the DL-UIP classifier, can identify fibrotic changes and the presence of a UIP pattern on LDCT, respectively. Results from both quantitative imaging algorithms were associated with increased all-cause mortality in the population based NLST cohort. Identifying evidence of pulmonary fibrosis on a screening CT in a large cohort of at-risk individuals has the promise to affect timely clinical decision making.

Supplementary Material

Online Supplement: This article has an online supplement, which is accessible at the Supplements Tab