Rates of positive lung cancer screening examinations in academic versus community practice

Abstract

The benefits and harms of lung cancer screening reported in the National Lung Screening Trial (NLST) likely differ from those observed in academic and community settings. High rates of positive screening findings in the NLST led to the development of the Lung CT Screening Reporting and Data System (Lung-RADS) to standardize interpretation and reporting. We conducted a prospective observational study of lung cancer screening data from four lung cancer screening sites in North Carolina to compare prospective use of Lung-RADS in a real-world screened population versus Lung-RADS retrospectively applied to the NLST, and to determine if Lung-RADS assessment use differs in academic versus community settings. We included 4,037 screening examinations from 11/2014 to 12/2018 in academic and community sites and 75,126 NLST LDCT screening exams. On baseline screening exams, the proportion of positive LDCT exams was higher in community versus academic sites or the NLST (17.7% vs. 11.4% and 13.6%, P value <0.01). On subsequent screens, the proportion of positive exams was lowest in the NLST and higher in community and academic sites (5.9% vs. 12.7% and 11.6%, P value <0.01). After adjusting for age, race, sex, and smoking status, patients screened at academic versus community sites were 34% less likely to have a positive screen at baseline [adjusted odds ratio (aOR) =0.66; 95% confidence interval (95% CI): 0.51–0.86] but on subsequent examinations, there was no difference in academic versus community sites (aOR =0.91; 95% CI: 0.58–1.43). Our findings may be due to differences in radiologists’ training or experiences or the availability of prior images for comparison.

Introduction

Based on National Lung Screening Trial (NLST) results, the US Preventive Services Task Force (USPSTF) recommends lung cancer screening with low-dose computed tomography (LDCT) for current and former heavy smokers ages 55–80 years (1,2). The NLST protocol defined a positive screening finding as a noncalcified nodule measuring 4mm or greater in the longest diameter (3). Using this threshold, false-positive rates in the NLST were 26.6% at baseline and 21.8% after baseline. Concern over these high false-positive rates observed in the NLST led to development of the Lung CT Screening Reporting and Data System (Lung-RADS) by the American College of Radiology (4).

Lung-RADS is currently in use for nodules detected on LDCT for lung cancer screening and was designed to standardize interpretation and reporting by radiologists, provide consistent management recommendations, and facilitate outcome monitoring (4). A retrospective analysis of the NLST data that applied Lung-RADS criteria found a reduction in false-positive rates to 12.8% at baseline and 5.3% after baseline (5); however, there is limited prospective data on the use of Lung-RADS assessment categories in real world clinical settings. Hence, we sought to compare use of Lung-RADS assessments in a real-world screening population versus the retrospective application to the NLST, and to determine if Lung-RADS use differs in academic versus community settings.

Methods

As part of the North Carolina Lung Screening Registry (NCLSR) we conducted a prospective observational study of lung cancer screening data from four lung cancer screening sites in North Carolina. The NCLSR is funded by the National Cancer Institute to collect patient, radiologist and outcomes data on lung cancer screening to evaluate the adoption of this cancer screening recommendation in population-based settings. We included all NCLSR patients who underwent LDCT for lung cancer screening from 11/2014 to 12/2018.

Using electronic medical record systems data and abstracted radiology text reports, we ascertained patient characteristics and Lung-RADS assessment categories for each screening exam. Patient characteristics included age, race, sex, and smoking status (current versus former). We dichotomized the radiologist reported Lung-RADS assessment into negative [Lung-RADS 1 (Negative) and Lung-RADS 2 (Benign Appearance or Behavior)] and positive [Lung-RADS 3 (Probably Benign) or 4A, 4B, or 4X (Suspicious)]. We dichotomized into these groups to be comparable to the retrospective analysis of the NLST (5). We classified the NCLSR screening sites as academic (n=2) or community (n=2).

We compared the Lung-RADS distribution for baseline and subsequent screening exams in the NCLSR with those in the retrospective analysis of the NLST (5) using chi-square tests. To adjust for potential differences in patient characteristics across screening sites, we employed a logistic regression to compare the likelihood of positive versus negative screening results in academic versus community sites. We stratified the analysis by baseline versus subsequent screen, and included the following covariates: age, race, sex, and smoking status. Because some patients had more than one subsequent screening exam, we controlled for patient-level correlations by including an R-side compound-symmetric random effect. This study was approved by the University of North Carolina Institutional Review Board (No. 17-2352).

Results

We included 4,037 lung cancer screening exams conducted in 2,861 NCLSR patients. The study population undergoing screening was 51.3% ages less than 65 years, although those screened at academic versus community sites were more likely to be ages 65 and older (59.0% vs. 45.2%, respectively, P value <0.01; Table 1). Among those with a known race (n=2,632), 80.6% were white, 16.9% were black, and 2.5% were another race. Overall, 8.0% of patients had missing race and these patients were from community sites. Compared with community sites, patients screened at academic sites were more likely to be black (25.7% vs. 13.5%, P value <0.01). The study population included 46.9% females and 53.4% current smokers, with no differences seen in academic versus community imaging sites.

Table 1. Characteristics of patients who underwent LDCT lung cancer screening in the North Carolina Lung Screening Registry (NCLSR) in academic and community sites.

Characteristic	NCLSR			P value*
	ALL (N=2,861), n (%)	Academic (N=731), n (%)	Community (N=2,130), N (%)
Age group, years				<0.01
<65	1,468 (51.3)	300 (41.0)	1,168 (54.8)
65+	1,393 (48.7)	431 (59.0)	962 (45.2)
Race				<0.01
White	2,122 (80.6)	520 (71.1)	1602 (84.3)
Black	444 (16.9)	188 (25.7)	256 (13.5)
Other	66 (2.5)	23 (3.2)	43 (2.3)
Unknown**	229 (8.0)	0 (0)	229 (10.8)
Sex				0.09
Female	1,341 (46.9)	323 (44.2)	1,018 (47.8)
Male	1,520 (53.1)	408 (55.8)	1,112 (52.2)
Smoking status				0.13
Current	1,527 (53.4)	408 (55.8)	1,119 (52.5)
Former	1,334 (46.6)	323 (44.2)	1,011 (47.5)

*, P value comparing academic vs. community in NCLSR. **, unknown values are excluded from the column percentages.

The distribution of Lung-RADS in the NLST, the NCLSR academic sites, and the NCLSR community sites, stratified by baseline and subsequent lung cancer screening examinations is shown in Table 2. On baseline screening exams, the proportion of positive LDCT exams was statistically significantly higher in NCLSR community sites (17.7%) versus in NCLSR academic sites (11.4%; P value <0.01) and in the NLST (13.6%; P value <0.01). Among subsequent screening exams, the proportion of positive exams was significantly lower in the NLST (5.9%) compared with the NLSR at academic (11.6%; P=0.01) or community (12.7%; P value <0.01) sites. Among NCLSR patients screened in subsequent rounds, the proportion with a positive LDCT result was similar in community and academic settings (P value =0.61). Even after adjusting for patient age, race, sex and smoking status, baseline lung cancer screening exams in NCLSR academic settings remained less likely to have a positive Lung-RADS [P value <0.01; adjusted odds ratio (aOR) =0.66, 95% confidence interval (95% CI): 0.51–0.86] compared to exams in community settings. There was no difference in the likelihood of a positive Lung-RADS assessment for subsequent screening exams in academic versus community settings after adjustment for patient characteristics (P value =0.68; aOR =0.91, 95% CI: 0.58–1.43).

Table 2. Comparison of lung-RADS assessment on baseline and subsequent lung cancer screening examinations in the National Lung Screening Trial and the North Carolina Lung Screening Registry by Site Type and Screening Round.

Lung-RADS	Baseline screening				Subsequent screening
	NLST* (N=26,455), n (%)	NCLSR academic (N=731), n (%)	NCLSR community (N=2,130), n (%)		NLST* (N=48,671), n (%)	NCLSR academic (N=319), n (%)	NCLSR community (N=857), n (%)
Lung-RADS assessment
1	14,709 (55.6)	272 (37.2)	989 (46.4)		25,201 (51.8)	79 (24.8)	325 (37.9)
2	8,145 (30.8)	376 (51.4)	765 (35.9)		20,606 (42.3)	203 (63.6)	423 (49.4)
3	1,697 (6.4)	45 (6.2)	204 (9.6)		588 (1.2)	13 (4.1)	63 (7.4)
3 or 4A	97 (0.4)	N/A	N/A		0	N/A	N/A
3, 4A, or 4B	193 (0.7)	N/A	N/A		0	N/A	N/A
4A	1,107 (4.2)	22 (3.0)	111 (5.2)		851 (1.7)	14 (4.4)	23 (2.7)
4A or 4B	0	N/A	N/A		280 (0.6)	N/A	N/A
4B	358 (1.4)	10 (1.4)	55 (2.6)		896 (1.8)	8 (2.5)	20 (2.3)
4X	149 (0.6)	6 (0.8)	6 (0.3)		249 (0.5)	2 (0.6)	3 (0.4)
Lung-RADS categorized
Negative	22,854 (86.4)	648 (88.6)	1,754 (82.3)		45,807 (94.1)	282 (88.4)	748 (87.3)
Positive	3,601 (13.6)	83 (11.4)	376 (17.7)		2,864 (5.9)	37 (11.6)	109 (12.7)

* REF: Pinsky PF et al. Ann Intern Med 2015.

Discussion

We found that on baseline lung cancer screening exams, positive rates in academic centers were similar to those in the NLST while rates in community sites were higher. On subsequent screening exams, the positive rate at academic sites did not decrease and while the positive rate did decrease for community sites, it did not decrease to levels observed in the NLST. The NLST was conducted primarily in urban, tertiary care hospitals with strict attention to image acquisition, regulation of radiologic interpretation conducted primarily by thoracic radiologists, and detailed follow-up protocols for nodules. These have led to concerns about the generalizability of the NLST results to lung cancer screening in community settings (1,6-8). Our results show the proportion of positive lung cancer screening examinations varied by screening round (baseline vs. subsequent) and imaging center type (academic vs. community), even after adjusting for patient characteristics.

The rates of positive lung cancer screening exams we observed (11.4–17.7% at baseline and 11.6–12.7% on subsequent exams) are similar to those of a 2017 study of lung cancer screening at Cancer Research Network Sites, which reported 10.7% to 21.5% of LDCT exams were positive, although this study did not examine baseline and subsequent screens separately (9). Data from 1,181 baseline lung cancer screening exams conducted as part of an academic screening program reported a positive rate of 12.7%, which is similar to our rate of 11.4% (10). Another study reported the rate of positive lung cancer screening exams among 500 baseline exams in community practice as 24.6%, which is higher than rates in our study (11). Reasons for differences between studies may include the underlying lung cancer risk in the study populations, or the type of radiologist. Our finding of a sustained increase in the proportion of positive screening exams on subsequent scans has implications for screening programs that will conduct annual screening beyond the 3-years analyzed in the NLST.

Our finding of a higher proportion of positive lung cancer screening exams in community versus academic settings at baseline but not on subsequent examinations may be due to radiologists’ training, the availability of prior images for comparison, or clinical experience. In our study, screening exams conducted at academic sites were interpreted by cardiothoracic trained radiologists while those performed at community sites were more likely to be interpreted by general radiologists. The availability of prior images for comparison likely reduced the proportion of patients with a positive LDCT exam (i.e., those who were recalled before the next annual screen) in community sites on subsequent screenings yet we did not see this finding for academic sites. Recall rates in breast cancer screening are significantly lower in mammograms with versus without a comparison mammogram (6.9% vs. 14.9%) (12). Radiologists’ clinical experience could also play a role in the rate of positive screening exams, similar to that observed in mammography interpretation (13).

The rate of positive lung cancer screening exams in the original NLST study (pre-Lung-RADS criteria) was 26.6% at baseline and 21.8% after baseline. Studies evaluating the net benefit and overall cost effectiveness of lung cancer screening have used the pre-Lung-RADS false positive rates but these rates are likely over estimates. In our study the rates of positive screening examinations from over 4,000 screening exams ranged from 12.3% to 16.0%. Subsequent work will evaluate false positive and false negative rates to help inform the benefit to harm ratio of lung cancer screening in real-world setting. Future research is needed to evaluate the impact of imaging site as well as radiologist training and experience on screening outcomes.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the University of North Carolina Institutional Review Board (No. 17-2352). Informed consent was taken from all individual participants.