Strategies for Reducing False-Positive Screening Results for Intermediate-Size Nodules Evaluated Using Lung-RADS: A Secondary Analysis of National Lung Screening Trial Data

Abstract

Background:

Lung-RADS version 1.1 (v1.1) classifies all solid nodules <6 mm as category 2. Lung-RADS v1.1 also classifies solid intermediate-size (6 to <10 mm) nodules as category 2 if perifissural and having triangular, polygonal, or ovoid shape (indicative of intrapulmonary lymph nodes). Additional category 2 criteria could reduce false-positive results of screening examinations.

Objective:

To evaluate the impact of proposed strategies to reduce false-positive results for intermediate-size nodules on lung cancer screening CT evaluated using Lung-RADS v1.1.

Methods:

This retrospective study entailed secondary analysis of National Lung Screening Trial (NLST) data. Of 1387 solid nodules measuring 6.0–9.5 mm on baseline screening CT examinations in NLST, all 38 nodules in patients who developed cancer, and a random sample of 200 nodules in patients who did not develop cancer, were selected for further evaluation. Cancers were required to correspond with the baseline nodule on manual review. After exclusions, the sample included 223 patients (median age, 62 years; 143 men, 53 women; 196 benign nodules, 27 malignant nodules). Two thoracic radiologists independently reviewed baseline examinations to record nodule diameter and volume using semiautomated software and classify whether nodules had perifissural location, other subpleural location, and triangular, polygonal, or ovoid shape. Different schemes for category 2 assignment were compared.

Results:

Across readers, Lung-RADS v1.1 had sensitivity of 89–93% and specificity of 26–31%. A modification assigning nodules <10 mm with triangular, polygonal, or ovoid shape in other subpleural locations (vs only perifissural location) as category 2 had sensitivity of 85–93% and specificity of 47–51%. Using volume cutoffs in Lung-RADS v1.1 had sensitivity of 89–93% and specificity of 37% (both readers). Both schemes’ sensitivity was not significantly different from Lung-RADS v1.1 (all p>.05). Both schemes’ specificity was significantly better than Lung-RADS v1.1 (all p<.05). Combining the two strategies yielded sensitivity of 85–93% and specificity of 58–59%.

Conclusion:

Classifying intermediate-size nodules with triangular, polygonal, or ovoid shape in any subpleural location (not just perifissural) as category 2, and use of volume- rather than diameter-based measurements, improves Lung-RADS specificity without decreased sensitivity.

Clinical Impact:

The findings can help reduce false-positives results, reducing 6-month follow-up examinations for benign findings.

Introduction

The National Lung Screening Trial (NLST) proved that lung cancer screening CT is effective in reducing mortality from lung cancer [1]. Further analyses of the NLST results and of other data show that lung cancer screening CT is cost-effective in improving patient life expectancy [2]. However, to be truly cost-effective, lung cancer screening programs must limit the number of false-positive scans, which can lead to additional cost, anxiety, radiation, and adverse events from biopsy of benign nodules. To establish a standardized framework for nodule reporting and follow-up, the American College of Radiology developed Lung-RADS [3]. Lung-RADS provides criteria for nodules to be deemed either benign (category 2), in which case a patient may return for an annual screening examination, or suspicious (categories 3 and 4), in which case the patients requires earlier follow-up.

In the initial version of Lung-RADS, only solid nodules measuring less than 6 mm in diameter were classified as category 2. Since the development of Lung-RADS, evidence has accumulated showing that certain pulmonary nodules can be confidently diagnosed as benign intrapulmonary lymph nodes [4–8] and should therefore also be classified as category 2. The typical features of intrapulmonary lymph nodes are a solid attenuation; triangular, polygonal, or oval shape; and a location along pleural surfaces [6]. The current version of Lung-RADS [version 1.1 (v1.1)] applies these features for characterizing a nodule as an intrapulmonary lymph node, and thus classifying the nodule as category 2, to only perifissural nodules measuring less than 10 mm [9]. Because all solid nodules less than 6 mm are classified as category 2, this recognition in Lung-RADS v1.1 of intrapulmonary lymph nodes specifically reduces false-positive results for nodules measuring at least 6 mm but under 10 mm (hereafter, intermediate-size nodules). Intermediate-size solid nodules with a typical shape are also likely to represent intrapulmonary nodules when in additional subpleural locations than currently reflected in Lung-RADS v1.1 [7, 10–12].

While Lung-RADS is the dominant schema for follow-up of nodules in lung cancer screening in the United States, other countries have adopted different approaches. Notably, the Dutch-Belgian NELSON trial followed a scheme that primarily relies on nodule volume, rather than the linear diameter measurements that are central in Lung-RADS [13]. While Lung-RADS v1.1 added nodule volume measurements as an alternative measure, standard radiology practice in the United States continues to rely on diameter measurements given the specialized software and additional time required for volumetric measurement. To date, comparisons between diameter- and volume-based nodule classification have yielded mixed results; some studies show slight advantages for volume measurement [14, 15], while other studies show superiority of diameter measurements [16, 17]. In the study by Silva et al. [14], volumetric measurements allowed improved risk stratification and reduction in follow-up examinations compared with standard diameter-based Lung-RADS evaluation.

This background suggests a couple of strategies to potentially reduce false-positive results for lung cancer screening examinations: (1) to expand the criteria for considering intermediate-size nodules to represent benign intrapulmonary lymph nodes; and (2) to use volume- instead of diameter-based measurements. To our knowledge, these strategies have not been evaluated in terms of their impact on the frequency of false-positive lung cancer screening results, and they have not been compared with each other. We therefore conducted this study to evaluate the impact of proposed strategies to reduce false-positive results for intermediate-size nodules on lung cancer screening CT evaluated using Lung-RADS v1.1.

Methods

Patient Population

This retrospective HIPAA-compliant study used solely nonidentifiable patient data from the NLST that was accessed for purposes of secondary data analysis. Consent to access NLST data was obtained from the National Cancer Data Access System of the National Cancer Institute through a data transfer agreement with the National Cancer Institute. The study was approved by the institutional review board at our institution. The requirement for written informed patient consent was waived for this post hoc analysis.

Briefly, the NLST was a randomized controlled trial of current and former smokers who were screened for lung cancer by either chest radiograph or low-dose chest CT. The CT protocol included one baseline examination and two subsequent annual screening rounds. Patients were followed after the three total annual examinations to evaluate for development of lung cancer and for death [1]. For purposes of this study, only baseline CT examinations were evaluated.

The NLST study readers annotated nodules on each CT examination, recording nodule features including location (lobe and slice number), attenuation, margins, and long- and short-axis diameters. Only the dominant (i.e., largest) annotated nodule on each patient’s baseline CT was considered for purposes of the present study. Patients were eligible for the present study if the dominant annotated nodule had a mean diameter (mean of short- and long-axis diameters) of 6.0 mm to 9.5 mm, was non-calcified with solid attenuation, and had non-spiculated margins, based on the NLST study reader’s annotations. Of the 25,844 patients in the CT arm of the NLST study, 1387 patients (with 1387 dominant nodules) met these criteria (Fig. 1).

Figure 1.

Patient selection flowchart. Dashed arrows represent nodules reassigned between study groups. NLST = National Lung Screening Trial.

The NLST data indicate cancers that developed during follow-up, but do not directly link nodules and cancer diagnoses. During follow-up, lung cancer developed in the same lobe as the dominant nodule in 38 of the 1387 patients. All 38 of these patients were selected for further evaluation. Of the remaining 1349 patients in whom cancer did not develop in the same lobe as the dominant nodule, 200 were randomly selected for further evaluation. The baseline examinations of the total of 238 patients (38 with, and 200 without, cancer in the same lobe as the dominant nodule) were initially reviewed by a board-certified subspecialty-trained thoracic radiologist (MMH with 6 years of posttraining experience). Nodules were excluded based on this review for the following reasons: images not available or not viewable [malignant group, n=0; benign group, n=4]; no nodule found corresponding with the NLST-annotated nodule [malignant group, n=1; benign group, n=3]; nodule deemed to represent atelectasis [malignant group, n=0; benign group, n=1]; nodule was not solid [malignant group, n=5 (part-solid in 3, cystic in 2); benign group, n=1 (ground glass)] (Fig. 1). At the time of the initial review, 11 nodules (all in the benign group) were found to have been annotated with the incorrect lobe by the NLST study reader; for these cases, the correct lobe was recorded for purposes of further analysis. The exclusions resulted in 223 remaining nodules, which underwent independent evaluation by two readers, as described later in this section.

After completion of the independent imaging review, the earlier noted investigator (MMH) performed a subsequent review of certain patients, unblinded to the details of patients’ follow-up. In this final session, the radiologist reviewed the nodules in patients who developed cancer to assess whether the cancer corresponded with the dominant nodule versus with a different nodule in that lobe (e.g., that developed later). The nodule and the subsequent cancer corresponded in 26 patients. In six patients, the cancer was attributed to a different nodule in the lobe with the dominant nodule; these six nodules were reassigned to the benign nodule group. Also in this final session, the radiologist reviewed the nodules in the benign group that had been assigned the incorrect lobe by the NLST study reader to assess whether a cancer later developed in the correct lobe. In one patient, a cancer later developed in the correct lobe, which corresponded with the dominant nodule; this nodule was reassigned to the malignant nodule group. During this final session, nodules were only reassigned between groups; no additional nodules were excluded.

This process resulted in a final study sample of 223 dominant nodules in 223 patients (median age 62 years; 143 men, 53 women; 196 with benign nodules, 27 with malignant nodules).

Image Review

Two investigators [the previously noted investigator (MMH) and a board-certified, subspeciality-trained thoracic radiologists (ARH with 28 years of posttraining experience)] independently reviewed the baseline CT examinations in 223 patients in random order, blinded to whether nodules were benign or malignant, using advanced visualization software (syngo.via, version VB40, Siemens Healthcare, Erlangen, Germany). The readers identified the dominant nodule in each patient based on the nodule locations in the NLST annotations. The readers measured each nodule using a semi-automated segmentation tool available in the advanced visualization software. After the radiologist marks anywhere within a nodule, the software automatically segments the nodule’s margins, which the radiologist may manually adjust. The software then reports the nodule’s volume as well as its short- and long-axis diameters on the slice with the nodule’s greatest long-axis diameter. These parameters were recorded, and the mean diameter (mean of short- and long-axis diameters) was calculated. Because the nodule diameters used for subsequent analysis were based on the measurements obtained using the automated software, the mean diameter may not have been within the range of 6.0 to 9.5 mm that was used for initial nodule selection based on the NLST annotations. The radiologists also recorded whether or not the nodule directly abutted a fissure (i.e., perifissural) and whether or not the nodule directly abutted the costal or mediastinal pleural surface (i.e., other subpleural). Finally, the radiologists assessed in a binary fashion whether or not the nodule’s shape was triangular, polygonal, or oval. Nodules could be considered to have triangular, polygonal, or oval shape only if having smooth margins. Using the readers’ assessments, nodules were classified as exhibiting lymph node characteristics if having perifissural or other subpleural location and triangular, polygonal, or oval shape. Data were entered in REDCap [18].

Lung-RADS Category Assignments

Based on the radiologists’ assessments (including nodules’ mean diameter from the semi-automated measurement) and the algorithm in the Lung-RADS v1.1 document [9], Lung-RADS categories were generated for each nodule. Per Lung-RADS for diameter-based assessment, all nodules < 6 mm as well as perifissural nodules < 10 mm with triangular, polygonal or ovoid shape were assigned category 2. Additional Lung-RADS categories were generated for each patient using three different schemes: (1) Lung-RADS categories when also including nodules in other subpleural locations and with mean diameter < 10 mm and triangular, polygonal or ovoid shape in category 2 (reflecting a potential modification to Lung-RADS v1.1); (2) Lung-RADS categories determined using the current Lung-RADS v1.1 volume cutoffs (i.e., for category 2: under 0.113 ml or under 0.524 ml if perifissural); (3) Lung-RADS categories when incorporating both the expanded criteria for category 2 and the use of current volume cutoffs (i.e., when incorporating both of the prior considerations). For purposes of analysis, Lung-RADS category 2 was considered negative, and category 3 or higher was considered positive. Nodules were also classified as negative or positive based on the NELSON trial algorithm [13], considering nodules negative if having volume < 0.05 ml, or if having a perifissural or other subpleural location and a short-axis diameter < 5 mm.

Extrapolation to NLST Cohort

Because the study sample only included intermediate-size nodules, the observed specificities of Lung-RADS were not anticipated to be representative of the overall specificity of Lung-RADS in the full NLST cohort. Specifically, of the 25,844 patients in NLST, 19,164 (74%) had no nodules (thus assigned category 1), and 3838 (15%) had nodules < 6 mm (thus assigned category 2). Because NLST predated Lung-RADS v1.1, nodules in NLST were not assigned category 2 if measuring at least 6 mm but under 10 mm and if perifissural in location. Thus, the anticipated impact in the overall NLST cohort of assigning category 2 for nodules likely representing lymph nodes was evaluated using the readers’ initially assigned Lung-RADS v1.1 categories and each of the three additionally generated Lung-RADS v1.1 categories for each patient. This assessment of impact in NLST was performed by extrapolating from the number of nodules assigned category 2 among the included 196 patients with benign nodules to the sample of 1349 patients in NLST without cancer from whom these patients had been randomly selected (i.e., presuming 6.9 additional nodules assigned category 2 in NLST for each category 2 nodule in the present study’s sample). Then, the overall percentage of patients in the NLST cohort who would have been assigned category 2 was calculated using the extrapolated value.

Statistical Analysis

Data were summarized descriptively using counts and percentages. Categorical variables were compared using Fisher’s exact test, and continuous variables were compared using the Wilcoxon rank-sum test. Interobserver agreement was calculated for diameter and volume measurements using intraclass correlation coefficients (ICCs) and for binary variables (including Lung-RADS categories dichotomized as negative or positive) using Cohen’s kappa coefficients. Sensitivities and specificities were compared between the different classification schemes for each reader using the McNemar test. A p value < .05 was considered statistically significant. Statistical analysis was conducted in JMP Pro (version 16, SAS Institute, Cary, NC).

Results

Patient and Nodule Characteristics

Table 1 summarizes characteristics of benign and malignant nodules. Patients with malignant nodules were significantly older than patients with benign nodules (median age, 66 vs 62 years, p=.04). The two groups were not significantly difference in terms of sex distribution (p=.36). The mean nodule diameters, as determined using the NLST annotations, ranged from 6.0 to 9.5 mm, using the software for reader 1 ranged from 3.5 to 16.0 mm, and using the software for reader 2 ranged from 2.5 to 15.0 mm. Median nodule size was significantly larger for malignant than benign nodules based on mean diameter (NLST: 7.5 vs 7.0, p=.002; reader 1: 9.0 vs 7.0, p<.001; reader 2: 9.0 vs 7.0m p<.001) and volume (reader 1: 0.30 vs 0.16 mL, p<.001; reader 2: 0.33 vs 0.17 mL, p<.001).

Table 1.

Characteristics of Included Nodules

Characteristic	Benign Nodules (n=196)	Malignant Nodules (n=27)	p
Age (y), median (IQR)	62 (58–66)	66 (61 – 70)	.04
Sex,  Male  Female	127 (65) 69 (35)	16 (59) 11 (41)	.36
Diameter (mm), median (IQR)  NLST  Reader 1  Reader 2	7.0 (6.5 – 7.5) 7.0 (6.0 – 8.5) 7.0 (6.0 – 8.5)	7.5 (7.0 – 9.0) 9.0 (7.5 – 10.0) 9.0 (8.0 – 10.5)	.002 <.001 <.001
Volume (ml), median (IQR)  Reader 1  Reader 2	0.16 (0.11 – 0.27) 0.17 (0.09 – 0.27)	0.30 (0.24 – 0.51) 0.33 (0.24 –0.51)	<.001 <.001
Perifissural location  Reader 1  Reader 2	37 (19) 25 (13)	2 (7) 1 (4)	.07 .14
Other subpleural location  Reader 1  Reader 2	55 (28) 60 (31)	3 (11) 3 (11)	.04 .02
Triangular, polygonal, or oval shape  Reader 1  Reader 2	175 (89) 95 (48)	7 (26) 6 (22)	<.001 .01
Lymph node characteristicsa  Reader 1  Reader 2	85 (43) 73 (37)	0 (0) 2 (7)	<.001 <.001

Unless otherwise specified, values represent count with percentage in parentheses.

^aDefined as exhibiting both (1) perifissural or other subpleural location, and (2) triangular, polygonal, or oval shape.

IQR = interquartile range; NLST = National Lung Screening Trial

The frequency of perifissural location was not significantly different between benign and malignant nodules for either reader 1 (19% vs 7%, p=.07) or reader 2 (13% vs 4%, p=.14). The frequency of other subpleural location was significantly higher for benign than malignant nodules for both reader 1 (55% vs 3%, p=.04) and reader 2 (60% vs 3%, p=.02). The frequency of triangular, polygonal, or oval shape was significantly higher for benign than malignant nodules for both reader 1 (89% vs 26%, p<.001) and reader 2 (48% vs 22%, p=.01). The frequency of lymph node characteristics was significantly higher for benign than malignant nodules for both reader 1 (85% vs 0%, p<.001) and reader 2 (73% vs 2%, p<.001).

Diagnostic Performance of Different Classification Schemes

Table 2 shows the sensitivity and specificity of the various classification schemes for lung cancer, defining category 2 as negative and category 3 or higher as positive. Lung-RADS v1.1 based on standard diameter measurements had a sensitivity and specificity of 93% (25/27) and 31% (60/196) for reader 1 and of 89% (24/27) and 26% (51/196) for reader 2. Two cancers were classified as category 2 by both readers. These cancers were categorized as category 2 because of mean diameter < 6 mm (reader 1: 4.5 and 5.0 mm; reader 2: 4.5 and 5.0 mm); neither was classified by either reader as having a perifissural or other subpleural location, or as having a triangular, polygonal, or oval shape. Figure 2 demonstrates one of these cancers. One additional cancer (Fig. 3) was assessed by reader 1 as having a mean diameter of 8.5 mm and perifissural location but not as having a triangular, polygonal, or ovoid shape (category 3); and by reader 2 as having a mean diameter of 8.5 mm, perifissural location, and a triangular, polygonal, or oval shape (category 2). Figure 4 shows a benign intermediate-size nodule assigned category 2 by both readers on the basis of size, perifissural location, and triangular, polygonal, or ovoid shape. Given the 51–60 nodules assigned category 2 based on perifissural location and size <10 mm, it was extrapolated that a total of 351–413 additional nodules would have been assigned category 2 in NLST, increasing the total number of category 2 nodules in NLST from 3838 to 4189–4251 (i.e., from 15% to 16% of all trial patients).

Table 2.

Sensitivity and Specificity of Subpleural Nodule Classification

Scheme and Reader	Sensitivity	p	Specificity	p
Lung-RADS v1.1 (diameter)  Reader 1  Reader 2	93 (77–98) [25/27] 89 (72–96) [24/27]	– –	31 (25–37) [60/196] 26 (20–33) [51/196]	– –
Modifieda Lung-RADS v1.1 (diameter)  Reader 1  Reader 2	93 (77–98) [25/27] 85 (68–94) [23/27]	>.99b .32b	151 (44–58) [100/196] 47 (41–54) [93/196]	<.001b <.001b
Lung-RADS v1.1 (volume)  Reader 1  Reader 2	93 (77–98) [25/27] 89 (72–96) [24/27]	>.99b >.99b	37 (30–44) [72/196] 37 (30–44) [72/196]	.007b <.001b
Modifieda Lung-RADS v1.1 (volume)  Reader 1  Reader 2	93 (77– 98) [25/27] 85 (68–94) [23/27]	>.99c >.99c	59 (52–66) [116/196] 58 (51–64) [113/196]	.001c <.001c
NELSON  Reader 1  Reader 2	93 (77–98) [25/27] 93 (77–98) [25/27]	>.99b .32b	12 (8–18) [24/196] 14 (10–19) [27/196]	<.001b <.001b

Values represent percentage with 95% CI in parentheses and numerator and denominator in brackets. P values obtained with McNemar’s test.

^aClassification of solid intermediate-size nodules with other subpleural location and triangular, polygonal, or ovoid shape as category 2

^bComparison to Lung-RADS v1.1 (diameter).

^cComparison to modified Lung-RADS v1.1 (diameter)

Figure 2.

59-year-old man undergoing lung cancer screening CT. (A) Axial image from baseline CT examination shows a nodule (arrow) classified as solid and with mean diameter of 4.5–5.0 mm (depending on reader). Nodule was classified as Lung-RADS category 2 because of size < 6 mm. (B) Axial image from 2-year follow-up CT examination shows that nodule has increased in size and became cavitary (arrow). Nodule was eventually diagnosed as a lung cancer. Category 2 assignment represented a false-negative interpretation.

Figure 3.

61-year-old man undergoing lung cancer screening CT. Axial image from baseline CT examination demonstrates a solid nodule (arrow) in the right upper lobe adjacent to the right minor fissure, with mean diameter of 8.5 mm (for both readers). Reader 1 did not consider the nodule to have triangular, polygonal, or ovoid shape and classified the nodule as Lung-RADS category 3. Reader 2 considered the nodule to have triangular, polygonal, or ovoid shape and classified the nodule as Lung-RADS category 2 given the size, perifissural location, and shape. The nodule slowly increased in size on follow-up imaging (not shown) and was eventually diagnosed as lung cancer. Category 2 assignment by reader 2 represented a false-negative interpretation.

Figure 4.

66-year-old man undergoing lung cancer screening CT. (A) Axial and (B) sagittal images from baseline CT examination show a solid nodule (arrow) in the right middle lobe adjacent to an incomplete minor fissure, with mean diameter of 9.5 mm (for both readers). Both readers considered the nodule to have triangular, polygonal, or ovoid shape. Nodule was classified as category 2 by both readers given its size, perifissural location, and shape, consistent with an intrapulmonary lymph node. No cancer developed during follow-up.

A modification of Lung-RADS v1.1 that also included other subpleural nodules (i.e., nodules abutting the costal or mediastinal pleura) < 10 mm as category 2 had no significant difference compared with Lung-RADS v1.1 for sensitivity [reader 1: 93% (25/27), p>.99; reader 2: 85% (23/27), p=.32], but had significantly higher specificity [reader 1: 51% (100/196), p<.001; reader 2: 47% (93/196), p<.001]. Figure 5 shows a benign intermediate-size nodule that was assigned category 3 by both readers using standard Lung-RADS v1.1 but assigned category 2 by both readers on the basis of the modification because of size, other subpleural location, and triangular, polygonal, or ovoid shape. Compared with Lung-RADS v1.1, no additional malignant nodule was assigned category 2 by reader 1, and one additional malignant nodule was assigned category 2 by reader 2; reader 2 assessed this nodule as having a mean diameter of 8.0 mm, other subpleural location, and a triangular, polygonal, or oval shape (Fig. 6). Given the 93–100 nodules assigned category 2 based on perifissural or other subpleural location and size <10 mm, it was extrapolated that a total of 640–688 additional nodules would have been assigned category 2 in NLST, increasing the total number of category 2 nodules in NLST from 3838 to 4478–4526 (i.e., from 15% to 17% of all trial patients).

Figure 5.

61-year-old woman undergoing lung cancer screening CT. Axial image from baseline CT examination shows a solid nodule (arrow) in the right upper lobe with a subpleural location (other than perifissural). Nodule had mean diameter of 9.5 mm (for both readers). Both readers considered the nodule to have triangular, polygonal, or ovoid shape. Both readers categorized the nodule as category 3 using Lung-RADS v1.1, but as category 2 using the explored modification of Lung-RADS v1.1 that considers solid nodules less than 10 mm with other subpleural location and triangular, polygonal, or ovoid shape to likely represent intrapulmonary lymph nodes. No cancer developed during follow-up. Category 3 assignment using standard Lung-RADS v1.1 represented a false-positive interpretation. v1.1 = version 1.1.

Figure 6.

58-year-old woman undergoing lung cancer screening CT. (a) Axial image from baseline CT image shows a solid nodule (arrow) in the right lower lobe with a subpleural location (other than perifissural). Nodule had mean diameter of 8.0 mm (for both readers). Reader 2, but not reader 1, considered the nodule to have triangular, polygonal, or ovoid shape. Both readers categorized the nodule as category 3 using Lung-RADS v1.1. Using the explored modification of Lung-RADS v1.1 that considers solid nodules less than 10 mm with other subpleural location and triangular, polygonal, or ovoid shape to likely represent intrapulmonary lymph nodes, nodule remained classified as category 3 by reader 1 but was classified as category 2 by reader 2. (B) Axial image from 1-year follow-up CT examination demonstrates slight increase in size of nodule (arrow). Nodule was eventually diagnosed as lung cancer. Category 2 assignment using modified Lung-RADS by reader 2 represented a false-negative interpretation. v1.1 = version 1.1.

Lung-RADS v1.1 using existing volume cutoffs had no significant difference compared with Lung-RADS v1.1 using standard diameter-based cutoffs for sensitivity [reader 1: 93% (25/27), p>.99; reader 2: 89% (24/27), p>.99], but had significantly higher specificity [reader 1: 37% (72/196), p=.007; reader 2: 37% (72/196), p<.001]. Compared with Lung-RADS v1.1, no additional malignant nodule was assigned category 2 by either reader. Given the 72 nodules assigned category 2 based on Lung-RADS v1.1 using volume measurements, it was extrapolated that a total of 496 additional nodules would have been assigned category 2 in NLST, increasing the total number of category 2 nodules in NLST from 3838 to 4334 (i.e., from 15% to 17% of all trial patients).

A modification of Lung-RADS v1.1 incorporating both considerations (inclusion of other subpleural nodules in category 2 and volume measurements) had no significant difference compared with the Lung-RADS v1.1 modification that included other subpleural nodules but with diameter measurements for sensitivity [reader 1: 93% (25/27), p>.99; reader 2: 85% (23/27), p>.99], but had significantly higher specificity [reader 1: 59% (116/196), p=.001; reader 2: 58% (113/196), p<.001]. Given the 113–116 nodules assigned category 2 using both subpleural location and volume measurements, it was extrapolated that a total of 778–798 additional nodules would have been assigned category 2 in NLST, increasing the total number of category 2 nodules in NLST from 3838 to 4616–4636 (i.e., from 15% to 18% of all trial patients).

The NELSON criteria showed no significant difference compared with standard Lung-RADS v1.1 for sensitivity [reader 1: 93% (25/27), p>.99; reader 2: 99% (25/27), p>.99], but had significantly lower specificity [reader 1: 12% (24/196), p<.001; reader 2: 14% (27/196), p<.001]

Interobserver Agreement

Interobserver agreement, expressed as ICC, was 0.87 for mean diameter and 0.96 for volume. Interobserver agreement, expressed as kappa, was 0.77 for perifissural location, 0.81 for other subpleural location, 0.20 for shape, and 0.73 for lymph node characteristics. Interobserver agreement, expressed as kappa, was 0.70 for Lung-RADS v1.1, 0.68 for Lung-RADS v1.1 also including other subpleural nodules < 10 mm as category 2, 0.83 for Lung-RADS using volume thresholds, 0.81 for Lung-RADS using both the expanded criteria for category 2 and volume thresholds, and 0.76 for NELSON.

Discussion

This study used a subset of intermediate-size solid nodules from the NLST to evaluate the impact of strategies for decreasing the frequency of false-positive interpretations relating to nodules likely representing benign intrapulmonary lymph nodes. Assigning category 2 for all subpleural nodules < 10 mm (rather than only perifissural nodules < 10 mm), in the presence of a characteristic shape, significantly increased the specificity of Lung-RADS without a significant loss in sensitivity. The use of current Lung-RADS v1.1 volume-based thresholds for assigning category 2, instead of standard diameter-based thresholds, also significantly increased specificity without a significant loss in sensitivity. Combining both of these strategies maximized specificity, still without significant loss in sensitivity. The specificity in the study sample from the combination of strategies would correspond with an anticipated increase in the frequency of category 2 nodules in the entire NLST cohort from 15% to 18% of patients. Adoption of these strategies could therefore help prevent unnecessary 6-month follow-up CT examinations.

The findings support previous studies that showed subpleural nodules with typical characteristics of intrapulmonary lymph nodes to almost always be benign [5, 10–12]. Nonetheless, caution is required during such evaluation. Using Lung-RADS v1.1, one malignant intermediate-size perifissural nodule was classified as category 2 by a single reader based on perceived triangular, polygonal, or ovoid shape. Using the explored modification of Lung-RADS v1.1, one malignant intermediate-size nodule with other subpleural location was classified as category 2 by a single reader based on perceived triangular, polygonal, or ovoid shape. Such cases highlight the importance of being rigorous when assessing the shape of intermediate-size nodules before considering such nodules to likely represent benign intrapulmonary lymph nodes warranting category 2 assignment.

A paucity of studies have compared diameter-based to volume-based criteria for triaging pulmonary nodules, and which method, if either, is better is currently considered inconclusive. The available studies include an analysis of subsolid nodules from the NLST that found the volume-based NELSON algorithm was not superior to Lung-RADS for cancer diagnosis [16], as well as an analysis that found the NELSON algorithm performed worse than Lung-RADS in evaluating nodules detected on follow-up lung cancer screening CT examinations [17]. However, in the present study, volume-based criteria increased the number of benign nodules assigned category 2, significantly improving the specificity of Lung-RADS without a significant loss in sensitivity. This benefit may reflect the relatively flat nature of many benign nodules, particularly intrapulmonary lymph nodes, such that the nodule’s actual volume is smaller than that of a sphere corresponding to its diameter [19]. In comparison, malignant nodules typically exhibit relatively isotropic growth, resulting in a larger volume for a given diameter.

Software for volumetric nodule measurement is not widely available, and its use is time-consuming. It is likely not necessary to measure the volume of all solid nodules–indeed, those nodules that already meet criteria for Lung-RADS category 2 using diameter-based measurement do not need volumetric measurement. However, in solid nodules that do not already meet criteria for category 2 assignment, it may be beneficial to measure the nodule’s volume to determine whether the nodule can be downgraded to category 2, thereby avoiding a follow-up examination. Interobserver agreement was also higher for volume- than diameter-based measurement, indicating an additional benefit of the method.

This study has a number of limitations. The primary limitation is that the NLST data do not specify direct correspondence between nodules and subsequent cancer diagnoses. However, baseline and follow-up imaging examinations were reviewed for all malignant nodules to assess whether the cancer could be attributed to a different nodule, including any that may have developed during follow-up. Second, the reference standard for benign nodules was imaging and clinical follow-up. It is unlikely that any cancers were missed given the multiple years of follow-up after baseline screening in all patients. Third, baseline imaging was evaluated by two subspecialist thoracic radiologists; performance of assessment for characteristics of intrapulmonary lymph nodes by general radiologists also needs to be studied. Fourth, although all malignant intermediate-size nodules were assessed, only a random sample of the benign intermediate-size nodules were evaluated. Finally, the impact of the proposed strategies for reducing follow-up when applied prospectively for real-world clinical cases remains unknown.

In conclusion, the classification of all intermediate-size nodules with perifissural or other subpleural location and with triangular, polygonal, or ovoid shape (suggestive of intrapulmonary lymph nodes) as Lung-RADS category 2 would substantially decrease false-positive lung cancer screening examinations without a significant reduction in sensitivity for lung cancer. Use of volume-based rather than diameter-based criteria would also substantially increase the number of nodules classified as category 2 without a significant reduction in sensitivity. The combination of the two methods achieved maximal specificity. Radiologists may consider using volumetric measurement for solid nodules when not already meeting criteria for category 2 assignment. The described strategies could help reduce recommendations for unnecessary 6-month follow-up CT examinations. Prospective trials would help confirm these findings.

Key Finding:

In comparison with standard Lung-RADS v1.1, classifying intermediate-size nodules with triangular, polygonal, or ovoid shape in any subpleural location (not just perifissural) as category 2, and use of volume- rather than diameter-based measurements, resulted in a significant increase in specificity from 26–31% to 58–89% without significant change in sensitivity.

Importance:

Classifying nodules in any subpleural location as category 2 based on characteristics indicative of intrapulmonary lymph nodes would reduce false-positive results of screening CT examinations.