Table 4:
Comparison of Specificity and Sensitivity of Sybil, an Open-Source Deep Learning Model, with Those of Lung-RADS v2022
| A: Matched Sensitivity | ||||
|---|---|---|---|---|
| Subgroup | Matched Sensitivity (%) | Specificity, Lung-RADS (%) | Specificity, Sybil Model (%) | P Value |
| Entire subset (n = 250) | 44.4 (31.9, 57.5) [28/63] | 95.2 (91.1, 97.8) [178/187] | 86.6 (80.9, 91.2) [162/187]* | .14 |
| Heavy smoking (n = 46) | 33.3 (9.9, 65.1) [4/12] | 88.2 (72.5, 96.7) [30/34] | 97.1 (84.7, 99.9) [33/34]† | .47 |
| Never- or light-smoking (n = 204) | 47.1 (32.9, 61.5) [24/51] | 96.7 (92.5, 98.9) [148/153] | 83.0 (76.1, 88.6) [127/153] | .002 |
| B: Matched Specificity | ||||
|---|---|---|---|---|
| Subgroup | Matched Sensitivity (%) | Specificity, Lung-RADS (%) | Specificity, Sybil Model (%) | P Value |
| Entire subset (n = 250) | 95.2 (91.1, 97.8) [178/187] | 44.4 (31.9, 57.5) [28/63] | 27.0 (16.6, 39.7) [17/63] | .19 |
| Heavy smoking (n = 46) | 88.2 (72.5, 96.7) [30/34] | 33.3 (9.9, 65.1) [4/12] | 50.0 (21.1, 78.9) [6/12] | >.99 |
| Never- or light-smoking (n = 204) | 96.7 (92.5, 98.9) [148/153] | 47.1 (32.9, 61.5) [24/51] | 25.5 (14.3, 39.6) [13/51] | .07 |
Note.—Data in parentheses are 95% CIs. Data in brackets indicate the numerator and denominator of true-positive findings over total positive findings for sensitivity and true-negative findings over total negative findings for specificity, as determined by the Lung CT Screening Reporting and Data System (Lung-RADS) or Sybil. Statistical comparisons were performed with the McNemar test. In a subset of 250 individuals—comprising 63 cases of visible lung cancer and 187 noncancer controls—no significant differences were observed between Sybil and Lung-RADS in terms of sensitivity or specificity. However, in the never- or light-smoking subgroup, Sybil showed lower specificity than Lung-RADS when sensitivity was matched. Individuals with heavy smoking history were defined as those with at least 20 pack-years of smoking, including those who currently smoke and those who had quit within the past 15 years. All other individuals who ever smoked but did not meet these criteria were classified as having a light smoking history (median, 2 pack-years in this study). Individuals with a neversmoking history were defined as those who had never smoked. An age- and sex-matched case-control sample was obtained at a 1:3 ratio (n = 250). Individuals with visible lung cancer were classified as true-positive cases (n = 63), while those without incident lung cancer were classified as true-negative cases (n = 187).
As an exact match in sensitivity with Lung-RADS was not available, the closest sensitivity (41.3%) was selected (Sybil cutoff, 1.2%).
As an exact match in sensitivity with Lung-RADS was not available, the closest sensitivity (25.0%) was selected (Sybil cutoff, 3.9%).