Comparison of Observed Harms and Expected Mortality Benefit for Persons in the Veterans Health Affairs Lung Cancer Screening Demonstration Project

Abstract

This study examines how a high false-positive rate in lung cancer screening influences the harm-to-benefit ratio for higher- vs lower-risk patients.

The Veterans Health Affairs (VHA) lung cancer screening (LCS) demonstration project identified a much higher false-positive rate following initial low-dose computed tomographic screening than did the National Lung Screening Trial (58.2% vs 26.3%). Most false-positive results (nodules not confirmed to be lung cancer [LC] after follow-up) resulted in repeated imaging, but 2.0% of people screened also required nonbeneficial downstream diagnostic evaluation to determine these nodules were not cancer. We sought to put these findings into context by examining how this high false-positive rate influences the harm-to-benefit ratio for higher- vs lower-risk patients.

Methods

From March 31, 2015, through June 30, 2015, 2106 patients were screened across 8 academic VAs. Screening processes and population-average outcomes for this project have been reported. In trials, LCS’s 20% relative risk reduction (RRR) in LC mortality did not vary by baseline LC risk, so we estimated each patient’s absolute risk reduction (ARR) by multiplying the 20% RRR by their baseline LC mortality risk (ARR = Baseline Risk × RRR). We estimated annual baseline LC mortality risk using the Bach risk model. Unlike other models, the Bach model’s inputs are obtainable in VHA’s Corporate Data Warehouse. In addition, a recent analysis indicates it is one of the best performing models.

Next, we separated patients into risk quintiles and assessed for each: number of LC cases observed; screening effectiveness (number needed to screen [NNS] per LC death prevented); and screening efficiency (number of false-positive results and downstream diagnostic procedures [eg, advanced imaging, bronchoscopies, biopsies] per LC death prevented). Following VHA policy and as part of the VA Quality Enhancement Research Initiative, this evaluation was not considered to be research and was declared to be nonresearch quality improvement activities by the VHA National Center for Health Promotion and Disease Prevention, and the Ann Arbor Veterans Affairs Medical Center institutional review board. As a quality improvement activity, patient consent was not required. Patient data were deidentified in analyses.

Results

Patients in higher quintiles of LC risk had significantly more lung cancers diagnosed during the project, supporting the Bach model's ability to risk stratify in this population (Figure, A: 4.8 LCs per 1000 in quintile 1 vs 29.7 per 1000 in quintile 5). Initial screens were least effective for veterans in quintile 1 (lowest LC risk) (NNS of 6903) and most effective for veterans in quintile 5 (NNS of 687) (Figure). Rates of false-positive results and downstream evaluations did not differ significantly across risk quintiles (P = .52 and P = .15 for trend, respectively). That is, the overall 56.2% rate of false-positive results requiring tracking remained relatively stable across risk quintiles (95% CI, 53.1%-62.6% in quintile 1 vs 51.9%-61.5% in quintile 5), as did the overall 2.0% rate of false-positive results requiring downstream diagnostic evaluations (95% CI, 0.3%-2.6% in quintile 1 vs 1.7%-5.2%). This relationship of increasing absolute benefit and relatively stable harms enhances the favorable harm vs benefit balance for higher-risk vs lower-risk persons. The initial screen was least efficient for patients in quintile 1 (2749 false-positive results and 68 nonbeneficial diagnostic procedures per LC death prevented) and most efficient for those in quintile 5 (eg, 363 false-positive results and 22 nonbeneficial diagnostic procedures per death prevented) (Table).

Figure. Observed Rate of Lung Cancer Diagnosis and Predicted Effectiveness With Initial Low-Dose Computed Tomography Screening.

A, Observed rate of lung cancer diagnoses (per 1000 persons screened once). B, Screening effectiveness: number needed to screen (NNS) to prevent 1 lung cancer death. Error bars indicate 95% CIs.

Table. Outcomes of Initial Low-Dose Computed Tomography Screening According to Risk Quintile.

Quintile of Risk (1-y Cumulative Risk LC)a	No. (%)				Predicteda LC Deaths		Efficiency Calculation (No. of Harms per LC Death Prevented)
	Participantsb	Observed (During VHA Demonstration Project)			Total, No.c	Prevented, No.c
		Observed LC Casesb,c	FPs Requiring Tracking	FPs Requiring Diagnostic Evaluation			FPs per LC Death Preventedc	Nonbeneficial Diagnostic Evaluations per LC Death Preventedc
All quintiles	2084 (100)	31 (100)	1175 (100)	42 (100)	7.97	1.59	737	26
Quintile 1	420 (20.2)	2 (6.5)	243 (20.7)	6 (14.3)	0.45	0.09	2749	68
Quintile 2	459 (22.0)	2 (6.5)	249 (21.2)	5 (11.9)	1.07	0.22	1152	23
Quintile 3	379 (18.2)	5 (16.1)	205 (17.4)	8 (19.0)	1.29	0.26	793	31
Quintile 4	422 (20.3)	10 (32.3)	249 (21.2)	9 (21.4)	2.0	0.40	622	22
Quintile 5	404 (19.4)	12 (38.7)	229 (19.5)	14 (33.3)	3.16	0.63	363	22

Abbreviations: FP, false-positive screening result; LC, lung cancer; VHA, Veterans Health Administration.

^aBased on lung cancer risk prediction model of Bach et al, which uses the following inputs to calculate an individual’s 1-year cumulative risk of LC: sex, age, smoking status (current/former smoker), years since quitting if former smoker, mean number cigarettes per day while smoking, and asbestos exposure. Asbestos exposure was not available for participants and was not considered in these calculations. The Bach model has been shown to have excellent predictiveness without this variable. For example, the Bach model (in the absence of asbestos exposure information) showed satisfactory calibration and excellent discriminative ability in a recent external validation study (areas under curves of 0.68 to 0.8 for predicting LC death).

^bTwenty-two of the 2106 participants had incomplete smoking history and were excluded from this analysis.

^cP < .05 by linear test of trend for continuous outcomes.

Discussion

The high rate of false-positive results identified in the VHA’s LCS demonstration project has caused concern about whether LCS should be implemented in this population. We reexamined these data and found that the high false-positive rate results in a more concerning harm-to-benefit ratio for those eligible persons at lower LC risk, but a much better harm-to-benefit ratio for high-risk patients (Table). We found that even given these very high false-positive rates, the overall balance of pros and cons among patients at high LC risk still surpasses those of most established cancer screening programs.

These results should be interpreted with several caveats in mind. The high rate of false-positive results found in the VA demonstration project may represent a substantial overestimate of future rates for 2 reasons: (1) initial screens likely have more false-positive results than recurrent screening, and (2) newer nodule management guidelines are showing great promise in lowering false-positive rates. Reducing the rate of false-positive findings would improve the harm-to-benefit balance for all quintiles. However, our analysis did not include all potential harms of LCS, such as overdiagnosis and psychological effects from false-positive results. In addition, effectiveness studies are still needed to confirm the extent to which the mortality benefit observed in the National Lung Screening Trial, a 20.0% reduction in lung cancer and a 6.7% reduction in all-cause mortality, applies in actual practice.

These real-world findings reinforce the need to risk-stratify patients for LCS and provide support for personalized, risk-based harm-benefit estimates for all eligible persons during LCS decision-making.