Variation in eligible patients’ agreeing to and receiving lung cancer screening: a cohort study

Abstract

Introduction:

Little is known about how clinicians make low-dose computed tomography lung cancer screening (LCS) decisions in clinical practice. We assessed factors associated with real-world decision making, hypothesizing that lung cancer risk and comorbidity would not be associated with agreeing to or receiving screening in practice. Even though these factors are key determinants of the benefit of LCS, they are often difficult to incorporate into decisions without the aid of decision tools.

Methods:

Retrospective cohort study of patients meeting current national eligibility criteria and deemed appropriate candidates for LCS, based on clinical reminders completed over a 2-year period (2013-2015) at 8 VA medical facilities. Multilevel mixed-effects logistic regression models (conducted 2019-2020) assessed predictors (age, gender, lung cancer risk, Charlson Index, travel distance to VA facility, and central vs outlying decision making location) of primary outcomes of agreeing to and receiving screening.

Results:

Of 5,551 patients (mean age 67; 97% male; mean lung cancer risk 0.7%; mean Charlson Index 1.14; median travel distance 24.2 miles), 3,720 (67%) agreed to LCS and 2,398 (43%) received screening. Lung cancer risk and comorbidity score were not strong predictors of agreeing to or receiving LCS. Empirical Bayes adjusted rates of agreeing to and receiving LCS ranged from 22%-84% across facilities, and 19%-85% across clinicians. 33.7% of the variance in agreeing-to and 34.2% of the variance in receiving LCS was associated with the facility or the clinician offering screening.

Conclusions

Substantial variation in Veterans agreeing to and receiving LCS during the VA LCS Demonstration Project was found. This variation wasn’t explained by differences in key determinants of patient benefit, while the facility and clinician advising the patient had a large impact on LCS decisions.

Introduction

Lung cancer is the leading cause of cancer death in the world, with an estimated 1.76 million deaths in 2018.¹ Most of this mortality is concentrated in a relatively small high-risk group -- heavy smokers.^2–4 In 2011, the National Lung Screening Trial (NLST) demonstrated that annual screening with low-dose computed tomography lung cancer screening (LCS) could substantially decrease lung cancer mortality among older heavy smokers.⁵ Yet, screening uptake has remained quite low.⁶ A number of studies have examined barriers to screening,^7–9 including concerns about screening harms, particularly for persons with a higher burden of comorbidity than were included in the NLST.¹⁰ These concerns have manifested in controversy about how to feasibly select good screening candidates.^10–12

At the same time, a growing body of research on LCS has produced evidence for how to better identify good candidates for screening.^13,4,14–16 Net-benefit (screening benefits minus its harms) varies dramatically across patients, mainly as a function of lung cancer risk and competing risk (life-expectancy).^15,16 Prior work has assessed how individualized risk-based and benefit-based approaches to screening can substantially improve the effectiveness and safety of screening programs.^2,4,17,18 Individualized approaches use validated lung cancer risk prediction models, or life year gain models (that incorporate comorbidity and life-expectancy),¹⁶ to select patients for screening. Prior work examining randomized trial evidence as well as evidence regarding the remaining scientific uncertainties with LCS found that, for eligible patients with at least a fair life-expectancy (≥ 10 years), an individualized approach to LCS (tailoring screening decisions to a person’s estimated lung cancer risk and life-expectancy) could accurately identify people likely to have a high benefit.¹⁵ Little is known, however, about how clinicians make LCS decisions in day-to-day practice and how patient characteristics play a role in LCS decision-making. Understanding these factors is critical for identifying gaps in current practice and how to target interventions for improving screening decisions. For example, if patient-specific lung cancer risk and competing risk are found not to be highly associated with decision-making in practice, this would indicate a need for deploying easy-to-use decision tools to help clinicians better incorporate these important factors.

The Veterans Affairs (VA) Lung Cancer Screening Demonstration Project examined patient outcomes from one round of LCS and projected resource needs for the VHA.¹⁹ Unadjusted rates of agreeing to screening were reported but factors associated with screening decisions were not examined. Nonetheless, clinical reminders developed during this project recorded whether a clinician and screening-eligible patient agreed to LCS and whether LCS was received, providing a unique opportunity to examine LCS decision making. These reminders were triggered by criteria that followed US Preventive Services Task Force guidelines for LCS at the time. Leveraging this data, a retrospective cohort study was conducted to examine the extent to which an individual patient’s lung cancer risk and comorbidity status impact LCS decisions outside of a randomized controlled trial, compared to the impact of non-patient factors (i.e. variation attributable to the facility and PCP advising the patient). We hypothesized that lung cancer risk and comorbidity would not be highly associated with agreeing to or receiving screening during the demonstration project, due to the lack at that time of decision tools that help clinicians incorporate validated prediction models into their decision-making with patients.

Methods

Data Sources –

Data for this study was obtained through VA Corporate Data Warehouse, which stores patient information captured through clinical reminders (see supplemental methods for further details).

Study Cohort –

The study cohort included LCS-eligible patients from eight geographically diverse VA academic medical centers from 2013 to 2015. The study focused on the time-period of the VA LCS Demonstration Project because clinical reminders documenting LCS decision-making were used more consistently across medical facilities during this period. Eligibility for screening was determined by LCS clinical reminders.¹⁹ Cohort inclusion reflected current national guidelines ²⁰ (age 55-80, completed reminder documenting ≥ 30 pack years smoking history, current smoker or quit <15 years ago) and also 2 additional criteria: timing – i.e., initial provider LCS reminder completed between 10/1/13 to 9/30/15; and clinician judgment about appropriateness – i.e., no documented clinical exclusions and clinicians felt patient to be an appropriate candidate for LCS (see supplemental methods for details).

Predictor Variables –

Initial lung cancer risk and comorbidity were included as key patient factors.¹⁵ Baseline lung cancer risk was estimated using the Bach risk model, an accurate and well-calibrated model on external validation (inputs: age, sex, asbestos exposure history, and smoking history).^17,18,21 The Charlson Comorbidity Index was calculated for each patient as a surrogate marker for competing mortality risk.²² Patient-level demographic covariates were also included in both models (i.e., age, gender, race, and distance to the VA medical facility offering LCS). The patient distance to the medical facility offering LCS was calculated (see supplement). As a measure of familiarity to the central medical facility delivering the actual CT screening, a binary variable was included indicating whether a patient’s decision-making location (where the LCS clinical reminder was completed) was at the central facility or an outlying community-based clinic. If a patient is typically seen at an outlying clinic, they may be less familiar with the central facility where they receive the actual CT screen. In addition, central facilities and outlying clinics differ in other important ways: academic (central) vs. non-academic (outlying clinic) as well as clinical resources (outlying clinics have fewer in general).

Primary Outcomes: Agreeing to and Receiving LCS –

Clinicians documented “agreed/does-not-agree to LCS” in the LCS provider reminder. Current Procedural Terminology (CPT) codes were used, supplemented by an approach developed for this project (see supplement for details), to determine receipt of LCS within 3 months of the initial decision-making documentation (i.e., screening utilization ≤ 3 months after the LCS provider reminder was completed).

Data Analysis -

A 3-level hierarchical mixed-effect logistic regression model was fit for each of the two primary outcomes, with patients nested within PCP and PCP nested within medical facility. The following patient level attributes were assessed as predictors of agreement and receipt of LCS: lung cancer risk, comorbidity index, distance to medical facility offering LCS, decision making location, race, gender, and age. To aid readers in understanding the absolute magnitude of relative associations, the average marginal effects for key patient factors was estimated. Outcome estimates for PCP and facility were “shrunken” using empirical Bayes methods to address the problem that clinics/providers with fewer patients would have more variance and less stable estimates.^23,24 Next, the intraclass correlation coefficient (ICC) were estimated to assess the degree of variation in outcomes due to clustering within PCP and medical facility.

The following transformations were applied to covariates: age was centered at the mean age of 67; initial lung cancer risk was log-transformed; and distance was log-transformed. Because the benefit of screening for those at higher lung cancer risk can be attenuated if the patient also has a limited life-expectancy (high competing mortality), the study also tested whether the effects of lung cancer risk might be different with varying age and for different degrees of comorbidity, including interaction terms (risk*comorbidity, risk*age) into the models. To further explore the relationship between lung cancer risk and the primary outcomes -- and because interaction effects (risk*comorbidity) can be challenging to interpret -- a subset analysis of patients with no comorbidity (comorbidity index = 0) was planned.

Analyses were performed in R version 3.6.0 and SAS version 9.4 (see supplemental methods for more information on statistical methods). This project was part of a VA Quality Enhancement Research Initiative, was considered non-research quality improvement based on VHA policy,²⁵ and was declared as non-research quality improvement activity by the VA Ann Arbor Healthcare System institutional review board. Investigators used a fully de-identified dataset for all analyses (i.e. they were blinded to patient, provider and site).

Results

A total of 23,123 patients were identified as potentially eligible based on age and smoking history, but 16,780 of these patients did not have an appropriateness assessment documented by a provider in the study timeframe and were excluded. An additional 792 patients were excluded due to documented clinical exclusions or missing data. This left 5,551 patients in the study cohort (see Figure 1), 3,720 (67%) of whom agreed to lung cancer screening and 2,398 (43%) received screening within 3 months of the initial decision-making. 4,075 (73%) had the decision-making discussion at the central medical facility, with the rest of the cohort having discussions at an outlying clinic (based on where the LCS clinical reminder was completed). Cohort demographics are listed in Table 1.

Figure 1:

Study Cohort

Table 1.

Study Cohort Characteristics

	Received LCS		Agree to screen
	Yes (N=2398)	No (N=3150)	Yes (N=3720)	No (N=1831)
Sex
Male	2321 (96.8%)	3047 (96.7%)	3579 (96.2%)	1792 (97.9%)
Female	77 (3.2%)	103 (3.3%)	141 (3.8%)	39 (2.1%)
Age
Mean	66.88	67.22	66.63	67.97
SD	5.16	5.53	5.23	5.53
Lung Cancer Risk
Mean	0.0072	0.0078	0.0072	0.0082
SD	0.0048	0.0054	0.0049	0.0056
Charlson Comorbidity Index
Mean	1.13	1.15	1.12	1.18
SD	1.31	1.31	1.30	1.32
Distance from VAMC
Mean	31.93	40.43	35.76	38.73
SD	43.00	44.57	46.75	38.05
Race
White	1782 (74.3%)	2227 (70.7%)	2658 (71.5%)	1354 (73.9%)
Black	316 (13.2%)	468 (14.9%)	575 (15.5%)	209 (11.4%)
Other	300 (12.5%)	455 (14.4%)	487 (13.1%)	268 (14.6%)
Decision Making Location
VAMC	1930 (80.5%)	2143 (68.0%)	3051 (82.0%)	1024 (55.9%)
Other	468 (19.5%)	1007 (32.0%)	669 (18.0%)	807 (44.1%)

Agreeing to screen

Age, gender, and decision-making location were the only patient-level factors found to be significant in predicting agreement for LCS (see Table S.1): older patients were less likely to agree (OR = 0.95 {95% CI 0.94-0.97}, P < 0.001); women were more likely to agree (OR = 1.56 {95% CI 1.00-2.41}, P = 0.05) and those with decision making at the central facility offering LCS were more likely to agree (OR = 13.54{95% CI 10.40-17.63}, P < 0.001). The probability of agreeing to screen decreased by 0.8%, on average for every additional year of age (CI = −0.5% to −1.1%). For example, it was found that 84.1% of men aged 57 were predicted to agree to screening, compared to 66.7% for men aged 77 (an absolute difference of −17.4 percentage points [CI = −5.1 to −23.7 percentage points]). Regarding average marginal effects for gender, 76.5% of men were predicted to agree to screening compared to 83.5% for women (absolute difference of 7.0 percentage points [CI = 1.6 to 11.0 percentage points]). It was also found that 86.7% of patients with decision making at the central location were predicted to agree to screening, on average, compared to 32.4% of patients with decision making at an outlying clinic (absolute difference of 54.3 percentage points [CI = 11.0 to 65.0 percentage points]). Those at higher pre-screening lung cancer risk were not substantially more likely to agree to screening (OR 0.99 {95% CI 0.89, 1.11}; P = 0.89). Similarly, comorbidity and distance were not strong predictors of agreeing to screen (see Table S.1).

The effect of pre-screening lung cancer risk on agreeing to screen did not differ by comorbidity score. Thus, despite a higher expected net benefit for those with higher lung cancer risk who are otherwise healthy,^15,26 those with no comorbidities and at higher lung cancer risk were not substantially more likely to agree to screening (in the subset with 0 comorbidities, adjusted probability of agreeing to LCS was 75.9% (CI = 59.9%, 86.9%) in the lowest lung cancer risk quintile compared to 76.7% (CI = 61.5%, 87.1%) in the highest lung cancer risk quintile (see supplemental analyses for additional information on interaction effects)).

In contrast, there was substantial variation in agreeing to LCS across the 8 VA medical facilities and across the 363 PCP’s. Empirical Bayes adjusted (“shrunken”)²⁷ rates of agreeing to LCS varied from 22% at the lowest site to 84% at the highest site, and ranged from a 12%- -85% agreement rate across PCPs. Overall, 33.7% of the variance in agreeing to screening was attributable to the medical facility (19.7%) or PCP (14.0%) who offered the screening.

Receiving screening within 3 months of decision-making

Among all patients deemed clinically appropriate for screening (not limited to only those agreeing to screening), decision making location (central VAMC vs. outlying clinic) was the only patient-level factor found to significantly predict receipt of LCS (see Table S.2; OR = 2.34 {1.86, 2.93}, P = <0.001). On average, 52.7% of patients with decision making at the central location were predicted to be screened compared to 32.3% of patients with decision making at an outlying clinic (95% CI on the absolute difference 5.1 to 21.2 percentage points]).

Pre-screening lung cancer risk, comorbidity, age, gender, race, and distance were not found to be significant predictors for receipt of LCS (see Table S.2). Patients with higher initial lung cancer risk may, if anything, be less likely to receive screening (OR = 0.91{0.82, 1.00}, P = 0.057). An estimated 52.7% (95% CI 33.7% to 71.0%) of patients in the lowest quintile of lung cancer risk were predicted to be screened, on average, compared to 43.5% (CI 26.3% to 62.3%) of patients in the highest risk quintile (95% CI on the absolute difference of −2.4 to −9.3 percentage points). The effect of lung cancer risk also differed by age (significant interaction effect): higher-risk patients were even less likely to be screened with older age. Additionally, in the subset of patients with no comorbidities (CCI = 0; pre-specified analysis), those with higher lung cancer risk were less likely to be screened: the predicted probabilities of receiving LCS by risk quintile ranged from 52.2% (CI 33.2% to 70.6%) in the lowest lung cancer risk quintile to 42.9% (CI 25.9% to 61.9%) in the highest lung cancer risk quintile (see supplemental analyses for details on interaction effects).

There was substantial variation in the receipt of screening across medical facilities and providers. Bayesian (“shrunken”) rates of receiving LCS varied from 19%-85% across the 8 medical facilities and from 20% to 88% for the 363 PCP’s. Overall, 34.2% of the variance in receiving screening was attributable to the medical facility (22.7%) or PCP (11.5%) who offered the screening.

Discussion:

Key determinants of a patient’s chance of benefiting from screening (their lung cancer risk and degree of comorbidity) explained little of the variation in patients’ willingness to undergo screening. Even more concerning, the likelihood of getting screened, if anything, actually decreased as lung cancer risk rose—the opposite of what would be desirable for an effective screening program. These results suggest that other patient factors and behavioral determinants (unrelated to clinical benefit) as well as PCP and facility-level factors are more influential in decision-making. Improving the way lung cancer screening is offered to patients across providers and facilities, by better aligning decision-making with estimated benefit for individual patients, could lead to substantial improvements in the effectiveness, safety, and patient-centeredness of lung cancer screening programs.^2–4,15 As a result, the VA is studying how to improve LCS decision-making by incorporating decision tools that can help primary care teams quickly estimate a patient’s lung cancer risk and also help them understand how their patients’ competing risk and comorbid illnesses might impact decisions.²⁵

Importantly, decisions to pursue lung cancer screening (LCS) varied substantially for eligible patients across 8 VA facilities: rates of agreeing to (22%-84%) and receiving screening (19%-85%) differed to a great extent, even in the context of a standardized implementation process ¹⁹ and even after “shrinking” these estimates.²⁴ Compared to unadjusted rates of agreeing to screening reported previously (for a more restricted cohort of demonstration project patients and not looking at variation in receipt of screening),¹⁹ the estimates from the multilevel regression framework provide a better, more reliable indication of the extent to which rates of agreeing-to and receiving screening might vary across health systems.^24,28

This study quantifies the impact that non-patient factors can have on real-world LCS decisions. The PCP offering screening and the facility where screening takes place explained a large degree of the variation in decisions to pursue screening: 33.7% of the variation in agreeing to LCS and 34.2% of the variation in receiving LCS could be attributed to PCP and medical facility. While variation across providers and medical facilities is to be expected,²⁹ particularly in the context of implementing a new intervention, the magnitude of the provider- and facility-level variation that was identified is concerning in light of the finding that screening decisions are largely based on something other than important clinical characteristics (i.e., lung cancer risk & comorbidity). This finding aligns with prior literature on drivers of large geographic variations in medical care, which has consistently identified physician preferences and beliefs about treatment effectiveness as influential in variation ^30,31 – and that organizational factors are often predictive of the care received.³²

Decision making location (central VAMC vs. outlying clinic) was an important predictor of agreeing to and receiving screening in the cohort. This and other patient-level associations with agreeing-to and receiving screening are discussed further in the supplement and placed into the context of other literature and additional data from an ongoing quality improvement project.²⁵

Limitations –

The degree of variation identified may underestimate the true range of variation across US health systems, since the findings reflect decision-making during a period where decision-making processes were more standardized and coordinated across sites than is typical. Decision-making in the VA context may not be representative of other contexts where financial incentives for increasing utilizations of imaging tests and procedures can be a larger driver of screening decisions, and such factors could not be explored in this study. Further, the data are from 2013-2015. These results may not fully reflect current practice if decision-making has systematically changed due to additional coverage decisions ³³, new trial results,³⁴ or other influential events since then. However, we would not expect to observe substantial changes in the influence of lung cancer risk and comorbidity on LCS decisions without targeted policy changes or decision support interventions. As with any study using an administrative database, there may be minor inaccuracies with the data. Additionally, the Charlson Comorbidity Index was used to estimate competing mortality risk, which is a simple count of comorbidities and is an imperfect measure of competing risk/life-expectancy. A more accurate life-expectancy estimate would better account for competing risks, which is particularly important for those patients with high lung cancer risk who are older and may have limited potential life-year gains.^26,35 Nonetheless, the findings of a possible decrease in LCS utilization for those at highest risk persisted across all levels of comorbidity, even for those without any comorbidity (CCI = 0). Last, the degree to which the observed variation in LCS decision-making might represent “warranted” variation due to differences in patient preferences across sites could not be assessed.³⁶ However, it is very unlikely that informed patient preferences would systematically vary to the degree observed across the providers and facilities in the study.³¹

Conclusion –

These results provide important lessons as LCS uptake grows in the US and elsewhere, raising concerns that patient-provider decision-making about LCS is sub-optimal. This may be due in part to barriers in supporting PCP’s capacity to quickly identify ideal candidates for screening during busy clinic visits with many competing demands.³⁷ A growing number of tools are being developed to help clinicians identify good screening candidates (i.e., very high lung cancer risk and > 10-year life-expectancy), such as the tool that is currently being studied in a VA implementation trial.^25,38. These results identify a critical need for decision support tools that make it easier for clinicians to align lung cancer screening decision-making with clinical benefit and patient preferences.³⁹

Figure 2: Variation in Agreeing to Lung Cancer Screening due to Medical Facility and Primary Care Provider.

a: Displays the adjusted predicted probability of agreeing to screen by facility. An average site has an adjusted probability of 50%. Sites with darker CI’s in the plot have significantly different adjusted rates of agreeing to screen compared to an average site.

b: Displays the adjusted predicted probability of agreeing to screen by primary care clinician (PCP). An average PCP has an adjusted probability of 50%. PCP’s with darker CI’s in the plot have significantly different adjusted rates of agreeing to screen compared to an average PCP.

Figure 3: Variation in Receiving Lung Cancer Screening due to Medical Facility and Primary Care Provider.

a: Displays the adjusted predicted probability of receiving screening within 3 months of initial decision making, by site. An average site has an adjusted probability of 50%. Sites with darker CI’s in the plot have significantly different adjusted rates of receiving screening compared to an average site.

b: Displays the adjusted predicted probability of receiving screening within 3 months of initial decision making, by primary care clinician (PCP). An average PCP has an adjusted probability of 50%. PCP’s with darker CI’s in the plot have significantly different adjusted rates of receiving screening compared to an average PCP.