Cost-Effectiveness of Management Algorithms for Lung-RADS Category 4 Nodules

Abstract

Purpose

To evaluate nodule management guidelines in a simulated cohort of Lung Reporting and Data System (Lung-RADS) 4 nodules based on real-world data.

Materials and Methods

In this retrospective study, 100 000 patients were simulated from 151 patients with Lung-RADS 4 nodules (from January 2010 to August 2018). Each patient in the simulation was managed with each algorithm, and health outcomes were accumulated based on interventions and delays to cancer diagnosis. If the algorithm missed a cancer, it was diagnosed at the next annual screening round, although it would grow in the interim. Patient age-specific or cancer-specific mortality was assigned depending on whether the nodule was malignant, and quality-adjusted life years (QALYs) were calculated. Costs of interventions and cancer treatment were accumulated. One-way sensitivity analyses were performed.

Results

The most effective algorithm was the British Thoracic Society (BTS; 10.041 QALYs), followed by the American College of Chest Physicians (10.035 QALYs) and Lung-RADS (10.021 QALYs). Only the BTS and Lung-RADS were on the efficient frontier, with an incremental cost-effectiveness ratio (ICER) of $52 634 (95% CI: $45 122, $60 619). Under nearly all sensitivity analyses, the only algorithms on the efficient frontier were BTS and Lung-RADS. The ICERs for BTS versus Lung-RADS were under $100 000 for all scenarios except an increased life expectancy in patients without cancer, in which case the ICER was $109 273.

Conclusion

The BTS algorithm and Lung-RADS were cost-effective for managing category 4 nodules, with BTS yielding greater QALYs.

Supplemental material is available for this article.

© RSNA, 2021

See also the commentary by Elicker in this issue.

Summary

The British Thoracic Society (BTS) guidelines and Lung Reporting and Data System (Lung-RADS) were cost-effective for managing Lung-RADS category 4 nodules, with BTS giving the greatest quality-adjusted life years (QALYs).

Key Points

• ■ Using real-world data coupled with a simulation, management guidelines for Lung Reporting and Data System (Lung-RADS) 4 nodules were evaluated for cost-effectiveness.

• ■ Among the guidelines tested, the British Thoracic Society (BTS) and Lung-RADS were cost-effective.

• ■ The BTS algorithm yielded the greatest quality-adjusted life years, with an incremental cost-effectiveness ratio of $52 634 compared with Lung-RADS.

Introduction

Pulmonary nodules may be discovered incidentally or through lung cancer screening and present a management dilemma for radiologists and clinicians alike. A number of management algorithms have been developed by specialty societies, including the Fleischner Society, the British Thoracic Society (BTS), and the American College of Chest Physicians (ACCP) (1–3). For lung cancer screening, the American College of Radiology developed Lung Reporting and Data System (Lung-RADS) (4). Whereas all these guidelines provide follow-up schedules for smaller, low-risk nodules, not all provide specific guidance for larger, high-risk nodules. In particular, the Fleischner Society guidelines and Lung-RADS list a menu of options (PET/CT, noninvasive biopsy, or surgical biopsy) for larger nodules, without specific recommendations for which nodules should get which option.

The BTS and ACCP guidelines use risk calculators to help triage large pulmonary nodules. These include the Brock University calculator and the Herder model for patients who underwent fluorine 18 (¹⁸F) fluorodeoxyglucose (FDG) PET/CT (5,6). These risk calculators consider not only nodule characteristics but also patient characteristics to better determine the risk of malignancy. Indeed, the Brock University model, which adds nodule location, patient demographics, emphysema, and family history of lung cancer to standard nodule attributes, has been shown to perform better than Lung-RADS in the National Lung Cancer Screening Trial (7,8). However, the accuracy of risk models for larger nodules may be less than it is for smaller nodules (9,10). In the BTS and ACCP guidelines, in contrast to Lung-RADS or Fleischner Society guidelines, specific recommendations are given for nodule management (eg, PET/CT, follow-up, or surgery) based on the results of the risk calculator. The BTS guidelines use the Herder model to stratify PET/CT results, while ACCP uses only a qualitative scale.

Cost-effectiveness research aims to optimize care across health care systems by evaluating the cost of various interventions (or management strategies) compared with life years gained for the patients. Such research often relies on simulations using data from the literature to test a set of different strategies and find the best one. The cost-effective strategies provide information to payers and policymakers about how best to spend money within a health care system to provide the greatest quality-adjusted life expectancy for the population given limited resources. To our knowledge, no prospective trials or cost-effectiveness analyses have been performed to evaluate the utility of nodule management algorithms for larger nodules.

In this study, we set out to evaluate the cost-effectiveness of the BTS, ACCP, and Lung-RADS management algorithms for Lung-RADS category 4 nodules using a simulated cohort derived from patients in our health care network.

Materials and Methods

Patient Cohort

This retrospective study was approved by the institutional review board, with waiver of consent. The patients used for this analysis were drawn from a previously described cohort (10). The inclusion criteria were patients who underwent lung cancer screening CT within our health care network between January 2010 and August 2018 and had been assigned a Lung-RADS category of 4A, 4B, or 4X. The dominant nodule that determined the Lung-RADS category for each case was assessed. Patients without a definitive diagnosis, either based on pathologic findings or stability, were excluded as detailed in the prior publication. A random subset of 151 such patients was selected, and detailed data were collected, including nodule characteristics, patient demographics, and the results of ¹⁸F-FDG PET/CT and follow-up chest CT, if performed.

Imputation and Sampling

For this analysis, a simulated cohort of 100 000 patients was selected from the 151 patients by sampling with replacement. This was performed using a Perl script. Of the 151 patients, 78 patients had undergone PET/CT, and 113 had undergone follow-up chest CT. To impute values for the remaining patients, we performed multivariable logistic regression analysis to predict the results of PET/CT and of follow-up chest CT from patient and nodule characteristics as well as the nodule diagnosis (benign vs malignant). The details of these logistic regression models are given in Appendix E1 (supplement).

For each of the 100 000 samples in which PET/CT or follow-up CT results were not available, an outcome for those variables was selected randomly from the distribution given by the logistic regression model. Thus, even for the same input patient from the real cohort, patients in the simulation cohort may differ (in the outcome of PET/CT or follow-up CT).

Nodule Management Algorithms

For each patient in the simulated cohort, the nodule was evaluated with the following management algorithms: BTS guidelines, ACCP guidelines, and Lung-RADS version 1.1, as well as algorithms that only considered the PET/CT result or follow-up CT result. Over the course of the algorithm, costs, health outcomes, and eventual life expectancy were accumulated with equations in JMP Pro (v15, SAS Institute). Details of the implementation of these algorithms are given in Figures E1–E5 (supplement), with an overview of the nodule stratification techniques used in each algorithm shown in Table 1. Of note, the only algorithm that explicitly employed nonsurgical lung biopsy was BTS; the sensitivity of lung biopsy for malignancy was taken at 92% (11) and assumed to have a specificity of 100%. The eventual diagnosis of the algorithm was recorded and used to determine potential delayed diagnosis.

Table 1:

Nodule Stratification Methods in the Triage Algorithms

Patient survival depended on the nodule diagnosis (benign or malignant): patients with benign nodules were assigned age-specific life expectancies, whereas patients with malignant nodules were assigned the median cancer-specific survival data by stage. For benign nodules in which the algorithm diagnosed malignancy, surgical mortality was also applied. For malignant nodules in which the algorithm misdiagnosed the lesion as benign, we assumed that the malignancy would be diagnosed at the next annual screening round; thus, a delay of 1 year was simulated. The nodule would grow within that year, with the potential to become a higher stage malignancy, yielding worse survival at the time of delayed diagnosis. Nodules that either grew at follow-up CT or had moderate and/or intense FDG uptake at PET/CT were assigned a faster growth rate. The details of this growth model are given in Appendix E1 (supplement).

Quality-Adjusted Life Years and Cost-effectiveness Analyses

Quality of life utilities for patients with active cancer depend on cancer stage and are given in Table E6 (supplement). Patients with benign nodules who underwent surgical resection had a utility of 0.9 for the 1st year following surgery, then normal utility thereafter (see Appendix E1 [supplement]). Costs and quality-adjusted life years (QALYs) were discounted at 3% per year.

The costs and QALYs for each nodule management algorithm were calculated for each patient in the simulated cohort. The average cost and QALY was then calculated, and the algorithms were compared. An efficient frontier analysis was performed to identify the cost-effective strategies (ie, strategies that provide increased QALYs with the least cost). Incremental cost-effectiveness ratios (ICERs), representing the additional cost in dollars per QALY gained, were calculated for algorithms on the efficient frontier. The 95% CIs were generated by bootstrapping.

Sensitivity Analyses

To test the robustness of the cost-effectiveness results, a series of one-way sensitivity analyses was performed. One sensitivity analysis was to generate a second simulated cohort (with the same parameters). Another sensitivity analysis altered the logistic regression model for predicting PET/CT outcomes, to increase the rate of positive findings at PET/CT examinations in benign nodules. A third simulated cohort was generated using the modified regression model. We also performed a sensitivity analysis using a modified version of the Lung-RADS algorithm, incorporating biopsy for patients with positive PET/CT findings. Additional sensitivity analyses were performed on the original simulated cohort by altering cancer growth rates, age-based and cancer-based life expectancy, and costs. Finally, a sensitivity analysis was performed including the possibility of distant metastatic disease for the malignant nodules. Details of the parameters of the sensitivity analyses are given in Appendix E1, section V (supplement).

Results

Simulated Cohort

A simulated cohort of 100 000 patients (nodules) was generated from the 151 real Lung-RADS 4 nodules. Patient and nodule characteristics shown in Table 2. Most nodules (81%) were solid, with mean nodule size of 13.5 mm. Malignant nodules comprised 43 125 (43%) of the cohort. Malignancy rates by Lung-RADS category were 21% (11 212 of 52 377) for Lung-RADS 4A, 58% (18 628 of 32 383) for Lung-RADS 4B, and 91% (13 285 of 14 615) for Lung-RADS 4X nodules.

Table 2:

Characteristics of Simulation Cohort

Management Algorithms

Management algorithm diagnostic accuracies are shown in Table 3. The algorithm with the highest accuracy was BTS (87%) followed by ACCP (80%); the least accurate algorithm was follow-up CT only (72%). The most sensitive algorithm for malignant nodules was BTS (80%), followed by ACCP (74%); the least sensitive was follow-up CT only at 44%. The follow-up only algorithm yielded the highest specificity (94%) followed by Lung-RADS (93%) and BTS (92%). These corresponded to the lowest rates of benign resections: follow-up CT with 7%, followed by Lung-RADS (8%) and BTS (9%).

Table 3:

Performance of Management Algorithms

Cost-effectiveness Analysis

Costs and QALYs for each management algorithm, using base case parameters, are shown in Table 4. The most effective algorithm was BTS (10.041 QALYs), followed by ACCP (10.035 QALYs) (Fig 1). However, of the five algorithms, only the BTS and Lung-RADS lie on the efficient frontier, meaning that they provide increased QALYs for the lowest cost. The ICER between Lung-RADS and BTS was $52 634 per QALY gained (95% CI: $45 122, $60 619). Thus, while the 95% CIs of these strategies appear to overlap in Table 4, the difference between the strategies is maintained, with BTS yielding greater QALYs than Lung-RADS with an ICER of less than $100 000. Within the subset of category 4A nodules, the ICER was $656 507, and within the category 4B/4X nodules, it was $24 654.

Table 4:

Cost-effectiveness Analysis, Base Case

Figure 1:

Cost-effectiveness plot of the nodule management strategies analyzed. ACCP = American College of Chest Physicians, BTS = British Thoracic Society, ICER = incremental cost-effectiveness ratio, Lung-RADS = Lung Reporting and Data System, QALY = quality-adjusted life year.

The results of a number of one-way sensitivity analyses are given in Table 5. Under nearly all conditions, the only algorithms on the efficient frontier were BTS and Lung-RADS. The ICERs for BTS (Fig 2) versus Lung-RADS were under $100 000 for all scenarios except an increased life expectancy in patients without cancer, in which case the ICER was $109 273. Under one condition, an increase in the growth rate of fast-growing lung cancers, the ACCP algorithm was on the efficient frontier and yielded higher QALY and cost than BTS; however, the ICER was very high at $1 384 951.

Table 5:

Incremental Cost-effectiveness Ratios, Sensitivity Analysis

Figure 2:

Sensitivity analyses for incremental cost-effectiveness ratios (ICERs) of British Thoracic Society (BTS) management algorithm.

Discussion

In summary, we simulated a cohort of 100 000 Lung-RADS 4 nodules based on real-world data from our health care system, and we evaluated a number of nodule management algorithms for their cost-effectiveness using this cohort. We found that the two management algorithms on the efficient frontier were Lung-RADS and BTS, with BTS yielding the greatest QALYs. The advantage of the BTS algorithm was seen by its ICER compared with Lung-RADS in statistical analysis by confidence interval and all except one sensitivity analysis we performed; in a condition where the growth rate of faster growing nodules was increased, the ACCP algorithm yielded higher QALYs but at a substantial cost (ICER of over $1 million).

Overall, the management algorithms that yielded higher QALYs were those that had more accuracy in discerning malignant nodules. This is because malignant nodules that were missed would grow in the interval before they were eventually diagnosed, yielding worse patient survival. Whereas the ACCP algorithm also yielded higher QALYs, it was not on the efficient frontier. This is likely because it included a higher rate of unnecessary procedures (resections for benign nodules), contributing to greater cost.

The one scenario in which the ACCP was on the efficient frontier, and indeed yielded greater QALYs than BTS, was that in which the growth rate of faster growing nodules was increased. In that scenario, even missing a few malignancies yielded substantial detriment in patient outcomes. The malignant nodules missed by BTS were caused by its use of percutaneous lung biopsies, which have good but imperfect sensitivity. While percutaneous lung biopsies are included as options in the other management algorithms (ACCP, Lung-RADS), they are not explicitly recommended as part of the algorithm except in BTS. Thus, except in one sensitivity analysis, we did not include biopsies in the ACCP or Lung-RADS algorithms because it is not clear from the algorithms in exactly which cases one should use biopsy versus going directly to surgery.

The algorithms that use nodule risk calculators (ie, BTS and ACCP) had higher accuracy than algorithms that did not use risk calculators. This translated into fewer missed cancer diagnoses and thus higher QALYs. The BTS algorithm uses both the Brock and Herder models, which is likely why it performed the best, compared with other algorithms that used simpler threshold-based decisions (particularly for FDG PET/CT uptake). Lung-RADS version 1.1 now suggests use of the Brock model for triage of high-risk nodules, but it does not provide details of how to do so.

This study had a number of limitations. The lack of PET/CT and follow-up CT information on all cases required imputation to estimate their outcomes, inherently subject to error. However, by using randomly determined results of PET/CT and follow-up CT for each sample in the cohort, a distribution of outcomes for each case was included in the cohort. Because a large number of simulated nodules made up our cohort, this should balance out errors in the imputation. We were also limited by the simple model we used to estimate growth of missed lung cancers. However, sensitivity analyses modifying the parameters of this model did, in general, not lead to substantial differences in the outcome of the analysis. Notably, our cost input and willingness-to-pay threshold of $100 000 may not be applicable to non–United States health care systems; however, the efficient frontier should be stable under such changes. Another limitation was that we did not specifically evaluate for the potential of overtreatment of indolent (subsolid) lung cancers. Finally, we note that the QALYs of the strategies differed by only small amounts. However, by looking at both the confidence interval of the ICER and sensitivity analyses, we showed that the results of the superiority of the BTS algorithm were robust.

In conclusion, the BTS algorithm was the cost-effective option with the best outcomes for managing high-risk (Lung-RADS 4) pulmonary nodules. This finding held true under multiple sensitivity analyses, suggesting that it may be generalizable, at least within the United States health care system. Radiologists should consider using this algorithm, or at least algorithms that employ risk calculators, rather than simple size-based or FDG uptake threshold-based decision models, for triage of pulmonary nodules. Prospective trials are needed to verify these findings.