Using Clinical Risk Models for Lung Nodule Classification

Abstract

Evaluation and diagnosis of indeterminate pulmonary nodules is a significant and increasing burden on our healthcare system. The advent of lung cancer screening with low-dose computed tomography only exacerbates this problem and more surgeons will be evaluating smaller and screening discovered nodules. Multiple calculators exist that can help the clinician diagnose lung cancer at the bedside. The PLCO model helps determine who needs lung cancer screening and the McWilliams or Mayo models help guide the primary care clinician or pulmonologist with diagnosis by estimating the probability of cancer in patients with indeterminate pulmonary nodules. The TREAT model assists surgeons to determine who needs a surgical biopsy in patients referred with suspicious lesions. Additional work is needed to develop decision support tools that will facilitate the use of these models in clinical practice, to complement the clinician’s judgement and enhance shared decision making with the patient at the bedside.

INTRODUCTION

Lung cancer is the most common cancer in the world with an estimated 1.8 million new cases a year.¹ It is the leading cause of cancer-related mortality among men and women with an estimated 224,000 new cases and 159,000 annual deaths due to lung cancer in the United States.² The U.S. spends an estimated $10.3 billion on lung cancer care which comprises 10% of all cancer related healthcare expenditures.³ Lung cancer prognosis remains poor despite the steady decline in lung cancer incidence, declining smoking rates, increased awareness in the general population, and the advent of new technologies to detect lung cancer early. A lung nodule suspicious for lung cancer can present to the clinician in three ways: symptomatically, incidental discovery of the nodule after imaging for another clinical indication, and from periodic screening. Irrespective of source, the diagnosis of lung cancer begins with radiographic imaging of the chest and review of a detailed history and physical of the individual. Previously, most lung cancer was diagnosed symptomatically (estimated 75%), but incidental discovery has increased recently with the proliferation of imaging modalities like Computed Tomographic (CT) scans.⁴ Once a national screening program is fully initiated in the US, a dramatic increase in asymptomatic lung nodules requiring diagnosis will occur. Irrespective of the source of a lung nodule, clinical risk models have been developed and employed to determine who should be screened, who should receive continued radiographic surveillance and who should be referred to a surgeon. We discuss these models and their use across the spectrum of lung cancer risk and look in depth at those models designed for evaluation of lung nodules.

PREDICTIVE MODELS IN LUNG CANCER

When considering the spectrum of lung cancer risk, we begin with the “at risk” patient prior to developing a suspicious lung lesion and move through time until a definitive diagnosis of cancer or benign disease occurs at the time of an operation. At each step along their evaluation, patients accumulate more diagnostic information until presenting to the surgeon for an operative decision (Figure 1). Risk models exploit the increasing amount of diagnostic information to create more accurate estimates of malignancy risk. In the first step, an individual is asymptomatic. Each person has genetic, demographic, environmental, clinical and behavioral risk factors for the development of lung cancer. So, for example, lung cancer risk in non-smokers is 23 per 100,000 person years in individuals between the ages of 60 and 80. This risk is 20-fold higher in current smokers.⁵ The person then undergoes imaging and a lesion is either present or absent. If a lung lesion is present, the individual is then further evaluated. In the second step, a lung lesion has been radiographically detected and clinical symptoms may or may not be present. Lesion imaging characteristics that change the likelihood for malignancy are added to the epidemiological data generated by a history and physical. Nodules are then evaluated by a clinician who makes an assessment of the probability of cancer and decides to either follow the lesion with continued CT imaging or obtain additional diagnostic testing such as F¹⁸-fluoro-deoxyglucose positron emission tomography (FDG-PET) scans in accordance to current clinical guidelines. In the third step, the nodule is suspicious enough to warrant a referral to a surgeon for additional assessment. At this point in the evaluation, surgeons have access to all available data including patient risk factors, clinical and imaging characteristics, and the results of additional procedural and clinical evaluations. A final estimate of cancer or benign disease occurs and the operative decision is made.

Figure 1. Existing predictive models.

At each step in the pathway, the patient accumulates more diagnostic information. Predictive models with disease prevalence are located at the 3 steps.

Screening Models

Population level predictive risk models at step 1, such as the PLCO model published by Tammemägi and colleagues that was developed from the Prostate, Lung, Colorectal and Ovarian Cancer cohort, the Liverpool model created by Cassidy and colleagues or the Bach model, exist to assess a patient’s multi-year risk for developing cancer prior to conducting imaging. These models are epidemiological in nature and assess large, asymptomatic populations. Their primary purpose is to determine who most benefits from lung cancer screening.^6–8 They have been extensively used to estimate cancer risk by age and tobacco burden and to help assess the efficacy of lung cancer screening across strata of risk.^7,9,10 In North America the two most popular models for this population are the Bach and PLCO models.^6,8 The Bach model is known for its simplicity in estimating the likelihood of cancer over 6 years. Only age, gender, smoking history (pack years and years quit), and any work related asbestos exposure are needed to estimate risk. The PLCO model is more complex than Bach’s and has been extended to include non-smokers. It requires age, race, years of education, body mass index, diagnosis of chronic pulmonary disease, personal and family history of lung cancer, smoking history including duration, and packs per day. Both models are available as web-based calculators.

Diagnostic Models

In the second step of Figure 1, a nodule has been verified on radiographic imaging. Models such as Mayo Clinic, VA, and SPN are applicable to incidentally or symptomatically discovered nodules. The new McWilliams model estimates the probability of cancer in patients with radiographically discovered abnormalities on CT scan who have been screened for lung cancer.^11–13 These diagnostic models help the clinician decide who needs watchful waiting, additional testing, or surgical referral in step 2. The Mayo Clinic model is an older model that does not incorporate FDG-PET scan results or patient symptoms, but it is the only validated model to receive recommendation by the American College of Chest Physicians (ACCP).^4,12 The VA model has limited input for radiographic characteristics of the lesion but was specifically developed in a VA population.

Treatment Models

The final step in nodule evaluation is referral to a surgeon for evaluation and decision to operate. The prevalence of disease is significantly higher in this population with between 40 and 80% of nodules evaluated being malignant. Patients evaluated by surgeons usually have a significant body of diagnostic information compiled from previous medical specialists such as multiple radiographic scans, biopsy results, and pulmonary function. The TREAT model (Thoracic surgery, Research, Epidemiology, Diagnosis And Treatment) is a treatment model developed by Deppen and colleagues to exploit the added information available to surgeons and to provide a predictive risk estimate designed for surgeons at the point of the critical operative decision.¹⁴ When considering which model to use one must understand where in the course of nodule evaluation a patient is and what information is available to personalize the risk assessment to the patient.

INDETERMINATE PULMONARY NODULES

The discovery of an indeterminate lung nodule is common in clinical practice. An indeterminate pulmonary nodule is a pulmonary lesion < 3cm found radiographically that does not have a tissue diagnosis.¹⁵ They are found in 17–51% of chest CT scans and a conservative extrapolation suggests that at least 2 million indeterminate nodules are found annually^15,16 with a 1–12% chance of malignancy.¹⁷ These symptomatic or incidentally discovered lung nodules represent a significant existing diagnostic burden. However, the burden of indeterminate lung nodules may well double or treble in the coming years because of lung cancer screening. Low-dose CT was recently shown to reduce deaths from lung cancer by 20% in the National Lung Screening Trial (NLST). Based on this study, the U.S. Preventative Services Task Force, Center for Medicare Services, major patient advocacy groups and clinical societies all recommended lung cancer screening with annual low dose CT scans. In 2015, screening of asymptomatic individuals who are at high risk for lung cancer with low-dose CT scans will be covered by private insurers and Medicare.^10,18

IMPACT OF LUNG CANCER SCREENING ON INDETERMINATE NODULES

In the NLST trial, 39% of patients had a positive screening test (identification of an anomaly) in the course of their three screening CT scans and 96% of those positive screens were false positives.¹⁹ Extrapolating from the results of the NLST and the proposed 7.8 million individuals eligible for lung cancer screening, an estimated 80,000 additional diagnostic operations will be conducted as part of a US national lung cancer screening program.⁹ In the NLST, 24% of invasive procedures were benign and 1.2% of those patients with benign procedures died within 90 days of their procedure. Using the results of the NLST and aggregating those results to the 7.8 million at risk individuals across the U.S., an estimated 3.1 million lung anomalies will be discovered in the first three years of a national lung cancer screening program.^20,21 These findings will generate over 1.5 million follow-up CT scans, an additional 250,000 FDG-PET scans and 120,000 diagnostic operations. If the benign procedure rate from the NLST continues in a national screening program, approximately 29,000 diagnostic procedures in the first three years would result in benign disease.¹⁹ Some evidence demonstrates that predictive models and clinician estimates are potentially complementary and may have additional benefit when used together for the evaluation of lung nodules.²² Systematically assessing the likelihood a nodule is cancerous has the potential to reduce individual bias and improve diagnostic evaluation.

PREDICTIVE MODELING IN CLINICAL PRACTICE

The ACCP and the National Cancer Comprehensive Network practice guidelines detail an evidence-based algorithm for the diagnosis of lung cancer^4,23 Upon discovery of a nodule, lesion or lung mass, non-invasive diagnosis and staging of the patient is conducted using chest CT. The guidelines recommend that in patients who are operative candidates, clinicians estimate the probability of lung cancer using their clinical expertise or by use of a validated diagnostic prediction model.¹² In patients with lung nodules found by CT screening, the American College of Radiology recommends using the McWilliams model to estimate the probability of cancer in screening discovered nodules.²⁴ The ACCP suggests using the Mayo Clinic model to assess lung nodules, as it is the only risk model with extensive external validation.

Once the risk estimate has been made, the guidelines recommend patients be reevaluated with decreasing frequent CT scans when the probability of lung cancer is less than 5%. Patients with an intermediate probability of cancer, between 5% and 65%, are suggested to undergo additional testing such as FDG-PET scan or less invasive testing such as fine needle aspiration or bronchoscopy prior to surgical evaluation. Patients with a probability of cancer greater than 65% are recommended to undergo evaluation by a surgeon.⁴ If the lesion has an intermediate or high probability for cancer and the patient is an operative candidate, then surgical biopsy is recommended; if the likelihood of cancer is a low probability, then additional testing or surveillance is recommended.

Predictive Modeling for Surgeons

We have previously shown that the Mayo Clinic model is poorly calibrated in surgical populations with their higher prevalence of malignancy.²⁵ Deppen and colleagues developed the TREAT model for nodules evaluated for possible resection and externally validated it in a Veteran Affairs population.²⁶ The TREAT model provides a noninvasive tool at the point of the critical operative decision when patients have a comprehensive data set. Common variables exist across these models as shown in Table 1

Table 1.

Validated models for estimating lung cancer risk in screening populations and pulmonary nodules.

Model Type	Screening	Diagnostic			Treatment
Model	PLCOM2012	McWilliams	Mayo Clinic	VA	TREAT
Variable Age	√	√	√	√	√
Gender	√	√			√
Race	√
Education	√
Body Mass Index	√				√
COPD	√	√			√
Family history of cancer	√	√
Smoking (Y/N)	√		√	√
Smoking - Pack Years	√a				√
Years quit smoking	√			√
Hemoptysis					√
Previous Cancer	√		√		√
Lesion Size		√	√	√	√
Number of lesions		√
Lesion Growth					√
Spiculation		√	√		√
Lesion Location		√	√		√
FDG-PET Avidity					√

^acombination of cigarettes per day and smoking duration

Prior to our work, no models existed to estimate the lesion’s probability of malignancy at the point of surgical evaluation. We developed and validated the TREAT lung cancer prediction model and compared the performance of our model to the Mayo Clinic model in two populations being evaluated for lung resection. In the development cohort, the area under the receiver operating curve (AUC) and Brier score were 0.87 (95%CI: 0.83–0.92) and 0.12 respectively compared to the AUC 0.87 (95%CI: 0.83–0.92) and Brier score 0.13 in the validation dataset (Figure 2). AUC visualizes the various combinations of sensitivity and specificity that a predictive model demonstrates and allows direct comparisons between clinical prediction model accuracy. The Brier score compares how frequently risk estimated by a model matches observed risk and is a more robust measure of calibration.²⁷ The TREAT model had significantly higher accuracy (p<0.001) and better calibration than the Mayo model (AUC=0.80, 95%CI: 75–85; Brier score=0.17). The validated TREAT model had better diagnostic accuracy than the Mayo Clinic model in preoperative assessment of suspicious lung lesions in a population being evaluated for lung resection.

Figure 2. Area under the curve comparing TREAT and Mayo Clinic models.

TREAT model with VA validation performs better than the Mayo Clinic model in predicting lung cancer in nodules being evaluated by surgeons prior to operative resection. (From Deppen et al. JTO, 2014)

Translating Predictive Modeling Research into Practice

The ability of medical science to improve human health is dependent on the details of its application. Most physicians do not reliably adhere to best evidence, even when that information is provided through a decision support tool at the point of care. Thus, the translation and implementation of knowledge are critically important to successful use. Many healthcare interventions do not attain their expected benefits because of insufficient attention to their “human factors”— attributes that enable humans to do “the right thing” in the complexity of the real world. For example, a poorly designed user interface on a decision support tool can predispose clinicians to ignore the tool’s advice or even to misread it. Additional research and careful implementation of these predictive models with the assistance of clinicians with expertise in human factors will ensure that these tools are useful at the bedside.

SUMMARY

Multiple calculators exist that can help the clinician in clinical practice diagnose lung cancer at the bedside. The PLCO model helps determine who needs lung cancer screening and the McWilliams or Mayo models help guide the primary care clinician or pulmonologist with estimating the probability of cancer in patients with indeterminate pulmonary nodules. The TREAT model assists surgeons to determine who needs a surgical biopsy in patients referred with suspicious lesions. Additional work is needed to develop decision support tools that will facilitate the use of these models in clinical practice, to complement the clinician’s judgement and enhance shared decision making with the patient at the bedside.