Validation of an Updated Algorithm to Identify Patients With Incident Non–Small Cell Lung Cancer in Administrative Claims Databases

Abstract

PURPOSE

Real-world lung cancer data in administrative claims databases often lack staging information and specific diagnostic codes for lung cancer histology subtypes. This study updates and validates Turner's 2017 treatment-based algorithm using more recent claims and electronic health record (EHR) data.

METHODS

This study used Optum's deidentified Market Clarity Data of linked medical and pharmacy claims with EHR data. Eligible patients had an incident lung cancer diagnosis (January 2014-December 2020) and ≥one valid histology code for lung cancer 30 days before to 60 days after diagnosis. Histology and stage information from the EHR were used to evaluate the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We evaluated the Turner algorithm using cohort 1 patients diagnosed between June 2014 and October 2015 (step 1) and between November 2015 and December 2020 after approval of immunotherapies (step 2). Next, we evaluated cohort 2 patients diagnosed between November 2015 and December 2020 using an updated algorithm incorporating the latest US treatment guidelines (step 3), and compared the results for cohort 2 (Turner algorithm, step 2 patients). Furthermore, an algorithm to determine early NSCLC (eNSCLC; stage I-III) versus metastatic or advanced/metastatic non–small cell lung cancer (stage IV) was evaluated among patients with available histology and stage information.

RESULTS

A total of 5,012 patients were included (cohort 1, step 1: n = 406; cohort 1, step 2: n = 2,573; cohort 2, step 3: n = 2,744). The updated algorithm showed improved performance relative to the previous Turner algorithm for sensitivity (0.920-0.932), specificity (0.865-0.923), PPV (0.976-0.988), and NPV (0.640-0.673). The eNSCLC algorithm showed high specificity (0.874) and relatively low sensitivity (0.539).

CONCLUSION

An updated treatment-based algorithm identifying patients with incident NSCLC was validated using EHR data and distinguished lung cancer subtypes in claims databases when EHR data were not available.

A validated algorithm for identifying patients with NSCLC in administrative claims databases.

INTRODUCTION

Non–small cell lung cancer (NSCLC) accounts for approximately 82% of all lung cancer cases in the United States.¹ Most patients with NSCLC are diagnosed with stage I, stage II, or resectable stage III disease, known as early NSCLC (eNSCLC).² Immunotherapies and targeted therapies have recently been approved for patients with eNSCLC in addition to chemotherapy-based regimens, which were previously the standard of care.^3,4 Evaluating treatment patterns and outcomes in real-world data can help understand treatment gaps in clinical practice as new therapies become available.

CONTEXT

• Key Objective

• This study validated and updated a published treatment-based algorithm by Turner et al⁸ using electronic health record data to identify patients with non–small cell lung cancer (NSCLC) in administrative claims data.

• Knowledge Generated

• Using data from 5,012 patients, the updated algorithm showed improved performance from the published algorithm for sensitivity (0.920-0.932), specificity (0.865-0.923), positive predictive value (0.976-0.988), and negative predictive value (0.640-0.673). An algorithm to identify patients with early NSCLC showed high specificity (0.874) and relatively low sensitivity (0.539).

• Relevance

• This study updates and validates a previous algorithm to classify lung cancer patients in claims databases using their first-line treatments. The current study integrates recent US guidelines for lung cancer treatment. The method can separate patients with early or advanced lung cancer as well as NSCLC and small cell lung cancer. This can be valuable when health records are not accessible.

Administrative claims databases are often used for real-world data research since they are readily available and generally offer large sample sizes. However, the International Classification of Diseases (ICD), Clinical Modification, Ninth Revision (ICD-9) and Tenth Revision (ICD-10) coding systems do not provide adequate detail to differentiate lung cancer subtypes such as NSCLC from small cell lung cancer (SCLC). Uncertainty in identifying a research population of interest presents a significant challenge to the use of administrative claims data for NSCLC research, where including patients without NSCLC (low specificity) or those who are not representative of the NSCLC population (low sensitivity) would return spurious results.

To this end, previous studies have attempted to identify patients with NSCLC using proxies on the basis of treatment patterns identified from administrative claims data.^5-7 In particular, Turner et al⁸ developed and validated a treatment-based algorithm to identify NSCLC cases in administrative claims databases using ICD-9 and ICD-10 codes and first-line (1L) treatment information based on 2015-2016 clinical practice guidelines, the most recent at the time of the study. As such, the Turner algorithm does not account for newer NSCLC treatments approved since 2015, including immunotherapies and targeted therapies. It should also be noted that administrative claims databases lack staging information, and the absence of a metastatic diagnosis code does not necessarily mean that patients have early-stage disease. Thus, there is also a need for algorithms to distinguish early-stage NSCLC from later-stage NSCLC on the basis of what treatment the patient received (ie, surgery).

This study sought to develop an updated 1L treatment-based algorithm to differentiate patients with NSCLC or SCLC in administrative claims databases on the basis of the latest lung cancer treatment guidelines in the United States. The primary objective was to update and validate the previous algorithm by Turner et al. To further differentiate eNSCLC from advanced/metastatic NSCLC (advNSCLC), this study included a secondary objective to evaluate the performance of a treatment-based algorithm to identify patients with eNSCLC versus advNSCLC.

METHODS

Study Data and Patient Cohorts

This retrospective observational study used Optum's deidentified Market Clarity Data, which deterministically links medical and pharmacy claims with electronic health record (EHR) data from providers across the continuum of care. The Market Clarity Data uses natural language processing to identify cancer histology and stage from EHR data with acceptable performance statistics (positive predictive value [PPV] >0.8), and has been used in peer-reviewed published research.⁹ The EHR was used to determine histology, stage, and race/ethnicity. The American Joint Committee on Cancer version of the stage variable was unavailable; thus, the stage as documented in the EHR was directly used in the analyses. All other variables were identified or derived from the administrative claims.

Eligible patients had an incident lung cancer diagnosis identified in claims between January 1, 2014, and December 31, 2020. A 3-month washout period before the diagnosis date was used to ensure the initial diagnosis was captured. Continuous enrollment with medical coverage was required during the 3-month period before the initial diagnosis. Patients also had to have ≥one valid histology code for lung cancer identified in the EHR data within 30 days before or 60 days after the diagnosis date, and ≥3 months of continuous health plan enrollment with medical and pharmacy coverage after the diagnosis date. Patients were excluded if their histology records indicated contradictory lung cancer types.

After updating the lung cancer treatment list based on the latest US cancer treatment guidelines, two cohorts were created on the basis of evidence in claims of receipt of treatments on the older treatment list or the updated treatment list. Cohort 1 comprised patients with any 1L lung cancer treatment included in the previous Turner algorithm on or after lung cancer diagnosis. Cohort 2 comprised patients with any 1L lung cancer treatment included in the updated algorithm on or after lung cancer diagnosis. For both cohorts, the 1L treatment regimen was defined by the combination of systemic therapies received within 60 days of the first treatment date.

For the secondary objective of developing an algorithm to distinguish eNSCLC from advNSCLC, patients with stage I-III NSCLC (as documented in the EHR) were considered the eNSCLC group and patients with stage IV NSCLC were considered the advNSCLC group. Patients were excluded if their stage was classified as unknown, extensive, or limited. If a patient had records of multiple stages, the stage closest to their diagnosis date was used; if multiple stages were available on the same date, the most advanced stage was used.

Updated NSCLC Algorithm Development and Classification Rules

The Turner algorithm was updated with NSCLC treatments approved since October 2015 in concordance with the latest US treatment guidelines at the time of analysis (November 2022) and in collaboration with a clinical expert (Appendix Table A1). Consistent with the original Turner algorithm, NSCLC treatments were used as inclusion criteria and SCLC treatments were used as exclusion criteria.

Furthermore, patients were classified by the secondary algorithm as having eNSCLC if 1L therapy identified in claims included surgery procedure codes and did not include treatment used specifically in locally advanced or metastatic settings; otherwise, the patient was classified as having advNSCLC (Appendix Table A2).

NSCLC Algorithm Validation

Three distinct steps were executed to validate the updated 1L treatment-based algorithm for classifying NSCLC versus SCLC. Step 1 replicated the Turner algorithm using cohort 1 patients with an initial lung cancer diagnosis between June 2014 and October 2015, which was the study period used by Turner.⁸ Step 2 then applied the Turner algorithm using cohort 1 patients with an initial lung cancer diagnosis between November 2015 and December 2020 to test the hypothesis that performance would decrease after the approval of immunotherapies in 2015. Finally, step 3 then evaluated the updated algorithm using cohort 2 patients with an initial diagnosis between November 2015 and December 2020, using the same time period as cohort 2 for comparison purposes. For all analyses, histology and stage information from the EHR were considered the gold standard when compared with the claims-based algorithm for classification when calculating performance statistics.

Statistical Analysis

Descriptive statistics were used to summarize patient demographic and clinical characteristics. Median and interquartile range were reported for continuous variables, and counts and proportions were reported for categorical variables. Algorithm performance at each step was measured using sensitivity, specificity, PPV, and negative predictive value. All results were summarized using R version 4.2.1 (2022-06-23 release; The R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Study Cohort

A total of 5,012 patients met the eligibility criteria and were included in the study (Fig 1). Cohort 1 included a total of 3,291 patients, of whom 2,836 (86%) were classified as having NSCLC and 455 (14%) were classified as having SCLC. Cohort 2 included a total of 3,533 patients, which consisted of 3,078 (87%) patients with NSCLC and 455 (13%) with SCLC. Among the 3,078 patients with NSCLC in cohort 2, stage information was available for 2,582 (84%) patients, of whom 1,527 (59%) were classified as having eNSCLC (stage I-III) and 1,055 (41%) were classified as having advNSCLC (stage IV).

FIG 1.

Cohort attrition. The EHR was used to determine race/ethnicity, histology, and stage; all other variables were identified from the administrative claims. EHR, electronic health record; ICD-9, International Classification of Diseases, Ninth Revision; ICD-10, International Classification of Diseases, Tenth Revision; NSCLC, non–small cell lung cancer; SCLC, small cell lung cancer.

Characteristics were generally similar among patients in steps 1, 2, and 3 (Table 1). Within each step, characteristics were also similar between patients with NSCLC and those with SCLC, although a slightly greater proportion of patients with SCLC were women (data not shown). Patients in cohorts 1 and 2 had similar mean ages (68.2 and 68.4 years, respectively), and similar proportions were women (52% and 53%, respectively), White (82% and 83%, respectively) or African American (11% and 11%, respectively), and from the US Midwest (42% and 41%, respectively).

TABLE 1.

Cohort Characteristics

Characteristic	Step 1 (n = 406)	Step 2 (n = 2,573)	Step 3 (n = 2,744)
Age, years, median (IQR)	68 (60-75)	69 (62-76)	69 (62-76)
Sex, female, No. (%)	193 (48)	1,364 (53)	1,455 (53)
Race, No. (%)
White	332 (82)	2,143 (83)	2,302 (84)
Black or African American	44 (11)	265 (10)	275 (10)
Asian	12 (3)	31 (1)	31 (1)
Other/unknown	18 (4)	134 (5)	136 (5)
Ethnicity, No. (%)
Hispanic	a	76 (3)	77 (3)
Not Hispanic	386 (95)	2,222 (86)	2,374 (87)
Unknown	a	275 (11)	293 (11)
US region, No. (%)
Midwest	139 (34)	1,154 (45)	1,218 (44)
Northeast	125 (31)	770 (30)	851 (31)
South	108 (27)	492 (19)	515 (19)
West	a	77 (3)	76 (3)
Other/unknown	a	80 (3)	84 (3)
Lung cancer, No. (%)
NSCLC	346 (85)	2,209 (86)	2,381 (87)
SCLC	60 (15)	364 (14)	363 (13)
Stage, No. (%)
0	a	a	a
I	45 (11)	442 (17)	476 (17)
II	31 (8)	228 (9)	232 (8)
III	71 (17)	468 (18)	491 (18)
IV	131 (32)	716 (28)	810 (30)
Extensive	20 (5)	112 (4)	115 (4)
Limited	a	a	a
Unknown	96 (24)	531 (21)	546 (20)

NOTE. The EHR was used to determine race/ethnicity, histology, and stage; all other variables were identified from the administrative claims. Proportions may not sum to 100% because of rounding.

Abbreviations: EHR, electronic health record; HIPAA, Health Insurance Portability and Accountability Act; NSCLC, non–small cell lung cancer; SCLC, small cell lung cancer.

^aCell values ≤11 have been redacted according to the Guidance Regarding Methods for De-identification of Protected Health Information in Accordance with the HIPAA Privacy Rule.¹⁰

NSCLC Algorithm Testing and Validation

Step 1 provided a sample size of 406 patients with NSCLC or SCLC (June 2014-October 2015) with similar PPV as that reported by Turner et al⁸ (Table 2). From step 2 (n = 2,573) to step 3 (n = 2,744), where the updated algorithm was compared with a post-2015 Turner algorithm, performance improved slightly for all performance measures (Table 2).

TABLE 2.

Algorithm Performance at Each Testing and Validation Step

Performance Measure	Step 1: Turner Replication	Step 2: Turner 2015-2020	Step 3: Updated Turner 2015-2020
Date range	June 2014-October 2015	November 2015-December 2020	November 2015-December 2020
Sample size, No.	406	2,573	2,744
Sensitivity	0.873	0.920	0.932
Specificity	0.933	0.865	0.923
PPV	0.987	0.976	0.988
NPV	0.560	0.640	0.673

NOTE. Bold indicates improved performance.

Abbreviations: NPV, negative predictive value; PPV, positive predictive value.

eNSCLC Algorithm Performance

A total of 2,582 patients were included in the eNSCLC algorithm evaluation step. The algorithm performance showed a high specificity (0.874). The sensitivity was relatively low (0.539; Table 3).

TABLE 3.

eNSCLC Algorithm Performance

Performance Measure	eNSCLC Histology	advNSCLC Histology
eNSCLC algorithm, No. (%)	823 (32)	133 (5)
advNSCLC algorithm, No. (%)	704 (27)	922 (36)
Sensitivity	0.539
Specificity	0.874
PPV	0.861
NPV	0.567

Abbreviations: advNSCLC, advanced/metastatic non–small cell lung cancer; eNSCLC, early non–small cell lung cancer; NPV, negative predictive value; PPV, positive predictive value.

DISCUSSION

This study developed and validated a treatment-based algorithm for identifying patients with NSCLC in administrative claims databases, accounting for newer treatments approved since a previous version of this approach was developed. When applying the Turner algorithm, performance statistics were similar to those in the original publication,⁸ supporting the face validity of the histology variable available in the Optum deidentified Market Clarity Data as well as the replicability and reliability of the Turner algorithm.

The face validity of the updated NSCLC algorithm was also improved by updating the treatment list on the basis of the latest NSCLC and SCLC treatments available in the United States. Ultimately, the overall performance was improved across all measures after updating the previous NSCLC algorithm. In particular, specificity increased to 92.3%, likely because of improved SCLC identification after adding the updated SCLC treatments. Thus, the algorithm more accurately identifies patients with SCLC the majority of the time, which means researchers would now be able to exclude these patients and have more confidence that they are appropriately identifying patients with NSCLC in their study. Sensitivity also remained high at 93.2%, which is important as this means that the NSCLC patient sample identified by the algorithm is representative of the NSCLC patient population.

Although the algorithm distinguishing eNSCLC had higher specificity and PPV, its sensitivity and negative predictive value were lower, likely because of heterogeneity of treatment decisions among stage III patients, which can include some locally advanced patients. Not all stage III patients are resectable, potentially because of comorbidities, patient preference, or other factors related to patient resectability and operability,¹¹ which are difficult to characterize in claims data. Thus, there are likely some stage III patients who are resectable but did not undergo surgery and were therefore misclassified as having advNSCLC.

The patient population for this study was generally similar to those of relevant previous studies, with a median age of 68-69 years, aligned with the majority of lung cancer diagnoses in people older than 65 years,¹² and slightly older than the population used by Turner et al⁸ (NSCLC mean 60 years). Women comprised 48%-53% of this study population, similar to published findings from the SEER Program (50% women)¹³ and from the study by Turner et al⁸ (49%-50% women). Consistent with previous reports of 82% NSCLC among patients with lung cancer,¹ the majority of patients in this study (85%-87%) were identified with NSCLC, and fewer than half had stage IV disease.

Although this study provides a validated updated algorithm for identifying patients with NSCLC in administrative claims databases, some limitations should be noted. This is an iterative update to the work by Turner et al⁸ that will need to be updated with developments in the treatment landscape, clinical practice, and data science. The Optum deidentified Market Clarity Data primarily comprises commercially insured patients (younger than 65 years). Therefore, the algorithm may not perform similarly in Medicare or Medicaid claims databases. The study population may not be generalizable to those without health insurance coverage or those receiving care in other countries or settings. This study was not able to account for certain considerations related to the use of generic and biosimilar formulations, restrictions on the basis of unapproved (off-label) treatment use, treatment access, and treatment shortages that may affect real-world treatment use. It should also be noted that administrative claims databases may have an inherent potential for coding inaccuracies. Future work may consider additional information beyond the EHR, such as tumor registry data, which may be considered a strong source of tumor diagnosis and staging information.

In conclusion, EHR data were used to validate an updated treatment-based algorithm to accurately identify patients with NSCLC. This updated algorithm can be used in future real-world data research to distinguish lung cancer subtypes in claims databases when EHR data are not available.

APPENDIX

TABLE A1.

Treatment Regimens Used to Identify NSCLC v SCLC

NSCLC Regimens	SCLC Regimens
Lung removal or resection surgery	Atezolizumab + etoposidea + carboplatin
Abraxane (nab-paclitaxel)	Bendamustine
Ado-trastuzumab emtansine	Carboplatin + durvalumab + etoposidea
Afatinib	Carboplatin, etoposide
Alectinib	Carboplatin, irinotecan
Amivantamab-vmjw	Cisplatin + durvalumab + etoposidea
Atezolizumab	Cisplatin, etoposide
Atezolizumab, carboplatin, abraxane	Cisplatin, irinotecan
Atezolizumab, carboplatin, paclitaxel, bevacizumabb	Cyclophosphamide
Bevacizumab,b carboplatin, paclitaxel	Cyclophosphamide, doxorubicin, vincristine
Bevacizumab,b carboplatin, pemetrexed	Doxorubicin
Bevacizumab,b cisplatin, pemetrexed	Doxorubicin, liposomal
Brigatinib	Etoposide/etoposide phosphate
Cabozantinib	Ifosfamide
Capmatinib	Irinotecan
Carboplatin, docetaxel	Irinotecan, liposomal
Carboplatin, gemcitabine	Lurbinectedin
Carboplatin, paclitaxel	Prophylactic cranial radiation
Carboplatin, pemetrexed	Temozolomide
Carboplatin, vinorelbine	Topotecan
Cemiplimab-rwlc	Vincristine sulfate
Ceritinib	Vinorelbine tartrate
Cisplatin, docetaxel
Cisplatin, gemcitabine
Cisplatin, paclitaxel
Cisplatin, pemetrexed
Cisplatin, vinblastine
Cisplatin, vinorelbine
Crizotinib
Dabrafenib
Dabrafenib, trametinib
Dacomitinib
Durvalumab
Entrectinib
Erlotinib
Erlotinib, bevacizumab
Erlotinib, ramucirumab
Fam-trastuzumab deruxtecan-nxki
Gefitinib
Gemcitabine, docetaxel
Gemcitabine, vinorelbine
Larotrectinib
Lorlatinib
Mobocertinib
Nivolumab
Nivolumab, ipilimumab
Nivolumab, ipilimumab, paclitaxel, carboplatin
Nivolumab, ipilimumab, pemetrexed, carboplatin
Nivolumab, ipilimumab, pemetrexed, cisplatin
Nivolumab + chemotherapy without ipilimumab
Osimertinib
Pembrolizumab
Pembrolizumab, carboplatin, abraxane
Pembrolizumab, carboplatin, paclitaxel
Pembrolizumab, carboplatin, pemetrexed
Pembrolizumab, cisplatin, pemetrexed
Pralsetinib
Ramucirumab, docetaxel
Selpercatinib
Sotorasib
Tepotinib
Vemurafenib

Abbreviations: NSCLC, non–small cell lung cancer; SCLC, small cell lung cancer.

^aEtoposide (oral) may have been interchanged with etoposide phosphate (intravenous) formulation.

^bIncludes biosimilars.

TABLE A2.

Treatment Regimens Used to Identify advNSCLC

Treatment Regimen
Abraxane (nab-paclitaxel)
Ado-trastuzumab emtansine
Afatinib
Alectinib
Amivantamab-vmjw
Atezolizumab
Atezolizumab, carboplatin, abraxane
Atezolizumab, carboplatin, paclitaxel, bevacizumaba
Bevacizumab,a carboplatin, paclitaxel
Bevacizumab,a carboplatin, pemetrexed
Bevacizumab,a cisplatin, pemetrexed
Brigatinib
Cabozantinib
Capmatinib
Carboplatin, docetaxel
Carboplatin, gemcitabine
Carboplatin, paclitaxel
Carboplatin, pemetrexed
Carboplatin, vinorelbine
Cemiplimab-rwlc
Cisplatin, docetaxel
Cisplatin, gemcitabine
Cisplatin, paclitaxel
Cisplatin, pemetrexed
Cisplatin, vinblastine
Cisplatin, vinorelbine
Crizotinib
Dabrafenib
Dabrafenib, trametinib
Dacomitinib
Durvalumab
Entrectinib
Erlotinib
Erlotinib, bevacizumaba
Erlotinib, ramucirumab
Fam-trastuzumab deruxtecan-nxki
Gefitinib
Gemcitabine, docetaxel
Gemcitabine, vinorelbine
Larotrectinib
Lorlatinib
Mobocertinib
Nivolumab + chemotherapy without ipilimumab
Nivolumab
Nivolumab, ipilimumab
Nivolumab, ipilimumab, paclitaxel, carboplatin
Nivolumab, ipilimumab, pemetrexed, carboplatin
Nivolumab, ipilimumab, pemetrexed, cisplatin
Osimertinib
Pembrolizumab
Pembrolizumab, carboplatin, abraxane
Pembrolizumab, carboplatin, paclitaxel
Pembrolizumab, carboplatin, pemetrexed
Pembrolizumab, cisplatin, pemetrexed
Pralsetinib
Ramucirumab, docetaxel
Selpercatinib
Sotorasib
Tepotinib
Vemurafenib

Abbreviations: advNSCLC, advanced/metastatic non–small cell lung cancer; eNSCLC, early non–small cell lung cancer.

^aIncludes biosimilars.

AUTHOR CONTRIBUTIONS

Conception and design: Rongrong Wang, Daniel Sheinson, Ann Johnson, Janet Shin Lee

Financial support: Ann Johnson

Administrative support: Ann Johnson

Collection and assembly of data: Sandip Pravin Patel, Summera Qiheng Zhou

Data analysis and interpretation: Sandip Pravin Patel, Rongrong Wang, Summera Qiheng Zhou, Daniel Sheinson, Janet Shin Lee

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/cci/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).