Monitoring Lung Cancer Screening Use and Outcomes at Four Cancer Research Network Sites

Abstract

Rationale: Lung cancer screening registries can monitor screening outcomes and improve quality of care.

Objectives: To describe nascent lung cancer screening programs and share efficient data collection approaches for mandatory registry reporting in four integrated health care systems of the National Cancer Institute–funded Cancer Research Network.

Methods: We documented the distinctive characteristics of lung cancer screening programs, and we provide examples of strategies to facilitate data collection and describe early challenges and possible solutions. In addition, we report preliminary data on use and outcomes of screening with low-dose computed tomography at each of the participating sites.

Results: Programs varied in approaches to confirming patient eligibility, ordering screening low-dose computed tomographic scans, and coordinating follow-up care. Most data elements were collected from structured fields in electronic health records, but sites also made use of standardized order templates, local procedure codes, identifiable hashtags in radiology reports, and natural language processing algorithms. Common challenges included incomplete documentation of tobacco smoking history, difficulty distinguishing between scans performed for screening versus diagnosis or surveillance, and variable adherence with use of standardized templates. Adherence with eligibility criteria as well as the accuracy and completeness of data collection appeared to depend at least partly on availability of personnel and other resources to support the successful implementation of screening.

Conclusions: To maximize the effectiveness of lung cancer screening, minimize the burden of data collection, and facilitate research and quality improvement, clinical workflow and information technology should be purposefully designed to ensure that patients meet eligibility criteria and receive appropriate follow-up testing.

The National Lung Screening Trial (NLST) demonstrated that annual screening with low-dose computed tomography (LDCT) resulted in a 20% relative reduction in lung cancer mortality compared with annual screening with chest radiography (1). In absolute terms, this translated to 3 fewer deaths resulting from lung cancer for every 1,000 high-risk current and former smokers who underwent screening, a magnitude of benefit that is comparable to that reported for annual breast cancer screening with mammography in women 50 to 74 years of age (2, 3).

In the idealized settings of the trial, however, the observed reduction in lung cancer mortality was accompanied by a 39% risk of at least one false-positive test result after three rounds of annual screening (1). Almost 9 million Americans currently meet the NLST eligibility criteria for screening (4, 5), and even more meet the expanded criteria put forth by the United States Preventive Services Task Force (USPSTF) (6).

To determine the safety and effectiveness of LDCT screening in usual clinical practice, it is necessary to examine use and outcomes in community settings where most screening will take place. The integrated care delivery systems of the member organizations of the Cancer Research Network (CRN) are an ideal laboratory to begin to generate this evidence (7–9). Development of screening registries in these and other typical practice settings will make it possible to determine who is receiving screening and whether they meet eligibility criteria. In addition, registries can capture adherence with subsequent rounds of screening, rates of true-positive and false-positive screening test results, frequencies of noninvasive imaging tests and invasive procedures performed to evaluate findings suspicious for lung cancer, complications of invasive testing, and incidental findings of clinical importance.

The Centers for Medicare and Medicaid Services now requires participation in an approved national registry as a condition for reimbursement of LDCT screening examinations. Currently, the only approved registry is sponsored by the American College of Radiology (10). Required data elements include (1) identifiers for the patient, interpreting radiologist, and ordering provider; (2) computed tomography (CT) scanner make, model, and modality; (3) patient age, sex, height, weight, and smoking history; (4) documentation of shared decision-making; (5) date of screening examination, indication, examination results, and effective radiation dose; and (6) follow-up information, including subsequent diagnostic procedures, tissue diagnosis, and lung cancer stage.

Although it may be feasible for smaller screening programs to collect registry data elements prospectively and manually, this task may be considerably more challenging for larger hospitals and health care systems. Accordingly, we endeavored to develop approaches to optimizing the efficiency and accuracy of the data collection process. In this paper, we describe a variety of methodological approaches that we employed to monitor lung cancer screening in four integrated health systems, all members of the CRN: the Henry Ford Health System (HFHS) and three Kaiser Permanente (KP) regions (Colorado, Northern California [NCal], and Southern California [SCal]). Specifically, we describe approaches for data collection that may help nascent lung cancer screening programs to improve quality and fulfill reporting requirements. In addition, we provide context by describing the characteristics of the health care systems and the screening programs, and we demonstrate the output of data collection by providing information about screening use and short-term outcomes.

Methods

Through the CRN, researchers at four institutions developed automated methods of data collection and demonstrated the feasibility of using these methods to fulfill registry reporting requirements. Sites worked independently to develop methods to identify (1) eligibility for screening, (2) the occurrence of screening, (3) results of screening, (4) follow-up of positive screening examinations, and (5) screen-detected lung cancers. Site-specific study protocols were reviewed and approved by the respective institutional review boards at all sites.

Framework for Data Collection Methods

Development of data elements was guided by the following principles: (1) use data collected as part of routine clinical care and entered into the electronic health record (EHR); (2) leverage variables already present in the Health Care Systems Research Network Virtual Data Warehouse (VDW) (11), a set of common data elements that is used by all CRN sites; (3) minimize the need for manual chart review or prospective data entry to the greatest extent possible; and (4) develop approaches to enabling data collection in diverse health care environments. By using this framework, we aimed to strike an appropriate balance between feasibility, comprehensiveness, accuracy, and the ability to share data collection methods with others. Although the underlying goal was to capture LDCT data in standard formats across systems, different strategies to extract data from local electronic systems were used because of differences in the source data at each site.

Data Analysis

We describe the distinctive characteristics of the screening process that were implemented separately at each participating CRN site and provide illustrative examples of data collection strategies, including a description of some early challenges and possible solutions. To examine the use of LDCT screening to date, we report baseline characteristics of screened patients, including eligibility criteria, for all four sites. In addition, we summarize information about screening test results and follow-up at three of the sites with available data (HFHS, KP Colorado, and KP NCal). The dates of screening examinations differed across sites on the basis of data availability: November 1, 2013, to February 29, 2016 (HFHS); January 1, 2015, to December 31, 2015 (KP Colorado); July 1, 2014, to June 30, 2015 (KP NCal); and November 1, 2015, to February 29, 2016 (KP SCal). At KP NCal, data were collected during the pilot testing of screening workflows and tools in 3 of 15 medical service areas. Screening outcomes were ascertained through February 29, 2016, at all sites.

Results

Participating health systems varied by geographic location and size (Table 1). Comprehensive EHRs were used at all sites during the study period. Characteristics of lung cancer screening programs varied widely across sites (Table 1).

Table 1.

Characteristics of health systems and lung cancer screening program at participating Cancer Research Network sites

	HFHS	KPCO	KPNC	KPSC
Characteristic of the health system
Location	Metropolitan Detroit	Denver/Boulder and Front Range, Colorado	Northern California	Southern California
Members/population served, n	700,000	625,000	3,700,000	4,200,000
Medical offices, n	33	24	>200	>200
Hospitals, n	5	3 (contract)	21	14
Sites where LDCT is performed, n	9	3	23*	24
EHR system	Epic	Epic	Epic	Epic
Characteristics of the lung cancer  screening program
Year screening was first introduced	2011†	2014	2014 (pilot)	2014
Dashboard alert present in EHR to flag eligible patients	No	Yes	No	No
Electronic consult mechanism to process referrals	No	Yes	Yes	No
Standardized electronic order set	Yes	Yes	No	Yes
Method to confirm eligibility for screening	Checklist in order set‡	Checklist in order set	Checklist in electronic consult‡	Checklist in order set
Coordinator or navigator to confirm eligibility and/or facilitate care	Yes	Yes	Yes§	No
Tools available to facilitate referral for smoking cessation	Yes	Yes	Yes	Yes
Scans interpreted by dedicated chest radiologist	Yes	Most	Some	Some
Lung-RADS used to report screening test results	Yes	Yes	Yes	Yes
Population-based outreach to patients who meet eligibility criteria	Yes (as of March 2017)	No	No	No
Approaches to implementation
Initiated and sponsored by executive leadership	Yes	Yes	Yes	Yes
Based primarily in primary care practices	Yes	Yes	Yes	Yes
Mandatory participation by PCPs	No	Yes	No	No
Formal link to smoking cessation program(s)	Yes (as of May 2017)	Yes	No	No
Formal link to QI program(s)	No	No	Yes	No
Metrics to evaluate provider performance	No	No	No	No

Definition of abbreviations: EHR = electronic health record; HFHS = Henry Ford Health System; KPCO = Kaiser Permanente Colorado; KPNC = Kaiser Permanente Northern California; KPSC = Kaiser Permanente Southern California; LDCT = low-dose computed tomography; Lung-RADS = Lung CT Screening Reporting and Data System; PCP = primary care physician; QI = quality improvement.

^*LDCT was performed at five sites during the pilot study period.

^†Prior to 2011, HFHS was an enrollment site for the National Lung Screening Trial.

^‡Original check box format subsequently converted to free-text data entry to improve accuracy of smoking information.

^§KPNC adopted a navigator model near the end of the pilot testing/data collection period.

Practices for Referral and Determination of Eligibility

Decisions to screen at all sites were triggered by individual provider suggestion or patient request. None of the sites employed population-based methods of outreach, although one site (KP Colorado) implemented a dashboard alert in the EHR to flag patients who were eligible for screening based on age and smoking history. Most of the sites implemented a standardized order set to confirm eligibility for LDCT screening, although one site (KP NCal) developed an electronic consult (eConsult) mechanism to process referrals. Some sites initially used categorical check boxes and drop-down menus to document age and smoking history, but HFHS and KP NCal subsequently converted to free-text data entry to minimize referral of ineligible patients. Three sites ultimately employed a navigator or coordinator to confirm eligibility and/or coordinate care, and all sites provided tools to support shared decision-making and referral to smoking cessation programs.

Practices for Tobacco Cessation Treatment and Scan Interpretation

At HFHS, referrals to the screening program trigger notification of the smoking cessation clinic, whereas at KP Colorado, orders for nicotine replacement therapy and varenicline treatment are embedded in the lung cancer screening order set. Dedicated chest radiologists interpreted most or all of the screening low-dose computed tomographic scans at HFHS and KP Colorado, whereas general radiologists interpreted most scans at the other sites. Radiologists at all sites use the Lung CT Screening Reporting and Data System (Lung-RADS) (12).

Data Collection Strategies and Challenges

Researchers at all sites used EHR data to collect information about patient demographic characteristics and smoking history (Table 2). At KP NCal, the eConsult mechanism was also used to improve smoking history documentation. Standardized order templates and/or unique local procedure codes were used at all sites to distinguish between screening LDCT and chest CT performed for diagnosis or follow-up of a previously identified nodule. Similarly, standardized templates and/or identifying hashtags (e.g., #LCS2 to denote a Lung-RADS category 2 finding) were used to capture data about screening test results. These methods were complemented by the use of natural language processing (NLP) algorithms at KP Colorado and KP SCal. Finally, information about follow-up tests was derived from Current Procedural Terminology and International Classification of Diseases codes, whereas information about cancer diagnoses was obtained from local tumor registries. Below, we provide additional details about some of these methods for illustrative purposes.

Table 2.

Data collection strategies at participating Cancer Research Network sites

	HFHS	KPCO	KPNC (Pilot)	KPSC
Data domain
Demographic characteristics	EHR/VDW	EHR/VDW	EHR/VDW	EHR/VDW
Smoking	EHR/VDW	EHR/VDW	EHR/VDW, eConsult template	EHR/VDW
LDCT indication	Epic SmartSet template; local procedure codes	Epic SmartSet template	eConsult template; history section of radiology reports	Epic SmartSet template; local procedure codes
LDCT results	Lung-RADS category templates	Radiologist-dictated “track”; NLP to extract nodule characteristics	Radiologist-dictated Lung-RADS hashtag	Lung-RADS category templates; NLP to extract nodule characteristics
Follow-up diagnostic procedures	CPT/ICD codes	CPT/ICD codes	CPT/ICD codes	CPT/ICD codes
Cancer diagnoses	Tumor registry	Tumor registry	Tumor registry	Tumor registry

Definition of abbreviations: CPT = Current Procedural Terminology; eConsult = electronic consult; EHR = electronic health record; HFHS = Henry Ford Health System; ICD = International Classification of Diseases; KPCO = Kaiser Permanente Colorado; KPNC = Kaiser Permanente Northern California; KPSC = Kaiser Permanente Southern California; LDCT = low-dose computed tomography; Lung-RADS = Lung CT Screening Reporting and Data System; NLP = natural language processing; VDW = Health Care Systems Research Network Virtual Data Warehouse.

Determining eligibility for screening

Incomplete information on smoking behavior in the EHR is an important barrier to determining eligibility for screening. For example, among all current and former smokers aged 55–80 years at HFHS, pack-year history and quit date (if applicable) could be determined for only 44.3%. Similarly at KP NCal, smoking status, pack-year history, and time since quitting (among former smokers) were documented in the VDW for 99.7%, 46.0%, and 53.7% of screened members, respectively, with complete smoking history documented for only 44%. However, by combining VDW and eConsult data, KP NCal improved the completeness of both pack-year history (70.7%) and time since quitting (67.7%), with complete smoking history documented for 66% of screened members. The completeness and accuracy of smoking history information was further improved at KP NCal due in part to the implementation of a patient navigator model near the end of the data collection period.

Determining the occurrence of screening

The use of standardized order templates has enabled researchers at three sites to ascertain when screening examinations have been performed. At HFHS, KP Colorado, and KP SCal, these orders are linked to unique local “examination type” codes to help distinguish between orders for diagnostic CT, screening LDCT, and LDCT for nodule follow-up. The codes can subsequently be retrieved from the files of the local radiological information system and/or EHR. Although this is helpful for data collection, it does not necessarily prevent the provider from ordering the wrong test—some screening tests are still being ordered as full-dose diagnostic examinations by mistake, requiring the rejection of the initial request and resubmission of an appropriate order for LDCT.

At KP NCal, several sources of information were used to distinguish between screening and diagnostic or surveillance chest computed tomographic scans because local codes to distinguish between chest computed tomographic examination type were not yet implemented during the data collection period. Screening examinations were identified by searching eConsult records for a local indication code for lung cancer screening and by text mining of chest CT reports to find a specific hashtag denoting the Lung-RADS category. Information on patient history was then used secondarily to identify and exclude nonscreening examinations.

Determining the results of screening

Radiologists at all sites are encouraged to use a standardized template to report characteristics of each pulmonary nodule identified by screening LDCT, creating a structured report. However, among KP NCal patients with identified nodules, only 33.6% had nodule characteristics documented in a structured text format. Similar issues with nonadherence have been encountered at KP SCal, suggesting that NLP and/or other text mining approaches are needed to extract results of interest from radiology transcripts.

At KP Colorado, to capture standardized text descriptors of Lung-RADS category, radiology reports are extracted, and structured query language text mining techniques are employed; this is done to assign each patient to a specific track for surveillance imaging. The track assignment is included on the patient’s health maintenance “dashboard” in a list of gaps in care that need to be closed.

At KP SCal, an NLP algorithm was developed and refined to identify patients with one or more lung nodules noted on a dictated radiology transcript. The initially published version of the NLP had a sensitivity and specificity of 96% and 86% for nodule identification, respectively (13). In an external validation, sensitivity and specificity were 90% and 86%, respectively (14). The algorithm was used to identify almost 70,000 unique adult members of KP SCal with at least one radiology transcript that was positive for a lung nodule between January 2006 and December 2012 (15). Although most of these lung nodules were incidental findings, the same methods are now being used to identify patients with nodules detected by screening.

To extend the functionality of the existing NLP program, similar iterative methods are being used to extract information from dictated radiology transcripts about the number, size, location, edge characteristics, and attenuation characteristics (density) of pulmonary nodules and masses. To facilitate adoption by other health systems, pseudocode that describes the key elements of the algorithm and is not specific to any programming language has been developed.

Determining the timing, type, and results of follow-up testing

We had little difficulty identifying information about downstream use of imaging tests and biopsy procedures. The availability of Current Procedural Terminology and International Classification of Diseases codes as a structured data element in the CRN VDW greatly facilitated ascertainment of specific procedures.

Use and Results of Screening

Characteristics of over 3,800 patients screened for lung cancer by LDCT in four health care systems are shown in Table 3. Despite attempts to verify eligibility, screening outside USPSTF criteria was common. At the four sites, as many as 7% of screenees were under 55 years of age, and up to 6% of screenees were older than 80 years of age. Variable percentages (11 to 32%) of current and former smokers had less than a 30–pack-year history of cigarette smoking, and, among former smokers, up to 29% had quit more than 15 years prior to screening. Missing data pertaining to smoking status were also common, with over 50% having an unknown smoking status at HFHS. The burden of comorbid conditions was variable across sites, with as many as 44% of patients having a Charlson comorbidity index greater than or equal to 2 at one site.

Table 3.

Characteristics of patients who underwent screening

	Dates of LDCT Screening Examinations
	11/2013–2/2016 HFHS (n = 899)	1/2015–12/2015 KPCO (n = 1,247)	7/2014–6/2015 KPNC (n = 359)	11/2015–2/2016 KPSC (n = 1,317)
Age, yr
<55	19 (2.1)	23 (1.8)	17 (4.7)	93 (7.1)
55–59	250 (27.8)	221 (17.7)	76 (21.2)	207 (15.7)
60–64	265 (29.5)	337 (27)	94 (26.2)	293 (22.2)
65–69	219 (24.4)	362 (29)	92 (25.6)	325 (24.7)
70–74	115 (12.8)	208 (16.7)	56 (15.6)	209 (15.9)
75–80	27 (3.0)	92 (7.4)	24 (6.7)	127 (9.6)
81+	4 (0.4)	4 (0.3)	0	63 (4.8)
Sex
Male	499 (55.5)	718 (57.6)	219 (61.0)	762 (57.9)
Female	400 (44.5)	529 (42.4)	140 (39.0)	555 (42.1)
Race/ethnicity
Non-Hispanic white	585 (65.1)	974 (78.1)	263 (73.3)	818 (62.1)
Asian	8 (0.9)	21 (1.7)	19 (5.3)	104 (7.9)
Black	92 (10.2)	53 (4.3)	31 (8.6)	156 (11.8)
Hispanic	5 (0.6)	96 (7.7)	33 (9.2)	180 (13.7)
Native American/Alaskan/Pacific Islander	2 (0.2)	19 (1.5)	4 (1.1)	23 (1.7)
Other	10 (1.1)	26 (2.1)	0 (0.0)	14 (1.1)
Unknown	197 (21.9)	58 (4.7)	9 (2.5)	22 (1.7)
Smoking status*
Never	20 (2.2)	10 (0.8)	3 (0.8)	137 (10.4)
Former	235 (26.1)	481 (38.6)	133 (37)	608 (46.2)
Current	224 (24.9)	756 (60.6)	222 (61.8)	572 (43.4)
Unknown	420 (46.7)	0	1 (0.3)	0
Pack-year smoking history (current and former smokers)
<30	95 (20.7)	142 (11.5)	38 (10.7)	379 (32.1)
30+	169 (36.8)	1095 (88.5)	213 (60)	483 (40.9)
Unknown	195 (42.5)	0 (0.0)	104 (29.3)	318 (26.9)
Time since quit, yr (former smokers only)
<15	112 (47.7)	452 (94)	78 (58.6)	331 (54.4)
15+	30 (12.8)	29 (6)	12 (9)	174 (28.6)
Unknown	93 (39.6)	0	43 (32.3)	103 (16.9)
Charlson comorbidity index
0	653 (72.6)	496 (39.8)	198 (55.2)	406 (30.8)
1	128 (14.2)	342 (27.4)	78 (21.7)	326 (24.8)
2	56 (6.2)	185 (14.8)	30 (8.4)	233 (17.7)
3+	62 (6.9)	224 (18)	53 (14.8)	346 (26.3)
Missing	0	0	0	6 (0.5)

Definition of abbreviations: HFHS = Henry Ford Health System; KPCO = Kaiser Permanente Colorado; KPNC = Kaiser Permanente Northern California; KPSC = Kaiser Permanente Southern California; LDCT = low-dose computed tomography.

^*Smoking status determined by Health Care Systems Research Network Virtual Data Warehouse and electronic consult combined at KPNC.

Results of screening

Screening results by Lung-RADS category were available for three of the four sites (Table 4). There was some variability across sites in the distribution of screening results, primarily due to differences in scans coded in Lung-RADS categories 1 (negative) and 2 (benign). Approximately half of screening LDCT examinations were coded as Lung-RADS category 2 at HFHS (44%) and KP Colorado (57%), whereas less than one-fourth were placed in this category at KP NCal (23%). Category 4 findings (suspicious for cancer) were recorded for 9% of screened patients at KP Colorado and 6% of screened patients at KP NCal, but they were less common at HFHS (<2%).

Table 4.

Results of screening and use of follow-up testing

	Dates of LDCT Screening Examinations
	11/2013–2/2016 HFHS (n = 899)	1/2015–12/2015 KPCO (n = 1,247)	7/2014–6/2015 KPNC (n = 359)
Lung-RADS category, n (% of those with Lung-RADS category)*
0	0	53 (4.3)	0
1	86 (29.7)	218 (17.5)	222 (61.8)
2	173 (59.7)	709 (56.9)	84 (23.4)
3	24 (8.3)	158 (12.7)	29 (8.1)
4A	4 (1.4)	68 (5.5)	9 (2.5)
4B	2 (0.7)	40 (3.2)	11 (3.1)
4X	1 (0.3)	1 (0.1)	0
Missing	0	0	4 (1.1)
Total with Lung-RADS category	290 (32.3)	1,247 (100.0)	359 (100.0)
Follow-up diagnostic procedures within 3 mo, n (% of patients in Lung-RADS category 4A, 4B, or 4X)
Chest CT	1 (16.7)	18 (16.5)	7 (35.0)
PET/CT	3 (50.0)	30 (27.5)	9 (45.0)
Any subsequent (either CT or PET)	4 (66.7)	40 (36.7)	14 (70.0)
Nonsurgical biopsy	1 (16.7)	14 (12.8)	9 (45.0)
Surgical biopsy	1 (16.7)	6 (5.5)	5 (25.0)
Any biopsy	1 (16.7)	16 (14.7)	9 (45.0)
Either CT/PET or biopsy	4 (66.7)	43 (39.4)	15 (75.0)
Total in Lung-RADS category 4A, 4B, or 4X	7	129	20
Lung cancer diagnoses through February 29, 2016, n (% of those screened)	6 (0.7)	20 (1.6)	6 (1.7)

Definition of abbreviations: CT = computed tomography; HFHS = Henry Ford Health System; KPCO = Kaiser Permanente Colorado; KPNC = Kaiser Permanente Northern California; LDCT = low-dose computed tomography; Lung-RADS = Lung CT Screening Reporting and Data System; PET = positron emission tomography.

^*Excludes 609 patients at HFHS for whom Fleischner Society guidelines were used before the introduction of Lung-RADS.

Follow-up testing and cancer diagnoses

Due to limited follow-up time after screening, we restricted examination of downstream follow-up procedures to those occurring within 3 months of the initial scan for patients classified as Lung-RADS category 4 (Table 4). Lung-RADS recommends prompt follow-up in these cases, either by additional imaging or by biopsy. Again, there was variability by screening site, with 39% of category 4 patients at KP Colorado and 75% of category 4 patients at KP NCal receiving some type of diagnostic examination within 3 months. Invasive biopsy was performed in 16% (HFHS), 15% (KP Colorado), and 45% (KP NCal) of category 4 patients. Lung cancer was diagnosed in 0.7% (HFHS), 1.6% (KP Colorado), and 1.7% (KP NCal) of those screened by the end of follow-up.

Discussion

In this paper, we describe the characteristics of screening programs at four integrated health care systems and the methods they employed to monitor the use and outcomes of lung cancer screening with LDCT. These care practices and data collection methods are broadly generalizable to other community settings where most lung cancer screening will take place.

Common challenges across participating sites included incomplete documentation of tobacco smoking history in the EHR, difficulty distinguishing between scans performed for screening and those performed for diagnostic purposes or for surveillance of a previously identified nodule, and variable adherence with use of templates and dictation scripts to facilitate structured reporting. Because these and other related challenges are not unique to the four participating sites, we provide a list of potential solutions that were implemented in our practice settings in Tables 5 and 6.

Table 5.

Challenges and potential solutions to facilitate successful care processes for lung cancer screening based on combined experience at participating sites

Care Process	Challenges	Potential Solutions
Determine eligibility	Not offering screening to eligible patients	Implement dashboard alert in EHR to flag eligible patients based on age and smoking history
	Offering screening to patients who are not eligible	Employ navigator or eConsult mechanism to confirm eligibility
		Use standardized order templates that require free-text entry of age and smoking history
Engage in shared decision-making	Offering screening without sufficient provision of information	Make tools for SDM available to patients and providers in multiple formats, including paper based, Internet based, and video based
Provide smoking cessation counseling	Not offering smoking cessation counseling to those eligible	Include option to refer for smoking cessation counseling in lung cancer screening order sets
		Use EHR to trigger automatic referral for smoking cessation counseling
		Embed electronic orders for nicotine replacement therapy and varenicline in lung cancer screening order sets
Interpret low-dose CT scans	Lack of standardization in documentation of results	Create standardized templates with Lung-RADS categories and/or specific fields for nodule size, attenuation, and other characteristics
		Employ dedicated chest radiologists or other specialized readers
Coordinate referrals; encourage adherence with follow-up testing and repeat annual screening	Delays in care and losses to follow-up	Employ navigator to coordinate care
		Assign patients to specific tracks for follow-up and include track assignment on dashboard in EHR

Definition of abbreviations: CT = computed tomographic; eConsult = electronic consult; EHR = electronic health record; Lung-RADS = Lung CT Screening Reporting and Data System; SDM = shared decision-making.

Table 6.

Challenges and potential solutions to facilitate successful data collection for lung cancer screening based on combined experience at participating sites

Required Data Elements	Challenges	Potential Solutions
Patient demographic characteristics	Effort and cost associated with manual chart review and prospective data entry	Leverage EHR and other existing data sources
Age and smoking history	Incomplete or erroneous data entries	Use standardized order templates that require free-text entry of age and smoking history
		Include specific fields in EHR for pack-years of smoking and years since quitting
Shared decision-making and referral for smoking cessation counseling	Incomplete or erroneous data entries	Include specific fields in EHR or standardized order sets for documentation of SDM and smoking cessation counseling
Distinguish between screening LDCT and diagnostic CT	Erroneous data entries	Insert identifiable hashtags in radiology reports
		Introduce unique local procedure codes to distinguish between screening LDCT and diagnostic CT
Results of screening, including Lung-RADS category and nodule characteristics	Incomplete or erroneous data entries	Insert identifiable hashtags in radiology reports
		Use text mining or natural language processing to identify nodules and their characteristics from radiology reports
Follow-up imaging and biopsy procedures	Incomplete or erroneous data entries	Leverage EHR and billing records
Cancer diagnosis	Incomplete or erroneous data entries Delayed reporting	Obtain information from local cancer registry Employ rapid case ascertainment methods

Definition of abbreviations: CT = computed tomography; EHR = electronic health record; LDCT = low-dose computed tomography; Lung-RADS = Lung CT Screening Reporting and Data System; SDM = shared decision-making.

Regarding the ascertainment of smoking status, it may take time to improve EHR documentation of specific information about pack-years and time since quitting; this information had less practical importance prior to the introduction of lung cancer screening into clinical practice. Better data about smoking behaviors in the EHR may provide other benefits for chronic disease management or the monitoring of smoking cessation programs.

Structured reporting has been embraced by many chest radiologists and other radiologists whose practice includes a large volume of screening low-dose computed tomographic scans, but it requires changes in usual work flow that might be viewed with less enthusiasm by general radiologists who interpret low-dose computed tomographic scans much less often. However, we believe that structured reporting of nodule size and characteristics, in addition to Lung-RADS category, is an important component of effective screening that would help to improve quality of care and facilitate future research.

Screening lower-risk patients who do not meet standard eligibility criteria will make screening less efficient and more costly at the population level. We observed variably poor adherence with USPSTF eligibility criteria for screening. However, it is not clear whether this reflects inappropriate screening of individuals at lower risk or if it is an artifact caused by incomplete or erroneous documentation of smoking behavior in the EHR, ordering a low-dose computed tomographic scan for symptom evaluation instead of a diagnostic computed tomographic scan to reduce radiation exposure, or gaming by providers to help patients avoid copays for diagnostic examinations. Most likely, some or all of these factors are jointly responsible, suggesting that it will be necessary not only to educate providers but also to design better information systems that make it easy for providers to do the right thing. Of note, we found that screening outside the eligibility criteria appeared to be less common at the two sites that adopted the most rigorous methods for documenting patient eligibility (KP Colorado and KP NCal). In contrast, screening outside eligibility criteria and incompleteness of data were most apparent at the site with the least extensive infrastructure to support screening (KP SCal).

Because Lung-RADS category is entered directly by the interpreting radiologist, the observed variability in its distribution across three of the sites is probably not an artifact, with coding of a much higher percentage of category 2 findings (e.g., small nodules that are likely to be benign) at HFHS and KP Colorado, where most low-dose computed tomographic scans were interpreted by chest radiologists, than at KP NCal, where most studies were read by general radiologists. Similar variation in the frequency of nodule identification was reported across the mostly academic centers that participated in the NLST (16), so it is not surprising that variation would be present in our study and other community settings. Somewhat more surprising is the relatively low prevalence of category 4 findings at HFHS (2.4%), especially when compared with the higher prevalence of suspicious findings at KP NCal (5.6%) or KP Colorado (8.8%) or in the baseline (6.2%) or subsequent (4.6%) rounds of screening during the NLST (17).

Observed differences in the frequency of follow-up testing for patients with Lung-RADS category 4 findings probably represents true practice variation. At our sites, one or more tests for follow-up were performed within 3 months of the abnormal screening test result in 39 to 75% of these individuals, representing less than optimal adherence with prompt evaluation. However, longer follow-up intervals are required to capture more complete information about testing. Unfortunately, Lung-RADS and other guidelines for pulmonary nodule evaluation are complicated and difficult to implement. Technology-enabled, population-based approaches will be necessary to make sure that patients do not fall through the cracks.

Our findings add to a growing literature on the implementation of lung cancer screening in clinical practice (18–20). The use of patient navigators, care coordinators, and standardized reporting of results has been described previously (21), but researchers in prior studies have not provided recommendations for data collection. Confirming our experience, the Veterans Health Administration demonstration project concluded that implementing lung cancer screening at the eight participating sites was complex and challenging, requiring new tools, care processes, and dedicated care coordination (22). Likewise, the results of a recent population-based study using data from the National Health Interview Survey confirm our observation that a sizable fraction of screening examinations are being performed in nonsmokers or lower-risk smokers who do not meet standard eligibility criteria (23).

This work has several limitations. First, the data presented are preliminary and reflect practices for LDCT screening that are new and evolving as they disseminate more widely within these and other health systems. Longitudinal data over longer periods of time are necessary to evaluate the outcomes of screening in diverse practice settings.

Second, our methods for data collection rely on the availability of EHRs and an information technology infrastructure that permits access to structured data. Although these resources are becoming increasingly common, they are not available in all health care settings, and relatively few systems routinely use such data for research and evaluation purposes. The large size and integrated character of the four participating delivery systems offer additional advantages that might not be replicable elsewhere.

Finally, we did not systematically examine whether shared decision-making or referral for smoking cessation was performed or documented. However, referral for smoking cessation within 3 months of screening was observed in approximately 70% of current smokers who underwent screening during an earlier time period at KP SCal (data not shown). Shared decision-making and smoking cessation are two of the nine core components of an effective screening program as outlined by the American College of Chest Physicians and the American Thoracic Society (24, 25). However, the quality of shared decision-making likely varies by center and provider, and —novel approaches and tools should be developed to facilitate this approach in practice.

In summary, we describe novel methods for facilitating data collection to monitor the occurrence and results of lung cancer screening with LDCT in four integrated health care systems. Sites adopted a variety of approaches to implement screening in their practice settings, which may have contributed to between-site differences in eligibility of those screened, LDCT results, and the use of follow-up tests. Integral components include the use of standardized order sets, improved documentation of smoking history, structured reporting of results, unique procedure codes to capture the occurrence of LDCT screening, and clinical navigators to verify eligibility and coordinate repeat screenings and follow-up. To maximize the effectiveness of screening, minimize the burden of data collection, and make such data available for research and quality improvement, clinical workflow and information technology must be purposefully designed to ensure that patients referred for lung cancer screening meet eligibility criteria and receive appropriate follow-up testing.