Effect of an Automated Tracking Registry on the Rate of Tracking Failure in Incidental Pulmonary Nodules

Abstract

Objective:

Following incidental lung nodules with interval CT scanning is an accepted method to detect early lung cancer, but delayed tracking or failure to track is reported in up to 40% of patients.

Methods:

Our institution developed and implemented an automated lung nodule registry tracking system. This system uses a code at the time that a suspicious nodule is discovered to populate the registry. Suspicious nodules were defined as any nodule, solid or ground glass, <3 cm that the radiologist recorded as a potential malignancy or recommended for follow-up imaging. We exported the system to eight other Veterans Administration Medical Centers (VAMCs) with over 10,000 patients enrolled. We retrospectively reviewed 200 sequential CT scan reports containing incidental nodule(s) from two tertiary care university-affiliated VAMCs, both before and after the implementation of the registry tracking system. The primary outcome was the rate of tracking failure, defined as suspicious nodules that had no follow-up imaging or whose follow-up was delayed when compared with published guidelines. Secondary outcomes were predictors of tracking failure and reasons for tracking failure.

Results:

After implementation of the registry tracking system in the two VAMCs, we found a significant decrease in tracking failure, from a preimplementation rate of 74% to a postimplementation rate of 10% (P < .001). We found that age, nodule size, number, and nodule characteristics were significant predictors.

Conclusions:

The automated lung nodule registry tracking system can be exported to other health care facilities and significantly reduces the rate of tracking failure.

INTRODUCTION

The failure to pursue abnormal test results is a common problem in medicine [1,2]. Published reports indicate that pulmonary nodules incidentally discovered on CT imaging are not pursued according to published guidelines in up to 70% of cases, potentially resulting in delays in diagnosis and harm to patients [3-10]. With the recent recommendation for lung cancer screening with low-dose chest CT in high-risk patient populations, the incidence of pulmonary nodules will increase, highlighting the need to find novel solutions for efficient and reliable monitoring of these patients for the development of lung cancer [11].

The goal for providers is to track and evaluate incidental pulmonary nodules using established guidelines to identify early-stage malignancies where interventions can impact outcome [8,9,12]. Results from the National Lung Screening Trial demonstrate that early recognition and intervention of lung cancer improves survival [13]. Institutions across the country have approached this problem by developing interventions to reduce the loss-to-follow-up rates, with varying degrees of success [3,4,14-17]. With the expansion of the electronic health record (EHR), there are new tools to reduce the frequency of failed follow-up and potentially save lives [17].

Our local institution developed an automated tracking registry to monitor incidentally found lung nodules that is directly linked to our EHR and follows published guidelines [8,9]. Patients are entered into the automated registry via a code initiated by the radiologist when a nodule is identified on any imaging that includes any portion of the chest. This prototype was implemented at the Minneapolis Veterans Administration Medical Center (VAMC) in early 2012 and showed marked improvement in the rate of tracking failure (nodules lost to follow-up or with significant delays) at our home institution. After the initial success at our home institution, the system was expanded to all of the regional VAMCs, including implementation at two tertiary care facilities similar to our own. The system was implemented at site A in January 2013 and site B in March 2014. We set out to determine if multi-site implementation of this tracking system would decrease the rate of tracking failure in patients with incidentally detected lung nodules.

METHODS

Lung Nodule Tracking Program

Our lung nodule tracking registry uses a facet of the standard VAMC EHR to assign patients unique identifiers when a pulmonary nodule that warrants tracking is found. We use these identifiers as a means for populating an automatic lung nodule registry tracking system that interacts with the EHR to ensure that tracking is completed on a schedule following established guidelines [13]. Patients are automatically entered into the registry if the radiologist assigns a code to the imaging report when a lung nodule meeting guideline criterion is identified. Nonradiologist providers can also place patients into the registry tracking system through an EHR request. The EHRs of patients identified as candidates in the registry tracking system are reviewed by support staff to determine if the nodule meets criteria for serial chest CT surveillance. If tracking is indicated, the support staff completes a templated EHR note indicating the time to the next CT scan based on published guidelines. The EHR template automatically interacts with the registry via embedded data fields, creating an expected timeline to the next CT scan. The process repeats until tracking is completed or a definitive workup for suspected lung cancer is initiated. The registry tracking system contains an automated system to identify overdue scans, allowing patient contact and rescheduling, if indicated.

Subjects

The Minneapolis Veterans Administration Health Care System Institutional Review Board approved this study protocol and waived informed consent and HIPAA requirements. We retrospectively reviewed the management of 50 patients with lung nodules incidentally found on CT scans at each of two university-affiliated VAMCs. At site A we reviewed imaging immediately before and roughly two years after implementation of the automated tracking registry. At site B we reviewed patients before and six months after implementation. In the preimplementation review, nodules were identified via a review of sequential chest CT scan reports until we identified 50 patients at each site with a report of an incidental pulmonary nodule. We defined a nodule as any solid, mixed, or ground glass opacity <3 cm in largest diameter for which the radiologist recommended followup or that was described as concerning for malignancy. After implementation of the registry tracking system, we repeated the audit to identify 50 unique patients at each site with incidental nodules.

Outcome Measures

Our primary outcome is “tracking failure,” defined as a composite of (1) lost to follow-up and (2) delayed followup. We defined “lost to follow-up” as any patient who did not complete nodule tracking per guidelines in the absence of having a separate diagnosis that would have made nodule tracking unnecessary or documentation that the patient declined nodule tracking. We defined “delayed follow-up” as follow-up imaging that was delayed by at least 30 days beyond guideline criteria, as previously published [7]. CT scans performed earlier than recommended by guideline criteria were not deemed tracking failure.

For the analysis of postimplementation tracking failure, we defined human error as failure of the radiologist to enter the code despite noting a nodule requiring tracking in the report or registry tracking staff failing to use the proper template during enrollment, resulting in exclusion from the registry tracking system. Patient preference leading to delays in follow-up imaging despite imaging ordered at the appropriate timeline was also identified and considered a form of tracking failure, though this was based on the individual patient’s travel and availability rather than an error or fault in the system.

Statistical Analysis

Patient and site characteristics before and after implementing the lung nodule tracking program are described as means and standard deviations or proportions expressed as percentages. The differences in lung nodules that were tracked before and after implementing the registry tracking system were compared using logistic regression. The few cases of delayed tracking were combined with those that were not tracked. Sites were compared by testing for an interaction between the two sites and use of the tracking program. Multivariable logistic regression was employed to control for the effects of site, age, gender, and nodule size, multiplicity, and character. Analyses were clustered by site to adjust the standard errors, and hence P values and confidence intervals, for within-site correlation. A P value ≤ .05 was considered significant without adjusting for multiple comparisons. All analyses were performed using Stata software (version 12.1).

RESULTS

Patient Population and Nodule Characteristics

The patient characteristics were similar both pre- and post-implementation (Table 1). The mean age in the preimplementation cases was 63.6 compared with 66.5 in the postimplementation group. Our population was almost exclusively male in both groups, which is consistent with the VAMC patient population in that age range. The nodule size distribution was similar in both the pre- and postimplementation groups, with almost half of the nodules >8 mm in size (48% for both). The majority of patients in both groups had >1 nodule (79% and 67%, respectively), and the majority of nodules in both groups were solid (>90%).

Table 1.

Patient characteristics

	Preimplementation (N = 100)	Postimplementation (N = 100)
Age (years), mean (SD)	63.6 (11.8)	66.5 (9.9)
Gender
Male	99	92
Female	1	8
Nodule size (largest)
≤4 mm	23	24
>4 to ≤6 mm	12	13
>6 to ≤8 mm	17	15
>8 mm	48	48
Number of nodules
Single	21	33
Multiple	79	67
Characterization
Solid	95	91
Mixed	2	5
Ground glass	3	4

Tracking Failure of Lung Nodules

After implementation of the registry tracking system in the two VAMCs, we found a significant decrease in aggregate tracking failure for both sites, our primary outcome, from a preimplementation rate of 74% to a postimplementation rate of 10% (P < .001, Fig. 1a). Results from the individual sites showed that site A had a lost to follow-up rate of 50% in the preimplementation phase with delayed tracking in an additional 10% of patients, for an overall 60% tracking failure rate (Fig. 1b). After registry tracking system implementation, site A decreased both the lost to followup rate to 6% and delayed follow-up to 4%, for an overall tracking failure rate of 10%. Site B had a lost to follow-up rate of 78% in the preimplementation phase and an additional 10% with delayed tracking, for an overall tracking failure rate of 88% (Fig. 1b). After registry tracking system implementation, site B significantly decreased their lost to follow-up rate to 8% and delayed tracking to 2%, for an overall 10% tracking failure rate. In the postimplementation phase, all loss to follow-up was due to human error (n = 7). In four instances the radiologist did not assign the code for tracking enrollment despite identifying a suspicious nodule. In three instances the registry tracking support staff had template errors that prevented the nodule from entry in the registry tracking system. Delays in tracking beyond the recommendation by >30 days for those patients enrolled into the registry tracking system were all due to patient preference.

Fig 1.

(a) Preimplementation lost and delayed follow-up. The majority of patients experiencing tracking failure were due to loss to follow-up compared with delayed tracking, (b) Postimplementation lost and delayed follow-up. There were no significant differences between lost and delayed follow-up.

We found that patient age, nodule size, number of nodules, and nodule density significantly correlated with tracking failure. Tracking failure was more likely in patients with nodules < 4 mm, in both the pre- and postimplementation periods (95% Confidence Interval (CI) 0.569–0.893). Increased age was associated with a decreased risk of tracking failure both pre- and postimplementation (CI 0.974-0.996). Compared with solid nodules, nodules with mixed characteristics were less likely to experience tracking failure (CI 0.420-0.699). Although female sex suggested a reduced risk for tracking failure, the sample size was too small to be interpreted with confidence.

DISCUSSION

In this retrospective analysis we assessed the effect of implementation of an automated tracking system on the rate of tracking failure in managing patients with incidental lung nodules at two tertiary care, academic-affiliated VAMC hospitals. In the preimplementation phase we found a very high rate of tracking failure, with the majority of tracking failure due to loss to follow-up. After implementation of the registry tracking system, we saw a significant decrease in tracking failure due to both an improvement in the loss to follow-up and a decline in delayed follow-up. These results were consistent with a quality improvement project at our home institution that showed a reduction in tracking failure from roughly 30% to 5% after implementation of the tracking system.

On analysis of tracking failure in the pooled pre- and postimplementation data, a few key patient and nodule characteristics emerged as significant predictors of tracking failure. Small, solid nodules in younger patients were more likely to experience tracking failure. One can hypothesize that the reason for this is that each of these characteristics is linked to increased risk for malignancy. Larger nodules, nodules in the elderly, and nodules with mixed characteristics are all independent risk factors for malignancy, so one could assume that these findings would be more likely to be closely and properly followed up. To confirm this assumption, though, would require further study.

Human error was the primary driver of tracking failure in the postimplementation phase. In three cases loss to follow-up was due to inexperienced personnel entering data incorrectly into the registry. In four other cases the radiologist documented a nodule requiring follow-up but failed to enter the code for inclusion into tracking, nor did the ordering physician follow up on the nodule reported in the EHR. These findings suggest the need for ongoing system audits to assure compliance.

One method to limit human error is to require nodule coding of the radiology report to be a forced function when reporting any image that involves the chest. One critique to this approach is that it adds another forced step in an era of alert/process-heavy EHRs that may not be well received by practicing providers [18].

There is also some question as to how effective a forced function would be. A previous quality improvement project at our local institution addressed this by requiring all CTs with nodules generating an alert to ordering providers (data not shown). Unfortunately, this system did not alter the pre- and postimplementation lost-to-follow-up rate. To make a definitive determination, a more formal study would be necessary.

A centralized registry with standardized timelines for scheduling imaging shifts the primary responsibility of nodule tracking from the primary care provider to the registry. Tracking throughout the system is then standardized for all patients, limiting unnecessary radiation exposure or the deviation from published guidelines. Deviation from accepted recommendations is a common occurrence. Nodules are often tracked more frequently and for longer periods of time than what is recommended by the literature when following the recommendations of the radiologist reading the CT [4-9,19,20]. This failure in following published guidelines does not occur just with radiologists, as pulmonologists yield similar results in terms of frequency and duration of imaging [5,6]. Our centralized system of tracking incorporates published guidelines to avoid both over- and under-imaging. However, this system comes with an inherent labor cost, which in our system equates to roughly 1 nonphysician full-time employee per 2,000 registry patients.

In this study we did not include early imaging (ie, imaging before the recommended timeline) as inappropriate follow-up because the perceived risk for patients is higher in missed nodules versus any increased risk associated with more frequent CT scanning. In the majority of cases, early imaging was due to a non–nodule-related medical evaluation [4].

Another approach to capture abnormal radiologic findings is natural language processing (NLP). NLP could be used to identify the use of certain terms, such as “nodule,” in radiology reports to identify patients for lung nodule tracking. NLP has shortcomings, however, as shown in a publication by Wieneke et al in 2015. They highlighted the inability of NLP to properly navigate free-text pathology reports due to the high variability in the descriptive language used in the reports. This resulted in more than 30% of the evaluation set being incorrectly coded and a further 49.1% of the reports being sent for manual review [14]. Other studies have had similar variable results [21,22]. Compared with use ofNLP, our results suggest that the registry tracking system is superior to an automated attempt to capture specific words or phrases at this point in time. The technology behind NLP continues to improve and develop, and with the increasing effectiveness of “learning algorithms,” a time may come when NLP paired with an automated registry further improves the rate of tracking failure.

To our knowledge, this is the first published report of an automated system demonstrating a significant decrease in tracking failure of incidental pulmonary nodules. We show that the incidental lung nodule registry tracking system performs substantially better compared with usual practice, and that it can be disseminated and implemented at other institutions, resulting in consistent improvement in nodule tracking.

In conclusion, our study shows that implementation of an automated lung nodule registry tracking system linked to the EHR significantly reduces the rate of tracking failure in lung nodule tracking.

TAKE-HOME POINTS.

• Incidental pulmonary nodules lost to proper follow-up is a problem in pulmonary medicine.

• An automated nodule tracking registry system significantly reduced the rate of nodules with tracking failure.

• The automated nodule tracking registry system reduces the rate of tracking failure when exported to other systems.