Use of a Deep Learning Algorithm for Detection and Triage of Cancer-associated Incidental Pulmonary Embolism

Abstract

Incidental pulmonary embolism (iPE) is a common complication in patients with cancer, and there is often a delay in reporting these studies and a delay between the finalized report and time to treatment. In addition, unreported iPE is common. This retrospective single-center cross-sectional study evaluated the effect of an artificial intelligence (AI) algorithm on the report turnaround time, time to treatment, and detection rate in patients with cancer-associated iPE. Adult patients with cancer were included either before (July 1, 2018, to June 30, 2019) or after (November 1, 2020, to April 30, 2021) implementation of an AI algorithm for iPE detection and triage. The results demonstrated that reported iPE prevalence was significantly higher in the period after AI implementation (2.5% [26 of 1036 studies] vs 0.8% [16 of 1892 studies], P < .001). Both report that the turnaround time (median, 0.66 hour vs 24.68 hours, P < .001) and time to treatment (median, 0.98 hour vs 28.05 hours, P < .001) were significantly shorter after AI implementation. In conclusion, the use of AI for detection and triage of iPE in clinical practice resulted in an increased detection rate of iPE and significantly shorter report turnaround time and time to treatment for patients with cancer-associated iPE.

Keywords: Cancer-associated Incidental Pulmonary Embolism, Pulmonary Embolism, Artificial Intelligence, Cancer, CT Imaging

© RSNA, 2023

Summary

Use of a deep learning algorithm for detection and triage of incidental pulmonary embolism resulted in an increased detection rate and decreased report turnaround time and time to treatment for patients with cancer-associated incidental pulmonary embolism.

Key Points

• ■ The prevalence of reported incidental pulmonary embolism (iPE) in patients with cancer was significantly higher after implementing a deep learning artificial intelligence (AI) algorithm for detection and triage of iPE (after AI, 2.5% vs before AI, 0.8%, P < .001).

• ■ Of the 26 reported cases of iPE after AI implementation, 24 were AI true-positive and two were AI false-negative.

• ■ The report turnaround time (median, 0.66 hour vs 24.68 hours, P < .001) and time to treatment (median, 0.98 hour vs 28.05 hours, P < .001) were significantly shorter after AI implementation.

Introduction

Pulmonary embolism (PE) is a common complication in patients with cancer and is often incidentally detected on CT scans for cancer staging or treatment response evaluation (1,2). There is often a delay in reporting these nonurgent CT studies, with additional delay from the report-to-treatment initiation (3). In addition, incidental PE (iPE) is commonly missed, with previous retrospective studies indicating rates of 32%–79% (4–8).

Artificial intelligence (AI) algorithms could be used to address both issues. A previous retrospective study investigated iPE prevalence and evaluated a deep learning AI algorithm for detection of iPE (8). The AI algorithm had 90.7% sensitivity and 99.8% specificity, and iPE prevalence was 4.0%, with 79% of cases unreported. However, the AI algorithm may not demonstrate the same performance when evaluated prospectively. This is exemplified by studies reporting on an AI algorithm for PE diagnosis using CT pulmonary angiography, where the algorithm had 92.7% sensitivity and 95.5% specificity in the retrospective setting (9) and 79.6% sensitivity and 95.0% specificity in clinical practice (10). There were also no significant reductions in report turnaround time or time to treatment. However, a greater impact could be expected for iPE in nonurgent studies.

The study objectives were to assess the prevalence of reported iPE and report the turnaround time and time to treatment for patients with cancer-associated iPE before and after implementing an AI algorithm for detection of iPE.

Materials and Methods

Study Design

This retrospective single-center cross-sectional study was conducted in Halland Hospital Halmstad, Region Halland, Sweden. The study protocol was approved by the Swedish Ethical Review Authority. Informed consent was waived.

Patients

Patients at least 18 years old with a confirmed diagnosis of cancer and at least one contrast-enhanced chest CT performed for cancer evaluation were included between July 1, 2018, and June 30, 2019 (before AI implementation), or between November 1, 2020, and April 30, 2021 (6 months after AI implementation). The prevalence of iPE was previously reported (8) in these patients prior to the implementation of AI. The current study reports on the effect of AI implementation on workflow metrics and the iPE detection rate.

CT Scan Parameters

Studies were performed with a 64-row multidetector CT scanner (Revolution CT; GE HealthCare) with 120 kVp and automatic tube current modulation. Patients received 375 mg of iodine per kilogram for chest CT imaging and 500 mg of iodine per kilogram for chest and abdominal CT imaging up to a maximum dose corresponding to a body weight of 80 kg. The scans were triggered when attenuation in the descending aorta reached 100 HU with a fixed delay of 18 seconds for the chest.

AI Algorithm and Clinical Workflow

The AI algorithm is a commercially available cloud-based solution for iPE and PE detection and triage (Aidoc BriefCase; Aidoc Medical). Details concerning training and validation have been previously published (9). In brief, the AI algorithm was trained and validated on tens of thousands of CT examinations and optimized for study-level classification.

Data upload and AI analysis are fully automated, and results are presented in a widget separate from the picture archiving and communication system (PACS) installed at radiologist and radiographer CT workstations. All positive AI results (suspicion of PE) were immediately evaluated by a radiologist. In cases of a true-positive iPE finding, contact was made by phone with the referring physician, referring department, or the on-call physician for the referring department. All patient communication was managed by the radiographers with support from an attending radiologist.

Image Review

A retrospective review of all reported iPE cases and all AI-positive cases was conducted by the authors, a general radiologist with 9 years of experience (P.W.) and a radiologist with 6 years of experience, including 1 year of subspecialty training in thoracic radiology (K.M.). Images were reviewed in the PACS (Sectra; Sectra AB). iPE was split into four groups: (a) single subsegmental, (b) multiple subsegmental, (c) segmental, and (d) lobar or more proximal iPE. The number of involved segmental and/or subsegmental vessels was estimated, with final characterization of the iPE by consensus.

Clinical Evaluation and Workflow Metrics

Report turnaround time was calculated according to time stamps in the PACS. Time to treatment was calculated according to time stamps in the electronic health record, defined as the time of arrival at the emergency department, time of arrival at an outpatient specialty clinic, or the time of evaluation if managed differently. The AI processing time was defined as the time between the start of the study to when the AI results were presented to the radiologists.

Statistical Analysis

Standard descriptive statistics were used on a per-study basis to compare iPE prevalence and characteristics. The Mann-Whitney U test was used for comparing the median report turnaround time, time to treatment, and number of involved vessels. The χ² test was used for comparing categorical variables. Categorical data are presented as percentages and continuous data as either the means ± 1 SD or medians with IQRs. A P value less than .05 was considered statistically significant. SPSS Statistics for Windows (version 27; IBM) was used for all analyses.

Results

Patient Characteristics

The study included 1892 examinations before AI implementation and 1036 examinations after AI implementation (Fig 1). The mean age was 69.4 years ± 11.4; 50.8% were female patients, and 49.2% were male patients.

Figure 1:

Flowchart of the inclusion process. AI = artificial intelligence, EHR = electronic health record, iPE = incidental pulmonary embolism, PACS = picture archiving and communication system, VTE = venous thromboembolism.

Clinical Evaluation and Workflow Metrics

The reported iPE prevalence was higher after implementation (2.5% [26 of 1036 studies] vs 0.8% [16 of 1892 studies], P < .001). The mean AI processing time was 10.8 minutes. Of the 27 AI-positive cases, 24 were reported as iPE (positive predictive value, 88.9%). We found no evidence of differences in the number of involved vessels in reported iPE (four vs five vessels, P = .18) (Fig 2) or when comparing the most proximal extent of the iPE (P > .05).

Figure 2:

Graphs show the number of involved vessels and most proximal extent of the incidental pulmonary embolism (iPE) cases. (A) Graph shows the distribution of the number of involved vessels. (B) Graph shows the number of patients in each iPE group. AI = artificial intelligence.

Two patients with reported iPE were AI-negative (AI false-negative) (Fig 3). In the retrospective image review, the three AI-positive but report-negative cases (AI false-positive in the original interpretation) were reclassified as AI true-positive (Fig 4).

Figure 3:

Two patients with reported incidental pulmonary embolism (iPE) on standard contrast-enhanced chest CT images that were artificial intelligence–negative. (A) Axial CT image obtained for treatment evaluation shows iPE in the right middle lobe artery and segmental arteries in the right lower lobe in a male patient with colorectal cancer. (B) Coronal CT image obtained for treatment evaluation shows iPE at the division of the right upper lobar artery and interlobar artery in a female patient with renal cancer.

Figure 4:

Contrast-enhanced CT images in three patients with artificial intelligence–detected incidental pulmonary embolisms (iPEs) that were unreported. (A) Axial CT images obtained for treatment evaluation show subsegmental iPE in the right lower lobe in a female patient with lung cancer. (B) Axial preoperative CT images show segmental iPE in the right lower lobe in a female patient with breast cancer. (C) Axial CT images (left and middle) obtained for treatment evaluation show segmental iPE in the left lower lobe in a male patient with lung cancer. Sagittal CT image (right) shows an additional segmental clot in the right middle lobe.

Report turnaround time (median, 0.66 hour vs 24.68 hours, P < .001) and time to treatment (median, 0.98 hour vs 28.05 hours, P < .001) were shorter after AI implementation (Table). After AI implementation, 20 of 24 patients were directly referred from the radiology department to the emergency department or a specialty outpatient clinic, which did not occur for any patients before implementation.

Report Turnaround Time, Time to Treatment, and Patient Management before and after Artificial Intelligence Implementation in Patients with Cancer-associated Incidental Pulmonary Embolism

Discussion

While iPEs are common in patients with cancer, many are unreported, and even in reported cases treatment is often delayed. In the current study, the use of AI for iPE detection and triage was associated with higher reported iPE prevalence (2.5% vs 0.8%, P < .001) and a shorter report turnaround time (median, 0.66 hour vs 24.68 hours, P < .001) and time to treatment (median, 0.98 hour vs 28.05 hours, P < .001).

However, even with AI support, three AI true-positive cases were misclassified as AI false-positives (Fig 4), and two reported iPE cases were not detected by AI (Fig 3), highlighting the need to carefully assess the pulmonary arteries.

There was no difference in the number of involved vessels. A lower embolic burden was anticipated, as a previous retrospective study showed an underreporting of iPE involving only a few vessels (8). In previous studies, patients with cancer-associated subsegmental iPE had a comparable recurrent venous thromboembolism risk compared with patients with more proximal iPE (11,12). While it is recommended that cancer-associated isolated subsegmental iPEs are offered treatment on a case-by-case basis (13), the majority of reported iPEs in the current study were proximal to the subsegmental level. The increased iPE detection rate could potentially mean that fewer patients now risk progression of a missed iPE or a recurrent venous thromboembolism at a later time point.

It is unlikely that a shortened time to treatment from 28 hours to 1 hour is associated with a better prognosis. A previous study showed that a delay of a few days at the start of treatment in cancer-associated iPE did not impact the risk of sudden death (3). However, at least one CT scan had a report turnaround time of 60 days, and in 25% of patients, the delay between report and treatment initiation was 3–18 days, attributed to difficulty in contacting patients and unawareness of the report. An AI algorithm for triaging iPE could ensure that even in periods with long report turnaround times, most patients with iPE would still get a timely evaluation.

A limitation is that the study was conducted at a single center. As cases that were both AI- and report-negative were not rereviewed, the true sensitivity of the AI algorithm could not be evaluated. While the second time period overlaps with the COVID-19 pandemic, it is unlikely that it affected iPE prevalence as included studies were nonurgent studies performed for cancer evaluation, and COVID-19–specific referrals, such as for pulmonary changes after COVID-19, were not included.

In summary, implementation of an AI algorithm for the detection and triage of iPE increased the detection rate of cancer-associated iPE and decreased the report turnaround time. The majority of patients were immediately redirected for further clinical evaluation with a shorter time to treatment. As the study focused on clinical workflow metrics and not long-term outcomes, future research could investigate the effects on morbidity and mortality of an increased detection rate and early diagnosis and treatment of iPE.