Abstract
Introduction: The incidence of venous thromboembolism is estimated to be around 3% of cancer patients. However, a majority of incidental pulmonary embolism (iPE) can be overlooked by radiologists in asymptomatic patients, performing CT scans for disease surveillance, which may significantly impact the patient’s health and management. Routine imaging in oncology is usually reviewed with delayed hours after the acquisition of images. Nevertheless, the advent of AI in radiology could reduce the risk of the diagnostic delay of iPE by an optimal triage immediately at the acquisition console. This study aimed to determine the accuracy rate of an AI algorithm (CINA-iPE) in detecting iPE and the duration until the management of cancer patients in our center, in addition to describing the characteristics of patients with a confirmed pulmonary embolism (PE). Materials and Methods: This is a retrospective analysis of the role of Avicenna’s CE-certified and FDA-cleared CINA-iPE algorithm in oncology patients treated at Gustave Roussy Cancer Campus. The results obtained from the AI algorithm were compared with the attending radiologist’s report and were analyzed by both a radiology resident and a senior radiologist. In case of any discordant results, the reason for this discrepancy was further investigated. The duration between the exact time of the CT scan and analysis was assessed, as well as the duration from the result’s report and the start of active management. Results: Out of 3047 patients, 104 alerts were detected for iPE (prevalence of 1.3%), while 2942 had negative findings. In total, 36 of the 104 patients had confirmed PE, while 68 alerts were false positives. Only one patient reported as negative by the AI tool was deemed to have a PE by the radiologist. The sensitivity and specificity of the AI model were 97.3% and 97.74%, while the PPV and NPV were 34.62% and 99.97%, respectively. Most causes of FP were artifacts (22 cases, 32.3%) and lymph nodes (11 cases, 16.2%). Seven patients experienced delayed diagnosis, requiring them to return to the ER for treatment after being sent home following their scan. The remaining patients received prompt care immediately after their testing, with a mean delay time of 8.13 h. Conclusions: The addition of an AI system for the detection of unsuspected PEs on chest CT scans in routine oncology care demonstrated a promising efficacy in comparison to human performance. Despite a low prevalence, the sensitivity and specificity of the AI tool reached 97.3% and 97.7%, respectively, with detection of all the reported clinical PEs, except one single case. This study describes the potential synergy between AI and radiologists for an optimal diagnosis of iPE in routine clinical cancer care. Clinical relevance statement: In the oncology field, iPEs are common, with an increased risk of morbidity when missed with a delayed diagnosis. With the assistance of a reliable AI tool, the radiologist can focus on the challenging analysis of oncology results while dealing with urgent diagnosis such as PE by sending the patient straight to the ER (Emergency Room) for prompt treatment.
Keywords: artificial intelligence, incidental, pulmonary embolism, oncology, assessment scans
1. Introduction
Cancer-associated pulmonary embolism (PE), constituting a substantial burden with increased cancer-related mortality, is dependent on the primary site of malignancy and the extent of the disease, but also on the characteristics of patients with significant comorbidities [1,2]. At autopsy, up to 12.4% of cancer patients exhibited one or more PE events, thus highlighting the high incidence of PE in this subset of patients [3]. In comparison to non-cancer patients, patients with malignant tumors have a 4–8-fold increase in the risk of mortality following an acute thrombotic event [4,5]. Moreover, the prevalence of venous thromboembolism events, including PE in cancer patients, is 12-fold higher than the general patients, but also more significant when receiving active anti-cancer therapy (23-fold) [6]. Therefore, a prompt diagnosis of PE is warranted in cancer patients to optimize their survival outcomes.
On the other hand, incidental pulmonary embolism (iPE) is commonly diagnosed during routine surveillance imaging, with an incidence varying between 1% and 5% [7]. Nevertheless, iPE is associated with respiratory symptoms that are mistakenly attributed to cancer itself. Therefore, iPE should not be considered a benign situation since it has been shown that the risk of recurrent events or PE-related events does not differ significantly from symptomatic PE. Chest computed tomography (CT) angiography remains the standard of care of imaging for the characterization of suspected PEs [8,9]. Current therapeutic strategies of iPE are similar to symptomatic PE, requiring the rapid instauration of active anticoagulation to prevent serious outcomes [7,10].
With the advent of artificial intelligence (AI) into daily medical activities, there is an unmet need to revolutionize and support the work of radiologists in the rapid diagnosis and management of acute medical conditions [11]. Improvement in the quality and technology of computed tomography (CT) scans has drastically changed the sensitivity in detecting PEs, with better visualization of the pulmonary artery and spatial resolution [12]. It is certain that AI, through the recognition of complex patterns in radiological data and the elaboration of a quantitative assessment of imaging features, would empower radiologists in their quest for enhanced accuracy and efficiency while minimizing the risks of diagnostic errors [13]. With the increasing number of FDA (Food and Drug Administration)-approved AI algorithms, radiologists and AI should become partners, not competitors, to enhance their performance and accuracy [14].
Therefore, numerous retrospective trials tried to assess the accuracy of AI-based models in detecting PEs [15,16,17,18,19]. AI improved the efficacy in the diagnostic process of suspected PEs; consequently, its implementation in daily routine care would enhance the detection of iPEs in populations like cancer patients [20]. In this study, with the aid of an AI-powered tool (CINA-PE Avicenna.AI, Avicenna.AI, La Ciotat, France), we hypothesized that the application of this model could support the radiologists in the diagnosis of iPE in an asymptomatic oncology population undergoing routine clinical CT scans to improve the clinical outcomes and reduce the time to the optimal therapeutic approach.
2. Methods and Materials
This study was approved by the Institutional Review Board of our hospital (no.: 2024-380). The need for written informed consent was waived. Avicenna.AI provided the PE-detection DL-based algorithm for this study. The study received no financial support. The authors had control of the data and the information submitted for publication at all times.
2.1. Study Design
The main objective of this study was to assess the role of AI in the accurate diagnosis of iPEs among cancer patients undergoing routine contrast-enhanced CT scans at Gustave Roussy Cancer Campus. The data were retrospectively and consecutively collected for a time period of 6 months from December 2022 to June 2023. All adult oncology patients who underwent a surveillance CT scan without suspected symptoms of PEs were included.
The patients included were being followed up for stage IV or metastatic neoplasia and had undergone an oncological evaluation CT scan with a thoraco–abdomino–pelvic CT scan with contrast.
Patients who had undergone a thoracic angioscan or thoracoabdominal angioscan and scans performed as part of an emergency or complication were excluded, as were initial assessments of tumors without histology.
Demographic data of patients and their disease, as well as information regarding the management of patients after detection of PEs, were also included.
The date and time of the CT exam (image acquisition) and the date the report was validated by the secretaries and medical assistants after the radiological report by the junior radiologist and correction by the senior radiologist was recorded.
The time and date of the patient’s admission to the emergency department for anticoagulant treatment was also recorded. Figure 1 shows a summary on the management with or without AI of incidentally discovered pulmonary embolism in cancer patients.
2.2. Imaging Feature and Analysis
2.2.1. CT Scan Data
The scanner datasets were provided by a clinical center and were distributed appropriately in terms of suppliers (GE Healthcare GE Optima CT660, GE Healthcare, Milwaukee, WI, USA, which is a 64-slice CT scanner, Siemens Healthineers SOMATOM® Force Dual Source CT system, Erlangen, Germany).
Two specialized radiologists (A.O. and S.A.) reviewed the positive results reported by the AI system, and data were filtered between true PEs and false positives. The specific causes of FPs were also detailed in the results section.
2.2.2. AI System
The iPE deep learning (DL)-based algorithm consists of a series of Unets, a particular type of Convolutional Neural Networks (CNNs) trained to detect specific anatomical structures (refUNET). Unets, known for their effectiveness in handling biomedical images, implement an encoder–decoder framework. The encoder captures features at different levels of abstraction through a series of convolutional layers. The decoder then samples the abstract features to generate voxel-wise segmentation maps. This strategy allows the Unet to capture both high-level contextual information and fine-grained details of medical images, essential for accurate segmentation.
The detection of PEs is achieved by leveraging multiple stages, beginning with the preprocessing of medical images to enhance relevant features and normalize intensity levels. The preprocessed images are then fed into the Unet models, which generate voxel-wise segmentation maps of key anatomical landmarks and potential PE regions.
This DL-based tool was trained on 7630 CT scans and validated on 381 CT scans from 39 different CT scans models. The algorithm achieved a sensitivity of 87.8% (95% CI: 82.2–92.2%) and a specificity of 92.0% (87.3–95.4%).
2.3. Statistical Analysis
On a per patient level, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for the detection of PE were calculated. The 95% confidence intervals (95% CI) were calculated using the exact binomial distribution (Clopper–Pearson) (ref CP). To further investigate the potential impact on clinical workflow, the mean time between the exam and the physician’s interpretation was calculated. The number of patients that were discharged (patients that left the hospital) due to the missing PE diagnostics and then rehospitalized once the diagnosis was established was evaluated. Furthermore, we analyzed the number of patients that needed hospitalization and that initiated active anticoagulation treatment once the PE was detected. All the statistical analyses were performed using MedCalc Statistical Software (v20.015, MedCalc Software Ltd., Ostend, Belgium).
3. Results
A total of 3050 patients were included in the analysis, in whom the prevalence of iPEs was 1.3% (39 patients). The most common primary site of malignancy among patients with iPE was digestive (nine out of thirty-three with available clinical data; 27.27%), followed by thoracic (eight patients; 24.2%) and gynecological neoplasms (five patients; 15.2%) (Table 1).
Table 1.
N | 3050 Patients |
Age (mean ± SD) | 60.86 ± 12.44 |
Sex | |
Male (%) | 65.20% |
Female (%) | 34.80% |
Characteristics of PEs | |
Prevalence of PEs | 1.3% (39 pts) |
Primary Tumors (33 pts) | |
Digestive | 27.7% (9 pts) |
Thoracix | 24.2% (8 pts) |
Gynecological | 15.2% (5 pts) |
Urological | 9.1% (3 pts) |
Head and Neck | 9.1% (3 pts) |
Breast | 9.1% (3 pts) |
Others | 6 pts (2 pts) |
Time between interpretation and exam | |
Mean ± SD | 8.13 ± 15.48 |
95% CI | [3.21–13.05] |
In the first version of the model, out of 3049 evaluated patients, 233 alerts for suspicious iPE were detected, while 2816 tests were found to be negative. Out of the 233 patients, 39 patients had confirmed iPE on their routine CT scans, while 194 patients had false positive (FP) results. Therefore, the sensitivity and specificity of the AI model were 100% (95% CI: 90.97–100%) and 93.55% (95% CI: 92.62–94.41%), respectively, while the positive predictive value (PPV) and negative predictive value (NPV) reached 16.74% and 100%. In the FP analysis, the most common causes were an artifact (80 cases, 41.2%), an FP (36 cases, 18.6%), and lymph nodes (21 cases, 10.8%) (Table 2).
Table 2.
CONFUSION MATRIX Version 1 |
CONFUSION MATRIX Version 2 |
||||||
---|---|---|---|---|---|---|---|
CINA-iPE | CINA-iPE | ||||||
iPE | NOT iPE | ALL | iPE | NOT iPE | ALL | ||
GT | iPE | 39 | 0 | 39 | 36 | 1 | 37 |
NOT iPE | 194 | 2816 | 3010 | 68 | 2942 | 3010 | |
ALL | 233 | 2816 | 3049 | 104 | 2943 | 3047 | |
Sensitivity | 100.0% | 97.3% | |||||
Specifcity | 93.6% | 97.7% | |||||
Accuracy | 93.6% | 97.7% | |||||
PPV | 16.7% | 34.6% | |||||
NPV | 100.0% | 100.0% |
The mean time of delay between the exam and the interpretation by the physician to detect the PE reached 8.13 h, with a standard deviation of 15.48 h (95% CI: 3.21–13.05). Following the test interpretation, there was a delay in the initial management in 5 out of 32 patients (15.6%) (patients left the hospital after the tests). Five patients (15.6%) needed hospitalization following iPE detection for surveillance and instauration of therapeutic anticoagulation, while two patients (6.25%) had serious complications. Active anticoagulation with low molecular weight heparin was initiated in those with newly diagnosed PE (25 patients).
In an effort to improve the performance, the model was revised, using different datasets for training and testing, and a second set of data was performed. Two patients from the first set of patients were not included in the second analysis due to processing issues.
In the second set of analysis, out of 3047 patients, 104 alerts were detected for iPE, while 2942 had negative findings. In total, 36 of the 104 patients had confirmed PE, while 68 alerts were false negatives. In this set of analyses, only one patient reported as negative by CINA-iPE was deemed to have a PE by the radiologist. The sensitivity and specificity of the second version of the AI model were 97.3% (95% CI: 85.84–99.93%) and 97.74% (95% CI: 97.14–98.24%), while the PPV and NPV were 34.62% and 99.97%, respectively. In the FP analysis, the most common causes were an artifact (22 cases, 32.3%), an FP (17 cases, 25%), and lymph nodes (11 cases, 16.2%).
4. Discussion
The purpose of this paper is to assess the value of adding CINA-iPE, an AI system for the detection of unsuspected PEs on chest CT scans, in routine oncology care outside urgent medical conditions. With a reported prevalence of 1.3% of PEs, the sensitivity and specificity reached 97.3% and 97.7%, respectively. The AI tool detected all the reported clinical PEs except one case, while no cases of missed PEs were noted. These data, with two datasets that showed reduced FPs in the second dataset, support the fact that ML is a continuous process that would ultimately lead to an optimal AI system in support of radiologists.
An AI medical tool, to be considered a valuable diagnostic solution, must be fast and cheap with a significant impact on survival outcomes. Earlier data showed that AI tools used for the detection of intracranial hemorrhage (ICH) on non-contrast CT scans can reduce both image wait and turnaround times through ML algorithms, thus potentially improving the diagnosis and management of patients [21]. Therefore, with the knowledge that cancer patients have an increased risk of thrombotic events including PEs, there is an unmet need to optimize the diagnosis and treatment of PEs with the aid of AI. However, the vast majority of the published data assessed the role of AI tools in detecting PEs on CT pulmonary angiography (CTPA) in an emergency setting. In fact, a recent meta-analysis evaluated the actual role of DL in PEs on a total of 36,847 CTPA scans in seven studies, in a retrospective fashion, which led to a pooled sensitivity and specificity of 88% and 86%, respectively. With an acceptable rate of FPs, the DL model’s performance was considered satisfactory, despite the retrospective nature of these works [22]. Moreover, the implementation of a validated commercialized AI algorithm in 1202 patients with suspected PE has led to increased sensitivity (92.6%) and negative predictive value (NPV), thus increasing the self-confidence of radiologists in their final diagnosis [20]. This AI tool detected a number of missed diagnoses in the emergency setting which would have significant better outcomes in cancer patients [20] (Figure 2).
Nevertheless, in the oncology field, iPEs are common, with an increased risk of morbidity when missed with a delayed diagnosis, most particularly in patients undergoing multiple routine surveillance CT scans in both primary and secondary care. In a meta-analysis by Meyer et al., the occurrence of iPE among oncology patients reached 3.36% (963 out of 2826 screened patients), with the highest prevalence among prostate cancer patients followed by hepato-biliairy carcinoma and pancreatic tumors [23]. On the other hand, the sensitivity for PE diagnosis by the radiologists is commonly low, ranging between 67 and 87%, with a high specificity between 89 and 99% [24,25]. In specialized cancer centers, radiologists are commonly overwhelmed by the overflow of CT scans, thus highlighting the need for validated support in their routine daily activities. Several AI companies operating in the healthcare industry have gained FDA (Food and Drug Administration) clearances in various indications including iPEs. AIDOC medical, the only AI model used in the published retrospective trials for iPE detection, gained FDA clearance for triage of iPE in October 2020 [26]. In March 2024, Avicenna.AI, a medical imaging AI company, received clearance from the FDA forCINA-iPE, an AI tool for the detection of iPE on routine CT scans in different health conditions, thus reducing the risk of PE-related morbidity and mortality [27]. To the best of our knowledge, this is the first study evaluating CINA-iPE in the characterization of iPE in an oncology population undergoing routine surveillance imaging.
Several retrospective analyses with distinct populations and CT scan models were designed to address the dilemma of iPE in both the general and oncology populations [15,16,17,18,19] (Table 3). First, Wildman-Tobriner et al. implemented an AI model for reporting iPE in a total of 11,913 CT examinations in all comers, including oncology patients (both chest, abdominal, and pelvic (CAP) CT scans and abdominal and pelvic CT scans), and reported a rate of 0.47% and 0.34% of FNs on CAP scans and AP scans, respectively. Demonstrating a significant agreement with radiologists, this AI tool was capable of detecting 49 missed iPEs (0.41%), thus constituting around 38% of missed PEs by radiologists (49 out of 128 PEs) [18]. It is worth mentioning that assessing iPE in the general population might undermine the value of AI where the risk of thrombotic events is lower in comparison to oncology patients. Moreover, the high risk of missed PEs in this study in comparison to our paper (0%) highlights the difference in the qualifications of radiologists between centers, an issue that was not addressed in these different trials. In a similar fashion, Batra et al. performed a retrospective analysis on 2555 patients, with or without a cancer diagnosis, to detect iPE on conventional CT scans with AI (Briefcase for PE detection by AIDOC). The frequency of iPE reached 1.3% (40 patients), with seven cases only detected by the radiologists while four were only detected by AI, thus highlighting the importance of the synergistic work between physicians and AI. The specificity and sensitivity were estimated at 99.8% and 86.3%, both significantly lower than clinical results, and there were no subgroup differences in terms of Se and Sp despite a small sample size, but there was also no difference in terms of NPV and PPV. Reported false positives were attributed to lymph nodes and pulmonary venous filling defects, while FNs were due to altered anatomy [15]. More recent data from Langius-Wiffen and colleagues demonstrated that 2.2% of acute iPE (67 out of 3089 patients) with routine portal venous contrast-enhanced chest CT scans (37.1% of oncology patients) had an additional 25 missed cases (37.3%), thus increasing the sensitivity of the AI-tool (AIDOC Medical, Tel Aviv, Israel). However, the AI tool had a lower PPV due to a high rate of reported FPs, most commonly related to suboptimal pulmonary artery opacification [17]. We may expect that low PPV would be overcome in the future through fast adaptation and the deep learning of ML models.
Table 3.
Author | AI Model | Number of Patients | Number of CT Scans | Population | iPE Prevalence (%) | Se | Sp | PPV | NPV | Missed PEs (%) |
---|---|---|---|---|---|---|---|---|---|---|
Batra et al (2022) [15] | AIDOC | 2555 | 3003 | All comers | 1.3% | 82.5% | 99.8% | 86.8% | 99.8% | 10% (4 PEs) |
Topff et al. (2023) [16] | AIDOC | 6447 | 11,736 | Cancer pts | 1.3% | 91.6% | 99.7% | 80.9% | 99.9% | 44.8% (47 PEs) |
Langius-Wiffen et al. (2023) [17] | AIDOC | 3089 | 3089 | All comers | 2.2% | 95.5% | 99.6% | 85.3% | 99.9% | 37.3% (25 PEs) |
Wildman-Tobriner et al. (2021) [18] | AIDOC | 4087 (CAP)–4779 (AP) | 11,913 | All comers | 0.66% | 62%/ 61.2% |
99.97%/ 99.98% |
96.1%/ 96.8% |
99.5%/ 99.7% |
38% (49 PEs) |
Wiklund et al. (2022) [19] | AIDOC | 1004 | 1892 | Cancer pts | 4% | 90.7% | 99.8% | 95.6% | 99.6% | 81.5% (53 PEs) |
Our Study (2024) | CINA-IPE | 3049 | 3049 | Cancer pts | 1.3% | 97.3% | 97.74% | 34.62% | 99.97% |
Two other retrospective analyses were conducted on oncology patients. In an analysis by Wiklund et al. on 1069 cancer patients undergoing an elective CT, the prevalence of iPE was 4%, with a sensitivity and specificity reaching 90.7% and 99.8%, respectively, after the application of a DL cloud-based AI algorithm (AIDOC Medical) with a confirmed shorter turnaround time. A total of 59 initially missed PEs were detected, but it should be noted that all included CT images were reviewed by a radiologist, which does not reflect daily routine care [19]. In addition, Topff et al. applied the same AI-model on 6447 oncology patients, which found a prevalence of iPE of 1.2% in the overall population, and achieved a sensitivity and specificity of 91.6% and 99.7%, respectively. Interestingly the missed PE diagnosis dropped significantly from 44.8% to 2.6% with the AI algorithm [16]. The most common causes of FNs were a flow artifact (41.9%) followed by an abnormality adjacent to a pulmonary artery (29%) and technical artifact (22.6%). With a median process time of 3 min, almost all of the studies were available for analysis at the opening of the study, but the number of FPs reached 18.8%. An interesting finding worthy of debate in this trial is the detection of a majority of segmental and subsegmental PEs (83.3%), which can suggest an overdiagnosis of a medical condition with no impact on survival; but, nevertheless, data so far support an active anticoagulation in this setting [10,16]. Also, data analysis and retrieval might constitute a limitation for AI software (AIDOC Medical), with a risk of delaying the reporting of the results (7.3% of non-analyzed exams) [16]. The SAFE-SSPE is a non-inferiority trial testing clinical surveillance in comparison to active anticoagulation in patients with low-risk subsegmental PE, which would help physicians understand the value of reporting these missed PEs by the AI system [28].
Implementation of AI models has tremendous advantages in terms of increased diagnostic accuracy, but also represents several limitations which would implicate the necessity for a leading role for the radiologists to guide its optimal use in the clinical setting. One optimal approach was conducted by Vallee et al., who evaluated the value of adding a DL algorithm to the clinical performance of radiologists in the detection of PEs on CT angiographies, using the already validated AI tool, CINA-iPE. A total of 15.8% of PEs were detected, and the good classification performance was higher with the human–AI combination (p < 0.0001). Interestingly, the sensitivity of the combo increased to 92.5% in the detection of PEs in comparison to 81.7% when physicians worked alone, but also enhanced the diagnosis agreement between residents with better concordance [29]. This finding is a clear message that the current flow of trials should address the harmony of work between radiologists and AI models and not try to compare their performances alone. Therefore, we believe that our paper has several strengths. First, CINA-iPE successfully matched the clinical results reported by the radiologists with optimal agreement between both counterparts. Second, we have evaluated the clinical impact in those suffering from iPEs, which was also reported with the initiation of an optimal anticoagulation and the hospitalization of those suffering from severe consequences. Third, we highlight the wide difference in the prevalence of missed PEs between the different trials, thus confirming the major role of the human factor in the diagnosis of PE. Nevertheless, there are relative limitations to our study, such as a limited generalizability of the outcomes to the general population in the clinical context of cancer patients which should be carefully monitored (high risk of thrombotic events); however, some might consider it to be an advantage for AI for its implementation in the general population to be a matter of debate with the low incidence of PEs. The relatively low PPV translates the need for radiologists to rule out 2/3 of the cases flagged by the AI. We may be concerned by this additional workload for the radiologist, while we may expect an improved PPV through continuous training and optimization of the models. Another common challenge between all trials including ours is that some iPEs might be missed by both physicians and AI, since AI-negative findings are not routinely reviewed by physicians. Moreover, the low prevalence of iPEs might be attributed to the fact that all patients admitted to our hospital are offered prophylactic anticoagulation to reduce the occurrence of this complication in a vulnerable population.
That said, two clinical approaches can be proposed for an optimal alliance between AI and radiologists: first, AI can serve as a second reader to catch those PEs missed by radiologists, while the second strategy is that the radiologists review the red alarms detected by the AI tools to detect additional PEs at the expanse of missing the FNs of the AI system. Nevertheless, both options might pose a serious issue of time-consumption, given the high rate of FPs reported in numerous trials such as our paper. Therefore, a minimization of FPs reported by the AI algorithms is a must to increase their efficiency but also reduce the risk of burnout among physicians. These triage models can be extremely helpful in reducing the time to diagnose and treat PEs among patients undergoing CT scans during weekends, but also exams analyzed by teleradiology, a flourishing medical field mainly during the COVID period, therefore optimizing the quality of care [30]. With the absence of prospective studies in a clinical setting, there will be a need to assess the performance of these AI tools with a direct comparison of two clinical settings, the performance of radiologists with and without the aid of AI. In the near future, radiologists will have the privilege of time, with an emphasis on complex cases, while AI will take care of filtering normal CT scans.
In conclusion, this paper demonstrates the efficacy of CINA-iPE, an AI tool approved for detecting iPEs on chest CT scans in oncology patients with a high sensitivity and specificity, thus reducing the time for reporting positive findings. In the era of AI euphoria, the conception of an AI–radiologist alliance is a must for optimal outcomes in the healthcare field. In routine clinical practice, earlier detection of pulmonary embolism in oncology patients can enhance therapeutic strategies by reducing disease burden and enabling the timely initiation of treatment.
Our results demonstrate the good performance of this AI-based algorithm, opening the way for its use to help radiologists by highlighting the positive exams for PE in the worklist, thereby accelerating the diagnosis and communication workflow.
Author Contributions
Conceptualization, S.A., A.O.C., T.A., S.Q., N.L., A.Z., A.A., A.Z., E.M.M., Y.C., A.F., L.C., F.B., N.L., C.B. and S.M.; methodology, S.A., A.O.C., T.A., S.Q., A.Z. and A.A.; validation, T.A., S.Q. and A.Z.; formal analysis, A.A., A.Z., E.M.M., Y.C., A.F., L.C., F.B., N.L. and C.B.; investigation, A.A., A.Z., E.M.M., Y.C., A.F., L.C., F.B., N.L. and C.B.; resources, A.A., A.Z., E.M.M., Y.C., A.F., L.C., F.B., N.L. and C.B.; data curation, A.A., A.Z., E.M.M., Y.C., A.F., L.C., F.B., N.L. and C.B; writing—original draft preparation, A.O.C., T.A., S.Q., N.L, A.Z. and A.A.; writing—review and editing, A.O.C., T.A., S.Q., N.L, A.Z. and A.A.; visualization, A.O.C., T.A., S.Q., N.L, A.Z. and A.A.; supervision, A.O.C., T.A., S.Q., N.L., A.Z. and A.A.; project administration, S.A., A.O.C., T.A. and S.Q.; funding acquisition, S.A., A.O.C., T.A. and S.Q. All authors have read and agreed to the published version of the manuscript.
Institutional Review Board Statement
The study was conducted according to the guidelines of the Declaration of Helsinki and was approved by the Bioethics Committee of the Medical University of Silesia in Katowice (protocol code PCN/CBN/0022/KB1/68/21 on 15 May 2021 and PCN/CBN/0052/KB1/68/1/21/22 on 29 March 2022).
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.
Conflicts of Interest
Angela Ayobi, Sarah Quenet, Yasmina Chaibi, Angelo Franciosini, Louis Clavel: employees at Avicenna.AI. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Funding Statement
This research received no external funding.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
References
- 1.Hasenberg U., Paul T., Feuersenger A., Goyen M., Kröger K. Cancer Patients and Characteristics of Pulmonary Embolism. Eur. J. Radiol. 2009;69:478–482. doi: 10.1016/j.ejrad.2007.11.022. [DOI] [PubMed] [Google Scholar]
- 2.Kearon C. Natural History of Venous Thromboembolism. Circulation. 2003;107:I-22. doi: 10.1161/01.CIR.0000078464.82671.78. [DOI] [PubMed] [Google Scholar]
- 3.Gimbel I.A., Mulder F.I., Bosch F.T.M., Freund J.E., Guman N., van Es N., Kamphuisen P.W., Büller H.R., Middeldorp S. Pulmonary Embolism at Autopsy in Cancer Patients. J. Thromb. Haemost. 2021;19:1228–1235. doi: 10.1111/jth.15250. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Prandoni P. The Long-Term Clinical Course of Acute Deep Venous Thrombosis. Ann. Intern. Med. 1996;125:1. doi: 10.7326/0003-4819-125-1-199607010-00001. [DOI] [PubMed] [Google Scholar]
- 5.Carson J.L., Kelley M.A., Duff A., Weg J.G., Fulkerson W.J., Palevsky H.I., Schwartz J.S., Thompson B.T., Popovich J., Hobbins T.E., et al. The Clinical Course of Pulmonary Embolism. N. Engl. J. Med. 1992;326:1240–1245. doi: 10.1056/NEJM199205073261902. [DOI] [PubMed] [Google Scholar]
- 6.Mulder F.I., Horvàth-Puhó E., van Es N., Van Laarhoven H.W.M., Pedersen L., Moik F., Ay C., Büller H.R., Sørensen H.T. Venous Thromboembolism in Cancer Patients: A Population-Based Cohort Study. Blood J. Am. Soc. Hematol. 2021;137:1959–1969. doi: 10.1182/blood.2020007338. [DOI] [PubMed] [Google Scholar]
- 7.van Es N., Bleker S.M., Di Nisio M. Cancer-Associated Unsuspected Pulmonary Embolism. Thromb. Res. 2014;133:S172–S178. doi: 10.1016/S0049-3848(14)50028-X. [DOI] [PubMed] [Google Scholar]
- 8.Schoepf U.J., Costello P. CT Angiography for Diagnosis of Pulmonary Embolism: State of the Art. Radiology. 2004;230:329–337. doi: 10.1148/radiol.2302021489. [DOI] [PubMed] [Google Scholar]
- 9.Albrecht M.H., Bickford M.W., Nance J.W., Zhang L., De Cecco C.N., Wichmann J.L., Vogl T.J., Schoepf U.J. State-of-the-Art Pulmonary CT Angiography for Acute Pulmonary Embolism. Am. J. Roentgenol. 2017;208:495–504. doi: 10.2214/AJR.16.17202. [DOI] [PubMed] [Google Scholar]
- 10.Smith S.B., Geske J.B., Maguire J.M., Zane N.A., Carter R.E., Morgenthaler T.I. Early Anticoagulation Is Associated with Reduced Mortality for Acute Pulmonary Embolism. Chest. 2010;137:1382–1390. doi: 10.1378/chest.09-0959. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Barragán-Montero A., Javaid U., Valdés G., Nguyen D., Desbordes P., Macq B., Willems S., Vandewinckele L., Holmström M., Löfman F. Artificial Intelligence and Machine Learning for Medical Imaging: A Technology Review. Phys. Medica. 2021;83:242–256. doi: 10.1016/j.ejmp.2021.04.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Patel S., Kazerooni E.A., Cascade P.N. Pulmonary Embolism: Optimization of Small Pulmonary Artery Visualization at Multi–Detector Row CT. Radiology. 2003;227:455–460. doi: 10.1148/radiol.2272011139. [DOI] [PubMed] [Google Scholar]
- 13.Hosny A., Parmar C., Quackenbush J., Schwartz L.H., Aerts H.J. Artificial Intelligence in Radiology. Nat. Rev. Cancer. 2018;18:500–510. doi: 10.1038/s41568-018-0016-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Cacciamani G.E., Sanford D.I., Chu T.N., Kaneko M., De Castro Abreu A.L., Duddalwar V., Gill I.S. Is Artificial Intelligence Replacing Our Radiology Stars? Not Yet! Eur. Urol. Open Sci. 2023;48:14–16. doi: 10.1016/j.euros.2022.09.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Batra K., Xi Y., Al-Hreish K.M., Kay F.U., Browning T., Baker C., Peshock R.M. Detection of Incidental Pulmonary Embolism on Conventional Contrast-Enhanced Chest CT: Comparison of an Artificial Intelligence Algorithm and Clinical Reports. Am. J. Roentgenol. 2022;219:895–902. doi: 10.2214/AJR.22.27895. [DOI] [PubMed] [Google Scholar]
- 16.Topff L., Ranschaert E.R., Bartels-Rutten A., Negoita A., Menezes R., Beets-Tan R.G.H., Visser J.J. Artificial Intelligence Tool for Detection and Worklist Prioritization Reduces Time to Diagnosis of Incidental Pulmonary Embolism at CT. Radiol. Cardiothorac. Imaging. 2023;5:e220163. doi: 10.1148/ryct.220163. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Langius-Wiffen E., de Jong P.A., Mohamed Hoesein F.A., Dekker L., van den Hoven A.F., Nijholt I.M., Boomsma M.F., Veldhuis W.B. Added Value of an Artificial Intelligence Algorithm in Reducing the Number of Missed Incidental Acute Pulmonary Embolism in Routine Portal Venous Phase Chest CT. Eur. Radiol. 2024;34:367–373. doi: 10.1007/s00330-023-10029-z. [DOI] [PubMed] [Google Scholar]
- 18.Wildman-Tobriner B., Ngo L., Mammarappallil J.G., Konkel B., Johnson J.M., Bashir M.R. Missed Incidental Pulmonary Embolism: Harnessing Artificial Intelligence to Assess Prevalence and Improve Quality Improvement Opportunities. J. Am. Coll. Radiol. 2021;18:992–999. doi: 10.1016/j.jacr.2021.01.014. [DOI] [PubMed] [Google Scholar]
- 19.Wiklund P., Medson K., Elf J. Incidental Pulmonary Embolism in Patients with Cancer: Prevalence, Underdiagnosis and Evaluation of an AI Algorithm for Automatic Detection of Pulmonary Embolism. Eur. Radiol. 2023;33:1185–1193. doi: 10.1007/s00330-022-09071-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Cheikh A.B., Gorincour G., Nivet H., May J., Seux M., Calame P., Thomson V., Delabrousse E., Crombé A. How Artificial Intelligence Improves Radiological Interpretation in Suspected Pulmonary Embolism. Eur. Radiol. 2022;32:5831–5842. doi: 10.1007/s00330-022-08645-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.O’Neill T.J., Xi Y., Stehel E., Browning T., Ng Y.S., Baker C., Peshock R.M. Active Reprioritization of the Reading Worklist Using Artificial Intelligence Has a Beneficial Effect on the Turnaround Time for Interpretation of Head CT with Intracranial Hemorrhage. Radiol. Artif. Intell. 2021;3:e200024. doi: 10.1148/ryai.2020200024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Soffer S., Klang E., Shimon O., Barash Y., Cahan N., Greenspana H., Konen E. Deep Learning for Pulmonary Embolism Detection on Computed Tomography Pulmonary Angiogram: A Systematic Review and Meta-Analysis. Sci. Rep. 2021;11:15814. doi: 10.1038/s41598-021-95249-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Meyer H.-J., Wienke A., Surov A. Incidental Pulmonary Embolism in Oncologic Patients-a Systematic Review and Meta-Analysis. Support. Care Cancer. 2021;29:1293–1302. doi: 10.1007/s00520-020-05601-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Eng J., Krishnan J.A., Segal J.B., Bolger D.T., Tamariz L.J., Streiff M.B., Jenckes M.W., Bass E.B. Accuracy of CT in the Diagnosis of Pulmonary Embolism: A Systematic Literature Review. Am. J. Roentgenol. 2004;183:1819–1827. doi: 10.2214/ajr.183.6.01831819. [DOI] [PubMed] [Google Scholar]
- 25.Kligerman S.J., Mitchell J.W., Sechrist J.W., Meeks A.K., Galvin J.R., White C.S. Radiologist Performance in the Detection of Pulmonary Embolism: Features That Favor Correct Interpretation and Risk Factors for Errors. J. Thorac. Imaging. 2018;33:350–357. doi: 10.1097/RTI.0000000000000361. [DOI] [PubMed] [Google Scholar]
- 26.Aidoc’s 6th FDA Clearance for AI Solution. [(accessed on 1 April 2024)]. Available online: https://www.aidoc.com/about/news/fda-incidental-pulmonary-embolism/
- 27.Avicenna.AI—Avicenna.AI Secures Two FDA Clearances For Stroke Assessment And Opportunistic PE. [(accessed on 20 September 2024)]. Available online: https://avicenna.ai/fda-clearances-aspects-and-ipe/
- 28.Baumgartner C., Klok F.A., Carrier M., Limacher A., Moor J., Righini M., Beer J.-H., Peluso M., Rakovic D., Huisman M.V. Clinical Surveillance vs. Anticoagulation for Low-Risk patiEnts with Isolated SubSegmental Pulmonary Embolism: Protocol for a Multicentre Randomised Placebo-Controlled Non-Inferiority Trial (SAFE-SSPE) BMJ Open. 2020;10:e040151. doi: 10.1136/bmjopen-2020-040151. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Vallée A., Quint R., Brun A.L., Mellot F., Grenier P.A. A Deep Learning-Based Algorithm Improves Radiology Residents’ Diagnoses of Acute Pulmonary Embolism on CT Pulmonary Angiograms. Eur. J. Radiol. 2024;171:111324. doi: 10.1016/j.ejrad.2024.111324. [DOI] [PubMed] [Google Scholar]
- 30.Hanna T.N., Steenburg S.D., Rosenkrantz A.B., Pyatt R.S., Duszak R., Friedberg E.B. Emerging Challenges and Opportunities in the Evolution of Teleradiology. Am. J. Roentgenol. 2020;215:1411–1416. doi: 10.2214/AJR.20.23007. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.