Validation Study of Two Artificial Intelligence–Based Preplanning Methods for Transcatheter Aortic Valve Replacement Procedures

Introduction

Transcatheter aortic valve replacement (TAVR) is a critical procedure for patients with aortic stenosis, requiring precise preoperative planning for optimal outcomes. These measurements can be time-consuming to obtain and may be subject to substantial interindividual variability, especially for inexperienced cardiologists. Two unique and fully automatic artificial intelligence (AI)-based planning methods were used to assess the validity of AI to quantify anatomical characteristics necessary for TAVR patient evaluation. These methods could aid in reducing the time required by an expert in imaging. This study builds upon previous advancements1, 2, 3, 4, 5 by merging the TAVI-PREP¹ algorithm with the Materialise algorithm, focusing on evaluating the combined efficacy and precision of these tools in the context of TAVR planning measurements. It was compared to 2 expert users planning with the standard-practice clinical planning method.

Methods

Two AI algorithms, TAVI-PREP and Materialise, were used to automatically generate measurements on pre-TAVR computed tomography (CT) scans. Both algorithms comprise 3 main components: a segmentation module, a landmark identification module, and a measurement extraction module. For TAVI-PREP,¹ the segmentation module relies on MeshDeformNet⁶ (TensorFlow), the landmark identification module is built on the foundation of the 3D Residual U-Net,⁷ and the measurement extraction algorithm utilizes the outcomes of both preceding algorithms in the sequence to extract measurements. For training, the segmentation module was trained using a dataset consisting of 20 public CT scans and 15 scans from the Montreal Heart Institute (MHI). Meanwhile, the landmark identification algorithm was trained on a dataset comprising 104 scans from MHI. Both datasets were annotated by a single expert cardiologist. Regarding the algorithm developed by Materialise, the segmentation module relies on a 3D U-Net⁸ for initialization, which results in the identification of the heart chambers and main vessels. The blood pool for both sides of the heart is then estimated, and a graph cut technique is employed to split the resulting segmentation mask into the desired structures. Training was performed on 260 CT scans; permission was granted for these scans to be used for research purposes. The landmark detection module relies on the same 3D U-Net and was trained on a dataset composed of 449 CT scans. The data was obtained from multiple European centers, more specifically three Belgian, two Dutch, one German and one British center. All scans were segmented and annotated by multiple engineers trained on cardiac imaging data. Two hundred pre-TAVR CT scans from the MHI were used to validate and compare both algorithms. The average age in this dataset was 79.8 ± 6.4 years for men and 79.2 ± 6.9 years for women, with men comprising 55% of the cohort. Subjects with congenital heart defects and bicuspid aortic valves were excluded from this study. Independently, 2 experts manually indicated the same measurements on the CT images using the 3Mensio Structural Heart software version 10.3 (Pie Medical Imaging BV). Automatic measurements from both algorithms were then compared to the manual measurements to evaluate accuracy.

Results

While the 2 algorithms agreed on most measurements, TAVI-PREP was more precise on perimeter-associated measurements, and Materialise yielded more accurate predictions of sinus measurements. Therefore, the combination of the strengths of each algorithm allowed expert-level accuracy in all measurements, except for the right and left coronary heights (Figure 1). Both coronary height measurements were off by at least 5% compared with the interexpert variability, which is still not adequate for a clinical setting.

Figure 1.

Absolute mean relative error with their corresponding CIs for each measurement are reported. Values are reported as mean (95% CI). The coronary scale is larger due to a lower predictive performance for these values.

Discussion

The comparative analysis revealed that both TAVI-PREP and Materialise algorithms perform within expert-level agreement for most TAVR planning metrics, validating the efficacy of AI in this domain. However, distinct advantages of each algorithm emerged in specific measurement categories. The strength of TAVI-PREP lies in its precision in perimeter-related measurements crucial for selecting appropriate prosthesis size and positioning. The Materialise algorithm, on the other hand, demonstrates higher accuracy in sinus (NCC/RCC/LCC) measurements, essential for valve selection and prequalification. Accurately predicting coronary heights is a persistent challenge for both algorithms, yet this remains a critical aspect of TAVR planning to ensure the prevention of coronary obstruction. The difficulty in precisely computing coronary heights arises from 3 main sources of error. First, the plane defined by the 3 approaches (3Mensio, TAVI-PREP, and Materialise) does not define the aortic plane exactly at the same exact location and angulation. Second, there is also a difference in all 3 algorithms in identifying the coronary ostia position. Third, there is an accumulation of errors caused by the aortic plane definition and the ostia position when the measurement is derived from projecting the ostia on the annulus plane. The synergy of these algorithms presents a novel approach in TAVR planning. By combining the TAVI-PREP and Materialise algorithms, a combined tool can be developed for identifying key anatomical measurements on pre-TAVR CT scans. This tool aims to leverage the advantages of both methods to ensure comprehensive and accurate preoperative planning. Future research will focus on integrating a third expert to enhance the accuracy of interindividual variability assessments, including a second center, refining the coronary height detection algorithm, and developing a new algorithm for femoral and alternative access, aiming to comprehensively cover the entire pre-TAVR workflow and to further validate the algorithms. Additionally, with the comprehensive nature of this enhanced algorithm, it can also facilitate making suggestions for valve selection.

Conclusion

This study showcased expert-level agreement for most TAVR planning metrics, thus validating the efficacy of AI in this domain. This showcases the potential AI can have to support clinical decision making, by eliminating the mundane effort required to identify anatomical measures. This in-depth analysis of combining 2 leading AI algorithms in TAVR planning showcases the strengths and limitations of each. The integration of the TAVI-PREP and Materialise algorithms can potentially set a new standard in automated TAVR planning, offering a more accurate, efficient, and personalized approach. The algorithmic solutions proposed here have the potential to provide more objective and uniform measurements and facilitate the preoperative planning process. As the field progresses toward AI-augmented medical practices, such collaborations will be pivotal in enhancing patient care and procedural success.