REPLY: Leveraging Machine Learning to Generate Prediction Models for Structural Valve Interventions

We thank Dr. Pollari and colleagues for their interest in our study on the application of machine learning (ML) to generate predictive models for in-hospital mortality in patients undergoing transcatheter aortic valve replacement (TAVR) (1). They mention a common clinical scenario in which ML and deep learning could aid in shared decision making: whether TAVR or surgical aortic valve replacement is the best treatment option for an individual patient.

Recent approval of TAVR for low-risk patients by the U.S. Food and Drug Administration has shifted this decision toward TAVR. The PARTNER (Placement of Aortic Transcatheter Valves) 3 and EVOLUT trials have shown that outcomes after TAVR are superior to or as good as those after surgical aortic valve replacement (2,3). With this in mind, current risk prediction models for TAVR-related outcomes are in need of thorough reevaluation given their limited extrapolation to the low-risk patient population. Hopefully, ML will be considered a reasonable tool to either improve prior scores or develop a novel model with superior performance.

Although our results are promising, the clinical application of ML remains unknown for transcatheter valve interventions. There is simply a lack of vali-dated predictive models currently available for clinicians. We can speculate that with proper granular data, ML could replace current mortality prediction models such as the Society of Thoracic Surgeons/American College of Cardiology TVT (Transcatheter Valve Therapy) Registry score. However, we must be wise and recognize that ML will be no more than a complementary tool within the complex clinical armamentarium. In the end, the ultimate decision would still be made by the multidisciplinary heart team.

Unstructured data, such as physician’s notes and diagnostic reports, account for about 80% of patient information, but uploading them into ML algorithms can be difficult. Most ML models rely on administrative or claims data. However, there is a tremendous amount of valuable insight in clinical data (i.e., imaging reports and physicians’ examination notes). With the right data and approach in place, ML can certainly accelerate diagnosis and treatment. For example, a diagnosis of frailty, difficult vascular access, or other coded evidence from a physician assessment may have a confidence score of only 70%. However, accuracy and confidence would be sub-stantially improved if evidence of frailty based on a quantitative score, percentage of arterial stenosis of the iliofemoral vasculature from a computed tomog-raphy report, and/or a physician’s observation of vessel tortuosity or low coronary ostium were added to the equation. The ability to pull this into a ML analysis can dramatically improve accuracy and confidence in the ultimate output.

In contrast, Dr. Baladrón and colleagues raise insightful concerns about the “bad performance” of the artificial neural network, which could have been improved if parameter optimization had been used. Although we agree with the last point, the area under the curve of 0.85 obtained for the artificial neural network model with the Weka default parameters is still very good. We recognize that a statistical comparison of all models’ discrimination using the DeLong test (4) would have helped determine whether statistically significant differences were present among all models. Moreover, we fully agree with the statement of Dr. Baladrón and colleagues that “for ML to reach its full potential, we need to create a new generation of clinical databases, using dynamic and continuous parameters.” ML techniques are for those problems for which solutions must be learned from data, thus artificial neural networks or any other ML models may have the power to show their full potential.

We believe that our study raises awareness of the potential use of artificial intelligence in structural valve interventions. Consequently, we hope to have set the benchmark for this novel methodology in future studies within the field. ML is a powerful tool that can help clinicians discover new clinical associations among patient populations and refine preventive care and treatment protocols. As ML continues to evolve and more robust clinical data emerge, there may be no better option but to embrace ML in our daily clinical practice.