Prediction model’s establishment and validation for aortic dissection: a long way to go

We are very grateful to hear from our prestigious colleague Dr. Wan commenting on our study entitled “Predicting in-hospital rupture of type A aortic dissection using Random Forest” published in Journal of Thoracic Disease” (1). Dr. Wan vividly termed type A aortic dissection (TAAD) as a “ticking time-bomb” and kindly appraised what we have done in the aforementioned study as “decode a ticking time-bomb”. Undoubtedly, Dr. Wan himself is a well-known “bomb disposal expert”, so that he was able to dissect our study in a profound way, both clinically and statistically.

Human beings are eager to predict the future, so as to grasp the future and obtain some psychological stability. This human nature is fully reflected in astrology, tarot cards, crystal balls, and other ancient and enduring prediction games and legends. Nowadays, the prediction model based on growingly powerful statistics has gained more and more attention and popularity. As medicine evolved from empirical medicine to evidence-based medicine, from evidence-based medicine to precision medicine, the value of data has received unprecedented attention. As a quantitative tool of risk and benefit assessment, the clinical prediction model can provide more intuitive and rational information for doctors, patients, and medical policymakers.

For aortic dissection, the International Registry of Acute Aortic Dissection (IRAD) study has proposed an in-hospital mortality prediction model for acute TAAD to assist clinicians for optimal treatment (2). We have also proposed a nomogram model based on logistic regression to predict the small dissections previously (3). Altogether, the prediction model researches in the field of the aortic disease are increasing, with some challenges emerging at the same time.

First and foremost, we need to standardize and strengthen the methodology of prediction model research. A prediction model should undergo both internal and external validations. In the above examples, the IRAD study has only done an internal verification, that is, to develop and validate the model with the same batch of data. However, one cannot play the athlete and referee role at the same time. A solid verification is particularly important before the model is applied clinically. In our research, we divide the development set and validation set according to the proportion of 70%:30%. Dr. Wan rightly pointed out that “A logical next step would be to externally validate this model using prospective multicenter data from an even larger sample size”. External validation is essential because of the potential heterogeneity between the model development population and the application population. The case presented by Dr. Wan (4) shows that the four most important variables identified by us were absent. Of course, we cannot overthrow a model according to a single special case, but this example appropriately reminds us of the importance of external verification.

Second, Dr. Wan stated that the Fuwai cohort was somewhat unique. Chinese TAAD patients indeed demonstrate some uniqueness, just as Sino-RAD reported (5). There are two important implications for us: (I) prediction models based on different groups of the population need to be thoroughly and extensively validated before they are adopted; (II) compared with our Western colleagues, we have enough data to develop, and we should develop our specific models given the different TAAD characteristics. Unfortunately, we are a little behind this goal. In our study, we aimed to resolve a unique problem that may only apply to Chinese TAAD patients: how to select the most urgent TAAD surgical candidates. The incidence rate of aortic dissection in China seems to be much higher than that in Western countries, which leads to greater emergency operation pressure for us, especially in the winter, and there are few specialized aortic centers in China. Therefore, we often face multiple TAAD patients, but we cannot arrange timely surgery for each of them where routine operations have occupied lots of clinical resources (over 14,000 cardiac operations annually). Dr. Wan further asked “can this random forest classification model help us safely delay an operation for several hours and still ensure improved survival for some selected patients with type A aortic dissection?” This is an interesting question, necessitating more research in the future. We have to point out that we are forced to triage patients. Ideally, if conditions permit (sufficient medical resources, patients’ willingness, and patients’ tolerance), we still recommend an emergency operation for all TAAD patients, without any delay.

Third, as Dr. Wan rightly pointed out, “Compared with other classification algorithms, the random forest algorithm has a high prediction accuracy and can process high-dimensional data and a large number of training samples.” Traditionally, we usually choose logistic regression to build the prediction model, just as the IRAD study did. Our research adopted the random forest model, a machine learning approach. Simply speaking, random forest, as the name implies, is to build a forest randomly. There are many independent decision trees in the forest. It can predict the effect of up to thousands of explanatory variables. Admittedly, as Dr. Wan pointed out, “a relatively high number of variables are collected from a single patient to feed in this model.” We need to balance the accuracy, applicability, and simplicity of the model. In addition, we agree with Dr. Wan that “even if all existing risk score systems have been proven to be extremely valuable, they are meant to be complementary and can never completely replace a physicians’ clinical judgment.” It is especially true for aortic dissection, a dangerous and variable disease. Of note, the variables used to establish a scoring system are based on the physicians’ clinical judgment. In turn, the scoring system can help to verify and improve a physicians’ clinical judgment. The upgraded physicians’ clinical judgment can further promote the scoring system. They are mutually evolving and complementary, not competing.

In general, although many teams, including ourselves, have been trying to predict some aspects of aortic dissection with certain achievements, we still have a long way to go.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.