We would like to thank Prof. Varma [1] for his insightful comments on our manuscript. We agree with most of his views which, as a matter of fact, re-enforces the message we have provided in our study.
We are in agreement with his comment that the area under the curve (AUC) of receiver operating characteristic (ROC) curves between 0.8 and 0.9 is considered excellent and that between 0.7 and 0.8 is considered acceptable. This is why we have refrained from using the term “excellent” and stated that both EuroSCORE (AUC 0.71) and STS Scores (AUC 0.72) are “fairly accurate” [2].
With regard to the discriminatory ability, we have very clearly stated in the manuscript that “while the AUC in our study for the STS Score and the EuroSCORE was 0.72 and 0.71 respectively, it was found to be 0.84 for both the scores in the patients undergoing cardiac surgery in the USA” [2]. Thus, we have not left the readers with any ambiguity about these scores being superior in the North American population. However, whether or not this superior discrimination applies to the subgroups as well is not known to us, and as no reference has been cited in this context in the letter, it is not possible to comment on it.
The point made regarding the need for a central registry and population-specific risk assessment tools further echoes the conclusion we have drawn that “creating a database of Indian patients may allow population-specific adjustments and should be aimed for” [2].
With regard to the comment on calibration using the goodness of fit tests, we had highlighted the paradox that exists with the Hosmer–Lemeshow (HL) test in our published manuscript [2]. It is well known that as sample size increases, the power of traditional goodness of fit tests, such as the HL test, increases. However, practically irrelevant discrepancies, between estimated and true probabilities, also increase and are likely to cause the rejection of the hypothesis of a perfect fit in larger samples such as ours [3]. It is precisely due to this paradox that it has been suggested that simpler models one can understand should be preferred, even if they do not fit, rather than the compulsion to obtain fitting models [4].
The simplest model, which all cardiac surgeons are familiar with, and one which is clinically relevant too, is the ratio between observed and predicted mortality. The observed-to-predicted mortality ratio in our study was 0.79 for the EuroSCORE II and 1.25 for the STS score. Compared to this, the reported observed-to-predicted ratio in the North American population was only 0.56 for EuroSCORE II and 0.68 for the STS Score [5]. This confirms that the ability to predict mortality for both these scores was in fact better in our study.
We also agree with Prof Varma’s comments [1] that low-risk patients comprise the majority undergoing coronary artery bypass grafting at most institutions, and for the score to be valid, it must be applicable to this category of patients. Again, we have reported that using the EuroSCORE II criteria for low risk (< 3), 4,373 patients were operated on in our series, with an observed-to-predicted mortality ratio of 0.79 (p = 0.24). Similarly, 4,766 patients were considered to be low risk according to the STS Score (< 4). The observed-to-predicted ratio, in this case, was 1.2 (p = 0.33). The p-values clearly demonstrate that there was no significant difference between the observed-to-predicted mortality predictions in the low-risk patients using either of these scores. Therefore, our conclusion that both EuroSCORE and STS Score are “fairly accurate” remains valid not only in the overall cohort but also in the larger subgroup of low-risk patients.
While statistical arguments and limitations may remain, one has to remember that the main purpose of these risk scores is to predict mortality which is done satisfactorily by both the scores. Until that time, that a central Indian registry comes into existence and a more accurate risk prediction model is available, based on our study, we have no hesitation in reiterating that both the EuroSCORE II and the STS Score predict the mortality risk in the Indian population “fairly accurately”.
Funding
None.
Declarations
Ethics approval
Not applicable.
Informed consent
Not applicable.
Conflict of interest
The authors declare no competing interests.
Footnotes
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Varma PK. Application of EuroSCORE II and STS Score for risk assessment in Indian patients-are they they useful?. 2021. 10.1007/s12055-021-01232-y [DOI] [PMC free article] [PubMed]
- 2.Shales S, Uma Maheswara Rao S, Khapli S, et al. Comparison of European System for Cardiac Operative Risk Evaluation (EuroSCORE) and the Society of Thoracic Surgeons (STS) score for risk prediction in Indian patients undergoing coronary artery bypass grafting. Indian J Thorac Cardiovasc Surg. 2021. 10.1007/s12055-021-01186-1. [DOI] [PMC free article] [PubMed]
- 3.Nattino G, Pennell ML, Lemeshow S. Assessing the goodness of fit of logistic regression models in large samples: a modification of the Hosmer-Lemeshow test. Biometrics. 2020;76:549–560. doi: 10.1111/biom.13249. [DOI] [PubMed] [Google Scholar]
- 4.Royle JA, Chandler RB, Sollmann R, Gardner B. Chapter 3 - GLMs and Bayesian analysis. In: Royle JA, Chandler RB, Sollmann R, Gardner B, editors. Spatial capture-recapture [Internet]. Boston: Academic Press; 2014 [cited 2021 Jun 13]. p. 47–85. Available from: https://www.sciencedirect.com/science/article/pii/B9780124059399000037.
- 5.Ad N, Holmes SD, Patel J, Pritchard G, Shuman DJ, Halpin L. Comparison of EuroSCORE II, original EuroSCORE, and The Society of Thoracic Surgeons Risk Score in cardiac surgery patients. Ann Thorac Surg. 2016;102:573–579. doi: 10.1016/j.athoracsur.2016.01.105. [DOI] [PubMed] [Google Scholar]