Response regarding application of the CHALF Score in pediatric acute liver failure

We thank the group from the Institute of Liver and Biliary Sciences in New Delhi, India for their interest in our research. Since being published in October 2023, our novel predictive tool, the CHLA Acute Liver Failure (CHALF) Score, has generated much interest within the pediatric liver transplant community¹. As indicated in our manuscript, we sought to develop an easily interpreted predictive score of clinical outcomes in pediatric acute liver failure (PALF) exclusively based on common admission lab values to 1) guide early referral/transfer to a liver transplant center for patients who present in the community and 2) inform transplant providers at the time of admission about the urgency to initiate a liver transplant evaluation. The CHALF Score predicts likelihood of survival with native liver versus liver transplant or death, based on data at the time of admission. Many factors, including underlying etiology, clinical trajectory as assessed in real-time by the transplant team, and evolution of multi-organ dysfunction impact the decision to actually proceed with transplant. The CHALF Score was not developed to be used dynamically to predict or inform that decision, although that is something to evaluate moving forward. Ultimately, by rapidly referring patients with higher CHALF Scores to a transplant center, who may then be quickly evaluated for placement on the waitlist, the CHALF Score will likely decrease the chance that patients at risk for rapid clinical deterioration are not yet admitted to a transplant center to receive organ offers at the time when they need them.

Yadav et al state that there are ‘methodological limitations’ to our study. With regards to etiology, it is uncommon to know the underlying etiology of PALF at the time of admission, and thus this variable was not included in our statistical analysis. Indeed, both our discovery CHLA and validation NIH PALF Study Group cohorts reported that up to 50% or more of patients are categorized as ‘indeterminant PALF’^1,2. We agree that further refinement by etiology is warranted and is something we will consider in our ongoing international, multi-center, prospective validation study of the CHALF Score.

With regards to our statistical approach, our lead co-author (SB) is a data scientist with expertise in biostatistics and machine learning techniques. When comparing AUROC between different models, it is standard practice to implement each model, calculate performance metrics, and then choose the model with the best performance. This strategy is well-established and has been used to develop, validate, and compare predictive models of clinical outcomes in liver transplant candidates including MELD 3.0^3,4. In terms of feature selection, we used a bootstrapping technique with 5000 iterations, which is widely used in predictive modeling to rank variables. This technique is agnostic and results in feature importance ranking, as shown in Figure 4a¹. A cut-off of 90% identified three variables (bilirubin, ammonia, albumin). This cut-off means that, in our 5000 iterations, 90% of the models created (i.e. 4500 models) included these three variables in their final, optimized model. In reviewing the ranking, these three variables are present in nearly 100% of the models created. Yadav et al hypothesize that international normalized ratio (INR) may not have been selected in our final model due to the impact of blood product transfusion or bridging therapies like continuous renal replacement. However, our model is based on admission variables and thus it is highly unlikely that these patients received any transfusion or critical care interventions like dialysis prior to labs being drawn at the time of presentation.

Finally, with regards to their own center cohort, several questions remain. They describe a single center population of 391 patients with PALF, which is more than double our high-referral center population accrued over a 20-year period and approaches the total enrollment of the international, multi-center, observational cohort generated in the NIH PALF Study Group. It would be important to know if they are a regional referral center, and if so, how accurately their first set of labs reflect ‘admission’ labs for their population, rather than a set of values that would be impacted by delays in transfer and may represent a later time point in the course of disease. In our study, we were careful to review the charts for the first set of labs, even if they were obtained at an outside facility. Yadav et al do not provide any demographic data for their patients in terms of age, distribution of all PALF etiologies beyond hepatitis A, presenting lab values, placement on the waitlist, or clinical outcomes. Thus, it is difficult to respond to their statistical interpretation of the CHALF Score, particularly when they refer to ‘unnecessary liver transplant’ as the CHALF Score does not indicate whether a patient should undergo transplant, it rather can predict the likelihood of undergoing transplant or dying. Given their high population of patients with hepatitis A, an etiology that was largely absent from both our CHLA and the PALF Study Group populations, we would welcome further investigation and refinement of the CHALF Score by other centers, similar to the natural evolution of the MELD Score over the last two decades⁵.

Abbreviations

PALF

Pediatric acute liver failure

CHALF

CHLA Acute Liver Failure

INR

international normalized ratio

MELD

Model for End-Stage Liver Disease

NIH

National Institutes of Health