Selecting the Most Robust Imipenem Pharmacokinetic Model Requires Paired Forecasts and Explicit Metric Weighting [Response to Letter]

Dear editor

We sincerely thank Dr. Xiaoxin Zhang for the constructive comments on our published article, “Toward Model-Informed Precision Dosing of Imipenem: Multicenter External Validation of Population Pharmacokinetic Models in Adult Critically Ill Patients”.1,2 The concerns raised by Dr. Zhang reflect core methodological issues in the field of external validation of population pharmacokinetic (popPK) models, including the weighting of model selection metrics, the comparability between a priori and Bayesian predictions, and the necessity of paired analysis. These points provide valuable insights for conducting systematic external validation of popPK models. We would like to share our aspects.

First of all, it should be noted that all of the tested popPK models for imipenem had unsatisfactory performance. The a priori prediction-based diagonistic method resulted in a large prediction error, and Bayesian estimation improved performance, but most F50s were approximately 50%. Simulation-based diagnosis revealed systemic bias across all the models. Thus, selecting the best model seems to involve catching a large fish on a small pod. We have previously reviewed the available imipenem models.3 The unsatisfactory external validation performance may be attributed to several reasons. Critically ill patients are a group of patients with high heterogeneity. Although the tested popPK models were developed for critically ill patients, the focused subpopulation included generally critically ill patients, pneumonia patients, deceased renal function patients and some other types of patients. Interindividual variability in imipenem PK is high for these patients. Current popPK models attempt to explain the variability using some covariates (often renal function and body weight), but the residual error is large. When selecting the appropriate model, a detailed comparison of target patients and modeling patients should be performed. Local validation and re-estimation of parameters using a small set are recommended. At the current stage, the integration of the popPK model and therapeutic drug monitoring may be the only feasible way to achieve optimized dosing of imipenem.

In terms of model comparison, Dr. Zhang noted that the Bai et al (2024) model was superior to the Truong et al (2025) model across the four reported Bayesian prediction metrics. This highlights a long-standing methodological challenge in the external validation of popPK models, that is, the handling of inconsistent rankings by multiple diagnostic dimensions. In our original study, the Truong et al (2025) model was designated “most robust” on the basis of a synthesis of three aspects: a priori predictive performance, Bayesian predictive performance, and simulation-based diagnostics (NPDE). Without a prespecified and explicit weighting rule, such a comprehensive judgment is inherently subjective. As demonstrated by recent external validation studies of vancomycin and polymyxin B popPK models, different models often exhibit heterogeneous performance across metrics of bias, precision, F20/F50, and NPDE and may even present a “trade-off” pattern in which a model performs better on one metric and worse on another.4,5 The aim of external validation was to identify which model was suitable for application to a specific population, not to rank models. The selection of a model should also consider the purpose of the application model, not only the prediction accuracy. Prediction-based and simulation-based diagnostics address fundamentally different questions. Prediction-based diagnostics focus on the accuracy of individual concentration predictions, whereas simulation-based diagnostics evaluate the ability of the model to characterize the data-generating process. They do not necessarily yield consistent model rankings.6 We have illustrated all the indicators, and the readers can select the models according to specific objectives.

Another important concern is the comparability between a priori and Bayesian predictions. In our study, a priori predictions were based on all 201 concentrations, whereas Bayesian predictions incorporated only 45 concentrations. This mismatch in sample size means that the difference between a priori and Bayesian summary metrics reflects not only the information gain from Bayesian updating but also the difference in the evaluation samples themselves. Dr. Zhang recommended that for each concentration following the first TDM sample, both a priori and Bayesian predictions should be generated simultaneously, and paired prediction errors should be used to isolate the true incremental value of Bayesian updating for the same observation target, with confidence intervals estimated using patient-level bootstrap methods. This recommendation is methodologically sound. We acknowledge that the original study did not adopt such a paired design, mainly owing to the practical constraints of retrospective ICU data: the timing and frequency of TDM sampling were sparse, and the number of available concentrations was small and unevenly distributed (only three patients had three TDM samples). However, the current method is acceptable and has been applied in many external studies.7,8

Furthermore, the patient-level bootstrap intervals mentioned by Dr. Zhang are particularly important for handling the hierarchical structure of ICU patient data. Because multiple concentration observations from the same patient exhibit strong within-individual correlations, conventional observation-level confidence interval estimation underestimates the standard error, leading to overly confident inference. The patient-level bootstrap method preserves the clustering structure of the data by resampling at the patient level and therefore provides more accurate statistical inference. However, few studies have used patient-level bootstrap resampling to construct confidence intervals for predictive performance metrics in external validation. Patient-level bootstrap resampling should be incorporated into future external validation study protocols to systematically evaluate the confidence intervals of prediction errors.

In summary, Dr. Zhang provides valuable comments that could benefit future studies about the external validation of popPK models, and our response could help readers to better understand the work.

Funding Statement

The author received no funding for this work.

Disclosure

The authors report no conflicts of interest in this communication.