A machine learning approach to predict daptomycin exposure from two concentrations based on Monte Carlo simulations

ABSTRACT

Daptomycin is a concentration-dependent lipopeptide antibiotic for which exposure/effect relationships have been shown. Machine learning (ML) algorithms, developed to predict the individual exposure to drugs, have shown very good performances in comparison to maximum a posteriori Bayesian estimation (MAP-BE). The aim of this work was to predict the area under the blood concentration curve (AUC) of daptomycin from two samples and a few covariates using XGBoost ML algorithm trained on Monte Carlo simulations. Five thousand one hundred fifty patients were simulated from two literature population pharmacokinetics models. Data from the first model were split into a training set (75%) and a testing set (25%). Four ML algorithms were built to learn AUC based on daptomycin blood concentration samples at pre-dose and 1 h post-dose. The XGBoost model (best ML algorithm) with the lowest root mean square error (RMSE) in a 10-fold cross-validation experiment was evaluated in both the test set and the simulations from the second population pharmacokinetic model (validation). The ML model based on the two concentrations, the differences between these concentrations, and five other covariates (sex, weight, daptomycin dose, creatinine clearance, and body temperature) yielded very good AUC estimation in the test (relative bias/RMSE = 0.43/7.69%) and validation sets (relative bias/RMSE = 4.61/6.63%). The XGBoost ML model developed allowed accurate estimation of daptomycin AUC using C0, C1h, and a few covariates and could be used for exposure estimation and dose adjustment. This ML approach can facilitate the conduct of future therapeutic drug monitoring (TDM) studies.

INTRODUCTION

Daptomycin is a concentration-dependent lipopeptide antibiotic with bactericidal activity. It can be used against staphylococci, streptococci, and enterococci including resistant bacteria such as methicillin-resistant Staphylococcus aureus(MRSA) and vancomycin-resistant enterococci (VRE) (1–3). It causes bacterial membrane function disruption and inhibits protein, DNA, and RNA synthesis. According to the summary of product characteristics, daptomycin can be used to treat children and adults for complicated skin and soft tissue infections, endocarditis of the right heart, and in Staphylococcus aureus bacteremia. However, a wider routine use against Gram-positive infections is common (4, 5) particularly in bone and joint infections (6), with the exception of lung infections (7).

Due to its lack of endothelial cell toxicity and nephrotoxicity, the fact that it requires only 30 min of intravenous infusion a day, and with a prolonged post-antibiotic effect of between 6 and 8 h (3), daptomycin is used more and more frequently in place of other Gram-positive intravenous antibiotics such as vancomycin or betalactamins. This drug is approved by the Food and Drug Administration (FDA) and the European Medicines Agency (EMA), and the French guidelines propose a dose of 8 to 12 mg/kg/d (validated by two French-speaking learned societies, the “Société de Pathologie Infectieuse de Langue Française” [SPILF] and the “Société Française de Pharmacologie et de Thérapeutique” [SFPT]). An important adverse effect caused by daptomycin administration is rhabdomyolysis, which may be asymptomatic when it starts (8, 9). Weekly creatine phosphokinase (CPK) monitoring is therefore required, and because daptomycin is primarily cleared by the kidneys, dose adjustment is necessary when creatinine clearance is <30 mL/min (1, 10).

More and more studies tend to emphasize the importance of pharmacological monitoring of daptomycin blood concentrations. Indeed, the area under the blood concentration curve (AUC) divided by the minimal inhibitory concentration (MIC) (AUC/MIC ratio) has been shown, as the pharmacokinetics/pharmacodynamics (PK/PD) index, to be associated with daptomycin efficacy (11). Some studies have shown an increased mortality for S. aureus infections in the case of AUC/MIC <666 (12, 13). An increased risk of myotoxicity associated with CPK elevation has been observed when trough concentration (C0) is >24.3 mg/L (9) and, very recently, for AUC >939 mg.h/L (14). Our team has already shown that machine learning (ML) algorithms based on Monte Carlo simulations from a population pharmacokinetic (POPPK) model allowed good predictions of tacrolimus AUC (with a decrease in the number of samples required in comparison to maximum a posteriori Bayesian estimation) or vancomycin AUC (15, 16). Recently, Tuloup et al. (17) proposed a Bayesian estimator for daptomycin AUC based on a two-sample strategy (time T0 and T6h post-dose) built from 77 patients (mean prediction bias of 0.13 mg·h/L and absolute imprecision of 3%) or C0 and C1 (mean bias of −10 mg·h/L and imprecision of 3%). However, the authors did not use an independent data set to evaluate the performances, leading to a risk of overfitting, and T6h is not easy to draw in outpatients.

The aim of this study was to develop and compare ML models to estimate the AUC of daptomycin based on two blood concentrations that are easy to collect in routine practice (at time T0 and T1) and a few demographic predictors.

RESULTS

Patients and data

A total of 4,740 AUC out of the 4,950 simulated patients were used to train the ML algorithms. Characteristics of the training and testing sets are reported in Table 1. The area under the curve at steady state obtained by dose/simulated clearance (AUC0-24ss) ranged between 208.9 and 3,087.0 mg.h/L for daptomycin qd.

TABLE 1.

Characteristics of training, testing, and validation populations^a

	Daptomycin
	Train (n = 3,552)	Test (n = 1,188)	Validation (n = 194)
Patient sex, male (%)	2,053 (57.80)	705 (59.34)	114 (58.76)
Patient weight, kg	85.68 [68.74–106.12]	85.12 [69.23–103.92]	80.06 [68.99–94.78]
Patient renal function (C&G), mL/min/1.73 m2	87.48 [64.68–107.37]	87.70 [64.12–107.53]	103.30 [89.01–115.51]
Body temperature, °C	37.61 [36.90–38.37]	37.56 [36.92–38.34]	37.20 [37.20–37.20]
Daptomycin dose (mg)	661.00 [473.00–882.40]	655.30 [465.3–895.90]	720.00 [600.00–840.00]
Trough level (C0), mg/L	11.36 [4.87–23.45]	11.22 [4.70–22.49]	24.74 [19.19–31.72]
Concentration at 1 h (C1), mg/L	92.38 [65.12–128.36]	91.24 [64.83–132.93]	93.90 [81.25–108.18]
Difference between C1 and C0 concentrations, mg/L	76.68 [53.89–106.98]	76.18 [51.82–110.74]	69.09 [59.29–77.32]
Reference AUC0-24ss, mg.h/L	866.30 [564.90–1,318.80]	866.70 [566.10–1,318.60]	1140.30 [935.30–1,393.10]

^a Median [IQR] is presented for continuous data and n (%) for categorical data. C&G, Cockcroft & Gault; AUC0-24ss, area under the curve at steady state obtained by dose/simulated clearance; IQR, interquartile range.

The correlation matrix between features and AUC0-24ss is presented in Fig. 1.

Fig 1.

Correlation matrix and scatterplots between features and AUC0-24ss. WT, patient body weight; SEX, patient sex; CREATCL, creatinine clearance using Cockcroft & Gault formula; TEMP, body temperature; dose, daptomycin dose administrated; conc_C0, trough concentration at steady state (T0); conc_C1, concentration at 1h after drug administration at steady state (T1); Diff_C1C0, difference between C1 and C0 concentrations.* corresponds to p<0.05, ** to p<0.01 and *** to p<0.001.

ML model

The best-tuned parameter values for each ML algorithm are presented in Table S1 in the section “complementary data” in Cyrielle Codde Github (https://github.com/CyrielleCodde/Daptomycin-AUC-prediction-from-2-samples). The results obtained in the training set after 10-fold cross-validation for the four ML algorithms and in the testing set for the XGBoost algorithm (best algorithm in the training set) are presented in Table 2. They show excellent results with no difference (no overfitting). The relative mean prediction errors (MPEs) were close to zero, and the relative RMSEs were <8% in both the training and the testing sets. The scatterplots obtained in the training and testing sets for XGBoost are presented in Fig. 2. The variable importance plot in the training set for XGBoost is presented in Fig. 3 and shows that trough concentration (C0), concentration at 1 h after drug intake (C1), and dose were the most important variables. In case of infections due to S. aureus, the proportion of good prediction for the AUC between 666 and 939 mg.h/L in the testing set showed a sensitivity (Se) of 80.3%, a specificity (Sp) of 94.8%, a positive predictive value (PPV) of 80.3%, and a negative predictive value (NPV) of 94.7% (Table 3). Results investigating independently the proportion of good prediction for AUC >666 mg.h/L (efficiency) or for AUC <939 mg.h/L (no toxicity) exhibited excellent performances for both with a value >0.9 for all the metrics (available in Table S2 in the section “complementary data” at https://github.com/CyrielleCodde/Daptomycin-AUC-prediction-from-2-samples).

TABLE 2.

Performances of the models in the training, testing, and validation data sets to estimate daptomycin AUCs obtained from two samples

	Daptomycin qd two samples
	Training_ML	Testing_ML	Validating_ML	Testing_MAP-BEc	Validating_MAP-BEc
RMSE, mg.h/La	77.50 ± 1.89b	79.60	58.28	92.0	182
R2a	0.985	0.984	0.963	0.977	0.721
Relative MPE, %	0.64	0.43	4.61	−0.94	3.84
Relative RMSE, %	7.83	7.69	6.63	7.29	15.9
Number of MPE of the ±20% interval n	85 (2.4 %)	26 (2.2 %)	0 (0.0 %)	15 (1.3 %)	36 (18.8 %)

^a Values obtained after 10-fold cross-validation.

^b SDs obtained after 10-fold cross-validation.

^c MAP-BE, maximum a posteriori Bayesian estimation.

Fig 2.

Scatterplots of ML predicted vs reference daptomycin AUCs in the training (A) and testing (B) sets using two blood concentrations. Sex: 0 = female, 1 = male.

Fig 3.

Importance plot of ML for daptomycin AUC predictions using two samples. conc_C0, trough concentration at steady state (T0); conc_C1, concentration at 1 h after drug administration at steady state (T1); CREATCL, creatinine clearance using Cockcroft & Gault formula; Diff_C1C0, difference between C1 and C0 concentrations; dose, daptomycin dose administrated; SEX, patient sex; TEMP, body temperature; WT, patient body weight.

TABLE 3.

Contingency table of categorized predictions from the testing set and external data set using XGBoost daptomycin AUC prediction model, applicable in the case of an S. aureus infection for a MIC at a breakpoint of 1 mg/L^a

	Reference AUC in 666–939 mg.h/L	Reference AUC out of 666–939 mg.h/L
Testing set (n = 1,188)
Predicted AUC in 666–939 mg.h/L	200	49
Predicted AUC out of 666–939 mg.h/L	49	890
External set (n = 194)
Predicted AUC in 666–939 mg.h/L	33	5
Predicted AUC out of 666–939 mg.h/L	11	145

^a AUC, area under the curve.

External evaluation vs the trapezoidal AUC

The characteristics of the simulated patients from the Garreau et al. model used for external validation are presented in Table 1. Overall, they showed a higher dose, C0, and AUC0-24ss, in comparison to the simulations of the POPPK model we used to train the ML model. Their performances are presented in Table 2 and yielded good results, with a bias close to 5% and an imprecision of about 6%. The scatterplot of estimated vs reference daptomycin AUCs split by sex can be seen in Fig. 4. We also investigated the proportion of good prediction in case of infections due to Staphylococcus for AUC between 666 and 939 mg.h/L in the validation set and showed an Se = 75.0%, Sp = 96.7 %, PPV = 86.8%, and PNV = 92.9% (Table 3). Results showing independently the proportion of good prediction for AUC >666 mg.h/L or for AUC <939 mg.h/L also exhibited excellent prediction except for the prediction of AUC <666 mg.h/L with only one out of six values predicted correctly (available in Table S2 in the section “complementary data” in this link: https://github.com/CyrielleCodde/Daptomycin-AUC-prediction-from-2-samples).

Fig 4.

Scatterplot of XGBoost ML vs reference daptomycin AUCs in the external validation data set using two blood concentrations. Sex: 0 = female, 1 = male.

Comparison to maximum a posteriori Bayesian estimation

The results of the MAP-BE are presented in Table 2 and showed overall numerically better performances for the ML vs MAP-BE in the testing set and in the external simulations from the Garreau et al. model.

DISCUSSION

In this work, we trained ML models to estimate the AUC0-24ss of daptomycin obtained from Monte Carlo simulations using two samples at C0 and C1h, the difference between C1 and C0, and five other covariates. As observed in some of our previous works in the literature, the XGBoost outperformed the linear regression, linear SVM, or random forest in drug exposure prediction (18–20). A possible explanation is that XGBoost is less sensitive to correlation between features, as it randomly selects a subset of them at each split. This allows XGBoost to effectively handle features that are correlated, such as weight, sex, and creatinine clearance (calculated with Cockcroft-Gault), which were all included in the same model. The covariate selection was not a priori performed in this study because all simulated covariates were known to have a relationship with PK parameters in the original models (no other independent covariates were simulated). XGBoost selects covariates based on their gain, which quantifies the effectiveness of a split in a given feature to reduce the model’s loss across all trees in the model. This prioritizes covariates associated with higher gains for their potential to significantly enhance model accuracy. Additionally, XGBoost incorporates regularization that penalizes model complexity to prevent overfitting and to improve generalizability. This balanced approach, combining gain-based selection with regularization, allows XGBoost to effectively identify and include informative covariates. An example is provided in the github of the article (in the file “Example of gain calculation in the XGBoost model”).

The performances in the training and testing set were excellent with a bias close to zero and an imprecision of <8%. This is in accordance with previous studies performed by our group and other teams, showing that ML approaches led to accurate or more accurate performances in terms of individual estimation of exposure indices than POPPK and MAP-BE (16, 21, 22). We compared the performances to the MAP-BE obtained with the Dvorchik model in the testing set and in the external simulations, and we obtained numerically better performances metrics. In addition, in terms of MPE and RMSE, our ML results are comparable to the ones obtained by Tuloup et al., who used a Bayesian approach to estimate AUCs from two points or only one (T0-T1, T0-T6, and T6 strategies) (17), but in their study, they developed and evaluated their limited sampling strategy in the same patients, leading to a risk of overfitting.

As no individual full PK profiles were available for an external validation, we decided to generate new patients from another independent POPPK model developed for this drug by another group. The patients obtained from the second model were simulated with different doses (corresponding to the ones used in routine practice in our center) and exposure indices, and the performances exhibited a higher bias but with a lower imprecision. The lower imprecision is likely attributed to the residual error being set to zero, which has been demonstrated in our previous study to enhance performance in real external patients (15, 23). In other words, the ML algorithm constructed from Dvorchik simulations slightly overestimates the AUC from Garreau simulations. However, this slight decrease in performance was anticipated because the Dvorchik model exhibits a weaker sex effect on clearance compared to the Garreau model, and the Garreau model uniquely reports a sex effect on central volume of distribution. Nevertheless, this mirrors a true external data set and demonstrates the robustness and generalizability of our algorithm’s results.

Daptomycin is particularly used in the context of S. aureus infections with an AUC efficacy cutoff established at 666 mg/L for a breakpoint MIC of 1 mg/L. Thus, we explored error in predictions (bias) in comparison to the critical values used as thresholds (AUC >666 mg.h/L and AUC <939 mg.h/L, respectively). For example, true AUCs simulated using the Garreau model of 666 and 939 were predicted at 729.8 and 994 mg.h/L respectively using our XGBoost model. These overestimations are quite small with a probably limited clinical impact. By applying our ML to AUC threshold values between 496 and 1,179 mg.h/L according to the Dvorchik publication for doses from 4 to 8 mg/kg, we obtained predictions of 386 and 1,179 mg.h/L, respectively (24). It can be assumed that a difference of more than 100 mg.h/L between the prediction and the real value could have a real impact in clinical practice. Nevertheless, the real clinical implications of this bias have to be evaluated in “real” patients.

The XGBoost algorithm is not a simple formula, and to improve its use, we developed a shiny interface allowing real-time calculation of AUC based on different features. The application available in https://ccodde.shinyapps.io/Daptomycin_AUC/ is for demonstration only and has not yet been validated in real patients.

The relevance of developing an ML algorithm when population pharmacokinetic models are available to perform Bayesian estimation is questionable. As stated above, we observed numerically better performances for the XGBoost ML vs Bayesian estimation. A possible explanation is related to the ability of XGBoost to manage complex, nonlinear relationships within the data, even if it was trained in a simulated data. Furthermore, due to its ensemble learning nature, XGBoost exhibits a heightened robustness against overfitting.

Therapeutic drug monitoring (TDM) of daptomycin is currently not widespread, but its interest, based on AUC monitoring, has already been described for patients treated with high doses to monitor the risk of toxicity and the efficacy (25). It could also benefit some population at risk of disturbed PK, such as patients with decreased renal function, obese patients, or geriatric patients.

For example, analysis of daptomycin PK in geriatric population has been studied, and exposure (AUC) was 58% higher in subjects 75 yr of age compared with younger subjects (26).

Regarding obese patients, when considering non-weight normalized parameters, the volume of distribution and the daptomycin’s elimination clearance have been shown to increase with body weight. However, after administration of a weight-proportional dose of daptomycin, exposure (AUC, Cmax, Cmin) was 25%–93% higher in obese subjects compared with non-obese subjects (27–29). There are applications to help prescribe antibiotics in the obese patient population; in France, an application is particularly used and was designed by a multidisciplinary group of doctors with the scientific partnership of SPILF (https://abxbmi.com/). The objective is to provide clinicians with the basis for recommendations based on a review of the scientific literature for an appropriate dose of antibiotic. Concerning daptomycin, three publications are at the origin of these recommendations (30–32).

According to a recent systematic review by Cairns et al. (33), no study has investigated the relevance of routine daptomycin TDM in comparison to fixed dosing regimens in adults with Gram-positive infection. This knowledge gap necessitates studies on daptomycin TDM early in routine practice, to establish clear and concise guidelines. In addition, model-informed precision dosing (MIPD) is not routinely performed because the collection of samples required to estimate AUC can be difficult, expensive, and rather uncomfortable for the patients. Decreasing the number of samples without decreasing the estimation performance should facilitate the use of daptomycin AUC for TDM. Further studies would be needed to assess how frequently a simplified TDM would result in prescription modifications compared to routine clinical recommendations.

This study has some limitations. First, the initial simulations in this study did not account for potential correlations between covariates, despite known relationships like those between weight and sex. However, this information is only rarely available in the literature and descriptive tables, thus we did not use that based on the practical constraints of the available data rather than on theoretical considerations. Nevertheless, the impact of not accounting for these correlations on our final algorithm is likely to be small. Indeed, we used a boosting-based approach, generally less sensitive to correlation between feature and extreme values. Second, no real patient profile was available, and it is very important to evaluate the performance in real patients and not only in simulations. Our simulated AUC values are closely tied to the two POPPK models and to the characteristics of the patient populations used to develop them. However, they may not fully encapsulate the variability and heterogeneity present in larger, more diverse populations. This underscores a fundamental limitation of in silico approaches. Furthermore, caution should be exercised when using the algorithm developed to new patients, especially if their characteristics are different from those of the patients used to develop the POPPK models. Future efforts could aim to include actual external patients and pursue opportunities for external validation, even in silico, to enhance the generalizability of our results.

In conclusion, we developed the XGBoost ML models, allowing the accurate estimation of daptomycin AUC, and validated them in an independent simulation group. This new tool, even if it requires further validation in real patients, provides an opportunity to assess more easily, as fewer samples are needed than in a classical approach, the relevance of daptomycin TDM in clinical studies and may facilitate the development of subsequent recommendations for daptomycin TDM in routine practice.

MATERIALS AND METHODS

Patients and data

We simulated 4,500 pharmacokinetic profiles in the MRGsolve R package (34) for daptomycin doses of 4, 5, 6, 7, 8, 9, 10, 11, and 12 mg/kg (500 PK profiles per dose) using the population pharmacokinetic model of daptomycin published by Dvorchik et al. (35). This two-compartment model with a first-order elimination included creatinine clearance calculated using the Cockcroft & Gault formula, body weight, sex, and body temperature on the clearance PK parameter. We set the residual variability to almost zero to only account for the covariate effect and inter-individual variability as previously described (15, 23). The covariable distributions were simulated in accordance with those described in Dvorchik et al. study (35): body weight (mean ± SD [min-max] = 75.1 ± 30 [48–153] kg), body temperature (mean ± SD [min-max] = 37.2 ± 1.5 [36.1–140.1]°C), creatinine clearance (mean ± SD [min-max] = 91.2 ± 30 [14–150] mL/min), and 59% male. We also simulated 450 additional PK profiles with extreme weight (≥100 kg) and creatinine clearances (≤60 mL/min) for daptomycin doses of 4, 5, 6, 7, 8, 9, 10, 11, and 12 mg/kg (50 PK profiles per dose). The covariable distributions in these extreme patients were as follows: body weight (mean ± SD [min-max] = 120 ± 30 [100–153] kg), low creatinine clearances (mean ± SD [min-max] = 30 ± 15 [14–60] mL/min), and 59% male. In simulations, the doses were administered using 30-min infusions with an interdose of 24 h.

The simulated trough concentration (C0 sampled at t = 0 min at steady state) and C1 (blood concentration of daptomycin at time 1 h after infusion at steady state) were extracted, and the reference AUC0-24ss was calculated as dose/simulated clearance). After the elimination of aberrant profiles (defined as daptomycin C0, Cmax, and AUC out of the 1st and 99th percentiles), 4,740 profiles were used to develop the ML algorithms.

In order to externally validate our ML models, 200 additional pharmacokinetic profiles were simulated from Garreau et al. (36) using the same methodology. The dose used for simulation was 8 or 10 mg/kg (100 profiles per dose) to reproduce the doses used in routine medical care in our center. This two-compartment model with a first-order elimination included creatinine clearance calculated using the Cockcroft & Gault formula, body weight, body temperature, sex, age, effect of weight, sex, and age on distribution volume and effect of sex on creatinine clearance. As for the Dvorchik model, the residual error was set to zero. Extreme simulated profiles (C0, Cmax, and AUC out of the 1st and 99th percentiles) were filtered out.

Feature engineering

The difference between C1 and C0 was calculated to improve the model, as previously described (19, 21). Finally, the features used to predict AUCs using the ML algorithms were the patients’ sex, age, daptomycin dose, creatinine clearance, body temperature, daptomycin blood sampling at time T0 and T1, and the difference between these two concentrations. We excluded 1% extreme percentile in both the XGBoost model and the external data set.

Exploratory data analyses

Correlation matrix and scatterplots were drawn to explore the correlation between AUC and the seven predictors with the Ggally R package (37).

Preprocessing of the data

For all the ML analyses, the Tidymodels framework was used (https://www.tidymodels.org/). Data splitting was performed, allocating 75% to the training set (3,552 PK profiles) and 25% to the testing set (1,188 PK profiles). We developed the XGBoost boosting, random forest (ensemble tree), glmnet (linear model with a mixture of ridge and Least Absolute Shrinkage and Selection Operator [LASSO] penalization), linear support vector machine (SVM) models using the training set to predict the AUC. This training set was used to tune the hyperparameters and evaluate model performance by 10-fold cross-validation for each algorithm. The best-tuned ML model was then evaluated in the testing set by measuring the root mean square error (RMSE; expressed in mg.h/L) between the predicted and the reference AUC0-24ss. The performances were evaluated by calculation of the RMSE, R², mean prediction error (MPE; expressed in mg.h/L), relative MPE (%), relative RMSE (%), number, and proportion of estimations with an MPE value of the ±20% interval. Variable importance plots obtained by random permutation were drawn to assess the importance of predictors. Finally, scatterplots of predicted vs reference AUCs and residual as function of reference values were also drawn.

External evaluation and comparison to simulated trapezoidal POPPK

The performances of the best ML algorithm were investigated on simulations from another POPPK model to mimic an independent data set validation. The same methodology as described above was used for the comparison between predicted AUC0-24ss and reference AUC obtained from the simulated profiles.

Comparison to population pharmacokinetics and maximum a posteriori Bayesian estimation

The performances of the Bayesian posterior predictions using C0 and C1 compared to the full simulated curves to estimate steady-state AUC were calculated in the testing set simulated from Dvorchik and in the “external” set simulated from Le Louedec et al. using the mapbayR package (38). The same performances metrics as for the ML analysis were calculated. All HTML scripts, csv files, and complementary data are available in Cyrielle Codde’s github at https://github.com/CyrielleCodde/Daptomycin-AUC-prediction-from-2-samples.