Prospective validation of a model‐informed precision dosing tool for vancomycin in intensive care patients

Abstract

Aims

Vancomycin is an important antibiotic for critically ill patients with Gram‐positive bacterial infections. Critically ill patients typically have severely altered pathophysiology, which leads to inefficacy or toxicity. Model‐informed precision dosing may aid in optimizing the dose, but prospectively validated tools are not available for this drug in these patients. We aimed to prospectively validate a population pharmacokinetic model for purpose model‐informed precision dosing of vancomycin in critically ill patients.

Methods

We first performed a systematic evaluation of various models on retrospectively collected pharmacokinetic data in critically ill patients and then selected the best performing model. This model was implemented in the Insight Rx clinical decision support tool and prospectively validated in a multicentre study in critically ill patients. The predictive performance was obtained as mean prediction error and relative root mean squared error.

Results

We identified 5 suitable population pharmacokinetic models. The most suitable model was carried forward to a prospective validation. We found in a prospective multicentre study that the selected model could accurately and precisely predict the vancomycin pharmacokinetics based on a previous measurement, with a mean prediction error and relative root mean squared error of respectively 8.84% (95% confidence interval 5.72–11.96%) and 19.8% (95% confidence interval 17.47–22.13%).

Conclusion

Using a systematic approach, with a retrospective evaluation and prospective verification we showed the suitability of a model to predict vancomycin pharmacokinetics for purposes of model‐informed precision dosing in clinical practice. The presented methodology may serve a generic approach for evaluation of pharmacometric models for the use of model‐informed precision dosing in the clinic.

1. What is already known about this subject

• Population pharmacokinetics of vancomycin have been widely studied in critically ill patients.

• Critically ill patients have severely altered pathophysiology, which may lead to differences in pharmacokinetics of vancomycin with subsequent inadequate drug concentrations.

• Software using population pharmacokinetic models from critically ill patients can optimize antibiotic exposure by accurately predicting dosing requirements for the individual patient.

What this study adds

• This is the first study to both retrospectively validate and prospectively evaluate the ability of a pharmacokinetic model of vancomycin in critically ill patients.

• Predictions of vancomycin concentrations from the best performing pharmacokinetic model can be made with acceptable precision for individualized dosing in our population.

1. INTRODUCTION

Vancomycin is an important antibiotic for critically ill patients with Gram‐positive bacterial infections such as Staphylococcus aureus, Staphylococcus epidermidis and vancomycin‐susceptible Enterococcus. ¹

Critically ill patients typically have severely altered pathophysiology that may lead to clinically relevant changes in pharmacokinetics (PK) of vancomycin resulting in subtherapeutic or toxic drug exposure ² , ³ Therapeutic drug monitoring (TDM) can aid in achieving desired exposure to optimize therapy by adaptations of dosages. TDM is therefore recommended to guide dosing of vancomycin. ⁴ The pharmacodynamic index of vancomycin is best described as the AUC24/MIC (area under the serum concentration–time curve over 24 h divided by the minimum inhibitory concentration) with a target ratio of ≥400 best correlated with efficacy for infections with Staphylococcus aureus. ⁵ Concentrations of vancomycin below this threshold are associated with the development of resistance and clinical failure. ¹ , ⁶ Prolonged high exposure to vancomycin is associated with a higher likelihood of encountering nephrotoxicity. There is evidence that TDM significantly increases the rate of clinical efficacy and decreases the rate of nephrotoxicity in patients treated with vancomycin. ⁷ For practical reasons, trough concentrations are often advised as a PK target in clinical practice. ¹

Achieving adequate antibiotic exposure in the critically ill population is challenging. The DALI‐study showed that only 57% of the intensive care (IC) unit patients achieved target trough concentrations (≥15 mg/L) ⁸ and Baptista et al. showed that in patients admitted to the IC unit with augmented renal clearance only 52% of the patients reached the therapeutic concentration within 3 days. ⁹

Most clinicians rely on dosing nomograms for dose adjustment of vancomycin, yet the majority of the published nomograms were developed in small groups of patients and have not been clinically validated in external populations. ¹⁰ Software programs using population PK models can provide a better alternative to nomograms to optimize exposure and are capable to more accurately predict dosing requirements for the individual patient. ¹¹ , ¹²

To fully deploy the benefits of model‐informed precision dosing (MIPD), the best possible population PK models that best describe the local population have to be selected. This should be followed by a prospective validation of PK models. In most cases, prospective validation of PK models for this purpose is lacking thereby failing to meet above mentioned requirements. ¹³ The aim of this study was to retrospectively evaluate the performance of existing population PK models for vancomycin in critically ill patients and, subsequently, prospectively evaluate the ability of the best performing model to predict vancomycin serum concentrations in this population using the Insight Rx platform (San Francisco, CA, USA).

2. METHODS

We performed a structured 3‐step‐approach for our study, consisting of the following steps:

1. Selection of population PK models of vancomycin in adult patients from literature

2. Retrospective external evaluation of the predictive performance of published PK models

3. Prospective evaluation of the selected PK model

2.1. Selection of population PK models of vancomycin in adult patients from literature

We performed a systematic literature search in PubMed for all population PK studies of vancomycin in adult patients from 2006 until 2016. We combined the following keywords (MeSH and free text) in our search strategies: vancomycin, pharmacokinetic, adult and population pharmacokinetics. Reference lists of identified articles were then manually screened for additional relevant studies. Furthermore, the publicly available abstracts from the Population Approach Group Europe meeting (www.page-meeting.org) were screened.

Models including patients requiring extracorporeal membrane oxygenation (ECMO) and/or dialysis and models that were based on <25 patients, were excluded. Publications with no full text available or focusing on 1 specific patient population (for example haemato‐oncology, obese, neurosurgical) were also excluded because it is known that the PK of vancomycin is different in this population compared to the mixed cohort of IC patients. ¹⁴ , ¹⁵ The remaining models were evaluated by a team of experts, consisting of a clinical pharmacist, a clinical pharmacologist/clinical pharmacist and a pharmacometrician for their relevance for practice.

2.2. Retrospective external evaluation of the predictive performance of published PK models

The included models were evaluated using retrospective PK data of IC patients. IC patients with an age of >18 years, receiving vancomycin (both intermittent and continuous infusion) during their stay in the IC ward at the Radboudumc University Medical Center (Nijmegen, the Netherlands) between January 2011 and January 2016 were considered for inclusion in this retrospective study if at least 1 vancomycin serum concentration was available. Patients receiving ECMO and/or dialysis and haemato‐oncology patients were excluded for reasons mentioned above.

The following data were collected from the electronic hospital system (Epic Systems, Verona, WI, USA): age (y), weight (kg), length (cm), sex, serum creatinine concentrations (μmol/L), vancomycin dose (mg), infusion length (h), dosing information (including time of administration), plasma vancomycin concentrations (mg/L) and blood sampling times. Patients with incomplete information were excluded.

The published PK models were evaluated on the retrospectively collected data. For this purpose, normalized prediction distribution error (NPDE) plots and visual predictive checks (VPC) were evaluated. This methodology was previously proposed previously by Zhao et al. ¹⁶ to evaluate the transferability of published models to clinical practice. Lastly, a fit‐for‐purpose evaluation was performed. For this purpose, the a priori and a posteriori predictive performance was evaluated: meaning their performance before TDM would be performed, and after 1 or more TDM samples have become available to perform an individual fit, respectively. Performance was evaluated by calculating the relative root mean squared error (RRMSE) as measure for precision and mean prediction error (MPE) as a measure for bias. ¹⁷ Also, the individually predicted vs observed concentrations were plotted for each model for both the a priori and a posteriori situation. The retrospective analysis was performed in R v3.5 (R Foundation for Statistical Computing, Vienna, Austria).

2.3. Prospective evaluation of the selected PK model

The best performing model according to judgement of the panel of experts was carried forward for prospective fit‐for‐purpose evaluation. In a prospective study, patients with an age of >18 years, receiving vancomycin (both intermittent and continuous infusion) during their stay in the IC unit at the Radboudumc University Medical Center (Nijmegen, the Netherlands), Jeroen Bosch Ziekenhuis (Den Bosch, the Netherlands), Isala (Zwolle, the Netherlands) and Rijnstate (Arnhem, the Netherlands) between June 2017 and June 2018 were considered for inclusion if at least 1 vancomycin serum concentration available. In all hospitals, sampling of trough concentrations was implemented as standard of care to guide vancomycin dosing. The same inclusion and exclusion criteria were applied as were used in the retrospective evaluation. The data were anonymously recorded in InsightRX. Dosing of vancomycin was according to local practice and at the discretion of the treating physician. Both intermittent and continuous infusions were allowed. Dose adaptations based on TDM results and InsightRX recommendations were allowed during the course of treatment.

Predictive performance of the PK model was assessed by calculating the RRMSE and MPE. We defined an acceptable bias and imprecision as <25%, based on the fact that such a bias and imprecision fall well within the PK trough target that, depending on population, dose and pathogen, is usually in the range of 10–25 mg/L. For all analyses, we performed maximum a posteriori Bayesian fitting, without adaptive weighting using the InsightRX algorithm.

2.4. Ethics

The medical ethics committee (Commissie Mensgebonden Onderzoek Arnhem‐Nijmegen, Nijmegen, The Netherlands) waived the necessity of formal approval and informed consent.

They declared that this research project could be exempted from obtaining informed consent because all data were obtained during routine clinical practice and extracted anonymously.

3. RESULTS

3.1. Selection of population PK models of vancomycin in adult patients from literature

A total of 5 suitable population PK models were identified and these are listed in Table 1. All these models had in common that body weight was a covariate for the volume of distribution and that estimated creatinine clearance was a covariate for clearance. The disposition of vancomycin in the body was described with either 1 or 2 compartments.

TABLE 1.

Identified population pharmacokinetic models for vancomycin

Model number and authors	Population	Structural model	Covariates
1. van Maarseveen et al. 18	Surgical ward patients, internal ward patients and critically ill patients (number of patients for model development not reported)	1 compartment	Total body weight on volume of distribution
			Creatinine clearance of clearance of vancomycin
2. Llopis‐Salvia et al. 19	Critically ill patients (n = 30)	2 compartments	Total body weight on volume of distribution
			Creatinine clearance of clearance of vancomycin
3. Roberts et al. 20	Critically ill patients (n = 206)	1 compartment	Total body weight on volume of distribution
			Creatinine clearance of clearance of vancomycin
4. Thomson et al. 21	Hospitalized patients, independent of condition (n = 398)	2 compartments	Total body weight on volume of distribution
			Creatinine clearance of clearance of vancomycin
5. Zdovc et al. 22	Critically ill patients (n = 33)	1 compartment	Total body weight on volume of distribution
			Creatinine clearance of clearance of vancomycin

3.2. External evaluation of the predictive performance of published PK models on retrospectively collected data

Vancomycin data from 30 patients were collected with a median of 3 paired observations of time and vancomycin serum concentrations (range 2–16). A summary of the data is presented in Table 2.

TABLE 2.

Retrospective data summary

Characteristic	Results
Sex
Male	18 (60%)
Female	12 (40%)
Age, median (range)	59 (20–82) y
Weight, median (range)	80 (54–133) kg
Height, median (range)	1.72 (1.37–1.90) m
Serum creatinine, median (range)	84 (40–189) μmol/L
Number of observations per patient, median (range)	3 (2–16)

Figure 1 shows the prediction‐corrected VPCs, based on 1,000 simulations on the retrospective data for each model. The blue shaded areas in this figure show the 95% of the 10^th, 50^th and 90^th of the simulated data. The lines in the different panels connect the respective percentiles of the observed data (open circles). As observed, the models by Zdovc et al. ²² Thomson et al. ²¹ and Roberts et al. ²⁰ best described the data, as the observed 10th, 50th and 90th percentiles (lines) matched the simulated prediction intervals for these percentiles (blue shaded areas). Furthermore, it can be observed in all plots that at timepoints >12 hours after last administration all models are under‐predicting. This phenomenon can be explained by the fact that usually vancomycin in adults is dosed twice or 3 times daily and that because of TDM dosing intervals are extended to 24 hours. Therefore, at timepoints later than 12 HOURS, there is a selection bias for data from patients who have a slower clearance than the population, which may be misinterpreted as model under‐prediction.

FIGURE 1.

Visual predictive checks of the identified models on the retrospective data. The blue shaded areas in this figure show the 95% of the 10^th, 50^th and 90^th of the simulated data. The lines in the different panels connect the respective percentiles of the observed data (open circles)

Figure 2 shows the QQ‐plot of the distribution of the NPDE vs the theoretical N (0,1) distribution (Figure 2A) as well as the histogram of the distribution of the NPDE, with the density of the standard Gaussian distribution overlaid for each model (Figure 2B). It can be seen that the observed quantiles follow the predicted theoretical quantiles (line of unity) for most of the data, but curve off at the extreme values. The latter indicates that the observed PK data have more extreme values than would be expected if they would come from a normal distribution.

FIGURE 2.

QQ‐plot of the distribution of the normalized prediction distribution error (NPDE) vs the theoretical N (0,1) distribution (A) as well as the histogram of the distribution of the NPDE, with the density of the standard N(0,1) distribution overlaid for each model (B). It can be seen that the observed quantiles seem to follow a normal distribution, but are more dispersed than the theoretical N(0,1) distribution

The results of the fit‐for‐purpose evaluation on the retrospectively collected data are graphically depicted in Figure 3, showing the MPE as a measure for bias, the RRMSE as a measure for precision and the predicted vs observed concentrations in Figure 3A, B and C, respectively. The models by Maarseveen et al. ¹⁸ Roberts et al. ²⁰ Thomson et al. ²¹ and Zdovc et al. ²² performed best on the retrospectively collected data, as observed in the lowest bias and best precision.

FIGURE 3.

(A) Mean prediction error (MPE) including 95% confidence intervals obtained with the various models on the retrospective data. (B) Relative root mean squared error (RRMSE) including 95% confidence intervals obtained with the various models on the retrospective data. (C) Predicted vs observed concentrations obtained with the various models on the retrospective data

3.3. Model selection and prospective evaluation

Based on overall best performance on the retrospective data, the model by Thomson et al. ²¹ was implemented in the Insight Rx framework and prospectively evaluated. This decision was based on the lowest bias and imprecision and the best performance of the VPC, based on visual assessment.

We included 50 patients in the prospective study. No patients were excluded from the analysis. Characteristics of the population used for prospective evaluation of the Thomson model are shown in Table 3.

TABLE 3.

Population characteristics

Demographic characteristics (n = 50)
Sex, n (%)
Male	28 (56)
Female	22 (44)
Age, median (range)	68 (36–84) y
Weight, median (range)	82 (50–110) kg
Height median (range)	1.72 (1.57–1.93) m
Serum creatinine median (range)	79 (73–285) μmol/L
Number of observations per patient, median (range)	4 (1–23)

Figure 4A and B show the MPE and overall RRMSE for the a priori and a posteriori situations with the Thomson model for the prospective evaluation. Figure 4C shows the RRMSE vs the TDM sample. It can be observed from Figure 4A and B that the accuracy and imprecision in the prospectively collected data were improved compared to retrospectively collected data. Furthermore, it can be observed that the RRMSE and MPE for the a priori situation are insufficient (>25%), indicating that vancomycin exposure cannot be adequately predicted without a previous assessment of the vancomycin plasma concentration. The bias and imprecision are well below the 25% margin for the a posteriori situation, indicating suitability for use of the model in clinical practice when the model is used for dose forecasting. Lastly, in Figure 4C, a trend can be observed that the precision of the prediction improves when more previous TDM samples are available. Figure 4D shows the predicted vs the observed concentration. These concentrations are scattered evenly around the line of unity, showing that the model performs well.

FIGURE 4.

(A) Mean prediction error (MPE) including 95% confidence intervals in the prospective study. (B) Relative root mean squared error (RRMSE) including 95% confidence intervals in the prospective study. (C) Relative root mean squared error vs therapeutic drug monitoring (TDM) sample in the prospective study. (D) Predicted vs observed concentrations in the prospective study

4. DISCUSSION

We have shown, using a systematic fit‐for‐purpose evaluation of existing population PK models for vancomycin in critically ill patients, that forecasting of vancomycin PK in a critically ill population can be performed accurately and precisely based on an existing PK model and routinely measured vancomycin concentrations. The predictive performance of the evaluated models was insufficient to dose vancomycin in this population without assessment of vancomycin plasma concentrations, underlining the necessity of TDM of vancomycin in critically ill patients.

The analysis described here is the first prospective evaluation of an MIPD tool for vancomycin in critically ill patients. We found that the model by Thomson et al. ²¹ performed best in a retrospective evaluation and then showed in a prospective multicentre study that it could precisely and accurately predict vancomycin exposure. It could be observed that the predictive performance in the prospective evaluation was better than in the retrospective evaluation. It is most likely that this is a result of the better quality of prospectively collected data and this shows the usefulness of a prospective evaluation. We found a small bias in the a posteriori predictive performance (MPE of 8.84%). We think this structural bias is negligible compared to the therapeutic window, that is usually in the range of 10–25 mg/L. Therefore, dose advices should always be verified with follow‐up measurements of the vancomycin serum concentration.

Furthermore, in both the retrospective and prospective evaluation, it could be observed that for the a priori situation, the situation where vancomycin PK are only predicted based on covariates and dose, the predictive performance was poor. This underlines the current consensus that vancomycin dosing should be guided by TDM.

Guo et al. ²³ recently performed a retrospective evaluation of the predictive performance of several population PK models for vancomycin in 2 Dutch hospitals. It was concluded that, on a population level, the model by Roberts et al.²⁰ performed satisfactorily. These results are in line with our retrospective results, where the model by Roberts et al. ²⁰ performed adequately. Unfortunately, Guo et al. did not evaluate the model by Thomson et al.²¹ Furthermore, a prospective fit‐for‐purpose evaluation ¹³ was lacking in the analysis by Guo et al. ²³ to evaluate how well vancomycin exposure could be predicted from a previous PK assessment. A fit‐for‐purpose evaluation of various population PK models for vancomycin in adult patients was recently also performed by Broeker et al. ²⁴ This study performed a similar evaluation as performed by us, albeit in a retrospective setting. The strength of the analysis by Broeker et al. is that a cross‐sectional evaluation was performed in a general hospital population. This may, however, also be a limitation, as it is known that some populations, such as critically ill ²⁵ or neutropenic ²⁶ patients, exhibit altered vancomycin PK. We therefore postulate that our prospective fit‐for‐purpose evaluation in the critically ill population is of added value.

As the ultimate goal of MIPD of vancomycin is to forecast the dose based on previous vancomycin PK observations, we performed a prospective multicentre study where we evaluated the individual predictive performance of the model by Thomson et al. ²¹ to predict future vancomycin PK from a previous observation of its PK, dose and covariates.

One may argue that a shortcoming of our analysis is that the primary endpoint of our prospective fit‐for‐purpose evaluation is a numerical outcome, and not a clinical outcome such as time‐to‐cure or time‐to‐therapeutic target attainment. We have chosen the current endpoint as multimorbidity and high variable vancomycin PK in critically ill patients, as well as differences in local TDM practice in the participating centres may all blur these outcomes.

Furthermore, a shortcoming of our study is that we did not include patients on haemodialysis, continuous venovenous haemofiltration or ECMO in our study. A next step in MIPD of vancomycin in the critically ill population should, therefore, be development and prospective validation of PK models that can adequately capture vancomycin PK for these patients as well, despite the potential limitations. ²⁷ Until then, we advise to perform intensified PK monitoring of vancomycin when dialysis techniques or ECMO are used to prevent inadequate therapy.

Although practice guidelines currently advocate the use of trough levels to guide vancomycin dosing, ⁴ there is increasing evidence that the AUC is a better surrogate endpoint for efficacy than trough concentrations and that using trough levels alone to guide vancomycin dosing should be reconsidered. ²⁸ The purpose of our evaluation was to prospectively evaluate the predictive performance of model‐based TDM in a multicentre study, where trough sampling was implemented as standard of care. It should be noted that, if the AUC has to be predicted from a previous PK assessment, one should first establish a sampling schedule for reliable assessment of AUC and then evaluate the predictive performance of this scheme for purpose of MIPD. This was, however, not the purpose of our analysis, where we evaluated the predictive performance various model for prediction of subsequent trough concentrations of vancomycin. As stated earlier, the therapeutic window for vancomycin depends on both the dosing regimen, exposure‐related toxicity and the pathogen susceptibility. The predictive performance of the used MIPD strategy should always be put in context with the therapeutic window and it should be noted that the therapeutic window for vancomycin may become small with pathogens with reduced vancomycin susceptibility. It is, therefore, advisable to always verify the MIPD intervention with a follow‐up PK assessment. Furthermore, if of vancomycin the AUC24/MIC target is used for MIPD, sources of variability in the assessment of both the AUC24 as the MIC should be considered. ²⁹

In conclusion, we have systematically evaluated the fitness of various model for use of MIPD of vancomycin in critically ill patients. The model by Thomson et al. ²¹ is now used in clinical practice for this purpose. Our approach, using a retrospective evaluation and prospective verification, may be used for evaluation of models for other drugs and populations.

COMPETING INTERESTS

Ron Keizer is shareholder of Insight Rx, San Francisco, CA, USA. The other authors have no competing interest to declare.

CONTRIBUTORS

R. ter Heine designed and performed the study and drafted the manuscript. R.J. Keizer designed and performed the statistical analysis and drafted the manuscript. K. van Steeg performed the study and reviewed the manuscript. E.J. Smolders performed the study and reviewed the manuscript. M. van Luin performed the study and reviewed the manuscript. H.J. Derijks performed the study and reviewed the manuscript. C.P.C. de Jager performed the study and reviewed the manuscript. T. Frenzel performed the study and reviewed the manuscript. R. Brüggemann designed and performed the study and drafted the manuscript.

ter Heine R, Keizer RJ, van Steeg K, et al. Prospective validation of a model‐informed precision dosing tool for vancomycin in intensive care patients. Br J Clin Pharmacol. 2020;86:2497–2506. 10.1111/bcp.14360

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.