External Evaluation of a Gentamicin Infant Population Pharmacokinetic Model Using Data from a National Electronic Health Record Database

Gentamicin is a common antibiotic used in neonates and infants. A recently published population pharmacokinetic (PK) model was developed using data from multiple studies, and the objective of our analyses was to evaluate the feasibility of using a national electronic health record (EHR) database for further external evaluation of this model.

ABSTRACT

Gentamicin is a common antibiotic used in neonates and infants. A recently published population pharmacokinetic (PK) model was developed using data from multiple studies, and the objective of our analyses was to evaluate the feasibility of using a national electronic health record (EHR) database for further external evaluation of this model. Our results suggest that, with proper data capture procedures, EHR data can serve as a potential data source for external evaluation of PK models.

TEXT

Gentamicin is one of the most commonly used antibiotics prescribed for treatment of or prophylaxis against Gram-negative infections in infants (1–3). Nephrotoxicity and ototoxicity are major adverse reactions that are associated with supratherapeutic gentamicin concentrations (4). Due to its narrow therapeutic index and wide pharmacokinetic (PK) variability, therapeutic drug monitoring of gentamicin is required (5, 6). Target peak concentrations of gentamicin should range from 4 to 10 mg/liter (conventional dosing), and trough levels should be <2 mg/liter (7).

Gentamicin population PK models were developed for infants in previous studies. Both two-compartment and three-compartment models were used to characterize the disposition of gentamicin in infants (8–12). Since gentamicin is almost entirely renally eliminated, age, weight, and serum creatinine (SCR) concentrations were commonly identified as important covariates of gentamicin clearance. Those studies either did not perform external evaluations or performed evaluations using external data sets consisting of 70 to ∼160 subjects (7–11).

Unlike traditional clinical trials, which are challenging to perform with children due to ethical, logistical, and financial factors (13), electronic health records (EHRs) allow researchers to access large volumes of clinical data easily and efficiently. The large sample size and widely distributed profiles in EHRs make them an ideal data source for evaluation of PK models. In previous studies, EHR data were used to develop PK models or to assess the relationship between drug exposure and safety (14, 15). To date, however, we are not aware of any studies that have used a national EHR database for external evaluation of a population PK model. The objective of this study was to use gentamicin as a case study to explore the potential use of EHR data in the evaluation of population PK models.

In this study, EHR data from 348 Pediatrix Medical Group neonatal intensive care units, from 1997 to 2014, were used to evaluate a previously reported gentamicin population PK model. Information in the EHRs included age, weight, sex, dose records, SCR concentrations, and peak and trough plasma concentrations of gentamicin. The population PK model developed by Germovsek et al. (8) is a three-compartment model with weight, postmenstrual age (PMA) and postnatal age (PNA), and SCR concentrations as covariates of clearance. The model was developed on the basis of 1,325 gentamicin serum concentration measurements from 205 infants and was evaluated using 483 gentamicin serum concentration measurements from 163 infants (8).

The following assumptions and criteria were used to extract relevant and reliable EHR data: (i) only infants receiving intravenous injections were included; (ii) the infusion time was assumed to be 30 min; (iii) only concentrations ranging from 4 to 20 mg/liter (peak) and from 0.3 to 10 mg/liter (trough) were included; (iv) peak samples were assumed to be collected 1 h after dosing and trough samples 2 min before dosing; (v) observations recorded from infants with SCR concentrations of >10 mg/dl were excluded; (vi) infants with PNA of >60 days and gestational age (GA) of <23 weeks were excluded; (vii) observations with doses of >6 mg/kg/day were excluded; (viii) to avoid model misspecification caused by data entry errors during regimen switches, only observations recorded during the first dosing regimen were included; and (ix) an occasion was defined as a dose with subsequently obtained gentamicin samples. These assumptions and criteria were based on common clinical practice and infant demographics in the model-building data set. A summary of demographics and dosing for the model-building data set (8) and filtered EHR data is shown in Table 1.

TABLE 1.

Population demographics for model-building and EHR data^a

Data set	No. of participants	No. of measurements	GA (median [range]) (wk)	PNA (median [range]) (days)	Body weight (median [range]) (kg)	Dose (median [range])
Model-building data setb	205	1,325	34 (23.3–42.1)	5.4 (1–66)	2.12 (0.53–5.05)	Initial dose of 2–3 mg/kg (twice daily) or 4 mg/kg (every 24 h)
EHR data	4,519	6,753	29 (23–42)	1 (1–59)	1.26 (0.31–4.79)	3.50 mg/kg/day (0.49–6.00 mg/kg/day)

^aGA, gestational age; PNA, postnatal age.

^bFrom reference 8.

To assess the predictive performance of the model, plots of population predicted concentrations versus observations for peak and trough concentrations were generated. Parameters were fixed to the final estimates reported in the original publication. The relationships between relevant covariates, i.e., body weight (WT) (in kilograms), measured SCR concentration (MSCR) (in micromoles per liter), typical SCR concentration (TSCR) (in micromoles per liter) (equal to −2.849·PMA [in weeks] + 166.48), PMA (in weeks), and PNA (in days), and PK parameters are described as follows: CL (in liters per hour) = 6.2 × PMA^3.33/(PMA^3.33 + 55.4^3.33) × (WT/70)^0.632 × (MSCR/TSCR)^−0.13 × [PNA/(1.70 + PNA)], V₁ (in liters) = 26.5 × (WT/70), V₂ (in liters) = 21.2 × (WT/70), V₃ (in liters) = 147.9 × (WT/70), Q₁ (in liters per hour) = 2.2 × (WT/70)^0.75, and Q₂ (in liters per hour) = 0.3 × (WT/70)^0.75, where CL is clearance, V is volume of distribution, and Q is intercompartmental clearance. Analyses were performed using NONMEM (version 7.3; Icon Development Solutions, Ellicott City, MD, USA); the first-order conditional estimation method with interaction was used. Data manipulation was performed with R (version 3.3.2) and RStudio (version 1.0.136). The xpose4 and lattice packages in R and RStudio were used for data visualization (16–18). Visual predictive checks (VPCs) were performed on the basis of 1,000 simulations, using Perl-speaks-NONMEM (version 4.6.0). The bias and precision of the model were evaluated by calculating the jth prediction error (PE_j) and relative prediction error (RPE_j), the mean prediction error (MPE), and the mean absolute prediction error (MAPE) (equations 1 to 4).

$${\text{PE}}_j=({\text{Prediction}}_j-{\text{Observation}}_j)$$ (1)

$${\text{RPE}}_j=\frac{{\text{PE}}_j\hspace{0.17em}\times 100}{{\text{Observation}}_j}$$ (2)

$$\text{MPE}=\text{Mean}\left(\frac{{\text{PE}}_j\times 100}{{\text{Observation}}_j}\right)$$ (3)

$$\text{MAPE}=\text{Mean}\hspace{0.17em}\left(\frac{\left|{\text{PE}}_j\right|\times 100}{{\text{Observation}}_j}\right)$$ (4)

Filtered EHR data contained 6,753 measurements, including 2,580 peak concentrations and 4,173 trough concentrations, from 4,519 infants. The EHR population had a similar age range, compared to the model-building data set (Table 1). Figure 1 shows box plots of the prediction errors and relative prediction errors for peak and trough concentrations. In the VPC plot (Fig. 2), 27.7% of observations were below the 80% prediction interval and 8.2% were above. There was a trend toward gentamicin concentrations plateauing after 24 h (Fig. 2), which might be related to large variations in gentamicin trough concentrations due to the timing of sample collection and different degrees of renal dysfunction among the infants. The median prediction errors were 3.43 mg/liter (2.5th to 97.5th percentile range, −6.20 to 12.95 mg/liter) and 0.35 mg/liter (2.5th to 97.5th percentile range, −2.03 to 1.78 mg/liter) for peak and trough concentrations, respectively. The median relative prediction errors were 40.82% (2.5th to 97.5th percentile range, −49.72 to 213.55%) and 47.14% (2.5th to 97.5th percentile range, −73.22 to 344.92%) for peak and trough concentrations, respectively (negative values indicate underprediction of concentrations). The MPEs from predictions of peak and trough concentrations were 51.0% and 71.0%, respectively. The precision values (measured as MAPEs) for peak and trough concentrations were 62.9% and 92.3%, respectively.

FIG 1.

Box plots of prediction errors (A) and relative prediction errors (B) for peak and trough concentrations. The bottom and top of the box are the 25th and 75th percentiles, respectively, and the thick line in the middle of the box is the 50th percentile. The height of the box is the interquartile range (IQR). The upper whisker indicates the 75th percentile + 1.5·IQR, and the lower whisker indicates the 25th percentile − 1.5·IQR. Open circles are outlier points that are outside 1.5·IQR above the upper quartile and below the lower quartile.

FIG 2.

VPC plot of gentamicin concentrations versus time after the last dose. The shaded regions denote the 95% confidence intervals around the 10th, 50th, and 90th percentiles of simulated concentrations. The dashed lines represent the 10th, 50th, and 90th percentiles for the observed data. The solid lines represent the 10th, 50th, and 90th percentiles for the predicted data. Open circles indicate the observed values.

Our results demonstrated that the model developed by Germovsek et al. (8) successfully captured the central tendency of the gentamicin concentrations in the EHR database (Fig. 2), with some notable overprediction (i.e., the distribution of relative prediction errors was skewed to the right) of peak and trough concentrations (Fig. 1). Peak concentrations were predicted with greater accuracy and precision, compared to trough concentrations, which is consistent with the findings from the original analysis. Overall, the model appears to have less accuracy and precision when evaluated with the EHR data, compared to the initial external database (8); this may be explained by assumptions we made in modeling the EHR data, particularly the lack of exact sampling times, which might lead to misspecification. There are variations in clinical practice regarding when peak concentrations are obtained, and a significant number of samples being drawn 1 h, rather than 30 min, after dosing might lead to overprediction of gentamicin concentrations. Additionally, differences in the gentamicin assays used by the centers might introduce measurement error, especially for trough concentrations falling near the lower limit of quantification. Since therapeutic hypothermia is associated with alterations in gentamicin PK and we could not capture this from the current data set, this factor may explain some of the observed misspecification (19). Therefore, it is likely that the model misspecification we observed in our analyses is related to the assumptions we made in developing our gentamicin EHR database for external evaluation. Given that this model performed well in previous external evaluations (8), further study focused on clinical implementation and evaluation of this model's use in facilitating dose individualization is justified.

While the use of EHR databases can significantly enhance the quantity of clinical data, ensuring that the data are of high quality is still crucially important. The major challenge we encountered in performing population PK modeling of EHR data was the lack of accurate documentation of sampling times and appropriate formatting of clinical data, which required us to apply reasonable assumptions to estimate missing information, as well as necessitating significant effort to prepare analysis-ready data sets. As a result, the misspecification we identified might result from either model error or data inaccuracy, which makes the evaluation of PK models more challenging. To maximize the use of EHR data in building and evaluating population PK models, more studies are needed to identify efficient procedures for extraction of large volumes of accurate clinical data from EHR databases. In addition, the widespread use of EHR databases in model evaluations could benefit from improvements in the protocols for clinical data collection, particularly the timing of dosing and PK measurements.

In conclusion, a national EHR database was used for external evaluation of a published population PK model for gentamicin in infants. Despite notable misspecifications, the model captured the central tendency of the gentamicin concentrations in the EHR database. Improvements in EHR data collection are required to maximize the robustness of EHR databases for population PK model evaluations.