Validation of the Predictive Value of Modeled Human Chorionic Gonadotrophin Residual Production in Low-Risk Gestational Trophoblastic Neoplasia Patients Treated in NRG Oncology/Gynecologic Oncology GroupY174 Phase III Trial

Abstract

Objectives:

In low-risk gestational trophoblastic neoplasia, chemotherapy effect is monitored and adjusted with serum human chorionic gonadotrophin (hCG) levels. Mathematical modeling of hCG kinetics may allow prediction of methotrexate (MTX) resistance, with production parameter “hCGres.” This approach was evaluated using the GOG-174 (NRG Oncology/Gynecologic Oncology GroupY174) trial database, in which weekly MTX (arm 1) was compared with dactinomycin (arm 2).

Methods:

Database (210 patients, including 78 with resistance) was split into 2 sets. A 126-patient training set was initially used to estimate model parameters. Patient hCG kinetics from days 7 to 45 were fit to: [hCG(time)] = hCG7 * exp(–k * time) + hCGres, where hCGres is residual hCG tumor production, hCG7 is the initial hCG level, and k is the elimination rate constant. Receiver operating characteristic (ROC) analyses defined putative hCGRes predictor of resistance. An 84-patient test set was used to assess prediction validity.

Results:

The hCGres was predictive of outcome in both arms, with no impact of treatment arm on unexplained variability of kinetic parameter estimates. The best hCGres cutoffs to discriminate resistant versus sensitive patients were 7.7 and 74.0 IU/L in arms 1 and 2, respectively. By combining them, 2 predictive groups were defined (ROC area under the curve, 0.82; sensitivity, 93.8%; specificity, 70.5%). The predictive value of hCGres-based groups regarding resistance was reproducible in test set (ROC area under the curve, 0.81; sensitivity, 88.9%; specificity, 73.1%). Both hCGres and treatment arm were associated with resistance by logistic regression analysis.

Conclusions:

The early predictive value of the modeled kinetic parameter hCGres regarding resistance seems promising in the GOG-174 study. This is the second positive evaluation of this approach. Prospective validation is warranted.

Patients with low-risk gestational trophoblastic neoplasia (GTN), characterized by an International Federation of Gynecology and Obstetrics (FIGO) 2000 risk scores 6 or less, are treated with single-agent chemotherapy. Treatment is usually continued until either normalization of serum human chorionic gonadotrophin (hCG) concentration followed by 1 to 3 additional cycles or resistance to this drug is detected.^1,2 Tu-mor chemoresistance is frequently characterized as an increase or stagnation of hCG levels during a 2- to 3-week period, but no consensus guideline has been defined.³ An early assessment of single-agent resistance risk is desirable because it may reduce the likelihood that patients will receive repetitive ineffective chemotherapy cycles. New methods for earlier identification, before the seventh week of treatment, of patients who will develop subsequent methotrexate (MTX) resistance have been reported.^4–9 Among them, a model-based approach developed by the French Reference Centre for treatment of GTN (Centre de Référence des Maladies Trophoblastiques, Lyon, France) was reported in a first study in 2010 and subsequently validated with a large independent database with patients treated at Charing Cross Hospital.¹⁰ It was shown that mathematical modeling of individual declining hCG measurements using a population kinetic approach is feasible. Moreover, a derived parameter called hCGres, related to residual production at the end of hCG decline, may be a strong early predictor of MTX resistance in patients treated with Bagshawe et al modified protocol commonly used in Europe (50 mg intramuscular [IM] MTX on days 1, 3, 5, and 7 combined with 15 mg oral folinic acid on days 2, 4, 6, and 8 every 2 weeks).^11,12 It was suggested that the estimation of modeled hCGres after 50 treatment days may help reduce the duration of MTX therapy by 2 weeks in 50% patients (and by 4 weeks in 25% patients) in resistant patients.¹⁰

However, the predictive value of modeled hCGres regarding resistance risk in patients prescribed with other regimens is still to be determined. The objective of the present study was to assess the predictive value of the hCG kinetic modeling approach with respect to treatment resistance in low-risk GTN patients treated with biweekly intravenous dactinomycin (ACT-D) regimen or weekly IM MTX in the GOG- 174 (NRG Oncology/Gynecologic Oncology GroupY174) phase III trial.

MATERIALS AND METHODS

GOG-174 Trial, Patient Selection, and hCG Analysis

The data from patients enrolled in the GOG-174 phase III trial were retrieved.¹³ All patients gave written informed consent before study entry in compliance with all local in stitutional review board and federal guidelines. Treatment with weekly IM MTX 30 mg/m² (arm 1) or with biweekly intravenous ACT-D 1.25 mg/m² (arm 2) was randomly allocated as first-line treatment for 240 patients with low-risk GTN according to FIGO 2000 guidelines. Patients continued on treatment until the A-hCG assay had reached the institutional normal or until either a rise (a >20% rise in the value for any 2 consecutive weekly assays) or a plateau (a <10% decrease in 3 consecutive weekly values) in the β-hCG level was observed. Once normal, the β-hCG level was determined biweekly for 1 month and then monthly for a further 11 months (12-month follow-up). In the event of a rise or plateau, study treatment was to be discontinued and treatment response was classified as a treatment failure. These patients were managed thereafter at the treating physician’s discretion. Patients who had a complete response received 1 additional cycle of their assigned treatment regimen after the first normal β-hCG level. Patient β-hCG values were measured every week while the patient was receiving study treatment and weekly for an additional 4 weeks after normalization. Written informed consent consistent with institutional, state, and federal regulations was obtained before entry into the study and random assignment. Local institutional review board approval was obtained.

The primary end point of the GOG-174 trial was objective complete response, as determined by a normal β-hCG sustained for a minimum of 4 consecutive weeks. The β-hCG serum concentrations (total β-hCG) were determined locally by participating North American institutions using different immunoassays. No central measurement of hCG titers had been planned. Exclusion criteria for the present study included ineligibility criteria for the initial protocol (World Health Organization score >6, inadequate pathologic documentation of disease, undocumented low-risk/persistent disease, ineligible histology), along with treatment with less than 2 courses or less than 2 available hCG values.

The main objective of the present study was to assess the early predictive value of hCGres modeled during the first 45 days in patients with low-risk GTN treated with different regimens regarding treatment resistance, as defined by β-hCG relapse or lack of normalization.

Population Kinetic Modeling of hCG Measurements

Two independent data sets were randomly built using the data from 210 patients (Fig. 1): a training data set including 60% of the patients (n = 126), used to adjust the parameters of the hCG model to regimens given in GOG-174, and a test data set composed of 40% of the patients (n = 84) with blinded-resistance status to assess the validity of predictions. The need for a readjustment of the hCG model parameter estimates in a training data set, with respect to those reported in previous European studies, was justified by the specific doses and routes of treatment regimen given in the GOG-174 trial.

FIGURE 1.

Flowchart CONSORT diagram.

The population pharmacokinetic model developed by the French national center in 2010 was used to fit overall declining hCG measurements across time from patients.¹⁴ Details of the monoexponential model have been previously described.¹⁴ The modeling work was performed between days 7 and 45 after start of chemotherapy. Indeed, an initial surge, defined as an increase of β-hCG between days 0 and 7, identified in 14% patients, worsened the fit of the predefined decrease shape by the mathematical model. Moreover, such a small proportion of patients would not allow estimation of a specific function integrating this initial surge. The model was refined and the kinetic parameter “hCGres,” related to residual production at the end of the β-hCG decline, was derived after fitting of declining β-hCG values using the final model hCG_ij(t) = (hCG7_i * e^−K_i * ^t + hCGres_i) (1 + ϵ_ij), where hCG_ij (t) is the jth β-hCG measurement in patient i at time t from start of study treatment; hCGres_i is the β-hCG residual production of patient i; and ϵ_ij represents the proportional residual variability at the jth measurement of patient i, which is assumed to be normally and independently distributed with 0 mean and σ² variance. Parameters K_i and hCGres_i were assumed to vary between patients according to a log normal distribution and parameters hCG7_i according to a Box-Cox distribution. Because the data included concentrations below the limit of quantification (eg, BLOQ ≤ 5 IU/L), the first concentration in a series of BLOQ observations was replaced by LOQ/2 and later observations were censored.^15,16

The following covariates were tested to estimate their impact on hCG kinetic parameters for reducing unexplained interindividual variability: treatment arm, serum creatinine, and choriocarcinoma. Analysis of covariates was guided by visual inspection for potential relationships between interindividual variability and the subject factor. When a covariate demonstrated significant relationships with any kinetic parameter, it was introduced into the model describing the fixed effects by forward inclusion. The covariate was kept in the model only if a significant decrease in the objective function was obtained (decrease, >7; P < 0.01). The choice of the final model was based on likelihood ratio tests (LRTs), goodness of fit, and simulation-based diagnostics. The modeling methodology has been described in detail elsewhere.¹⁴

Internal Graphical Validation of the Model Fit: Visual Predictive Check

The predictive performance of the final model was evaluated using the visual predictive check method, an approach commonly adopted in modeling studies.^17,18 As such, 1000 replicates were simulated using parameter estimates of the final model. Observed β-hCG values, along with their median and 5% and 95% quantiles were compared with the median and 5% and 95% quantiles (and their associated 95% confidence interval [95% CI]) of the simulated values.

Assessment of Predictive Value for Treatment Resistance Using Receiver Operating Characteristic Curve Analysis

Receiver operating characteristic (ROC) curve analyses were used to assess the predictive value of modeled hCGres in each arm.¹⁹ The optimal hCGres cutoffs in each arm for discriminating resistant versus sensitive patients were determined by the intersection of minimal overlap of the true-positive and true-negative populations for the training data set, that is, where sensitivity + specificity was maximized.

Analysis of Test Data Set

The final model and parameter distributions issued from the training data set were applied to independently estimate individual values of hCGres in the test data set patients. As previously described, overall declining β-hCG values observed between days 7 and 45 were fitted using Maximum A Posteriori algorithm in NONMEM software.²⁰ The predicted resistance risk of the test data set patients was defined based on the predefined cutoffs in every subject, except for 3 patients who did not have the required number of assessable hCG values. The performance of prediction was then independently assessed by sensitivity and specificity using the predicted treatment resistance against unrevealed observed treatment resistance.¹⁹

Predictive Value of hCGres Regarding Risk of Resistance With Treatment Arms

Logistic regression was used to examine the associations between the binary response to study treatment (ie, resistance vs sensitive) and modeled hCGres and treatment arm in test data set patients.²¹ Based on model selection criteria of Akaike information criterion²² and generalized coefficient of determination (R²)²³ (results not shown), the final logistic regression model was logit[Prob(response = resistance)] = β_0 + β_1 × hCGres + β_2 × treatment arm. An arbitrary significance level of 0.05 was used to classify individual statistical hypothesis test results as statistically significant. There was no adjustment for multiple tests.

Analysis Algorithms/Software

The modeling analysis was performed using a nonlinear mixed-effects modeling strategy implemented in NONMEM version 7.²⁰ Data analysis was performed using the first-order conditional estimation method with an interaction computational method algorithm.²⁴ Graphical representations were performed in R using the Xpose package.^25,26

Statistical analyses were performed using XLSAT and SAS 9.2.

RESULTS

Patient Characteristics

Among 216 eligible patients, β-hCG measurements and clinical data from 210 patients could be explored for the present study (Fig. 1). Summary characteristics of the patient cohort are presented in Table 1.

TABLE 1.

Characteristics of patients included in the analysis

		Training Data Set Patients, n (%)	Test Data Set Patients, n (%)
Treatment arm	1: MTX	68 (54.0)	37 (45.7)
	2: ACT-D	58 (46.0)	44 (54.3)
Age (IQR), y		24.0–35.0	22.9–33.7
Interval between evacuation and first treatment dosing (IQR), mo		1.1—2.1	1.2–2.2
Choriocarcinoma		8 (6.3)	2 (2.5)
WHO-FIGO score	0	19 (15.1)	8 (9.9)
	1	44 (34.9)	32 (39.5)
	2	26 (20.6)	20 (24.7)
	3	16 (12.7)	12 (14.8)
	4	9 (7.1)	6 (7.4)
	5	6 (4.8)	2 (2.5)
	6	6 (4.8)	1 (1.2)
Response to study treatment	Sensitive	78 (61.9)	52 (64.2)
	Resistant	48 (38.1)	28 (34.6)
	Missing	0(0)	1 (1.2)
Total (N = 207 patients)		126	81

IQR, interquartile range.

β-hCG Population Kinetic Modeling of Training Data Set Patients

Declining β-hCG measurement data retrieved for training data set patients were analyzed using the population kinetic modeling approaches. Parameter estimates and their respective coefficients of variation are reported in Table 2. The estimated means and corresponding coefficients of variation (CV) for hCG7, K, and hCGres were 3350 IU/L (CV, 247%), 0.191/d (CV, 400%), and 42.7 IU/L (CV, 37%), respectively. No significant impacts of covariates, especially treatment arm, were found on interindividual variabilityof parameter estimates.

TABLE 2.

Model parameter estimates and coefficients of variation

Parameter	Population Value	Interindividual Variability: Coefficient of Variation, %	Residual Variability, %
hCG7, IU/L	3350	247	38
K, /d	0.191	400
hCGres, IU/L	42.7	37
Box-Cox transformation parameter	0.152	N/A

The goodness-of-fit plots show that the model fit individual β-hCG profiles well (Fig. 2A, B). Visual predictive check displays that the median and 5% and 95% quantiles of observed β-hCG values across time were included within 95% CI boundaries of the median and 5% and 95% quantiles of simulated β-hCG in keeping with the good predictive performance of the model (Fig. 3).

FIGURE 2.

Goodness-of-fit plots. A, Individual predicted versus observed hCG values (IU/L). B, Observed and individual predicted hCG versus time in 4 typical patients.

FIGURE 3.

Visual predictive check of the hCG levels versus time. The red lines represent the median and 5% and 95% quantiles of the observed hCG values along with time. The blue areas are the nonparametric 95% confidence intervals for the median and 5% and 95% quantiles computed from the 1000 simulated replicates issued from the model.

Prediction of Treatment Resistance Using Derived hCGres in Training Data Set Patients

Area under the curve (AUC) values with continuous ROC tests were 0.85 (95% CI, 0.76–0.94) and 0.85 (95% CI, 0.75–0.95) in arms 1 and 2, respectively, for prediction of resistance. The optimal hCGres cutoffs for discriminating resistant versus sensitive were 7.7 IU/L in arm 1 and 74.0 IU/L in arm 2 (Table 3). When the combined training data set patients were reassessed by discrete ROC analysis after classifying them into 2 predictive groups as either high resistance risk (>7.7 IU/L in arm 1 or >74.0 IU/L in arm 2) or low risk (≤7.7 IU/L in arm 1 or ≤74.0 in arm 2) to ascertain hCGres predictive diagnostic performance at the optimal cutoffs, the AUC was 0.82 (95% CI, 0.51–1.00), with 93.8% sensitivity (95% CI, 82.4%–98.4%), 70.5% specificity (95% CI, 59.5%–79.5%), 66.2% positive predictive value (PPV), and 94.8% negative predictive value (NPV).

TABLE 3.

Predictive value of hCGres regarding resistance risk in training and tefst data sets

	Training Set					Test Set
ARM	Sensitivity, %	Specificity, %	PPV, %	NPV, %	ROC AUC	Sensitivity, %	Specificity, %	PPV, %	NPV, %	ROC AUC
1	100.0 (85.4–100.0)	60.0 (44.6–73.6)	63.6	100	0.80	95.0 (75.1–99.9)	75.0 (47.6–92.7)	82.6	92.3	0.85
2	85.0 (62.9–95.4)	81.6 (66.2–91.0)	70.8	91.2	0.83	75.0 (34.9–96.8)	72.2 (54.8–85.8)	37.5	92.9	0.74
1 + 2	93.8 (82.4–98.4)	70.5 (59.5–79.5)	66.2*	94.8*	0.82	89.3 (71.8–97.7)	73.1 (59.0–84.4)	64.1*	92.7*	0.81

^*PPV and NPV only applied to the population similar to that of the GOG-174 trial where patients are with equal probability to receive either weekly IM MTX or biweekly intravenous ACT-D.

Challenging the Model and the Predictive Value of hCGres in Test Data Set Patients to Estimate the Test’s Performance if Applied Clinically

Individual values of hCGres were estimated in test data set patients using the same model as defined above. Patient risk of resistance was predicted based on the hCGres thresholds defined above (Table 3). The outcomes regarding the predictive value of modeled hCGres in the test patient data set were similar to those observed with training set patientsVAUC, 0.81 (95% asymptotic CI, 0.73–0.89); sensitivity, 89.3% (95% exact CI, 71.8%–97.7%); specificity, 73.1% (95% exact CI, 59.0%–84.4%); PPV, 64.1%; NPV, 92.7%Vconfirming the high reproducibility of the methodological approach.

Logistic Regression for Prediction of Resistance in Test Data Set Patients

The corresponding P values for deviance and Pearson goodness-of-fit tests were 0.46 and 0.47, which indicate that this logistic model was fit. At the 0.05 level, patient response to study treatment was significantly associated with modeled hCGres and treatment arm by LRTs after adjusting treatment arm and hCGres, respectively; their corresponding values of P were 0.0002 and <0.0001. The estimated area under the ROC curve for the final logistic model was 0.88.

DISCUSSION

Management of GTN is highly reliant on serum hCG monitoring. The utility of hCGres, a modeled hCG kinetic parameter, as an early predictor of MTX resistance after 50 treatment days, suggested in the initial French study, was subsequently confirmed in a large independent cohort of British patients in 2013.¹⁰ As a consequence, assessment of the role of modeled hCGres for early treatment adjustment in patients treated with the 8-day MTX Bagshawe et al modified protocol regimen is planned within the International Society for the Study of Trophoblastic Diseases.¹¹

Although the latter MTX regimen is the most commonly used regimen in Europe, the pulsed biweekly ACT-D regimen is more commonly prescribed in the United States.^1,2,7,11,27 In the present study, we investigated the reproducibility of modeled hCGres predictive value in GTN patients enrolled in the GOG-174 trial, in which 2 regimens commonly used in North America were compared. Although the same model as reported in previous European studies could be used in the present study, the model kinetic parameter estimates had to be readjusted to the specific doses and routes of treatments given in the GOG-174 trial in the training set. Interestingly, post hoc mean population values of kinetic parameter estimates, K and hCGres (0.191/d and 42.7 IU/L, respectively), were close to those found in the British study (0.169/d and 24.7 IU/L).¹⁰

The same model could be used for MTX and ACT-D arms, but different hCGres predictive cutoffs regarding resistance risk were found. These predictive threshold values were different from those reported in the British study. This may be understood by variations in hCG decline rates induced by different drugs, administration routes, and doses. The present study meant to assess the reproducibility of hCGres predictive value in patients enrolled in the GOG-174 trial using the model previously reported but did not aim at comparing the hCG decline profiles of the 2 tested regimens.

The predictive value of hCGres regarding resistance risk was found reproducible in training and test data sets. These findings suggest that the predictive kinetic model developed in European patients treated with Bagshawe et al MTX regimen may be applicable to patients with the same disease but treated with other chemotherapies, provided adjustments are done to modeling time scale and predictive cutoffs.¹¹ Based on 3 to 4 time points measured during the first 50 treatments days, it is easily possible to characterize the patient hCG decline profile and the risk of resistance early. As suggested in the previous British study, changing to second-line treatment immediately after prediction of MTX resistance using the kinetic method may help reduce administration of unnecessary treatment cycles in resistant patients.¹⁰ Thisstrategy might also be considered in patients treated with ACT-D, which is commonly prescribed in North America. In the training data set, if patients with a high risk of resistance based on modeled hCGres were changed to second-line treatment after only 6 weeks of treatment, a total of 107 chemotherapy administrations to 29 patients, whowere later determined to be resistant and treated with second-line treatment, would have been saved. Such a strategy would reduce the duration of first-line treatment by 4 weeks in 50% of patients (and by 7 weeks in 25% of patients) and limit unnecessary use of chemotherapy. On the other hand, this approach might also have led to unnecessary treatment change in about 30% of patients, who, despite their high modeled hCGres values, were cured with first-line therapy. The strategy seems a fair one as the treatment of most patients would have been switched to MTX (preferentially the European 8-day Bagshawe et al regimen sinceweekly MTX was shown to be inferior) or ACT-D, expected to be highly curative in such patients as shown by the outcomes of the GOG- 174 trial and of another study.¹ However, these assumptions need to be validated in other independent databases.^9,27

The study presents some limitations. Although the training and test data sets showed similar predictability for the modeled hCGres regarding resistance risk, there is a degree of discrepancy for modeled hCGres predictability between 2 treatment arms. This discrepancy could be caused by unbalanced treatment arms in training and test data sets (Table 1). Furthermore, the test data set demonstrated a statistically significant association between binary response to study treatment and treatment arm after adjusting the modeled hCGres by the LRT at the 0.05 level. This implies the limited application of the marginal estimates for PPVand NPV (Table 3), which were only valid to the population similar to that of the GOG-174 trial where patients are with equal probability to receive either weekly IM MTX 30 mg/m² or biweekly intravenous ACT-D 1.25 mg/m². Although most of the patients enrolled in the GOG-174 trial were treated in different North American centers using various immunoassays, the present outcomes, especially that of the predictive value of the reported hCGres thresholds, may not necessarily be applicable in other cohorts of patients. The use of different assays may have increased the interindividual variability in hCGres estimations, although this effect may have been limited by the longitudinal assessment of hCG kinetics by the model. A validation of these thresholds in other North American databases would be warranted.

Despite these limitations, the present work is the third study that suggests that modeled kinetic parameters of hCG decline may be used to predict early the risk of resistance to front-line treatment in low-risk GTN. Although model building is a complex process, our results may have practical clinical application. An online or smart phone program integrating the model could easily calculate individual modeled kinetic parameters, such as hCGres, based on hCG levels, and predict the risk of resistance.