LATE RECTAL TOXICITY ON RTOG 94-06: ANALYSIS USING A MIXTURE LYMAN MODEL

Abstract

Purpose

To estimate the parameters of the Lyman normal-tissue complication probability (NTCP) model using censored time-to-event data for grade ≥2 late rectal toxicity among patients treated on Radiation Therapy Oncology Group (RTOG) 94-06, a dose-escalation trial designed to determine the maximum tolerated dose for 3D conformal radiotherapy (3D-CRT) of prostate cancer.

Methods and Materials

The Lyman NTCP model was fitted to data from 1010 of the 1084 patients accrued on RTOG 94-06 using an approach that accounts for censored observations. Separate fits were obtained using dose-volume histograms (DVH) for whole rectum and dose-wall histograms (DWH) for rectal wall.

Results

With a median follow-up of 7.2 years, the crude incidence of grade ≥2 late rectal toxicity was 15% (N=148). The parameters of the Lyman model fitted to DVH data, with 95% profile-likelihood confidence intervals, were TD₅₀=79.1 Gy (75.3 Gy, 84.3 Gy), m=0.146 (0.107, 0.225), and n=0.077 (0.041, 0.156). The fit based on DWH data was not significantly different. Patients with cardiovascular disease had a significantly higher incidence of late rectal toxicity (P=0.015), corresponding to a dose-modifying factor of 5.3%. No significant association with late rectal toxicity was found for diabetes, hypertension, rectal volume, rectal length, neoadjuvant hormone therapy, or prescribed dose per fraction (1.8 Gy versus 2 Gy).

Conclusions

These results, based on a large cohort of patients from a multi-institutional trial, are expected to be widely representative of the ability of the Lyman model to describe the long-term risk of grade ≥2 late rectal toxicity after 3D-CRT of prostate cancer.

INTRODUCTION

Patients undergoing external beam radiotherapy (RT) of prostate cancer may develop late rectal complications, and many investigators have sought to identify characteristics of the rectal dose-volume histogram (DVH) associated with toxicity risk. At least four studies (1–4) have analyzed late rectal data using the Lyman normal-tissue complication probability (NTCP) model (5) combined with the DVH reduction scheme of Kutcher and Burman (6). In fitting the Lyman-Kutcher-Burman (LKB) model to data, the approach used in previous studies of rectal toxicity has been to score patients as responders or non-responders according to whether or not toxicity was observed, although it is recognized that some “non-responders” would have experienced the endpoint with longer follow-up. As a consequence of these false negatives, NTCP models fitted to such data tend to underestimate the true complication risk. To address this problem, some previous studies have specified a time point for risk assessment, e.g. 18 months for the study of Rancati et al. (1) and 3 years for the study of Peeters et al. (3). However, excluding patients with shorter follow-up leads to a loss of data, and estimates of risk at specified time points can be substantially lower than the ultimate long-term risk level, depending on the time course over which toxicity appears.

The goal of the present study was to fit the LKB model to late rectal toxicity data using a technique that explicitly takes into account the possibility of censored events, and thereby avoids false negatives as well as the loss of data that results from specification of a minimum follow-up time (7). The generalized LKB model can be called the “mixture Lyman model” by analogy with “mixture cure” models from the statistical literature (8). Such models assume that some patients will experience the event of interest with sufficiently long follow-up whereas others will not, and the goal is to estimate the risk that a patient belongs to the former category (the NTCP). In fitting the model to data, patients without toxicity at last follow-up are regarded as having some probability of experiencing the endpoint at a later time, depending on the current length of follow-up as well as on the DVH and other relevant risk factors. As noted previously (7), censored cases include patients who die without toxicity, which implies that NTCP estimates from the mixture Lyman model are expected to remain accurate in settings where survival is improved.

The data analyzed in this study are from RTOG 94-06, a large multi-institutional trial designed to establish the maximum tolerated dose during 3D-conformal RT (3D-CRT) of clinically localized (T1–T3) adenocarcinoma of the prostate. Participating institutions were required to meet specific criteria for technology and quality assurance, although each institution used its own in-house conformal techniques for treatment. Patients enrolled on the trial were followed regularly at prescribed intervals and scored prospectively according to strictly defined toxicity criteria. The uniformity in the collection and scoring of data from a large number of patients, combined with the variation in treatment designs among participating institutions, makes the data from RTOG 94-06 an excellent resource for investigating the dose-volume response of the rectum.

METHODS AND MATERIALS

Patient accrual and treatments on protocol RTOG 94-06

Protocol RTOG 94-06 accrued 1084 patients, from 42 different institutions, between 1994 and 2000. Details of the trial and results of primary analyses have been presented elsewhere (9–12). Briefly, the trial included 5 prescription dose levels: 68.4 Gy, 73.8 Gy and 79.2 Gy (levels I– III) given in 1.8-Gy fractions, and 74.0 Gy and 78.0 Gy (levels IV–V) given in 2-Gy fractions. Patients were stratified into 3 groups according to the estimated risk of seminal vesicle (SV) invasion (13). Patients in group 1 were treated to the prostate only, patients in group 2 were treated to the prostate and bilateral SVs for the first 55.8 Gy (levels I–III) or 54 Gy (levels IV–V) and to the prostate only for the remainder of treatment, and group 3 patients were treated to the prostate plus bilateral SVs throughout RT. Neoadjuvant androgen suppression was permitted if it began 2–6 months before study registration.

DVH data

Treatment planning CT scans were acquired in the same position and under the same conditions (e.g. full versus empty bladder) as for treatment. The rectum was empty unless contrast was used, and was contoured from the level of the ischial tuberosities to the rectosigmoid flexure. The rectal DVH was computed for rectum as a solid volume based on the dose matrix submitted by the participating institution. For a few cases in which the dose matrix did not encompass the entire rectum, the volume of rectum outside the dose matrix was added to the 0-Gy dose bin of the DVH. For patients who did not complete planned treatment, the rectal DVH represented the dose actually delivered.

A dose-wall histogram (DWH) was calculated for an approximate rectal wall structure obtained by retracting the outer rectal contour inward by 3mm. As described previously (14), the choice of 3-mm is supported by a study in which rectal wall thicknesses were measured by ultrasound (15).

Patient follow-up and toxicity scoring

Following treatment, patients were followed every 3 months for the first year, every 4 months during the second year, every 6 months during the next 3 years, and annually thereafter. Toxicity was scored using RTOG criteria (16). Consistent with previous analyses of these data (9–12), late rectal toxicity was defined as toxicity starting or persisting at least 120 days after the start of radiotherapy. Time to grade ≥ 2 late rectal toxicity was computed from the RT start date, and patients not experiencing the endpoint were censored at the date of last follow-up.

The data analyzed here were extracted from the RTOG database in October 2007. This retrospective secondary analysis was approved by the RTOG Publications Committee and by the Institutional Review Boards of UTMDACC, the Washington University Medical Center, and the American College of Radiology.

Data analysis

Data were analyzed using the mixture Lyman model (7), in which the NTCP after indefinitely long follow-up is modeled using the standard LKB formula, with parameters TD₅₀, m, and n:

$$\text{NTCP}=\frac{1}{\sqrt{2\pi }}·{\displaystyle {\int }_{-\mathrm{\infty }}^t{e}^{-u^2/2}\mathit{\text{du}}}$$ (1)

where

$$t=\frac{D_{\mathit{\text{eff}}}-{\mathit{\text{TD}}}_{50}}{m·{\mathit{\text{TD}}}_{50}}$$ (2)

and

$$D_{\mathit{\text{eff}}}={(\mathrm{\Sigma }{(D_i)}^{1/n}·v_i)}^n$$ (3)

In the expression for effective dose, given by equation (3), D_i is the dose to relative organ volume v_i, and the sum extends over all dose bins in the DVH (17).

The mixture Lyman model also includes a formula for the distribution of times at which toxicity occurs among patients who will experience the endpoint. In the present study, latent times were modeled using a lognormal distribution, which has parameters μ and σ and probability density function

$$f(\tau )=\frac{1}{\sigma \tau \sqrt{2\pi }}·e^{-{(\text{ln}\tau -\mu )}^2/2{\sigma }^2}$$ (4)

The mixture Lyman model was fitted to data using maximum likelihood (ML) analysis (18). As described in detail elsewhere (7), the contribution to the likelihood for a patient experiencing toxicity at time τ is NTCP· f (τ), while for a patient followed to time τ without experiencing toxicity, the contribution to the likelihood is 1−NTCP· F(τ), where F(τ) is the cumulative distribution function corresponding to f(τ).

To assess goodness-of-fit, the squared residual was computed for each patient as (O–E)², where O is the observed outcome (=1 for patients experiencing the endpoint and 0 otherwise), and E= NTCP· F(τ) is the expected probability of observing the endpoint within the available follow-up. A simulation study was performed using 10,000 iterates to estimate the distribution in the sum of squared residuals (SSR), assuming the fitted model to be correct. The proportion of iterates for which the simulated SSR was larger than the SSR computed from the observed data was used as a measure of significance for model fit.

Confidence intervals for the ML parameter estimates were computed using the profile likelihood method (18). The method of Kaplan and Meier (KM) was used to compute freedom from toxicity as a function of time. Incidence levels of toxicity were computed as 1 minus the KM estimate of freedom from toxicity, with standard errors calculated using the method of Greenwood (19).

Clinical characteristics were included in the model by representing their effects as dose-modifying factors on D_eff (3,7). Factors investigated for their ability to improve the model fit were: patient age; presence of cardiovascular disease, diabetes or hypertension at the time of study registration; rectal volume; rectal length; neoadjuvant hormonal therapy; and prescribed dose per fraction (1.8 Gy versus 2 Gy). The statistical significance of each factor was assessed using the likelihood ratio test.

RESULTS

Study cohort

Of the 1084 patients enrolled in RTOG 94-06, 1010 patients (93%) were included in the present analysis (Table 1). Twenty-nine patients withdrew consent or were determined to be ineligible, 36 had missing or incomplete dosimetry data, 6 were excluded because they had treatment breaks longer than 2 weeks (range 22–72 days), 2 were excluded because of pre-existing bowel toxicities, and 1 died during the course of RT, unrelated to toxicity. Four patients in the analysis did not complete treatment as planned because of refusal (N=2) or other reasons (N=2). There were 9 patients for whom the 0-Gy dose bin of the DVH was adjusted to take into account the volume of rectum outside the dose matrix; the resulting increase in rectal volume ranged from <0.1% to 22% (median 7%). At the time of data extraction for the present analysis (October 2007), median patient follow-up was 7.2 years (range 3 months to 12.4 years). Table 1 shows follow-up by prescription dose level. The crude incidence of grade ≥2 late rectal toxicity was 15% (N=148); 121 patients had grade 2 toxicity, 25 had grade 3, and 2 had grade 4. Among the patients with grade ≥2 late rectal toxicity, approximately half (N=73) had earlier grade 1 toxicity, at times ranging from 1.3 months to 7.0 years (median 1.1 years) prior to detection of grade ≥2 toxicity. Another 299 patients experienced grade 1 late rectal toxicity without progressing to grade ≥2 during follow-up.

Table 1.

Numbers of patients from RTOG 94-06 included in the present analysis by prescription dose level and disease group.

	Disease Group*			Follow-up (years)***
Dose level**	1	2	3
I	69	30	4	9.4 (1.5 – 12.4)
II	98	106	91	8.3 (0.3 – 12.0)
III	100	60	0	8.9 (1.0 – 10.6 )
IV	115	136	0	7.3 (0.9 – 8.7)
V	109	92	0	6.1 (0.5 – 7.3)

^*Disease groups: Group 1 = Clinical stages T1b-c or T2a-b with PSA + ([Gleason − 6] × 10) ≤ 15; Group 2 = Clinical stages T1b-c or T2a-b with PSA + ([Gleason − 6] × 10) > 15; Group 3 = Clinical stage T3 with PSA < 70

^**Prescribed dose levels: Levels I–III = 68.4 Gy, 73.8 Gy, and 79.2 Gy, respectively, given in 1.8 Gy fractions; Levels IV–V = 74 Gy and 78 Gy, respectively, given in 2 Gy fractions.

^***Median patient follow-up and range is shown for each prescription dose level

Fit of the mixture Lyman model

The parameter estimates from the fit of the mixture Lyman model to the grade ≥2 late rectal toxicity data using DVHs from solid rectum are listed in Table 2, and the model fit is illustrated in Figure 1. Figure 1A shows freedom from toxicity in subsets of patients grouped according to estimated NTCP (equation 1), which ranged from 0% to 41% (median 14%). Figure 1B shows incidence of toxicity at 8 years in subgroups of patients grouped by D_eff (equation 3). There was no evidence for lack of fit of the model using the goodness-of-fit test described in Methods (P=0.189).

Table 2.

Parameter estimates of the mixture Lyman model for grade ≥2 late rectal toxicity fitted using rectal DVH data.

Parameter	Estimate	Confidence interval*
TD50	79.1 Gy	(75.3 Gy – 84.3 Gy)
m	0.146	(0.107 – 0.225)
n	0.077	(0.041 – 0.156)
μ	0.442	(0.312 – 0.587)
σ	0.756	(0.665 – 0.877)

^*95% profile-likelihood confidence intervals are shown.

Figure 1.

Panel A) Freedom from grade ≥2 late rectal toxicity in patient subgroups defined by NTCP (equations 1–3), computed using the parameter estimates in Table 2. Panel B) Incidence of grade ≥2 late rectal toxicity at 8 years in each of 10 subgroups of 101 patients each, grouped by D_eff (equation 3), computed using n=0.077. Points are plotted at the mean value of D_eff per subgroup. Horizontal error bars show ±1 standard deviation; vertical error bars show ±1 standard error.

A fit of the model using DWH data from rectal wall was found to be very similar to the fit based on DVH data from solid rectum. NTCP values predicted by the two fits differed by <1% on average (range ± 6%, with 96% of patients having NTCP estimates that differed by <3%). A bootstrap analysis in which DVH- and DWH-based models were fitted to each of 1000 bootstrap datasets obtained by sampling from the original data with replacement (N=1010 for each bootstrap sample) also found no significant difference between the two fits.

Time to late rectal toxicity

The distribution of observed times to grade ≥2 late rectal toxicity, shown in Figure 2, did not differ significantly from a lognormal distribution (P = 0.263, Shapiro-Wilk W test), supporting the choice of the lognormal latent-time function (equation 4) in the generalized LKB model. The fitted latent-time model, also shown in Figure 2, predicts that 52% of grade ≥2 late rectal toxicities occur later than 18 months, 37% occur later than 2 years, 19% occur later than 3 years, and 11% occur later than 4 years. Although average follow-up in the present cohort is relatively long (>7 years), the model predicts that an additional 10 events would have occurred if all patients had complete follow-up.

Figure 2.

Distribution of times to grade ≥2 late rectal toxicity among patients experiencing the endpoint (N=148). The dashed curve shows the fitted latent-time model (equation 4; parameter values in Table 2).

To test whether latent time varied with severity of injury, the model was re-fitted with the parameter μ (equation 4) replaced by a linear function of NTCP. Although there was a trend toward shorter latency (smaller μ) with increasing NTCP, the effect was not statistically significant (P=0.107, likelihood ratio test).

Clinical factors

Table 3 lists the significance of each variable tested as a potential dose-modifying factor. Patients with cardiovascular disease had a significantly increased risk of grade ≥2 late rectal toxicity (P=0.015), with a dose-modifying effect equal to a 5.3% increase in D_eff. This corresponds to a median increase in NTCP of 7 percentage points (range 0.2% to 12%) over the range of D_eff values in the present cohort. None of the other factors (age, diabetes, hypertension, rectal volume, rectal length, neoadjuvant hormone therapy, or dose per fraction) had a significant effect on late rectal toxicity.

Table 3.

Effect of including covariates in the generalized Lyman model.

Factor	Values*	N**	DMF+	P++
Age (years)	69 (41 – 84)	1010	0.2%/yr	0.231
Cardiovascular disease	Yes	315	5.3%	0.015
	No	622
	Unknown	73
Diabetes	Yes	129	−0.3%	0.906
	No	809
	Unknown	72
Hypertension	Yes	432	0.6%	0.770
	No	505
	Unknown	73
Rectal volume (mL)	87.8 (21.3 – 446.1)	1010	<0.1%/mL	0.606
Rectal length (cm)	10.5 (4.5 – 16.8)	1010	0.3%/cm	0.637
Hormone therapy	Yes	364	−0.8%	0.640
	No	646
Dose per fraction	2 Gy	452	3.4%	0.088
	1.8 Gy	558

^*Median and range are shown for patient age, rectal volume, and rectal length.

^**N = number of patients

⁺DMF = Dose-modifying factor, expressed as a change in D_eff

⁺⁺Patients with missing data are excluded from analysis

DISCUSSION

The recent QUANTEC review (QUalitative Analysis of Normal Tissue Effects in the Clinic) identified 5 published studies in which the LKB model was fitted to late rectal toxicity data (20). One was an abstract with preliminary results of the present analysis, based on data extracted from the RTOG database approximately 2 years earlier than the data analyzed here (21). The other four studies were those of Cheung et al. (2), Peeters et al. (3), Rancati et al. (1), and Söhn et al. (4). As noted in the QUANTEC review, most of the published LKB parameter estimates for grade ≥2 late rectal toxicity or rectal bleeding are in good agreement with one another, despite some variation in the study endpoints. A meta-analysis yielded “best overall” LKB parameter estimates of TD₅₀=76.9 (73.7, 80.1) Gy, m=0.13 (0.10, 0.17) and n=0.09 (0.04, 0.14). The only source of heterogeneity was the study of Cheung et al. (2), for which TD₅₀ was considerably smaller (53.6 Gy) and n considerably larger (3.91) than for the other studies.

Figure 3 shows how the NTCP values derived here compare with NTCP values computed for patients in the present study using previously published LKB parameters. As illustrated in Figures 3A–3C, NTCP estimates obtained using the parameters in Table 2 are, with very few exceptions, higher than those obtained using the parameter estimates of Peeters et al. (3), Rancati et al. (1), or Söhn et al. (4). Compared to present estimates, NTCP values computed using the parameters of Peeters et al. are a median of 9 percentage points (range 0% to 19%) lower, those computed using the model fit of Rancati et al. are 10 percentage points (range −6 to 24) lower, and those computed using the parameters of Söhn et al. are 7 points (range 0 to 8) lower.

Figure 3.

Panel A) NTCP computed for each patient in RTOG 94-06 using the parameter estimates of Peeters et al. (3) for late rectal bleeding (TD₅₀=81 Gy, m=0.14, n=0.13), plotted against NTCP computed using the current parameter estimates (Table 2). Panel B) NTCP computed using the parameters of Rancati et al. (1) for grade ≥2 rectal bleeding among patients without prostatectomy (TD₅₀=75.7 Gy, m=0.14, n=0.24), plotted against current NTCP. Panel C) NTCP computed using the parameters of Söhn et al. (4) for grade ≥2 late rectal toxicity scored using CTCAE version 3.0 criteria (22) (TD₅₀=78.4 Gy, m=0.108, n=0.08), plotted against current NTCP. Panel D) NTCP computed using the “best overall” LKB parameter estimates from the QUANTEC meta-analysis (18) (TD₅₀=76.9 Gy, m=0.13, n=0.09), plotted against current NTCP.

To some extent, these differences in predicted NTCP are explained by differences in the toxicity endpoints. In the study of Peeters et al., the endpoint was severe rectal bleeding requiring laser treatment or transfusion, and for the studies of Rancati et al. and Söhn et al. it was grade ≥2 late rectal bleeding, whereas in RTOG 94-06, the grade ≥2 late rectal endpoint included any intermittent bleeding, as well as moderate diarrhea and colic, >5 bowel movements daily, or excessive rectal mucus (16).

However, the differences in estimated NTCP shown in Figures 3A–3C are also due, in part, to the use of the mixture Lyman model in the present study. Our analysis takes censored observations into account and predicts complication probabilities after indefinitely long follow-up. In contrast, the studies of Rancati et al. and Peeters et al. estimated rectal toxicity at the 18-month and the 3-year time points, respectively, Patients with shorter follow-up were excluded from analysis, and patients were scored as non-bleeders if they experienced toxicity at later times. The study of Söhn et al. did not impose a minimum follow-up time, but patients without complications during follow-up were counted as non-responders although some might have experienced toxicity later. Consistent with these examples, we expect NTCP estimates from our study to be higher than those from studies that do not take censoring into account.

Figure 3D shows the comparison between NTCP predictions from the current study and those obtained using the “best overall” parameter estimates from the QUANTEC meta-analysis (20). The agreement is good (median difference of 1 percentage point in NTCP, range −6% to 3%), as expected from the fact that preliminary results of the present study (21) represented the largest contribution, in terms of patient numbers, to the meta-analysis. The QUANTEC estimates are somewhat more conservative than the estimates derived here, in the sense that the QUANTEC parameters predict higher NTCP values for 20% of the patients in RTOG 94-06, and the discrepancy is largest among patients with the highest estimated values of NTCP.

Among the clinical factors investigated here for their association with late rectal toxicity, only cardiovascular disease was found to significantly improve the fit of the generalized Lyman model (Table 3). In contrast, previous studies have found no significant relationship between cardiovascular disease and rectal bleeding (23,24). However, Choe et al. found that grade ≥3 rectal bleeding was significantly worse among patients receiving warfarin or clopidogrel (25). Their finding suggests that perhaps anticoagulant use, and not cardiovascular disease per se, is the relevant factor, although Fiorino et al. (26) found no effect of anticoagulants or antiaggregants in their analysis of grade ≥2 bleeding. Information about anticoagulant use was not available for the current analysis.

Patients with diabetes were not found to be at higher risk for late rectal toxicity in the present study, consistent with most previous reports (23,24,26–29), although some studies have found an association (30–32). Our finding regarding a lack of significance for hypertension is also consistent with previous reports (26,27), although one study found a protective effect (28). Similarly, the majority of previous studies have also found no relationship between patient age and late rectal toxicity (23,24,27–30), although one study found a significant adverse effect (31) and one observed a protective effect (33) of advanced age.

As reported previously for the patients on RTOG 94-06 (34), there is no significant difference in late rectal toxicity among patients receiving neoadjuvant hormone therapy. Although a few studies have reported an effect of hormone treatment (28,32), most studies have also found no association with late rectal toxicity (23,26,27,29,31,33). There could of course be differing effects of hormone therapy depending on the particular agents used and on dosage and timing.

A recent analysis of the data from RTOG 94-06 by prescription dose level and disease group reported a higher incidence of grade ≥2 late rectal toxicity among patients receiving 78 Gy in 2 Gy per fraction (level V) (35). Figure 4 illustrates that this finding is consistent with the dose-volume effects described by the LKB model. Patients in level V were treated with significantly larger margins (35), which resulted in increased volumes of rectum exposed to high doses and therefore a substantially increased average risk of rectal complications.

Figure 4.

Incidence of grade ≥2 late rectal toxicity at 5 years by prescription dose level in RTOG 94-06. Points are plotted at the mean value of NTCP per dose level. Horizontal error bars show ±1 standard deviation; vertical error bars show ±1 standard error.

Finally, it should be noted that the rectal DVHs used in the current analysis represent total physical dose to rectum, with no corrections made for differences in dose per fraction. Rancati et al. reported no effect on LKB parameter estimates of correcting the rectal DVHs for fractionation (1). However, future work will test whether use of fractionation-corrected DVHs improves LKB model accuracy for the RTOG 94-06 cohort. In addition, the present study does not take into account organ motion occurring during RT. Therefore, it may be necessary to revise the estimated parameters of the LKB model for settings in which daily localization techniques are used.