Evaluation of dose prediction errors and optimization convergence errors of deliverable-based head-and-neck IMRT plans computed with a superposition∕convolution dose algorithm

Abstract

The purpose of this study is to evaluate dose prediction errors (DPEs) and optimization convergence errors (OCEs) resulting from use of a superposition∕convolution dose calculation algorithm in deliverable intensity-modulated radiation therapy (IMRT) optimization for head-and-neck (HN) patients. Thirteen HN IMRT patient plans were retrospectively reoptimized. The IMRT optimization was performed in three sequential steps: (1) fast optimization in which an initial nondeliverable IMRT solution was achieved and then converted to multileaf collimator (MLC) leaf sequences; (2) mixed deliverable optimization that used a Monte Carlo (MC) algorithm to account for the incident photon fluence modulation by the MLC, whereas a superposition∕convolution (SC) dose calculation algorithm was utilized for the patient dose calculations; and (3) MC deliverable-based optimization in which both fluence and patient dose calculations were performed with a MC algorithm. DPEs of the mixed method were quantified by evaluating the differences between the mixed optimization SC dose result and a MC dose recalculation of the mixed optimization solution. OCEs of the mixed method were quantified by evaluating the differences between the MC recalculation of the mixed optimization solution and the final MC optimization solution. The results were analyzed through dose volume indices derived from the cumulative dose-volume histograms for selected anatomic structures. Statistical equivalence tests were used to determine the significance of the DPEs and the OCEs. Furthermore, a correlation analysis between DPEs and OCEs was performed. The evaluated DPEs were within ±2.8% while the OCEs were within 5.5%, indicating that OCEs can be clinically significant even when DPEs are clinically insignificant. The full MC-dose-based optimization reduced normal tissue dose by as much as 8.5% compared with the mixed-method optimization results. The DPEs and the OCEs in the targets had correlation coefficients greater than 0.71, and there was no correlation for the organs at risk. Because full MC-based optimization results in lower normal tissue doses, this method proves advantageous for HN IMRT optimization.

INTRODUCTION

Intensity-modulated radiation therapy (IMRT) dose calculation is a two-step process. During the first step, radiation fluence exiting the treatment head and incident upon the patient is predicted. In the second step, the dose deposition to the patient∕phantom resulting from the predicted incident fluence is computed. Radiation transport and energy deposition approximations inherent to the computation algorithms used in each step lead to differences between the actual and the estimated doses, termed dose prediction errors (DPEs).^{1, 2, 3, 4} In practice, the impossibility of measuring the actual doses received by a patient prevents exact evaluation of true DPEs. Therefore, an alternative approach of substituting a highly accurate dose calculation algorithm, such as Monte Carlo (MC), for the actual dose is often^{2, 5, 6, 7, 8, 9, 10, 11, 12, 13} used to evaluate DPEs, given that the MC method itself is well benchmarked.^{14, 15, 16}

Sophisticated IMRT plans that require complex intensity patterns are generated through an inverse optimization process. The goal of the optimization is to achieve an optimal treatment plan, subject to many variables, objectives, and constraints. Optimization convergence errors (OCEs)^{3, 17, 18} result from imperfect convergence of the optimization algorithm (“optimizer”) to the global or local optimal IMRT solution. OCEs can be caused by the lack of feedback in the optimization loop (e.g., separating plan optimization and leaf sequencing processes) or by the presence of DPEs^{2, 8, 12, 16, 19, 20} in the dose calculation algorithms (e.g., fluence modeling errors or incomplete modeling of tissue heterogeneities) used in the IMRT optimization.

Previous studies have evaluated the DPEs^{1, 12} and the OCEs³ in IMRT optimization as either a combination of fluence and heterogeneity effects² or separately for fluence estimation¹ and heterogeneity^{1, 12} corrections. We recently developed a mixed MC∕analytic dose calculation method¹⁶ to compute IMRT dose distributions. Briefly, in this method, accurate MC-derived energy fluence transmission maps are used to estimate the photon fluence modulation whereas an analytic dose computation algorithm built into a commercial treatment planning system¹ (PINNACLE³ v7.6c) is used to compute the patient dose. This approach enables the use of highly accurate incident fluence estimates for analytic dose calculation algorithms. Furthermore, it allows evaluation of the DPEs and OCEs inherent to the analytic dose calculation algorithms with respect to MC dose calculations because the same incident fluence estimates are used for each algorithm.

The purpose of this work is to evaluate the DPEs and OCEs that result from the use of superposition∕convolution (SC) and MC dose calculations used in deliverable IMRT optimization for head-and-neck (HN) patients. This work differs from the MC deliverable optimization of Dogan et al.³ in that Dogan’s deliverable optimization used analytic models to predict the multileaf collimator (MLC)-induced fluence modulation when the SC algorithm was used for patient dose calculations. Thus, Dogan’s MC-based deliverable optimization corrected for OCEs due to DPEs in the fluence prediction and in the patient dose computation (the difference between using SC and MC for the patient dose calculation). In this work, the DPE caused by fluence prediction is avoided through use of MC-generated fluence with the SC dose algorithm. Therefore, this work singles out and quantifies the improvement in the use of a MC dose algorithm over a SC algorithm for the patient dose calculation. Furthermore, whereas Dogan et al.³ presented patient-by-patient analysis, this work uses statistical tests to determine the overall likely clinical significance of using MC during IMRT optimization and correlates DPEs to OCEs.

MATERIALS AND METHODS

IMRT plans

A retrospective investigation was performed using 13 HN IMRT patient plans. The locally advanced cases were planned according to an IMRT dose-escalation protocol using a simultaneous integrated boost (SIB) technique.²¹ The site was chosen because the treatments were achieved with highly modulated fields in the presence of heterogeneities (i.e., air cavities, bones) and organs at risk, such as spinal cord, brain stem, and parotid glands, in close proximity to the targets. This highly heterogeneous geometry of the HN region is where DPEs and OCEs were expected to be clinically significant.

The plans consisted of six or nine coplanar, split,²² dynamic IMRT, 6 and∕or 18 MV beams. Complete details of the SIB HN protocol, the dose prescriptions, the targets, and the clinical treatment plans are reported elsewhere.²¹ The clinical outcomes and analysis of the clinically applied plans are reported elsewhere.^{2, 23} This study uses the same target volumes, dose prescriptions, beam energies and arrangements, and planning criteria as the clinical plans. Briefly, three dose levels were used for the gross tumor volume (GTV): 68.1, 70.8, and 73.8 Gy. The clinical tumor volume (CTV), formed by expanding the GTV by approximately 1 cm, was planned with a dose of 60 Gy to 95% of the CTV volume. The planning objectives also included sparing of the parotid glands, where the volume receiving more than 30 Gy was kept below 50%, as well as sparing of the cord and brain stem, where the maximum doses were constrained by 45 and 55 Gy, respectively.

Optimization process

The IMRT optimization minimized the plan quality scores for dose-volume-based quadratic objective functions in an in-house IMRT program, the implementation of which has been described in detail.^{14, 15, 24, 25} This study used a three-stage optimization sequence. In the first stage, preoptimization, the objective function was evaluated with doses computed from nondeliverable idealized fluence maps. Generated at each iteration step by the optimizer, these maps did not contain any information about the effect or limitations of the MLC on the dose delivery. Preoptimization reduces the number of iterations of later optimization stages by providing those stages a good starting point.

In the second optimization stage, Mixed^Opt, beam-by-beam idealized fluence maps from the preoptimization served as a starting point. This stage used deliverable-based optimization, which has been described in the literature.^{3, 15, 26} Briefly, during the deliverable optimization (in contrast to the nondeliverable optimization), desired intensities were converted to MLC leaf sequences at each iteration step. Fluence maps resulting from the MLC leaf sequences were used for the dose evaluation during the optimization. Thus, deliverable optimization resulted in dose distributions constrained by the MLC position and motion limitations and incorporated radiation transmitted through and scattered from the MLC. Deliverable-based optimization has been described in further detail.^{3, 15, 26} A SC dose calculation algorithm with MC-computed fluence modulation $(D_{\mathrm{SC}}\left({\mid \Psi \phantom{\mid }}_{\mathrm{MC}}\right)=D_{\mathrm{SC}}^{{\Psi }_{\mathrm{MC}}})$ was used during the Mixed^Opt iterations. The $D_{\mathrm{SC}}^{{\Psi }_{\mathrm{MC}}}$ method has been described in detail.^{14, 16} Briefly, in $D_{\mathrm{SC}}^{{\Psi }_{\mathrm{MC}}}$, a MC algorithm^{14, 16} was used to transport particles through the treatment head with and without the MLC, yielding energy fluence transmission matrices, Ψ_MC. These matrices were then used to modulate the incident fluence for the PINNACLE³ (v. 7.6c) SC dose calculation algorithm.

In the third stage, a MC deliverable-optimization technique (MC^Opt) was used in the final IMRT optimization.^{3, 26} The MC^Opt was started from the final solution of the Mixed^Opt method. Again, at each iteration the optimizer-generated fluence maps were converted into MLC leaf trajectories. In the MC^Opt method, MC simulation was used for both the fluence and the dose calculations. For the MC dose computations, particles sampled from a saved phase space²⁷ were transported through the beam-defining jaws using BEAM,²⁸ through the MLC with a published MC method,¹⁴ and through the patient CT data set with the VMC++^{29, 30} MC algorithm. In VMC++, absorbed dose to water was directly scored³¹ so that dose results would be directly comparable to the doses computed with PINNACLE.

A sample of the convergence of the plan quality score for the two IMRT optimizations, together with the effects of the DPEs and OCEs, are presented in Fig. 1. The objective function was based on dose-volume constraints with a gradient-based optimization, which started from a set of uniform intensities. The absolute initial intensity distribution was normalized such that the mean target dose matched the prescription. Additional details and the utilization of the IMRT cost function used in the optimization process (data not shown) have been previously described.²⁵ All optimization methods used the same optimization algorithm, thus separating the dose calculation process from the topographical search of the IMRT parameter space. DPEs attributed to the differences between the SC and the MC patient dose algorithms were quantified by recomputing (using the same MLC leaf-sequence files) with MC (MC^Opt,0) the Mixed^Opt solution (Mixed^{Opt,Converged}) and comparing the computed dose distributions. In evaluation of the DPEs, MC^Opt,0 results were used as a reference standard. OCEs attributable to the differences between using a SC and a MC patient dose calculation algorithm during optimization were quantified by further optimizing with MC^Opt (starting with MC^Opt,0, defined earlier) and comparing MC^Opt,0- and MC^{Opt,Converged}-computed doses. The OCE evaluation isolates the effect of using a SC instead of a MC dose calculation algorithm during the optimization on “achieved” (MC-computed) doses.

Figure 1.

The effects of DPEs and OCEs on a typical plan score convergence. Mixed^{Opt, Converged} is the optimization solution achieved with the mixed optimization method. MC^Opt,0 is the optimization solution for the first iteration, and MC^{Opt, Converged} is the final solution, when full MC is used in the IMRT optimization. When Mixed^{Opt, Converged} is reevaluated by MC (MC^Opt,0), the difference MC^Opt,0−Mixed^{Opt, Converged} quantifies the effect of the DPEs. The difference MC^{Opt, Converged}−MC^Opt,0 quantifies the effect of the OCEs, demonstrating that the presence of DPEs may result in OCEs.

Analysis

DPEs and OCEs were evaluated based on target and critical structure dose-volume histogram (DVH) indices. The DVH indices are numerically equal to a dose covering a certain (absolute or fractional) volume of a given anatomic structure. The HN targets of interest were the GTVs, the CTVs, and the nodal volumes, whereas the critical structures were the spinal cord, the brain stem, and the parotid glands (parotids). The evaluated indices were the same as those used for plan optimization, clinical decision-making, and previous dosimetric analyses.^{1, 2, 21}

The evaluated dose indices were GTVs D_98%, CTVs D_95%, nodal D_90%, cord and brain stem D_02%, and parotid D_50%. For each dose index, the DPE was defined as

$${\mathrm{DPE}}_{\text{fractional}\text{volume}}^{{\text{Mixed}}^{\mathrm{Opt}}}={\text{Dose}}_{\text{fractional}\text{volume}}^{{\mathrm{MC}}^{\mathrm{Opt},0}}-{\text{Dose}}_{\text{fractional}\text{volume}}^{{\text{Mixed}}^{\mathrm{Opt},\text{Converged}}},$$

where ${\text{Dose}}_{\text{fractional}\text{volume}}^{\text{Method}}$ is the dose index to the fractional volume computed with the calculation Method equal to MC^opt,0 or Mixed^Opt,Converge For example, the DPE for the GTV dose index D_98% is ${\mathrm{DPE}}_{\mathrm{GTV},{\mathrm{D}}_{98}}^{{\text{Mixed}}^{\mathrm{Opt}}}={\mathrm{GTVsD}}_{\mathrm{GTV},{\mathrm{D}}_{98}}^{{\mathrm{MC}}^{\mathrm{Opt},0}}-{\mathrm{GTVsD}}_{\mathrm{GTV},{\mathrm{D}}_{98}}^{{\text{Mixed}}^{\mathrm{Opt},\text{Converged}}}$. Similarly, the OCE was defined as ${\mathrm{OCE}}_{\text{fractional}\text{volume}}^{{\mathrm{MC}}^{\mathrm{Opt},0}}={\text{Dose}}_{\text{fractional}\text{volume}}^{{\mathrm{MC}}^{\mathrm{Opt},\text{Converged}}}-{\text{Dose}}_{\text{fractional}\text{volume}}^{{\mathrm{MC}}^{\mathrm{Opt},0}}$. The above-presented equations can be normalized to allow looking at fractional (or percentage) changes with respect to the reference dose. Thus, the fractional DPEs (fDPEs) can be expressed as

$${\mathrm{fDPE}}_{\text{fractional}\text{volume}}^{{\text{Mixed}}^{\mathrm{Opt}}}={\mathrm{DPE}}_{\text{fractional}\text{volume}}^{{\text{Mixed}}^{\mathrm{Opt}}}∕{\text{Dose}}_{\text{fractional}\text{volume}}^{{\mathrm{MC}}^{\mathrm{Opt},0}}=1-{\text{Dose}}_{\text{fractional}\text{volume}}^{{\text{Mixed}}^{\mathrm{Opt},\text{Converged}}}∕{\text{Dose}}_{\text{fractional}\text{volume}}^{{\mathrm{MC}}^{\mathrm{Opt},0}},$$

where the normalization is performed with respect to the reference dose computation method. Solving for the dose index ratio results in

$${\text{Dose}}_{\text{fractional}\text{volume}}^{{\text{Mixed}}^{\mathrm{Opt},\text{Converged}}}∕{\text{Dose}}_{\text{fractional}\text{volume}}^{{\mathrm{MC}}^{\mathrm{Opt},0}}=1-{\mathrm{DPE}}_{\text{fractional}\text{volume}}^{{\text{Mixed}}^{\mathrm{Opt},\text{Converged}}}∕{\text{Dose}}_{\text{fractional}\text{volume}}^{{\mathrm{MC}}^{\mathrm{Opt},0}}=1-{\mathrm{fDPE}}_{\text{fractional}\text{volume}}^{{\text{Mixed}}^{\mathrm{Opt},\text{Converged}}}.$$

Note, if fDPE is zero, then ${\text{Dose}}_{\text{fractional}\text{volume}}^{{\text{Mixed}}^{\mathrm{Opt},\text{Converge}}}={\text{Dose}}_{\text{fractional}\text{volume}}^{{\mathrm{MC}}^{\mathrm{Opt},0}}$. Casting the definitions of the DPEs and the OCEs in the form of fractional quantities and dose index ratios eases the intercomparisons because various patients have various dose prescriptions. Note that the dose ratios have no units and their values are near unity.

An equivalence test was used to determine the minimum dose interval around the reference dose index values such that t tests concluded that the reference- and test-dose index values used in the DPE and OCE evaluations were equivalent with p<0.05. The equivalence test was performed for each dose index using two one-tailed paired t tests.³² The dose interval was initially set to zero, then the t and p values computed. The p value represented the probability observing a value greater than t for the set of data observations assuming normally distributed data.³³ The dose interval was progressively increased in 0.01 cGy steps until the statistical tests indicated equivalence between the methods with p>0.05. The percentage of the dose interval with respect to the dose index value for the reference dose index was then compared with 3% to determine if it was clinically significant. All statistical calculations were performed with the built-in functions in the Microsoft EXCEL software package.

In addition to the DPE and OCE effect on the IMRT plans, correlations between the two types of errors were investigated. As a result of the analysis, correlation coefficients were derived.

RESULTS

To evaluate the impact of a DPE or a OCE on the quality of the IMRT plan, a threshold of 3% difference in each dose index was used to state clinical significance. This value was based on the results from several studies,^{34, 35, 36} concluding that for 3D conformal radiation therapy, dose differences of 3%–7% have an observable clinical effect.

DPE

The results from the per-patient DPE evaluation are presented in Fig. 2. The results were normalized to Dose^MCOpt,0 to allow a direct comparison of the results for various patients (who had slightly different dose levels). Seven of 78 dose indices evaluated for the HN site were outside the ±3% limit. Two of the tallied indices were in the targets (nodal volumes for patients 1 and 4; Fig. 2), and the other five were in the critical structures. Notably, the dose index differences in the OARs indicate dose underestimation by the mixed optimization method, but the majority of the normalized dose indices were less than unity, and in some cases the difference was as much as 4.5%.

Figure 2.

Normalized dose indices for selected anatomic structures demonstrating the differences resulting from the DPEs when Mixed^{Opt, Converged} and MC^Opt,0 methods are compared. The dotted lines are plotted to aid in the visualization of the 3% difference threshold, whereas the dashed lines indicate the baseline of 1.0.

Statistical results are presented in Table 1. The first column of Table 1 shows the average doses (over the 13 patients) computed with the MC^Opt,0 method, which was used as a standard in the DPE quantification. The statistical equivalence tests demonstrate that within 2.8% (of the average dose for each structure), the Mixed^{Opt,Converged}-computed doses are statistically indistinguishable from the MC^Opt,0-computed doses. Thus, although 7 of 78 (9%) dose indices were outside the 3% tolerance, the DPE of Mixed^{Opt,Converged} is not clinically significant.

Table 1.

Dose intervals at which statistical equivalence test indicate that the DPEs and OCEs are statistically significant (p<0.05) for each dose index tallied. Equivalence between any two calculation methods used in the dose computation is established when p values from t tests performed on both the upper and the lower bounds of the dose difference intervals are less than 0.05 (5%).

Structure	Error type
	Average MC dose used as DPE standard (cGy)	DPE dose interval (cGy)	Average MC dose used as OCE standard (cGy)	OCE dose interval (cGy)
GTVs D98%	6887	65	6910	57
CTVs D95%	6104	37	6084	48
Nodes D90%	3869	67	3771	130
Cord D02%	4148	72	4022	171
Brain stem D02%	2694	74	2586	141
Parotids D50%	5440	103	5486	80

OCE

A plot quantifying the impact of the OCEs on deliverable IMRT plans is presented in Fig. 3. Only selected data for the HN site are presented in a graphical format. Of 78 dose indices, 21 are outside the ±3% interval. Twenty of the dose indices (e.g., cord and parotids in Fig. 3) are for the critical structures, and only one is in the targets (Fig. 3; nodes for patient 1). Figure 3 demonstrates that the converged solution differs in the sparing of the critical structures. The normalized dose indices are systematically greater than unity.

Figure 3.

Normalized dose indices resulting from the comparison of MC^Opt,0 with MC^{Opt, Converged}. The observed differences quantify the magnitude of the OCEs. The dotted and dashed lines are the same as in Fig. 2.

The statistical results are presented in the last two columns of Table 1, in which the average dose (computed with MC^{Opt,Converged} method) and the dose equivalence interval are given. The statistical test shows that the MC^Opt,0 and MC^{Opt,Converged} results are statistically equivalent over all tallied indices at the 5.5% level of the local dose, which is almost double that of the DPEs. The OCE of the Mixed^{Opt,Converged} is considered clinically significant.

OCE and DPE correlation

Figure 4 shows the correlation plots between selected HN targets and critical structures. The correlation between the fOCEs and the fDPEs is strong in the targets in which the correlation coefficients range from 0.71 (in the CTVs, which are not shown) to 0.94 (in the GTVs; Fig. 4, top panel). The correlation plots for the critical structures are much more scattered, indicating weak or nonexistent correlation between the two types of errors.

Figure 4.

Correlation plot between fOCEs and fDPEs, expressed as percentages, for several HN targets and critical structures. The dashed lines denote a linear fit to the data, with correlation coefficients given in the panel legends.

DISCUSSION AND CONCLUSIONS

This investigation quantifies the impact of DPEs and OCEs resulting from differences in SC and MC dose calculation algorithms when used in IMRT optimization. Through use of the same fluence prediction algorithms for both SC and MC optimizations, the investigation isolates the dosimetric gains of using MC dose computations for the patient dose calculation during IMRT optimization. The DPEs and OCEs evaluated in the study result from the rigor with which the dose algorithm incorporates the radiation transport through human tissue, especially heterogeneities. MC does a full radiation transport though the heterogeneous geometry, whereas SC scores the TERMA³⁷ throughout the geometry and convolves it with MC-derived energy deposition kernels using density scaling to account for heterogeneities.

The statistical equivalence tests indicate that the DPEs that result from the use of a SC algorithm for the patient dose calculation are within ±2.8%. This finding is consistent with the results of Wang et al.,¹² who found that the differences between MC and pencil-beam (PB) dose calculation algorithms for HN cases are within 10%. Wang et al.¹² did not study OCEs.

In this work, a ±5.5% dose interval was required to encompass the OCE of the mixed method. This interval is larger than the 3% interval at which differences are considered to be clinically significant and larger than the associated DPEs. Correlation analysis revealed a correlation between the DPEs and the OCEs in the targets, but there is no obvious correlation in the critical structures.

Although the OCEs do not affect the target coverage, the reduced (by as much as 8.5%) critical structure doses imply the existence of optimization solutions with more favorable dosimetric properties than were found with the analytic dose calculation algorithm. Evidently, correcting for the DPEs made available previously unexplored portions of the solution space. The reduced normal tissue doses afforded by these portions of the solution space can potentially open the door to dose escalation.

It has been reported in several publications^{5, 7, 8, 10, 12, 18, 38, 39, 40} that the differences between measured∕MC-derived doses and doses calculated by correction∕PB-based algorithms are larger than the differences between measured∕MC doses and SC doses. In other words, the DPEs of PB∕correction-based algorithms are larger than the DPEs of SC algorithms. It is therefore expected that the corresponding OCEs will also be larger. Thus, it is likely that MC-based IMRT optimization will also be more beneficial in the cases where the alternative is PB- or correction-based IMRT optimization.

In summary, this work shows that, even when the DPEs of optimizations that use SC for the patient dose calculation are clinically insignificant (below 3%), the OCEs can be clinically significant, indicating that the use of MC for HN patient dose computations (to model tissue heterogeneities) during IMRT optimization yields treatment plans with improved normal tissue sparing.