Characterization of automatic treatment planning approaches in radiotherapy

Highlights

• •Auto-Planning is widely used, yet creation of high quality treatment plans remains challenging.
• •Systematic investigation of behavior and optimal use of Auto-Planning.
• •Widely applicable solutions to create optimal plans.
• •Auto-Planning outperforms manual plans in DVH metrics and blind comparisons.

Abstract

Background and purpose

Automatic approaches are widely implemented to automate dose optimization in radiotherapy treatment planning. This study systematically investigates how to configure automatic planning in order to create the best possible plans.

Materials and methods

Automatic plans were generated using protocol based automatic iterative optimization. Starting from a simple automation protocol which consisted of the constraints for targets and organs at risk (OAR), the performance of the automatic approach was evaluated in terms of target coverage, OAR sparing, conformity, beam complexity, and plan quality. More complex protocols were systematically explored to improve the quality of the automatic plans. The protocols could be improved by adding a dose goal on the outer 2 mm of the PTV, by setting goals on strategically chosen subparts of OARs, by adding goals for conformity, and by limiting the leaf motion. For prostate plans, development of an automated post-optimization procedure was required to achieve precise control over the dose distribution. Automatic and manually optimized plans were compared for 20 head and neck (H&N), 20 prostate, and 20 rectum cancer patients.

Results

Based on simple automation protocols, the automatic optimizer was not always able to generate adequate treatment plans. For the improved final configurations for the three sites, the dose was lower in automatic plans compared to the manual plans in 12 out of 13 considered OARs. In blind tests, the automatic plans were preferred in 80% of cases.

Conclusions

With adequate, advanced, protocols the automatic planning approach is able to create high-quality treatment plans.

1. Introduction

Automation is a hot topic in radiotherapy treatment planning. Important benefits of automation include time saving, high quality planning, and protocol standardization. Different commercially available Treatment Planning Systems offer different automation methods [1], [2]. This paper concerns automatic planning via protocol based automatic iterative optimization [1].

The challenge in clinical implementation of automatic planning is finding an automation protocol that results in optimal treatment plans that adhere to the institute’s planning protocol and desired trade-offs. Development of such protocols is a challenging task [3], [4], [5]. The behaviour of the iterative optimizer can be unexpected and a large part of protocol development consists of trial-and-error [3], [4], [6]. Several studies have been published that compare protocol based automatic plans to manual plans for treatment sites H&N [3], [6], [7], [8], [9], [10], [11], lung [4], [12], [13], oesophagus [14], hippocampus sparing whole brain [15], and prostate [11], [16]. Most studies report that the quality of the automatic plans is at least similar to manual plans. Some report a difference in plan style and trade-offs, making the comparison subjective. In most cases automatic plans have better OAR sparing, at the possible cost of target homogeneity and conformity. Automatic plans tend to be more modulated and use more monitor units compared to manual plans [4], [9], [11], [14].

Most works in automatic planning literature report on a comparison between automatic and manual plans for a specific treatment protocol. Having their focus on plan quality comparison, details on the automation protocol and its development are often limited.

The aim of this work was to investigate how to configure the automation protocols in order for them to lead to optimal treatment plans. This required understanding of the automatic planning approach, which was obtained by systematically investigating the functioning and performance of the approach on all relevant aspects of treatment plans. The challenges identified and solutions presented can be widely applied by those wanting to make optimal use of automatic planning approaches.

2. Materials and methods

2.1. Automatic planning system

The automatic planning approach used in this work is Pinnacle³ 16.2 Auto-Planning (Philips Radiation Oncology, Madison, WI). In this module, the iterative optimization is performed by the Auto-Planning engine (APE), and the automation protocol is defined in the “treatment technique” (TT); a template that contains optimization goals for targets and OAR and advanced settings.

The TT consists of a number of configurable plan criteria. The target goals only contain the prescribed dose. For OARs, a mean dose, max dose, or max DVH point can be configured. In addition, a priority (high, medium, low) can be assigned to each OAR goal. In case an OAR overlaps with a target, a compromise switch can be used to define whether the overlapping voxels should be treated or spared. There are a number of advanced settings [3], including the tuning balance, which controls the overall priority between targets and OAR, and the hot spot maximum dose.

The generation of Auto-Plans occurs according to a fixed pattern. Based on the TT, the APE sets objectives and uses the Pinnacle optimization module to generate a plan. Objectives to achieve conformity, set on a ring around the PTV and “BodyMinusTarget”, are automatically added. After an initial round of optimization, objectives are modified or added based on the dose distribution and objective costs. For the generated target objectives the weights are fixed, for the OAR the weights are adjusted over the course of the iterations. The APE strongly aims to meet the configured OAR constraints, and sets accompanying “extra push” objectives to further lower the dose. After six rounds of optimization, the final dose distribution is computed. The treatment plans generated by Auto-Planning are conventional Pinnacle plans that can be further optimized manually.

2.2. Treatment protocols and patient data

In this work we studied Auto-Planning for prostate, rectum and sequential H&N plans. A description of the discussed treatment protocols is given in Suppl Mat A. For TT development, the 20 (10 for development, 10 for evaluation) most recently treated patients per site were used. The study was approved by the Institutional Review Board.

2.3. Coverage, OAR sparing and conformity

The behaviour of the APE was first investigated for the 46 Gy part of the sequential H&N plans to the elective nodes. Auto-Plans were generated using an elementary TT, which consisted of the target dose and OAR constraints as defined in the treatment protocol, and default advanced settings. This elementary TT was then systematically modified to investigate whether its initial performance on target coverage, OAR sparing, and dose conformity could be improved.

First, the target coverage, which has to be at least 98%, was evaluated on automatic plans made using the elementary TT for the 10 H&N evaluation patients. The coverage was then compared to that of plans made using a TT that contained three target optimization goals on different parts of the PTV. These goals were set on the outer 2 mm of the PTV, which was split in a part overlapping with the parotid glands and the part outside of the parotids, and on the remaining inner part of the PTV (Fig. 1).

Fig. 1.

Dose distributions resulting from the elementary automation protocol (A) and the modified protocol with 3 target goals (B). The PTV is shown in purple, the parotids gland in green (R) and yellow (L), the oral cavity in red. In B, the auxiliary inner ring structures on which goals are configured are shown in black and blue. As can be seen from the light blue 95% isodose line, large patches of the PTV close to the OAR receive insufficient dose. In B it can be seen that using 3 target goals attains proper coverage but deteriorates conformity and OAR sparing, which implies that additional goals are required to improve these aspects of the plan. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

To investigate the OAR sparing behaviour of the APE, the OAR goals in elementary TT, including the additional target goals, were modified by systematically varying the dose goal on the parotid glands. By comparing the results of this “dose goal sweep” for five of the development patients, conclusions were drawn on the existence of a single optimization goal leading to optimal sparing for all patients.

In addition, plans were made using a TT that divided the parotid glands into shells at fixed distances from the PTV, to which separate goals were set. Three shells were used that respectively cover the first 5 mm, the part between 5 and 25 mm and the part beyond 25 mm away from the PTV. As the distance of each shell to the PTV is fixed, a representative dose value could be assigned to each goal. The resulting plans were compared to the most optimal plans from the dose goal sweep.

Two sets of plans were made to investigate conformity. The first were generated using only the advanced target and OAR goals. For the second, a max dose goal on a 5 mm ring directly adjacent to the PTV and max DVH goals on a ring between 5 and 10 mm from the PTV and the body contour minus the PTV + 10 mm were added. Plans were compared visually and on conformity index (CI ≡ V_95%/V_PTV).

2.4. Complexity and dosimetric verification

The automatic plans’ beam complexity and deliverability were evaluated. Initially, the automatic plans were created without limitation of the leaf motion. The complexity was first assessed by interpolation of the control point (CP) spacing from 4° to 2° and MU/cGy ratio [17]. Plans were considered too complex if the PTV D_mean increased more than 1% after CP interpolation and/or the MU/cGy ratio was over 3. If these tests passed, the plan delivery was validated by phantom-based EPID dosimetry [18]. If the corresponding complexity or deliverability tests failed, the TT was modified by constraining the maximum leaf motion and delivery time.

A TT was created for 24 Gy primary tumour H&N plans using all techniques described above. The unconstrained versions of the plans were made without limitation of the leaf motion and with a maximum beam delivery time of 90 s. As these plans proved to be too complex, constrained plans were made with maximum leaf motion of 0.3 cm/° and a delivery time of 120 s for a full arc.

2.5. Configurational limitations and automatic post optimization

Several treatment plan requirements of our prostate protocol could not be captured in the TT. It was not possible to (i) precisely define the desired dose in an overlap region between target and OAR, and in a low dose PTV close to a high dose PTV, (ii) define that for a certain OAR goal, no further sparing is wanted once the configured value has been reached, (iii) both control the mean and max dose for each separate target.

In order to correct for these limitations of Auto-Planning, an automatic prostate postscript was developed that created additional help structures and objectives, and executed another round of optimization after Auto-Planning was completed. Objectives were added to improve the coverage and conformity, to precisely control the dose in the overlap region between the PTV and the rectum, and to just meet the dose constraints on the femoral heads. Automatic prostate plans before and after the postscript were compared on clinical acceptability.

2.6. Evaluation of automatic plans

Using the methods described above, TTs were developed for treatment sites prostate, rectum, and H&N (the TTs are included in Suppl Mat B). For each site, automatic plans were created for the evaluation patients, and evaluated in terms of DVH metrics and a blind comparison between clinically used manual plans and automatic plans (for H&N for the 46 Gy plans). Each patient in the blind test was evaluated by a radiation oncologist and a planning dosimetrist, who were each asked to judge the two plans on clinical acceptability as well as to express their preference between the two. Per site, four radiation oncologists and four planning dosimetrists were involved in the test to avoid one sided evaluations.

3. Results

3.1. Coverage, OAR sparing and conformity

None of the 46 Gy H&N automatic plans generated using the elementary automation protocol achieved the required target coverage. The average V_95% of the automatic plans was 96.9%. Using the advanced protocol with 3 target goals, the average V_95% increased to 99.7%. Example dose distributions are shown in Fig. 1.

Optimal parotid sparing was only reached for protocol dose goals that were close to the lowest achievable value (Fig. 2). For an OAR goal value close to the parotid mean dose constraint of 26 Gy, the resulting parotid mean dose was on average 4.6 Gy (max 9.7 Gy) above what could be achieved with an optimal case dependent goal. The best case independent goal dose was 12 Gy, which led to an average dose 1.8 Gy (max 5.7 Gy) above the optimum. No single mean dose goal was found that led to optimal sparing for all geometries. The OAR shell method resulted in an average mean dose 0.1 Gy below that of the optimal but a priori unknown dose goal. An example of the parotid shells is shown in Fig. 3.

Fig. 2.

The achieved parotid mean dose as a function of the parotid mean dose goal for 10 cases. The dashed red lines is a diagonal. If a goal was chosen far above or below the achievable dose, the automatic optimizer failed to reach the optimum. In cases where the goal was comparable to the achievable dose, the automatic optimizer reached a dose just below that goal. The dashed horizontal lines, which represent the parotid mean dose as achieved by an automation protocol that utilizes the OAR shell method, are on average close to the minimum of the corresponding curve and indicate that this method reached optimal sparing using a patient independent automation protocol. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3.

Dose distributions generated without (A) and with (B) 3 additional goals to ensure target conformity. The rest of the protocol is built using the 3 targets and OAR shell method. In the right parotid, the OAR shells are visualized in blue, yellow and red. The plan with conformity goals in B has better high dose conformity (CI = 1.23) compared to the plan without conformity goals in A (CI = 1.40). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

The automatic plans did not naturally achieve optimal high dose conformity. The plans generated using the automation protocol with explicit high dose instructions improved the conformity, without deteriorating the OAR doses and target homogeneity (Fig. 3).

3.2. Complexity and dosimetric verification

The unconstrained 24 Gy H&N plans were too complex. After interpolating the CP spacing from 4° to 2°, the PTV D_mean increased by on average 0.9% and by >1% in 4 out of 10 patients. Unacceptable hot spots appeared in the PTV. On average, the plans used 2.3 MU/cGy.

In the constrained plans, the mean increase of PTV D_mean upon CP interpolation was limited to 0.2%, and hotspots no longer appeared. The average of MU/cGy increased to 2.5. In the EPID dosimetry verification, the average γ-mean and γ-pass rate were 0.42 and 96% for 3%/3mm criteria, which fell well within our clinical thresholds and values obtained for manual plans.

3.3. Automatic post optimization

An experienced planning dosimetrist considered 0 out of 10 automatic prostate plans clinically acceptable before running the post optimization. The main reasons for rejecting the automatic plans were: insufficient coverage (7/10), hotspots on the rectum wall (5/10), insufficient high dose conformity (4/10), high femur dose (2/10). After post optimization, all 10 plans were acceptable. For H&N and rectum plans, having less strict OAR and hotspot constraints, no post-processing was required.

3.4. Plan quality comparison

In 12 of 13 OARs for the three treatment sites, the dose was lower (by on average 1.2 Gy) in the automatic plans compared to the manual plans (Table 1). In the blinds tests, there was an 11.5–8.5 (prostate), 20–0 (rectum) and 16–4 (H&N 46 Gy) preference for the automatic plans, mostly due to improved OAR sparing. All 30 automatic plans were considered clinically acceptable. An example of an automatic vs. manual plan comparison for prostate is shown in Fig. 4.

Table 1.

The difference in OAR dose between automatic and manual plans for 10 prostate, 10 H&N, and 10 rectum patients that were used for the evaluation and blind comparison test. A negative mean difference means the automatic plans performs better.

Treatment Protocol	Metric	Mean ± σ [Gy] Auto - Manual
Prostate 77 Gy	Rectum Dmean	−2.5 ± 3.3
	Anal Sphincter Dmean	−1.1 ± 2.3
Head & Neck 46 Gy	Spinal Cord Dmax	−0.5 ± 2.1
	Brainstem Dmax	−3.6 ± 3.5
	Parotid_gl_R Dmean	−0.9 ± 1.8
	Parotid_gl_L Dmean	−0.5 ± 1.8
	Submnd_gl_R Dmean	−0.3 ± 1.8
	Submnd_gl_L Dmean	−0.1 ± 1.9
	Oral Cavity Dmean	0.3 ± 1.8
	Base of Tongue Dmean	−0.9 ± 4.0
	Constrictor Muscle Dmean	−1.6 ± 1.7
	Larynx Dmean	−1.5 ± 1.6
Rectum 50 Gy	Bowel + Bladder Dmean	−2.5 ± 1.3

Fig. 4.

Example of the blind manual (A) vs. automatic plan (B) comparison test for prostate plans. The PTVs are shown in pink and purple, the rectum in brown. As in most prostate cases, the two plans are similar. In this case, both the radiation oncologist and the planning dosimetrist preferred the automatic plan because of the (2.2 Gy) lower rectum D_mean. Also the higher mean dose in the high dose PTV in the automatic plan (76.4 Gy vs. 77.1 Gy) is favored. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

4. Discussion

In this work we discussed the development of automation protocols that lead to optimal treatment plans. An automation protocol that simply contained the prescribed target dose and OAR constraints generally did not provide optimal plans. More intricate protocols, that contained additional optimization goals and help structures, were required. Different tumour sites, depending on their geometry and constraints, required different approaches.

Reaching sufficient target coverage with automatic planning is difficult. For target optimization goals, the automation protocol only contains the prescribed dose. The coverage criterion cannot be specified. Only the relative importance between the targets and OARs can be set using the tuning balance. In order to achieve target coverage, the tuning balance is often set below default to give more priority to the target [3], [8], [16]. A simple way to achieve target coverage is by increasing the plan’s monitor units. However, this can lead to hotspots, high PTV mean dose, and deterioration of plan conformity. An extreme measure to reach target coverage is to set target goals on an expansion of the PTV [3], [8]. The method used in this paper, setting a separate goal on the outer 2 mm of the PTV, can be applied to many planning protocols for automatic as well as conventional planning. A similar method was recently published for automatic lung planning [12].

For most planning protocols, it is not possible to spare OARs optimally with a single optimization goal which is the same for all patients [3], [5], [6], [8], [9], [10], [13], [14]. The automatic optimizer needs a reasonable seed dose value to reach optimal sparing. For reasonable seeds, the optimizer strongly aims to meet the goal, often at the expense of target coverage. When using a constant seed, a low value works better than a high one. For H&N plans, we achieved optimal parotid gland sparing by dividing the OAR into shells for which a reachable dose can be estimated. This approach can be thought of as inverse application of an overlap volume histogram (OVH), as is often used in knowledge based planning (KBP) [5], [19], [20]. In KBP, the achievable dose is predicted based on the position of the OAR with respect to the PTV and previously generated plans. In the shell method, conversely, the dose at a certain distance from the PTV is estimated and the OAR shells are created such that they correspond to this dose. The advantage of the shell method is that no further functionality, such as OVH generation, has to be developed. To further improve the automation protocol performance, even more advanced auxiliary structures can be applied.

Even though the automatic optimizer aims to automatically achieve good dose conformity, we found that conformity could be improved using specific optimization goals [13], [21].

Automatic plans are mostly more modulated and use more MU compared to manual plans [4], [9], [11], [14]. In cases where this leads to unacceptable QA results, segment modulation can be limited by configuring a suitable combination of maximum leaf speed and delivery time. When limiting leaf speed, it may be necessary to increase the delivery time to reach the desired plan quality.

A powerful method to improve generated automatic plans is by post-optimization using conventional objectives. The necessity of this was reported in previous works [6], [9], [10], [11], [15], [16]. Using scripting, such post-optimization can be automated [9]. The advantage of post-optimization over further improvement of an automation protocol is that it offers more direct control over the dose distribution.

We have shown that for prostate, rectum [21] and sequential head and neck treatment, automatic plans outperformed manual plans in both DVH and blind comparison. Comparing these improvements to those obtained by others is difficult as this strongly depends on the quality of the manual plans [3], [11]. Development of an automation protocol that leads to optimal and deliverable plans with trade-offs precisely as desired is challenging [3], [4], [5] and took us several months of work for a physicist and dosimetrist.

Different treatment planning systems offer different types of automatic plan generation [1]. A first type is the template based approach, which can be assisted by a personalized dose prediction. Second are the knowledge-based planning methods, which mimic dose distributions predicted based on anatomy and a database of plans. Third are the approaches that generate a large number of optimal plans, allowing the user to select the one with the most preferable trade-offs. In a comprehensive comparison of different automatic planning solutions by Krayenbuehl et al. [2], protocol based automatic iterative optimization was found to achieve the best OAR sparing and above average time saving.

After clinical introduction it is important to monitor the quality and acceptance rates of automatic plans. For the 190 prostate cases treated in our clinic between January 2017 and July 2018 using automatic planning as discussed in this paper, the acceptance rate of the automatic plans was 75% [22]. In the remaining 25%, the automatic plan was used as starting point and substantial time was saved compared to full manual planning. Independent KBP QA revealed that 92% of our clinically generated automatic prostate plans are optimal [19].

In conclusion, the automatic planning approach is able to generate high quality treatment plans fully automatically, however development of automation protocols that lead to optimal plans that adhere to local planning protocols remains challenging and requires solutions as presented in this work.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following are the Supplementary data to this article: