Comparison of dose metrics between automated and manual radiotherapy planning for advanced stage non-small cell lung cancer with volumetric modulated arc therapy

1. Introduction

Advanced stage non-small cell lung cancer (AS-NSCLC) is generally treated with concurrent chemoradiotherapy using a radiation dose of 60–66 Gray (Gy) in 30–33 fractions over six-seven weeks [1]. Intensity-modulated radiation therapy (IMRT) and volumetric modulated arc therapy (VMAT) are widely used treatment techniques [2].

Radiation treatment plans are becoming increasingly complex [3] and manual optimization of treatment plans is time-consuming and the quality of the plan is operator dependent [4]. The increasing complexity of treatment plans complicates the optimization procedure and thereby augments the rate of inconsistency between manually derived treatment plans [3], [5]. Several trial-and-error optimization processes are usually required to achieve clinically acceptable plans. More manual actions could influence consistency and plan quality of the manual treatment plans [3], [4]. The experience of the planner has a large impact on plan quality and dissemination of best practices could help improve these variations [6].

Aforementioned drawbacks of manual treatment planning might be overcome by automating treatment planning. Most treatment planning systems currently have integrated an automated treatment planning solution. In addition, there are also in-house developed automated treatment planning systems [7]. Automated treatment planning methods are aimed to reduce the inter-planner variability and the planning time during the optimization process and to improve plan quality. Different sites investigated were already investigated such as head and neck (H&N) [4], [5], [8], [9], prostate [10] and oesophagus [11].

Automated treatment plans for AS-NSCLC radiation therapy are not much represented yet. The location of a tumor in the lung varies more than the location of a tumor in the H&N area or the prostate. This variability may cause a difference in the result of automated treatment planning techniques. Della Gala et al. [7] investigated different radiation techniques for AS-NSCLC by comparing originally manually planned IMRT treatment plans versus automated VMAT treatment plans with their in-house developed treatment planning system Erasmus-iCycle. In this study we have investigated, if automated treatment planning is able to create treatment plans with consistent quality using a single optimization preset including beam set-up, dose prescription, objectives and priorities for organs at risk (OARs), and planning target volumes (PTVs) for AS-NSCLC. A comparison is presented between automated and manually generated VMAT treatment plans for AS-NSCLC.

2. Materials and methods

2.1. Study design and inclusion criteria

We performed a quantitative retrospective planning study to develop an automated VMAT treatment planning procedure for AS-NSCLC. The treatment plans of twenty-five consecutive AS-NSCLC patients originally planned in the period 2016 – 2017 were re-planned using the Auto-Planning module in Pinnacle 9.10. Three cases were excluded when either a second dose level was specified to part of the PTV or when adjustments to the protocol were done. All remaining twenty-two plans were designed to deliver 66 Gy in daily fractions of 2 Gy to the primary tumor and the lymph node metastases.

Male and female patients were included regardless of age, localisation of the primary tumor or tumor size. The median age of twenty-two patients (six female; sixteen male) was 63.5 years (range 53–83 years). The primary tumor was located in the left lung in seven patients, and in the right lung in fifteen patients. The median PTV was 326 cm³ (range 103–940 cm³).

The PTV consist of the CTV + 10 mm margin, whereby the CTV was formed by tumor GTV + 5 mm margin and lymph node GTV with no margin. The internal target volume (ITV) was used as the motion management technique for the treatment plans. The OARs, which were eligible for contouring, were oesophagus, lungs, myelum, heart, plexus brachialis, sternum and ribs.

2.2. Planning technique

The clinical treatment plans were planned with the clinical version of Pinnacle 9.10 (Philips Radiation Oncology Systems, Fitchburg, WI, USA) using the VMAT technique. The Auto-Planning module within the clinical version of Pinnacle 9.10 was used to plan the automated treatment plans. Treatment plans were calculated with a dose grid of 3 mm. The clinical plans were made using a model for an Elekta Synergy linear accelerator (Elekta AB, Stockholm, Sweden) with a Agility™ multileaf collimator for 10 megavolt (MV) photons. The number of beams were doubled for the automated treatment plans, giving these plans more space to fulfil the dose criteria.

Clinical plans were made using one partial VMAT arc. However, one partial VMAT arc did not appear to be sufficient for automated plans to make clinical acceptable plans for all cases. Therefore, the partial VMAT arc of the manual plan was copied and therefore two VMAT arcs were used for the automated plans.

The planning templates were tested through trial-and-error to investigate the possible settings for the automated treatment plans. The settings were first tested on four treatment plans. After this, the settings were evaluated on the remaining treatment plans. High priority is not used in this setting, because this setting appears to have a strong impact on the dose distribution of the PTV.

If the tumor was located left ventral or right ventral, the beam angles for the manual and automated plans were the same. If the tumor was located left dorsal or right dorsal, the beam angles differ from each other. The manual angles for left dorsal were 216–20° counter clockwise (CCW) and for right dorsal the angles were 20−140° CCW. For the automated plans the beam angles were changed a little, because we did not want the beam angle go through the 180°, because this was physical not possible. The template setting for the automated treatment plan were described in Table S1 see Supplementary material.

The template setting was developed to meet the prescription and dose criteria of the automated treatment plan for AS-NSCLC (see Table 1).

Table 1.

Prescription for stage III AS-NSCLC.

Prescription 66 Gy in 33 fractions (33x2Gy)
ROI	Dose
PTV	V95% ≥99%
PTV	Dmean: 66 Gy (maximum 67,5 Gy)
Hot spots in PTV	107%≤ “hot spot” ≤110% “hot spot” ≤2 cm3
Oesophagus in PTV	Dmax: 66 Gy (100%) Dmin: 62,7 Gy (95%)
Oesopahgus + 2 mm	Dmax: V35 Gy < 50% Dmax: V50 Gy < 40% Dmax: V65 Gy < 35% Dmax: V70 Gy < 20%
Both lungs (lungs – GTV)	Dmax: V20 < 30%
Both lungs (lungs – GTV)	Dmean: <19 Gy
Spinal cord	Dmax: 50 Gy
Spinal cord + 3 mm	Dmax: 50 Gy
Heart	Dmax: V25 Gy < 10% Dmax: V30 Gy < 46% Dmax: V50 Gy < 40%
Heart	Dmean: <26 Gy
Plexus brachialis	Dmax: 66 Gy
Sternum	Dmax: 66 Gy
Ribs	Dmax: 66 Gy

Gy: Gray, ROI: Region Of Interest, V: Volume, Dmean: Mean Dose, Dmax: Maximum Dose, Dmin: Minimum Dose, GTV: Gross Tumor Volume.

In clinical practice the plan quality always improved after performing a warm re-start, i.e. calculate the treatment plan without resetting the previous optimization results. After the warm re-start scorecards were used to check if the planning goals were met. If a goal was not met, another warm re-start is performed after manual adaptation of an objective.

2.3. Data collection and analysis

The plan quality between the original manual optimized and automated VMAT treatment plans were compared by means of conformity index (CI) and homogeneity index (HI). CI assesses the dose conformity of a region of interest (ROI), and HI assesses the dose homogeneity of a ROI [4], [12], [13].

The CI was calculated by the formula of Van ‘t Riet [14], which is the most clean way of describing CI, because it simultaneously takes into account irradiation of the target volume and irradiation of healthy tissues [15]:

$$\mathit{CI}=\frac{{\left(V_{95\%}^{\mathit{PTV}}\right)}^2}{V^{\mathit{PTV}}\mathrm{x}{V}_{95\%}}$$

${\mathrm{V}}_{95\%}^{\mathrm{P}\mathrm{T}\mathrm{V}}$ = volume of the PTV receiving 95% of the prescribed dose.

$V^{\mathit{PTV}}$ = volume of the PTV.

$V_{95\%}$ = volume of the 95% isodose.

The optimal value of the CI is 1 [12]. The lower the CI, the lower the conformity of the PTV in a treatment plan [12].

The HI was calculated by the formula of the International Commission on Radiation Units and Measurements (ICRU) [13]:

$$\mathit{HI}=\frac{D_{2\%}^{\mathit{PTV}}-D_{98\%}^{\mathit{PTV}}}{D_{50\%}^{\mathit{PTV}}}$$

$D_{2\%}^{\mathit{PTV}}$ = 2% of the volume of the PTV receives that dose or more (near maximum dose).

$D_{98\%}^{\mathit{PTV}}$ = 98% of the volume of the PTV receives that dose or more (near minimum dose).

$D_{50\%}^{\mathit{PTV}}$ = 50% of the PTV received dose (median dose).

The ideal value for HI is zero and it increases as homogeneity decreases [13], [16].

Treatment plans were also compared using dose-volume histograms (DVHs) and monitor units (MUs) [4], [17]. Distributions of the CIs, HIs, DVHs and MUs were visually assessed using histograms. In case of a skewed data, the median and interquartile range were used to describe the data. Further statistical analysis was performed with the Wilcoxon signed rank test. Differences between plans were considered statistically significant if p < 0.05. An in-house developed DVH-tool, using Matlab version R2016a (The MathWorks Inc., Natick, MA, United States) software was used for comparing dose values of PTVs and OARs for all plans.

Plan QA was performed on a Delta4 phantom (Scandidos, Uppsala, Sweden) [18]. A plan is accepted if more than 95% of the measured point within the 40% isodose surface (related to the prescribed dose) fulfilled γ-criteria of 3% (related to measured maximum dose) or 3 mm distance.

2.4. Evaluation of planning time

To evaluate the cost effectiveness of the automated planning procedure we analysed hands-on time for both the automated and the manual planning procedure. The hands-on-time was estimated for the time a planner is actively working on the treatment plan.

3. Results

For eleven of the twenty-two automated plans, the plans were ready after automated planning and a seperate warm optimization re-start were done. For the remaining eleven plans, another warm re-start was performed after manual adaptation of an objective weight or value for some OARs before the plan was clinical acceptable. The treatment plans obtained by automated planning were superior compared to the manually derived plans, an example is given in Fig. 1. The spider-web diagram of Fig. 1 shows visually the reduction in dose to the OARs by the automated planning procedure.

Fig. 1.

A subset of the plan quality metrics for PTV and dose metrics comparison for OARs. Upper panels show an example of the transversal dose distribution for the original clinical (upper left panel) and automated plan (upper right panel). In the lower panels, the plan quality is evaluated for all plans comparing the manual (light blue) with the automated plans (dark blue). The plan quality criteria for PTV are plotted and scaled (if necessary) for visualisation: V95% (divided by 100), HI (multiplied with factor 10) and CI. In the lower right panel, the dose metrics to the OARs are shown. PTV: Planning Target Volume, OAR: Organ At Risk, V: Volume, HI: Homogeneity Index, CI: Conformity Index, Oes: Oesophagus, GTV: Gross Tumor Volume, Dmean: Mean Dose, Dmax: Maximum dose. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

3.1. PTV

As shown in Table 2, there was a significant improvement in HI for the PTV from 0.09 to 0.07 favouring the automated plans.

Table 2.

Dose metrics results for PTV and OARs.

Organs	Parameters	Median Manual	Median AP	IQR Manual	IQR AP	P-value
PTV	V95Gy (%)	99.0	99.1	0.6	0.5	0.5
PTV	Dmean (Gy)	66.3	66.3	0.5	0.3	0.4
PTV	CI	0.8	0.8	0.1	0.0	0.4
PTV*	HI (x10)	0.9	0.7	0.2	0.1	0.001
Oesophagus + 2 mm	Dmean (Gy)	58.0	55.1	7.8	5.5	<0.001
Oesophagus + 2 mm	V35Gy (%)	26.2	23.2	14.0	19.1	<0.001
Oesophagus + 2 mm	V50Gy (%)	14.5	12.9	15.0	15.7	<0.001
Oesophagus + 2 mm	V65Gy (%)	2.6	4.0	7.8	8.2	0.2
Oesophagus + 2 mm	V70Gy (%)	0.0	0.0	–	0.0	0.2
Lungs – GTV	V20Gy (%)	26.0	25.6	10.3	9.8	0.001
Lungs – GTV	Dmean (Gy)	15.8	15.7	5.9	5.9	0.002
Spinal cord	Dmax (Gy)	44.8	39.4	5.5	14.8	<0.001
Spinal cord + 3 mm	Dmax (Gy)	47.4	45.7	7.7	16.3	0.002
Heart	Dmean (Gy)	8.2	4.8	9.3	7.3	<0.001
Heart	V25Gy (%)	9.2	3.2	17.1	12.5	0.001
Heart	V30Gy (%)	7.2	2.6	14.5	10.4	<0.001
Heart	V50Gy (%)	2.9	1.0	6.2	3.4	<0.001
Plexus brachialis	Dmax (Gy)	65.9	65.4	–	–	0.6
Monitor Units	MUs (#)	350	460	90.2	120.7	<0.001

P < 0.05 is statistical significant.

AP: Auto-Planning, IQR: Interquartile Range, PTV: Planning Target Volume, V: Volume, Dmean: Mean Dose, Gy: Gray, CI: Conformity Index, HI: Homogeneity Index, * HI values are multiplied with factor 10 in this Table, GTV: Gross Tumor Volume, Dmax: Maximum Dose, MUs: Monitor Units.

No significant difference was seen in PTV V95% coverage, PTV mean dose (Dmean) and conformity between the automated treatment plans and the manually derived treatment plans.

Twenty automated plans showed a PTV V95% coverage of 99% or higher, whereas only eleven manual plans had a PTV V95% coverage of 99% or higher. Of the two automated plans with a PTV V95% coverage lower than 99% (98% and 97% respectively), the corresponding manual PTV V95% coverage was also lower than 99%. A large part of the PTV of these two patients was in the vicinity of the oesophagus, and a V95% coverage of 99% or more have resulted in a large number of hotspots. Given these results, these automated treatment plans were clinically acceptable plans and approved by a clinical physician, physicist and an experienced treatment planner.

3.2. OARs and dose metrics

With automated planning a significant reduction of dose delivered to OARs (Table 1) was found favoring the automated plans (Table 2).

This was the case for several parameters: ‘oesophagus + 2 mm’ (Dmean, V35Gy and V50Gy), ‘lungs – GTV’ (V20Gy and Dmean), ‘spinal cord’ (Dmax), ‘spinal cord + 3 mm’ (Dmax), and ‘heart’ (Dmean, V25Gy, V35Gy and V50Gy).

The Dmean, V35Gy and the V50Gy of the volume ‘oesophagus + 2 mm’ were significantly reduced with 2.9 Gy, 3% and 2%, respectively. However, the V65Gy of the volume ‘oesophagus + 2 mm’ was increased with 1.4% in the automated plans, but this was not a significant difference.

The ‘lungs – GTV’ volume received a reduced dose in the automated plans. However, the significant differences for V20Gy and Dmean were 0.4% and 0.1 Gy, respectively.

The maximum doses in the spinal cord and in the spinal cord + 3 mm were significant reduced in the automated treatment plans than in the manual treatment plans. The differences between the plans were 5.4 Gy and 1.7 Gy respectively in favour of the automated plans.

The heart also received a reduced dose in the automated treatment plans than in the manual treatment plans. Statistical significant differences of the heart Dmean, V25Gy, V30Gy and V50Gy were found. The differences were 3.4 Gy, 6%, 5% and 2%, respectively.

3.3. MUs

The number of MUs in the automated treatment plans was significantly higher than the manual treatment plans. This was the case in 21 of the 22 automated treatment plans. The difference was 110 MUs.

3.4. Planning time

The total estimated planning time for both the automated and the manual treatment plans was four hours. However, the estimated hands-on-time of the manual treatment plans was about two hours, whereas the estimated hands-on-time of the automated treatment plans was only about thirty minutes.

4. Discussion

The aim of this study was to develop an automated VMAT treatment planning for AS-NSCLC patients and evaluate the plan quality between the original manual optimized and automated VMAT treatment plans, for daily clinical practice. It was reassuring that no significant differences between the PTV V95% and the PTV Dmean were found. This could be explained by the fact that these criteria are determined beforehand. The template was adjusted to meet these criteria. The dose to the PTV V95% was similar in the automated and manual treatment plans, as well as PTV Dmean and the CI. Automated and manual treatment plans performed equally with respect to these criteria. The automated treatment plans achieved a significant improvement in homogeneity of the PTV and dose reduction for OARs, whereby the quality of the treatment plans were comparable with the clinical plans.

We showed that a single template setup for automated treatment planning lead to plans of consistent and high quality, fulfilling the clinical dose criteria for the PTV with maximal sparing of the OARs for patients with AS-NSCLC. The hands-on planning time was reduced from two hours to thirty minutes. Besides its efficiency and consistency, the automated plans gave significantly lower doses to the surrounding organs at risk: ‘oesophagus + 2 mm’ (V35Gy and V50Gy), ‘lungs – GTV’ (V20Gy and Dmean), ‘spinal cord’ (Dmax), ‘spinal cord + 3 mm’ (Dmax), and ‘heart’ (Dmean, V25Gy, V35Gy and V50Gy). This dose reduction could result in a significant decrease in acute side effects for instance for the oesophagus and late toxicity for the heart. Although a reduction of an already low dose of the spinal cord has no direct advantage for the patient, this indicates that in case of closer vicinity of the PTV to the spinal cord automated planning may allow a high dose to the PTV while maintaining an low complication probability of the spinal cord, thereby enhancing the therapeutic window.

Automated treatment planning gave a significant reduction of dose delivered to several parameters of various OARs. So even though our manual treatment plans were made by experienced treatment planners, the automated planning technique was able to achieve better sparing of the OARs. Again this has also been reported before: for AS-NSCLC by Della Gala et al. [8] and for head and neck tumors by Hazell et al. [5], Hansen et al. [8], Krayenbuehl et al. [4] and Gintz et al. [19]. However, in our study, the V65Gy of the volume ‘oesophagus + 2 mm’ was slightly higher in the automated plans than in the manual treatment plans. This was due to the vicinity of the PTV to the oesophagus in combination with increased homogeneity of PTV coverage in the automated treatment plans. The automated plans were clinically acceptable for this parameter.

This study was most comparable with the article of Della Gala et al. [7] who also examined treatment planning for AS-NSCLC. The results of the HI, ‘lungs – GTV’ V20Gy and Dmean and heart Dmean were compared. These dose metrics were all improved with the automated treatment planning. The aforementioned dose metrics of the manual and automated plans are in this study superior to the results of the dose metrics of the article of Della Gala et al. [7]. Most likely, because we made manual VMAT plans instead of manual IMRT plans and probably the different use of arcs. The hands-on time was also seen to be decreased in both articles.

Implementation of the automated planning within the clinic is not straightforward and needs to be carefully introduced into the clinic by an expert team who have the skills and experience to understand how automated planning can deliver high quality plans. Thorough testing needs to be done to ensure that the generated automated plans are as least as good as the manually clinical plans in all cases. Therefore, a representative subset of clinical cases needs to be tested and compared with the originally manual plans before introducing automated planning into the clinic.

Generally, a single template is used for all patients, which may limit finding the optimal plan for all patients. However, it would be better to make a patient-specific planning template. There are several alternative options for personalization of planning goals, including Knowledge-Based solutions like Rapidplan available in Eclipse (Varian Medical Systems, Palo Alto, USA) and Multicriteria Optimization (MCO) available in Raystation (Raysearch Laboratories, Stockholm, Sweden) and Erasmus-iCycle. Pinnacle introduced PlanIQ Feasibility, which provides a patient-specific estimation of the best-case scenario dose distribution, we did not yet use this.

We developed an overlap volume histogram (OVH)-based method for automated prostate plans to check the plan quality of the automated plans [20]. This tool could also be used to set a patient-specific planning template before starting the treatment process. This tool was not used in this study. Currently we are investigating the implementation of this tool for the automated lung plans obtained in this study. Although the automated planning should be robust, a plan quality control tool is necessary to guarantee that for each patient the generated plan fulfills clinical criteria for the patient-specific anatomy. For Pinnacle typically a scorecard which contains clinical goals is used to evaluate the plan quality. However, these scorecards are based on general protocols and do not take into account the possible patient specific endpoints. Currently, different machine learning techniques [21], [22] are developed to predict the 3D dose distribution and integrated in the planning process. Also our in-house developed plan quality control tool can be used the further improve the plan quality.

Furthermore, the beam configuration like for example collimator rotation [23] may influence the plan quality. In this study we used a collimator rotation of 20° and did not investigate how other values could influence the plan quality.

The number of MU in the automated treatment plans was significantly higher than in the manual treatment plans. This was caused by the use of two VMAT arcs instead of one arc and by increased multileaf collimator (MLC) modulation. Considering the benefits of automated treatment planning, we found the increase in MU acceptable. In head and neck tumors, Hansen et al. [8] found an average increase of 75 MU per treatment plan for automated treatment plans in comparison with manual VMAT treatment plans, which was in the same order of magnitude of our study.

Manual and automated plans differ in (partial) arcs arrangement: manual plans exploit one arc, while automated plans used two arc arrangements. Despite the fact that a double arc arrangement increases the robustness of the automated planning technique, it might also improve the plans itself. We were forced to choose different numbers of beams, because one partial VMAT arc was not sufficient for the automatic plans to meet the criteria. To keep the other settings as much the same as possible, we decided to double the beam setup for the automated treatment plans.

Moreover, the four treatment plans from the trial-and-error template were included in the results. These four treatment plans were chosen by a second person and not the investigator. If the template was only focused on these four plans, the template settings would give no good results for the other plans, which was not the case in this study population. The sternum and the ribs were not classified as OARs in any plan by the radiation oncologist. These OARs had not been examined in the template. The brachial plexus is included in three of the twenty-two radiation plans. If the brachial plexus was present in a treatment plan, manual optimization was used, so that the plan clinically met the criteria. In this study no conclusions could be drawn about these critical organs.

The process to generate automated plans completely automatic only succeeded for half of the patient plans. This was due to the fact that only a single template setting was used for all patient plans. However, the other plans could been made clinical acceptable very easy by doing a warm restart and adapting the settings based on the scorecard results. Therefore, we concluded that automated plans were less dependent on the experience of the planner and on the planning process in general. Although there is no limit in making more template settings, our approach was to use one single template for all patient plans. In the future individual adaption of the template could lead to an more individualized automated plan approach.

Our findings indicate, that the plan quality of the automated treatment plans was better than the clinically manually optimized treatment plans for patients with AS-NSCLC and the hands-on-planning time was reduced from two hours to thirty minutes. Our department has implemented automated treatment planning into the standard clinical practice for these patients.

Conflict of interest

All of the authors have no conflict of interest.

Appendix A. Supplementary data

The following are the Supplementary data to this article: