Development and validation of a decision support tool to select IMRT as radiotherapy treatment planning modality for patients with locoregionally advanced non-small cell lung cancers (NSCLC)

Abstract

Objective:

Radiation planning for locally-advanced non-small cell lung cancer (NSCLC) can be time-consuming and iterative. Many cases cannot be planned satisfactorily using multisegment three-dimensional conformal radiotherapy (3DCRT). We sought to develop and validate a predictive model which could estimate the probability that acceptable target volume coverage would need intensity modulated radiotherapy (IMRT).

Methods:

Variables related to the planning target volume (PTV) and topography were identified heuristically. These included the PTV, it’s craniocaudal extent, the ratio of PTV to total lung volume, distance of the centroid of the PTV from the spinal canal, and the extent PTV crossed the midline. Metrics were chosen such that they could be measured objectively, quickly and reproducibly. A logistic regression model was trained and validated on 202 patients with NSCLC. A group of patients who had both complex 3DCRT and IMRT planned was then used to derive the utility of the use of such a model in the clinic based on the time taken for planning such complex 3DCRT.

Results:

Of the 202 patients, 93 received IMRT, as they had larger volumes crossing midline. The final model showed a good rank discrimination (Harrell’s C-index 0.84) and low calibration error (mean absolute error of 0.014). Predictive accuracy in an external dataset was 92%. The final model was presented as a nomogram. Using this model, the dosimetrist can save a median planning time of 168 min per case.

Conclusion:

We developed and validated a data-driven, decision aid which can reproducibly determine the best planning technique for locally-advanced NSCLC.

Advances in knowledge:

Our validated, data-driven decision aid can help the planner to determine the need for IMRT in locally advanced NSCLC saving significant planning time in the process.

Introduction

Three-dimensional conformal radiotherapy (3DCRT) is widely used as the curative treatment for patients with inoperable locoregionally-advanced non-small cell lung cancer (NSCLC).¹ Intensity modulated radiotherapy (IMRT) is used if 3DCRT cannot yield an acceptable radiotherapy plan, as target volume coverage is unsatisfactory or organ at risk constraints are not being met or both.² However, there is a lack of conclusive evidence that IMRT improves outcomes when compared to 3DCRT.^3–5 In lung cancers, better conformity of IMRT is associated with a tradeoff in terms of low dose spillage in the lung. Additionally, IMRT needs robust quality assurance and stringent image guidance, because of the sharp dose gradients.⁶ Hence, if an acceptable treatment plan can be generated, 3DCRT still remains the desirable technique.^3–5

For an individual patient with complex target volume geometry, complex multisegment 3DCRT planning is often attempted before a decision is made to go for IMRT.⁷ The iterative nature of this planning makes this a time consuming process. While experienced dosimetrists or planners may be able to judge the required treatment modality (3DCRT vs IMRT), this is an empirical decision in most cases. The time used in generating multiple complex 3DCRT plans may be saved if a decision support tool is available. Such a decision support system could inform the dosimetrist about the likelihood that IMRT treatment plan would be needed to satisfy the planning objectives, thereby saving time and effort spent in generating a complex 3DCRT plan.

The objective of the current study was to develop a data-driven, decision support tool employing simple to measure metrics from the planning scan, to determine if IMRT will be necessary for a given patient. In order to be clinically useful, we pre-specified that such a tool should have a cross-validated concordance-index (C-index) of 0.80 or better and be available as a nomogram. The model C-index is a measure of the goodness-of-fit for binary outcomes in a logistic regression model and is equal to the area under the curve for a receiver operating characteristic (ROC) curve.⁸ If such a validated model could be developed, then we would also determine the time saved by avoiding attempts at creating complex 3DCRT plans in a separate set of patients.

Methods and materials

Patient population

Medical and radiotherapy planning records of consecutive patients with NSCLC (July 2013–December 2017) treated with curative intent radiotherapy at a tertiary cancer center were retrieved. Treatment plans of 202 patients, of whom 93 had required IMRT, were analyzed. The final treatment plan for these patients had been evaluated and approved using standard planning criteria which are presented in Table 1.^9–12 Eclipse treatment planning system (v. 10.0.42 till September 2017, and v. 15.1.52.01 subsequently, Varian Medical Systems, Palo Alto, CA) was used to generate all the plans. The authors have described the typical planning process used for generating complex 3DCRT and for IMRT (including volumetric arc therapy) plans for lung cancers in an earlier publication.¹³ For all cases, the treatment plan evaluation and approval were done by senior medical physicists and senior clinical oncologists with more than 10 years of experience. Until mid-2014, for all patients, the planning process started with the development of a 3DCRT plan. If the planners failed to create a 3DCRT plan that achieved all of the objective criteria, they would attempt complex multifield and multisegment 3DCRT, before deciding on changing to an IMRT plan. As can be expected, for several cases, significant time would have been spent in iterating a 3DCRT plan before the decision for planning IMRT would be taken. Subsequently, as the team gained more experience senior physicists estimated whether IMRT was needed upfront.

Table 1.

Institutional plan evaluation criteria

Volume	Metric	Criteria	Priority
PTV	D95	≥95%	1
	D107	<1cc	2
	Dmax	<110%	2
Lung-PTV	V20	<35%	2
	V10	<50%	2
	V5	<70%	2
	Mean	<18 Gy	2
Spinal cord	Dmax	<48 Gy (Conventional fractionation) <44 Gy (Accelerated radiotherapy)	1
Heart	Mean	<26 Gy	3
	V30	<46%	3
Esophagus	Dmax	≤Prescribed dose	4

PTV, planning target volume.

Heuristic decision-making

At the start of the study, we asked an experienced clinical oncologist and a senior physicist to review the planning CT for 25 randomly selected patients from this data set. The oncologist and the physicist were free to visualize the target volumes, location and spread in the treatment planning system in any manner required. They were then asked to decide if the patient would need an IMRT plan or would a 3DCRT plan adequately achieve the planning objectives shown in Table 1. The corresponding author noted the time taken for decision making as well as the decision itself on a separate sheet. The result of this heuristic decision making process was then used to determine the accuracy benchmark for the decision support tool. In order to be useful, the decision support tool should have an accuracy equal to or better than that of this heuristic decision making.

Model specification and development

After completing the heuristic decision making, both the clinical oncologist and the senior physicist were asked to describe the reasons that they felt influenced their choice. The factors identified were:

1. Planning target olume (PTV)

2. The ratio of the PTV to the total lung volume

3. The distance between midline to the PTV centroid

4. The distance of the PTV centroid with respect to the spinal canal

5. The craniocaudal extent of the PTV

6. The distance of PTV across the midline

Based on these factors, the following quantitative metrics were identified for developing the model. We selected quantitative metrics to facilitate objective, measurable and reproducible estimation of the parameters for use in the decision support model (Figure 1).

Figure 1.

The figure showing the measures to be obtained (See text and Supplementary Material 1) for details of the measurements.

1. Volume of the PTV: This was measured in cubic centimetres (cc) from the treatment planning system (TPS).

2. Ratio of the PTV to that of the total lung volume (PTV_TLV_ratio): The TLV was the auto-segmented lung volume in the TPS. The volume was also measured directly in the TPS.

3. Lateral distance of the PTV centroid from the midline (Midline_to_Centroid): Measured on the axial plane where the centroid was located. First, an anterior field of 10 × 10 cm was with the isocenter placed at the centroid of the PTV using the automatic centring algorithm of the TPS. The midline was taken as a straight line passing through the mid of the anterior vertebral body. The distance was measured using the distance measurement tool in the TPS.

4. Centroid ventral distance (Centroid_Ant_Distance): Anterior distance of the PTV centroid from the anterior border of the spinal canal along the midline as measured on the axial plane where the centroid was placed. This distance was also measured in the same plane as above.

5. Extent of PTV crossing the midline (Dist_PTV_to_midline): To measure this we first estimated the extent of the PTV which was medial to the centroid of the PTV on the frontal beam’s eye view of the field. This value was subtracted from the lateral distance of the PTV centroid to the midline to obtain the distance by which the PTV crossed the midline, into the contralateral side-of the body. This method was adopted because the maximum extension of the PTV across the midline often lay in a plane which was different from the plane of the centroid.

6. Craniocaudal extent of the PTV (PTV_CCextent): This was measured as the distance between axial slices having the superior most and the inferior most contours of the PTV.

These values were collected for all patients and a logistic regression model was developed to predict the probability that the patient would have a clinically acceptable plan using IMRT. We used the methodology outlined by Harrell et al to develop the model.¹⁴ The detailed model development methodology followed is provided in (Supplementary Material 2) . Briefly the process involved the following:

1. Evaluation of the relationship between the variables and the independent variable. Differences in the mean values for the variables in IMRT vs 3DCRT were explored using the Wilcoxon signed rank test.

2. Redundancy analysis for the variables with a flexible parametric additive model investigating how well each variable could be predicted from the others. All continuous variables were expanded using restricted cubic splines (RCS) with three knots.

3. Evaluation of a full model which all continuous variables were expanded with RCS with three knots in order to evaluate the relationship between the log odds ratio and the covariate value. This relationship was evaluated using the analysis of variance test and graphically in order to determine which of the covariates fulfilled the assumption of linearity.

4. Testing of prespecified interactions to evaluate if a model with interactions would give a better predictive ability.

5. Bootstrap resampling (500 resamples) to check model discrimination and calibration. Model discrimination allows us to predict how well the given model will predict in other samples, while calibration checks the errors in the prediction across a range of predicted probabilities.

6. Plotting of the nomogram for the final model.

Internal validation

First, this model was used for predicting the probabilities for requiring IMRT or 3DCRT in a population of 17 patients treated in 2018. This cohort of patients was not used in the model development process. The accuracy, sensitivity and specificity of the model were calculated for the predicted values. Model-predicted probabilities of >50% were categorized as IMRT as rest as 3DCRT.

Utility of model

The primary utility of this model lies in saving the time a dosimetrist would use, for creating a 3DCRT plan which would be found inadequate, before deciding to change the planning approach to IMRT. In order to estimate the time spent by dosimetrists in creating the complex 3DCRT plans, that were subsequently deemed unsatisfactory, we obtained planning time data for 15 patients with NSCLC (three patients with small cell cancers were excluded) in whom both complex 3DCRT and IMRT plans were done.¹³ In these patients, both plans had been done as a part of a service development audit. The results from this audit which have been published previously demonstrated that IMRT resulted in improved target volume coverage as compared to complex, multi segment 3DCRT.¹³ The time required for planning 3DCRT was obtained by reviewing the editing log available in the treatment planning system. All of these patients were a part of the model building data set and hence data of these patients were not used for the internal validation.

Results

Subject population

Table 2 shows the distribution of the model parameters in patients undergoing 3DCRT vs IMRT. As expected patients requiring IMRT had larger tumor volumes and more centrally located tumors and all parameters were significantly different between the two groups at a p-value of <0.05.

Table 2.

Distribution of model parameters in patients undergoing 3DCRT vs IMRT

Parameter	3DCRT (n = 109)	IMRT (n = 93)	p-value
PTV (cc)	690.54 (325.27)	865.63 (360.02)	<0.01
PTV:TLV ratio	0.27 (0.16)	0.33 (0.16)	<0.01
Midline to centroid distance (cm)	5.10 (1.61)	4.33 (1.36)	<0.01
Centroid distance from spinal canal (cm)	4.14 (1.97)	3.51 (2.05)	0.01
PTV craniocaudal extent (cm)	11.96 (2.59)	13.88 (3.62)	<0.01
Extent of PTV crossing the midline (cm)	1.51 (1.93)	3.60 (1.86)	<0.01

3DCRT, three-dimensional conformal radiotherapy; IMRT, intensity modulated radiotherapy; PTV, planning target volume; TLV, total lung volume.

Mean and standard deviation are displayed. Difference calculated using the Wilcoxon test.

Initial heuristic decision-making

In the initial round of the testing the heuristic decision-making, both the oncologist and the physicist accurately predicted the planning modality in 65% of the cases. There was a disagreement in three cases between the raters. The majority of disagreement and incorrect prediction occurred in tumors which were central in location but not closely abutting the spinal canal. An interview with the two raters after this exercise confirmed the choice for the variables. Our choice of at least 80% accuracy was deemed to be reasonable after this round.

Model results

The initial model with all continuous terms expanded using RCS with three knots had a Harrell’s C index of 0.867 and a Brier score of 0.147. Examination of the plots of the log odds ratio against the values of the individual variables showed that PTV:TLV ratio, PTV centroid to spinal canal distance, and craniocaudal PTV extent had a non-linear relationship. The results of the analysis of variance test confirmed the graphical findings. Hence, the decision was made to expand these three variables with RCS. The reduced model had a C-index of 0.866 and a Brier score of 0.148 indicating that the predictive accuracy would be maintained (Supplementary Material 1 for the details of the models and the analysis).

Redundancy analysis showed that PTV volume could be considered as a redundant variable at a threshold R2 of 0.7. Hence, a further model reduction was made where PTV was dropped from the model and ratio was retained. This final reduced model with 8 degrees of freedom had a C-index of 0.867 and a brier score of 0.148 (Figure 2, panel A).(Figure 2) The variables Dist_PTV_to_midline and the Centroid_Ant_Distance influenced the model significantly with p-values of < 0.05. Cross-validation of the model discrimination showed that the model C-index was maintained at 0.843. The calibration curve shown in Figure 3, panel B shows that the model maintained good predictive accuracy across the range of predictions with a mean absolute error of 0.015.

Figure 2. .

Panel A Shows the summary of the logistic regression model represented as the odds ratio. For each continuous variable comparisons are made between the values corresponding to the first and the third quartile. Solid blue lines represent the 95% confidence intervals of the estimate of the odds ratio. Panel B: Represents the calibration plot for the model with the dark line being the bias corrected line obtained from 500 bootstrap resamples.

Figure 3. .

Nomogram showing the probability of the patient requiring IMRT. PTV_TLV_ratio = PTV to TLV ratio, Midline_to_centroid = Distance between PTV centroid to midline, PTV_CC_extent = Craniocaudal extent of the PTV, Centroid_Ant_Distance = Distance between the PTV centroid and the anterior spinal canal, Dist_PTV_to_Midline = Distance of the PTV across the midline. Total points of > 98 would indicate that the patient has a > 50% probability of needing IMRT.

The nomogram of the final model is shown in Figure 3 numerical summary of the model is further presented in Appendix A to ease calculations.

Internal validation

Internal validation of the model in a discreet data set not used for model development showed that the accuracy of the model in predicting that a patient would need IMRT was 94.12% (95% confidence intervals: 71.3–99.9%). The positive predictive value was 91.7% and the negative predictive value was 100%.

Utility

In the sample of 15 patients chosen for analysis of utility, the nomogram correctly predicted the use of IMRT in 12 patients (accuracy of 80%). There were three patients where there was a discrepancy in the nomogram prediction and the actual planning modality chosen. These cases were analyzed retrospectively, and of the three patients, one had a deliverable 3DCRT plan but IMRT had been used. In another patient, IMRT was used as two separate PTV volumes were present and use of 3DCRT would have necessitated a dual isocenter plan which would be more complex. However, a dual isocenter 3DCRT plan was feasible for that patient. Finally, in the third patient, the choice of IMRT was dictated by the proximity to the heart and consequently higher doses to the heart. The median total time spent in obtaining an acceptable 3DCRT plan was 168 min (interquartile range: 82.5–399.0 min). Using this nomogram would have saved a median time of 2 h 48 min of a planners time for each patient, in this situation.

Discussion

Radiotherapy treatment planning for lung cancer is complex and often involves trade-offs between target volume coverage and sparing of the organs at risk (OARs). Critical organs with a serial architecture like spinal cord are significant impediments as the nodal target volume often overlaps with the cord. Lateral fields expose the opposite lung to significant radiation doses, which increase the risk of radiation-induced pneumonitis.

IMRT for lung cancer enables better coverage of large or complex-shaped target volumes, while allowing adequate sparing of the OARs.^2,13,15 It allows the simultaneous treatment of multiple discrete targets, using a single isocentre.⁶ However, a steep dose gradient renders the plan susceptible to motion interplay and necessitates robust image guidance. While the use of IMRT is being explored in isotoxic dose-escalation,^16,17 the routine clinical use of IMRT is often limited in NSCLC.^2,13,15

The advantages of 3DCRT include no contouring of control structures, simpler forward planning and plan assessment, and no requirement for patient-specific quality assurance.⁶ For many patients, IMRT does not confer any clinically significant dosimetric advantage, and 3DCRT meets all the planning criteria.^2,4

It stands to reason that it would be an advantage if the dosimetrist can predict if a complex 3DCRT plan would meet the dosimetric acceptance criteria, before spending several hours on planning. Planning complex 3DCRT is often time-consuming, as shown in the current study. This estimate of time has been obtained from the editing logs of the TPS and is likely to be an underestimate as the log does not record time spent in adjusting weights or multileaf collimators.

Use of this model allows the dosimetrist to predict the probability that a patient would need IMRT to ensure appropriate PTV coverage. Any patient with greater than 50% model predicted probability should be planned with IMRT upfront. This prediction is a function of the target volume, topography, and proximity to the spinal canal. The strongest predictor is the extent by which the PTV crosses the midline. However, additional variables are needed, as complex 3DCRT may be used for PTV which are sufficiently ventral to the spinal canal.

While deciding on the list of variables, we opted for the variables that would be simple to measure and be reproducible. Astute dosimetrists would know that IMRT is needed when the PTV has a concave surface in proximity to the spinal canal. However, this concavity is difficult to measure reproducibly. While an angle of contact metric may be used (as is used to determine inoperability),¹⁸ the measurement will be more variable.

Our model does not incorporate variables which are related to the doses to the OARs directly, but the variables like the ratio of the PTV to the total lung volume, midline_to_centroid distance and volume of PTV are all surrogates for doses to these structures. The choice of the variables was intentional as we wanted to develop the model using data which would be available to the planner before the first plan is generated. The model was developed in clinically approved plans in which doses of the critical structures met the acceptable standards and thus the use of this model allows the planner to choose the modality for planning (viz. 3DCRT or IMRT) with a greater degree of confidence.

In one of the patients, the model showed that an acceptable 3DCRT would be achievable, but the final choice for IMRT plan was made to reduce heart dose.^1,19 Dose constraints for the heart are poorly defined for NSCLC, though emerging evidence suggests a need to reduce the dose.²⁰ In the current model, there is no variable which accounts for the proximity to the heart. A variable which determines the extent of contact or overlap with cardiac contour may be used but would increase the complexity of collecting data significantly.

Data-driven decision-making is an emerging field in radiation oncology.²¹ Existing TPS allows knowledge-based planning to inform the planner regarding the achievable doses with IMRT.^22,23 The current tool enables the planner to choose the planning modality viz. 3DCRT or IMRT, with a good degree of confidence. To our knowledge, none of the commercial knowledge-based planning tools provide this information. Our model does not inform about the possible doses to the OARs and this could be an area of active research in the future.

Another limitation of the nomogram is the lack of external validation in a cohort of patients treated outside our hospital (Tata Medical Center) and we are actively exploring avenues to validate this nomogram in this manner. Part of the model performance may be related to the similarity in the target volume delineation practice and disease burden in our center. The heuristic decision-making process employed at the beginning of the model development process is not ideal as heuristic decision-making incorporates the inherent bias that accompanies decision making by humans. The ideal way to get around this problem would have been using automatic planning for 3DCRT and IMRT but none of the commercially available planning systems allow this. We are planning an external validation study and will also be exploring the real-world gains in treatment planning time reduction achieved by the use of this planning technique.

Conclusion

A model that predicts the need to use IMRT for lung cancer with a high degree of accuracy was developed and validated. We expect that the use of this model can significantly reduce the time taken in the treatment planning for complex, locoregionally advanced NSCLC patients. A dynamic nomogram of this model is being used in clinical practice at our institute.