Comparison of complexity metrics for multi-institutional evaluations of treatment plans in radiotherapy

Victor Hernandez; Jordi Saez; Marlies Pasler; Diego Jurado-Bruggeman; Nuria Jornet

doi:10.1016/j.phro.2018.02.002

. 2018 Feb 22;5:37–43. doi: 10.1016/j.phro.2018.02.002

Comparison of complexity metrics for multi-institutional evaluations of treatment plans in radiotherapy

Victor Hernandez ^a,^1,^⁎, Jordi Saez ^b,¹, Marlies Pasler ^c, Diego Jurado-Bruggeman ^d, Nuria Jornet ^e

PMCID: PMC7807588 PMID: 33458367

Highlights

•
Several complexity metrics were highly correlated and can be considered equivalent.
•
Other metrics produced different results, especially for plans from different TPSs.
•
Different TPSs prioritise modulation of different plan parameters.
•
The ranking of plan complexity can greatly depend on the metric used.
•
This must be carefully considered in multi-institutional plan comparisons.

Keywords: Plan complexity, Complexity metrics, Audits, Clinical trials

Abstract

Background and purpose

It is known that intensity-modulated radiotherapy plans that are highly complex might be less accurate in dose calculation and treatment delivery. Multiple complexity metrics have been proposed, but the relationships between them have not been thoroughly investigated. This study investigated these relationships in multi-institutional comparisons of treatment plans, where plans from multiple treatment planning systems (TPSs) are typically evaluated.

Materials and methods

A program was developed to compute several complexity indices and provide analysis of dynamic plan parameters. This in-house software was used to analyse plans from a recent multi-institutional audit. Additionally, 100 clinical volumetric modulated arc therapy (VMAT) plans from two institutions using different TPSs were analysed.

Results

All plans produced satisfactory pre-treatment verification results and, hence, complexity metrics could not be used to predict plans failing QA. Regarding the relationship among complexity indices, some very strong correlations were found (r > 0.9 with p < 0.01). However, some relevant discrepancies between complexity indices were obtained, even with negative correlation coefficients (r ∼ −0.6) which were expected to be positive. These discrepancies could be explained because each complexity index focused on different features of the plan and different TPSs prioritised modulation of different plan parameters.

Conclusions

Some complexity indices provided similar information and can be considered equivalent. However, indices that focused on different plan parameters yielded different results and it was unclear which complexity index should be used. Careful consideration should be given to the use of complexity metrics in multi-institutional studies.

1. Introduction

Advances in the technology for planning and delivery of radiotherapy treatments allow for highly conformal dose distributions to be achieved. However, these distributions require modulation of many machine parameters [1], [2], [3], [4], [5]. Since additional sources of variability are thus introduced, treatment plans with similar dose distributions may differ greatly in their complexity. Many investigators have reported that the degree of plan complexity may affect the accuracy of dose calculations and treatment delivery [6], [7], [8], [9], [10], [11], [12], [13], which is crucial in dosimetry audits and clinical trials, as well as for big data analysis [14], [15], [16]. Therefore, aspects such as quality and complexity of treatment plans have to be carefully evaluated in multi-institutional plan comparisons [17].

Several investigators have proposed different complexity metrics and have reported correlations with overall accuracy and the resulting quality assurance (QA) metrics [6], [7], [8], [9], [10], [11], [12], [13]. Thus, less complex plans offer several benefits such as more accurate dose calculations, more accurate and robust treatment delivery, better QA metrics and even lower risk of intra-fraction movements and patient variations [6], [7], [8], [9], [10]. For all these reasons plans with low complexity are associated with lower uncertainties and can be considered, in general, more robust than highly complex plans.

AAPM pointed out the need to incorporate measures of beam modulation to ensure that centres achieve intensity-modulated radiation-therapy (IMRT) plans that are comparable with regards to their complexity [18]. However, it is not clear which of the proposed complexity indices should be used and the relationship between these multiple indices in multi-TPS environments has not been previously addressed. In this study we investigated the use of complexity metrics in multi-institutional comparisons where multiple TPSs, planners and linac types are typically involved. The study focused on volumetric modulated arc therapy (VMAT) treatments, but most of the indices evaluated can also be applied to other techniques such as sliding window and step-and-shoot IMRT.

2. Materials and methods

2.1. Complexity metrics

In this study several complexity indices that are computed from the treatment plan parameters defined at each of the control points of the plan were investigated. These indices allow for a detailed analysis of the dynamic parameters involved in treatment plans, which makes them more appropriate for VMAT than ‘fluence-based’ indices. The following indices were evaluated:

a)
Modulation Complexity Score (MCS) [6]. This score integrates two contributions to complexity: variability in the shape of segments and variations in their area. MCS uses a fixed range from 0 to 1 and, unlike the rest of the complexity indices, it is defined in such a way that the lower the value of the MCS the higher the complexity. It was initially designed for step-and-shoot treatments and later adapted to sliding window and VMAT [9], [19].
b)
Edge metric (EM) [8]. This metric computes the complexity of multileaf collimator (MLC) apertures based on the ratio of MLC side edge length and aperture area. In this study the original recommendation for the parameters (C1 = 0 and C2 = 1) was followed. Thus, the greater the differences between the positions of adjacent leaves the higher the EM index, which is closely related to the amount of tongue-and-groove effect.
c)
Leaf travel (LT) [9]. This index indicates the average distance travelled by the moving leaves. LT was devised for VMAT treatments consisting of a single full arc. To allow for simple comparisons between plans with a different number of arcs or with partial arcs, we divided LT by the corresponding arc length (typically about 360 deg for single arcs and about 720 deg for double arcs) and we named this index as ‘LT/AL’.
d)
Plan irregularity (PI) and Plan modulation (PM) [11]. PI describes the deviations of aperture shapes from a circle, being 1 for a perfect circle. PM indicates to what extent a beam is modulated with multiple smaller segments.
e)
Modulation index total (MItotal) [10]. This index evaluates the variations in speed and acceleration of the MLC as well as variations of the gantry speed and the dose rate. MItotal is, to our knowledge, the only complexity index that takes into account the modulation of the dose rate and the gantry speed.

2.2. Treatment plans evaluated

The first group of plans evaluated in this study included forty plans from a recent audit promoted by the Catalan Association of Medical Physics within the framework of the Catalan-Occitan Oncology Group (GOCO). This audit included local pre-treatment verification results and independent dosimetry audit measurements [20]. A mock head-and-neck and a mock prostate case adapted from those proposed in TG119 were used. Most plans (twenty-eight) were produced with Eclipse™ (Varian Medical Systems), eight plans were generated with Pinnacle Auto-Planning (Philips Radiation Oncology Systems) and four plans with Monaco (Elekta AB). Hereafter these TPSs will be called TPS-A, TPS-B and TPS-C, respectively. Details on the TPSs, the linacs and the methodology used can be found in the aforementioned publication.

Additionally, in the present study clinical plans from TPS-A and TPS-B were also analysed. In particular, fifty head-and-neck VMAT plans and fifty prostate VMAT plans from each TPS were randomly selected and evaluated. Plans from TPS-A and TPS-B were produced for a Varian Clinac iX (Millennium 120 MLC) and an Elekta Synergy (MLCi2, binned dose rate), respectively.

2.3. Software and equipment used

To compute the previously described complexity indices, an in-house program called PlanAnalyser was developed in MATLAB (Mathworks, Massachusetts, USA). This software reads the DICOM plan as exported from the TPS and computes complexity indices using the data contained in the DICOM plan. Plan complexity indices were computed by joining all beams and performing the calculations for the ‘combined’ beam.

PlanAnalyser incorporates an emulator that predicts the variations of the dynamic plan parameters during treatment delivery. Since one of the complexity metrics (MItotal) evaluates the variations of the dose rate and gantry speed, we investigated the modulation of these parameters. Mean variations were defined as the total variation (i.e., sum of all variations between consecutive control points) divided by the total arc length. To verify the predictions from the emulator they were compared to results from log files for both Elekta and Varian linacs. Varian log files were analysed with in-house software [21] and log files from Elekta were recorded with the service graphing module of the linac controller (Integrity 1.2).

Pre-treatment verifications were carried out for all plans. Audit plans were measured with both independent QA equipment (ArcCHECK, Sun Nuclear Corporation) and a large variety of local QA devices [20]. Clinical plans from TPS-A and TPS-B were measured with ArcCHECK and Octavius II – 2D array seven29 (PTW Freiburg), respectively. Since audit plans corresponded to the same mock cases, a plan quality score was computed with the software PlanIQ™ (Sun Nuclear Corporation) in order to identify which plans achieved the best trade-off between target coverage, homogeneity, conformity, and doses to organs at risk [20].

To investigate the dependencies among these indices, the Spearman’s rank correlation coefficients r, sensitive to both linear and non-linear correlations, were calculated. The strength of the association, for absolute values of r, 0–0.19 was regarded as ‘no correlation’, 0.20–0.39 as ‘weak’, 0.40–0.59 as ‘moderate’, 0.60–0.79 as ‘strong’ and 0.80–1 as ‘very strong’. To account for multiple testing, false discovery rates (q-values) [22], [23] were calculated. Reported p-values represent statistical analysis without multiple testing correction and statistical significance was considered at p < 0.05 with q-value < 0.1. All statistical analysis was performed in R-3.3.2 (R: A Language and Environment for Statistical Computing, 2016, Vienna, Austria).

3. Results

3.1. Audit plans

All participating centres fulfilled all the requested planning goals regarding both target coverage requirements and dose limits to organs at risk. Large differences in the degrees of plan complexity were observed, but no statistically significant correlation was found between dosimetric plan complexity and plan quality [20]. Pre-treatment verification results were clinically acceptable for all plans (>95% of points with gamma 3%/3 mm < 1), hence complexity metrics could not be used to predict plans failing QA.

Regarding the comparison between complexity indices, strong correlations were found between MCS, PI and EM. However, we also observed some evident discrepancies, meaning that some plans were more complex than others according to a particular complexity index, while the opposite result was found when another complexity index was considered. In Fig. 1 the comparison between four pairs of indices is shown and a more detailed comparison between all pairs of indices is provided as Supplementary material (Fig. S1). Spearman’s correlation coefficients are given in Fig. 2.

Fig. 2 — Spearman’s correlation coefficients and their statistical significance p and false discovery rates q for the (a) head-and-neck and (b) prostate plans from the audit. Statistically significant correlations are marked (*) and coded in colour.

Fig. 1a shows an inverse correlation between MCS and EM. This was expected because lower MCS values are linked to higher plan complexities and results in negative correlation coefficients. Indeed, a strong correlation was found between MCS and EM, with r = −0.63 (p = 0.003) for head-and-neck and r = −0.68 (p = 0.001) for prostate mock cases. However, six plans from TPS-A yielded a high complexity according to EM (with values 0.5–0.55) while their complexity according to MCS was very similar to the other plans with the same TPS. All plans from TPS-A were more complex than plans from TPS-C according to these indices (i.e., higher EM and lower MCS values), while plans from TPS-B fell between them.

As shown in Fig. 1b, a very strong linear correlation was also found between PI (related to MLC aperture irregularity) and EM (related to the tongue-and-groove effect) for a given treatment site, with r > 0.95 (p < 0.0001). The ratio between the two indices depended on the treatment site, probably because the size of the target volumes was quite different and EM is not dimensionless. Thus, PI revealed higher complexities for head-and-neck plans than for prostate plans, while EM indicated similar complexities in both cases. According to PI, plans from TPS-C were also the least complex and plans from TPS-A were the most complex.

On the contrary, no statistically significant correlation between MItotal and EM was found for prostate cases and a negative moderate correlation (r = −0.58, p = 0.007) was found for head-and-neck plans. In general, Fig. 1c shows that plans with higher EM values did not produce higher values of MItotal. Interestingly, MItotal provided similar or higher plan complexities for both TPS-B and TPS-C with respect to TPS-A, while EM yielded opposite results. As it can be observed, there was no evident correlation between MItotal and EM for any TPS and the negative correlation coefficient for head-and-neck plans was caused by these differences between TPSs.

The relationship between MItotal and LT/AL is illustrated in Fig. 1d. Although these two indices focus on different features of the plan, for plans from TPS-A a very strong correlation was obtained regardless of the treatment site, as the regression line in Fig. 1d shows (r > 0.99, p < 0.0001). On the contrary, when all TPSs were considered the overall correlation was much weaker and only statistically significant for the head-and-neck plans (see Fig. 2).

The variations in dose rate and gantry speed for plans from different TPSs are shown in Fig. 3. Variations for TPS-B and TPS-C were much larger than those produced by TPS-A, which indicates that the optimisation engines in TPS-B and TPS-C further modulate both the dose rate and the gantry speed.

To confirm these variations and validate our predictions, log files were collected and analysed. Data from log files was in good agreement with predictions from the emulator and confirmed the different degree of modulation of these parameters depending on the TPS (see Fig. 4).

Fig. 4 — Dose rate and gantry speed as a function of the gantry angle for plans from TPS-B (top) and TPS-A (bottom). Data from log files and predictions from the emulator are given for two representative plans corresponding to the same head-and-neck case.

3.2. Clinical plans

Pre-treatment verification was carried out for all the clinical plans evaluated. All QA results were also clinically acceptable (>95% of points with gamma 3%/3 mm < 1). Complexity indices from clinical plans were, in general, similar to those found in the audit, which confirmed that plans from the audit were representative of clinical practice. For clinical plans the variability in complexity scores was higher because they included a wide variety of cases with large anatomical variations, especially for head-and-neck cases.

The relationships between complexity indices and their correlations were similar to those found in the audit. A detailed comparison of all the indices and the correlations obtained is provided as Supplementary material (Figs. S2 and S3). For comparison purposes, the relationship between MItotal and EM for clinical plans is shown in Fig. 5a and b. Discrepancies were very similar to those observed in the audit, with plans from different TPSs being more complex according to EM and less complex according to MItotal. Again, plans from TPS-A involved much lower variations of dose rate and gantry speed, with practically no variations in gantry speed when multiple arcs were used (see Fig. 5c and d). TPS-B, on the contrary, produced much larger variations in both dose rate and gantry speed regardless of the number of arcs involved. Boxplots illustrating these differences are provided as Supplementary material (Fig. S4).

Fig. 5 — The comparison between MItotal and EM is shown for (a) head-and-neck and (b) prostate clinical plans. In (c) and (d) the corresponding mean gantry speed and mean dose rate variations are shown.

4. Discussion

We found very strong correlations between some of the complexity indices that evaluate similar parameters (e.g., MCS, PI and EM), which means they provide similar information. In general, indices that focused on different features of the plan produced much weaker correlations or no statistically significant correlations. Some discrepancies appeared between complexity metrics such as MItotal and EM for both audit plans and clinical plans when plans produced by different TPSs were compared. These discrepancies can be explained by the large differences in the degree of modulation of the dose rate and the gantry speed depending on the TPS. Indeed, MItotal is further increased in plans that further modulate the dose rate and the gantry speed, while the other complexity indices do not take these variations into account. The variability of complexity metrics depended on the TPS, which indicates that the ability of each index to discriminate between plans depends on the TPS. However, it is not clear which index is more relevant for each TPS or linac. This should probably be investigated in every particular situation and based on specific QA results.

The discrepancies between different complexity metrics indicate that no individual metric is sufficient for all TPSs and that several complexity metrics should be evaluated. Additionally, in order to identify excessively complex plans, multi-institutional comparisons should be carried out per TPS. Similarly, acceptable ranges or threshold levels in complexity metrics depend on the TPS model and hence must be evaluated for each specific TPS.

The analysis of complexity indices revealed differences in the optimisation engines and sequencers of each TPS, which prioritise the modulation of different plan parameters. A large spread in some of the complexity indices was observed depending on the TPS, in EM for TPS-A, for instance, and in MItotal for TPS-B and TPS-C, although all these plans achieved similar dosimetric plan quality. This indicates that more complex plans do not necessarily produce better dose distributions, which has been reported by other investigators [8], [24]. A certain degree of plan complexity is not per se negative, because some degree of complexity is needed to achieve clinically acceptable dose distributions [25]. However, unnecessary complexity should be avoided and excessively complex plans might compromise the accuracy of dose calculations in the TPS and the accuracy of treatment delivery [6], [7], [8], [9], [10], [11], [12], [13]. For that reason some authors recommend incorporating complexity metrics into the cost function used by optimisation algorithms [6], [7], [8], [24], [26]. In our audit, we found several plans with a much higher degree of complexity that produced dose distributions of similar dosimetric quality. By incorporating these metrics into optimisation algorithms, the degree of modulation of dynamic plan parameters could be further controlled and this unnecessary complexity might be greatly reduced.

The fact that plans produced by different TPSs prioritise the modulation of different plan parameters can also have implications in the commissioning and QA of linacs and TPSs. For instance, plans from TPS-A were more demanding for the MLC, which makes them more sensitive to uncertainties associated to the use of small MLC openings, as well as to potential errors in the MLC calibration. On the other hand, plans from TPS-B and TPS-C further modulated the gantry speed and the dose rate and were, therefore, more demanding for the gantry assembly and beam modulation. Complexity metrics can be useful to understand these differences and to better adapt QA programs to each particular situation.

Many investigators have reported that plan complexity indices are correlated with QA metrics [6], [7], [8], [9], [10], [11], [12], [13]. However, in this study all plans produced satisfactory QA results and more stringent QA criteria might be needed to investigate these potential correlations. If multi-institutional comparisons (such as in audits and clinical trials) include some plans that fail QA, complexity metrics can indicate if these plans are particularly complex, which might help understand the causes for poor QA results. In general, these causes may depend on the linac model and its proper maintenance, as well as on the limitations of the TPS used (beam model, dose engine and potential commissioning inaccuracies). Since different causes would originate different correlations between QA results and complexity metrics, these correlations are not generic and might be harder to find in multi-institutional comparisons, where different causes for poor QA results could interfere.

One limitation of this study is that plans from only a few TPSs were evaluated. We found, however, clear differences in the degree of modulation of their plan parameters and in some of their corresponding complexity metrics. Another limitation is that only VMAT plans were analysed, but the software developed can also be applied to other IMRT techniques.

In conclusion, strong correlations were found between several complexity metrics, which show that some indices provide similar information and can be considered equivalent. However, some relevant discrepancies between complexity metrics were also found and it is unclear which complexity index should be used. The ranking of plans according to their degree of complexity greatly depends on the metric used and on the features evaluated by each index, especially for plans from different TPSs. This must be carefully considered in multi-institutional plan comparisons, such as audits and clinical trials.

Conflict of interest statement

None declared.

Footnotes

^{Appendix A}

Supplementary data associated with this article can be found, in the online version, at https://doi.org/10.1016/j.phro.2018.02.002.

Appendix A. Supplementary data

Supplementary data 1

mmc1.doc^{(2.9MB, doc)}

References

1.Ling C.C., Zhang P., Archambault Y., Bocanek J., Tang G., LoSasso T. Commissioning and quality assurance of RapidArc radiotherapy delivery system. Int J Radiat Oncol Biol Phys. 2008;72(2):575–581. doi: 10.1016/j.ijrobp.2008.05.060. [DOI] [PubMed] [Google Scholar]
2.Van Esch A., Huyskens D.P., Behrens C.F., Samsoe E., Sjolin M., Bjelkengren U. Implementing RapidArc into clinical routine: a comprehensive program from machine QA to TPS validation and patient QA. Med Phys. 2011;38(9):5146–5166. doi: 10.1118/1.3622672. [DOI] [PubMed] [Google Scholar]
3.Thwaites D. Accuracy required and achievable in RT dosimetry: have modern technology and techniques changed our views? J Phys Conf Ser. 2013;444:012006. [Google Scholar]
4.van der Merwe D., Van Dyk J., Healy B., Zubizarreta E., Izewska J., Mijnheer B. Accuracy requirements and uncertainties in radiotherapy: a report of the International Atomic Energy Agency. Acta Oncol. 2017;56(1):1–6. doi: 10.1080/0284186X.2016.1246801. [DOI] [PubMed] [Google Scholar]
5.Li J.S., Lin T., Chen L., Price R.A., Jr, Ma C.M. Uncertainties in IMRT dosimetry. Med Phys. 2010;37:2491–2500. doi: 10.1118/1.3413997. [DOI] [PubMed] [Google Scholar]
6.McNiven A.L., Sharpe M.B., Purdie T.G. A new metric for assessing IMRT modulation complexity and plan deliverability. Med Phys. 2010;37(2):505–515. doi: 10.1118/1.3276775. [DOI] [PubMed] [Google Scholar]
7.Nauta M., Villarreal-Barajas J.E., Tambasco M. Fractal analysis for assessing the level of modulation of IMRT fields. Med Phys. 2011;38(10):5385–5393. doi: 10.1118/1.3633912. [DOI] [PubMed] [Google Scholar]
8.Younge K.C., Matuszak M.M., Moran J.M., McShan D.L., Fraass B.A., Roberts D.A. Penalization of aperture complexity in inversely planned volumetric modulated arc therapy. Med Phys. 2012;39(11):7160–7170. doi: 10.1118/1.4762566. [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Masi L., Doro R., Favuzza V., Cipressi S., Livi L. Impact of plan parameters on the dosimetric accuracy of volumetric modulated arc therapy. Med Phys. 2013;40(7):071718. doi: 10.1118/1.4810969. [DOI] [PubMed] [Google Scholar]
10.Park J.M., Park S., Kim H., Kim J.H., Carlson J., Ye S.J. Modulation indices for volumetric modulated arc therapy. Phys Med Biol. 2014;59(23):7315–7340. doi: 10.1088/0031-9155/59/23/7315. [DOI] [PubMed] [Google Scholar]
11.Du W., Cho S.H., Zhang X., Hoffman K.E., Kudchadker R.J. Quantification of beam complexity in intensity-modulated radiation therapy treatment plans. Med Phys. 2014;41(2):21716. doi: 10.1118/1.4861821. [DOI] [PubMed] [Google Scholar]
12.Götstedt J., Hauer A.K., Bäck A. Development and evaluation of aperture-based complexity metrics using film and EPID measurements of static MLC openings. Med Phys. 2015;42:3911–3921. doi: 10.1118/1.4921733. [DOI] [PubMed] [Google Scholar]
13.Crowe S.B., Kairn T., Kenny J., Knight R.T., Hill B., Langton C.M. Treatment plan complexity metrics for predicting IMRT pre-treatment quality assurance results. Australas Phys Eng Sci Med. 2014;37(3):475–482. doi: 10.1007/s13246-014-0274-9. [DOI] [PubMed] [Google Scholar]
14.Kouloulias V.E., Poortmans P.M., Bernier J., Horiot J.C., Johansson K.A., Davis B. The Quality Assurance programme of the radiotherapy group or European Organisations for Research and Treatment of Cancer (EORTC): a critical appraisal of 20 years of continuous efforts. Eur J Cancer. 2003;29(4):430–437. doi: 10.1016/s0959-8049(02)00113-2. [DOI] [PubMed] [Google Scholar]
15.Ibbott G.S., Followill D.S., Molineu H.A., Lowenstein J.R., Alvarez P.E., Roll J.E. Challenges in Credentialing Institutions and Participants in advanced technology Multi-institutional Clinical trials. Int J Radiat Oncol Biol Phys. 2008;71:S71–S75. doi: 10.1016/j.ijrobp.2007.08.083. [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Budiharto T., Musat E., Poortmans P., Hurkmans C., Monti A., Bar-Deroma R. Profile of European radiotherapy departments contributing to the EORTC Radiation Oncology Group (ROG) in the 21st century. Radiother Oncol. 2008;88(3):403–410. doi: 10.1016/j.radonc.2008.05.013. [DOI] [PubMed] [Google Scholar]
17.McGarry C.K., Agnew C.E., Hussein M., Tsang Y., McWilliam A., Hounsell A.R. The role of complexity metrics in a multi-institutional dosimetry audit of VMAT. Br J Radiol. 2016;1057(89) doi: 10.1259/bjr.20150445. [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Ezzell G.A., Burmeister J.W., Dogan N., LoSasso T.J., Mechalakos J.G., Mihailidis D. IMRT commissioning: Multiple institution planning and dosimetry comparisons, a report from AAPM Task Group 119. Med Phys. 2009;36(11):5359–5373. doi: 10.1118/1.3238104. [DOI] [PubMed] [Google Scholar]
19.Svenssons E. University of Gothemburg; 2011. Evaluation of complexity and deliverability of IMRT-treatment plans. (Master thesis) [Google Scholar]
20.Jurado-Bruggeman D., Hernández V., Sáez J., Navarro D., Pino F., Martínez T. Multi-centre audit of VMAT planning and pre-treatment verification. Radiother Oncol. 2017;124(2):302–310. doi: 10.1016/j.radonc.2017.05.019. [DOI] [PubMed] [Google Scholar]
21.Hernandez V., Abella R., Calvo J.F., Jurado-Bruggemann D., Sancho I., Carrasco P. Determination of the optimal tolerance for MLC positioning in sliding window and VMAT techniques. Med Phys. 2015;4:1911–1916. doi: 10.1118/1.4915541. [DOI] [PubMed] [Google Scholar]
22.Storey J.D. A direct approach to false discovery rates. J R Stat Soc Ser B. 2002;64:479–498. [Google Scholar]
23.Storey JD, Bass AJ, Dabney A, Robinson D. qvalue: Q-value estimation for false discovery rate control. R Package 2.6.0. 2015. http://github.com/jdstorey/qvalue.
24.Younge K.C., Roberts D., Janes L.A., Anderson C., Moran J.M., Matuszak M.M. Predicting deliverability of volumetric-modulated arc therapy (VMAT) plans using aperture complexity analysis. J Appl Clin Med Phys. 2016;17(4):124–131. doi: 10.1120/jacmp.v17i4.6241. [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Webb S. Use of a quantitative index of beam modulation to characterize dose conformality: illustration by a comparison of full beamlet IMRT, few-segment IMRT (fsIMRT) and conformal unmodulated radiotherapy. Phys Med Biol. 2003;48(14):2051–2062. doi: 10.1088/0031-9155/48/14/301. [DOI] [PubMed] [Google Scholar]
26.McGarry C.K., Chinneck C.D., O’Toole M.M., O’Sullivan J.M., Prise K.M., Hounsell A.R. Assessing software upgrades, plan properties and patient geometry using intensity modulated radiation therapy (IMRT) complexity metrics. Med Phys. 2011;38(4):2027–2034. doi: 10.1118/1.3562897. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data 1

mmc1.doc^{(2.9MB, doc)}

[b0005] 1.Ling C.C., Zhang P., Archambault Y., Bocanek J., Tang G., LoSasso T. Commissioning and quality assurance of RapidArc radiotherapy delivery system. Int J Radiat Oncol Biol Phys. 2008;72(2):575–581. doi: 10.1016/j.ijrobp.2008.05.060. [DOI] [PubMed] [Google Scholar]

[b0010] 2.Van Esch A., Huyskens D.P., Behrens C.F., Samsoe E., Sjolin M., Bjelkengren U. Implementing RapidArc into clinical routine: a comprehensive program from machine QA to TPS validation and patient QA. Med Phys. 2011;38(9):5146–5166. doi: 10.1118/1.3622672. [DOI] [PubMed] [Google Scholar]

[b0015] 3.Thwaites D. Accuracy required and achievable in RT dosimetry: have modern technology and techniques changed our views? J Phys Conf Ser. 2013;444:012006. [Google Scholar]

[b0020] 4.van der Merwe D., Van Dyk J., Healy B., Zubizarreta E., Izewska J., Mijnheer B. Accuracy requirements and uncertainties in radiotherapy: a report of the International Atomic Energy Agency. Acta Oncol. 2017;56(1):1–6. doi: 10.1080/0284186X.2016.1246801. [DOI] [PubMed] [Google Scholar]

[b0025] 5.Li J.S., Lin T., Chen L., Price R.A., Jr, Ma C.M. Uncertainties in IMRT dosimetry. Med Phys. 2010;37:2491–2500. doi: 10.1118/1.3413997. [DOI] [PubMed] [Google Scholar]

[b0030] 6.McNiven A.L., Sharpe M.B., Purdie T.G. A new metric for assessing IMRT modulation complexity and plan deliverability. Med Phys. 2010;37(2):505–515. doi: 10.1118/1.3276775. [DOI] [PubMed] [Google Scholar]

[b0035] 7.Nauta M., Villarreal-Barajas J.E., Tambasco M. Fractal analysis for assessing the level of modulation of IMRT fields. Med Phys. 2011;38(10):5385–5393. doi: 10.1118/1.3633912. [DOI] [PubMed] [Google Scholar]

[b0040] 8.Younge K.C., Matuszak M.M., Moran J.M., McShan D.L., Fraass B.A., Roberts D.A. Penalization of aperture complexity in inversely planned volumetric modulated arc therapy. Med Phys. 2012;39(11):7160–7170. doi: 10.1118/1.4762566. [DOI] [PMC free article] [PubMed] [Google Scholar]

[b0045] 9.Masi L., Doro R., Favuzza V., Cipressi S., Livi L. Impact of plan parameters on the dosimetric accuracy of volumetric modulated arc therapy. Med Phys. 2013;40(7):071718. doi: 10.1118/1.4810969. [DOI] [PubMed] [Google Scholar]

[b0050] 10.Park J.M., Park S., Kim H., Kim J.H., Carlson J., Ye S.J. Modulation indices for volumetric modulated arc therapy. Phys Med Biol. 2014;59(23):7315–7340. doi: 10.1088/0031-9155/59/23/7315. [DOI] [PubMed] [Google Scholar]

[b0055] 11.Du W., Cho S.H., Zhang X., Hoffman K.E., Kudchadker R.J. Quantification of beam complexity in intensity-modulated radiation therapy treatment plans. Med Phys. 2014;41(2):21716. doi: 10.1118/1.4861821. [DOI] [PubMed] [Google Scholar]

[b0060] 12.Götstedt J., Hauer A.K., Bäck A. Development and evaluation of aperture-based complexity metrics using film and EPID measurements of static MLC openings. Med Phys. 2015;42:3911–3921. doi: 10.1118/1.4921733. [DOI] [PubMed] [Google Scholar]

[b0065] 13.Crowe S.B., Kairn T., Kenny J., Knight R.T., Hill B., Langton C.M. Treatment plan complexity metrics for predicting IMRT pre-treatment quality assurance results. Australas Phys Eng Sci Med. 2014;37(3):475–482. doi: 10.1007/s13246-014-0274-9. [DOI] [PubMed] [Google Scholar]

[b0070] 14.Kouloulias V.E., Poortmans P.M., Bernier J., Horiot J.C., Johansson K.A., Davis B. The Quality Assurance programme of the radiotherapy group or European Organisations for Research and Treatment of Cancer (EORTC): a critical appraisal of 20 years of continuous efforts. Eur J Cancer. 2003;29(4):430–437. doi: 10.1016/s0959-8049(02)00113-2. [DOI] [PubMed] [Google Scholar]

[b0075] 15.Ibbott G.S., Followill D.S., Molineu H.A., Lowenstein J.R., Alvarez P.E., Roll J.E. Challenges in Credentialing Institutions and Participants in advanced technology Multi-institutional Clinical trials. Int J Radiat Oncol Biol Phys. 2008;71:S71–S75. doi: 10.1016/j.ijrobp.2007.08.083. [DOI] [PMC free article] [PubMed] [Google Scholar]

[b0080] 16.Budiharto T., Musat E., Poortmans P., Hurkmans C., Monti A., Bar-Deroma R. Profile of European radiotherapy departments contributing to the EORTC Radiation Oncology Group (ROG) in the 21st century. Radiother Oncol. 2008;88(3):403–410. doi: 10.1016/j.radonc.2008.05.013. [DOI] [PubMed] [Google Scholar]

[b0085] 17.McGarry C.K., Agnew C.E., Hussein M., Tsang Y., McWilliam A., Hounsell A.R. The role of complexity metrics in a multi-institutional dosimetry audit of VMAT. Br J Radiol. 2016;1057(89) doi: 10.1259/bjr.20150445. [DOI] [PMC free article] [PubMed] [Google Scholar]

[b0090] 18.Ezzell G.A., Burmeister J.W., Dogan N., LoSasso T.J., Mechalakos J.G., Mihailidis D. IMRT commissioning: Multiple institution planning and dosimetry comparisons, a report from AAPM Task Group 119. Med Phys. 2009;36(11):5359–5373. doi: 10.1118/1.3238104. [DOI] [PubMed] [Google Scholar]

[b0095] 19.Svenssons E. University of Gothemburg; 2011. Evaluation of complexity and deliverability of IMRT-treatment plans. (Master thesis) [Google Scholar]

[b0100] 20.Jurado-Bruggeman D., Hernández V., Sáez J., Navarro D., Pino F., Martínez T. Multi-centre audit of VMAT planning and pre-treatment verification. Radiother Oncol. 2017;124(2):302–310. doi: 10.1016/j.radonc.2017.05.019. [DOI] [PubMed] [Google Scholar]

[b0105] 21.Hernandez V., Abella R., Calvo J.F., Jurado-Bruggemann D., Sancho I., Carrasco P. Determination of the optimal tolerance for MLC positioning in sliding window and VMAT techniques. Med Phys. 2015;4:1911–1916. doi: 10.1118/1.4915541. [DOI] [PubMed] [Google Scholar]

[b0110] 22.Storey J.D. A direct approach to false discovery rates. J R Stat Soc Ser B. 2002;64:479–498. [Google Scholar]

[b0115] 23.Storey JD, Bass AJ, Dabney A, Robinson D. qvalue: Q-value estimation for false discovery rate control. R Package 2.6.0. 2015. http://github.com/jdstorey/qvalue.

[b0120] 24.Younge K.C., Roberts D., Janes L.A., Anderson C., Moran J.M., Matuszak M.M. Predicting deliverability of volumetric-modulated arc therapy (VMAT) plans using aperture complexity analysis. J Appl Clin Med Phys. 2016;17(4):124–131. doi: 10.1120/jacmp.v17i4.6241. [DOI] [PMC free article] [PubMed] [Google Scholar]

[b0125] 25.Webb S. Use of a quantitative index of beam modulation to characterize dose conformality: illustration by a comparison of full beamlet IMRT, few-segment IMRT (fsIMRT) and conformal unmodulated radiotherapy. Phys Med Biol. 2003;48(14):2051–2062. doi: 10.1088/0031-9155/48/14/301. [DOI] [PubMed] [Google Scholar]

[b0130] 26.McGarry C.K., Chinneck C.D., O’Toole M.M., O’Sullivan J.M., Prise K.M., Hounsell A.R. Assessing software upgrades, plan properties and patient geometry using intensity modulated radiation therapy (IMRT) complexity metrics. Med Phys. 2011;38(4):2027–2034. doi: 10.1118/1.3562897. [DOI] [PubMed] [Google Scholar]

PERMALINK

Comparison of complexity metrics for multi-institutional evaluations of treatment plans in radiotherapy

Victor Hernandez

Jordi Saez

Marlies Pasler

Diego Jurado-Bruggeman

Nuria Jornet

Highlights