How to compare treatment plans? Personalized perspective

Recently Kumar et al.¹ presented a comparative study of two different volumetric modulated arc therapy delivery techniques. The growing numbers of new planning algorithms, delivery methods, and technologies in the recent years has caused the number of published planning comparisons to increase almost exponentially. Therefore, discussion on how to find the best way of comparing treatment plans is quite urgent.

We fully agree with the authors of the commented study that simple planning comparisons are susceptible to bias. However, even a simple comparison should be made according to clearly defined criteria, which allows to compare treatment plans prepared for two equivalent techniques implemented on two different systems (e.g. VMAT on Varian and Elekta). Clearly defined criteria enable standardization of the obtained results. Unfortunately, the authors of the commented study do not provide elementary information about normalization of doses for the two equivalent VMAT plans created on the Eclipse and Monaco treatment planning systems nor any information about the constraints of the doses to organs at risk and to the tumor that were used during the optimization process. We agree with the authors that technological differences between two different solutions, such as Varian and Elekta, could pose a problem in data interpretation, but these differences can also be minimized. For example, according to the paper published by Knoos et al.,² the algorithm-specific differences of dose distribution could be ignored if the compared algorithms include similar methods of accounting for changes in electron transport. In the commented paper, algorithms are equivalent and, therefore, do not bias the comparisons. Unfortunately, another important aspect concentrated on the arcs used for plan preparation was not deeply discussed in the commented study. Generally, if the comparisons are performed, operator/planner dependant degrees of freedom should be minimized. The comparison between two-arc Varian VMAT and one-arc Elekta VMAT is controversial. Even if the two-arc technique (Varian VMAT) is an accepted standard of care, the results of one-arc VMAT Varian technique should be added. The reason is that the two-arc VMAT technique doubles the number of potential beamlets used during the optimization process as compared to the one-arc VMAT technique. Finally, the comparisons of parameters that describe dose distribution and parameters that describe dose delivery (e.g. estimated times or monitor units) should be carefully checked by appropriate statistical tests, which should be clearly reported in the study. Referring to the commented study, we would like to note that selecting the t-Student test is not always a good idea. Sometimes, a wrong selection of a statistical test leads to misinterpretation of results. An appropriate selection of statistical tests depends on many independent factors, among which the most important are: (i) dependence or independence of the compared groups; (ii) number of compared groups and observations made for each group; (iii) normality or non-normality of the distribution and (iv) information of the statistical dispersion of the observations made for the compared groups.³

It should be noted, that comparisons including only averaged parameters relating to the compared treatment plans rather than those related to the specific requirements of individual patients can lead to misinterpretation. To resolve this problem, the proposal by Phillips and Holdsworth⁴ should be considered. Phillips and Holdsworth proposed to introduce two types of objectives in planning comparisons: inverse planning objective and decision objective to deal with the multicriteria nature of radiotherapy planning. The idea initially developed as a tool for comparisons of different treatment modalities was then adopted to resolve the selection problems of the best treatment dependent on patient-specific condition. In other words, they developed a multiobjective optimization algorithm that provides the decision maker with plans that span the range of a complete tumor coverage with high normal tissue doses to the opposite of good sparing of normal tissues but poor tumor coverage. This offers the physician a greater latitude in choosing a plan that meets the patient's condition and wishes.^5,6 It was the first step from population-based comparisons based on averaged and non-connected parameters (e.g. dose to the tumor and integral dose at estimated treatment time), to the personalized comparisons that take into account patient-specific requirements which determine the final selection of the most adequate solution. Unfortunately, decision objective component of this method is based only on the temporal decision of the physician that is not supported with potential outcomes of the treatment related to the specific patient. Yartsev and Mackie⁷ proposed the method of machine learning from databases that include all needed data (e.g. patient-specific information received before, during and after treatment, and technology-specific information) to estimate potential outcomes of treatment. An interesting idea as it is, the statistical management of data and the adequate capacity and performance of these databases seem questionable. Therefore, an interesting alternative is to develop decision models of the entire radiation therapy process using Bayesian networks.^8–10 These mechanistic models of cancer and radiation response can incorporate all the tradeoffs inherent in the case. They also provide a very flexible method of introducing new data on outcomes.

In conclusion, we appreciate the work of Kumar et al.¹ as a classical solution of the comparison problem. On the basis of their work, we tried to highlight that even a simple evaluation must have well-defined criteria and should be supported by advanced statistical tests. Additionally, we tried to present the global tendency of switching from the population-based comparisons to the personalized solutions that has strongly developed recently.

Conflict of interest

None declared.

Financial disclosure

None declared.