The use of log file analysis within VMAT audits

Abstract

Objective:

This work investigated the delivery accuracy of different Varian linear accelerator models using log file-derived multileaf collimator (MLC) root mean square (RMS) values.

Methods:

Seven centres independently created a plan on the same virtual phantom using their own planning system, and the log files were analyzed following delivery of the plan in each centre to assess MLC positioning accuracy. A single standard plan was also delivered by the seven centres to remove variations in complexity, and the log files were analyzed for Varian TrueBeams and Clinacs (2300IX or 2100CD models).

Results:

Varian TrueBeam accelerators had better MLC positioning accuracy (<1.0 mm) than the 2300IX (<2.5 mm) following delivery of the plans created by each centre and also the standard plan. In one case, log files provided evidence that reduced delivery accuracy was not associated with the linear accelerator model but was due to planning issues.

Conclusion:

Log files are useful in identifying differences between linear accelerator models and isolate errors during end-to-end testing in volumetric-modulated arc therapy (VMAT) audits. Log file analysis can rapidly eliminate the machine delivery from the process and divert attention with confidence to other aspects.

Advances in knowledge:

Log file evaluation was shown to be an effective method to rapidly verify satisfactory treatment delivery when a dosimetric evaluation fails during end-to-end dosimetry audits. MLC RMS values for Varian TrueBeams were shown to be much smaller than those for Varian Clinacs for VMAT deliveries.

INTRODUCTION

Intensity-modulated radiotherapy (IMRT), volumetric-modulated arc therapy (VMAT) and tomotherapy are implemented in many radiotherapy clinics within the UK.¹ An audit to verify treatment planning system (TPS) modelling and treatment delivery in rotational IMRT (VMAT and tomotherapy) has been undertaken and has demonstrated accurate clinical implementation.² Within this audit, deliverability was defined using a two-dimensional array within a phantom. This type of end-to-end testing in audits has the advantage that centres can benchmark their planning and delivery systems against others. However, if there are suboptimal results, it can sometimes be difficult to isolate which part of the system is the cause. Therefore, further information on each stage of the process including planning parameters, planning technique/class solutions, planning optimization and delivery method is important.

In the Varian Clinac series linear accelerators (Varian Medical Systems, Palo Alto, CA), communication between the multileaf collimator (MLC) and linear accelerator control systems (two control system) takes approximately 50–80 ms. However, the new generation Varian TrueBeam linear accelerators (Varian Medical Systems) have an integrated MLC and linear accelerator control system which communicates with each of the components retrospectively every 10 ms and uses this information to prospectively instruct each component for the subsequent 10 and 20 ms, in order to synchronize the planned and actual treatment delivery. The integrated control system of the TrueBeam has been found to improve the dose delivery accuracy of segmented IMRT fields.³ VMAT has been shown to have higher MLC errors than IMRT for Clinac series linear accelerators.⁴ No reports have so far compared the differences in MLC positional errors between the integrated systems and the previous control system for VMAT delivery.

Log file analysis has also been proposed to improve the efficiency of patient-specific quality assurance^5–8 and provide insight into machine parameters not possible with phantom-based measurements.^9–11 In addition, log file analysis can be used to assess the actual/relative delivered dose reconstructed on the patient anatomy using the patient's original CT image set^10,12 or the on-treatment cone-beam CT image set.¹³ The use of log files within a radiotherapy VMAT dosimetry audit has been reported for Elekta linear accelerators¹⁴ (Elekta AB, Stockholm, Sweden) but not for Varian linear accelerators. Pasler et al¹⁴ investigated the delivery accuracy of a single challenging plan sent to each centre using log files. Delivery inaccuracies were independent of TPS and centre technique, and the mean leaf error (±1 standard deviation) was reported as 0.3 ± 0.2 mm for all linear accelerators.

Kerns et al⁴ provided a framework where 85,000 log files were used to compare machine delivery errors across different institutions. Differences in root mean square (RMS) leaf errors were reported between delivery modalities (step-and-shoot IMRT—0.008 mm, dynamic IMRT—0.32 mm and VMAT—0.46 mm) and between institutions. Important conclusions such as optimization of MLC performance and specific tolerances were suggested based on analysis solely using the log files. These results did not differentiate between TPS types and versions, linear accelerator type and treatment site/techniques. Hernandez et al¹⁵ investigated Varian Trilogy and Clinac log files with plans delivered using a single TPS to determine optimal MLC tolerances for IMRT and VMAT.

The aim of this work was to assess the benefit of adding log file information to end-to-end VMAT dosimetry audits. This was performed using both: (a) a subset of seven plans created by each centre and (b) by investigating a single plan delivered by a subset of centres participating in a national rotational audit. Further to this, analysis of the differences in MLC RMS values between Clinac and TrueBeam control systems was undertaken.

METHODS AND MATERIALS

Centre-specific “three-dimensional treatment planning system” plan analysis

As part of the UK National VMAT audit, plans were created by each centre on the virtual phantom shown in Figure 1 with pre-delineated volumes, called the three-dimensional treatment planning system (3DTPS) test.¹⁶ The test included five planning target volumes (PTVs) and one organ at risk, each of which had different specified dose constraints per fraction (2.5 Gy: primary PTV2, 2.0 Gy: PTV3 and PTV5 and 1.5 Gy: PTV1 and PTV4, with maximum dose to the organ at risk <1.0 Gy). The plan was validated on each of the planning systems before the audit began and was designed to be challenging. Centres were asked to use their typical VMAT planning technique and TPS settings. There were no requirements on the number of arcs to be utilized for planning. Each centre was provided with a standard set of CT scans of the Octavius^® II phantom and was instructed to import this scan into their TPS. Each centre was instructed to apply their normal procedure for setting the size of dose calculation grid and couch correction, e.g. inserting or ignoring a couch structure in the planning system.

Figure 1.

The dimensions and positions of the three-dimensional (3D) treatment planning system volumes in a virtual cylindrical phantom in (a) the transverse plane and (b) 3D visualization. OAR, organ at risk; PTV, planning target volume.

All centres were given detailed instructions to ensure that the position of the dose distribution relative to the phantom was consistently reproduced, and thus the dose planes were measured by the audit team using the same Octavius^® II phantom (PTW, Freiburg, Germany) in the same part of the plan for each centre. These consisted of two coronal (horizontal) planes and a sagittal (vertical) plane, each of which intersected a different selection of targets and critical structures.¹⁶ Analysis of each measured dose plane was made using the PTW VeriSoft^® software (PTW). Global gamma analysis was performed by the co-ordinating centre, using 2%/2 mm and 3%/3 mm (dose difference and distance to agreement) with a minimum dose threshold of 20%. The normalization dose for the gamma analysis was inside the high-dose PTV in a low-dose gradient region.

Seven centres (centres 1–7) provided log files following delivery of their centre-specific 3DTPS plan during the UK National VMAT audit. TPS used were Varian Eclipse™ (Varian Oncology systems, Palo Alto, CA), Nucletron Oncentra^® (Elekta Ltd, Crawley, UK) and Phillips Pinnacle (Philips, Reigate, UK). The complexity of each plan was recorded using the modulation complexity score (MCS), as previously defined by McNiven et al.¹⁷ The MCS assesses plan complexity based on the variability of leaf positions, aperture area between segments and segment weight, as defined by McNiven et al,¹⁷ and can be determined with each control point considered as a beam segment for VMAT, as described previously.¹⁸ Plans were delivered using Varian TrueBeams and Varian Clinac 2300IXs. Combinations of setups are shown in Table 1. These log files were analyzed using the in-house code.¹⁹ This software has the ability to analyze the MLC leaf pairs in DynaLog files created using Varian Clinacs (.DLG files) and trajectory log files created by Varian TrueBeams (.INI files) compared with the planned positions defined in the plan files (digital imaging and communications in medicine). This software creates a report of MLC error histograms and the average RMS of MLC positioning errors. It has been previously validated for step-and-shoot IMRT,^3,8 VMAT in TrueBeams¹⁹ and also for VMAT using Clinacs with this data set. All data were presented as median values with associated ranges (minimum and maximum).

Table 1.

Treatment planning system (Varian, Phillips and Elekta) and delivery system (all Varian) combinations for all centres investigated. Also shown are the modulation complexity score (MCS). The standard plan was created using Varian Eclipse v. 8

Centre-specific “3DTPS” plan				Standard “3DTPS” plan analysis
Centre	Treatment planning system	Delivery system	Average MCS	Centre	Delivery system
1	Eclipse v. 10	TrueBeam	0.268	A	TrueBeam
2	Pinnacle v. 9.2	TrueBeam	0.169	B	TrueBeam
3	Oncentra v. 4.1	2300IX	0.269	C	TrueBeam
4	Eclipse v. 8	2300IX	0.252	D	2300IX
5	Eclipse v. 8	2300IX	0.219	E	2300IX
6	Eclipse v. 8	2300IX	0.244	F	2300IX
7	Eclipse v. 10	2300IX	0.333	G	2100CD

3DTPS, three-dimensional treatment planning system.

Standard “three-dimensional treatment planning system” plan analysis

Seven centres (centres A–G) provided log files following delivery using different linear accelerator models (Table 1) of the same “standard” 3DTPS plan which had an MCS = 0.219. This meant that all the centres used the same control points, MLC positions, gantry speeds and monitor units during the delivery. This plan was provided by the co-ordinating centre as a single digital imaging and communications in medicine plan, which removed the variability in planning between different centres. These log files were also analyzed using the in-house code¹⁹ following delivery. Histograms of MLC positions were plotted and the average RMS of MLC positioning errors were calculated. All data were presented as median values with associated ranges (minimum and maximum).

RESULTS

Table 1 shows that there was a variation in the complexity of plans created by the seven centres. This demonstrates the range of planning and delivery approaches, for all centres attempting to achieve the same planning objective.

Figure 2 shows that all positional errors were <1mm for TrueBeams and <2 mm for 2300IXs. Over 99.99% of MLC positional errors were <2 mm for the 2100CD linear accelerator. Table 2 confirms the reduced MLC positional error for TrueBeam linear accelerators. The median “average RMS values” were 0.13 mm (0.11–0.14 mm) and 0.41 mm (0.24–0.47 mm) for TrueBeams and Clinacs, respectively. The data presented in Table 2 show poor gamma pass rates for one of the TrueBeams. The fact that the log files clearly show excellent delivery accuracy demonstrates that the differences in gamma results are more likely to be due to how the plans are modelled and unlikely to be due to significant MLC positional errors. Two of the measurement planes had >95% of pixels passing the 3% per 3-mm criteria for this TrueBeam, and the third plane was found to be below this value.

Figure 2.

Histogram of multileaf collimator (MLC) positional errors following delivery of centre-specific plans for TrueBeams (TB) (blue) and Clinac 2300s (orange). For colour image see online.

Table 2.

Multileaf collimator errors and gamma pass rates following delivery of centre specific plans. Median values are also shown (minimum and maximum)

Parameter	TrueBeam (N = 2) (n = 4)	2300IX (N = 5) (n = 10)
RMS values for leaves in Bank A (mm)	0.13 (0.11–0.14)	0.41 (0.24–0.46)
RMS values for leaves in Bank B (mm)	0.13 (0.12–0.14)	0.42 (0.25–0.47)
Gamma pass rate (3%3mm)	97.4 (84.0–100.0)	100.0 (98.9–100.0)
Gamma pass rate (2%2mm)	78.3 (50.5–97.9)	98.2 (85.4–99.7)

n, number of arcs analyzed; N, number of centres; RMS, root mean square.

Leaf tolerances were set to 5 mm for all Clinac deliveries. If an error goes beyond this value in VMAT delivery, an interlock is triggered by the MLC in order to interrupt the gantry rotation.¹⁵ There are no tolerances for TrueBeams as they prioritize the MLC and adjust other beam delivery parameters to allow MLCs to reach planned positions, negating the requirement for a specific tolerance value.

Figure 3 shows MLC delivery accuracy following delivery of the same standard plan (MCS = 0.219) by different centres. Again, the TrueBeam linear accelerators had >99.95% of MLC errors <1 mm and the Clinacs had no errors >2 mm. The median “average RMS values” were 0.16 mm (0.16–0.19 mm) and 0.42 mm (0.40–0.46 mm) for TrueBeams and Clinacs, respectively, which were of similar magnitude to centre-specific plans (Table 3).

Figure 3.

Histogram of multileaf collimator (MLC) positional errors following delivery of standard plans for TrueBeams (TB) (blue), Clinac 2300s (orange) and Clinac 2100s (green). For colour image see online.

Table 3.

Multileaf collimator errors for standard plan delivery. Median values are also shown (minimum and maximum)

Parameter	TrueBeam (N = 3) (n = 6)	2300IX (N = 2) (n = 4)	2100CD (N = 2) (n = 4)
RMS values for leaves in Bank A (mm)	0.16 (0.16–0.18)	0.43 (0.41–0.45)	0.42 (0.40–0.44)
RMS values for leaves in Bank B (mm)	0.16 (0.16–0.19)	0.44 (0.41–0.46)	0.42 (0.40–0.45)

n, number of arcs analyzed: N, number of centers: RMS, root mean square.

DISCUSSION

Log file analysis has been shown to be an excellent tool, to determine systematic differences between linear accelerator types and help source possible errors during end-to-end testing in radiotherapy VMAT audits. It is clear that the TPS modelling needed to be checked rather than the delivery for one of the centres studied.

The magnitude of the MLC errors for the Clinacs was similar to that reported for VMAT deliveries by Kerns et al⁴ and also for Elekta linear accelerators reported by Pasler et al.¹⁴ MLC errors for TrueBeams were similar to those reported by Agnew et al.¹⁹ The improved accuracy is owing to the integrated controller design of TrueBeam linear accelerators, which record logs every 20 ms and utilize predictive systems. Log file errors appeared to be consistent across different centres for each linear accelerator type. The results presented by Kerns et al⁴ were not differentiated between TPS types and versions, linear accelerator type and treatment site/techniques. This work showed that MLC errors for VMAT are linear accelerator dependent. Further work is required, with a larger data set, to set tolerances for TrueBeam linear accelerators and to check the dependence on TPS.

One of the TrueBeam linear accelerators had significantly fewer pixels passing the gamma criteria following delivery to the Octavius seven29 array (PTW). This did not correlate with the RMS values for the centre-specific or standard plan, which were much less than those observed for the Clinacs. For this centre, the planning system used for VMAT modelling was independent of the vendor providing VMAT delivery. This was clearly shown to affect delivery accuracy, when all centres in the audit were analyzed.² Other potential sources of error could have been the complexity which was much higher for this plan. It can be difficult to decipher whether a poor result is owing to complexity or modelling of delivery parameters by the TPS. Previous work¹⁸ shows that the delivery accuracy was significantly better when VMAT modelling was specifically designed for the linear accelerator manufacturer's own treatment delivery system (Type 1) than when modelling was designed by an independent vendor for VMAT delivery (Type 2), and that Type 2 algorithms produce more complex plans. The work presented in this short communication shows that although the complexity was higher, the delivery accuracy determined by the log file, which is independent of the TPS modelling, was similar to other less complex plans. Hence, this would suggest that this TPS may not have modelled the delivery in this case as accurately as in other systems.

The delivery of a standard plan with the same complexity at different centres (i.e. a single MCS) had the advantage that there were no differences in plan complexities between centres, although there were still some variations in MLC positional errors between linear accelerators. This could be due to variations in the age of the linear accelerators, total beam-on time or MLC usage during the lifetime of the linear accelerator. TrueBeams were found to have much lower MLC positional errors than Clinacs for the VMAT deliveries analyzed. This improvement has previously been observed for step-and-shoot IMRT plans⁴ and can be attributed to the integrated communication system. It is most likely that the lower errors found for VMAT observed in this article were also due to the integrated communication system. Improvements were observed for both centre-specific and standard plans.

CONCLUSION

Log files are a valuable tool to gain information during audits, which may help explain sources of error detected in end-to-end testing. MLC RMS values for TrueBeams were much smaller than those for Clinacs for VMAT deliveries. Delivering the same plan independently created by each centre and delivering a single pre-prepared standard plan can help differentiate between TPS errors and errors based on delivery.