Correlation between gamma index passing rate and clinical dosimetric difference for pre-treatment 2D and 3D volumetric modulated arc therapy dosimetric verification

Abstract

Objective:

To investigate comparatively the percentage gamma passing rate (%GP) of two-dimensional (2D) and three-dimensional (3D) pre-treatment volumetric modulated arc therapy (VMAT) dosimetric verification and their correlation and sensitivity with percentage dosimetric errors (%DE).

Methods:

%GP of 2D and 3D pre-treatment VMAT quality assurance (QA) with different acceptance criteria was obtained by ArcCHECK® (Sun Nuclear Corporation, Melbourne, FL) for 20 patients with nasopharyngeal cancer (NPC) and 20 patients with oesophageal cancer. %DE were calculated from planned dose–volume histogram (DVH) and patients' predicted DVH calculated by 3DVH® software (Sun Nuclear Corporation). Correlation and sensitivity between %GP and %DE were investigated using Pearson's correlation coefficient (r) and receiver operating characteristics (ROCs).

Results:

Relatively higher %DE on some DVH-based metrics were observed for both patients with NPC and oesophageal cancer. Except for 2%/2 mm criterion, the average %GPs for all patients undergoing VMAT were acceptable with average rates of 97.11% ± 1.54% and 97.39% ± 1.37% for 2D and 3D 3%/3 mm criteria, respectively. The number of correlations for 3D was higher than that for 2D (21 vs 8). However, the general correlation was still poor for all the analysed metrics (9 out of 26 for 3D 3%/3 mm criterion). The average area under the curve (AUC) of ROCs was 0.66 ± 0.12 and 0.71 ± 0.21 for 2D and 3D evaluations, respectively.

Conclusions:

There is a lack of correlation between %GP and %DE for both 2D and 3D pre-treatment VMAT dosimetric evaluation. DVH-based dose metrics evaluation obtained from 3DVH will provide more useful analysis.

Advances in knowledge:

Correlation and sensitivity of %GP with %DE for VMAT QA were studied for the first time.

Volumetric modulated arc therapy (VMAT) is a novel delivery method of intensity-modulated radiotherapy (IMRT). It is capable of delivering highly conformal dose distributions through concomitant continuous gantry rotation, dynamic beam modulation and variable dose rate.^1,2 Owing to its rotational delivery features, VMAT is more complex than conventional IMRT in both planning and dosimetric evaluations.^3,4

Quality assurance (QA) for VMAT is relatively new with respect to the established dosimetric verification of fixed-beam IMRT with two-dimensional (2D) arrays. Verifying the whole plan while the gantry is rotating is rather challenging.^5,6 Numerous approaches and phantoms have been investigated for the QA of VMAT, including Monte Carlo simulation,⁷ ScandiDos Delta⁴® (ScandiDos, Uppsala, Sweden),⁸ GAFCHROMIC® EBT (International Specialty Products, Wayne, NJ) films,⁹ MatriXX™ 2D ionization chamber array with a MultiCube™ phantom (IBA Dosimetry Inc., Memphis, TN),¹⁰ 2D-ARRAT seven29 and Octavius phantom (PTW, Freiburg, Germany), electronic portal imaging device and three-dimensional (3D) diode array ArcCHECK® (Sun Nuclear Corporation, Melbourne, FL).⁶

Until now, no standardized QA procedure and acceptance criteria specific for VMAT have been established. Those performing VMAT QA are typically using QA methods and action levels taken from fixed-beam IMRT QA methods. Phantom dose verification, gamma index with 3% dose difference and 3-mm dose-to-distance criteria are most commonly used by physicists in pre-treatment IMRT and VMAT QA as reported in the AAPM Task Group 119 and some other articles.^11–13 However, recent studies demonstrated that there is no correlation between the percentage gamma passing rate (%GP) and the magnitude of dose discrepancy between the planned dose and the actual delivered dose for IMRT.^14,15 This also raises concern about whether the %GP is correlated with clinical dosimetric difference for VMAT.

The main purpose of this study is to investigate comparatively the %GP of 2D and 3D VMAT dosimetric verification with different acceptance criteria, and their correlation and sensitivity with percentage dosimetric errors (%DE) between planned dose–volume histogram (DVH) and patients' predicted DVH calculated by 3DVH® software (Sun Nuclear Corporation).

METHODS AND MATERIALS

Patients

20 patients with nasopharyngeal cancer (NPC) treated by dual-arc simultaneous integrated boost VMAT and 20 patients with oesophageal cancer treated by one-arc VMAT were enrolled in this study. VMAT plans were optimized with the SmartArc algorithm in Pinnacle treatment planning system (TPS) (Philips Healthcare, Fitchburg, WI) for a 6-MV X-ray beam on an Elekta Synergy® linac (Elekta Ltd, Crawley, UK) equipped with an 80-leaf MLCi2™. VMAT objective settings and optimization parameters for patients with NPC have been reported in our previous study.¹⁶ For one-arc VMAT plan of patients with oesophageal cancer, contours and optimization parameters have also been reported in our previous study.¹⁷ All plans were delivered through a MosaiQ® record and verify system v. 1.60Q3 (IMPAC Medical Systems, Inc., Sunnyvale, CA). The study was approved by the ethics committee of the 1st Affiliated Hospital of Wenzhou Medical University with written informed consent obtained from the patients for publication of this report.

Two-dimensional and three-dimensional dosimetric verification

Pre-treatment VMAT QA was performed using a 3D diode array ArcCHECK (Model 1220) that consists of 1386 n-type solid-state diode detectors that are curved to form a cylindrical surface inside a doughnut-shaped phantom. The phantom has an outer diameter of 26.6 cm and an inner-hole diameter of 15.1 cm, with the curved plane of diodes at a distance of 10.4 cm from the centre. The arcrylic plug, a 15-cm diameter cylinder with a hole for an ionization chamber, was inserted in the ArcCHECK phantom.¹⁸

SNC Patient v. 6.2.1 (Sun Nuclear Corporation) was used to measure the 2D %GP for dosimetric differences between the delivered dose to the QA device and the dose calculated in the TPS by comparing the unfolded 2D profiles from the cylindrical surface, as shown in Figure 1. Relative gamma analysis with acceptance criteria of 4%/4 mm, 3%/3 mm, 2%/2 mm and 10% lower dose threshold (TH) was applied.

Figure 1.

Two-dimensional gamma index analysis with SNP on unfolded profile from cylindrical surface.

3D dosimetric verification was conducted by 3DVH. An ArcCHECK movie (ACML) file generated by the SNC Patient software during the 2D phantom dosimetric verification, which contains calculated gantry angles as a function of time, together with RTPlan [digital imaging and communications in medicine (DICOM) file containing all the information about the plan's parameters] and RTDose (DICOM file containing all the information about the dose distribution) exported from TPS was exported into the 3DVH program. The delivered 3D dose distribution in the phantom was reconstructed with the planned dose perturbation (PDP) algorithm and compared with the dose distribution in Pinnacle.¹⁹ For this 3D dosimetric comparison, three different acceptance criteria, 4%/4 mm, 3%/3 mm, 2%/2 mm and 10% lower dose threshold (TH), were applied for %GP analysis.

Dose–volume histogram-based dose metrics evaluation

The 3DVH software with the PDP algorithm is able to output a 3D patient dose grid that has built into it the manifestation of any errors detected by the dosimetry array system. PDP does not introduce new sources of variation or errors that may occur with an independent 3D dose algorithm.^14,20 Through the 3D dosimetric verification with all the input of ArcCHECK measurement files and TPS files, PDP derives the per-voxel dose contribution of volumes from each beam, beams' control points and machine unit settings, and partitions each dose voxel in the 3D dose grid. Measured errors were then rendered into the patient's volume and “perturbed” using the PDP algorithm. Accumulating the total dose perturbation over all voxels and beams can then be used to modify (perturb) the input patient dose grid calculated by the TPS, which includes all necessary heterogeneity dependencies.¹⁴

%DE between the DVH values from 3DVH and TPS were calculated according to the following equation:

$$\text{DE}\%=\left|\frac{{\mathit{D}}_{3\text{DVH}}-D_{\text{TPS}}}{{\mathit{D}}_{\text{TPS}}}\right|\times 100$$

where D_3DVH is the dose value calculated from 3DVH software and D_TPS is the dose value extracted from TPS. This analysis was performed for each patient on the region of interest (ROI) and point of interest (POI). ROI includes PTV and some organs at risk (OARs). Two POIs of simulation isocentre and calculating point were also evaluated. For each planning target volume (PTV), mean dose (D_mean), maximum dose (D_max), D_2% and D_98% (dose to 2% and 98% volume) were calculated and compared. For OARs of patients with NPC, D_max and D_1% (dose to 1% volume) of the spinal cord and brainstem, and D_mean and D_50% of the parotids were evaluated. For patients with oesophageal cancer, D_max and D_1% of the spinal cord, D_mean and D_50% of the lung and heart were considered. All patient plans were calculated on a 2 × 2 × 2-mm dose grid.

Correlation and sensitivity analysis

Statistical correlation between %GP and %DE was investigated using Pearson's correlation coefficient (r) with SSPS® 17.0 (SPSS Inc., Chicago, IL). %DE were assumed to be correlated with a determined %GP when p < 0.05, which was obtained from r. In order to compare the sensitivity of 2D and 3D dosimetric evaluation, the number of “false negative” (FN) cases (cases where high QA passing rates implied large errors in ROI and POI dose metrics) and “true positive” (TP) cases (cases where low QA passing rates implied large errors in ROI and POI dose metrics) were calculated. In particular, we considered all those structures “FN” that had DVH errors >5% among those patients with %GP >95%. We considered all the cases “TP” that had DVH errors >5% and %GP <95%. From the FN and TP rates, receiver operating characteristic (ROC) curves were generated to investigate the ability of 2D and 3D methods to identify accurately the plan with dose errors >5%.

RESULTS

The percentage dose errors obtained from TPS and 3DVH are shown in Table 1. Relatively higher dose errors on the D_max of PTV and spinal cord, D_mean and D_50% of the right and left parotid are presented for patients with NPC. The errors on the D_max, D_1% of the spinal cord and D_50% of the heart for patients with oesophageal cancer were large as well. The average %GPs of 2D and 3D QA for patients with NPC and oesophageal cancer with different acceptance criteria are presented in Table 2. The passing rates of 3D for patients with oesophageal cancer with one-arc VMAT was higher than those of patients with NPC with dual-arc VMAT. The average passing rate of 2%/2 mm acceptance criteria was <95% for 2D dosimetric evaluation.

Table 1.

Percentage dose difference of dose–volume histogram-based dose metrics between treatment planning system and 3DVH® for patients with nasopharyngeal cancer (NPC) and oesophageal cancer (ESO)

Parameters	NPC	ESO
Point
Simulation isocentre	−1.26 ± 1.91	−2.07 ± 3.09
Calculating point	−2.97 ± 2.14	−3.01 ± 5.24
PTV
Dmean	−0.71 ± 0.90	−1.04 ± 0.79
Dmax	6.91 ± 7.49	1.57 ± 1.79
D2%	0.66 ± 1.22	0.20 ± 0.81
D98%	−0.67 ± 1.42	−1.44 ± 1.64
Brainstem
Dmax	0.38 ± 2.60
D1%	−1.74 ± 3.01
Spinal cord
Dmax	−3.07 ± 4.80	−2.93 ± 4.65
D1%	−3.27 ± 1.82	−2.86 ± 4.10
Right parotid
Dmean	−5.00 ± 1.97
D50%	−12.74 ± 5.00
Left parotid
Dmean	−2.96 ± 2.38
D50%	−11.47 ± 5.84
Lung
Dmean		−1.85 ± 0.92
D50%		−2.34 ± 1.63
Heart
Dmean		−1.62 ± 1.19
D50%		−4.04 ± 4.22

D_x%, dose to percentage volume; D_max, maximum dose; D_mean, mean dose; PTV, planning target volume.

3DVH software was obtained from Sun Nuclear Corporation, Melbourne, FL.

Table 2.

Percentage gamma index passing rate between two-dimensional (2D) and three-dimensional (3D) volumetric modulated arc therapy quality assurance for patients with nasopharyngeal cancer (NPC) and oesophageal cancer (ESO)

Acceptance criteria (%/mm)	NPC	ESO	p-value
4%/4 mm
2D	98.57 ± 2.83	99.58 ± 0.49	0.13
3D	98.84 ± 0.57	99.21 ± 0.45	0.03
3%/3 mm
2D	96.67 ± 1.36	97.57 ± 1.63	0.07
3D	96.86 ± 1.60	97.95 ± 0.78	0.01
2%/2 mm
2D	85.81 ± 3.80	87.65 ± 4.43	0.17
3D	92.74 ± 4.10	95.38 ± 1.82	0.01

Tables 3 and 4 show the results of statistical correlation with the respective p-values between %DE and %GP of 2D and 3D QA for patients with NPC and oesophageal cancer, respectively. For 2D 4%/4 mm criterion, no correlation between %GP and %DE was observed for NPC, while the %GP of 3%/3 mm and 2%/2 mm acceptance criteria were correlated with the %DE of the D_98% of PTV (r = 0.49, p = 0.03; r = 0.57, p = 0.009), D_max (r = 0.45, p = 0.049; r = 0.61, p = 0.004) and D_1% (r = 0.56, p = 0.01; r = 0.52, p = 0.02) of the brainstem. For patients with oesophageal cancer, only the %GP of 4%/4 mm and 3%/3 mm showed a correlation with the %DE of the calculating point with r = 0.6, p = 0.007 and r = 0.49, p = 0.04, respectively.

Table 3.

Correlation between two-dimensional gamma passing rate and dose difference for patients with nasopharyngeal cancer (NPC) and oesophageal cancer

Acceptance criteria	4%/4 mm		3%/3 mm		2%/2 mm
	r	p	r	p	r	p
NPC
Point
Simulation isocentre	0.05	0.83	0.11	0.64	0.17	0.84
Calculating point	0.21	0.38	−0.10	0.69	−0.12	0.61
PTV
Dmax	−0.25	0.29	0.35	0.13	0.42	0.07
Dmean	−0.15	0.53	−0.21	0.37	−0.11	0.65
D2%	−0.21	0.38	0.24	0.32	0.33	0.15
D98%	−0.14	0.55	0.49*	0.03	0.57*	0.009
Brainstem
Dmax	0.19	0.41	0.45*	0.049	0.61*	0.004
D1%	−0.23	0.34	0.56*	0.01	0.52*	0.02
Spinal cord
Dmax	0.03	0.9	0.02	0.93	0.01	0.96
D1%	0.15	0.54	−0.38	0.10	−0.21	0.37
Right parotid
Dmean	−0.29	0.22	−0.08	0.74	−0.13	0.58
D50%	0.19	0.31	−0.24	0.30	−0.16	0.50
Left parotid
Dmean	−0.10	0.67	0.09	0.71	0.30	0.20
D50%	0.12	0.61	0.01	0.96	0.12	0.62
Oesophageal cancer
Point
Simulation isocentre	0.37	0.12	0.35	0.14	0.46	0.05
Calculating point	0.60*	0.007	0.49*	0.04	0.40	0.09
PTV
Dmax	0.01	0.97	−0.15	0.54	−0.06	0.81
Dmean	0.19	0.43	0.16	0.51	0.17	0.49
D2%	0.05	0.85	−0.11	0.65	−0.04	0.86
D98%	0.40	0.09	0.26	0.29	0.32	0.19
Spinal cord
Dmax	−0.26	0.29	−0.36	0.13	−0.14	0.57
D1%	−0.16	0.51	−0.34	0.16	−0.18	0.47
Lung
Dmean	−0.29	0.25	−0.46	0.06	−0.36	0.14
D50%	−0.31	0.21	−0.41	0.09	−0.36	0.15
Heart
Dmean	−0.17	0.52	−0.28	0.27	−0.2	0.44
D50%	−0.27	0.29	−0.24	0.35	−0.42	0.10

D_x%, dose to percentage volume; D_max, maximum dose; D_mean, mean dose; PTV, planning target volume.

Asterisks indicate statistically significant values.

Table 4.

Correlation between three-dimensional gamma index passing rate and dose difference for nasopharyngeal cancer (NPC) and oesophageal cancer patients

3DVH®	4%/4 mm		3%/3 mm		2%/2 mm
	r	p	r	p	r	p
NPC
Point
Simulation isocentre	0.22	0.36	0.42	0.065	0.51	0.022
Calculating point	0.15	0.54	0.17	0.47	0.17	0.47
PTV
Dmax	0.66*	0.001	0.77*	<0.001	0.71*	<0.001
Dmean	0.20	0.40	0.17	0.49	0.12	0.61
D2%	0.54*	0.01	0.56*	0.01	0.47*	0.04
D98%	0.51*	0.02	0.63*	0.003	0.59*	0.006
Brainstem
Dmax	0.37	0.11	0.44	0.05	0.41	0.07
D1%	0.13	0.59	0.10	0.68	0.03	0.91
Spinal cord
Dmax	−0.13	0.58	0.08	0.74	0.23	0.33
D1%	−0.81	0.73	−0.10	0.67	−0.08	0.74
Right parotid
Dmean	0.38	0.10	0.59*	0.006	0.62*	0.004
D50%	0.17	0.47	0.30	0.21	0.36	0.12
Left parotid
Dmean	0.30	0.20	0.48*	0.03	0.51*	0.02
D50%	0.10	0.67	0.08	0.74	0.06	0.80
Oesophageal cancer
Point
Simulation isocentre	0.02	0.94	0.11	0.66	−0.71	0.77
Calculating point	0.29	0.22	0.15	0.54	−0.002	0.99
PTV
Dmax	0.37	0.12	0.68*	0.001	0.51*	0.03
Dmean	0.02	0.95	0.25	0.30	0.30	0.21
D2%	0.23	0.34	0.58*	0.01	0.55*	0.02
D98%	0.19	0.45	0.33	0.16	0.20	0.42
Spinal cord
Dmax	−0.21	0.39	0.08	0.76	0.02	0.93
D1%	−0.05	0.83	0.11	0.66	−0.01	0.96
Lung
Dmean	0.45	0.06	0.82*	<0.001	0.80*	<0.001
D50%	0.15	0.56	0.59*	0.01	0.72*	0.001
Heart
Dmean	−0.13	0.62	0.14	0.60	0.23	0.37
D50%	−0.33	0.20	−0.22	0.40	0.03	0.92

D_x%, dose to percentage volume; D_max, maximum dose; D_mean, mean dose; PTV, planning target volume.

3DVH software was obtained from Sun Nuclear Corporation, Melbourne, FL.

Asterisks indicate statistically significant values.

The number of correlations between 3D %GP and %DE in Table 4 was higher than that between 2D %GP and %DE in Table 3 (21 vs 8). The number of correlations with the criteria of 3%/3 mm and 2%/2 mm was higher than that of 4%/4 mm in 3D QA (9 vs 3). The %DE of the D_max, D_2% and D_98% of PTV for all three acceptance criteria were correlated with %GP in patients with NPC, while the %DE of the D_mean of the left and right parotid were correlated with %GP for only the acceptance criteria of 3%/3 mm and 2%/2 mm. There was no correlation between %GP and %DE for 4%/4 mm acceptance criteria in patients with oesophageal cancer, while the %DE of the D_max and D_2% of PTV, D_mean and D_50% of the lung were correlated with %GP for 3%/3 mm and 2%/2 mm criteria.

Further analysis on sensitivity was performed using the 3%/3 mm acceptance criterion, since 4%/4 mm showed less correlation and 2%/2 mm showed a lower passing rate. The average area under curve (AUC) values of ROCs for DVH metrics were 0.66 ± 0.12 and 0.71 ± 0.21 for 2D and 3D, respectively. Figure 2 presents comparative ROC curves for PTV D_max and isocentre point dose together with their AUC values.

Figure 2.

Comparative two-dimensional and three-dimensional receiver operating characteristic (ROC) curves with area under the curve (AUC) values for (a) planning target volume (PTV) maximum dose (D_max); (b) point dose. The 3DVH® software was obtained from Sun Nuclear Corporation, Melbourne, FL.

DISCUSSION

The correlation between %GP of 2D and 3D pre-treatment VMAT dosimetric evaluation, and DVH-based dose metric %DE were evaluated and compared for patients with NPC and oesophageal cancer. Relatively weak correlation and sensitivity between %GP and %DE were observed for both 2D and 3D pre-treatment VMAT dosimetric evaluation.

In agreement with the literature, the %GP of patients with NPC in this study was lower than those with oesophageal cancer.²¹ Except for 2%/2 mm acceptance criterion of 2D gamma index, the average %GP of 2D and 3D were all acceptable for clinical QA practice, as presented in Table 2. The average %GP with the commonly used 3%/3 mm criterion for 2D and 3D dosimetric valuation was 97.11% ± 1.54% and 97.39% ± 1.37%, respectively. These were higher than the generally accepted 95% passing rate and similar to the previous reported %GP.^3,19 However, the relatively higher dose errors of some dose metrics presented in Table 1 were not implied by the %GP of both 2D and 3D methods.

Similar to the reported weak correlation between gamma index passing rate and clinical dosimetric differences in IMRT QA,^14,15 our study also demonstrated a weak correlation between %GP and the DVH-based dose metrics %DE for both 2D and 3D VMAT QA. 3D dosimetric verification with 3DVH improved the correlation and sensitivity with respect to 2D dose verification on some dose metrics. However, the general rate of correlation was still poor for all the analysed metrics (9 out of 26 for 3%/3 mm criterion). 3DVH had a better prediction on the %DE than did 2D as shown by the ROC analysis, but the accuracy of both 3DVH and gamma index were not good enough for clinical acceptable with AUC values <0.8. This lack of correlation and sensitivity in patient-specific VMAT QA could be understood as the general gamma passing rate is a non-patient-specific metric. The dose differences of individual voxels in different ROIs have different clinical impacts. The %GP tells only how many voxels fail to pass the criteria and provides no information on the anatomic location of the failure or at which dose level it failed.

As it has been suggested to use the predicted patient DVH instead of gamma passing rate analysis for patient-specific IMRT QA,³ this study also implies that for VMAT pre-treatment QA, we should take into account the clinical tolerance of PTV and OARs and not rely on gamma passing rate only. As it has been mentioned that the average passing rate of gamma index that would be clinically acceptable for one patient could be unacceptable for another.²² A software such as 3DVH that presents both the 3D %GP and the impact of measured dose difference on DVH-based dose metrics will provide helpful solution for VMAT QA.

The accuracy and reliability of 3DVH employing a PDP algorithm to estimate the dose delivered to patients from the phantom measurement have been demonstrated both with homogeneous and heterogeneous phantoms for VMAT.^3,23 The large dose differences observed in some DVH metrics in this study could have resulted from the insufficient spatial resolution of the detector. The insufficient spatial resolution could introduce lower dose points around a diode in areas with a high-dose gradient.²⁴ VMAT QA using DVH-based dose metrics allows per-patient dose QA to be based on metrics that are both sensitive and specific. On the other hand, these DVH metrics could grow to be numerous and complex depending on the number of ROIs and POIs investigated, and introduce inefficiencies in a busy clinic. For the sake of analysis efficiency, only some points in the DVH curves, such as the D_50% and D_max, were considered for dose metric analysis in this study instead of comparing all the information available in the DVH curves.

CONCLUSION

There is a lack of correlation and sensitivity between gamma index passing rate and dose error of DVH dose metrics for both 2D and 3D pre-treatment VMAT dosimetric evaluation. DVH-based dose metrics evaluation obtained from 3DVH will provide more helpful analysis. However, new studies on a new process for DVH-based dose metrics for pre-treatment VMAT QA are urgently needed to fit it into practical constraints of time and resources.

FUNDING

This study was supported by a grant from the Wenzhou Science and Technology Bureau (H20100068).