Statistical process control analysis for patient-specific IMRT and VMAT QA

Abstract

This work applied statistical process control to establish the control limits of the % gamma pass of patient-specific intensity modulated radiotherapy (IMRT) and volumetric modulated arc therapy (VMAT) quality assurance (QA), and to evaluate the efficiency of the QA process by using the process capability index (C_pml). A total of 278 IMRT QA plans in nasopharyngeal carcinoma were measured with MapCHECK, while 159 VMAT QA plans were undertaken with ArcCHECK. Six megavolts with nine fields were used for the IMRT plan and 2.5 arcs were used to generate the VMAT plans. The gamma (3%/3 mm) criteria were used to evaluate the QA plans. The % gamma passes were plotted on a control chart. The first 50 data points were employed to calculate the control limits. The C_pml was calculated to evaluate the capability of the IMRT/VMAT QA process. The results showed higher systematic errors in IMRT QA than VMAT QA due to the more complicated setup used in IMRT QA. The variation of random errors was also larger in IMRT QA than VMAT QA because the VMAT plan has more continuity of dose distribution. The average % gamma pass was 93.7% ± 3.7% for IMRT and 96.7% ± 2.2% for VMAT. The C_pml value of IMRT QA was 1.60 and VMAT QA was 1.99, which implied that the VMAT QA process was more accurate than the IMRT QA process. Our lower control limit for % gamma pass of IMRT is 85.0%, while the limit for VMAT is 90%. Both the IMRT and VMAT QA processes are good quality because C_pml values are higher than 1.0.

INTRODUCTION

Radiotherapy is one of the principal treatment modalities in the management of cancer. The radiation treatment technique has been developed from two-dimensional radiotherapy (2D), three-dimensional conformal radiotherapy (3D-CRT) to intensity modulated radiotherapy (IMRT) and volumetric modulated arc therapy (VMAT) [1]. IMRT is an advanced treatment technique whereby radiation beams can be modulated by a moving multileaf collimator (MLC) or by using a compensator. MLC is commonly used to modulate the beam. VMAT is a more advanced technique; the beam is delivered by two-ways of MLC movement including the dose rate and gantry speed variation for beam modulation as well. IMRT and VMAT have become standard treatment techniques instead of conventional radiotherapy technique for head and neck cancer in many cancer centers [2] because of the clinical benefits of reduced radiation toxicity to surrounding normal tissues, especially the salivary glands, and benefits of dose conformity to target volume [3, 4]. Because the radiation dose in both the IMRT and VMAT techniques is modulated by MLC movement, a robust patient-specific quality assurance (QA) program of dose distribution is recommended to verify the correct treatment delivery before clinical treatment [5]. Traditional ways of evaluating isodose distribution have been based on prior experience and training. New methods of the distance-to-agreement (DTA) and gamma index are commonly used as the quantitative evaluation [6].

The control chart is a new tool that it is possible to apply for QA in radiotherapy. It is one of the seven quality control tools for statistical process control used to monitor variation of the process, and allows the separation of systematic error from random error. Control charts were originally used in industrial manufacturing and have been previously applied to radiotherapy QA. Pawlicki et al. [7] investigated control charts for radiotherapy QA. In their work, control charts were applied to daily QA for output and flatness/symmetry as a pilot project. The results showed that the control charts were able to identify systematic change while standard deviation methods cannot detect the error. Breen et al. [8] used control charts to evaluate the percentage of dose differences between measurement and calculation using PINNACLE treatment planning (versions 6.2 and 7.6; Philips Radiation Oncology Systems, Madison, WI, USA). They found that the upper and lower limits of version 6.2 were larger than version 7.6 because of the limitations of the MLC in the older version. Pawlicki et al. [9] applied the control chart and process ability to IMRT QA. Better process ability was observed in prostate IMRT QA compared with head and neck IMRT QA. They also showed only 11 of the 24 processes were within control limits. Gerard et al. [10] used control charts to distinguish significant change from random variation and employed performance indices to quantify the performance of patient-specific IMRT QA for head-and-neck and prostate plans. The result showed efficient point drift detection from the control chart before observing from clinical specification. The performance indices demonstrated that the QA process was within the control compared to specification limits in both head-and-neck and prostate plans. However, prostate plans are more process efficient because they are less complex than head-and-neck plans.

In this work, we selected MapCHECK (Sun Nuclear Corporation, Melbourne, FL) as a 2D diode array to verify the patient-specific IMRT QA, and used ArcCHECK for 3D diode arrays to perform patient-specific VMAT QA. The purpose of this study was to establish the appropriate guidelines for % gamma pass of patient-specific IMRT and VMAT QA of nasopharyngeal carcinoma by using an X control chart, and to evaluate the QA performance by using a process capability index.

MATERIALS AND METHODS

IMRT planning and QA

A relatively large cohort of 278 nasopharyngeal carcinoma plans in almost 2 years (2007–2009) of patient-specific IMRT QA has been considered using a MapCHECK 2D diode array. This device is composed of 445 n-type solid state detectors with an area of 22.0 × 22.0 cm². The inner 221 detectors cover the central part of 10 × 10 cm² with 0.707 cm diagonal spacing, while the outer 224 detectors have 1.414 cm diagonal spacing. The active area of each detector is 0.8 × 0.8 mm [2, 11]. Almost all of the nasopharyngeal carcinoma IMRT plans were performed with nine equi-angular beam arrangements of coplanar technique with dynamic MLC generated by the Eclipse treatment planning system (version 7.3.10; Varian Medical Systems, Inc., Palo Alto, CA, USA), using 6 MV on a Varian Clinac 21 (80 leaves) or 23EX linear accelerator (120 leaves) (Varian Medical Systems, Inc.). The isodose distribution is shown in Fig. 1. After the patient plan was approved by a radiation oncologist, medical physicists created the pretreatment verification plan based on the composite plan at a 0° gantry angle with the beam directed perpendicular to the verification plan. MapCHECK has 1.35 cm inherent buildup equivalent to 2 g/cm² water, the source-to-skin distance (SSD) on the MapCHECK surface was set at 98.65 cm and then it was covered by 3 cm of solid water phantom as shown in Fig. 2a. Therefore, the water equivalent depth of total buildup was 5 cm, which was approximately equivalent to the average depth of the head and neck region. However, before the IMRT plans were verified, the absolute dose of 200 cGy was calibrated at every QA session to reduce the effect of output variation. Because with version 6.1 of the SNC Patient software (Sun Nuclear Corp., Melbourne, FL, USA) it is possible to compare the dosage in composite beams, this study evaluated the plan in composited beam verification with an absolute dose comparison using the gamma index. The criteria were 3% dose difference and 3 mm distance-to-agreement (^γ_3%/3mm) between the measurement and calculation with 10% threshold [12]. The gamma index was applied from the ellipse formula by using the dose and distance difference between measurement and calculation. A point that had a gamma value higher than 1.00 would not pass the criteria. The percentage of points that pass the criteria can be called the % gamma pass or gamma pass rate.

Fig. 1.

The isodose distribution of the nasopharyngeal carcinoma plan for IMRT and VMAT treatment techniques.

Fig. 2.

The setting up of (a) MapCHECK for patient-specific IMRT verification; and (b) ArcCHECK for patient-specific VMAT verification.

VMAT planning and QA

One year after the VMAT machine (Varian Clinac iX linear accelerator; RapidArc; Varian Medical Systems) had been installed in our institute (year 2011), 159 VMAT plans of nasopharyngeal carcinoma were optimized and calculated using version 8.9.17of the Eclipse treatment planning system. From our experience, 6 MV with a 2.5 arc (185° to 175° CW and CCW rotation and 185° to 0° CW) was an optimal technique to treat the head and neck region. The maximum repetition rate was set to 600 MU/min. The collimator rotation was set at 340° for the CW arc and 20° for the CCW arc to reduce the tongue-and-groove effect. The field size was manually set by the planner before optimization. A medium thickness of the Exact IGRT couch top structure was inserted into the planning to improve the accuracy of dose calculation. The medical physicist planned both IMRT and VMAT techniques as shown in Fig. 1, the IMRT treatments were planned first and the dose constraints from IMRT were used to optimize the VMAT plan. The oncologist selected the appropriate VMAT or IMRT plan to treat the patient by considering dose-volume histogram and isodose evaluation. If the VMAT plan was selected, the ArcCHECK cylindrical diode array was employed for patient-specific QA. This device is composed of 1386 n-type solid state detectors with an active size of 0.8 × 0.8 mm² in each detector. The detectors are arranged in a spiral array with 1 cm spacing along the cylinder and 1 cm along the circumference for 21.0 cm in detector array length (spiral height). The detectors are embedded in a 2.85-cm depth of acrylic buildup that is equivalent to a 3.28-cm water equivalent depth [13]. All detectors of ArcCHECK were perpendicular to the beam for all gantry angles. ArcCHECK was divided into two sections. The outer section has 6 cm thickness and the inner section is 15 cm in diameter of acrylic insertion capable to insert a thimble ionization chamber for central axis dose measurement. If the inner section is removed, the VMAT plan can be used to check the inhomogeneity correction of air in treatment planning. After the VMAT plan was approved by a radiation oncologist, the VMAT plan was recalculated in the ArcCHECK phantom based on a composite plan for the same patient plan of monitor unit to get the same MLC movement, the same dose rate variation and the same gantry speed modulation. The isocenter was set at the center of ArcCHECK by using lasers and cross-hairs, the SSD of 86.60 cm would read on the ArcCHECK surface as shown in Fig. 2 b. The ArcCHECK was also calibrated with an absolute dose of 200 cGy at 10 × 10 cm² field size for every QA session date. After dose calibration, the VMAT QA plan can be measured in a composite plan with the actual gantry angle. The measured doses from ArcCHECK were compared with calculated doses from planning in ArcCHECK software in absolute dose evaluation. The γ_3%/3mm with 10% dose threshold was used to analyze the VMAT QA results using the same criteria as IMRT QA [12].

Statistical process control analysis

The control chart consists of an upper control limit (UCL), center line (CL), lower control limit (LCL) and data points. The control chart is used to monitor and control the variation in the process. If the data are within the control limits, the data will be in-control process with only random variation of data. When some of the data are outside the control limits, systematic errors can occur. Those data points outside of the control limits should be removed to bring the process back into control. There are many types of control chart depending on the data type and number of subgroups. The individual (X) control chart is one type of Shewhart control chart that is commonly used to control the process variation. The % gamma pass of IMRT and VMAT QA plans showing the differences in MapCHECK/ArcCHECK measurement and Eclipse calculation are plotted on X control chart with plan number. The CL and limits of IMRT and VMAT QA are calculated from equation 1.

(1)

The MR is the absolute value between two consecutive points of % gamma pass data (MR_i = | X_i – X_i-1|). The and are the average % gamma pass and average moving range of all % gamma pass data. The constant d₂ depends on n value, where n is the number of values in the subgroup. In this case, n is equal to 2, so d₂ is 1.128 [14].

The first 50 data points were used to calculate the control limits. If there were any out-of-control data points and we could find the sources of error, those out-of-control points were removed. Then the CL and control limits were recalculated to get the corrected control limits from random error only.

The process capability indices are the measure of process performance according to the specification limits. C_p and C_pk are two common process capability indices that have been used in industrial engineering for long time. The C_p parameter explains how much the data are dispersed but does not consider the process mean, while the C_pk index illustrates how close to the process center the data points are but does not take the dispersion into account. The new index, C_pm, combines those two indices together to adequately describe process performance in one index. C_pm can be calculated from equation 2.

(2)

is an average of % gamma pass, while σ is the standard deviation of % gamma pass. USL and LSL are the upper and lower specification limits, respectively. If there are no standard criteria for specification limit, we can replace LSL with LCL. T is the process target value that can be assumed to be the average of the gamma pass value in conditions of no target value. In the case of patient-specific IMRT and VMAT QA, however, a one-sided lower specification should be set. There should not be an upper specification limit because the higher the passing rate, the better the result. The C_pm can be modified to C_pml as shown in equation 3.

(3)

A constant of 1.46 is recommended by Pillet [15] in the case of a one-sided specification limit. The higher the C_pml value calculated, the more efficient the result obtained. In industrial engineering, a process capability index value of 1.00 indicates a high quality QA process.

RESULTS

The CL, UCL and LCL of the X chart for 278 nasopharyngeal IMRT plans and 159 nasopharyngeal VMAT plans were calculated. When the limitations of the X control chart were calculated from the first 50 data points of the IMRT plans, a systematic error was identified for point numbers 26, 32, 33 and 34 (from Fig. 3a). Therefore, these four points were removed and then new limitations were recalculated. For the VMAT QA, data point number 26 was lower than LCL. This data point was not actually a systematic error because a very small field size with a complicated plan was employed in this case and also the ArcCHECK detectors were low resolution. We decided to remove this data point from the control limit calculation. The comparison of limitation results between the first 50 point calculation and the systematic error removed in X charts of IMRT and VMAT QA is displayed in Table 1. When those systematic errors were removed, the calculation limits in both IMRT and VMAT QA were narrower. The UCL, CL and LCL of IMRT QA after the systematic errors were removed were 105.1%, 95.1% and 85.0%, respectively. The UCL, CL and LCL of VMAT QA when systematic errors were removed were 103.1%, 96.7% and 90.3%, respectively. Because the maximum value of gamma evaluation was 100%, the UCL should not be considered. The LCL was the only parameter to define the passing criteria of percentage gamma pass.

Fig. 3.

Individual (X) control chart of % gamma pass of patient-specific (a) IMRT QA and (b) VMAT QA for nasopharyngeal carcinoma plans with center line (CL) and lower control limit (LCL). The open circle sign represents the calculated points of the control limits (without systematic error points), while the open square points are the data points of the remaining IMRT QA results.

Table 1.

The control limits of the X chart using all first 50 plans and with out-of-control points removed for nasopharyngeal carcinoma IMRT and VMAT plans

Techniques	Control limits calculated from first 50 plans			Control limits calculated from first 50 plans with systematic errors removed
	UCL	CL	LCL	UCL	CL	LCL
IMRT	107.9	92.9	77.8	105.1	95.1	85.0
VMAT	103.5	96.5	89.5	103.1	96.7	90.3

Figure 3a is the X control chart of the IMRT QA results. The control limits from the first 50 points were used to calculate for the removal of systematic error. The data in the control chart showed less than 3.6% of the points were (10 from 278 points) outside the control limit due to systematic error, however, there were some out-of-control points (6 from 278 points) that wereclose to the LCL but did not demonstrate systematic error. Most of these points were due to complicated plans with large areas of high dosage and high dose gradients. The remaining 262 plans from 278 plans (>94%) were within the control limits that implied random error of the QA process. Almost all of the systematic error points were due to human error, so the source of errors should be found and removed.

Figure 3b is the X control chart of the VMAT QA results. The control limits were calculated from the first 50 points. We found one out-of-control point from a complicated plan with a small field size. Therefore, we removed that out-of-control point and used only 49 points to calculate the control limit. The chart detected two systematic errors and three near-miss error points.

When we evaluated the plan by using the average and standard deviation of all IMRT and VMAT plans with systematic error removed, it showed almost the same numbers as the SPC method, with the average percentage gamma pass at 93.7% ± 3.7% for IMRT QA and 96.6% ± 2.2% for VMAT QA. This implied that the VMAT technique had a closer measured dose distribution to the predicted dose from Eclipse treatment planning than the IMRT study. The result was confirmed by process capability index C_pml values of 1.60 and 1.99 for IMRT and VMAT, respectively. These C_pml values were based on results with systematic errors removed.

DISCUSSION

The IMRT or VMAT techniques are the standard treatment techniques for the head and neck region at our center. There have been some publications that have compared the clinical evaluation between IMRT and VMAT treatment techniques. VMAT had smoother dose distribution as demonstrated in Fig. 1, and the treatment time was reduced. The dose conformity to PTV and dosage to normal tissues were comparable for the two techniques. However, there have only been a few dosimetric studies that have compared the QA results between IMRT and VMAT. We know that these two treatment techniques are quite complicated plans for the head and neck region. The QA results are important and would be one of the indicators to select the technique.

The MapCHECK 2D diode array is a device to verify the patient-specific IMRT QA in our routine, while the ArcCHECK 3D diode array is used for the VMAT technique. Although gamma evaluation is a standard tool for 2D to 3D planar dose QA, in this work, the control chart and process capability index are the optional tools applied to analyze IMRT and VMAT QA results for nasopharyngeal carcinoma study. When the data fall within the control limits, we can assume there are random errors only. In contrast, when the data are out-of-control limits, the chart will remind physicists to immediately check that QA result. If we can find the source of errors from the out-of-control point, those points can be called systematic errors that we should get rid of and bring the systematic error points back to in-control.

From Fig. 3, the systematic errors presented in the charts are due to: incorrect case comparison (point 26) of the same patient first name but different surname; incorrect depth of additional buildup (points 32, 33, 34) from 3 cm to 4 cm; exporting the wrong slice plane from planning for dose comparison (point 65); incomplete composite field (point 77) using a combination of seven fields instead of nine fields; SSD setup error (points 113–114) from confusing student setup of 2.0 cm depth instead of 1.35 cm PMMA inherent buildup; and incorrect calibration dose (points 158–159) by using the monitor unit of 10 MV to calibrate 6 MV energy. The remaining out-of-control points are due to the inherently complicated nature of the plans. Although the VMAT planning employed more beam parameters to modulate the radiation dose than the IMRT technique, the VMAT QA results are surprisingly better than IMRT QA. This is because the VMAT in Eclipse is better than IMRT for fluence map segmentation. It uses more of an aperture optimization that results in a more homogeneous dose distribution, lower plan complexity and a reduced high dose gradient with fewer MUs as shown in Fig. 1. The consequence is better matched in VMAT dose distribution between measurement and planning. The results of IMRT and VMAT QA can not be directly compared because they use different techniques and devices to verify the QA plan, but it seems from the process capability index that the VMAT QA process is more capable. The process capability index, C_pml, value of the VMAT plan is higher than the IMRT QA. This means that the passing rate of VMAT QA is closer to 100% with lesser variation in results than for IMRT QA. Because the C_pml of IMRT QA is 1.60, this implies the process of IMRT QA is quite satisfactory. For the VMAT QA, the C_pml is very high at 1.99, demonstrating the very good quality of the VMAT QA process.

The average % gamma pass is 93.7% ± 3.7% for IMRT QA and 96.6% ± 2.2% for VMAT QA. Our % gamma pass rate of IMRT QA from MapCHECK is not too difference from Lucas's study [16], which showed a pass rate of 92.7% ± 4.7% for head and neck QA plans. Also, our VMAT QA result is comparable with Scorsetti's [17], who presented an average gamma agreement index of head and neck VMAT QA of 96.7% ± 2.1%. One more reason that the pass rate of VMAT QA is higher than IMRT QA in our result is that our VMAT plan doses were calculated with a newer, more accurate algorithm. The IMRT plans were calculated from version 7.3.10 of the AAA algorithm and the VMAT plans were calculated with version 8.9.17. However, a direct gamma pass comparison between institutes or between techniques should not be performed because the gamma pass rate depends on several factors, such as the model used in treatment planning, the dose delivery system and the measuring instrument.

We have found many systematic errors in IMRT QA due to human error but few systematic errors occurred in the VMAT nasopharyngeal carcinoma plans. This is due to the ease of using the ArcCHECK: no need for additional buildup, simple set-up, and no plane selection in planning, therefore, the chance of systematic error is reduced. We found one systematic error in VMAT QA (point number 118), which was caused by calculating without an acrylic insertion but measuring with the insertion in place. Another one was not an actual systematic error (point 24) but was the effect of the low resolution of the detector of ArcCHECK. This plan was a boost to only the gross tumor, so the field size was very small. The areas that did not pass the gamma criteria were only at the edge of the field.

The tolerance level of the patient-specific QA for the acceptance process has been proposed by many groups, such as Venselaar et al. [18], Palta et al. [19], Stock et al. [20], De Martin et al. [21], Basran and Woo [22], Both et al. [23], etc., however, those limits are mainly focused on only IMRT QA. For example, Basran and Woo [22] and Both et al. [23] recommended that the local control limits for IMRT head and neck cases using MapCHECK should be 88% and 90% gamma pass, respectively. There are no standard criteria to set the threshold limit of the VMAT treatment technique. In this work, the calculated limit can be set at 85.0% gamma pass for nasopharyngeal carcinoma IMRT QA and about 90.0% for VMAT QA when the X control chart is used. These LCLs are the cut-off limit for separating systematic errors from random variation at our institute.

CONCLUSION

The concept of industrial engineering QA can potentially be applied to patient-specific QA as a modern QA tool. The control chart is an effective tool to detect uncontrolled variation. If the chart indicates that the process is under control then it can be used with confidence. However, if the chart indicates that the process is not under control, the pattern will assist the medical physicist to determine the source of variation, then eliminate it and bring the result back into control simultaneously. Because our results show IMRT QA to have lower efficiency (1.60) than VMAT QA (1.99), the IMRT QA process should be improved to increase its capability even both QA process are already capable. Most of the out-of-limit points from IMRT QA are due to human error, which indicates that efforts should be made toward more training and standardization. The VMAT treatment technique is not only better than the IMRT technique in clinical and dosimetric areas, but also in terms of QA. Therefore, we can treat the patient with more confidence with this new technique. The lower control limit of percentage gamma pass of IMRT for the head and neck is 85.0%, while the limit of VMAT for the head and neck is 90%.