Comparative dosimetric evaluation of independent verification systems: ArcherQA versus AccuCheck for volumetric modulated Arc therapy plan validation

Abstract

Background

Volumetric Modulated Arc Therapy (VMAT), as an advanced radiotherapy technique places higher demands on quality assurance (QA) during treatment planning. Although multiple verification systems have been widely applied in clinical practice. Comparative studies between ArcherQA and AccuCheck for VMAT plan verification have been relatively scarce. This study provides a comprehensive comparison of dose verification results between ArcherQA and AccuCheck systems for VMAT plans with the aim of providing practical evidence to guide the selection of appropriate quality assurance tools in clinical practice.

Methods

A total of 89 VMAT plans delivered with the EDGE linear accelerator were retrospectively collected from routine clinical practice, including 24 cranial, 23 thoracic, and 42 abdominal cases. All plans had passed institutional clinical dose verification prior to treatment. Independent secondary dose calculations were performed using both ArcherQA and AccuCheck to ensure cross-platform consistency. The gamma passing rate (GPR) (3%/3 mm) was used as the primary metric to evaluate agreement between TPS and secondary-calculated dose distribution, while mean dose differences (MDD) provided complementary information on PTV and OAR dose variations. Receiver operating characteristic (ROC) curves were generated to determine the optimal global GPR threshold under the 3%/3 mm criteria using Youden’s index, and the sensitivity and specificity of plan evaluation at this threshold were subsequently analyzed for both systems.

Results

The MDDs against TPS for PTV and OARs were within 3% and 4%, respectively. The GPRs (mean ± SD) for cranial, thoracic, and abdominal cases were (97.03 ± 1.84) % vs. (99.79 ± 0.20) %, (99.52 ± 0.21) % vs. (98.21 ± 0.67) %, and (99.63 ± 0.15) % vs. (99.49 ± 0.22) % for ArcherQA and AccuCheck, respectively. Significant statistical differences were observed in cranial, thoracic, and abdominal cases (t = -7.403, p < 0.001; t = 8.549, p < 0.001 and t = 3.309, p = 0.002). The optimal GPR thresholds under the 3%/3 mm criterion were 99.05% and 99.51%, with sensitivity/specificity of 1.000/0.687 and 0.842/0.673, respectively.

Conclusion

Both ArcherQA and AccuCheck demonstrated excellent dosimetric agreement with TPS satisfied clinical validation requirements, and can be regarded as reliable independent verification tools for VMAT plans.

Clinical trial number

Not applicable.

Introduction

With the advancement of computer technology and radiotherapy planning systems, modern radiotherapy techniques have evolved from traditional modalities to precise techniques such as 3D Conformal Radiotherapy (3D-CRT), Intensity Modulated Radiation Therapy (IMRT) and Volumetric Modulated Arc Therapy (VMAT) [1–3], thereby achieving higher accuracy than traditional approaches. Among these, VMAT technology achieves highly conformal dose distribution through rotational irradiation, offering advantages such as shorter treatment times, fewer monitor units, and uniform dose distribution [4]. VMAT has been widely adopted in the treatment of various malignancies, including glioma [5–6], breast cancer [7–8] and cervical cancer [9–10], demonstrating advantages in dose conformity and treatment efficiency across these anatomical sites. However, its complex dynamic modulation demands very high accuracy in implementation [11–13]. Therefore, prior to treatment, the dose distribution from the Treatment Planning System (TPS) must be verified, this process is known as pre-treatment Patient Specific Quality Assurance (PSQA).

As radiotherapy technology has progressed, QA methods have evolved from early one-dimensional point dose to comprehensive three-dimensional dose verification, The focus of verification has shifted from traditional phantoms to patients themselves [14]. Currently, two main types of verification methods are commonly used: Measurement-based QA (MQA) and Calculation-based QA (CQA). While MQA is widely employed, it suffers from drawbacks such as being time-consuming, heavy reliance on equipment, and occasional tissue heterogeneity and TPS errors [15]. In contrast, CQA enables rapid and efficient dose verification through computer simulations, detects data errors of data transmission and plan execution, and improves verification accuracy [16–19]. Several commercial quality assurance verification systems are currently employed in clinical practice. Among these, The Archer QA (v1.0.201015_R; HeiFei Wisdom Medical, China) verification system utilizes a Monte Carlo algorithm for dose calculation, a widely accepted high-precision method that yields results closely approximating actual physical dose distributions. Furthermore, the system incorporates a high-performance GPU platform, allowing it to perform complex 3D dose verification within 0.5 s. This rapid verification capability eliminates the need for accelerator beam time, significantly enhancing clinical workflow efficiency [20–22]. In parallel, AccuCheck (v1.2.3.18; Manteia Tecnologies) —an online treatment plan quality assurance system developed by Manteia—is designed specifically for adaptive radiation therapy (ART) planning and delivery. Its core technology combines deep learning with conventional algorithms, and integrates real-time image guidance to enable online dose verification and adaptive treatment support. This facilitates efficient dose validation and real-time adjustments. It is particularly suitable for complex cases requiring adaptive management while ensuring precision throughout the treatment course. Additionally, AccuCheck supports multi-modal image fusion (e.g., CT, MRI), enabling more comprehensive evaluation of anatomical changes during therapy. Therefore, differences in algorithms, the performance of the two systems may vary significantly across anatomical regions and tissue types.

Although numerous studies have examined computational verification systems, research on the emerging AccuCheck system remains scarce, and comparative studies on ArcherQA and AccuCheck in VMAT plan verification remains limited. This study aims to compare the performance of ArcherQA and AccuCheck in terms of PTV, OAR protection, and verification efficiency in VMAT plans to provide evidence for selecting suitable verification tools in clinical practice.

Materials and methods

Patient medical records

We retrospectively collected VMAT radiotherapy plans of 89 patients who underwent postoperative adjuvant radiotherapy at the Radiation Therapy Centre, Department of Oncology, First Affiliated Hospital of Army Medical University, using the Varian EDGE linear accelerator between January 2023 and December 2024. Among these patients, 24 had cranial tumors (12 males and 12 females; mean age 52 years, range 20–78), 23 had thoracic tumors (mean age 49 years, range 25–77), and 42 had abdominal tumors (mean age 56 years, range 29–74). Table 1 provides a summary of patient demographics, tumor characteristics, and treatment-related variables at baseline.

Table 1.

Patients’ baseline demographics and tumour characteristics

Characteristic	Cranial (n = 24)	Thorax (n = 23)	Abdomen (n = 42)
Tumor type	Glioma	Breast cancer	Cervical cancer
Age (years)	52 (20–78)	49 (25–77)	56 (29–74)
Gender (M/F)	12/12	0/23	0/43
PTV volume (cm³)	332.24 ± 118.31	713.77 ± 195.38	1080.07 ± 152.12
CI	0.90 ± 0.05	1.23 ± 0.44	0.98 ± 0.09
HI	0.04 ± 0.01	0.13 ± 0.20	0.05 ± 0.01
GI	2.58 ± 0.24	2.42 ± 0.29	3.88 ± 0.22
MU	483.58 ± 100.50	569.35 ± 146.96	515.70 ± 54.88

Note: All dosimetric values are presented as mean ± SD for the full cohort within each anatomical site

Radiotherapy planning

All patients underwent simulation using a Philips 16-slice large-bore CT simulator with an 85-cm bore diameter ((Philips Medical Systems, The Netherlands)), which suitable for different radiotherapy positioning requirements. Following simulation, the imaging data were transferred to the Varian Eclipse 16.1 treatment planning system, and all treatment plans were developed by the same senior medical physicist. Patients with thoracic tumors received four-arc VMAT with a prescribed dose of 5000 cGy to the PTV, while cranial and abdominal tumor patients were treated with two-arc VMAT, with prescribed doses of 6000 cGy and 5040 cGy, respectively. The Acuros External Beam (AXB) algorithm was employed for plan optimization, and treatments were delivered using 6 MV X-rays in flattened beam mode. All plans successfully passed measurement-based dose verification prior to treatment delivery.

Dose validation and assessment metrics

A total of 89 VMAT plans were imported into both the ArcherQA and AccuCheck verification systems, including full DICOM datasets (RT Plan, RT Dose, RT Structure, and CT simulation images). Both systems were utilized to compute secondary verification dose distributions, generate dose-volume histograms (DVHs), and determine GPRs. The gamma analysis employed the criteria recommended by Van Dyke et al. [23], with a dose difference of 3%, and distance-to-agreement of 3 mm, a 10% dose threshold. Plans with GPRs ≥ 95% were considered clinically acceptable. Figure 1 displays the working interfaces of the two verification systems. A dosimetric comparison was then performed between the two systems. The mean doses (Dmean) to PTVs and OARs and their corresponding GPRs, including overall GPRs, were recorded. Dosimetric differences were quantified using the percentage dose difference formula: MDD%= (Dose_Val- Dose_TPS) / Dose_TPS× 100%, where Dose_Val represents the dose calculated by the verification software and Dose_TPS denotes the TPS calculated dose. A comparison of the dose cloud maps between the two systems and the TPS is presented in Fig. 2. In this context, the initial dose distribution calculated by the treatment planning system (TPS) served as the reference standard for the two secondary verification systems, representing a computational reference rather than a physical gold standard.

Fig. 1.

Illustrates the verification interfaces of the two quality assurance systems: (a) ArcherQA, and (b) AccuCheck

Fig. 2.

Comparison of dose cloud maps of the two systems: (a) ArcherQA, and (b) AccuCheck

TPS data were analyzed to determine target volumes and calculate three indices. The homogeneity index (HI = (Dmax − Dmin)/Dmean) was used to assess dose uniformity. The conformity index (CI = (VT, ref/VT) × (Vref/VT, ref)) was calculated, where VT, ref is the target volume covered by the 95% isodose line, VT is the actual target volume, and Vref is the reference isodose volume; CI values range from 0 to 1, with values closer to 1 indicating superior conformity. The gradient index (GI = V50%/VT) was also determined, where V50% is the 50% isodose volume and VT is the prescription dose volume; lower GI values represent steeper dose gradients.

Organ-specific dosimetric analysis was performed by tumor site. For cranial tumors, mean doses to the brainstem and lenses were evaluated. For thoracic tumors, mean doses to the heart and lungs were recorded, and for abdominal tumors, mean doses to the small intestine, rectum, and bladder were analyzed. GPRs were evaluated for both PTVs and OARs, as well as overall plans. This allowed comparison of dose distribution agreement between the two verification systems in terms of PTV coverage and OAR sparing across different anatomical sites.

Determination of optimal threshold values

The Receiver Operating Characteristic (ROC) analysis is a standard method for evaluating binary classification [24–26]. This analytical approach involves plotting sensitivity (true positive rate) against 1 − specificity (false positive rate) to identify the optimal balance between these parameters, thereby effectively minimizing both false-positive and false-negative errors. In this study, treatment plans were categorized as unacceptable if any of the following criteria were satisfied: (1) the absolute percentage difference in Dmean exceeded 5% for any PTV or OAR, or (2) the GPR for the PTV fell below 95%. Plans failing to meet these criteria were considered clinically acceptable [27]. Crystalline lenses were excluded from this classification due to consistently large discrepancies observed with both verification systems, which reflect known limitations of gamma analysis for very small structures rather than clinically meaningful dose deviations.

Using this classification as the reference standard, ROC analysis of overall-plan GPRs was performed in GraphPad Prism (GraphPad Software, San Diego, CA), and the area under the curve (AUC) was calculated. The AUC quantifies discriminative performance, with standard interpretation: values > 0.9 indicate excellent accuracy; 0.7–0.9 represent good accuracy; 0.5–0.7 suggest limited accuracy; and < 0.5 indicate no better than chance. To identify the optimal GPR threshold, the Youden index (J = sensitivity + specificity − 1) was calculated across all candidate cut-offs. The threshold yielding the maximal Youden index was selected as optimal, and the corresponding sensitivity and specificity were recorded. This approach provides a threshold that balances sensitivity and specificity for clinical decision-making.

Statistical methods

All statistical analyses were performed using SPSS version 29.0 (IBM Corp., Armonk, NY). The normality of GPR distributions was assessed using the Shapiro-Wilk test. For data following a normal distribution, paired t-tests were used. Results are presented as mean ± standard deviation (x̄ ± SD). Non-normally distributed data were analyzed using the Wilcoxon signed-rank test. Statistical significance was set at p < 0.05 (two-tailed). Effect sizes were quantified using Cohen’s d, with interpreted as follows: small (0.2 ≤ d < 0.5), medium (0.5 ≤ d < 0.8), and large (d ≥ 0.8).

Results

PTV parameters in the TPS system

Analysis of TPS-derived data revealed significant variations in target volume, CI, HI, GI, and MU values among cranial, thoracic, and abdominal VMAT plans, as shown in Table 1. The number of entries for each dosimetric value is consistent across all cases (n = 24 for cranial, n = 23 for thoracic, n = 42 for abdominal). However, some values (e.g. CI, HI, GI, MU) are presented as mean ± SD for each site, and the same number of cases applies to each parameter within each anatomical group. The table has been revised to clarify that all dosimetric values are reported for the full cohort within each site. Target volumes increased progressively from cranial to abdominal regions. Thoracic plans showed higher CI and HI values than cranial and abdominal plans, corresponding to poorer dose conformity and homogeneity in thoracic targets. The abdominal plans had the highest GI values, reflecting steeper dose gradients. Notably, thoracic plans required significantly more MUs, suggesting greater planning complexity.

Comparative analysis of MDD

Figure 3 presents boxplot of mean dose differences between the two verification systems for both OARs. As shown in Table 2, MDDs for all PTVs and OARs were compared between the two systems. In cranial plans, significant statistically differences (p < 0.05) were observed between the two systems for mean dose in PTV, brainstem, and lenses. The brainstem exhibited relatively smaller discrepancies with a medium effect size (Cohen’s d = 0.512). Notably, an opposite trend was observed between target volume and OAR verification: AccuCheck showed better agreement with TPS for PTV mean doses, but larger discrepancies for OARs. The pronounced differences observed in bilateral lens doses with AccuCheck may be attributed to its limited accuraacy in detecting small-volume structures such as crystalline lenses.

Fig. 3.

Box plots comparing mean dose differences between ArcherQA and AccuCheck for PTV and OARs

Table 2.

Comparative analysis of dose parameter differences (%) between two verification systems (Mean ± SD)

Anatomical Site	Structure	ArcherQA	AccuCheck	t-value	p-value	Cohen’s d
Cranial	PTV	3.20 ± 0.95	0.13 ± 0.75	22.631	< 0.001	4.619
	Brainstem	2.99 ± 3.22	6.80 ± 6.93	-2.509	0.020	0.512
	Len-L	-7.38 ± 5.69	17.45 ± 8.97	-16.362	< 0.001	3.340
	Len-R	-7.88 ± 7.30	14.42 ± 7.79	-9.919	< 0.001	2.025
Thorax	PTV	0.90 ± 0.86	-1.62 ± 0.75	24.108	< 0.001	5.027
	Heart	-3.79 ± 2.88	1.71 ± 1.46	-7.929	< 0.001	1.653
	Lung-L	-2.17 ± 2.35	0.06 ± 0.16	-4.645	< 0.001	0.969
Abdomen	PTV	0.88 ± 0.24	-1.18 ± 0.37	32.017	< 0.001	4.883
	Small intestine	-0.61 ± 2.27	-2.20 ± 1.72	4.285	< 0.001	0.654
	Rectum	0.72 ± 0.90	-2.17 ± 9.68	1.928	0.061	0.294
	Bladder	0.98 ± 1.00	-0.53 ± 1.54	6.847	< 0.001	1.044

This opposite trend persisted in thoracic plans. ArcherQA maintained better consistency with TPS for thoracic target volumes, showing smaller mean dose differences (0.90 ± 0.86%), while but showed larger discrepancies in OAR doses (all Cohen’s d > 0.8). Both systems achieved mean dose differences within 2% for target volumes and 5% for OARs, with AccuCheck showing higher precision (< 2% differences) for both.

For abdominal plans, ArcherQA consistently outperformed AccuCheck, with mean dose differences within 1% for both PTVs and OARs, compared to ~ 3% with AccuCheck. Although significant statistical differences (p < 0.05) were observed for small intestine doses, the effect size was moderate (Cohen’s d = 0.654). No significant statistical difference was found between the two systems for rectal mean doses (p > 0.05).

GPRs analysis comparison

Figure 4 shows 3D GPR boxplots for both verification systems across the three anatomical sites, together with scatter plots for individual patients. The greatest inter-system discrepancy in 3D GPRs was found in cranial cases, whereas abdominal cases showed the smallest variation. ArcherQA demonstrated superior overall 3D GPRs in thoracic and abdominal regions, whereas AccuCheck exhibited better performance in cranial cases. For PTV-GPRs, ArcherQA showed lower performance in cranial cases, whereas AccuCheck consistently maintained mean PTV-GPRs above 95% across all sites. Both verification systems achieved GPRs > 99% for OARs (Fig. 5).

Fig. 4.

(a) Box plots comparing 3D GPRs between ArcherQA and AccuCheck across different anatomical sites, (b) Scatter plot of the 3D GPRs for all patients across the three anatomical sites

Fig. 5.

Box plots comparing GPRs for PTV and OARs between Archer QA and AccuCheck

Table 3 compares GPRs between ArcherQA and AccuCheck for PTVs and OARs. Detailed analysis of cranial cases revealed that AccuCheck significantly outperformed ArcherQA in both 3D GPRs and PTV-GPRs (p < 0.05, Cohen’s d > 0.8). The lowest GPR was recorded for ArcherQA in cranial PTV (81.44%). No significant inter-system difference was observed for brainstem GPRs (p > 0.05, Cohen’s d = 0.158), with comparable performance between systems for lenses.

Table 3.

Comparative analysis of GPRs (%) between two verification systems (Mean ± SD)

Anatomical Site	Structure	ArcherQA	AccuCheck	t-value	p-value	Cohen’s d
Cranial	3D	97.03 ± 1.84	99.79 ± 0.20	-7.403	< 0.001	1.511
	PTV	81.44 ± 13.62	99.01 ± 0.97	-6.410	< 0.001	1.309
	Brainstem	99.37 ± 1.24	99.53 ± 0.82	-0.773	0.447	0.158
	Len-L	100.00 ± 0.00	100.00 ± 0.00	\	\	\
	Len-R	100.00 ± 0.00	100.00 ± 0.00	\	\	\
Thorax	3D	99.52 ± 0.21	98.21 ± 0.67	8.549	< 0.001	1.783
	PTV	98.30 ± 1.17	91.69 ± 3.26	8.297	< 0.001	1.730
	Heart	100.00 ± 0.01	99.91 ± 0.27	1.618	0.120	0.337
	Lung-L	99.91 ± 0.17	100.00 ± 0.00	-2.563	0.018	0.534
Abdomen	3D	99.63 ± 0.15	99.49 ± 0.22	3.309	0.002	0.505
	PTV	99.01 ± 0.32	95.59 ± 1.64	13.206	< 0.001	2.014
	Small intestine	99.90 ± 0.08	99.53 ± 0.36	6.982	< 0.001	1.065
	Rectum	99.39 ± 0.67	99.62 ± 0.47	-2.026	0.049	0.309
	Bladder	99.41 ± 0.45	99.64 ± 0.54	-2.461	0.018	0.375

For thoracic cases, ArcherQA demonstrated significantly better performance in both 3D γ and PTV-GPRs compared to AccuCheck (p < 0.05, Cohen’s d > 0.8). AccuCheck showed lower PTV-GPRs (91.69%) in this region. No significant statistical difference was found for cardiac GPRs. A small but significant difference (p < 0.05, Cohen’s d = 0.534) was noted for left lung GPRs.

In abdominal cases, both systems maintained GPRs > 95% for all targets and OARs. The overall 3D GPRs exceeded 99% for both systems, with only minor inter-system differences (Cohen’s d = 0.505). ArcherQA showed significantly better performance in PTV and small intestine GPRs, while AccuCheck demonstrated slightly higher results for rectum and bladder GPRs, though these differences were not clinically significant (Cohen’s d < 0.4).

Determination of optimal GPR threshold points

I In this study, plans were considered clinically acceptable if they met all of the following criteria: (1) global GPR ≥ 95%, (2) PTV-GPR ≥ 95%, and (3) MDD < 5% for both PTVs and OARs. Crystalline lenses were excluded from this assessment due to significant dose discrepancies, which reflect known limitations of gamma analysis for very small structures rather than clinically meaningful dosimetric errors. ROC curve analysis was performed on 89 VMAT plans using the 3%/3 mm criteria (Fig. 6). The area under the curve (AUC) values was 0.864 for ArcherQA and 0.765 for AccuCheck, both exceeding the 0.7 threshold for satisfactory discriminatory ability.

Fig. 6.

ROC curves of two verification systems under 3%/3 mm criteria

The optimal GPR thresholds were determined based on maximal Youden index values. Optimal GPR thresholds were determined from the maximal Youden index values. ArcherQA demonstrated perfect sensitivity (1.00) with specificity of 0.687 at its optimal threshold of 99.05%. AccuCheck showed balanced performance with sensitivity of 0.842 and specificity of 0.673 at its optimal threshold of 99.51%. Notably, crystalline lens data were excluded from ROC analysis due to consistently large discrepancies between verification systems.

Discussion

VMAT, as one of the advanced techniques in modern radiotherapy, employs continuous single- or multi-arc rotational delivery to achieve highly conformal dose distributions while significantly reducing treatment time, monitor units, and improving dose homogeneity [11–28]. However, VMAT implementation requires precise synchronization of multiple dynamic parameters including gantry rotation, real-time multi-leaf collimator (MLC) motion, dose rate modulation, and gantry speed adjustment, which imposes stringent demands on system accuracy and stability [29–31]. Consequently, comprehensive QA for VMAT plans is essential.

In modern radiotherapy, plan QA is essential to ensure treatment safety and efficacy. Conventional measurement-based QA (MQA) uses ion chamber arrays for pre-treatment dose verification, with a 10% dose threshold and 3%/3 mm gamma criteria commonly applied to IMRT plans. Plans with GPR ≥ 95% are generally considered clinically acceptable [32–33], whereas those below this threshold require further review and possible re-verification or re-optimization. However, MQA has limitations: it is time- and labor-intensive, depends on detector performance, and cannot fully account for tissue heterogeneity or detect potential calculation errors in the TPS [34–35]. In contrast, calculation-based QA (CQA) provides several advantages over MQA. As an independent verification tool, CQA calculates doses using baseline parameters established during commissioning, under conditions consistent with the TPS. This enables detection of TPS modeling or configuration errors, which are often missed by conventional measurement methods. It can also verify the accuracy of data transfer. In addition to GPRs, CQA provides dose-volume histogram (DVH) parameter analysis, thus enabling more comprehensive plan evaluation [36–39].

This study employed two verification systems, ArcherQA and AccuCheck, to perform three-dimensional dose verification on 89 VMAT plans (cranial, thoracic, and abdominal). Global GPRs, as well as GPRs and mean doses for PTVs and OARs, were analyzed to provide detailed information on patient-specific dosimetry. Our results confirm our hypothesis that the fundamental algorithmic differences between Monte Carlo-based ArcherQA and the deep learning-based AccuCheck lead to site-specific variations in performance. Analysis of TPS-derived parameters (PTV, CI, HI, GI, MU) revealed poorer dose consistency and homogeneity in thoracic targets, likely due to variability in breast tumor morphology, complex target shapes, and respiratory motion [40, 41]. Higher GI values in abdominal targets, indicating steeper dose gradients, may reflect the clinical need for rapid dose fall-off to protect adjacent organs such as the intestines. These findings are consistent with those of Yamamoto T and Dai et al. [42, 43]. For cranial, thoracic, and abdominal cases, significant statistical differences were observed between verification results from ArcherQA and AccuCheck and TPS calculations for all parameters except rectal doses. Notably, the two systems showed opposite trends in mean dose differences between PTVs and OARs. In thoracic cases, ArcherQA showed smaller differences for PTVs, whereas AccuCheck showed better agreement for OARs—a pattern also seen in cranial and abdominal cases. Overall, both systems maintained mean dose differences within 3% for PTVs (mostly within 2%), and within 4% for OARs (mostly within 3%) excluding cranial OARs. Boxplots showed greater variability in cranial and thoracic cases, with both systems overestimating cranial mean doses relative to TPS. Both systems showed substantial discrepancies in lens dose verification, likely due to difficulties in identifying these small-volume structures. This is consistent with reports by Geng C et al. [44] and Azimi R et al. [45], and highlights a key area for improvement in current computational verification systems.

Across all three anatomical sites, both ArcherQA and AccuCheck achieved average 3D GPRs exceeding 97%. While AccuCheck demonstrated a higher overall GPR in cranial tumors, ArcherQA showed better agreement with the TPS in the thoracic region. Notably, these inter-system differences progressively decreased from cranial to abdominal cases, leading to excellent agreement with the TPS for both systems in the abdomen.

Inter-system differences were more pronounced for PTV GPRs, though minimal in abdominal cases. OAR GPRs consistently exceeded 95% across sites, with thoracic OARs showing the smallest differences and best agreement with TPS. These results suggest that both systems perform well for abdominal VMAT plans but show greater variability in cranial cases. In the cranial region, with its complex anatomy, heterogeneous tissues, and irregular target shapes, Monte Carlo may introduce systematic biases, whereas deep learning models appear to perform better, likely due to training on diverse datasets. In this study, the cranial volume of PTV was (332.24 ± 118.31) cm³, which was smaller than the thoracic and abdominal PTV volumes ((713.77 ± 195.38) cm³ and (1080.07 ± 152.12) cm³) respectively. The relatively low PTV-GPRs observed for cranial cases using ArcherQA (mean 81.44%) warrant particular attention. This may be attributed to the small volume of cranial targets combined with steep dose gradients; under such conditions, even minor discrepancies in dose calculation algorithms or spatial registration between ArcherQA and the TPS can be amplified, resulting in lower PTV-GPRs despite clinically acceptable dose distributions. This likely explains why the PTV-GPRs for cranial targets was lower than that for thoracic and abdominal targets. In thoracic cases, respiratory motion and low-density lung tissue posed challenges, but Monte Carlo retained advantages in modeling dose distribution. For abdominal tumors, where tissue density was more uniform and target volumes larger, both algorithms achieved high accuracy, reducing inter-system differences.

VMAT plans demand higher delivery precision and parameter coordination compared to conventional static or dynamic IMRT. However, current VMAT dose verification commonly adopts IMRT thresholds, with GPR ≥ 95% typically regarded as acceptable. Moreover, these evaluation criteria do not fully capture dose variations in clinically relevant anatomical regions, thereby limiting their clinical significance. Previous studies have also noted misclassification risks when relying solely on GPRs: false negatives, where high GPRs occur despite clinically significant dose deviations, and false positives, where low GPRs occur despite clinically adequate dose delivery. T To define more appropriate GPR thresholds, we applied ROC curve analysis based on GPRs and mean dose parameters from 3D verification. This approach was used to identify optimal thresholds for VMAT plan evaluation. For ArcherQA and AccuCheck, the optimal γ passing rate thresholds were 99.05% and 99.51% (3%/3 mm). Raising thresholds reduced false negatives and false positives, providing a better balance between sensitivity and specificity.

This study has several limitations. First, as a single-center retrospective study based solely on plans generated with the Varian Eclipse TPS and Acuros AXB algorithm, the generalizability of our findings to other institutions using different planning systems or algorithms may be limited. This study is a single-center retrospective study with varying sample sizes across sites, reflecting real-world case distribution. Although sample sizes were not fully balanced, the smallest group (23 cases) still had adequate statistical power for the primary analyses. Future multicenter prospective studies are needed to validate these findings.

Second, the reference standard was the TPS-calculated dose; although clinically accepted as the benchmark, any inherent inaccuracies would systematically affect evaluation of both systems. As noted in the Methods section, the TPS-calculated dose was used as computational reference rather than a physical gold standard. Finally, our analysis primarily relied on GPRs and MDDs; incorporating a more comprehensive set of DVH parameters for a broader range of OAR in future studies could provide deeper clinical insights.

Conclusion

Dose verification of VMAT plans using ArcherQA and AccuCheck demonstrates feasible, efficient, and easy to implement. Both software systems exhibit strong agreement with the TPS, making them suitable for use as independent third-party tools for clinical radiotherapy dose validation. They may enhance the rigor of existing verification workflows and improve treatment plan safety. Overall, both systems are clinically acceptable for VMAT dose verification. The choice between them may be guided by clinical priorities: AccuCheck may be preferred in cranial SBRT where target dose accuracy is critical, whereas ArcherQA demonstrated excellent consistency in thoracic and abdominal cases, potentially offering advantages for OAR sparing evaluation.

Abbreviations

VMAT

Volumetric Modulated Arc Therapy

IMRT

Intensity Modulated Radiation Therapy

3D-CRT

3D Conformal Radiotherapy

QA

Quality Assurance

PSQA

Patient Specific Quality Assurance

MQA

Measurement-based QA

CQA

Calculation-based QA

TPS

Treatment Planning System

PTV

Planning Target Volume

OAR

Organ at Risk

GPR

Gamma Passing Rate

MDD

Mean Dose Difference

DVH

Dose Volume Histogram

HI

Homogeneity Index

CI

Conformity Index

GI

Gradient Index

ROC

Receiver Operating Characteristic

AUC

Area Under the Curve

SD

Standard Deviation

MU

Monitor Unit

AXB

Acuros External Beam

DICOM

Digital Imaging and Communications in Medicine

CT

Computed Tomography

MRI

Magnetic Resonance Imaging

Author contributions

Conceptualization, F.R.; Methodology, T.W. and F.R.; Validation, T.W.; Formal analysis, T.W. and J.C.; Investigation, T.W.; Resources, J.C.,K.Z., and H.G.; Data curation, J.C., J.W. and Y.C.; Writing—original draft preparation, T.W.; Writing— review and editing, F.R., Z.W., and J.L.; Visualization, T.W.; Supervision, T.W. and F.R.; Project administration, F.R.; All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Natural Science Foundation of Xizang Autonomous Region (Grants No. XZ202501ZR0081) to Z.W., the Natural Science Foundation of Chongqing, China.

Data availability

No datasets were generated or analyzed during the current study.

Declarations

Ethics approval and consent to participate

This study was approved by the Research Ethics Committee of the First Affiliated Hospital of Army Medical University (Approval ID: (B) KY2025062). This study is a retrospective analysis that dose not involve any organizations, direct patient contact, or the disclosure of patient personal privacy.

Consent for publication

No applicable.

Competing interests

The authors declare no competing interests.