Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Mar 15.
Published in final edited form as: Int J Radiat Oncol Biol Phys. 2020 Oct 24;109(4):1054–1075. doi: 10.1016/j.ijrobp.2020.10.021

Adaptive radiation therapy (ART) strategies and technical considerations: A state of the ART review from NRG Oncology

Carri K Glide-Hurst 1, Percy Lee 2, Adam D Yock 3, Jeffrey R Olsen 4, Minsong Cao 5, Farzan Siddiqui 1, William Parker 6, Anthony Doemer 1, Yi Rong 7, Amar U Kishan 5, Stanley H Benedict 7, X Allen Li 8, Beth A Erickson 8, Jason W Sohn 9, Ying Xiao 10, Evan Wuthrick 11
PMCID: PMC8290862  NIHMSID: NIHMS1681147  PMID: 33470210

Abstract

The integration of adaptive radiation therapy (ART), or modifying the treatment plan during the treatment course, is becoming more widely available in clinical practice. ART offers strong potential for minimizing treatment-related toxicity while escalating or de-escalating target doses based on the dose to organs at risk. Yet, ART workflows add complexity into the radiation therapy planning and delivery process that may introduce additional uncertainties. This work sought to review presently available ART workflows as well as technological considerations such as image quality, deformable image registration, and dose accumulation. Quality assurance considerations for ART components and minimum recommendations are described. Personnel and workflow efficiency recommendations are provided as well as a summary of currently available clinical evidence supporting the implementation of ART. Finally, to guide future clinical trial protocols, an example ART physician directive and a physics template following standard NRG Oncology protocol is provided.

1. Introduction

Adaptive radiation therapy (ART) was introduced in the late 1990s as “a closed loop radiation treatment process where the treatment plan can be modified using a systematic feedback of measurements with the intention to improve radiation treatment by systematically monitoring treatment variations and incorporating them to re-optimize the treatment plan early on during the course of treatment”1. By accounting for changes in the patient’s anatomy during the treatment course, isotoxic based radiotherapy (escalating or de-escalating target doses to maintain a constant, acceptable risk of clinical toxicity based on the dose to organs at risk (OARs)) has been demonstrated24. ART may be implemented to address patient-specific treatment variations including systematic changes in weight, tumor and organ geometric and biological response, as well as stochastic variations such as organ deformation, filling change, and respiration and peristaltic motion. These variations may occur at different time scales ranging from seconds, to hours, to days. As a result, the implementation of ART is often binned into three major classes: (1) offline between treatment fractions, (2) online immediately prior to a treatment fraction, and (3) real-time during a treatment fraction, with major steps outlined in Figure 1.

Figure 1:

Figure 1:

Typical elements in adaptive radiation therapy workflows including online, offline, and real-time approaches.

Offline ART mostly addresses systematic and progressive changes that occur during the treatment course such as patient weight loss and tumor morphological changes5. For offline ART, adjustments to a patient’s treatment plan parameters based on these observed changes are modified after the current treatment fraction, typically following the same clinical workflow as regular initial treatment planning. Repeat simulation may be required if the acquired in-room image is not sufficient for treatment planning, followed by contouring, planning, and patient specific quality assurance (PSQA) and the resultant new treatment plan is reviewed by the physician and then implemented in subsequent delivery sessions. Offline ART has been shown in prospective clinical trials in the prostate, head and neck, and lung to yield improved target coverage and OAR sparing2,69.

Many treatment variations such as interfractional target and organ displacement, particularly in the abdomen and pelvis, and deformation of OARs will occur during a shorter time scale and thus, offline ART is not sufficient to account for these variations. Online ART is a process where the patient’s treatment plan is adjusted prior to the treatment delivery to account for temporal and stochastic changes detected in a single treatment fraction while the patient remains in the treatment position. As a result, online ART requires imaging, rapid replanning, plan review, and an acceptable form of PSQA. While resource and time intensive, online ART has shown promise for conventional and stereotactic body radiation therapy to enable better OAR sparing and improved target coverage, particularly in the head and neck10, abdomen1114, pelvis1518, and more recently presented for ultra-central lung cancer19. Promising data has emerged for using daily online MR-guided ART for localized prostate cancer showing low incidence of early gastrointestinal and genitourinary toxicities for clinician and patient-reported outcomes20. A multi-institutional prospective Phase II Study of stereotactic MR-guided on-table adaptive radiation therapy with real-time respiratory gating for patients with locally advanced pancreatic cancer is currently enrolling to evaluate toxicity, overall/progression free survival, and patient-reported quality of life (QOL) using daily online ART21. Daily target dose escalation has also been proposed when OARs are in a favorable location although clinical evidence is still emerging.

To account for variations that occur within a treatment fraction such as respiration, internal status changes, and peristalsis motion, real-time ART has been introduced where the treatment plan is automatically adapted during treatment delivery without operator intervention. Real-time ART may be performed through treatment gating, dynamic tracking by the treatment machine (e.g. CyberKnife or Vero22 systems), by the MLCs, and intrafraction re-planning although such an approach typically requires continuous imaging with constant re-planning and rapid dose calculation2325. The CyberKnife Stereotactic Radiosurgery System with Synchrony® Respiratory Tracking System dynamically tracks tumors that move during respiration via an external to internal motion correlation model updated throughout treatment using x-ray imaging26,27. More recently, the Radixact® (Accuray Incorporated, Sunnyvale, CA), a next-generation helical tomotherapy system has integrated intrafraction motion management based on the Synchrony® to predict motion based on implanted fiducials or the tumor itself28. Real-time ART has also been realized using simultaneous intrafraction monitoring for target identification and MLC tracking to align the beam to the target for SBRT prostate cases using a standard linear accelerator29.

In addition to classification based on the time domain, ART may be characterized as anatomically or biologically driven based on treatment variations. Biologically-guided ART holds great promise because changes at physiological and molecular levels usually occur prior to anatomical change leading to early treatment adaption. Patient-specific biological changes during treatment have been shown to correlate with clinical outcomes and toxicity profiles, suggesting strong clinical benefits of biological guided ART in personalized treatment. However, biologically-guided ART still remains limited in clinical practice. The majority of the current studies focus on adjustment of target volumes based on functional imaging obtained during treatment course with a comprehensive review of biologically-guided ART provided in the literature30. Several ongoing clinical trials are underway (e.g. NCT02031250, NCT03416153, NCT03224000, NCT01504815) that aim to investigate functional sub-volume boosting and dose scheme changes based on functional imaging.

Regardless of the class of ART implementation or combination thereof, the increasing interest in ART along with emerging vendor-provided products and workflows will undoubtedly increase ART utilization. Yet, ART introduces complexities into the clinical workflow that will necessitate rigorous benchmarking and evaluation. This need will become even greater as ART is applied to clinical trials, where safe and consistent implementation is of paramount importance to ensure high fidelity in trial outcomes. Overall, this work describes considerations pertinent to the implementation of ART techniques and establishes a foundation for the safe and effective implementation of ART both in conventional clinical contexts and in clinical trials.

2. Technological considerations for ART

a. Image acquisition

Performing the necessary steps for ART requires adequate information for tumor/OAR delineation, accurate dose calculation, and sufficient image quality. Table 1 summarizes the major imaging modalities used in different ART workflows at the present time, their advantages and disadvantages, special considerations for their implementation, accompanied by a consensus subjective grading system for the merits of each modality. While Table 1 highlights the current imaging modalities being implemented, new image reconstruction algorithms are emerging that may have implications for ART performance such as iterative CBCT, which has shown promise for improved image quality and more complete FOVs than conventional CBCT31,32. Recently, Ethos™ (Varian, Palo Alto, California, USA) was introduced that integrates iterative CBCT for dose calculation on the anatomy of the day with clinical integration efforts ongoing33. Furthermore, with the recent trend toward hypofractionated treatment regimens, imaging doses are expected to become less of a concern in the future.

Table 1:

Image modality considerations for online and offline ART implementations. Grading system: (least to most advantageous, 1 * to 5 *) based on consensus grading by authors.

ART Imaging Modality Considerations Computed Tomography (on rails or simulation) Cone beam Computed Tomography Megavoltage Computed Tomography Magnetic Resonance Imaging Positron Emission Tomography
Contrast - Soft tissue differentiation Diagnostic quality Same as planning CT scan Good contrast for large density differences such as bone/ tissue/air. Scatter significantly decreases contrast. Good contrast for large density differences such as bone/ tissue/air. Scatter significantly decreases contrast. Excellent soft tissue contrast No soft tissue differentiation but provides quantitative functional information
**** *** ** ***** ***
Spatial resolution Same as planning CT scan – sub millimeter, can be limited longitudinally Same as planning CT scan – sub millimeter, longitudinally typically 1 mm Same as planning CT scan – sub millimeter, longitudinally typically 2 mm Similar to planning CT scan – sub millimeter, longitudinally typically 1 mm Typically few mm in each direction PET, depends on body site
***** **** **** **** ***
Motion artifacts Fast scan but motion must be managed to avoid artifacts Long scan times prone to motion artifacts Long scan times prone to motion artifacts Scan times can be long or short - prone to motion artifacts Very long scan times, prone to blurring from motion
Clinical motion management solutions Yes Yes No Yes No
**** ** ** *** *
Reconstruction artifacts Prone to hardening artifacts from high Z materials or elongated body shape, motion Same artifacts as CT, as well as scatter, ring, aliasing, and misalignment artifacts Same artifacts as CT, zipper artifacts Susceptibility, motion, distortion Same artifacts as CT, attenuation correction, motion, CT reconstruction as well as partial volume artifacts
*** *** *** ** **
Geometry
Anatomy Anatomy changes from CT, organ localization Anatomy changes, organ localization Anatomy changes, organ localization Anatomy changes, organ localization Metabolic uptake changes
**** *** *** ***** ****
FOV limitations 60 cm FOV Up to 50 cm FOV, large FOV results in poor image quality Up to 50 cm FOV, large FOV results in poor image quality Up to 50 cm FOV Up to 70cm FOV
**** *** *** **** ****
Patient position issues Same as planning CT scan Can affect image quality depending on position on treatment couch Can affect image quality depending on position on treatment couch Bore size may limit patient position, coil placement may limit use of accessories May not be same as simulation setup, PET scan bore size may limit patient position
**** *** *** *** ****
Truncated structures Same as planning CT scan FOV limitations may truncate structures FOV limitations may truncate structures FOV limitations may truncate structures and external contour Same as planning CT scan
Tracking organ motion Not available during treatment Not available during treatment Not available during treatment Available Not available during treatment
* * * **** *
Density
HU table management Same as planning CT scan Can build custom HU table 1–2% accuracy in dose calculation MVCT number, similar to HU table, must be monitored at high frequency Not available, surrogate needed Same as planning CT scan if PET-CT
***** ***** *** ** *****
Online/Offline ART
Modality for Online CT on rails CBCT Tomotherapy able to sum plans and “dose of the day” MRI-Cobalt, MRI-Linac Under development
*** *** *** **** *
Additional dose to patient Up to 3 cGy per scan Up to 10 cGy per scan Up to 5 cGy per scan Not applicable Up to 3 cGy whole body plus CT dose
*** ** ** ***** **

b. Deformable image registration

Deformable image registration (DIR) is an important step commonly used during ART to account for changes in the shape and size of internal organs between the initial and adaptive planning images acquired during the treatment course. For offline ART, DIR is used as needed during the treatment course and the adaptive planning image may include high-quality CT images, images that were used for IGRT (i.e., CBCT and MRI), or an interim functional image such as a PET-CT or MRI. For online ART workflows, DIR is often employed at every fraction prior to treatment delivery to perform tasks such as deforming contours or performing electron density mapping between the initial planning dataset and the daily images used for patient positioning. At present, many treatment planning vendors and standalone image registration software suites offer DIR and ART workflows as summarized in Table 2.

Table 2:

Summary of currently available deformable image registration and relevant adaptive radiation therapy components

Vendor DIR Algorithm Unimodal Registration (CT, CBCT) Multimodal Registration (CT, PET, MR) Contour Propagation Dose Warping and Summation Offline ART Online ART
Stand-alone deformable image registration products
MIM (v6.8) Free-form, Demons optical flow27, 28 YES YES YES YES NO NO
Velocity (v3.2) Multipass B-spline29 YES YES YES YES YES* NO
Mirada (vRTX1.8) Unimodal: free-form Multimodal: Radial basis function30 YES YES YES YES YES* NO
Treatment planning systems with deformable image registration modules
Raystation (v9A) ANACONDA31, MORFEUS32 YES YES YES YES YES NO
Eclipse (v15.6) Unimodal: Accelerated Demons33 Multimodal: Adaptive gridbased Radial basis function YES YES YES YES YES NO
Pinnacle (v9.10) Fast symmetric Demons, Salient-Feature-Based Registration (SFBR)34, 35 YES YES YES YES NO NO
Monaco (v5.51) Gradient-free Dense Hybrid MI Deformation36, 37 YES YES YES YES YES YES
Precision (v2.1) Multi-organ B-Spline38 YES YES YES YES YES YES
ViewRay (v5.2.5) Free-form Unimodal: Correlation coefficient Multimodal: Mutual Information YES YES YES NO YES YES

ANACONDA= ANAtomically CONstrained Deformation Algorithm; MORFEUS= Multi-organ Finite Element Modeling Algorithm

*

Partial offline ART functionality (no treatment planning)

Deformation vector fields (DVFs), or the voxel-by-voxel 3D transformation matrix obtained from DIR34, are often applied for tasks such as contour propagation, plan adjustment, and fractional dose accumulation23,35. Therefore, any errors introduced in the image deformation process may be propagated downstream in the ART process. The major sources of error and uncertainty originating from DIR often arise from the image quality of the two input images, inaccuracy of the DIR algorithms, and any parameter selection or manual adjustment during the registration process. For online and offline ART, the input images include the original planning dataset (the moving image) and the stationary image acquired during treatment. It is important that both the moving and stationary image are evaluated for image quality as errors from the input images often arise from image artifacts (e.g. noise, blur caused by motion, image truncation, etc.) or image distortion such as in MRI.

In 1998, Maintz and Viergever36 summarized image registration variables and categorized them using nine criteria, including dimensionality, nature of the registration basis, domain of the transformation, degree of interaction, optimization procedure, image modalities, involved subjects, and body sites. Despite being 20 years later, these classifications still hold with minor revisions37. All of these variables introduce various degrees of errors and uncertainties during DIR that are convoluted in the DVF obtained from image deformation, which will then be applied for contour mapping and dose deformation/accumulation tasks. Therefore, it is essential for the end user to perform validation of the DIR algorithm.

However, direct quantitative validation of DIR using the DVF has proven difficult due to the lack of ground truth. Recently, AAPM Task Group 132 (TG-132) has provided guidelines on using qualitative and quantitative measures for evaluating image registration accuracy34. Qualitative methods include visual checking with various display methods including image-to-image comparison with or without mapped contour/structure overlays. Quantitative metrics include target registration error (TRE), mean distance to agreement (MDA), Dice similarity coefficient (DSC), Jacobian matrix (identifying local volume changes such as expansion or contraction that may indicate erroneous regions of interest) and consistency (or the independence of the algorithm to the direction of the registration). TG-132 has provided expected tolerances to each of these metrics based on the application and image voxel dimensions. Validation of DIR performance often consists of landmark verification such as bifurcations or implanted markers38. In addition, subjective scoring methods for evaluating the mapped structures have also been proposed39,40. Phantom datasets for multiple modalities (i.e. CT, CBCT, PET, and MRI) have been made available by TG-132 for DIR validation and are currently under evaluation by the NRG Image Deformation Working Group. Publicly available data sets have also been created for image registration validation including brain MR images41, head and neck CT images42, prostate CT images43, and thoracic CT and 4DCT images4346 to benchmark DIR performance as outlined in Section 4a.

c. Dose accumulation and tracking

ART may yield significant improvements in accommodating tumor and OAR changes during the treatment course when the original planning dataset is not fully representative of the anatomy of the day. However, as the anatomy and corresponding contours change, the initial dose calculated by using the planning dataset may have limited accuracy and may not continue to represent the actual delivered dose. For example, in a head and neck cohort of 13 patients, a dose reduction of 0.2–7.4 Gy was observed in the planning target volume (PTV) coverage (D95%) with increased maximum doses of 0.6–8.1 Gy and 0.2–15.4 Gy in the brainstem and cord, respectively47. Recently, MR-guided ART has shown that for pancreas SBRT, the dose to the duodenal loop increased up to 6 Gy while the PTV coverage reduced up to 4.5% if the plan had not been adapted48. Therefore, ART calls for an updated 3D dataset representing the current anatomy, an adaptive plan tailored to the anatomy change, and, ideally, an accurate summary of the ‘as delivered’ dose. Here, ‘as delivered’ refers to updated dose reporting that takes into account tumor and adjacent OAR anatomy changes, with a determined dose (DVH) that was delivered to the patient. To provide such an updated delivered dose, a voxel-by-voxel dose accumulation for each delivery timepoint needs to be performed by deforming the dose based on the calculated DVF from DIR over the treatment course with the dose warped back to the initial planning CT for dose accumulation over the total fractions to date49.

An alternative approach to obtain the daily delivered dose is to deform the initial planning dataset to match the daily IGRT image (i.e., CBCT, MVCT, CT-on rails, or MRI) for calculating the “dose of the day”50. By applying DIR, the dose calculated based on a deformed planning CT has been shown have 95% of the voxels agree at 2 mm/2% with the re-simulated CT dose50, which may be considered clinically acceptable. This methodology of deforming the adapted planning image yields improved dose estimation as compared to conventional dose calculations based on the rigid registration of the planning CT or directly on the CBCT itself.

Nevertheless, estimating the cumulative dose is still highly dependent on the choice of DIR algorithm and the underlying image quality. For online ART, ideally, a fully integrated treatment planning, imaging, and dose delivery system accompanied by an optimized DIR algorithm would be needed to implement this computationally intense adaptive workflow in an efficient fashion. The calculated “dose of the day” for each fraction can be warped back to the reference CT (i.e. the planning CT) or MRI dataset and summed to obtain the estimated accumulated dose. The accuracy of this accumulated dose is highly dependent on the DVF generated from the initial steps of image deformation. A wide range of DIR accuracy has been reported, ranging from <1 mm up to 10 mm depending on the disease site and DIR algorithm used41,43,51. Corresponding dose deviations illuminated via accumulation may have clinical impact, depending on the cancer site, image modality and quality, DIR algorithm, parameter choices, dose evaluation metrics (i.e. mean, max, min, D95, etc.), organ volume/motion, and other factors5255.

The accuracy of dose warping and accumulation depends on the accuracy of the DVF which may be limited by internal target changes (i.e., shrinkage or growth) and movement of the adjacent organs that may challenge boundary detection. Mass changes are a particular challenge for DIR and other methods to accommodate them, such as integrating models of tumor regression56 have been described in the literature. To date, limited studies provide reliable QA methods to ensure the accuracy of dose warping and accumulation for patient datasets, thus caution must be taken when applying to ART decision making.

d. Rapid Replanning

Re-planning cases for ART involves consideration of the strategy (offline vs. online), timely delineation of targets and/or OARs, the time it takes to re-plan, and the clinical criteria as to what necessitates the adaptation. RTOG 1106 is an example of an offline ART clinical trial schema for advanced stage lung cancer where the experimental arm includes a PET/CT and CT re-simulation acquired after Fraction 18, offline replanning, and a new treatment plan beginning on Fraction 22 to allow sufficient time for the development and QA of the adapted plan57. A recently published offline ART protocol for oropharynx cancer included weekly adaptation using geometrical criteria (when the GTV shrinkage exceeded 2 mm) via CT and MR-simulation data acquired at intervals of 5 ± 2 fractions58. An offline adaptive scheme using CBCT-generated contours from the initial six fractions of radiation therapy has been used to generate average positions of the CTV and rectum with ~7 ± 0.5 hours additional time needed to perform the additional replanning59. Offline ART workflow solutions are becoming commercially available to help the decision-making process regarding adaptation. For example, Accuray’s PreciseART® treatment planning system allows for automated dose monitoring and volume-based statistics that may be reviewed offline to assess the need for adaptation, with example cases taking between 2–8 fractions between plan evaluation and treating with a new adapted plan60. The total time required for offline ART will depend on several factors including the amount of multimodality imaging needed, total number of organs that need to be recontoured, dose accumulation/plan evaluation, PSQA if warranted, and any treatment planning dose constraint challenges that may be introduced.

To facilitate online ART, contours required for replanning must be generated rapidly while the patient is on the treatment table. Strategies to expedite recontouring have included implementing rigid or DIR to propagate delineated volumes from the initial simulation images or previous fractions to the daily image. Another strategy is to perform manual re-contouring limited to regions in close proximity to the target volume such as in an MR-guided online ART scenario in the abdomen where only the OARs within a 3 cm expansion of the PTV are delineated61,62. The rationale for using a subset of the OAR volume is that OAR dose tolerances are often expressed as a small volume dose constraint (typically D0.5cc) and presumably, these will be located close to the target volume. Recent results presented for MR-guided online ART showed clinically acceptable contouring times (median = 9 minutes, range 2–24 minutes) to allow for daily adaptation in a clinical trial setting using this contouring strategy11. Recent efforts using rapid autocontouring approaches such as deep learning63,64 are emerging and offer great potential to facilitate more efficient online ART. One such example is Varian’s Ethos online x-ray based ART solution that employs neural networks that uses a large library of images and ground truth contours to autosegment the anatomy of the day33.

Aside from recontouring, plan re-optimization must also be performed quickly for online ART. For head and neck cancer cases using CT on-rails and a conventional linac, an online ART workflow has been achieved in 5–8 minutes for plan reoptimization10. To perform a more expedited optimization, one strategy includes combining all OARs into a single optimization structure to decrease the total objectives that need to be achieved by the optimizer and thereby simplifying the optimization process. This also makes for a more robust planning approach since the achieved dose distribution will be less sensitive to expected daily changes, although caution must be exercised to ensure all necessary OARs are included in the optimization. Sophisticated workflows for online optimization have been implemented including using an artificial neural network that provided robust parameters that consistently met the OAR constraints, compared to a failure rate of 36% of fractions where a conventional optimization approach was used62. Re-optimization times ranging from 10 to 223 seconds for full reoptimization of lumbar spine bone metastases have been achieved on a 1.5T MR-Linac65.

To ensure accurate dose calculation, accurate CT number (and hence, electron density) is required. In an MR-guided ART workflow, multi-modality DIRs between the CT, daily electron density map, and MRI are performed that may severely warp the images as shown in Fig 2 and introduce uncertainties in dose calculation. This also requires substantial personnel effort to fix the underlying electron density map via manual overrides of air and tissue. Indeed, the evaluation and correction of electron density maps is a rate-limiting quality assurance step requiring significant resources in many MR-only and ART workflows6668 thus appropriate QA steps are required to address these uncertainties during the online replanning steps.

Figure 2:

Figure 2:

Deformable image registration is used in many aspects of the ART workflow including contour propagation and deforming electron densities as shown in the figure. Note the deformed right kidney and erroneously warped deformed density map highlighting the need for per-fraction review and quality assurance of the electron density map and contours used for online ART.

Overall, the time required to perform the entire online MR-guided ART process including IGRT, re-contouring, re-planning, and QA has been reported to have a median on-table time of ~80 minutes per fraction (range 36–160 min) for abdominal malignancies11 and 45 minutes (40–70 minutes) for prostate SBRT (typically 7.25 Gy/fx)66. For high-field MR-guided ART of abdominal SBRT using 4D-MRI guidance, the median overall total treatment was ~62 min using an adapt-to-shape workflow69 while for an online adapt-to-shape prostate SBRT (35 Gy/5fx) workflow, a median fraction treatment time of 50 min (range, 46–65) has been reported70.

e. Pre-Treatment Plan and Delivery QA

Offline ART strategies follow standard treatment plan and delivery QA approaches that external beam treatment planning workflow. Online ART, on the other hand, requires plan and delivery checks be performed in an accelerated timeframe. At present, commercially available options are limited, thus many groups are developing in-house QA tools. One such example is for MR-guided ART where a software program reads DICOM data from the base and daily adapted plan and compares the beam angles, number of segments, beam-on time, fluence patterns, and volumes (initial vs. replanned)71. An independent secondary dose calculation was also developed to ensure the adapted plan’s integrity before treatment. Another in-house QA tool built in C++ can be used for conventional and MR-guided linacs and checks demographics, imaging information (i.e., patient orientation, electron density), contour integrity, monitor units, MLCs and jaws within machine specifications, and dose calculation accuracy72. The MRIdian Linac (ViewRay, Inc., Mountain View, CA) has a vendor-provided online adaptive QA tool that runs on the treatment console and performs a rapid secondary dose calculation of the new adapted treatment plan. A report is automatically generated with plan comparisons, 3D gamma analysis, contour/dose statistics, and per-beam fluence comparisons. ART workflows may integrate 3rd party independent dose calculation and adaptive plan QA such as the Varian Ethos system that uses the Mobius QA platform33 while other commercial options are emerging or are being customized that provide independent dose calculation checks, such as MU2Net®, RadCalc®, etc. Ideally, automated plan checks and secondary dose calculation tools would perform an independent evaluation of plan quality, however these are still works in progress for many vendors and remain an unmet need.

f. Dose Reporting

A methodology for dose reporting is also required to ensure homogeneity across clinical trial study sites. For example, the protocol could state that full OAR delineation is required (whether in real-time or post-delivery) or that a more limited delineation scheme (i.e., a few cm from PTV) can be performed to facilitated more rapid contouring for online dose evaluation and subsequently reviewed offline with full delineations. An alternative strategy would be to implement concepts from brachytherapy as carried out in NRG-GY00673. Here, volume dose parameters are defined for dose tracking and reporting per ICRU Report #8974. In GY006, the specific reference point locations and volume definitions are well-defined (i.e., D2cc of the bladder, rectum, and sigmoid) and are recorded for each fraction73.

4. Quality assurance needs

a. Deformable image registration and contour propagation benchmarking

Direct qualitative evaluation of a DIR result can be performed via what TG-132 defines as an image-image visual validation of the deformed image with respect to the stationary image, including split screen displays, ROI overlapping, overlay assessment, or side-by-side display via a linked cursor34. Many commercially available software packages include functionality to visually display DVFs that can be overlaid on the deformed dataset, including incorporating color coding and vector length displays to highlight potential regions of non-physical or erroneous deformations. While TG-132 recommends that DIR programs are able to export a DVF in DICOM format, vendor compliance is still a work in progress. Nevertheless, to properly perform a quantitative benchmarking of DIR accuracy, appropriate physical or digital phantoms are required. While deformable physical phantoms with implanted landmarks have been built, at present they are not widely commercially available7577.

Benchmarking multi-modality ART workflows such as MRI/CT or PET-CT/CT with physical phantoms also introduces challenges in phantom construction and landmark visibility. The major advantage of using physical phantoms are to perform end-to-end testing in a clinical setting with consideration of the entire ART workflow. However, more straightforward DIR benchmarking can be achieved via the use of a digital phantom for comparing a user-obtained DVF generated by the DIR software with a gold-standard DVF. A digital validation set can be created in software from virtual phantoms or patient scans by generating a warped image and its associated structure set from an original image (and its associated structure set) with a known DVF78. Ideally, the original image and structure set, warped image and structure set, and ground-truth DVFs can all be imported to user’s DIR software for testing. Several studies have explored this approach51,79, while TG-132 also provided datasets created from ImSimQA™ software and recommended commercial DIR software vendors to provide feasible tools for user validation80. TG-132 provides guidelines for DIR evaluation metrics such as setting the tolerance of point-wise registration error or mean distance-to-agreement between two surfaces to within the magnitude of the maximum voxel dimension (approx. 2–3 mm).

Yet, few commercial systems adopted by radiation oncology departments have the recommended function for importing or comparing DVF files. In this case, indirect validation metrics (i.e. TRE, DSC, MDA, etc.) may have to be adopted for clinically feasible evaluation of DIR quality comparing propagated landmarks and structures with ground-truth34,80. A detailed multi-institution evaluation of DIR commissioning and QA is currently underway by NRG Oncology to provide benchmarking guideline for clinical trials involving DIR and ART. The testing criteria include TG-132 compliance, rigid registration accuracy, deformable registration accuracy between the planning CT and other image modalities (CT, CBCT, PET, and MRI) for various body sites including head and neck, lung, and prostate.

Auto-segmentation may also be part of an ART workflow. These contours, whether created de novo or through DIR contour propagation, should be reviewed by a radiation oncologist or other appropriately trained personnel. A rate-limiting step in the process may rely on a physician to re-contour the relevant organs at risk or target. Opportunities to make this more efficient include: workflows that enable safe remote contouring and viewing81, training therapists or other auxiliary staff to perform the initial re-contouring with physician approval82, and systematic applications of auto-contouring tools. Evaluation of the auto-contouring functionality should be assessed prior to clinical implementation, and a protocol for the review of contours generated during online ART should be established. Ultimately, the accuracy of the final contours should be within the uncertainty of an expert contouring the structure from scratch with a tolerance for the DSC value between two contours to be within the contouring uncertainty (approx. 0.8–0.9)34. Generally speaking, many online ART workflows consist of rigidly copying the target volumes to the daily image in lieu of deforming or modifying the target during the online process11. The clinical rationale for this decision is that complimentary, multi-modality diagnostic images as well as the consultation of surgical or diagnostic reports are often used to assist in target delineation which are not typically available at the time of online replanning. For example, in a prospective clinical trial for prostate SBRT, the prostate target volume was rigidly registered to the anatomy of the day and only edited as needed, such as with rotational differences62.

Dose accumulation may be used for retrospective evaluation of the delivered dose with the verification carrying particular significance when plans are created based on images other than a conventional simulation CT (e.g. dose based on a cone-beam CT or a synthetic CT generated from MRI). Dose accumulation accuracy depends upon the DIR accuracy as well as mass changes occurring during the treatment course83. Efforts are currently underway to further develop validation schemes such as developing new methods for dose mapping84, using energy conservation criterion485, developing uncertainty metrics86, and validation using computational78 or deformable phantoms87,88.

b. Machine-specific quality assurance

As is the case with non-ART workflows, the treatment machine needs to perform within specifications for reliable radiation delivery. For conventional mechanical and dosimetric assessment of machine performance, a standard QA program as described in AAPM Task Group Reports 142 is appropriate89. ART features an increased dependence on imaging systems in the treatment room. This underscores the need for appropriate periodic QA regarding image quality. Robust examination of factors such as geometric distortion, image artifacts, and HU-to-electron density calibration curves is necessary if the imaging system is to be used for ART replanning.

c. End-to-end testing of ART workflows

An end-to-end verification test should be conducted prior to clinical implementation of ART to evaluate the system holistically and to establish confidence in the dose delivered to the patient. To benchmark offline ART, digital phantoms such as the TG-132 test suite or POPI model can be implemented to benchmark DIR and dose accumulation depending on the imaging modality used in the workflow. For online ART, end-to-end verification should include the imaging of at least two geometries of a physical phantom using the modalities used in the ART workflow (i.e., CT, CBCT, MRI, etc.), the clinical utilization of ART subsystems (e.g. DIR, auto-contouring, dose accumulation, and plan re-optimization), and ultimately, the comparison of cumulative delivered dose with the intended dose modeled by the treatment planning system. A verification of the secondary dose calculation or verification system should also be performed using the modified geometry. Regardless of the additional tasks and subsystems involved in an ART workflow, the final dosimetric accuracy should be within the conventional guideline of ±5% of the intended dose.5 At present, only a few physical phantoms have been made by independent investigators to meet all these needs. Multi-modality anthropomorphic pelvis phantoms that mimic internal organ kinematics have been built recently88,90. Deformable lung91 and abdominal phantoms75,92 have also been devised and have been implemented to evaluate accumulated dose. In addition to end-to-end tests to verify the planning and delivery of an ART workflow, system analysis such as failure mode effects analysis (FMEA) may be used to characterize the ART process and to further direct efforts of the associated QA program such as described for real-time93 and online94 ART.

d. Adaptive plan-specific quality assurance

For online ART, patient-specific QA (PSQA) options may be limited prior to treatment delivery. Performing measurements on each plan can become impractical if additional plans are created frequently, and pre-delivery measurements may not be feasible for online workflows when the patient is on the table95. As a result, one must balance the practical costs of plan-specific QA while ensuring the dose delivered is safe and appropriate. In-vivo portal dosimetry allows for patient specific or transmission measurements that have been applied in several ART scenarios including using an electronic portal imager integrated into a 1.5T MR-linac96. Reoptimization methods such as implementing MLC aperture morphing from a base plan as opposed to a fully generated reoptimization may lessen the likelihood of a PSQA failure. Finally, clinical trial endpoints (e.g. dose that causes a specific toxicity, etc.) may need to consider the possibility that protocols will adopt limited manual re-contouring of OARs within some distance of the PTV.

Generally speaking, adaptive plans should be held to a similar standard with prescribed clinical criteria as the original plan. AAPM Task Group Report 218 discusses techniques for plan-specific intensity modulated radiation therapy (IMRT) QA and recommends tolerance limits and action limits of 95% and 90% γ passing rates, respectively, for 3%(global)/2 mm with a 10% dose threshold for both the Perpendicular Field-by-Field and True Composite methods.6 Another approach receiving increased attention is to simulate rather than measure the delivered dose. For offline workflows, machine log files generated during QA delivery of the plan with or without a phantom can be used.7, 8 For online workflows, it may be possible to perform a dry run where the mechanical components of the delivery are enacted, but with no dose being delivered. This could be performed with the patient on the table, but with obvious caveats regarding added time and risk. Lastly, for online workflows, various systems could be used to monitor the delivered dose in real-time in lieu of pre-treatment QA. Machine log files can be used in this way, retrospectively determining the fidelity of the delivered plan.7 Additionally, transmission detectors attached to the treatment machine or portal imaging devices may verify treatment field apertures and instantaneous output during delivery.

While various PSQA methods are available, it remains imperative that rigorous plan-specific checks are performed prior to treatment including: verification of plan data integrity, plan dosimetric quality, monitor unit calculations, and correct data transfer from the treatment planning system to the record and verify system.9 Software solutions are likely to play an increasing role in verifying consistent treatment parameters and accurate data transfer in the accelerated workflow of ART.10 Where feasible, QA on the deliverability of the treatment plan should be conducted prior to the patient’s treatment. This applies to offline ART plans as well as to the initial treatment plan for both online ART and offline ART. Post-treatment analysis of the delivered parameters will suffice where pre-treatment measurements are not feasible (e.g. adapted online ART plans) provided the other checks on plan integrity and data transfer have been performed properly.9, 11

e. ART action levels and evaluation criteria

Clear ART directives are required a priori to facilitate both online or offline ART in a systematic fashion. For offline ART, directives may be based on empirical data (i.e., at set time points for replanning) or practical clinical considerations (weight loss, tumor volume changes, review of anatomical changes in daily setup images, treatment breaks, or ill-fitting immobilization devices as examples). Example online ART objectives may include violations of predetermined OAR dose limits or target dose coverage considerations, although it is important to note that these should be evaluated using the dose expected on the geometry and delineated organs of the day.

Daily planning objectives will often be similar to those used for the generation of the initial plan and, whenever possible, should be pre-specified and imported into the treatment planning system to minimize time required for adaptive plan generation.

Ideally, to facilitate routine practice of offline ART, an automated dose-volume evaluation based on the daily treatment fraction would be ideal. The PreciseART tool (Accuray Inc., Sunnyvale, CA) is a semi-automated tool that initiates the dose-volume evaluation process as soon as each fraction delivery is completed60. The tool automatically creates the merged daily and plan images, deforms the plan contours, calculates dose on the daily image, accumulates the daily dose onto the planning CT, and generates a structured report with dose-volume data, user-defined metrics, flags, trends, and triggers for ART. The plan reviewer can thus identify at a glance if a particular dose-volume objective is no longer being met and if an adaptive plan is needed based on a pre-defined action level for future fractions.

For online ART, a solution to automatically and objectively determine when online ART is required immediately after the acquisition of the daily image is highly desirable. For example, Lim et al. reported a method to rapidly determine the need for online ART by analyzing the Jacobian determinant histogram obtained from the DIR between the plan and daily images without time-consuming and labor-intensive structure delineation based on the daily image97. The recently introduced iterative CBCT-based online ART platform incorporates guided clinical decision-making at several steps in the ART process including image approval, auto-contouring, and plan approval33. It is anticipated that further development of rapid evaluation solutions will be an active area of development.

f. Summary of minimum requirements and recommendations

Table 3 provides a summary of minimum elements and QA requirements to integrate ART into clinical trials, with associated clinical rationale provided for potential clinical impact.

Table 3:

Summary of minimum requirements to consider for ART workflow components.

ART Component Element Suggested Minimum Requirement Potential Clinical Impact
Imaging Hounsfield Unit Accuracy CT number accuracy within 10% 20% variation in HU value may result in a systematic dose error of 1.5%94
Geometric Integrity ≤ 1 mm (within 10 cm radial distance of isocenter)
≤ 2 mm (>10 cm radial distance away from isocenter)95
Inaccurate localization of organs Inaccuracies in dose calculation
Low Contrast Resolution Per AAPM TG recommendations for the ART planning modality Limited boundary detection that may adversely impact accurate delineation
Consistent physiological state as reference dataset (breathhold/internal filling) Motion managed within TG-76 recommendations (< 5 mm)96 Over/underestimate of target/OAR doses Incorrect state of internal anatomy for treatment planning
FOV Contains all relevant anatomy and full integrity skin contour Inaccurate dose calculation for missing anatomy Lack of one-to-one correspondence may lead to erroneous deformable image registration
Artifacts ART planning image shall be free of artifacts in the clinically useable FOV May obscure relevant anatomy Delineation accuracy adversely impacted Dose calculation may be adversely impacted
Image Deformable Registration Visual assessment
Point-wise registration error or mean distance-to-agreement within magnitude of maximum voxel dimension39
Erroneous deformations may warp images leading to inaccurate geometry and underlying electron densities
Registration Contour propagation Visual assessment
Dice similarity coefficient >0.839
Inaccurate dose evaluation due to incorrect target volumes Under or overestimated volumes
End-toend/workflow testing Localization using clinically applicable ART workflows and imaging modalities Data integrity verified via TG-5397 Concordance of external laser system/landmarks: Preferred: 1 mm Acceptable: 2 mm Localization uncertainties Input/output discrepancies Systematic offsets introduced
Dosimetric accuracy ±5% of intended dose5 Inaccurate dose evaluation

Currently, the most comprehensive benchmarking of ART was that implemented by the Trans Tasman Radiation Oncology Group (TROG) for a multi-institutional clinical trial of ART for bladder cancer98. The ART schema consisted of delivery of a conventional plan for the first 7 days of treatment with the remainder of the treatment delivered using one of three plan options with varied bladder filling conditions. The three different plans were generated based on a hybrid of the original planning CT and five CBCT bladder volumes acquired over the first week of treatment. Credentialing consisted of the following: (1) a facility questionnaire, (2) a treatment planning exercise, and (3) a site visit including a phantom-based implementation of image guidance. For clinical treatments, the presence of a trained team member is required during daily IGRT. The training of this individual consisted of a one-day course or an e-Learning module. The treatment planning exercise included the delineation of structures and the generation of plans with varied treatment planning margins based on the union of contours generated from several treatment fractions. Of interest is the onsite visit by trial coordinators that included discussions, lectures, review of the planning exercise and past clinical CBCT datasets, as well as a mimicked ART workflow procedure. Here, treatment planning was conducted on digital phantom data with an initial bladder filling condition and then IGRT and ART plan selection were performed based on differing anatomy. Dosimetric verification with thermoluminescent dosimeters was also conducted.

Another such example of multi-institutional implementation and credentialing for ART clinical trials is the Radiotherapy Trials Quality Assurance (RTTQA) group that has coordinated efforts across 10 centers and 71 radiation therapists in the United Kingdom99. Here, real patient data was used for credentialling, including contouring, treatment planning, IGRT, the plan selection process, and rapid review of the first enrolled patient. Overall, the credentialing process tested the main components of the trial ART workflow including hardware and software while also including the decision-making process. This broad benchmarking underscores the fact that ART is dependent not only technology, but also on workflow and procedure. For that reason, both pre-implementation and periodic quality assurance needs to evaluate the technique from a comprehensive perspective.

5. Personnel recommendations

a. Online ART physician directive and approval

Regardless of ART approach, the attending physicians must first specify quantitative adaptation criteria based on a physician directive to determine the necessity of adaptive replanning. Typical components include the structures to be recontoured, OAR volumetric constraints, and minimum target coverage criteria subject to the OAR constraints. If all OAR constraints are met due to favorable geometry, another ART action criteria may be target coverage improvement above a certain threshold, such as >10%, as compared to the original plan.

When online ART is anticipated, substantial physician involvement may be required and analogous to that required for non-adaptive or offline treatment planning, but with increased time constraints for online ART. For online ART, physician approval may be required of patient localization and positioning, which is analogous to approval of simulation in the offline setting. Subsequent delineation and thorough review of target and OAR segmentation are required to evaluate the need for online adaptation and facilitate plan reoptimization if clinically indicated. Similar to offline ART processes, review and approval is required of target and OAR segmentation prior to re-planning, with objectives provided for target coverage and OAR sparing. While physicians often participate in this process, recent efforts have been implemented to train radiation therapists or other staff members to perform contouring81. Ultimately, if a new plan is found to be justified based on predefined clinical criteria, documented physician approval of the new plan and associated QA are required to confirm the adapted prescribed dose, volume, and technique, and to document any planned escalation or deescalation in the prescribed target dose based on the patient anatomy of the day. In the context of clinical trial implementation, the physician directive should also include the objective indication for ART to generate data regarding prevalence of specific ART indications.

b. Online ART Tasks and responsibilities

An online ART workflow can be best described as a choreographed process involving contributions from several team members including radiation therapists, medical physicists, and physicians with typical roles as outlined in Table 4. An example low-field online MRgART workflow is described as it has been previously published in several clinical trials11,20,48 with similar workflows also being reported for high-field MRgART100. First, the radiation therapists bring the patient into the room, perform initial setup, and acquire a volumetric MRI suitable for target alignment and with a large enough field of view to facilitate online treatment planning. The radiation therapist then aligns the treatment target in the image-guided radiotherapy workspace, and then pages the covering physician and adaptive planner (typically a physicist or dosimetrist). Deformable-registration based auto-segmentation is initiated, followed by manual edits of auto-generated critical structure contours. Critical structure contours may be edited by the adaptive planner and may be reviewed by the covering physician or other qualified personnel. The gross tumor volume is rigidly propagated (not deformed) and may be edited by the covering physician as deemed necessary. Derived structures, such as PTV expansions or optimization volumes, are generated based on pre-determined workflows that can be rapidly applied online. Dose is then recalculated on an electron density dataset that is derived from the registration of the initial plan’s electron density to the daily setup image. The “predicted dose,” or the dose that would have been delivered if the plan were not adapted, is then evaluated using dose-volume histograms based on the new anatomy and re-contoured structures. Based on the predicted dose, the current anatomy visualized in the setup image, and predetermined clinical criteria, a decision is made whether to treat as-is or to adapt.

Table 4.

Example online adaptive workflow actions and potential corresponding staff roles. Roles may be adjusted based on internal credentialing processes.

Action Therapist Dosimetrist Physicist Physician
Acquire setup imaging and align patient
Critical structure re-contouring
Gross tumor volume contour, as needed
Create derived contour structures
Pre-adaptation evaluation
Plan re-optimization
Plan evaluation
Quality assurance checks
Configuration of gating and beam-on

Performed by

Reviewed by

Attending physicians specify quantitative adaption criteria per plan based on a physician directive planning sheet is then utilized to determine necessity of adaptive replanning. If the decision is made to adapt, in one example online workflow, IMRT optimization is performed with the same structure weights and beam angles as the offline plan (only the structures themselves, as well as the electron density map, having changed). Beam angles and structure weights can be edited if needed, but usually, are not edited because of the corresponding increase in time required. Dosimetry of the adaptive plan is evaluated and a decision is made whether to treat the adaptive plan, treat the initial plan, or abort the fraction. Finally, gating parameters are set, if applicable, and the treatment is initiated.

c. Offline ART Physician Directive and Approval

Offline ART is often triggered by clinical observations such as loose fitting masks, patient weight loss, or changes observed over time on volumetric on-board imaging such as CBCT. One such example is highlighted in Fig 3 for a Stage III cT3N2cM0 nasopharyngeal carcinoma who was scheduled to receive 70 Gy in 33 fractions with concurrent chemotherapy. Major reductions in the primary tumor and bilateral neck nodes were observed on the CBCT by the 12th fraction while a weight loss of ~10 lbs. was also observed, prompting a re-simulation and new plan generation. DIR was conducted between the initial TPCT and the re-simulation CT using a Demons-based algorithm (SmartAdapt, version 13.0, Varian Medial Systems). Local regions of deformation and tumor regression can be observed.

Figure 3.

Figure 3.

Stage III cT3N2cM0 nasopharyngeal carcinoma patient who underwent an offline adaptive replan due to volume reductions in the primary tumor and bilateral neck nodes. (a) Initial planning CT scan at the level of the maximum extent of the primary nasopharyngeal cancer; (b) resimulation CT scan at fraction 12 showing a major reduction in the primary tumor volume with the original extent of the primary tumor in red; (c) resultant deformation map at the level of the primary tumor; (d) initial planning CT scan at the level of the neck nodes; (e) resimulation CT scan (fraction 12) at the level of the neck nodes showing the original extent of the neck nodes outside of the external anatomy; (f) resultant deformation map at the level of the neck nodes. Scale shown is the 3D vector displacement in mm.

In offline ART settings, requests are often made ad hoc by the physician and documented in the electronic chart. However, to implement offline ART more systematically in clinical trials, more rigid criteria are required such as defining a predetermined timepoint (e.g., after an initial dose or specific fraction57) or using a geometric constraint (i.e., for a head and neck trial when GTV shrinkage exceeded 2 mm via weekly imaging58. The offline directive should include the adaptive criteria, dose limits of the plan summation (either rigid or deformable as validated by the physics team), and physician approval of the final plan.

6. Efficiency recommendations

a. Frequency of plan adaptation

In an ART workflow, the frequency of plan adaptation can have many practical and dosimetric ramifications. In principle, increasing the frequency with which plans are adapted to changes in patient position, anatomy, and dose will maintain or improve the clinical goals of treatment including the therapeutic ratio. However, the dosimetric improvement – and therefore the cost-benefit ratio – of increasingly frequent adaptation is dependent on the clinical context, and may exhibit diminishing returns101. Specifically, increasing the frequency of adaptation when organs at risk are anatomically stable and tumor response occurs over the course of weeks may result in a decreasing incremental benefit and an increasing use of clinical resources, as has been demonstrated in lung cancer treatment planning studies.3 In contrast, daily online adaptation has been shown in a prospective clinical trial to allow substantial simultaneous dose escalation and OAR sparing for abdominal SBRT, where daily anatomic variation both in tumor and OAR anatomy is present.4 The optimal timing and frequency of adaptation, therefore, depends on anatomic changes characteristic of the treatment site, on the time interval of anatomic change, and on the proximity of a given target or an OAR to a steep dose gradient. These factors, in conjunction with the increased workload of repeating plan preparation steps like contouring, optimization, and verification, affects the optimal frequency of ART. In the context of clinical trial implementation, it is important to clarify the specific goal of adaptation, with objective action thresholds to allow multi-institutional uniformity of the adaptive technique. For example, if anatomic change results in violation of a previously defined OAR constraint or coverage goal, adaptive replanning may be objectively warranted.

b. Offline optimization and replanning

For disease sites in which anatomical changes occur gradually (over the course of several treatment fractions), offline optimization is generally preferred to online ART due to greater flexibility in time constraints and fewer required personnel. Definitive head and neck radiotherapy is a scenario where the anatomy exhibits small changes that trend over the course of treatment. Examples include the decrease in volume and movement towards midline of head and neck tumors.5 This pattern of change suggests that an ART workflow focused on occasional offline adjustments in response to the observed anatomic trends is appropriate, but must be based on objective criteria of decreased target coverage or clinically impactful increases in OAR dose. Offline ART is also appropriate when the adaptation is determined based on imaging findings that are available sufficiently in advance of the planned adaptation. This generally is the case when adaptation is based on planned interim imaging assessments as was performed in the RTOG 1106 trial utilizing FDG-PET/CT based adaptation for lung cancer as shown in Figure 4.

Figure 4.

Figure 4.

An example of initial (left) and adaptive (right) radiotherapy treatment plans from RTOG 1106. The initial plan was based on pre-treatment PET and CT target volumes and treated to 46.2 Gy in 21 fractions (Fx). The adaptive plan was designed to boost the residual CT and PET volumes based on during-treatment imaging. The boost dose was prescribed up to 3.8 Gy/Fx, limited by normal tissue constraints. PET volumes were based on auto-thresholding and dose summation was performed on the initial planning CT using rigid registration. Figure credit: Martha Matuszak, PhD.

c. Online optimization and replanning

Online ART offers greater theoretical benefit for clinical scenarios in which significant random interfraction anatomic change occurs, in particular when the change corresponds to a region with a steep dose gradient. In prostate radiotherapy, large random changes like bladder and rectal filling can affect the delivered dose. Because of the random nature of these changes, online corrections may be more appropriate for maintaining the desired dose distribution than offline corrections intended to address trending changes.6, 7 Furthermore, intrafractional changes may occur on the timescale of an individual treatment fraction, and so the duration of the imaging and replanning effort is of particular concern. A potential benefit to online ART has also been suggested for abdominal SBRT, which provides a similar clinical scenario of random interfractional changes including variable OAR position along a high dose gradient and high dose per fraction, all of which may significantly impact the delivered dose and associated toxicity risk.8

d. Example ART implementation for cancer sites

Appendix A highlights key evidence outlining the potential clinical benefits for major cancer disease sites. Considerations were given for online, offline, prospective, and retrospective trials with clinical and dosimetric endpoints summarized.

e. Specialty ART Planning Considerations

i. Proton therapy

Due to the sensitivity of protons to interfractional uncertainties relative to that of photons, ART is particularly advantageous for proton therapy. In a retrospective study of advanced NSCLC, 61% of patients re-planned with intensity modulated proton therapy (IMPT) would have required adaptation during treatment due to anatomical changes102. For online adaptive proton therapy, CBCT has been implemented with post-processing corrections to correct for the Hounsfield numbers as small inaccuracies may lead to large range uncertainties103. An ART proton therapy workflow has been described that generates a virtual CT scan (derived from CBCT coupled with DIR) to produce more accurate CT numbers and improved image quality for replanning104. Mobile helical CT has also been implemented for online ART proton planning to produce a high quality datasets in the ART process105. Further reduction of the range uncertainty is currently being investigated at multiple institutions106, but as of yet, they do not appear to be available in the treatment room, which will only permit offline adaptive regimens.

ii. Brachytherapy

Brachytherapy is perhaps one of the most conformal and adaptive approaches to deliver dose to a defined target. With the advent of the GEC ESTRO guidelines107 outlining the definition of a GTV, high risk (HR) HR CTV and IR CTV on MRI at the time of cervical brachytherapy, as well as the recent ICRU 69 Report108 further elaborating on volumetric brachytherapy, we have moved from film-based point dosimetry to volume-based brachytherapy for both the targets and the adjacent organs at risk. Brachytherapy is now referred to as image guided adaptive brachytherapy (IGABT) Potter, Tanderup, Kirisits, de Leeuw, Kirchheiner, Nout, Tan, Haie-Meder, Mahantshetty, Segedin, Hoskin, Bruheim, Rai, Huang, Van Limbergen, Schmid, Nesvacil, Sturdza, Fokdal, Jensen, Georg, Assenholt, Seppenwoolde, Nomden, Fortin, Chopra, van der Heide, Rumpold, Lindegaard, Jurgenliemk-Schulz, Group 109. With the advent of CT and MR-compatible applicators as well as sophisticated 3D digital images, radiation plans can be generated on these images with the applicators in place. This reveals the doses to key volumes of these targets as well as the OARs, so that modifications can be made to enhance target coverage and decrease dose to the critical organs. Manipulation of dwell times and positions as well as use of interstitial in addition to intracavitary applicators can be done for each fraction to optimally balance these competing dose constraints. Given the fact that usually 4–5 fractions are delivered for cervical cancer, each implant offers a new opportunity to adapt the dose distribution. This has led to a decrease in complications and an increase in both local control and survival that parallel and exceed the impact of concurrent chemotherapy109111. Combining the doses delivered with external beam and brachytherapy remains a challenge and a dedicated working group has been formed at NRG to address this topic. Current state of the art uses an EQD2 worksheet (downloadable at: https://www.americanbrachytherapy.org/ABS/assets/file/public/consensus-statements/gyn_HDR_BT_docu_sheets.xls) which converts both the brachytherapy and external beam doses to equivalent 2Gy doses for dose summation. Ideally, voxel-by-voxel dose accumulation of the external beam and brachytherapy components of treatment would be implemented, however these are currently works in progress.

7. Forward Looking Statements and Unmet Needs

a. Isotoxic dose escalation

Isotoxic based radiotherapy refers to treatment planning that is driven primarily by the acceptable clinical toxicity risk rather than a mandated target dose. For isotoxic planning, the target dose is escalated or de-escalated to maintain a constant, acceptable risk of clinical toxicity based on the dose to OARs. Isotoxic planning is not new, with prior implementations described for multiple disease sites including lung, prostate, and liver malignancies24. Prior reports of isotoxic planning are driven by inter-patient variability assumed to remain stable during a treatment course. In contrast, adaptive isotoxic treatment also allows treatment modification for a given patient due to anatomic changes that occur on an interfraction or intrafraction basis.

Adaptive isotoxic treatment planning has several implications that must be accounted in the context of clinical trial implementation. First, the maximum dose felt to be of clinical benefit should be determined a priori to prevent adaptive delivery of a higher target dose than is clinically warranted when OAR anatomy is favorable. Similarly, if the relationship between target and OAR anatomy is unfavorable, investigators must decide if a sacrifice in target coverage is truly warranted in order to maintain OAR isotoxicity.

In the absence of ART, application of an initial treatment plan to variable patient anatomy is known to frequently result in a dose to OARs that violates traditional hard planning constraints112. While ART re-optimization may be performed to avoid violation of constraints, previously established dose constraints in the non-adaptive setting may not accurately reflect true OAR tolerance. Since previously established constraints are based on static OAR anatomy, it is plausible that such toxicity metrics did not account for drift of OARs into a high-dose region unknown to the clinician. Such variability may be accounted for with current ART techniques. Therefore, in the context of clinical trial implementation, the delivered dose to OARs with isotoxic planning should be carefully documented so that clinical toxicity rates observed with isotoxic ART may be verified relative to prior expected values. In addition, online adaptive therapy allows the possibility to explore multiple novel facets of dose delivery, including daily alterations in dose per fraction, daily changes in dose homogeneity, and daily dose escalation or de-escalation.

b. Biological or functional guided ART

Traditional treatment response assessment based on tumor size and anatomical change is not always timely or necessarily correlates with final treatment outcome. Changes at physiological and molecular levels characterize the true underlying biological response to radiation treatment and usually occur much earlier than detectable morphological changes. Therefore, imaging biomarkers hold great promises for adaptive radiotherapy, wherein the treatment plan can be adjusted during therapy based on individual patient’s biological response. Recent studies have shown promising results of monitoring tumor biological and functional changes using image guidance system of radiotherapy treatment machines for potential biological image guided ART1,113. Recently, a prototype PET scanner coupled with a linear accelerator (RefleXion™ (Hayward, California, USA)) was introduced to conduct biologically adapted radiation therapy114, offering potential for PET-guided online ART in the future115. In order to deploy these advanced techniques in clinical trials, a few key challenges need to be overcome. Standardization of imaging acquisition protocols, measurement and analysis methods is essential for reproducible and consistent assessment of treatment response among multiple institutions. A rigorous quality assurance program needs to be established to allow for accurate quantification with sufficient validation. Most importantly, strategies and methods of incorporation of biological information into decision making of treatment planning need to be developed.

c. Integration of advanced computing

Several advancements in computing and programming offer strong potential to make both online and offline ART more efficient. One such example is the integration of a graphics processing unit (GPU) that enables high processing efficiency and yielding accelerated processing speeds for RT tasks at a relatively low cost116118. Many vendors have integrated GPU into their clinical software solutions, often for dose calculation and treatment planning. Current major unmet needs in the online ART workflow include rapid delineation and replanning that may be improved by the integration of deep or machine learning techniques into the workflow. For example, a convolutional neural network deep learning model was trained in ~12 hours for 100 patient abdominal datasets for online MR-guided ART, generating contours in ~5 seconds with good overall accuracy119. Deep and machine learning offer great potential for several other ART tasks such as automating treatment planning via accurate dose distributionsError! Hyperlink reference not valid., generating high quality planning datasets for accurate dose calculation120,121, and performing automated plan quality evaluation122.

d. Clinical trial integration

When incorporating ART into clinical trial design, the role of ART as a primary or secondary trial endpoint should be clearly defined. When characterization of ART benefit is a primary endpoint, the trial will generally be designed to report outcomes from a population 1) treated exclusively with ART in a phase I/II manner with descriptive clinical and toxicity outcomes, or 2) in a randomized phase II/III setting with patients either receiving or not receiving ART based on trial randomization, with direct comparison between trial arms. Given that online ART in particular is a relatively new approach, to facilitate more rapid evaluation of ART it may often be more feasible to incorporate ART as a secondary trial endpoint. As a secondary endpoint, ART may be incorporated or allowed for a broad spectrum of trials, where ancillary data may be generated to characterize ART benefits in the context of a non-adaptive primary study question. For such secondary integration, use and allowance of ART is similar in concept to current trial designs which often allow for variable planning techniques including 3D, IMRT, or proton-based treatment depending on institutional preference.

For any trial where ART is a primary or secondary endpoint, objective criteria that determine the specific action threshold to trigger an ART intervention are mandatory to ensure treatment uniformity. Such thresholds may be based on observed violations of initial study constraints during the ART evaluation, or a pre-specified improvement in target or OAR dose resulting from ART that is deemed to be clinically significant. An alternative clinical trial strategy that may be implementing ART as the stratification approach would be to apply ART for each treatment fraction in a manner that has been reported in the literature20,62,100. Another important consideration is the extent of plan review performed for ART. One institution evaluated their clinical practice of having the physicians and physicists perform a visual assessment of daily MR images without a full dose prediction for 7 pancreas patients (35 datasets) to determine the need for daily ART123. Importantly, a more thorough offline dosimetric analysis revealed that daily image review was not reliable and insufficient to determine the benefit of ART for a patient; visual assessment only resulted in 14/35 fractions undergoing ART whereas 25/35 were revealed to have potential clinical benefit. Thus, it is recommended in an online ART clinical trial setting, that daily contouring and dose prediction with a full dosimetric evaluation is performed with the appropriate time allocated for a safe and effective implementation of this process.

Given that noncompliance with radiotherapy protocol guidelines is known to correlate with inferior clinical outcomes124, it is imperative to verify that both physician re-contouring and adaptive plan quality are in accordance with protocol recommendations. The uniformity of physician re-contouring may be particularly challenging if imaging obtained for adaptive replanning does not clearly differentiate the extent of tumor response or of residual subclinical disease. Although protocol-mandated central review of physician contours and the treatment plan are widely implemented in current trials, such central review is not feasible for online ART due to the immediate nature of plan adaptation. Potential alternatives may be to develop a site-specific delineation atlas using the ART imaging modality or to require initial delineation cases for physician benchmarking. It is also recommended that a process be incorporated in the clinical trial design that before patient enrollment, in addition to physics and machine credentialing, institutional ART workflow will be confirmed. Such a process will confirm appropriate departmental workflow as per the personnel requirements section, with central review of the first ART case to include physician recontouring, adaptive planning, and appropriate evaluation regarding the clinical indication for adaptive treatment. It is also recommended that the initial (minimum 3) clinical adaptive cases for a new institution are retrospectively reviewed centrally after each adaptive fraction to ensure adherence to protocol.

Different adaptive strategies are appropriate for different treatment sites due to site-specific adaptive radiotherapy goals, and tumor and OAR characteristics. Recommendations on the range of possible adaptive frequencies and timing can be established based on estimates of inter- and intrafractional motion and their dosimetric impact.9 For example, plan adaptations could be triggered when the volume of the target has changed by a pre-specified action threshold or when dose to an OAR exceeds a tolerance level. Such action thresholds may often be defined by the baseline coverage and OAR sparing goals of the trial. Recommendations can also take the form of action levels based on assessments made at predetermined time points or intervals (e.g. based on a single interval FDG-PET/CT as assessed in the RTOG 1106 trial, or on more frequent intervals). Regardless of the details of a particular ART workflow, the timing and frequency of adaptation should balance objectively the clinical value-added to the patient with considerations the finite resources of the clinic. A template for clinical trial language supporting an online ART workflow has been provided in Appendix B including considerations for IGRT, daily adaptation, and ART-related QA that would be added to standard treatment planning and credentialing protocols for new clinical trials.

8. Conclusions

Overall, while resource intensive, ART shows incredible promise for offering gains in OAR sparing and improving target coverage. As vendor offerings increase and our ability to perform workflows within standard clinical operation become easier, the likelihood of implementing ART more routinely—when clinically indicated—is rapidly expanding.

Supplementary Material

Appendix

Acknowledgments

Funding: Research reported in this publication was supported by National Cancer Instituteof the National Institutes of Health under Award Numbers R01CA204189, U24CA180803, and U10CA180868.

Footnotes

Conflict of Interest statement:

Dr. Benedict, Rong, Sohn, and Xiao have nothing to disclose. Dr. Cao reports personal fees from ViewRay Inc. and Varian Inc., outside the submitted work. Mr. Doemer reports honoraria and lecturing for ViewRay, Inc. Dr. Erickson reports Departmental Research Support from Elekta, outside the submitted work. Dr. Glide-Hurst reports Travel and honorarium for speaking engagements from ViewRay, Inc., Advisory Board, Master Research Agreement from Philips Healthcare, grants from NIH, National Cancer Institute of the National Institutes of Health under Award Number R01CA204189, Research agreement from Modus Medical, outside the submitted work. Dr. Kishan reports Honoraria, consulting from Varian Medical Systems, Grant, honoraria from ViewRay, Inc., Advisory board from Janssen, Consulting fees from Intelligent Automation, outside the submitted work. Dr. Lee reports personal fees and non-financial support from ViewRay Inc., grants, personal fees and non-financial support from AstraZeneca, Inc., personal fees and non-financial support from Varian, Inc., outside the submitted work. Dr. Li reports research grants to institution from Elekta AB, Manteia Med, and Siemens Med, outside the submitted work; In addition, Dr. Li has a patent/license on ART techniques. Dr. Olsen reports in the past (>48 months ago) I have received industry research funding in support of MR guided radiation therapy, and travel/speaking fees to give presentations on MR guided radiation therapy. Dr. Parker reports employee from McGill University Health Centre, outside the submitted work. Dr. Siddiqui reports Honorarium for talks and lectures from Varian Medical Systems, Inc., Honorarium for site surveys from American College of Radiology, during the conduct of the study. Dr. Wuthrick reports personal fees from Varian and Sanofi/Regeneron, outside the submitted work. Dr. Yock reports Vendor-supported research grant regarding implementing online adaptive radiotherapy; received compensation for one-time consultation regarding adaptive radiotherapy from Varian Medical Systems, outside the submitted work; In addition, Dr. Yock has a provisional application planned for a phantom designed for adaptive radiotherapy pending.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Yan D, Vicini F, Wong J, Martinez A. Adaptive radiation therapy. Physics in Medicine & Biology. 1997;42(1):123. [DOI] [PubMed] [Google Scholar]
  • 2.Vargas C, Yan D, Kestin LL, et al. Phase II dose escalation study of image-guided adaptive radiotherapy for prostate cancer: use of dose-volume constraints to achieve rectal isotoxicity. Int J Radiat Oncol Biol Phys. 2005;63(1):141–149. [DOI] [PubMed] [Google Scholar]
  • 3.Liu M, Wang Z, Zhou T, et al. Individual isotoxic radiation dose escalation based on V20 and advanced technologies benefits unresectable stage III non-small cell lung cancer patients treated with concurrent chemoradiotherapy: long term follow-up. Oncotarget. 2017;8(31):51848–51858. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Dawson LA, Eccles C, Craig T. Individualized image guided iso-NTCP based liver cancer SBRT. Acta Oncol. 2006;45(7):856–864. [DOI] [PubMed] [Google Scholar]
  • 5.Yan D. Adaptive Radiotherapy. Seminars in Radiation Oncology. 2010;20:79–146. [DOI] [PubMed] [Google Scholar]
  • 6.Vargas C, Martinez A, Kestin LL, et al. Dose-volume analysis of predictors for chronic rectal toxicity after treatment of prostate cancer with adaptive image-guided radiotherapy. Int J Radiat Oncol Biol Phys. 2005;62(5):1297–1308. [DOI] [PubMed] [Google Scholar]
  • 7.Spoelstra FO, Pantarotto JR, van Sornsen de Koste JR, Slotman BJ, Senan S. Role of adaptive radiotherapy during concomitant chemoradiotherapy for lung cancer: analysis of data from a prospective clinical trial. Int J Radiat Oncol Biol Phys. 2009;75(4):1092–1097. [DOI] [PubMed] [Google Scholar]
  • 8.Schwartz DL, Garden AS, Thomas J, et al. Adaptive radiotherapy for head-and-neck cancer: initial clinical outcomes from a prospective trial. Int J Radiat Oncol Biol Phys. 2012;83(3):986–993. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Li X, Quan EM, Li Y, et al. A fully automated method for CT-on-rails-guided online adaptive planning for prostate cancer intensity modulated radiation therapy. International Journal of Radiation Oncology* Biology* Physics. 2013;86(5):835–841. [DOI] [PubMed] [Google Scholar]
  • 10.Ahunbay EE, Peng C, Godley A, Schultz C, Li XA. An on-line replanning method for head and neck adaptive radiotherapy. Med Phys. 2009;36(10):4776–4790. [DOI] [PubMed] [Google Scholar]
  • 11.Henke L, Kashani R, Robinson C, et al. Phase I trial of stereotactic MR-guided online adaptive radiation therapy (SMART) for the treatment of oligometastatic or unresectable primary malignancies of the abdomen. Radiother Oncol. 2018;126(3):519–526. [DOI] [PubMed] [Google Scholar]
  • 12.El-Bared N, Portelance L, Spieler BO, et al. Dosimetric Benefits and Practical Pitfalls of Daily Online Adaptive MRI-Guided Stereotactic Radiation Therapy for Pancreatic Cancer. Practical radiation oncology. 2019;9(1):e46–e54. [DOI] [PubMed] [Google Scholar]
  • 13.Li XA, Liu F, Tai A, et al. Development of an online adaptive solution to account for inter-and intra-fractional variations. Radiotherapy and Oncology. 2011;100(3):370–374. [DOI] [PubMed] [Google Scholar]
  • 14.Liu F, Erickson B, Peng C, Li XA. Characterization and management of interfractional anatomic changes for pancreatic cancer radiotherapy. International Journal of Radiation Oncology* Biology* Physics. 2012;83(3):e423–e429. [DOI] [PubMed] [Google Scholar]
  • 15.Court LE, Dong L, Lee AK, et al. An automatic CT-guided adaptive radiation therapy technique by online modification of multileaf collimator leaf positions for prostate cancer. International Journal of Radiation Oncology* Biology* Physics. 2005;62(1):154–163. [DOI] [PubMed] [Google Scholar]
  • 16.Ahunbay EE, Peng C, Holmes S, Godley A, Lawton C, Li XA. Online adaptive replanning method for prostate radiotherapy. International Journal of Radiation Oncology* Biology* Physics. 2010;77(5):1561–1572. [DOI] [PubMed] [Google Scholar]
  • 17.Mohan R, Zhang X, Wang H, et al. Use of deformed intensity distributions for on-line modification of image-guided IMRT to account for interfractional anatomic changes. International Journal of Radiation Oncology* Biology* Physics. 2005;61(4):1258–1266. Adaptive radiation therapy review [DOI] [PubMed] [Google Scholar]
  • 18.Heijkoop ST, Langerak TR, Quint S, et al. Clinical implementation of an online adaptive plan-of-the-day protocol for nonrigid motion management in locally advanced cervical cancer IMRT. International Journal of Radiation Oncology* Biology* Physics. 2014;90(3):673–679. [DOI] [PubMed] [Google Scholar]
  • 19.Henke LE, Olsen JR, Contreras JA, et al. Stereotactic MR-guided online adaptive radiation therapy (SMART) for ultracentral thorax malignancies: Results of a phase 1 trial. Advances in Radiation Oncology. 2019;4(1):201–209. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Bruynzeel AM, Tetar SU, Oei SS, et al. A prospective single-arm phase 2 study of stereotactic magnetic resonance guided adaptive radiation therapy for prostate cancer: early toxicity results. International Journal of Radiation Oncology* Biology* Physics. 2019;105(5):1086–1094. [DOI] [PubMed] [Google Scholar]
  • 21.NCT03621644 Stereotactic MRI-guided On-table Adaptive Radiation Therapy (SMART) for Locally Advanced Pancreatic Cancer. https://clinicaltrials.gov/ct2/show/NCT03621644. Published 2018. Accessed October, 2020.
  • 22.Hiraoka M, Matsuo Y, Sawada A, et al. Realization of dynamic tumor tracking irradiation with real-time monitoring in lung tumor patients using a gimbaled x-ray head radiation therapy equipment. International Journal of Radiation Oncology• Biology• Physics. 2012;84(3):S560–S561. [Google Scholar]
  • 23.Pathmanathan AU, van As NJ, Kerkmeijer LGW, et al. Magnetic Resonance Imaging-Guided Adaptive Radiation Therapy: A “Game Changer” for Prostate Treatment? Int J Radiat Oncol Biol Phys. 2018;100(2):361–373. [DOI] [PubMed] [Google Scholar]
  • 24.Kontaxis C, Bol G, Stemkens B, et al. Towards fast online intrafraction replanning for free-870 breathing stereotactic body radiation therapy with the MR-linac. Physics in Medicine & Biology. 2017;62(18):7233. [DOI] [PubMed] [Google Scholar]
  • 25.Klawikowski S, Tai A, Ates O, Ahunbay E, Li XA. A fast 4D IMRT/VMAT planning method based on segment aperture morphing. Medical physics. 2018;45(4):1594–1602. [DOI] [PubMed] [Google Scholar]
  • 26.Hara W, Soltys SG, Gibbs IC. CyberKnife® Robotic Radiosurgery system for tumor treatment. Expert review of anticancer therapy. 2007;7(11):1507–1515. [DOI] [PubMed] [Google Scholar]
  • 27.Schweikard A, Shiomi H, Adler J. Respiration tracking in radiosurgery. Medical physics. 2004;31(10):2738–2741. [DOI] [PubMed] [Google Scholar]
  • 28.Ferris WS, Kissick MW, Bayouth JE, Culberson WS, Smilowitz JB. Evaluation of radixact motion synchrony for 3D respiratory motion: Modeling accuracy and dosimetric fidelity. Journal of Applied Clinical Medical Physics. 2020;21(9):96–106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Keall PJ, Nguyen DT, O’Brien R, et al. The first clinical implementation of real-time image-guided adaptive radiotherapy using a standard linear accelerator. Radiotherapy and Oncology. 2018;127(1):6–11. [DOI] [PubMed] [Google Scholar]
  • 30.Matuszak MM, Kashani R, Green M, Owen D, Jolly S, Mierzwa M. Functional adaptation in radiation therapy. Seminars in radiation oncology. 2019;29(3):236–244. [DOI] [PubMed] [Google Scholar]
  • 31.Mao W, Liu C, Gardner SJ, et al. Evaluation and Clinical Application of a Commercially Available Iterative Reconstruction Algorithm for CBCT-Based IGRT. Technology in Cancer Research & Treatment. 2019;18:1533033818823054. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Gardner SJ, Mao W, Liu C, et al. Improvements in CBCT image quality using a novel iterative reconstruction algorithm: a clinical evaluation. Advances in Radiation Oncology. 2019;4(2):390–400. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Archambault A, Boylan C, Bullock D, et al. Making On-Line Adaptive Radiotherapy Possible Using Artificial Intelligence and Machine Learning For Efficient Daily Re-Planning. Medical Physics International Journal. 2020;8(2):77–86. [Google Scholar]
  • 34.Brock KK, Mutic S, McNutt TR, Li H, Kessler ML. Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132. Medical physics. 2017;44(7):e43–e76. [DOI] [PubMed] [Google Scholar]
  • 35.Lim-Reinders S, Keller BM, Al-Ward S, Sahgal A, Kim A. Online Adaptive Radiation Therapy. Int J Radiat Oncol Biol Phys. 2017;99(4):994–1003. [DOI] [PubMed] [Google Scholar]
  • 36.Maintz JB, Viergever MA. A survey of medical image registration. Med Image Anal. 1998;2(1):1–36. [DOI] [PubMed] [Google Scholar]
  • 37.Viergever MA, Maintz JBA, Klein S, Murphy K, Staring M, Pluim JPW. A survey of medical image registration - under review. Medical Image Analysis. 2016;33:140–144. [DOI] [PubMed] [Google Scholar]
  • 38.Jaffray DA, Lindsay PE, Brock KK, Deasy JO, Tomé WA. Accurate Accumulation of Dose for Improved Understanding of Radiation Effects in Normal Tissue. International Journal of Radiation Oncology*Biology*Physics. 2010;76(3, Supplement):S135–S139. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Hardcastle N, van Elmpt W, De Ruysscher D, Bzdusek K, Tome WA. Accuracy of deformable image registration for contour propagation in adaptive lung radiotherapy. Radiat Oncol. 2013;8:1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Hardcastle N, Tome WA, Cannon DM, et al. A multi-institution evaluation of deformable image registration algorithms for automatic organ delineation in adaptive head and neck radiotherapy. Radiat Oncol. 2012;90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Klein A, Andersson J, Ardekani BA, et al. Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration. Neuroimage. 2009;46(3):786–802. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Pukala J, Meeks SL, Staton RJ, Bova FJ, Manon RR, Langen KM. A virtual phantom library for the quantification of deformable image registration uncertainties in patients with cancers of the head and neck. Med Phys. 2013;40(11):111703. [DOI] [PubMed] [Google Scholar]
  • 43.Zhong HL, Kim J, Chetty IJ. Analysis of deformable image registration accuracy using computational modeling. Medical Physics. 2010;37(3):970–979. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Vandemeulebroucke J, Rit S, Kybic J, Clarysse P, Sarrut D. Spatiotemporal motion estimation for respiratory-correlated imaging of the lungs. Medical Physics. 2011;38(1):166–178. [DOI] [PubMed] [Google Scholar]
  • 45.Castillo R, Castillo E, Fuentes D, et al. A reference dataset for deformable image registration spatial accuracy evaluation using the COPDgene study archive. Phys Med Biol. 2013;58(9):2861–2877. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Murphy K, van Ginneken B, Reinhardt JM, et al. Evaluation of Registration Methods on Thoracic CT: The EMPIRE10 Challenge. Ieee T Med Imaging. 2011;30(11):1901–1920. [DOI] [PubMed] [Google Scholar]
  • 47.Hansen EK, Bucci MK, Quivey JM, Weinberg V, Xia P. Repeat CT imaging and replanning during the course of IMRT for head-and-neck cancer. International Journal of Radiation Oncology* Biology* Physics. 2006;64(2):355–362. [DOI] [PubMed] [Google Scholar]
  • 48.Luterstein E, Cao M, Lamb J, et al. Stereotactic MRI-guided adaptive radiation therapy (SMART) for locally advanced pancreatic cancer: a promising approach. Cureus. 2018;10(3). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Yan D, Jaffray DA, Wong JW. A model to accumulate fractionated dose in a deforming organ. Int J Radiat Oncol Biol Phys. 1999;44(3):665–675. [DOI] [PubMed] [Google Scholar]
  • 50.Veiga C, McClelland J, Moinuddin S, et al. Toward adaptive radiotherapy for head and neck patients: Feasibility study on using CT-to-CBCT deformable registration for “dose of the day” calculations. Medical Physics. 2014;41(3):1–12. [DOI] [PubMed] [Google Scholar]
  • 51.Varadhan R, Karangelis G, Krishnan K, Hui S. A framework for deformable image registration validation in radiotherapy clinical applications. J Appl Clin Med Phys. 2013;14(1):192–213. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Zhang L, Wang Z, Shi C, Long T, Xu XG. The impact of robustness of deformable image registration on contour propagation and dose accumulation for head and neck adaptive radiotherapy. J Appl Clin Med Phys. 2018;19(4):185–194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Nobnop W, Chitapanarux I, Neamin H, Wanwilairat S, Lorvidhaya V, Sanghangthum T. Evaluation of Deformable Image Registration (DIR) Methods for Dose Accumulation in Nasopharyngeal Cancer Patients during Radiotherapy. Radiol Oncol. 2017;51(4):438–446. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Velec M, Moseley JL, Eccles CL, et al. Effect of Breathing Motion on Radiotherapy Dose Accumulation in the Abdomen Using Deformable Registration. Int J Radiat Oncol. 2011;80(1):265–272. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Veiga C, Lourenco AM, Mouinuddin S, et al. Toward adaptive radiotherapy for head and neck patients: Uncertainties in dose warping due to the choice of deformable registration algorithm. Medical Physics. 2015;42(2):760–769. [DOI] [PubMed] [Google Scholar]
  • 56.Hugo GD, Cao K, Guy C, Weiss E, Jan N. and Christensen GE Measurement of local deformation due to lung tumor response to radiation therapy. In The Fifth International Workshop on Pulmonary Image Analysis. 2013:97–108. [Google Scholar]
  • 57.Kong F, Machtay M, Bradley J, et al. RTOG 1106/ACRIN 6697: Randomized phase II trial of individualized adaptive radiotherapy using during treatment FDG-PET. CT and modern technology in locally advanced non-small lung cancer (NSCLC). 2012. [Google Scholar]
  • 58.Bahig H, Yuan Y, Mohamed ASR, et al. Magnetic Resonance-based Response Assessment and Dose Adaptation in Human Papilloma Virus Positive Tumors of the Oropharynx treated with Radiotherapy (MR-ADAPTOR): An R-IDEAL stage 2a-2b/Bayesian phase II trial. Clin Transl Radiat Oncol. 2018;13:19–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Nijkamp J, Pos FJ, Nuver TT, et al. Adaptive radiotherapy for prostate cancer using kilovoltage cone-beam computed tomography: first clinical results. Int J Radiat Oncol Biol Phys. 2008;70(1):75–82. [DOI] [PubMed] [Google Scholar]
  • 60.Kainz SL K, G-P Chen,Li XA PreciseART Adaptive Radiation Therapy Software: Dose Monitoring, Re-Planning, and Delivery Verification. Accuray White Paper. 2017. [Google Scholar]
  • 61.Olberg S, Green O, Cai B, et al. Optimization of treatment planning workflow and tumor coverage during daily adaptive magnetic resonance image guided radiation therapy (MR-IGRT) of pancreatic cancer. Radiat Oncol. 2018;13:1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Bohoudi O, Bruynzeel AME, Senan S, et al. Fast and robust online adaptive planning in stereotactic MR-guided adaptive radiation therapy (SMART) for pancreatic cancer. Radiother Oncol. 2017;125(3):439–444. [DOI] [PubMed] [Google Scholar]
  • 63.van Dijk LV, Van den Bosch L, Aljabar P, et al. Improving automatic delineation for head and neck organs at risk by Deep Learning Contouring. Radiotherapy and Oncology. 2020;142:115–123. [DOI] [PubMed] [Google Scholar]
  • 64.Boldrini L, Bibault J-E, Masciocchi C, Shen Y, Bittner M-I. Deep learning: a review for the radiation oncologist. Frontiers in Oncology. 2019;9:977. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Winkel D, Bol GH, Kiekebosch IH, et al. Evaluation of online plan adaptation strategies for the 1.5 T MR-linac based on “First-In-Man” treatments. Cureus. 2018;10(4). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Tetar SU, Bruynzeel AM, Lagerwaard FJ, Slotman BJ, Bohoudi O, Palacios MA. Clinical implementation of magnetic resonance imaging guided adaptive radiotherapy for localized prostate cancer. Physics and Imaging in Radiation Oncology. 2019;9:69–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Werensteijn-Honingh AM, Kroon PS, Winkel D, et al. Feasibility of stereotactic radiotherapy using a 1.5 T MR-linac: multi-fraction treatment of pelvic lymph node oligometastases. Radiotherapy and Oncology. 2019;134:50–54. [DOI] [PubMed] [Google Scholar]
  • 68.Roach M, Glide-Hurst CK. MRI at the Time of External Beam Treatment. In: MRI for Radiotherapy. Springer, Cham; 2019:169–188. [Google Scholar]
  • 69.Paulson ES, Ahunbay E, Chen X, et al. 4D-MRI driven MR-guided online adaptive radiotherapy for abdominal stereotactic body radiation therapy on a high field MR-Linac: Implementation and initial clinical experience. Clinical and Translational Radiation Oncology. 2020(23):72–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Cuccia F, Mazzola R, Nicosia L, et al. Impact of hydrogel peri-rectal spacer insertion on prostate gland intra-fraction motion during 1.5 T MR-guided stereotactic body radiotherapy. Radiat Oncol. 2020;15(1):1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Acharya S, Fischer-Valuck BW, Kashani R, et al. Online Magnetic Resonance Image Guided Adaptive Radiation Therapy: First Clinical Applications. Int J Radiat Oncol Biol Phys. 2016;94(2):394–403. [DOI] [PubMed] [Google Scholar]
  • 72.Chen GP, Ahunbay E, Li XA. Technical Note: Development and performance of a software tool for quality assurance of online replanning with a conventional Linac or MR-Linac. Med Phys. 2016;43(4):1713. [DOI] [PubMed] [Google Scholar]
  • 73.Leath C, Mell L, Mackay H. NRG-GY006: A Randomized Phase II Trial of Radiation Therapy and Cisplatin Alone or in Combination with Intravenous Triapine in Women with Newly Diagnosed Bulky Stage IB2, Stage II, IIIB, or IVA Cancer of the Uterine Cervix or Stage II-IVA Vaginal Cancer. In:2018. [Google Scholar]
  • 74.Prescribing, Recording, and Reporting Brachytherapy for Cancer of the Cervix. Journal of the International Commission on Radiation Units and Measurements. 2013;13(1–2). [DOI] [PubMed] [Google Scholar]
  • 75.Liao Y, Wang L, Xu X, et al. An anthropomorphic abdominal phantom for deformable image registration accuracy validation in adaptive radiation therapy. Med Phys. 2017;44(6):2369–2378. [DOI] [PubMed] [Google Scholar]
  • 76.Kirby N, Chuang C, Pouliot J. A two-dimensional deformable phantom for quantitatively verifying deformation algorithms. Med Phys. 2011;38(8):4583–4586. [DOI] [PubMed] [Google Scholar]
  • 77.Graves YJ, Smith AA, McIlvena D, et al. A deformable head and neck phantom with in-vivo dosimetry for adaptive radiotherapy quality assurance. Med Phys. 2015;42(4):1490–1497. [DOI] [PubMed] [Google Scholar]
  • 78.Stanley N, Glide-Hurst C, Kim J, et al. Using patient-specific phantoms to evaluate deformable image registration algorithms for adaptive radiation therapy. Journal of applied clinical medical physics. 2013;14(6):177–194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Kim H, Park SB, Monroe JI, et al. Quantitative Analysis Tools and Digital Phantoms for Deformable Image Registration Quality Assurance. Technology in Cancer Research & Treatment. 2015;14(4):428–439. [DOI] [PubMed] [Google Scholar]
  • 80.Brock KK, Mutic S, McNutt TR, Li H, Kessler ML. Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task. Medical Physics. 2017;44(7):E43–E76. [DOI] [PubMed] [Google Scholar]
  • 81.Price A, Kim H, Henke LE, et al. Implementing a Novel Remote Physician Treatment Coverage Practice for Adaptive Radiotherapy during the Coronavirus Pandemic. Advances in Radiation Oncology. 2020(5):737–742. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82.Hales RB, Rodgers J, Whiteside L, et al. Therapeutic Radiographers at the Helm: Moving Towards Radiographer-Led MR-Guided Radiotherapy. Journal of Medical Imaging and Radiation Sciences. 2020;51(3):364–372. [DOI] [PubMed] [Google Scholar]
  • 83.Zhong H, Chetty IJ. Caution must be exercised when performing deformable dose accumulation for tumors undergoing mass changes during fractionated radiation therapy. International Journal of Radiation Oncology• Biology• Physics. 2017;97(1):182–183. [DOI] [PubMed] [Google Scholar]
  • 84.Li HS, Zhong H, Kim J, et al. Direct dose mapping versus energy/mass transfer mapping for 4D dose accumulation: fundamental differences and dosimetric consequences. Physics in Medicine & Biology. 2013;59(1):173. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 85.Li S, Glide-Hurst C, Lu M, et al. Voxel-based statistical analysis of uncertainties associated with deformable image registration. Physics in Medicine & Biology. 2013;58(18):6481–6494. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 86.Takemura A, Nagano A, Kojima H, et al. An uncertainty metric to evaluate deformation vector fields for dose accumulation in radiotherapy. Physics and imaging in radiation oncology. 2018;6:77–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 87.Bohoudi O, Lagerwaard FJ, Bruynzeel AM, et al. End-to-end empirical validation of dose accumulation in MRI-guided adaptive radiotherapy for prostate cancer using an anthropomorphic deformable pelvis phantom. Radiotherapy and Oncology. 2019:200–207. [DOI] [PubMed] [Google Scholar]
  • 88.Cunningham JM, Barberi EA, Miller J, Kim JP, Glide-Hurst CK. Development and evaluation of a novel MR-compatible pelvic end-to-end phantom. Journal of Applied Clinical Medical Physics. 2019;20(1):265–275. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 89.Klein EE, Hanley J, Bayouth J, et al. Task Group 142 report: Quality assurance of medical accelerators a. Medical physics. 2009;36(9Part1):4197–4212. [DOI] [PubMed] [Google Scholar]
  • 90.Niebuhr NI, Johnen W, Echner G, et al. The ADAM-pelvis phantom-an anthropomorphic, deformable and multimodal phantom for MRgRT. Phys Med Biol. 2019:13. [DOI] [PubMed] [Google Scholar]
  • 91.Zhong H, Adams J, Glide-Hurst C, Zhang H, Li H, Chetty IJ. Development of a deformable dosimetric phantom to verify dose accumulation algorithms for adaptive radiotherapy. Journal of Medical Physics/Association of Medical Physicists of India. 2016;41(2):106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 92.Ehrbar S, Jöhl A, Kühni M, et al. ELPHA: Dynamically deformable liver phantom for real-time motion-adaptive radiotherapy treatments. Medical physics. 2019;46(2):839–850. [DOI] [PubMed] [Google Scholar]
  • 93.Sawant A, Dieterich S, Svatos M, Keall P. Failure mode and effect analysis-based quality assurance for dynamic MLC tracking systems. Medical physics. 2010;37(12):6466–6479. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 94.Noel CE, Santanam L, Parikh PJ, Mutic S. Process-based quality management for clinical implementation of adaptive radiotherapy. Medical physics. 2014;41(8Part1):081717. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 95.Green OL, Henke LE, Hugo GD. Practical clinical workflows for online and offline adaptive radiation therapy. Seminars in radiation oncology. 2019;29(3):219–227. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 96.Torres-Xirau I, Olaciregui-Ruiz I, Baldvinsson G, Mijnheer BJ, van der Heide UA, Mans A. Characterization of the a-Si EPID in the unity MR-linac for dosimetric applications. Phys Med Biol. 2018;63(2):025006. [DOI] [PubMed] [Google Scholar]
  • 97.Lim SN, Ahunbay EE, Nasief H, Zheng C, Lawton C, Li XA. Indications of Online Adaptive Replanning Based On Organ Deformation. Practical radiation oncology. 2020;10(2):e95–e102. [DOI] [PubMed] [Google Scholar]
  • 98.Kron T, Pham D, Roxby P, Rolfo A, Foroudi F. Credentialing of radiotherapy centres for a clinical trial of adaptive radiotherapy for bladder cancer (TROG 10.01). Radiotherapy and Oncology. 2012;103(3):293–298. [DOI] [PubMed] [Google Scholar]
  • 99.Tsang Y, Baker A, Patel E, Miles E. A new era for clinical trial quality assurance: a credentialing programme for RTT led adaptive radiotherapy. Technical Innovations & Patient Support in Radiation Oncology. 2018;5:1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 100.Winkel D, Bol GH, Kroon PS, et al. Adaptive radiotherapy: The Elekta Unity MR-linac concept. Clinical and Translational Radiation Oncology. 2019:54–59. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 101.Wu Q, Chi Y, Chen PY, Krauss DJ, Yan D, Martinez A. Adaptive replanning strategies accounting for shrinkage in head and neck IMRT. International Journal of Radiation Oncology* Biology* Physics. 2009;75(3):924–932. [DOI] [PubMed] [Google Scholar]
  • 102.Hoffmann L, Alber M, Jensen MF, Holt MI, Moller DS. Adaptation is mandatory for intensity modulated proton therapy of advanced lung cancer to ensure target coverage. Radiother Oncol. 2017;122(3):400–405. [DOI] [PubMed] [Google Scholar]
  • 103.Park YK, Sharp GC, Phillips J, Winey BA. Proton dose calculation on scatter-corrected CBCT image: Feasibility study for adaptive proton therapy. Med Phys. 2015;42(8):4449–4459. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 104.Landry G, Nijhuis R, Dedes G, et al. Investigating CT to CBCT image registration for head and neck proton therapy as a tool for daily dose recalculation. Medical physics. 2015;42(3):1354–1366. [DOI] [PubMed] [Google Scholar]
  • 105.Sun B, Yang D, Lam D, et al. Toward adaptive proton therapy guided with a mobile helical CT scanner. Radiother Oncol. 2018:479–485. [DOI] [PubMed] [Google Scholar]
  • 106.Mohan R, Das IJ, Ling CC. Empowering Intensity Modulated Proton Therapy Through Physics and Technology: An Overview. Int J Radiat Oncol Biol Phys. 2017;99(2):304–316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 107.Pötter R, Haie-Meder C, Van Limbergen E, et al. Recommendations from gynaecological (GYN) GEC ESTRO working group (II): concepts and terms in 3D image-based treatment planning in cervix cancer brachytherapy—3D dose volume parameters and aspects of 3D image-based anatomy, radiation physics, radiobiology. Radiotherapy and Oncology. 2006;78(1):67–77. [DOI] [PubMed] [Google Scholar]
  • 108.Griffith R, Bergmann H, Fry FA, et al. Report 69. Journal of the International Commission on Radiation Units and Measurements. 2016;3(1). [Google Scholar]
  • 109.Potter R, Tanderup K, Kirisits C, et al. The EMBRACE II study: The outcome and prospect of two decades of evolution within the GEC-ESTRO GYN working group and the EMBRACE studies. Clin Transl Radiat Oncol. 2018;9:48–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 110.Sturdza A, Potter R, Fokdal LU, et al. Image guided brachytherapy in locally advanced cervical cancer: Improved pelvic control and survival in RetroEMBRACE, a multicenter cohort study. Radiother Oncol. 2016;120(3):428–433. [DOI] [PubMed] [Google Scholar]
  • 111.Fokdal L, Sturdza A, Mazeron R, et al. Image guided adaptive brachytherapy with combined intracavitary and interstitial technique improves the therapeutic ratio in locally advanced cervical cancer: Analysis from the retroEMBRACE study. Radiother Oncol. 2016;120(3):434–440. [DOI] [PubMed] [Google Scholar]
  • 112.Henke L, Kashani R, Yang D, et al. Simulated Online Adaptive Magnetic Resonance-Guided Stereotactic Body Radiation Therapy for the Treatment of Oligometastatic Disease of the Abdomen and Central Thorax: Characterization of Potential Advantages. Int J Radiat Oncol Biol Phys. 2016;96(5):1078–1086. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 113.Hunt A, Hansen V, Oelfke U, Nill S, Hafeez S. Adaptive radiotherapy enabled by MRI guidance. Clinical Oncology. 2018;30(11):711–719. [DOI] [PubMed] [Google Scholar]
  • 114.Mazin S, Nanduri A, Inventors; US Patent App. 16/834,956, assignee. SYSTEMS AND METHODS FOR USE IN EMISSION GUIDED RADIATION THERAPY. 2020. [Google Scholar]
  • 115.Crehange G, Soussan M, Gensanne D, Decazes P, Thariat J, Thureau S. Interest of positron-emission tomography and magnetic resonance imaging for radiotherapy planning and control. Cancer/Radiothérapie. 2020:398–402. [DOI] [PubMed] [Google Scholar]
  • 116.Jia X, Ziegenhein P, Jiang SB. GPU-based high-performance computing for radiation therapy. Physics in Medicine & Biology. 2014;59(4):R151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 117.Men C, Jia X, Jiang SB. GPU-based ultra-fast direct aperture optimization for online adaptive radiation therapy. Physics in Medicine & Biology. 2010;55(15):4309. [DOI] [PubMed] [Google Scholar]
  • 118.Han X, Hibbard LS, Willcut V. GPU-accelerated, Gradient-free MI Deformable Registration for Atlas-based MR Brain Image Segmentation. IEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. 2009;1:204–211. [Google Scholar]
  • 119.Fu Y, Mazur TR, Wu X, et al. A novel MRI segmentation method using CNN-based correction network for MRI-guided adaptive radiotherapy. Medical physics. 2018;45(11):5129–5137. [DOI] [PubMed] [Google Scholar]
  • 120.Liang X, Chen L, Nguyen D, et al. Generating synthesized computed tomography (CT) from cone-beam computed tomography (CBCT) using CycleGAN for adaptive radiation therapy. Physics in Medicine & Biology. 2019;64(12):13. [DOI] [PubMed] [Google Scholar]
  • 121.Li Y, Zhu J, Liu Z, et al. A preliminary study of using a deep convolution neural network to generate synthesized CT images based on CBCT for adaptive radiotherapy of nasopharyngeal carcinoma. Physics in Medicine & Biology. 2019;64(14):12. [DOI] [PubMed] [Google Scholar]
  • 122.Jarrett D, Stride E, Vallis K, Gooding MJ. Applications and limitations of machine learning in radiation oncology. The British journal of radiology. 2019;92(1100):20190001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 123.Tyran M, Jiang N, Cao M, et al. Retrospective evaluation of decision-making for pancreatic stereotactic MR-guided adaptive radiotherapy. Radiother Oncol. 2018:319–325. [DOI] [PubMed] [Google Scholar]
  • 124.Ohri N, Shen X, Dicker AP, Doyle LA, Harrison AS, Showalter TN. Radiotherapy protocol deviations and clinical outcomes: a meta-analysis of cooperative group clinical trials. J Natl Cancer Inst. 2013;105(6):387–393. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix

RESOURCES