The Importance of Quality Assurance in Radiation Oncology Clinical Trials

Abstract

Clinical trials have been the center of progress in modern medicine. In oncology, we are fortunate to have a structure in place through the National Clinical Trials Network (NCTN). The NCTN provides the infrastructure and a forum for scientific discussion to develop clinical concepts for trial design. The NCTN also provides network group structure to administer trials for successful trial management and outcome analyses. There are many important aspects to trial design and conduct. Modern trials need to ensure appropriate trial conduct and secure data management processes. Of equal importance is the quality assurance of a clinical trial. If progress is to be made in oncology clinical medicine, investigators and patient care providers of service need to feel secure that trial data is complete, accurate, and well controlled in order to be confident in trial analysis and move trial outcome results into daily practice. As our technology has matured, so has our need to apply technology in a uniform manner for appropriate interpretation of trial outcome. In this article, we review the importance of quality assurance in clinical trials involving radiation therapy. We will include important aspects of institution and investigator credentialing for participation as well as ongoing processes to ensure that each trial is being managed in a compliant manner. We will provide examples of the importance of complete datasets to ensure study interpretation. We will describe how successful strategies for quality assurance in the past will support new initiatives moving forward.

Introduction

Quality has become an important and elemental metric in modern clinical medicine. Senior leadership groups in modern hospital systems participate in daily review of process and performance. Metrics for quality and safety are now imbedded in our daily inpatient and outpatient work scope and reported on a daily basis. Outcomes in oncology are increasingly measured by regulatory agencies and insurance providers for performance analysis. Processes in all aspects of medical care have been adjusted to ensure that quality and safety indicators remain a priority in the modern workplace. While this aspect of workplace behavior has accelerated in the past decade, the National Cancer Institute (NCI)-led trials including the current National Clinical Trial Network (NCTN) have had quality assurance (QA) for more than five decades. Changing clinical trial environments and modernization of therapy technology demand process improvements in the QA centers.^1–5

In the mid 1960’s, early pioneers in oncology practice saw worth in being able to share information about patient management and outcome. Collaborative meetings were held in common venues to have conversations and create protocols that could be shared among institutions to evaluate and test clinical concepts in patient care management. Clinical trials established to test hypotheses of management and outcomes were reviewed periodically at biannual meetings. The process of trial development and outcome analysis became an important vehicle because trial outcome would potentially influence standards of care. As clinical trial development broadened, it was recognized that administrative structure would need to be established to generate clinical trials and simultaneously collect and collate information for study analysis. The NCI established a network for participation in what was called the cooperative groups now the NCTN. The cooperative groups became an extraordinary vehicle to develop management strategies and test the hypotheses in a national and international venue. Prior to 2014, there were 10 cooperative groups participating in all aspect of liquid and epithelial oncology.⁶ Many of the groups had strengths in most adult and pediatric disease disciplines and others had strength in individual disease areas (Gynecologic Oncology Group-GOG) and specific therapy disciplines (Radiation Therapy Oncology Group-RTOG).

The NCI serves multiple purposes including supporting phase 1 activity when required and validating potential new advances in care through a phase 3 randomized approach. The groups were supported by individual group data/statistical centers and tissue banks. In 2014, the cooperative system was consolidated into five groups, one pediatric group and four adult groups. The NCI consolidated all the QA review centers in imaging and radiation oncology into one group called the Imaging and Radiation Oncology Core (IROC). IROC includes the former Radiological Physics Center (RPC, IROC Houston), the Quality Assurance Review Center (QARC, IROC Rhode Island (RI)), The University of Ohio (IROC Ohio), and the core labs for American College of Radiology Imaging Network (ACRIN) and RTOG, now IROC Philadelphia.^1–5 The current schema is listed in Figure 1. Each IROC QA Center has individual strengths, and the consolidation has provided an opportunity to leverage these strengths to support imaging and radiation oncology core service function for the NCTN. This provides an opportunity to harmonize protocol review for each group and align data management and acquisition pathways for translational research. As the pathways for data management become transparent, the data can be reformatted for upload to national archives including The Cancer Imaging Archive (TCIA) for secondary analyses research and development in artificial intelligence (AI).^7,8 If the trials are well designed and the data acquired are in standard formats, the data provide an extraordinary opportunity to not only support and validate primary clinical trial endpoints but also serve to be re-purposed for secondary analyses by site and study investigators. Upload to archives such as TCIA make the data available for analyses by scientists worldwide.

Figure 1.

NCTN Schema. This image is in the public domain and is courtesy of the National Cancer Institute (NCI). https://www.cancer.gov/research/infrastructure/clinical-trials/nctn.

For confidence in the primary and secondary analyses, the data need to be adherent to trial guidelines and as uniform as possible so that the results of the analysis can be applied to the science of patient care. This is why QA of the clinical trial matters. In the next section we will review the history of QA and how processes have matured over the past five decades.

History of QA in Radiation Oncology Clinical Trials

When the first patient was treated in the United States on a linear accelerator by Henry Kaplan in 1956, radiation oncology was introduced as a maturing discipline to medical science. With increasing technology changes generated from post-World War II repurposed radar systems, compact linear accelerators were developed for patient care.^9,10 During the 1960’s, linear accelerator technology had developed to a degree that daily treatment could be delivered in a reliable and consistent manner and academic centers and community hospitals were able to acquire the equipment for patient care. With advances in radiation oncology coupled with the evolving specialty of medical oncology, early oncology investigators were committed to sharing patient treatment strategies and developing protocols for treatment. Commitments were made by investigators to meet twice yearly, to review protocol progress and design strategies for the next iteration of trials. Initially the commitments were informal, however the potential strength of collaboration to move clinical care forward was recognized by the newly formed NCI and a network of cooperative groups was established over the ensuing years to create protocols for clinical trials, collect data, assess outcome, and help standardize and improve clinical care.¹¹

Over time, 10 cooperative groups were established in both pediatric and adult diseases to address disease areas and technology disciplines. Radiation therapy (RT) was considered an important partner in clinical trial development. Strategies for protocol development were finalized by discussions within specific RT committees. Although desired, initially there was no vehicle or mechanism to collate and collect RT treatment data for each trial. However, when a limited number of completion notes were reviewed by committee, it was recognized by committee members there was significant variance within the RT community in treatment plans with respect to both radiation dose and volume of treatment. Because there was limited commercialization of RT planning systems at that time, variances in dose computation and calculation were likewise identified reviewing objects and planning information.¹² A decision was made within committees to acquire information on patient treatment which included dose computation data and RT treatment fields obtained at the time of fluoroscopic imaging during simulation and megavoltage portal images obtained from accelerators each week on treatment. Initially there was resistance to the data submission process, however the groups supported data acquisition and within a year, submission rates improved to 95% from 30% as institutions became more familiar with the data submission process. In the early phase of data acquisition, the materials were reviewed in retrospect often at trial completion. Despite the presence of RT guidelines in the study, deviation rates remained significant. Because the data acquisition process was proving to be successful, a decision was made by committee to require an on-treatment review of the planned treatment to ensure that the goals and objectives of the trial were being met. Cases were to be reviewed by day three of treatment to provide what was then thought as a reasonable time frame to make an adjustment in the care plan to meet protocol guidelines. The RT committee had multiple roles including protocol development, protocol management, and the new role of directing QA for the trial. In 1980, the pediatric divisions of the Cancer and Leukemia Group B (CALGB) and Southwest Oncology Group (SWOG) separated from their respective group to form the Pediatric Oncology Group (POG). At that time, the QA program for data acquisition and management for the CALGB, Intergroup Rhabdomyosarcoma Group (IRSG), National Wilms Tumor Study Group (NWTS), and POG became independently funded and known as QARC. The RTOG established an internal RT QA process. In parallel the RPC established dose computation validation processes for RT facilities participating in NCI trials. The development of these organizations confirmed the importance of QA in the clinical trials process. The QA organizations provided an economy of scale for the cooperative group process as each group could use services by the individual QA centers.^12,13

As processes of clinical care matured, so did the QA process. Fluoroscopic RT planning was only able to provide volumetric data at a rudimentary level. Volumetric based treatment planning with computer tomography (CT)-based systems reshaped the planning process and workflow within radiation oncology. Physicians, physics staff, dosimetrists, and therapists all adapted to volume-based planning. Physician workflow now required skill in contouring tumor targets using CT tools and therapy was planned to a volume, not a point, with metrics applied to constrain dose to normal tissue. In two-dimensional fluoroscopic planning, the majority of work required of the physician was completed at the time of simulation. With volumetric planning, the work of the physician changed, and contours were drawn after digital imaging was processed by physics well after the patient left the department. This would alter patient care as the planning effort could only begin once the contours were complete and impose delays from the time of simulation to the patient initiating therapy. For patients on clinical trials requiring data submission, the process of forwarding digital planning information became a challenge.^14,15

A pivotal moment facilitating RT data transmission for clinical trials was provided with the introduction of RTOG data exchange. This permitted planning information to be forwarded in a digital package to QA centers for protocol review. For targets thought to be well seen on RT three dimensional systems including prostate, the data submission evaluating the image and the proposed plan could be completed with review of the objects. Drs. James Purdy (Washington University) and James Deye (NCI) were instrumental in providing this form of digital data exchange to the cooperative group community and the QA centers were able to align data acquisition process centered on digital exchange of data.¹⁶

As volumetric planning became more standard practice in radiation oncology, computational tools in radiation oncology for treatment planning evolved from an institution-driven practice towards alignment with commercial systems. Therefore, credentialing and benchmark analysis of treatment plans for qualification of activity in clinical trials became standardized as departments adopted more consistent procedures.^17–20 Although metrics for dose computation became more uniform, the role of diagnostic imaging in defining target volumes for RT became the next important step for protocol alignment among radiation oncologists. This became a focal point for radiation oncology clinical trials managed through QARC.^21,22 Imaging was needed to validate the RT target volume and for assessing response to therapy and treatment outcome. As imaging became Digital Imaging and Communications in Medicine (DICOM) compliant, digital transfer tools matured, and imaging was acquired through similar mechanisms previously established for radiation oncology. As institutions became more familiar with these processes, the volume of imaging acquired for clinical trial management began to include imaging associated with chemotherapy including response assessment imaging. Because QA centers were collecting imaging and RT treatment objects on the same patient for response assessment and review of RT treatment plans, a unique relationship matured between imaging and radiation oncology committees of each group. Since response assessment imaging was needed to design RT treatment objects, diagnostic imaging and radiation oncology committee members developed a working relationship with increasing respect for each other’s discipline as patient care became more interdependent. As each discipline gained more familiarity with each other’s work and expertise, radiologists were able to identify disease progression more readily from therapy effect. As the digital world expanded, objects could be reviewed in real time on a simultaneous basis by site and study investigators using web-based tools. This moved QA processes and clinical trials to a new level. Clinical trials were written to accommodate adaptive endpoints with RT treatment guidelines written for adaptive clinical and radiographic response. Therefore, modern QA processes include site credentialing for participation in clinical trials, data acquisition and management, real time review of protocol data including imaging and RT objects, data analysis, and research including secondary clinical trial objectives not necessarily anticipated at the time of trial development or execution.²³ The QA centers, now known as IROC, house information from nearly 1,000 NCI-sponsored and associated clinical trials with millions of images and thousands of RT treatment objects all managed in a uniform format consistent with clinical trial processes. With bridges built into clinical outcome data and tissue banks, the information becomes a powerful vehicle for additional research including program development for processes in AI. Figure 2 represents imaging and RT objects archived side by side in the integrated database at IROC RI.

Figure 2.

Protocol patient digital file. Image courtesy of IROC RI.

With the infrastructure in place and processes aligned for enterprise function in data management, we need to examine why the processes came to be of importance to the clinical trials community.

Why QA Matters

Although it is acknowledged that clinical trials remain a cornerstone in the improvement of clinical care for oncology patients, the processes involved in data management and clinical trial investigation for site investigators can be challenging for even seasoned investigators. Internal review board (IRB) approvals, consent, data management and patient care before, during and after the time of study can be demanding and may limit perceived clinical productivity when measured in relative value units (RVU). At times the data management and submission processes can be a perceived barrier to protocol participation. This can influence how protocols are written and what data are collected, managed, and archived by QA centers. If data submission is perceived as a barrier to participation, often the compromise is to limit the data acquisition process in order to encourage accrual to study. This can result in a decrease in submitted imaging, elimination of the real time review process, and no oversight into trial management. This has both strength and weakness. The perceived immediate strength is the goal of meeting study accrual objectives. However, for every perceived strength there is a consequence and often limiting data acquisition can potentially decrease the confidence in the outcome of the clinical trial. The experience at QARC, now IROC RI, in Hodgkin lymphoma (HL) typifies how processes in clinical trials have matured with the commensurate need for optimal clinical trial management.²⁴

POG clinical trial 8725 would today be classified as a study of patients with intermediate and early high-risk HL. The trial design was a point of randomization to determine if RT after eight cycles of hybrid chemotherapy including MOPP and ABVD was of benefit. RT was designed to treat all initial sites of disease after completion of chemotherapy with response adapted blocking approved for the mediastinum to limit radiation dose to pulmonary parenchyma. CT images of disease at presentation and at completion of chemotherapy were collected in order to confirm the volume of therapy to be treated. Response, not the rapidity of response, was not intentionally reviewed for protocol management. In this era, digital information was not available, and all RT information was performed by study investigators as a retrospective process after the patient completed therapy. There was no real time review of information by study investigators, or the QA centers. In the original publication,²⁵ there appeared to be no benefit to the addition of RT after chemotherapy for this patient population.

A secondary analysis of the RT information on this study was performed to determine why the deviation rate was in excess of 30% and what could be done to improve process for the next iteration of Hodgkin studies. What was uncovered was the following: the deviation rate was driven exclusively by radiation investigators titrating the volume of therapy and excluding segments of original disease at presentation from the RT fields due to the perception that titrating volumes limited the late sequelae. Therefore, in this study, optimal outcome was achieved when all sites of original disease were included in the RT treatment field and RT added no statistical benefit when original disease was excluded in the therapy field.^1–5 The findings were important at several levels. It was clear to POG leadership that investigator-driven application of RT in HL was variable and had the likely potential of influencing trial outcome. It was likewise clear that RT object review done in a retrospective manner had importance however the process was going to have to be adjusted if RT was going to have a meaningful impact in clinical trials involving HL. After consideration by the RT committee, a decision was made to move forward with pre-treatment approval of RT treatment objects in the next generation of studies.

COG Studies 9425 and 9426 were clinical trials designed to address trials in intermediate and low risk HL. Each study incorporated RT for consolidation management with the intent to include all soft tissue and bone sites of disease at presentation with response-adapted titration of mediastinal adenopathy where disease extended into pulmonary parenchyma. COG 9426, the low-risk study, also included response-adaptive titration of chemotherapy following review of imaging after two cycles of chemotherapy. The trial was conducted in the pre-digital era; however, consensus was the review of RT objects prior to the treatment was feasible because the RT would be executed after chemotherapy, therefore offering time for the data to be submitted and reviewed. The response imaging, however, was performed in retrospect. The pre-treatment review process proved to be a success with near 100% data submission pre-therapy and significant improvement in study compliance. However, what was unanticipated was the 50% discordance in the assessment of therapy response between local and central review. This prompted an additional review of process, and a decision was made in concert with the imaging committee to review imaging objects in real time to ensure the studies were conducted in a manner consistent with trial objectives. The QA centers were developing digital acquisition; therefore, it was an uncomplicated process to include digital imaging review with the review of RT treatment objects. Intermediate risk COG AHOD0031 became the first study to integrate real time review of imaging and RT objects into the protocol schema (Fig. 3). The study was designed to identify patients with rapid and slow early responses to chemotherapy and separate these populations into different treatment strata as a secondary point of randomization. There was a third randomization in patients who were deemed both a rapid early responder to chemotherapy who in addition had a complete response after four cycles of chemotherapy. In order to conduct the trial with real time review of multiple image sets including RT treatment objects, QA data management and information services (IS) teams developed digital data transfer system with reminders to sites about what data was due and when. The study was complex and included both anatomic and metabolic imaging for response assessment and digital RT objects for pre-plan review. Nearly 1800 patients were accrued to study with data sets on each patient reviewed on multiple occasions for each patient to ensure that the data collected was study compliant including outcome images. The process was highly successful and led to several important publications on study process and outcome. In submitting a paper evaluating the pattern of failure of patients relative to the RT treatment fields, the reviewers each commented on the importance of the QA process and indicated that the process provided trust to the reviewers that the study outcome could be believed. Today, real time review of objects has become standard in the pediatric and adult clinical trials community. This has led to successful studies in very limited stage HL and studies evaluating volume titration with RT after more comprehensive chemotherapy for advanced stage patients.^24,26

Figure 3.

Real-time review of imaging and RT in COG AHOD0031.³ © 2013 FitzGerald, Bishop-Jodoin, Bosch, Curran, Followill, Galvin, Hanusik, King, Knopp, Laurie, O’Meara, Michalski, Saltz, Schnall, Schwartz, Ulin, Xiao and Urie. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.

The current status of clinical trials for HL represents the growth and development of QA processes that have matured in parallel with technology development in radiation oncology and diagnostic imaging. Radiation oncology has extraordinary technology which can now be applied with precision and accuracy. QA programs have likewise matured with the technology improvements and can move/share data in real time between site and study investigators. As digital data transfer systems became available for use within institutions, they were readily repurposed for use at an enterprise level by the clinical trial community. This provided a mechanism for study and site investigators to communicate in real time over issues with specific patients and their appropriateness for the specific trial in a manner that could not occur in the era of hard copy data transfer. The ability to perform real time interactions optimized placement of patient on study and more importantly, applied the correct care plan to the right patient.

The HL treatment community worldwide recognizes the need to perform clinical trials with real time data management strategies including collection of outcome imaging to validate outcome, recurrence patterns, and normal tissue injury.^27–30 The progress being made in our understanding of therapeutic titration and therapy augmentation is driven in part by our ability to acquire complete datasets and re-purpose them for primary and secondary study analysis. Nimble query tools permit us to ask questions not anticipated at the time of trial design for example, treatment response velocity coupled with the need for RT. Providing the datasets are complete without flaw, data interpretation can be secure. However, in circumstances where data are incomplete, at times the flaws and gaps of information can bring investigators and the research community to conclusions which may not be accurate. In the following section, several examples of limitations of this issue are presented.

Problems with Incomplete Data or Limitations in the QA Process

If AI models are going to be useful in clinical care, they need to be built on comprehensive datasets which are accurate and complete. One of the purposes of QA programs is to ensure the completeness and accuracy of datasets to make certain the datasets provide all of the necessary information to draw accurate conclusions for primary and secondary translational science. This is important in radiomic and pathomic research and just as vital in research involving radiation oncology. In a similar manner, if our trials and trial results are to be accepted by colleagues for clinical practice, there must be full confidence in the process of the clinical trial for the outcome to be believed and subsequently accepted as standard of care.

The HeadSTART trial was designed to address the role of Tirapazamine in patients with locally advanced non-metastatic squamous cell carcinoma of the head and neck. The trial was conducted in an era where there was limited information about the role of human papillomavirus (HPV) in tumor development. Successful phase 2 data from a single institutional trial generated significant enthusiasm for a phase 3 randomized trial evaluating Tirapazamine as a hypoxic sensitizer to RT. Trial accrual crossed six continents and with more than 800 patients enrolled. The trial committee wanted QA as part of the trial charter due to the global nature of the study. Digital transfer tools at that time were embryonic and data was accepted through multiple media in order to facilitate imaging and radiation oncology data transfer. The data was acquired and managed through QARC. In discussions before the trial charter was completed, the decision was made by the trial management committee to have an on-treatment review of objects at QARC and not a pre-treatment review. This decision was made because of the perception of delay in data transfer and response. The management committee required an on-treatment review within the first three days. This was a time when radiation oncology was transitioning to volumetric treatment planning. Up until this time, most radiation oncology investigators used fluoroscopy for treatment planning and the use of CT three-dimensional planning tools, digital data transfer, and tumor disease contouring was nascent to the global radiation oncology community.^31,32

During the course of the study, adjustments were requested in defining the tumor volume in 211 patients, greater than 25% of the study population. This was due to under contouring the tumor target resulting in non-protocol coverage of disease in both primary and nodal regions. Of the 211 patients requested to have adjustments in targeting, 95 patients had plans resubmitted during treatment and 116 patients did not have adjustments made. There was a significant difference in survival between patients who had a compliant plan at the time of review and those who had a protocol compliant review after plan adjustments were made. The trial management committee was blinded to the interventional review comments and on re-review of each case were in 100% compliance with all cases deemed study deviations, therefore there was QA on the on-treatment review process. At closure of the study, the trial management committee asked QARC to review all study deviations and determine whether or not they were clinically meaningful. For example, at that time and even today, computer algorithm systems cannot accurately define dose to and beyond the dermal surface. If the nodal contour unintentionally extended to the skin, the dose volume histogram would not be consistent with 95% coverage to 95% of the volume. Therefore, unless there was unambiguous evidence of dermal involvement, this was considered a deviation that would not be clinically meaningful. Exclusion of tumor from protocol dose was considered clinically meaningful. As seen in Figure 4, patient survival for those with deviations not considered clinically meaningful had survival near identical to patients who had protocol compliant adjustments in their care plan during treatment. Those who were scored as deviations clinically meaningful had survival significantly worse than other groups. Similar data was seen for local control seen in Figure 5.

Figure 4.

Overall survival by deviation status: (1) compliant from the outset (n = 502), (2) made compliant following a review by the Quality Assurance Review Center (n = 86), (3) noncompliant but without predicted major adverse impact on tumor control (n = 105), and (4) noncompliant with predicted major adverse impact on tumor control (n = 87). Overall P < .001. Pair-wise tests: not statistically significant except for cohort 1 versus cohort 4 (P < .001), cohort 2 versus cohort 4 (P = .041), and cohort 3 versus cohort 4 (P = .006). TCP, tumor control probability; RT, radiotherapy.³¹ Image courtesy of Wolters Klower Health, Inc.

Figure 5.

Time to locoregional failure by deviation status. The four cohorts are (1) compliant from the outset (n = 502), (2) made compliant following a review by the Quality Assurance Review Center (n = 86), (3) noncompliant but without predicted major adverse impact on tumor control (n = 105), and (4) noncompliant with predicted major adverse impact on tumor control (n = 87). Overall P < .001. Pair-wise tests were not statistically significant except for cohort 1 versus cohort 4 (P < .001), cohort 2 versus cohort 4 (P = .004), and cohort 3 versus cohort 4 (P = .006). TCP, tumor control probability; RT, radiotherapy. ³¹ Image courtesy of Wolters Klower Health, Inc.

The phase 3 study failed to confirm the highly favorable data from the phase 2 trial and the deviations on study likely overwhelmed the ability to demonstrate benefit from the study as the deviations had influence on study outcome. RT was not the study question; however, RT became an unintended focal point of the protocol as seen in the publication. The study called attention to the fact that the quality of RT matters and can negatively impact trial performance if not managed in a proactive manner. Although trial design seemingly addressed this point, in execution a more comprehensive approach would be required to maintain the rigor needed for trial management. The study demonstrated the need for pre-treatment intervention, similar to the experience in HL. It was interesting to note that even those patients with adjusted plans on treatment review had a an 8% statistically significant decrease in survival even though the adjusted plan was study compliant. This indicates that every treatment mattered, and dose volume relationships could not be secure in validation until one could ensure that the correct volume was treated accurately every day. One of the primary goals of modern daily image guidance is to ensure this objective. This study established a tone for developing a strategy to ensure compliance to study for all future industry and NCI clinical trials. The data was collected and collated in a timely and protocol compliant manner, however a greater effort in pre-treatment review would have addressed what proved to be an important study weakness. This will be especially important as we initiate protocols asking important questions for both dose and volume titration for head and neck patients.

The cost of QA is largely driven with building the infrastructure to acquire and manage data in a secure digital manner and training staff as data managers. It is a misperception that cost is driven by data acquisition. Once the infrastructure is in situ and data security systems are in place, it can be used in a nimble manner to collect all RT data and imaging, including outcome imaging in a study compliant manner. Institutions can develop internal strategies for data submission which become facile for data management staff as patients are entered on clinical trials. It has proven to be a mistake to limit data submission under the impression of cost, especially for outcome imaging which serves to validate outcome and pattern of failure, both essential components to modern clinical trial management. Despite decades of clinical trial management in the delivery of RT for breast cancer, we have not yet learned enough from clinical trial experience to indicate with measurable confidence which nodal basins to treat in node negative or positive breast cancer management. The former CALGB managed a series of breast cancer trials testing the incorporation and dose density application of chemotherapy to node positive breast cancer patients. These trials were highly successful and met accrual objectives.^33,34 Because RT was not thought to influence patient survival, information about radiation treatment including volume treated and dose were intentionally not collected. When publications defined the significant survival benefit associated with RT in breast cancer, the information arrived mid-study, therefore amendments could not be introduced to alter study management. An effort was made to acquire RT information and treatment objects by QARC as part of a retrospective analysis, however information could only be collected on approximately one-third of patients on study as many institutions required an amendment to the institutional IRB to permit this information to be forwarded to the QA center. The data that could be acquired was reviewed and published by Carolyn Sartor, MD and colleagues at QARC.³⁵ The information revealed a vast asymmetry on both radiation dose and volume treated without the use of protocol guidelines and review of objects. The data suggested there was a tendency for those patients who received Taxol to also be irradiated, however no further information could be gained from data acquisition and review in this manner. This was disappointing because more than 3,000 patients were treated on these trials and collection of objects with patterns of local regional relapse would have been an invaluable addition to our clinical acumen and would have served to influence an important aspect of clinical care for the radiation oncology community. The lack of data acquisition in this regard also influenced interpretation of ACOSOG trial Z0011. This trial was designed to test titration of surgical and RT to the axilla with the hypothesis that early stage and limited volume patients would benefit from a clinical perspective without vigorous attention to axillary volumes. In the trial management of this study, guidelines were indicated in the protocol however the decision was made not to acquire objects as part of the study. The initial publication suggested that therapy titration was reasonable for this group of patients, however a retrospective review of limited data acquired by QARC revealed that a significant number of patients thought to be of high risk actually had comprehensive regional nodal RT, therefore raising a question concerning the role of limited volume therapy.^36,37 In traditional tangential RT to the breast, approximately 60% of the axillary contents and likely the greater majority of level one are included in the therapy field.³⁸ However, with tangential therapy, levels two and three are not intentionally included. Therefore, the asymmetry associated with the volume of therapy treated makes it challenging for applying the results of Z0011 to real world management of the patients and, accordingly, management of the axilla remains a point of debate among colleagues in a tumor board and breast cancer management group environment which influences current clinical trials as investigators need to permit diversity in approach to make certain study accrual objectives are met. Positron emission tomography studies are now obtained in a more commonplace manner as part of breast cancer management for patients with triple negative disease before consideration for surgery and studies are revealing occasional different locations of lymph nodes which may/may not have been included in traditional comprehensive regional fields as defined by anatomical location.³⁹ Therefore, more work is needed to re-address these points which we did not define as part of the QA process when the opportunity was available. Patients are currently treated off study in a non-uniform manner without the benefit of comprehensive failure pattern data to support regional clinical application of RT.

Management of patients with non-small cell lung carcinoma (NSCLC) has undergone considerable change in the past two decades. Volumetric planning has provided the infrastructure for defining dose volume constraints for multiple critical structures in the thorax including pulmonary and cardiac volumes. This has helped put structure to clinical trials and provide a framework for treatment planning on protocols managed by the NCTN and industry. RTOG protocol 0617 was a two-tiered randomization study designed to evaluate the use of cetuximab in patients being treated for NSCLC with two different RT treatment schedules of 60 Gy and 74 Gy. RTOG 0617 revealed no benefit to the higher dose RT. Proponents point to the changing molecular landscape in the treatment of lung carcinoma including the increasing use of immune and targeted therapies. The study has had significant influence on the direction of lung cancer clinical trials both in industry and the NCTN over the past decade, especially with the increasing use of immunotherapy. Most industry and NCTN clinical trials, including those involving immunotherapy, limit the dose delivered to lung cancer to 60 Gy under the assumption that “less is better” to improve tumor control, meet protocol defined normal tissue metrics, and limit toxicity.⁴⁰

While these are reasonable conclusions with available data, a closer evaluation of the data reveals that for the first few years on study, the local control in the high dose arm was statistically inferior to the low dose arm as seen in Table 1. This would be counterintuitive as one would assume that local control would be equivalent if not superior in the high dose arm. The QA process for this study collected RT data sets and compliance standards were established by dose to volume on the data set. Other groups routinely collect diagnostic images to validate the targeting of tumor as part of the data management process. In this protocol, it was generally assumed the tumor contours were study appropriate. This may explain why in the initial phase of the study, local control was inferior on the high dose arm, suggesting that tumor contours possibly undertreated areas of disease. Although local control balanced between the arms in later analysis, if this did not occur in the early part of the study, would the narrative of “less is better” remain valid? We will not know the answer to this question as diagnostic imaging was not collected.

Table 1.

RTOG 0617- For the first 3 years on study, local failure 74 Gy: 36%, local failure 60 Gy: 24%.⁴⁰

	60Gy (n=217)*	74Gy (n=207)	Cetuximab (n=237)*	No cetuximab (n=228)
Overall survival
Dead	127	140	140	139
1 year	80.0% (73.9–84.7)	69.8% (63.1–75.6)	76.2% (70.2–81.1)	71.1% (64.7–76.6)
2 year	57.6% (50.6–63.9)	44.6% (37.7–51.3)	52.3% (45.6–58.5)	50.1% (43.3–56.6)
Median (months)	28.7 (24.1–36.9)	20.3 (17.7–25.0)	25.0 (20.2–30.5)	24.0 (19.8–28.6)
HR	1.38 (1.09–1.76)	..	1.07 (0.84–1.35)	..
p value (log-rank, one-sided)	0.004	..	0.29	..
Progression-free survival
Fail	164	164	181	168
1 year	49.2% (42.3–55.6)	41.2% (34.4–47.8)	44.3% (37.8–50.5)	46.3% (39.6–52.7)
2 year	29.1% (23.1–35.3)	21.4% (16.1–27.3)	24.2% (18.8–29.9)	27.5% (21.7–33.6)
Median (months)	11.8 (10.2–14.3)	9.8 (8.8–11.6)	10.8 (9.8–12.3)	10.7 (9.3–13.2)
HR	1.19 (0.95–1.47)	..	0.99 (0.80–1.22)	..
p value (log-rank, one-sided)	0.12	..	0.89	..
Local failure
Fail	77	86	95	77
1 year	16.3% (11.4–21.3)	24.8% (18.9–30.7)	22.2% (16.8–27.5)	17.6% (12.6–22.7)
2 year	30.7% (24.5–36.9)	38.6% (31.9–45.3)	38.2% (31.9–44.5)	30.7% (24.6–36.9)
HR	1.26 (0.93–1.71)	..	0.82 (0.61–1.11)	..
p value (Gray, two-sided)	0.13	..	0.20	..
Distant metastasis
Fail	106	107	124	98
1 year	32.2% (25.9–38.5)	35.1% (28.5–41.6)	35.0% (28.9–41.2)	29.8% (23.8–35.9)
2 year	46.6% (39.9–53.4}	51.0% (44.1–57.9)	52.6% (46.0–59.1)	42.0% (35.4–48.6)
HR	1.10 (0.84–1.43)	..	0.80 (0.61–1.04)	..
p value (Gray, two-sided)	0.48	..	0.09	..

Data are % (95% CI), median (95% CI), or HR (95% CI).

^*Reference level for the hazard ratios presented.

In the initial phase of applying new technology into clinical trials including review of tumor contours, there is uniformly a growth phase in the RT clinical community as to how the technology and change are applied to patient care. This is a vulnerable time as site investigators are adjusting to protocol language more familiar to the investigators who wrote the study. Questions will be asked by site investigators that may have not been anticipated at the time of initiating the study and challenges of meeting normal tissue constraints due to target size and tumor location require real time communication between investigators and QA centers to ensure study compliance. Sites will be opening trials at varied timepoints during the study, and each will experience similar issues, perhaps to a lesser degree as the QA centers would be familiar with issues and may better anticipate issues de novo to educate those new to the study. Nevertheless, the education of sites on study remains an exercise of self-renewal and this is where the QA centers can have significant impact in the success of a clinical trial by maintaining protocol standards during the study. Working with study investigators, QA centers maintain the consistency of trial conduct which will serve to strengthen and validate study conclusions.^1–5

Study 0617 also raised important questions about normal tissue tolerance and toxicity. Since the initiation of the study, there have been considerable advances in technology for conformal avoidance of normal tissues and AI is supporting contouring of normal tissue and tumor anatomy. Treatment planning for lung patients remains challenging as optimal planning requires a balance of constraints as cardiac, pulmonary, esophageal, and spinal cord constraints can compete and place emphasis on one can place unintended dose to another structure. For example, a hard cardiac constraint can unintentionally increase pulmonary V20 and V5. Therefore, one has to prioritize constraints and ask harder questions as RT is not a drug and needs to be carefully applied for optimal normal tissue outcome.⁴¹

QA processes can review diagnostic imaging and evaluate tumor targeting and normal tissue dose to ensure compliance to study objectives. When patients are treated per protocol, outcomes can be trusted and more readily applied to clinical practice without the caveat of doubt created by ambiguity of incomplete data, deviation from standard, or limitations in the processing of data. It will also be important to have a better understanding of pre-existing conditions prior to initiation of protocol therapy. Oncologists need to apply treatment to clinical situations and often have limited tools before treatment is applied a priori. Moving forward for clinical trials and toxicity assessment, it will be important to have a nimble functional tool for pre-therapy normal tissue assessment before applying protocol therapy. There are imbedded processes for audiology function in the delivery of cisplatin and similar processes will need to be imbedded in clinical trials for normal tissue function for cardiac, pulmonary, and additional targets to optimize assessment of function post therapy and devise strategies to mitigate decrease in function. This will require large datasets which can only be generated from clinical trial data management centers with QA review of the data to ensure completeness and accuracy of the dataset. QA processes can ensure this information is made available to investigators as part of the clinical trial process.

Recent studies in the application of post-operative RT for resected lung cancer also point to the need for accurate datasets with QA review. One of these studies indicated a 20% improvement in local control; however, the study also suggested the advantage was negatively influenced by a significant increase in toxicity. It was also noted that nearly 90% of the patients treated with RT were treated with three-dimensional therapy, not intensity modulation and volumes were not reviewed. Post-operative patients have a decrease in functional pulmonary volume; therefore, radiation oncologists need to be cognizant that metrics such as V20 and V5 have different meaning in surgical patients as the denominator has changed, therefore QA processes need to reflect this matter and guidelines for care require a thoughtful approach to ensure benefits of therapy without the consequence.⁴²

COG standard risk medulloblastoma protocol ACNS0331 was a study with Jeff Michalski MD as principal investigator asking important radiation oncology questions in this disease including volume titration of the boost field to an image-guided target volume and titration of the spine dose in age-appropriate patients. The protocol required 189 sites to meet accrual objectives. Retrospective diagnostic imaging review revealed more than 10% of patients entered onto study were not eligible due to more than 1.5 cm³ residual disease post-surgery in the posterior fossa or evidence of intramedullary lesions on spine imaging. This is important for many reasons including that these patients unintentionally entered on the standard risk trial had a statistically significant decrease in survival compared to the primary study population. The RT components to the study were reviewed in real time, however imaging by study investigators was performed in retrospect and the time of the final review of RT objects for RT volume compliance. Adjustments in approach are planned for the next series of studies which include real time review by QA centers with referral to study neuro-radiologist for challenging and ambiguous cases. QA performed in real time can ensure that the correct patient is entered on the correct study to ensure optimal protocol management.⁴³

Summary

Fairchild and colleagues performed a literature review of multiple clinical trials involving RT and noted that protocol compliant RT was associated with a decrease in failure rates and an increase in survival.⁴⁴ The investigators concluded that study compliance contributed to the ability of the collected data to answer the study question. Limitations and factors contributing to the ability to comply with protocol management included misinterpretation of ambiguities written in the study, institutional differences in problem solving and approach to patient care, inter-physician variation in interpretation of the target volume, and intentional changes in approach due to the perception of acute and late toxicity. While these reasons have merit, most, if not all, can be addressed through real time dialogue to determine if protocol guidelines can be met. The interactions can center on study appropriateness for the individual patient and where compromises can be met in targeting to meet trial objectives and needs of the patient. These exchanges can help secure study compliance and meet study objectives. QA processes have matured as our technology has evolved. Processes today can function at an enterprise level not anticipated by our mentors and data collection can be robust and complete.

As can be seen in the examples provided in the text, the quality of data matters, however of equal importance, the process of data acquisition and management matters. This is the purpose of QA. For primary study endpoints, pathomics, radiomics, RT objects, outcome imaging, and surgery/medical oncology information all play an important role in trial analysis and are included in the primary manuscript. The integrity of the data is ensured by the QA process which in turn promotes confidence and clinical applicability of trial outcome. Involved investigators likewise want to evaluate additional trial endpoints not always anticipated at the time of trial design and execution. To do so and to do this well, availability and completeness of data are essential to make certain the review is timely and accurate. We have learned that on-treatment review in several disease sites may be less helpful than pre-treatment review in selected disease areas, especially when new concepts are introduced to the RT treatment community. We have seen where partial acquisition of data can limit our ability to ask secondary questions often not recognized at the time of trial design limiting confidence in the clinical applicability of the results. We have shown how corrective measures can be applied to improve data acquisition and management to support the success of the trial. The HL QA process evolution led investigators to ask more modern questions in protocol format including response-driven adaptive RT and therapy titration. The ability to acquire and share information worldwide on a real time basis opens doors to pathways previously hidden. The future of clinical trials is bright with processes in place to ensure acquisition and management of complete and accurate data.

We have much to learn. We need to optimize our targeting and dose to primary brain tumors relative to advanced technology imaging. Can we titrate radiation dose and volume to selected head and neck patients, optimize function, and maintain local control? Can we adjust RT to gross tumor only for molecular driven targets in primary lung cancer to optimize cardio-pulmonary tolerance? Can we optimize targeting for gastrointestinal volumes and limit toxicity to normal tissue? Metabolic imaging is suggesting a higher number of node positive patients in advanced cervix cancer than was previously recognized. Can we define the correct dose to nodal targets, and do we need to adjust for size and metabolic activity? Can we dose paint image-defined targets in the prostate and consider the prostate gland a clinical target volume (CTV), not a gross tumor volume (GTV)? Can we limit toxicity for our pediatric populations?^45–60 These important questions and others can only be answered in the context of a trial with large datasets. The quality of the data provided by QA has direct bearing on the answers.