Practice-Based Evidence to Evidence-Based Practice: Building the National Radiation Oncology Registry

Successful implementation of the National Radiation Oncology Registry should improve the quality of care for patients with cancer treated with radiation.

Abstract

The National Radiation Oncology Registry (NROR), sponsored by the Radiation Oncology Institute and the American Society for Radiation Oncology, is designed to collect standardized information on cancer care delivery among patients treated with radiotherapy in the United States and will focus on patients with prostate cancer. Stakeholders were engaged through a forum that emphasized the need for patient-centered outcomes, minimal data burden, and maximal connectivity to existing registries and databases. An electronic infrastructure is under development to provide connectivity across radiation oncology and hospital information systems. The NROR Gateway features automatic abstraction as well as aggregation of treatment and outcome data. The prostate cancer data dictionary provides standardized elements in four domains: facility, physician, patient, and treatment. The pilot phase will consist of clinical centers chosen to provide a representative mix of radiation treatment modalities, facility types, population-based settings, and regional locations. The initial set of radiation practice metrics includes physician board certification and maintenance, ordering of staging scans, active surveillance discussion, dose prescriptions for low-risk/high-risk disease, radiation fields for low-risk/high-risk disease, image-guided radiation therapy use, androgen deprivation therapy use, post-brachytherapy implant computed tomography dosimetry, collection of toxicity assessments, and longitudinal patient follow-up. The NROR pilot study will provide the framework for expansion to a nationwide electronic registry for radiation oncology.

Introduction

Patient registries have emerged as important tools for collection of data on the quality, safety, effectiveness, and value of medical therapies.¹ Examples of successful registries include the American College of Cardiology Registries,² the Society of Thoracic Surgeons National Database,³ the National Oncologic PET Registry,⁴ and the Quality Oncology Practice Initiative by the American Society for Clinical Oncology.⁵ A radiation oncology–specific national registry comprising standardized aggregated data on therapies and technologies used to treat specific types of cancers and their outcomes would yield invaluable quality improvement potential by focusing on best practices, treatment effectiveness, and practice patterns of care as provided by radiation oncologists.

The National Radiation Oncology Registry (NROR) is a national collaborative initiative between the Radiation Oncology Institute and the American Society for Radiation Oncology. The overarching mission of the NROR is to improve the care of patients with cancer by capturing reliable information on treatment delivery and health outcomes.⁶ Prostate cancer (PCa) has been selected for the NROR pilot as a model disease site given its high incidence, multiple management options, potential public health and economic implications, and its identification by the Institute of Medicine as the area of oncology most in need of comparative-effectiveness research.⁷ A group of committed volunteers has provided expert input at every stage of development. As the regulatory sponsor, the Radiation Oncology Institute provides leadership at the national level, promoting the NROR and its goals within the radiation oncology community.

The most significant accomplishments to date include the finalization of a standardized set of data elements for the prostate pilot; the development of an information technology infrastructure to support the efficient collection, aggregation, and analysis of clinical data; and the establishment of a standardized set of radiation practice metrics.

Registry Development

Stakeholder Engagement

To build consensus for the goals of the NROR through stakeholder engagement, a Stakeholder Forum was held on April 19, 2012. Participants included representatives from the American Society for Radiation Oncology and other professional societies, physicians and physicists, patient advocates, participating vendors, the federal government, payers, and other registries (Appendix Table A1, online only). The Stakeholder Forum emphasized the need for minimal data collection burden, maximal connectivity to existing registries and databases, and patient-centered outcomes. A strong foundation for long-term engagement among participants and the organizations and constituencies they represent was established.

Electronic Infrastructure

We have developed the detailed functional specifications for a software platform to fulfill the NROR mission of collecting patient-specific radiotherapy treatment and outcome data electronically in close collaboration with radiotherapy information system vendors. A unique feature of the NROR will be the automatic abstraction and aggregation of treatment and outcome data from electronic medical records, patient portals, oncology information systems, and radiation therapy treatment planning systems.

Clinical data will be submitted from multiple clinical sites and undergo quality review consisting of source document verification at the site before the data are uploaded into the NROR database to be aggregated and subsequently analyzed. The registry database fulfills all security and privacy considerations and maintains redundancy to minimize data loss, as illustrated in Figure 1. A gateway web application (NROR Gateway) performs the initial data aggregation for each institution, supporting manual data entry and automated transfers via open web services application programming interfaces. Deidentified, quality assured data are submitted from the Gateway via web services and stored in a relational database optimized for analytics. The decoupling of the data collection from the analytic repository facilitates a hybrid system of both electronic and manual data entry that can support all clinics regardless of their level of electronic medical record implementation.

Figure 1.

National Radiation Oncology Registry (NROR) electronic infrastructure. Personal health information will be collected at the sites and used for quality assurance at the local level, but only a limited data set will be transferred to the registry database for aggregation and analysis.

Prostate Data Dictionary

The NROR Prostate Data Dictionary comprises data elements derived from authoritative sources in radiation oncology, including the Radiation Therapy Oncology Group radiotherapy trials, Cancer of the Prostate Strategic Urologic Research Endeavor,⁸ the Quantitative Analysis of Normal Tissue Effects in the Clinic review of radiotherapy toxicity,⁹ the Quality Research in Radiation Oncology Patterns of Care studies,¹⁰ the National Cancer Institute Common Toxicity Criteria, Agency for Healthcare Research and Quality processes of care elements, and the American College of Radiology facility descriptors. When possible, an effort was made to adhere to established data standards.

The Data Dictionary has been reduced to the essential elements needed for the pilot in PCa through review by PCa health services and technical experts. The data elements are organized into a comprehensive taxonomy for the collection of facility, physician, patient, disease, treatment, and outcomes data, as illustrated in Figure 2. The Data Dictionary can be downloaded at http://www.roinstitute.org/NROR/Documents/DDV2.pdf.

Figure 2.

National Radiation Oncology Registry data dictionary hierarchy. Abbreviation: MOC, maintenance of certification.

Pilot in PCa

More than 80 clinical centers indicated willingness to participate in the PCa pilot by completing a comprehensive online questionnaire. The pilot phase will consist of clinical centers chosen to provide a representative mix of radiation treatment modalities, facility types, settings, regional locations, and PCa patient volumes selected through an objective, blinded, and transparent process. The NROR is being rolled out in phases, with the first phase consisting of seven beta testing sites that will rigorously test the electronic infrastructure and develop efficient workflows to minimize the burden of data collection and submission at the pilot sites. The beta testing phase began in December 2012 and will continue through the middle of 2013. After the beta testing phase, the NROR will grow to a total of approximately 30 pilot sites in 2013, with planned expansion to additional radiation facilities in 2014 and additional diseases in 2015. This phased approach will provide an opportunity for incremental improvements and implementation of lessons learned along the way.

Radiation Practice Metrics

We have developed a set of top 10 radiation oncology–specific quality measures for PCa treatment known as the radiation practice metrics (RPMs), which will serve as benchmarks for quality improvement. Sources for the RPMs include National Comprehensive Cancer Network Clinical Practice Guidelines, European Association of Urology Guidelines, American College of Radiology Appropriateness Criteria, American Urological Association Prostate Guidelines, Radiation Therapy Oncology Group Consensus Panel Atlas, American Brachytherapy Society consensus guidelines, Transatlantic Consensus Group, the National Quality Forum, Quality Oncology Practice Initiative, and the National Cancer Institute Cancer Therapy Evaluation Program.

RPM benchmarking reports will be provided to participating sites on a regular and timely basis. The initial list of RPMs includes physician board certification and maintenance,¹¹ ordering of staging scans,^12–15 active surveillance discussion,^12,16,17 dose prescriptions for low-risk/high-risk disease,^12,18–20 radiation fields for low-risk/high-risk disease,^18,19,21,22 image-guided radiation therapy use,^{17–19,21,23} androgen deprivation therapy use,^{12–14,16,20} postbrachytherapy implant computed tomography dosimetry,^24,25 collection of toxicity assessments,²⁶ and longitudinal patient follow-up.^12,20,27,28 Over time, the NROR registry will attempt to validate the relationship of these measures to patient-centered health outcomes. Table 1 lists RPMs and their sources.

Table 1.

RPMs

Metric	Source
Physician board certification and maintenance status	American Board of Radiology Maintenance of Certification
Ordering of staging scans: bone, CT, etc	EUA Guidelines, QOPI, NCCN Clinical Practice Guidelines, NQF
Active surveillance discussion	Transatlantic Consensus Group, EAU Guidelines, AUA Guidelines
Dose prescriptions for low-risk/high-risk disease	ACR Appropriateness Criteria, EAU Guidelines
Radiation fields for low-risk/high-risk disease	RTOG Consensus Panel Atlas, ACR Appropriateness Criteria
Image-guided radiation therapy utilization	Transatlantic Consensus Group, ACR Appropriateness Criteria
Androgen deprivation therapy utilization	EAU Guidelines, AUA Guidelines, NCCN Clinical Practice Guidelines, NQF
Postbrachytherapy implant CT dosimetry	ABS Consensus Guidelines, ACR Appropriateness Criteria
Collection of toxicity assessments	NCI CTEP Common Terminology Criteria for Adverse Events 4.0
Longitudinal patient follow-up	EUA Guidelines, ACR Appropriateness Criteria

Abbreviations: ABS, American Brachytherapy Society; ACR, American College of Radiology; AUA, American Urological Association; CT, computed tomography; CTEP, Cancer Therapy Evaluation Program; EUA, European Association of Urology; NCCN, National Comprehensive Cancer Network; NCI, National Cancer Institute; NQF, National Quality Forum; QOPI, Quality Oncology Practice Initiative; RPM, radiation practice metric; RTOG, Radiation Therapy Oncology Group.

Regulatory

To streamline the review/approval process and minimize the regulatory burden on the sites, we selected a central institutional review board, New England Institutional Review Board. After consultation with both the New England Institutional Review Board and other registries, we developed a regulatory policy using an opt-out informed consent: an informational handout describing the pilot and providing the option to refuse participation will be distributed to eligible patients. Participating sites will sign a participation agreement governing data sharing in compliance with the Health Insurance Portability and Accountability Act and the Health Information Technology for Economic and Clinical Health Act.

Discussion

The overall goal of the NROR is to improve the quality of cancer care through real-world and near–real-time assessment of care delivery and outcomes. We have established a pilot registry using a consensus-based set of PCa data elements as a model. The NROR pilot will provide the framework for expansion to a national electronic registry for radiation oncology in the United States. Benchmarking across institutions will promote rapid learning and accountable cancer care that is anticipated to benefit patients. The electronic infrastructure will facilitate population-based examination of patient-level dosimetric and treatment data, transforming efforts to improve quality and safety in radiation oncology.

With its great potential, the NROR still faces several challenges, including developing incentives for broad participation, identification of sources of sustainable funding, establishing links to other registries, implementation of the patient portal for patient-reported outcomes, and further streamlining of the data extraction process to maximize data collection while minimizing burden on participating sites. Successful implementation of the NROR should improve the quality of care for patients with cancer treated with radiation.

Appendix

Table A1.

National Radiation Oncology Registry Stakeholder Forum Participating Organizations

Organization	Constituency
Accuray	Industry
Agency for Healthcare Research and Quality	Federal government
American Association for Physicists in Medicine	Professional society
American Board of Radiology	Professional society
American College of Cardiology/American Heart Association Task Force on Clinical Data Standards	Other registries
American College of Radiology	Professional society
American Medical Association	Professional society
American Society for Radiation Oncology	Professional society
American Society of Clinical Oncology	Other registries
American Urological Association	Professional society
Blue Cross Blue Shield Association	Payer
Center for Medical Technology Policy	Nonprofit
Centers for Medicare and Medicaid Services	Federal government
Christiana Care	Payer
Duke University	Physician/physicist
Elekta	Industry
Food and Drug Administration	Federal government
Friends of Cancer Research	Patient advocate
HealthCare IT	Vendor
IBA Group	Industry
Johns Hopkins University	Physician/physicist
Massachusetts General Hospital	Physician/physicist
Medical College of Wisconsin	Physician/physicist
Mevion	Industry
National Cancer Institute	Federal government
OncoChart	Industry
Procure	Industry
Radiation Oncology Institute	Professional society
Siemens	Industry
St Luke's Hospital of Kansas City	Physician/physicist
UnitedHealthcare	Payer
University of California Los Angeles	Payer
University of Florida	Physician/physicist
University of Michigan	Physician/physicist
University of Oklahoma	Physician/physicist
University of Pennsylvania	Physician/physicist
Us, Too, Prostate Cancer Support Group	Patient advocate
Valley Radiotherapy Associates Medical Group	Physician/physicist
Vanderbilt School of Medicine	Physician/physicist
Varian	Industry
Washington University	Physician/physicist
WellPoint	Payer
21st Century Oncology	Physician group

Authors' Disclosures of Potential Conflicts of Interest

Although all authors completed the disclosure declaration, the following author(s) and/or an author's immediate family member(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO's conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors.

Employment or Leadership Position: Jason A. Efstathiou, Radiation Oncology Institute (U); C. Bob Bogardus, Radiation Oncology Institute (U), Bogardus Medical Systems (C); Jeffrey Carlin, Bogardus Medical Systems (C); Dave Eggert, Elekta AB (C); Benedick A. Fraass, Radiation Oncology Institute (U); Joel Goldwein, Elekta (C); Ken Hotz, Varian Medical Systems (C); Colleen A. Lawton, American Society for Radiation Oncology (U), Elsevier (C); Sasa Mutic, Radiation Oncology Institute (U); Christopher M. Rose, Radiation Oncology Institute (U), Terry Wall, Radiation Oncology Institute (U); Anthony L. Zietman, American Society for Radiation Oncology (U); Peter E. Gabriel, Radiation Oncology Institute (U); Justin E. Bekelman, Radiation Oncology Institute (U) Consultant or Advisory Role: None Stock Ownership: Ken Hotz, Varian Medical Systems Honoraria: Benedick A. Fraass, Varian Medical Systems Research Funding: Todd R. McNutt, Philips Healthcare, Elekta; Margie Hunt, Varian Medical Systems; Charles Mayo, Varian Medical Systems; Wolfgang Tomé, Philips Radiation Oncology Systems Expert Testimony: None Other Remuneration: None

Author Contributions

Conception and design: All authors

Administrative support: Deborah S. Nassif

Provision of study materials or patients: Jason A. Efstathiou, Deborah S. Nassif, Todd R. McNutt, Walter Bosch, Ron C. Chen, Henry Chou, Joel Goldwein, Karen E. Hoffman, Margie Hunt, Colleen A.F. Lawton, Charles Mayo, Jeff M. Michalski, Sasa Mutic, Louis Potters, Howard M. Sandler, Gregory Sharp, Wolfgang Tomé, Phuoc Tran, Anthony L. Zietman, Peter E. Gabriel, Justin E. Bekelman

Collection and assembly of data: Jason A. Efstathiou, Deborah S. Nassif, Todd R. McNutt, Walter Bosch, Ron C. Chen, Henry Chou, Joel Goldwein, Karen E. Hoffman, Margie Hunt, Colleen A.F. Lawton, Charles Mayo, Jeff M. Michalski, Sasa Mutic, Louis Potters, Howard M. Sandler, Gregory Sharp, Wolfgang Tomé, Phuoc Tran, Anthony L. Zietman, Peter E. Gabriel, Justin E. Bekelman

Data analysis and interpretation: Jason A. Efstathiou, Deborah S. Nassif, Todd R. McNutt, Peter E. Gabriel, Justin E. Bekelman

Manuscript writing: Jason A. Efstathiou, Deborah S. Nassif, Todd R. McNutt, Ronald C. Chen, Joel Goldwein, Karen E. Hoffman, Margie Hunt, Colleen A.F. Lawton, Christopher M. Rose, Phuoc T. Tran, Anthony L. Zietman, Peter E. Gabriel, Justin E. Bekelman

Final approval of manuscript: All authors