ORBIT-RT: A Real-Time, Open Platform for Knowledge-Based Quality Control of Radiotherapy Treatment Planning

Abstract

PURPOSE

Access to knowledge-based treatment plan quality control has been hindered by the complexity of developing models and integration with different treatment planning systems (TPS). Online Real-time Benchmarking Information Technology for RadioTherapy (ORBIT-RT) provides a free, web-based platform for knowledge-based dose estimation that can be used by clinicians worldwide to benchmark the quality of their radiotherapy plans.

MATERIALS AND METHODS

The ORBIT-RT platform was developed to satisfy four primary design criteria: web-based access, TPS independence, Health Insurance Portability and Accountability Act compliance, and autonomous operation. ORBIT-RT uses a cloud-based server to automatically anonymize a user's Digital Imaging and Communications in Medicine for RadioTherapy (DICOM-RT) file before upload and processing of the case. From there, ORBIT-RT uses established knowledge-based dose-volume histogram (DVH) estimation methods to autonomously create DVH estimations for the uploaded DICOM-RT. ORBIT-RT performance was evaluated with an independent validation set of 45 volumetric modulated arc therapy prostate plans with two key metrics: (i) accuracy of the DVH estimations, as quantified by their error, DVH_clinical − DVH_prediction and (ii) time to process and display the DVH estimations on the ORBIT-RT platform.

RESULTS

ORBIT-RT organ DVH predictions show < 1% bias and 3% error uncertainty at doses > 80% of prescription for the prostate validation set. The ORBIT-RT extensions require 3.0 seconds per organ to analyze. The DICOM upload, data transfer, and DVH output display extend the entire system workflow to 2.5-3 minutes.

CONCLUSION

ORBIT-RT demonstrated fast and fully autonomous knowledge-based feedback on a web-based platform that takes only anonymized DICOM-RT as input. The ORBIT-RT system can be used for real-time quality control feedback that provides users with objective comparisons for final plan DVHs.

INTRODUCTION

The quality of radiotherapy (RT) treatment plans—sufficient dose delivery to the planning target volume (PTV) combined with adequate sparing of organs-at-risk (OARs)—is susceptible to subjectivity at all stages of planning. Plan quality variations have been observed in instances in which treatment planning guidelines were not patient-specific,^1,2 as the tools to have patient-specific quality criteria are not universally available. This can result in avoidable patient toxicity where OARs receive higher dose than necessary for cancer treatment. Even exemplary data-driven policies for treatment plan quality control face technical challenges in clinics where commercial knowledge-based quality control is only available from within the commercial treatment planning system (TPS), still requiring several manual interventions to complete the workflow.

CONTEXT

• Key Objective

• How can knowledge-based dose-volume histogram (DVH) prediction models be more widely disseminated for accessible quality control of organs-at-risk (OAR) radiotherapy dose sparing? Online Real-time Benchmarking Information Technology for RadioTherapy (ORBIT-RT) is the first real-time, automated, web-based platform for this purpose.

• Knowledge Generated

• ORBIT-RT provides prostate OAR DVH predictions with accuracy and speed suitable for clinical use. ORBIT-RT uses only standard Digital Imaging and Communications in Medicine for RadioTherapy (DICOM-RT) as input, is Health Insurance Portability and Accountability Act-compliant, and functions independently of any local treatment planning system.

• Relevance

• Clinicians as registered users may use ORBIT-RT for reliable, real-time quality control of their radiotherapy treatment plans, regardless of local resources or available software.

The Online Real-time Benchmarking Informatics Technology for RadioTherapy (ORBIT-RT)³ project was thus launched to respond to the need for (1) patient-specific quality control of RT plans^1,4-12 and (2) uninhibited access to this quality control from large and small clinics everywhere, completely independent of the TPS in use. ORBIT-RT is designed to analyze existing coplanar external beam treatment plans by giving knowledge-based estimates of dose-volume histograms (DVHs), built from previously treated cases. Effectively, ORBIT-RT tells the user what DVHs their plan would have received if treated at our institution. Submitted OARs with larger dose-receiving volumes relative to the ORBIT-RT DVH predictions may benefit from further plan improvement. The platform is fully automated and entirely web-based.

To ensure accuracy, accessibility, practicality, and confidentiality, the ORBIT-RT platform was required to meet several design criteria:

1. Universally accessible web-based platform—The ORBIT-RT platform can be accessed from anywhere in the world with standard internet access (no virtual private network [VPN] required).

2. Digital Imaging and Communications in Medicine (DICOM) file–based operation—The ORBIT-RT platform must be independent of any particular commercial TPS installation.

3. Minimal user input for case submission—To begin a new plan analysis, ORBIT-RT users need to only identify the disease site, Rx dose, and PTV structure name(s) before uploading the study.

4. Health Insurance Portability and Accountability Act (HIPAA) compliance—Uploaded DICOM images can only be accepted by the ORBIT-RT image server after having gone through a rigorous two-step anonymization process that removes personally identifiable information.

5. Completely autonomous dose prediction workflow—The knowledge-based dose estimation functions are triggered by the arrival of an uploaded treatment planning study (DICOM-RT objects: RT image, RT structure set, RT plan, and RT dose) at the ORBIT-RT image server.

6. System intelligence to aid efficient operation—Structure identification logic is used to complete in seconds the potentially tedious process of matching free text structure names (eg, femur_r, rt_femur, and RT FEMUR) to dose prediction models.

MATERIALS AND METHODS

ORBIT-RT Architecture

To meet ORBIT-RT's necessary design criteria, a commercial back-end for the DICOM-RT was developed using the MIM software suite¹³ (version 7.0.2), in conjunction with MIMCloud architecture. MIM has all the required native functionality (eg, contour operations, DVH calculations, extension support, and automated workflows) to process the DICOM-RT. The ORBIT-RT knowledge-based DVH estimation module within MIM operates as a set of customizable Java scripts, using an existing, published Java application programming interface (API). New features in MIMCloud and MIMCloud Assistant (MCA) ensure HIPAA-compliant encryption and anonymization of DICOM data. The front-end process responsible for predictions is activated automatically via MIM Assistant upon the receipt of a new patient's DICOM to the image server.

To encourage uptake by a broad user base, the ORBIT-RT aim was to provide the simplest and fastest user experience possible. The process for DVH estimation follows these generic steps (Fig 1):

1. Case Metadata. After registration as an ORBIT-RT user and linking to an institution, the user logs in to the ORBIT-RT website and submits the basic plan metadata (disease type, Rx dose(s), and target name). A customized case number is automatically generated for the user (eg, ClinicName_Prostate_0023).

2. DICOM Anonymization and Upload. The user exports the relevant structure DICOM from within their own TPS via MCA, made available to registered ORBIT-RT users. Local MCA installations are configured with a template that anonymizes patient-identifying DICOM fields according to safe harbor standards before export. In turn, ORBIT-RT's MIMCloud destination is configured to reject any DICOM uploads that have not been processed by the anonymization template. In this way, anonymization is checked at both ends before transfer can occur. Anonymization templates are configured during institution registration with ORBIT-RT. Patients anonymized with the same template from the same institution are assigned the same anonymous header scheme, so that any previous RT plans may be grouped appropriately. Cases are stored in the user's institutional directory on the ORBIT-RT system, wherein they are then transferred to the local ORBIT-RT image server for processing.

3. MIM Workflow. On the ORBIT-RT server, scripts using the MIM API are triggered by the arrival of new DICOM-RT data. The user-entered metadata ensure that the proper disease site model is applied and that the target identification and Rx dose are correct.

4. Structure Matching. Because incorrect structure identification could yield erroneous results, ORBIT-RT automatches organ structure names found in the DICOM-RT file to available organ model names using a precompiled set of custom regular expressions. For example, the TG 263-compliant structure name BrachialPlex_L is searched in the DICOM-RT file as brac.*ple?x(?!\\s*_*-*r) (case insensitive). The system sends the prospective contour matches back to the ORBIT-RT website, where any corrections by the user are made with an intuitive drag-drop interface. After the user confirms the final structure matching, the extension makes any necessary corrections and completes processing of the case.

5. DVH Display. Once the DVH estimation is complete, ORBIT-RT presents interactive OAR DVH predictions, including error bands, to the user via the website, alongside any clinical DVHs already present in the input DICOM. For user convenience and easy pattern recognition, ORBIT-RT DVHs retain the same display colors as their represented OARs in the original TPS.

FIG 1.

Autonomous knowledge-based radiotherapy plan quality control on ORBIT-RT. Users upload basic plan metadata (top left) to the ORBIT-RT website before uploading the anonymized patient DICOM using MCA. OAR structure names are automatched and confirmed by the user (top center). Interactive DVH predictions with error estimates are presented on the website (top right). DICOM, Digital Imaging and Communications in Medicine; DVH, dose-volume histogram; MCA, MIMCloud Assistant; OAR, organs-at-risk; ORBIT-RT, Online Real-time Benchmarking Informatics Technology for RadioTherapy.

Once the DVH predictions have arrived at the website, interactive features are available to assist the user in analysis. DVHs may be toggled on or off for each OAR (the top right image of Fig 1), and plot axes may be represented in absolute or relative units. Users may enter specific dose or volume values (eg, V_20Gy) in an entry table, which will return the corresponding ORBIT-RT DVH and DVH error estimates, alongside the current clinical DVH values.

All ORBIT-RT DVH predictions are stored on the website for future recall on the My Cases page. Users are able to view the DVHs of other users from the same institution. Users may filter DVH collections according to username and disease site.

ORBIT-RT DVH Estimation Model

The knowledge-based algorithm used by ORBIT-RT follows closely the methodology described by Appenzoller et al.¹⁰ Where possible, names of variables have been preserved for ease of reference. Several free parameters arise from the model build; they are identified explicitly in the text and in the logic for their chosen values.

Volume discretization.

The ORBIT-RT model is a 1D radial model, making the discrete cells concentric shells expanding around the PTV, where the zeroth shell (k = 0) is the PTV itself (Fig 2). The OAR discretization is the OAR overlap with this shell discretization (ie, OAR shells). The discretization is subject to a number of free parameters, tunable by the modeler to optimize the estimation: the total number of PTV shells, a high-resolution shell interval δ_high, a low-resolution shell interval δ_low, and the distance from the PTV in which the interval changes from high resolution to low resolution. The shell resolution is chosen based on the expected radial dose gradients and dose extent around the PTV for the given disease site. The total number of shells is then chosen to encompass only the spatial region of clinical interest as calculations at distances far from the treated volume are clinically unimportant and computationally expensive. Discretization parameters are customized to each disease site to optimize performance on the basis of the expected scale lengths of the dose gradients and the clinical range of the dose distribution.

FIG 2.

1D radial PTV shells discretize the dose volume. Strong dose anisotropies are handled by splitting each shell into in-field and out-of-field components, which are treated separately throughout the workflow. An OAR is thus discretized into regions defined by overlaps with the shells and the beam field (ie, OAR shells). Any OAR volume remaining outside the shells is assumed to be low-dose and will be used as a correction factor to the final DVH prediction (see DVH prediction). DVH, dose-volume histogram; OAR, organs-at-risk; PTV, planning target volume.

To account for differing dose gradients found in-field (coplanar region where beams are incident and modulated) and out-of-field (region where radiation is always shielded by collimating elements), each PTV shell is broken into in-field and out-of-field components (Fig 2), which are handled separately throughout the entire kernel build.¹⁰ The free parameter determining the boundary between these regions, nominally 3 mm, is an axial distance from the PTV craniocaudal extrema (Fig 2).

Knowledge-based kernel.

The treatment plan training set is the foundation of the ORBIT-RT knowledge-based kernel.^10,14 In the case of the prostate, 90 past treatment plans (2017 or later) were hand-selected and filtered based on the dual quality criteria of effective Rx dose delivery and OAR sparing. Each training plan contributes a single sub-DVH to the kernel. This sub-DVH χ_ijk (D) is a normalized differential DVH (functionally, a dose probability density) for the k-th shell of the j-th OAR of the i-th treatment plan from the training set.

From the volume discretization, it is possible for an OAR shell to be inordinately small. A sub-DVH from such an overlap would carry equal weight with sub-DVHs from larger overlaps in the kernel calculation because of the volume normalization. Therefore, a minimum volume threshold was introduced as a site-specific free parameter to filter out insignificant overlap volumes. This threshold is nominally set as a relative value equal to 1% of the total OAR volume.

The resultant kernel ${\overline{\text{χ}}}_{\text{jk}}\left(\text{D}\right)$ is an arithmetic mean of all the sub-DVHs χ_ijk (D), calculated binwise for each dose value (Fig 3). This yields one representative curve for the entire training set per OAR shell (and in-field or out-of-field component). These kernels are written to a file, wherein they are used as a linear basis for new DVH predictions by the ORBIT-RT front-end script (see DVH prediction).

FIG 3.

An example of how OAR sub-DVHs contribute to the ORBIT-RT model for the k = 3 shell of a bladder. All sub-DVHs (light blue) from training plans are mapped to the same normalized dose axis. The ORBIT-RT model (dark blue) is the pointwise arithmetic mean of these sub-DVHs. This model kernel is functionally equivalent to the probability density of having a given dose in shell k of OAR j. DVH, dose-volume histogram; OAR, organs-at-risk; ORBIT-RT, Online Real-time Benchmarking Informatics Technology for RadioTherapy.

It is common for several OAR shells not to have training set data, especially if the shell is located far from the OAR. However, it is always possible that a new patient's OAR could overlap a shell previously unobserved in the training set. In such a case, there would be no kernel from which to derive a predicted DVH in that shell. To address this remote possibility, alongside the ORBIT-RT kernel build, a generic phantom kernel is also available, irrespective of the OAR j. This model consists of an isotropically irradiated spherical PTV within a cylindrical water phantom. The kernel is derived from a single set of shell sub-DVHs, ${\text{χ}}_{\text{k}}^{\text{phant}}\left(\text{D}\right)$; that is, the phantom volume from which the sub-DVH is calculated is simply the entire shell. In this way, all shell volumes of the phantom are guaranteed to be nonzero, and as there is only one phantom plan, the phantom kernel ${\overline{\text{χ}}}_{\text{k}}^{\text{phant}}\left(\text{D}\right)$ is exactly the phantom sub-DVH ${\text{χ}}_{\text{k}}^{\text{phant}}\left(\text{D}\right)$. The phantom sub-DVHs and kernels are stored alongside the training set's sub-DVHs and kernels. DVH prediction describes in greater detail how the phantom may be called in DVH prediction.

ORBIT-RT Prediction

DVH prediction.

For DVH predictions, the new patient's OARs are discretized in the exact manner as those of the training patients (see Volume discretization). The volumes of the new OAR shells ${\text{V}}_{\text{jk}}$ function as weighting coefficients to the ORBIT-RT kernels ${\overline{\text{χ}}}_{\text{jk}}\left(\text{D}\right)$. The weighted sum of the kernels over all shells equals the differential DVH for the OAR—or in this case, either the in-field or out-of-field component of the OAR:

$${\frac{\text{dV}}{\text{dD}}}_{\text{j}}\left(\text{D}\right)={\displaystyle \sum }_{\text{k}}{\text{V}}_{\text{jk}}\cdot {\overline{\text{χ}}}_{\text{jk}}\left(\text{D}\right)$$ (1)

The sum of the in-field and out-of-field DVH components is the OAR's total differential DVH, which is then converted into a cumulative DVH.

A patient-specific DVH prediction error estimate is also calculated from the ORBIT-RT kernel. Training sub-DVHs with expected dose lower than the expected dose of the kernel ${\overline{\text{χ}}}_{\text{jk}}\left(\text{D}\right)$ contribute exclusively to a lower error kernel; analogously, sub-DVHs with higher expected dose contribute to an upper error kernel. Consequently, upper and lower error estimates are oppositely biased DVH predictions arising from these error kernels according to the same formulation in Equation 1. In this way, we expect the range between the upper and lower DVH error estimates to resemble an interquartile range although rigorously speaking it is an interval between oppositely biased DVH predictions arising from cleaved subsets of the training set.

Two special cases may arise from new patient OAR volume in unmodeled regions of the dose volume. The first special case occurs when the new OAR is unlike any analogous OAR in the training set, such that a particular OAR shell k has no available kernel. As alluded in knowledge-based kernel, the solution is to substitute the absent knowledge-based kernel for a generic phantom kernel in the affected shell. In this scenario, a warning is signaled to the user, and ORBIT-RT reports the percent OAR volume that used the phantom kernel. The user should decide, based on the percent volume, if the generic phantom contribution represents a significant correction to the DVH prediction. Additionally, it may be prudent to re-examine the accuracy of the OAR contouring.

The second special case occurs when the OAR contour extends beyond the range that one would normally expect to find significant dose (see the OAR remainder in Fig 2), such as a large femur structure in a prostate treatment. As noted in volume discretization, the shell domain is customized for each disease site to encompass the dose volume of expected clinical relevance to OARs. If a structure is contoured beyond the range of all PTV shells, ORBIT-RT records the remainder OAR volume and rescales the DVH contribution of the outermost shell, such that the last shell now additionally includes all the remainder volume. The remainder volume is thus implicitly assumed to have a dose characteristic similar to the last shell, and the cumulative DVH at 0 Gy will reflect the true OAR volume the user observes in the TPS (Fig 4). A warning is again signaled to the user, and ORBIT-RT reports the total OAR volume appended in this way. The user should decide, based on the disease site, if the remainder constitutes a correction affecting clinical decision making.

FIG 4.

OARs contoured far outside the dose volume are likely not to be encompassed by the shell discretization (Fig 3). Such remainder volume is assumed to be low-dose and added artificially to the differential DVH of the last (ie, outermost) OAR shell (upper plot). The remainder volume is distributed in the same manner as the shell volume—according to the kernel. This correction ensures that the cumulative DVH prediction (lower plot) reflects the total OAR volume as quoted in the TPS. Note the relatively low dose range over which this correction differs from the uncorrected DVH. DVH, dose-volume histogram; OAR, organs-at-risk.

Validation and performance.

To validate ORBIT-RT DVH predictions for prostate treatment, a validation set of 45 treatment plans (2017 or later) was tested, and its ORBIT-predicted DVHs were compared with the actual clinical DVHs to assess prediction accuracy. The mean error $\text{μ}\equiv \overline{{\text{V}}_{\text{clin}}-{\text{V}}_{\text{ORBIT}}}$ indicates prediction bias, whereas the standard deviation σ of ${\text{V}}_{\text{clin}}-{\text{V}}_{\text{ORBIT}}$ indicates prediction error uncertainty. A combination of low-bias and low-error uncertainty yields low root-mean-square error (${\text{RMSE}}_{\text{ORBIT}}$) and thus high prediction accuracy:

$${\text{RMSE}}_{\text{ORBIT}}\equiv \sqrt{\frac{1}{\text{N}}{\displaystyle \sum }_{\text{i}}{\left({\text{V}}_{\text{clin},\text{i}}-{\text{V}}_{\text{ORBIT},\text{i}}\right)}^2}\cong \sqrt{{\text{σ}}^2+{\text{μ}}^2}$$ (2)

where

$$\text{σ}\equiv \sqrt{\frac{1}{\text{N}-1}{\displaystyle \sum }_{\text{i}}{\left(\left({\text{V}}_{\text{clin},\text{i}}-{\text{V}}_{\text{ORBIT},\text{i}}\right)-\text{μ}\right)}^2} \text{and }\mu \equiv \overline{{\text{V}}_{\text{clin}}-{\text{V}}_{\text{ORBIT}}}$$

ORBIT-RT DVH calculation time is expected to scale linearly with the number of OARs to process. To quantify this, the full ORBIT-RT workflow was run repeatedly on ten randomly selected plans from the prostate validation set, with an increasing number of OARs to analyze. All the calculation time measurements were grouped as a single dataset, and a least-squares analysis was performed to derive a trendline. Concurrently, total runtimes were recorded for data transfer to the ORBIT-RT server, MIM workflow execution, and data transfer back to the website.

RESULTS

Figure 5 summarizes the results of the prostate model validation exercise (see Validation and performance). The validation set's mean error μ (blue curves, Fig 5) is a measure of ORBIT-RT prediction bias. The standard deviation σ of the error (blue bands, Fig 5) is a measure of ORBIT-RT prediction error uncertainty.

FIG 5.

Mean (curve) and standard deviation (band) of ORBIT-RT OAR DVH prediction error for a prostate validation set (45 plans). Smaller mean and SD indicate less bias and uncertainty, respectively; the combination thereof indicates lower RMSE and greater accuracy. At 40% Rx dose or higher, for all OARs, prediction bias is < 2%, with uncertainty between 5% and 10%. At 80% Rx dose or higher, prediction bias falls to < 1%, with < 2% uncertainty. DVH, dose-volume histogram; OAR, organs-at-risk; ORBIT-RT, Online Real-time Benchmarking Informatics Technology for RadioTherapy.

At 40% Rx dose and higher, μ was < 2% of OAR volume for all OARs, with σ varying by OAR but generally at 5%-10%. At 80% Rx dose and higher, for all prostate OARs, μ fell to < 1% of OAR volume, with σ < 2%. From Equation 2, low systematic bias in combination with low error uncertainty yields high prediction accuracy, with ${\text{RMSE}}_{\text{ORBIT}}$ < 1%-3% of OAR volume above 80% Rx dose.

Figure 6A shows the scaling of ORBIT-RT DVH calculation time with number of OARs for the ten randomly selected prostate plans. As expected, the trend was strongly linear for any given plan, but the slope of the trendline varied from plan to plan. On average, ORBIT-RT required 3.0 additional seconds of runtime per OAR to analyze, and 4.3 seconds of program overhead, owing mostly to PTV shell creation. Regardless, the upward trend in DVH calculation time was small compared with runtime spent in general data transfer and workflow execution (Fig 6B), making the total ORBIT-RT runtime effectively constant. Following initial file upload, it required 90 seconds for the DICOM-RT to arrive at the MIMCloud server, an additional 30 seconds to transfer the DICOM-RT to the MIM image server, and 35 seconds to transfer the completed DVH predictions back to the website (Fig 1). In total, the ORBIT-RT service can be expected to take 2.5-3 minutes for prostate plans.

FIG 6.

Ten prostate plans were tested on ORBIT-RT with various numbers of OARs to process. (A) DVH calculation runtime scales linearly with the number of OARs to analyze, although the scaling varies from patient to patient (dotted lines). A least-squares model (solid line) indicates that each OAR required 3.0 seconds, with 4.3 seconds of program overhead. (B) However, the majority of the total ORBIT-RT runtime (average in blue) is dedicated to the DICOM transfer (average in red) and other aspects of the MIM workflow (average in green), making the runtime effectively constant at 2.5-3 minutes for prostate plans. DICOM, Digital Imaging and Communications in Medicine; DVH, dose-volume histogram; OAR, organs-at-risk; ORBIT-RT, Online Real-time Benchmarking Informatics Technology for RadioTherapy.

DISCUSSION

ORBIT-RT demonstrated fast and fully autonomous knowledge-based feedback on a web-based platform that requires only anonymized DICOM-RT as input. The ORBIT-RT system can be used for real-time quality control feedback that provides users with objective comparisons for final plan DVHs. Clinicians anywhere can, at their own discretion, using only standard DICOM-RT objects common to all modern treatment planning systems, obtain externally benchmarked knowledge-based feedback on individual treatment plans to improve their quality and consistency. ORBIT's efficient performance makes it suitable for real-time clinical use.

ORBIT-RT DVH predictions have been independently validated against past clinical DVHs, showing clinically acceptable accuracy. Limitations to the ORBIT-RT model arising from unusual OAR structures can be addressed with a combination of phantom models and/or DVH corrections (see DVH prediction), with appropriate notifications to the user.

ORBIT-RT has been developed modularly to enable new functionality in the future. Development has already extended into analogous brachytherapy applications, which may be added to the website as a second treatment modality option. Currently, the ORBIT-RT model only handles single-PTV cases, but multi-PTV models can also be incorporated in future versions. In the meantime, new disease site models will continue to be rolled out for single-PTV external beam treatment.

AUTHOR CONTRIBUTIONS

Conception and design: Brent M. Covele, Kartikeya S. Puri, Karoline Kallis, James D. Murphy, Kevin L. Moore

Collection and assembly of data: Brent M. Covele, Kevin L. Moore

Data analysis and interpretation: Brent M. Covele, Karoline Kallis, Kevin L. Moore

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/cci/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).

James D. Murphy

Consulting or Advisory Role: Boston Consulting Group

Research Funding: eContour

Kevin L. Moore

Honoraria: Varian Medical Systems

Research Funding: Varian Medical Systems

Patents, Royalties, Other Intellectual Property: Licensing agreement

No other potential conflicts of interest were reported.