The Body of Evidence for Advanced Technology in Radiation Oncology

Proton therapy has struck a nerve in the national dialogue about cancer comparative effectiveness research in the United States. Unlike photon-based radiotherapy, proton therapy delivers radiation within a finite range, depositing dose in a tumor target with essentially no residual radiation beyond the tumor. Proton therapy radiation dose distributions often appear superior to photon-based treatments like intensity-modulated radiotherapy (IMRT) and three-dimensional conformal radiotherapy, particularly in the reduction of low and intermediate radiation dose to normal tissues.

However, proton therapy has greater intrinsic uncertainties than photon-based treatments, both biological and physical. For example, uncertainty exists about where the finite range of protons terminates in tissue; to compensate, proton treatment centers routinely overshoot tumor targets to ensure adequate radiation coverage (1). Proton therapy uncertainties and methods by which such uncertainties are mitigated could impact important clinical outcomes. Thus, the comparative effectiveness test for proton therapy is whether it leads to incremental reductions in morbidity or improvements in survival and disease control, not whether the dose distribution looks better or mechanism of radiation delivery is novel (2).

In this issue of the Journal, Yu et al. (3) report the results of a retrospective observational comparison of proton therapy to IMRT for prostate cancer using 2008 and 2009 Medicare claims data from the Chronic Condition Warehouse, a national database of 100% of Medicare fee-for-service claims. Among men aged greater than 65 years, the investigators found that proton therapy was associated with a small reduction in genitourinary complications at 6 months relative to IMRT (5.9% vs 9.5%, respectively), but they found no statistically significant difference at 12 months. There was no statistically significant difference in gastrointestinal or other complications at 6 or 12 months.

Moreover, the median Medicare reimbursement for proton therapy patients was $32428, whereas the median for IMRT patients was $18575. This report is the most comprehensive study to date on outcomes of proton therapy and IMRT among US men aged greater than 65 years—a population that comprises more than two-thirds of all men who receive radiation for prostate cancer—and reports marginal and transient reductions, at best, in acute genitourinary complications with proton therapy at substantially increased treatment costs.

The Yu et al. report (3) is an example of careful observational comparative effectiveness research (CER) and augments the growing observational evidence base for advanced radiotherapy technologies. However, three notable systematic biases may threaten the validity of nonrandomized studies and are particularly pertinent to claims-based radiotherapy CER: 1) outcome misclassification, 2) exposure misclassification, and 3) uncontrolled confounding (4).

Outcome misclassification is endemic to CER that uses large administrative datasets because few studies have validated claims-based algorithms to serve as surrogates for clinical toxicity endpoints such as radiotherapy-induced bowel, urinary, or other complications. Two previous claims-based studies of proton therapy reached materially different conclusions from the Yu et al. report: that relative to IMRT, proton therapy was associated with increased long-term bowel complications and with no statistically significant difference in urinary complications (5,6). Yet, these studies and the Yu et al. report defined complications using such different methodologies and billing codes that a comparison of effect estimates between the three studies is not possible. Without studies to validate the surrogacy of claims-based endpoints, outcome misclassification could lead to false-negative or false-positive results.

Exposure misclassification is also a major challenge in claims-based radiotherapy CER. For example, patients assigned to the proton therapy group may have received a portion of their treatment with photon-based therapy or those assigned to the IMRT group may have actually received three-dimensional conformal radiotherapy instead (three-dimensional conformal radiotherapy was employed in 10% to 15% of patients during the study period). Even in claims datasets that track the type and number of radiotherapy fractions delivered, misclassification can occur because radiotherapy dose, field, image guidance, and delivery quality are not captured.

Uncontrolled confounding could also impact the findings reported by Yu et al. (3). An underlying assumption of any statistical matching (or adjustment) technique is that controlling for measured confounders also controls unmeasured confounders. However, patients in the proton therapy group were not only younger and healthier than those in the IMRT group but also traveled a great distance for therapy. Although age and comorbidity were included in the matching algorithm, travel distance was not, but likely correlates with unmeasured factors like frailty, baseline urinary or bowel dysfunction, or patient preferences. These factors plausibly influence both treatment selection and the outcomes under study and could explain the observed differences in acute urinary morbidity.

Despite limitations, observational studies can play an important role in radiotherapy CER (7). What can be done to minimize these systematic biases? First, validation studies are necessary to demonstrate the sensitivity and specificity of claims-based algorithms to detect important radiotherapy outcomes and exposures across disease sites. Second, equipment manufacturers should develop unique technology identifiers for radiotherapy devices and image guidance that would facilitate identification in registries or possibly claims data. For example, the proposed National Radiation Oncology Registry will ascertain finer details on radiotherapy delivery quality at a population level through an integrated electronic infrastructure (8). Third, investigators should explore novel, prospective CER clinical trial designs. Randomization is the most appropriate study design to minimize confounding when investigators hypothesize modest to moderate differences in outcomes between treatment exposures (as with proton therapy for prostate cancer); large-scale, pragmatic, randomized trials or parallel, randomized and observational cohort studies can extend the generalizability of traditional randomized studies (9).

Is a randomized trial of proton therapy vs IMRT worth the costs? A rough calculation of the incremental health-care expenditures associated with replacing IMRT with proton therapy for even just one-third of the nearly 28000 Medicare beneficiaries who received treatment in 2008 and 2009 would be at least $100 million of excess spending. The costs of a randomized trial that would compare the two radiation modalities range from $5 to $15 million. For such a scientifically important question in radiotherapy CER, a randomized trial of proton therapy vs IMRT would appear to be a good investment for patients and clinicians. The University of Pennsylvania and the Massachusetts General Hospital have partnered with other centers to conduct this randomized trial. Similar efforts, combined with important findings from Yu et al. (3), will continue to build the body of evidence for advanced radiotherapy technologies.

Funding

JEB is supported by the National Cancer Institute (1K07CA16316-01).

The study sponsors had no role in the writing of the editorial or the decision to submit it for publication.