Randomization, But Without Comparison: A Missed Opportunity

We read with great interest the phase II PANACHE01-PRODIGE48 trial results reported by Schwarz et al.¹ The authors should be commended for randomizing 153 patients with resectable pancreatic adenocarcinoma to neoadjuvant mFOLFIRINOX, neoadjuvant FOLFOX, or upfront surgery. In doing so, they sought to answer the important question of whether adding neoadjuvant systemic therapy would be beneficial in this patient population. Although this undertaking deserves commendation, herein we reflect on the so-called “randomized, non-comparative trial” design of this trial and raise the question of whether this design limits the value of the evidence provided.

In the PANACHE01-PRODIGE48 trial, the primary objective was to determine whether the 12-month survival probability of and feasibility of each individual neoadjuvant systemic therapy arm exceeded certain point thresholds of interest, based on what would be considered clinically acceptable. However, the primary reason for randomizing patients in a clinical trial is to facilitate comparisons of randomized arms.² Randomization eliminates systematic confounding factors affecting treatment choice, a goal also achievable by using other allocation methods that remove treatment decision-making from participants and patients.³ The unique value of randomization lies in its ability to provide a robust statistical basis for reliably estimating the uncertainty of comparative treatment effects between randomized groups.^2,3 Consequently, randomized non-comparative designs, which formally prohibit comparative inferences, fundamentally undercut the primary value added by the randomization.⁴ In this context, the contribution of data from patients enrolled on the surgical arm is also not immediately evident.

The definition of the primary endpoint in the PANACHE01-PRODIGE48 trial also requires that the individual treatment effects in each arm are reliably estimated. In fact, without robust transportability assumptions, estimating treatment-group specific inferences can be exceedingly challenging in randomized trials, as randomized trials typically and intentionally enroll non-representative patient samples that do not correspond well to any underlying general patient population.⁵ Consequently, the confidence intervals for 12-month survival probability in each arm in the PANACHE01-PRODIGE48 trial have far less meaning than would the confidence interval for the comparative treatment effect between arms.^3,6 The limitations of such inferences have been well recognized by methodologists, including the CONSORT guidelines which directly advocate against treatment-group specific inferences, although such inferences do commonly appear in modern oncology trials.^3,6 The U.S. Food and Drug Administration further advises against comparisons of individual arms with historical controls, particularly for time-to-event outcomes like overall survival.⁷ Thus, interpretation of the primary endpoint of the PANACHE01-PRODIGE48 trial is as challenging as that for any single-arm trial.

Lastly, despite the design label of a non-comparative trial, comparisons between arms are both implicit and explicit in the article by Schwarz and colleagues, such as the presentation of Kaplan-Meier survival curves and Cox regression results in the appendix. In a randomized non-comparative trial, such comparisons are by definition post-hoc. Post-hoc analyses are often associated with presentation bias and generally discouraged in randomized trial reports, although such comparisons are quite common in randomized non-comparative trials and are difficult to interpret.⁶

If the overarching goal of the PANACHE01-PRODIGE48 trial was to understand which neoadjuvant systemic therapy regimen should be evaluated against upfront resection in subsequent phase III trials, we posit that a randomized trial comparing the two neoadjuvant systemic therapy regimens would have lent more interpretable insights. If estimation of treatment-group specific effects were the primary question of interest, inferences from a single-arm study that emphasized representative sample selection and modeling would be more reliable to interpret. Altogether, these notable limitations apply to other randomized non-comparative trials of patients with resectable pancreatic cancer such as the NEONAX trial, the SWOG S1505 trial, and the JASPAC-04 trial.^8–10 Overall, the myriad drawbacks of randomized non-comparative trials should give the trialist community substantial pause in the use of such designs in pancreatic cancer trials.