In DEPOSEIN (NCT01645839), intrathecal liposomal cytarabine plus systemic therapy for leptomeningeal metastases (LM) from breast cancer significantly extended LM-related progression-free survival (LM-PFS) compared with systemic therapy alone.1 DEPOSEIN is an accomplishment in effort, design, methods, and transparency. The authors briefly acknowledged problems with selection, imbalance, and response measurement, but the devil is in the details. Here, we comment on the study methods and the strength of evidence behind a positive P-value.
Despite randomization, DEPOSEIN has a high risk of bias. The baseline covariate imbalance is more apparent than what we expect by chance; it is widespread (we identified 10 unbalanced covariates) and weighs more against controls (Table 1). Human epidermal growth factor receptor 2 (HER-2) positivity is unlikely to affect results because of no favorable impact on LM.2 Better Eastern Cooperative Oncology Group (ECOG) scores and less neurologic disability in controls suggest a flawed concealment of the allocation sequence (second-order selection bias).3,4 Moreover, the risk of bias in response measurement is real because of nonvalidated scorecards and lack of central imaging review to minimize cognitive bias.5 We concluded that selection bias subverted randomization in DEPOSEIN. This vulnerability is a well-known plague in randomized controlled trials.6
When and how solid was the clinical benefit in DEPOSEIN? At 2 months, the LM-PFS rate in controls dropped by 46%, and by only 20% in the experimental group. At 4 months, this difference had disappeared (32.5% controls and 30.6% in the intrathecal-treated group). Therefore, the statistical significance in LM-PFS could be due to the faster and higher progression rate in controls within 2 months of enrollment. A cumulative hazard function can support our observation. Finally, how important is a P-value of 0.04? Assuming that bias did not affect the study (DEPOSEIN was vulnerable to second-order selection bias), this P-value and a reduction in the risk of death from LM progression (hazard ratio, 0.61; 95% confidence interval, 0.38‒0.98) support a significant yet imprecise treatment effect. We computed a Bayes factor (BF10) of 5.42 using LM-PFS summary data, which roughly indicates moderate evidence. Is this meaningful for patients and clinicians? With no favorable effect on survival or even on PFS, the answer is unclear to us.
Table 1.
Baseline prognostic covariates with an imbalance in DEPOSEIN (NCT01645839)
| Covariate | Control, n (%) | Experimental, n (%) |
|---|---|---|
| Poor differentiation | 20 (54) | 12 (33) |
| N2 or T4 status | 5 (14) | 2 (6) |
| M1 status | 5 (14) | 8 (22) |
| HER-2+ | 9 (24) | 2 (6) |
| Prior CNS radiotherapy | 10 (27) | 5 (14) |
| Brain metastasis | 15 (41) | 8 (22) |
| ECOG 0 | 7 (19) | 3 (8) |
| ECOG 2–4 | 14 (38) | 18 (50) |
| No neurologic deficit | 11 (30) | 5 (14) |
| Major neurologic deficit | 1 (3) | 4 (11) |
Funding and other potential conflicts of interest
None of the 4 authors have funding sources or other potential conflicts of interest.
References
- 1. Le Rhun E, Wallet J, Mailliez A, et al. Intrathecal liposomal cytarabine plus systemic therapy versus systemic chemotherapy alone for newly diagnosed leptomeningeal metastasis from breast cancer. Neuro Oncol. 2020;22(4):524–538. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Morikawa A, Jordan L, Rozner R, et al. Characteristics and outcomes of patients with breast cancer with leptomeningeal metastasis. Clin Breast Cancer. 2017;17(1):23–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Higgins JPT, Savović J, Page MJ, Elbers RJ, Sterne JAC. Assessing risk of bias in a randomized trial. Cochrane Handboook for Systematic Reviews of Interventions v. 6.0 2020; https://training.cochrane.org/handbook/current/chapter-08. Accessed June 1, 2020.
- 4. Berger VW. Quantifying the magnitude of baseline covariate imbalances resulting from selection bias in randomized clinical trials. Biom J. 2005;47(2):119–127. [DOI] [PubMed] [Google Scholar]
- 5. Busby LP, Courtier JL, Glastonbury CM. Bias in radiology: the how and why of misses and misinterpretations. Radiographics. 2018;38(1):236–247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Kennedy ADM, Torgerson DJ, Campbell MK, Grant AM. Subversion of allocation concealment in a randomised controlled trial: a historical case study. Trials. 2017;18(1):204. [DOI] [PMC free article] [PubMed] [Google Scholar]