Abstract
Objective
Re-randomization trials allow patients to be re-enrolled for multiple treatment episodes. However, it remains uncertain to what extent re-randomization improves recruitment compared to parallel group designs or whether treatment estimates might be affected.
Study Design and Setting
We evaluated trials included in a recent Cochrane review of granulocyte colony-stimulating factors for patients with febrile neutropenia. We assessed the recruitment benefits of re-randomization trials; compared treatment effect estimates between re-randomization and parallel group designs; and assessed whether re-randomization led to higher rates of non-compliance and loss to follow-up in subsequent episodes.
Results
We included 14 trials (5 re-randomization and 9 parallel group). The re-randomization trials recruited a median of 25% (range 16–66%) more episodes on average than they would have under a parallel-group design. Treatment effect estimates were similar between re-randomization and parallel group trials across all outcomes, though confidence intervals were wide. The re-randomization trials in this review reported no loss to follow-up and low rates of non-compliance (median 1.7%, range 0–8.9%).
Conclusions
In the setting of febrile neutropenia, re-randomization increased recruitment while providing similar estimates of treatment effect to parallel group trials, with minimal loss to follow-up or non-compliance. It appears to be safe and efficient alternative to parallel group designs in this setting.
Keywords: Re-randomization, Randomized controlled trials, Clinical trials, Febrile neutropenia, Re-enrolment, Poor recruitment
What is new?
-
•
Previous work has shown that re-randomization trials can provide unbiased estimates of treatment effect and increase patient recruitment, but this has never been evaluated empirically.
-
•
Our review found that re-randomization trials increased patient recruitment while providing similar estimates of treatment effect to parallel group designs, with minimal loss to follow-up or non-compliance.
-
•
Re-randomization appears to offer a safe and efficient alternative to parallel group trials.
1. Background
Febrile neutropenia occurs when neutropenic patients (those with abnormally low neutrophil granulocyte counts) develop fever. It is often a complication for patients with cancer who receive chemotherapy regimens which suppress bone marrow activity. Because chemotherapy is usually given in multiple cycles, patients may develop febrile neutropenia multiple times during the course of their cancer treatment, and each episode of febrile neutropenia would require medical intervention. Standard care for febrile neutropenia is broad-spectrum antibiotics [1]. However, it has been suggested that granulocyte colony-stimulating factor (G-CSF) could be useful in this setting, as it regulates the production of the neutrophil lineage [1]. A number of clinical trials have compared the use G-CSF with antibiotics vs. antibiotics alone in patients with febrile neutropenia.
In a parallel group trial, patients would be enrolled for one episode of febrile neutropenia only; if they experienced further episodes of febrile neutropenia, they would no longer be eligible to participate in the trial. This approach can be inefficient, as a large proportion of febrile neutropenia episodes may be ineligible for the trial, which can affect recruitment. The majority of trials in this area have recruited fewer than 50 patients per treatment arm [1], which would lead to underpowered analyses for important outcomes such as mortality.
An alternative approach is a re-randomization trial (Fig. 1) [2], [3], [4]. In re-randomization trials, patients can be re-enrolled and re-randomized for each new episode of febrile neutropenia they experience. The number of times each patient is enrolled in the trial is not specified in advance, but instead depends on the number of febrile neutropenia episodes they experience during the course of the trial; some patients may be enrolled only once, and others may be enrolled multiple times. Because patients can be enrolled for multiple episodes, re-randomization can increase the recruitment rate compared to parallel group designs, which could facilitate quicker and more efficient trials [2], [3].
Fig. 1.
Re-randomization vs. parallel group trials. This figure depicts the treatment episodes occurring during the trial recruitment period that are eligible for enrollment under a parallel group and re-randomization design. Gray episodes denote the patient was not eligible, A = allocated to treatment A, B = allocated to treatment B.
However, there has been little empirical evaluation of re-randomization trials, and so, it is unclear how much of a recruitment benefit might be expected in practice or whether treatment effect estimates from re-randomization trials might differ to those from parallel group designs. Furthermore, there may be concern that repeated enrollments in re-randomization trials may place undue burden on patients due to increased treatment or follow-up burden and may lead to higher rates of non-compliance or loss to follow-up in subsequent enrollments. We there undertook a review of trials in febrile neutropenia to evaluate (1) the impact re-randomization had on recruitment; (2) whether treatment effect estimates from re-randomization trials were different to those from parallel group trials; and (3) whether re-randomization led to higher rates of non-compliance and loss to follow-up in subsequent episodes.
2. Methods
2.1. Overview of re-randomization trials
We begin by providing a brief overview of the re-randomization design (Table 1). This design is appropriate in settings where at least some patients may require treatment on multiple occasions, and in practice, the intervention(s) under study would be used for each new treatment episode that occurred [2], [3]. Furthermore, the duration of the intervention and the length of the patient follow-up period must be less than the overall length of the trial recruitment period [2], [3]. This design is therefore suitable in the setting of febrile neutropenia, as some patients experience multiple episodes and require treatment for each episode, and the intervention (G-CSF) and patient follow-up duration are typically short-term.
Table 1.
Overview of re-randomization trials
| Settings requirements for re-randomization trials |
|
| Design requirements for re-randomization trials |
|
| Implementation of re-randomization trials |
|
Reproduced from Kahan BC. Using re-randomization to increase the recruitment rate in clinical trials–an assessment of three clinical areas. Trials. 2016;17:595. doi:10.1186/s13063-016-1736-z with permission.
There are two core design requirements for re-randomization trials [2], [3]; (1) patients are only re-enrolled and re-randomized after the follow-up period from their previous enrollment is complete (i.e., there cannot be overlapping follow-up periods from different enrollments); and (2) randomizations for the same patient are performed independently (e.g., patients are not forced to crossover from one treatment arm to another between episodes).
Analysis of re-randomization trials can be via an “independence” analysis [2], where each episode is analyzed independently (i.e., the correlation between episodes from the same patient is ignored in the analysis). This approach can provide unbiased estimates and correct type I error rates [2]. It will also provide the same power as a parallel group design with an equivalent number of observations in many settings, provided the overall variance is not increased through the use of re-randomization; further details are available in another article [2]. Therefore, in these settings, the same sample size calculation as in a parallel group design could be used; however, instead of recruiting the required number of patients, the re-randomization trial could recruit the required number of treatment episodes. For example, if the sample size calculation for a parallel group trial required 100 patients, a re-randomization trial would require 100 episodes of febrile neutropenia from fewer patients, for example, 100 episodes from 75 patients (where 50 patients contributed one episode each, and 25 patients contributed two episodes each) or some other combination.
2.2. Review of trials in febrile neutropenia
We used the Cochrane Collaboration systematic review and meta-analysis conducted by Mhaskar et al. [1] as the basis of our study. Full details of the search strategy, inclusion criteria, and data collection procedure are available in their publication [1]. The review included 14 trials; 9 used a parallel-group design, and 5 allowed re-randomization [5], [6], [7], [8], [9]. Information was extracted by two reviewers (B.C.K and T.P.M.), and discrepancies were resolved through discussion.
2.3. Impact of re-randomization on recruitment
For the five trials using re-randomization, we extracted information on the number of individual patients enrolled in the trial and the number of treatment episodes enrolled. Based on this information, we estimated the increased sample size obtained through the use of re-randomization for each trial by dividing the total number of treatment episodes by the number of individual patients. This measure represents the extra number of episodes each trial gained by using a re-randomization design instead of a parallel group design.
2.4. Difference in treatment effect estimates between re-randomization and parallel group trials
For each trial, we extracted the treatment effect estimate and 95% confidence interval (95% CI) for each outcome reported. In our analysis, we only included outcomes that were available for at least two re-randomization and two parallel group trials.
For each outcome, we used the Stata package metareg to conduct a random-effects meta-regression model to estimate the difference in treatment effect estimates between re-randomization and parallel group trials [10]. Treatment effect estimates were log(hazard ratio) for time-to-event outcomes, log(risk ratio) for binary outcomes, and standardized mean differences for continuous outcomes. A negative difference in effect sizes indicates that re-randomization trials show a more beneficial treatment effect than parallel group trials.
Two re-randomization trials did not have any events for overall mortality, and two trials (one re-randomization and one parallel group) did not have any events for infection-related mortality. These trials were excluded from the analysis of these outcomes, as it was impossible to estimate either a treatment effect or standard error. One parallel group trial included three treatment arms (two active and one control) and involved two treatment comparisons (both active interventions vs. control). The meta-analysis by Mhaskar et al. [1] included both treatment comparisons in the analysis as separate trials; we have used the same approach here.
2.5. Treatment compliance and loss to follow-up in re-randomization trials
In order to assess whether re-randomization led to higher rates of non-compliance or loss to follow-up in later episodes, we extracted data on the number of treatment episodes which did not comply with the treatment protocol, and for outcomes reported by two or more re-randomization trials, the number of treatment episodes excluded from the analysis due to missing data. We attempted to extract these data separately for each episode, to assess whether non-compliance and missing data were higher in later episodes. However, this information was not reported in any trial, and so, we extracted the overall rates of non-compliance and loss to follow-up across all episodes.
2.6. Characteristics of re-randomization trials
For the five trials employing re-randomization, we extracted information on characteristics related to the re-randomization aspects of the trials. This included information on the number of treatment episodes for each patient, how the sample size calculation was conducted, and how the analysis was conducted.
3. Results
3.1. Impact of re-randomization on recruitment
Results are shown in Fig. 2. Among the five trials using re-randomization, the median number of individual patients recruited was 40 (range 28–112) and the median number of episodes of febrile neutropenia enrolled was 58 (35–186). The median increase in the sample size obtained through the use of re-randomization was 25% (range 16–66%), indicating that using a re-randomization design allowed trials to recruit between 16% and 66% more episodes of febrile neutropenia than they would have under a parallel group design.
Fig. 2.
Percentage increase in number of episodes recruited due to re-randomization.
3.2. Difference in treatment effect estimates between re-randomization and parallel group trials
Differences in treatment effect estimates between re-randomization and parallel group trials are shown in Fig. 3. Treatment effect estimates for the two designs were similar for each of the five outcomes, and none of the differences were statistically significant. However, CIs were wide, indicating that differences were possible.
Fig. 3.
Difference in effect sizes between re-randomization and parallel group trials. The blue lines represent the estimated treatment effect and 95% CI from the re-randomization and parallel group trials. The red lines represent the difference in the treatment effect estimates between re-randomization and parallel group trials (and a 95% CI for this difference). If the red line is close to 0, it means that re-randomization and parallel group trials are providing similar estimates of treatment effect; if it is far away from 0, then re-randomization and parallel group trials are giving different estimates of treatment effect. The x-axis shows the size of the effect for both the blue and red lines; however, the x-axis text (whether re-randomization or parallel group trials show more beneficial effects) applies only to the red line. RR, re-randomization; PG, parallel group; CI, confidence interval; Std. Mean Diff., standardized mean difference. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)
3.3. Treatment compliance and loss to follow-up in re-randomization trials
Results are shown in Table 2. The median percentage of episodes which were not compliant with the protocol was 1.7% (range 0–8.9%). None of the five outcomes we assessed excluded any episodes from the analysis due to missing data.
Table 2.
Compliance and loss to follow-up in re-randomization trials
| Number of trials reporting measure | Median (range) | |
|---|---|---|
| Percent of episodes not complying with treatment protocol | 5 | 1.7 (0–8.9) |
| Percent of episodes excluded from analysis due to missing data | ||
| Overall mortality | 5 | 0 (0–0) |
| Infection-related mortality | 4 | 0 (0–0) |
| Hospitalization >10 days | 3 | 0 (0–0) |
| Duration of grade IV neutropenia | 2 | 0 (0–0) |
| Time to recovery from fever | 3 | 0 (0–0) |
3.4. Characteristics of re-randomization trials
Results are shown in Table 3. None of the five trials explicitly stated whether randomizations for the same patient were independent or that patients were only re-enrolled once the follow-up period from their previous enrollment was complete.
Table 3.
Design and analysis characteristics of re-randomization trials
| Re-randomization trials (n = 5) | |
|---|---|
| Explicitly stated that randomizations for the same patient were independent | |
| No | 5 (100) |
| Yes | 0 (0) |
| Explicitly stated that patients were only re-randomized when the follow-up period from their previous enrollment was complete | |
| No | 5 (100) |
| Yes | 0 (0) |
| Placed limit on maximum number of times each patient could be enrolled in the trial? | |
| No | 0 (0) |
| Yesa | 1 (20) |
| Not reported | 4 (80) |
| Based sample size calculation on a parallel group design | |
| No | 0 (0) |
| Yes | 3 (60) |
| No sample size calculation reported | 2 (40) |
| Analyzed data on a per-episode or per-patient basis | |
| Per-episode | 5 (100) |
| Per-patient | 0 (100) |
| Both | 0 (100) |
| Did analysis account for correlation between treatment episodes from the same patient? | |
| No (independence analysis) | 5 (100) |
| Yes (e.g., GEE or mixed-effects models) | 0 (0) |
| Did analysis adjust for any factors associated with re-randomization, such as episode number? | |
| No | 5 (100) |
| Yes | 0 (0) |
| Reported the number of treatment episodes for each patientb | |
| No | 2 (40) |
| Yes | 3 (60) |
One trial used a limit of four treatment episodes.
In two trials, all patients were enrolled for one or two treatment episodes. In the third trial, most patients were enrolled for one or two treatment episodes, and a small proportion of patients were enrolled for three or more episodes.
Of the trials that reported a sample size calculation, each based their calculation on a parallel group design. All trials analyzed the data on a per-episode basis and all used an independence analysis (i.e., ignored correlation between episodes from the same patient), and none of the trials adjusted for any factors associated with re-randomization in the analysis (e.g., whether it was the patient's first or second time enrolled in the trial).
4. Discussion
We found that using re-randomization in trials of febrile neutropenia increased the number of episodes recruited by between 16% and 66%. Furthermore, we found that re-randomization trials provided similar estimates of treatment effect to parallel group designs. Given that many trials face challenges in recruitment [11], [12], [13], [14], re-randomization may be a viable option in appropriate settings to facilitate more efficient recruitment than parallel group designs, while providing similar results. We found that re-randomization trials had minimal non-compliance and no loss to follow-up, indicating that increased patient burden due to trial re-enrollment was not an issue in this setting. This may be because most outcomes were recorded by the trial team, rather than by the patients themselves.
Most re-randomization trials were designed and analyzed using very simple approaches; they used the same sample size calculation and analysis method as would be used in a parallel group trial, except instead of recruiting and analyzing patients, they recruited and analyzed episodes. This approach will lead to adequately powered trials in many settings [2] and can provide unbiased estimates of treatment effect and correct type I error rates [2]. Therefore, using a re-randomization design rather than a parallel group trial design does not necessarily require additional methodological complexity and can be done in a very simple way.
This is the first review to assess the use of re-randomization trials in practice. However, there were some limitations. There were only a small number of trials available. This led to wide CIs for the differences in treatment effect estimates, meaning that we could not rule out differences between designs. We also focused only on one clinical area, and so, these results may not be generalizable to other settings. Furthermore, reporting of key trial characteristics in re-randomization trials was often inadequate, which may in part reflect a lack of guidance on good reporting practice.
5. Conclusions
In the setting of febrile neutropenia, re-randomization increased recruitment while providing similar estimates of treatment effect to parallel group trials, with minimal loss to follow-up or non-compliance. It appears to be safe and efficient alternative to parallel group designs in this setting.
Acknowledgments
Table 1 was reproduced from Kahan BC. Using re-randomization to increase the recruitment rate in clinical trials—an assessment of three clinical areas. Trials. 2016; 17:595. doi:10.1186/s13063-016-1736-z with permission.
Authors' contributions: B.C.K. concepted the study, developed review, performed data extraction, and wrote the manuscript. T.P.M. developed review, performed data extraction, and contributed to the manuscript. E.H. developed review and contributed to the manuscript. R.P., R.H., and S.E. contributed to the manuscript.
Footnotes
Funding: B.C.K. is funded by a National Institute for Health Research Doctoral Research Fellowship (DRF-2016-09-005). This article presents independent research funded by the National Institute for Health Research (NIHR). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. This work was supported by the Medical Research Council [Morris, grant numbers C_UU_12023/21 and MC_UU_12023/29].
Conflict of interest: The authors declare they have no conflicts of interest.
References
- 1.Mhaskar R., Clark O.A., Lyman G., Engel Ayer Botrel T., Morganti Paladini L., Djulbegovic B. Colony-stimulating factors for chemotherapy-induced febrile neutropenia. Cochrane Database Syst Rev. 2014;(10):CD003039. doi: 10.1002/14651858.CD003039.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Kahan B.C., Forbes A.B., Dore C.J., Morris T.P. A re-randomisation design for clinical trials. BMC Med Res Methodol. 2015;15:96. doi: 10.1186/s12874-015-0082-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kahan B.C. Using re-randomization to increase the recruitment rate in clinical trials - an assessment of three clinical areas. Trials. 2016;17(1):595. doi: 10.1186/s13063-016-1736-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Dunning A.J., Reeves J. Control of type 1 error in a hybrid complete two-period vaccine efficacy trial. Pharm Stat. 2014;13(6):397–402. doi: 10.1002/pst.1649. [DOI] [PubMed] [Google Scholar]
- 5.Anaissie E.J., Vartivarian S., Bodey G.P., Legrand C., Kantarjian H., Abi-Said D. Randomized comparison between antibiotics alone and antibiotics plus granulocyte-macrophage colony-stimulating factor (Escherichia coli-derived in cancer patients with fever and neutropenia. Am J Med. 1996;100:17–23. doi: 10.1016/s0002-9343(96)90006-6. [DOI] [PubMed] [Google Scholar]
- 6.Lopez-Hernandez M.A., Jimenez-Alvarado R., Borbolla-Escoboza R., de Diego Flores-Chapa J., Alvarado-Ibarra M., Gonzalez-Avante M. [Granulocyte colony-stimulating factor in the treatment of febrile neutropenia] Gac Med Mex. 2000;136(2):99–105. [PubMed] [Google Scholar]
- 7.Mitchell P.L., Morland B., Stevens M.C., Dick G., Easlea D., Meyer L.C. Granulocyte colony-stimulating factor in established febrile neutropenia: a randomized study of pediatric patients. J Clin Oncol. 1997;15:1163–1170. doi: 10.1200/JCO.1997.15.3.1163. [DOI] [PubMed] [Google Scholar]
- 8.Riikonen P., Saarinen U.M., Makipernaa A., Hovi L., Komulainen A., Pihkala J. Recombinant human granulocyte-macrophage colony-stimulating factor in the treatment of febrile neutropenia: a double blind placebo-controlled study in children. Pediatr Infect Dis J. 1994;13(3):197–202. doi: 10.1097/00006454-199403000-00006. [DOI] [PubMed] [Google Scholar]
- 9.Rodriguez Z.N., Tordecilla C.J., Campbell B.M., Joannon S.P., Rizzardini L.C., Soto A.V. [Usefulness of G-CSF in pediatric high risk cancer patients with fever and neutropenia] Rev Chilena Infectol. 2005;22(3):223–237. doi: 10.4067/s0716-10182005000300001. [DOI] [PubMed] [Google Scholar]
- 10.Sterne J.A., Juni P., Schulz K.F., Altman D.G., Bartlett C., Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in “meta-epidemiological” research. Stat Med. 2002;21:1513–1524. doi: 10.1002/sim.1184. [DOI] [PubMed] [Google Scholar]
- 11.Carlisle B., Kimmelman J., Ramsay T., MacKinnon N. Unsuccessful trial accrual and human subjects protections: an empirical analysis of recently closed trials. Clin Trials. 2015;12:77–83. doi: 10.1177/1740774514558307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.McDonald A.M., Knight R.C., Campbell M.K., Entwistle V.A., Grant A.M., Cook J.A. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7:9. doi: 10.1186/1745-6215-7-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Sully B.G., Julious S.A., Nicholl J. A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies. Trials. 2013;14:166. doi: 10.1186/1745-6215-14-166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Williams R.J., Tse T., DiPiazza K., Zarin D.A. Terminated trials in the ClinicalTrials.gov results database: evaluation of availability of primary outcome data and reasons for termination. PLoS One. 2015;10(5):e0127242. doi: 10.1371/journal.pone.0127242. [DOI] [PMC free article] [PubMed] [Google Scholar]