Change in study randomization allocation needs to be included in statistical analysis: Comment on ‘Randomized controlled trial of weight loss versus usual care on telomere length in women with breast cancer: the lifestyle, exercise, and nutrition (LEAN) study.’

Data are often combined across multiple studies, sites, strata or phases of data collection, for a variety of reasons. In a randomized controlled trial (RCT), employing proper methods when combining data collected in separate contexts ensures unbiased estimates of the combined treatment effect. Collapsing (or “lumping”) data across studies or strata without statistical adjustment can provide misleading results [1], such as occurs in Simpson’s paradox where treatment effects that are consistent across each strata separately are reversed when data are collapsed [2-4]. This paradox occurs specifically when there are differences between the two or more strata (or studies) in the ratio of people in each treatment group [3]. Altman wrote recently of dangers of bias in combining data across studies with varied randomization allocation ratios [5].

Sanft et al. report results from data analysis combining data from an earlier RCT with additional data collected at a later time point, under a different study design, without accounting for these two phases of study in the description of their statistical analysis [6, 7]. The primary study was an RCT with three arms comparing in-person counseling, telephone counseling, and usual care on weight loss, with an equal 1:1:1 randomization allocation [7]. The second phase of the study randomized participants into only two arms: counseling intervention or usual care, with an ostensibly similar 1:1 equal allocation [6]. However, because the two interventions in the primary study were grouped together for analysis [6], the allocation ratio was effectively 2:1 for intervention in the primary study but changed to 1:1 in the later study. This change in allocation ratio can lead to bias [3, 5].

Because biased estimation of treatment effects can result from differences between the two periods, data should be compared and reported between participants in the two periods, by treatment group. For example, baseline BMI may differ between the two recruitment periods in Sanft et al., where significant differences between treatment groups are reported in the combined data [6] but not the primary data [7]. More critically, it is essential that statistical analyses should be adjusted by including study period as a stratification or blocking variable and testing for interactions [1-5, 8]. Bangdiwala et al. discuss multiple options for pooling data across heterogeneous studies [8].

While the overall conclusions in the Sanft et al. paper may remain unchanged, this is an important methodologic issue for researchers to avoid bias in analyses combining data across multiple strata, sites, or phases of data collection in RCTs. We encourage Sanft and colleagues to consider re-analyzing their data taking these factors into account and publish corrected results, and we offer to assist in the re-analysis if needed.