Cluster randomized trials offer a design that allows researchers to randomize intact groups of subjects to treatment and to analyze groups of individuals where the effects of treatments are confounded with the groups. However, it is crucial that data are analyzed per the unit of randomization to draw valid conclusions.
We read with interest Coknaz et al.’s work [1], where they randomized four schools with three schools allocated to the active video game treatment and one school to the control group. In the data analysis, the authors justified not accounting for a clustered design by saying “There was no difference between the schools in terms of SES due to their public nature. Therefore, each student was assumed to act individually regardless of the attended school.”[1] This leads to two points of concern.
First and most importantly, the authors used only one cluster in the control arm. According to the CONSORT extension for cluster randomized trials, “Trials with one cluster per arm should be avoided as they cannot give a valid analysis, as the intervention effect is completely confounded with the cluster effect”. [2]
The second concern is how the data were analyzed. The authors referenced the CONSORT extension for cluster randomized trials [2], but it contradicts their analytical decision to ignore clustering because it explicitly states “Even if cluster specific characteristics are balanced (that is, characteristics of the randomly allocated clusters), researchers have little control over the participants within each cluster (this is the case whether the number of clusters is large or small).” When clustering and nesting are not accounted for in the analyses, the type I error rate is likely markedly inflated.
Taken together, the data from this study should be re-evaluated and conclusions revised accordingly.
Acknowledgments
Funding
Supported in part by NIH grants R25DK099080 and R25HL124208 and the Gordon and Betty Moore Foundation. The opinions expressed are those of the authors and do not necessarily represent those of the NIH or any other organization.
Disclosures
Dr. Allison has received personal payments or promises for same from: American Society for Nutrition; American Statistical Association; Biofortis; California Walnut Commission; Columbia University; Fish & Richardson, P.C.; Frontiers Publishing; Henry Stewart Talks; IKEA; Indiana University; Laura and John Arnold Foundation; Johns Hopkins University; Law Offices of Ronald Marron; MD Anderson Cancer Center; Medical College of Wisconsin; National Institutes of Health (NIH); Sage Publishing; The Obesity Society; Tomasik, Kotin & Kasserman LLC; University of Alabama at Birmingham; University of Miami; Nestle; WW (formerly Weight Watchers International, LLC). Donations to a foundation have been made on his behalf by the Northarvest Bean Growers Association. Dr. Allison is an unpaid member of the International Life Sciences Institute North America Board of Trustees. Dr. Allison’s institution, Indiana University, has received funds to support his research or educational activities from: NIH; Alliance for Potato Research and Education; American Federation for Aging Research; Dairy Management Inc; Herbalife; Laura and John Arnold Foundation; National Cattlemen’s Beef Association, Oxford University Press, the Sloan Foundation, The Gordan and Betty Moore Foundation, and numerous other for-profit and non-profit organizations to support the work of the School of Public Health and the university more broadly. Dr. Allison’s prior institution, the University of Alabama at Birmingham, received gifts, contracts, and grants from other organizations including the Coca-Cola Company, Pepsi, and Dr. Pepper/Snapple. In the last 12 months, Dr. Brown has received travel expenses from the University of Louisville and grants through his institution from Dairy Management, Inc. and the National Cattlemen’s Beef Association. He has been involved in research for which his institution or colleagues have received grants from the Gordon and Betty Moore Foundation, NIH/NHLBI, NIH/NIA, NIH/NIDDK, and Sloan Foundation. The other authors declare that they have no conflict of interest.
Footnotes
Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of a an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.
References
- [1].Coknaz D, Mirzeoglu AD, Atasoy HI, Alkoy S, Coknaz H, Goral K. A digital movement in the world of inactive children: favourable outcomes of playing active video games in a pilot randomized trial. Eur J Pediatr (2019) 178: 1567 10.1007/s00431-019-03457-x [DOI] [PubMed] [Google Scholar]
- [2].Campbell MK, Piaggio G, Elbourne DR, Altman DG (2012) Consort 2010 statement: extension to cluster randomised trials. BMJ 345:e5661 10.1136/bmj.e5661 [DOI] [PubMed] [Google Scholar]