Dear Sir,
Rshikesan et al. conducted a parallel randomized controlled trial (RCT) in which participants were assigned to either a yoga or a control condition and effects on sleep and body composition variables were observed.[1] In parallel RCTs, the correct analyses for inferences about treatment effects entail comparing outcomes between the treatment and control groups.[2,3,4,5] Rshikesan et al. conducted the correct analyses and found no statistically significant effects. Therefore, the correct conclusions of the study would have been that the data did not show that the treatment was effective.
Surprisingly, those were not the conclusions reported by Rshikesan et al. Rshikesan et al. incorrectly concluded that “The results indicate the beneficial effects of integrated approach of yoga therapy on body composition and sleep quality in obese males.” They drew these conclusions by mistakenly relying on the fact that some of the within-group changes in outcome variables were statistically significant for the yoga group but not for the control group. Unfortunately, such an analytic strategy is invalid. The invalidity of this approach has been noted repeatedly in the literature.[3,4,5,6,7] The use of this inappropriate analytic approach, referred to as the difference in nominal significance (DINS) error, can result in an inflated Type I (false-positive) error rate as high as 50% (i.e., 0.50 instead of the usual 0.05) when sample sizes are equal across groups.[3,4,5]
Other papers have had to be corrected[8,9,10] or retracted[11,12] because of making the DINS error. Ironically, in this very journal, this same error occurred in a different paper and was noted last year.[13]
Given that the primary conclusions offered in the paper by Rshikesan et al. are incorrect, an erratum or retraction should be issued.[14]
Financial support and sponsorship
This study was financially supported in part by NIH grants R25DK099080 and R25HL124208.
Conflicts of interest
There are no conflicts of interest.
Acknowledgment
Supported in part by NIH grants R25DK099080 and R25HL124208. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or any other organization.
References
- 1.Rshikesan PB, Subramanya P, Singh D. Sleep quality and body composition variations in obese male adults after 14 weeks of yoga intervention: A randomized controlled trial. Int J Yoga. 2017;10:128–37. doi: 10.4103/ijoy.IJOY_53_16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. Br Med J. 2010;340:c869. doi: 10.1136/bmj.c869. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Bland JM, Altman DG. Comparisons against baseline within randomised groups are often used and can be highly misleading. Trials. 2011;12:264. doi: 10.1186/1745-6215-12-264. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Bland JM, Altman DG. Best (but oft forgotten) practices: Testing for treatment effects in randomized trials by separate analyses of changes from baseline in each group is a misleading approach. Am J Clin Nutr. 2015;102:991–4. doi: 10.3945/ajcn.115.119768. [DOI] [PubMed] [Google Scholar]
- 5.George BJ, Beasley TM, Brown AW, Dawson J, Dimova R, Divers J, et al. Common scientific and statistical errors in obesity research. Obesity (Silver Spring) 2016;24:781–90. doi: 10.1002/oby.21449. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Allison DB, Brown AW, George BJ, Kaiser KA. Reproducibility: A tragedy of errors. Nature. 2016;530:27–9. doi: 10.1038/530027a. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Gelman A, Stern H. The difference between “significant” and “not significant” is not itself statistically significant. Am Stat. 2006;60:328–31. [Google Scholar]
- 8.Allison DB, Antoine LH, George BJ. Incorrect statistical method in parallel-groups RCT led to unsubstantiated conclusions. Lipids Health Dis. 2016;15:77. doi: 10.1186/s12944-016-0242-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Allison DB. RE: Statistical Interpretation Error in Metformin Trail Paper; Published online June 25, 2017. 2017. [Last accessed on 2017 Dec 01]. Available from: http://www.pediatrics.aappublications.org/content/early/2017/06/08/peds.2016-4285.comments#re-statistical-interpretation-error-in-metformin-trail-paper . [DOI] [PubMed]
- 10.Allison DB, Williams MS, Hand GA, Jakicic JM, Fontaine KR. Conclusion of “Nordic walking for geriatric rehabilitation: A randomized pilot trial” is based on faulty statistical analysis and is inaccurate. Disabil Rehabil. 2015;37:1692–3. doi: 10.3109/09638288.2014.1002580. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Dimova RB, Allison DB. Inappropriate statistical method in a parallel-group randomized controlled trial results in unsubstantiated conclusions. Nutr J. 2016;15:58. doi: 10.1186/s12937-016-0163-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Cassani RS, Fassini PG, Silvah JH, Lima CMM, Marchini JS. Retraction note: Impact of weight loss diet associated with flaxseed on inflammatory markers in men with cardiovascular risk factors: A clinical study. Nutr J. 2016;15:59. doi: 10.1186/s12937-016-0165-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.George BJ, Goldsby TU, Brown AW, Li P, Allison DB. Unsubstantiated conclusions from improper statistical design and analysis of a randomized controlled trial. Int J Yoga. 2016;9:87–8. doi: 10.4103/0973-6131.171726. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Committee on Publication Ethics. Guidelines for Retracting Articles. 2017. [Last accessed on 2017 Sep 06]. Available from: http://www.publicationethics.org/files/retraction%20guidelines.pdf .