In reading Qiu et al. [1], we found errors that require correction. While researchers may disagree with the inclusion of studies and diets, we retained the study selections from Qiu et al. and focused our corrections on statistical concerns. We address two statistical issues here: (1) not accounting for correlation in cross-over studies, and (2) including multiple diets from the same study without adjustment. Additionally, we noticed data extraction errors. We posted a detailed version of this comment including supporting information at https://osf.io/urb7q/ (files deposited on 23 January 2023).
The first concern is cross-over studies that were included without accounting for correlation within subjects. In cross-over trials, each study participant receives all treatments and serves as their own control. When calculating the treatment effect for a cross-over study, one needs to account for the correlation within subjects [2]. This affects the studies of Baum et al. [3], Bellissimo et al. [4], Kral et al. [5], Leidy & Racki [6], Liu et al. [7], and Mehrabani et al. [8]. We elaborate on our calculations at https://osf.io/urb7q/ (files deposited on 23 January 2023).
The second statistical issue is including multiple treatments from the same study without proper adjustment. Three studies (Bellissimo et al. [4], Kral et al. [5], Mehrabani et al. [5]) included comparisons with more than two treatment or control diets entered as separate studies in the meta-analysis without adjustments. The Cochrane Handbook recommends either combining groups/conditions into a single comparison (recommended), omitting irrelevant groups for the comparison, adjusting sample size in the shared group, or conducting a network meta-analysis [2]. We combined conditions and included one single comparison into our revised meta-analysis (for more details see https://osf.io/urb7q/ (files deposited on 23 January 2023)).
We reanalyzed the data addressing the issues above and correcting data extraction errors. The overall estimate of the unstandardized mean difference in subsequent energy intake between protein-rich and control breakfasts changed from the reported −111.2 kcal to −106.8 kcal (Table 1). We did not extract and reanalyze the data for the relationship with fullness (Figure 5) and hunger (Figure 6) or other analyses in the paper, but we expect similar issues in all analyses. Although the main conclusion does not change considering the association of breakfast protein content with subsequent energy intake, the errors made by Qiu et al. require formal correction for all analyses. Until corrected, the results and conclusions of the current manuscript are unproven [9]. Other readers who notice the issues outlined above might dismiss the results and conclusions due to incorrect analyses. Correcting those errors may prevent similar errors in future studies (for details see https://osf.io/urb7q/ (files deposited on 23 January 2023)).
Table 1.
Associations between protein-rich breakfast and subsequent energy intake (kcal) from a meta-analysis with random effects model.
| Treatment Effect (kcal) | Heterogeneity | |
|---|---|---|
| As reported in Qiu et al. [1] | −111.2 [−145.42, −76.9] | I2 = 67% |
| Reanalysis: Adjusting for correlation in cross-over studies and combine treatments/control if multiple were listed | −106.8 [−130.3, −83.2] | I2 = 51% |
Acknowledgments
We would like to thank the first author of Qiu et al. for providing an electronic copy of the Wang 2014 article when requested.
Author Contributions
B.H. and X.C. reviewed the original papers, re-calculated effects, and noted discrepancies. B.H., X.C., S.L.D., A.W.B. and D.B.A. discussed the errors identified and how to correct and present them. All authors have read and agreed to the published version of the manuscript.
Conflicts of Interest
In the last thirty-six months, Dr. Allison has received personal payments or promises for the same from: Alkermes, Inc.; Amin Talati Wasserman for KSF Acquisition Corp (Glanbia); Big Sky Health, Inc.; Clark Hill PLC; Kaleido Biosciences; Law Offices of Ronald Marron; Medpace/Gelesis; Novo Nordisk Fonden; and Sports Research Corp. Donations to a foundation have been made on his behalf by the Northarvest Bean Growers Association. Indiana University, which employs D.B.A., B.H., X.C. and S.L.D., and the Indiana University Foundation have received funds or donations to support research or educational activities from: the Alliance for Potato Research and Education; American Egg Board; Arnold Ventures; Eli Lilly and Company; Mars, Inc.; National Cattlemen’s Beef Association; Pfizer Pharmaceuticals; USDA; Soleno Therapeutics; WW (formerly Weight Watchers); and numerous other for-profit and non-profit organizations to support the work of the School of Public Health and the university more broadly. In the last thirty-six months, Dr. Brown has received travel expenses from the International Food Information Council and Potatoes USA; speaking honoraria from the Calorie Control Council, Eastern North American Region of the International Biometric Society, Purchaser Business Group on Health, and The Obesity Society; a monetary award from American Society for Nutrition; consulting fees from Pennington Biomedical Research Center, and Soy Nutrition Institute Global; and grants through his current or former institution from Alliance for Potato Research & Education, American Egg Board, National Cattlemen’s Beef Association, NIH/NHLBI, NIH/NIDDK, and NIH/NIGMS. He has been involved in research for which his current or former institution or colleagues have received grants or contracts from the Alliance for Potato Research & Education, Center for Open Science, Gordon and Betty Moore Foundation, Hass Avocado Board, Indiana CTSI, National Cattlemen’s Beef Association, NIH/NCATS, NIH/NCI, NIH/NHLBI, NIH/NIA, NIH/NIGMS, NIH/NLM, and Sloan Foundation. His wife is employed by Reckitt. The remaining authors (B.H., X.C. and S.L.D.) have no conflicts of interest to declare. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.
Funding Statement
This work was funded in part by the National Institutes of Health (NIH): R25HL124208, R25DK099080, and P20GM109096, and the Gordon and Betty Moore Foundation. The opinions expressed are those of the authors and not necessarily of the NIH or any other organization.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
References
- 1.Qiu M., Zhang Y., Long Z., He Y. Effect of Protein-Rich Breakfast on Subsequent Energy Intake and Subjective Appetite in Children and Adolescents: Systematic Review and Meta–Analysis of Randomized Controlled Trials. Nutrients. 2021;13:2840. doi: 10.3390/nu13082840. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Higgins J.P.T., Eldridge S., Li T. Chapter 23: Including variants on randomized trials; Section 23.3.4. In: Higgins J.P.T., Thomas J., Chandler J., Cumpston M., Li T., Page M.J., Welch V.A., editors. Cochrane Handbook for Systematic Reviews of Interventions Version 6.2 (Updated February 2021) Cochrane; 2021. [(accessed on 16 September 2021)]. Available online: www.training.cochrane.org/handbook. [Google Scholar]
- 3.Baum J.I., Gray M., Binns A. Breakfasts higher in protein increase postprandial energy expenditure, increase fat oxidation, and reduce hunger in overweight children from 8 to 12 years of age. J. Nutr. 2015;145:2229–2235. doi: 10.3945/jn.115.214551. [DOI] [PubMed] [Google Scholar]
- 4.Bellissimo N., Fansabedian T., Wong V., de Zepetnek J.T., Brett N., Schwartz A., Cassin S., Suitor K., Rousseau D. Effect of increasing the dietary protein content of breakfast on subjective appetite, short-term food intake and diet-induced thermogenesis in children. Nutrients. 2020;12:3025. doi: 10.3390/nu12103025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kral T.V., Bannon A.L., Chittams J.L., Moore R.H. Comparison of the satiating properties of egg- versus cereal grain-based breakfasts for appetite and energy intake control in children. Eat. Behav. 2016;20:14–20. doi: 10.1016/j.eatbeh.2015.11.004. [DOI] [PubMed] [Google Scholar]
- 6.Leidy H.J., Racki E.M. The addition of a protein-rich breakfast and its effects on acute appetite control and food intake in ‘breakfast-skipping’ adolescents. Int. J. Obes. 2010;34:1125–1133. doi: 10.1038/ijo.2010.3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Liu A.G., Puyau R.S., Han H., Johnson W.D., Greenway F.L., Dhurandhar N.V. The effect of an egg breakfast on satiety in children and adolescents: A randomized crossover trial. J. Am. Coll. Nutr. 2015;34:185–190. doi: 10.1080/07315724.2014.942471. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Mehrabani S., Safavi S.M., Mehrabani S., Asemi M., Feizi A., Bellissimo N., Salehi-Abargouei A. Effects of low-fat milk consumption at breakfast on satiety and short-term energy intake in 10- to 12-year-old obese boys. Eur. J. Nutr. 2015;55:1389–1396. doi: 10.1007/s00394-015-0956-4. [DOI] [PubMed] [Google Scholar]
- 9.Jamshidi-Naeini Y., Golzarri-Arroyo L., Vorland C.J., Brown A.W., Dickinson S., Allison D.B. Accounting for the clustering and nesting effects verifies most conclusions. Corrected analysis of: “Randomized nutrient bar supplementation improves exercise-associated changes in plasma metabolome in adolescents and adult family members at cardiometabolic risk”. PLoS ONE. 2022;17:e0275242. doi: 10.1371/journal.pone.0275242. [DOI] [PMC free article] [PubMed] [Google Scholar]