Dear Editor:
We read with interest the recent overviews of milk and dairy product consumption and cardiovascular disease (1) and milk and dairy product consumption and risk of mortality (2). We congratulate and applaud the important work of Fontecha et al. and Cavero-Redondo et al., but several important issues of these 2 overviews should be noted.
First, after reviewing the included systematic reviews, we identified some reviews that address the same research question but share the same primary studies, indicating that the authors did not address the overlapping data of original studies. The authors have meta-analyzed findings from all available published systematic reviews together, although they presented this issue as a limitation in the overview. It would be better to deal with this issue when conducting meta-analyses.
Second, since some systematic reviews may be updated many times, there may be many versions, but the authors did not clarify how to deal with this issue. Instead, all versions were included, which has a great impact on the credibility of the results.
Third, these 2 overviews used the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) (3) to evaluate the risk of bias of each included systematic review, and ranked each systematic review as excellent, 15–16 points; very good, 12–14 points; good, 9–11 points; acceptable, 6–8 points; or deficient, 3–5 points. However, the responses to the AMSTAR 2 items should not be used to derive an overall score, because an overall score may disguise critical weaknesses that should diminish confidence in the results of a systematic review (3–5). Furthermore, the developers of AMSTAR 2 recommend that users adopt the rating process based on the identification of critical domains and provide a solution for assessing the quality of systematic reviews based on critical and noncritical items. The AMSTAR 2 contains 16 items, among which 7 are critical domains. Based on the critical and noncritical items, the overall confidence of the quality of the review is rated into 4 levels, high, moderate, low, and critically low. Therefore, the authors should not use their own scoring criteria to rank the quality of the included systematic reviews. We believe that addressing this issue could further increase the value of their studies.
Notes
Author disclosures: YG, YS, JC, and JT, no conflicts of interest.
References
- 1. Fontecha J, Calvo MV, Juarez M, Gil A, Martínez-Vizcaino V. Milk and dairy product consumption and cardiovascular diseases: an overview of systematic reviews and meta-analyses. Adv Nutr. 2019;10:S164–89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Cavero-Redondo I, Alvarez-Bueno C, Sotos-Prieto M, Gil A, Martinez-Vizcaino V, Ruiz JR. Milk and dairy product consumption and risk of mortality: an overview of systematic reviews and meta-analyses. Adv Nutr. 2019;10:S97–104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E et al.. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Greenland S, O'Rourke K.. On the bias produced by quality scores in meta-analysis, and a hierarchical view of proposed solutions. Biostatistics. 2001;2:463–71. [DOI] [PubMed] [Google Scholar]
- 5. Jüni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;282:1054–60. [DOI] [PubMed] [Google Scholar]