Reply to X Chen et al.

Dear Editor:

We thank Chen et al. for their interest in reading our work entitled “Effects of a Paleolithic Diet on Cardiovascular Disease Risk Factors: A Systematic Review and Meta-Analysis of Randomized Controlled Trials.” In that study (1), we evaluated the effects of a Paleolithic diet on cardiovascular disease risk factors using data from randomized controlled trials. However, Chen et al. raised some questions that must be addressed.

According to their classification regarding possible questions, we answer those questions in the same order accordingly:

1. Discrepancies in effect sizes reported:

   1. In the study of Irish et al. (2), only the baseline data, but not after-intervention data, were reported in Table 1. Only the percentage change in the mean was reported in the Result section, meaning that this percentage cannot be used to calculate the SE. Therefore, data on C-reactive protein were extracted from Figure 5E in that article using Plot Digitizer (http://plotdigitizer.sourceforge.net/). This kind of figure-based estimation is a routine procedure to derive the means and their variations.

   2. In all included studies in our meta-analysis, the unit of lipids was mmol/L except for the study of Masharani et al. (3) and the unit conversion from mg/dL to mmol/L was performed for this study. Therefore, the correct unit of lipids is mmol/L, whereas unfortunately it has been reported in mg/dL for lipid markers. The effect size of −0.20 mmol/L for triglycerides is correct. However, the findings from reanalysis did not tangibly change from those reported in the article [weighted mean difference (WMD) = −0.22 mmol/L; 95% CI: −0.43, −0.02 mmol/L; P = 0.031 compared with WMD = −0.24 mmol/L; 95% CI: −0.46, −0.01 mmol/L; P = 0.037] (Figure 4B). We thank Chen et al. for their precise point.

   3. As observed in Figure 4B, we used a mean difference of −0.14 for the study of Genoni et al. (4). In fact, the effect size of −0.41, which is seen in Table 1, was written by mistake. Unfortunately, this is a typing error in Table 1; however, the effect size used for the meta-analysis is right.

   4. and e) It can be seen in Figure 2C (BMI) that we used an effect size of −1.8 from the study of Mellberg et al. (5) and −0.8 from the study of Boers et al. (6), but these were swapped by mistake (−0.8 for Mellberg et al.’s study and −1.8 for Boers et al.’s study in Table 1); unfortunately this is a typing error in Table 1. In fact the effect sizes reported in Figure 2C are right.

2. Selection of effect sizes included

   • a) and b) Stomby et al.’s article (7) is a substudy of the larger main study of Mellberg et al. (5), and therefore they have the same design. In these 2 studies, we extracted data on all outcomes from the first period only (6-mo period) and no data were extracted from the other intervention periods (12, 18, and 24 mo). We reported the whole follow-up period for the studies of Stomby et al. (7) and Mellberg et al. (5) (720 d) in Table 1. However, all outcomes were extracted over a 6-mo period in both articles. Maybe it would have been better had we reported the 6-mo intervention period for both studies in Table 1 to prevent misconceptions. Data on body weight were also extracted from Figure 2B for the 6-mo period (intervention group: baseline, 87 kg; end, 79.1 kg; control group: baseline, 86.8 kg; end, 83.8 kg).

   • c) Stomby et al.’s article (7) is a substudy of the larger main study of Mellberg et al. (5). Indeed, Stomby et al. (sample size = 49) replicated data from Mellberg et al.’s study (sample size = 61), and so we only included Mellberg et al.’s study in our meta-analysis. But because the data on body fat percentage had not been reported in this study, we only extracted the body fat data from Stomby et al.’s study, and the other outcomes were only extracted from the study of Mellberg et al. (the main study).

   • d) We sent an email to 2 authors of this study (B Ahrén and T Jönsson) and asked them to send us data on other outcomes, if possible. Finally, on 16 January, 2018, T Jönsson responded to our request and sent us the data on blood pressure and lipid profiles.

3. CIs that could not be reproduced

   1. The reason for these differences is related to the correlation coefficient. In the study of Boers et al. (6), we did not use the assumed constant correlation coefficient (e.g., 0.5 and 0.8) to calculate the mean differences between end of study and baseline. If we can calculate the correlation coefficient from available data from studies included in the analyses, it is more accurate than the assumed constant correlation coefficient. Therefore, using the studies that reported mean ± SD values pre and post intervention and also change from baseline to endpoint for outcomes, we calculated the correlation coefficient via SD baseline (SD_E), SD final (SD_C), and SD change (SD_diff) by the following formula:

      (1)

      Finally, we estimated the SD for mean change from baseline to endpoint by averaging the calculated correlation coefficients.

   2. In the study of Jönsson et al. (8), we did not use the assumed constant correlation coefficient (e.g., 0.5 and 0.8) to calculate the mean differences between end of study and baseline. In this study, mean ± SD values pre and post intervention for outcome measures were reported, and we could impute SD for the mean change from baseline to endpoint using the P value for means of outcomes in the intervention and control groups. To resolve the concerns in this regard, we performed reanalyses by using a constant correlation coefficient of 0.8, such that for the studies that did not report the changes from baseline to follow-up values, the correlation coefficient of 0.8 was used to compute the SDs for the mean change values. The findings from this reanalysis did not tangibly change from those reported previously.

Notes

Supported by the Nutrition and Food Security Research Center, Shahid Sadoughi University of Medical Sciences, Yazd, Iran.

Author disclosures: The authors report no conflicts of interest.