Reply to T Kalayjian and EC Westman

Dear Editor:

Thank you for the opportunity to respond to the Letter to the Editor received from Drs Kalayjian and Westman (1). In their letter, Drs Kalayjian and Westman raise several points about methodological limitations of our Keto-Med trial (2) that appear to show bias against the well-formulated ketogenic diet (WFKD). Prior to responding to each point, we want to emphasize that when designing studies comparing health effects we feel a truly meaningful and informative comparison requires giving each diet an equal opportunity to succeed and avoid “straw man” comparisons, that is, equipoise (3). We selected the WFKD (4), an exemplary form of a very-low-carbohydrate dietary pattern, to maximize the potential benefits.

1) Medication reduction. We agree that differential changes in antihyperglycemic medications complicate the analysis of glycated hemoglobin (HbA1c) changes. Sulfonylurea medications often need to be adjusted on a low-carbohydrate diet (5), and we designed our protocol to balance the study objectives with the safety of participants with type 2 diabetes mellitus (T2DM). The medication reduction was matched to each dietary pattern based on clinical recommendations. The effect of following a practical guideline improves the generalizability but reduces the fairness of the comparison. This could have been highlighted better in the article, we agree. Fortunately, the majority of the participants with T2DM (9/13, 69%) maintained the same medication regimen on both diets. We do present a sensitivity analysis excluding the 4 participants with differential usage of sulfonylureas, appreciating that medication changes could affect HbA1c, and state these results in the article in the section on sensitivity analyses.

2) Adjustment for weight. We agree that weight change differential on the 2 diets could be an important component of clinical impact. The primary analysis is not adjusted for weight change. It was only in the sensitivity analyses that we report the findings in terms of an adjustment for weight change, for those readers who might be interested in that specific question.

3) Crossover without a washout period. Although washout periods are often utilized in crossover design studies, the usual intent of having endpoints return to baseline levels before initiating the second phase is challenging in terms of a) duration, and b) whether the focus is just the primary outcome or also secondary outcomes. In this study, with HbA1c as the primary outcome, a washout of ≤12 wk might have been used because this is the time period of interest reflected by HbA1c values (6). But such a lengthy washout would likely have led to increased dropouts and thus be problematic for participants with data for only the first phase of the study. Instead, the intervention periods were designed to be 12 wk, and the initial 4 wk of each study phase involved food delivery to maximize rapid adherence to each new diet phase, which we did observe (7). Thus, HbA1c levels at the end of each 12-wk diet phase should have reflected a full and appropriate exposure to the primary outcome. Plus, the time required by each patient to return to baseline would have likely varied across patients, making the selection of a single duration period for the washout problematic. In addition, although this study did not involve calorie restriction, and weight loss was not a specific focus of the intervention, weight loss—a secondary outcome of interest—was expected and was in fact achieved for both diets in the first phase of the study. Asking and waiting for study participants to regain weight and return to their baseline weight would have been highly unrealistic and unethical for this group of study participants. It was primarily for these reasons, among others, that we chose to not utilize a washout phase in the crossover study.

4) Due to lack of washout, treat the crossover study as a parallel design study, and use only the first phase, and provide a post hoc power calculation. In fact, this analysis was done as part of the sensitivity analyses, and is reported briefly in the main article and in more detail in the Supplemental Materials. However, there is a strong rationale for not conducting post hoc power calculations (8). Hoenig and Heisey (8) illustrate “the one-to-one relationship between p-values and observed power” and declare “nonsignificant p-values always correspond to low observed powers” in post hoc power analyses. They advocate for confidence intervals (CIs), explaining “the breadth of the interval tells us how confident we can be…[and] once we have construted a CI, power calculations yield no additional insights.”

In summary, the comments and suggestions of Drs Kalayjian and Westman were anticipated and provided for in this study by an extensive set of sensitivity analyses that were preplanned in the Statistical Analysis Plan (SAP) and reported in the article (2). The conclusions of our study appropriately focus on the results of the preplanned primary analyses of the SAP; substituting selected findings from the sensitivity analyses would be methodologically inappropriate. Overall, we are pleased to report that the comments we respond to here were anticipated, and the sensitivity analyses results are reported, so that individual readers can use these in their own interpretation of the study findings, as should be done for all studies.

Data Availability

The data that support the findings of the Keto-Med trial are available from the corresponding author, CDG, upon reasonable request.