Reply to DM Thomas et al

Dear Editor:

We thank Thomas et al.'s group for their comments on our study simulating long-term calorie restriction using the mathematical models underlying both the NIH of Health Body Weight Planner (BWP) and the Pennington Biomedical Research Center's Weight Loss Predictor (PBRC WLP) (1). It was disappointing to read that the authors believe that we misrepresented their model. We worked diligently to reproduce the model described in their 2011 publication (2), which was described on the PBRC WLP website as the basis for the PBRC WLP since its inception in approximately 2012 until at least the date of acceptance of our study (1). This can be confirmed by the Internet archive capture of the PBRC WLP website at the following URL: https://web.archive.org/web/20171220103129/http://www.pbrc.edu/research-and-faculty/calculators/weight-loss-predictor/about/.

Subsequent to the publication of our study (1), we noticed that the website description of the PBRC WLP model was changed to eliminate mention of Thomas et al. (2), or any subsequent peer-reviewed model publication. For many years, the PBRC WLP website listed the model equations corresponding to Thomas et al. (2), but the recently updated website provides an incomplete list of equations with some changes from those listed in the peer-reviewed publication. Therefore, the PBRC WLP website has not adequately documented any unpublished evolution of the model. Whatever model now underlies the PBRC WLP, it has not been subject to peer review.

As with any scientific methodology, mathematical models must be fully documented and their results need to be reproducible. Our efforts to reproduce the model described in Thomas et al. (2) showed errors and inadequate model specifications that were corrected as described in our study (1). In this process, we sought several clarifications from Dr. Thomas through an e-mail exchange between 2014 and 2017. Dr. Thomas described how various model components had been changed over time, but she was uncertain as to what specific equations had been used to generate the results reported in the 2011 publication (2) or had been implemented in the PBRC WLP. This uncertainty and lack of documentation hampered our efforts to precisely reproduce the PBRC WLP model. We are thankful that Dr. Thomas and colleagues finally released what we presume is the current PBRC WLP model code in conjunction with their letter. The model equations and code underlying the NIH BWP have been fully documented and available since the model's publication in 2011 (3) and have been successfully reproduced and implemented by several investigators around the world [e.g., (4–6)].

With regard to the model validation methods used in our study (1), the authors raised questions regarding the relation between initial measured body weight, weight change, and final body weight. They claimed that our study comparing model-predicted weight changes with measured weight changes (1) was inappropriate because final body weights should have been used instead. However, the ordinary differential equations underlying these models require the initial measured body weight as an input to compute subsequent changes in body weight over time. The resulting model-predicted final body weight is simply the initial measured body weight plus the model-predicted weight change. Because the initial body-weight measurements for each individual subject were commonly known inputs to both models, comparing model-predicted with measured weight changes is identical to comparing model-predicted with measured final body weights. Thus, criticisms of our model validation methods on the basis of the use of weight change instead of final body weight are unfounded.

In an effort to show the validity of the current PBRC WLP model, the authors plotted the measured compared with model-predicted final body weights for all subjects on the same graph. However, such a plot can give a false sense of model validity because much of the variance in both the model-predicted final body weights and measured final body weights is due to between-subject differences in the initial measured body weights. A more rigorous model validation procedure involves plotting the individual model residuals against the measured weight changes, as we did in our study (1). Alternatively, comparing measured with model-predicted final body weights across subjects should control for the initial measured body weights in the regression procedure.

The current PBRC WLP continues to exhibit the initialization error described in our study (1), resulting in a violation of the first law of thermodynamics when baseline physical activity level (PAL; ratio of total to resting energy expenditure) is <1.67. To illustrate, the online PBRC WLP was used to simulate a 30-y-old, 100-kg woman who is 155 cm tall. The PBRC WLP describes using the Livingston-Kohlstadt equation (which does not depend on height) to calculate the initial resting energy expenditure of 1691 kcal/d. Initial total energy expenditure (which is height-dependent in the current online PBRC WLP model and conveniently allows for a way of adjusting initial PAL) was 2708 kcal/d, thereby resulting in an initial PAL of 1.6. Despite setting calorie restriction to zero, the online PBRC WLP resulted in ∼1.5 kg of weight loss after 12 mo. Increasing height to 179 cm increases initial total energy expenditure to 2826 kcal/d, thereby bumping PAL just above the 1.67 threshold, such that no weight loss occurs when calorie restriction is zero.

Half of the subjects with complete data in phase 2 of the Comprehensive Assessment of the Long-term Effects of Restriction of the INtake of Energy (CALERIE) study had an initial PAL <1.67. In these subjects, the PBRC WLP simulates weight loss even without calorie restriction. Therefore, simulating calorie restriction in these subjects using the PBRC WLP model results in an inappropriately enhanced negative energy imbalance and increased weight loss. As described in our study (1), we corrected this PBRC WLP model error and the CALERIE phase 2 study was simulated to result in overall mean (95% CI) underestimations of model-predicted weight losses of 4.02 kg (3.40, 4.63 kg) at 12 mo (P < 0.0001) and 4.35 kg (3.51, 5,18 kg) at 24 mo (P < 0.0001). Interestingly, failing to correct this error resulted in improved model biases of −0.43 kg (−1.63, 0.78 kg) at 12 mo (P = 0.49) and −0.79 kg (−2.22, 0.64 kg) at 24 mo (P = 0.28). We suspect that Figures 1 A, B in the letter from Thomas et al. were also affected by this error, thereby resulting in apparently favorable model results, but for the wrong reason.

The validity of a mathematical model is ideally established by an unbiased prospective test of a fully documented and reproducible model without post hoc adjustment of model components or parameters. In this regard, our study (1) validated the NIH BWP model for simulating long-term human caloric restriction using data published years after the model's peer-reviewed publication (3). The NIH BWP model outperformed the published model of Thomas et al. (2), which was previously acknowledged to be the basis for the PBRC WLP. The model currently underlying the PBRC WLP has yet to be fully described or validated in a peer-reviewed publication.