Regression to the Mean, Apparent Data Errors, and Biologically Extraordinary Results: Letter Regarding “Changes in telomere length 3–5 years after gastric bypass surgery”

The paper by Dersham et al. contains an extraordinary claim, namely that “This study confirms that TL lengthening is observed post bariatric surgery…” (1). “Extraordinary claims require extraordinary evidence.” So wrote Carl Sagan in 1979 (2). Nearly 40 years later, we have noted that many extraordinary claims in the scientific literature result from data and statistical errors (3), and many authors are seeking to promote greater rigor in science in general (4) and nutrition and obesity research in particular (5–7).

The possibility of telomere length increasing during an individual’s lifetime is conceivable and has been previously raised. Yet because current understanding of telomere biology is that they shorten with aging, at a relatively slow rate (25–35 basepair/year) across multiple tissues including leukocytes (8, 9), a compelling finding to the contrary would be ground-breaking (8).

Given the extraordinary claim, we were eager to examine the supporting evidence and have noticed several issues that cause us to find the claim doubtful.

Regression to the Mean

Presumably, the primary analysis should be an analysis of the overall sample. In the overall sample, the authors actually found that “…the median TL from baseline to follow-up increased but was not significant…p=0.167).” This presumable primary analysis was not mentioned in the published abstract. In contrast, in the abstract and the body of the manuscript the authors concluded that 1) telomere length (TL) increased 3–5 years after bariatric surgery, and 2) telomere lengthening was significant only for those with baseline TL < 3.9 (baseline median TL 5), and telomere lengthening was not significant for the intermediate TL group (TL 3.9 to 6.7) and the long TL group (TL > 6.7) (Figure). Unfortunately, without a control group for comparison, these conclusions are misleading. The results are indistinguishable from those that would merely be a function of regression to the mean (RTM) (7).

In the low baseline TL subgroup (N=17), subjects were selected to have baseline TL (< 3.9, median 1.07) below the baseline median TL of 5. Even in the absence of any true change in TL or any intervention, the average post-surgery TL values in this low baseline TL subgroup is always expected to rise when they are followed over time due to regression towards the mean (RTM), if TL is measured imperfectly (i.e., with error). Likewise, the average follow-up TL value in the high baseline TL subgroup is always expected to fall due to RTM, whether the subjects were treated or not. This is a classic example of the statistical phenomenon, RTM, and not an indication of a treatment effect. As predicted by RTM, the greatest changes in TL values from baseline to follow-up tend to occur, on average, in subgroups with larger baseline TL differences from the overall population mean TL (=5). 

RTM seems to lead many investigators astray and to erroneous conclusions (7, 10–13), including in the study of TL (14). RTM in such situations will occur whenever the baseline and follow-up measurements are not perfectly correlated. Bias due to RTM can be avoided by the use of a control group in the study design. Although the Dersham et al briefly mention the possibility of RTM in their discussion, doing so does not alter the unreasonableness of the published conclusions.

Mathematically Incommensurable Data in Figure 1

Figure 1 appears to contain a numerical error. The median of the original sample (N=50) should be equal to the median of the intermediate group described in Figure 1. The median value is the value that divides the observations into two equal groups so that half the observations are smaller than the median and half are larger than the median (15). In the case where there are an even number of observations, the median is calculated as the average of the smallest value in the top half and largest in the bottom half. Consequently, the median of the Intermediate group should be equal to the median of the entire sample. In Figure 1, however, these two medians are not equal, suggesting a mislabeling and/or data error.

Measurement Concerns

The change in TL reported in Dersham et al falls well outside the range that is normally encountered (25-35bp/year loss), particularly in the context of such a short follow-up period of 3-5 years (8). Considering the measurement precision of TL by quantitative polymerase chain reaction (qPCR) which is at best in the order of a couple hundred basepairs (8, 16), a much larger sample size would be appropriate for investigating differences in LTL dynamics over a period of 3-5 years (8, 17).

Conclusion

In conclusion, we believe that the study by Dersham et al is not sufficient to support the published conclusion of the paper. Increased attention to rigor in study design, data, analysis, and interpretation is warranted.