Robust statistics show no evidence for a relationship between fiber density and memory performance

Schott et al. (1) reported significant Pearson's correlations between memory performance and the density of fibers between the prefrontal ventral cortex and the rhinal cortex. More precisely, the authors reported correlations between the density of fibers connecting the prefrontal ventral cortex to the entorhinal cortex and (i) deep encoding [r = 0.6414 (95% confidence interval 0.3527, 0.8186), t = 4.2627, P = 0.000235] and (ii) shallow encoding [r = 0.5329 (0.1995, 0.7557), t = 3.2113, P = 0.003503], as well as correlations between the density of fibers connecting the prefrontal ventral cortex to the perirhinal cortex and (iii) deep encoding [r = 0.7029 (0.4471, 0.8524), t = 5.0391, P = 0.00003031] and (iv) shallow encoding [r = 0.5861 (0.2727, 0.7870), t = 3.6887, P = 0.001047].

From these correlations, the authors concluded that fiber density between the prefrontal ventral cortex and the rhinal cortex predicts memory performance. This conclusion is, however, unjustified by the data, as suggested by the scatterplots in figure 3b of Schott et al. (1). In these scatterplots, most of the points are clustered together and show no obvious linear relationship between fiber density and memory. The relationships between fiber density and memory seem to be due to outliers (i.e., few subjects with larger fiber density than the majority of the subjects). In keeping with this observation, correlations must be computed after removing outliers (2). Outliers can be detected in the bivariate space using multivariate projection techniques, such as the minimum covariance determinant (MCD) or the minimum-volume ellipsoid (2). For instance, MCD flags between three and five outliers for correlations mentioned previously. Using a robust technique called a skipped correlation, whereby the multivariate outliers are removed and Spearman's correlation is applied to the remaining data points, with some adjustments for pruning the data (2), none of the correlations are significant: (i) r = −0.0399, t = 0.2034; (ii) r = −0.2294, t = 1.2018; (iii) r = 0.4730, t = 2.7375; (iv) r = 0.0798, t = 0.4080 (critical t value = 2.939; α = 0.05, with adjustment for performing multiple tests) (2).

Hence, the misleading results reported by Schott et al. (1) provide a good illustration of a well-known problem: Pearson's correlation is not robust. In fact, in the presence of a significant Pearson's correlation we should start wondering whether there is a true association between two variables or whether we are facing one of many problems that can affect Pearson's correlation (2).

Even if we consider the significant correlations reported by Schott et al., the data suggest that there is too much uncertainty about the correlations to justify strong conclusions. Indeed, the 95% confidence intervals are quite large (see above) and get even larger with the use of more accurate percentile bootstrap 95% confidence intervals (2): (i) (0.1779, 0.8494); (ii) (−0.1433, 0.7888); (iii) (0.3351, 0.8888); (iv) (−0.0158, 0.8319). Thus, as advocated by many, effect sizes must be taken into account when interpreting results (3, 4). If one suspects that fiber density predicts memory performance, it would also be more informative to report a linear regression, with confidence intervals around its parameters, to start building even a simple model linking structure and function. Finally, an interesting problem will be to determine why some subjects have such large fiber densities.