Concerns regarding the prediction of behavioral measures from multilayer network switching

In a recent PNAS paper by Pedersen et al. (1), the authors report impressive performance when attempting to predict phenotypic variables (e.g., duration of previous night’s sleep) using individual differences in brain network switching. Although the results are intriguing, we have major concerns regarding the prediction model used to generate the results reported in figure 3 of ref. 1.

The results obtained with the elastic net regression are reasonable. Based on their analysis, the authors used behavior to predict network switching and reported that “behavioral data accounted for ∼3% of the total variance of fMRI network switching data” (ref. 1, p. 13379). A concern that we bear is how this or any of the ensuing analyses translated into the claim of having found that network switching predicts behavior, and with what appears to be a surprisingly high accuracy reported in figure 3 of ref. 1 (r = [0.86, 0.54, 0.46]). To make the claims reported by the authors, one would have needed to employ a model that treats network switching parameters as independent variables and behavior as the predicted. In the text, the authors state, “We defined prediction as Prediction = Xβ + β_o, where X is the original values of our 3 behavioral variables and β_o is the intercept of the elastic net regression model” (ref. 1, p. 13381). While their methodology is not entirely clear as written, we see little evidence to suggest that network switching was used as a predictor in their model—a key requirement for any claim of brain predicting behavior.

Additionally, the description of the sleep measure is inaccurate. The authors claim they are using a measure from the Human Connectome Project (HCP) database quantifying the amount of sleep that participants had the night before the MRI scan: “Notably, network switching predicted the amount of sleep that participants had the night before the MRI scan” (ref. 1, p. 13379). The HCP inquires about participants’ average sleep over the last 30 d as part of the Pittsburgh Sleep Quality Index. Thus, the interpretation of sleep-related results is imprecise at best.

To summarize, caution should be taken when interpreting the relationship between network switching and behavioral outcomes in this paper (especially ref. 1, figure 3), given the mismatch between methods, results, and conclusion in their analytic approach.