Comments on Brodin et al. “Sleep deprivation and dendritic architecture: a systematic review and meta-analysis”

Dear Editor,

We write regarding the meta-analysis recently published in SLEEP by Brodin et al. [1] entitled “Sleep deprivation and dendritic architecture: a systematic review and meta-analysis.” The authors reach two conclusions: (1) short sleep deprivation does not consistently affect spine density or dendrite length, while long sleep deprivation decreases both parameters; (2) the literature on this topic is highly heterogenous and suffers from many methodological and statistical limitations.

While a meta-analysis of changes in neuronal architecture in response to sleep deprivation is welcome, it can only be helpful if it is comprehensive, accurate, and scholarly, providing an overview that distills trustworthy conclusions from a potentially confusing literature, guiding rather than misleading the reader. Unfortunately, this is not the case here.

Recognizing this, in 2009, the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines were published, and they have become the standard for many journals [2]. While Brodin et al. reference similar guidelines (Cochrane), there are critical flaws in their approach that compromise the quality of the analysis and raise questions about their conclusions. These flaws include the sole reliance on the abstract to select the papers, unwarranted assumptions, and several technical inadequacies.

Brodin et al. choice of search terms is already problematic, focusing on the “rodent cerebrum” while ignoring other animal models (e.g. [3–5]) and brain regions (e.g. [6]). They also ignored studies that measured spine/synapse turnover (formation/elimination) after sleep deprivation, a parameter that is quite relevant for the analysis of “dendritic architecture”. Most importantly, out of 5149 papers initially identified in their area of focus, they selected 30 papers identified solely based on the abstract. According to Dr. Karlsson (personal communication), reading only the abstract was “the only way to review more than 5000 papers”. By this logic, any paper reporting the analysis of spine/synapse density in the main text and tables but not in the abstract would have been excluded, no matter how relevant (e.g. [7]). While it certainly takes time to accurately review the literature, there are efficient methods to address this challenge. AI-powered search engines, which can scan the entire text of a paper or crowd-sourcing, are accepted methods [2] that would have identified the cited paper [7] and likely several other publications. A high-quality meta-analysis would also include a review of the literature cited by the selected articles, to be certain that all relevant papers are included, and would have scoured the “grey” literature to reduce the bias that comes from failure to publish null results.

Another issue is that, as acknowledged by Brodin et al., several key assumptions were made when the methods in the selected papers did not provide enough details. According to Dr. Karlsson (personal communication), attempts to clarify these details by contacting the authors were not successful, as “very few authors replied”. Thus, Brodin et al. state: “… In several of the studies the unit given was deemed to be highly unlikely and, in these cases, we changed to the unit that would give a spine density within the same order of magnitude as the rest of the studies … ”. Also “ ….19 of the 30 studies (63%) did not clearly state if they treated neurons from the same animal as dependent data points.”

There are other problems. Brodin et al. opt to apply random-effects models to subsamples due to the considerable heterogeneity. However, according to Cochrane guidelines, this is appropriate when heterogeneity cannot be explained, which is not the case here. They also draw an unjustified line between studies of sleep deprivation with a duration of at least 24 h and one of at least 72 h, which they call acute and chronic, respectively. In fact, duration of sleep deprivation varied enough across studies that it would best be considered as a continuous variable. Furthermore, sleep deprivation duration and other factors (e.g. brain region, protocol) would best be considered as moderators of the primary effects, allowing for more consolidated analyses, rather than as factors for subgroup analyses, which, as admitted by the authors, are severely underpowered. Finally, essential to any analytic approach is the use of homogenous metrics. For example, an analysis of dendritic length measured in absolute terms (μm) and in percent values together is incorrect. In a recent meta-analysis by one of us (RS) [8], we asked the authors for raw data in order to compare apples to apples and ensure an accurate outcome.

A major “discrepancy” among studies discussed by Brodin et al. relates to the method of sleep deprivation. As argued in a recent review [9], scientists use different methods and the choice can “fundamentally alter the experimental outcome with regard to synaptic plasticity” [9]. For example, some use novel object exploration to keep mice awake because the goal is to mimic ongoing learning, which is assumed to be a default feature of wakefulness [7]. Other laboratories use gentle handling, in some cases with the specific intention of minimizing sensory stimulation, novelty, and plasticity, under the assumption that this would reveal the synaptic effects of “waking per se” [9]. It is not so much that one method is “better” than the other, but that the questions and assumptions are different [9]. Similarly, in many studies of long sleep deprivation, the goal is not to avoid “stress”, which is inherent to chronic extreme sleep loss, but to measure how that specific stress affects the brain.

It is reassuring that Brodin et al. suggest the use of linear mixed effect models for complex analyses in which spine density or dendrite length are not the only parameters to consider. In our work (CC), LME models have been systematically employed to study the effects of sleep/wake on synaptic size and number (e.g. [6, 7]). This is because spine/synapse density and synapse size are not independent at the dendrite level, as documented both in cortex [7] and in the CA1 region of the hippocampus, the focus of most of the analyses by Brodin et al. In CA1, electrically induced long-term potentiation leads to opposite changes in spine density (a decrease) and synapse size (an increase) [10]. Thus, measuring the number of spines alone is not a reliable way to assess synaptic strength [7].

We share the interest of Brodin et al. in making the field stronger and more rigorous, and emphasize that this should apply to the primary empirical work as much as to secondary reviews and meta-analyses. For this reason, we hope that readers of SLEEP consider the conclusions of Brodin et al. in light of the points raised here.