Skip to main content
NCBI home page
Sleep logoLink to Sleep
. 2021 Aug 1;44(12):zsab197. doi: 10.1093/sleep/zsab197

Cognitive throughput and working memory raw scores consistently differentiate resilient and vulnerable groups to sleep loss

Tess E Brieva 1, Courtney E Casale 1, Erika M Yamazaki 1, Caroline A Antler 1, Namni Goel 1,✉
PMCID: PMC8664585  PMID: 34333658

Abstract

Study Objectives

Substantial individual differences exist in cognitive deficits due to sleep restriction (SR) and total sleep deprivation (TSD), with various methods used to define such neurobehavioral differences. We comprehensively compared numerous methods for defining cognitive throughput and working memory resiliency and vulnerability.

Methods

Forty-one adults participated in a 13-day experiment: 2 baseline, 5 SR, 4 recovery, and one 36 h TSD night. The Digit Symbol Substitution Test (DSST) and Digit Span Test (DS) were administered every 2 h. Three approaches (Raw Score [average SR performance], Change from Baseline [average SR minus average baseline performance], and Variance [intraindividual variance of SR performance]), and six thresholds (±1 standard deviation, and the best/worst performing 12.5%, 20%, 25%, 33%, 50%) classified Resilient/Vulnerable groups. Kendall’s tau-b correlations compared the group categorizations’ concordance within and between DSST number correct and DS total number correct. Bias-corrected and accelerated bootstrapped t-tests compared group performance. 

Results

The approaches generally did not categorize the same participants into Resilient/Vulnerable groups within or between measures. The Resilient groups categorized by the Raw Score approach had significantly better DSST and DS performance across all thresholds on all study days, while the Resilient groups categorized by the Change from Baseline approach had significantly better DSST and DS performance for several thresholds on most study days. By contrast, the Variance approach showed no significant DSST and DS performance group differences.

Conclusion

Various approaches to define cognitive throughput and working memory resilience/vulnerability to sleep loss are not synonymous. The Raw Score approach can be reliably used to differentiate resilient and vulnerable groups using DSST and DS performance during sleep loss.

Keywords: individual differences, sleep deprivation, cognitive performance, resilient, vulnerable, recovery, DSST, DS

Statement of Significance.

Robust individual differences are observed in cognitive deficits resulting from sleep loss, with different methods used to define these neurobehavioral differences. For the first time, this study systematically compared numerous methods for defining cognitive throughput (via the Digit Symbol Substitution Test [DSST]) and working memory (via the Digit Span Test [DS]) resiliency and vulnerability to sleep loss. The approaches generally failed to categorize the same individuals into Resilient and Vulnerable groups within or between performance measures, and thus are not synonymous. An approach using raw scores, rather than variance or change from baseline, can be reliably used to differentiate resilient and vulnerable groups using DSST and DS performance during sleep loss. Our results inform biomarker identification and countermeasure development.

Introduction

Sleep deprivation induces deficits in cognitive throughput, working memory, attention, sleepiness, mood, and a variety of other neurobehavioral functions [1–3]. However, some individuals are minimally affected by sleep loss (i.e. resilient) and some individuals are markedly affected by sleep loss (i.e. vulnerable) [3, 4]. Such neurobehavioral resilience or vulnerability to sleep loss has been demonstrated to be trait-like and stable over time [3–6]. The characterization of differential sleep loss responses is critical given that deficits in vulnerable individuals impede work performance, safety, and overall well-being [2, 4, 7–11]. Yet, the literature on individual differences includes various methods to define neurobehavioral resilience and vulnerability to sleep loss, which must be directly compared to better understand research in this area and its implications.

Numerous approaches have been used to define neurobehavioral resilience and vulnerability to sleep loss. An approach using raw scores has been commonly utilized, wherein individuals are categorized as resilient or vulnerable by their raw score on neurobehavioral tasks and self-report measures during sleep loss [12–16]. Other studies have used a change from baseline score approach to define resilient and vulnerable groups, whereby individuals are categorized by a difference score that accounts for baseline scores [17–23]. Intra-individual variance, which captures instability within an individual, has been suggested as another possible factor contributing to differential sleep loss response because variability may indicate cognitive vulnerability [24–27] and may involve time-of-day variation [28–33]. However, defining neurobehavioral resilience and vulnerability to sleep loss based on intra-individual variance has not yet been explicitly examined.

Within these various approaches, different thresholds have been applied to classify neurobehavioral resilience and vulnerability. The median split (50% threshold) [12, 14, 18, 21–23, 34–38] and the tertile split (33% threshold) [13, 20, 23, 39] have been used most frequently. The quartile split (25% threshold) [16, 40] and other numeric thresholds (e.g. the best and worst N = 5 performing participants [41] and the best and worst N = 8 performing [19]) have also been utilized. Additionally, ±1 standard deviation (SD) is another possible threshold for classifying resilience and vulnerability that warrants examination.

Sleep deprivation causes deficits in many neurobehavioral domains critical for optimal functioning, including cognitive throughput and working memory [1]. Cognitive throughput (a corrected response rate measure representing the number of correct responses within a unit of time [i.e. processing speed]) is negatively impacted by total sleep deprivation (TSD) and chronic sleep restriction (SR) [29, 42–44]. The effect of sleep loss on working memory (a layered system for short-term information storage and usage [45]) is less consistent; deficits may or may not occur after TSD [46–48], but do not occur after chronic SR [6, 49, 50]. Nevertheless, stable, trait-like individual differences in cognitive throughput and working memory performance occur following sleep loss [4–6]. However, research on the specific characterization of such resilient and vulnerable cognitive responses to sleep loss is sparse, with only a few studies using DSST and DS performance to explicitly define resilient and vulnerable performance groups [35, 46, 51–53].

Given the various definitions of resilient and vulnerable responses to sleep loss, our study for the first time comprehensively compared numerous methods of defining cognitive throughput and working memory resilience and vulnerability to sleep loss. Specifically, we defined cognitive throughput and working memory performance resilience and vulnerability based on three approaches and six discrete thresholds, some of which have thus far not been investigated. The aims of the study were to: (1) compare the resilient and vulnerable categorizations of the approaches and thresholds within each measure; (2) analyze the differences between the resilient and vulnerable groups’ cognitive throughput and working memory performance defined by the different approaches and thresholds across the study; and (3) compare the resilient and vulnerable categorizations of the approaches and thresholds between cognitive throughput and working memory measures. We hypothesized the following: (1) individuals would be categorized into resilient and vulnerable groups by the three approaches in a similar manner within a measure; (2) for all approaches and at all thresholds, significantly higher scores (reflecting better cognitive throughput and working memory) would be found in resilient individuals relative to vulnerable individuals on all SR days and during TSD; and (3) individuals would be categorized into resilient and vulnerable groups by the three approaches in a similar manner between measures of cognitive throughput and working memory.

Methods

Participants

Forty-one healthy adults (ages 21–49; mean ± SD, 33.9 ± 8.9 years; 18 females; 31 African Americans) were recruited in response to study advertisements. Participants were monitored at home with actigraphy, sleep-wake diaries, and time-stamped call-ins to determine bedtimes and waketimes during the 7–14 days before the laboratory phase. Participants were not allowed to use caffeine, alcohol, medications (except oral contraceptives), or tobacco for 7 days before the study, as verified by blood and urine screenings. To view detailed participant qualifications, the pre-study protocol, and prohibited activities, see Yamazaki et al. [42].

The protocol was approved by the University of Pennsylvania’s Institutional Review Board. All participants provided written informed consent in accordance with the Declaration of Helsinki. They received compensation for participation.

Procedures

Participants engaged in a 13-day laboratory study in which they were studied continuously and received daily checks of vital signs and symptoms by nurses (with a physician on call). The study consisted of two nights of baseline sleep of 10 h (baseline day 1 [B1], 2200 h–0800 h) and 12 h (baseline day 2 [B2], 2200 h–1000 h) time in bed (TIB) respectively, followed by five consecutive nights of 4 h TIB per night [sleep restriction days 1–5 (SR1–SR5, 0400 h–0800 h)], four consecutive nights of 12 h TIB per night recovery sleep opportunity (recovery days 1–4, R1–R4, 2200 h–1000 h), and 36 h of TSD (0 h TIB, wakefulness from 1000 h to 2200 h the following day). Participants were monitored continuously by trained staff throughout the study to ensure adherence to the protocol. Additionally, polysomnography (PSG) was recorded on certain nights, including B2. For further information on participant permitted activities and a detailed protocol, see Yamazaki et al. [42].

Neurobehavioral measures

A precise computer-based neurobehavioral test battery was administered every 2 h during wakefulness on all days of the study. The test battery included the Digit Symbol Substitution Test (DSST) [54], which measures cognitive throughput and the Digit Span Task (DS) [55], which measures working memory. For the DSST, participants matched a set of symbols to digits with a pairing key within a prescribed time limit with performance defined as the total number of correct symbols [43, 55]. DSST number correct was the outcome variable for this test [5, 6, 42]. The DS involved memorization of a sequence of numbers that participants recited forward or backward [55, 56]. DS total number correct, which is the sum of forward and backward scores, was the outcome variable for this test. B1 served as an adaptation day and thus these DSST and DS data were excluded from analyses. Due to protocol scheduling conflicts, DSST and DS data were missing for the B2 2000 h (for N = 26 participants), SR5 0800 h (for N = 22 participants), and R1 1000 h (for N = 22 participants) test bouts.

Resilient, vulnerable, and intermediate group determination

Resilient (Res), Vulnerable (Vul), and Intermediate (Int) groups were defined by three approaches, as follows: (1) The Raw Score approach, which calculated a participant’s average performance (DSST: number correct; DS: total number correct) across the SR1 0800 h–SR5 2000 h test bouts; (2) The Change from Baseline approach, which subtracted a participant’s average performance across the B2 1000 h–2000 h test bouts from their own average performance across the SR1 0800 h–SR5 2000 h test bouts; (3) The Variance approach, which calculated the intraindividual variance of a participant’s performance across the SR1 0800 h–SR5 2000 h test bouts.

The median and interquartile range (IQR) of average score, average change from baseline, and average variance for each measure were as follows: DSST number correct Raw Score approach, 57.000 (11.558); DSST number correct Change from Baseline approach, −0.977 (3.997); DSST number correct Variance approach, 33.204 (33.814); DS total number correct Raw Score approach, 10.222 (5.045); DS total number correct Change from Baseline approach, −0.023 (2.350); and DS total number correct Variance approach, 9.074 (5.389).

Within each approach, Res and Vul groups were defined by six thresholds as follows: (1) ±1 SD (Res and Vul groups, each N = 0–7; see Figures 1–6 captions and Supplementary Table S1 for exact N for each measure and approach); (2) the best and worst performing 12.5% (Res and Vul groups, each N = 5); (3) the best and worst performing 20% (Res and Vul groups, each N = 8); (4) the best and worst performing 25% (Res and Vul groups, each N = 10); (5) the best and worst performing 33% (Res and Vul groups, each N = 13); and (6) the best and worst performing 50% (Res group N=20, Vul group N = 21). For the Raw Score approach categorization of DSST and DS performance, the +1 SD and best performing percentage groups comprised the Res groups (e.g. the greater the number correct, the more resilient) and the −1 SD and worst performing percentage groups comprised the Vul groups (e.g. the fewer the number correct, the more vulnerable). For the Change from Baseline approach categorization of DSST and DS performance, the +1 SD and best performing percentage groups comprised the Res groups (e.g. the greater the average change from baseline, the more resilient) and the −1 SD and worst performing percentage groups comprised the Vul groups (e.g. the lower the average change from baseline, the more vulnerable). For the Variance approach categorization of DSST and DS performance, the −1 SD and best performing percentage groups comprised the Res groups (e.g. the less variance, the more resilient) and the +1 SD and worst performing percentage groups comprised the Vul groups (e.g. the more variance, the more vulnerable). At each threshold, the remaining participants who were not categorized into the Res or Vul groups were classified as part of the Int group.

Figure 1.

Figure 1.

Open in a new tab

Resilient (Res), Vulnerable (Vul), and Intermediate (Int) group Digit Symbol Substitution Test (DSST) number correct performance profiles across the study using six different thresholds within the Raw Score approach. Res, Vul, and Int groups were determined by averaging DSST number correct from all test administrations during sleep restriction days 1–5 (SR1–SR5) (e.g. the higher DSST number correct, the more resilient) and using the following six thresholds: (A) ±1 standard deviation (SD) (Res N = 5; Vul N = 7; Int N = 29); (B) the best and worst performing 12.5% (Res N = 5; Vul N = 5; Int N = 31); (C) the best and worst performing 20% (Res N = 8; Vul N = 8; Int N = 25); (D) the best and worst performing 25% (Res N = 10; Vul N = 10; Int N = 21); (E) the best and worst performing 33% (Res N = 13; Vul N = 13; Int N = 15); (F) the best and worst performing 50% (Res N = 20; Vul N = 21; All N = 41). See Table 2 for t-test comparisons between Res and Vul groups. The top and bottom axis labels depict the study design: Baseline day 2 (B2, 1000 h–2400 h), SR1 (0200 h, 0800 h–0200 h), SR2–SR4 (0800 h–0200 h), SR5 (0800 h–2000 h), Recovery days 1–4 (R1–R4, 1000 h–2000 h), and total sleep deprivation day (TSD, 2200 h–2000 h). Light blue lines and light gray lines depict individual DSST number correct profiles for the Res and Vul groups, respectively; the dark blue line and the dark gray line depict averaged DSST number correct profiles for the Res and Vul groups, respectively. The black dotted line depicts the Int group (except for 50%, for which this line depicts all participants) average DSST number correct profile. Breaks in the lines indicate missing data.

Figure 2.

Figure 2.

Open in a new tab

Resilient (Res), Vulnerable (Vul), and Intermediate (Int) group Digit Symbol Substitution Test (DSST) number correct performance profiles across the study using six different thresholds within the Change from Baseline approach. Res, Vul, and Int groups were determined by subtracting each participant’s mean DSST number correct across baseline day 2 (B2) from their mean DSST number correct across sleep restriction days 1–5 (SR1–SR5) (e.g. the greater the average change from baseline score, the more resilient) and using the following six thresholds: (A) ±1 standard deviation (SD) (Res N = 5; Vul N = 5; Int N = 31); (B) the best and worst performing 12.5% (Res N = 5; Vul N = 5; Int N = 31); (C) the best and worst performing 20% (Res N = 8; Vul N = 8; Int N = 25); (D) the best and worst performing 25% (Res N = 10; Vul N = 10; Int N = 21); (E) the best and worst performing 33% (Res N = 13; Vul N = 13; Int N = 15); (F) the best and worst performing 50% (Res N = 20; Vul N = 21; All N = 41). See Table 2 for t-test comparisons between Res and Vul groups. The top and bottom axis labels depict the study design: B2 (1000 h–2400 h), SR1 (0200 h, 0800 h–0200 h), SR2–SR4 (0800 h–0200 h), SR5 (0800 h–2000 h), Recovery days 1–4 (R1–R4, 1000 h–2000 h), and total sleep deprivation day (TSD, 2200 h–2000 h). Light blue lines and light gray lines depict individual DSST number correct profiles for the Res and Vul groups, respectively; the dark blue line and the dark gray line depict averaged DSST number correct profiles for the Res and Vul groups, respectively. The black dotted line depicts the Int group (except for 50%, for which this line depicts all participants) average DSST number correct profile. Breaks in the lines indicate missing data.

Figure 3.

Figure 3.

Open in a new tab

Resilient (Res), Vulnerable (Vul), and Intermediate (Int) group Digit Symbol Substitution Test (DSST) number correct performance profiles across the study using six different thresholds within the Variance approach. Res, Vul, and Int groups were determined by intraindividual variance in DSST number correct from all test administrations during sleep restriction days 1–5 (SR1–SR5) (e.g. the less variance, the more resilient) and using the following six thresholds: (A) ±1 standard deviation (SD) (Res N = 0; Vul N = 5; Int N = 36); (B) the best and worst performing 12.5% (Res N = 5; Vul N = 5; Int N = 31); (C) the best and worst performing 20% (Res N = 8; Vul N = 8; Int N = 25); (D) the best and worst performing 25% (Res N = 10; Vul N = 10; Int N = 21); (E) the best and worst performing 33% (Res N = 13; Vul N = 13; Int N = 15); (F) the best and worst performing 50% (Res N = 20; Vul N = 21; All N = 41). See Table 2 for t-test comparisons between Res and Vul groups. The top and bottom axis labels depict the study design: Baseline day 2 (B2, 1000 h–2400 h), SR1 (0200 h, 0800 h–0200 h), SR2–SR4 (0800 h–0200 h), SR5 (0800 h–2000 h), Recovery days 1–4 (R1–R4, 1000 h–2000 h), and total sleep deprivation day (TSD, 2200 h–2000 h). Light blue lines and light gray lines depict individual DSST number correct profiles for the Res and Vul groups, respectively; the dark blue line and the dark gray line depict averaged DSST number correct profiles for the Res and Vul groups, respectively. There was no Res group for the ±1 SD threshold due to no participants having a z-score > 1.0. The black dotted line depicts the Int group (except for 50%, for which this line depicts all participants) average DSST number correct profile. Breaks in the lines indicate missing data.

Figure 4.

Figure 4.

Open in a new tab

Resilient (Res), Vulnerable (Vul), and Intermediate (Int) group Digit Span (DS) total number correct performance profiles across the study using six different thresholds within the Raw Score approach. Res, Vul, and Int groups were determined by averaging DS total number correct from all test administrations during sleep restriction days 1–5 (SR1–SR5) (e.g. the higher DS total number correct, the more resilient) and using the following six thresholds: (A) ±1 standard deviation (SD) (Res N = 5; Vul N = 6; Int N = 30); (B) the best and worst performing 12.5% (Res N = 5; Vul N = 5; Int N = 31); (C) the best and worst performing 20% (Res N = 8; Vul N = 8; Int N = 25); (D) the best and worst performing 25% (Res N = 10; Vul N = 10; Int N = 21); (E) the best and worst performing 33% (Res N = 13; Vul N = 13; Int N = 15); (F) the best and worst performing 50% (Res N = 20; Vul N = 21; All N = 41). See Table 2 for t-test comparisons between Res and Vul groups. The top and bottom axis labels depict the study design: Baseline day 2 (B2, 1000 h–2400 h), SR1 (0200 h, 0800 h–0200 h), SR2–SR4 (0800 h–0200 h), SR5 (0800 h–2000 h), Recovery days 1–4 (R1–R4, 1000 h–2000 h), and total sleep deprivation day (TSD, 2200 h–2000 h). Light blue lines and light gray lines depict individual DS total number correct profiles for the Res and Vul groups, respectively; the dark blue line and the dark gray line depict averaged DS total number correct profiles for the Res and Vul groups, respectively. The black dotted line depicts the Int group (except for 50%, for which this line depicts all participants) average DS total number correct profile. Breaks in the lines indicate missing data.

Figure 5.

Figure 5.

Open in a new tab

Resilient (Res), Vulnerable (Vul), and Intermediate (Int) group Digit Span (DS) total number correct performance profiles across the study using six different thresholds within the Change from Baseline approach. Res, Vul, and Int groups were determined by subtracting each participant’s mean DS total number correct across baseline day 2 (B2) from their mean DS total number correct across sleep restriction days 1–5 (SR1–SR5) (e.g. the greater the average change from baseline score, the more resilient) and using the following six thresholds: (A) ±1 standard deviation (SD) (Res N = 7; Vul N = 5; Int N = 29); (B) the best and worst performing 12.5% (Res N = 5; Vul N = 5; Int N = 31); (C) the best and worst performing 20% (Res N = 8; Vul N = 8; Int N = 25); (D) the best and worst performing 25% (Res N = 10; Vul N = 10; Int N = 21); (E) the best and worst performing 33% (Res N = 13; Vul N = 13; Int N = 15); (F) the best and worst performing 50% (Res N = 20; Vul N = 21; All N = 41). See Table 2 for t-test comparisons between Res and Vul groups. The top and bottom axis labels depict the study design: B2 (1000 h–2400 h), SR1 (0200 h, 0800 h–0200 h), SR2–SR4 (0800 h–0200 h), SR5 (0800 h–2000 h), Recovery days 1–4 (R1–R4, 1000 h–2000 h), and total sleep deprivation day (TSD, 2200 h–2000 h). Light blue lines and light gray lines depict individual DS total number correct profiles for the Res and Vul groups, respectively; the dark blue line and the dark gray line depict averaged DS total number correct profiles for the Res and Vul groups, respectively. The black dotted line depicts the Int group (except for 50%, for which this line depicts all participants) average DS total number correct profile. Breaks in the lines indicate missing data.

Figure 6.

Figure 6.

Open in a new tab

Resilient (Res), Vulnerable (Vul), and Intermediate (Int) group Digit Span (DS) total number correct performance profiles across the study using six different thresholds within the Variance approach. Res, Vul, and Int groups were determined by intraindividual variance in DS total number correct from all test administrations during sleep restriction days 1–5 (SR1–SR5) (e.g. the less variance, the more resilient) and using the following six thresholds: (A) ±1 standard deviation (SD) (Res N = 5; Vul N = 4; Int N = 32); (B) the best and worst performing 12.5% (Res N = 5; Vul N = 5; Int N = 31); (C) the best and worst performing 20% (Res N = 8; Vul N = 8; Int N = 25); (D) the best and worst performing 25% (Res N = 10; Vul N = 10; Int N = 21); (E) the best and worst performing 33% (Res N = 13; Vul N = 13; Int N = 15); (F) the best and worst performing 50% (Res N = 20; Vul N = 21; All N = 41). See Table 2 for t-test comparisons between Res and Vul groups. The top and bottom axis labels depict the study design: Baseline day 2 (B2, 1000 h–2400 h), SR1 (0200 h, 0800 h–0200 h), SR2–SR4 (0800 h–0200 h), SR5 (0800 h–2000 h), Recovery days 1–4 (R1–R4, 1000 h–2000 h), and total sleep deprivation day (TSD, 2200 h–2000 h). Light blue lines and light gray lines depict individual DS total number correct profiles for the Res and Vul groups, respectively; the dark blue line and the dark gray line depict averaged DS total number correct profiles for the Res and Vul groups, respectively. The black dotted line depicts the Int group (except for 50%, for which this line depicts all participants) average DS total number correct profile. Breaks in the lines indicate missing data.

Statistical analysis

Statistical analyses were conducted in the R software environment [57]. BMI, age, and sex composition of the Res and Vul groups were compared for each approach at the 12.5%, 33%, and 50% thresholds for DSST and DS (comparisons for these supplemental analyses were restricted to these three thresholds because the 12.5% threshold is the most restrictive, and the 33% and 50% thresholds are the least restrictive and most commonly utilized divisions) via one-way analysis of variance (ANOVA) for BMI and age and via chi-square tests for sex. Race comparisons were not evaluated between the Res and Vul groups at any threshold since the chi-squared sample size requirements were not met in each cell. Additionally, pre-study total sleep time (TST, measured by actigraphy from 7 to 14 days before the study) and B2 TST (measured by PSG) were evaluated for Res and Vul groups at the 12.5%, 33%, and 50% thresholds for both measures via one-way ANOVA.

Kendall’s tau-b correlations [58, 59] compared the categorizations of participants (i.e. whether they were in the Res, Vul, or Int group) across the three approaches within each measure at each threshold (e.g. the DSST number correct Raw Score approach at the 12.5% threshold compared to the DSST number correct Change from Baseline approach at the 12.5% threshold). Additionally, Kendall’s tau-b correlations compared the categorizations of participants between measures and approaches at all thresholds (e.g. Approach and threshold categorization of DSST number correct compared to approach and threshold categorization of DS total number correct). Kendall’s tau-b was used for these comparisons due to its nonparametric nature, and its ability to account for the repeating of values (e.g. ties in the ranking of data points) and to analyze ordinal data; given these criteria, it is considered more accurate relative to Spearman’s rank correlation for analyzing this dataset [59, 60]. Tau-b strength was defined as tau-b = 0.00 to ±0.09: zero; ±0.10 to ±0.39: weak; ±0.40 to ±0.69: moderate; ±0.70 to ±0.99: strong; ±1.00: perfect [61].

Bias-corrected and accelerated (BCa) bootstrapped t-tests with 5,000 iterations [62, 63] compared average DSST number correct or average DS total number correct from the 1000 h to 2000 h test bouts between the Res and Vul groups for each approach and at each threshold on each day of the study (e.g. DSST number correct for the Raw Score approach Res group at the 12.5% threshold compared to DSST number correct for the Raw Score approach Vul group at the 12.5% threshold on B2). BCa bootstrapped t-tests with 5,000 iterations also compared average DSST or DS performance of Res and Vul groups across SR1–SR5 (e.g. the Raw Score approach Res group for DSST number correct at the 12.5% threshold compared to the Raw Score approach Vul group for DSST number correct at the 12.5% threshold across SR1–SR5).

The Benjamini-Hochberg False Discovery Rate (FDR) correction [64] was applied to all bootstrapped t-test p-values and Kendall’s tau-b correlation p-values to account for multiplicity. Only 10.64% of these p-values became nonsignificant with FDR correction, and all presented p-values from t-tests and Kendall’s tau-b correlations are corrected.

Results

Participant characteristics

The Res and Vul groups defined by any approach or at any threshold did not significantly differ in BMI, age, or sex at the 12.5%, 33%, or 50% thresholds for either DSST number correct or DS total number correct (F(1) = 0.002–2.862, p = 0.099–0.964; χ(1) = 0.000–0.650, p = 0.420–1.000; Supplementary Table S2). Additionally, the DSST number correct and DS total number correct Res and Vul groups did not differ significantly in pre-study or B2 TST (F(1) = 0–3.355, p = 0.117–0.995) at the 12.5%, 33%, or 50% thresholds, except for DSST number correct by the Variance approach at the 50% threshold (F(1) = 4.338, p = 0.044), whereby the Res group had significantly longer pre-study TST than the Vul group (Supplementary Table S2).

Digit symbol substitution task

Participants were grouped into Res, Vul, and Int groups by all three approaches (Raw Score, Change from Baseline, and Variance) at all thresholds, except for by the Variance approach at the ±1 SD threshold, whereby a Res group was not formed (N = 0) due to the absence of individuals whose variance in DSST number correct across SR1–SR5 was greater than 1 SD below the mean. For the Raw Score approach, the Res group had a significantly greater average DSST number correct across SR1–SR5 than the Vul group at all thresholds (p < 0.001). For the Change from Baseline approach, the Res group had significantly greater average DSST number correct across SR1–SR5 than the Vul group at the ±1 SD, 12.5% and 20% thresholds (p ≤ 0.001–0.003), but not at the 25%, 33%, and 50% thresholds (p = 0.133–0.344). For the Variance approach, the Res and Vul groups were not significantly different in average DSST number correct across SR1–SR5 at any threshold (p = 0.089–0.944). The DSST number correct profiles of the Res, Vul, and Int groups across the entire study defined by the Raw Score, Change from Baseline, and Variance approaches at the six thresholds are depicted in Figures 1–3, respectively.

Comparison of DSST number correct resilient and vulnerable approaches.

The Kendall’s tau-b correlation comparing the categorizations of the Raw Score and Variance approaches was significant at the ±1 SD threshold and the tau-b value was weak (τb = 0.373; p = 0.044; Table 1). Kendall’s tau-b correlations comparing the categorizations of the Change from Baseline and Variance approaches were significant at the ±1 SD, 20%, and 33% thresholds and the tau-b values were moderate (τb = 0.411–0.586; p = 0.002–0.031; Table 1). All remaining comparisons were not significant (τb = 0.024–0.370; p = 0.051–0.880; Table 1).

Table 1.

Kendall’s tau-b correlations comparing the categorization of participants into the Resilient, Intermediate, and Vulnerable groups for Digit Symbol Substitution Test (DSST) number correct and Digit Span Task (DS) total number correct based on three approaches*

DSST number correct DS total number correct
Threshold Approach 1 Approach 2 Tau-b P Threshold Approach 1 Approach 2 Tau-b P
±1 SDa Rawb Baselinec 0.346 0.051 ±1 SD Raw Baseline 0.250 0.234
Raw Varianced 0.373 0.044 Raw Variance −0.096 0.741
Baseline Variance 0.586 0.002 Baseline Variance −0.009 1.000
12.5% Raw Baseline 0.278 0.113 12.5% Raw Baseline 0.284 0.234
Raw Variance 0.185 0.264 Raw Variance −0.093 0.741
Baseline Variance 0.370 0.051 Baseline Variance 0.000 1.000
20% Raw Baseline 0.278 0.110 20% Raw Baseline 0.282 0.234
Raw Variance 0.233 0.162 Raw Variance −0.116 0.741
Baseline Variance 0.461 0.026 Baseline Variance −0.056 0.839
25% Raw Baseline 0.175 0.264 25% Raw Baseline 0.312 0.234
Raw Variance 0.281 0.110 Raw Variance −0.227 0.251
Baseline Variance 0.350 0.051 Baseline Variance −0.090 0.741
33% Raw Baseline 0.102 0.496 33% Raw Baseline 0.301 0.234
Raw Variance 0.140 0.361 Raw Variance −0.240 0.234
Baseline Variance 0.411 0.031 Baseline Variance 0.000 1.000
50% Raw Baseline 0.219 0.230 50% Raw Baseline 0.219 0.332
Raw Variance 0.024 0.880 Raw Variance −0.269 0.234
Baseline Variance 0.317 0.110 Baseline Variance −0.074 0.824
Open in a new tab

*Three different approaches (Raw Score, Change from Baseline, and Variance) defined Resilient and Vulnerable groups based on sleep restriction performance within each measure.

a: SD = standard deviation; b: Raw = Raw Score approach; c: Baseline = Change from Baseline approach; d: Variance = Variance approach.

Kendall’s tau-b correlation coefficients and Benjamini-Hochberg corrected P-values are presented.

Comparison of DSST number correct resilient and vulnerable groups by day.

For the Raw Score approach, the Res group had significantly greater average DSST number correct than the Vul group on all days and all thresholds (p ≤ 0.001; Table 2; Figure 1). The Res and Vul groups determined by the Change from Baseline approach were significantly different on B2 (33% threshold), SR2 (±1 SD and 12.5% thresholds), SR3–SR5 (±1 SD, 12.5%, and 20% thresholds), R1 (20% threshold), R2 and R3 (±1 SD and 12.5% thresholds), and TSD (±1 SD, 12.5%, and 20% thresholds) (p ≤ 0.001–0.047; Table 2; Figure 2). For the Change from Baseline approach, the Res group had greater DSST number correct than the Vul group, except on B2 at the 33% threshold whereby the Vul group performed better than the Res group (p = 0.016; Table 2). For the Variance approach, the Res group had significantly greater average DSST number correct than the Vul group only on SR5 at the 12.5% threshold (p = 0.021; Table 2; Figure 3). All remaining comparisons were not significant (p = 0.077–0.988; Table 2).

Table 2.

Comparisons of Resilient and Vulnerable group means for Digit Symbol Substitution Test (DSST) number correct and Digit Span Task (DS) total number correct on each study day within each approach*

Study day DSST number correct DS total number correct
Threshold Raw score P Change from Baseline P Variance P Threshold Raw score P Change from Baseline P Variance P
B2a ±1 SDe <0.001 0.163 — ±1 SD <0.001 0.673 0.871
12.5% <0.001 0.140 0.400 12.5% <0.001 0.988 0.591
20% <0.001 0.326 0.724 20% <0.001 0.684 0.938
25% <0.001 0.090 0.986 25% <0.001 0.813 0.949
33% <0.001 0.016 0.484 33% <0.001 0.972 0.949
50% <0.001 0.660 0.319 50% <0.001 0.972 0.327
SR1b ±1 SD <0.001 0.295 — ±1 SD 0.001 0.198 0.938
12.5% <0.001 0.272 0.988 12.5% <0.001 0.070 0.800
20% <0.001 0.399 0.862 20% <0.001 0.272 0.967
25% <0.001 0.948 0.621 25% <0.001 0.121 0.949
33% <0.001 0.769 0.844 33% <0.001 0.095 0.853
50% <0.001 0.603 0.641 50% <0.001 0.099 0.137
SR2 ±1 SD <0.001 <0.001 — ±1 SD <0.001 0.210 0.972
12.5% <0.001 <0.001 0.621 12.5% <0.001 0.037 0.917
20% <0.001 0.069 0.450 20% <0.001 0.210 0.967
25% <0.001 0.548 0.310 25% <0.001 0.064 0.917
33% <0.001 0.926 0.862 33% <0.001 0.083 0.758
50% <0.001 0.400 0.789 50% <0.001 0.121 0.200
SR3 ±1 SD <0.001 <0.001 — ±1 SD <0.001 0.004 0.949
12.5% <0.001 <0.001 0.355 12.5% <0.001 0.001 0.970
20% <0.001 0.014 0.310 20% <0.001 0.019 0.949
25% <0.001 0.250 0.260 25% <0.001 0.013 0.949
33% <0.001 0.494 0.703 33% <0.001 0.003 0.725
50% <0.001 0.220 0.983 50% <0.001 0.041 0.272
SR4 ±1 SD <0.001 0.001 — ±1 SD 0.002 0.002 0.972
12.5% <0.001 <0.001 0.421 12.5% 0.002 <0.001 0.881
20% <0.001 <0.001 0.450 20% <0.001 <0.001 0.881
25% <0.001 0.140 0.306 25% <0.001 <0.001 0.967
33% <0.001 0.457 0.724 33% <0.001 <0.001 0.949
50% <0.001 0.094 0.960 50% <0.001 0.008 0.412
SR5 ±1 SD <0.001 0.007 — ±1 SD <0.001 0.001 0.917
12.5% <0.001 0.007 0.021 12.5% <0.001 <0.001 0.753
20% <0.001 <0.001 0.097 20% <0.001 <0.001 0.972
25% <0.001 0.185 0.077 25% <0.001 0.002 0.949
33% <0.001 0.455 0.475 33% <0.001 0.003 0.853
50% <0.001 0.186 0.862 50% <0.001 0.006 0.272
R1c ±1 SD <0.001 0.080 — ±1 SD <0.001 <0.001 0.686
12.5% <0.001 0.093 0.862 12.5% <0.001 <0.001 0.598
20% <0.001 0.042 0.662 20% <0.001 <0.001 0.972
25% <0.001 0.399 0.450 25% <0.001 <0.001 0.972
33% <0.001 0.621 0.988 33% <0.001 0.002 0.970
50% <0.001 0.301 0.494 50% <0.001 0.013 0.456
R2 ±1 SD <0.001 0.026 — ±1 SD 0.002 0.005 0.598
12.5% <0.001 0.026 0.988 12.5% 0.002 0.004 0.616
20% <0.001 0.144 0.953 20% <0.001 0.023 0.773
25% <0.001 0.621 0.871 25% <0.001 0.012 0.949
33% <0.001 0.986 0.642 33% <0.001 0.009 0.972
50% <0.001 0.476 0.400 50% <0.001 0.025 0.400
R3 ±1 SD <0.001 0.032 — ±1 SD 0.002 0.014 0.949
12.5% <0.001 0.041 0.862 12.5% 0.002 0.003 0.967
20% <0.001 0.269 0.937 20% <0.001 0.018 0.967
25% <0.001 0.739 0.925 25% <0.001 0.004 0.938
33% <0.001 0.953 0.619 33% <0.001 0.008 0.870
50% <0.001 0.337 0.377 50% <0.001 0.053 0.237
R4 ±1 SD <0.001 0.300 — ±1 SD 0.007 0.037 0.591
12.5% <0.001 0.295 0.399 12.5% 0.006 0.017 0.507
20% <0.001 0.494 0.876 20% <0.001 0.016 0.739
25% <0.001 0.872 0.975 25% <0.001 0.008 0.735
33% <0.001 0.953 0.511 33% <0.001 0.005 0.758
50% <0.001 0.457 0.341 50% <0.001 0.036 0.547
TSD d ±1 SD <0.001 0.001 — ±1 SD 0.002 0.016 0.588
12.5% <0.001 0.001 0.301 12.5% 0.002 0.067 0.598
20% <0.001 0.047 0.092 20% <0.001 0.022 0.551
25% 0.001 0.202 0.080 25% <0.001 0.008 0.591
33% 0.001 0.308 0.328 33% <0.001 0.013 0.476
50% <0.001 0.069 0.603 50% <0.001 0.179 0.881
Open in a new tab

*Three different approaches (Raw Score, Change from Baseline, and Variance) defined Resilient and Vulnerable groups based on sleep restriction performance within each measure.

a: B2 = Baseline day 2; b: SR = sleep restriction day; c: R = recovery day; d: TSD = total sleep deprivation day; e: SD = standard deviation.

Bias-corrected and accelerated bootstrapped t-test P-values are presented. The Benjamini-Hochberg correction for multiple comparisons was applied to all P-values. Analyses were not conducted for the Variance approach for DSST number correct at the ±1 SD threshold due to the absence of a Resilient group.

Digit span task

Participants were grouped into Res, Vul, and Int groups by all three approaches (Raw Score, Change from Baseline, and Variance) at all thresholds. For the Raw Score approach, the Res group had significantly greater average DS total number correct across SR1–SR5 than the Vul group at all thresholds (p < 0.001). For the Change from Baseline approach, the Res group had significantly greater average DS total number correct across SR1–SR5 than the Vul group at all thresholds (p = 0.001–0.013) excluding the ±1 SD threshold (p = 0.114). For the Variance approach, the Res and Vul groups did not significantly differ in average DS total number correct across SR1–SR5 at any threshold (p = 0.269–0.973). The DS total number correct profiles of the Res, Vul, and Int groups across the entire study defined by the Raw Score, Change from Baseline, and Variance approaches at the six thresholds are depicted in Figures 4–6, respectively.

Comparison of DS total number correct resilient and vulnerable approaches.

Kendall’s tau-b correlations indicated that the categorizations of the Raw Score, Change from Baseline, and Variance approaches were weakly correlated and nonsignificant within any comparison or at any threshold (τb = −0.269–0.312; p = 0.234–1.000; Table 1).

Comparison of DS total number correct resilient and vulnerable groups by day.

For the Raw Score approach, the Res group had significantly greater average DS total number correct than the Vul group on all days at all thresholds (p ≤ 0.001–0.007; Table 2; Figure 4). For the Change from Baseline approach, the Res group had significantly greater average DS total number correct than the Vul group on SR2 (12.5% threshold), SR3–R2 (all thresholds), R3 (all thresholds except 50%), R4 (all thresholds), and TSD (±1 SD, 20%, 25%, and 33% thresholds) (p ≤ 0.001–0.041; Table 2; Figure 5). For the Variance approach, the Res and Vul groups were not significantly different for any day or at any threshold (p = 0.137–0.972; Table 2; Figure 6). All remaining comparisons were not significant (p = 0.053–0.972; Table 2).

Comparison of DSST number correct and DS total number correct resilient and vulnerable approaches

Kendall’s tau-b correlations indicated that the categorizations of the approaches and thresholds were not significantly correlated between DSST number correct and DS total number correct (τb = −0.190–0.530; p = 0.112–1.000; Table 3). Although not significant, there were tau-b values of moderate strength (τb = 0.40–0.53; p = 0.112–0.230; Table 3) between the DSST number correct Raw Score approach and the DS total number correct Raw Score approach at several thresholds. All other nonsignificant correlations ranged from zero to weak in strength.

Table 3.

Kendall’s tau-b correlations comparing the categorization of participants into the Resilient, Intermediate, and Vulnerable groups as defined by the three approaches* between Digit Symbol Substitution Test (DSST) number correct and Digit Span Task (DS) total number correct

DSST number correct
Raw score Change from Baseline Variance
Threshold ±1 SDa 12.5% 20% 25% 33% 50% ±1 SD 12.5% 20% 25% 33% 50% ±1 SD 12.5% 20% 25% 33% 50%
DS total number correct Raw score ±1 SD 0.50 0.45 0.43 0.44 0.28 0.32 0.00 0.00 0.00 −0.06 −0.05 0.13 0.12 0.00 0.07 0.25 0.11 0.04
12.5% 0.53 0.48 0.45 0.46 0.35 0.38 0.00 0.00 0.00 −0.06 −0.11 0.10 0.15 0.00 0.07 0.20 0.06 0.00
20% 0.35 0.30 0.29 0.26 0.14 0.22 0.00 0.00 0.00 0.00 0.09 0.22 0.11 0.00 0.05 0.15 0.00 0.00
25% 0.37 0.27 0.31 0.27 0.12 0.27 0.07 0.07 0.00 0.04 0.12 0.27 0.10 0.00 −0.06 0.04 −0.08 −0.07
33% 0.32 0.23 0.31 0.27 0.13 0.23 0.06 0.06 0.00 0.03 0.03 0.17 0.09 0.06 0.00 0.08 −0.07 −0.17
50% 0.26 0.19 0.37 0.40 0.23 0.32 0.10 0.10 0.15 0.20 0.17 0.22 0.21 0.19 0.15 0.20 0.12 −0.07
Change from Baseline ±1 SD 0.32 0.26 0.33 0.29 0.36 0.35 0.34 0.34 0.34 0.30 0.26 0.18 0.16 0.08 0.20 0.18 0.15 0.09
12.5% 0.35 0.28 0.37 0.33 0.34 0.38 0.28 0.28 0.30 0.26 0.23 0.19 0.15 0.10 0.23 0.20 0.17 0.00
20% 0.27 0.22 0.28 0.25 0.18 0.15 0.30 0.30 0.24 0.21 0.22 0.15 0.11 0.07 0.23 0.25 0.17 0.08
25% 0.36 0.33 0.35 0.31 0.23 0.27 0.33 0.33 0.26 0.23 0.24 0.20 0.10 0.07 0.20 0.22 0.11 0.00
33% 0.37 0.28 0.35 0.35 0.28 0.29 0.23 0.23 0.18 0.16 0.14 0.06 0.00 0.00 0.04 0.16 0.07 −0.06
50% 0.25 0.19 0.22 0.27 0.23 0.32 0.19 0.19 0.22 0.27 0.29 0.32 −0.08 −0.19 −0.08 0.07 0.06 0.02
Variance ±1 SD 0.19 0.20 0.31 0.21 0.24 0.15 0.20 0.20 0.15 0.13 0.18 0.05 0.17 0.21 0.08 0.14 0.25 0.26
12.5% 0.17 0.19 0.29 0.20 0.23 0.19 0.19 0.19 0.14 0.13 0.17 0.10 0.15 0.19 0.07 0.13 0.23 0.19
20% 0.19 0.22 0.28 0.25 0.31 0.30 0.07 0.07 0.11 0.00 0.00 0.00 0.11 0.15 0.11 0.20 0.26 0.08
25% 0.23 0.26 0.30 0.27 0.34 0.33 0.07 0.07 0.25 0.13 0.08 0.07 0.20 0.20 0.20 0.23 0.32 0.13
33% 0.14 0.22 0.26 0.19 0.26 0.23 0.11 0.11 0.22 0.16 0.10 0.06 0.18 0.17 0.22 0.20 0.24 0.12
50% −0.01 0.10 0.08 0.00 0.12 0.12 0.19 0.19 0.30 0.20 0.17 0.02 0.21 0.19 0.15 0.13 0.17 0.12
Open in a new tab

*Three different approaches (Raw Score, Change from Baseline, and Variance) defined Resilient and Vulnerable groups based on sleep restriction performance within each measure.

a: SD = standard deviation.

Kendall’s tau-b correlation coefficients are presented. The Benjamini-Hochberg correction was applied to all P-values. Bolded tau-b values indicate comparisons of the same thresholds between each measure.

Discussion

In the present study, we systematically examined three approaches and six thresholds to categorize individuals into Res and Vul groups based on cognitive throughput and working memory performance during sleep loss. The three approaches generally did not categorize participants similarly based on DSST performance, although the concordance between the Change from Baseline and Variance approaches was moderate. For DS performance, categorizations of Res and Vul groups by all three approaches were not significantly correlated. Additionally, there were no significant associations between the DSST and DS categorizations for any approach. The Res groups defined by the Raw Score approach had significantly better DSST and DS performance across all thresholds on all study days compared to the respective Vul groups, while the Res groups defined by the Change from Baseline approach had significantly better DSST and DS performance for several thresholds on most study days compared to the respective Vul groups. By contrast, groups categorized by the Variance approach generally did not have significantly different DSST and DS performance throughout the study. For the first time, our study demonstrated that several approaches and thresholds for defining cognitive throughput and working memory resilience and vulnerability to sleep loss are not synonymous.

Our hypothesis that the three approaches would show high concordance within each measure was not supported. For DSST performance, the Raw Score and Variance approaches were significantly and weakly correlated at one threshold (±1 SD) and the Change from Baseline and Variance approaches were significantly and moderately correlated at three thresholds (±1 SD, 20%, and 33%), while all other comparisons were not significant. The Change from Baseline and Variance approaches were the most comparable for DSST performance: those who had improvement or little change in SR DSST performance from baseline also had more stable performance during SR. Overall, however, the three approaches did not similarly categorize participants into Res and Vul groups for cognitive throughput performance. Similarly, based on DS performance, the three approaches were not significantly correlated at any threshold, demonstrating that the approaches did not categorize working memory resilience and vulnerability in a similar manner. Since the three approaches we examined likely denote distinct representations of resilience and vulnerability with minimal crossover, the within-measure correlations suggest that selecting an approach for resilient and vulnerable categorization based on DSST and DS performance should be done carefully.

Our hypothesis that the three approaches would show high concordance between the DSST and DS measures also was not supported. The comparisons of Res and Vul groups by the three approaches between the measures revealed no significant correlations. Of note, the comparisons between the Raw Score approach for DSST and the Raw Score approach for DS revealed moderate (although nonsignificant) correlations, suggesting the greatest crossover of individuals in Res and Vul groups when categorizing DSST and DS resilience or vulnerability may occur using Raw Score. Nevertheless, the absence of significant correlations between the DSST and DS Res and Vul groups suggests that cognitive throughput and working memory display differential resilience and vulnerability. Our results align with prior findings that the DSST and the DS show differential sleep loss effects [6, 42, 43, 50]. However, some studies suggest that the DSST and DS are related, exemplified by significant positive Spearman rank correlations on averaged scores between these measures during SR and TSD exposures [5, 6]. Thus, further research is needed to clarify the relationship between the DSST and DS following sleep loss.

Our hypothesis that the resilient groups would perform better than their respective vulnerable groups for all approaches and thresholds across all sleep loss days was partially supported. The Res groups categorized by the Raw Score approach had significantly better DSST and DS performance across all thresholds on all study days compared to the respective Vul groups. This finding demonstrates that the DSST and DS performance differences detected during SR (i.e. the basis of our resilient and vulnerable categorization) were also apparent across the other study phases including baseline, recovery, and TSD. This result was expected given that working memory and cognitive throughput resilience or vulnerability have been found to be consistent when individuals are exposed to both chronic SR and TSD [5, 6]. Given that we categorized resilient and vulnerable groups based on performance during SR, future studies should categorize groups based on TSD performance and investigate whether the group differences remain stable across chronic SR and recovery. Notably, this is the first study to demonstrate that resilient or vulnerable groups defined by DSST and DS SR performance maintain this designation throughout an extended recovery period. Thus, the Raw Score approach can be reliably used to differentiate between resilient and vulnerable groups based on DSST and DS performance during chronic SR.

The Res groups categorized by the Change from Baseline approach had significantly better performance compared to the respective Vul groups on all study days for DSST (except B2, SR1, and R4) and for DS (except B2 and SR1). The nonsignificant R4 group differences suggest a return to baseline performance for both groups on the last recovery day for DSST. Although the Res and Vul significant differences for the DSST were generally present at the more restrictive performance thresholds (±1 SD, 12.5%, and 20%), the Res and Vul significant differences for the DS were generally present across all thresholds. Thus, the restrictive thresholds may have produced more extreme performance differences for the DSST compared to the other thresholds. Future research should investigate these threshold differences between the DSST and DS and consider a learning effect (i.e. better performance with repeated administrations) as a possible underlying factor, given that this effect is apparent for the DSST [43, 65] but is not as strongly observed for the DS. Additionally, the differences between Res and Vul groups were apparent following SR1 and throughout recovery, suggesting that those individuals with improvement or little change in SR performance from baseline also demonstrate better DSST and DS performance during recovery and TSD.

Lastly, the Res and Vul groups categorized by the Variance approach did not have significantly different DSST and DS performance for any threshold on any day across the study, except for the 12.5% threshold on SR5 for DSST performance. Therefore, groups defined by intra-individual variance during sleep loss did not demonstrate DSST or DS performance differences across baseline, SR, recovery, or TSD; thus, stability, as defined in our study, is not a useful approach for categorizing resilient and vulnerable groups to sleep loss based on working memory and cognitive throughput measures. Interestingly, previous studies without sleep loss have posited that a resilient or vulnerable designation based on intraindividual variability is task-dependent, whereby inconsistent performance may signal vulnerability on reaction-time based-tasks but may signal improvement on executive function tests [27, 66, 67]. Since our study used the latter, we might have expected significant inverse relationships between the Raw Score and Variance approaches, but these were not detected. The sleep loss component of our study may have altered this relationship; therefore, further research is needed to clarify how resilience and vulnerability of intraindividual variance relate to average performance for executive function tests such as the DSST and DS during sleep loss.

There are a few limitations to the present study. First, the approaches and thresholds we investigated are not an exhaustive list of methods used to define cognitive throughput and working memory resilient and vulnerable groups. Additionally, we examined the DSST and DS in this study, therefore our findings may not be generalizable to other neurobehavioral measures, particularly subjective variables [5, 6, 41, 44, 68, 69]. Our sample also included only healthy adults; thus, we are unable to generalize our findings to populations with sleep disorders or other medical conditions. Furthermore, since our sample was between the ages of 21 and 49 and predominantly included African American participants, we cannot generalize our findings to adolescents or to adults above the age of 50, or to other racial and/or ethnic populations. Lastly, since we used averaged scores, we did not directly assess the impact of time-of-day fluctuations on performance; however, the Variance approach served as a proxy to capture such time-of-day effects on cognitive performance scores, in that participants exhibited greater variation/fluctuation in performance if they were more vulnerable or sensitive to time-of-day effects [24–33].

Our findings are relevant to the broader literature on individual differences in executive functioning following sleep loss, particularly in relation to the definitions of resilience and vulnerability. A few genetic studies have established links to resilience or vulnerability in executive functioning during sleep deprivation [70, 71], and several neuroimaging studies have demonstrated greater left frontoparietal region activation among resilient individuals during working memory tasks at baseline [19, 53, 72, 73]. Additionally, vulnerable individuals have been shown to experience safety, well-being, and work performance-related deficits following sleep loss [2, 4, 7–11, 74], and may be at risk for future health concerns [75]. Importantly, consistent definitions of cognitive throughput and working memory resilience and vulnerability are imperative for understanding real-world implications, and for developing biomarkers and effective mitigation strategies for the impairing effects of sleep loss.

Supplementary Material

zsab197_suppl_Supplementary_Materials
Click here for additional data file. (74.5KB, docx)

Acknowledgments

We thank the faculty and staff of the Unit of Experimental Psychiatry for their contributions to this study in terms of data collection. N.G. designed the overall study, T.E.B. conducted statistical analyses of the data, and N.G. provided financial support. T.E.B., C.E.C, E.M.Y., C.A.A., and N.G. prepared the manuscript. All authors reviewed and approved the final manuscript.

Funding

This work was primarily supported by Department of the Navy, Office of Naval Research (Award No. N00014-11-1-0361) to N.G. Other support was provided by National Aeronautics and Space Administration (NASA) grant NNX14AN49G and grant 80NSSC20K0243 (to N.G.), National Institutes of Health grant NIH R01DK117488 (to N.G.), and Clinical and Translational Research Center grant UL1TR000003. None of the sponsors had any role in the following: design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

Disclosure Statements

Financial Disclosure: None.

Nonfinancial Disclosure: None.

Data Availability

The data underlying this article will be shared on reasonable request to the corresponding author.

References

  • 1. Goel N, et al. Neurocognitive consequences of sleep deprivation. Semin Neurol. 2009;29(4):320–339. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Belenky G, et al. Patterns of performance degradation and restoration during sleep restriction and subsequent recovery: a sleep dose-response study. J Sleep Res. 2003;12(1):1–12. [DOI] [PubMed] [Google Scholar]
  • 3. Goel N. Neurobehavioral effects and biomarkers of sleep loss in healthy adults. Curr Neurol Neurosci Rep. 2017;17(11):89. [DOI] [PubMed] [Google Scholar]
  • 4. Van Dongen HP, et al. The cumulative cost of additional wakefulness: dose-response effects on neurobehavioral functions and sleep physiology from chronic sleep restriction and total sleep deprivation. Sleep. 2003;26(2):117–126. doi: 10.1093/sleep/26.2.117. [DOI] [PubMed] [Google Scholar]
  • 5. Yamazaki EM, et al. Robust stability of trait-like vulnerability or resilience to common types of sleep deprivation in a large sample of adults. Sleep. 2020;43(6). doi: 10.1093/sleep/zsz292. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Dennis LE, et al. Healthy adults display long-term trait-like neurobehavioral resilience and vulnerability to sleep loss. Sci Rep. 2017;7(1):14889. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Dawson D, et al. Fatigue, alcohol and performance impairment. Nature. 1997;388(6639):235. [DOI] [PubMed] [Google Scholar]
  • 8. Fairclough SH, et al. Impairment of driving performance caused by sleep deprivation or alcohol: a comparative study. Hum Factors. 1999;41(1):118–128. [DOI] [PubMed] [Google Scholar]
  • 9. Williamson AM, et al. Moderate sleep deprivation produces impairments in cognitive and motor performance equivalent to legally prescribed levels of alcohol intoxication. Occup Environ Med. 2000;57(10):649–655. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Lockley SW, et al. ; Harvard Work Hours, Health and Safety Group. Effects of health care provider work hours and sleep deprivation on safety and performance. Jt Comm J Qual Patient Saf. 2007;33(Suppl 11):7–18. [DOI] [PubMed] [Google Scholar]
  • 11. Leonard C, et al. The effect of fatigue, sleep deprivation and onerous working hours on the physical and mental wellbeing of pre-registration house officers. Ir J Med Sci. 1998;167(1):22–25. [DOI] [PubMed] [Google Scholar]
  • 12. Caldwell JL, et al. Differential effects of modafinil on performance or low-performing and high-performing individuals during total sleep deprivation. Pharmacol Biochem Behav. 2020;196:172968. [DOI] [PubMed] [Google Scholar]
  • 13. Chua EC, et al. Sustained attention performance during sleep deprivation associates with instability in behavior and physiologic measures at baseline. Sleep. 2014;37(1):27–39. doi: 10.5665/sleep.3302. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Moreno-Villanueva M, et al. The degree of radiation-induced DNA strand breaks is altered by acute sleep deprivation and psychological stress and is associated with cognitive performance in humans. Sleep. 2018;41(7). doi: 10.1093/sleep/zsy067. [DOI] [PubMed] [Google Scholar]
  • 15. Patanaik A, et al. Classifying vulnerability to sleep deprivation using baseline measures of psychomotor vigilance. Sleep. 2015;38(5):723–734. doi: 10.5665/sleep.4664. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Chua EC, et al. Classifying attentional vulnerability to total sleep deprivation using baseline features of Psychomotor Vigilance Test performance. Sci Rep. 2019;9(1):12102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Riontino L, et al. Individual differences in working memory efficiency modulate proactive interference after sleep deprivation. Psychol Res. 2021;85(2):480–490. [DOI] [PubMed] [Google Scholar]
  • 18. Chee MW, et al. Lapsing when sleep deprived: neural activation characteristics of resistant and vulnerable individuals. Neuroimage. 2010;51(2):835–843. [DOI] [PubMed] [Google Scholar]
  • 19. Chee MWL, et al. Functional imaging of working memory following normal sleep and after 24 and 35 h of sleep deprivation: correlations of fronto-parietal activation with performance. Neuroimage. 2005;31(1):419–428. [DOI] [PubMed] [Google Scholar]
  • 20. Kong D, et al. Increased automaticity and altered temporal preparation following sleep deprivation. Sleep. 2015;38(8):1219–1227. doi: 10.5665/sleep.4896. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Chuah LYM, et al.. Donepezil improved episodic memory in young individuals vulnerable to the effects of sleep deprivation. Sleep. 2009;32(8):999–1010. doi: 10.1093/sleep/32.8.999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Xu J, et al.. Frontal metabolic activity contributes to individual differences in vulnerability towards total sleep deprivation-induced changes in cognitive function. J Sleep Res. 2016;25(2):169–180. [DOI] [PubMed] [Google Scholar]
  • 23. Patanaik A, et al. Predicting vulnerability to sleep deprivation using diffusion model parameters. J Sleep Res. 2014;23(5):576–584. [DOI] [PubMed] [Google Scholar]
  • 24. Bruhn P, et al. Reaction time variability in epileptic and brain-damaged patients. Cortex. 1977;13(4):373–384. [DOI] [PubMed] [Google Scholar]
  • 25. Fuentes K, et al. Intraindividual variability in cognitive performance in persons with chronic fatigue syndrome. Clin Neuropsychol. 2001;15(2):210–227. [DOI] [PubMed] [Google Scholar]
  • 26. Hultsch DF, et al. Intraindividual variability in cognitive performance in older adults: comparison of adults with mild dementia, adults with arthritis, and healthy adults. Neuropsychology. 2000;14(4):588–598. [DOI] [PubMed] [Google Scholar]
  • 27. Allaire JC, et al. Intraindividual variability may not always indicate vulnerability in elders’ cognitive performance. Psychol Aging. 2005;20(3):390–401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Blatter K, et al. Circadian rhythms in cognitive performance: methodological constraints, protocols, theoretical underpinnings. Physiol Behav. 2007;90(2-3):196–208. [DOI] [PubMed] [Google Scholar]
  • 29. Banks S, et al. Behavioral and physiological consequences of sleep restriction. J Clin Sleep Med. 2007;3(5):519–528. [PMC free article] [PubMed] [Google Scholar]
  • 30. Doran SM, et al. Sustained attention performance during sleep deprivation: evidence of state instability. Arch Ital Biol. 2001;139(3):253–267. [PubMed] [Google Scholar]
  • 31. Schmidt C, et al. A time to think: circadian rhythms in human cognition. Cogn Neuropsychol. 2007;24(7):755–789. [DOI] [PubMed] [Google Scholar]
  • 32. Rabbitt P, et al. There are stable individual differences in performance variability, both from moment to moment and from day to day. Q J Exp Psychol A. 2001;54(4):981–1003. [DOI] [PubMed] [Google Scholar]
  • 33. Adam M, et al. Age-related changes in the time course of vigilant attention during 40 hours without sleep in men. Sleep. 2006;29(1):55–57. doi: 10.1093/sleep/29.1.55. [DOI] [PubMed] [Google Scholar]
  • 34. Rocklage M, et al. White matter differences predict cognitive vulnerability to sleep deprivation. Sleep. 2009;32(8):1100–1103. doi: 10.1093/sleep/32.8.1100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Diekelmann S, et al. Sleep enhances false memories depending on general memory performance. Behav Brain Res. 2010;208(2):425–429. [DOI] [PubMed] [Google Scholar]
  • 36. Michael L, et al. Electrodermal lability as an indicator for subjective sleepiness during total sleep deprivation. J Sleep Res. 2012;21(4):470–478. [DOI] [PubMed] [Google Scholar]
  • 37. Salfi F, et al. Effects of total and partial sleep deprivation on reflection impulsivity and risk-taking in deliberative decision-making. Nat Sci Sleep. 2020;12:309–324. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Yeo BT, et al. Functional connectivity during rested wakefulness predicts vulnerability to sleep deprivation. Neuroimage. 2015;111:147–158. [DOI] [PubMed] [Google Scholar]
  • 39. Galli O. Predictors of Interindividual Differences in Vulnerability to Neurobehavioral Consequences of Chronic Partial Sleep Restriction [doctoral dissertation]. Philadelphia, PA: University of Pennsylvania; 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. St Hilaire MA, et al. Using a single daytime performance test to identify most individuals at high-risk for performance impairment during extended wake. Sci Rep. 2019;9(1):16681. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41. Frey DJ, et al. Inter- and intra-individual variability in performance near the circadian nadir during sleep deprivation. J Sleep Res. 2004;13(4):305–315. [DOI] [PubMed] [Google Scholar]
  • 42. Yamazaki EM, et al. Residual, differential neurobehavioral deficits linger after multiple recovery night following chronic sleep restriction or acute total sleep deprivation. Sleep. 2021;44(4). doi: 10.1093/sleep/zsaa224. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Honn KA, et al.. New insights into the cognitive effects of sleep deprivation by decomposition of a cognitive throughput task. Sleep. 2020; 43(7):1–14. doi: 10.1093/sleep/zsz319. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44. Tucker AM, et al. Effects of sleep deprivation on dissociated components of executive functioning. Sleep. 2010;33(1):47–57. doi: 10.1093/sleep/33.1.47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45. Chai WJ, et al.. Working memory from the psychological and neurosciences perspectives: a review. Front Psychol. 2018;9:401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46. Taylor DJ, et al.. Cognitive performance after sleep deprivation: does personality make a difference? Pers Individ Dif. 2003;34(7):1179–1193. [Google Scholar]
  • 47. Alhola P, et al. Sleep deprivation: impact on cognitive performance. Neuropsychiatr Dis Treat. 2007;3(5):553–567. [PMC free article] [PubMed] [Google Scholar]
  • 48. Kim DJ, et al. The effect of total sleep deprivation on cognitive functions in normal adult male subjects. Int J Neurosci. 2001;109(1–2):127–137. [DOI] [PubMed] [Google Scholar]
  • 49. Freidmann J, et al. Performance and mood during and after gradual sleep reduction. Psychophysiology. 1977;14(3):245–250. [DOI] [PubMed] [Google Scholar]
  • 50. Voderholzer U, et al.. Sleep restriction over several days does not affect long-term recall of declarative and procedural memories in adolescents. Sleep Med. 2011; 12(2):170–178. [DOI] [PubMed] [Google Scholar]
  • 51. Merikanto I, et al.. Influence of seasonal variation in mood and behavior on cognitive test performance among young adults. Nord J Psychiatry. 2012;66(5):303–310. [DOI] [PubMed] [Google Scholar]
  • 52. Gilmore GC, et al. Symbol-digit substitution and individual differences in visual search ability. Intelligence. 2004;32(1):47–64. [Google Scholar]
  • 53. Mu Q, et al. Decreased brain activation during a working memory task at rested baseline is associated with vulnerability to sleep deprivation. Sleep. 2005;28(4):433–446. doi: 10.1093/sleep/28.4.433. [DOI] [PubMed] [Google Scholar]
  • 54. Wechsler D. The Measurement of Adult Intelligence. Baltimore, MD: The Williams and Wilkins Company; 1939. [Google Scholar]
  • 55. Wechsler D. WAIS-R Manual: Wechsler Adult Intelligence Scale-revised. San Antonio, TX: The Psychological Corporation; 1981. [Google Scholar]
  • 56. Blackburn HL, et al. Revised administration and scoring of the digit span test. J Consult Psychol. 1957;21(2):139–143. [DOI] [PubMed] [Google Scholar]
  • 57. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2020. Retrieved from https://www.R-project.org/. [Google Scholar]
  • 58. Kassambara A. rstatix: Pipe-Friendly Framework for Basic Statistical Tests [R Package Version 0.6.0]. Retrieved from https://CRAN.R-project.org/package=rstatix. [Google Scholar]
  • 59. Kendall MG, et al. Rank Correlation Methods. 5th ed. New York, NY: Oxford University Press; 1990. [Google Scholar]
  • 60. Dodge Y. The Concise Encyclopedia of Statistics. Berlin, Germany: Springer Science + Business Media; 2008. [Google Scholar]
  • 61. Dancey CP, et al. Statistics Without Maths for Psychology. 4th ed. Essex, England: Pearson Education Limited; 2007. [Google Scholar]
  • 62. Weiss NA. wBoot: Bootstrap Methods. R Package Version 1.0.3; 2016. Retrieved from https://CRAN.R-project.org/package=wBoot. [Google Scholar]
  • 63. Mantua J, et al. Self-reported sleep need, subjective resilience, and cognitive performance following sleep loss and recovery sleep. Psychol Rep. 2021;124(1):210–226. [DOI] [PubMed] [Google Scholar]
  • 64. Benjamini Y, et al. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol. 1995;57(1):289–300. [Google Scholar]
  • 65. Cornelis C, et al. Preserved learning during the symbol-digit substitution test in patients with schizophrenia, age-matched controls, and elderly. Front Psychiatry. 2014;5:189. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66. Li S, et al. Short-term fluctuations in elderly people’s sensorimotor functioning predict text and spatial memory performance: the Macarthur successful aging studies. Gerontology. 2001;47(2):100–116. [DOI] [PubMed] [Google Scholar]
  • 67. Li SC, et al. Aging and attenuated processing robustness: evidence from cognitive and sensorimotor functioning. Gerontology. 2004;50(1):28–34. [DOI] [PubMed] [Google Scholar]
  • 68. Tkachenko O, et al. Interindividual variability in neurobehavioral response to sleep loss: a comprehensive review. Neurosci Biobehav Rev. 2018;89:29–48. [DOI] [PubMed] [Google Scholar]
  • 69. Louca M, et al. The effect of one night’s sleep deprivation on adolescent neurobehavioral performance. Sleep. 2014;37(11):1799–1807. doi: 10.5665/sleep.4174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70. Bachmann V, et al. The BDNF Val66Met polymorphism modulates sleep intensity: EEG frequency- and state-specificity. Sleep. 2012;35(3):335–344. doi: 10.5665/sleep.1690. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71. Grant LK, et al. Impaired cognitive flexibility during sleep deprivation among carriers of the Brain Derived Neurotrophic Factor (BDNF) Val66Met allele. Behav Brain Res. 2018;338:51–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72. Caldwell JA, et al. Are individual differences in fatigue vulnerability related to baseline differences in cortical activation? Behav Neurosci. 2005;119(3):694–707. [DOI] [PubMed] [Google Scholar]
  • 73. Cui J, et al. Microstructure of frontoparietal connections predicts individual resistance to sleep deprivation. Neuroimage. 2015;106:123–133. [DOI] [PubMed] [Google Scholar]
  • 74. Samkoff JS, et al. A review of studies concerning effects of sleep deprivation and fatigue on residents’ performance. Acad Med. 1991;66(11):687–693. [DOI] [PubMed] [Google Scholar]
  • 75. Sin NL, et al. Emotional vulnerability to short sleep predicts increases in chronic health conditions across 8 years. Ann Behav Med. 2021;kaab018. [published online ahead of print]. https://pubmed.ncbi.nlm.nih.gov/33821929/ [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

zsab197_suppl_Supplementary_Materials
Click here for additional data file. (74.5KB, docx)

Data Availability Statement

The data underlying this article will be shared on reasonable request to the corresponding author.

RESOURCES