Trait-like vulnerability of higher-order cognition and ability to maintain wakefulness during combined sleep restriction and circadian misalignment

Abstract

Study Objectives

Determine stability of individual differences in executive function, cognitive processing speed, selective visual attention, and maintenance of wakefulness during simulated sustained operations with combined sleep restriction and circadian misalignment.

Methods

Twenty healthy adults (eight female), aged 25.7 (±4.2 SD), body mass index (BMI) 22.3 (±2.1) kg/m² completed an 18-day protocol twice. Participants maintained habitual self-selected 8-hour sleep schedules for 2 weeks at home prior to a 4-day laboratory visit that included one sleep opportunity per day: 8 hours on night 1, 3 hours on night 2, and 3 hours on mornings 3 and 4. After 3 days of unscheduled sleep at home, participants repeated the entire protocol. Stability and task dependency of individual differences in performance were quantified by intra-class correlation coefficients (ICC) and Kendall’s Tau, respectively.

Results

Performance on Stroop, Visual Search, and the Maintenance of Wakefulness Test were highly consistent within individuals during combined sleep restriction and circadian misalignment. Individual differences were trait-like as indicated by ICCs (0.54–0.96) classified according to standard criteria as moderate to almost perfect. Individual differences on other performance tasks commonly reported in sleep studies showed fair to almost perfect ICCs (0.22–0.94). Kendall’s rank correlations showed that individual vulnerability to sleep restriction and circadian misalignment varied by task and by metric within a task.

Conclusions

Consistent vulnerability of higher-order cognition and maintenance of wakefulness to combined sleep restriction and circadian misalignment has implications for the development of precision countermeasure strategies for workers performing safety-critical tasks, e.g. military, police, health care workers and emergency responders.

Statement of Significance.

Combined sleep restriction and circadian misalignment produce robust, trait-like effects on higher-order cognitive functions and ability to maintain wakefulness. Individual trait vulnerability varied across tasks and within different metrics from the same tasks. This suggests that tests and countermeasures for sleep and circadian disruption that address impairments across multiple cognitive domains are likely to be optimally effective.

Introduction

Approximately, 20% of the US labor force works at night, in occupations such as mining, health care, the police force, and the military; these rates are similar in other nations [1–3]. These workers are often awake at night and asleep during the day, out of phase with their internal circadian clock. This state is termed circadian misalignment. Many night shift workers also sleep less than 6 hours per night [3]. Short sleep duration is very common in the military, where 41.8% report sleeping 5 hours or less per night [4], well below the 7 hours recommended for optimal health [5]. Military personnel are often required to maintain performance while experiencing profound sleep and circadian disturbance, due to rapid travel across time zones and continuous or sustained operations. Sleep restriction and circadian misalignment have negative health, performance, and safety consequences, including reduced effectiveness, efficiency, resilience, and readiness [6, 7], a greater risk of motor vehicle crashes and injuries [8], higher rates of posttraumatic stress disorder and risky health behaviors in military personnel [4, 9], more safety violations and citizen complaints for police officers [10], and greater risk of metabolic disease and cancer [11, 12]. To mitigate these health, performance, and safety risks, a greater understanding of vulnerability to sleep restriction and circadian misalignment is needed.

The degree of cognitive impairment induced by sleep loss and circadian misalignment varies considerably between individuals. Vulnerability also varies by cognitive domain within an individual, i.e. an individual may be vulnerable to sleep loss in one domain, and more resilient in another [13–16]. A trait is determined by testing participants repeatedly, and statistically comparing the between-subject variance to the within-subject variance. When performance is different between individuals but highly consistent within individuals it is considered-trait like [16, 17]. Trait-like responses have been demonstrated for vigilance, short-term memory, speed of visual and cognitive processing, self-reported sleepiness, and mood [14, 16], in response to both total sleep deprivation and to sleep restriction, and are stable across days and weeks [14, 16]. Stability of higher-order cognitive performance during combined sleep restriction and circadian misalignment, however, remains largely unexplored. It is also unclear whether individual differences in impairment on commonly examined vigilance, self-reported sleepiness, and memory tasks generalize to higher cognitive functions. Therefore, this study assessed individual differences in executive function, cognitive processing speed, selective visual attention, maintenance of wakefulness, vigilance, mood, and sleepiness during days of combined sleep restriction (3 hours of sleep per day) and circadian misalignment (wakefulness during habitual sleep periods, and sleep during habitual wake periods). This schedule was designed to simulate sustained military operations. We primarily focused on measures that are widely used to assess higher-order cognitive performance and maintenance of wakefulness, whose individual stability during combined sleep restriction and circadian misalignment have not been reported.

Methods

Participants

Healthy adults aged 18–35 years were recruited from the community by online advertisements, flyers, and word-of-mouth. A medical exam, blood chemistries, clinical electrocardiogram, psychological interview, health history interview, and polysomnography were conducted at either the Clinical and Translational Research Center or the Sleep and Chronobiology Laboratory to confirm that participants were free of medical, psychiatric, and sleep disorders, had a habitual sleep duration of 7–9 hours, body mass index (BMI) 18.5–27 kg/m², were not pregnant, had resided at the local altitude (~1600 m) or above for at least 1 year, had not travelled >1 time zone in the past 3 weeks, and had not worked night shift in the past year. Urine toxicology and alcohol breath testing (Lifeloc Technologies Model FC10) were conducted at the start of each laboratory visit to verify that participants were free of medication and recreational drugs. All participants provided written informed consent, and the protocol was approved by the Institutional Review Board of the University of Colorado Boulder and the Colorado Clinical and Translational Sciences Institute Scientific Advisory Review Committee.

Study design

Participants maintained habitual self-selected 8-hour sleep schedules for 2 weeks, then stayed in the laboratory for 4 days. After 3 days of ad libitum sleep at home this procedure was repeated, lasting a total of 39 days (Figure 1). Prior to study start, participants abstained from antibiotics for at least 3 months and from medications and prebiotics or probiotics for at least 1 month, to minimize changes in the microbiome as part of a separate study aim not examined in this report. Prior to each in-laboratory visit, participants abstained from over-the-counter pain relievers for 1 week and from caffeine, alcohol, and exercise for 3 days. Home sleep schedules were verified by wrist-worn actigraphy with concurrent light exposure monitoring (Actiwatch Spectrum Plus, Respironics), sleep diary, and time-stamped sleep-wake times (voicemail or online form). To control metabolic influences on performance and physiology, a scheduled macronutrient-controlled, energy-balanced diet was provided for 2 days prior and throughout the in-laboratory visit (15% protein, 30% fat, 55% carbohydrate). Caloric content was matched to participants’ resting metabolic rate (RMR × an activity factor of 1.5), determined during the medical exam and was increased by 6% for in-laboratory meals due to increased energy expenditure during sleep loss [18, 19]. The laboratory visits consisted of an 8-hour sleep opportunity at the participant’s habitual time, followed by one night-time and two day-time 3-hour sleep opportunities (sleep restriction and circadian misalignment). Throughout in-laboratory visits, participants were housed and tested individually in a temperature-controlled, sound-attenuated private suite. Light exposure was dim (<10 lux maximum, ~1.9 lux, ~0.6 W/m² in the angle of gaze) during scheduled wakefulness and was dark (0 lux) during scheduled sleep. Sleep and wakefulness were monitored continuously with electroencephalography (F3, C3, C4, O1 referenced to contralateral mastoids), electrooculography, chin electromyogram, and electrocardiogram (Siesta, Compumedics USA Inc., Charlotte, NC). Sleep was scored according to standard guidelines [20]. Computerized cognitive testing batteries were performed every 3 hours during scheduled wakefulness. A Maintenance of Wakefulness Test was performed 25 minutes before each cognitive battery. On the first evening of each laboratory visit, participants practiced cognitive tasks until the steep portion of the learning curve was eradicated [13]. These practice tests were not included in analyses. The cognitive battery comprised the Karolinska Sleepiness Scale (KSS), Positive and Negative Affect Scale (PANAS), Mathematical Addition Test (ADD), Psychomotor Vigilance Test (PVT), Conjunction Visual Search Task (CONJ), Stroop Color Word and Visual Analog Scales (VAS) of alertness and mood (Supplementary Table S1 shows task order within the battery).

Figure 1.

Experimental protocol. Timing was based on participants’ habitual sleep schedules with habitual bedtime denoted as the relative clock hour of midnight. Black bars indicate scheduled sleep episodes; grey areas indicate scheduled wakefulness in the laboratory; numbers in grey areas indicate cognitive batteries; hashed bars indicate unscheduled sleep at home.

Task parameters

Cognitive processing speed and the inhibitory control component of executive function were tested with a computerized version of the Stroop Color-Word Test [21]. The Stroop is a test of cognitive speed and executive functions including attention, inhibitory control, and mental flexibility [22]. Participants were instructed to respond as quickly and accurately as possible by pressing a response button with milliseconds (ms) accuracy, indicating the color of displayed text. Two hundred stimuli were presented in random order. Neutral stimuli were the text “XXXX” colored blue, green, red, or yellow; congruent stimuli were text in the same color as the word (e.g. the word “GREEN” colored green); incongruent stimuli were text in a different color to the word (e.g. the word “RED” colored green). Performance outcomes were accuracy (% correct) and median reaction times (RTs; ms) on correct trials to congruent, neutral and incongruent stimuli, and differences in accuracy and RT on congruent compared to incongruent stimuli, and neutral compared to incongruent stimuli [23].

Selective visual attention was assessed with a conjunction visual search task [23, 24]. The CONJ is a test of visual selective attention and goal-directed behavior requiring determination of whether or not a target is present among nontarget distractor stimuli. Participants were required to press a response button with ms accuracy as quickly and accurately as possible, indicating whether a target (a vertical red bar) was present among a set of distractors (horizontal red bars, and horizontal and vertical green bars). Ninety self-paced trials with sets of 10, 20, 30, or 40 stimuli, with a target present on 50% of trials. Set size was equally distributed across the task. Performance outcomes were median RT (ms) and change in median RT as a function of set size (slope, ms/number) [24], computed for trials with correct responses to a target present (hits), target absent (blanks), and both; cognitive throughput (number of correct/minute), and % of trials missed [23].

One participant was color blind (confirmed by an abbreviated Ishihara test [25]), and therefore their performance may have been impaired when responding to Stroop and Visual Search Task stimuli. To ensure similar cognitive loads across all participants, that participant performed the Stroop and Visual Search tasks, but their data from those tasks were not included in analyses.

Cognitive processing speed and working memory for mathematical operations were assessed with an addition task (ADD) [26], where participants were given 2 minutes to sum pairs of two-digit numbers as quickly and accurately as possible. As soon as a response was entered a new set of randomly generated numbers was displayed on the screen. Performance was assessed by the total number of sums attempted in 2 minutes, the number of correct responses and the % correct.

Sustained attention was measured with the 10-minute PVT [27], with inter-stimulus intervals of 1–9 seconds. RTs were measured in ms and log or reciprocal transformations were used to address violations of homogeneity of variance and normality. PVT outcomes were 1/median RT, 1/mean RT, log of the standard deviation of the mean RT (log STD RT), 1/the 10% slowest RT (1/Slow RT), the 10% fastest RT (Fastest RT), number of lapses (RT > 500 ms) and number of false starts (RT < 100ms).

Participants were instructed that speed and accuracy were equally important on performance tasks.

Self-reported alertness was assessed using a VAS, a 100-mm line with the ends labeled “sleepy” and “alert.” Participants clicked the position on the line that they felt best described their level of alertness, scored as mm from the “sleepy” end, such that higher scores indicate greater alertness.

Mood was assessed at the beginning and end of each battery with the PANAS [28], modified to obtain responses via a 100-mm VAS. Self-reported sleepiness was assessed with the KSS [29] at the beginning, middle, and end of each testing battery (Supplementary Table S2). KSS and PANAS scores from the beginning of each battery were used for the primary analyses. Statistical justification for this decision is described in the Supplementary Material.

Objective alertness was assessed with a 20-minute Maintenance of Wakefulness Test (MWT) [30]. Sleep onset latency (SOL) was defined as minutes elapsed until the first epoch of any sleep stage. The test was ended after 20 minutes or by waking participants immediately after the first epoch containing a spindle or K-complex. To ensure that participants were not motivated to end the task sooner by falling asleep quickly, staff then entered the room and sat quietly while the subject remained in quiet wakefulness in the bed until the remainder of the 20 minutes had elapsed. Participants who did not fall asleep on a given test were given a score of 20 minutes for that test.

Statistical analysis

Change in performance between visits and across the experimental protocol was tested using linear mixed effects models with participant as a random factor and visit and battery number (time in the protocol relative to the participant’s habitual sleep schedule) as fixed factors. The effect of sex on performance (averaged across batteries 3 to 17) was tested in a multiple regression model with performance as the dependent variable and sex and visit (1 or 2) as the independent variables (Supplementary Table S7). To assess individual differences in performance, test scores were averaged across all batteries performed under combined sleep restriction and circadian misalignment (batteries 3–17). Between-participants and within-participant variance for each metric was calculated using linear mixed effects models with participant as a random factor and visit as a fixed factor. Performance was assessed with and without controlling for baseline (battery 1) as a covariate, and by comparing baseline tests across visits (Supplementary Table S4). Stability of individual differences were quantified by intra-class correlation coefficients (ICC), calculated according to the ICC(A,1) formula defined by McGraw and Wong for a single score, two-way mixed effects model [17]. The strength of ICC scores were interpreted using standard criteria: poor (<0.00), slight (0.00–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.81), almost perfect (0.81–1.00). Kendall’s rank correlation was used to determine whether the rank order of participants (best to worst performers) on each metric was correlated with rank order on other metrics (performance averaged across batteries 3–17). To account for multiple comparisons p-values were adjusted with a 5% false discovery rate procedure for correlated test statistics, i.e. accounting for the fact that metrics may be correlated within tests [31]. To examine the relationships among task metrics, exploratory factor analysis was conducted, limited to factors with eigenvalues greater than one (Varimax rotation, normalized). Statistical analyses were conducted with Statistica 13 (StatSoft Inc.), apart from Kendall’s tau rank correlation, which was conducted with Matlab R2018a (Mathworks Inc.).

Results

The 39-day study was completed by 20 healthy adults (eight female, sex ascertained by self-report), aged 25.7 (±4.2 SD), BMI 22.3 (±2.1 SD) kg/m². Changes in performance, sleepiness, and mood over time in the combined sleep restriction and circadian misalignment protocol are shown in Figure 2, and the main effects of time and visit are summarized in Table 1 for a selected primary metric from each task. Main effects of time and visit on remaining metrics are summarized in Supplementary Table S3 and shown in Supplementary Figures S1–S4. A significant main effect of time was observed for all metrics except ADD Correct (%), Negative Affect Score, Stroop Incongruent-Congruent median RT to correct trials (MEDRTC), Incon-Neutral MEDRTC, Incon-Neutral Correct (%), and CONJ Visual Search Slope on target absent trials.

Figure 2.

Performance (A, B, E, G, H), sleepiness (C, F), and mood (D) over time. Time is relative to the participant’s habitual sleep schedule. Grey bars indicate scheduled sleep episodes. Error bars show standard error of the mean.

Table 1.

Main effect of visit and time for selected metrics for each task

Task	Metric	Visit		Time
		F	p	F	p
Stroop	Incongruent Correct (%)	9.34	<.01	10.59	<.0001
CONJ Visual Search	MEDRTC (ms)	9.09	<.01	7.27	<.0001
MWT	SOL (minute)	3.14	.08	42.03	<.0001
ADD	Correct (number)	64.95	<.0001	6.79	<.0001
PVT	1 / median RT (/ms)	137.52	<.0001	0.00	<.0001
VAS	Alert (mm)	7.31	.01	58.21	<.0001
KSS	KSS	1.09	.30	44.48	<.0001
PANAS	Positive Affect Score (mm)	35.35	<.0001	11.43	<.0001

Time is relative to the participant’s habitual sleep schedule.

Consistency of individual differences in performance during sleep restriction and circadian misalignment are summarized in Table 2. Stability of individual differences in performance were moderate to almost perfect for Stroop (ICCs 0.54–0.92, Figure 3) and almost perfect for CONJ Visual Search (ICCs 0.81–0.96, Figure 4). Stability of individual differences was substantial for the ability to maintain wakefulness on the MWT (ICC 0.76, Figure 5A), KSS (ICC 0.68, Figure 5E) and VAS alertness (ICC 0.79, Figure 5B), was fair to almost perfect for the ADD (ICCs 0.22–0.94, Figure 5, C, F, and G), and was substantial for positive and negative affect (ICC 0.66–0.75, Figure 5, D and H). Stability of individual differences in performance was substantial for the PVT (ICCs 0.68–0.79, Figure 6) apart from false starts, which was moderate (ICC 0.53, Figure 6E).

Table 2.

Consistency of individual differences in performance, sleepiness, and mood. Incon – Neutral, Incongruent – Neutral; CT, cognitive throughput; log STD, log standard deviation; b1, baseline (test battery 1, also see Figure 1)

Task	Metric	ICC	ICC strength	b1 as Covariate
				ICC	Strength
Stroop	Congruent MEDRTC (ms)	0.89	Almost perfect	0.68	Substantial
Stroop	Neutral MEDRTC (ms)	0.88	Almost perfect	0.80	Substantial
Stroop	Incongruent MEDRTC (ms)	0.92	Almost perfect	0.68	Substantial
Stroop	Incon – Con MEDRTC (ms)	0.82	Almost perfect	0.63	Substantial
Stroop	Incon – Neutral MEDRTC (ms)	0.54	Moderate	0.36	Fair
Stroop	Congruent Correct (%)	0.65	Substantial	0.55	Moderate
Stroop	Neutral Correct (%)	0.66	Substantial	0.53	Moderate
Stroop	Incongruent Correct (%)	0.80	Almost perfect	0.73	Substantial
Stroop	Incon – Con Correct (%)	0.76	Substantial	0.59	Moderate
Stroop	Incon – Neutral Correct (%)	0.56	Moderate	0.48	Moderate
CONJ Visual Search	MEDRTC (ms)	0.96	Almost perfect	0.92	Almost perfect
CONJ Visual Search	MEDRTC Target Present (ms)	0.92	Almost perfect	0.87	Almost perfect
CONJ Visual Search	MEDRTC Target Absent (ms)	0.95	Almost perfect	0.93	Almost perfect
CONJ Visual Search	CT (items/minute)	0.91	Almost perfect	0.88	Almost perfect
CONJ Visual Search	Slope Target Present	0.81	Almost perfect	0.82	Almost perfect
CONJ Visual Search	Slope Target Absent	0.93	Almost perfect	0.88	Almost perfect
CONJ Visual Search	Missed Targets (%)	0.87	Almost perfect	0.65	Substantial
MWT	SOL (minute)	0.76	Substantial	0.80	Almost perfect
ADD	Attempted (number)	0.94	Almost perfect	0.66	Substantial
ADD	Correct (number)	0.93	Almost perfect	0.62	Substantial
ADD	Correct (%)	0.22	Fair	0.66	Substantial
PVT	False Starts (number)	0.53	Moderate	0.47	Moderate
PVT	Fastest RT (ms)	0.70	Substantial	0.71	Substantial
PVT	Lapses (number)	0.73	Substantial	0.79	Substantial
PVT	1 / median RT (/ms)	0.68	Substantial	0.66	Substantial
PVT	1 / mean RT (/ms)	0.73	Substantial	0.74	Substantial
PVT	log STD RT	0.74	Substantial	0.74	Substantial
PVT	1 / Slowest RT (/ms)	0.74	Substantial	0.75	Substantial
VAS	Alert (mm)	0.79	Substantial	0.68	Substantial
KSS	KSS (mm)	0.68	Substantial	0.61	Substantial
PANAS	PAS (mm)	0.66	Substantial	0.44	Moderate
PANAS	NAS (mm)	0.75	Substantial	0.32	Fair

Figure 3.

Individual Differences in Stroop Performance. Participants are ordered from worst to best performers separately for each metric. (A–E) MEDRTC, median reaction time to correct trials; (F–J) percent of trials answered correctly.

Figure 4.

Individual differences in visual search performance (A–G). Participants are ordered from worst to best performers separately for each metric. CT, cognitive throughput.

Figure 5.

Individual differences in sleepiness (A, B, E), mathematical addition performance (C, F, G), and mood (D, H). Participants are ordered from worst to best performers separately for each metric.

Figure 6.

Individual differences in PVT performance (A–G). Participants are ordered from worst to best performers separately for each metric. log STD, log standard deviation.

When controlling for baseline performance, ICCs were reduced for the Stroop, CONJ Visual Search, ADD, and PANAS, but mostly remained moderate to almost perfect (Table 2). At baseline, consistency of individual differences was generally weaker than during sleep restriction and circadian misalignment (ICCs slight to moderate strength, Supplementary Table S3).

Participants’ ranking from lowest performers to highest performers varied by task (Table 3 visit 1, Supplementary Table S5 visit 2, Figures 3–6). That is, individuals with the lowest performance on one task were not necessarily the lowest performers on other tasks. At visit 1, there were only two significant correlations between metrics from different tasks: rank on Visual Search Missed Targets correlated with rank on two Stroop accuracy metrics. At visit 2, there were 14 significant rank correlations between metrics from different tasks. Rank on Stroop Neutral MEDRTC correlated with rank on Visual Search MEDRTC Target Present. Rank on Visual Search Missed Targets was negatively correlated with rank on three Stroop accuracy metrics, and Stroop Congruent Correct (%) rank correlated with rank on PVT log STD RT and 1/ Slowest RT. Visual Search Cognitive Throughput rank was correlated with rank on VAS Alertness and with all PVT metrics apart from False Starts. VAS Alertness rank was negatively correlated with KSS rank. PANAS, ADD, and MWT metrics were not rank correlated with metrics from any other task at either visit.

Table 3.

Kendall’s tau rank correlation matrix

Task Metric	Congruent MEDRTC	Neutral MEDRTC	Incongruent MEDRTC	Incon- Con MEDRTC	Incon- Neutral MEDRTC	Congruent Correct (%)	Neutral Correct (%)	Incongruent Correct (%)	Incon-Con Correct (%)	Incon-Neutral Correct (%)	MEDRTC	CT	Slope Target Present	Slope Target Absent	Missed Targets	MEDRTC Target Present	MEDRTC Target Absent	Positive Affect Score	Negative Affect Score	Alert	False Starts	Fastest RT	1 / median RT	1 / mean RT	log STD RT	1 / Slowest RT	Lapses	KSS	Attempted	Correct (number)	Correct (%)
Stroop Congruent MEDRTC
Stroop Neutral MEDRTC	0.79
Stroop Incongruent MEDRTC	0.54	0.66
Stroop Incon-Con MEDRTC	0.23	0.37	0.68
Stroop Incon-Neutral MEDRTC	0.16	0.23	0.57	0.74
Stroop Congruent Correct (%)	0.19	0.26	0.39	0.42	0.45
Stroop Neutral Correct (%)	0.23	0.26	0.34	0.26	0.28	0.70
Stroop Incongruent Correct (%)	0.25	0.23	0.33	0.23	0.32	0.53	0.61
Stroop Incon-Con Correct (%)	0.29	0.22	0.30	0.20	0.25	0.36	0.48	0.83
Stroop Incon-Neutral Correct (%)	0.32	0.30	0.34	0.26	0.19	0.23	0.22	0.61	0.70
VS MEDRTC	0.43	0.40	0.42	0.17	0.19	0.09	0.12	0.19	0.21	0.23
VS CT	−0.29	−0.24	−0.20	−0.01	−0.01	0.29	0.14	0.16	0.06	0.07	−0.56
VS Slope Target Present	0.19	0.16	0.32	0.32	0.32	0.23	0.19	0.16	0.14	0.14	0.46	−0.38
VS Slope Target Absent	0.19	0.19	0.32	0.22	0.22	0.14	0.26	0.26	0.25	0.18	0.60	−0.32	0.29
VS Missed Targets	−0.19	−0.21	−0.27	−0.15	−0.15	−0.34	−0.42	−0.55	−0.50	−0.42	−0.15	−0.18	−0.06	−0.36
VS MEDRTC Target Present	0.45	0.40	0.44	0.17	0.22	0.07	0.09	0.22	0.23	0.28	0.91	−0.58	0.48	0.53	−0.15
VS MEDRTC Target Absent	0.37	0.35	0.38	0.18	0.20	0.11	0.20	0.23	0.26	0.25	0.87	−0.50	0.47	0.71	−0.21	0.80
PANAS Positive Affect Score	−0.05	−0.03	0.03	0.16	0.23	0.20	0.07	0.09	0.00	0.16	−0.02	0.16	0.16	0.01	0.08	−0.06	−0.01
PANAS Negative Affect Score	−0.25	−0.16	−0.24	−0.18	−0.25	−0.02	−0.02	−0.11	−0.08	−0.04	−0.29	0.36	−0.21	−0.24	−0.01	−0.26	−0.27	0.05
VAS Alert	−0.23	−0.11	−0.05	0.05	0.05	0.33	0.19	0.17	0.05	0.01	−0.22	0.43	−0.06	−0.15	−0.07	−0.26	−0.23	0.29	0.18
PVT False Starts	−0.01	0.02	−0.09	−0.16	−0.26	−0.28	−0.29	−0.25	−0.21	−0.02	0.10	−0.17	0.01	0.09	0.17	0.15	0.02	−0.24	0.02	−0.28
PVT Fastest RT	0.26	0.31	0.30	0.22	0.12	−0.14	0.00	−0.16	−0.06	−0.16	0.27	−0.44	0.28	0.18	0.11	0.27	0.29	−0.20	−0.10	−0.36	−0.01
PVT 1 / median RT	−0.16	−0.21	−0.16	−0.07	0.09	0.15	−0.05	0.18	0.10	0.19	−0.19	0.39	−0.26	−0.24	0.07	−0.18	−0.22	0.19	0.06	0.30	−0.09	−0.71
PVT 1 / mean RT	−0.12	−0.12	0.02	0.13	0.28	0.40	0.14	0.28	0.16	0.20	−0.13	0.41	−0.14	−0.15	−0.02	−0.14	−0.15	0.20	−0.01	0.35	−0.21	−0.55	0.80
PVT log STD RT	0.11	0.06	−0.12	−0.28	−0.38	−0.45	−0.21	−0.25	−0.12	−0.16	0.17	−0.34	−0.02	0.19	0.01	0.19	0.15	−0.35	0.05	−0.42	0.38	0.31	−0.47	−0.71
PVT 1 / Slowest RT	−0.12	−0.07	0.11	0.26	0.35	0.43	0.19	0.23	0.11	0.15	−0.16	0.33	0.01	−0.19	0.01	−0.19	−0.15	0.33	−0.06	0.42	−0.35	−0.34	0.51	0.74	−0.98
PVT Lapses	0.06	0.06	−0.06	−0.19	−0.29	−0.36	−0.13	−0.24	−0.19	−0.25	0.12	−0.31	0.04	0.19	0.00	0.15	0.11	−0.24	0.10	−0.39	0.36	0.40	−0.67	−0.80	0.79	−0.82
KSS	−0.02	−0.08	−0.04	−0.05	−0.09	−0.27	−0.22	−0.02	0.10	0.10	0.04	−0.21	0.05	−0.12	−0.05	0.13	−0.02	−0.45	−0.11	−0.34	0.29	0.08	−0.17	−0.23	0.26	−0.27	0.17
ADD Attempted	−0.36	−0.43	−0.43	−0.23	−0.23	−0.28	−0.23	−0.21	−0.16	−0.18	−0.43	0.14	−0.18	−0.28	0.04	−0.43	−0.32	−0.01	0.08	0.01	−0.19	−0.09	0.04	−0.04	0.04	−0.02	−0.01	0.04
ADD Correct (number)	−0.37	−0.44	−0.45	−0.25	−0.25	−0.32	−0.27	−0.25	−0.20	−0.22	−0.38	0.10	−0.14	−0.24	0.07	−0.38	−0.27	−0.04	0.09	−0.03	−0.17	−0.05	0.00	−0.08	0.09	−0.07	0.03	0.07	0.96
ADD Correct (%)	−0.15	−0.22	−0.27	−0.24	−0.22	−0.13	0.04	−0.05	0.00	−0.13	0.15	−0.15	0.16	0.22	−0.06	0.15	0.25	−0.10	0.08	−0.23	0.05	0.24	−0.32	−0.30	0.34	−0.32	0.25	0.14	0.31	0.36
MWT SOL	0.01	0.03	0.04	0.15	0.19	0.26	0.15	0.15	0.06	0.13	0.02	0.15	0.08	0.06	0.00	−0.03	0.10	0.47	−0.03	0.16	−0.32	−0.17	0.33	0.39	−0.47	0.46	−0.42	−0.47	0.04	−0.01	−0.09

Blue shaded boxes indicate significant within-task rank correlations, orange shaded boxes indicate significant between-task rank correlations. Data shown is from visit 1, correlation matrix for visit 2 is in Supplementary Table S5. Incon – Con, Incongruent – Congruent; Incon – Neutral, Incongruent – Neutral; VS, Conjunction Visual Search Task; CT, cognitive throughput; log STD, log standard deviation.

Furthermore, within a task, individuals with the lowest performance on one metric were not necessarily the lowest performers on other metrics. For example, on the Stroop task, rank on Incongruent-Congruent MEDRTC correlated with Incongruent-Neutral MEDRTC at both visits, but did not correlate with any other Stroop metrics. Positive Affect and Negative Affect were not rank correlated with each other. Rank on most metrics for the PVT were correlated within task, apart from False Starts.

Exploratory factor analysis using only metrics that showed a significant change during the protocol identified six factors that explained 89.6% of the variance at visit 1, and 89.7% of the variance at visit 2. Factors loadings are shown in Table 4 with descriptive factor titles. Factor structure was highly consistent between visits. The factor designated Accurate Response Speed included all PVT metrics except False Starts at both visits. From the Stroop, speed measures loaded on a separate factor to accuracy measures, which loaded with CONJ Visual Search Missed Targets at both visits. The factor designated Visual Search comprised four Visual Search metrics at Visit 1, and three at visit 2, with nontrivial loading of CT. The factor designated Mathematical Addition comprised Attempted and Correct number at both visits, and with PVT False Starts at visit 1. MWT SOL did not load on any factor, but showed nontrivial loading with KSS and Positive Affect Score at visit 1. Factor structure using all metrics is shown in Supplementary Table S6.

Table 4.

Factor loadings, varimax rotation (normalized)

Task	Metric	Visual search		Accurate response speed		Stroop accuracy		KSS		Stroop speed		Mathematical addition
		V1	V2	V1	V2	V1	V2	V1	V2	V1	V2	V1	V2
Stroop	Congruent MEDRTC	0.22	0.22	0.02	−0.08	0.10	0.05	0.00	0.01	0.94	0.93	0.04	0.17
Stroop	Neutral MEDRTC	0.19	0.16	0.09	−0.01	0.12	0.09	0.03	0.09	0.93	0.95	0.16	0.14
Stroop	Incongruent MEDRTC	0.21	0.21	−0.03	0.08	0.17	0.10	0.09	0.01	0.91	0.92	0.14	0.18
Stroop	Congruent Correct (%)	0.04	0.02	−0.36	0.40	0.84	0.80	0.22	0.21	0.06	0.15	0.15	−0.04
Stroop	Neutral Correct (%)	0.00	0.10	−0.13	0.41	0.89	0.83	0.18	0.22	0.19	0.16	0.04	−0.05
Stroop	Incongruent Correct (%)	0.16	0.11	−0.28	0.40	0.85	0.81	−0.03	0.09	0.29	0.29	−0.02	−0.13
PVT	False Starts	0.00	−0.16	0.29	−0.45	−0.10	−0.17	−0.43	−0.48	−0.30	−0.29	0.69	0.44
PVT	Fastest RT	0.15	0.12	0.80	−0.75	−0.22	−0.41	0.12	0.05	0.34	0.24	0.01	−0.01
PVT	1 / median RT	−0.12	−0.17	−0.95	0.90	0.13	0.29	0.04	−0.02	−0.13	−0.10	−0.04	0.01
PVT	1 / mean RT	−0.13	−0.27	−0.95	0.92	0.13	0.22	0.22	0.12	−0.04	0.03	0.06	0.05
PVT	log STD RT	0.16	0.27	0.84	−0.86	−0.11	−0.18	−0.42	−0.28	−0.15	−0.13	−0.08	−0.01
PVT	1 / Slowest RT	−0.18	−0.26	−0.87	0.91	0.07	0.07	0.36	0.19	0.18	0.10	0.08	0.05
PVT	Lapses	0.14	0.09	0.94	−0.96	−0.06	−0.15	−0.20	−0.10	−0.07	0.03	0.10	0.03
CONJ Visual Search	MEDRTC	0.95	0.93	0.15	−0.23	0.11	0.12	−0.05	−0.06	0.19	0.19	0.14	0.14
CONJ Visual Search	CT	−0.71	−0.59	−0.39	0.63	0.46	0.35	0.14	0.15	−0.22	−0.15	−0.05	0.04
CONJ Visual Search	MEDRTC Target Present	0.92	0.88	0.11	−0.25	0.09	0.10	−0.09	−0.12	0.25	0.28	0.17	0.21
CONJ Visual Search	MEDRTC Target Absent	0.94	0.92	0.18	−0.26	0.18	0.17	−0.03	0.00	0.15	0.18	0.07	0.13
CONJ Visual Search	Missed Targets	−0.03	−0.12	−0.08	−0.07	−0.87	−0.76	0.06	0.08	0.00	0.11	−0.05	−0.28
VAS	Alert	−0.42	−0.40	−0.38	0.24	0.23	0.09	0.48	0.71	−0.07	−0.22	0.08	0.12
KSS	KSS	0.20	0.03	0.13	0.07	−0.20	−0.31	−0.81	−0.82	−0.18	−0.39	−0.03	−0.08
PANAS	Positive Affect Score	0.05	0.01	−0.28	0.45	−0.09	−0.29	0.85	0.60	−0.11	−0.24	0.05	0.13
MWT	SOL	−0.01	−0.02	−0.55	0.52	0.10	0.20	0.59	0.50	0.15	0.18	−0.15	−0.25
ADD	Attempted	−0.26	−0.20	0.10	−0.04	−0.18	−0.07	−0.15	−0.07	−0.41	−0.27	−0.81	−0.92
ADD	Correct (number)	−0.25	−0.19	0.12	−0.06	−0.16	−0.04	−0.15	−0.06	−0.43	−0.28	−0.81	−0.92
	Eigenvalue	1.94	2.20	8.23	9.02	2.48	1.29	1.33	1.44	5.98	5.64	1.56	1.94
	% Total variance	8.07	9.17	34.27	37.57	10.34	5.38	5.53	6.00	24.91	23.51	6.50	8.08

Bold indicates values with loading greater than 0.65. log STD, log standard deviation; V1, visit 1; V2, visit 2.

Discussion

We found substantial to almost perfect trait-like individual differences in higher-order cognitive functions involved in goal-directed behaviors and cognitive control, mood, and the ability to maintain wakefulness during combined sleep restriction and circadian misalignment. Others have reported trait-like mood, sleepiness, evening sleep propensity, vigilance, and working memory performance during chronic sleep restriction or total sleep deprivation [14–16]. Our results confirm the latter findings, and extend them to higher-order cognitive functions of inhibitory control and visual search, and to the ability to maintain wakefulness during combined sleep restriction and circadian misalignment. Inhibitory control, selective visual attention, and the ability to stay awake are important for safety-critical tasks conducted in professions that also commonly experience sleep restriction and circadian misalignment. For example, by combat personnel identifying and responding to friend or foe and camouflaged or airborne targets, nurses reading medication labels that are visually similar, or security personnel monitoring for weapons.

Our findings have implications for the detection, prediction, and counteraction of impairments due to combined sleep restriction and circadian misalignment. That individual differences in vulnerability to combined sleep restriction and circadian misalignment varied across cognitive domains highlights the need to develop effective countermeasures that target multiple cognitive domains. To date, the best strategies to optimize performance appear to be combinations of countermeasures with different modes of action (e.g. stimulants combined with naps or bright light therapy) [32–34], although these remain insufficient and further countermeasure development is needed. Our findings also highlight the need for tools that can predict impairment on multiple cognitive domains during combined sleep restriction and circadian misalignment. Finally, our findings suggest that it is important to assess multiple cognitive domains during sleep loss and circadian misalignment studies, when the goal is to determine individual differences in performance vulnerability, develop biomarkers that may correlate only with some specific domains of performance loss versus multiple domains, and when testing effectiveness of countermeasure strategies.

Additional research is necessary to examine the consistency of individual differences in other cognitive domains during sleep loss and circadian misalignment. The factor analysis showed relative independence of various cognitive domains, including that self-reported sleepiness/alertness (KSS/VAS) was orthogonal to performance on other tasks. This is consistent with previous factor analyses of performance during sleep restriction and circadian misalignment, which found that self-reported alertness loaded on a separate factor to response speed and accuracy [13, 16, 35]. Similarly, in military personnel performing a simulated marksmanship task, RTs slowed and discrimination of friend or foe was reduced following sleep deprivation, even though participants did not perceive their performance to have deteriorated [36]. This aligns with the view that self-report measures of alertness are insufficient to predict functional impairment.

The strength of individual differences on many measures was reduced when statistically controlling for baseline performance. This suggests that individual differences in performance under combined sleep restriction and circadian misalignment are related, at least in part, to individual differences in performance capability at baseline. However, baseline performance was previously shown to be insufficient to predict subsequent impairment due to sleep deprivation [16].

In general, ICCs at baseline were weaker than ICCs during sleep restriction and circadian misalignment. This could reflect some continued learning, as participants practiced until the steep portion of the learning curve was eradicated, but not until performance plateaued (e.g. performance on some tests has been shown to continue to improve over multiple weeks of daily testing after the steep portion of the learning curve has been eradicated) [37]. A poor ICC for MWT SOL at baseline was likely due to a ceiling effect as the majority of participants remained awake for the entire test and there was thus insufficient variability between participants to observe trait-like differences.

It has previously been shown that individual vulnerability to sleep deprivation is task dependent [13]. Here we extend that finding to higher-order cognitive tasks, and further show that even within one task, individual vulnerability may vary across performance metrics. The task dependency of performance is evidenced by the factor analysis, which showed that performance on the Stroop and Visual Search tasks was orthogonal to each other and to other tasks. Furthermore, participant rankings (i.e. best to worst) on PVT metrics of speed were not correlated with rank on Stroop speed. The metric dependency of vulnerability is evidenced by the finding that when participants were ranked according to performance, their rank order could vary across metrics of the same task. For example, on the Stroop task, rank on Incongruent-Congruent MEDRTC correlated with rank on Incongruent-Congruent MEDRTC, but did not correlate with any other Stroop metrics. Performance on several metrics was consistently high and unimpaired by sleep restriction and circadian misalignment, and remained high and stable across the protocol: ADD Correct (%), Negative Affect Score, Stroop Incongruent-Con MEDRTC, Incongruent-Neutral MEDRTC, Incongruent-Neutral Correct (%), and CONJ Visual Search Slope on target present and target absent trials. It has been previously shown that within a single working memory task, individuals exposed to total sleep deprivation showed different degrees of vulnerability or resilience of different components of working memory performance [38]. We extend these findings to show metric-dependency within tasks that test several cognitive domains including selective attention, processing speed, working memory, and inhibitory control. Our findings demonstrate that in tasks that involve multiple cognitive domains, different domains (and thus different metrics) may be more or less vulnerable to sleep restriction and circadian misalignment.

Given that individual trait vulnerability varied across tests and between different metrics from the same test, the selection of tests, and metrics for testing vulnerability and responses to countermeasures is not straight forward. These findings suggest that multiple metrics from multiple tasks should be employed when assessing performance, predicting vulnerability and testing responses to countermeasures. The design of a multi-domain test would depend on its intended use. A test could address multiple cognitive domains by simultaneously engaging multiple domains, or it could be comprised of sub-tasks that each test different cognitive processes (i.e. a battery of tasks). For applications in which distinguishing the contributions of particular cognitive domains is of interest, a battery of targeted sub-tasks would be preferable (e.g. in a research study to draw conclusions regarding which cognitive domains are impacted by a sleep/circadian intervention). On the other hand, if a test is intended for use in a real-world occupational setting to predict impairment (e.g. to predict whether a pilot will be fit to fly after a given sleep schedule), dissociation of the cognitive components involved may not be of interest. Rather, the important features of the test would be its ability to predict the consequence of interest (e.g. vehicle crashes) and the ease of implementation.

RTs on the Stroop have high test–retest reliability in the absence of sleep manipulations [39]. Here, we show that performance is also consistent during combined sleep restriction and circadian misalignment, and that there are substantial and consistent differences between individuals. The finding that responses to incongruent stimuli are slower and less accurate than responses to congruent or neutral stimuli is termed the “Stroop effect,” and can be attributed to multiple cognitive processes including conflict resolution, interference control, selective attention, and the ability to inhibit an automatic response (i.e. inhibitory control) [22, 40]. Cognitive speed measures on the Stroop were more sensitive to combined sleep restriction and circadian misalignment than was the Stroop effect. The latter is consistent with our prior findings that the Stroop effect is sensitive to circadian variation, but less so to homeostatic sleep pressure [23].

The consistency of performance gives confidence in the utility of these tasks in research protocols using sleep and circadian disturbance. However, within-participant study designs should be used, and data should be analyzed with linear mixed effects models, so that the contribution of individual participants to variance is accounted for separately from that due to the manipulation/condition [41]. Such designs should also be considered for other aspects of sleep shown to have trait-like individual differences, including sleep EEG spectra [42, 43], EEG topography [44, 45], and homeostatic dynamics of slow-wave sleep [46].

Some individual differences in performance could be due to varying interpretations of the task directions. Participants were directed to weigh speed and accuracy equally; however, some may have favored one over the other [47]. If so, the strong trait-like performance suggests that participants’ varying strategies were retained throughout both visits. Some of the individual differences may be genetically determined, as has been reported for PVT performance and sleepiness during sleep restriction [48], PVT performance during sleep deprivation [49, 50], and for sleep EEG spectra [44]. Other potential factors underlying individual differences include brain structure and function, such as hippocampal volume, activation patterns of the default mode network and white matter integrity (for a comprehensive review see [51]). Van Dongen et al. [16] previously showed that individual differences in performance were not altered by prior sleep history.

Whether sex contributes to individual differences in response to sleep and circadian disruption is largely unknown. Findings regarding sex differences in the effects of sleep restriction and circadian misalignment on performance during sleep deprivation are mixed [13, 52, 53]. In the present study, 40% of participants were female; however, the study was not powered to assess sex differences. Exploratory analyses of sex differences in performance outcomes are provided in the Supplementary Material, with men showing faster Visual Search performance but more missed targets, better PVT RT performance, and higher ratings of VAS alertness. Future research is warranted to characterize sex differences in higher-order cognitive performance and ability to maintain wakefulness during sleep restriction and circadian misalignment.

This study was conducted in a population of young, healthy adults without sleep disorders, who had adequate habitual sleep durations and who were not night shift workers. How these findings generalize to different age groups and those with sleep disorders, chronic sleep restriction, or chronic circadian misalignment warrants further study. It will also be important to test whether these findings generalize to other schedules of sleep/circadian disturbance, such as the chronic sleep restriction and circadian misalignment experienced by night shift workers. Ramakrishnan et al. [54] showed that PVT performance following total sleep deprivation predicted PVT performance under chronic sleep restriction, and vice versa. Further research is needed to determine whether other higher cognitive functions show similar trait stability under other sleep/circadian challenges. Future research should also test whether these effects observed in the laboratory are similarly present in field settings during real-world operations.

Conclusions

Stable individual differences in performance during combined sleep restriction and circadian misalignment represent an opportunity for development of personalized countermeasure strategies. Given that the participants in this study were a carefully screened homogeneous group, individual differences in the general population may be even greater than those observed here. Tailoring countermeasures towards individual vulnerabilities could increase efficiency, e.g. by removing the need to administer countermeasures that target a domain in which an individual is resilient. Alternatively, creating tests and countermeasures that address impairments across multiple cognitive domains may be a more effective strategy, until better predictive tools are developed to characterize and predict individual vulnerabilities.

Funding

This work was supported by the Office of Naval Research MURIgrant N00014-15-1-2809, NIH/NCATS Colorado CTSA Grant UL1TR002535 and the University of Colorado Boulder Undergraduate Research Opportunities Grant.Contents are the authors’ sole responsibility and do not necessarily represent official NIH views.

Conflict of interest statement. K.P.W. has received consulting fees from or served as a paid member of scientific advisory boards for the Sleep Disorders Research Advisory Board - National Heart, Lung and Blood Institute, CurAegis Technologies, Circadian Therapeutics, LTD, Kellogg, and has received speaker/educational/travel consultant honorarium fees from the American Academy of Sleep Medicine, American College of Chest Physicians, American College of Sports Medicine, American Diabetes Association, and Associated Professional Sleep Societies.