Paradoxical effects from stimulus density manipulation provide new insight into the impact of sleep deprivation on PVT performance

Abstract

Study Objectives

The psychomotor vigilance test (PVT), a 10-min one-choice reaction time task with random response-stimulus intervals (RSIs) between 2 and 10 s, is highly sensitive to behavioral alertness deficits due to sleep loss. To investigate what drives the performance deficits, we conducted an in-laboratory total sleep deprivation (TSD) study and compared performance on the PVT to performance on a 10-min high-density PVT (HD-PVT) with increased stimulus density and truncated RSI range between 2 and 5 s. We hypothesized that the HD-PVT would show greater impairments from TSD than the standard PVT.

Methods

n = 86 healthy adults were randomized (2:1 ratio) to 38 h of TSD (n = 56) or corresponding well-rested control (n = 30). The HD-PVT was administered when subjects had been awake for 34 h (TSD group) or 10 h (control group). Performance on the HD-PVT was compared to performance on the standard PVTs administered 1 h earlier and 1 h later.

Results

The HD-PVT yielded approximately 60% more trials than the standard PVT. The HD-PVT had faster mean response times (RTs) and equivalent lapses (RTs > 500 ms) compared to the standard PVT, with no differences between the TSD effects on mean RT and lapses between tasks. Further, the HD-PVT had a dampened time-on-task effect in both the TSD and control conditions.

Conclusions

Contrary to expectation, the HD-PVT did not show greater performance impairment during TSD, indicating that stimulus density and RSI range are not primary drivers of the PVT’s responsiveness to sleep loss.

Statement of Significance.

Based on theoretical perspectives on the PVT’s remarkable sensitivity to sleep loss, we predicted that increasing the task’s stimulus density (i.e. high-density PVT) would be associated with a higher probability of capturing attentional instability, amplification of the time-on-task effect, and enhancement of the RSI effect—and thus even greater detection of behavioral alertness deficits during sleep deprivation. Our empirical findings contradict these predictions and instead suggest that the exertion of top-down compensatory effort during the RSIs to compensate for bottom-up degradation of neuronal processing is a critical mediator of performance impairment on the PVT. Our results help to explain what makes the PVT a gold standard for measuring behavioral alertness deficits due to sleep loss.

Introduction

Since the original invention, first publication [1], and further development [2–5] of the Psychomotor Vigilance Test (PVT) by the research team led by David Dinges, this performance task has become the gold standard measure of behavioral alertness [5] or vigilant attention [6] in research on sleep loss, circadian rhythms, and operational fatigue [5, 7]. The PVT is a one-choice serial reaction time task in which a visual stimulus—typically a rolling millisecond counter—is presented at random response-stimulus intervals (RSIs) ranging from 2 to 10 s in steps of 1 s (or as a continuous range in later versions). Individuals are instructed to respond to the stimulus, by pressing a button or tapping a screen, as quickly as possible without making false starts (i.e. responding prior to the stimulus presentation). After each response, the response time (RT) is displayed for 1 s; then the screen is blank until the next stimulus appears. The PVT was originally administered with custom-built hardware (PVT-192) and was 10 min in duration [1], which entails approximately 80–100 stimuli. The PVT is now widely used in both standard (10-min) and abbreviated (e.g. 3-min or 5-min) durations and is available across a range of platforms, including desktop computers, laptops, tablets, smartphones, and personal digital assistants (e.g. [8–15]). In some versions, an auditory (rather than visual) stimulus can be used [16], although this is less commonly applied.

PVT performance is highly sensitive to [17–18], and considerably compromised by, sleep loss [19], circadian misalignment [20], sleep inertia [21], obstructive sleep apnea [22], and various other sleep-related conditions and challenges (e.g. [23–27]). With no practice effect [28, 29] and little variation among people in aptitude for the task [4], the PVT has been a valuable tool in studies of the temporal dynamics of performance in response to acute total sleep deprivation [3], chronic sleep restriction [28, 30], nap sleep [31, 32], recovery sleep [33], sleep extension [34], circadian rhythmicity [35], stimulant use [36, 37], etc., as well as individual differences therein [38, 39]. The PVT has been used in laboratory and field settings and across many different populations, including police officers, fire fighters, doctors, nurses, emergency medical technicians, truck drivers, sailors, pilots, astronauts, and others (e.g. [40–48]).

The PVT’s exceptional sensitivity makes it a valuable research tool, but it also raises important theoretical questions about neurobehavioral functioning under conditions of sleep loss—that is, what exactly makes PVT performance so reactive to sleep loss, and what does that reveal about what happens in the sleep-deprived brain? Paradoxically, classic vigilance tasks [49, 50] were thought to derive their sensitivity to sleep loss from long task durations (in the order of hours) and long intervals between stimuli (in the order of one to several minutes). The task duration and RSI range of the PVT are substantially shorter, but PVT performance exhibits a robust time-on-task effect (i.e. degradation of performance across the duration of the task; [51]), which interacts strongly with sleep deprivation and circadian rhythmicity and amplifies the task’s reactivity thereto [3, 52]; see Figure 1A. It has also been observed that performance impairment is most prominently seen in responses associated with the shortest of the RSIs [53], particularly during sleep deprivation [54, 55]; see Figure 1B. To this day, the neurobiological mechanisms underlying the time-on-task and RSI effects on the PVT and its overall sensitivity to performance changes in response to sleep loss are topics of investigation and debate.

Figure 1.

Signature effects captured by the PVT. (A) Performance degrades with increasing time on task, which is exacerbated by sleep loss. A 10-min standard PVT was administered every 3 h throughout 38 h of total sleep deprivation (TSD), which lasted from 08:00 until 22:00 the following evening. Mean RT (± SE) from 16 healthy adults is plotted per 1-min interval across the 10-min task duration, showing progressive slowing within each test bout, especially in the later part of the TSD period. Note that placement of the 1-min bins in each test is not to scale on the hours awake axis. Adapted from Fig. 2 in Grant et al. [12]. (B) Performance is worse for shorter response-stimulus intervals (RSIs), which is exacerbated by sleep loss. The percentage of RTs classified as lapses (RT > 500 ms) from 54 healthy adults who completed a 10-min PVT at 10:00 after 3.5 h awake and 24 h later after 27.5 h awake is plotted as a function of RSI sub-groups (with continuous-range RSIs between 2 and 10 s divided into 7 equal intervals on a log scale). Curves redrawn from data provided as part of Fig. 3 in Yang et al. [55]. (C) Extended wakefulness causes a rightward skewing of the RT distribution. The distribution of raw RTs from 9 healthy adults performing the PVT at 01:30 after 16 hours of wakefulness and after 40 hours of wakefulness is plotted. Adapted from Fig. 4.1 in Dinges and Kribbs [2]. (D) Sleep deprivation produces “lapses” evident as the emergence of slow RTs interspersed among fast RTs. Individual RTs for a healthy adult performing a PVT at 20:00 after 12 h awake (top panel) and again after 60 h awake (bottom panel). Gaps indicate false starts, the prevalence of which increases after sleep deprivation as well. Adapted from Fig. 3 in Doran et al. [3].

A hallmark characteristic of the effect of sleep loss on PVT performance, as compared to well-rested baseline, is a pronounced rightward skewing of the RT distribution [2]; see Figure 1C. This phenomenon has also been observed in other RT tasks [56], in particular in the context of time-on-task effects [57] and the effects of sleep deprivation on performance [58] and their interaction [59, 60]. It has been interpreted as “gaps”, blocks”, or “lapses” evident through the emergence of slow RTs in response to stimuli being interspersed among otherwise fast responses that are also still observed, even in severely sleep-deprived individuals [3]; see Figure 1D. Several metrics extracted from PVT data to quantify the degree of performance impairment have focused on capturing this phenomenon, giving rise to the number of “lapses of attention” (traditionally defined as RTs > 500 ms) as one frequently used PVT outcome measure [4]. While other metrics have been developed to capture changes in the RT distribution more fully [61, 62], recognition of the “lapses” phenomenon has led to the idea that vigilant attention waxes and wanes over time scales of seconds and minutes and that frequent sampling is critical to capture this attentional instability [3]. Consequently, the impact of sleep deprivation on PVT performance may be determined at least in part by the density of stimuli.

An early theoretical interpretation posited that lapses are discrete events indicative of alertness being intermittently blocked [58]. However, this “lapse hypothesis” does not account for the previously observed, transiently mitigating effect of high incentives on performance impairment during sleep deprivation [63] or the interacting effects of sleep deprivation and stimulus quality on PVT performance [64]. Dinges and colleagues, noting that performing the PVT while sleep-deprived and across time on task requires compensatory effort, introduced the wake “state instability” hypothesis [3]. It proposes that the effects of sleep deprivation on performance involve escalating wake state instability from elevated homeostatic (and/or circadian) sleep drive resulting in rapid and uncontrolled sleep initiation, which subjects seek to resist using increasingly greater top-down compensatory effort to maintain task performance [65]. In this view, the number of lapses is seen as an index of state instability, whereas the number of false starts (which should be small) is seen as an index of compensatory effort. This way, the PVT is thought to expose the moment-to-moment variability in attention brought about by the interaction of the biological sleep drive with sustained compensatory effort exerted by participants performing the task [3].

Van Dongen and colleagues, building on “local sleep” theory [66], later proposed a bottom-up explanation for attentional instability on the PVT [67]. In this perspective, performance variability on the PVT is attributed to the task’s requirement for repetitive use of the same neuronal pathway(s), which would thereby be driven into a local (i.e. pathway-specific) sleep-like state in a use-dependent manner as previously demonstrated in a rodent model [68]. Consistent with this notion, a study of neurosurgical patients in which local neuronal activity was recorded through intracranial electrodes while they were performing a face/non-face categorization vigilance task found that performance lapses on the task were preceded by sleep-like changes in local field potentials [69]. As postulated [52], local sleep theory predicts that the greater the use (through extended wakefulness, longer task duration, and/or higher stimulus density), the greater the degradation of neuronal processing, and thus the greater the performance impairment on the PVT [62]. Regardless of whether or not the state instability hypothesis or the local sleep explanation will ultimately hold up to scientific scrutiny [6], from all the empirical and theoretical evidence discussed thus far it is evident that influential factors in the impact of sleep loss on PVT performance should include at least the following:

• Task duration—a longer task duration should be associated with extended neuronal use and a more pronounced time-on-task effect, and therefore should contribute to performance impairment on the PVT.

• Stimulus density—a greater number of stimuli per minute of task duration should increase the probability of capturing attentional instability, and therefore should contribute to task sensitivity; it may also be associated with more intense neuronal use, thereby amplify the time-on-task effect, and thus further contribute to impairment captured by the PVT.

• Response-stimulus intervals—the effect of sleep deprivation is most noticeable for shorter RSIs, and thus restricting the range of RSIs to the shorter RSIs should intensify performance impairment on the PVT.

Basner and colleagues [10, 12] leveraged these factors to reduce the duration of the PVT while maintaining relatively high sensitivity to sleep loss. Specifically, they showed that an abbreviated version of the PVT with a 70% decrease in task duration (down to 3 min), a near-doubling of the stimulus density, and a restricted RSI range (from 2 to 5 s only) was associated with a disproportionately modest reduction in statistical effect sizes during total and partial sleep deprivation experiments, as compared to the standard (i.e. original) 10-min PVT. The authors had expected even better sensitivity for the brief version of the PVT, however, and suggested that the use of a different device for the 3-min high-density PVT than for the standard 10-min PVT in their study alerted subjects of the shorter task duration. As such, they potentially could more easily exert compensatory effort to maintain performance even when sleep-deprived [10]. Thus, even though the shorter RSIs and consequently increased stimulus density allowed for a shorter task duration while maintaining reasonably high sensitivity to sleep loss, whether stimulus density would prove to be a key driver of the impact of sleep loss on PVT performance if task duration and other task characteristics were held constant remains to be determined.

Here, in a controlled laboratory study, we juxtaposed the effects of sleep deprivation on a standard desktop PVT with a high stimulus density version, while keeping the task duration fixed at 10 min and all other hardware and software specifications the same. We predicted that after sleep deprivation, relative to baseline, the high-density PVT (HD-PVT) would:

• Capture more attentional instability;

• Exhibit a steeper time-on-task effect;

• Bring about the enhanced sleep deprivation effect associated with shorter RSIs;

and compared to the standard PVT should therefore be unequivocally more sensitive to sleep loss, operationally defined as showing a higher level of performance impairment. Our objective was to test this prediction so as to gain deeper insights into the PVT’s exceptional sensitivity to sleep loss and its neurobehavioral underpinnings.

Methods

Subjects

n = 86 healthy, adult subjects (39 males, 47 females) between the ages of 21 y and 38 y (mean age ± SD: 26.6 ± 5.2 y) completed a 4-day/3-night, in-laboratory study. They were randomly assigned with a 2:1 probability ratio to a total sleep deprivation (TSD) condition (n = 56) or a well-rested control condition (n = 30). In the TSD condition, the mean age ± SD was 25.9 ± 5.3 y, with 24 males and 32 females. In the control condition, the mean age was 27.9 ± 4.9 y, with 15 males and 15 females.

Subjects were screened through a telephone interview, two in-laboratory screenings, an at-home week of sleep monitoring with actigraphy and a sleep diary, and baseline polysomnography. All were found to be physically and psychologically healthy, with no medical or drug treatment (except oral contraceptives) and no current drug or alcohol use (verified with an alcohol breathalyzer and urine drug screen both at screening and at the start of the laboratory study). Subjects had no history of drug or alcohol abuse in the past year, and no history of methamphetamine abuse; were not a current smoker; had no history of moderate to severe brain injury; had no present or presently clinically relevant history of psychiatric illness; were not pregnant; reported no previous adverse reaction to sleep deprivation; had no history of a learning disability; and were not vision impaired (unless corrected to normal). They had no sleep or circadian disorders; reported good habitual sleep, between 6 and 10 h in duration; had regular bedtimes, habitually getting up between 06:00 and 09:00; did not travel across time-zones within one month of entering the study; and did not engage in shift work within 3 months of entering the study.

Experimental design

On the first day of the study, subjects entered the laboratory at 15:30, were oriented to the laboratory environment, and then were trained on a number of cognitive tests, including the standard PVT. During this training, members of the research team worked one-on-one with each of the subjects at their testing station in their individual room to instruct them on the tasks and clarify the demand characteristics. This included general instructions for testing (e.g. maintain good and stable posture, maintain attention without fidgeting, etc.) and instructions for completing specific tests (e.g. whether to respond using the keyboard/mouse, etc.). For the PVT, the researcher indicated that the subject should press the designated button on a response box as fast as possible whenever the rolling millisecond counter stimulus appeared. Specific instructions were provided to emphasize that subjects should try their very hardest to respond as fast as they could, even if they experienced this to be difficult to sustain (i.e. avoid errors of omission), and to avoid making false starts (i.e. avoid errors of commission). Subjects were asked to give their best performance every time, to use determination and focus, and to aim to have consistently fast RTs. The researcher then positioned themself behind the subject and observed the performance throughout the entire 10-min test, providing feedback as needed so that the subject understood the expectations and could produce fast and relatively consistent RTs at baseline.

At the end of the first day, subjects had a 10-h sleep opportunity from 22:00 to 08:00. At 18:00 on day 2, subjects were told whether they had been assigned to the TSD condition or the control condition. Subjects in the TSD condition were kept awake from 08:00 on day 2 until 22:00 on day 3 (38 h of TSD), while subjects in the control condition had another 10-h sleep opportunity on the second night (22:00–08:00). All subjects had a final 10-h sleep opportunity from 22:00 until 08:00 on the final night; they left the laboratory around 11:30 on day 4.

Sleep periods were recorded polysomnographically. During scheduled wakefulness, various performance tasks were administered, and meals and snacks were served every 4 h (including during the night when subjects in the TSD condition were awake). Subjects were in their own individual room while sleeping and during performance testing. When not eating, sleeping, or testing, they were only allowed non-vigorous activities such as reading or chatting with other participants or research staff. No visitors and no contact with people outside the laboratory were allowed. Trained research assistants monitored the subjects’ behavior throughout the study.

During the scheduled wake periods, a computerized test battery was administered at approximately 2-h intervals. The test battery, which took approximately 25 min to complete, included the Karolinska Sleepiness Scale (KSS; [70]), a performance prediction scale, the standard PVT, a 9-point computerized variation on a mood visual analog scale [38], the Positive and Negative Affect Schedule (PANAS; [71]), another KSS, a continuous performance matching task [72], and a computerized digit symbol substitution test (4-min standard version; [73]). The HD-PVT was administered once during the study, as a standalone test at 18:00 on day 3, when subjects in the control condition had been awake for 10 h and those in the TSD condition had been awake for 34 h.

For the present purposes, only the HD-PVT at 18:00 and the surrounding standard PVTs at 17:00 and 19:00 on day 3 were analyzed. To avoid carry-over effects, there was a break of more than 30 min between the standard PVTs and the HD-PVT; see Figure 2. The standard PVT and the HD-PVT were administered on the same desktop computer using the same base software and same stimulus presentation. Responses were made with a custom-made response box, positioned flat on the desk in front of the subject, and calibrated to record RTs with a precision standard deviation of no more than 10 ms, as recommended by Basner and colleagues [74]. The tasks were administered while subjects were seated at a computer in their individual bedrooms, with continuous monitoring by the research team from a separate observation room via camera and microphone. Both the standard PVT and the HD-PVT tasks were 10 min in duration. Any impact of task novelty from the single administration of the HD-PVT should be mitigated by these experimental and test platform standardizations.

Figure 2.

Testing timeline, task specifications, and descriptive statistics.

While the standard PVT had an RSI range from 2 to 10 s (in steps of 1 s) and averaged 95 trials per test bout, the HD-PVT had an RSI range from 2 to 5 s (in steps of 1 s) and averaged 157 trials per test bout. The probability of each RSI occurring was evenly distributed within each test version, with approximately 11.1% of trials having each RSI value (2 to 10 s) on the standard PVT and approximately 25% of trials having each RSI value (2 to 5 s) on the HD-PVT. Other than the RSI range, the two PVT versions were identical. Thus, the modified RSI range and resultant difference in stimulus density were expected to drive any differences in results between the task versions.

Statistical analyses

To verify the PVT manipulation (standard versus high-density version), the number of trials in each test bout (17:00 PVT, 18:00 HD-PVT, and 19:00 PVT) was compared using a mixed-effects analysis of variance (ANOVA) with fixed effects for condition (TSD, control), bout (17:00, 18:00, 19:00), and their interaction. To investigate the overall effect of TSD on the different versions of the PVT, the mean RT, number of lapses (RTs > 500 ms), mean RT of lapses, mean percent of responses that were lapses, and number of false starts for each test bout were analyzed using mixed-effects ANOVA with fixed effects for condition, bout, and their interaction. Additionally, as part of this analysis, the standard PVT and the HD-PVT were compared head-to-head through planned contrasts, both with regard to the mean outcome (for mean RT, number of lapses, mean RT of lapses, percent lapses, and false starts) between tasks and with regard to the TSD effect (difference Δ between the TSD and control conditions) within and between tasks. To investigate the time-on-task effect, the mean RT per 1-min bin in each test bout was analyzed for the two conditions separately using mixed-effects ANOVA with fixed effects for bout, bin (1–10), and their interaction. The magnitude of the time-on-task effect, quantified as the difference in mean RT from the first minute (bin 1) to the last minute (bin 10) of the task, was compared between tasks by means of a planned contrast. Lastly, to investigate the RSI effect, the mean RT for each test bout was analyzed for the two conditions separately using mixed-effects ANOVA with fixed effects for bout, RSI (using only the RSIs in common between the tests: 2–5 s), and their interaction. The magnitude of the RSI effect, quantified as the difference in mean RT from trials with RSIs of 2 and 5 s, was compared between tasks by means of a planned contrast. All analyses included a random effect over subjects on the intercept. The type I error threshold was set at α = 0.05; significance levels between 0.05 and 0.10 are reported as trends.

Results

Figure 2 shows specifications and descriptive statistics for the three test bouts of interest (17:00 PVT, 18:00 HD-PVT, and 19:00 PVT). The difference in the RSI range between the two PVT versions caused a more than 60% increase in the number of trials in the HD-PVT compared to the standard PVT (main effect of bout: F_2,168 = 9423.5, p < 0.001, η_p² = 0.990), confirming that the HD-PVT did indeed have high stimulus density as intended; see Figure 2. TSD caused no change to the number of trials (main effect of condition: F_1,168 = 0.7, p = 0.41, η_p² = 0.004), with a trend for the condition by bout interaction: (F_2,168 = 2.4, p = 0.097, η_p² = 0.027).

As expected, the mean RT was significantly slower in the TSD condition than in the control condition (main effect of condition: F_1,168 = 10.7, p = 0.001, η_p² = 0.060). Surprisingly, in both conditions the HD-PVT exhibited faster mean RT than the standard PVT (main effect of bout: F_2,168 = 15.1, p < 0.001, η_p² = 0.152), with no differential effect of TSD on the two task versions (condition by bout interaction: F_2,168 = 1.6, p = 0.21, η_p² = 0.018); see Figure 3A (top left). Head-to-head comparison of the two task versions across both the sleep deprivation and control conditions through planned contrast revealed that the mean difference (± SE) was –34.3 ± 6.6 ms (planned contrast: F_1,168 = 27.3, p < 0.001, η_p² = 0.140), with a faster mean RT on the HD-PVT (mean ± SE: 277.8 ± 11.6 ms) than on the standard PVT (mean ± SE: 312.1 ± 11.0 ms). There were significant effects of TSD on the HD-PVT (ΔRT mean ± SE: 61.5 ± 23.2 ms; F_1,168 = 7.0, p = 0.009, η_p² = 0.040) and on the standard PVT (ΔRT mean ± SE: 74.7 ± 21.9 ms; F_1,168 = 11.6, p < 0.001, η_p² = 0.064), but no difference between the TSD effects on the two tasks (F_1,168 = 1.0, p = 0.32, η_p² = 0.006). Thus, despite arguably being associated with more intense neuronal use due to the higher stimulus density, the HD-PVT exhibited no greater performance changes in response to sleep loss than the standard PVT.

Figure 3.

(A) Mean RT, (B) number of lapses, (C) percent of lapses, and (D) false starts. Data shown are mean ± SE on the standard PVT (open circles) and the HD-PVT (closed circles), by condition, as a function of measurement time. Data from control subjects are shown in blue and from TSD subjects in red.

Again as expected, there were more lapses on the PVT in the TSD condition than in the control condition (main effect of condition: F_1,168 = 20.2, p < 0.001, η_p² = 0.108). Counter to expectation, however, there was no significant difference between the two task versions regardless of condition (main effect of bout: F_2,168 = 0.9, p = 0.42, η_p² = 0.010; condition by bout interaction: F_2,168 = 0.6, p = 0.58, η_p² = 0.007); see Figure 3B (top right). Head-to-head comparison of the two task versions across both the sleep deprivation and control conditions through planned contrast also showed no significant difference (F_1,168 = 0.2, p = 0.62, η_p² = 0.001), with a similar number of lapses on the HD-PVT (mean ± SE: 4.2 ± 0.7 lapses) and on the standard PVT (mean ± SE: 4.0 ± 0.7 lapses). There were significant effects of TSD on the HD-PVT (Δlapses mean ± SE: 6.5 ± 1.4 lapses; F_1,168 = 20.4, p < 0.001, η_p² = 0.108) and on the standard PVT (Δlapses mean ± SE: 5.6 ± 1.3 lapses; F_1,168 = 17.6, p < 0.001, η_p² = 0.095), but no difference between the TSD effects on the two tasks (F_1,168 = 0.9, p = 0.34, η_p² = 0.006). Further, an analysis of mean RT for lapses only revealed no significant effects for condition (F_1,120 = 2.29, p = 0.13), bout (F_2,120 = 0.88, p = 0.42), or condition by bout interaction (F_2,120 = 0.05, p = 0.95). That is to say, despite its higher sampling frequency, the HD-PVT did not capture more lapses of attention than the standard PVT, nor were mean lapse RTs in the HD-PVT any longer than in the standard PVT.

Repeating the lapse analysis with the percent of responses that were lapses instead of the raw number of lapses similarly revealed a larger percent of lapses in the TSD condition than in the control condition (main effect of condition: F_1,168 = 20.1, p < 0.001, η_p² = 0.107). Unlike the raw lapse analysis, percent lapses also showed a significant difference between the two task versions (main effect of bout: F_2,168 = 7.8, p < 0.001, η_p² = 0.085; condition by bout interaction: F_2,168 = 3.0, p = 0.052, η_p² = 0.034); see Figure 3C (bottom right). Head-to-head comparison of the two task versions across both the sleep deprivation and control conditions through planned contrast showed a significant difference (F_1,168 = 13.4, p < 0.001, η_p² = 0.074), with a smaller percent of responses that were lapses on the HD-PVT (mean ± SE: 3.0% ± 0.7%) than on the standard PVT (mean ± SE: 4.4% ± 0.7%). While there were significant effects of TSD on the HD-PVT (Δpercent lapses mean ± SE: 4.4% ± 1.4%; F_1,168 = 10.4, p = 0.002, η_p² = 0.058) and on the standard PVT (Δpercent lapses mean ± SE: 6.3% ± 1.3%; F_1,168 = 23.8, p < 0.001, η_p² = 0.124), there was a notable difference between the TSD effects on the two tasks (F_1,168 = 5.8, p = 0.017, η_p² = 0.033). Notwithstanding the increased sampling frequency, the HD-PVT displayed a lower percent of responses that were lapses than the standard PVT.

There was a trend for more false starts on the PVT in the TSD condition compared to the control condition (main effect of condition: F_1,168 = 2.7, p = 0.10, η_p² = 0.016). There were also more false starts in the HD-PVT than in the standard PVT (main effect of bout: F_2,168 = 50.1, p < 0.001, η_p² = 0.374), but no differential effect of TSD on the two task versions (condition by bout interaction: F_2,168 = 1.4, p = 0.24, η_p² = 0.017); see Figure 3D (bottom left). Head-to-head comparison of the two task versions across both the sleep deprivation and control conditions through planned contrast revealed that the mean difference (± SE) was 2.9 ± 0.3 false starts (planned contrast: F_1,168 = 99.2, p < 0.001, η_p² = 0.371), with more false starts on the HD-PVT (mean ± SE: 4.4 ± 0.3 false starts) than on the standard PVT (mean ± SE: 1.5 ± 0.3 false starts). There was a significant effect of TSD on the HD-PVT (Δfalse starts mean ± SE: 1.6 ± 0.7 false starts; F_1,168 = 5.3, p = 0.022, η_p² = 0.031), but not on the standard PVT (Δfalse starts mean ± SE: 0.6 ± 0.6 false starts; F_1,168 = 1.0, p = 0.33, η_p² = 0.006), and a trend of a difference between the TSD effects on the HD-PVT and the standard PVT (F_1,168 = 2.9, p = 0.092, η_p² = 0.017). While the difference in false starts between the two task versions may simply be a consequence of having more stimuli to respond to on the HD-PVT, 98% of HD-PVT bouts and in 99% of standard PVT bouts had fewer than 10% of responses as false starts, which is comparable to previously reported false start rates on the standard PVT during TSD [3].¹

There was a significant time-on-task effect in the TSD condition (main effect of bin: F_9,1594 = 15.7, p < 0.001, η_p² = 0.081); see Figure 4. Overall, mean RT per bin was lower in the HD-PVT than in the standard PVT (main effect of bout: F_2,1594 = 21.2, p < 0.001, η_p² = 0.026), but there was no significant difference in the manifestation of the time-on-task effect between the two task versions in the TSD condition (bout by bin interaction: F_18,1594 = 0.6, p = 0.91, η_p² = 0.007). Still, the magnitude of the time-on-task effect, quantified as the difference in mean RT from the first to the last minute of the task, was trending toward smaller (planned contrast: F_1,1594 = 3.0, p = 0.083, η_p² = 0.002) for the HD-PVT (mean ± SE: 51.7 ± 26.0 ms; F_1,1594 = 4.0, p = 0.047, η_p² = 0.002) than for the standard PVT (mean ± SE: 106.8 ± 18.3 ms; F_1,1594 = 34.1, p < 0.001, η_p² = 0.021).

Figure 4.

Time-on-task effect displayed as mean RT per 1-min bin on the standard PVT and the HD-PVT. Data shown are mean ± SE by condition as a function of measurement time.

There was also a significant time-on-task effect in the control condition (main effect of bin: F_9,841 = 6.0, p < 0.001, η_p² = 0.060), albeit much more modest in magnitude; see Figure 4. Overall, mean RT per bin was again lower in the HD-PVT than in the standard PVT (main effect of bout: F_2,841 = 122.7, p < 0.001, η_p² = 0.226), with a significant difference in the manifestation of the time-on-task effect between task versions (bout by bin interaction: F_18,841 = 1.8, p = 0.018, η_p² = 0.038). The time-on-task effect was smaller (planned contrast: F_1,841 = 2.4, p = 0.13, η_p² = 0.003) and essentially eliminated on the control condition for the HD-PVT (mean ± SE: 5.2 ± 6.4 ms; F_1,841 = 0.7, p = 0.42, η_p² < 0.001) as compared to its already modest magnitude on the standard PVT (mean ± SE: 17.2 ± 4.5 ms; F_1,841 = 14.43, p < 0.001, η_p² = 0.017). Thus, contrary to expectation, the higher stimulus density of the HD-PVT did not amplify the time-on-task effect, but significantly dampened it in both the TSD and control conditions.

There was a significant RSI effect in the TSD condition (main effect of RSI: F_3,605 = 13.4, p < 0.001, η_p² = 0.062); see Figure 5 (left). Overall, the mean RT at each RSI from 2 to 5 s (the RSI range both task versions had in common) was faster in the HD-PVT than in the standard PVT (main effect of bout: F_2,605 = 25.0, p < 0.001, η_p² = 0.076), but there was no other difference in the manifestation of the RSI effect between the two task versions in the TSD condition (bout by RSI interaction: F_6,605 = 0.5, p = 0.84, η_p² = 0.005). The magnitude of the RSI effect, quantified as the difference in mean RT between the RSIs of 2 and 5 s, was not significantly different (planned contrast: F_1,605 = 1.6, p = 0.20, η_p² = 0.003) between the HD-PVT (mean ± SE: 60.2 ± 24.4 ms; F_1,605 = 6.07, p = 0.014, η_p² = 0.010) and the standard PVT (mean ± SE: 98.5 ± 17.3 ms; F_1,605 = 32.6, p < 0.001, η_p² = 0.051).

Figure 5.

Response-stimulus interval (RSI) effect displayed as mean RT for each RSI on the standard PVT and the HD-PVT. Data shown are mean ± SE per RSI duration in the TSD condition (left) and the control condition (right) on the standard PVT at 17:00 (open circles), the HD-PVT at 18:00 (filled circles), and the standard PVT at 19:00 (open triangles). Note that although the figure shows all RSIs for each test (i.e. 2–10 s for the standard PVT and 2–5 s for the HD-PVT), the analyses of the RSI effect included only RSIs common between the two task versions, as indicated by the gray rectangles (i.e. 2–5 s).

There was also a significant RSI effect in the control condition (main effect of RSI: F_3,319 = 102.8, p < 0.001, η_p² = 0.492); see Figure 5 (right). Overall, the mean RT at each RSI from 2 to 5 s was faster in the HD-PVT than in the standard PVT (main effect of bout: F_2,319 = 185.0, p < 0.001, η_p² = 0.537), with no significant difference in the manifestation of the RSI effect between task versions (bout by RSI interaction: F_6,319 = 1.8, p = 0.10, η_p² = 0.032). The magnitude of the RSI effect was significantly smaller (planned contrast: F_1,319 = 7.1, p = 0.008, η_p² = 0.022) for the HD-PVT (mean ± SE: 41.7 ± 5.7 ms; F_1,605 = 52.9, p < 0.001, η_p² = 0.142) than for the standard PVT (mean ± SE: 60.3 ± 4.1 ms; F_1,605 = 221.7, p < 0.001, η_p² = 0.410).

Discussion

In this controlled laboratory study with a total sleep deprivation (TSD) condition and a corresponding control condition, we made within-subjects comparisons of the effects of sleep deprivation on two versions of the PVT: one being the standard PVT [1, 5] and one with a higher stimulus density (HD-PVT) implemented specifically for this study. Administration of the HD-PVT was flanked by administration of the standard PVT within a 2-h timeframe (Figure 2), eliminating sleep homeostatic (time awake) and circadian (time of day) effects as potential confounds; and with 86 carefully screened, healthy subjects, our statistical power was high. Accordingly, both the standard PVT and the HD-PVT showed significant performance impairment due to TSD, including slower mean RT, more lapses, and increased time-on-task effect as compared to the control condition (Figures 3 and 4).

However, the effects of TSD on the two task versions were not identical. As described above, we had a priori predicted that the greater stimulus density and truncated RSI range of the HD-PVT would result in increased performance impairment in response to sleep loss compared to the standard PVT, given the fixed 10-min task duration and all other task characteristics being the same. Yet, that is clearly not what the data showed. Contrary to expectation, there were multiple indications that performance on the HD-PVT is not more, and in some respects is even less, impaired due to TSD than on the standard PVT. This includes faster mean RTs and an equivalent number of lapses on the HD-PVT, with no differential impact of TSD relative to the standard PVT—except when considering the percent of responses that were lapses, for which the HD-PVT showed a relatively dampened response to TSD.

The faster mean RTs are by themselves unexpected, as the more restricted, lower RSI range of the HD-PVT would have been anticipated to emphasize the RSI effect and thus increase mean RT. That said, the overall faster RTs on the HD-PVT could have artificially reduced the number of lapses, as greater skewing of the RT distribution would be needed for the tail of the RT distribution to cross the conventional 500 ms lapse threshold. Indeed, it has been pointed out that an increase in sensitivity could be achieved by lowering the lapse threshold [10, 12], which would also make effective use of the greater number of trials in the HD-PVT. The increase in sensitivity from such an adaptation would however be strictly statistical in nature (i.e. pertaining to effect size only). Note also that responses on the HD-PVT included more false starts than on the standard PVT, but false starts remained rare and should have little relevance for understanding the differential sensitivity of the two task versions.

Surprisingly, the magnitude of the time-on-task effect—a hallmark response of PVT performance during TSD—was reduced on the HD-PVT compared to the standard PVT. The dampening of the time-on-task effect was observed in the TSD condition as well as the control condition, indicating that it is at least partially characteristic of the HD-PVT per se and not solely associated with the effect of TSD. In every case, the differences between the two tasks in the magnitude of the response to TSD were associated with commensurate differences in the statistical effect size (η_p²), underlining that the HD-PVT is not more sensitive to sleep loss than the standard PVT, but rather the opposite. Thus, our prediction that the greater stimulus density and reduced RSI range of the HD-PVT would render this task version more responsive to sleep loss was incorrect. It follows that by itself, stimulus density cannot be the sole or main driver of the impact of sleep loss on PVT performance.

This finding has important theoretical implications. Firstly, as evidenced by the equivalent mean number of lapses per bout and the smaller fraction of trials observed as lapses during TSD, the increased stimulus density of the HD-PVT did not result in greater detection of lapses. This is congruent with David Dinges’ assertion that lapses are not a discrete phenomenon, but rather a reflection of a more gradual, continuous waxing and waning of attention indicative of wake state instability [3], as has also been recognized in more recent PVT research [75]. As such, enhancing the sampling frequency by increasing the stimulus density does not meaningfully improve the PVT’s effectiveness of capturing lapses of attention during task performance.

Secondly, our results do not support the ideas that the increased stimulus load of the HD-PVT is associated with more workload [10] or greater neuronal use [52] leading to greater performance impairment and amplified time-on-task and RSI effects. It would seem that some other facet of the HD-PVT must in fact provide some relief from these challenges of performing the task when sleep-deprived. A bottom-up perspective on performance impairment from sleep loss [67] does not, by itself, offer any clues as to what that facet might be.

Computational cognitive modeling accounts of the effects of sleep deprivation on the PVT [75, 76] may offer a further insight. Specifically, the effects of TSD on performance of the standard PVT have been described with a one-choice diffusion decision model [75]; see Figure 6. In this computational model, after encoding of the stimulus, a decision process takes place that involves accumulation of evidence through neuronal processing until an evidence criterion is met, upon which a response is made. Stimulus encoding and motor responses are relatively unaffected by sleep loss [77, 78], leaving the decision process as the model component of interest. This decision process can be described as a random walk with a drift rate that varies naturally across trials and is the main driver of the RT distributions produced when subjects perform the task. Consistent with the idea of bottom-up degradation of neural processing [67], the primary effect of TSD was an increase in the mean drift rate [75], which is associated with the characteristic skewing of the RT distribution and the manifestation of state instability (Figure 1C and D).

Figure 6.

Illustration of the one-choice diffusion decision model for the PVT. After presentation of a stimulus and subsequent stimulus encoding, a decision process takes place during which evidence is accumulated through neuronal processing resembling a random walk with a certain drift rate. When the accumulated evidence reaches an evidence criterion, the decision process stops and a response is made. While performing the PVT, the evidence criterion must be set in advance of the stimulus onset and maintained throughout the RSI, which in the face of sleep deprivation may require sustained effort. Figure adapted from Ratcliff and Van Dongen [75].

In this account of the PVT, the evidence criterion—which reflects the demand characteristics of the task and represents the balance between trying one’s hardest to respond and not making any false starts (i.e. between avoiding errors of omission and avoiding errors of commission; [3])—was essentially unchanged by TSD [75]. However, in a go/no-go variant of the PVT, which required either responding or withholding a response based on which of two different stimuli appeared after a variable RSI from 2 to 10 s, subjects traded speed for accuracy (i.e. avoiding false starts) when sleep deprived [79]. In the diffusion decision model, this corresponds with subjects raising the evidence criterion, which is under behavioral control [80], and doing so reduces sensitivity to sleep loss [79].

For the present study, the demand characteristics associated with both the standard PVT and the HD-PVT emphasized response speed and urged subjects to apply compensatory effort to try to maintain optimal performance in the face of sleep deprivation. This was reinforced during training on the standard PVT at the beginning of the laboratory study and via continuous behavioral monitoring throughout the study. Furthermore, the PVT and HD-PVT provided performance feedback by displaying the RT for 1 s immediately after each trial, which further emphasized the importance of compensatory effort to maintain good performance [81]. Effectively, subjects were driven to exert sustained effort during the variable durations of the RSIs to maintain a tight evidence criterion, even when sleep deprivation—and the associated wake state instability and hypothesized bottom-up degradation of neuronal processing due to local sleep—made it difficult to do so. It stands to reason that this exertion of sustained effort to maintain a tight evidence criterion throughout the RSIs would have come at a performance cost, could give rise to the increased time-on-task effect, and may in fact be a central aspect of what makes the PVT sensitive to sleep loss. Indeed, as Dinges and colleagues pointed out [3], the increasing top-down compensatory effort that subjects must make to resist the escalating wake state instability from elevated homeostatic (and/or circadian) sleep drive during TSD appears to be pivotal in understanding the impact of TSD on PVT performance.

Previously published work on the one-choice diffusion decision model of the PVT [75] has not explicitly addressed this aspect of the PVT. It has also not provided an explanation for the time-on-task effect and its interaction with TSD (Figure 1A). The present results from a head-to-head comparison between the standard PVT and the HD-PVT may help to fill this gap. That is, if exerting compensatory effort to maintain a tight evidence criterion throughout the RSIs is key to the impact of sleep loss on PVT performance, then increasing the stimulus density and making the RSIs shorter (while holding task duration constant) would reduce the requirement to sustain such effort through long RSIs, which should lessen the time-on-task effect and its interaction with TSD and thereby the task’s overall performance responsiveness to sleep loss—as indeed we did observe for the HD-PVT (Figures 3 and 4, respectively).

Thus, our current, empirical results support previous speculation that PVT performance relies on a combination of bottom-up and top-down attentional mechanisms [82]. That is, although the sleep loss-induced degradation of cognitive processing for stimulus detection and response is principally a bottom-up problem, maintaining good performance to meet the task demands requires top-down attentional control. The latter is not only essential for task compliance, but may also be where the perceived cognitive demand comes from [83]. This sheds new light on why subjective sleepiness is increased right after performing the PVT [84] and why it is so hard to perform the task when sleep-deprived. Follow-up research in older populations with potentially differential, age-related changes in bottom-up versus top-down attentional functioning [85] could provide helpful data to further evaluate this explanation. Furthermore, we speculate that other performance tasks with RSIs of unpredictable duration, if they require rapid responses (in order to be as fast as possible or to meet short response deadlines), would be similarly impacted by sleep loss.

Submitted for inclusion in the special issue, Festschrift for David Dinges

This paper is part of the David F. Dinges Festschrift Collection. This collection is sponsored by Pulsar Informatics and the Department of Psychiatry in the Perelman School of Medicine at the University of Pennsylvania.

Funding

This work was supported by the Office of the Assistant Secretary of Defense for Health Affairs, through the Peer Reviewed Medical Research Program under Award No. W81XWH-16-1-0319 and through the Peer Reviewed Medical Research Program Expansion Award under Award No. W81XWH-20-1-0442. Opinions, interpretations, conclusions, and recommendations are those of the author and are not necessarily endorsed by the Department of Defense.

Disclosure Statement

The authors have no financial or non-financial conflicts of interest.

Data Availability

Upon reasonable request to K.A.H., the deidentified data can be shared with researchers.