Signal-to-Noise Ratio in PVT Performance as a Cognitive Measure of the Effect of Sleep Deprivation on the Fidelity of Information Processing

Abstract

Study Objectives:

There is a long-standing debate about the best way to characterize performance deficits on the psychomotor vigilance test (PVT), a widely used assay of cognitive impairment in human sleep deprivation studies. Here, we address this issue through the theoretical framework of the diffusion model and propose to express PVT performance in terms of signal-to-noise ratio (SNR).

Methods:

From the equations of the diffusion model for one-choice, reaction-time tasks, we derived an expression for a novel SNR metric for PVT performance. We also showed that LSNR—a commonly used log-transformation of SNR—can be reasonably well approximated by a linear function of the mean response speed, LSNR_apx. We computed SNR, LSNR, LSNR_apx, and number of lapses for 1284 PVT sessions collected from 99 healthy young adults who participated in laboratory studies with 38 hr of total sleep deprivation.

Results:

All four PVT metrics captured the effects of time awake and time of day on cognitive performance during sleep deprivation. The LSNR had the best psychometric properties, including high sensitivity, high stability, high degree of normality, absence of floor and ceiling effects, and no bias in the meaning of change scores related to absolute baseline performance.

Conclusions:

The theoretical motivation of SNR and LSNR permits quantitative interpretation of PVT performance as an assay of the fidelity of information processing in cognition. Furthermore, with a conceptual and statistical meaning grounded in information theory and generalizable across scientific fields, LSNR in particular is a useful tool for systems-integrated fatigue risk management.

Statement of Significance

By introducing a novel, signal-to-noise-ratio metric for a widely used fatigue assay, the psychomotor vigilance test (PVT), this research connects performance impairment in human sleep deprivation studies with a theoretically grounded, benchmark index of the fidelity of information processing in human central cognition. This work contributes theoretical insight into the effects of sleep loss and circadian misalignment on brain functioning and informs the debate about which metrics to extract from the PVT response time distribution. Furthermore, this work is relevant in applied settings by providing a basis for quantifying fatigue risks in sleep-deprived individuals and for spurring a new line of research into the reliability of partially automated, integrated systems with sleep-deprived humans in the loop.

INTRODUCTION

The psychomotor vigilance test (PVT) is one of the most widely used assays of impairment in behavioral alertness in human sleep deprivation studies.¹ The PVT is a one-choice, reaction-time task requiring subjects to respond as quickly as possible to a visual stimulus. In the standard implementation of the task, the duration of a test session is 10 min, and trials occur at random intervals of 2–10 s. Performance on the PVT is highly sensitive to fatigue from sleep loss, circadian misalignment, and time on task,² while showing minimal effects of aptitude or practice. Interindividual differences in performance impairment on the task are stable and can be partially explained by genetic makeup.³

The effects of sleep loss, circadian misalignment, and time on task on performance in a PVT session are characterized by an increase in response time (RT) variability across trials—particularly in the form of a skewing of the RT distribution to the right such that the biggest impact tends to be observed in the right tail of the distribution (Figure 1a)—in conjunction with a more modest increase in false starts.¹ Evidence from neuroimaging studies indicates that these effects, and interindividual differences therein, may be explained in terms of reduced processing capacity in the brain.⁴ Furthermore, through cognitive modeling, these effects have been linked to deficits in central cognition (rather than stimulus detection and response execution), which result in the brain superimposing noise onto stimuli while they are being processed.⁵

Figure 1.

Psychomotor vigilance test (PVT) metrics and the effects of sleep deprivation. (a) Illustration of the response time (RT) distribution at baseline (solid) and after a night of total sleep deprivation (dotted). The inset shows the number of responses (lapses) in the right tail of the distribution (RTs ≥ 500 ms) at baseline (BL; solid) and after a night of total sleep deprivation (SD; hatched). Figure adapted from Honn et al.¹⁰ with permission. (b) Illustration of the diffusion model for one-choice, reaction-time tasks. Figure adapted from Ratcliff and Van Dongen⁵ with permission. (c–f) PVT metrics (means ± SE) at 2-hr intervals across 38 hr of total sleep deprivation (c: SNR, d: LSNR, e: LSNR_apx, f: number of lapses). The curves in each graph correspond to four different laboratory studies (Study 1: black; Study 2: dark gray; Study 3: light gray; and Study 4: dotted). The insets show the difference (mean and standard error) between sleep deprivation (28–38 hr awake) and the same times of day 24 hr earlier at baseline (4–14 hr awake) in each of the four studies. In the bottom right graph the vertical scales are inverted such that downward consistently corresponds to worse performance on the PVT.

There is no consensus, however, about which metric(s) to extract from the RT distribution to best capture this phenomenon.⁶ To address this issue, we can make use of a theoretical framework called the diffusion model,⁵ which describes task performance in terms of a diffusion process representing the accumulation of evidence in central cognition. The diffusion process is characterized by a drift rate that represents the speed of evidence accumulation. The process triggers a response when the evidence accumulation reaches a decision threshold (Figure 1b). In the diffusion model for one-choice, reaction-time tasks such as the PVT, the effects of sleep deprivation on the RT distribution have been shown to predominantly involve a decrease in the drift ratio, which is defined as the drift rate relative to the variability in drift rate across trials.⁵

The drift ratio is comparable to the discriminability index in signal detection theory and can be seen as a measure of the fidelity of information processing in central cognition. This suggests that the skewing of the RT distribution due to sleep loss is a consequence of intrinsic degradation of information processing due to increased neuronal processing noise.⁷ Here, we propose to express PVT performance in terms of the fidelity of information processing—specifically in terms of the well- established concept of the signal-to-noise ratio (SNR).

EXPRESSIONS FOR SNR

In the diffusion model for one-choice, reaction-time tasks, cognitive processing of a stimulus involves an encoding step, a central information-processing step (or decision step), and a response preparation step (Figure 1b). The central information-processing step is represented as a one-boundary diffusion process in which evidence is accumulated from a starting point to the decision threshold, with an accumulation rate (drift rate) that is normally distributed across trials and superimposed on Brownian noise.⁵ The encoding and response preparation steps are considered together as non-decision processes, which are assumed to vary in duration across trials according to a uniform distribution. The overall RT to a stimulus is the time it takes for the central information-processing step to terminate plus the duration of the non-decision processes.

The SNR is the ratio of the average power of the signal to the average power of the noise. To derive an expression for the SNR, we consider the amount of evidence accumulated in the central information-processing step as a function of time, in which the signal is a ramp function determined by the drift rate and the noise is the Brownian noise. Because the SNR increases as time passes, we must define it over a fixed unit of time. We can approximate the SNR from the RT data of a PVT session, as follows (see Online Supplement):

$$SNR\approx \frac{N{\left({\displaystyle \sum _{i=1}^N{w}_i{S}_i}\right)}^2}{{\displaystyle \sum _{i=1}^N\left[w_i{\left(S_i{\displaystyle \sum _{i=1}^N{w}_i-{\displaystyle \sum _{i=1}^N{w}_i{S}_i}}\right)}^2\right]}}+1$$ (1)

where S_i = 1 / (RT_i – C), w_i = 1 / (r²S_i + 1), C = 100 ms, r² = 196 ms, RT_i is the i^th RT (in ms), and N is the number of trials in the PVT session. False starts (i.e., trials with responses prior to 150 ms after stimulus onset) are not included.

In other fields that make use of the SNR, such as information theory and electrical engineering, the metric is typically computed in log-transformed form:

$$LSNR=10{\log }_{10}(SNR)$$ (2)

which is expressed in units of decibel (dB).

If the speed/accuracy trade-off⁸ (i.e., the ratio of the decision threshold to the magnitude of the Brownian noise) is stable such that the criterion individuals use to emphasize speed over avoiding false starts is nearly constant, LSNR is approximately proportional to the mean response speed (i.e., the mean of 1/RT). It is then possible to use a simple approximation of LSNR:

$$LSN{R}_{\text{apx}}=B_{}\left(\frac{1}{N}{\displaystyle \sum _{i=1}^N\frac{1}{R{T}_i}}\right)$$ (3)

Using mixed-effects linear regression on the data set presented subsequently, we found that B = 3855 ms,^a and the correlation between LSNR and LSNR_apx was 0.65.

EFFECT OF SLEEP DEPRIVATION ON SNR

We computed PVT metrics for a total of 1284 test sessions in sleep deprivation studies conducted in the Sleep and Performance Research Center at Washington State University Spokane.³ A group of 99 healthy young adults (ages 22–37, 50 females) participated in one of four in-laboratory studies (study 1: n = 37; study 2: n = 12; study 3: n = 39; study 4: n = 11). In each of the studies, after baseline sleep (10 hr time in bed ending at 08:00) subjects were exposed to 38 hr of total sleep deprivation under constant supervision. Subjects performed the PVT on a desktop computer at 2-hr intervals. During testing, they were seated at a desk in a private laboratory room with dim ambient light (fixed below 100 lux). Subjects in Study 4 also had their head position fixed with a chin rest. For all test sessions that took place at least 4 hr after baseline sleep (after any sleep inertia had fully dissipated), we extracted the following outcome metrics: SNR (Eq. 1), LSNR (Eq. 2), LSNR_apx (Eq. 3), and number of lapses (RTs ≥ 500 ms). These outcomes were analyzed as a function of time awake using mixed-effects analysis of variance (ANOVA) with fixed effects for time and study and their interaction and a random effect over subjects on the intercept.

Figure 1c–f shows the sample means (± SE) as a function of time awake for each of the PVT outcome metrics. All four metrics displayed the well-established homeostatic and circadian regulation of fatigue,¹ with performance degrading across time awake and modulated by time of day (main effect of time: F_16,1132 > 36.6, p < .001). The SNR metric (Figure 1c) exhibited the greatest dynamic range (relative to SE). Also, SNR was the only metric to reveal that well-rested baseline performance (at 4–14 hr awake) was improved by fixation of head position with a chin rest in Study 4 (dotted curve) compared to the other studies. However, the difference among studies in baseline SNR impacted the study-specific estimates of the effect of sleep deprivation (at 28–38 hr awake) relative to baseline (Figure 1c, inset)—suggesting that on the basis of SNR, the sleep deprivation effect in Study 4 was overestimated.

This was corroborated by the LSNR metric (Figure 1d), for which the difference between sleep deprivation and baseline is not biased by the absolute baseline value. Based on LSNR, the effect of sleep deprivation (28–38 hr awake) relative to baseline (4–14 h awake) was –1.9 ± 0.1 dB (grand mean ± SE), corresponding to a 36.0% reduction in the fidelity of information processing. At the trough of performance in the early morning (22 hr awake), the LSNR change from baseline was –3.0 ± 0.2 dB, corresponding to a 49.9% reduction in the fidelity of information processing. Results were similar but less pronounced for LSNR_apx (Figure 1e).

For the number of lapses (Figure 1f), the degradation of performance due to extension of wakefulness (beyond ~16 hr) seemed to be more abrupt than for the other metrics, but this is an artifact caused by a floor effect in the number of lapses at baseline. The different PVT metrics were fairly similar in their sensitivity to total sleep deprivation across all time points according to Cohen’s effect size measure f ². As indicated in Figure 1c–f, the effect sizes approached or exceeded the benchmark for “large” (f² ≥ 0.35) for all four metrics, being the highest for LSNR (f² = 0.40). However, LSNR_apx had considerably greater intraindividual stability after accounting for the mean effect of time, as determined with the intraclass correlation coefficient (ICC). Furthermore, the LSNR and LSNR_apx metrics exhibited relatively high degrees of normality around the sample means, which is a desirable property for purposes of statistical testing.

DISCUSSION

The theoretical foundation for SNR and LSNR through the diffusion model permits interpretation of these metrics as quantitative measures of the fidelity of information processing in cognitive performance. The LSNR metric outperforms the SNR metric and the number of lapses as PVT outcome measures on a range of psychometric properties, including high sensitivity to sleep deprivation and circadian misalignment, high degree of statistical normality, and absence of floor and ceiling effects. The LSNR metric has the additional advantage that, by definition, a given change in LSNR always has the same meaning regardless of absolute values. For instance, a reduction in LSNR of 3 units (i.e., a −3 dB change) from baseline to sleep deprivation may be interpreted as a 50% drop in the fidelity of information processing regardless of the absolute baseline value. Interestingly, it follows that the baseline value for LSNR may be set to 0 dB to anchor the metric, without loss of generality and without causing a floor or ceiling effect. This provides a novel solution in the area of mathematical modeling of fatigue, where metric anchoring has been a topic of debate.⁹

The approximation LSNR_apx exhibits greater intraindividual stability than the original LSNR, albeit at a cost of less precision and somewhat less sensitivity. LSNR_apx links LSNR with mean response speed (i.e., the mean of 1 / RT), which has previously been shown to be a sensitive measure of fatigue from sleep loss.⁶ As such, LSNR_apx connects the effects of sleep loss on the estimated fidelity of information processing with the existing literature on sleep deprivation and the PVT.¹ However, LSNR_apx is less precise when the speed/accuracy trade-off may vary, as in studies involving, for example, reward manipulations or administration of stimulants.

The present work connects literature on sleep loss with literature in information theory and other fields regarding the fate of information under noisy conditions and enables new lines of research to better understand, predict, and mitigate deficits in cognitive performance when people are sleep deprived (see Online Supplement for examples). Because LSNR has a conceptual and statistical meaning that generalizes across scientific fields, it also provides a basis for calculations of the overall reliability of partially automated operational systems with sleep-deprived humans in the loop. As such, the LSNR metric for the PVT may be a useful addition to the currently available tools for systems-integrated fatigue risk management.

SUPPLEMENTARY MATERIAL

Supplementary material is available at SLEEP online.

FUNDING

This work was supported by National Science Foundation grant CNS-1545104. Data collection was funded by National Institutes of Health grants R21CA167691 (study 1) and R01HL105768 (study 2), Office of Naval Research grant N00014-13-1-0302 (study 3), and Congressionally Directed Medical Research Program grant W81XWH-05-1-0099 (study 4).

INSTITUTE

This work was performed at Washington State University.

AUTHORS’ NOTE

The paper does not address any off-label or investigational drug use. This research was not a clinical trial.

DISCLOSURE STATEMENT

There are no conflicts of interest for any of the authors.