The confounding effects of eye blinking on pupillometry, and their remedy

Kyung Yoo; Jeongyeol Ahn; Sang-Hun Lee

doi:10.1371/journal.pone.0261463

. 2021 Dec 17;16(12):e0261463. doi: 10.1371/journal.pone.0261463

The confounding effects of eye blinking on pupillometry, and their remedy

Kyung Yoo ¹, Jeongyeol Ahn ¹, Sang-Hun Lee ^1,^*

Editor: Manuel Spitschan²

PMCID: PMC8683032 PMID: 34919586

Abstract

Pupillometry, thanks to its strong relationship with cognitive factors and recent advancements in measuring techniques, has become popular among cognitive or neural scientists as a tool for studying the physiological processes involved in mental or neural processes. Despite this growing popularity of pupillometry, the methodological understanding of pupillometry is limited, especially regarding potential factors that may threaten pupillary measurements’ validity. Eye blinking can be a factor because it frequently occurs in a manner dependent on many cognitive components and induces a pulse-like pupillary change consisting of constriction and dilation with substantive magnitude and length. We set out to characterize the basic properties of this “blink-locked pupillary response (BPR),” including the shape and magnitude of BPR and their variability across subjects and blinks, as the first step of studying the confounding nature of eye blinking. Then, we demonstrated how the dependency of eye blinking on cognitive factors could confound, via BPR, the pupillary responses that are supposed to reflect the cognitive states of interest. By building a statistical model of how the confounding effects of eye blinking occur, we proposed a probabilistic-inference algorithm of de-confounding raw pupillary measurements and showed that the proposed algorithm selectively removed BPR and enhanced the statistical power of pupillometry experiments. Our findings call for attention to the presence and confounding nature of BPR in pupillometry. The algorithm we developed here can be used as an effective remedy for the confounding effects of BPR on pupillometry.

Introduction

Even when ambient light is controlled, our pupil keeps dilating and constricting as mental operations or internal states of various sorts transpire in the course of carrying out diverse cognitive tasks [1–8]. This pupil-size dynamics is known to be tightly coupled with the activation of norepinephrine-containing neurons in the locus coeruleus (LC-NE system) and in other brain regions associated with the LC-NE system such as the colliculi and cingulate cortex [9–12]. Moreover, thanks to recent technical advancements, pupil size can be measured with high temporal resolution in non-invasive manners even when animals or humans are allowed to move their eyes and head relatively freely. For these reasons, pupil size became a popular physiological measure among cognitive scientists and neuroscientists, being considered a “peripheral window” into internal cognitive and neural processes [13,14].

Despite the growing need and popularity of pupillometry, there is a lack of research on eye blinking as a confounding factor in pupillary experiments. Intriguingly, the pupil constricts and re-dilates within a few seconds after each event of eye blinking (Fig 1A, Supplementary S1 Video). This phenomenon, which will be referred to as “blink-locked pupillary response (BPR)” hereafter, is, in itself, not a discovery [15]. Although it is not entirely clear what causes BPR, the visual changes caused by the pupil’s transient occlusion by the eyelid are considered its most likely source [16,17]. However, the presence of BPR seems not well known among researchers using pupillometry. They are not referred to in most pupillometry studies, including where the relationship between blinks and pupil size was investigated [18,19]. Even when BPR was mentioned, BPR was described as an “unexpected” phenomenon, and the data that were considered contaminated by BPR were simply discarded from further analysis [20].

Fig 1 — Raw pupil diameter time course example (top panel), and snapshots of recorded eye video (bottom panel). At the top panel, red bars indicate blinks, and blue vertical lines indicate the time of each eye image snapshot. At the bottom panel, the blue circle is a pupil area defined by the eye tracker, and the cyan circle is a corneal reflection. The snapshots are 4-s long and 2Hz frame rate. It clearly shows blink-locked pupillary response (BPR), that pupil constricts and re-dilates transiently after blinking. See the video (S1 Video) for vivid BPR examples.

There are reasons to believe that BPR should be considered a serious confounder that potentially threatens pupillometry experiments’ internal validity or statistical power. First, humans blink spontaneously 3–15 times per minute [21–24]. Because each of such frequent blinks engenders BPR with substantial magnitude (constriction about 0.1–0.3 mm on average) and length (at least 2–3 s), a significant fraction of raw pupillometry data is expected to be dominated by BPR. Secondly, these seemingly “spontaneous” blinks do not occur randomly. Instead, the blink rate and patterns are highly dependent on subjects, task structures, and cognitive factors [25–28]. For example, blinks tend to be suppressed during task epochs important for task performance, such as when a target is presented, and concentrated around implicit breakpoints of a task, such as right after a response is made [19,29–31]. Furthermore, the blink rate depends on the activity of the dopamine (DA) system [32–34] or the executive functions that are known to be modulated by the system [35–38]. These studies suggest that BPR is likely to interact with cognitive factors of researchers’ interest and the task or subject components of a given experimental design and that a substantial fraction of pupillary measurements is likely to be under the influence of such interactions.

Being motivated by these important implications of BPR on pupillary measurement, we set out to conduct a series of pupillometry experiments to understand how BPR confounds pupillary responses and figure out the proper way of handling BPR. Specifically, we proceeded as follows. First, by acquiring several BPR samples without asking subjects to perform any cognitive tasks, we learned about the general profile of BPR and how that profile varies across individual subjects and blinks. Second, by analyzing how the blink rate and pattern are associated with task structures and cognitive states, we learned how BPR interferes with the pupillary responses that are supposedly associated with the task components and cognitive variables of interest. Third, based on this learning, we built a statistical, generative model of how pupillary measurements are stochastically determined by the cognitive-state, spontaneous-state, and blink variables. Fourth, we developed a probabilistic-inference algorithm of correcting raw pupillary measurements for BPR with this generative model in hand. Lastly, we verified the effectiveness of the developed algorithm in selectively filtering out BPR and demonstrated that the algorithm could enhance the experiments’ statistical power by applying it to the data in the cognitive pupillometry experiments. Our studies suggest that BPR should not be treated as a mere nuisance or trivial phenomenon that can be averaged or deleted away, but rather as a serious confounder, something that potentially leads to misinterpretation or reduced statistical power of experimental results. Our correction algorithm illustrates one proper way of handling BPR for a wide range of cognitive experiments using pupillometry.

Results

The shape of blink-locked pupillary responses and its variability across luminance levels, subjects, and trials

The first goal of the current work is to describe the generic shape of BPR and to assess whether and, if so, how the shape of BPR varies depending on experimental factors, including background-luminance levels, subjects, and trials. To keep the other factors from influencing BPR, we measured the pupil size while subjects kept fixating at the center of the screen without performing any cognitive tasks at all (Fig 2A). To avoid any unwanted changes in cognitive state due to intentional suppression of natural blinks [39], we did not give subjects any instructions about eye blinking (e.g., “when they are, or are not, allowed to blink”; “how often they can blink”), except “blink naturally.” To define the shape of BPRs induced by sufficiently isolated single blink events, we only used the BPR samples associated with the blinks that were apart more than 3 s from their nearest blink (see Materials and Methods for details).

In line with previous work [15], the pupil size initially constricted right after an eye blink and returned to the baseline size (Fig 2B). When averaged across the luminance conditions, subjects, and trials, the BPR reached its peak constriction (−0.2 mm) at about 0.9 s after an eye blink and returned to its baseline size at about 3.0 s after an eye blink (Fig 2C). Given that the difference in pupil diameter by a manipulated variable in general pupillometry usually ranges from 0.05 to 0.5mm [40], pupil diameter change by BPR was considerable.

We found that the general shape of BPR, i.e., an initial fast constriction followed by a slow dilation back to the baseline size, remained unchanged, but its peak amplitude and time-to-the-peak substantially varied across different luminance levels, individual subjects, and individual blinks. The peak amplitude increased as the background luminance increased up to the second-highest level (5.35 cd m^-2) but somewhat decreased under the highest level (97.60 cd m^–2; Fig 2D). This result is consistent with the view that BPR reflects the pupillary response to transient changes in retinal illuminance [15,41]. The slight decrease under the highest level of luminance appears to reflect the fact that the baseline pupil size (mean ± SD across subjects, 3.13 ± 0.51 mm) was likely to be close to the minimum pupil size [42] so that BPR constriction was bounded.

Both the peak amplitude and the time-to-the-peak of BPR also varied substantially across individual subjects (Fig 2E). The shape of BPR appears idiosyncratic to each individual, as supported by the high degrees of split-half (even versus odd runs) correlation in both peak amplitude (Pearson correlation r > 0.9 under any backgrounds) and in time-to-the-peak (r > 0.7 under any backgrounds) (see S1 Fig for details).

Lastly, the blink-by-blink variability of peak amplitude was also substantial. To examine the blink-by-blink variability independent of luminance level and idiosyncrasy across subjects, BPRs were sorted into five quantiles by their peak amplitudes under balanced effects of luminance level and subject (see Materials and Methods for details). The peak amplitude of the highest quantile was 7.85 times greater than the peak amplitude of the lowest quantile (Fig 2F). It means that the peak amplitude of each blink varies considerably within the same background-luminance level and subject.

To summarize, single blinks instigate a three-second-long BPR in which the pupil quickly constricts and then slowly dilates back to its baseline size. The shape of BPR, especially the peak amplitude, varies substantially across luminance levels, subjects, and blinks.

BPR, not just a nuisance but a confounder

We argue that BPR not only makes pupil-size measurements noisy but also can confound them in various manners if eye blinks, the trigger of BPR, do not occur randomly, but their frequency is systematically dependent on task phases or experimental variables. In such cases, BPR can threaten the internal validity of pupillometry experiments. Previous work suggests such dependencies of eye blinks. In humans, eye blinking does not occur just reflexively such as when the eyes need to be lubricated [43]. Instead, the rate of spontaneous blinks is closely associated with cognitive factors [22]. For instance, eye blinking tends to be held during the task epochs important for information processing (e.g., target presentation) and then vigorously occurs during the intervals between those “important” epochs, which are often dubbed “implicit breakpoints” [19,29–31]. The rate of spontaneous blink also depends on the activity of the dopamine system in the brain [32–34] or on the executive functions and reward processes, which are modulated by the dopamine system [35–38].

To illustrate how BPR can confound the pupillary signal of interest, we measured pupil size while subjects performed two cognitive tasks, “auditory oddball detection” and “delayed orientation estimation” tasks. These two tasks were chosen because the cognitive processes underlying the tasks, i.e., ‘surprise by oddity’ and ‘working memory load’, are relatively well established, and those processes are known to be tightly associated with pupillary responses [5,44–47].

In the auditory oddball task, in which subjects had to discriminate the sounds of two tones every 2 s (Fig 3A; see Materials and Methods for details), the length of the trial (2 s) is shorter than the length of BPR (3 s). This implies that the impact of single blinks on pupillary responses is not confined to current trials but extended to subsequent trials, which complicate the time course of pupillary responses such that the application of conventional analyses to raw pupil data leads to serious misinterpretation. To illustrate such confounding effects, we simulated synthetic pupillary responses by assuming that each blink generates a canonical BPR, the average of isolated BPR samples collected under the second-highest level of the fixation task (Fig 2D), where retinal illuminance was most similar, and there is no change in the pupillary responses other than BPRs. Specifically, we simulated the time courses of pupillary responses in many pairs of two consecutive trials. In line with previous studies that showed suppression of blink rate during target stimulus presentation (or other cognitively important moments) and concentration of blink rate during implicit breakpoints [19,29–31], the blink rate was low during the stimulus presentation and increased rapidly as well, and slowly dwindled afterward (Fig 3B, top). When averaged over many simulated pairs of trials, the occurrences of blinks with that distribution resulted in the pupillary responses that (negatively) peaked at around the boundary between the first and second trials and returned to the baseline level at around the end of the second trial (Fig 3B, bottom). These simulation results imply that BPR will make pupillary responses deviate from the true pupillary signal of interest (the black dotted line in Fig 3B, bottom). Such deviations due to BPR, when combined with the procedure of setting the baseline of single-trial pupillary responses to the pupil diameter at the trial onset, which is conventionally practiced, will result in large spurious biases, decreasing and increasing time courses of pupillary responses in the blinking (first) and post-blinking (second) trials, respectively.

In the delayed orientation estimation task, we asked subjects to estimate the orientation of a post-cued target after a short delay while varying the number of bars to be remembered (Fig 3C; see Materials and Methods for details). Suppose the independent variable of interest is the number of bars to be remembered (i.e., visual working-memory loads). Intriguingly, we found that the actual rate of eye blinking systematically varied depending on the number of to-be-remembered bars during the estimation epoch: the blink rate tended to increase with an increasing number of bars (Fig 3D, top). When the synthetic pupillary responses were simulated as were done for the auditory oddball task data, the differences in the blink rate between the conditions resulted in the mirroring, spurious differences in pupillary responses: the negative peak amplitude of pupil size increased with an increasing number of bars (Fig 3D, bottom). These simulation results imply that BPR can confound the pupillary responses of interest—i.e., those associated with “visual working-memory load”—with the blink rate because BPR differentially biases the pupil size depending on the blink rate that happens to be associated with the independent variable of interest.

Put together, the simulation results show that BPR can readily confound the pupil responses of interest, i.e., those associated with a cognitive state of interest, whenever the blink rate changes depending on the cognitive state of interest. It is important to be aware of such confounded-by-BPR pupillary responses and treat them properly because they will be directly translated into the “signals of interest” by researchers who are unaware of the presence of BPR per se (Fig 3E, left), which severely threatens the internal validity of findings leading to either type I (false positive) or type II (false negative) errors in statistical inference. If researchers are aware of BPR but do not fully recognize the confounding nature of BPR (Fig 3E, right), they might simply treat BPR as a noisy nuisance by getting rid of the trials that are affected with blinks selectively. Unfortunately, this “selective screening” approach cannot “de-confound” the pupillary responses for BPR but instead may invite other confounders of a complicated nature because such a selection itself is confounded with eye blinks, which confounds a cognitive state of interest.

Model-based correction of pupillary responses for BPR

From the fixation task, we have learned about the shape and variability of BPR (Fig 2). From the changes in the blink rate during the auditory oddball task and delayed orientation estimation tasks, we have also learned about the confounding nature of BPR (Fig 3). Based on this discovery, we built a generative model of pupillary responses. According to this model (Fig 4A, left), a pupillary measurement is a random variable that has three random variables as parents, namely, the “cognitive-state,” “spontaneous-state,” and “blink” variables. Based on what was discovered from the fixation task, we approximated the causal function from the blink variable to the pupil size (i.e., BPR) with an inverted gamma probability density function, the parameters of which are unique to a given individual and the amplitude of which varies across blinks. Importantly, we incorporated the confounding nature of BPR into the generative model by positing that the cognitive state has another indirect causal route to the pupil size through the blink variable.

Fig 4 — (A) According to the realistic model, observed pupil size is an amalgam of response to cognitive state, BPR, and spontaneous fluctuation from other cognitive-independent sources. The goal is to decompose the pupil size as the sum of the components with a generative model and eliminate BPR specifically based on the estimated BPR component, making the data free from BPR confound. The algorithm estimates the BPR shape for each subject and BPR amplitude for each blink (e.g., sth subject’s jth blink is estimated here) (see Materials and Methods for the detailed description of the algorithm). (B) An example of BPR correction results. Note that the correction algorithm estimates the BPR amplitude for each blink (denoted as numbers above time course) based on the generative model. (C) Comparison between blink-locked time course from pre-BPR correction (black), post-BPR correction (green), and blink-free period (blue). Note that the characteristic short-pipe shape of BPR before correction is flattened and resembles a blink-free period after being corrected for BPR. Line and shade indicate the mean and standard errors of the mean (SEM), respectively. (D) Euclidean distance between the mean blink-locked time course and the mean blink-free time course in each subject. The distance decreases significantly after BPR correction (*p <* 0.001; ***).

Given this generative model of pupil-size measurements, the normative way of de-confounding pupil measurements must block the backdoor route from the cognitive state to pupil measurement via the blink variable. Blocking this back door means inferring “counterfactually” what pupil measurements would be like if blinks did not occur. This counterfactual inference can be achieved only when we can estimate the subject-specific shape and the blink-specific amplitude of BPR and then take out that specific BPR selectively from the raw measurements of the pupil size (Fig 4A, right). Such selective filtering will leave the causal influences from both the cognitive state and the spontaneous state intact in pupil measurements.

We stress that our intended purpose of correction is not to infer the cognitive state per se but to get rid of the influence of blink events selectively on pupil measurements so that the corrected pupil measurements still reflect the influences both from the cognitive state and from the spontaneous state. We intended so not simply because it is difficult to distinguish the influence from the spontaneous state from the cognitive state but rather because the spontaneous state itself can be of interest to some researchers (e.g., those who want to know the impact of the spontaneous state on pupil measurements).

The critical part of our correction method is to accurately estimate the subject-specific shape parameters and the blink-specific amplitude parameter of BPR (BPR_js in Fig 4A). By incorporating what we learned about BPR from the fixation task into the generative model as prior knowledge, we developed an algorithm inferring subject-specific BPR profile and blink-by-blink BPR amplitude by combining that prior knowledge and the likelihood function acquired from the observed pupil measurements (see Materials and Methods for the detailed description of the algorithm). Given the substantial blink-to-blink variability of BPR amplitudes (Fig 2F), we stress the importance of estimating the amplitude of BPR on a blink-to-blink basis. In this regard, our algorithm is distinguished from and goes beyond the previously proposed correction method that assumed the constant amplitude of BPR [17].

As an initial step of validating our correction algorithm, we applied it to the data acquired from the fixation task (Fig 4B). One important merit of this data set is the fact that an effective ground-truth time course of pupillary responses, which are free from BPR, can be approximated by the average of the sufficiently long periods during which blinks did not occur (see Materials and Methods for the detailed description of blink-free time courses). It is important to compare the blink-affected time course of pupillary responses against this ground-truth time course before and after, respectively, the correction for BPR because it allows us to verify whether our correction algorithm selectively removes the pupillary responses associated with BPR but not those associated with other components, including cognitive and spontaneous fluctuations. In this regard, note that the blink-free, ground-truth time course of pupillary responses might not be necessarily flat because there could be many uncontrolled factors that might affect the pupillary responses (e.g., gradual decrement or increment in pupil size over time due to fatigue or arousal). In such cases, the simple comparison of the pupillary responses before and after the application of BPR correction does not provide sufficient information about whether the algorithm successfully corrected the pupillary responses only for BPR. If the algorithm succeeds in selectively removing BPR from the raw pupil-size measurements, the time course of the corrected measurements must be close to the ground-truth time course. This prediction was confirmed: the original average profile of the pupillary responses with blinks showed the typical shape and amplitude of BPR before correction (gray line in Fig 4C) but became close to the average profile of the pupillary responses without blinks after the correction (green and blue lines in Fig 4C). Statistical tests showed that the correction significantly reduced the distance between the profile of the pupillary responses with blinks and that without blinks (Wilcoxon signed-rank test p < 0.001) (Fig 4D).

Correcting the auditory oddball data for BPR

As the second step of verifying the effectiveness of the correction algorithm, we applied it to the data acquired in the auditory oddball task. Unlike the fixation task, the auditory oddball task was designed to test a particular prediction of a specific cognitive hypothesis. For example, in line with previous studies [44,45], suppose an experimenter predicts that the pupil size will increase as the oddity increases according to the hypothesis that the oddity is associated with the degree of a surprise based on the strong relationship between pupil size and degree of surprise. To describe in terms of the generative model (Fig 3F), this means that the “cognitive-state” variable, i.e., “oddity,” is manipulated by the experimenter and expected to influence the pupillary measurement variable in the auditory oddball task. Thus, unlike the fixation task in which no specific cognitive state was manipulated, the oddball task offers an opportunity to further verify the correction algorithm by testing whether our correction algorithm can selectively remove BPR in the presence of the influence of a cognitive state on pupillary measurements.

Before testing the algorithm’s effectiveness, we first validated the generative model by testing how successfully it can predict the time course of raw pupillary responses in the presence of both cognitive and BPR influences on pupillary measurements. To do so, we simulated the time series of pupillary responses (as depicted by the black line in Fig 5A) by positing that a beep sound and an ensuing manual motor response together constantly generate an increasing and decreasing profile of pupillary responses (as depicted by the blue line in Fig 5A) and, in parallel, that an event of eye blinking also constantly generates the canonical BPR (as depicted by the red line in Fig 5A). The procedure for this simulation was identical to that for the one depicted in Fig 3B, except that the constant pupillary response associated with the sound and the manual motor response is linearly added (see Materials and Methods for details). As was done previously, to illustrate the varying impacts of BPR on the data acquired in the auditory oddball task, we contrasted the averaged, 2-second-long profile of simulated pupillary responses for two cases: (i) in which blinking occurred on the current trial but neither previous nor following trials (left panel in Fig 5B); (ii) in which blinking occurred on the immediately preceding trial but not on the current trial nor 2, 3 back trials (middle panel in Fig 5B). Additionally, to illustrate the overall impact of BPR, we also plotted the grand average of simulated pupillary responses (right panel in Fig 5B). For the first case, the generative model predicts that the profile of raw pupillary responses initially follows that of the sound-locked responses (signal) and then rapidly drops due to the constriction caused by BPR. By contrast, for the second case, the profile of raw pupillary responses is predicted to increase more rapidly than that of the sound-locked reactions and continues to increase due to the dilation caused by BPR, while that of the sound-locked responses decreases. Note that this exaggerated profile is the spurious outcome of the practice of adapting the baseline to trial onset (as indicated by the cross point between the dotted vertical blue line and black line in Fig 5A) in conjunction with the “return-to-baseline” dynamics of BPR that was triggered by the blink in the preceding trial (red solid line in Fig 5A). For the third case, the generative model predicts that BPR makes even the all-trial averaged profile of the raw pupillary responses substantially deviate from that of the sound-locked responses. The averaged profile of the raw responses is predicted to be initially greater and then smaller than that of the sound-locked responses. These predictions well matched the observed profiles for all three cases (as indicated by the sound correspondences between the simulated black lines in Fig 5B and the observed ones in the left panel of Fig 5C).

Fig 5 — (A) Schematic example addressing how BPR confounds pupillometry in the auditory oddball task. The pupil dilates by an auditory tone, which is the signal experimenters are interested in (blue). However, subjects spontaneously blink during the experiment, and it evokes BPR (red). Subjects tend to frequently blink about 500 ms after the stimulus (shown in Fig 3B). Therefore, BPR would distort the pupil response to the stimulus by decreasing it during the trial in which blink occurred and by increasing it during the subsequent trial. Linear interpolation of 3 s after a blink (orange) slightly reduced the pupil response in this case. (B) Simulation of pupil time course shape by BPR. BPR would induce significant and spurious distortion in shape. To identify the spurious distortion pattern by BPR clearly, two cases were selected. The cases where a blink occurred at the current trial (left panel) or right before the trial (middle panel) are shown (see Materials and Methods for details). The observed pupil time course (black) would be the sum of the pupil response to cognitive state (blue) and BPR (red). Note that it has increased and decreased by BPR (red) as shown in A. (right panel) In all trials, the increasing and decreasing effects are combined and happened such that predicted mean profile of pupil-size measurements (blue) so somewhat exaggerated the peak of the pupil dilation. (C) The spurious distortion in shape was identified as anticipated and fixed by the correction method. (Left panel) Pre-correction pupil time courses (black). When a blink occurred, the spurious decrease in the blink-occurred trials (dash-dotted line), the spurious increase in the post-blink trials (dashed line), and the spurious increase for the first 1 s and the following decrease in all trials (solid line) were observed as anticipated by the generative model. (Middle panel) Most of these spurious distortions were removed or attenuated after applying the model-based BPR correction method (green). (Right panel) The spurious distortions were over-corrected after applying the 3-s long linear interpolation correction method such that the all-trial average profiles of pupil-size measurements was substantially reduced (orange). Line and shade indicate the mean and standard errors of the mean (SEM) across subjects, respectively.

Having confirmed the predictions of the generative model upon which our correction algorithm is built, we turn to check whether our model-based counterfactual-inference method selectively filters out the biases due to BPR in the raw pupillary responses. If so, the corrected profiles of pupillary responses in the three cases must become much close to one another. The close matches between the observed profiles of the three cases (middle panel of Fig 5C) indicate that the biases due to BPR were successfully corrected. We stress that this selective removal of BPR cannot be accomplished simply by deleting the raw pupil-size data under the influence of BPR and linearly interpolating those deleted data (as indicated by the orange line in Fig 5A). The linear interpolation method appeared to reduce the biases due to BPR to some degrees but invited the systematic errors opposite to those observed in the uncorrected data: the amplitude of the interpolated profile was increased and decreased for the first and second cases, respectively, compared to the third case (Fig 5C, right). This pattern can readily be expected from the spurious effects due to baseline setting (the orange line in Fig 5A).

As another way of verifying the effectiveness of the proposed algorithm, we sorted trials into those that were affected by blinks (“blink-affected” trials) and those that were free from blinks (“blink-free” trials), and compared the two types of trials in pupil-size time course before and after the correction algorithm was applied, as we have done in the fixation experiment. The rationale and prediction of this comparison analysis are the same as those stated in the fixation experiment: If the algorithm succeeds in selectively removing BPR from the raw pupil-size measurements, the time course of the corrected measurements must be close to the time course of blink-free, which is a proxy of ground-truth signal, after BPR correction. Considering that BPR lasts for about 3 s, the blink-free trials were defined as the trials whereby blinks occurred neither in the 2-back trial, in the 1-back trial, nor in the current trial while the blink-affected trials were all the rest of the trials (see Methods and Materials for details). Using the time course of pupil-size changes in the blink-free trials as effective ground truth of pupillary responses to the cognitive event of interest, we examined: (1) how much the time course of pupil-size measurements in the blink-affected trials deviates from that in the blink-free trials; (2) whether, and how effectively if so, the correction algorithm reduces such deviations. We found that the time course of pupil size in the blink-free trials (gray lines in Fig 6A) exhibits double peaks. The first and second peaks can be interpreted to reflect the sound stimulus and the following manual response. This interpretation is consistent with previous reports where similar double-peak pupillary responses were reliably observed in the auditory oddball tasks in which subjects made a manual response on each and every trial like in our study [44,48]. Before the correction, the time course of pupil size was significantly exaggerated in the blink-affected trials, particularly around the peak (left panels in Fig 6A and 6B). Some might think that this exaggerated increase is weird because BPR appears to ‘increase’ the pupil size. However, this increase can be anticipated because BPR lasts longer than a single trial, and the time course of pupil size is baselined. In the trials in which blinks occurred in the current trial, the effect of BPR appears as a dip. The effect usually affects only the second half part (1–2 s) of the trial (see 0–2 sec of the bottom panel of Fig 3B) since blinks are concentrated at 0.5–1 s and (as described in the top panel of Fig 3B) and BPR has some delay (0.3 s) before a substantial decrease. In the trials in which blinks occurred in the 1-back trial, the BPR from the 1-back trial typically reaches its negative peak around the onset of the current trial, so the baseline is spuriously decreased. As the pupil diameter at stimulus onset is set as the baseline, BPR in the recovery (dilation) phase appears as a spurious increase of the first half part (0–1 s) of the trial (see 2–4 sec of the bottom panel of Fig 3B). In the trials in which blinks occurred in the 2-back trial, the BPR effect is similar to that in the 1-back trial case, but with a smaller effect size since BPR from the 2-back trial substantially recovered prior to the current trial. The combination of the decrease in the last half part (from the trials in which blinks occurred in the current trial) and the increase in the first half part (from the trials in which blinks occurred in the 1-back and 2-back trial) eventually made spuriously exaggerated dilation of the baselined time course in this task. We stress that these spurious patterns were exactly those that were predicted by the simulation based on our generative model (Fig 5A and 5B), and the observed patterns (Fig 5C) confirmed this anticipation. The thick black line in Fig 6A, which is the same as the thick black line in the left panel of Fig 5C, is the observed time course of the “blink-affected” trials, which is quite similar to that predicted by the model simulation (the right panel of Fig 5B) and the mean of the spurious patterns (dashed and dash-dotted lines in the left panel of Fig 5C). This deviation became substantially reduced after being corrected by the model-based method (middle panels in Fig 6A and 6B). Specifically, the corrected time course in the blink-affected trials was similar to the time course in the blink-free trials in shape, exhibiting double peaks. However, in amplitude, the corrected time course in the blink-affected trials was slightly smaller than that in the blink-free trials. This slight deviation may be taken as a limitation of the correction method. However, alternatively, it may reflect the genuine difference between the blink-affected trials and the blink-free trials, which is a matter of empirical investigation. By contrast, when the interpolation-based algorithm was applied, the corrected time course in the blink-affected trials was significantly below the time course in the blink-free trials (right panels in Fig 6A and 6B). As demonstrated in Fig 5, this understated pattern can readily be explained by the limitation of the interpolation approach, which not only removes BPR but also throws out the genuine pupillary responses associated with the cognitive factors of interest (i.e., the sound and the manual response to it).

Fig 6 — (A) The time courses of pupil size before (left) and after correction (middle and right). The gray lines represent the mean time course of pupil-size measurements in the blink-free trials. The black line is the mean time course of pupil size in the blink-affected trials before correction. The green and orange lines represent the mean time courses of pupil size after correction with the model-based method and the linear interpolation method. Shades represent standard errors of the mean (SEM) across subjects. (B) The time courses of t statistics (“blink-affected”–“blink-free,” df = 23) of the difference between the blink-affected and the blink-free trials. Horizontal dashed lines indicate the values of t statistics where the significance level is 0.01, and t = 0.

Lastly, we evaluated the utility of the correction algorithm by assessing the extent to which it contributes to the statistical power of revealing the differences in pupillary responses between the experimental conditions of interest, i.e., oddity levels in the auditory oddball detection task. This evaluation is important and cannot be guaranteed on the basis of the above test, which only demonstrated the algorithm’s effectiveness in removing the mean bias due to BPR in specific cases. The statistical power, in principle, can be improved or deteriorated not just due to the mean difference between conditions but also due to the trial-to-trial variability. In this regard, it should be reminded that the key feature of our correction algorithm is to estimate the shape and amplitude of BPR on the subject-to-subject and blink-to-blink bases, respectively. Thus, if this estimation is successfully carried out, the statistical power must be improved. As an initial step of testing such improvement in statistical power, we examined how much the model-based method enhanced the statistical differences between the levels of the independent variable, i.e., the oddity of sound frequency, over the trial-locked time course of pupil size. As anticipated, the pupil dilation increased as a function of oddity even before correction (left panels in Fig 7A). However, after being corrected for BPR with the model-based method, the differences between the oddity levels substantially increased, especially in the later part of the time course (middle panels in Fig 7A). By contrast, the interpolation method substantially decreased the statistical differences between the oddity levels (right panels in Fig 7A). Next, we assessed the contributions of the correction algorithm to the changes in the mean difference and standard errors of the mean (SEM) across trials, respectively. We found that the correction algorithm did not affect the mean difference (green symbols in Fig 7B) but robustly decreased the SEM (green symbols in Fig 7C). As a result, the correction algorithm significantly improved the statistical power when the power was assessed both by the area-under-the-curve analysis (green symbols in Fig 7D; see Materials and Methods for details) and by the simulation analysis in which the minimum number of trials required to reject the null hypothesis was calculated (green line in Fig 7E; see Materials and Methods for details). In contrast with our correction algorithm, the interpolation method ended up decreasing the statistical power (orange symbols in Fig 7D; orange line in Fig 7E) by substantially reducing the mean differences (orange symbols in Fig 7B).

Fig 7 — (A) (top panels) Time courses of pupil size for different levels of sound-frequency oddity. Lines and shades correspond to the means and SEMs across subjects. (bottom panels) Time courses of t statistics (paired t-tests, df = 23) for the deviations of the high oddity (magenta) and low oddity (cyan) conditions from the medium oddity (blue) conditions. The left panel shows the results before correction; the middle and right panels show the results after being corrected using the model-based method and the interpolation method, respectively. (**B-D**) (left panels) Diagonal plot. Each marker indicates each condition pair, a subject has three markers each (high vs medium, medium vs low, and high vs low). Filled marker and error bars indicate the mean and SEM across subjects. Empty markers correspond to individual subjects, with different colors indicating different correction methods (green for the model-based method; orange for the interpolation method) and different symbols indicating different oddity pairs (see inset for detailed labels). (right panels) Boxplot. Corrected–uncorrected, which are equivalents to deviance from the diagonal line in the corresponding left panels. (Wilcoxon signed-rank test, (*, *p <* 0.05; **, *p <* 0.01; ***, *p <* 0.001)). The color scheme matched that shown in the left panels. (B) Mean difference in pupil dilation, (C) pooled SEM, (D) AUC between the conditions. (E) Comparison of the data uncorrected (black line), the data corrected with the model-based method (green), and the data corrected with the linear interpolation method (orange) in statistical power. The fraction of significant tests was plotted against the number of trials (see Methods and Materials for the detailed procedure).

Correcting delayed orientation estimation data for BPR

In line with previous work [19,29–31], the mean rate of eye blinking significantly fluctuated depending on the task epochs in the delayed orientation estimation task (black dotted line in Fig 8; see Materials and Methods for details about the task). The blink rate increased within each of the four task epochs, reaching the global maximum in the preparation epoch, a local maximum right after the bar presentation, another local maximum right after masking stimulus presentation, and another at the start of the estimation epoch. The visual comparison of the average profile of the corrected pupillary responses (green line in Fig 8) with that of the pre-correction responses (black line in Fig 8) suggests that the correction algorithm selectively carved out BPR from the pre-correction pupillary responses in two aspects. First, the pupil size increased overall after correction, reflecting the baseline level of blink rate. Second, the first local dimple (bottom red arrow in Fig 8) in the preparation epoch (top red arrow in Fig 8), but not the second local dimple in the masking epoch, was removed after correction, which makes sense that the latter dimple is likely to reflect the pupil constriction due to the sudden increase of retinal illuminance at the beginning of the masking epoch.

Fig 8 — Pupil (solid line) and blink rate (dotted line) time course during the delayed orientation estimation task. Since BPR induces pupillary constriction, the pupil-size time courses corrected by the model-based method (green) and the linear interpolation method (orange) were larger than pre-correction (black) overall, but note that the linear interpolation method decreased the time course when it was a hill (upper blue arrow). Note that subjects tend to blink more often at the preparation epoch, and it induces a spurious dip at stimulus onset (red arrows). It was flattened after applying both of the correction methods. Line and shade indicate the mean and SEM, respectively.

Overall, the pupil began to dilate substantially right after stimulus offset and became maximal in amplitude during the estimation period (top blue arrow in Fig 8) regardless of whether the pupil data were corrected or not. This is anticipated from previous work on working memory, in which the pupil size increased as memoranda of stimuli need to be retrieved [5]. Additionally, the peak (as indicated by the vertical dashed lines in top panels of Fig 9A) increased as a function of the working-memory load (the number of bars to be remembered; Fig 9A, top). Notably, the blink rate also increased as the working-memory load increased, as shown before (the top panel of Fig 3D, which is redrawn in the top panels of Fig 9A). This implies that the blink rate, via BPR, works as a serious confounder that counteracts the pupil-size changes associated with the cognitive state of interest, i.e., working-memory load. After being corrected for BPR with the model-based method, the differences between the working-memory load conditions increased (middle top panel of Fig 9A), resulting in increased statistical differences between the conditions (middle bottom panel of Fig 9A). By contrast, the interpolation method rather substantially decreased the mean differences (right top panel of Fig 9A) and the statistical differences (right bottom panel of Fig 9A). These results showcase the effectiveness of our model-based method in selectively removing BPR while keeping the pupillary responses of interest intact.

Fig 9 — (A) (top panels) Pupil (solid line) and blink rate (dotted line) time course across the working-memory load conditions. The time 0 is the onset of the mask epoch. The pupil size increased as a function of working-memory load. The color indicates the number of bars to be memorized (N = 1,2,4 and 8 are denoted as black, blue, magenta, and red lines, respectively). Solid lines indicate the means across subjects, and the vertical dotted lines indicate the time span where the mean pupil of each trial is computed. Note that the more memory load is given, the higher the blink rate was, and thereby BPR would be more biased to the higher memory load conditions. The model-based method captured this bias so that the pupil-size time course under high memory loads increased more than low memory loads after the correction. (bottom panels) T values of the paired t-tests (df = 20) for the difference of each bar condition from the 2-bar condition. The left panel shows the results before correction; the middle and right panels show the results after being corrected using the model-based method and the interpolation method, respectively. (**B-D**) (left panels) Diagonal plot. Each marker indicates each condition pair, a subject has three markers each (1 bar vs 2 bars, 4 bars vs 2 bars, and 8 bars vs 2 bars). Filled marker and error bars indicate the mean and SEM across subjects. Empty markers correspond to individual subjects, with different colors indicating different correction methods (green for the model-based method; orange for the interpolation method) and different symbols indicating different working-memory load (the number of bars) pairs (see inset for detailed labels). (right panels) Boxplot. Corrected–uncorrected, which are equivalents to deviance from the diagonal line in the corresponding left panels (Wilcoxon signed-rank test, (*, *p <* 0.05; **, *p <* 0.01; ***, *p <* 0.001)). The color scheme matched that shown in the left panels. (B) Mean difference in pupil dilation, (C) pooled SEM, (D) AUC between the conditions. Panel B and C were zoomed in for clear visualization. See S2 Fig for zoomed-out plots. (E) Effects of the correction methods on statistical power in the delayed orientation estimation task. The number of trials required to reject the null hypothesis decreased by about 40% by the model-based method, whereas it increased 100% by the linear interpolation method.

Using the same procedure used for the auditory oddball task, we assessed the extent to which the correction algorithms contribute to the statistical power of revealing the differences in pupillary responses between the working-memory load conditions. We obtained the results that were similar to those of the auditory oddball task data: The discriminability between the working-memory load conditions across subjects increased after the BPR correction by the model-based method, whereas it is decreased after the correction by the interpolation method (Fig 9A bottom panels). Additionally, the model-based method enhanced the mean differences (green symbols in Fig 9B) and decreased the SEMs (green symbols in Fig 9C). As a result, the statistical power was significantly improved after correction (green symbols in Fig 9D; green line in Fig 9E). By contrast, the interpolation method decreased the statistical power (orange symbols in Fig 9D; orange line in Fig 9E) by substantially reducing the mean differences (orange symbols in Fig 9B).

Discussion

Summary of findings

By inspecting the pupillary response profiles that were associated with isolated single events of eye blinking, we learned that pupil size reacts to a single blink similar to how it responds to an abrupt change in retinal illuminance, which we dubbed BPR. We also learned that BPR varies substantially, mainly in amplitude and slightly in phase, across display luminance levels, subjects, and blinks. By inspecting how the blink rate changes as a function of task epoch or as a function of cognitive states, we also learned that BPR is not a mere nuisance but should be treated as a serious confounder that threatens the internal validity of experiments. Upon these empirical findings and understandings, we built a generative model in which a given pupillary measurement is the stochastic outcome of the linear summation of the cognitive-state, spontaneous-state, and blink variables, while the blink variable may confound pupillary measurements via the backdoor route involving BPR. With this generative model in hand, we argued that pupillary measurements should be corrected for BPR by counterfactually inferring BPR on a subject-to-subject and blink-to-blink basis, and developed an algorithm of doing such counterfactual inference. By analyzing the three data sets that were acquired using the fixation, auditory oddball, and delayed orientation estimation tasks, respectively, we showed that the generative model can explain away the seemingly peculiar event-locked profiles of the raw pupillary responses with the confounding acts of BPR. Lastly, we demonstrated that our newly proposed algorithm could effectively and selectively remove BPR and significantly increase the statistical power of revealing the pupillary-response differences between the cognitive states of interest. Our findings call for the attention of researchers who analyze pupillometry data to the presence of BPR, the serious nature of its confounding acts, and the proper way of addressing it.

Potential causes of BPR

As mentioned briefly in INTRODUCTION, there are many reasons to believe that BPR is mainly caused by the transient changes in retinal input that occur as the eyelid briefly blocks the light pass through the pupil. First, previous work reported that “dark flashes”–turning off and on the light for a short period approximately matching the blink duration–under constant illumination make the pupil constrict and re-dilate with the dynamics close to BPR [15,41]. Specifically, successful description of the pupillary responses to dark flashes with a linear time-invariant system [41] suggests that the short-pipe shape of BPR dynamics can be accounted for by a linear mixture of slow and weak dynamics of dilation associated with the abrupt decrease in illuminance due to eye closing and a fast and strong dynamic of constriction associated with the abrupt increase in illuminance due to eye-opening. Second, saccadic eye movements also trigger the dynamics of pupil size that are similar to BPR [16,17,49,50]. Thus, pronounced pupil constriction followed by saccadic eye movements is similar to the pupillary responses to abrupt changes in visual input, such as a checkerboard undergoing polarity inversion. However, we also note that motor activity during blinks may also contribute to BPR, especially to the slow dilation component of BPR because a motor activity is known to promote pupil dilation [51].

The shape and variability of BPR

As suggested by a few previous studies [15,41], the general profile of BPR identified in the current work resembled the known pupillary responses to a dark flash, which is probably caused by the abrupt change in retinal illuminance due to the eyelids’ act of blocking the light entering the pupil. Furthermore, although a previous [17] study characterized the generic profile of BPR as a biphasic shape with a rebounding tail using the finite impulse response method, the rebound was largely absent or negligible in our data. Considering also that our correction algorithm, in which BPR was modeled as a single inverted gamma probability density function, did not exhibit any noticeable systematic bias, we think that a “short-pipe” shape can sufficiently describe BPR without a rebounding tail.

One novel contribution of the current work to understanding the properties of BPR is that the amplitude of BPR markedly varies across individual instances of eye blinking. For example, on average, the amplitude of BPR of the top 20 percentile was 7.85 times greater than that of the bottom 20 percentile. Although we have not specified the origin of this large variability of BPR amplitude, our studies strongly suggest that any correction methods based on the assumption of BPR with a constant amplitude are likely to leave substantial amounts of unwanted variability in the supposedly corrected pupillary measurements. This unaddressed variability may decrease the statistical power of the experiment and impose severe limits on any investigation of pupillary responses on a trial-to-trial basis, which provides crucial information about cognitive systems. In this regard, our correction algorithm can be considered one effective way of addressing the blink-by-blink variability of BPR amplitude.

Confounding acts of BPR

One of the most important contributions of the current work to the field of pupillometry studies on cognitive processing is our demonstration that BPR might act as a serious confounder. We demonstrated two concrete examples of such confounding acts of BPR. When the trial length is shorter than the length of BPR, as in the auditory oddball task, the overarching influence of BPR, in conjunction with the practice of adapting the baseline to trial onset, can lead to spurious yet great amounts of distortion depending on when blinking events occur in relative to the current trial. We also demonstrated that when the cognitive state of interest is associated with the blink rate, as in the delayed orientation estimation task, the influence of the cognitive state on pupillary measurements is also distorted depending on the nature of such association. Considering that these two situations, i.e., short-event-related design and condition-dependent blink-rate modulation, are not rare cases in cognitive experiments [38,52], we argue that BPR should not be treated lightly as a mere nuisance but rather seriously as a confounder that would threaten the internal validity or statistical power of experiments.

Another important contribution of the current work is that we built a generative model of pupillary measurements. We stress that this generative model was built upon the knowledge, which was empirically earned from a set of controlled observations designed to characterize the key properties of BPR, such as its shape, variability, and confounding nature. By incorporating this empirical knowledge into the generative model in which the major variables shape a causal network of random variables, we conceptualized BPR as a backdoor route through which a confounding flow of information propagates [53]. One of many virtues of having a generative model is to allow for conducting ex-ante simulations, which allow for validating certain assumptions upon which the model is built. We exercised that virtue when validating the key assumption of the generative model regarding the aforementioned backdoor route (Fig 3) and when explaining away the seemingly peculiar profiles of pupillary measurements encountered in the observed data (Figs 5 and 8). To be sure, we do not claim that the generative model presented in the current work sufficiently includes the variables involved in pupillary measurements (e.g., a few obvious factors such as the light variable were omitted) nor that all assumptions have been empirically grounded (e.g., the linear summation of the outputs from the three variables or Independence of BPR from each blink). Our generative model should be considered a good approximation of how BPR relates to cognitive states, spontaneous state, and pupillary measurements. More future studies are required to validate further and extend this model.

The proper way of correcting pupil-size measurements for BPR

We stress that our algorithm of correcting pupil-size measurements for BPR is a normative solution derived from the generative model. Given the causal structure of the variables in the generative model, the proper solution of correcting pupil-size measurements for BPR must be to selectively block the backdoor route from the cognitive state to the pupil size via the blink variable. Put in terms of the Bayesian network, the proposed solution can be understood as the strategy of “counterfactually” fixing the blink variable at the “off” state although the observed (actual) state of the blink variable is “on.” Such fixing requires the correct specification of the causal function from the blink variable to pupil-size measurements, BPR. From the fixation task, we learned the three crucial features of this causal function: (i) in general, BPR has a 3-s-long temporal profile of fast constriction followed by slow re-dilation; (ii) individual subjects have their own unique, and fixed shape of BPR; (iii) for a given individual, the amplitude of BPR stochastically varies on a blink-to-blink basis. This means that the shape and amplitude of BPR must be “probabilistically inferred” from the observed pupil-size measurements and the prior knowledge about BPR. Consistent with this implication, we showed that the pupil-size measurements could not be corrected for BPR with conventional approaches such as averaging a substantially large number of trials or replacing the blink-affected epoch of measurements with an interpolated line. One may wonder whether this blink-to-blink probabilistic inference of the shape and amplitude of BPR can be approximated by applying interpolation-based methods, including sophisticated versions that perform correction in a blink-to-blink manner such as “cili” [54]. However, such interpolation-based methods are limited because the interpolation procedure not just removes BPR but also throws away the genuine pupil-size changes of interest, i.e., pupillary responses to certain cognitive factors. Because of this limitation, the interpolation-based correction of BPR might end up decreasing the statistical power of experiments, as we demonstrated in our results.

To our best knowledge, there has been only one previous work that attempts to correct pupil-size measurements for BPR by estimating the shape and amplitude of BPR [17]. However, this approach differs from ours in two crucial aspects. First, it is assumed that the shape of BPR might be unique to a given individual, as we did, but the amplitude of BPR was fixed at a constant value. Secondly, it estimated the shape of BPR using the finite impulse response method, which requires the design matrix of events because the pupillary measurements were modeled as the sum of pupillary responses to events. As mentioned above, the assumption of the fixed amplitude of BPR will lead to substantial inflation of blink-by-blink variability, given the high degree of variability of BPR amplitude found in the current work. In addition, unlike this approach, our correction algorithm does not require any design matrix of events and thus can be applied to a wide range of experimental designs.

Although our correction algorithm is grounded in the empirically constrained generative model, effective in selectively filtering out BPR, and contributes to the statistical power, a few of its limitations should be mentioned. First, our algorithm may require a large volume of data that is sufficient to build a reliable probability distribution because it estimates the amplitude of BPR on a blink-to-blink basis based on the probability distribution. We think that the volume of data can be compromised by assuming the constant-amplitude BPR in specific experimental designs, whereby the trial-to-trial variability is not important, or by using the canonical shape of BPR when individual differences do not matter much. Second, in principle, it would be difficult to distinguish between BPR and the pupillary responses associated with the cognitive factor of interest when the latter’s shape and timing are close to those of BPR. However, we stress that it is very rare for the pupil time course of a certain cognitive event to match BPR exactly in timing and shape. Lastly, our algorithm relies on the fitting procedure that is rather slow, computationally costly, and still imprecise, especially when blinks occur too frequently or in a burst-like manner. However, these problems are not specific to our algorithm but rather general to all model-based correction methods.

To help readers readily apply the BRP-correction algorithm developed in the current work to their own data, we provided it as an out-of-the-box solution. This toolbox is coded in MATLAB and requires little manual editing to work: all users have to do is feed in time series of pupil-size measurements and blink data as inputs to the toolbox. The MATLAB codes, prerequisite data-preprocessing steps, and a brief description of the correction procedure with an example data set can be downloaded at the following link: https://github.com/yookyung1310/BPR_toolbox.

Practical considerations for avoiding BPR-related problems

Given the prevalence of spontaneous blinks and the confounding nature of BPR, we considered a few points that pupillometry experimenters should keep in mind at the various stages of conducting experiments.

At the experimental design stage, consider making the inter-trial interval sufficiently long, at least more than 3 s. This recommendation is worth being considered even when blinks do not occur because the pupil size needs some time to return to its baseline size. However, the presence of BPR even further recommends researchers to make the inter-trial interval long, especially given that subjects tend to blink a lot at implicit breakpoints such as the moment right after making responses. Thus, the separation of consecutive trials by more than 3 s will tend to minimize–although it will not prevent completely–the influences of BPR on raw pupil-size measurements.

At the data acquisition stage, it is recommended to record a whole time series of eye-tracking data over an entire run. Suppose data are acquired separately for individual trials and baselined to the pupil-size measurement at trial onset. In that case, there is no way to correct these baselined time courses of measurements for BPR. Alternatively, even when data are collected in a trial-by-trial manner, we recommend experimenters to keep the time series of at least 3 s before the trial onset so that BPR can be delineated before adapting the data to a baseline level. Note that this practice is worth being considered also because it allows for measuring the pupil size before any experimental manipulation of interest.

At the stage of giving subjects instructions regarding task performance, we recommend that experimenters consider not asking subjects to suppress spontaneous blinks for several reasons, despite the existence of BPR. First, the blink rate is highly informative, known as a correlate of many important neuro-cognitive states, including reward, attention, and dopamine activity [32–38]. Intentional suppression of blinks would distort such valuable information. Second, the instruction of blink suppression will aggravate eye fatigue from eye-tracking experiments. Third, in a typical video-based pupillometry setup, being a subject is already quite demanding: the eyes being exposed to the infrared light; the gaze direction is fixed at the fixation target; the body posture is maintained at a specific position which is best for data collection for experimenters but might be uncomfortable for subjects. Thus, adding another request of suppressing blinks on top of an already straining set of requests will likely make subjects weary and lose focus on task performance. Lastly, an attempt to suppress blinks for a prolonged period may induce unwanted cognitive and neural responses such as the accumulation of a natural bodily urge [39].

At the stage of data preprocessing, we do not recommend experimenters discard blink-affected trials. The selective removal of blink-affected trials reduces the total number of trials for data analysis and is likely to invite another confounding factor, especially when the blink rate is tightly linked with a cognitive state of interest. For example, suppose the true time course of pupil size differs between blink-affected and blink-free trials, possibly due to the difference in dopamine activity or working-memory load, for instance. In such cases, deleting only the blink-affected trials would bias results such that the underlying difference between blink-affected and blink-free trials is not taken into account in the observed differences between experimental conditions.

Materials and methods

Subjects

Paid 28 (21 females, aged 23.7 ± 3.3), 27 (19 females, 25.6±3.6), and 22 (12 females, aged 26.4±3.3) human subjects participated in the fixation, auditory oddball, and delayed orientation estimation tasks, respectively. Thirteen subjects participated in more than one experiment: 11 in all the three experiments; 2 in the fixation and auditory oddball experiments. All had normal or corrected-to-normal vision with contact lenses or glasses. Subjects were recommended not to wear glasses because the light reflected from glasses often occludes the pupil. The Research Ethics Committee of Seoul National University approved the experimental procedures (IRB No. 2007/003-029). All participants gave informed consent before the experiments and were naïve to the purpose of the experiments. The data quality was not good when the eyelids occluded the pupil substantively, the blink rate was too high (> 40/mins), or subjects dozed off during data collection. As result, one, three, and one subjects were excluded from data analysis for the fixation, auditory oddball, and delayed orientation estimation tasks, respectively. However, we did not exclude trials or subjects based on task performance because task performance was generally good for all the participants. In both of the auditory oddball and delayed estimation tasks, subjects in our study showed a reasonable range of performance in accuracy (mean±SD = 97.74±2.59%, range = 90.4%-99.9% for the auditory oddball experiment; absolute error mean±SD = 6.19±1.74, 8.93±3.06, 12.94±4.43, 21.89±5.88 deg, range = 4.35–10.90, 4.86–18.22, 8.01–23.03, 15.89–35.11 deg for each memory load of the delayed estimation experiment) and RT (mean±SD = 360±63 msec, range = 201–454 msec for the auditory oddball experiment; mean±SD = 1.95±0.42, 2.02±0.49, 2.17±0.55, 2.31±0.62, range = 1.45–3.24, 1.46–3.70, 1.58–4.09, 1.67–4.35 sec for each memory load of the delayed estimation experiment), which were quite comparable to those reported in the previous work using the same tasks [44,47,55].

Apparatus and eye-tracking setup

All experiments were conducted in a scotopic room. Subjects’ heads were positioned on a chin rest in front of a monitor (LG FLATRON 19-inch monitor for the fixation task; Dell 4k monitor for the remaining two tasks). The distance between the chin rest and the monitor was 60 cm (in the fixation task) or 90cm (in the remaining two tasks). Pupil diameters and gaze positions were binocularly recorded with the Eyelink 1000 plus in the fixation task or the Eyelink 1000 system (SR Research) in the remaining two tasks at a sampling rate of 500Hz. The eye tracker was calibrated using the built-in five-point calibration routine (HV5) at the beginning of each experimental session, after each break, whenever the background luminance changed, or occasionally when the subjects’ head positions were found to move from the original position during runs. Eye-tracking signals were acquired in a ‘pupil–corneal reflection (P-CR)’ mode. The pupil size was estimated using the built-in ellipsoid fitting method, which is known to be robust for pupil occlusion by the eyelids.

Stimuli and procedure

The stimuli were generated using MATLAB (MathWorks) in conjunction with Psychtoolbox-3 [56–58]. To prevent any unwanted processes associated with voluntary blink suppression, such as a natural body urge [39], we intentionally did not give the subject any instructions about eye blinking at all throughout the entire experimental sessions. Subjects were allowed to take as many breaks as desired at the end of each run, disengaging from the eye-tracking setup and moisturizing the eyes using disposable artificial tears as needed. Experimenters monitored the state of subjects to check whether they fall asleep or have dry eyes. Subjects were recommended to take a break or nap for a few minutes if they felt sleepy.

Fixation task

Each subject participated in three sessions, each of which consisted of twelve 140-s-long eye-tracking runs. To check whether and how the shape and amplitude of BPR change as a function of baseline pupil size, we varied the background luminance of the display with four levels. To do so, the lowest and highest possible levels of luminance were included, and a black strawboard was attached to the monitor to block the light thoroughly for the darkest background-luminance condition. The two intermediate levels were chosen such that the corneal flux density (retinal illuminance), a product of visual area and luminance from the area, which determines baseline pupil size [42], fell in a dynamic range. As a result, the background-luminance levels were 0.00, 0.03, 5.35, and 97.60 cd/m², which corresponded to 7.8, 62.3, 5256.8, and 95790.0 cd/m²/deg² of corneal flux density in the current experimental setup with fixation stimuli and the infrared light from the eye tracker. Each background-luminance run was repeated in a row over three runs. The order of the highest three background-luminance conditions was determined using the Latin square method, while the lowest background-luminance condition was always assigned to the last three runs because the strawboard had to be attached unlike the other conditions. Subjects were instructed to fixate on a fixation cross (height/width: 1.47 visual angular degrees (v.a.d.)). The luminance of the fixation cross was 97.6 cd/m² for the lowest two background-luminance conditions and 0.03 cd/m² for the two highest conditions for visibility. However, note that both the luminance of the fixation cross and the background always remain unchanged within each eye-tracking run. Gaze-related trial exclusion was not applied.

Auditory oddball task

Two brief sine-tone auditory stimuli with different frequencies (200 ms in length, 1.5 kHz and 1 kHz) were heard via external speakers. To help subjects remain attentive and prevent unwanted pupil-size measurement errors due to visual gaze, we instructed subjects to fixate at a bull’s eye stimulus (outer radius: 0.15 v.a.d., inner radius: 0.03 v.a.d., 0.06 cd/m²) on the darkest background-luminance screen with a large circle (18 v.a.d., 31.10 cd/m²). To measure the changes in pupil size associated with stimulus oddity while preventing those associated with motor responses or mental counting, we asked subjects to perform a two-alternative forced-choice (2AFC) task so that a button press was made every trial. Subjects were instructed to respond as fast and accurately as possible by pressing the”1” or”2” key of a number pad to low-tone or high-tone stimuli while fixating to the fixation target throughout entire runs. The tone-key assignment was counterbalanced across subjects. There was no feedback for performance. There were three types of runs: (i) in the ‘high-tone-odd’ run, the high-tone and low-tone stimuli were presented for 20% and 80%, respectively, of trials; (ii) in the ‘low-tone-odd’ run, the high-tone and low-tone stimuli were presented for 80% and 20%, respectively, of trials; (iii) in the ‘no-odd’ run, the high-tone and low-tone stimuli were presented for 50% and 50%, respectively, of trials. A single run consisted of 70 trials, and subjects participated in 14 to 18 runs (14 runs for one subject; 15 runs for 19 subjects; 18 runs for 4 subjects). The sequence of stimuli was pseudo-randomized. The inter-stimulus interval was fixed at 2 s. Gaze-related trial exclusion was not applied. Deviance from bull’s eye was 1.79 ± 0.94 deg (mean ± SD across subjects)

Delayed orientation estimation task

Subjects were instructed to fixate their gaze at a bull’s-eye fixation stimulus (outer radius: 0.15 v.a.d., inner radius: 0.03 v.a.d., 57.10 cd/m²) on the center of the gray (27.72 cd/m²) background. Each trial consisted of six epochs, preparation (0.5–1 s), stimulus presentation (2 s), masking (1 s), orientation estimation (up to 6 sec), confidence reporting (no time limit), and waiting (1 s) epochs. Note that the duration of trial varied because the length of the orientation estimation (2.11 ± 0.51 sec (mean ± SD across subjects, min = 1.55, max = 3.85 sec)) and confidence reporting epochs (0.77 ± 0.29 sec (mean ± SD across subjects, min = 0.41, max = 1.54 sec)) depend on how quickly subjects respond. In the preparation epoch, by acquiring subjects’ gaze data online, we aborted trials in which the gaze deviated more than 2 v.a.d. from the fixation center and re-administered those trials later. This abortion procedure was omitted in some runs, where online gaze data were unstable. In the stimulus presentation and masking epochs, oriented bar stimuli with different colors, the number of which varied from 1 to 2, 4, and 8 bars, were presented and followed by masking stimuli, which consisted of 90 bars that changed their color and orientation every video frame (60 Hz). In the orientation estimation epoch, subjects had to estimate the post-cued (by color) orientation of the presented bars by rotating a probe bar stimulus as close to its original orientation as possible. In the confidence reporting epoch, subjects reported their subjective confidence about orientation estimation with a slider. The sequence of bar number, orientation and location was pseudo-randomized over trials to minimize any order-related confounding effects. Each run consisted of 48 trials. Mean estimation errors (in angular degrees) and mean confidence was shown as feedback for overall performance at the end of each run. Subjects conducted 8 runs in a single daily session (for 11 subjects) or 20 runs in 2 or 5 daily sessions (for 10 subjects).

Data analysis

Preprocessing of eye-tracking data

The data from the eye tracker, which was provided in a digitized format called EDF, were imported to MATLAB using an open-source script (https://github.com/iandol/opticka/blob/master/communication/edfmex.m) and analyzed using the custom MATLAB scripts. In all experiments, pupil size and gaze positions were measured binocularly. However, we opted to use the data from one of the two eyes that provided more reliable data for the following reasons. First, there were subtle but noticeable differences in blink offset timing between the two eyes. Because we wanted to define the time course of BPR precisely, we were concerned that averaging two signals that differ in timing might introduce unwanted blurs. Second, occasionally in some individuals, only one eye’s data was not detected or became unstable probably due to eyelid occlusion, which is known to occur more frequently in Asian people. If we average the signals from the two eyes, the time courses are expected to show weak but spurious fluctuations due to the missed signals from the unreliable eye.

The definition of single blink events

Blink events were defined in the following procedure. Initially, we identified the time points where pupil data were missing, extremely small (<1 mm), or underwent abrupt changes. Here, to identify such “abrupt changes,” we high-passed the data using the 3rd order Butterworth filter with 10 Hz cutoff frequency and detected the time points where the absolute values of the high-passed data exceeded 0.25 mm. Next, by assuming that the consecutive eye blinks are unlikely to occur in a row within 200 ms, we treated the initial time points that were apart less than 200 ms as belonging to a single blink event. The start and end time points of this “single blink event” were defined as the onset and offset of an individual eye blink. The mean blink duration defined in the above procedure was 288 ± 108 ms (mean ± SD across subjects) for the fixation task, 311 ± 82 ms for the auditory oddball task, and 349 ± 107 ms for the delayed orientation estimation task. The mean blink rate defined in the above procedure was 0.28 ± 0.12 Hz (mean ± SD across subjects) for the fixation task, 0.39 ± 0.18 Hz for the auditory oddball task, 0.28 ± 0.11 Hz for the delayed orientation estimation task.

The removal of artifacts around single blink events

It is known that the video-based eye-tracking data are typically contaminated with artifacts immediately before and after each blink event. For this reason, previous studies adopted a custom of getting rid of 150 ms around single blink events, i.e., replacing the data of those time points with not-a-number (NaN) values [59–61]. We also applied the same procedure for the time points before each blink but slightly modified the procedure for those after each blink, as follows, because the post-blink artifacts appeared more prolonged compared to the pre-blink artifacts. First, we computed the derivatives of the raw data and smoothed them by taking their moving averages with a boxcar time window (0.25 s in size). The smoothing procedure was applied not to be oversensitive for detection in the following step. Next, we determined the endpoint of the post-blink artifact by identifying the time point where the smoothed derivative starts to be smaller than 0.1 mm/s. We also constrained that the post-blink artifact period is longer than 200 ms and smaller than 500 ms. As the final step, the pre-blink and post-blink artifact time points were replaced with NaN values and linearly interpolated. Note that the smoothing with a boxcar window was applied only when defining the endpoint of the post-blink artifact, but not when preprocessing the pupil time courses that were used for analysis. Also, we note that a recent work [62] provided a more advanced algorithm to define blink events and to avoid pre-blink and post-blink artifacts. We also stress that this procedure only removes the artifacts that occur within a short period around blink events and should be distinguished from the procedure of correcting the pupil data for BPR, which is provided in the current work.

After interpolating the blink events and associated artifact time points, the pupil time courses were band-pass filtered (0.02 and 4Hz) with the 3rd order Butterworth filter to remove electric noise and slow drift components. Then the data, which was originally provided in an arbitrary unit, was converted into square millimeters by measuring the relationship between the original arbitrary unit and physical square millimeters. In doing so, we printed black dots, and then measured the size of the dots with a pupil-only mode (Eyelink 1000), or with pinching holes into the dots and attaching a silver foil underneath the holes (Eyelink 1000 plus). The converted square-millimeter unit was re-converted into millimeters for data analysis as follows: $d i a m t e t e r = \sqrt{A r e a / π}$ , assuming that the shape of the pupil is a perfect circle.

Analyses on gaze deviance and potential dependency between blink and gaze

Since it is known that large changes in gaze position can distort pupil-size measurements by tilting eye images [63], we checked the extent to which the actual gaze position deviated from the fixation target. The amount of gaze deviation was 1.45 ± 0.09, 1.79 ± 0.94, and 1.84 ± 0.64 v.a.d. (mean ± SD across subjects), and 0.52 ± 0.23, 0.75 ± 0.40, 0.94 ± 0.38 v.a.d. (mean ± SD of the within-subject SDs) for the fixation, the auditory oddball, the delayed orientation estimation tasks, respectively. Although these amounts of deviances can be considered small, there might be some noise from gaze deviance in our pupil data. Thus, we checked the possibility that the potential pupil change confounded by gaze deviance could have affected our claims about the effect of blinks on pupil-size measurements. We reasoned that our claim is threatened only when blinks are correlated, either positively or negatively, with the amount of gaze deviances. However, we found that the cross-correlation between blink events and gaze deviances was close to zero or almost negligible (maximum cross correlation < 0.025 in any runs, in both horizontal and vertical gaze deviances). Based on these results, we concluded that the pupil changes due to gaze deviances might have added some noise to our data but are unlikely to affect our findings regarding the relationship between blinks and pupillary responses because blinks were not correlated with pupil-size measurements.

Definition of isolated blink-affected and blink-free time courses

To characterize the shape of BPR that is associated with a single, isolated blink in the fixation task, we identified only the blinks that are apart from neighboring blinks by more than 3 s. To ensure the reliability of data, we included only the data from the subjects for whom more than 25 isolated blinks were identified for each background-luminance condition. As a result, the data from 21 subjects contributed to the definition of isolated blink-affected time courses (Fig 2C–2F). To define the blink-free time courses, we took the following steps. First, we identified only the inter-blink intervals with more than 6 s. Next, we removed the initial 3-s portion, which is not free from a blink. Then, we divided the remaining portion into as many 3-s portions as possible (e.g., if the remaining portion is 7-s long, then we obtained two segments of blink-free time courses and discarded the 1-s portion at the end). The grand average of these blink-free time courses is shown as the blue curve in Fig 4C.

Analysis of the blink-by-blink variability of BPR in peak amplitude and time-to-the-peak

By inspecting the grand average of blink-affected time courses across subjects and background-luminance conditions, we found that it reached the negative peak at around 0.5–1.2 s after blink offset (Fig 2C). Based on this finding, we quantified the amplitude of individual BPRs by taking the lowest pupil size within the period of 0.5–1.2 s after blink offset. Next, for each subject, these individual BPRs were separately grouped according to the four background-luminance levels and then sorted by their amplitudes into five quantile bins within each background-luminance group. Then, for each quantile, BPR time courses were averaged across the background-luminance groups such that the BPRs from the 4 luminance levels equally contributed to the averaged time course. For reliable estimation of the peak amplitude and time-to-the-peak, these averaged time courses were further corrected for noise with a cubic splicing method. Lastly, for each quantile, we defined the peak amplitude and the time-to-the-peak by finding the negative maximum pupil size and the time point when that maximum pupil size was found, respectively (Fig 2F).

Prediction of BPR confound via simulation

To illustratively demonstrate how BPR can affect the pupil time courses and lead to incorrect interpretation of the pupil data acquired in cognitive experiments, we conducted ex-ante simulations using the BPR learned from the fixation task on both the auditory oddball and the delayed orientation estimation tasks. The detailed procedure was as follows. First, the BPR that was defined at the second-highest background-luminance condition in the fixation task (the second brightest curve in Fig 2D) was used as the model BPR because the retinal illuminance under that condition was closest to the background-luminance level used in the auditory oddball and the delayed orientation estimation tasks. Second, to demonstrate the pure impact of BPR, we convoluted with this model BPR the time courses of blink rates that were empirically acquired in the auditory oddball and the delayed orientation estimation tasks (Fig 3B and 3D). Next, to demonstrate the impact of BPR in the presence of a certain pupillary response that is associated with a cognitive process of interest (i.e., responses to the sound and the manual response to it), we initially convoluted the blink rate from the auditory oddball task with the model BPR (as was done in the previous step) and add this BPR-convoluted time course (the red curve in Fig 5B) to the presumed pupillary response to a cognitive event (the blue curve in Fig 5B). In doing so, to specifically demonstrate how the BPR spuriously distorts the pupil responses to a cognitive event of interest, we contrasted the simulated responses associated with two specific sequences of trials, namely the “blink-in-current-trial” sequence and the “blink-in-previous-trial” sequence. These two sequences were both defined by the series of blink events in current and 1, 2, and 3-back trials ([t-3,t-2,t-1,t], where t stands for a current trial). To isolate the impact of BPR that occurred in previous (t-1) and current (t), both the two sequences were constrained for cases where blink events did not occur on t-2 and t-3 trials. Thus, the trial series of blink events were [0 0 0 1] for the “blink-in-current-trial” sequence and [0 0 1 0] for the “blink-in-previous-trial” sequence, where 0 and 1 stands for the absence and presence of blink events, respectively.

Analysis of the blink rate and pupil data in the auditory oddball and delayed orientation estimation tasks

To quantify how the frequency of blinks fluctuates over time within a trial, we computed the blink rate by taking the moving (boxcar, size = 0.1 sec) averages of blink offset pulses across trials (as shown in the top panels of Fig 3B and 3D and dotted line in the bottom panel of Figs 8 and 9A). To control for the differences in overall pupil size across runs, the pupil time courses were demeaned run-by-run for the mean difference and pooled SEM analyses, or z-scored run by run for the AUC and bootstrapping analysis. We demeaned the mean difference and pooled SEM analyses because z-scoring applies different denominators across runs, so cannot compare the change in mean or SEM directly. Next, trial-locked pupil time courses were baselined at the start of each trial. To quantify the magnitude of those trial-locked pupil responses that were associated with cognitive events of interest (i.e., “hearing expected or unexpected sounds” and “retrieval of a bar orientation with different working-memory loads”), we averaged the data within certain time windows. In the auditory oddball task, the start point of the time window was set to 0.5 after event onset because it appears to take at least 0.5 s for the pupil size to respond to events. The endpoint of the time window was set to the endpoint of the trial (2 s). In the delayed orientation estimation task, we used the time points where the grand average of pupil-size measurements across memory load conditions were above 75% of its maximum as the start and endpoint of the integration window (1.22–2.38 s). Having quantified these response magnitudes for each trial, we sorted them into the experimental conditions and computed their mean and its standard error for each condition and each subject, which was used to plot the data in Figs 7B, 7C, 9B and 9C. To evaluate whether and how much the BPR correction methods improve the discriminability in pupil size between conditions, we derived the receiver operation characteristic (ROC) curve from a given pair of distributions of trial-to-trial response magnitudes and computed the area-under-the-ROC curve for each pair of conditions, as shown in Figs 7D and 9D.

Statistical power analysis

To evaluate how much the BPR-correction algorithms contribute to the statistical power of experiments, we carried out the regression analysis on the bootstrap samples drawn from the original data of the auditory oddball and the delayed orientation estimation tasks while varying the bootstrap sample size. First, for a given sample size, bootstrap samples were repeatedly (10,000 times) drawn with replacement from the original pool of trial-to-trial response magnitudes, which were merged across subjects. Then, we linearly regressed the data onto the oddity levels (for the auditory oddball task) or the working-memory loads (for the delayed orientation estimation task). Specifically, the tones with 80%, 50%, and 20% probability were coded as -0.5, 0, and 0.5, respectively, as the regressor for the auditory oddball task; the 1, 2, 4, 8 bar conditions were coded as 1, 2, 3, and 4, respectively, as the regressor for the delayed orientation estimation task. The linear regression was implemented with the “fitlm.m” function in the Statistical and Machine Learning Toolbox of MATLAB. Lastly, we plotted the percentage of the bootstrap samples in which the regression was statistically significant (p < 0.01) as a function of sample size and computed how many trials are needed to reach the null hypothesis rejection fraction of 95% for the BPR-uncorrected data and the BPR-corrected data (Figs 7E and 9E).

The algorithm of correcting pupillary responses for BPR

The generative model

At the core of our BPR correction algorithm lies the probabilistic inference about the generic shape of BPR, h, and its blink-to-blink amplitude, θ_j (where j stands for jth blink). Based on what we learned from the fixation task, we built the generative model for this inference, which consisted of two parts, one for the “blink-free” time courses and the other for the “blink-affected” time courses (see the subsection titled “Definition of blink-affected and blink-free time courses” in the above for the detailed definition of these two types of times courses). The original, 3-s-long, time courses were down-sampled from 500 Hz to 5 Hz to minimize computing load, which resulted in vectors of 16 ({0 s, 0.2 s, 0.4 s, …, 3 s}) × 1 dimensions.

We modeled the blink-free time courses collected at the ith sample under the lighting condition c from the subject s, $Y_{i s c}^{f r e e}$ , as the linear sum of the time course of ongoing spontaneous fluctuation, SF_isc, and that of responses associated with a cognitive state of interest, $P R C_{i s c}^{f r e e}$ :

Y_{i s c}^{f r e e} = S F_{i s c} + P R C_{i s c}^{f r e e}

(Eq 1a)

Note that s and c are used simply to indicate that the algorithm should be applied separately to the data collected from a particular individual under a particular lighting condition.

Next, we assumed that the power transform of $Y_{i s c}^{f r e e}$ is a stochastic sample drawn from a multivariate normal distribution with the mean, $μ_{s c}^{f r e e}$ , and the covariance, ∑_sc:

g (Y_{i s c}^{f r e e}; λ_{s c}) ~ M V N (μ_{s c}^{f r e e}, Σ_{s c})

(Eq 1b)

where g is the box-cox power transformation with a free power parameter, λ. Here, $μ_{s c}^{f r e e}$ can be approximated with the mean time course of the blink-free time courses for the c lighting condition from the subject s, ${\hat{μ}}_{s c}^{f r e e}$ . As current pupil sizes are strongly correlated with neighboring values, ∑_sc was approximated as the first-order autoregressive function with two parameters, σ_sc and ρ_sc:

{({\hat{Σ}}_{s c})}_{k, l} = σ_{s c}^{2} ρ_{s c}^{| k - l |} (k, l = 1, 2, 3, \dots, 16)

(Eq 1c)

In sum, Eq 1a–1c describes the generative process for the blink-free samples with three parameters, {λ_ssc, σ_sc, ρ_sc}.

Next, the 3-s-long, individual time courses that were affected by jth blink, $Y_{j s c}^{a f f e c t e d}$ , were modeled as the linear sum of the time course of ongoing spontaneous fluctuation, SF_jsc, that of responses associated with a cognitive state of interest, $P R C_{j s c}^{a f f e c t e d}$ , and that of BPR responses, BPR_jsc:

Y_{j s c}^{a f f e c t e d} = B P R_{j s c} + S F_{j s c} + P R C_{j s c}^{a f f e c t e d}

(Eq 2a)

where, based on what was learned in the fixation task, BPR_jsc was assumed to be constant in shape under a given lighting condition for a given subject, h_sc, but vary in amplitude from blink to blink, θ_jsc:

B P R_{j s c} = θ_{j s c} * h_{s c}

(Eq 2b)

where h_sc was modeled as a gamma probability density function, f, with two shape parameters, α and β, one amplitude parameter, γ, and one temporal shift parameter, t, as follows:

h_{s c} (x; α_{s c}, β_{s c}, γ_{s c}, t_{s c}) = γ_{s c} * f (x - t_{s c}; α_{s c}, β_{s c})

(Eq 2c)

where x is a time vector comprising the blink-affected time course. As was done for the blink-free time courses, the power transform of $Y_{j s c}^{a f f e c t e d} - B P R_{j s c}$ was assumed to be a stochastic sample drawn from a multivariate normal distribution with the mean, $μ_{s c}^{a f f e c t e d}$ , and the covariance, ∑_sc:

g (Y_{j s c}^{a f f e c t e d} - B P R_{j s c}; λ_{s c}) = g (S F_{j s c} + P R C_{j s c}^{a f f e c t e d}; λ_{s c}) ~ M V N (μ_{s c}^{a f f e c t e d}, Σ_{s c})

(Eq 2d)

where $μ_{s c}^{a f f e c t e d}$ can be approximated with the cubic spline of both ends of the mean of the blink-affected time courses, ${\hat{μ}}_{s c}^{a f f e c t e d}$ . In sum, Eq 2a–2d describes the generative process for the blink-affected samples with additional five parameters, {θ_jsc, α_sc, β_sc, γ_sc, t_sc}.

Inference of generative model parameters

Having defined the generative model as above, the goal of the BPR-correction algorithm becomes to infer the parameters of the generative models, a total of seven parameters for a given lighting condition for a given subject, {λ_sc, σ_sc, ρ_sc, α_sc, β_sc, γ_sc, t_sc} and 1 parameter for each blink θ_jsc. These parameters can be inferred as follows. First, λ_sc for the power transform can be estimated by the box-cox method. Second, σ_sc and ρ_sc for ∑_sc can be inferred by finding their values that maximize the log-likelihood of the observed $Y_{i s c}^{f r e e}$ :

{{\hat{σ}}_{s c}, {\hat{ρ}}_{s c}} = \underset{σ, ρ}{argmax} [\sum_{i = 1}^{n_{f r e e}} ln (M V N (g (Y_{i s c}^{f r e e}; λ_{s c}); {\hat{μ}}_{s c}^{f r e e}, {\hat{Σ}}_{s c}))]

(Eq 3a)

where n_free is the number of blink-free samples. Third, {α_sc, β_sc, γ_sc, t_sc} for h_sc can be inferred by finding the values that maximize the likelihood of the power transform of the subtraction of BPR from the mean of all the blink-free samples, $Y_{j s c}^{a f f e c t e d}$ :

{{\hat{α}}_{s c}, {\hat{β}}_{s c}, {\hat{γ}}_{s c}, {\hat{t}}_{s c}} = \underset{α, β, γ, t}{argmax} [M V N (g (\bar{Y_{s c}^{a f f e c t e d}} - h_{s c}; λ_{s c}); {\hat{μ}}_{s c}^{a f f e c t e d}, {\hat{Σ}}_{s c} / n_{a f f e c t e d})]

where n_affected is the number of blink-affected samples, and $\bar{Y_{s c}^{a f f e c t e d}}$ is the mean of $Y_{j s c}^{a f f e c t e d}$ over all those samples. Here, the power transform of the subtraction of BPR from the mean of all the blink-affected samples can be approximated as below:

g (\bar{Y_{s c}^{a f f e c t e d}} - h_{s c}; λ_{s c}) ~ M V N ({\hat{μ}}_{s c}^{a f f e c t e d}, {\hat{Σ}}_{s c} / n_{a f f e c t e d})

Because

\frac{1}{n_{a f f e c t e d}} \sum_{j = 1}^{n_{a f f e c t e d}} g (S F_{j s c} + P R C_{j s c}^{a f f e c t e d}; λ_{s c}) ~ M V N ({\hat{μ}}_{s c}^{a f f e c t e d}, {\hat{Σ}}_{s c} / n_{a f f e c t e d}) .

The inference of {α_sc, β_sc, γ_sc, t_sc} allows for replacing h_sc with ${\hat{h}}_{s c}$ , from which the power transform of the BPR-affected time courses can be derived as follows:

\begin{array}{l} g (Y_{j s c}^{a f f e c t e d} - B P R_{j s c}; λ_{s c}) = g (Y_{j s c}^{a f f e c t e d} - θ_{j s c} * {\hat{h}}_{s c}; λ_{s c}) = \\ g (S F_{j s c} + P R C_{j s c}^{a f f e c t e d}; λ_{s c}) ~ M V N ({\hat{μ}}_{s c}^{a f f e c t e d}, {\hat{Σ}}_{s c}) . \end{array}

Lastly, θ_jsc, the blink-by-blink amplitude of h_sc, can be inferred by finding the value of θ_jsc that maximizes the power transform of the BPR-affected time courses:

{\hat{θ}}_{j s c} = \underset{θ}{argmax} [M V N (g (Y_{j s c}^{a f f e c t e d} - θ_{j s c} * {\hat{h}}_{s c}; λ_{s c}); {\hat{μ}}_{s c}^{a f f e c t e d}, {\hat{Σ}}_{s c})] .

We note that when two or more blinks occurred within 3 s, BPRs were overlapped with one another, which made it hard to estimate non-BPR components. Thus, in such cases, ${\hat{μ}}_{s c}^{a f f e c t e d}$ was replaced with a vector whose elements were padded with the first or last values at each blink, and then averaged across blinks. We implemented this BPR-correction algorithm into MATLAB scripts (bprcorrect.m in the BPR-correction toolbox) as an out-of-the-box solution so that users can readily apply the algorithm to their data (https://github.com/yookyung1310/BPR_toolbox).

Supporting information

S1 Fig. Individual’s BPR shape remains constant across runs.

(left panel) Split-half correlation in peak amplitudes (right panel) Split-half correlation in peak time.

(TIFF)

Click here for additional data file.^{(229.1KB, tiff)}

S2 Fig. Zoomed out diagonal plot for Fig 9B and 9C.

(TIFF)

Click here for additional data file.^{(2.4MB, tiff)}

S1 Video. Example of blink-locked pupillary response (BPR).

(MOV)

Click here for additional data file.^{(4.6MB, mov)}

Acknowledgments

This article originated as part of a dissertation completed at Seoul National University by the first author under the direction of the last author. We acknowledge Joonwon Lee and Jaeseob Lim for advising some analyses and visualization and Minjin Choe for part of data collection.

Data Availability

All data files are available from the OSF database (https://osf.io/uvs2d/).

Funding Statement

This work was supported by the National Research Foundation of Korea Grants NRF-2015M3C7A1031969, NRF-2018R1A4A1025891, and NRF-2017M3C7A1047860 All authors received the grant http://nrf.re.kr/ The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

1.Bradshaw J. Pupil Size as a Measure of Arousal During Information Processing. Nature. 1967;216: 515–&. doi: 10.1038/216515a0 [DOI] [PubMed] [Google Scholar]
2.Preuschoff K, Hart BM’, Einhaeuser W. Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making. Front Neurosci. 2011;5. doi: 10.3389/fnins.2011.00005 [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Henson DB, Emuh T. Monitoring Vigilance during Perimetry by Using Pupillography. Invest Ophthalmol Vis Sci. 2010;51: 3540–4. doi: 10.1167/iovs.09-4413 [DOI] [PubMed] [Google Scholar]
4.Hess EH, Polt JM. Pupil Size in Relation to Mental Activity During Simple Problem-Solving. Science. 1964;143: 1190–&. doi: 10.1126/science.143.3611.1190 [DOI] [PubMed] [Google Scholar]
5.Kahneman D, Beatty J. Pupil Diameter and Load on Memory. Science. 1966;154: 1583–&. doi: 10.1126/science.154.3756.1583 [DOI] [PubMed] [Google Scholar]
6.Lavin C, San Martin R, Rosales Jubal E. Pupil dilation signals uncertainty and surprise in a learning gambling task. Front Behav Neurosci. 2013;7: 218. doi: 10.3389/fnbeh.2013.00218 [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Kloosterman NA, Meindertsma T, van Loon AM, Lamme VAF, Bonneh YS, Donner TH. Pupil size tracks perceptual content and surprise. Eur J Neurosci. 2015;41: 1068–1078. doi: 10.1111/ejn.12859 [DOI] [PubMed] [Google Scholar]
8.de Gee JW, Knapen T, Donner TH. Decision-related pupil dilation reflects upcoming choice and individual bias. Proceedings of the National Academy of Sciences of the United States of America. 2014;111: E618–E625. doi: 10.1073/pnas.1317557111 [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Joshi S, Li Y, Kalwani RM, Gold JI. Relationships between Pupil Diameter and Neuronal Activity in the Locus Coeruleus, Colliculi, and Cingulate Cortex. Neuron. Elsevier Inc; 2016;89: 221–234. doi: 10.1016/j.neuron.2015.11.028 [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Szabadi E. Modulation of physiological reflexes by pain: role of the locus coeruleus. 2016;: 1–15. doi: 10.3389/fnint.2012.00094/abstract [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Murphy PR, O’Connell RG, O’Sullivan M, Robertson IH, Balsters JH. Pupil diameter covaries with BOLD activity in human locus coeruleus. Hum Brain Mapp. 2014;35: 4140–4154. doi: 10.1002/hbm.22466 [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Joshi S, Gold JI. Pupil Size as a Window on Neural Substrates of Cognition. Trends in Cognitive Sciences. 2020;24: 466–480. doi: 10.1016/j.tics.2020.03.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Kahneman D. Attention and effort. Englewood Cliffs, NJ: Prentice-Hall; 1973. doi:doi=10.1.1.398.5285&rep=rep1&type=pdf [Google Scholar]
14.van der Wel P, van Steenbergen H. Pupil dilation as an index of effort in cognitive control tasks: A review. Psychonomic Bulletin & Review; 2018;: 1–11. doi: 10.3758/s13423-018-1432-y [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Loewenfeld IE. The Pupil. Anatomy, Physiology and Clinical Applications. 2nd ed. Boston: Butterworth- Heinemann; 1999. pp. 1–1590. doi: 10.1023/a:1002134106425 [DOI] [Google Scholar]
16.Mathôt S, Van der Stigchel S. New Light on the Mind’s Eye: The Pupillary Light Response as Active Vision. Curr Dir Psychol Sci. 2015;24: 374–378. doi: 10.1177/0963721415593725 [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Knapen T, de Gee JW, Brascamp J, Nuiten S, Hoppenbrouwers S, Theeuwes J. Cognitive and Ocular Factors Jointly Determine Pupil Responses under Equiluminance. Verguts T, editor. PLoS ONE. Public Library of Science; 2016;11: e0155574–13. doi: 10.1371/journal.pone.0155574 [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Fukuda K, Stern JA, Brown TB, Russo MB. Cognition, blinks, eye-movements, and pupillary movements during performance of a running memory task. Aviat Space Environ Med. 2005;76: C75–85. [PubMed] [Google Scholar]
19.Siegle GJ, Ichikawa N, Steinhauer S. Blink before and after you think: Blinks occur prior to and following cognitive load indexed by pupillary responses. Psychophysiology. 2008;45: 679–687. doi: 10.1111/j.1469-8986.2008.00681.x [DOI] [PubMed] [Google Scholar]
20.Hupe JM, Lamirel C, Lorenceau J. Pupil dynamics during bistable motion perception. Journal of Vision. 2009;9: 10–10. doi: 10.1167/9.7.10 [DOI] [PubMed] [Google Scholar]
21.Collins M, Seeto R, Campbell L, Ross M. Blinking and corneal sensitivity. Acta Ophthalmol (Copenh). 1989;67: 525–531. doi: 10.1111/j.1755-3768.1989.tb04103.x [DOI] [PubMed] [Google Scholar]
22.STERN JA, WALRATH LC, GOLDSTEIN R. The Endogenous Eyeblink. Psychophysiology. 1984;21: 22–33. doi: 10.1111/j.1469-8986.1984.tb02312.x [DOI] [PubMed] [Google Scholar]
23.Finnemore VM. Is the dry eye contact lens wearer at risk? Not usually. Cornea. 1990;9 Suppl 1: S51–3– discussion S54. doi: 10.1097/00003226-199010001-00021 [DOI] [PubMed] [Google Scholar]
24.Carney LG, Hill RM. The nature of normal blinking patterns. Acta Ophthalmol (Copenh). John Wiley & Sons, Ltd; 1982;60: 427–433. doi: 10.1111/j.1755-3768.1982.tb03034.x [DOI] [PubMed] [Google Scholar]
25.Doughty MJ. Consideration of Three Types of Spontaneous Eyeblink Activity in Normal Humans: during Reading and Video Display Terminal Use, in Primary Gaze, and while in Conversation. Optometry and Vision Science. 2001;78: 712–725. doi: 10.1097/00006324-200110000-00011 [DOI] [PubMed] [Google Scholar]
26.Doughty MJ. Effects of Background Lighting and Retinal Illuminance on Spontaneous Eyeblink Activity of Human Subjects in Primary Eye Gaze. Eye & Contact Lens: Science & Clinical Practice. 2013;39: 138–146. doi: 10.1097/ICL.0b013e31827124b7 [DOI] [PubMed] [Google Scholar]
27.Doughty MJ. Further assessment of gender- and blink pattern-related differences in the spontaneous eyeblink activity in primary gaze in young adult humans. Optometry and Vision Science. 2002;79: 439–447. doi: 10.1097/00006324-200207000-00013 [DOI] [PubMed] [Google Scholar]
28.Daniels LB, Nichols DF, Seifert MS, Hock HS. Changes in pupil diameter entrained by cortically initiated changes in attention. Vis Neurosci. 2012;29: 131–142. doi: 10.1017/S0952523812000077 [DOI] [PubMed] [Google Scholar]
29.Orchard LN, STERN JA. Blinks as an index of cognitive activity during reading. Integr Physiol Behav Sci. 1991;26: 108–116. doi: 10.1007/BF02691032 [DOI] [PubMed] [Google Scholar]
30.Nakano T, Yamamoto Y, Kitajo K, Takahashi T, Kitazawa S. Synchronization of spontaneous eyeblinks while viewing video stories. Proceedings of the Royal Society B: Biological Sciences. 2009;276: 3635–3644. doi: 10.1098/rspb.2009.0828 [DOI] [PMC free article] [PubMed] [Google Scholar]
31.Liao H-I, Yoneya M, Kidani S, Kashino M, Furukawa S. Human Pupillary Dilation Response to Deviant Auditory Stimuli: Effects of Stimulus Properties and Voluntary Attention. Front Neurosci. 2016;10: 403–14. doi: 10.3389/fnins.2016.00403 [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Groman SM, James AS, Seu E, Tran S, Clark TA, Harpster SN, et al. In the Blink of an Eye: Relating Positive-Feedback Sensitivity to Striatal Dopamine D2-Like Receptors through Blink Rate. Journal of Neuroscience. 2014;34: 14443–14454. doi: 10.1523/JNEUROSCI.3037-14.2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
33.Taylor JR, Elsworth JD, Lawrence MS, Sladek JR Jr., Roth RH, Redmond DE Jr. Spontaneous Blink Rates Correlate with Dopamine Levels in the Caudate Nucleus of MPTP-Treated Monkeys. Experimental Neurology. 1999;158: 214–220. doi: 10.1006/exnr.1999.7093 [DOI] [PubMed] [Google Scholar]
34.Kotani M, Kiyoshi A, Murai T, Nakako T, Matsumoto K, Matsumoto A, et al. The dopamine D1 receptor agonist SKF-82958 effectively increases eye blinking count in common marmosets. Behavioural Brain Research. 2016;300: 25–30. doi: 10.1016/j.bbr.2015.11.028 [DOI] [PubMed] [Google Scholar]
35.Zhang T, Mou Di, Wang C, Tan F, Jiang Y, Lijun Z, et al. Dopamine and executive function: Increased spontaneous eye blink rates correlate with better set-shifting and inhibition, but poorer updating. International Journal of Psychophysiology. Elsevier B.V; 2015;96: 155–161. doi: 10.1016/j.ijpsycho.2015.04.010 [DOI] [PubMed] [Google Scholar]
36.Colzato LS, van den Wildenberg WPM, van Wouwe NC, Pannebakker MM, Hommel B. Dopamine and inhibitory action control: evidence from spontaneous eye blink rates. Exp Brain Res. 2009;196: 467–474. doi: 10.1007/s00221-009-1862-x [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Aarts H, Bijleveld E, Custers R, Dogge M, Deelder M, Schutter D, et al. Positive priming and intentional binding: Eye-blink rate predicts reward information effects on the sense of agency. Social Neuroscience. 2012;7: 105–112. doi: 10.1080/17470919.2011.590602 [DOI] [PubMed] [Google Scholar]
38.van Bochove ME, Van der Haegen L, Notebaert W, Verguts T. Blinking predicts enhanced cognitive control. Cogn Affect Behav Neurosci. 2012;13: 346–354. doi: 10.3758/s13415-012-0138-2 [DOI] [PubMed] [Google Scholar]
39.Berman BD, Horovitz SG, Morel B, Hallett M. Neural correlates of blink suppression and the buildup of a natural bodily urge. NeuroImage. Elsevier Inc; 2012;59: 1441–1450. doi: 10.1016/j.neuroimage.2011.08.050 [DOI] [PMC free article] [PubMed] [Google Scholar]
40.Beatty J, Lucero-Wagoner B. The pupillary system. In Cacioppo J T, Tassinary L G, & Berntson G (Eds), Handbook of psychophysiology. Cambridge university press; 2003. pp. 142–162. [Google Scholar]
41.Korn CW, Bach DR. A solid frame for the window on cognition: Modeling event-related pupil responses. Journal of Vision. 2016;16: 28–16. doi: 10.1167/16.3.28 [DOI] [PMC free article] [PubMed] [Google Scholar]
42.Watson AB, Yellott JI. A unified formula for light-adapted pupil size. Journal of Vision. 2012;12: 12. doi: 10.1167/12.10.12 [DOI] [PubMed] [Google Scholar]
43.Ponder E, Kennedy WP. On the act of blinking. Quarterly Journal of Experimental Physiology. 1928;18: 88–109. [Google Scholar]
44.Hong L, Walz JM, Sajda P. Your eyes give you away: prestimulus changes in pupil diameter correlate with poststimulus task-related EEG dynamics. PLoS ONE. Public Library of Science; 2014;9: e91321. doi: 10.1371/journal.pone.0091321 [DOI] [PMC free article] [PubMed] [Google Scholar]
45.Kamp SM, Donchin E. ERP and pupil responses to deviance in an oddball paradigm. Psychophysiology. John Wiley & Sons, Ltd; 2015;52: 460–471. doi: 10.1111/psyp.12378 [DOI] [PubMed] [Google Scholar]
46.Denison RN, Parker JA, Carrasco M. Modeling pupil responses to rapid sequential events. Behavior Research Methods; 2020;: 1–17. doi: 10.3758/s13428-020-01368-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
47.Bays PM. Noise in Neural Populations Accounts for Errors in Working Memory. Journal of Neuroscience. 2014;34: 3632–3645. doi: 10.1523/JNEUROSCI.3204-13.2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
48.Murphy PR, Robertson IH, Balsters JH, O’connell RG. Pupillometry and P3 index the locus coeruleus-noradrenergic arousal function in humans. Psychophysiology. 2011;48: 1532–1543. doi: 10.1111/j.1469-8986.2011.01226.x [DOI] [PubMed] [Google Scholar]
49.Mathôt S. Tuning the Senses: How the Pupil Shapes Vision at the Earliest Stage. Annu Rev Vis Sci. 2020. doi: 10.1146/annurev-vision-030320-062352 [DOI] [PubMed] [Google Scholar]
50.Zuber BL, Stark L, Lorber M. Saccadic suppression of the pupillary light reflex. Experimental Neurology. 1966;14: 351–370. doi: 10.1016/0014-4886(66)90120-8 [DOI] [PubMed] [Google Scholar]
51.Einhauser W, Koch C, Carter O. Pupil dilation betrays the timing of decisions. Front Hum Neurosci. 2010;4. doi: 10.3389/neuro.09.004.2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
52.Gilzenrat MS, Nieuwenhuis S, Jepma M, Cohen JD. Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn Affect Behav Neurosci. 2010;10: 252–269. doi: 10.3758/CABN.10.2.252 [DOI] [PMC free article] [PubMed] [Google Scholar]
53.Pearl J, Glymour M, Jewell NP. Causal inference in statistics: A primer. John Wiley & Sons; 2016. [Google Scholar]
54.Acland BT, Braver TS (2014). Cili (v0.5.4) [Software] Available from 10.5281/zenodo.48843 [DOI]
55.Kiehl KA, Stevens MC, Laurens KR, Pearlson G, Calhoun VD, Liddle PF. An adaptive reflexive processing model of neurocognitive function: supporting evidence from a large scale (n = 100) fMRI study of an auditory oddball task. NeuroImage. 2005;25: 899–915. doi: 10.1016/j.neuroimage.2004.12.035 [DOI] [PubMed] [Google Scholar]
56.Kleiner M, Brainard D, Pelli D. What’s new in Psychtoolbox-3? 2007. pp. 1–89. [Google Scholar]
57.Pelli DG. The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spat Vis. 1997;10: 437–442. [PubMed] [Google Scholar]
58.Brainard DH. The psychophysics toolbox. Spat Vis. 1997;10: 433–436. [PubMed] [Google Scholar]
59.McCamy MB, Otero-Millan J, Macknik SL, Yang Y, Troncoso XG, Baer SM, et al. Microsaccadic efficacy and contribution to foveal and peripheral vision. J Neurosci. Society for Neuroscience; 2012;32: 9194–9204. doi: 10.1523/JNEUROSCI.0515-12.2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
60.Otero-Millan J, Troncoso XG, Macknik SL, Serrano-Pedraza I, Martinez-Conde S. Saccades and microsaccades during visual fixation, exploration, and search: foundations for a common saccadic generator. Journal of Vision. The Association for Research in Vision and Ophthalmology; 2008;8: 21.1–18. doi: 10.1167/8.14.21 [DOI] [PubMed] [Google Scholar]
61.Choe KW, Blake R, Lee S-H. Pupil size dynamics during fixation impact the accuracy and precision of video-based gaze estimation. VISION RESEARCH. Elsevier Ltd; 2017;: 1–12. doi: 10.1016/j.visres.2014.12.018 [DOI] [PubMed] [Google Scholar]
62.Hershman R, Henik A, Cohen N. A novel blink detection method based on pupillometry noise. Behav Res. Springer US; 2018;50: 107–114. doi: 10.3758/s13428-017-1008-1 [DOI] [PubMed] [Google Scholar]
63.Hayes TR, Petrov AA. Mapping and correcting the influence of gaze position on pupil size measurements. Behav Res. 2nd ed. 2015;48: 510–527. doi: 10.3758/s13428-015-0588-x [DOI] [PMC free article] [PubMed] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0261463.r001

Decision Letter 0

Manuel Spitschan

13 May 2021

PONE-D-21-12636

The Confounding Effects of Eye Blinking on Pupillometry, and a Remedy for Them

PLOS ONE

Dear Dr. Lee,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Thank you for submitting this article, which has now been reviewed by three reviewers. All three reviewers are generally positive about the work, and make concrete suggestions. All points raised by the reviewers should be addressed comprehensively in your revision.

Please submit your revised manuscript by Jun 27 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Manuel Spitschan

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

3.Thank you for stating the following in your Competing Interests section:

"No"

Please complete your Competing Interests on the online submission form to state any Competing Interests. If you have no competing interests, please state "The authors have declared that no competing interests exist.", as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now

This information should be included in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: # Review

Yoo, Ahn, & Lee (2021) The confounding effects of eye blinking on pupillometry, and a remedy for them. *PLoS ONE*.

---

In this article, the authors describe the blink-induced pupil response (BPR), which is a pronounced pupil constriction from about 200 to 3000 ms after a blink. They point out that this not only increases noise when measuring pupil size, but is also likely to introduce confounds when blink rate covaries with the experimental conditions (as it often does). They then outline a technique to remove the PBR from the pupil time series, and show that this increases statistical power.

I feel that this is a very well-done study. It's well-written, despite some sentence-level English mistakes (but the structure is well done). And the aim and conclusions are clear and reasonable. I have a few suggestions to further strengthen the manuscript.

The introduction does not address the question of *why* the pupil constricts after blinks. In a sense, this is not important, because in the context of the current manuscript it's mostly relevant that it does (and not necessarily why). However, I suspect most readers will still be left wondering, and so I would add a short paragraph about this. I know this probably comes across as a cheap excuse to ask for a citation (I don't mean it as such), but I speculated on this in a recent review article, in which I basically suggest that it's the same phenomenon as the pupil constriction that is induced by visual change:

- <https: 10.1146="" annurev-vision-030320-062352="" doi="" full="" www.annualreviews.org="">

More importantly, I feel that the manuscript would benefit from practical guidelines for researchers to actually apply this method. It seems that the OSF repository includes code to conduct the analyses, or at least there's a `codes.zip` file. But this file seems to be corrupted, and in any case the organization of the OSF and to what extent it contains useful code is unclear. So cleaning up and documenting the OSF project would be low-hanging fruit.

And what kind of practical considerations should researchers keep in mind? For example, when using an EyeLink, you generally start recording at the start of a trial, and then end recording at the end of the trial. This means that there's no continuous pupil time series that spans multiple trials. To what extent is this a problem for applying the proposed technique? More generally, a paragraph with some hands-on advice would be worthwhile, in my opinion.

---

Sebastiaan Mathôt \\

Department of Experiment Psychology \\

University of Groningen \\

<http: smathot="" www.cogsci.nl=""></http:></https:>

Reviewer #2: In the present study the authors explain the BPR problem, the possible artifact that it might cause and also an algorithm to solve it. While the main idea is important and should be discussed and solved, I found three main issues that should be addressed before publication. Please see my comments attached in a PDF file.

Reviewer #3: The Confounding Effects of Eye Blinking on Pupillometry, and a Remedy for Them

===================================================

The manuscript describes the phenomenon of the blink-locked-pupillary response (BPR), which is the transient pupil constriction that occurs after an eye blink. In cognitive pupillometry research, the BPR is often overlooked completely or regarded simply as a nuisance artefact that must be removed from analysis; but here the authors suggest that the BPR is a 'serious confounder' with the potential to reduce statistical power or to seriously undermine the validity of pupillometry experiments. The manuscript first explores the general shape of the BPR in a simple fixation task and highlights its variability with respect to luminance, individual subjects and trials. With data simulations, it is subsequently demonstrated how the BPR can confound the pupillary signal in the two cognitive tasks (auditory oddball and delayed orientation estimation). Finally, a novel generative model is proposed to account for the BPR and then applied to 'de-confound' the pupil signal of the BPRs influence in the cognitive tasks. When compared to the common standard of using linear interpolation, the model-based correction is shown to substantially improve statistical power.

This is an important contribution to cognitive pupillometry methods, especially given that only one previous study has experimented with a modelling approach to correct for the BPR. The convincing demonstration of improved statistical power would however be more exciting if I felt that the BPR modelling approach–described by the authors as 'The proper way of correcting pupillary measurements for BPR'–would be easy to implement on my own data. The author's refer to a 'BPR correction toolbox' throughout the manuscript, but it remains unclear whether this is actually something that is being offered. What does a researcher have to do to apply this method to their own data? If it is overly complicated, I confess that I would probably stick with linear interpolation and incur the penalty to statistical power.

On the topic of linear interpolation, it may be worth acknowledging that some approaches do attempt to account for the BPR on a blink-by-blink basis. For example, I've previously processed EyeLink data with a toolbox called cili (https://github.com/beOn/cili), which has a function to extend blink end points to the first sample where the z-scored rate of change of the pupil time course drops below 10% of the average within a 100 ms moving window. This does a fairly good job of drawing a line across the BPR and is very easy to implement.

I therefore have the following general recommendations:

1. Describe the steps that a researcher must take in order to implement this approach on their own pupil data, and indicate whether a 'BPR correction toolbox' will be made available. If there is not going to be a toolbox and it is time-consuming to implement, this should be listed as a limitation.

2. Indicate whether there are some situations that this modelling approach would be impractical.

3. Acknowledge alternatives to dealing with the BPR, such as the linear interpolation approach on a blink-by-blink basis that I mentioned above.

Is the manuscript technically sound, and do the data support the conclusions?

---------------------------------------------------------------------------------------

The manuscript appears technically sound with data that support the conclusions. I have the following minor suggestions for improvement:

1. With EyeLink systems, optical distortion of the pupil image (caused by changes in gaze position) affects the pupil data by up to 10%. This should be acknowledged, and ideally a simple analysis provided showing that participants performed the fixation task adequately. It would also be helpful to indicate whether gaze related trial exclusion criteria were applied for the fixation and oddball tasks.

2. Please clarify whether any of the participants took part in more than one of the three studies outlined in the manuscript.

3. Please indicate whether a 'BPR toolbox' will be made available, and if so, where it is maintained and and how to use it.

4. Corneal flux density (c) is included as a parameter in the model. Does this mean it is necessary for researchers to obtain their own measurements of corneal flux density in order to accurately model the BPR?

Has the statistical analysis been performed appropriately and rigorously?

---------------------------------------------------------------------------------

The analysis appears to be in good order, but I am not an expert in modelling and therefore do not feel qualified to make comments on the finer details of the analysis. My general feeling is that this may be overcomplicated for the routine analysis of pupillometry data.

Minor points:

1. The meaning of the small lines on the x-axis in Figure 2 should be clarified in the figure caption. I assume they are indicating the time-to-peak for the respective traces?

2. The readability of the figures could be improved by increasing the DPI and possibly font size.

Have the authors made all data underlying the findings in their manuscript fully available?

----------------------------------------------------------------------------------------------------

Yes, all code and data are available on the OSF.

Is the manuscript presented in an intelligible fashion and written in standard English?

-----------------------------------------------------------------------------------------------

The manuscript is presented in an intelligible fashion but lapses frequently in grammar and phrasing, sometimes tone. With this in mind, I recommend additional proof-reading and copy editing. Special attention should be given to the abstract, figure captions and the Materials and Methods Section.

Example minor points:

1. Line 45 - BPR is defined here as blink-locked pupillary response, but was defined as as blink-induced pupillary response in the abstract (line 22).

2. Line 61 - (about xx in z-score unit) - Estimated value is missing

3. Line 563 - 'eyet racker'

4. Line 580 - 'Fixaion'

5. Line 626 - Review this sentence

6. Line 706 - 'we collected acquired'

7. Lines 457-460 - review sentence

8. Line 481 - 'In average'

9. Line 502 - 'vicious confounder' - revise word choice?

10. Line 564 - 'corneal reflex' - shouldn't this be 'corneal reflection'?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Sebastiaan Mathôt[Review attached as PDF]

Reviewer #2: Yes: Ronen Hershman

Reviewer #3: Yes: Joel T. Martin

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachment

Submitted filename: review.pdf

Click here for additional data file.^{(94.4KB, pdf)}

Attachment

Submitted filename: PONE-D-21-12636.pdf

Click here for additional data file.^{(301.4KB, pdf)}

PLoS One. 2021 Dec 17;16(12):e0261463. doi: 10.1371/journal.pone.0261463.r002

Author response to Decision Letter 0

11 Jul 2021

We previously submitted a manuscript titled “The Confounding Effects of Eye Blinking on Pupillometry, and a Remedy for Them” as a Research Article to PLOS ONE.

Our manuscript had been reviewed, and the editor asked us to revise the original manuscript by addressing the reviewers’ comments. We thank the reviewers for their constructive comments on the previously submitted version of our manuscript.

The co-authors and I carefully studied the reviewer’s comments, and carried out additional analyses on the data and conducted further literature search to fully address the concerns raised in those comments. Based on these new analyses and literature search, we revised our manuscript, including Abstract, Introduction, Results, Discussion, Materials and Methods, with an additional figure and several additional panels in the old figures. We also modified the title of the manuscript (newly titled “The Confounding Effects of Eye Blinking on Pupillometry, and their Remedy”).

Thanks to the reviewers’ comments, we believe that the revised manuscript now presents our findings on more solid methodological grounds and in a form more accessible to the researchers in the field of cognitive pupillometry than its previous version.

Please see the attached file (titled "Responses to Reviewers") for our point-by-point replies to the reviewers' comments.

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(46.8KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0261463.r003

Decision Letter 1

Manuel Spitschan

6 Oct 2021

PONE-D-21-12636R1The Confounding Effects of Eye Blinking on Pupillometry, and their RemedyPLOS ONE

Dear Dr. Lee,

==============================

Please respond to the queries from Reviewer #2.

==============================

Please submit your revised manuscript by Nov 20 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Manuel Spitschan

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

I would kindly ask you to respond to the remaining questions from Reviewer 2.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: (No Response)

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: Yes

Reviewer #2: Partly

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Reviewer #1: I feel that the authors have adequately addressed all my comments. This is a very useful manuscript for pupillometry researchers.

Reviewer #2: I have read the authors' responses and the revised manuscript and find that most of my concerns are still there. Moreover, it made me wonder about additional issues that I didn’t notice in the first version. Please see my comments to the authors in the attached PDF.

Reviewer #3: Having read the revised manuscript I feel that all of the comments were addressed and that the paper should be accepted for publication. I was also able to run the MATLAB toolbox tutorial without issue.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Sebastiaan Mathôt

Reviewer #2: Yes: Ronen Hershman

Reviewer #3: Yes: Joel T. Martin

Attachment

Submitted filename: review_PONE-D-21-12636R1.pdf

Click here for additional data file.^{(702.9KB, pdf)}

PLoS One. 2021 Dec 17;16(12):e0261463. doi: 10.1371/journal.pone.0261463.r004

Author response to Decision Letter 1

29 Nov 2021

General comments ([G1] ~ [G5])

“I have read the authors' responses and the revised manuscript and find that most of my concerns are still there. Moreover, it made me wonder about additional issues that I didn’t notice in the first version.

As I mentioned in my previous review, while the main idea is important and should be discussed and solved, there are several issues that should be addressed before publication should be considered.

[G1] The most important issue is the validity of the algorithm. Specifically, the authors presented partial analysis (i.e., only for two experiments) that estimated the efficiency of their algorithm. [G2] Another important issue is about the way they estimated the efficiency of their algorithm; namely, what they compared (or didn’t compare). In my opinion, there are difficulties that make the conclusion unclear. [G3] The third main issue (and maybe the most serious one) is about the methodology that the authors used in their studies. Specifically, it seems that the second experiment had short durations that cannot be used in pupillometry studies (due to lack of time to get back to baseline). [G4] In addition, it seems there are several methodological issues in their analysis; specifically, the smoothing that might destroy valid data, comparison that cannot provide information about the influence of the manipulation on the pupil size, and possibly wrong blink corrections. These are all a little bit problematic in a study that aims to solve a methodological issue. [G5] Another important issue is that the authors didn’t provide any real indication of the importance of the BPR correction. Yes, indeed, statistical power is highly important, but without a clear-cut example for the problematic uncorrected BPR, I can’t see any reason to adopt the authors’ algorithm. In other words, if it is impossible to present an example that will show how the correction helps researchers, there is no actual reason to use it.”

Our replies to the general comments ([G1]~[G5]): As indicated right below, the reviewer here gives a brief and general summary of the key issues that will be specified in the following specific comments. Specifically, [G1] and [G2] appear to be specified mainly in his comment labeled with [C2-1]; [G4] in [New15]; [G5] in [C2-2]. Thus, our replies to [G1]~[G5] will be addressed in our point-by-point replies to those specific comments.

Although we appreciate (and agree with) a few of the “comments” newly made in the second round of revision and revised the manuscript accordingly, we, in general, do not agree with the reviewer’s main assertions summarized in [G1]~[G5]. The corresponding grounds for our disagreement with [G1]~[G5] will be presented in our replies to the reviewer’s specific comments, which consist of the reviewer’s rebuttals to our previous replies to the reviewer’s comments in the first round of review (a total of 5 comments, [C2-1], [C2-2], [M2-3], [M2-4], and [M2-6]) and the reviewer’s new comments in the second round of review (a total of 5 comments, [New1]~[New17]).

However, [G3], the comment about the second (auditory oddball detection) experiment was not specified in any of the following specific comments. Thus, here we provide our reply to [G3].

[G3]: “The third main issue (and maybe the most serious one) is about the methodology that the authors used in their studies. Specifically, it seems that the second experiment had short durations that cannot be used in pupillometry studies (due to lack of time to get back to baseline).”

Our reply to [G3]: We do not understand why the second experiment threatens the validity of our work. We would like to remind the reviewer of the purposes of the second experiment, as written in the current manuscript: (1) “To specifically illustrate how BPR can confound the pupillary signal of interest, we measured the pupil size while subjects were performing two cognitive tasks, ‘auditory oddball detection’ and ‘delayed orientation estimation’ tasks.” (the second paragraph of the Results section titled “BPR, not just a nuisance but a confounder”); (2) “As the second step of verifying the effectiveness of the correction algorithm, we applied it to the data acquired in the auditory oddball task.” (the first paragraph of the Results section titled “Correcting the auditory oddball data for BPR”. As explicitly stated in (1), the first purpose of the auditory oddball detection experiment was to concretely show an example case in which spurious increments or decrements appear depending on the timing of BPR when researchers design experiments with a short inter-trial interval and adopt the practice of adapting the pupillary time course to the pupil size at trial onset. Importantly, this design and practice have been exercised in previous studies, including those cited in the current manuscript. For example, Hong and his colleagues (Hong L, Walz JM, Sajda P (2014) Your Eyes Give You Away: Prestimulus Changes in Pupil Diameter Correlate with Poststimulus Task-Related EEG Dynamics. PLoS ONE 9(3): e91321.) measured the pupil size with an auditory oddball task in which ITI was 2 or 3 seconds and quantified the evoked pupil responses by adapting the trial-chopped time course of pupil size to the trial onset (as seen in their Figure 3). There have been several pupillometry studies like Hong et al. (2014), such as Keute et al. (2019) (Keute, M., Demirezen, M., Graf, A., Mueller, N. G., & Zaehle, T. (2019). No modulation of pupil size and event-related pupil response by transcutaneous auricular vagus nerve stimulation (taVNS). Scientific reports, 9(1), 1-10.) and Nakakoga et al. (2021) (Nakakoga, S., Shimizu, K., Muramatsu, J., Kitagawa, T., Nakauchi, S., & Minami, T. (2021). Pupillary response reflects attentional modulation to sound after emotional arousal. Scientific reports, 11(1), 1-10.).The second purpose of the auditory oddball detection experiment was to demonstrate the effectiveness of our BPR correction algorithm by showing that the spurious decrements and increments due to BPR almost disappeared when the BPR correction algorithm was applied. Thus, the data and analysis outcomes of the auditory oddball detection experiment supported the claims we make in the current study. Then, why are the data and the analysis carried out on the auditory oddball experiment “the most serious” problems?

Specific comments originating from the first round of reviews ([C2-1], [C2-2], [M2-3], [M2-4], and [M2-6])

As I mentioned above, I read the author's comments and the revised manuscript and would now like to refer to specific difficulties that I found in their responses and in the manuscript. First, I would like to address the authors’ responses to the main comments (the identification of the comment is what the authors used in their response).

[The issues related to the major point [C2-1] in the first-round of review/revision]

(To clarify the origin of issues and track the past exchanges between the reviewer and us, we copied and pasted the corresponding parts from our first-round point-by-point replies.)

The reviewer’s original comment [C2-1]: “In the present study the authors explain the BPR problem, the possible artifact that it might cause and also an algorithm to solve it. While the main idea is important and should be discussed and solved, I found three main issues that should be addressed before publication.

1. The authors present and analyze the pupil size that was changed due to the BPR. However, it’s

not clear whether eye blinks were caused / associated with the observed pattern. For example, in

Fig. 5C it seems that there are dips after about 5 seconds and 20 seconds. These dips looks pretty

much the same as other dips that were associated by the authors to eye blinks and therefore to

BPR. Moreover, it seems that there are oscillations in pupil size that cause dipping about every 5

seconds (0.2 Hz). There are several possible options to solve this problematic issue. One possible

options is to compare two groups – one group that will be allowed to blink, and another group

that won’t be able to blink (by using dedicated eye drops). Another possible option is by

comparison between trials / time windows with blinks and those without blinks. If the authors

find meaningful differences between the groups, the interpretation of the original results will be

more reliable. Without this comparison it is difficult to understand the weight of this possible

artifact in pupillometry studies.”

Our reply to [C2-1] in the first-round revision: We agree with the reviewer that the weight of BPR in pupillometry would be better appreciated if there is a more straightforward demonstration of the effects of BPR correction. We thank the reviewer for suggesting two possible options for making such a demonstration. We reasoned that the second option, i.e., comparing the ‘blink-affected’ trials and the ‘blink-free’ trials, is more suitable than the first option, i.e., comparing the ‘blink-allowed’ condition and the ‘blink-suppression’ condition. Of course, the first option requires additional data collection. However, a more important reason for preferring the second option to the first one is our concern that active suppression of blinks is likely to distort the genuine time course of pupil size (see a new section of DISCUSSION in the revised manuscript (“Practical considerations for avoiding BPR-related problems”, which was added as our reply to one of the Reviewer #1’s comments ([C1-4])). So, we carried out the second analysis on the auditory oddball detection data (wherein individual trials are brief (2 s) and thus ideal for trial-to-trial analysis) and summarized the results with a new paragraph (the fourth paragraph under the RESULTS section titled “Correcting the auditory oddball data for BPR”) and a new figure (Figure 6) in the revised manuscript.

We note that the reviewer, while raising a possibility that the results shown in Figure 5 might not indicate the association (or causal relationship) between blinks and BPR, pointed out the wavy modulation of pupil size in the BPR-corrected time course of pupil size (“For example, in

Fig. 5C it seems that there are dips after about 5 seconds and 20 seconds. These dips looks pretty

much the same as other dips that were associated by the authors to eye blinks and therefore to

BPR. Moreover, it seems that there are oscillations in pupil size that cause dipping about every 5

seconds (0.2 Hz).”). This wavy time course was referred to again in one of the minor points of the reviewer (see [M2-5] in the below: “5. Lines 603-609: Why were the fixation colors different between trials? With respect to the first major point, maybe these contrasts were the reason for the 0.2Hz oscillations.”). We were quite perplexed with this comment and cautiously guess that the reviewer was confused about the time scale at which the time course of pupil size is described: the time unit used in all the plots of Figure 5 was “second” and thus the wavy modulation occurs not at 0.2 Hz but at 2 Hz. If we misunderstood the reviewer’s point, please let us know. However, more importantly, the reviewer’s pointing out this wavy modulation (and associated possible suspicion that comes with it) made us think that it’s better to offer some explanation for the wavy curve (we thank the reviewer for that). Although not 100% sure, we think that the first and second peaks are likely to reflect the previously known pupil responses that are associated with a sound stimulus and a manual response to it, given that subjects heard a sound and judged the tone of the sound by pressing one of the two assigned keys each and every trial. This conjecture is in line with the results that the blink-free trials (the gray lines in Figure 6A in the revised manuscript) and the corrected blink-affected trials (the green line in Figure 6A) both exhibited the double-peak pattern at 2 Hz. And we looked up the literature on pupillometry experiments with the auditory oddball task and found that the wavy pattern has been reported in the studies where subjects pressed a button on each trial like in our study [1,2]. In the revised manuscript, we offered our interpretation for this wavy modulation at 2 Hz in the fourth paragraph of the RESULTS section titled “Correcting the auditory oddball data for BPR”.

The reviewer’s rebuttal to our reply to [C2-1] in the first round of revision: “[C1-2](probably mislabeling [C2-1] with [C1-2]): I don’t really understand the rationale behind the authors’ test. In each trial there were two options: there was a correction of BPR (by using the authors’ algorithm) or there was no correction. Why did the authors choose a different separation of trials? Moreover, in my opinion the analysis should be applied to all the experiments (to support the existence of the problem) and not only to the first two experiments. In addition, the comparisons have to be identical across experiments in terms of separation criteria (related to [G1][G2]). Actually, the presented analysis (see more comments about the presented analysis below) does not answer the important question: Does the authors’ algorithm solve the problem?

I think I wasn’t clear in my previous review so I’ll try to explain it now. In Figure 4B (the one with a time window of 35 seconds), there is a decrement of the pupils about every 5 seconds = 0.2 Hz. Hz means occurrences per second. 2Hz means 2Hz occurrences per second. Here there is less than one occurrence per second. Hence, the frequency has to be smaller than 1. In Figure 4B there is one occurrence every 5 seconds, namely 1/5s = 0.2Hz. These decrements (or oscillations in the terminology of the original comment) are very strange and made me think maybe it might explain part of the results of the authors. These decrements (as far as reflected in the figure) are not associated with eye blinks (because their number is larger than the number of the observed eye blinks). If this is the case, I’m not entirely sure what the meaning of the analysis is and moreover, what the validity of the present study is.”

Our re-reply to the reviewer’s rebuttal to our reply to [C2-1] in the first round of revision (Although the reviewer labeled this rebuttal with [C1-2], he seemed to mean [C2-1] by it when the context was considered and because [C1-2] refers to the second major comment of the other reviewer (Reviewer 1).):

Regarding the issue of “rationale of separation of trials”. The reviewer repeatedly stated that it is difficult to understand the rationale behind our comparing the BPR-affected time course of pupil size against that of the BPR-free time course of pupil size before and after correction, respectively. We have a clear rationale for this comparison, and there is a reason why we thought this comparison is appropriate for the fixation and the auditory oddball experiments but not so for the delayed orientation estimation experiment, as follows.

One important merit of the data set acquired from the first two experiments (i.e., fixation and auditory oddball detection) is the fact that an effective ground-truth time course of pupillary responses, which are free from BPR, can be approximated by the average of the sufficiently long periods during which blinks did not occur (see Materials and Methods for the detailed description of BPR-free time courses). It is important to compare the BPR-affected time course of pupillary responses against this ground-truth time course before and after, respectively, the correction for BPR because it allows us to verify whether our correction algorithm selectively removes the pupillary responses associated with BPR but not those associated with other components, including spontaneous fluctuations. In this regard, note that the blink-free, ground-truth time course of pupillary responses might not be necessarily flat because there could be many uncontrolled factors that potentially affect the pupillary responses (e.g., gradual decrement or increment in pupil size over time due to fatigue or arousal, or peculiar fluctuations associated with task structure). In such cases, the simple comparison of the pupillary responses before and after the application of BPR correction (, which seems to be suggested by the reviewer based on his comment, “In each trial there were two options: there was a correction of BPR (by using the authors’ algorithm) or there was no correction. Why did the authors choose a different separation of trials?”) does not provide the sufficient information about whether the algorithm successfully corrected the pupillary responses only for BPR, which is the exact goal of the BPR correction algorithm. In the revised manuscript, we added several sentences by explicitly stating why this comparison is required both for the fixation experiment (the last paragraph of the Results section titled “Model-based correction of pupillary responses for BPR”) and for the auditory oddball detection experiment (the fourth paragraph of the Results section titled “Correcting the auditory oddball data for BPR”).

The reviewer also asked why we didn’t apply this comparison for the delayed orientation estimation task. There were two main reasons. First, the trial structure was not suitable for this type of comparison. Unlike the other two experiments, a single trial lasted for more than 10 seconds (including the phase of confidence reporting and inter-trial interval; see Methods & Materials), which made it virtually impossible to find blink-free trial because blink occurs typically several times during single trials. Second, the main purpose of the delayed orientation estimation experiment was to demonstrate (i) a case in which BPR can directly confound the independent variables of interest and (ii) our algorithm can be effective in de-confounding the pupillometry data by selectively filtering out BPR component. That’s why we focused on comparing the differences between the different levels of the independent variables before and after BPR correction. As mentioned below, what’s important is the fact that we used the same algorithm consistently and successfully demonstrated that our algorithm can be applied for BPR correction in two example experimental situations where our algorithm helps experimenters to get rid of spurious increments and decrements in pupil size (in the auditory oddball detection experiment) and de-confound the confounded pupillometry signals (in the delayed orientation estimation experiment).

Regarding the issue of “applying the same analysis to the three experiments”. We do not agree with the reviewer’s assertion that trials should be separated exactly in the same way for all the experiments. We neither agree that the same analysis should be applied to all the experiments. What’s important is - not to apply the one and only analysis repeatedly to different data sets - but to choose a type of analysis that is appropriate for a given purpose.

To be sure, we applied the same algorithm to the three sets of data (namely, the fixation, auditory oddball, and delayed orientation estimation task) but wanted to demonstrate, firstly, the diverse contexts in which BPR occurs and acts as confounders and, secondly, that our proposed algorithm is effective in correcting the pupil time courses for BPR under those different contexts. The three data sets were acquired with different purposes and can be examined using different analyses or comparisons that best serve their respective purposes. We do not believe there is such a rule dictating that one and only type of analysis should be applied to the data sets belonging to a given study.

In the fixation experiment, there was no cognitive task except for constant (i.e., time-invariant) fixation, which creates an ideal situation where the effective ground-truth time course that is affected neither by BPR nor by task-related responses can be acquired and utilized as a reference for verifying the effectiveness of the algorithm. That’s the reason why we opted to compare the BPR-corrected time course to the effective ground-truth time course (as described in the last paragraph of the Results section titled “Model-based correction of pupillary responses for BPR” and Fig 4C,D). Unlike in the fixation experiment, subjects performed specific cognitive tasks in the remaining two experiments, namely the auditory oddball detection and delayed orientation estimation tasks. The two experiments create quite different contexts, especially in trial length. Because the trial length is shorter than the length of BPR (>3 sec) in the auditory oddball experiment, we predicted, via simulation based on our generative model (Fig 4A,B), that the practice of adapting the trial-chopped pupil size time courses to the pupil size at trial onset would generate spurious upward or downward fluctuations due to BPR depending on when a blink occurs unless the raw time courses are corrected for BPR. That’s why we separated four consecutive trials into two different sequences ([0 0 1 0] and [0 0 0 1]) and tested whether the proposed algorithm can get rid of such spurious fluctuations (as described in Fig 4C). Note that this type of analysis is neither appropriate nor possible in the fixation or delayed orientation estimation experiments because there was no trial structure in the former and the trial length was much longer than the length of BPR in the latter (as mentioned above). Next, in the delayed orientation estimation experiment, we focused on a context where the blink rate is associated with the main cognitive variable, i.e., working memory load – the number of memoranda to keep in mind. Here, we focused on the task phase in which such an association was apparent (right after estimation onset, when the confounding due to BPR are expected to occur, as shown in the top panel of Fig 3D and Fig 9A), predicted, via simulation based on the generative model (Fig 4C,D), that the association between blink rate and memory load would underestimate the changes in pupil size between the different memory load conditions, and tested whether the proposed algorithm can address such underestimation due to BPR (as described in Fig 9).

Put together, we demonstrated the effectiveness of the proposed algorithm in different contexts by applying the analyses that were tailored to those contexts. Contrary to what the reviewer asserted, the approach taken by the current work has a merit, rather than a problem, by showing that it can be applied to different situations to address various problems due to BPR. We could not find a reason to follow reviewer’s strict request that “the [same] analysis should be applied to all the experiments”. It is “an algorithm” but not “an analysis” that must be applied consistently.

Regarding the issue of “0.2 Hz fluctuations”. On a separate note, we should stress that the reviewer initially (on the first round of review) referred to Fig 5C, not to Fig 4B, when describing 0.2 Hz fluctuations (look above (The reviewer’s original comment [C2-1]) for those highlighted in red). In our previous reply (Our reply to [C2-1] in the first-round revision), we couldn’t find 0.2 Hz fluctuations in Fig 5C and instead guessed that the reviewer might have referred to 2Hz ripple-like fluctuations. From our perspective, this issue comes as completely different issues. We clarify that this is not our fault but the reviewer’s fault, which shouldn’t be tersely excused with a comment like “I think I wasn’t clear in my previous review.” We spent a lot of time due to the reviewer’s mistake, being puzzled to figure out what the reviewer means by that. Unfortunately, we now have to deal with another totally different issue raised by the reviewer (, again from our standpoint to be sure).

Anyway, we now understand what the reviewer implies regarding Fig 4B. However, we do not agree with the reviewer’s assertion that the presence of decrements that are not associated with blinks threaten the validity of our work. In the current work, we never claimed that our algorithm can fix all the problematic fluctuations existing in pupillometry data (e.g., fluctuations associated with saccadic or micro-saccadic eye movements). Instead, what we do claim in the current work is that (i) blinks are followed by a short-pipe-shape fluctuation of pupil size (BPR), which varies in amplitude across blinks and in shape across individuals; (ii) BPR is not just a nuisance but a confounder because blinks do not randomly occur but can be associated with task phases or experimental conditions; (iii) the proposed model-based algorithm can effectively (we never said ‘perfectly’ or ‘efficiently’, to be sure) remove BPR without affecting non-BPR components in pupillary responses. We stated clearly throughout the entire manuscript that these three are our main claims, i.e., “the meaning of the analysis”. The decrements that are not associated with blinks might be due to (micro)saccades or spontaneous pupil fluctuations (we do not know). However, their presence and addressing them are simply beyond the scope of the current study. Thus, we do not find a reason why the reviewer thinks the presence of the decrements, which are based on the reviewer’s anecdotal and subjective eyeballing observation, threatens “the validity of the present study”, and why the current study should address them.

By the way, we confirmed that the 0.2Hz – seemingly oscillatory – fluctuations, which the reviewer generalized from the sample time course of pupil size depicted in Fig. 4B, did not exist in the data collected in the fixation experiment: our power spectrum analysis did not indicate any peculiar increase in power at around 0.2 Hz (see the figure in the right). Line and shade indicate the mean and standard errors of the mean (SEM) across runs.

[The issues related to the major point [C2-2] in the first-round of review/revision]

The reviewer’s original comment [C2-2]:“2. In addition to the (relatively) steady state measurement of the pupils (the fixation task), the authors ran more two independent cognitive tasks. While the results of the fixation task were relatively replicated in these tasks, the main idea behind the possible artifact is still not clear. Yes, indeed, the mean was pretty much the same and the scattering was decreased (this makes sense with respect to the first main issue that caused interpolation of a lot of dips). However, the influence of this possible artifact on the actual effects unclear. From the study perspective, the BPR might cause systematic pupil decrement that is not associated with the task (this is a bit of a problematic statement due to the fact that the authors cited studies that associated eye blink rate with task difficulty), therefore, effects should appear (false positive) / disappear (false negative) after the BPR correction. If this is the case here, the authors should present these possible scenarios with / without BPR corrections. Specifically, on one hand, the authors should show effects (in terms of the actual task – e.g., easy vs. difficult) that will appear after correction, and on the other hand, the authors should show effects that will disappear after correction. Without these comparisons, the contribution of the correction is not clear (at least in cognitive tasks).”

Our reply to [C2-2] in the first-round revision: As the reviewer pointed out, it would have been a much more straightforward (or more dramatic) demonstration of the effects of BPR correction if we had shown that the effects of interest (i.e., ‘oddity’ and ‘memory load’ in the first and second cognitive experiments, respectively) “appear (false positive) / disappear (false negative) after the BPR correction”. However, the effects were already statistically significant even before the BPR correction in both of the cognitive experiments. That’s also consistent with what has been previously reported. That’s why we decided to focus on the power analysis and identifying the exact contribution of the BPR correction, which was to decrease the variabilities of pupil-size measurements at the subject-to-subject and blink-to-blink levels (as summarized in Figure 6 and 8 in the previously submitted manuscript). We stress that this specific contribution is highly consistent with the results in the fixation experiment, where we found that BPR substantively varied in both shape and amplitude across individuals and mainly in amplitude across blinks within given individuals. We also stress that what were depicted in Figure 6D and 8E in the old manuscript partly demonstrate what the reviewer wanted to see. That is, there is a certain range of trial numbers (or subject numbers in a realistic situation) wherein the effects of interest disappear (insignificant) before correction and appear (significant) after correction. This was one of the motivations for drawing those plots.

Having explained why we focused on the power analysis, we also understand why the reviewer requested a straightforward demonstration of “false positive” or “false negative”. Although we cannot carry out such a demonstration for the reason pointed out above, we thought that it would help readers ‘tangibly sense’ the contribution of the BPR correction if we visually summarize how much the BPR correction increases the effect size in the two cognitive experiments. Thus, in the revised manuscript, we added new figure panels wherein the mean trajectories of pupil size are shown for each level of the main variable and the time courses of t statistics are shown in parallel for the differences between the levels (Figure 7A and 9A) and added several sentences for describing the results (the fifth paragraph of the RESULTS section titled “Correcting the auditory oddball data for BPR” and the second paragraph of the RESULTS section titled “Correcting the delayed orientation estimation data for BPR”).

The reviewer’s rebuttal to our reply to [C2-2] in the first round of revision: “[C2-2]: If I understand correctly, the authors said that the BPR might not cause a statistical problem (in terms of type I / type 2 errors) but could cause a decrease the statistical power. Moreover, in a series of three experiments, they failed to show why the correction (that it is not clear to me whether it is valid or not) is good and important. Yes, indeed, statistical power is important and is the basis of any statistical test. However, in the presented work, I can’t see any reason to use it. This is because the problem (i.e., in what scenarios the BPR will cause trouble) is not clear or well defined, and because the correction wasn’t compared (convincingly) with different valid tools / cases without BPR correction.” (related to [G5])

Our re-reply to the reviewer’s rebuttal to our reply to [C2-2] in the first round of revision: We do not agree with the reviewer’s assertion that we “failed to show why the correction (that it is not clear to me whether it is valid or not) is good and important.” As we stated in re-replying to the reviewer’s rebuttal to our reply to [C2-1] in the above, we clearly demonstrated that our algorithm is effective in various contexts: (i) by comparing ‘the BPR-corrected time courses of pupil size’ with ‘the effective ground-truth time courses of pupil size’ (fixation experiment, Fig 4C,D); (ii) by demonstrating that the proposed algorithm can effectively remove the spurious fluctuations that arise from the customary procedure of adapting trial-chunked pupil time series to trial onset (auditory oddball experiment, Fig 5-7), and (iii) by demonstrating that the proposed algorithm can increase the differences between the conditions of interest by de-confounding the raw pupil-size responses from BPR (Fig 9). Based on these results, we’ve shown that our method of correction is not only important in that it deals with BPR that is prevalent and can potentially be a serious confounding factor in cognitive pupillometry experiments, but also good in that it effectively corrects the raw pupil size data for BPR under various contexts. Therefore, we do not understand on what basis the reviewer asserts that we “failed to show why the correction is good and important.”

Next, contrary to what the reviewer insinuated, the statistical power is intimately associated with type I (false positive, its probability being quantified by alpha) and II (false negative, its fraction being quantified by beta) errors because it becomes increasingly difficult to distinguish between null and alternative hypotheses as the statistical power decreases (Krzywinski, M., Altman, N. Power and sample size. Nat Methods 10, 1139–1140 (2013). https://doi.org/10.1038/nmeth.2738). Considering that cognitive neuroscience literature is infamous for a low level of statistical power (~0.2; Button, K., Ioannidis, J., Mokrysz, C. et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 14, 365–376 (2013). https://doi.org/10.1038/nrn3475), the issue of increasing statistical power is not just important in itself but also critically associated with type I and II errors. In this sense, the reviewer mischaracterizes our reply by stating “the authors said that the BPR might not cause a statistical problem (in terms of type I / type 2 errors) but could cause a decrease the statistical power” (, which is an impossible statement by itself for the reason mentioned above). We said such a statement neither in our reply nor in the manuscript. Actually, as we clearly stated in our reply, the main motivation of carrying out the simulation analysis in which we plotted how statistical power increases as the number of trials increases (Fig7E & Fig9E) was exactly to show, in a principled way, that our BPR correction method can substantially affect both type I and type II errors by increasing the statistical power. The statistical power matters and is often expensive to buy for empirical scientists.

[The issues related to the minor point [M2-3] in the first-round of review/revision]

The reviewer’s original comment [M2-3]: “3. Something doesn’t make sense in Fig. 7. After 4 seconds there are a lot blinks (the EBR is relatively large), but the pupil size is still increased (with / without corrections). How it is in line with the main idea of BPR? Why the pupil is sometimes increased and sometimes decreased?”

Our reply to [M2-3] in the first-round revision: First of all, it should be stressed that the pupil-size measurements are not just an outcome of BPR but also influenced by cognitive and spontaneous states, as we assumed in our generative model (as depicted in Figure 4A). We interpret that the pupil size increases after 4 seconds despite of the increase of EBR because the pupil size tends to increase when subjects are required to (‘cognitively’) retrieve the memory of the target orientation and make manual adjustments for estimation report. Both of these processes are known to increase the pupil size (Hong et al., 2014; Murphy et al., 2011; Robison and Unsworth, 2019). Of course, blinks will decrease the pupil size, but blinks do not occur not that often (~0.3Hz) and overshadowed by the rapid increase due to the cognitive demand. As demonstrated in Figure 9A, when BPR is corrected with our algorithm, the corrected time course was greater than the uncorrected time course. In the revised manuscript, we explained the rapid rise during the estimation epoch of the delayed orientation estimation task (the second paragraph of “Correcting the delayed orientation estimation data for BPR”).

The reviewer’s rebuttal to our reply to [M2-3] in the first round of revision: “The authors’ response is a bit problematic. If there are no decrements after any blink, why do the authors associate the decrements after blinks to blinks – this is the main idea of the study (unless I missed something). If the aim is to define the BPR (in terms of shape and latency), this issue should be of concern. Otherwise, it means that in several cases (that cannot be predicted – at least by me) the algorithm might destroy valid data.”

Our re-reply to the reviewer’s rebuttal to our reply to [M2-3] in the first round of revision:

We do not agree with the reviewer’s assertion, “If there are no decrements after any blink, why do the authors associate the decrements after blinks to blinks – this is the main idea of the study (unless I missed something).” As we clearly stated in our first-round reply to [M2-3], “it should be stressed that the pupil-size measurements are not just an outcome of BPR but also influenced by cognitive and spontaneous states, as we assumed in our generative model (as depicted in Fig 4A),” which is why cognitive scientists acquire pupil size measurements and read out such states from those measurements. Thus, there could be no visible decrements in the across-trial-averaged time course of pupil size if there is a counteracting increase in pupil size that is induced by cognitive factors. Again, as clearly stated in our previous reply, this is a very plausible scenario given that memory retrieval and estimation report “are known to increase the pupil size” and that “blinks do not occur that often (~0.3Hz) and overshadowed by the rapid increase due to the cognitive demand.” Thus, the fact that “there are no decrements after any blink” in the across-trial-averaged time course of pupil size does not necessarily negate the presence of BPR. We already revised the manuscript in the first round of revision to clarify this issue by explaining “the rapid rise during the estimation epoch of the delayed orientation estimation task (the second paragraph of “Correcting the delayed orientation estimation data for BPR”)”.

[The issues related to the minor point [M2-4] in the first-round of review/revision]

The reviewer’s original comment [M2-4]: “4. Lines 569-574: Was the exclusion (of both trials and subjects) also based on task performance? If not, why?”

Our reply to [M2-4] in the first-round revision: We did not exclude trials or subjects based on task performance. Task performance was generally good for all the participants, and we could not find any particular reasons to exclude based on task performance. Instead, we excluded the entire data sets from a few subjects when the recording quality was poor, when subjects dozed off during experiments, or when the blink rate was unusually high. We revised the “Subjects” section of METHODS and MATERIALS and stated that how many subjects are discarded from further data analysis on what basis.

The reviewer’s rebuttal to our reply to one of the minor points [M2-4]: “Performances should be added to support the authors’ claim.”

Our re-reply to the reviewer’s rebuttal to our reply to [M2-4] in the first round of revision: Yes, we provided the data that support our claim at the end of “Subjects” subsection of METHODS and MATERIALS in the revised manuscript. In both tasks, subjects in our study showed a reasonable range of performance in accuracy and RT, which were quite comparable to those reported in the previous work the same tasks. In the revised manuscript, we reported the mean±SD and range [min, max] both in accuracy and RT, with the previous work as references.

[The issues related to the minor point [M2-6] in the first-round of review/revision]

The reviewer’s original comment [M2-6]: “6. Line 653: How did the authors separate between eye-blinks and other artefacts (e.g., noise / head movements).”

Our reply to [M2-6] in the first-round revision: First of all, as we stated in the “Stimuli and procedure” section of METHODS and MATERIALS, we used a chinrest to avoid head movements and to keep the distance between the eyes and the eye-tracker. Thus, the outputs from the eye-tracker were not that noisy and blinks were readily discriminated from other miscellaneous background noises.

The reviewer’s rebuttal to our reply to one of the minor points [M2-6]: “Every device has noise. It means that any device might show extreme pupillometry values by mistake. My recommendation is to exclude extreme values by using Z-scores (in most cases 2.5 Z-scores) based on the mean and the SD of each trial separately. Please note that eye-movements can also cause missing values that might be associated accidently to eye-blinks (e.g., if the participant looks at the floor). In addition, if the participant touches his/her face (it might happen sometimes), missing values will be observed. Other scenarios that I can think about are associated with moving the head back. Yes, indeed, the chinrest is supposed to decrease the possibility of head movements, but it might happen. I highly recommend the authors check this, specifically in methodological work like this.”

Our re-reply to the reviewer’s rebuttal to our reply to [M2-6] in the first round of revision: We appreciate the reviewer’s thoughtful suggestion of excluding “extreme values by using Z-scores (in most cases 2.5 Z-scores) based on the mean and the SD of each trial separately.” However, we opted not to incorporate this suggestion for the following reasons. First, in our previously published work (Choe KW, Blake R, Lee SH. Pupil size dynamics during fixation impact the accuracy and precision of video-based gaze estimation. Vision Res. 2016 Jan; 118:48-59; Choe KW, Blake R, Lee SH. Dissociation between neural signatures of stimulus and choice in population activity of human V1 during perceptual decision-making. J Neurosci. 2014 Feb 12;34(7):2725-43), we learned that making head motions is extremely rare in our setup with a chin-rest. We did not observe that subjects were off the chin rest during the experiments. If they did, our gaze data would pick up that huge missing data. Additionally, we monitored and recorded subject’s gaze data throughout the entire experiments, but we haven’t observed any large-size shifts in gaze data that are expected by the head or eye or face-touching motions mentioned by the reviewer. Second, even if such large gaze shifts occurred during our experiments, our preprocessing procedure of band-pass filtering would catch such extreme values of changes and filter them out. Finally, and most importantly, we avoided the method of excluding data based on Z-scoring because we learned from the fixation experiment that the observed range of BPR amplitudes go beyond 1 mm, which will readily fall outside -2.5 in z score thus will be discarded. Thus, the Z-scoring procedure is likely to exclude the informative part of BPR-related data.

Specific comments originating from the second round of review ([New1]~[New17])

Beside my comments to the authors’ answers, I have a (relatively long) list of specific comments (the numbers represent the relevant line in the manuscript).”

[New1]: “30-38: This pupil-size dynamics, which is dissociated with light”: this is not absolutely true. Steinhauer found an interaction between them (Steinhauer, S. R., Siegle, G. J., Condray, R., & Pless, M. (2004). Sympathetic and parasympathetic innervation of pupillary dilation during sustained processing. International Journal of Psychophysiology, 52(1), 77-86).”

Our reply to [New1]: We thank the reviewer for directing us to this important reference. In the revised manuscript, we removed “which is dissociated with light” (the second sentence of Introduction).

[New2]: “152-155: I looked for studies about the “delayed orientation estimation” and pupillometry, and I didn’t find anything relevant. Moreover, the citation (44) is not relevant to this task. Hence, I’m not entirely sure why this task was chosen nor why the authors declared that “These two tasks were chosen because the cognitive processes underlying task performance are relatively well established, and those processes are known to be tightly associated with pupillary responses.”

Our reply to [New2]: First of all, we used the auditory oddball task and the delayed orientation estimation task because the cognitive processes underlying these tasks, such as ‘surprise by oddity’ and ’ working memory load’, are known to be tightly linked to pupil size. As pointed out by the reviewer, the references 44 and 45 are relevant only to the (auditory) oddball task. In the revised manuscript, we added old and new references (Kahneman D, Beatty J. Pupil Diameter and Load on Memory. Science. 1966;154: 1583–&. doi:10.1126/science.154.3756.1583; Bays PM. Noise in Neural Populations Accounts for Errors in Working Memory. Journal of Neuroscience. 2014;34: 3632–3645. doi:10.1523/JNEUROSCI.3204-13.2014; Denison, R.N., Parker, J.A. & Carrasco, M. Modeling pupil responses to rapid sequential events. Behav Res 52, 1991–2007 (2020)) as references associated with the delayed orientation estimation task, and modified the sentence a little.

[New3]: “156-173: I took a look at the cited paper and the authors in that paper made a different analysis (depending on the luminance). It is difficult to understand whether the results of the authors of the present manuscript are good or not in terms of replication.”

Our reply to [New3]: We are quite puzzled by this comment because the cited references (19, 29-31) were clearly in line with our results by reporting that the blink rate tends to decrease during stimulus presentation or during important task phases and increase during implicit break points. Specifically put, Siegle et al. (2008) showed (in their Figure 1a) that the blink rate decreased during the stimulation phase and then started to increase during the mask period, which can be considered as an implicit break point. Orchard et al. (1991) showed that the blink rate decreased during a cognitively important period (i.e., when fixation pauses during normal reading) and increased during the moment of line change, which can also be considered as an implicit break point. Nakano (2009) showed that the blink rate decreased when subjects viewed important movie scenes but increased when they viewed unimportant movie scenes, as the authors explicitly stated “we found the synchronization of blink timing within and across individuals at implicit breaks in video stories.” Lastly, Liao et al. (2016) showed (in their Figure 7) that the blink rate decreased during stimulus presentation and then increased afterwards. Having indicated the relevance of these references, we slightly modified the sentences considering that Orchard et al. (1991) and Nakano (2009) did not directly show the time course of blink rate as in our study (the last paragraph of the Results section titled “BPR, not just a nuisance but a confounder”).

We do not understand the part about luminance (“the authors in that paper made a different analysis (depending on the luminance)”) because we couldn’t find any indication that the authors in the referenced papers carried out different analyses depending on the luminance.

[New4]: “193-204: Again, it is difficult to understand if the results are good or not due to the fact that they weren't compared with previous results in terms of replication.”

Our reply to [New4]: It is true that, to our best knowledge, there is no previous study directly showing that the blink rate increases as a function of working memory load during the estimation/reporting phase in delayed estimation tasks although Rac-Lubashevsky et al. (2017) and Bochove et al. (2012) showed that the blink rate increases as a function of cognitive load during task switching and in incongruent trials of a flanker task, respectively. That’s the reason why we did not cite any references and instead used the expression “intriguingly” when describing the results. However, we do not agree with the reviewer’s assertion that the absence of previous report threatens the validity of our findings because it is simply the observed fact that the blink rate increased as a function of working memory load in our task, and this fact stands alone as an implication that BPR can potentially confound the main variable of interest (working memory load).

[New5]: “353-379: Something is a bit unclear in Fig 6A. It seems that the pupil size in the trials with BPR and without corrections was larger than in trials when no corrections were required. I expected to see a huge dip (due to the BPR) that would be fixed after the presented correction. However, it seems that during these trials, the pupils had no decrements that associated with BPR. If I understand correctly, this figure suggests that only one peak was found during these trials. It seems to me that something went wrong with the analysis of those trials. In addition, I'm wondering why this analysis wasn’t done on the third experiment also? In my opinion, the same test is supposed to be done on all the three experiments.”

Our reply to [New5]: To understand why the blink-affected time course of pupil size increases, rather than decreases, in Fig 6A, one needs to know the accurate definition of the “blink-affected trials.” The sentence in the fourth paragraph of the Results section titled “Correcting the auditory oddball data for BPR” of the submitted manuscript clearly defines what the “blink-free trials” and “blink-affected trials” are: “Considering that BPR lasts for about 3 s, the blink-free trials were defined as the trials whereby blinks occurred neither in the 2-back trial, in the 1-back trial, nor in the current trial while the blink-affected trials were all the rest of the trials (see Methods and Materials for details)”. That is, the blink-affected trials include a set of trials in which blinks occurred either in the 2-back trial, in the 1-back trial, or in the current trial. In the trials in which blinks occurred in the current trial, the effect of BPR appears as a dip. The effect usually affects only the second half part (1-2 s) of the trial (see 0-2 sec of the bottom panel of Fig 3B) since blink rate is concentrated at 0.5-1 s and (as described in the top panel of Fig 3B) and BPR has some delay (0.3 s) before a substantial decrease. In the trials in which blinks occurred in the 1-back trial, the BPR from 1-back trial typically reaches its negative peak at the onset of the current trial, so the baseline is spuriously decreased. As the pupil diameter at stimulus onset is set as baseline, BPR in the recovery (dilation) phase appears as a spurious increase of the first half part of the trial (see 2-4 sec of the bottom panel of Fig 3B). In the trials in which blinks occurred in the 2-back trial, the BPR affects similarly as the 1-back trial case, but with much smaller effect since BPR from the 2-back trial already substantially recovered prior to the current trial. The combination of the decrease in the last half part (from the trials in which blinks occurred in the current trial) and the increase in the first half part (from the trials in which blinks occurred in the 1-back and 2-back trial) eventually made spuriously exaggerated dilation of the baselined time course in this task. We anticipated these spurious patterns via simulation based on our generative model (Fig 5A-B), and the observed patterns supported (Fig 5C) confirmed this anticipation. The thick black line in Fig 6A, which is the same as the thick black line in the left panel of Fig 5C, is the observed time course of the “blink-affected trials”, which is quite similar to that predicted by the model simulation (the right panel of Fig 5B) and the mean of the spurious patterns (dashed and dash-dotted lines in the left panel of Fig 5C). Although we think the currently submitted manuscript provides an account for why and how the time course of the “blink-affected trials” increase (not decrease), we further elaborated this account in detail to help readers readily follow our account in the revised manuscript (the fourth paragraph of the Results section titled “Correcting the auditory oddball data for BPR”).

As for the comment regarding the issue of “why this analysis wasn’t done on the third experiment also?”, please read our reply to [The issues related to the major point [C2-1] in the first-round of review/revision] in the above.

[New6]: “454-466: There were differences between the conditions at time 0. Therefore, it is difficult to conclude that the observed differences after the manipulation (i.e., the presentation of the stimulus) were caused by the differences themselves. In my opinion, the authors should present relative changes (compared to the stimulus onset, for example?).

Moreover, at the trial offset (the beginning of trial n+1) the pupil size was not equal to that at the trial onset (the beginning of trial n). Due to the fact that trial n and trial n+1 are identical in terms of the presentation of the mean response, it is problematic that the trial onset wasn’t identical to the trial offset. It means that there was an influence of trial n on trial n+1 (in addition to the theoretical influence of the BPR). In other words, there was a methodological issue in the experiment. A study that deals with (relatively minor?) artifacts like BPR cannot use an experiment with a critical problem like that.”

Our reply to [New6]: By the way, we clarify that, as we explicitly wrote in the figure caption for Fig 9A, the ‘time 0’ demarcates not the stimulus onset but the onset of the estimation epoch. To address the reviewer’s point, we simply expanded the time window of plotting around from the time point when the sensory stimulus was masked, because we were interested in knowing how the peak amplitude in pupil size during the mnemonic period (i.e., the period from stimulus offset and estimation response, during which mnemonic representation is formed, maintained, and used for estimation) differ across the different memory load conditions. In this way, readers can see the developing time course of pupil size corresponding to the (mnemonic) period relevant to our purpose. In the revised manuscript, we modified the figure caption of Fig 9 and the main text (the second paragraph in the Results section titled “Correcting the delayed orientation estimation data for BPR”).

As for the mismatch between the trial offset and onset in pupil size in Fig 8, we clarify that the data at the end of the plot is not the data at the trial offset. As we described in Methods (lines 744-745) in the previous manuscript, the estimation epoch and the confidence report epoch varied in length from trial to trial depending on how quickly subjects estimated and judged their confidence (mean±SD and range for estimation duration, 2.11 ± 0.51 sec [min = 1.55, max = 3.85 sec]; mean±SD and range for confidence judgment duration, 0.77 ± 0.29 sec [min = 0.41, max = 1.54 sec]), resulting in varying trial duration across trials. This explains the superficial mismatch between the leftmost points and the rightmost points in pupil size in Fig 8. In the revised manuscript, we further specified the temporal structure of the delayed orientation estimation task (in Methods & Materials subsection titled “delayed orientation estimation task”).

[New7]: “504-522: The summary talks about the algorithm itself that wasn’t described at all in the manuscript. What is the purpose of the study? To describe the BPR issue? To address the BPR issue? Up until this part in the manuscript, no details about the algorithm were provided. How are the readers supposed to judge the algorithm?”

Our reply to [New7]: Because the details of the BPR correction algorithm were too lengthy and complex to be described in Results, we elected to give only a conceptual and intuitive description of the algorithm in the Results section titled “Model-based correction of pupillary responses for BPR” – just before testing its validity and effectiveness – and provide the details and corresponding mathematical equations comprising the algorithm in the Methods section titled “The algorithm of correcting pupillary responses for BPR”. We still believe that this option works best. Note that, in the submitted manuscript, we explicitly directed readers’ attention to this Methods & Materials section for the specifics and details of the algorithm (“By incorporating what we learned about BPR from the fixation task into the generative model as prior knowledge, we developed an algorithm inferring subject-specific BPR profile and blink-by-blink BPR amplitude by combining that prior knowledge and the likelihood function acquired from the observed pupil measurements (see Materials and Methods for the detailed description of the algorithm)”).

We believe that it is more appropriate and effective to state the purpose of the study in the Introduction, not in the first paragraph of the Discussion, which only recapitulates the gist of important findings in the study before delving into an in-depth discussion of those findings. Note that the motivation, purpose, and anticipatory summary of our study are compactly and explicitly provided in the last paragraph of Introduction.

[New8]: “610-618: The authors said that the other approach is not good enough because of the two mentioned crucial aspects. However, no comparison between the approaches was applied. If the authors suggest that their approach (that wasn’t discussed in the manuscript except for in the last section) is better, a comparison should be done. In my opinion, a comparison between the 2 approaches and between the data after the BPR corrections to trials with no BPR in the three experiments has to be done.”

Our reply to [New8]: We do not agree with the reviewer’s accusation: we never stated or indicated that “the other approach is not good enough”. Here we simply indicated in what aspects the other approach (Knapen et al., 2016) and our approach differ (“However, this approach differs from ours in two crucial aspects. First, … Secondly, … In addition,…”). We cannot claim that our approach is better than that approach in the current work because it’s difficult for us to evaluate Knapen et al. (2016)’s method because, as described in the text, it requires us to build an event matrix unlike ours and outcomes would be highly affected depending on the event matrix. For this reason, we opted to let readers know how our approach differs from Knapen et al. (2016)’s method, but not to claim the superiority of our approach over it.

[New9]: “625-627: This is a serious problem. It means that the algorithm might destroy valid data with a wrong interpolation. Isn't it safer to ignore the BPR?”

Our reply to [New9]: Although we exercised scientific rigor here by bringing up a theoretical possibility that the pupil time course of a certain cognitive event matches BRP exactly in timing and shape, such cases must be very rare. Our demonstration of increases in statistical power in the auditory oddball and delayed estimation task support this. In the revised manuscript, we added a sentence to indicate that such cases are very rare.

[New10]: “641-646: In my opinion, the “back to baseline” is a good reason for a relatively long ITI. As it is reflected in Exp. 2, this might also be problematic if there are no blinks (and BPR) at all.”

Our reply to [New10]: We agree with the reviewer. We incorporated the reviewer’s point by modifying this paragraph.

[New11]: “647-652: Please note that this is also really relevant in case of blink corrections. Actually, in general it is highly recommended to measure pupil size before the manipulation onset. As reflected from Fig. 9, there are differences in the manipulation onset. Actually, we don’t really know what was happening before the manipulation onset in terms of the pupil size. Therefore, it is difficult to conclude that the manipulation itself caused the results we see. Specifically, in addition to the comparison relative to the stimulus onset, the researcher has to be sure that any changes in the pupil size didn’t start before the manipulation itself. Hence, the recommendation about recording the data before the manipulation onset is not only relevant for BPR, but for pupillometry studies in general (also for studies when there are no eye blink at all by using special drags ...).”

Our reply to [New11]: We agree with the reviewer that it’s “highly recommended to measure pupil size before the manipulation onset.” Accordingly, we incorporated the reviewer’s point by adding a sentence in this paragraph. (However, See our reply to [New5] in the above regarding the reviewer’s extended comment on Fig. 9).

[New12]: “653-663: In contrast to RT studies (but not only), in pupillometry studies it is important to get data from the eyes. Hence, voluntary eye blinks (for a rest?) are not highly recommended. Honestly, I don’t believe that it is possible to avoid involuntary eye blinks (that are associated with all the aspects that the authors mentioned). Hence, I think this suggestion (avoid researchers to ask participants to avoid eye blinks) is wrong and might cause loss of data.”

Our reply to [New12]: We are a bit sympathetic to the reviewer’s comment: because subjects cannot completely suppress blinks even if experimenters ask them to suppress blinks, experimenters can maximize the blink-free data by instructing subjects to suppress eye blinking. However, we do not agree with the reviewer’s assertion that “this suggestion (avoid researchers to ask participants to avoid eye blinks) is wrong and might cause loss of data.” ‘An attempt of suppressing blinks intentionally’ itself, even if that’s not completely successful, comes with many cons as we listed in the manuscript: (1) loss and distortion of valuable blink data, (2) fatigue associated with blink suppression, and (3) creation of unwanted cognitive or brain states due to blink suppression. Therefore, we think it is better for experimenters not to say anything to subjects about blinks at all and preprocess BPR, as we showed in the current work. However, we admit that we have not compared the two conditions (instruct to blink naturally vs. suppressing blink), so we modified the expression from “do not recommend” to “recommend to consider not giving”. Note that the Discussion section titled “practical considerations for avoiding BPR-related problems” is a set of practical guides we can offer with the premise that our findings presented in the current work are valid. On that premise, we think it is reasonable to recommend that experimenters consider not giving the instruction of blink suppression.

[New13]: “701-702: The authors decided to not give the subjects any instructions about eye blinking. They also recommended other researchers do the same (as reflected from lines 653-663). Hence, it seems that there is a conflict of interest in their recommendation, specifically if they didn’t examine it directly; namely, if they did not compare between two groups that are different based on the instructions.”

Our reply to [New13]: We do not agree with the reviewer’s assertion that “there is a conflict of interest in their recommendation”. We did not give instruction about blinking for several reasons. Instructing subjects to suppress blinks is an act intervening in the distribution of spontaneous blinks. We intended to observe how pupil data is confounded by BPR from spontaneous blinks without intervention, so we did not give any instruction about blinking. Also, many pupillometry experiments are being conducted without instruction about blinking as well, so it shows well how pupillometry is confounded by BPR under natural and conventional circumstances. Additionally, there is a logical reason why we did not recommend instruction of blink suppression. Suppressing blinks can reduce BPR but induce other problems as we described. Therefore, we believe that it is better to let subjects blink naturally and preprocess BPR as much as possible, rather than instructing suppressing subjects’ blinking. To sum up, we have a reasonable ground about not giving instruction about blinking, and recommend other researchers to consider doing that, and we do not see any inconsistency or conflict of interest here.

[New14]: “763-766: Did the authors find meaningful differences between the two eyes? If the authors decided to use binocular measurement, why wasn’t the mean pupil size (of the two eyes) used for the analysis?”

Our reply to [New14]: Yes, there were differences in quality occasionally between the two eyes, and we opted to use the data from one of the two eyes that provided more reliable data for the following reasons. First, there were subtle but noticeable differences in blink offset timing between the two eyes. Because we wanted to define the time course of BPR precisely, we were concerned that averaging two signals that differ in timing might introduce unwanted blurs. Second, occasionally in some individuals, only one eye’s data was not detected or became unstable probably due to eyelid occlusion, which is known to occur more frequently in Asian people. If we average the signals from the two eyes, the time courses are expected to show weak but spurious fluctuations due to the missed signals from the unreliable eye. In the revised manuscript, we stated the reason why we opted to analyze one eye’s signal for pupil data (the first paragraph in the section titled “Data analysis”).

[New15]: “785-787: In my previous review I asked why the smoothing was required. I understand that the authors asked to reduce the noise of the data, however, smoothing with a boxcar time window of 250 ms not only reduces the noise, but also the signal itself. Actually, a boxcar time window of 10 ms should remove most of the noise. Therefore, it seems to me that the smoothing was too aggressive and in my opinion should be a new concern. EyeLink devices are well known to be high quality devices. Hence, smoothing is not really required in pupillometry studies. For reference, in our lab we are using the eye tribe devices (that have a sampling rate of 60 Hz and are well known to be relatively noisy devices). After blink correction, no smoothing is required. The reason that I mention it again (after I asked about it in my previous review) is due to the data in Figure 6. In this figure, the data looks too processed and it made me question whether the smoothing might have caused these patterns.” (related to [G4])

[New16]: “794-795: If the reason behind the smoothing is to remove noise, why was the band-pass filter also required? Was the smoothing used only for the detection of the blinks? It is not clear to me.”

Our reply to [New15] & [New16]: We clarify that the smoothing with a boxcar time window (0.25 sec) was applied only to define the endpoint of the post-blink artifact (for the purpose of being not too sensitive for detection), but was never applied to the pupil time course that was used for analysis. The data used for analysis were smoothed only by the band-pass filter (0.02Hz and 4Hz with the 3rd order Butterworth filter), which is often practiced in other pupillometry studies. To avoid confusion, we added a sentence to stress that the boxcar smoothing was not applied to the data used for analysis (section titled “The removal of artifacts around single blink events”).

As for the impression that “the data [in Fig 6.] looks too processed”, we think it looks smooth because, firstly, the lines represent the averages across many subjects (n=27) and many trials/subject (n=980~1,260 trials) and, secondly, (as we just mentioned above), the high-frequency noises were filtered out with the bandpass filter.

[New17]: “817-826: Similar to the general comment about the separation between trials, I don’t really understand this rationale. In each trial there were two options—there was a correction of BPR (by using the authors’ algorithm) or there was no correction. Why do the authors separate the trials differently?”

Our reply to [New17]: This is the same comment as [C2-1]. Please see above for our reply on this issue (Our re-reply to the reviewer’s rebuttal to our reply to [C2-1] in the first round of revision: Regarding the issue of “applying the same analysis to the three experiments”.

Attachment

Submitted filename: KyungEtal_2021_ResponsesToReviewers.pdf

Click here for additional data file.^{(312.1KB, pdf)}

PLoS One. doi: 10.1371/journal.pone.0261463.r005

Decision Letter 2

Manuel Spitschan

3 Dec 2021

The Confounding Effects of Eye Blinking on Pupillometry, and their Remedy

PONE-D-21-12636R2

Dear Dr. Lee,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Manuel Spitschan

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

PLoS One. doi: 10.1371/journal.pone.0261463.r006

Acceptance letter

Manuel Spitschan

9 Dec 2021

PONE-D-21-12636R2

The Confounding Effects of Eye Blinking on Pupillometry, and their Remedy

Dear Dr. Lee:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Manuel Spitschan

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 Fig. Individual’s BPR shape remains constant across runs.

(left panel) Split-half correlation in peak amplitudes (right panel) Split-half correlation in peak time.

(TIFF)

Click here for additional data file.^{(229.1KB, tiff)}

S2 Fig. Zoomed out diagonal plot for Fig 9B and 9C.

(TIFF)

Click here for additional data file.^{(2.4MB, tiff)}

S1 Video. Example of blink-locked pupillary response (BPR).

(MOV)

Click here for additional data file.^{(4.6MB, mov)}

Attachment

Submitted filename: review.pdf

Click here for additional data file.^{(94.4KB, pdf)}

Attachment

Submitted filename: PONE-D-21-12636.pdf

Click here for additional data file.^{(301.4KB, pdf)}

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(46.8KB, docx)}

Attachment

Submitted filename: review_PONE-D-21-12636R1.pdf

Click here for additional data file.^{(702.9KB, pdf)}

Attachment

Submitted filename: KyungEtal_2021_ResponsesToReviewers.pdf

Click here for additional data file.^{(312.1KB, pdf)}

Data Availability Statement

All data files are available from the OSF database (https://osf.io/uvs2d/).

[pone.0261463.ref001] 1.Bradshaw J. Pupil Size as a Measure of Arousal During Information Processing. Nature. 1967;216: 515–&. doi: 10.1038/216515a0 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref002] 2.Preuschoff K, Hart BM’, Einhaeuser W. Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making. Front Neurosci. 2011;5. doi: 10.3389/fnins.2011.00005 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref003] 3.Henson DB, Emuh T. Monitoring Vigilance during Perimetry by Using Pupillography. Invest Ophthalmol Vis Sci. 2010;51: 3540–4. doi: 10.1167/iovs.09-4413 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref004] 4.Hess EH, Polt JM. Pupil Size in Relation to Mental Activity During Simple Problem-Solving. Science. 1964;143: 1190–&. doi: 10.1126/science.143.3611.1190 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref005] 5.Kahneman D, Beatty J. Pupil Diameter and Load on Memory. Science. 1966;154: 1583–&. doi: 10.1126/science.154.3756.1583 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref006] 6.Lavin C, San Martin R, Rosales Jubal E. Pupil dilation signals uncertainty and surprise in a learning gambling task. Front Behav Neurosci. 2013;7: 218. doi: 10.3389/fnbeh.2013.00218 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref007] 7.Kloosterman NA, Meindertsma T, van Loon AM, Lamme VAF, Bonneh YS, Donner TH. Pupil size tracks perceptual content and surprise. Eur J Neurosci. 2015;41: 1068–1078. doi: 10.1111/ejn.12859 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref008] 8.de Gee JW, Knapen T, Donner TH. Decision-related pupil dilation reflects upcoming choice and individual bias. Proceedings of the National Academy of Sciences of the United States of America. 2014;111: E618–E625. doi: 10.1073/pnas.1317557111 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref009] 9.Joshi S, Li Y, Kalwani RM, Gold JI. Relationships between Pupil Diameter and Neuronal Activity in the Locus Coeruleus, Colliculi, and Cingulate Cortex. Neuron. Elsevier Inc; 2016;89: 221–234. doi: 10.1016/j.neuron.2015.11.028 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref010] 10.Szabadi E. Modulation of physiological reflexes by pain: role of the locus coeruleus. 2016;: 1–15. doi: 10.3389/fnint.2012.00094/abstract [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref011] 11.Murphy PR, O’Connell RG, O’Sullivan M, Robertson IH, Balsters JH. Pupil diameter covaries with BOLD activity in human locus coeruleus. Hum Brain Mapp. 2014;35: 4140–4154. doi: 10.1002/hbm.22466 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref012] 12.Joshi S, Gold JI. Pupil Size as a Window on Neural Substrates of Cognition. Trends in Cognitive Sciences. 2020;24: 466–480. doi: 10.1016/j.tics.2020.03.005 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref013] 13.Kahneman D. Attention and effort. Englewood Cliffs, NJ: Prentice-Hall; 1973. doi:doi=10.1.1.398.5285&rep=rep1&type=pdf [Google Scholar]

[pone.0261463.ref014] 14.van der Wel P, van Steenbergen H. Pupil dilation as an index of effort in cognitive control tasks: A review. Psychonomic Bulletin & Review; 2018;: 1–11. doi: 10.3758/s13423-018-1432-y [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref015] 15.Loewenfeld IE. The Pupil. Anatomy, Physiology and Clinical Applications. 2nd ed. Boston: Butterworth- Heinemann; 1999. pp. 1–1590. doi: 10.1023/a:1002134106425 [DOI] [Google Scholar]

[pone.0261463.ref016] 16.Mathôt S, Van der Stigchel S. New Light on the Mind’s Eye: The Pupillary Light Response as Active Vision. Curr Dir Psychol Sci. 2015;24: 374–378. doi: 10.1177/0963721415593725 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref017] 17.Knapen T, de Gee JW, Brascamp J, Nuiten S, Hoppenbrouwers S, Theeuwes J. Cognitive and Ocular Factors Jointly Determine Pupil Responses under Equiluminance. Verguts T, editor. PLoS ONE. Public Library of Science; 2016;11: e0155574–13. doi: 10.1371/journal.pone.0155574 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref018] 18.Fukuda K, Stern JA, Brown TB, Russo MB. Cognition, blinks, eye-movements, and pupillary movements during performance of a running memory task. Aviat Space Environ Med. 2005;76: C75–85. [PubMed] [Google Scholar]

[pone.0261463.ref019] 19.Siegle GJ, Ichikawa N, Steinhauer S. Blink before and after you think: Blinks occur prior to and following cognitive load indexed by pupillary responses. Psychophysiology. 2008;45: 679–687. doi: 10.1111/j.1469-8986.2008.00681.x [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref020] 20.Hupe JM, Lamirel C, Lorenceau J. Pupil dynamics during bistable motion perception. Journal of Vision. 2009;9: 10–10. doi: 10.1167/9.7.10 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref021] 21.Collins M, Seeto R, Campbell L, Ross M. Blinking and corneal sensitivity. Acta Ophthalmol (Copenh). 1989;67: 525–531. doi: 10.1111/j.1755-3768.1989.tb04103.x [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref022] 22.STERN JA, WALRATH LC, GOLDSTEIN R. The Endogenous Eyeblink. Psychophysiology. 1984;21: 22–33. doi: 10.1111/j.1469-8986.1984.tb02312.x [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref023] 23.Finnemore VM. Is the dry eye contact lens wearer at risk? Not usually. Cornea. 1990;9 Suppl 1: S51–3– discussion S54. doi: 10.1097/00003226-199010001-00021 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref024] 24.Carney LG, Hill RM. The nature of normal blinking patterns. Acta Ophthalmol (Copenh). John Wiley & Sons, Ltd; 1982;60: 427–433. doi: 10.1111/j.1755-3768.1982.tb03034.x [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref025] 25.Doughty MJ. Consideration of Three Types of Spontaneous Eyeblink Activity in Normal Humans: during Reading and Video Display Terminal Use, in Primary Gaze, and while in Conversation. Optometry and Vision Science. 2001;78: 712–725. doi: 10.1097/00006324-200110000-00011 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref026] 26.Doughty MJ. Effects of Background Lighting and Retinal Illuminance on Spontaneous Eyeblink Activity of Human Subjects in Primary Eye Gaze. Eye & Contact Lens: Science & Clinical Practice. 2013;39: 138–146. doi: 10.1097/ICL.0b013e31827124b7 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref027] 27.Doughty MJ. Further assessment of gender- and blink pattern-related differences in the spontaneous eyeblink activity in primary gaze in young adult humans. Optometry and Vision Science. 2002;79: 439–447. doi: 10.1097/00006324-200207000-00013 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref028] 28.Daniels LB, Nichols DF, Seifert MS, Hock HS. Changes in pupil diameter entrained by cortically initiated changes in attention. Vis Neurosci. 2012;29: 131–142. doi: 10.1017/S0952523812000077 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref029] 29.Orchard LN, STERN JA. Blinks as an index of cognitive activity during reading. Integr Physiol Behav Sci. 1991;26: 108–116. doi: 10.1007/BF02691032 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref030] 30.Nakano T, Yamamoto Y, Kitajo K, Takahashi T, Kitazawa S. Synchronization of spontaneous eyeblinks while viewing video stories. Proceedings of the Royal Society B: Biological Sciences. 2009;276: 3635–3644. doi: 10.1098/rspb.2009.0828 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref031] 31.Liao H-I, Yoneya M, Kidani S, Kashino M, Furukawa S. Human Pupillary Dilation Response to Deviant Auditory Stimuli: Effects of Stimulus Properties and Voluntary Attention. Front Neurosci. 2016;10: 403–14. doi: 10.3389/fnins.2016.00403 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref032] 32.Groman SM, James AS, Seu E, Tran S, Clark TA, Harpster SN, et al. In the Blink of an Eye: Relating Positive-Feedback Sensitivity to Striatal Dopamine D2-Like Receptors through Blink Rate. Journal of Neuroscience. 2014;34: 14443–14454. doi: 10.1523/JNEUROSCI.3037-14.2014 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref033] 33.Taylor JR, Elsworth JD, Lawrence MS, Sladek JR Jr., Roth RH, Redmond DE Jr. Spontaneous Blink Rates Correlate with Dopamine Levels in the Caudate Nucleus of MPTP-Treated Monkeys. Experimental Neurology. 1999;158: 214–220. doi: 10.1006/exnr.1999.7093 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref034] 34.Kotani M, Kiyoshi A, Murai T, Nakako T, Matsumoto K, Matsumoto A, et al. The dopamine D1 receptor agonist SKF-82958 effectively increases eye blinking count in common marmosets. Behavioural Brain Research. 2016;300: 25–30. doi: 10.1016/j.bbr.2015.11.028 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref035] 35.Zhang T, Mou Di, Wang C, Tan F, Jiang Y, Lijun Z, et al. Dopamine and executive function: Increased spontaneous eye blink rates correlate with better set-shifting and inhibition, but poorer updating. International Journal of Psychophysiology. Elsevier B.V; 2015;96: 155–161. doi: 10.1016/j.ijpsycho.2015.04.010 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref036] 36.Colzato LS, van den Wildenberg WPM, van Wouwe NC, Pannebakker MM, Hommel B. Dopamine and inhibitory action control: evidence from spontaneous eye blink rates. Exp Brain Res. 2009;196: 467–474. doi: 10.1007/s00221-009-1862-x [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref037] 37.Aarts H, Bijleveld E, Custers R, Dogge M, Deelder M, Schutter D, et al. Positive priming and intentional binding: Eye-blink rate predicts reward information effects on the sense of agency. Social Neuroscience. 2012;7: 105–112. doi: 10.1080/17470919.2011.590602 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref038] 38.van Bochove ME, Van der Haegen L, Notebaert W, Verguts T. Blinking predicts enhanced cognitive control. Cogn Affect Behav Neurosci. 2012;13: 346–354. doi: 10.3758/s13415-012-0138-2 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref039] 39.Berman BD, Horovitz SG, Morel B, Hallett M. Neural correlates of blink suppression and the buildup of a natural bodily urge. NeuroImage. Elsevier Inc; 2012;59: 1441–1450. doi: 10.1016/j.neuroimage.2011.08.050 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref040] 40.Beatty J, Lucero-Wagoner B. The pupillary system. In Cacioppo J T, Tassinary L G, & Berntson G (Eds), Handbook of psychophysiology. Cambridge university press; 2003. pp. 142–162. [Google Scholar]

[pone.0261463.ref041] 41.Korn CW, Bach DR. A solid frame for the window on cognition: Modeling event-related pupil responses. Journal of Vision. 2016;16: 28–16. doi: 10.1167/16.3.28 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref042] 42.Watson AB, Yellott JI. A unified formula for light-adapted pupil size. Journal of Vision. 2012;12: 12. doi: 10.1167/12.10.12 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref043] 43.Ponder E, Kennedy WP. On the act of blinking. Quarterly Journal of Experimental Physiology. 1928;18: 88–109. [Google Scholar]

[pone.0261463.ref044] 44.Hong L, Walz JM, Sajda P. Your eyes give you away: prestimulus changes in pupil diameter correlate with poststimulus task-related EEG dynamics. PLoS ONE. Public Library of Science; 2014;9: e91321. doi: 10.1371/journal.pone.0091321 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref045] 45.Kamp SM, Donchin E. ERP and pupil responses to deviance in an oddball paradigm. Psychophysiology. John Wiley & Sons, Ltd; 2015;52: 460–471. doi: 10.1111/psyp.12378 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref046] 46.Denison RN, Parker JA, Carrasco M. Modeling pupil responses to rapid sequential events. Behavior Research Methods; 2020;: 1–17. doi: 10.3758/s13428-020-01368-6 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref047] 47.Bays PM. Noise in Neural Populations Accounts for Errors in Working Memory. Journal of Neuroscience. 2014;34: 3632–3645. doi: 10.1523/JNEUROSCI.3204-13.2014 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref048] 48.Murphy PR, Robertson IH, Balsters JH, O’connell RG. Pupillometry and P3 index the locus coeruleus-noradrenergic arousal function in humans. Psychophysiology. 2011;48: 1532–1543. doi: 10.1111/j.1469-8986.2011.01226.x [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref049] 49.Mathôt S. Tuning the Senses: How the Pupil Shapes Vision at the Earliest Stage. Annu Rev Vis Sci. 2020. doi: 10.1146/annurev-vision-030320-062352 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref050] 50.Zuber BL, Stark L, Lorber M. Saccadic suppression of the pupillary light reflex. Experimental Neurology. 1966;14: 351–370. doi: 10.1016/0014-4886(66)90120-8 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref051] 51.Einhauser W, Koch C, Carter O. Pupil dilation betrays the timing of decisions. Front Hum Neurosci. 2010;4. doi: 10.3389/neuro.09.004.2010 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref052] 52.Gilzenrat MS, Nieuwenhuis S, Jepma M, Cohen JD. Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn Affect Behav Neurosci. 2010;10: 252–269. doi: 10.3758/CABN.10.2.252 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref053] 53.Pearl J, Glymour M, Jewell NP. Causal inference in statistics: A primer. John Wiley & Sons; 2016. [Google Scholar]

[pone.0261463.ref054] 54.Acland BT, Braver TS (2014). Cili (v0.5.4) [Software] Available from 10.5281/zenodo.48843 [DOI]

[pone.0261463.ref055] 55.Kiehl KA, Stevens MC, Laurens KR, Pearlson G, Calhoun VD, Liddle PF. An adaptive reflexive processing model of neurocognitive function: supporting evidence from a large scale (n = 100) fMRI study of an auditory oddball task. NeuroImage. 2005;25: 899–915. doi: 10.1016/j.neuroimage.2004.12.035 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref056] 56.Kleiner M, Brainard D, Pelli D. What’s new in Psychtoolbox-3? 2007. pp. 1–89. [Google Scholar]

[pone.0261463.ref057] 57.Pelli DG. The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spat Vis. 1997;10: 437–442. [PubMed] [Google Scholar]

[pone.0261463.ref058] 58.Brainard DH. The psychophysics toolbox. Spat Vis. 1997;10: 433–436. [PubMed] [Google Scholar]

[pone.0261463.ref059] 59.McCamy MB, Otero-Millan J, Macknik SL, Yang Y, Troncoso XG, Baer SM, et al. Microsaccadic efficacy and contribution to foveal and peripheral vision. J Neurosci. Society for Neuroscience; 2012;32: 9194–9204. doi: 10.1523/JNEUROSCI.0515-12.2012 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0261463.ref060] 60.Otero-Millan J, Troncoso XG, Macknik SL, Serrano-Pedraza I, Martinez-Conde S. Saccades and microsaccades during visual fixation, exploration, and search: foundations for a common saccadic generator. Journal of Vision. The Association for Research in Vision and Ophthalmology; 2008;8: 21.1–18. doi: 10.1167/8.14.21 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref061] 61.Choe KW, Blake R, Lee S-H. Pupil size dynamics during fixation impact the accuracy and precision of video-based gaze estimation. VISION RESEARCH. Elsevier Ltd; 2017;: 1–12. doi: 10.1016/j.visres.2014.12.018 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref062] 62.Hershman R, Henik A, Cohen N. A novel blink detection method based on pupillometry noise. Behav Res. Springer US; 2018;50: 107–114. doi: 10.3758/s13428-017-1008-1 [DOI] [PubMed] [Google Scholar]

[pone.0261463.ref063] 63.Hayes TR, Petrov AA. Mapping and correcting the influence of gaze position on pupil size measurements. Behav Res. 2nd ed. 2015;48: 510–527. doi: 10.3758/s13428-015-0588-x [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

The confounding effects of eye blinking on pupillometry, and their remedy

Kyung Yoo

Jeongyeol Ahn

Sang-Hun Lee

Roles

Abstract

Introduction

Fig 1. Example of blink-locked pupillary response (BPR).

Results

The shape of blink-locked pupillary responses and its variability across luminance levels, subjects, and trials

Fig 2. Experimental design and examination of factors affecting blink-locked pupillary responses.

BPR, not just a nuisance but a confounder

Fig 3. BPR is predicted to bias pupil size because blink rate and pattern vary across time, working-memory load, and subjects.

Model-based correction of pupillary responses for BPR

Fig 4. Generative model-based decomposition of pupil-size measurements and example correction result.

Correcting the auditory oddball data for BPR

Fig 5. Temporal bias in blink patterns induces spurious distortion of pupil time course in the auditory oddball task.

Fig 6. Comparison of the blink-affected trials and the blink-free trials before and after correction.

Fig 7. Contribution of BPR correction to the statistical power of the auditory oddball task.

Correcting delayed orientation estimation data for BPR

Fig 8. Temporal bias in blink patterns induces spurious distortion of pupil time course in the delayed orientation estimation task.

Fig 9. Contribution of BPR correction to the statistical power of the delayed orientation estimation task.

Discussion

Summary of findings

Potential causes of BPR

The shape and variability of BPR

Confounding acts of BPR

The proper way of correcting pupil-size measurements for BPR

Practical considerations for avoiding BPR-related problems

Materials and methods

Subjects

Apparatus and eye-tracking setup

Stimuli and procedure

Fixation task

Auditory oddball task

Delayed orientation estimation task

Data analysis

Preprocessing of eye-tracking data

The definition of single blink events

The removal of artifacts around single blink events

Analyses on gaze deviance and potential dependency between blink and gaze

Definition of isolated blink-affected and blink-free time courses

Analysis of the blink-by-blink variability of BPR in peak amplitude and time-to-the-peak

Prediction of BPR confound via simulation

Analysis of the blink rate and pupil data in the auditory oddball and delayed orientation estimation tasks

Statistical power analysis

The algorithm of correcting pupillary responses for BPR

The generative model

Inference of generative model parameters

Supporting information

Acknowledgments

Data Availability

Funding Statement

References

Decision Letter 0

Manuel Spitschan

Roles

Author response to Decision Letter 0

Decision Letter 1

Manuel Spitschan

Roles

Author response to Decision Letter 1

Decision Letter 2

Manuel Spitschan

Roles

Acceptance letter

Manuel Spitschan

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases