Emotion‐Guided Attention Impacts Deliberate Multi‐Evidence Emotion‐Related Perceptual Decision‐Making

ABSTRACT

Emotion‐guided endogenous attention (e.g., attending to fear) may play a crucial role in determining how humans integrate emotional evidence from various sources when assessing the general emotional tenor of the environment. For instance, what emotion a presenter focuses on can shape their perception of the overall emotion of the room. While there is an increasing interest in understanding how endogenous attention affects emotion perception, existing studies have largely focused on single‐stimulus perception. There is limited understanding of how endogenous attention influences emotion evidence integration across multiple sources. To investigate this question, human participants (N = 40) were invited to judge the average emotion across an array of faces ranging from fearful to happy. Endogenous attention was manipulated by instructing participants to decide whether the face array was “fearful or not” (fear attention), “happy or not” (happy attention). Eye movement results revealed an endogenous attention‐induced sampling bias such that participants paid more attention to extreme emotional evidence congruent with the target emotion. Computational modeling revealed that endogenous attention shifted the decision criterion to be more conservative, leading to reduced target‐category decisions. These findings unraveled the cognitive and computational mechanisms of how endogenous attention impacts the way we gather emotional evidence and make integrative decisions, shedding light on emotion‐related decision‐making.

Impact Statement

Assessing the emotional tone of our surroundings is crucial for guiding our cognition and reaction. Various factors, including attention, can affect the process of integrating emotional evidence from multiple sources. For example, a speaker's focus on the audience's emotions during a speech can shape their perception of the overall emotional atmosphere. This study explores how endogenous emotion‐guided attention impacts our ability to gauge emotions in different situations, providing insights into decision‐making and effective social interactions.

To effectively navigate our environment, it is essential to assess the emotional nature of our surroundings, which often involves integrating evidence from multiple pieces of emotional stimuli. The observer is thus tasked to gather and weigh these pieces of evidence before arriving at a summary perceptual decision. These perceptual decisions are influential in guiding subsequent cognitions and actions and can have an impact on memory and learning (Fox et al. 2018). For instance, when talking to a party of peers, the person needs to constantly assess whether the audience, as a whole, is enjoying the topic, a judgment that will help the speaker decide whether to continue or change the topic. Inaccurate judgment in this scenario may lead to embarrassment, contributing to a negative social memory and impacting the person's reputation.

Decades of affective science research has provided valuable knowledge about how humans make emotion‐related perceptual decisions (e.g., Cuthbert et al. 1998; Ekman and Friesen 1978; Lang and Bradley 2010; Méndez‐Bértolo et al. 2016; Russell 1980; Schupp et al. 2000). However, our current knowledge on this subject has largely come from paradigms asking people to judge a single stimulus. Little is known about how humans integrate multiple pieces of emotional evidence. Importantly, we do not passively perceive stimuli in our environment. The observer actively participates in the process of perceptual decision‐making by selectively attending to, sampling, and weighing different pieces of evidence. In the aforementioned example, the individual's tendency to attend to negative versus positive emotional expressions may lead to different evidence gathering and possibly different judgments. While traditionally, research on emotion processing has mainly focused on the stimulus‐driven bottom‐up process, a recent paradigm shift has stressed the impacts of endogenous, or top‐down, factors, such as attention and goal relevance, in the processing of emotional evidence (Barrett et al. 2011; Mohanty et al. 2023; Sussman et al. 2017). Here again, the research has mostly focused on single‐stimulus processing. Currently, it remains unknown how top‐down factors may influence the integration process of multiple pieces of emotional evidence in forming a perceptual decision. In this study, we specifically investigate the impact of one key top‐down factor, namely endogenous emotion‐guided attention, on perceptual decision‐making involving multiple pieces of emotional evidence. In the rest of the introduction, we first review the relevant background literature and then provide the rationale of the current study design.

Our perceptual system is limited such that we cannot take in various pieces of evidence all at once (Alvarez 2011; Broadbent 1953; Luck and Vogel 1997; Tsotsos 1997). Hence, attention allocation plays a crucial role in forming perceptual decisions, as it gates what evidence is available and prioritized for subsequent processing (Carrasco 2011; Chun et al. 2011; Lang et al. 1997a, 1997b; Posner and Boies 1971). In ensemble perception studies, humans can apply distributed visual attention to quickly extract the “gist” of a visual scene (e.g., within a fraction of a second) (Baek and Chong 2020; Cavanagh 2001; Chong and Treisman 2003; Whitney and Yamanashi Leib 2018). When applied to emotion‐related decision‐making, such as judging the average emotion of a set of emotional faces, it has been demonstrated that human perception is sensitive to the mean emotion, even when the stimuli are presented for a very short period (Whitney 2007; Haberman and Whitney 2009, 2010; Hubert‐Wallander and Boynton 2015; Ji et al. 2014; Wolfe et al. 2015). In contrast to perceptual decisions made when access to evidence is time‐limited, much less is known about deliberate emotion‐related decision‐making processes. When allowed sufficient time to sample the evidence space deliberately, humans may overcome perceptual limitations by shifting focused attention and thereby registering each piece of evidence properly (Choi and Chong 2020; Kiani et al. 2008; Ngai et al. 2025; Phelps et al. 2014; Wolfe et al. 2011). In this deliberate decision‐making process, it remains unexplored how endogenously guided visual attention directly influences which pieces of evidence are gathered and to what extent (Kaanders et al. 2022; Krajbich et al. 2010; Orquin and Loose 2013; Thomas et al. 2019).

Visual attention can be affected in both bottom‐up and top‐down manners (Itti and Koch 2000; Lang et al. 1997a, 1997b; Yantis 1998). From the bottom‐up perspective, given its inherent motivational saliency (Ledoux 1998; Lundqvist and Öhman 2005; Öhman 2002), emotional stimuli capture attention automatically, also known as the attention capturing effect (Brown 2022; Calvo and Nummenmaa 2016; Pessoa and Ungerleider 2004; Smith et al. 2003). While both positive (e.g., happy) and negative (e.g., fearful) stimuli have been found to possess a robust attention capturing effect, some argue that the effect is particularly strong for fearful stimuli due to the association with danger, triggering an automatic and rapid orienting of attention (Adolphs 2008; Bannerman et al. 2009; Carretié 2014; Eastwood et al. 2003; Méndez‐Bértolo et al. 2016; Öhman et al. 2001; Zadra and Clore 2011). Recent free‐viewing ensemble perception research demonstrated that when given ample time, participants attentively sampled extreme emotional evidence more than less extreme evidence before making decisions (Ngai et al. 2025). Thus, when presented multiple stimuli with varying levels of emotional intensity, visual attention may be naturally attracted to more extreme emotional evidence than less extreme ones, an effect particularly strong for fearful ones.

From the top‐down perspective, according to the predictive processing and constructivist view of emotion, recent theoretical and empirical works have supported that emotion processing is heavily influenced by the observer's internal factors, such as endogenous attention and current goals (Barrett and Simmons 2015; Bhatia 2014; Feldman Barrett 2011; Lee et al. 2021; Mohanty and Sussman 2013; Sussman et al. 2020). Endogenous attention, that is what the individual is purposefully and willingly attending to (often initiated prior to the stimulus presentation), dictates what evidence is relevant to the task at hand and therefore salient, facilitating the extraction of relevant evidence (Desimone and Duncan 1995; Lang et al. 1997a, 1997b; Posner and Boies 1971; Smith and Ratcliff 2009; Summerfield and Egner 2009; Summerfield et al. 2006; Taylor and Fragopanagos 2005). For instance, in single stimulus rapid‐judgment tasks, it has been shown that endogenous emotion‐guided attention (e.g., attending to fearful compared to neutral targets) enhances perceptual decision‐making performance in discriminating between emotional and neutral stimuli (Glasgow et al. 2022; Sussman, Jin, et al. 2016; Sussman et al. 2017). Computationally, these impacts are related to the influence of endogenous attention on the process of accumulating emotional evidence (Richter et al. 2019; Yiend 2010). Endogenous emotion‐guided attention, in the context of a deliberate multi‐evidence decision‐making task, can further enhance the saliency of the task‐relevant extreme evidence due to its relevance to the goals in decision‐making (Mohanty et al. 2023). Thus, this could lead to increased sampling of extreme evidence that is congruent with endogenous attention, for example, oversampling of very fearful evidence when attending to the fear emotion.

Visual attention has been associated with the weight of the evidence in value‐based binary choice tasks, culminating in a phenomenon called the gaze cascade effect. For example, when choosing between two options, people are more likely to choose the one they looked at longer (Kaanders et al. 2022; Sepulveda et al. 2020; Shimojo et al. 2003). This attention‐as‐weight finding has been observed in nonemotion‐related binary choice tasks (Armel et al. 2008; Hertwig and Erev 2009; Thomas et al. 2019). Also, research on non‐emotion‐related value judgments has revealed that salient stimuli (stimuli with extreme values) capture attention in a bottom‐up manner, leading to them being overweighed in the final decision (Kunar et al. 2017; Tsetsos et al. 2012). Thus, it is possible that endogenous attention‐congruent evidence may be overweighed in the final decision. However, findings from the free‐viewing emotion ensemble perception study mentioned above showed that while participants exhibited biased attention distribution, they weighed various emotional evidence equally (Ngai et al. 2025). It remains unexplored how endogenous attention would affect this process. Finally, attending to a particular emotion can lead to biases. In this regard, confirmatory bias has been reported, for example, an increased tendency to judge a stimulus as fearful when attending to fearful versus neutral or happy emotion (LaBar et al. 2003; Mohanty et al. 2023; Niedenthal et al. 2000; Wyart et al. 2012). Alternatively, attention can also increase the specificity of the attended emotional object, manifesting in a more conservative criterion; for example, being more reluctant to judge a stimulus as fearful under fear‐guided attention (Chipchase and Chapman 2013; Murray and Wojciulik 2004). While confirmatory and conservative biases have been observed in different contexts, the likelihood of these biases occurring is influenced by factors such as the nature of the emotional stimuli and individual differences (Aue and Okon‐Singer 2015; Barrett et al. 2019; Wieser and Brosch 2012). The interplay of these factors can result in the manifestation of either confirmatory or conservative biases, highlighting the complexity of emotion, attention, and their impact on decision‐making.

To investigate how endogenous emotion‐guided attention affects deliberate emotion‐related decision‐making, participants' endogenous attention was directed to a specific emotion (fearful‐ or happy‐guided attention) during a multi‐evidence decision‐making task. After sampling the multi‐evidence space consisting of facial expressions ranging from fearful to happy, participants decided whether the set of faces, on average, belonged to the attended emotion category. Their eye movement was tracked during sampling, as eye gaze has been shown to be a proxy of visual attention (Henderson 2003; Itti and Koch 2000; Rayner 1998). This study task and procedure were based on a previous study conducted by our team (Ngai et al. 2025). However, in the current study, participants' endogenous attention was manipulated to examine how endogenous attention influences sampling and integration of evidence. Understanding the role of endogenous attention in decision‐making is critical for illuminating the mechanisms behind real‐world decision‐making processes, where decision makers are often influenced by their emotions and expectations.

Based on the deliberate multi‐evidence perceptual decision‐making task in our previous study, endogenous emotion‐guided attention can alter one or more of the following components: (1) how the evidence is sampled, (2) how the sampled evidence is weighed, and (3) a shift in the decision‐making criterion. It was hypothesized that (1) more visual attention reflected in eye gaze would be allocated to extreme (outlying) emotional evidence than less extreme (inlying) emotional evidence overall given the naturally high salience of extreme emotional evidence; endogenous attention (2) would lead to an imbalanced attention distribution such that eye gaze would be biased toward more extreme emotional evidence that was attention‐congruent, compared to attention‐incongruent, (3) would be translated into a biased weighing of evidence, such that evidence that was attended to more was also weighed higher, and (4) would shift the decision‐making criterion to be either more conservative or relaxed. The findings from this study can provide valuable insights and contribute to the theories mentioned above. Specifically, (1) Does the attention capturing effect proposed by the evolutionary saliency theory of emotion and the top‐down modulatory effects suggested by the constructivist view of emotion persist when individuals engage in emotion ensemble perception with focused attention? (2) Furthermore, if these effects hold true, will we also observe the gaze cascade effect, characterized by attention influencing the weight of evidence? By investigating these questions, our study seeks to provide a more detailed understanding of the specific effects and mechanisms involved in emotion ensemble perception.

1. Method

1.1. Participants

The sample size was determined based on a previous experiment conducted by our team examining a purely bottom‐up effect in emotion‐related multi‐evidence decision‐making (Ngai et al. 2025). The sample size of 33 in that study yielded large partial eta (η _p ²) effect sizes largely above 0.14 in all statistical analyses. Hence, following preregistration, we chose a similar target sample size of 40 in this study.

Forty‐six young adults (mean age = 27.08 years old, SD = 9.79 years old; women = 70%, men = 30%; Chinese and Other Asian = 97.8%) from the University of Hong Kong were recruited. Data from six participants were discarded due to experiment incompletion (N = 2), technical failure (N = 2), and being outliers (i.e., mean ± 4 SD) in eye movement data (N = 2). Thus, the final data set consisted of 40 participants. Each participant provided written consent, reported being fluent in English and had normal or corrected‐to‐normal vision. The study was approved by the Human Research Ethics Committee at the University of Hong Kong. Participants were rewarded either with partial course credit or compensated with 50HKD per hour.

1.2. Stimuli

The study used original image stimuli consisting of 10 faces (five fearful, five happy). These faces were selected from five Asian actors (two females) from the hosting university's Face Database (Zhang et al. 2019). In order to match the local sociocultural context where this study was held, Asian faces were chosen (Masuda et al. 2008). Fearful and happy faces were chosen because they are both high arousal emotions with opposite valence (Watson et al. 1999). While anger is also a negative valence emotion, it was not chosen because it can provoke complex responses or evaluations in the participants (Harmon‐Jones et al. 2011; Pichon et al. 2009). Following our previously established procedure (Ngai et al. 2025), images were created using FantaMorph software (Abrosoft Fantamorph Version 5.6.2; www.fantamorph.com), which linearly interpolated between the fearful and happy facial images to create 99 synthetic images that varied in degrees of fearfulness (happiness). Using Adobe Photoshop (Version 22.1.1), the resulting images were manually retouched to ensure they looked natural. The numerically coded morphing units, where 0 corresponded to the actor's most fearful face and 100 corresponded to the happiest face, will be referred to as the objective emotional value (OEV) from here on. Finally, the stimuli were equalized for contrast and luminance using the SHINE Toolbox in MATLAB (Willenbockel et al. 2010). This resulted in 505 stimuli in total (101 fearful‐to‐happy images × 5 face actors).

1.3. Experimental Tasks

Building on a previous multi‐evidence perceptual decision‐making design (Ngai et al. 2025), participants completed two tasks: the emotion attention‐guided decision‐making task with eye‐tracking first, then the subjective emotion rating task.

1.3.1. The Emotion Attention‐Guided Decision‐Making Task

In this attention‐guided perceptual decision‐making task, endogenous attention was manipulated at the block level following an established paradigm in single‐stimulus judgment studies, where the binary choice was in a “X or Not X” format (Sussman, Szekely, et al. 2016; Sussman et al. 2017). Specifically, participants made decisions regarding whether a set of faces were on average “Fearful or Not Fearful” under the (F)ear‐attention blocks and “Happy or Not Happy” under the (H)appy‐attention blocks (Figure 1A). There were three F‐attention and three H‐attention blocks, each containing 15 trials and totaling to 90 trials. The F‐attention and H‐attention blocks alternated throughout the experiment, with the sequence counterbalanced between participants.

FIGURE 1.

Experimental tasks. (A) Participants were asked to make a “Fearful or Not Fearful” decision in the Fear‐attention block, and a “Happy or Not Happy” decision in the Happy‐attention block of the emotion attention‐guided decision‐making task. (B) Sample trial for the subjective emotion rating task.

Each trial started with a drift correction check, and stimuli were presented only when the participants' gaze was within one degree of visual angle from fixation. Next, nine empty squares were displayed for 1000 ms, after which they were replaced by nine grayscale face images (eight in the annulus and one reference face in the center). Please note that all nine face images were of the same face actor identity, following common ensemble perception practice (Haberman and Whitney 2009; Whitney and Yamanashi Leib 2018). Each face image subtended about 5° of visual angle horizontally and 7° of visual angle vertically and was presented 10° of visual angle away from the center reference face (when measuring from the center of each image). The eight faces in the annulus made up the emotion ensemble, whereas a central reference was placed in the middle of the screen. The central reference face served as a common benchmark where the face actor exhibited 50% fear and 50% happiness.

The faces were shown for 8000 ms, during which the subjects were free to view the images. After viewing, participants made the binary decision summarizing the eight faces in the annulus according to the block instruction. They were also instructed not to take the central reference face into their decision, but to use it as a benchmark or reference of 50% fearful/happy. Due to the subjectivity of emotion perception, there is no true correct answer in a binary choice task regarding emotions. Hence, the OEVs of the eight evidence‐bearing faces in each trial were drawn from a Gaussian distribution with a mean of 50% fearful–50% happy (see Supporting Information S1 for variance levels). There were 45 unique trials, each of which was presented once in F‐attention and once in H‐attention. Eye movements were tracked during this 8000 ms free viewing. The decision‐making task was delivered using Experiment Builder (Version 2.2.61) compatible with eye‐tracker software. It was ensured that the same face identity and gender were not shown more than four times in a row.

1.3.2. The Subjective Emotion Rating Task

For each of the 45 unique trials used in the emotion attention‐guided decision‐making task, participants rated the emotions of each individual face (Figure 1B). Specifically, subjects were presented with the same image arrays in the same trial sequence as used in the decision‐making task but were instructed to rate each of the eight faces shown on a scale ranging from 0 (“very fearful”) to 10 (“very happy”) with no time constraint. Once again, the subjects were instructed that the central reference face served as a benchmark of that face actor's 50% fearful and 50% happy face. Ratings obtained from this task will be referred to as the subjective emotional value (SEV). We purposefully obtained the SEVs of each stimulus in its trial context, as emotional value judgment of the same stimulus can change given different surrounding contexts (Barrett et al. 2011; Wieser and Brosch 2012). As emotion perception is highly subjective across (Hamann and Canli 2004; Winter and Kuiper 1997) and within individuals (Barrett 2017), the inclusion of this subjective emotion rating task allowed for the measurement of participant's perception of each piece of evidence within the trial context. In this way, we were obtaining as close as possible the SEVs contributing to the binary decision‐making task choices. This design differs from previous studies where the value of each element is typically defined orthogonal to the decision‐making task context. This can involve either relying on the objective value provided by the experimenter or obtaining a subjective rating outside of the trial context (e.g., Elias et al. 2017; Haberman and Whitney 2009). The subjective emotion rating task was delivered using PsychoPy (Version 3.2.4).

1.4. Apparatus

The EyeLink 1000 eye‐tracker (SR Research Ltd.) was used to record the eye movement of each participant's dominant eye. The participants were positioned 60 cm away from a 22″ monitor (1024 × 768 pixels), where the whole display was used. The system sampled the eye movements monocularly at a rate of 1000 Hz using pupil and corneal reflection. The data collection used the default settings of Eyelink, which included a saccade motion threshold of 0.1° visual angle, a saccade acceleration threshold of 8000°/s², and a saccade velocity threshold of 30°. At the beginning of the eye‐tracking task, a nine‐point calibration and validation were performed and repeated until the average error was less than one degree of visual angle from each fixation. All participants used a chin rest to minimize head movements.

1.5. Data Processing

To ensure the quality of the eye movement data, a quality check was performed by first finding the mean number of fixations per trial across all participants. Outliers were then detected by using the mean ± 4 SDs as the threshold as preregistered. Two participants were excluded from subsequent analyses as they had more than 20% of trials with fixations smaller than the overall average.

Eye movement data were analyzed in both the spatial and temporal dimensions, considering the fixation location, duration, and sequence of transitions among fixation locations. Specifically, we calculated the trial‐wise fixation count and gaze duration for each region of interest (ROI), which corresponded to the nine placeholders within a trial (a 180‐by‐180 pixel square for each ROI). Then, to model the temporal dynamics of eye movements for each trial, transition probability was estimated using the eye movement analysis with hidden Markov model (EMHMM), a data‐driven, machine‐learning approach MATLAB toolbox (version 0.77). EMHMM uses hidden Markov models (HMMs) to analyze the observable time‐series data provided from eye tracking. Transition probability measures the likelihood of a saccade toward an ROI, based on the location of the previous fixation (Chuk et al. 2014). In EMHMM, the observable eye movement data were modeled as resulting from an underlying dynamic process of the sequence of the ROIs viewed (Chan et al. 2018; Chuk, Chan, et al. 2017; Chuk, Crookes, et al. 2017). As the current task contains a natural set of nine ROIs per trial, we predefined the spatial ROIs based on the area of each face image's location (see Supporting Information S1) (Cho et al. 2022; Chuk et al. 2019). Using the fixation point sequence from the eye movement data, the HMM estimated the probability of starting at a particular ROI (i.e., estimated prior probability) and the probability of fixations transitioning between a pair of ROIs (i.e., estimated transition probability) (Chuk et al. 2014). All analyses regarding the eye movement data were conducted in the environment of MATLAB (Release 2020b; The MathWorks, Natick, MA).

1.6. Statistical Analyses

1.6.1. Examining Differences in Overall Eye Movement

The total fixation count and gaze duration of each endogenous attention condition (F‐attention and H‐attention) were computed for each participant. Explorative dependent t‐tests were conducted to compare any differences in total fixation count and gaze duration across the two endogenous attention conditions.

1.6.2. Examining Evidence Sampling Using Eye Movement Data

To examine how participants sampled emotional evidence, we analyzed the mean proportion of fixation counts, gaze durations, and transition probabilities to each ROI. First, the trials under the same condition of endogenous attention manipulation were grouped together. Fixations were grouped by ROI, excluding those outside the nine ROIs. Proportions of fixation counts and gaze durations were computed for each of the eight evidence‐bearing ROIs. Moreover, transition probabilities to each of the eight peripheral ROIs were calculated using marginal transition probabilities. Using the estimated outputs from EMHMM, the marginal transition probabilities were calculated as the weighted summation of the prior probability of the source ROI with all the transition probabilities to that ROI (see Supporting Information S1). Then the three eye measures of fixation counts, gaze durations, and transition probabilities were reordered based on ascending SEV. This resulted in eight levels of ROI rank order, with level 1 representing the most fearful face and level 8 representing the happiest face of each trial. This re‐ordering was conducted at the individual trial level, following the same rationale as previous studies (de Garlle and Summerfield 2011).

For statistical analyses, after taking the mean percentage of fixation count, percentage of gaze duration, and transition probabilities averaged across all trials for each participant, the trials were then separated by the condition of endogenous attention. Thereafter, repeated measures ANOVAs (rmANOVA) were applied to examine the effects of the eight levels of ROI rank order (from the most fearful to the happiest) and two levels of endogenous attention (F‐attention or H‐attention) on eye movement measures. This was conducted specifically to examine the distribution of overt visual attention across varying emotional evidence and whether the pattern differed between endogenous attention conditions.

Subsequently, to further investigate whether the extreme ROIs were attended more during attention‐congruent conditions (e.g., very fearful face ROIs under F‐attention) compared to incongruent (e.g., very fearful face ROIs under H‐attention), the ROIs were further separated into Outlying Fearful (OutF), Inlying (In), and Outlying Happy (OutH) in each attention condition. “OutF” was the average between the two most fearful ROIs of each trial (ROIs 1 and 2), “In” referred to the average of the inlying four ROIs (ROIs 3–6), and “OutH” was the average of the two happiest ROIs of each trial (ROIs 7 and 8). This distribution was chosen as the most extreme (outlying) images should be at the end of the spectrum among the ensemble images. Finally, rmANOVAs were applied to examine the effects of the three levels of extremities of ROIs (OutF, In, and OutH) and two levels of endogenous attention (F‐attention, H‐attention) on the three eye movement measures.

1.6.3. Computational Modeling of the Binary Decision‐Making

To gain deeper insight into how subjects integrated the emotion evidence, we compared 13 models differing only in how the decision value (DV) was computed (Table 1). To start, a basic model (M0) used the average OEV from each trial to compute the DV, meaning that this model did not account for variances in emotion valuation within and between subjects. Following this, models 1 and 2 (M1&2) were constructed where each piece of evidence held the same weight, and the DV was calculated by using the SEV of the eight stimuli rated by each participant during the emotion rating task. As an exploratory model, M1 used a categorical approach, transforming each SEV into 1 if it was interpreted as happier and −1 if viewed as more fearful, before averaging. This categorization was based on comparing the SEV to the midpoint of the scale. In contrast, M2 used a continuous approach, directly averaging the eight SEVs, and consequently preserved all the SEV information.

TABLE 1.

Decision value (DV) calculations.

Model	DV calculation
M0: Objective Emotional Value (OEV)	${\mathrm{DV}}_j=\frac{1}{8}\sum _{i=1}^8{\mathrm{OEV}}_i$
M1: Subjective Emotional Value (SEV) Categorical	${\mathrm{DV}}_j=\frac{1}{8}\sum _{i=1}^8{\text{cSEV}}_i,$ where cSEV = 1 if SEVi ≥ 5 and cSEV = −1 if SEVi < 5
M2: SEV Continuous	${\mathrm{DV}}_j=\frac{1}{8}\sum _{i=1}^8{\mathrm{SEV}}_i$
M3: Fixation Count × Categorical SEV	${\mathrm{DV}}_j=\sum _{i=1}^8{\omega }_i{\text{cSEV}}_i$, where ${\omega }_i$ is the corresponding image's fixation count% and cSEV = 1 if SEVi ≥ 5 and cSEV = −1 if SEVi < 5
M4: Gaze Duration × Categorical SEV	${\mathrm{DV}}_j=\sum _{i=1}^8{\omega }_i{\text{cSEV}}_i$, where ${\omega }_i$ is the corresponding image's gaze duration% and cSEV = 1 if SEVi ≥ 5 and cSEV = −1 if SEVi < 5
M5: Transition Probability × Categorical SEV	${\mathrm{DV}}_j=\sum _{i=1}^8{\omega }_i{\text{cSEV}}_i$, where ${\omega }_i$ is the corresponding image's marginal transition probability & cSEV = 1 if SEVi ≥ 5 and cSEV = −1 if SEVi < 5
M6: Fixation Count × Continuous SEV	${\mathrm{DV}}_j=\sum _{i=1}^8{\omega }_i{\mathrm{SEV}}_i$, where ${\omega }_i$ is the corresponding image's fixation count %
M7: Gaze Duration × Continuous SEV	${\mathit{DV}}_j=\sum _{i=1}^8{\omega }_i{\mathit{SEV}}_i$, where ${\omega }_i$ is the corresponding image's gaze duration %
M8: Transition Probability × Continuous SEV	${\mathrm{DV}}_j=\sum _{i=1}^8{\omega }_i{\mathrm{SEV}}_i$, where ${\omega }_i$ is the corresponding image's marginal transition probability
M9: Weights of the evidence as free parameters	${\mathrm{DV}}_j={\sum }_{i=1}^8{\beta }_i{\chi }_{i,j}$, where i is the ROI bin index, ${\beta }_i$ is a free parameter, and ${\chi }_i$ is the tallied value of an ROI bin
M10‐12: Weights of the evidence as free parameters, plus a trial‐level noise term	${\mathrm{DV}}_j={ϵ}_j+{\sum }_{i=1}^8{\beta }_i{\chi }_{i,j}$, where i is the ROI bin index, ${\beta }_i$ is a free parameter, and ${\chi }_i$ is the tallied value of an ROI bin; $ϵ$ an error value is drawn from G (0, $\xi$), where $\xi$ = 0.9 for M10, 1.8 for M11, and 2.7 for M12. M9 had no error value $ϵ$

Note: cSEV stands for categorical (i.e., dichotomized) SEV. j indexes trial.

Models 3–8 evaluated if visual attention allocation, gauged by the three eye movement measurements, influenced the weights in decision‐making. Models 3–5 employed the dichotomized SEV as in M1. For every eye movement measurement of an ROI, the measurement was multiplied by −1 if the face displayed fear and by +1 if it showed happiness. The DV was calculated by averaging all the adjusted values for the percentage of fixation count (M3), the percentage of gaze duration (M4), and marginal transition probability (M5). For Models 6–8, they utilized the continuous SEV as in M2, with the DV computed as a weighted total of the SEV. Consequently, the SEV of each facial stimulus was directly weighted by one of the eye movement measurements: the percentage of fixation count (M6), the percentage of gaze duration (M7), and the marginal transition probability (M8).

To more directly examine how participants weighed the different pieces of evidence, we also constructed linear regression models in an exploratory analysis, where the weights of each ROI were estimated as free parameters. Specifically, the linear regression models contained the variable ${\chi }_i$, defined following established approach (Li et al. 2017). This variable was calculated by first subtracting five (the midpoint in the SEV scale) from the SEV value of each ROI. Then the values of the SEV of all trials were categorized into eight bins. Then for each trial, the bin values were determined by tallying the SEV values that fell within each bin's range. The weight parameter ${\beta }_i$. was subsequently estimated for each bin. The betas contribute to the calculation of the DV and reflect how much weight participants allocate to each evidence. For models 10–12, we also introduced a noise term, in line with studies using similar modeling approaches (Dakin 2001; Li et al. 2017; Solomon et al. 2016). For these models, the random noise term, ${ϵ}_j$, was introduced at the DV calculation step, which was a number drawn from a Gaussian distribution with mean of 0 and variance of $\xi$ (0.9 for M10, 1.8 for M11, and 2.7 for M12) (Li et al. 2017). Compared to the preregistered models, the categorical approach (M1, 3–5) and the models where the weights of evidence were free parameters (M9–12) were further added to be more comprehensive.

All models assumed that the corresponding DV produced probabilistic preferences by applying the DV to a standard softmax function (Equation 1). For each subject, a free parameter that controlled how strongly related choices were to the DV, $\lambda$, was estimated. Moreover, the value of the DV at which each subject was expected to be indifferent (i.e., p(happy) = 0.5), c was estimated for F‐attention trials and H‐attention trials separately (c _F and c _H, respectively). When labeling the choice decisions, “Fearful” choices included “fearful” decisions in the F‐block and “not happy” decisions in the H‐block, whereas “happy” choices included “happy” decisions in the H‐block and “not fearful” decisions in the F‐block. Please note that the “Fearful” and “Happy” terms were for labeling purpose, and we did not assume that “not happy” decisions are equivalent to “fearful” decisions and vice versa.

$$\left\{\begin{array}{c}p\left(\text{happy}\right)=\frac{1}{1+e^{\left(-\lambda \left(\mathit{DV}-C_F\right)\right)}}\\ p\left(\text{happy}\right)=\frac{1}{1+e^{\left(-\lambda \left(\mathit{DV}-C_H\right)\right)}}\end{array}\right.$$ (1)

Parameter values were taken as those minimizing the total negative log‐likelihood of the choice data. These values were estimated for each subject and model using the optimization routines in the SciPy Python package (Virtanen et al. 2020). Following model fitting, the mean of each model's negative log‐likelihood were calculated to summarize which model best described the participants' task behavior (i.e., by which model yielded the lowest negative log‐likelihood). Last, a paired t‐test was applied to compare the c _F and c_H of the winning model to investigate any differences in decision‐making criterion under different endogenous attention conditions.

1.7. Transparency and Openness Statement

We preregistered the experiment on the Open Science Framework (OSF). Preregistrations and materials are made available on the OSF at https://osf.io/9dpcj. Preregistration was done subsequent to data collection but prior to data analysis. Minor deviations from the preregistered plan were justified in the Data Analyses portion of the Method section. Sample stimulus, processed data, and code are available for the review process upon request. A public link will be generated prior to publication. During manuscript editing, ChatGPT 3.5 available through HKU (https://chatgpt.hku.hk) was occasionally used to check grammar (OpenAI 2023).

2. Results

2.1. Behavioral Results

As shown in Figure 2, a considerable proportion of decisions of the same stimuli were inconsistent compared between F‐attention and H‐attention conditions. Participants, when faced with the exact same stimuli, chose “Not Fearful” under F‐attention and “Not Happy” under H‐attention for 26.6% of the trials (SD = 13.1%). Moreover, 11.5% of the trials (SD = 7.91%) were chosen as “Fearful” when under F‐attention but “Happy” under H‐attention on average.

FIGURE 2.

Decision‐making behavior. Pattern of choices made under different endogenous attention conditions. Each bar represents responses to the same stimuli under different endogenous attention conditions. Both light gray bars denote choices that were consistent under different endogenous attention conditions. The dark gray bars refer to choices that were inconsistent across the same stimuli but under different endogenous attention conditions. For example, the rightmost dark gray bar shows the proportion of trials where a “Fearful” choice was made in the F‐block, whereas a “Happy” choice was made in the H‐block when responding to the same stimuli. Error bars denote one standard error.

To explore whether there were any differences in overall eye movement features across the two endogenous attention conditions, dependent t‐tests were conducted. Participants exhibited 21.69 fixations on average in the F‐attention trials and 21.45 fixations on average in the H‐attention trials (SD_fear = 3.50; SD_happy = 3.84, t(39) = 0.88, p = 0.39, Cohen's d = 0.27). They fixated on F‐attention trials for 4836 ms on average, whereas H‐attention trials were fixated on for 4731 ms on average (SD_fear = 639 ms; SD_happy = 718 ms, t(39) = 1.72, p = 0.094, Cohen's d = 0.27), respectively. There was no evidence supporting that there were differences in eye movement summary statistics between the two endogenous attention conditions (p > 0.05).

2.2. Endogenous Attention Influences Visual Attention Allocation

Visual inspection of the eye movement data showed the U‐shape as hypothesized (Figure 3A–C). Participants sampled extreme emotions more, especially when congruent with the endogenous attention, showing a tilted U‐shape in favor of evidence of the attended emotion. This was further confirmed by a significant interaction between endogenous attention and ROI in fixation count (F(7, 273) = 4.89, p < 0.001, η _p ² = 0.11), gaze duration (F(7, 273) = 5.83, p < 0.001, η _p ² = 0.080) and marginal transition probability (F(7, 273) = 3.39, p = 0.002, η _p ² = 0.11) in the 8 × 2 rmANOVA conducted for each eye movement measure.

FIGURE 3.

Eye movement results. The mean (A) fixation count (%), (B) gaze duration (%), and (C) marginal transition probability toward the eight images across the two endogenous attention conditions in all participants. The mean (D) fixation count (%), (E) gaze duration (%), and (F) marginal transition probability toward the different levels of ROI extremity. OutF refers to outlying fearful (ROIs 1 and 2), Inlying refers to inlying (ROIs 3–6) and OutH refers to outlying happy (ROIs 7 & 8). Red refers to (F)earful‐attention, and blue refers to (H)appy‐attention. Error bars denote one standard error.

Across all eight ROI measures, ROIs with the most extreme values (i.e., the outlying ones) tend to be attended more when they were attention‐congruent than incongruent, while the ones with less extreme values (i.e., inlying ROIs) show no visible difference between F‐ and H‐attention conditions. We further quantified this interaction by grouping the eight ROIs into three extremity levels (Figure 3D–F). There was a significant interaction between endogenous attention and the three levels of ROI extremities in fixation count (F(2, 78) = 10.72, p < 0.001, η _p ² = 0.22), gaze duration (F(2, 78) = 12.07, p < 0.001, η _p ² = 0.24), and marginal transition probability (F(2, 78) = 10.23, p < 0.001, η _p ² = 0.21). Specifically, OutF had a significantly higher proportion of fixation count, gaze duration, and transition probability under F‐attention compared to H‐attention (p _fixcount = 0.012; p _gazedur = 0.005; p _transprob = 0.035). Likewise, OutH had a significantly higher proportion of eye movement under H‐attention compared to F‐attention (p _fixcount = 0.043; p _gazedur = 0.039; p _transprob = 0.048; see Table 2). Last, there was no significant difference between the inlying ROIs in both attention conditions (p > 0.05).

TABLE 2.

Mean and standard deviation of eye movement measurements to each level of ROI extremity in each attention condition.

ROI extremity	OutF				OutH
	F‐attn		H‐attn		F‐attn		H‐attn
	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Fixation count (%)	10.44	0.19	9.92	0.18	10.46	0.20	9.981	0.18
Gaze duration (%)	10.33	0.21	9.72	0.22	10.43	0.22	9.81	0.20
Transition robability	0.109	0.00061	0.108	0.00053	0.109	0.00052	0.108	0.00052

2.3. Endogenous Attention Affects Evidence Integration by Shifting Decision Criterion

As shown in Figure 4A, Model 2 fit the participants' choice data better than the 12 other models indicated by mean negative log likelihood. M2 was a simple average of the continuous values of the SEV, and it outperformed the basic model M0, any of the models taking a categorical approach (M1, M3–5), any of those incorporating eye movement as weights (M6–8) and those where the weight of each ROI bin was a free parameter (M9–12). The winning model, M2, suggests that all evidence was weighed equally. The results imply that no additional weight was given to evidence based on eye movement. This is further supported by the directly estimated weights from M9 to M12 showing a rather flat distribution (Figure S1).

FIGURE 4.

Computational modeling results. (A) Mean of each model's minimized cumulative log likelihood. The results show that Model 2, where the DV was calculated as a simple average of all SEV, had the lowest mean log likelihood, deeming it the winning model. (B) In the winning model (M2), the estimated decision criterion for F‐attention and H‐attention trials was compared to reveal that endogenous attention indeed leads to a more conservative decision criterion. Error bars denote standard error.

In the best‐fitting model, the trials of F‐attention had a lower indifference point (i.e., more fearful) than H‐attention trials (Figure 4B), indicating that overall, the decision criterion of a trial was affected by endogenous attention favoring a “no, not the target emotion” decision. This result suggests that endogenous attention‐guided decisions induced more conservative decision criteria. A paired samples t‐test showed that the decision criteria of F‐attention trials, c _F, were significantly lower (mean = 46.04, SD = 9.44) than H‐attention trials, c _H, in the winning model M2 (mean = 55.55, SD = 11.03; t = −3.47, p = 0.001, d = −0.55). The means and standard errors of the log likelihood and criterion (c _F and c _H) of every model are listed in Tables S1 – S3. Together, the modeling results revealed that (1) visual attention did not translate into how participants weighed the evidence and (2) endogenous attention induced a more conservative decision criterion.

3. Discussion

Judging the emotional nature of a scene not only depends on the environment but is also affected by endogenous factors such as attention and expectation. In an emotion‐guided multi‐evidence decision‐making task, we used eye‐tracking and computational modeling to dissect the impacts of endogenous attention on how people gather emotionally laden evidence and form the overall perceptual decision. The findings extend our previous work (Ngai et al. 2025), with consistent findings and patterns reflecting the reliability and robustness of this study.

The eye‐tracking results first revealed that extreme emotional stimuli attract more visual attention than less extreme evidence. This finding may be unique to emotional stimuli due to their inherent saliency (Calvo and Nummenmaa 2016; Maratos and Pessoa 2019; Treue 2003) and replicates the findings from our previous work involving deliberate emotion ensemble decision‐making (Ngai et al. 2025). Studies using nonemotional stimuli (e.g., judging colors or shapes) have in fact shown that extreme evidence is downsampled and downweighed (de Garlle and Summerfield 2011; Epstein et al. 2020; Li et al. 2017; Vandormael et al. 2017). These studies postulate that such evidence is being treated as outliers due to hierarchically structured pooling. This pooling mechanism proposes that outliers are dynamically discounted before the integration process, without a distinct selective mechanism (Lee and Chong 2024). On the other hand, it seems evolutionarily beneficial that when asked to sample multiple pieces of evidence over a longer period of time, people intentionally and purposefully attend to specific pieces of evidence that are deemed particularly important for their final decision. Especially in emotion‐related decision‐making, it can be adaptive to give extreme evidence more attention, as more intense evidence typically indicates more consequential outcomes (e.g., a highly fearful face versus a mildly surprised face mean quite different levels of threat) (Adolphs 2008; Feldman Barrett 2011). Thus, there is a natural motivation to be biased toward extreme emotional evidence (Yuan et al. 2007). In some ensemble perception studies using emotional stimuli, the outlying emotional faces of each trial were found to be downweighed in participants' decisions, which is in direct contrast with our findings (Epstein et al. 2020; Haberman and Whitney 2010). However, it is to be noted that the outliers in those studies refer to faces that are obvious outliers deviating from the trial's mean largely (i.e., having OEV above or below 3 SD of the mean), while the extreme emotional faces in the current task are not outliers. Instead, the emotional values of all faces for each trial were drawn from one Gaussian distribution. In other words, human participants should indeed consider all pieces of evidence in our task, confirming the attention‐capturing effect of evolutionarily salient stimuli.

On top of this bottom‐up stimulus emotion‐related effect, the current study also demonstrated the effect of endogenous emotion‐guided attention on evidence sampling. The eye movement results further showed that extreme emotional evidence attracted even more attention when it was the target of endogenous attention compared to when it was not. This finding is consistent with the cognitive conceptualization of feature‐based attention, according to which endogenous attention dictates what is relevant (Summerfield and Egner 2009), and this relevancy renders additional salience to extreme evidence, resulting in a bottom‐up and top‐down interaction. Also, theories of visual selection postulate that not only are eye movements driven by the properties of the stimuli field, but when given a larger time window, volitional control based on the observer's expectancies and goals will bias this process of visual attention (Awh et al. 2012; Theeuwes 2010). Here we show that such an attention allocation strategy is further enlarged under top‐down emotion guidance, consistent with the constructivist view of emotion (Barrett 2017; Feldman Barrett 2011).

The computational modeling findings are two‐fold, first unveiling the process of evidence weighing, then also examining the effects of endogenous attention on this process. First, it can be concluded that eye movement measures do not contribute more to the weight of the corresponding evidence than the subjective rating, replicating the findings of our previous work (Ngai et al. 2025). This can be seen by the success of the model which used a simple average of the stimuli's salience in each trial, rather than the models which incorporated eye movement as weights. The flat betas from the linear regression models further support this notion of participants weighing the evidence equally. Moreover, the results revealed that under emotion‐guided attention, humans are more conservative in making a confirmatory decision. In other words, humans exhibit caution under endogenous attention, raising the bar that leads to attention‐congruent decisions. This was supported by the computational modeling results that the indifference point of making a fear‐related decision is relatively more fearful than a happy‐related one. The above findings imply that contrary to previous research illustrating the gaze cascade effect, attention does not increase the value of the choice alternatives (Mormann and Russo 2021), or else participants would be more willing to make emotion‐congruent decisions when their attention was directed to a certain emotion. This dissociation also demonstrates that under endogenous emotion‐guided attention, humans are more cautious in making confirmatory decisions even though the eye movement data revealed that they sample the emotion‐congruent evidence more. Note that this finding further dissociated the overt visual attention and the preference of decision. While the existing gaze cascade literature shows that choices that are attended to more are weighed higher and eventually more likely to be chosen (Pärnamets et al. 2016; Towal et al. 2013), the discrepancy with our findings may be explained by the distinct nature of multi‐evidence decision‐making. Compared to single‐stimulus decision‐making tasks, the multi‐evidence space contains varying sources of evidence that need to be weighed and integrated. The complexity of this process may lead to an evidence‐weighing process distinctive from single‐stimulus judgment. Moreover, the existing gaze cascade effect literature lacks sufficient substantiation for this causal assertion, as alternative explanations that align with the same evidence remain plausible (see Mormann and Russo 2021 for review).

Based on our findings, it can be speculated that manipulating endogenous attention alters an observer's strategy when sampling available evidence and changes decision‐making criterion. Despite the potential impact of top‐down attention modulation on decision‐making, to our knowledge, this study is the first to thoroughly dissect top‐down effects on deliberate multi‐evidence emotion‐related decision‐making. Thus, further research in this area is critical for understanding the role of endogenous attention in decision‐making and its potential implications for real‐world decision‐making. For example, if a speaker walks into a presentation prompted by others to reflect upon whether they will “do well or not,” the outcome may be very different from being prompted on whether they will “perform poorly or not.” Moreover, other possible factors such as probability, feedback or attention training may be further explored in the future in order to see how else this process may be altered. For example, probability‐based expectation effect refers to the phenomenon where expectations about the likelihood of certain events influence perception and decision‐making (Jiang et al. 2013; Summerfield and Egner 2009; Summerfield et al. 2006). If signaled that one decision is more probable, expected stimuli are found to be detected quicker, with greater accuracy and heightened perceptual sensitivity (Bar 2004; De Lange et al. 2018; Stein and Peelen 2015). Future research may explore how the multi‐evidence decision‐making process is impacted when the likelihood of stimulus occurrence is explicitly provided.

The current findings have implications in the clinical field by shedding light on underlying mechanisms of psychological disorders. Certain disorders are characterized by too much attention and weight allocated to extreme evidence. For example, anxiety disorders typically exhibit negative‐emotion‐oriented bias, whereas mania and other risk‐taking behaviors tend to display positive emotion‐oriented bias (Berenbaum et al. 2003; Chan et al. 2009). Using pathological anxiety as a specific example: anxiety is characterized by hypervigilance (Grupe and Nitschke 2013; Power and Dalgleish 2015), which may manifest in an increase in threat‐oriented endogenous attention biasing the perceptual decision‐making process in favor of threatening evidence. This may lead to hypersensitivity towards negative stimuli or a preference toward positive valence stimuli in avoidance of negative valence stimuli (Craske 2010). Examining whether individuals with high anxiety may allocate more overt attention and weight to negative, positive, or neutral evidence may facilitate our understanding of the psychopathology of anxiety in the future. The current findings may contribute to our understanding of pathological emotion‐related decision‐making by identifying potential cognitive and attentional biases. In the future, the development of targeted interventions to address decision‐making difficulties in clinical populations may be achieved.

This study also contains some limitations that should be acknowledged. First, the sample size of participants in this study was limited to a certain demographic, neglecting potential contextual or situational influences. Future research could explore the role of external factors, such as age or cultural background, in modulating the sampling and integration processes of emotional evidence. Also, the study mainly investigated emotional integration in the context of positive and negative emotions. It would be valuable for future research to explore other complex emotions, such as surprise and anger, to provide a more comprehensive understanding of how humans perceive and integrate a wider range of emotional expressions. Last, the set of stimuli used in each trial was from the same face actor, which may not reflect real‐world scenarios as it is impossible to see varying emotional faces of the same person simultaneously. However, this deliberate intentional choice was made to maintain a high level of experimental control and to specifically investigate the dynamic nature of facial expressions within an individual. Future research may investigate a slightly different question by using different face identities in one trial. By addressing these questions, future research can further improve our knowledge about how humans integrate various emotional evidence and how intraindividual factors can yield different emotion perceptions.

In conclusion, the present study enhances our understanding of how endogenous attention influences emotion‐related decision‐making. By examining the impact of endogenous attention on decisions requiring the integration of multiple pieces of emotional evidence, this research contributes to a deeper comprehension of top‐down factors in this domain. These insights can contribute to a more comprehensive understanding of the complex interplay between attention, decision‐making, and emotions, offering potential avenues for real‐world and clinical applications.

Author Contributions

Hilary H. T. Ngai: conceptualization, investigation, writing – original draft, writing – review and editing, visualization, validation, methodology, software, formal analysis, project administration and data curation. Jingwen Jin: supervision, resources, writing – review and editing, investigation, funding acquisition and conceptualization.

Disclosure

Preregistration and open science: We preregistered the study on the Open Science Framework (OSF). Preregistrations, materials are available at https://osf.io/9dpcj. Sample stimulus, processed data, and code are available for review process upon request. A public link will be generated prior to publication. During manuscript editing, ChatGPT 3.5 available through The University of Hong Kong (https://chatgpt.hku.hk) was occasionally used to check grammar. This research received approval from the Human Research Ethics Committee of the University of Hong Kong (ID EA2003039).

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Data S1.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.