Affective State Influences Perception by Affecting Decision Parameters Underlying Bias and Sensitivity

Abstract

Studies of the effect of affect on perception often show consistent directional effects of a person’s affective state on perception. Unpleasant emotions have been associated with a “locally focused” style of stimulus evaluation, and positive emotions with a “globally focused” style. Typically, however, studies of affect and perception have not been conducted under the conditions of perceptual uncertainty and behavioral risk inherent to perceptual judgments outside the laboratory. We investigated the influence of perceivers’ experience affect (valence and arousal) on the utility of social threat perception by combining signal detection theory and behavioral economics. We created three perceptual decision environments that systematically differed with respect to factors that underlie uncertainty and risk: the base rate of threat, the costs of incorrect identification threat, and the perceptual similarity of threats and non-threats. We found that no single affective state yielded the best performance on the threat perception task across the three environments. Unpleasant valence promoted calibration of response bias to base rate and costs, high arousal promoted calibration of perceptual sensitivity to perceptual similarity, and low arousal was associated with an optimal adjustment of bias to sensitivity. However, the strength of these associations was conditional upon the difficulty of attaining optimal bias and high sensitivity, such that the effect of the perceiver’s affective state on perception differed with the cause and/or level of uncertainty and risk.

The perception of facial actions as threatening is not a simple act of decoding facial movements – it is influenced by both the internal state of the perceiver (e.g., accessible conceptual knowledge; Lindquist, Barrett, Bliss-Moreau, & Russell, 2006; Gendron, Lindquist, Barsalou, & Barrett, In press) and perceptual environment (e.g., background scene; Barrett & Kensinger, 2010) (for a review, see Barrett, Mesquita, & Gendron, 2011; Gendron, Mesquita, & Barrett, In press,). Here, we sought to evaluate how the perceiver’s affective state might serve as an internal context that influences social threat perception within different external, perceptual environments.

Prior research demonstrates widespread influence of perceivers’ affective state on perception. For example, affect influences changes in sensitivity to different spatial frequencies (Phelps, Ling, & Carrasco, 2006; Bocanegra & Zeelenberg, 2009), field of view (Schmitz, De Rosa, & Anderson, 2009), estimates of height (Stefanucci & Proffitt, 2006) and steepness (Stefanucci, Proffitt, Clore, & Parekh, 2008), estimates of temperature and weight (Avramova, Stapel, & Lerouge, 2010), conscious awareness of affective stimuli (Anderson, Siegel, & Barrett, 2011), and attribution of affect to neutral (Anderson, Siegel, White, & Barrett, In press) or ambiguous (Bouhuys, Bloem, & Groothuis, 1995) stimuli.

The influence of affect on perception have been characterized in several ways. Some research has shown that the perceiver’s affective state can bias perception of affect-laden stimuli in a direction congruent with that affective state (e.g., Bower, 1991; Bouhuys, et al., 1995). For example, participants induced to feel depressed (vs. elated) rated ambiguous facial expressions as more sad than did other participants (Bouhuys, et al., 1995). Other research argues that a person’s affective state can influence the style of processing, sometimes characterized as effort, depth, or global vs. local focus (reviewed by Schwarz & Clore, 2007). For example, people induced to feel happy expended less effort to evaluate stimuli (reviewed by Schwarz, 1990), showed greater reliance on heuristics when categorizing stimuli (Park & Banaji, 2000), showed less ability to adjust their behavior following errors (van Steenbergen, Band, & Hommel, 2010), and showed greater influence of perceptual context (Avramova, et al., 2010). Finally, other research argues that affect may be associated with perceivers’ propensity to switch processing style, rather than dictating the direction of the association itself (Huntsinger, Clore, & Bar-Anan, 2010). For example, when Huntsinger, et al., 2010 people induced to be in a positive state and primed to be locally focused stayed locally focused, while people induced to be in a negative state and primed to be locally focused adopted a globally focused style. Huntsinger et al. (2010) attribute the commonly reported link between negative state and local focus to a propensity for people to employ a global focus style, so that induction of negative state, typically causes a switch to a local focus style.

One limitation of prior studies is that they have tended not to investigate the interaction of affective state with the uncertainty (e.g., perceivers cannot always be sure of what they are seeing) and risk (e.g., being incorrect can be costly) inherent to perception outside of the laboratory. Current theories imply that perceivers’ affect should exert its influence on perception regardless of uncertainty or risk. For example, if one hypothesizes that negative valence will influence perception of scowling faces in a mood-congruent or locally-focused direction, current theories offer no expectation that the relationship should be different among environments that differ on: (1) the ease of distinguishing threats from non-threats (uncertainty), (2) the relative encounter rate with threats or costs of incorrect perception (risk), or (3) the difficulty of optimizing one’s perceptual decisions under different levels of uncertainty and risk. However, uncertainty and risk clearly influence perception, as documented in the psychophysics literature (e.g., Commons, Nevin, & Davison, 1991). It is conceivable that affect could influence how perceivers respond to risk and uncertainty separately from how affect influences the better understood characteristics, such as mood congruency and processing style. An understanding of how affect influences perception will be incomplete until interactions of affect with uncertainty and risk are delineated.

Here, we take the perspective that perception is a decision, albeit a decision that perceivers are typically unaware of making. As a decision, perception is characterized by uncertainty and risk. For example, sometimes scowling people look threatening when in fact they are not (they may be concentrating, or angry about something else) and sometimes people don’t clearly show their feelings (they might appear unthreatening when they in fact are). Furthermore, failing to correctly note when something is a threat can have different costs than seeing threat where it does not exist. For example, failing to respond appropriately when someone is angry at you may accrue different personal or social costs than does misattributing anger where it does not exist. Likewise, correctly identifying threat can have different benefits than correctly rejecting threat in favor of an alternative conclusion.

To capture the decision-like characteristics of perception, and examine how a perceiver’s affect might interact with them, we used signal detection theory (SDT) within a behavioral economic framework (Lynn, 2005, 2006, 2010). SDT characterizes a perceiver’s response to uncertainty and risk by quantifying the perceiver’s ability to discriminate signals of one kind (targets, e.g., facial and body actions indicating threat) from another (foils, e.g., non-threatening actions), called sensitivity; and the perceiver’s tendency to categorize any signal as target vs. foil, called bias. A tendency to categorize signals as targets, or liberal bias, incurs many correct detections of true threats, but also many false alarm reactions to non-threatening circumstances as if they were threats. A tendency to identify signals as foils, or conservative bias, incurs many correct rejections of non-threats, but also many missed detections of true threats.

In SDT, any given perceptual environment can be characterized by three signal parameters: (1) how frequently the perceiver encounters targets (e.g., threatening people), called base rate; (2) the payoffs associated with the four possible decision outcomes, i.e., costs of a missed detection of threat in another person or of a false alarm (incorrectly perceiving someone as threatening), and benefits of correctly detecting threat in another person or of correctly deciding that a person is not threatening; and, (3) the perceptual similarity of target and foil, e.g., the physical similarity of facial expressions indicative of threat (target), vs. non-threat (foil). A perceiver’s bias is largely influenced by base rates and payoffs (e.g., rare targets or costly false alarms each promote conservative bias), while his or her sensitivity is influenced by the perceptual similarity between targets and foils (e.g., people are more sensitive when targets and foils are less similar to one another) (Green & Swets, 1966; Macmillan & Creelman, 1991).

Within this parameterized SDT framework, overall performance is measured by utility, the net benefit accrued over a series of decisions. Utility is the outcome of a series of correct detections and correct rejections (which accrue benefits) minus the outcome of a series of false alarms and missed detections (which accrue costs). A perceiver’s sensitivity and bias both influence the utility of his or her perception. Sensitivity affects utility by its influence on absolute number of errors committed. Higher sensitivity results in fewer false alarms and missed detections. Bias affects utility by influencing the number of false alarm and missed detection errors relative to each other. The amount and direction of bias that is optimal in a given environment is a function of the environmental base rate and payoff values, but also of the perceiver’s level of sensitivity (Supplemental Figure S3 describes the relationship of bias to sensitivity).

The current study

We examined the association between perceivers’ feelings and their ability to judge social threat under three different perceptual environments. We created a threat perception task in which participants categorized faces depicting scowls of varying intensities as either “more threatening” or “less threatening.” We used affective images and music to induce variation in participants’ hedonic valence (feeling of pleasantness—unpleasantness) and arousal (feelings of low—high activation). We used different values of the three signal parameters (base rate, payoffs, and similarity) to create three environments with different levels of perceptual uncertainty and behavioral risk. A low base rate of threat created a conservatively biased environment, emphasizing risk of misidentification due to frequency of occurrence. A high cost of missed detections of threat created a liberally biased environment, emphasizing risk of misidentification due to differential costs of mistakes. A high perceptual similarity of threatening vs. non-threatening signals created an environment emphasizing risk of misidentification due to perceptual uncertainty. On each trial, participants viewed a scowling face (of variable scowl intensity from trial-to-trial) and judged the face as “more” or “less” threatening. They earned points for each correct judgment and lost points for each incorrect judgment. Participants were instructed to earn as many points as they could (i.e., optimize their perceptual judgments of the faces). As measures of performance, we analyzed points earned, response bias, perceptual sensitivity, and a novel index of signal detection optimality that measures perceivers’ bias and sensitivity relative to an optimality criterion calculated from the values of the three signal parameters (see Supplemental Material).

We hypothesized that the affective state associated with highest utility would differ across the three environments. We predicted that the regression relationship between affective state (subjective ratings of valence and arousal) and utility (points earned) would differ across perceptual environments for two reasons. First, the different levels or causes of uncertainty and risk implemented in our task may play to the strengths of different affective states, as characterized in the prior literature. For example, achieving optimal sensitivity under conditions of high similarity might be promoted by the increased influence of discriminative context on perception afforded by positive valence (Avramova, et al., 2010), while optimizing bias to account for changing base rates or payoffs might be impaired by the lack attention to details associated with positive valence (Stroessner, Hamilton, & Mackie, 1992). Thus, the strength and direction of relationships of affective state with overall performance could differ among the three conditions. Second, valence and arousal often have not been examined as separate influences on perception in prior studies. Although in the literature the influence of affect on perception is discussed in terms of valence, in practice, subjective affect has often been measured as endorsement of a categorical emotion, e.g., “happy” (positive valence, across a possibly wide range of arousal) vs. “sad” (negative valence, typically at low arousal) (Forgas, 1995). Without separate analysis of valence and arousal, effects attributed to positive vs. negative valence might also be due to high vs. low arousal. This possible confound leaves room for doubt as to the dominance of a particular state (combination of valence and arousal) across all three task conditions.

High utility results from both high sensitivity and optimal bias, and our design permitted us to examine the influence of affect on these components of utility separately. We predicted that valence would be associated with bias, such that people feeling more unpleasant would exhibit a more optimal amount of bias than would people feeling pleasant. We reasoned that unpleasant valence should cause more optimal bias due to “local focus” processing characteristics, such as improved attention to category details (e.g., base rate [Stroessner, et al., 1992]) and improved behavioral adjustment in response to errors (van Steenbergen, et al., 2010). Based on prior studies of the influence of arousal on signal detection (reviewed by Matthews & Davies, 2001), we hypothesized that arousal would be associated with sensitivity, such that people feeling more activated would exhibit better sensitivity. Because our measure of bias optimality–distance to a line of optimal response–is novel, we made no a priori predictions about how arousal might influence it.

Method

Participants

Participants were two hundred and fifteen people, largely undergraduate Psychology majors (118 females and 97 males; mean age = 20.2 ± 2.51 [SD] years). Participants gave informed consent according an IRB-approved protocol and were compensated with $15 or 1.5 research participation credits.

Affective State Inductions

Participants were assigned to receive pleasant, unpleasant, or neutral affective state inductions with the goal of creating continuous variation in valence and arousal within the sample, for use in regression analyses. Affective state induction comprised watching a 4.5 minute computer presentation of affective images (International Affective Picture System; Lang, Bradley, & Cuthbert, 2005) accompanied by affect-congruent music (Supplemental Table S1). Before and after affective state induction, participants rated their current level of arousal (sleepy–excited) and valence (unpleasant–pleasant) on separate 9-point scales. See Supplemental Material for data on efficacy of the inductions.

Participants were told that the aim of the study was to investigate the influence of “concentration” (i.e., the threat perception task) on how people feel (i.e., their response to the affective state inductions). This was the reverse of the study’s actual aim, and so participants were unaware that the purpose of experiencing the images and music was to induce an affective state with the intent of investigating its influence on their perception.

Threat Perception Task

Framework

We modeled threat perception by adapting a utility-based signal detection framework developed for studies of animal learning (Lynn, 2006, 2010), which we call the Signal Utility Estimator (SUE) model. The framework uses the SDT utility function (Green & Swets, 1966: Equation 1.14; Supplemental Material) to predict optimal decision criterion placement given values of the three signal parameters (base rate, payoff, and similarity).

The perceptual similarity parameter was implemented as two classes of stimuli, “more threatening” (targets) and “less threatening” (foils), which varied over a range of scowling face intensity (Figure 1). Each stimulus class was defined by a Gaussian distribution with particular mean and standard deviation, e.g., the 40%-scowling morph as the mean foil and the 60%-scowling morph as the mean target, with standard deviation = 10% for both distributions. The presence of variance meant that the exact same stimulus would be shown with some likelihood as a target exemplar on some trials, and with some alternative likelihood as a foil exemplar on other trials, the likelihoods being specified by the distribution of the respective Gaussian functions over the range of scowl intensities. The base rate parameter controlled the proportion of trials drawn from the target vs. foil distributions. There was thus a correct answer for every trial. However, due to overlap of the distributions over the stimulus range, participants experienced considerable uncertainty as to what the correct answer was (because the same physical stimulus could be a target on one trial and a foil on another trial). The payoff parameter was implemented as points earned or lost at each trial based on whether the trial resulted in a correct detection, false alarm, missed detection, or correct rejection. A task comprised 178 trials on which participants decided if a stimulus was from the “more threatening” or “less threatening” stimulus class.

Figure 1.

Threat perception as a signal detection issue. As input, the Signal Utility Estimator model uses experimenter-supplied signal parameters (e.g., mean and standard deviation of target and foil threat categories [comprising less <−> more scowling faces], modeled here by green and blue bell-shaped likelihood distributions that define the similarity parameter). The SUE output is the expected utility (red lines) of adopting a decision criterion at any given location on the range of faces. The point of maximum utility is the optimal decision criterion location for a given set of parameter values. Perceiving all faces right of criterion as “threat” will maximize utility (measured as points earned in our experiments). Different environmental conditions (sets of parameter values) have differently shaped utility functions. Some environments call for a liberal (leftward) criterion location, mildly scowling faces on the left of the continuum should be categorized as “more threatening” (thus incurring few missed detection mistakes but many false alarm mistakes). Other environments call for a conservative (rightward) criterion, in which perceivers should not respond to those same mildly scowling faces as “more threatening” (incurring more missed detections but fewer false alarms). For clarity, the y-axis for the signal distributions (Probability Density) and the distribution of foils for the low similarity condition are not shown.

We created four sets of parameter values—four perceptual environment conditions. The conditions differed from each other in the value of one of the three signal parameters (Table 1). In the Baseline Condition, we set base rate of threat to 0.50 (50% of trials were drawn from the target stimulus class). Correct detection of targets (faces drawn from the “more threatening” distribution) and correct rejection of foils (faces drawn from the “less threatening” distribution) each earned +10 points. False alarms to foils lost 7 points and missed detection of targets lost 3 points. The 40%-scowling morph was the mean foil and the 60%-scowling morph was the mean target, with standard deviation = 10% for both distributions. All else being equal, the slightly higher false alarm vs. missed detection cost created a mildly conservatively biased perceptual environment.

Table 1.

Signal parameter values defining four perceptual environment conditions. The low base rate condition implemented a lower base rate of occurrence of “more threatening” faces than other conditions. The costly-miss payoff condition implemented greater loss of points for missed detections and less loss of points for false alarms other conditions. The high similarity condition implemented more similar means of the “more” and “less” threatening signal distributions than other conditions. Payoffs are in units of points. Similarity mean and SD are in units of percent-scowling.

Condition name	Base rate	Payoffs				Similarity
		Correct detection	Correct rejection	False alarm	Missed detection	Targets		Foils
						Mean	SD	Mean	SD
Baseline	0.50	10	10	−7	−3	60%	10%	40%	10%
Low base rate	0.25	10	10	−7	−3	60%	10%	40%	10%
Costly-miss payoff	0.50	10	10	−1	−15	60%	10%	40%	10%
High similarity	0.50	10	10	−7	−3	60%	10%	50%	10%

In the Low Base Rate Condition, we set the base rate of threat to 0.25 (25% of trials were drawn from the target stimulus class). All other parameters were as in the Baseline Condition. The payoff values and low base rate combined to create a moderately conservatively biased perceptual environment (Figure 1). In the Costly-miss Payoff Condition we set the cost of a false alarm to −1 point and the cost of a missed detection to −15 points. All other parameters were as in the Baseline Condition. These payoff values created a liberally biased perceptual environment (Figure 1). In the High Similarity Condition we adjusted the perceptual uncertainty by setting the mean %-scowl of the “less threatening” (foil) distribution to the 50%-scowling morph. All other parameters were as in the Baseline Condition. The high perceptual uncertainty amplified the mild bias established by the payoff values to create a conservatively biased perceptual environment (Figure 1).

Stimuli

A stimulus set consisted of 11 faces ranging from a neutral, relaxed face to a scowling face in 10% increments (Figure 1). Faces were created by digitally blending (MorphMan 4, Stoik Imaging) the neutral and scowling end-point faces. Posed, scowling faces were used because they elicit “automatic” threat-related physiological and behavioral responses in a perceiver (Roelofs, Hagenaars, & Stins, 2010). Faces were converted from color to grey scale, placed against a black background, and rescaled to 500 × 625 pixels at 96 dpi resolution. Six such sets were created, comprising 3 female and 3 male photographic models, all college-aged Caucasians from the MacBrain¹ and IASLab² face sets. Face stimuli were shown for 67 ms then backward-masked with the neutral face of different model. We selected a 67 ms stimulus exposure to attenuate on-line deliberative processing while maintaining a supra-liminal visual experience (Szczepanowski & Pessoa, 2007). The mask remained on-screen until the participant’s behavioral response.

Procedure

During recruitment, we asked participants to refrain from eating for at least two hours prior to the experiment. During the study orientation, we informed participants that they could exchange earned points for rewards—small bags of snacks (M&M® candies, potato chips, cocoa-covered almonds, or graham crackers)—at a rate of 700 points per approximately 30 g serving. Prior to the threat perception tasks, participants sampled each food item and informed the experimenter of the item for which they anticipated exchanging their points. Each kind of snack was present in sealed jars on a table in front of the participant for the duration of the study.

During the experiment, participants sat in a comfortable armchair in a sound attenuating, dimly lit room. All visual stimuli were shown on a 40-inch LCD video monitor (Samsung LNT4065F, 1080p resolution) 1.5 m from the armchair. Faces subtended ca. 7.2 horizontal degrees by 9.5 vertical degrees.

Participants were assigned to one of three affective state inductions (pleasant, unpleasant, or neutral) and one of three test conditions (low base rate, costly-miss payoff, or high similarity), in a fully crossed design. Over the course of participating in a larger experiment, participants (1) experienced an affective state induction in their assigned valence condition, (2) performed a “practice” threat perception decision task in the baseline condition, (3) experienced a second “booster” affective state induction in their assigned valence condition, (4) performed the threat perception task in their assigned test condition, and (5) experienced a neutral affect induction (so that they left the laboratory in a more or less neutral affective state). Participants then exchanged total accumulated points for the snack(s) of their choosing, and were debriefed and dismissed.

At the beginning of the each threat perception task, on-screen instructions informed participants of the base rate and payoffs, and showed two exemplars from the centers of the target and foil distributions. We directed participants to learn how to best categorize the faces as “more threatening” or “less threatening” by attending to the points earned and lost, and to earn as many points as possible. Participants categorized the face stimuli by using their index fingers to press one of two keys on the computer keyboard, labeled “(+)” for “more threatening” and “(−)” for “less threatening”. Position of the appropriate response labels on the “1!” and “+=“ keys was randomized for each participant. On-screen feedback was given immediately after each perceptual decision with the text “Yes, that was right.” or “No, that was wrong.”, the points earned or lost for that trial, and total points accrued over the entire study thus far.

Data Analysis

For each trial, a computer logged the decision outcome (correct detection, correct rejection, false alarm, missed detection), response time, and points earned. Responses occurring in under 300 ms from face image on-set were excluded from analysis due to the high probability of their containing motor errors. From the remaining trials we calculated sensitivity (d′) and bias (c) (Macmillan & Creelman, 1991) and total points earned.

Additionally, for each condition, we used the SUE model to determine the optimal bias for any given sensitivity value, which we call the Line of Optimal Response (LOR, see Supplemental Material). The LOR traces the amount of bias that maximizes utility at any given sensitivity level for particular environmental base rate and payoff values. We determined each participant’s distance-to-LOR, which we call d_O, (distance-sub-Optimal) as shortest Euclidean distance from a point defined as (d′, c) to the LOR. Shorter distance-to-LOR reflects a more optimal bias: At a given level of sensitivity, perceivers whose bias places them closer to LOR accrue more utility (where utility can be indexed as, e.g., points earned over a series of decisions). Because d_O expresses a perceiver’s bias relative to the bias that is optimal in a given environment, it permits a measurement of whether a perceiver is too biased vs. not biased enough in a given environment, taking into account the perceiver’s sensitivity.

We used multiple regression to examine how a perceiver’s affective state (measured as valence and arousal ratings reported immediately after the “booster” affect induction) influenced threat perception during the subsequent test condition (low base rate, costly-miss payoff, or high similarity). Valence and arousal ratings were grand mean centered for each test condition. We dummy-coded the three test conditions to simultaneously compare the influence of affective state among the conditions in a single regression. The four dependent variables, utility (points earned), bias (c), distance-to-LOR (d_O), and sensitivity (d′), were analyzed separately. The influence of affective state on distance-to-LOR was examined controlling for sensitivity. Significant interaction of multiple regression predictor variables (centered valence and arousal ratings) were further examined with simple slopes analysis (Aiken, West, & Reno, 1991).

Results

We predicted that the relationship of affective state with utility would differ across the three environmental test conditions and that this difference would be accompanied associations between valence and bias, and arousal and sensitivity. These predictions were supported, but, unexpectedly, appeared conditional upon the apparent “difficulty” of adapting ones bias and sensitivity to a given environment.

Maximizing Utility

Low Base Rate Condition

The low base rate condition presented a low frequency of targets relative to the other conditions. In this condition, a perceiver’s valence and arousal had separate, additive effects on the utility of threat perception (Figure 2A, Table 2). Perceivers experiencing a pleasant affective state earned significantly fewer points over the course of the task than did perceivers experiencing an unpleasant state (p<0.022). Unrelated to effects of valence, participants experiencing low arousal affective state earned significantly fewer points than did those in a high arousal state (p<0.003). When the base rate of threat was low, it was most effective to feel unpleasant, high arousal.

Figure 2.

Simple slopes resulting from multiple regression of valence and arousal on four dependent variables (utility [points earned], sensitivity [d′], bias [c], and distance-to-LOR [d_O]) in each of three conditions (low base rate, costly-miss payoff, and high similarity). Valence and arousal ratings were grand mean centered for each condition prior to regression. The simple slopes for mean, mean + 1 SD, and mean − 1 SD of arousal (moderate, high and low arousal, respectively) are plotted against mean, mean + 1 SD, and mean − 1 SD valence (neutral, pleasant, and unpleasant valence, respectively).

Table 2.

Results of a single (simultaneous) multiple regression test of the effect of perceivers’ valence and arousal on perception of scowling faces across three perceptual environment conditions. Distance-to-LOR was controlled for sensitivity. P-values for the influence of valence on bias and for arousal on sensitivity are one-tailed, based on directional a priori predictions. All other p-values, including interactions, are two-tailed.

	Utility (points)				Bias (c)				Distance-to-LOR (dO)				Sensitivity (d′)
	B	β	t	p	B	β	t	p	B	β	t	p	B	β	t	p
Low base rate (n=75)
Intercept	900.89	0.66	29.74	<0.001	0.45	0.48	9.63	<0.001	1.20	0.94	23.26	<0.001	0.87	0.64	16.49	<0.001
Valence	−36.39	−0.05	−2.32	0.022	−0.05	−0.10	−1.88	0.031	0.04	0.06	2.27	0.024	−0.02	−0.03	−0.74	0.458
Arousal	46.07	0.07	3.03	0.003	0.01	0.03	0.55	0.585	−0.02	−0.03	−0.92	0.361	0.05	0.07	1.69	0.041
Interaction	−0.12	0.00	−0.02	0.986	0.00	0.01	0.10	0.923	0.00	0.00	0.02	0.986	0.00	−0.01	−0.23	0.816
Costly-miss payoff (n=67)
Intercept	757.75	0.53	23.19	<0.001	−0.52	−0.53	−10.43	<0.001	0.82	0.61	16.99	<0.001	0.72	0.50	12.57	<0.001
Valence	8.18	0.01	0.49	0.627	0.00	0.00	−0.04	0.483	−0.01	−0.01	−0.39	0.698	0.02	0.02	0.61	0.545
Arousal	17.56	0.02	1.07	0.287	0.02	0.05	0.88	0.378	0.01	0.01	0.47	0.642	0.03	0.04	1.03	0.103
Interaction	−18.80	−0.05	−2.35	0.020	0.02	0.08	1.57	0.119	0.02	0.05	1.65	0.100	−0.03	−0.07	−1.78	0.076
High similarity (n=73)
Intercept	624.95	0.46	19.28	<0.001	−0.10	−0.11	−1.99	0.048	0.79	0.61	19.60	<0.001	0.40	0.29	7.15	<0.001
Valence	12.18	0.02	0.90	0.368	0.02	0.06	1.10	0.136	−0.02	−0.05	−1.61	0.109	0.02	0.04	1.02	0.311
Arousal	4.67	0.01	0.24	0.807	−0.07	−0.12	−2.31	0.022	0.05	0.07	2.47	0.014	0.02	0.03	0.60	0.225
Interaction	−14.17	−0.05	−1.86	0.064	0.01	0.02	0.42	0.676	0.00	0.00	0.13	0.901	−0.02	−0.06	−1.38	0.170
Sensitivity (controlled for in Distance-to-LOR regression)									−0.461	−0.496	−10.309	<0.001

Costly-miss Payoff Condition

The costly-miss payoff condition presented missed detections as much more costly than false alarms (relative to the other conditions, in which false alarms were somewhat more costly than missed detections; Table 1). In this condition, valence and arousal interacted to influence the utility of threat perception (Figure 2B, Table 2). Points earned differed little for perceivers experiencing an unpleasant affective state. However, among perceivers experiencing pleasant affect, those who also experienced low arousal earned fewer points than those experiencing high arousal (interaction, p<0.020). Simple slopes analysis found that a pleasant, high arousal affective state was associated with more points earned than were all other affective states. Perceivers reporting pleasant valence (i.e., 1 SD above the mean valence of participants in the costly-miss payoff condition) showed a significant positive association between arousal and points earned (B=54.9, beta = 0.4, t=2.3, p<0.026). The association between valence and points at either 1 SD below or above mean arousal did not reach significance (B=−29.6, beta=−0.2, t=−1.3, p<0.195 and B=46.0, beta=0.3, t=1.7, p<0.085, respectively). When missing a threat was costly, it was most effective to feel pleasant, high arousal.

High Similarity Condition

The high similarity condition presented targets and foils that were more perceptually similar to each other compared to the other test conditions. In this condition, affect appeared to influence the utility of threat perception in several ways, albeit at marginal significance (Figure 2C, Table 2). Points earned was relatively high for both unpleasant, low arousal perceivers and pleasant, high arousal perceivers (interaction, p<0.064). Simple slopes analysis revealed a significant association between valence and points at 1 SD above mean arousal (B=36.7, beta=0.5, t=2.4, p<0.018). There was no significant association between valence and points earned at 1 SD below mean arousal (B=−12.4, beta=−0.2, t=−1.0, p<0.332) but weak associations between points earned and arousal at 1 SD below and above mean valence (B=−30.3, beta=−0.3, t=−1.7, p<0.097 and B=39.6, beta=0.4, t=1.9, p<0.065, respectively). Under high perceptual uncertainty between threat and non-threat, it was effective to feel either pleasant, high arousal or unpleasant, low arousal.

Summary

The relationships of valence and arousal with utility found to be significant in one environmental condition were not present in the other conditions. This inconsistency means that the relationship between how people felt and how they judged potential threats differed depending on the base rate of threats, the cost of missing a threat vs. false alarming, and the perceptual similarity between threat and foil. We found that the strength and direction of relationships between affective state and perception were conditional upon the level and/or causes of risk and uncertainty of the decision-making processes that comprise “perception.” Therefore, to understand why a particular combination of valence and arousal resulted in high utility in a given environment we must examine the relationships of affective state to the perceptual processes that underlie utility: sensitivity and bias.

Calibrating bias to base rate and payoff

As predicted, we found a significant relationship between bias and valence. In the low base rate condition, unpleasant valence was associated with more conservative-going bias (one-tailed p<0.031; Figure 2D, Table 2). When threats were infrequent, pleasant-feeling perceivers were insufficiently conservative, resulting in less optimal bias (distance-to-LOR; p<0.024, Figure 2G, Table 2). However, associations between valence and either measure of bias were not significant in the other conditions (Table 2).

We attributed the inconsistency of these relationships among conditions to differences in the difficulty of calibrating bias to base rate and payoffs. Participants in the low base rate condition achieved lower optimality of bias (higher distance-to-LOR, d_O) than did participants in the costly-miss payoff condition (Table 3, Supplemental Figure S2). The influence of valence on bias was, thus, only observed in the condition in which an adjustment of bias to either base rate or payoff was necessitated by the environment, and was also relatively difficult to accomplish: namely, the low base rate condition (the high similarity condition did not require an adjustment of bias to changes in base rate or payoff). Sampling artifacts did not explain the inconsistent influence of affective state across the conditions: The mean and variance of valence and arousal among participants in the low base rate and costly-miss payoff conditions did not differ, nor did the variance of bias (Table 3).

Table 3.

Comparison of mean and variance of valence, arousal, points earned, bias, distance-to-LOR, and sensitivity among the three perceptual environment test conditions. For each variable, results of the omnibus test across the three conditions are followed by results of paired comparisons. Conditions: Base rate denotes the low base rate condition, Payoff denotes the costly-miss payoff condition, and Similarity denotes the high similarity condition.

Variable	Equality of Mean (ANOVA)		Equality of Variance (Levene’s Test)
	F	p	F	p
Valence	0.5	0.629	3.6	0.029
Base rate vs. Payoff		0.973	0.9	0.935
Base rate vs. Similarity		0.409	5.4	0.022
Payoff vs. Similarity		0.403	4.9	0.028
Arousal	2.6	0.072	1.7	0.178
Base rate vs. Payoff		0.603	0.001	0.928
Base rate vs. Similarity		0.027	2.6	0.107
Payoff vs. Similarity		0.102	2.8	0.097
Utility	24.2	<0.001	8.6	<0.001
Base rate vs. Payoff		<0.001	1.0	0.309
Base rate vs. Similarity		<0.001	16.8	<0.001
Payoff vs. Similarity		0.003	9.6	0.002
Bias	103.9	<0.001	0.191	0.900
Base rate vs. Payoff		<0.001	0.2	0.723
Base rate vs. Similarity		<0.001	0.09	0.767
Payoff vs. Similarity		<0.001	0.3	0.555
Distance-to-LORa	39.1	<0.001	0.8	0.469
Base rate vs. Payoff		<0.001	0.2	0.619
Base rate vs. Similarity		<0.001	0.6	0.454
Payoff vs. Similarity		0.028	1.3	0.259
Sensitivity	23.3	<0.001	10.8	<0.001
Base rate vs. Payoff		0.015	0.09	0.771
Base rate vs. Similarity		<0.001	19.9	<0.001
Payoff vs. Similarity		<0.001	17.3	<0.001

^aMean distance-to-LOR was compared with ANCOVA using sensitivity as a covariate. Residuals from the correlation between distance-to-LOR and sensitivity were submitted to Levene’s test.

Calibrating bias to sensitivity

The LOR defines an inverse relationship between optimal bias and sensitivity. This relationship suggests that, in a perceiver, declining sensitivity (here, implemented as the transition from the baseline condition to the high similarity condition) necessitates an increase in bias (here, more conservative-going in the high similarity condition than in the baseline condition). We found that in the high similarity condition, optimality of bias (distance-to-LOR) was significantly associated with arousal. Perceivers reporting low arousal attained shorter distance-to-LOR than others (p<0.014; Figure 2I, Table 2) and more conservative bias (p<0.022; Figure 2F, Table 2). Associations between arousal and either measure of bias were not significant in the low base rate condition, but were possibly represented in the costly-miss payoff condition as a trend-level interaction on distance-to-LOR (p<0.100; Figure 2H, Table 2). In the costly-miss payoff condition low arousal was associated with more optimal bias for participants feeling unpleasant.

We attributed the inconsistency of these relationships among conditions to differences in the difficulty of calibrating bias to sensitivity. Participants in the costly-miss payoff and high similarity conditions exhibited significantly lower sensitivity than those in the low base rate condition (Table 3, Supplemental Figure S2). The influence of arousal on bias optimality was, thus, only observed in conditions which necessitated calibration of bias to challenged sensitivity: namely, the high similarity and costly-miss payoff conditions.

Achieving High Sensitivity

As predicted, we found a significant relationship between sensitivity and arousal. In the low base rate condition, high sensitivity was associated with high arousal (one-tailed p<0.046; Figure 2J, Table 2). When threats were infrequent, high arousal perceivers were better able to discriminate threats from non-threats. However, the association between arousal and sensitivity was not significant in the other conditions (Table 2). Instead, in the costly-miss payoff condition, we found evidence for a possible interaction of arousal and valence on sensitivity (p<0.076; Figure 2K, Table 2)).

We attributed the inconsistency of this relationship among conditions to differences in the difficulty of achieving high sensitivity. As mentioned above, participants in the low base rate condition exhibited higher sensitivity than did participants in the costly-miss payoff and high similarity conditions (Table 3, Supplemental Figure S2). The breakdown of a clear association between high arousal and high sensitivity, and possible emergence of an interaction with valence was, thus, observed in conditions in which high sensitivity was difficult to achieve, namely, the costly-miss payoff and high similarity conditions.

Discussion

We examined the influence of a perceiver’s affective state on perceptual decision-making in the domain of social threat discrimination. By combining signal detection theory with behavioral economics, we were able to measure the utility, bias, and sensitivity of threat perception over a series of decisions. We found that the exigencies of the environment influenced the details of how perceivers’ experience of valence and arousal affected perception. Inconsistencies in the strength and direction of affect—perception relationships complicate conclusions of prior studies that have shown consistent directional associations between affective state and perception. Our findings indicate that the influence of affective state on perception is conditional upon (i) the level and/or cause of uncertainty and risk, as instantiated by the values of the three signal parameters, and (ii) the “difficulty” of calibrating one’s perception to that uncertainty and risk.

Specifically, we found that experiencing unpleasant valence promoted calibration of bias to those environmental parameters that directly influence bias—the base rate of threat and the benefits and costs accrued for correct vs. incorrect perceptual decisions. However, this association was apparent only in the condition in which the overall bias exhibited by participants at the end of the task was relatively suboptimal. We found that experiencing low arousal promoted calibration of one’s bias to one’s sensitivity (distinct from calibrating bias to base rate or payoff). However, this association was apparent only in conditions necessitating calibration of bias to declining sensitivity. We found that experiencing high arousal promoted high sensitivity. However, this association was characterized by mild interaction with valence in conditions in which high sensitivity was difficult to achieve. In these conditions, the experience of arousal failed to influence sensitivity among participants who felt unpleasant.

The direction of the valence-by-arousal association with sensitivity suggests that the influence of arousal can be enabled or attenuated by the local- vs. global-focus processing style that prior studies have associated with unpleasant and pleasant valence, respectively. In the costly-miss payoff and high similarity conditions, participants experienced difficulty achieving high sensitivity (Supplemental Figure S2). Participants in these conditions experiencing unpleasant valence and high arousal failed to show the effect of high arousal on their sensitivity (Figures 2K & L). In prior literature, experiencing unpleasant emotional states (e.g., sadness) has been associated with a processing style particularly sensitive to “local” details of a perceptual environment or target stimulus class, at the expense of more “global” task features (reviewed by Schwarz & Clore, 2007). In particular, Avramova, et al. (2010) demonstrated that people in an unpleasant state can be insensitive to the discriminative perceptual context in which stimuli are evaluated, such as when stimuli are be evaluated relative to one another or in the presence of distractors. We speculate that high sensitivity—the discrimination of targets as a stimulus class distinct from foils—requires representing what targets look like in the “context” of what foils look like, and that when high sensitivity is difficult to achieve, unpleasant valence becomes a perceptual impairment. We propose that local focus interferes with the sensitivity-promoting effects of high arousal.

We found no consistent evidence of mood-congruent effects. When judging scowling faces, unpleasant valence might be expected to lead to consistently liberal bias. For example, Baumann & DeSteno (2010) have hypothesized that negative valence may be associated with over-estimation of the base rate of threat (see also Mayer & Gaschke, 1988). Our modeling indicates that base rate over-estimation should manifest as suboptimally liberal bias (i.e., large and liberal-going distance-to-LOR). We found instead that unpleasant valence was associated with most more optimal bias (i.e., short distance-to-LOR) in both liberally and conservatively biased perceptual environments.

We anticipated that sensitivity would be equal in the low base rate and costly-miss payoff conditions because those conditions used identical similarity parameter values. Nonetheless, participants in the costly-miss payoff condition did not attain sensitivity equal to that of participants in the low base rate condition. The finding suggests a lack of independence in how payoffs and similarity are “estimated” by the brain. As rewards and punishments, payoffs must figure prominently in informing the brain about both the base rate and similarity of targets to foils. Having to learn new values for costs and benefits (from baseline to the costly-miss payoff condition) may to have interfered with perceivers’ ability distinguish target from foil. Parameters may not be independently represented in the brain, in spite of their theoretical independence (see, e.g., Bohil & Maddox [2001] for a discussion).

Participants in the low similarity condition were largely unable to adapt their bias to their low sensitivity. The LOR indicates that, in environments in which some amount of bias is optimal, the low utility associated with reduced sensitivity can be somewhat mitigated by adopting a more extreme bias (Supplemental Figure S3). In between-subjects manipulations of similarity, we have found this relationship to hold (Lynn & Barrett, unpublished data). The failure of that relationship in the current within-subjects manipulation of similarity (from baseline to the high similarity condition) suggests that perceivers have difficulty adapting an established relationship between bias and sensitivity to a decrease in sensitivity.

Though human perceivers do adapt their behavior to changes in the three signal parameters in directions predicted by considerations of utility (e.g. Green & Swets, 1966, p 88; See, Warm, Dember, & Howe, 1997), the degree of optimization achieved can be quite variable (reviewed by Dusoir, 1975; Bohil & Maddox, 2001). In this context, our studies can be viewed as an examination of individual differences in perceiver affective state that contribute to variability in the degree of optimality perceivers achieve. We have proposed that how a person feels (valence and arousal) influences how he or she adapts perceptual decision-making to the environmental conditions in which the decisions are being made. This suggests that perceivers could optimize their threat perception by either strategically managing how they feel or perhaps by paying particular attention to parameters with which their current affective state may interfere.

Our studies implemented a limited number of environmental conditions. We manipulated the three parameters in a single direction only. Therefore, the generalizability of the associations of valence with bias and of arousal with distance-to-LOR and sensitivity, and of the influence of optimization “difficulty,” remains to be determined. For example, we have described two conditions in which the ability of perceivers to make effective (i.e., high utility) perceptual decisions unexpectedly appeared jeopardized: increasing the cost of missed detections (which impaired sensitivity) and decreasing similarity (which impaired optimal bias adjustment). However, this study does not indicate whether these impairments would also be found for decreasing the cost of missed detections and increasing similarity, respectively. We also found that how a person feels can have conflicting effects on perception depending on the environment, and these conflicts are another potentially fruitful area for further study. For example, high arousal was associated with high sensitivity, yet low arousal was associated with better adjustment of bias to sensitivity when sensitivity was poor. Sensitivity appears to have a stronger influence on utility than does bias in our data (Figure 2), suggesting that, in general, high arousal will serve perceivers better than low. Nonetheless, the interaction of arousal and valence on utility of perception in the high similarity condition (Figure 2C) indicates that this generalization may not be true in all environments.

Placing signal detection issues in the context of behavioral economics has benefits over the use of accuracy, or sensitivity and bias, alone. Utility permits determination of how much bias is optimal. For example, on its own, our finding that, in the low base rate condition, perceivers feeling pleasant valence exhibited less biased behavior than those feeling unpleasant valence means only that pleasant valence was associated with a tendency to label moderately scowling faces as “more threatening” while unpleasant affect tended to reserve that response for more intensely scowling faces. Bias alone says nothing about which behavioral pattern led to better decision-making. That information is only given by points earned over the series of decisions or a measure of bias optimality, such as distance-to-LOR.