Counterintuitive Effects of Negative Social Feedback on Attention

Abstract

Which stimuli we pay attention to is strongly influenced by learning. Stimuli previously associated with reward outcomes, such as money and food, and stimuli previously associated with aversive outcomes, such as monetary loss and electric shock, automatically capture attention. Social reward (happy expressions) can bias attention towards associated stimuli, but the role of negative social feedback in biasing attentional selection remains unexplored. On the one hand, negative social feedback often serves to discourage particular behaviors. If attentional selection can be curbed much like any other behavioral preference, we might expect stimuli associated with negative social feedback to be more readily ignored. On the other hand, if negative social feedback influences attention in the same way that other aversive outcomes do, such feedback might ironically bias attention towards the stimuli it is intended to discourage selection of. In the present study, participants first completed a training phase in which color targets were associated with negative social feedback. Then, in a subsequent test phase, these same color stimuli served as task-irrelevant distractors during a visual search task. The results strongly support the latter interpretation in that stimuli previously associated with negative social feedback impaired search performance.

Attention serves as the mechanism by which the brain selects which stimuli in the environment receive representation. Which stimuli we pay attention to is strongly influenced by learning. Stimuli previously associated with reward are preferentially attended (e.g., Della Libera & Chelazzi, 2009; Raymond & O’Brien, 2009), even when doing so runs counter to current behavioral goals (e.g., Anderson, Laurent, & Yantis, 2011, 2014; Hickey, Chelazzi, & Theeuwes, 2010). The capture of attention by previously reward-associated stimuli is supported by both behavioral (e.g., Anderson et al., 2011; Anderson & Yantis, 2012) and neurophysiological measures (e.g., Anderson et al., 2014; Hickey & Peelen, 2015; MacLean & Giesbrecht, 2015; Qi, Zeng, Ding, & Li, 2013; see Anderson, in press b, for a review). Similarly, stimuli associated with aversive outcomes such as monetary loss and electric shock also capture attention (e.g., Schmidt, Belopolsky, & Theeuwes, 2015a,b; Wang, Yu, & Zhou, 2013). Findings such as these support the existence of a mechanism by which attention is biased to select stimuli that have been learned to predict a biologically pertinent outcome (see Anderson, 2013).

Social information has also been shown to influence the allocation of attention. For example, attention is biased in the direction that another person is looking (e.g., Frischen, Bayliss, & Tipper, 2007). From the standpoint of associative learning, stimuli associated with the self, such as one’s own name, have high attentional priority (e.g., Harris, Pashler, & Coburn, 2004; Moray, 1959; Sui, Chechlacz, & Humphreys, 2012; Sui, He, & Humphreys, 2012). Positive social feedback can be processed much like other, more tangible rewards and thereby promote learning (e.g., Izuma, Saito, & Sadato, 2008; Klucharev, Hytonen, Rijpkema, Smidts, & Fernandez, 2009), and recent research demonstrates that stimuli associated with positive social feedback (a face bearing a happy expression) automatically capture attention (Anderson, in press a) much like stimuli associated with tangible rewards such as money and food (e.g., Anderson et al., 2011, 2014; Pool, Brosch, Delplanque, & Sander, 2014).

The role of negative social feedback in the biasing of attention remains unexplored, however, and could be subject to one of two very different psychological processes. On the one hand, negative social feedback often serves to discourage particular behaviors (e.g., Bandura, 1977; French & Raven, 1959), and this reflects how individuals such as parents and educators often strategically employ such feedback. Attentional selection can exhibit characteristics of a teachable behavior (e.g., Chelazzi, Perlato, Santandrea, & Della Libera, 2013; Hikosaka, Yamamoto, Yasuda, & Kim, 2013), and the ignoring of a stimulus can be facilitated when such ignoring is reinforced with positive outcomes (Della Libera & Chelazzi, 2006, 2009). If attentional selection can be curbed much like any other behavioral preference, and if negative social feedback is processed by the attention system as a teaching signal, we might expect stimuli associated with negative social feedback to be more readily ignored. On the other hand, attention is biased towards stimuli that predict aversive outcomes (e.g., Schmidt et al., 2015a,b; Wang et al., 2013), which likely serves to promote detection of these stimuli and thus facilitate the opportunity to flee such outcomes (Anderson, 2013). It is unclear whether negative social feedback affects the attention system in the same way as overt punishment, such as electric shock and monetary loss (Schmidt et al., 2015a,b; Wang et al., 2013). If negative social feedback influences attention in the same way that other aversive outcomes do, such feedback might ironically bias attention towards the stimuli it is intended to discourage selection of. To adjudicate between these two possibilities, the present study was modeled closely after the attentional capture paradigm of Anderson (in press a), but using negative (angry) rather than positive (happy) social feedback. Of interest was whether distractors previously associated with negative social feedback would impair performance, consistent with involuntary attentional capture, or whether these distractors would instead be associated with improved target identification consistent with facilitated ignoring.

Experiment 1

Methods

Participants

Twenty-four participants (18–30 y, mean = 22.4 y, 14 female) were recruited from the Johns Hopkins University community. All reported normal or corrected-to-normal visual acuity and normal color vision. The entire experiment took approximately 50 min to complete, and participants were compensated with $10 (US Dollars) for their time. All procedures were approved by the Johns Hopkins University Institutional Review Board. The sample size was determined a priori based on prior studies of value-driven attention (e.g., Anderson, in press a; Anderson et al., 2011).

Apparatus

A Mac Mini equipped with Matlab software and Psychophysics Toolbox extensions (Brainard, 1997) was used to present the stimuli on an Asus VE247 monitor. The participants viewed the monitor from a distance of approximately 70 cm in a dimly lit room. Manual responses were entered using a standard keyboard.

Training Phase

Stimuli

Each trial consisted of a fixation display, a search array, and a social feedback display (Figure 1A). The fixation display contained a white fixation cross (.5° × .5° visual angle) presented in the center of the screen against a black background, and the search array consisted of the fixation cross surrounded by six colored circles (each 2.3° × 2.3°) placed at equal intervals on an imaginary circle with a radius of 5°. The target was defined as the red or green circle, exactly one of which was presented on each trial; the color of each nontarget circle was drawn from the set {blue, cyan, pink, orange, yellow, white} without replacement. Inside the target circle, a white bar was oriented either vertically or horizontally, and inside each of the nontargets, a white bar was tilted at 45° to the left or to the right (randomly determined for each nontarget). The feedback display consisted of a picture of a face exhibiting either an angry or a neutral expression. The faces were those of 20 male and 20 female models taken from the AR face database (Martinez & Benavente, 1998).

Figure 1.

Sequence and time course of trial events. (A) During the training phase, participants reported the orientation of the bar within the color-defined (red or green) target with a keypress. Independent of whether the response was correct or not, the target display was followed by feedback consisting of the presentation of a face. One target color was associated with a greater probability of an angry face (80%) vs a neutral face (20%), while for the other target color this mapping was reversed. Note that in the actual experiment, the faces used were images of real people taken from the AR face database. (B) In the test phase, participants searched for a shape singleton target (diamond among circles or circle among diamonds) and reported the orientation of the bar within the target as vertical or horizontal. On a subset of trials, one of the nontargets was rendered in the color of a former target from the training phase.

Design

One of the two color targets (counterbalanced across participants) was followed by a face exhibiting an angry expression on 80% of trials and a face exhibiting a neutral expression on the remaining 20% (frequently negative target); for the other color target, these percentages were reversed (infrequently negative target). Each model face was presented a total of three times in each of the two expressions for each participant, with each instance paired pseudorandomly with a color target based on the proportion of angry vs neutral expressions assigned to each color. Each color target appeared in each of the six possible stimulus locations equally often, and trials were presented in a random order.

Procedure

The training phase consisted of 240 trials, which were preceded by 40 practice trials. Each trial began with the presentation of the fixation display for a randomly varying interval of 400, 500, or 600 ms. The search array then appeared and remained on screen until a response was made or 1000 ms had elapsed, after which the trial timed out. The search array was followed by a blank screen for 1000 ms, the social feedback display for 1500 ms, and a blank 1000 ms inter-trial interval (ITI).

Participants made a forced-choice target identification by pressing the “z” and the “m” keys for the vertically- and horizontally-orientated bars within the targets, respectively. They were instructed to respond both quickly and accurately. The content of the social feedback following each search array was independent of the participants’ actual behavior; that is, it was not affected by the speed or accuracy of the response on that (or any) trial. Participants were only informed that the faces would “react to what happened on each trial.” Participants were not explicitly informed that these reactions were related to the color of the target, which could only be learned through experience in the task. If the trial timed out, the computer emitted a 500 ms 1000 Hz tone. A mandatory 30-sec break period was provided every 60 trials.

Test Phase

Stimuli

Each trial consisted of a fixation display, a search array, and (in the event of an incorrect response) a feedback display (Figure 1B). The six shapes now consisted of either a diamond among circles or a circle among diamonds, and the target was defined as the unique shape. On a subset of the trials, one of the nontarget shapes was rendered in the color of a former target from the training phase (referred to as the distractor); the colors of the rest of the shapes were drawn from the same set used during the training phase, and the target was never red or green. The feedback display only informed participants if their prior response was incorrect.

Design

Target identity, target location, distractor identity, and distractor location were fully crossed and counterbalanced, and trials were presented in a random order. Distractors were presented on 50% of the trials, half of which were frequently followed by an angry expression during training and half of which were infrequently followed by an angry expression during training (negative-frequent and negative-infrequent conditions, respectively).

Procedure

Participants completed the test phase following a brief rest period that separated the training and test phases, which were performed in the same experimental session. Participants were instructed to ignore the color of the shapes and to focus on identifying the unique shape both quickly and accurately, using the same orientation-to-response mapping. The test phase consisted of 240 trials, which were preceded by 32 practice (distractor absent) trials. In the event of an incorrect response, the search array was followed immediately by the word “Incorrect” centrally presented for 1000 ms (feedback was omitted following a correct response); no faces were shown during the test phase. Each trial ended with a 500 ms ITI. Trials timed out after 1500 ms. As in the training phase, if the trial timed out, the computer emitted a 500 ms 1000 Hz tone. A mandatory 30-sec break period was provided every 60 trials.

Data Analysis

Only correct responses were included in all analyses of RT, and RTs more than 2.5 SDs above or below the mean of their respective condition for each participant were trimmed. All manipulations and measures are reported.

Results and Discussion

Training Phase

Participants were neither significantly slower, t(23) = 0.60, p = .552, nor less accurate, t(23) = 0.63, p = .535, to report a target associated with a high probability of negative social feedback (mean RT = 615 ms, mean accuracy = 89.6%) than a target associated with a low probability of negative social feedback (mean RT = 610 ms, mean accuracy = 90.2%). This suggests that participants searched for each target color with roughly equal priority, as is commonly observed in this paradigm using a reward manipulation (e.g., Anderson, in press a) and also using electric shock feedback (Wang et al., 2013). Of primary interest was how the experience of these stimulus–outcome associations would influence attention in the test phase, when the same color stimuli were presented as task-irrelevant distractors, thereby providing a sensitive measure of automatic attentional processing.

Test Phase

Planned comparisons probing the effect of the distractors on performance, following prior studies (e.g., Anderson et al., 2011; Anderson, in press a), yielded a clear and consistent pattern indicative of attentional capture as opposed to facilitated ignoring (Figure 2). Participants were slower to report the target on high-negative distractor trials compared to distractor-absent trials, t(23) = 2.23, p = .035, d = .46. RT was also numerically, but not significantly, slower on low-negative distractor trials compared to distractor-absent trials, t(23) = 1.48, p = .152, d = .30. Accuracy was lower on both high-negative and low-negative distractor trials compared to distractor-absent trials, t(23) = 4.11, p < .001, d = .84, and t(23) = 2.42, p = .024, d = .49, respectively. Performance appeared to be impacted little by the probability of negative feedback, as neither RT nor accuracy significantly differed between the high-negative and low-negative distractor conditions, t’s < 1, p’s > .35.

Figure 2.

Behavioral performance. (A) Mean response time and (B) accuracy by distractor condition in the test phase of Experiment 1. Error bars reflect the within-subjects S.E.M.

Experiment 2

The results of Experiment 1 suggest that negative social feedback biases attention towards the stimuli it is associated with, and are inconsistent with an account in which negative social feedback serves as a teaching signal discouraging selection. However, it is unclear whether the observed attentional capture was driven specifically by the negative valence of the social feedback, or whether it was driven by an association with social feedback more generally or by outcome-independent search history (i.e., the former target status of the distractors). This is because, unlike in Anderson (in press a), no differences were observed between distractors associated with a high and low probability of valenced feedback. If the attentional capture observed in Experiment 1 merely reflected attention to former targets, independent of the nature of the associated outcomes, or attention to stimuli previously associated with any social outcome, attentional capture should still be observed if all of the social outcomes were neutral. Experiment 2 tests for this possibility by using only neutral faces as feedback. No or significantly reduced attentional capture under these conditions would confirm the importance of the negative valence of the feedback in biasing attention.

Methods

Participants

Twenty new participants (18–29 y, mean = 21.5 y, 11 female) were recruited from the Johns Hopkins University community. Compensation and inclusion criteria were the same as in Experiment 1.

Apparatus

The apparatus was identical to that used in Experiment 1.

Training phase

The training phase was identical to that of Experiment 1 with the exception that only the neutral faces were used as feedback. This amounted to replacing each presentation of an angry face with its neutral counterpart.

Test phase

The test phase was identical to that of Experiment 1.

Data analysis

Because there was no difference in the nature of the social feedback associated with each target color during training, trials were categorized as distractor present vs absent during the test phase and a single measure of mean RT and accuracy was computed for the training phase. Otherwise, the data analysis procedures were the same as in Experiment 1.

Results and Discussion

Training phase

Mean RT in the training phase was 618 ms and mean accuracy was 87.5%.

Test phase

Performance was not significantly affected by the distractors when the social feedback was consistently neutral during training. Mean RT was 702 ms on distractor absent trials and 705 ms on distractor present trials, t(19) = 0.58, p = .570. Mean accuracy was 89% on distractor absent trials and 88.9% on distractor present trials, t(19) = 0.13, p = .901.

Between experiments comparison

Collapsing across target condition in Experiment 1, mean RT, t(42) = −0.38, p = .703, and mean accuracy, t(42) = 0.94 p = .355, did not differ between experiments in the training phase, nor did mean RT, t(42) = 0.31, p = .758, or mean accuracy, t(42) = 0.55, p = .586, differ on distractor absent trials between experiments during the test phase, suggesting similar levels of overall motivation. Because both RT and accuracy were affected by the distractors in Experiment 1, performance in the test phase was converted into a combined measure of inverse efficiency (IE; Townsend & Ashby, 1978) for the sake of comparison of attentional capture between experiments. The impairment caused by the distractors associated with a high probability of negative social feedback (relative to distractor absent trials) in Experiment 1 (mean IE difference = 42 ms) was significantly greater than the impairment caused by the distractors in Experiment 2 (mean IE difference = 4 ms), t(42) = 2.67, p = .011, d = .81. This provides direct evidence that the impairment caused by the distractors in Experiment 1 was attributable to the negative feedback associated with those stimuli rather than more general social feedback or outcome-independent search history. The difference between the impairment caused by distractors associated with a low probability of negative social feedback in Experiment 1 (mean IE difference = 38 ms) and the distractors in Experiment 2 was marginally significant, t(42) = 1.82, p = .076, d = .56.

General Discussion

Positive social feedback has been shown to bias attention towards the stimuli that predict it (Anderson, in press a). The present study examined the consequence of negative social feedback on subsequent attention. The findings demonstrate that when attending to a stimulus is associated with the receipt of negative social feedback, future selection of that stimulus is potentiated rather than discounted. A control experiment confirmed that the attentional capture observed in the main experiment was attributable to the negative valence of the social feedback rather than association with social outcomes more generally or outcome-independent search history. This suggests that negative social feedback biases attention much like other aversive outcomes (e.g., Schmidt et al., 2015a,b; Wang et al., 2013), in opposition to the manner in which negative social feedback discourages overt behavior as a teaching signal (e.g., Bandura, 1977; French & Raven, 1959).

More broadly, the findings of the present study speak to the learning mechanism underlying value-driven attention. These findings support the idea that associative learning between an arbitrary stimulus and a valenced outcome can give rise to changes in attentional priority, as suggested by studies dissociating value-driven attention from the motivational aspects of reward (Bucker, Belopolsky, & Theeuwes, 2015; Le Pelley, Pearson, Griffiths, & Beesley, 2015; Pearson, Donkin, Tran, Most, & Le Pelley, 2015; Sali, Anderson, & Yantis, 2014). The present study extends this principle of learning by association to negative outcomes. Were value-driven attentional priority the result of instrumental conditioning, the feedback signals provided by negative social outcomes would be expected to discourage the patterns of attentional selection that lead to such outcomes. Instead, a robust bias to select such outcome-predicting stimuli in the future was observed.

The act of attentional selection, in an instrumental sense, may play an important role in the learning of value-driven attentional biases. Importantly, however, when a stimulus is associated with a valenced outcome, the learning of that association appears to facilitate subsequent selection of the predictive stimulus regardless of whether the outcome is positive or negative. This is true even though negative social feedback, unlike other forms of punishment used in prior studies of value-driven attention such as electric shock (Schmidt et al., 2015a,b), does not itself reflect a danger that attentional biases might better prepare the organism for. Instead, negative social feedback can often be prevented by ignoring or otherwise avoiding the stimuli that predict it. Although a reduction in associated behavior patterns following punishment may be true of overt actions, it is not necessarily true of attention.

The neutral distractors of Experiment 2 differed from the negative feedback-associated distractors of Experiment 1 not only with regard to the associated valence but also in the predictability of the associated feedback more generally. That is, the valence of the feedback was invariant in Experiment 2, whereas it varied somewhat unpredictably in Experiment 1. There is evidence from experiments using monetary rewards that variable feedback is alone insufficient to give rise to subsequent attentional biases for associated stimuli (Sali et al., 2014), and using positive social feedback, the same probability structure did not give rise to observable attentional biases for stimuli associated with a low probability of a valenced social outcome (Anderson, in press a). Thus, it seems unlikely that the variability of the feedback is alone responsible for the attentional biases observed in the present study. However, the question remains as to why stimuli associated with a low probability of negative social feedback so robustly affected attention. One possibility is that negative social feedback is an especially potent driver of attentional learning, being sufficient to robustly bias attention even when occurring infrequently. Another possibility is that the negative quality of the feedback shifted the state of the participant (although not in a manner observable in search performance during training), for example by heightening vigilance, resulting in greater learning for goal-related stimuli (targets) or stimuli associated with any uncertain outcome. This latter possibility would reflect a more indirect influence of negative social feedback on attention. More broadly, future research is needed to explore the potential role of state-dependent changes caused by valenced feedback in the development of attentional biases.

The findings of the present study also have important practical implications. In order to curb the attention of another person away from certain kinds of stimuli, it may be more effective to reinforce attention to alternative stimuli that could compete for selection than it is to punish selection of the undesired stimulus. For example, there is some evidence that rewarding attention to neutral faces can alleviate attentional biases towards threatening faces in social anxiety disorder (Sigurjonsdottir, Bjornsson, Ludvigsdottir, & Kristjansson, 2015). Ironically, when parents and educators employ negative social feedback in the hopes of discouraging attention to materials they deem inappropriate, such as pornography, the result might be that these materials become all the more attention-grabbing.