Reward Devaluation: Dot-Probe Meta-Analytic Evidence of Avoidance of Positive Information in Depressed Persons

Abstract

Cognitive theories of depression and anxiety have traditionally emphasized the role of attentional biases in the processing of negative information. The dot-probe task has been widely used to study this phenomenon. Recent findings suggest that biased processing of positive information might also be an important aspect of developing psychopathological symptoms. However, despite some evidence suggesting persons with symptoms of depression and anxiety may avoid positive information, many dot-probe studies have produced null findings. The present review used conventional and novel meta-analytic methods to evaluate dot-probe attentional biases away from positive information and, for comparison, toward negative information, in depressed and anxious individuals. Results indicated that avoidance of positive information is a real effect exhibiting substantial evidential value among persons experiencing psychopathology, with individuals evidencing primary symptoms of depression clearly demonstrating this effect. Different theoretical explanations for these findings are evaluated, including those positing threat-processing structures, even-handedness, self-regulation, and reward devaluation, with the novel theory of reward devaluation emphasized and expanded. These novel findings and theory suggest that avoidance of prospective reward helps to explain the cause and sustainability of depressed states. Suggestions for future research and methodological advances are discussed.

Since first investigated (Gotlib & McCann, 1984; MacLeod, Mathews, & Tata, 1986), depressed and anxious individuals’ automatic and controlled emotional information-processing biases have received much empirical scrutiny (Bar-Haim, Lamy, Pergamin, Bakermans-Kranenburg, & van IJzendoorn, 2007; Williams, Mathews, & MacLeod, 1996). Persons with heightened levels of depression and anxiety are an attractive group for scientific inquiry, in line with cognitive models of psychopathology (e.g., Bandura, 1969; Beck, 1976; Bower, 1981; Lazarus, 1991) that posit that these individuals’ pathogenic self-schemas (i.e. associative networks or systems of appraisals) influence their interpretation and subsequent experience of emotional information. These interpretations may become automatic upon repetition, explaining why biased processing may seem readily apparent to an observer (such as a therapist), while remaining outside of the awareness of the perceiver.

For decades, research in cognitive (Bower, 1981), social (Pratto & John, 1991), and clinical (MacLeod et al., 1986) psychology has explored the possibility that people experiencing depression and anxiety are more attentive to emotional information that is negative or threatening than individuals without symptoms of distress. Indeed, individuals with depressed and anxious symptoms’ bias toward the processing of negative information is now considered well-established (Bar-Haim et al., 2007; Gotlib & Joormann, 2010; Teachman, Joormann, Steinman, & Gotlib, 2012; Yiend, 2010). Using both conventional and new meta-analytic techniques, we will (a) provide updated evidence of the evidential value of negative bias findings for depression and anxiety and, most importantly, (b) explore a related, but conceptually opposite possibility: that people experiencing depression and anxiety have attentional biases away from information that is positive.

People with heightened levels of depression or anxiety process negatively-toned information in a biased fashion. They respond more quickly to, and find it more difficult to disengage from, negative information, than do others (Gotlib & McCann, 1984; Williams & Nulty, 1986; Yiend, 2010). Advanced understanding of attentional biases of depressed and anxious individuals toward negative information is currently being translated into applied clinical science (e.g., Amir, Beard, Burns, & Bomyea, 2009; Hallion & Ruscio, 2011), although these potential advances come with some notable caveats (Rapee et al., 2013). However, far less is understood regarding biases for positive information.

Reviewing the Literature

Selective Attention in the Dot-Probe Task

This review will focus on one of the most common tasks in selective attention research, the dot-probe (i.e., attentional-probe; DP) task (MacLeod et al., 1986; Posner, Snyder, & Davidson, 1980). In the DP paradigm, stimulus-pairs consisting of words, pictures, or faces are presented either vertically, horizontally, or diagonally on opposite sides of a computer screen. After the stimulus-pairs, a probe is presented following one of the two stimuli, and the participant’s task is to identify the probe as quickly as possible. A valenced stimulus and a neutral stimulus comprise the stimulus pairs. Bias scores are calculated by comparing response times to probes that appear in the same location as the neutral stimulus with response times to probes that appear in the same location as the emotional stimulus. Because people respond faster to probes that appear in a place they were already looking, faster responding to probes that replace emotional stimuli is interpreted as evidence of attention having been directed toward the emotional cue, whereas faster responding to probes that replace neutral stimuli is interpreted as attentional avoidance of the emotional cue.

A minor variation of the classic probe task just described (i.e., the probe position task) is the probe classification task (Bradley et al., 1998). In the probe classification task, the participant classifies the type of probe presented (e.g., whether the probe was three vertical or three horizontal dots), instead of simply indicating the position of the probe. This alteration makes the DP task slightly more difficult, and thereby increases the mean response time on the task (Mogg & Bradley, 1999b). The change was introduced to ensure that participants were not simply focusing on the left visual field during probe tasks.

The DP task has been used in a large number of studies to examine processing of positive stimuli in depressed and anxious individuals. Moreover, there is inconsistency in the literature, suggesting a need for a meta-analytic and theoretical review of findings. In many DP studies, participants with symptoms of depression and/or anxiety are significantly more avoidant of positive cues than participants without clinical symptoms (e.g., Bradley, Mogg, Falla, & Hamilton, 1998; Joormann & Gotlib, 2007; Shane & Peterson, 2007; Taylor, Bomyea, & Amir, 2010). However, in other DP studies no such differences have been found (e.g., Klumpp & Amir, 2009; Mogg & Bradley, 2002).

Thus, the purpose of the following review and meta-analyses is to (a) empirically assess the current pattern of findings in the DP literature; (b) evaluate whether extant theories, many of which focus primarily on threat processing in depression and anxiety, can predict the results of our meta-analyses; and (c) provide updated theory and methodological guidelines in response to our findings.

Role of Conventional Meta-Analysis

Our conventional meta-analysis examines combined effects of all dot-probe studies that have included positive stimuli, and thus yields an estimate of the size of the effect of positive-based bias in depressed and anxious persons. This novel evaluation, which includes a large number of findings we calculated from response time tables listed in manuscripts (Borenstein, Hedges, Higgins, & Rothstein, 2009), will help to evaluate if avoidance of positivity is a real effect, and to gauge its robustness.

Conventional meta-analysis has become the coin of the realm of psychological reviews, as it allows different types of test statistics to be included in one synthesized meta-analytic estimate (Borenstein et al., 2009). It also allows for the evaluation of moderation to examine whether effects vary as a function of methodological or individual difference variables.

After reporting these conventional meta-analytic results, we then report results from a new meta-analytic technique, p-curve, to provide multiple ways of evaluating this corpus of findings.

Role of P-Curve

P-curve is a novel method of meta-analysis developed to counter selective reporting of significant findings (Simonsohn, Nelson, & Simmons, 2014). Selective reporting of studies is known as the file-drawer effect; this is when studies yielding statistically significant findings are published, while studies that produce null findings are either rejected by journals or are not submitted for publication (Rothstein, Sutton, & Bornstein, 2005). Selective reporting can also occur with individual analyses carried out within a given study, such as when statistically significant subsets of analyses are published, while other analyses, variables, manipulations, and groups that produced non-significant results are not published. All programs of research produce false positives from time to time, therefore it is necessary to know how often analyses testing a given effect produce null results in order to determine the value of significant results showing that effect.

P-curve provides a solution by testing the evidential value of a set of statistically significant findings. It does so by determining the likelihood that a set of p-values would occur in the absence of a real effect. In the present review we used p-curve to test the evidential value of DP findings showing avoidance of positive information in persons with elevated depressive or anxious symptoms by comparing the distribution of significant p-values showing this effect to the distribution of p-values that one would expect to see in the absence of a real effect. To provide a point of comparison that might further contextualize p-curve results for avoidance of positive information, we also used p-curve to test the evidential value of DP findings showing vigilance toward negative information in persons with elevated depressive or anxious symptoms. Vigilance toward negative information was well suited to serve as a reference point because most of the DP studies that included positive stimuli also included negative stimuli, and vigilance toward negative information is a well-established effect with the DP task.

Previous Reviews

Narrative (Cisler, Bacon, & Williams, 2009; Cisler & Koster, 2010; De Raedt & Kostner, 2010; Gotlib & Joormann, 2010; Joormann, Yoon, & Zetsche, 2007; Mathews & MacLeod, 2005; Mobini & Grant, 2007; Mogg & Bradley, 1998, 2004, 2005; Ruiz-Caballero & Bermudez, 1997; Teachman et al., 2012; Williams et al., 1996; Williams, Watts, MacLeod, & Matthews, 1988; Williams, Watts, MacLeod, & Matthews, 1997; Yiend, 2010) and meta-analytic (Bar-Haim et al., 2007; Cisler et al., 2011; Frewen, Dozois, Joanisse, & Neufeld, 2008; Peckham, McHugh, & Otto, 2010; Phaf & Kan, 2007) reviews document that depressed and anxious persons are biased toward the processing of negative information. Only portions of meta-analytic reviews, however, (Cisler et al., 2011; Frewen et al., 2008; Peckham et al., 2010)¹, have begun to address the topic pursued here, that is, whether depression and anxiety are associated with biases away from positive information. In a subset of 10 studies, Cisler et al. (2011) reviewed emotional Stroop performance on positive information in individuals with posttraumatic stress disorder (PTSD), finding no within-subject or between-subject differences. In another subset of 14 dot-probe studies accompanying a neural network model of threatening information, Frewen et al. (2008) found that normal participants were quicker to respond to positive than to neutral information, whereas anxious and depressed persons were slower to respond to positive than to neutral information, in comparison to controls. Lastly, in a subset of 4 dot-probe studies and 8 emotional Stroop studies, Peckham et al. (2010) found that normal individuals were quicker to respond to positive information than depressed individuals. These reviews provide somewhat equivocal findings and were limited in their theoretical scope, due to their primary focus on negative information. Indeed, only Frewen et al. (2008) provides any explanation of why differential processing of positive information might occur in depressed or anxious individuals.

Parameters of the Review

Literature base

Studies reviewed here were collected by cross-referencing recent reviews on attentional biases in depression and anxiety (Bar-Haim et al., 2007; Cisler, Bacon, & Williams, 2009; Cisler & Koster, 2010; De Raedt & Koster, 2010; Cisler et al., 2011; Frewen et al., 2008; Joormann, Yoon, & Zetsche, 2007; Phaf & Kan, 2007; Teachman et al., 2012; Yiend, 2010) as well as by examining older reviews for studies that might have been omitted in newer reviews (Williams, Watts, MacLeod, & Matthews, 1997; Ruiz-Caballero & Bermúdez, 1997). In addition, a search of PsycInfo databases was conducted using the key words “dot-probe,” “dot probe,” “attentional probe,” “attentional-probe” “probe detection,” “visual probe,” and “attentional bias” cross-checked with “anx*,” “depress*,” “dys*,” “PTSD,” “panic,” “posttraumatic stress disorder,” “obsessive-compulsive,” “social anxiety disorder,” “social phobia,” “positive,” “threat,” “overactive withdrawal,” “vantage sensitivity,” “reward devaluation,” and “even-handed.” Also, searches were conducted for studies referencing seminal works and figures in the dot-probe literature on PsycInfo and Google Scholar. Each prospective paper published, including online first publication, by May 2013, was assessed for initial relevance, then the method section was read, and if the study used the DP task and contained positive stimuli, neutral stimuli, and examined participants with elevated depressive, anxious, socially anxious, obsessive-compulsive, or post-traumatic stress symptoms, it was incorporated into the review (besides the exceptions noted in the exclusion criteria – please see Figures 1a and 1b).

Figure 1.

Exclusion criteria

The current review is restricted to selective attention to and away from emotional information in adults; therefore, studies wherein participants were children or adolescents were excluded. In addition, studies that examined elevated state anxiety were excluded to focus on the reliability of biases associated with person-based depression and anxiety (Robinson, Goetz, Wilkowski, & Hoffman, 2006).

Why Depression and Anxiety?

Depression and anxiety often co-occur, with lifetime comorbidity prevalence estimates ranging from 40% to 90% (Brown, Campbell, Lehman, Grisham, & Mancill, 2001; Clark, 1989; Kessler, 1997; Kessler, Berglund, Demler, Jin, & Walters, 2005; Kessler, Chiu, Demler, & Walters, 2005; Kessler, et al., 1996, 2004; Newman et al., 2013; Shankman & Klein, 2003). Separate diagnoses do not indicate separate ontological syndromes (Borsboom et al., 2003; Cervone, 2004; Cramer et al., 2010; Insel et al., 2010)², but examining depression and anxiety together allows for an inclusive investigation of reward-based biases as well as an examination of whether different primary symptoms (i.e., depression versus anxiety) moderate the strength of those effects.

Theoretical Models of Attention to Positive Information and Available Evidence

To contextualize the pattern of results presented in our meta-analyses we first here present three prevailing frameworks and their predictions regarding depressed and anxious individuals’ biased processing of positive information (see Table 1 for a summary of predictions of the three theoretical models, and Tables 2, 3, and 4 for findings supporting each model).

Table 1.

Theoretical frameworks and predictions for processing of positive information in depressed and anxious individuals

Framework	Value of Positive Information in ANX and DEP	Within-Group Differences from NEUT for ANX/DEP	Between-Group Difference for ANX/DEP vs. CON
Threat-Processing Structures	No explicit predictions	Extrapolated: ANX/DEP: POS=NEUT < NEG:	Extrapolated: ANX/DEP: POS=NEUT < NEG CON: NEG = POS = NEUT
Even-Handed (EH)/Self-Regulation (SR)	Non-biased attention toward positive information specific to DEP (EH) or ANX (SR)	DEP (EH)/ANX (SR): POS = NEUT	CON: POS > NEUT DEP: POS = NEUT
Reward Devaluation	Active avoidance of positive information in ANX and DEP	ANX/DEP: POS < NEUT	CON: POS ≥ NEUT ANX/DEP: POS < NEUT

Note. DEP, depressed persons; ANX, anxious persons; CON, control group; POS, positive stimuli; NEG, negative stimuli; NEUT, neutral stimuli

Table 2.

Findings included in p-curve(s) that support Even-Handed/Self-Regulation theory

							Within clinical/high- symptom		Clinical/high- symptom vs. controls
							Positive- neutral	Negative -neutral	Positive- neutral	Negative -neutral
Study	Sample type	Per group n and primary diagnosis or symptom(s)	Comorbid symptoms that differed between groups	Stimuli	Stimulus duration (ms)	Task type	p	p	p	p
Bradley, Mogg, & Millar (2000)a	Nonclinical	Dysphoria (Low, n = 18; Medium, n = 17; High, n = 19)	State anxiety, trait anxiety (participants regrouped to test hypotheses involving different symptoms)	Faces (angry, happy)	500	Detection	N/A	N/A	.04	ns
Fox (Experiment 2; 2002)a	Nonclinical	Trait anxiety (Low, n = 18; High, n = 18)	State anxiety (HTA > LTA)	Faces (fearful, happy)	17	Identification	ns	<.001	ns	<.001
Fritzsche et al. (2010)a	Clinical	MDD, n = 20; Remitted depressed, n = 20; Asthma, n = 20; Normal controls, n = 20	Recent anxiety (MDD > RMD > Asthma, Controls)	Faces (sad, happy)	1000	Detection	ns	.029	.032	.003
Gotlib, Kasch et al. (2004)a	Clinical	MDD, n = 88; GSP, n = 35; Normal controls, n = 55	Recent anxiety (MDD, GAD > Controls); depressive symptoms (MDD > GAD > Controls)	Faces (sad, angry, happy)	1000	Detection	<.001 (r within MDD)	.007	ns	.031
Joormann & Gotlib (2007)	Clinical	MDD, n = 26; Remitted depressed, n = 23; Never depressed, n = 19	N/A (not reported)	Faces (sad, happy)	1000	Detection	ns	.01	.044	.001
Miskovic & Schmidt (2012)a	Nonclinical	Social anxiety (Low, n = 16; High, n = 17)	Depressive symptoms (HSA > LSA)	Faces (angry, happy)	100, 500, 1250	Detection	ns	.027	ns	.039
Mogg & Bradley (1999a)	Nonclinical	Trait anxiety (Low, n = 19; High, n = 19)	Depressive symptoms, state anxiety (HTA > LTA)	Faces (angry, happy)	500	Detection	ns	.025	ns	.041
Oehlberg, Revelle, & Mineka (Study 1; 2012)a	Nonclinical	Anxious and depressive symptoms treated as continuous variables; N = 64	No groups used	Faces (angry, sad, happy)	300, 500, 1250	Detection	N/A	N/A	ns	.031

^aDenotes a study that is listed in more than one table because findings can be interpreted as supporting more than one theoretical framework.

Note. GSP = Generalized Social Phobia; RMD = Remitted Major Depression; LTA = Low Trait Anxious; HTA = High Trait Anxious; HSA = High Social Anxiety; LSA = Low Social Anxiety; MDD = Major Depressive Disorder; GAD = Generalized Anxiety Disorder; SAD = Social Anxiety Disorder; SP = Social Phobia; OCD = Obsessive Compulsive Disorder; PD = Panic Disorder; PTSD = Posttraumatic Stress Disorder. Results listed as “ns” were non-significant at the .05 level; results listed as “N/A” were not reported in the original paper.

Table 3.

Findings included in p-curve(s) that support theory of Threat-Processing Structures

							Within clinical/high- symptom		Clinical/high- symptom vs. controls
							Positive- neutral	Negative -neutral	Positive- neutral	Negative -neutral
Study	Sample type	Per group n and primary diagnosis or symptom(s)	Comorbid symptoms that differed between groups	Stimuli	Stimulus duration (ms)	Task type	p	p	p	p
Bradley et al. (Experiments 1 and 2 combined; 1997)	Nonclinical	Dysphoric, n = 29; Non-dysphoric, n = 33; 11 additional participants	Trait anxiety, social anxiety significantly correlated with depressive symptoms	Faces (angry, happy)	500	Detection	ns	ns	ns	.042
Donaldson, Lam, & Mathews (2007)	Clinical	MDD, n = 36; Normal controls, n = 36	Recent anxiety, rumination (MDD > Controls)	Words (negative, positive)	500, 1000	Detection	< .001	< .001	N/A	N/A
Eldar, Yankelevitch, Lamy, & Bar-Haim (2010)	Nonclinical	Trait anxiety (Low, n = 23; High, n = 23)	State anxiety (HTA > LTA)	Faces (angry, happy)	500	Identification	ns	.008	ns	ns
Fox (Experiment 2; 2002)a	Nonclinical	Trait anxiety (Low, n = 18; High, n = 18)	State anxiety (HTA > LTA)	Faces (fearful, happy)	17	Identification	ns	< .001	ns	<.001
Fritzsche et al. (2013)a	Clinical	MDD, n = 20; Normal controls, n = 19; Chronic obstructive pulmonary disorder (COPD) patients with MDD, n = 18; COPD patients without MDD, n = 21	General distress (Controls < COPD without MDD < COPD + MDD, MDD)	Faces (sad, happy)	1000	Detection	ns	< .001	.002	< .001
Gotlib, Krasnoperova, Yue, & Joormann (2004)	Clinical	MDD, n = 19; GAD, n = 18; Normal controls, n = 16	Recent anxiety (MDD, GAD > Controls); depressive symptoms (MDD > GAD > Controls)	Faces (sad, angry, happy)	1000	Detection	ns	.007	ns	.03
Klumpp & Amir (2009)	Nonclinical	Social anxiety (Low, n = 37; High, n = 39)	Trait anxiety, state anxiety, depressive symptoms (HSA > LSA)	Faces (angry, happy)	500	Identification	ns	.047	ns	.036
Miskovic & Schmidt (2012)a	Nonclinical	Social anxiety (Low, n = 16; High, n = 17)	Depressive symptoms (HSA > LSA)	Faces (angry, happy)	100, 500, 1250	Detection	ns	.027	ns	.039
Mogg & Bradley (Experiment 3; 1999b)	Nonclinical	Trait anxiety (Low, n = 11; High, n = 11)	Trait anxiety, state anxiety, depressive symptoms (HTA > LTA)	Faces (angry, happy)	17	Identification	ns	.039	ns	N/A
Mogg & Bradley (2002)	Nonclinical	Social anxiety (Low, n = 16; High, n = 11)	Trait anxiety, state anxiety, depressive symptoms (HSA > LSA)	Faces (angry, happy)	17	Identification	ns	.023	ns	.001
Mogg, Bradley, & Hallowell (1994)	Nonclinical	Trait anxiety (Low, n = 30; High, n = 36)	Depressive symptoms, state anxiety (HTA > LTA)	Words (achievement threat, physical threat, positive)	14, 500	Detection	ns	.049	ns	.009
Mogg, Bradley, & Williams (1995)	Clinical	MDD, n = 17; GAD, n = 17; Normal controls, n = 15	State anxiety, trait anxiety (MDD, GAD > Controls), depressive symptoms (MDD > GAD > Controls)	Words (depression-related, anxiety-related, positive)	14, 1000	Detection	ns	.009	ns	.005
Oehlberg, Revelle, & Mineka (Study 1; 2012)a	Nonclinical	Anxious and depressive symptoms treated as continuous variables; N = 64	No groups used	Faces (angry, sad, happy)	300, 500, 1250	Detection	N/A	N/A	ns	.031
Oehlberg, Revelle, & Mineka (Study 2; 2012)	Nonclinical	Anxious and depressive symptoms treated as continuous variables; N = 166	No groups used	Faces (angry, sad, happy)	300, 1000	Detection	N/A	N/A	ns	< .001
Reinecke, Cooper, Favaron, Massey-Chase, & Harmer (2011)a	Clinical	PD, n = 23; Normal controls, n = 22	Depressive symptoms, trait anxiety, agoraphobic cognitions (PD > Controls)	Faces (fearful, happy)	16, 100	Identification	ns	.013	ns	.029
Reinecke, Waldenmaier, Cooper, & Harmer (Waitlist group; 2013)	Clinical	PD, n = 14	Anxiety, depressive symptoms elevated	Faces (fearful, happy)	16, 100	Identification	ns	.017	N/A	N/A
Schrooten, Smulders, Mogg, & Bradley (2012)	Nonclinical	Trait anxiety (Low, n = 31; High, n = 30)	Depressive symptoms, state anxiety, social and physical worries (HTA > LTA)	Words (social threat, physical threat, positive)	14, 500	Identification	ns	.045	ns	.02
Stevens, Rist, & Gerlach (Placebo condition; 2009)	Clinical	SP, n = 20; Normal controls, n = 19	Depressive symptoms, anxiety sensitivity (SP > Controls)	Faces (angry, happy)	175, 600	Identification	ns	.001	ns	.013
Tran, Lamplmayr, Pintzinger, & Pfabigan (Females; 2013)	Nonclinical	Trait anxiety (Low, n = 71; High, n = 50)	Depressive symptoms (HTA > LTA)	Faces (angry, disgusted, fearful, sad, happy)	50	Detection	ns	.037	ns	.014

^aDenotes a study that is listed in more than one table because findings can be interpreted as supporting more than one theoretical framework.

Note. HSA = High Social Anxiety; LSA = Low Social Anxiety; COPD = Chronic Obstructive Pulmonary Disorder; LTA = Low Trait Anxious; HTA = High Trait Anxious; MDD = Major Depressive Disorder; GAD = Generalized Anxiety Disorder; SAD = Social Anxiety Disorder; SP = Social Phobia; OCD = Obsessive Compulsive Disorder; PD = Panic Disorder; PTSD = Posttraumatic Stress Disorder. Results listed as “ns” were non-significant at the .05 level; results listed as “N/A” were not reported in the original paper.

Table 4.

Findings included in p-curve(s) that support theory of Reward Devaluation

							Within clinical/high- symptom		Clinical/high- symptom vs. controls
							Positive- neutral	Negative -neutral	Positive- neutral	Negative -neutral
Study	Sample type	Per group n and primary diagnosis or symptom(s)	Comorbid symptoms that differed between groups	Stimuli	Stimulus duration (ms)	Task type	p	p	p	p
Bradley, Mogg, Falla, & Hamilton (1998)	Nonclinical	Trait anxiety (Low, n = 19; High, n = 19)	Depressive symptoms, social anxiety, anxious arousal, anhedonia, general distress (HTA > LTA)	Faces (angry, happy)	500, 1250	Identification	ns	.041	.014	.019
Bradley, Mogg, & Millar (2000)a	Nonclinical	Dysphoria (Low, n = 18; Medium, n = 17; High, n = 19)	State anxiety, trait anxiety (participants regrouped to test hypotheses involving different symptoms)	Faces (angry, happy)	500	Detection	N/A	N/A	.04	ns
Brosschot, de Ruiter, & Kindt (1999)	Nonclinical	Trait anxiety, defensiveness (Low anxious, n = 11; Repressors, n = 15; High anxious, n = 10; Defensive high anxious, n = 15)	State anxiety also measured, but not reported by group	Words (social threat, social positive, physical threat, general positive)	500	Detection	ns	ns	.018	ns
Donaldson, Lam, & Mathews (2007)	Clinical	MDD, n = 36; Controls, n = 36	Recent anxiety, rumination (MDD > Controls)	Words (negative, positive)	500, 1000	Detection	< .001	< .001	N/A	N/A
Fritzsche et al. (2010)a	Clinical	MDD, n = 20; Remitted depressed, n = 20; Asthma, n = 20; Controls, n = 20	Recent anxiety (MDD > RMD > Asthma, Controls)	Faces (sad, happy)	1000	Detection	ns	.029	.032	.003
Fritzsche et al. (2013)a	Clinical	MDD, n = 20; Controls, n = 19; Chronic obstructive pulmonary disorder (COPD) patients with MDD, n = 18; COPD patients without MDD, n = 21	General distress (Controls < COPD without MDD < COPD + MDD, MDD)	Faces (sad, happy)	1000	Detection	ns	< .001	.002	< .001
Gotlib, Kasch et al. (2004)a	Clinical	MDD, n = 88; GSP, n = 35; Controls, n = 55	Recent anxiety (MDD, GAD > Controls); depressive symptoms (MDD > GAD > Controls)	Faces (sad, angry, happy)	1000	Detection	<.001 (r within MDD)	.007	ns	.031
Hunt, Keogh, & French (Masked condition; 2006)	Nonclinical	Anxiety sensitivity (Low, n = 20; Moderate, n = 14; High, n = 19)	Recent anxiety, depressive symptoms (HAS > LAS)	Words (anxiety symptom, social threat, positive)	500	Detection	ns	N/A	ns	.006
Ioannou, Mogg, & Bradley (2004)	Nonclinical	Trait anxiety, defensiveness (Low anxious, n = 10; Repressors, n = 16; High anxious, n = 11; Defensive high anxious, n = 13)	State anxiety, depressive symptoms (HA, DHA > LA, R)	Faces (angry, happy)	500, 1250	Identification	.024	.048	N/A	.025
Keogh, Dillon, Georgiou, & Hunt (2001)	Nonclinical	Anxiety sensitivity (Low, n = 27; High, n = 24)	Trait anxiety, recent anxiety, stress, depressive symptoms (HAS > LAS)	Words (social threat, physical threat, positive)	500	Detection	N/A	N/A	.049	.037
Keogh, Ellery, Hunt, & Hannent (2001)	Nonclinical	Recent anxiety treated as a continuous variable*, N = 74	Fear of pain, stress, depressive symptoms, anxiety sensitivity all positively correlated with recent anxiety	Words (pain-related, social threat, positive)	500	Detection	N/A	N/A	< .001	ns
Lindstrom et al. (2011)	Clinical	PTSD, n = 9 adults; Non-PTSD, n = 36 adults	N/A (not reported)	Faces (angry, happy)	500	Detection	ns	ns	.004	ns
Mingtian, Xiongzhao, Jinyao, Shuqiao, & Atchley (2011)	Clinical	MDD, n = 24; Controls, n = 25	N/A (not reported)	Pictures (negative, positive)	100, 500	Detection	.032	ns	< .001	ns
Mogg, Philippot, & Bradley (2004)	Clinical	SP, n = 15; Controls, n = 15	Trait anxiety, state anxiety, depressive symptoms (SP > Controls)	Faces (angry, happy)	500, 1250	Identification	ns	.039	N/A	N/A
Mueller et al. (2009)	Clinical	SAD, n = 12; Controls, n = 15	Trait anxiety, depressive symptoms (SAD > Controls); comorbid diagnoses in SAD group of GAD (n = 8), specific phobia (n = 5), MDD (n = 7), OCD (n = 3)	Faces (angry, happy)	100	Identification	.007	ns	ns	ns
Pishyar, Harris, & Menzies (CGBT; 2008)	Clinical	SP, n = 16	Anxiety sensitivity, depressive symptoms elevated	Faces (disgusted, happy)	500	Detection	.002	.002	N/A	N/A
Pishyar, Harris, & Menzies (Waitlist; 2008)	Clinical	SP, n = 16	Anxiety sensitivity, depressive symptoms elevated	Faces (disgusted, happy)	500	Detection	< .001	.01	N/A	N/A
Reinecke, Cooper, Favaron, Massey-Chase, & Harmer (2011)a	Clinical	PD, n = 23; Controls, n = 22	Depressive symptoms, trait anxiety, agoraphobic cognitions (PD > Controls)	Faces (fearful, happy)	16, 100	Identification	ns	.013	ns	.029
Shane & Peterson (Study 1; 2007)	Nonclinical	Dysphoric, n = 30; Non-dysphoric, n = 43	Group difference in trait anxiety was non-significant	Pictures (negative, positive)	500, 1500	Detection	.026	ns	.037	ns
Shane & Peterson (Study 2; 2007)	Nonclinical	Dysphoric, n = 27; Controls, n = 39	Trait anxiety (Dysphoric > Controls)	Words (depression-related, positive)	200, 1500	Detection	.019	.026	.004	.021
Taylor, Bomyea, & Amir (2010)	Nonclinical	Social anxiety treated as a continuous variable, N= 43	State anxiety, anhedonia elevated	Words (social positive)	500	Identification	.006	N/A	N/A	N/A

^aDenotes a study that is listed in more than one table because findings can be interpreted as supporting more than one theoretical framework.

Note. HAS = High Anxiety Sensitivity; LAS = Low Anxiety Sensitivity; LA = Low Anxiety; HA = High Anxiety; R = Repressor; DHA = Defensive High Anxious; COPD = Chronic Obstructive Pulmonary Disorder; MDD = Major Depressive Disorder; GAD = Generalized Anxiety Disorder; SAD = Social Anxiety Disorder; SP = Social Phobia; OCD = Obsessive Compulsive Disorder; PD = Panic Disorder; PTSD = Posttraumatic Stress Disorder. Results listed as “ns” were non-significant at the .05 level; results listed as “N/A” were not reported in the original paper.

Threat Processing Structures

Bower (1981) initially theorized that negative moods restrict cognitive focus, such that dysphoric or distressed moods would lead to more processing of negative, and thus less positive, information. This has given way to a number of threat-processing structure models (Mogg & Bradley, 1998; Mathews & Mackintosh, 1998; Williams et al., 1988, 1997; also see Ouimet et al., 2009). In these models, a threat-processing structure initially evaluates the level of threat of a stimulus, and then a second processing structure alters attentional processes depending on the output of the threat-processing structure.

These models attempt to explain anxious—and to a lesser extent, depressed—individuals’ threat-processing biases (see Yiend, 2010). They are less specific regarding biased processing of positive information, however, beyond noting that valenced information may be processed in congruency with one’s mood (Bower, 1981). For example, Cognitive-Motivational Analysis (CMA; Mogg & Bradley, 2005) posits two stages of emotional processing in people with anxiety. Initially, a Valence Evaluation System (VES) assesses the threat level of a stimulus. The stimulus-threat is then fed into a Goal Engagement System (GES), which determines whether to interrupt current goals in order to engage with the potential threat, or to continue on as usual (see Lazarus, 1991). In CMA, positive affect occurs due to the interaction of positively-valenced stimuli and goal engagement; however, how depressed and anxious persons might differentially process positive stimuli is not specified. This seems to imply that non-pathological individuals are directed by default to reward-based stimuli by their GES, and that depressed individuals are not because of a chronically active VES feeding in negative information.

Mogg and Bradley (2005) briefly note that findings demonstrating biased processing of positive information in depressed and anxious persons are intriguing, but they do not include specific hypotheses regarding positive information-processing. However, a lack of normal processing of positive information due to chronic activation of the VES is noted. Thus, one may extrapolate that these models predict differences between depressed and anxious individuals and controls, such that controls would process positive information more quickly than neutral information, but depressed and anxious persons would not. However, depressed and anxious individuals would not exhibit within-subject differential biases in response to positive as opposed to neutral stimuli. These hypotheses relate to the even-handed and self-regulatory frameworks detailed below.

Even-Handed/Self-Regulation Frameworks

Gotlib and colleagues (Coyne & Gotlib, 1983; Gotlib et al., 1988; McCabe & Gotlib, 1995; McCabe, Gotlib, & Martin, 2000; McCabe & Toman, 2000) have advanced an “evenhanded” theory of underlying processing of positive information by depressed persons. In a similar vein, Kashdan and colleagues (Kashdan, Weeks, & Savostyanova, 2011) advocate a “self-regulatory” framework for the processing of positive information in socially anxious individuals. These frameworks argue that depressed and socially anxious persons may lack the bias toward positive information that is normally present in individuals without such symptoms (see Coyne & Gotlib, 1983; Gotlib & McCabe, 1992; Kashdan et al., 2011; Korn, Sharot, Walter, Heekeren, & Dolan, 2013; McCabe & Toman, 2000). This is due to depressed and socially anxious persons’ added focus on negative information (Gotlib et al., 1988; Yiend, 2010), and socially anxious persons’ difficulty self-regulating negative impulses (Kashdan et al., 2011). This lack of bias towards positive information may indicate an absence of unrealistic, yet self-protective, biases that are found in non-depressed and non-anxious individuals (Alloy & Abramson, 1979). In support of the even-handed framework, depressed individuals are sometimes more accurate in processing information than are non-depressed controls (e.g., Alloy & Abramson, 1979; Cummins & Nistico, 2002).

Kashdan et al. (2011) argue that socially anxious individuals tend to increase attempts to control themselves, which results in their experiencing less positivity than do non-anxious individuals. In other words, socially anxious individuals are so concerned with downregulating negative affect that they end up processing less positive information. In support of this framework, socially anxious individuals attempt to self-regulate negative impulses more (Kashdan & McKnight, 2010) and experience less positive affect (Kashdan, 2007; Weeks, Jakatdar, & Heimberg, 2010) than do non-anxious individuals.

The even-handed and self-regulation frameworks predict that non-depressed and non-anxious individuals will be faster to respond to positive information than to neutral information, whereas depressed and anxious individuals will not differ in processing neutral and positive information. Findings using the dot-probe have provided support of this hypothesis (Bradley et al., 1997; Fritzsche et al., 2010; Joormann & Gotlib, 2007).

Reward Devaluation

Diverging from the previous frameworks, the “reward devaluation” hypothesis claims that depressed and anxious individuals automatically avoid positive material (Frewen et al., 2008; Shane & Peterson, 2007; Tomarken & Keener, 1998; Winer et al., 2011). In contrast to the even-handed, self-regulatory, and threat-processing structure frameworks, the reward devaluation hypothesis conceptualizes avoidance not simply as a lack of valuing positive information, but as an active process of inhibition of rewarding stimuli (Atchley et al., 2012; Frischen et al., 2012; Frewen et al., 2008; Winer et al., 2011).

Frewen et al. (2008) have posited that depressed and/or anxious individuals may actively avoid positive stimuli due to a lack of exposure to positive information during development, which has led to chronic lack of approach motivation. Shane and Peterson (2007) argue that depressed individuals may actively avoid positive information due to biological diathesis, manifested by overactive withdrawal systems (Gray, 1994; Tomarken & Kenner, 1998) in relation to their approach systems. Although Shane and Peterson note that the withdrawal system may simply limit approach tendencies (i.e., even-handedness), they present the alternative that encoding of positive information may be impaired, thereby producing eventual anhedonia.

In addition, Pluess & Belsky (2013) posit that neurodevelopmentally-derived differential reactivity to positivity predicts whether individuals will or will not gain advantage from objectively rewarding environments. These individual differences, named vantage sensitivity and vantage resistance, refer to specific neurodevelopmental endogenous factors and early-state characteristics of personality (see Sweitzer et al., 2013) that predict variation in response to exclusively positive experience. That is, departing from previous frameworks that emphasize negative influences either exclusively or along with positive influences as causing or sustaining psychopathology, vantage sensitivity and resistance focus solely on differences in how individuals are influenced by positivity in relation to whether they eventually experience sustained distress. Individuals who are highly vantage sensitive have more promotive factors, i.e., factors which allow them to gain increased benefit from positive influences. Conversely, people who are vantage resistant possess more vantage-resistance factors, i.e., factors which diminish or completely eliminate positive response to supportive environments. However, although vantage resistance may play a role in predisposition to avoid or eliminate positive stimuli, the underlying cognitive mechanisms of action for this development are not specified.

Persons experiencing symptoms of psychopathology may avoid positive information because it has come to represent a threat, such that positive information is associated with negative outcomes (e.g., Winer et al., 2011). In this way, reward processing qualitatively differs from threat processing in depressed and anxious individuals; the associated danger with negative information is in not noticing warning signs and therefore failing to take action to avoid or brace for threat, whereas the associated danger with positive information is that one approaches what appear to be safety or reward signals, when in fact those signals are ultimately meretricious (i.e., harmful). Thus, positive information, counterintuitively, becomes prospectively more dangerous to depressed and anxious individuals than neutral information. This hypothesis yields the prediction that depressed and anxious individuals will be biased away from positive information in comparison to neutral information, as well as in comparison to control groups.

Tabular Review

As noted above and in our tabular review (see Tables 2–4), some significant findings support the hypothesis that anxious individuals take longer to respond to probes following positive information than to those following neutral or negative words. These findings have emerged primarily at 500 ms presentation durations (e.g., Pishyar, Harris, & Menzies, 2008). There is also some evidence that depressed and dysphoric groups differentially process positive information than negative information and that this differential processing diverges from control groups (e.g., Gotlib, Kasch, et al., 2004; Shane & Peterson, 2007). These findings have occurred primarily with 1000 ms and 500 ms presentation durations. It is unclear, however, whether these differences are due to depressed and anxious individuals responding slower to positive than to neutral information, or due to controls responding faster to positive than to neutral information. Moreover, some studies (e.g., Bradley et al., 1997; Gotlib, Krasnoperova, et al., 2004) report null findings, thus making it somewhat unclear if depressed and anxious individuals do indeed avoid positive information at all.

Meta-Analyses

Given the discordant array of findings regarding biased processing of positive information in the DP task in depressed and anxious individuals, we decided to examine whether findings of biases away from positive information have evidential value—that is, whether they reflect a real effect rather than merely false positives. To empirically evaluate this possibility we conducted both a conventional meta-analysis (Borenstein et al., 2009) and a p-curve meta-analysis (Simonsohn et al., 2014). We first present results from the conventional meta-analysis of dot-probe biases for positive and negative information, including an exploration of procedural and individual difference moderators. We then evaluate the evidential value of findings demonstrating biased processing away from positive information and toward negative information in depressed and anxious individuals.

Conventional Meta-Analyses of Dot-probe Findings

Overview of conventional meta-analyses

We conducted omnibus meta-analyses of DP attentional bias findings to test within-subjects bias for positive stimuli in high-symptom participants, within-subjects bias for positive stimuli in healthy controls, and between-subjects differences in bias for positive stimuli when comparing high-symptom participants to healthy controls. We carried out an analogous set of omnibus meta-analyses to test DP findings with negative stimuli. In addition, we examined a set of potential moderators including stimulus type, stimulus duration, primary symptoms of high-symptom group, and whether or not high-symptom participants were selected on the basis of a clinical diagnosis. For each meta-analysis, one effect size was included from each independent sample that met criteria, or in the case of between-subjects meta-analyses, each pair of samples. Because the goal of this analysis was to inclusively investigate both negative and positive DP biases, and because we used a random effects model to limit the influence of any single study (as noted below), all studies that otherwise met inclusion criteria were part of the meta-analysis³.

General selection rules

In addition to the inclusion and exclusion criteria for the narrative review (see “Parameters of the Review”), the following rules were used in selecting findings for inclusion in our conventional meta-analyses:

1. Attentional bias findings comparing positive to neutral stimuli were analyzed separately from findings comparing negative to neutral stimuli.

2. Within each valence category (i.e., positive-neutral and negative-neutral), three types of findings were eligible for inclusion in analyses: measurements of absolute or within-subjects bias toward or away from emotional stimuli in the high-symptom group, measurements of absolute or within-subjects bias in the control group, and measurements of between-subjects differences in attentional bias, wherein bias for emotional stimuli exhibited by the high-symptom group is compared to that of the low-symptom group.

3. If attentional bias scores with standard deviations (SD) or standard errors (SE) for high-symptom and/or low-symptom groups were reported by the original authors, these scores were used to compute effect sizes for the present meta-analyses (Borenstein et al., 2009).

4. In the absence of bias scores with SDs or SEs, attentional bias findings reported by the original authors in the form of t or F values for within-subjects or between-subjects bias, or in the form of r values of correlations between symptoms and attentional bias, were used to compute effect sizes for the present meta-analyses.

5. If correlations were reported in addition to bias scores or group-level significance tests, both/all results were averaged to create a single effect size to be included in each meta-analysis. Correlations were treated as between-subjects effects when they included all participants and as within-subjects effects when they only included participants in the high-symptom group (e.g., Gotlib, Kasch, et al., 2004). Correlations that excluded high symptom participants were not used (i.e., Mogg, Philippot, & Bradley, 2004).

6. If neither attentional bias scores with SDs or SEs nor eligible t, F, or r values were reported by the original authors, then bias scores and variances were computed using tabular listings of high-symptom and control participants’ mean response times (RTs) and SDs or SEs for each combination of probe location and emotional stimulus location (see “Procedure for computing bias scores and variances from RT tables” in Appendix A for further details).

7. For studies that compared two independent high-symptom groups to one control group, the control group n was split between the two high-symptom groups so that both between-subjects effects could be entered into our meta-analyses (Bar-Haim et al., 2007).

8. For studies wherein participants were assigned to high, medium, and low symptom groups, only the two extreme groups were included in our meta-analyses.

9. For omnibus meta-analyses, if participants were tested using more than one level of a moderator variable—for example, at two different stimulus durations (Shane & Peterson, 2007), or with both words and faces (Pishyar, Harris, & Menzies, 2008)—findings were averaged across conditions to produce a single mean effect size for each group. For meta-analyses examining moderators, only findings from the appropriate moderator condition were included, as described in the selection rules for meta-analyses of moderators.

10. For studies wherein multiple categories of positive or negative stimuli were presented, an average bias effect was computed for each valence. For example, a study by Brosschot, De Ruiter, and Kindt (1999) included general positive and social positive words, so an average bias effect for positive words was computed for inclusion in our meta-analyses.

11. In computing an average effect size for inclusion in each meta-analysis, redundant effects were excluded. For example, if the original authors reported the t-value for positive bias collapsed across 500 ms and 1250 ms stimulus durations in addition to the t-values for positive bias at each duration separately, then only the finding collapsing across durations was included in omnibus meta-analyses.

Selection rules for examining potential moderator of stimulus duration

1. Findings were sorted into the following categories: stimulus duration < 200 ms = 1; duration of 200–500 ms = 2; duration > 500 ms = 3.

2. For studies in which more than one stimulus duration was used, durations for which results were reported by the original authors were selected over durations for which results had to be computed using RT tables.

3. If results were reported for more than one duration category, then categories 1 or 2 were selected over category 3 for studies examining anxiety symptoms, whereas category 3 was prioritized over 1 or 2 for studies examining symptoms of depression, consistent with prevailing theory. Otherwise category 2 was selected. One study included an anxious group and a depressed group with one control group and used stimulus durations of 14 ms and 1000 ms (Mogg, Bradley, & Williams, 1995), and we selected the 14 ms finding for the within-subjects analysis in healthy controls.

4. If two durations within a single category were used (e.g., the 2005 study by Vassilopoulos included both 200 ms and 500 ms durations), the findings were averaged into a single mean effect size.

Selection rules for examining potential moderator of stimulus type

1. Findings were categorized according to the type of stimuli used: words or images (including faces).

2. A study wherein participants were tested using both words and images included two independent high-symptom groups, so we randomly chose word findings from one group and image findings from the other (Pishyar, Harris, & Menzies, 2008).

Selection rules for examining potential moderator of clinical vs. nonclinical sample

1. Findings were designated as either clinical or nonclinical based on whether or not high-symptom participants were selected on the basis of a clinical diagnosis.

Selection rules for examining potential moderator of primary symptom type

1. Findings were sorted into the following categories based on the primary symptom type used by the original authors to select and/or group participants: symptoms associated with depression, general anxiety (including trait and generalized anxiety), social anxiety, post-traumatic stress disorder, obsessive-compulsive disorder, or panic disorder.

2. Because only one study was identified in which obsessive compulsive symptoms were of primary interest (i.e., Harkness et al., 2009), this study was excluded from analyses of primary symptom type as a moderator.

3. For studies in which multiple symptom measures were administered and the original authors did not specify primary symptoms of interest, we used the most general assessment administered as the primary symptom type. For example, we designated general anxiety as the primary symptom type for the 2009 study by Berenson and colleagues. In one paper with two studies, both depressive and trait anxiety symptoms were reported, no diagnoses were made, and no preselection criteria was used (Oehlberg et al., 2012.). Thus, in this study we randomly chose study 1 as depressive and study 2 as general anxious, so as to not bias nonspecific samples toward either anxiety or depression.

4. When multiple stimulus categories were reported for a single valence, we combined all potential relevant categories for high symptom and control groups.

5. Findings from the comorbid symptom group in the 2012 study by LeMoult and Joormann were excluded from analyses of primary symptom type as a moderator, because these findings could not be assigned to a single symptom category.

6. Control comparisons were not included in the primary symptom analysis, as they could not be expected to differ by primary symptom and in some cases were comparison groups for multiple high symptom groups.

Computing individual effect sizes

Effect sizes were computed and meta-analyses carried out using Comprehensive Meta-Analysis software, Version 2.2064. The effect size Cohen’s d—defined as mean bias score over the within-groups standard deviation, pooled across groups—was computed for each finding included in meta-analyses. Hedges g was chosen as the overall primary effect size measure because it provides a more accurate estimate of effect size than Cohen’s d when working with small samples (Borenstein et al., 2009). For between-subjects comparisons of emotional information bias, a positive value for g means that the high-symptom group showed more attentional bias toward emotional, as opposed to neutral, stimuli, in comparison to healthy controls (see “Formulas used in computing Cohen’s d and Hedge’s g” in Appendix A for further details).

Computing meta-analysis effect size

All meta-analyses were carried out using a random-effects model so as to not assume equal variance. All findings meeting our inclusion criteria were included in the meta-analyses. Specificity was assessed via the p-curve analysis in the following section. Please see Tables 5–8 for meta-analytic results.

Meta-analysis of between-subject attentional biases for positive information

Across all of the studies that reported between-subjects effects and used positive stimuli, the combined positive-related bias was both negative and significant in depressed and anxious participants (k = 48, n = 2,562, g = −0.131, SE = 0.057, p = .02). Thus, a significant avoidance of rewarding information was found for depressed and anxious participants in comparison to asymptomatic controls (see Table 5).

Table 5.

Meta-Analytic Results of Between-Groups Bias for Positive Information for High Symptom vs. Control Participants

Moderator	k	N	g	SE	Q	p
Total data set	48	2,562	−0.131	0.057		.02
Primary symptoms					11.316	.003
Depression	15	793	−0.399	.084		<.001
General anxiety	16	884	0.001	.094		.991
Social anxiety	9	411	−0.078	.117		.507
Clinical v. nonclinical					<1	.421
Clinical	19	805	−0.060	0.108		.576
Nonclinical	29	1757	−0.163	0.066		.014
Stimulus duration					2.347	.309
< 200 ms	7	396	−0.013	0.115		.907
200–500 ms	31	1,7391	−0.141	0.068		.039
> 500 ms	10	425	−0.281	0.132		.033
Stimulus type					<1	.715
Words	14	619	−0.164	0.124		.184
Images	34	1,943	−0.114	0.063		.071

¹Participants from Oehlberg et al., 2012 varied slightly (+/−4) by analyses for duration and primary symptom.

Meta-analysis of within-subject attentional biases for positive information

Across all studies that reported within-subjects effects and used positive stimuli, depressed and anxious participants’ avoidance of reward was not significant (k = 46, n = 1,062 g = −0.046, SE = 0.024, p = .055). The positive-related bias was not significant in the opposite direction for asymptomatic controls (k = 36, n = 934, g = 0.035, SE = 0.019, p = .064). The difference between the combined within-subjects effect sizes of depressed and anxious participants in comparison to asymptomatic participants was significant, however (Q = 7.058, p = .008). Thus, a trend emerged suggesting avoidance of positive information in depressed and anxious individuals and approach of positive information in asymptomatic controls, and the two groups significantly differed from each other (see Table 6).

Table 6.

Meta-Analytic Results of Within-Group Bias for Positive Information for High Symptom and Control Participants

	Within high symptom						Within control						High symptom v. control
Moderator	k	N	g	SE	Q	p	k	N	g	SE	Q	p	Q	p
Total data set	46	1062	−0.046	0.024		.055	36	934	0.035	0.019		.064	7.058	.008
Primary symptoms					7.097	.029
Depression	11	302	−0.167	0.063		.008
General anxiety	15	332	0.015	0.029		.610
Social anxiety	13	275	−0.044	0.045		.323
Clinical v. nonclinical					<1	.331					1.267	.260
Clinical	24	540	−0.071	0.044		.102	15	371	0.063	0.035		.075	5.705	.017
Nonclinical	22	522	−0.023	0.023		.309	21	563	0.017	0.020		.411	1.706	.191
Stimulus duration					<1	.735					3.482	.175
< 200 ms	9	213	−0.052	0.041		.203	7	220	0.000	0.029		.990	1.058	.304
200–500 ms	26	540	−0.037	0.029		.199	20	466	0.023	0.030		.440	2.095	.148
> 500 ms	11	309	−0.100	0.077		.195	9	248	0.103	0.047		.029	5.041	.025
Stimulus type					1.430	.232					3.225	.073
Words	14	299	−0.095	0.053		.075	10	222	−0.017	0.034		.602	1.515	.218
Images	32	763	−0.024	0.026		.353	26	712	0.055	0.022		.014	5.342	.021

Do positive-related biases differ as a function of primary symptom?

We examined between- and within-subject reward avoidance at each level of primary symptom endorsed by the high-symptom group. We first examined all primary symptom groups (excluding obsessive-compulsive symptoms due to only one study existing), then performed a secondary analysis examining only the symptom groups with enough existing studies (i.e., depression, general anxiety, and social anxiety) to provide more power (i.e., k > 5).

Between-subjects combined effects at each level of primary symptom yielded a significant avoidance of positive information in participants with primary symptoms of depression (k = 15, g = −0.3999, SE = 0.084, p < .001), but no other primary symptom groups. Between-subjects group differences were significant with both five (Q = 19.364, p = .001) and three (Q = 11.316, p = .003) groups. The between-subjects effects for depression differed significantly from general anxiety (k = 16, Q = 10.129, p = .001) and social anxiety (k = 9, Q = 4.992, p = .025) (see Figure 2).

Figure 2.

Avoidance of positive information in individuals with primary symptoms of depression (between-subjects)

Consistent with the between-subjects findings, within-subject combined effects also yielded a significant avoidance of positive information in depressed participants (k = 11, g = −0.167, SE = 0.063, p = .008), but non-significant biases for the other groups. Within-subjects group differences were not significant with all five groups included in the model (Q = 7.369, p = .118), but were when the model was restricted to the three most powerful groups (Q = 7.097, p = .029). The within-subjects effects for depression differed significantly from general anxiety (k = 15, Q = 6.807, p = .009) but not social anxiety (k = 13, Q = 2.518, p = .113) (see Figure 3).

Figure 3.

Avoidance of positive information in individuals with primary symptoms of depression (within-subjects)

Taking between- and within-subject findings together, depressed participants evidenced a significant devaluation of reward, consistently differing from those reporting a primary symptom of general anxiety, and evidencing a less robust difference from those reporting a primary symptom of social anxiety.

Do positive-related biases differ due to clinical versus non-clinical samples?

The combined between-subject effect sizes for reward-based biases were significant for non-clinical (k = 29, g = −.163, SE = 0.066, p = .014) but not clinical (k = 19, g = −.060, SE = 0.108, p = .576) samples, although this difference was not significant (Q < 1, p = .421). The within-subjects effects for both non-clinical control participants (k = 21, g = 0.017, SE = 0.020, p = .411) and non-clinical but high symptom (k = 22, g = −0.023, SE = 0.023, p = .309) participants were not significant and did not significantly differ (Q = 1.706, p = .191). The combined within-subjects effects within studies using clinical samples for control (k = 15, g = 0.063, SE = 0.035, p = .075) and symptomatic participants (k = 24, g = −0.071, SE = 0.044, p = .102) were not significant. However, the comparison of symptomatic and asymptomatic participants was significant (Q = 5.705, p = .017).

Do positive-related biases differ as a function of whether stimuli were words or pictures?

The combined between-subject effect sizes for reward-based biases were not significant for pictures (k = 34, g = −0.114, SE = 0.063, p = .071) and words (k = 14, g = −0.164, SE = 0.124, p = .184). There was no significant difference between words and pictures for between-subjects findings (Q < 1, p = .715). For within-subject biases, high symptom participants evidenced a trend avoiding positive words (k = 14, g = −0.095, SE = 0.053, p = .075), whereas asymptomatic controls did not (k = 10, g = −0.017, SE = 0.034, p = .602). However, asymptomatic controls significantly were biased toward positive pictures (k = 26, g = 0.055, SE = 0.022, p = .014), whereas symptomatic participants were not (k = 32, g = −0.024, SE = 0.026, p = .353).

Do positive-related biases vary as a function of stimulus duration?

The combined between-subject effect sizes for reward-based biases were significant for durations above 500 ms (k = 10, g = −0.281, SE = 0.132, p = .033) and between 200 and 500 ms (k = 31, g = −0.141, SE = 0.068, p = .039), but not significant for durations below 200 ms (k = 7, g = −0.013, SE = 0.115, p = .907). There was no significant difference between durations (Q = 2.347, p = .309). Within-subjects findings yielded no significant differences besides control participants approaching reward at >500 ms (k = 9, g = 0.103, SE = .047, p = .029), and control and high symptom participants differing in their reward-related biases at >500 ms (Q = 5.041, p = .025).

Summary of positive-related bias findings

We found a small effect demonstrating that symptomatic individuals differ from controls in the manner in which they process positive information. We also found some evidence that asymptomatic control participants approach positive information, providing support for evenhandedness and self-regulatory frameworks, and helping to explain why some between-subjects effects were mildly larger than within-subjects effects (see Table 1).

Moderator analyses examining stimulus type, duration, and clinical versus nonclinical samples yielded equivocal findings, but our most striking finding was that participants endorsing depressive or dysphoric symptoms as their primary symptom were more likely to avoid positive information than other symptomatic individuals. This finding emerged in our meta-analysis of not only between-subjects, but also within-subjects effects, suggesting that it is driven by the devaluation of rewarding information. This is consistent with the underlying depressogenic process posited to date by proponents of the reward devaluation accounts. The level of specificity of this constellation of findings, such that participants whose main presenting symptom was depression were the only group to robustly demonstrate reward devaluation, suggests that (a) the specificity posited by the even-handedness framework may be correct but the hypothesis of a mere lack of valuation of positivity may need refinement and that (b) the avoidance of positivity posited by the reward devaluation framework may be correct but that the hypothesis that this process is equated in depression and anxiety may need refinement (see Table 1), although evidence regarding differences in selective attention of reward between persons with primary symptoms of depression and social anxiety remains somewhat equivocal. In light of these results, we will thus unpack and refine the reward devaluation framework below after summarizing the remaining meta-analytic findings.

These findings emerged with inclusive criteria (including full re-analysis of response time table data to calculate effect sizes), a conservative effect size estimate, and with the vast majority of studies primarily investigating threat-based biases, and thus commonly not using positive stimuli more specific than smiling faces. We therefore have presented the most comprehensive evidence to date supporting the hypothesis that depressed individuals devalue rewarding information. As noted below, these findings also diverge from those related to attentional biases for negative information.

Meta-analysis of between-subject attentional biases for negative information

Across all of the studies that reported between-subjects effects and used negative stimuli, the combined negative-related bias was both positive and significant in depressed and anxious participants (k = 53, n = 2,829, g = 0.289, SE = .048, p < .001). Thus, a significant vigilance to negative information was found for depressed and anxious participants in comparison to asymptomatic controls (see Table 7).

Table 7.

Meta-Analytic Results of Between-Groups Bias for Negative Information for High Symptom vs. Control Participants

Moderator	k	N	g	SE	Q	p
Total data set	53	2,829	0.289	0.048		<.001
Primary symptoms					4.606	.203
Depression	14	768	0.472	0.099		<.001
General anxiety	19	1,057	0.263	0.074		<.001
Social anxiety	9	437	0.210	0.102		.040
Panic	5	245	0.424	0.148		.004
Clinical v. nonclinical					1.377	.241
Clinical	23	986	0.361	0.084		<.001
Nonclinical	30	1,843	0.242	0.057		<.001
Stimulus duration					6.566	.038
< 200 ms	8	470	0.484	0.151		.001
200–500 ms	31	1,7401	0.181	0.048		<.001
> 500 ms	14	618	0.470	0.145		.001
Stimulus type					1.172	.279
Words	15	714	0.216	0.076		.005
Images	38	2,115	0.321	0.061		<.001

¹Participants from Oehlberg et al., 2012 varied slightly (+/− 4) by analyses for duration and primary symptom.

Meta-analysis of within-subject attentional biases for negative information

Across all studies that reported within-subjects effects and used negative stimuli, depressed and anxious participants evidenced significant biases toward negative information (k = 51, n = 1,177, g = 0.120, SE = .021, p < .001). The negative-related bias was significant in the opposite direction for asymptomatic controls (k = 37, n = 982, g = −0.044, SE = .018, p = .012). The difference between the combined within-subjects effect sizes of high symptom participants in comparison to asymptomatic participants was significant (Q = 36.039, p < .001). Thus, a significant vigilance toward negative information in depressed and anxious individuals was present, a smaller but significant avoidance of negative information emerged in asymptomatic controls, and the two groups significantly differed from each other (see Table 8).

Table 8.

Meta-Analytic Results of Within-Group Bias for Negative Information for High Symptom and Control Participants

	Within high symptom						Within control						High symptom v. control
Moderator	k	N	g	SE	Q	p	k	N	g	SE	Q	p	Q	p
Total data set	51	1,177	0.120	0.021			37	982	−0.044	0.018		.012	36.039	<.001
Primary symptoms					1.910	.591
Depression	11	322	0.125	0.054		.022
General anxiety	17	379	0.131	0.032		<.001
Social anxiety	14	294	0.103	0.045		.022
Panic	5	95	0.189	0.046		<.001
Clinical v. nonclinical					<1	.959					<1	.592
Clinical	26	589	0.120	0.033		<.001	15	369	−0.032	0.024		.181	13.925	<.001
Nonclinical	25	588	0.118	0.027		<.001	22	613	−0.050	0.025		.043	20.605	<.001
Stimulus duration					2.714	.257					<1	.778
< 200 ms	11	238	0.190	0.055		.001	7	221	−0.052	0.037		.157	13.450	<.001
200–500 ms	25	528	0.093	0.028		.001	22	518	−0.050	0.023		.029	15.616	<.001
> 500 ms	15	411	0.138	0.049		.005	8	243	−0.015	0.046		.738	5.235	.022
Stimulus type					1.633	.201					<1	.842
Words	15	335	0.075	0.041		.069	12	288	−0.051	0.034		.143	5.450	.020
Images	36	842	0.137	0.024		<.001	25	694	−0.042	0.021		.041	31.093	<.001

Do negative biases differ as a function of primary symptom?

We examined between- and within-subject negative biases at each level of primary symptom endorsed by the high-symptom group. As with the reward-bias analyses, we first examined all primary symptom groups, then performed a secondary analysis examining only the symptom groups with enough existing studies, with the exception that panic symptoms were included in the secondary analysis because they yielded a significant effect.

Between-subjects combined effects at each level of primary symptom yielded a significant vigilance toward negative information in participants with primary symptoms of depression (k = 14, g = 0.472, SE = 0.099, p < .001), general anxiety (k = 19, g = 0.263, SE = 0.074, p < .001), social anxiety (k = 9, g = 0.210, SE = 0.102, p = .040), and panic (k = 5, g = 0.424, SE = 0.148, p = .004), but not PTSD (k = 4, g = −0.015, SE = 0.122, p = .902). Between-subjects group differences were significant with five (Q = 11.044, p = .026) but not four (Q = 4.606, p = .203) groups. Unlike with reward-based analyses, between-subjects effects for depression did not differ significantly from general anxiety (Q = 2.846, p = .092) or social anxiety (Q = 3.414, p = .065).

Within-subject effects also yielded a significant vigilance toward negative information in depressed participants (k = 11, g = 0.125, SE = 0.054, p = .022), general anxiety (k = 17, g = 0.131, SE = 0.032, p < .001), social anxiety (k = 14, g = 0.103, SE = 0.045, p = .022), panic (k = 5, g = 0.189, SE = 0.046, p < .001), but not for PTSD (k = 3, g = 0.000, SE = 0.110, p = .998). Within-subjects group differences were not significant with all five groups (Q = 3.384, p = .496), or with four groups included in the model (Q = 1.910, p = .591). The within-subjects effects for depression did not differ significantly from general anxiety (Q < 1, p = .920) or social anxiety (Q < 1, p = .760), again differing from the reward-bias findings.

Do negative biases differ due to clinical versus non-clinical samples?

The combined between-subject effect sizes for negative biases were significant for clinical (k = 23, g = 0.361, SE = 0.084, p < .001) and nonclinical (k = 30, g = 0.242, SE = 0.057, p < .001) samples, and this difference was not significant (Q = 1.377, p = .241). The within-subjects effects for both non-clinical asymptomatic (k = 22, g = −0.050, SE = 0.025, p = .043) and non-clinical but symptomatic (k = 25, g = 0.118, SE = 0.027, p < .001) participants were significant and significantly differed from each other (Q = 20.605, p < .001). The combined within-subjects effects within studies using clinical samples for asymptomatic controls (k = 15, g = −0.032, SE = 0.024, p = .181) were not significant, whereas symptomatic participants were significant (k = 26, g = 0.120, SE = 0.033, p < .001). The comparison of symptomatic and asymptomatic participants was significant (Q = 13.925, p < .001).

Do negative biases vary as a function of stimulus duration?

The combined between-subject effect sizes for negative biases were significant for durations above 500 ms (k = 14, g = 0.470, SE = 0.145, p = .001), between 200 and 500 ms (k = 31, g = 0.181, SE = 0.048, p < .001), and durations below 200 ms (k = 8, g = 0.484, SE = 0.151, p = .001). There was a significant difference between durations (Q = 6.566, p = .038). Within-subjects findings yielded significant vigilant biases for depressed and anxious participants at each level of duration, but no significant difference by duration for symptomatic participants or asymptomatic controls.

Do negative biases differ as a function of whether stimuli were words or pictures?

The combined between-subject effect sizes for negative biases were significant for pictures (k = 38, g = 0.321, SE = 0.061, p < .001) and words (k = 15, g = 0.216, SE = 0.076, p = .005). There was no significant difference between words and pictures for between-subjects findings (Q = 1.172, p = .279). For the within-subjects effects, symptomatic participants (k = 15, g = 0.075, SE = 0.041, p = .069) and asymptomatic controls (k = 12, g = −0.051, SE = 0.034, p = .143) were not biased toward negative words, though they significantly differed (Q = 5.450, p = .020). However, asymptomatic controls significantly were biased away from negative pictures (k = 25, g = −0.042, SE = 0.021, p = .041), whereas symptomatic participants were biased toward negative pictures (k = 36, g = 0.137, SE = 0.024, p < .001), with these comparisons differing significantly (Q = 31.093, p < .001). Thus, symptomatic participants were significantly approaching negative stimuli.

Summary of negative bias findings

We found consistent evidence of vigilance toward negative information in depressed and anxious individuals similar to previous meta-analytic reviews (e.g., Bar-Haim et al., 2007), though our effect sizes were somewhat smaller (e.g., within-subjects threat bias, Bar-Haim et al.: d = 0.45; within-subjects negative bias reported here: g = 0.12) due to our use of an effect size with a built-in correction, our inclusion of heterogeneous primary symptom groups, and our inclusive use of reported data, including recalculation of response time table data to provide the most inclusive estimate of effect sizes. Unlike the reward-bias effects, these effects did not markedly differ by primary symptom, such that individuals reporting panic and trait anxiety as their primary symptoms robustly demonstrated vigilance toward negative information.

Summary of conventional meta-analysis

The most striking finding of our comparative meta-analyses was that individuals with a primary symptom of depression systematically avoided positive information, yielding evidence consistent with reward devaluation and even-handedness theoretical explanations. This pattern of avoidance in individuals with symptoms of depression differed markedly from other symptom groups, and yielded effect sizes comparable to those produced by our meta-analysis of negative DP biases in both within-subjects (depression avoidance of positivity: g = −0.167, SE = 0.063; high-symptom vigilance toward negativity: g = 0.120, SE = 0.021) and between-subjects (depression avoidance of positivity: g = −0.399, SE = 0.084; high symptom vigilance toward negativity: g = 0.289, SE = 0.048) analyses. Thus, consistent with our rationale for the inclusion of negative DP bias findings, these findings provide the strongest evidence to date that reward-based biases in individuals with symptoms of depression are real effects. Our other meta-analysis now assesses the specificity of this constellation of findings.

P-curves of Dot-probe Findings

Overview of p-curve

P-curve is a meta-analytic technique for determining the likelihood that a set of p-values would occur in the absence of a real effect. If a studied effect is non-existent, then all significant (p ≤ .05) findings showing that effect presumably occurred by chance. For any single significant finding the probability of p ≤ .01 is .2, the probability of p ≤ .02 is .4, the probability of p ≤ .03 is .6, the probability of p ≤ .04 is .8, and the probability of p ≤ .05 is 1. Thus, given enough independent observations, the set of significant p-values for a non-existent effect should settle into a uniform distribution that looks like a flat line when plotted. If a studied effect does exist, however, then the set of significant findings showing that effect should include a greater proportion of very low p-values than would have occurred by chance. In other words, the distribution of p-values for a true effect should be right-skewed. Conversely, selective reporting tends to produce a greater proportion of significant p-values near .05 than would be expected to occur by chance, because researchers generally stop running additional analyses when they achieve a significant effect. This pattern of data-analysis reduces the right skew of the p-value distribution for a real effect and can produce a left-skewed distribution for a small or nonexistent effect.

To test for right skew, p-curve takes each significant p-value in an observed set and calculates the probability of a p-value as small or smaller occurring if one assumes a uniform distribution of p-values. This produces a set of pp-values, which are then combined via Fisher’s method (Fisher, 1925; Simonsohn et al., 2014). The resulting χ² test statistic represents the likelihood that a distribution of p-values at least as right-skewed as the observed distribution would occur by chance, i.e., assuming the null of no true effect. If the χ² test for right-skew is statistically significant, then it is very unlikely the observed distribution would have occurred in the absence of a real effect. The null can be rejected in favor of the alternative hypothesis that at least some of the p-values included in the curve have evidential value. If the right-skew test is not significant, then the null cannot be rejected, that is, selective reporting of false positives cannot be ruled out as an explanation for the observed set of significant findings.

A set of p-values may fail to show significant right skew for a number of reasons: the studied effect may not exist, it may exist but be very small, the set of studies included in the p-curve may have been severely underpowered, or there may be too few p-values in the set for p-curve to be conclusive. To reduce this ambiguity, a second set of pp-values is calculated assuming a null of a small true effect (i.e., assuming a slightly right-skewed distribution). If this test is statistically significant, then one can reject the null of a small true effect and thus conclude that the observed set of p-values lacks evidential value. Finally, a test for left skew can be conducted by the same process used to test for right skew, except that in a left-skew test each pp-value represents the probability of a p-value as large as or larger than an observed value occurring by chance, assuming the null of no true effect. A statistically significant left-skew test indicates an extremely strong likelihood that at least some of the p-values in the observed set were selectively reported.

Selection rules

In addition to the inclusion and exclusion criteria for the narrative review (see “Parameters of the Review”), the following rules were used in selecting p-values for inclusion in our p-curve analyses:

1. In keeping with general guidelines for the use of p-curve, only p-values that are statistically significant at the .05 level qualified for inclusion in p-curves. Because non-significant p-values commonly go unpublished due to selective reporting or publication bias, the published record of p-values larger than .05 would not provide an unbiased picture of the full range of p-values obtained for a given phenomenon.

2. We included both within-subjects and between-subjects effects in p-curves because p-curve’s accuracy at detecting evidential value depends, in part, on the number of p-values entered into a curve. Including the broadest range of relevant findings when testing an effect maximizes the number of p-values included in p-curves and thus the accuracy of the meta-analyses.

3. P-values from analyses comparing symptomatic participants’ bias scores to a hypothetical mean of zero (within-subjects effects), analyses comparing symptomatic participants’ bias scores to bias scores of healthy controls (between-subjects effects), and correlational analyses measuring the relationship of bias scores to symptom levels, were considered for inclusion in p-curves. P-values from within-subjects analyses of healthy controls only, i.e., analyses that did not include participants with elevated depressive or anxious symptoms, were not used in any p-curves.

4. Because the p-curve technique requires that p-values used be independent of one another, only one p-value from each experiment was included in any one p-curve. This means that a p-value for a within-subjects analysis and a p-value for a between-subjects analysis from that same experiment could not both be included in a single p-curve. Thus, we computed all p-curves in two ways. For the first p-curve of each pair we prioritized within-subjects effects for inclusion and only used between-subjects effects from studies when within-subjects effects were non-significant or were not reported by the original authors. This procedure was reversed for the second p-curve of each pair, wherein significant between-subjects effects received priority for inclusion. In the event that neither within-subjects nor between-subjects effects reached significance, or for studies where participants were not split into groups, p-values from correlational analyses were included in p-curves. This p-value selection process is described in more detail in the section titled P-curves of Avoidance of Positive Information.

5. For studies that included two independent samples of participants with elevated clinical symptoms, one p-value from each sample could be included in the same p-curve without violating the rule requiring independent p-values. For example, the 2008 study by Pishyar and colleagues included two groups of socially phobic participants, one assigned to a cognitive-behavioral group therapy condition and the other assigned to a wait-list condition. Both groups exhibited biases away from positive information (compared to zero) at baseline, so both p-values could be included in a p-curve.

6. If p-values for more than one independent variable were equally qualified for inclusion in a p-curve, then the smallest p-value was used. For example, in a 2010 study by Taylor, Bomyea, and Amir, positive attentional bias was negatively correlated with both social anxiety and anxiety sensitivity, so we included the social anxiety finding because it had the smaller p-value. This yielded the most conservative approach for detecting any lack of evidential value.

7. Studies wherein clinical symptoms were associated with similar patterns of bias (either vigilance or avoidance) for both positive and negative stimuli (e.g. Bradley, Mogg, White, Groom, & de Bono, 1999; Mansell, Ehlers, Clark, & Chen, 2002; Sposari & Rapee, 2007) or for all faces (e.g. Chen, Ehlers, Clark, & Mansell, 2002) were not included in p-curves, because the purpose of this meta-analysis was to test evidential value of findings showing selective avoidance that is specific to positive information, in relation to depressive and anxious symptoms. Studies showing a broader vigilance toward or avoidance of both positive and negative emotional stimuli may be capturing a qualitatively different phenomenon than the one we set out to evaluate here and thus would not be relevant to our research question or analyses.

8. Studies that found vigilance for positive information were excluded from the positive bias p-curves (e.g., Fani, Bradley-Davino, Ressler, & McClure-Tone, 2011). Again, this decision was made because we were specifically interested in testing whether significant findings showing avoidance of positive information reflect a real effect.

9. In the case of studies that involved both an exam-related stress condition and lab-stressor condition (e.g., Mogg, Bradley, & Hallowell, 1994), we allowed p-values from the exam condition (i.e., students were completing the DP task in advance of an upcoming test) into curves because stress was naturally occurring based on factors in the participants’ lives. Likewise, when a study involved a therapeutic intervention (e.g., Pishyar, Harris, & Menzies, 2008) or administration of alcohol (Stevens, Rist, & Gerlach, 2009) only p-values for analyses of baseline data or from placebo groups were included in p-curves, consistent with our inclusion/exclusion criteria.

Please see Appendix A for details on selection of p-values from studies wherein the original authors hypothesized reversing or attenuated interactions.

P-curves of Avoidance of Positive Information

First, we tested the evidential value of dot-probe findings linking clinical symptoms to avoidance of positive information to determine the likelihood that these findings reflect a real effect. We used p-curve (http://www.p-curve.com/app) to compute p-curves of relevant positive bias effects in two different ways. Details about the studies and specific p-values included in these p-curves are presented in Tables 2–4 and in Appendix C, Table C1.

For the first curve, 19 independent p-values, taken from 18 studies, met inclusion criteria (see Appendix D, Table D1). In this curve we prioritized within-subjects findings for inclusion. Thus, whenever a single study showed significant avoidance of positive stimuli (compared to a hypothetical mean of zero bias) within the high-symptom group and significant differences in bias for positive stimuli at different symptom levels, we included the p-value for the within-subjects finding. For studies that did not show significant avoidance of positive stimuli within the high-symptom group, we included p-values for significant between-subjects differences in bias for positive stimuli. If neither within- nor between-subjects findings reached significance, or if symptom severity was analyzed as a continuous rather than a categorical variable, then we included p-values for significant negative correlations between depressive or anxious symptoms and bias for positive stimuli.

The resulting distribution of 19 independent p-values was tested for right skew, and the result was significant, χ²(38) = 75.48, p = .0003 (Figure 4A). This outcome indicates that, consistent with our prediction, the findings included in this p-curve have evidential value. Furthermore, neither the test for lack of evidential value, nor the test for left skew, approached significance, yielding chi-square values of χ²(38) = 37.88, p = .4748 and χ²(38) = 25.14, p = .9459, respectively. Thus, selective reporting can be ruled out as the sole explanation for dot-probe findings showing avoidance of positive information in participants with depression and anxiety.

Figure 4.

P-curves of p-values for findings showing avoidance of positive information in individuals with symptoms of depression or anxiety. Within-subjects effects were given priority for inclusion in the p-curve on the left (A), followed by between-subjects effects, followed by correlations. For the p-curve on the right (B) between-subjects effects were given priority for inclusion, followed by within-subjects effects, followed by correlations. Note. Blue line represents distribution of p-values.

To investigate whether there were differences between within-subject and between-subject findings showing avoidance of positive information, we also computed a second p-curve of findings from the same set of studies. This time, if a single study showed significant avoidance of positive stimuli within the high-symptom group and significant differences in bias for positive stimuli at different symptom levels, we included the p-value for the between-subjects finding. In the absence of significant between-subjects differences in bias for positive stimuli, we included p-values for significant within-subjects avoidance of positive stimuli in the high symptom groups. As before, p-values for significant negative correlations between anxious or depressive symptoms and bias for positive stimuli were included in the absence of significant within- or between-subjects findings (Appendix D, Table D2).

This distribution of p-values was tested for right skew, and again the result was significant, χ²(38) = 86.41, p < .0001 (Figure 4B). Also consistent with the first p-curve, neither the test for lack of evidential value nor the test for left skew approached significance, this time yielding chi-square values of χ²(38) = 34.95, p = .6111 and χ²(38) = 23.56, p = .9679, respectively. Thus, selective reporting can be ruled out as the sole explanation for dot-probe findings yielding differences in avoidance of positive information between participants with symptoms of anxiety and/or depression and controls.

P-curves for Vigilance Toward Negative Information

The p-curves of positive bias findings indicate that a true relationship exists between elevated symptoms of depression and anxiety and avoidance of positive information (see Conventional Meta-Analysis findings above). It is possible that previous null findings regarding positive information-processing biases were partially due to weaknesses of the dot-probe task. Indeed, previous dot-probe studies using neutral and threatening words and images suggest that the task has low split half reliability in non-clinical samples (Schmukle, 2005).

Beyond questions of task reliability, however, it is important to consider that findings showing avoidance of positive information emerged from studies primarily aimed at measuring attentional biases for negative information in relation to clinical symptoms. We reasoned that researchers may run more analyses to test their primary hypotheses, thus placing negative attentional bias findings at higher risk of Type I error, while placing positive attentional bias findings at higher risk of Type II error.

In order to explore the possibility of higher rates of Type I error in negative attentional bias findings in individuals with depression and anxiety, we computed p-curves of significant analyses showing vigilance to negative information in relation to clinical symptoms. Specifically, we compared p-curves of negative bias findings from studies that also found avoidance of positive information to those from studies that included positive stimuli but did not yield significant positive avoidance effects. The same rules were used to select significant p-values showing vigilance for negative information as were used to select p-values showing avoidance of positive information.

Negative bias findings from studies showing avoidance of positive information

Of the 17 studies that contributed p-values to the p-curves for avoidance of positive information, 16 also included negative stimuli, and 9 yielded 10 independent p-values demonstrating vigilance toward negative information in depressed or anxious individuals. Details about the studies and specific p-values included in these p-curves are presented in Table 2–4 and in Appendix C, Table C2.

Table C2.

P-curve Disclosure Table for Vigilance Toward Negative Information

Study	Quoted Text from Original Paper Stating Hypotheses	Study Design	Finding(s) to be P-curved	Quoted Text from Original Paper Describing Within- Subjects Results	Quoted Text from Original Paper Describing Between-Subjects Results	Quoted Text from Original Paper Describing Correlational Results	Why p-value(s) selected do not violate p-curve guidelines regarding attenuated or reversing interactions
Bradley, Mogg, Falla, & Hamilton (1998)	The main aims of the study were to examine: (1) whether high trait anxious individuals show an initial attentional bias towards threat faces (i.e. in 500msec condition); and (2) whether this predicted attentional bias will be maintained at the longer interval of 1250msec, or whether avoidant strategies may emerge in the latter condition.	2 × 2 × 2 ANOVA of bias scores with trait anxiety (high, low) as a between-subjects variable and exposure duration (500, 1250 ms) and emotional face type (threat, happy) as within-subjects variables	Difference of means (threat bias score for high anxious vs. low anxious participants)	Bias scores were also tested against zero (0 = no attentional bias). The high trait anxiety group showed significant vigilance for threat faces averaged across both durations [t(18) = 2.20, P < 0.05] …	A significant effect of group was found on both Threat [F(1,36) = 5.99, P < 0.05] … bias scores, indicating that, irrespective of exposure duration, the high trait anxiety group, compared with the low trait group, were more vigilant for threat …	N/A	Original authors’ stated hypotheses were exploratory in nature (“whether”) and constituted simple effects that may or may not have been attenuated by duration.
Bradley et al. (Experiments 1 and 2 combined; 1997)	No explicit hypotheses were stated regarding the results included in p-curves, which were from a combined data set that included data from Experiment 1, Experiment 2, and 11 additional participants	2 × 2 × 2 ANOVA of bias scores with group (dysphoric, nondysphoric) and gender (male, female) as between-subjects variables and emotional face valence (threat, happy) as a within-subjects variable	Difference of means (mean threat bias for dysphoric vs. nondysphoric)	*Non-significant	Furthermore, comparison of the two groups on their bias scores for threat faces confirmed that the nondysphoric group were significantly more avoidant of threat faces relative to neutral faces, than the dysphoric group [F(1, 69) = 4.29, P < .05]	*Not reported	Original authors did not state hypotheses regarding results of Experiments 1 & 2 combined. For Experiment 1 alone, original authors’ stated hypothesis regarding dysphoria was that “Nondysphoric subjects have an attentional bias favoring happy faces,” but no predictions were made about processing of happy faces among dysphoric participants or processing of threat faces. (The only hypothesis about bias for threat faces concerned social anxiety rather than dysphoria.)
Donaldson, Lam, & Mathews (2007)	First, we predicted that at baseline, depressed patients would show an attentional bias for negative words presented for 1000 ms but not for 500 ms; that these biases would not occur for positive or neutral words and that control participants would not show any biases at all regardless of exposure duration and word valence. Second, we predicted that the rumination induction would produce an increase in the attentional bias for negative words in depressed patients at 1000 ms, the distraction induction would produce a decrease and there would be no change in attentional responses for control participants. Third, we predicted that the strength of the baseline attentional bias shown by depressed patients at 1000 ms would be proportionally related to levels of trait rumination and inversely related to levels of trait distraction.	2 × 2 × 2 × 3 × 2 ANOVA of bias scores with group (controls, patients) and induction (distraction, rumination) as between-subjects variables and exposure duration (500 ms, 1000 ms), word type (negative, positive, neutral), and time (pre- and post-induction) as within-subjects variables	Difference of means (depressed group mean negative bias score vs. hypothetical mean of zero)	Results showed that bias scores for negative words were significantly greater than for positive or neutral words (negative and neutral words: t = 3.7, 35df, p ≤ .001).	*Not reported	N/A	The original authors hypothesized an attenuated interaction, but one including duration and induction, which was unlikely to result in selective reporting of simple effects (as the DP literature allows for the publication of simple effects with or without an interaction with duration even when multiple durations are part of the design). We thus examined the relevant simple effect, which is solely within-subjects and distinct from the irrelevant hypothesized interaction.
Eldar, Yankelevitch, Lamy, & Bar-Haim (2010)	We expected to replicate the established finding of attentional bias towards threat in anxious participants. That is, faster RTs to targets replacing angry faces than to targets replacing neutral faces in anxious individuals but not in nonanxious controls.	2 × 2 ANOVA of bias scores with group (HA, LA) as a between-subjects variable and emotional face type (angry, happy) as a within-subjects variable	Difference of means (HA group mean angry bias score vs. hypothetical mean of zero)	One-sample t-tests show that the attention bias towards angry faces was significantly greater than zero in the anxious group, t(22) = 2.87; *One-sample t-test recalculated using bias score and standard deviation, t(22) = 2.93.	*Non-significant	*Not reported	The interaction between bias toward threat and anxiety level was not significant, so the within-subjects threat bias for the high anxious group was not selectively reported on the basis of the hypothesized attenuated interaction.
Fox (Experiment 2; 2002)	The aim of Experiment 2 was to evaluate whether HA people allocate attention toward the location of masked fearful faces. On the basis of the results of Experiment 1, as well as of previous research (Mogg & Bradley, 1999), the hypothesis was that this propensity should be most apparent when the fearful face was presented in the LVF.	2 × 2 × 2 ANOVA of bias scores with group (HA, LA) as a between-subjects variable and emotional face type (fearful, happy) and location of emotional face (LVF, RVF) as within-subjects variables	Difference of means (mean fearful bias for HA vs. LA groups); Difference of means (HA group mean fearful bias score vs. hypothetical mean of zero)	Once again, with a one-tailed test, the HA group demonstrated vigilance for fearful faces appearing in the LVF [140.4; t(17) = 5.7, p < .001] and in the RVF …	With one-tailed tests of significance, it was found that the HA group was more vigilant than the LA group for fearful faces in both the LVF [140.4 vs. 25.7; t(34) = 4.6, p < .001] and the RVF …	N/A	The interaction between anxiety group, emotional face type, and location of emotional face was not significant, so within- and between-subjects bias effects were not selectively reported on the basis of the hypothesized attenuated interaction. Results were reported separately for each visual field; in line with our selection rules, p-values for the left visual field (LVF) were included in p-curves because they were smaller than those for the right visual field.
Fritzsche et al. (2010)	In the present study we examined whether formerly depressed individuals and patients with asthma exhibit cognitive biases similar to those observed in currently depressed individuals. In addition, we compared these groups with healthy participants.	4 × 2 ANOVA of bias scores with group (asthma, depressed, remitted depressed, controls) as a between-subjects variable and emotional face type (sad, happy) as a within-subjects variable	Difference of means (sad bias score for MDD vs. NC group)	Attentional bias score for the MDD group towards sad faces was positive and significantly different from zero [t(19) = 2.37, p < 0.05]	Both the MDD (p < 0.001) and RMD groups (p < 0.05), which did not differ from each other, were faster in detecting the dot probes behind sad faces than the NC group. *Calculated based on bias scores and standard deviations reported in a table: MDD vs. NC group sad bias scores, t(38) = 3.14	*Not reported	Original authors’ hypotheses were exploratory in nature with regard to control-group comparisons, and included simple effects of MDD toward sadness bias; also, the omnibus interaction incorporated asthma and remitted depressed groups and thus was unlikely to affect publication bias for between-subject and within-subject findings only including MDD or MDD and NC groups.
Fritzsche et al. (2013)	We hypothesized that compared with healthy controls, COPD patients with comorbid depression would show depression-specific biases in information processing, which are comparable to those observed in lung-healthy patients with depression.	4 × 2 ANOVA of bias scores with group (MDD, COPD + MDD, COPD, controls) as a between-subjects variable and emotional face type (sad, happy) as a within-subjects variable; One-sample t-tests comparing bias to a hypothetical mean of zero	Difference of means (MDD group mean sad bias score vs. hypothetical mean of zero); difference of means (sad bias score for MDD vs. NC group)	The positive bias score for sad faces of the MDD group differed significantly from zero, suggesting an approach bias towards sad faces, T(19) = 4.44	*Calculated based on mean bias score and standard deviation reported in a table: t(37) = 4.70	*Not reported	Original authors did not make specific predictions about the nature of attentional biases in lung-healthy MDD group; omnibus interaction incorporated COPD group and thus would be unlikely to result in publication bias; in addition, analyses within MDD group and between MDD group and Controls were exploratory in nature.
Gotlib, Kasch et al. (2004)	First, we predicted that MDD, GSP, and NC participants would differ with respect to biases in their processing of emotional stimuli. More specifically, we expected that, compared with the NC participants, both the MDD and the GSP participants would exhibit a negative bias on all tasks. Second, we predicted that we would find evidence of stimulus specificity, such that the MDD and GSP participants would demonstrate differential responses to the different categories of stimuli, with the GSP participants exhibiting stronger biases for socially threatening content stimuli and the MDD participants demonstrating stronger biases for sad-content stimuli.	3 × 3 ANOVA of bias scores with group (depressed, generalized anxiety, controls) as a between-subjects variable and emotional face type (sad, happy, angry) as a within-subjects variable	Difference of means (MDD group mean sad bias score vs. hypothetical mean of zero); difference of means (sad bias score for MDD vs. NC group)	*Calculated based on bias scores and standard deviations reported in a table: MDD group sad bias t(87) = 2.78	Follow-up Fisher LSD analyses indicated that the MDD participants had higher bias scores than did the NC participants in the sad face condition *Calculated based on bias scores and standard deviations reported in a table: MDD vs. NC sad bias t(141) = 2.18	*Non-significant	The Omnibus interaction investigates a research question irrelevant to our analysis. Both between-subjects and within-subjects hypotheses are simple effects. This does not appear to be an attenuated or reversing interaction, but instead hypotheses examining specificity of MDD and GSP. This could potentially be interpreted as a reversing interaction, but one reversing within negative stimuli, not positive stimuli, and thus is unlikely to have resulted in publication bias.
Gotlib, Krasnoperova, Yue, & Joormann (2004)	1. Depressed participants, compared with nonpsychiatric controls, will demonstrate an attentional bias for sad faces presented for 1,000 ms. 2. The depression-related attentional bias will be content specific. Thus, depressed participants, compared with nonpsychiatric controls, will demonstrate an attentional bias for sad faces but not for happy or angry faces. 3. The depression-related bias will be diagnosis specific. Thus, the attentional bias for sad faces exhibited by MDD participants will not be exhibited by individuals diagnosed with GAD.	Hypothesis 1: t-tests comparing MDD and NC participants’ bias scores for sad faces and comparing MDD participants’ bias score for sad faces to zero; Hypothesis 2: t-tests comparing MDD and NC participants’ bias scores for angry faces and for happy faces and comparing MDD participants’ bias scores to zero; Hypothesis 3: t-tests comparing MDD and GAD participants’ bias scores for sad faces and comparing GAD participants’ bias score for sad faces to zero	Difference of means (mean sad bias for MDD vs. NC groups); Difference of means (MDD group mean sad bias score vs. hypothetical mean of zero)	These analyses revealed that MDD participants’ attentional bias for sad faces was positive and significantly different from zero t(18) = 3.02, p < .01, whereas NC participants showed no attentional bias for sad faces…	As predicted, MDD participants demonstrated significantly greater vigilance for sad faces than did NC participants, t(33) = 2.27, p < .05.	*Not reported	Three hypotheses were tested with no omnibus ANOVA (in accordance with the APA Task Force on Statistical Interaction; Wilkinson, 1999); hypothesis 1 is the only investigation relevant to our analysis, and in hypothesis 1 the original authors predict a simple effect of depression on bias for sad faces, which is the finding we included in relevant p-curves prioritizing within-subjects effects [see Figure 4A and Figure 5A].
Hunt, Keogh, & French (Masked condition; 2006)	It was predicted that: 1. Those high in physical AS would exhibit attentional vigilance to words pertaining to anxiety symptomatology relative to neutral words; when such stimuli are presented below the threshold of conscious identification individuals with high physical AS will just exhibit a specific attentional vigilance to words pertaining to anxiety symptomatology; 2. Those low in physical AS would show attentional avoidance of threat words.	3 × 2 × 3 ANOVA of bias scores with anxiety sensitivity group (high, medium, low) and presentation condition (masked, unmasked) as between-subjects variables and valenced word type (anxiety symptomatology, social threat, positive) as a within-subjects variable	Difference of means (High AS group mean anxiety symptomatology bias score vs. hypothetical mean of zero)	*Non-significant	For the anxiety symptomatology index, a significant difference was found between the anxiety sensitivity groups in the masked condition, F(1, 103) = 8.02, p < .01	*N/A	Original authors’ stated hypotheses included simple within-subjects bias for anxiety symptomatology words in the high AS group; their prediction about low AS group pertained to social threat rather than anxiety symptomatology words so it may or may not represent a reversing interaction, and the simple effect of the masked condition is relevant to the present analysis.
Ioannou, Mogg, & Bradley (2004)	(1) High trait anxious individuals with low levels of defensiveness (HA) should show vigilance for threat. Moreover, they were expected to show a higher level of vigilance for threat when compared with either the low anxious or repressor groups (Bradley et al., 1998). (2) Repressors (REP) should show attentional avoidance of threat. (3) Defensive high anxious participants (DHA) should show an attentional bias for threat to at least the same degree as the HA group i.e., vigilance for threat. (4) Low anxious participants (LA) should not show an attentional bias. In addition, the task included trials in which a happy face was paired with a neutral face, in order to examine whether any attentional bias for threat faces was specific to negative stimuli or would also operate for emotional stimuli in general (i.e., for both threat and happy faces).	2 × 2 × 2 ANOVA of bias scores with trait anxiety (high, low) and defensiveness (high, low) as between-subjects variables and exposure duration (500, 1250 ms) and emotional face type (threat, happy) as within-subjects variables	Difference of means (HA group mean threat bias score vs. hypothetical mean of zero); difference of means (threat bias score for HA vs. LA group)	The results revealed that the HA group showed significant vigilance for threat faces in the 500 ms exposure duration condition t(10) = 2.25, p < 0.05)	A comparison of these groups’ threat bias scores under the 500 ms condition showed a significant result consistent with prediction t(19) = 2.43, p < 0.05).	*Not reported	Original authors’ hypothesis involved multi-directional, multi-level interaction not meeting criteria of mere attenuated or reversing interactions. Authors stated hypotheses included simple within-subjects bias for threat in the HA group and main effect of group for negative condition.
Joormann & Gotlib (2007)	If negative cognitive biases are not merely symptoms of depression, we expected that both currently and formerly depressed individuals, compared to never-disordered controls, would demonstrate an attentional bias for sad faces on the dot-probe task.	3 × 2 ANOVA of bias scores with group (depressed, remitted depressed, never-depressed controls) as a between-subjects variable and emotional face type (sad, happy) as a within-subjects variable	Difference of means (happy bias score for MDD vs. NC group)	These analyses revealed that the attentional bias scores for MDD and RMD participants toward sad faces at the 1-s exposure duration were positive and significantly different from zero: MDD, t(25) = 2.79 … both ps < .01	As hypothesized, both the MDD and RMD participants demonstrated significantly greater vigilance to the sad faces than did the NC participants: MDD, t(43) = 3.74 … both ps < .01	*Not reported	Original authors’ stated hypotheses included simple within-subjects bias for sad faces in the depressed group, and between-subjects differences between that group and controls; full ANOVA included remitted depressed individuals and thus were unlikely to result in publication bias regarding the within-subjects or between-subjects effects.
Keogh, Dillon, Georgiou, & Hunt (2001)	It was predicted that those high in anxiety sensitivity would exhibit specific selective attentional bias towards the location of physically threatening words relative to neutral controls. It was also expected that such individuals would not exhibit any such bias towards social threat-related or positive words. Finally, based on previous evidence with trait anxiety (MacLeod & Mathews, 1988) and anxiety sensitive pain patients (Asmundson et al., 1997), it was expected that those low in physical anxiety sensitivity would selectively avoid threatening material.	2 × 3 ANOVA of bias scores with physical anxiety sensitivity group (high, low) as a between-subjects variable and emotional word valence (physical threat, social threat, positive) as a within-subjects variable	Difference of means (physical threat bias score for high AS vs. low AS participants)	*Not reported	Those high in physical anxiety sensitivity were found to exhibit a selective attentional bias towards the location of physically threatening words, compared to those low in physical anxiety sensitivity who exhibited a relative avoidance of such material [F(1, 49) = 4.61, p < .05].	*N/A	The simple effects (i.e., within-subjects bias for physical threat in the high AS group and in the low AS group) are not included in the publication, so within- and between-subjects bias effects were not selectively reported on the basis of the hypothesized reversing interaction.
Klumpp & Amir (2009)	Based on the literature, our primary hypothesis was that socially anxious individuals would exhibit vigilance for threat when baseline trials were not taken into account. That is, they would be faster in their detection of threat faces on congruent trials (i.e., probes replace threat faces) than incongruent trials (probes replace neutral faces) compared to individuals without social anxiety. For our secondary hypothesis, we proposed that socially anxious individuals would be faster at detecting probes that replaced baseline (neutral neutral) trials compared to probes that replaced threat faces during congruent (threat neutral) trials.	2 × 2 × 2 × 2 ANOVA of RTs with group (LSA, HSA) as a between-subjects variable and congruency of emotional face and probe (congruent, incongruent), emotional face type, (angry, happy), and probe position (top, bottom) as within-subjects variables	Difference of means (HSA group mean RT for congruent angry trials vs. HSA group mean RT for neutral-neutral trials)	For the HSA group, results showed faster RTs on congruent trials compared to baseline trials (t(38)= 2.05, p < .05) suggesting vigilance for angry faces.	The results showed that the HSA group had faster RTs for angry faces than the LSA group (t(74) = 2.14, p < .04) *Comparison of RTs from congruent trials only; between-groups difference in RT was non-significant for incongruent trials	*Not reported	Original authors’ stated first hypothesis is a between-subjects main effect of negative bias and the second hypothesis examines simple within-subjects bias toward threat in the HSA group.
Miskovic & Schmidt (2012)	We hypothesized that socially anxious individuals, compared to their low anxious peers, would show vigilance toward angry faces at the early states of processing consistent with existing theoretical models (Rapee & Heimberg, 1997) and previous experiment (Mogg & Bradley, 2002; Mogg et al., 2004). We expected to observe either attenuated or avoidant responses to social threat at longer (1,250ms) durations. Furthermore, we expected that high, compared to low, socially anxious individuals would show greater orienting to moderately angry faces. We did not predict between-group differences for attention to happy expressions.	Separate 2 × 2 × 2 × 2 × 3 ANOVAs for each exposure duration (100 ms, 500 ms, 1250 ms) with group (high socially anxious, low socially anxious) as a between-subjects variable and congruence (congruent, incongruent), probe location (left, right), emotional face type (angry, happy), and emotional face intensity (mild, moderate, strong) as within-subjects variables	Difference of means (mean angry bias for high socially anxious vs. low socially anxious groups); Difference of means (high socially anxious group mean angry bias score vs. hypothetical mean of zero)	Within-group contrasts of the bias scores against zero (no bias) confirmed that the high socially anxious group showed selective attention toward angry faces, t(16) = 2.43, p = .03, but no preferential processing of happy faces (p > .58).	There was a significant main effect of Group for the angry-neutral, F(1, 31) = 4.63, p = .04, η2 = .13, but not the happy-neutral pairs (p > .13).	*N/A	Original authors’ stated hypotheses only predicted between-group differences (that socially anxious participants would show comparatively more vigilance to threat than low anxious participants) The prediction about socially anxious participants’ attention to threat at different durations leaves open the possibility of either an attenuated or reversing interaction (“either attenuated or avoidant responses to social threat at longer (1,250 ms) durations.
Mogg & Bradley (1999a)	In the present study, we investigated two issues: (1) whether this evidence of an anxiety-related bias for threat faces could be replicated and (2) whether the same pattern of results, and a similar effect size, would be obtained using a probe position task (“Is the probe on the left or right?”), rather than the probe classification task used by Bradley et al. (1998).	2 × 2 ANOVA of bias scores with group (high trait anxiety, low trait anxiety) as a between-subjects variable and emotional face type (happy, threat) as a within-subjects variable	Difference of means (mean threat bias for high trait anxious vs. low trait anxious groups); Difference of means (high trait anxious group mean threat bias score vs. hypothetical mean of zero)	Comparisons of the bias scores against zero (0 = no attentional bias) showed significant vigilance for threat faces, relative to neutral faces, in the high trait group (t(18) = 2.45, p < 0.05)…	The high trait anxious group showed significantly more vigilance for threat faces compared with the low trait group (t(36) = 2.12, p < 0.05)…	*Not reported	Original authors’ stated hypotheses were to replicate the pattern (anxiety-related bias for threat faces) found in Bradley et al. (1998) with a variant of the DP task. Relevant hypotheses revolved around between- and within-group differences in threat bias, not proposed interactions with positive bias.
Mogg & Bradley (Experiment 3; 1999b)	Our hypotheses remained the same: Attention will be preferentially allocated towards the spatial location of masked threat faces rather than happy faces (i.e. Face type x Face location x Probe location interaction). In addition, this bias will be more evident in high than low anxious individuals.	2 × 2 × 2 × 2 ANOVA of Face type x Face location x Probe location x Trait anxiety (high trait anxiety, low trait anxiety) as a between-subjects variable and emotional face type (happy, threat) as a within-subjects variable	Difference of means (high trait anxious group mean threat bias score vs. hypothetical mean of zero)	Paired contrasts were carried out for the high trait anxious group: For trials with Threat-Neutral face pairs, left probes were detected significantly faster if they were preceded in the same location by a masked threat face (496msec) rather than a neutral face [523msec; t(10) = 2.37, P < .05].	*Not reported (there is a reported interaction incorporating positive words, but this is not included because of our selection rules dictating separate positive and negative p-curves)	*Not reported	Original authors’ hypotheses seem to include an attenuated hypothesis, i.e., that both groups will show bias toward threat rather than happy faces, but this bias will be greater for the high anxious group. However, they do not report between-groups difference in mean bias scores for threatening faces without folding in effects from bias to positive faces as well (the Face Type x Face Location x Probe Location x Trait Anxiety interaction). Because no between-subjects analysis specifically testing threat bias was reported in the paper, no between-subjects findings are included. In addition, it is likely that the simple effect (bias toward threat within the high anxious group) was not selectively reported on the basis of the hypothesized attenuated interaction being significant, because it is separate from multiple-valenced hypotheses.
Mogg & Bradley (2002)	One aim of the present study was to examine whether the finding of an anxiety-related bias for masked threat faces would replicate (Mogg & Bradley, 1999b, Study 3). … If participants show automatic orienting towards threat, they should be faster to detect probes that replace masked threat, rather than masked neutral faces.	2 × 2 × 2 × 2 ANOVA of Face type x Face location x Probe location x Social anxiety	Difference of means (mean threat bias for high socially anxious vs. low socially anxious groups); Difference of means (high socially anxious group mean threat bias score vs. hypothetical mean of zero)	Further analyses of the RT data from trials with threat-neutral faces showed that the high social anxiety group was 12 ms faster in responding to probes that replaced masked threat rather than neutral faces (530 vs. 542 ms; paired t(10)=2.68, p < 0.05), which is consistent with an attentional bias towards the location of the masked threat stimuli.	The interaction effect of social anxiety x face location x probe location was significant for threat–neutral faces (F(1, 25) = 14.17, p < 0.01)…	*N/A	The four-way interaction incorporates positive valence and thus violates our selection criteria and is not included. The three-way follow-up interaction at the level of threat-neutral faces was included in p-curve, which is associated with the original authors’ hypothesis.
Mogg, Bradley, & Hallowell (1994)	Our main hypotheses were as follows: 1. Under examination-induced stress, high trait anxious subjects will shift attention towards the spatial location of unmasked threat stimuli, whereas low trait subjects will shift attention away from such stimuli (MacLeod & Mathews, 1988). 2. The pattern of attentional biases predicted above (Hypothesis 1) is a function of a prolonged rather than an acute stressor (Mogg et al., 1990). That is, the interaction effect of stress and trait anxiety on attentional responses to threat will be specific to exam-induced stress rather than lab-induced stress. In addition, if the pattern of attentional biases predicted above (Hypotheses 1 and 2) is mediated by preconscious processes (Williams et al., 1988), the masked and unmasked exposure conditions should yield a similar pattern of results. Thus, high trait anxious subjects should shift attention towards the spatial location of threat stimuli presented outside awareness, in comparison with low trait subjects, and this bias should be evident in the exam stress condition.	2 × 3 × 2 × 2 ANOVA of bias scores with trait anxiety group (low, high) as a between-subjects variable and stress (lab, exam, none), exposure (masked, unmasked), and threat word type (achievement-related, physical) as within-subjects variables; attentional bias for positive-neutral word pairs was examined separately and results were non-significant	Difference of means (mean threat bias for high trait anxious vs. low trait anxious groups); Difference of means (high trait anxious group mean threat bias score vs. hypothetical mean of zero)	Consistent with prediction, under exam stress, the bias scores of high trait subjects for unmasked threat scores were significantly greater than zero, t(35) = 2.04	These showed a significant effect of trait anxiety on bias scores for unmasked threat under exam stress, F(1, 64) = 7.34, p < 0.01…	*Not reported	Original authors predicted a reversing interaction (hypothesis 1), so p-values for both simple effects (within-subjects bias to threat in the high anxious group and in the low anxious group) would normally go into p-curve. The simple effect from the high anxious group is included in p-curves of negative vigilance with within-subjects effects prioritized [see Figure 4A and Figure 6A]. However, the simple effect for the low anxious group (i.e. within-subjects avoidance of negative stimuli) was not significant, t(29) = 1.89, p = .069, so the reversing interaction was not selectively reported on the basis of both simple effects being significant.
Mogg, Bradley, & Williams (1995)	(1) Attentional bias in anxious subjects (a) Do anxious subjects shift attention towards the spatial location of negative information, in comparison with normal controls (MacLeod et al., 1986; Williams et al., 1988)? (b) Is the attentional bias in anxious subjects specific to anxiety-relevant information rather than negative information in general? (c) Does the attentional bias in anxious subjects depend on conscious awareness of the stimuli? (2) Attentional bias in depressed subjects (a) Do depressed subjects, compared to normal control subjects, selectively attend to negative information (Beck et al., 1979; Bower, 1981)? (b) Is there an attentional bias in depressed subjects which is specific to depression-relevant information? (c) Does an attentional bias in depressed subjects depend on awareness of the stimuli? (3) Attentional bias in anxious vs. depressed subjects: Do anxious and depressed subjects show a different pattern of attentional bias? (4) Attentional bias for positive words: Do anxious or depressed subjects, in comparison with normal controls, show an attentional bias for positive words?	3 × 2 × 2 ANOVA of bias scores on negative-neutral trials with group (anxious, depressed, control) as a between-subjects variable and exposure (subliminal, supraliminal) and type of negative word (anxiety-relevant, depression-relevant as within-subjects variables; attentional bias for positive-neutral word pairs was examined separately and results were non-significant	Difference of means (mean negative word bias for depressed vs. control groups); Difference of means (depressed group mean negative word bias score vs. hypothetical mean of zero)	The bias scores of the depressed group for supraliminal negative words were significantly greater than zero (t(16) = 2.99	There was a significant main effect of group F(1, 30) = 9.40, p < .01; the depressed group showed greater overall vigilance for negative words than the control group (see Fig. 1b).	*N/A	Original authors’ stated hypotheses were exploratory in nature, and the relevant comparisons involve the effect of a single group as opposed to the specificity between anxiety and depression central to the hypotheses.
Mogg, Philippot, & Bradley (2004)	We predicted that individuals with clinical social phobia would show initial vigilance for angry faces at the shorter stimulus duration (500 ms) but not necessarily at the longer stimulus duration (1,250 ms) where they may show avoidance of threat.	2 × 2 × 2 ANOVA of bias scores with group (social phobia, control) as a between-subjects variable and emotional face type (angry, happy) and exposure duration (500 ms, 1250 ms) as within-subjects variables	Difference of means (social phobia group mean angry bias score vs. hypothetical mean of zero)	A contrast of their bias scores against a value of zero confirmed that the social anxiety group was more vigilant for angry faces, relative to neutral faces, t(13) = 2.29, p < .05).	*Not reported	*Non-significant in the social phobia group	Original authors’ stated hypotheses predicted a simple effect at the 500 ms duration (i.e., that participants with social phobia would show bias toward threat), and hypotheses about the 1,250 ms duration were exploratory/comparative in nature (e.g., anxious participants would not “necessarily” show vigilance, and they “may” show avoidance).
Oehlberg, Revelle, & Mineka (Study 1; 2012)	According to a general factor model for threat biases, anxiety, dysphoria, and general negative affectivity will all be associated with biased attention toward angry faces at the shorter (300 ms, 500 ms) but not the longer (1,250 ms) exposure durations. By contrast, the disorder model predicts that these threat biases will be associated primarily with anxiety. In this model, anxiety should significantly predict attention to threat over and above either general negative affect or dysphoria. Finally, in the specificity model, attention to threat at 300 and 500 ms would be positively associated with anxiety but negatively associated with dysphoria, after accounting for their covariance. Based on prior research using the dot-probe task, we also predicted that biases away from happy faces would be positively associated with dysphoria, but not anxiety.	Correlations and regression analyses with threat bias score as the dependent variable and symptoms of depression, symptoms of anxiety, and general negative affect as predictor variables	Correlation between composite anxiety index and angry bias scores	*Not reported	*Not reported	As predicted by this model, early attentional biases for angry faces were positively correlated with cNA, cAnx, and cDep at 300 ms (see Table 1). *Reported in a table: correlation between composite anxiety score and attentional bias for threatening stimuli, r = .27, n = 64	Original authors’ stated hypotheses were exploratory in nature, as they sought to evaluate whether attention to threat is predicted by each of three possible models.
Oehlberg, Revelle, & Mineka (Study 2; 2012)	Based on our findings in Study 1, we predicted the following: 1. Biases toward angry faces at 300 ms will follow the general factor model, being predicted equally well by cNA, cAnx, and cDep. Neither cAnx nor cDep will predict additional variance over and above that predicted by cNA. 2. Biases toward sad faces at 1,000 ms will be specific to individual differences in cDep. 3. Biases for positive stimuli will not be associated with cNA, cAnx, or cDep.	Correlation and regression analyses with threat bias score as the dependent variable and symptoms of depression, symptoms of anxiety, and general negative affect as predictors	Correlation between composite negative affectivity and angry bias scores	*Not reported	*Not reported	As predicted by the general factor model and results of Study 1, attentional biases for angry faces at 300 ms were positively correlated with cNA, cAnx, and cDep (see Table 2). *Reported in a table: correlation between composite negative affect score and attentional bias for threatening stimuli, r = .28, n = 166	Original authors’ stated hypotheses include (in hypothesis 1) simple effect of symptoms of negative affect on bias toward angry faces. That is, bias toward threat at 300 ms is stated as a hypothesis that is distinct from predictions at 1,000 ms. Thus, Hypothesis 1 would be supported if symptoms of negative affect predicted threat bias at 300 ms, regardless of whether or not Hypotheses 2 and 3 received support.
Pishyar, Harris, & Menzies (Cognitive Behavioral Group Therapy condition at Time 1; 2008).	It was predicted that there would be no difference between the CBGT and WLC groups on self-report and attentional bias measures on the first measurement occasion, but that after therapy the CBGT group only would have reduced anxiety and depression scores and altered responding on the measures of attentional bias. Consistent with Musa et al. (2003), Mogg et al. (2004) and Pishyar et al. (2004) it was predicted that a tendency to preferentially direct attention towards threatening words and faces would be evident on the dot-probe task at Time 1 for both the CBGT and WLC groups. This vigilance towards threatening words and faces would be diminished at Time 2 for the CBGT group only. … It would be expected that both groups would preferentially attend to words that suggested body sensations at Time 1, but at Time 2, only the WLC would preferentially attend to body-sensation words.	2 × 2 × 2 ANOVA of bias scores with group (CBGT, WLC) as a between-subjects factor and emotional facial expression (threatening, happy) and time of test (pre-treatment, post-treatment) as within-subjects factors	Difference of means (CBGT group mean social threat bias score at baseline vs. hypothetical mean of zero)	*Calculated based on bias score and standard deviation reported in a table: CBGT pre-treatment bias towards threatening faces t(15) = 3.79	*Not conducted because, although this study contained a comparison group, it did not include a healthy/low-anxious group	*Not reported	Original authors’ stated hypotheses included a simple effect at Time 1 (prediction that both groups of participants with social phobia would show bias toward threat).
Pishyar, Harris, & Menzies (Wait List Control condition at Time 1; 2008).	*See above	*See above	Difference of means (WLC group mean social threat bias score at baseline vs. hypothetical mean of zero)	*Calculated based on bias score and standard deviation reported in a table: WLC pre-treatment bias towards threatening faces, t(15) = 2.95	*Not conducted because, although this study contained a comparison group, it did not include a healthy/low-anxious group group	*Not reported	Original authors’ stated hypotheses included a main effect at Time 1 (prediction that all participants with social phobia would show bias toward threat).
Reinecke, Cooper, Favaron, Massey-Chase, & Harmer (2011)	The aim of this study was to assess emotional information processing in a less severely affected community-based sample of unmedicated and untreated participants with PD-specific panic attacks. We were interested in studying whether we would find similar cognitive biases as in commonly investigated outpatient or inpatient samples.	2 × 2 ANOVA with group (PD, controls) as a between-subjects factor and emotional face type (fearful, happy) as a within-subjects factor	Difference of means (PD group mean fearful bias vs. a hypothetical mean of zero); difference of means (mean fearful bias for PD vs. NC group)	PPD t0 fear(22) = 2.72	Independent t-test fear: t(43)=2.26	*Non-significant	Original authors’ stated hypotheses specified within-subject and between-subject simple negative bias effects, and this information was not selectively reported.
Reinecke, Waldenmaier, Cooper, & Harmer (Waitlist group; 2013)	It is usually assumed that CBT primarily targets more explicit and deliberate cognitive beliefs and control processes rather than automatic processes. Cognitive behavioral therapy would therefore be expected to reduce automatic threat processing only over time and only with repeated practice and learning. We have tested this hypothesis in panic disorder (PD), as a paradigm treatment target, with an acute-dose CBT paradigm.	2 × 2 ANOVA of bias scores with treatment group (CBT vs. Waitlist) as a between-subjects variable and emotional face type (fearful, happy) as a within-subjects variable; One sample t-tests comparing happy bias and fearful bias for each group to a hypothetical mean of zero	Difference of means (Waitlist group mean fearful bias vs. hypothetical mean of zero)	A significant vigilance effect for fearful faces was found in WG patients [t0(13) = 2.75, p = .02].	*Not conducted because this study did not include healthy/non-disordered controls	*Not reported	Original authors’ stated hypotheses did not include predictions about bias at baseline.
Shane & Peterson (Study 2; 2007).	Our predictions were similar to study one, in that we predicted the dysphoric group to manifest increased processing of the depression-specific words. In addition, based on the results of study one, we anticipated that the dysphoric group may also show increased avoidance of positive stimuli.	2 × 2 × 2 ANOVA of bias scores with dysphoria (high, low) as a between-subjects variable and emotional stimulus valence (positive, negative) and exposure duration (200 ms, 1500 ms) as within-subjects variables	Difference of means (dysphoric group mean threat bias score vs. hypothetical mean of zero)	Second, the dysphoric group showed significant vigilance toward dysphoric-specific stimuli on long-duration trials, t(25) = 2.37, d = .47, p = .03, which differed from the non-dysphoric group …	Second, the dysphoric group showed significant vigilance toward dysphoric-specific stimuli on long-duration trials … which differed from the non-dysphoric group, t(63) = −2.36, d = .61, p = .02.	*N/A	Original authors’ stated hypotheses included a simple effect of bias toward depression words in the dysphoric group as part of a reversing interaction.
Schrooten, Smulders, Mogg, & Bradley (2012)	It was hypothesized that, with 500 ms exposure, the high-anxious group would show a content-specific bias in orienting to threat-relevant words, relative to the low-anxious … That is, the high-anxious group would respond faster when threat-relevant word and probe spatially correspond than when they do not.	2 × 3 × 2 × 2 ANOVA of bias scores with group (HA, LA) as a between-subjects variable and emotional word type (physical threat, social threat, positive), duration (14 ms, 500 ms), and block (first, second), as within-subjects variables	Difference of means (mean physical threat bias for HA vs. LA group); Difference of means (HA group mean physical threat bias score vs. hypothetical mean of zero)	*Calculated using mean physical threat bias score and SE for HA group, t(29) = 2.09	For physical-threat words, the high-anxious group showed a more positive visual orienting score (M = 6.07, SE = 2.9) than the low-anxious group (M = −3.25, SE = 2.5), t(59) = 2.5; *Independent-samples t-test recalculated using means and standard errors, t(59) = 2.44	*Not reported	Original authors’ stated hypotheses include simple effects (i.e., between-subjects and within-subjects bias for threat in the HA group). There was no hypothesis about a difference between physical and social threat biases, and thus it is unlikely that selective reporting occurred on the basis of a difference between these two bias scores.
Stevens, Rist, & Gerlach (2009)	We assumed that the short presentation time is particularly suited to measure initial attentional vigilance. Threat-related stimuli should be processed preferentially by sober social phobia patients, as evidenced by faster reaction time (RT) to probes replacing angry expressions than probes replacing neutral faces (see Mogg et al., 2004). We expected that in individuals with social phobia, alcohol would reduce this bias in the short (175 ms) condition, because a detailed elaboration of the stimuli is impeded. As in the study of Mogg et al. (2004), we did not expect any initial attentional bias for the healthy control subjects during either alcohol or control conditions.	2 × 2 ANOVA of bias scores for threat faces in each exposure duration with beverage (orange juice, alcohol) and group (social phobic, control) as between-subjects variables; planned comparison of bias scores for the two groups (social phobic, controls) in the orange juice condition at the 175 ms duration	Difference of means (mean angry bias for social phobia vs. control groups); Difference of means (social phobia group mean angry bias score vs. hypothetical mean of zero)	When RT in the unambiguously angry face condition was compared separately for groups, only social phobia patients responded faster to probes replacing unambiguously angry faces instead of neutral faces (F(1, 19) = 15.10, p < .01).	Social phobia patients and controls differed significantly in their reaction times when the probe and an unambiguously angry expression appeared in the same location, compared to the condition when probe and the face appeared in different locations F(1, 39) = 6.81	*N/A	Original authors’ stated hypotheses include simple effect (i.e., within-subjects bias for threat in the social phobia group) and the attenuated interaction involving alcohol is unlikely to result in publication bias regarding the simple effect.
Tran, Lamplmayr, Pintzinger, & Pfabigan (2013)	We explored attentional biases in the dot probe task towards facial emotional expressions of anger, disgust, fear, happiness, and sadness in a large community sample, initially unscreened for anxiety, but evenly distributed with regard to sex and age. Main focuses of our study lay at examining whether attentional biases are similar among men and women, and to what extent threat-related attentional biases towards facial stimuli are anxiety-specific, i.e., confined to high-anxious subjects. Thus, our study served on the one hand as a direct test of the validity of the EGA in research on attentional biases. Given the broad range of investigated emotional expressions, our study served on the other hand also as a test of the emotionality hypothesis. We expected that men and women differ with regard to attentional biases towards facial stimuli; i.e., only women show a threat-related attentional bias.	Censored-inflated regression analyses of relationship between bias scores and symptoms of anxiety, controlling for symptoms of depression, age, sex; 2 × 3 ANOVA of bias scores for each emotional face type (angry, happy) with anxiety group (high, intermediate, low) and sex (male, female) as between-subjects variables	Difference of means (mean angry bias for high anxious females vs. low anxious females); Difference of means (high anxious females’ mean angry bias score vs. hypothetical mean of zero)	*Calculated based on bias scores and standard deviations reported in a table; attentional bias toward angry faces for high-anxious female participants, t(49) = 2.15	*Calculated based on bias scores and standard deviations reported in a table: difference in bias scores between high anxious and low anxious female participants, t(119) = 2.49	*N/A	Original authors’ stated hypotheses were exploratory in nature (centered around emotionality and EGA methodology), and also included that only women would evidence threat biases.

Note. Where hypotheses were not explicitly stated in an original paper, the clearest available statement of the purpose of the study was quoted from that paper. Results included in p- curves appear in bold. In columns labeled “Quoted text from original paper,” asterisk denotes comments by the current authors. Results are listed as “not reported” if original authors did not conduct or did not report results of relevant analyses; results are listed as “N/A” if relevant analyses were conducted and reported by original authors, but no p-values from these analyses were included in p-curves due to selection rules of the present meta-analysis.

For the first p-curve of negative bias effects from these studies, we prioritized p-values for significant vigilance to negative stimuli within high-symptom groups. In the absence of significant within-subjects findings, we included p-values for significant differences in negative vigilance when comparing particiants with high vs. low symptom severity. Finally, in the absence of significant within- and between-subjects negative vigilance effects, we included p-values for significant positive correlations between symptom severity and negative vigilance (Appendix D, Table D3). Though the resulting distribution of 10 independent p-values (Figure 5A) appears to contain more p-values ≤ .01 than one would expect to see in the absence of a real effect, the right-skew test yielding chi-square values of χ²(20) = 31.05, p = .0545, thus providing only promise of evidential value, due to marginal significance. Also, the test for lack of evidential value, χ²(20) = 24.91, p = .2048, and the left-skew test, χ²(20) = 17.14, p = .6441,were non-significant. Although this outcome yields promise of overall evidential value, this p-curve was unable to conclusively determine the evidential value of the studied effect based on these nine p-values.

Figure 5.

P-curves of findings showing vigilance toward negative information from studies that also showed avoidance of positive information. Within-subjects effects were given priority for inclusion in the p-curve on the left (A), followed by between-subjects effects, followed by correlations. For the p-curve on the right (B), between-subjects effects were given priority for inclusion, followed by within-subjects effects, followed by correlations. Note. Blue line represents distribution of p-values.

For the next p-curve, we prioritized between-subjects differences in negative bias, followed by within-subjects biases toward negative information in the clinical or high-symptom group, followed by correlations (Appendix D, Table D4). This distribution of 10 p-values (Figure 5B) was tested for right skew, and the result was significant, χ²(20) = 45.46, p = .001, indicating that at least some of the findings included in this curve have evidential value. Neither the test for lack of evidential value, χ²(20) = 15.37, p = .7549, nor the test for left skew, χ²(20) = 8.76, p = .9856, approached significance.

Negative bias findings from studies not showing avoidance of positive information

Twenty-one studies that met inclusion criteria yielded 21 independent p-values showing vigilance toward negative information in participants with depressive or anxious symptoms, but did not show significant biases away from positive information. We calculated two p-curves of these findings. Details about the studies and specific p-values included in these p-curves are presented in Table 2–4 and in Appendix C, Table C2⁴. As before, the first p-curve was calculated prioritizing within-subjects biases toward negative information in the depressed or anxious group, followed by between-subjects biases toward negative information in the depressed or anxious group, followed by correlations (Appendix D, Table D5). The resulting distribution of 21 independent p-values (Figure 6A) was tested for right skew and the result was significant, χ²(42) = 69.94, p = .0044, indicating that at least some of the findings included in this curve have evidential value. Neither the test for lack of evidential value nor the test for left skew were significant, yielding chi-square values of χ²(42) = 56.44, p = .0675, and χ²(42) = 39.58, p = .5776, respectively.

Figure 6.

P-curves of findings showing vigilance toward negative information from studies that did not show avoidance of positive information. Within-subjects effects were given priority for inclusion in the p-curve on the left (A), followed by between-subjects effects, followed by correlations. For the p-curve on the right (B), between-subjects effects were given priority for inclusion, followed by within-subjects effects, followed by correlations. Note. Blue line represents distribution of p-values.

In computing the second p-curve for this set of studies, we prioritized between-subjects differences in negative bias (Appendix D, Table D6). The distribution of p-values (Figure 6B) was tested for right skew and again the result was significant, χ²(42) = 72.64, p = .0023, indicating that at least some of the findings included in this curve have evidential value. Neither the test for lack of evidential value nor the test for left skew approached significance, yielding chi-square values of χ²(42) = 44.96, p = .3489 and χ²(42) = 28.37, p = .9466, respectively.

Discussion of p-curves for vigilance toward negative information

The purpose of computing separate p-curves of negative bias findings from studies that showed significant avoidance of positive information and those that did not was to compare the two sets of findings for evidential value. The results did not suggest differences in the evidential value of negative bias findings from these two sets of studies. However, both of the negative bias p-curves prioritizing within-subjects effects appeared to follow a bimodal distribution with one peak at p ≤ .01 and a second peak at .03 ≤ p ≤ .04. In addition, these two p-curves contained a greater proportion of significant p-values between .04 and .05 than would be expected assuming even a small true effect (i.e., compared to the curve for a true effect at 33% power; Simonsohn et al., 2014). The bimodal shape of these p-curves suggests that some of the findings in each curve have evidential value, whereas others likely are the result of Type I errors (U. Simonsohn, personal communication, August 6, 2013).

Comparison of Dot-probe Findings from Clinical and Non-Clinical Populations

The p-curves described thus far indicate that both avoidance of positive information and vigilance toward negative information are real effects that occur in individuals with symptoms of anxiety or depression. However, research indicates that the dot-probe task has low reliability when used to measure attentional biases in non-clinical populations (Schmukle, 2005). We viewed the distinction between clinical and non-clinical groups as a stand-in for symptom severity, and thus expected that clinical populations (i.e., individuals with more severe symptoms of anxiety or depression) would tend to exhibit more dramatic avoidance of positive information and vigilance toward negative information than non-clinical populations (i.e., individuals with less severe symptoms). In other words, the effect sizes may be larger—and thus the dot-probe task may be a more powerful instrument—with clinical populations.

We thus wished to examine whether the use of a non-clinical sample alters the evidential value of dot-probe findings. To do this, we sorted the studies that had been included in our p-curves into two groups: those that included samples from clinical populations and those that did not. For each group of studies we computed new p-curves of findings showing avoidance of positive information and vigilance toward negative information.

Negative bias findings with clinical populations

Our selection rules identified 13 independent significant findings showing vigilance toward negative information in 12 studies of clinical populations. These studies involved groups of participants who were evaluated at the time of the study and met criteria for a diagnosis of one or more of the following psychological disorders: Major Depressive Disorder (MDD), Generalized Anxiety Disorder (GAD), Panic Disorder (PD) and Social Anxiety Disorder (SAD). The first p-curve of these findings was calculated prioritizing within-subjects biases toward negative information in the clinical group, followed by between-subjects differences in negative bias scores, followed by correlations (Appendix D, Table D7). The resulting distribution of 13 independent p-values (Figure 7A) was tested for right skew, and the result was significant, χ²(26) = 57.72, p = .0003, whereas neither the test for lack of evidential value, χ²(26) = 15.09, p = .9557, nor the test for left skew, χ²(26) = 8.19, p = .9997, approached significance.

Figure 7.

P-curves of findings showing vigilance for negative information from studies with clinical populations. Within-subjects effects were given priority for inclusion in the p-curve on the left (A), followed by between-subjects effects, followed by correlations. For the p-curve on the right (B), between-subjects effects were given priority, followed by within-subjects effects, followed by correlations. Note. Blue line represents distribution of p-values.

In computing the second p-curve of negative bias findings in clinical populations, we prioritized between-subjects differences (Appendix D, Table D8). The distribution of 13 p-values (Figure 7B) was tested for right skew and again the result was significant, χ²(26) = 60.65, p = .0001. Neither the test for lack of evidential value nor the test for left skew approached significance, yielding chi-square values of χ²(26) = 18.75, p = .8469 and χ²(26) = 10.86, p = .996, respectively. Thus, both of these p-curves provide very strong support for the evidential value of dot-probe findings showing vigilance toward negative information in clinical populations.

Negative bias findings with non-clinical populations

Our selection rules identified 17 studies showing vigilance toward negative information with non-clinical populations. These studies did not use diagnostic evaluations or formal psychological diagnoses as inclusion criteria for any group of participants. The first p-curve of negative vigilance findings from these studies was calculated prioritizing within-subjects biases toward negative information in the high-symptom group, followed by between-subjects differences, followed by correlations (Appendix D, Table D9). The resulting distribution of 18 independent p-values (Figure 8A) was notably different from those obtained in studies with clinical populations. For this p-curve, the right-skew test was not significant, χ²(36) = 43.27, p = .1888. The test for lack of evidential value was significant, χ²(36) = 66.26, p = .0016, indicating that selective reporting could not be ruled out as an explanation for this set of findings. The left-skew test, which evaluates whether there is substantial evidence of selective reporting, was not significant, χ²(36) = 48.53, p = .0793.

Figure 8.

P-curves of findings showing vigilance for negative information from studies with non-clinical populations. Within-subjects effects were given priority for inclusion in the p-curve on the left (A), followed by between-subjects effects, followed by correlations. For the p-curve on the right (B), between-subjects effects were given priority, followed by within-subjects effects, followed by correlations. Note. Blue line represents distribution of p-values.

A second p-curve was computed for this set of studies, this time prioritizing between-subjects differences in negative bias, followed by within-subjects bias toward negative information in the high-symptom group, followed by correlations (Appendix D, Table D10). For this p-curve (Figure 8B) the right-skew test was significant, χ²(36) = 57.46, p =.013, whereas neither the test for lack of evidential value nor the left-skew test were significant, yielding chi-square values of χ²(36) = 41.58, p = .2406, and χ²(36) = 26.26, p = .8831, respectively. These results indicate that at least some of the findings included in this curve have evidential value, and suggest that between-subject findings of vigilance for negative information are more robust than within-subject findings in non-clinical populations.

Positive bias findings with clinical populations

Ten findings showing avoidance of positive information in relation to elevated symptoms came from studies with clinical populations. The p-curve prioritizing within-subjects bias in the clinical group (Appendix D, Table D11; Figure 9A) and the p-curve prioritizing between-subjects differences in positive bias (Appendix D, Table D12; Figure 9B) both clearly exhibited right skew that reached significance, yielding chi-square values of χ²(20) = 52.03, p = .0001, and χ²(20) = 60.32, p < .0001, respectively.

Figure 9.

P-curves of findings showing avoidance of positive information from studies with clinical populations. Within-subjects effects were given priority for inclusion in the p-curve on the left (A), followed by between-subjects effects, followed by correlations. For the p-curve on the right (B), between-subjects effects were given priority for inclusion, followed by within-subjects effects, followed by correlations. Note. Blue line represents distribution of p-values.

Neither the test for lack of evidential value nor the left-skew test approached significance for the p-curve prioritizing within-subjects bias, χ²(20) = 14.16, p = .8222 and χ²(20) = 9.09, p = .9818, respectively, nor for the p-curve prioritizing between-subjects differences, χ²(20) = 11.07, p = .9443 and χ²(20) = 7.12, p = .9963, respectively. Overall, these p-curves strongly indicate that dot-probe findings showing avoidance of positive information in clinical populations have evidential value.

Positive bias findings with non-clinical populations

Nine of the findings showing avoidance of positive information came from studies with non-clinical populations. Two p-curves were computed for these findings. For the p-curve prioritizing within-subjects bias (Appendix D, Table D13; Figure 10A), the right-skew test, test for lack of evidential value, and left-skew test produced non-significant chi-square values of χ²(18) = 24.02, p = .1544, χ²(18) = 23.72, p = .1644, and χ²(18) = 16.04, p = .5897, respectively. Similarly, for the p-curve prioritizing between-subjects differences (Appendix D, Table D14; Figure 10B) the right-skew test, test for lack of evidential value, and left-skew test produced non-significant chi-square values of χ²(18) = 26.66, p = .0856, χ²(18) = 23.88, p = .1589, χ²(18) = 16.43, p = .5624, respectively. Thus, p-curve analyses of findings from studies with non-clinical populations showing avoidance of positive information were inconclusive.

Figure 10.

P-curves of findings showing avoidance of positive information from studies with non-clinical populations. Within-subjects effects were given priority for inclusion in the p-curve on the left (A), followed by between-subjects effects, followed by correlations. For the p-curve on the right (B), between-subjects effects were given priority for inclusion, followed by within-subjects effects, followed by correlations. Note. Blue line represents distribution of p-values.

Discussion of P-curve Meta-Analysis

Our first p-curve meta-analytic results yielded evidence that both vigilance toward negative information and avoidance of positive information are real effects that occur in anxious and depressed persons (see Figures 4–6). This pattern of results is in line with previous findings (e.g., Frewen et al., 2008).

Also, our secondary set of meta-analytic findings indicate that the avoidance of positive and vigilance toward negative information in anxious and/or depressed individuals are real and reliable effects that occur in participants whose symptoms are severe enough to meet diagnostic criteria for major depression or an anxiety disorder (see Figure 7 and Figure 9). A novel pattern of findings emerged suggesting that findings of vigilance toward negative information in non-clinical samples may lack evidential value in comparison to similar findings produced with clinical samples. Equivocal evidence emerged from studies examining avoidance of positive information in nonclinical samples. These findings thus add to the novel constellation of findings reported in our conventional meta-analysis.

Limitations

Because the goal of our p-curve meta-analysis diverges somewhat from those of the original paper documenting the technique (Simonsohn et al., 2014), our selection rules occasionally raised ambiguity with regard to what should be included in each curve. Because we are examining the phenomena of vigilance toward negative and avoidance of positive information, we are examining different simple effects that, hypothetically, are part of a reversing interaction among depressed and anxious individuals. Per p-curve guidelines, we examined one side of these simple effects or the other for different curves so as to be able to investigate the evidential value of these simple effects independently. We also examined a number of p-values that emerged without authors’ explicit prediction, including simple effects emerging from within-subject comparisons. Again, this is directly related to our hypothesis that p-values in the dot-probe emerging outside of the researcher’s stated hypothesis would be less likely to be selectively reported. Our disclosure tables outline why we selected the effects, and why they are valid for examining the phenomena of interest.

Reward Devaluation: Predisposing Factors and Proposed Mechanisms of Avoidance of Positivity

Strong support emerged from our conventional meta-analysis that depressed individuals avoid positive stimuli on the dot-probe task, and thus devalue reward (Adams & Dickinson, 1981; Karniol & Miller, 1983). Mixed evidence emerged for anxious individuals, with socially anxious groups evidencing a weak trend toward avoidance, and generally anxious individuals not evidencing a similar effect. In addition, p-curve analyses yielded strong evidence that avoidance of positivity in symptomatic individuals is a real effect.

As noted in our summary of positive-related bias findings in our first meta-analysis, the finding that participants whose main presenting symptom was depression were the only group to robustly demonstrate devaluation of reward suggests that although the avoidance of positivity posited by the reward devaluation framework is likely correct, the hypothesis that this process is equated in depressed and anxious persons may need refinement. Thus, the unpacked discussion of reward devaluation that follows focuses on depressogenic processes that might underlie positive-related avoidance biases.

Importantly, there are no extant proposed mechanisms to synthesize reward devaluation predictions on the DP with prevailing models of threat processing (e.g., Mogg & Bradley, 1998). Furthermore, although the prospect that biological factors may underlie impaired reward-processing is certainly plausible (Shane & Peterson, 2007; Pluess & Belsky, 2013), there are also limited proposed mechanisms linking predispositional factors to attentional and cognitive biases away from positive information.

Pluess & Belsky (2013) briefly summarized DP findings in regard to the implications of vantage sensitivity and resistance, but they concluded that research on attentional biases for emotional information in individuals who possess vantage resilient factors were inconclusive. Our findings provide a solution as to why there have been inconsistent findings with negative stimuli (i.e., studies without sufficient power due to a lack of psychopathological symptom severity in their subjects), but also, importantly, provide evidence that positive information-processing biases are robust in depressed and anxious individuals. Thus, we now can expand the theoretical investigation of how vantage resistant individuals avoid positive stimuli.

Our meta-analytic evidence indicates that increased depressive symptom severity is associated with increased vantage resistance to, or avoidance of, positivity. That avoiding or inhibiting positivity would be associated with self-reported lack of positivity and distress (i.e., depression) is underemphasized in the literature, but also is relatively noncontroversial. However, the implications of the findings regarding the capability and function of underlying depressogenic processes are potentially profound. How might a process leading to the avoidance of objectively rewarding information develop, and how might it be reconciled with prevailing threat-processing frameworks?

Suggestive Evidence of Reward Devaluation through Inhibition and Negative Affect

Although an exhaustive review of other literatures is beyond our scope, findings from outside of the dot-probe literature may help to elucidate how depressed individuals come to avoid or inhibit positive information. In five studies establishing the relation between goal pursuit, negative affect, and inhibition, Aarts, Custers, and Holland (2007) differentially manipulated negative affect in congruence with reward-based goals. Findings suggested that when individuals are motivated by a goal for reward (even a nonconscious one), and are then quickly primed with negative affect, the desire to complete the goal is greatly reduced. Using modified evaluative conditioning paradigms pairing positive goals and negative affective stimuli, Aarts et al. (2007) found that participants repeatedly inhibited their desires for and behaviors toward goals, such that they were far less likely to complete goals associated with negativity. Thus, the association of positivity with negativity inhibits reward seeking.

Evidence from experimental manipulations of reward value in combination with forced behavioral inhibition provides further support for the reward devaluation hypothesis. Frischen et al. (2012) found that inhibition of positive stimuli results from pairing trustworthy faces with loss-associated patterns on a series of Go/No-Go experiments. In four experiments, individuals were made to inhibit their responses to positive stimuli, resulting in associated negative affect over time. Interestingly, these studies used both positive and negative stimuli (i.e., trustworthy and untrustworthy faces), and found that all ratings became increasingly negative, as opposed to increasingly neutral when paired with behavioral inhibition over time. Thus, inhibition results from devaluation of positive stimuli, not merely loss of affective valence of emotional stimuli.

Lastly, a series of studies by Veling and colleagues provide perhaps the most germane evidence of reward devaluation in relation to our meta-analytic evidence. Veling & Aarts (2009) found that individuals who were made thirsty and then made to repeatedly stop (via a Go/No Go paradigm) instead of quenching their thirst developed a reduced valuation of water. These biases were developed with little if any conscious thought. In addition, Veling, Aarts, & Stroebe (2011) manipulated motor inhibition of reward by pairing fearful facial expressions with rewarding stimuli (i.e., palatable foods). This slowed participant responses to subsequent probes, i.e., it caused an inhibitory tendency, similar to what was found in depressed individuals in our review. Interestingly, inhibition only emerged in those who were initially sensitive to the reward, emphasizing again a process resulting in devaluation of positivity, not solely lack of valuation. As emphasized in the reward devaluation hypothesis, individuals who initially valued the environmental reward ended up showing the greatest devaluation when the initially-rewarding stimulus was paired with fearful faces and inhibitory behaviors.

Thus, a range of evidence suggests that the incongruency of reward and threat pairings results in reward devaluation, not merely an initial lack of reward valuation. Moreover, the more positive or meaningful the initial reward was to the perceiver, the more likely that the reward will ultimately be inhibited when paired with negativity. This experimental evidence of reward devaluation dovetails with our DP meta-analytic findings, and further suggests that the mechanisms underlying the avoidance of positive information are inhibitory in nature. Whether or not these findings are related to the phenomenon of avoidance of positive stimuli in the dot probe, however, remains an empirical question and an interesting topic for future research.

The Attentional Timeframe of Reward-Processing Structures and Processes

Drawing upon the aforementioned studies and our meta-analytic findings, we here provide a close examination of the timeframe of avoidance of positive information via the DP to begin the expanded model of reward devaluation that we will present below. Avoidant DP findings have generally emerged at 500 ms durations, followed by 500–700 ms response times, most likely placing them after engagement (30–1000 ms) or disengagement (500–1000 ms) stages of attention (Posner, 1980). This follows theory and evidence that avoidant processes occur after an initial engagement (Derakshan, Eysenck, & Myers, 2007). However, motivational tendency models commonly operationalize these processes as related only to avoidance of threat (Elliot, 2008). Our findings suggest that besides threat assessment (occurring within the first 500 ms of processing), reward is also assessed, either contemporaneously, via parallel processes, or after the initial interpretation and validation of prospective threat, via a serial process. This valuation appraisal (e.g., Cervone, 2004) likely occurs via a psychological reward processing structure assessing situationally-advantageous or rewarding information, which submits the stimulus to a similar evaluative process as is posited in threat processing structure models (Mogg & Bradley, 2005). Reward processing structures differ from threat processing structures, however, because they may be shut down by the prospect, or contemporaneous processing, of threat. This inhibition of positivity would likely not cause a large inhibitory effect if only activated once (see Frischen, Exp. 4), but would lead to longstanding association of positivity with either ultimate negative outcome or contemporaneous negative affect, and result in positive stimuli ultimately being devalued over time.

As such it is not that positive information is no longer meaningful to the person, but instead that it is meaningfully inhibited. Ergo, when a word such as hope is processed, there is an initial ingrained motivational facilitatory tendency to approach this prospective reward, then a learned association that activates an incongruent inhibitory tendency to avoid, which overwhelms approach tendencies and results in ultimate avoidance (i.e., devaluation) of reward. Thus, information that activates both threat and reward processing structures and therefore an incongruent system of processes ultimately results in facilitation of threat, but inhibition of reward, such that objectively rewarding stimuli is ultimately processed as less rewarding than even objectively neutral stimuli. Importantly, this overarching process would develop over time; cross-sectional data would yield evidence of either even-handedness or reward devaluation depending on how active the inhibitory tendency had become.

Clinical Implications

Although future research is needed to empirically evaluate the translational value of the reward devaluation hypothesis, the current findings and our theoretical treatment may ultimately have significant clinical implications. Recent work outside of the attentional bias literature investigating deficits in reward anticipation in depressed individuals (McMakin, Siegle, & Shirk, 2011; Olino et al., 2011) has highlighted inability to value reward due to prospective threat (Gilbert, 2012) and a lack of stimulation from and savoring of positive affect. For example, the Positive Affect Stimulation and Sustainment (PASS) intervention model has produced initially heartening translational results of decreased depression beyond treatment as usual interventions in a developmental trial aimed at increasing positive affect functioning (McMakin et al., 2011). However, this intervention privileges increased and expanded experience of positive affect with the assumption that positive affect is merely being undervalued by depressed individuals due to fear of negative results. These interventions may be improved by incorporating assessments and therapeutic components evaluating and addressing whether positive affect is being devalued because of its positive content due to fear of negative results.

Implications for Methods of Future Research

One possible reason for the lack of theoretical investigation of avoidance of positivity to date may lie in the limitations of the dot-probe, which preclude definitively answering whether avoidance of positive information is occurring. Although highly unlikely in the face of our meta-analytic evidence, it is possible that findings purportedly showing avoidance of positive information are simply indicative of weaker versions of the threat-bias effects that have already been documented. In other words, prospective avoidance of positive information may simply be caused by an attraction toward neutral, but comparatively negative, stimuli (Fox et al., 2001; Frewen et al., 2008; Santesso, Meurat, Hofmann, Mueller, Ratner et al., 2008; Winer et al., 2011).

However, because the DP allows for attention to be directed either toward or away from a stimulus, it is a better fit to assess approach and avoidance than another widely-used measure of attentional bias, the emotional Stroop task (EST). In the EST, emotionally-toned and neutral words are presented one at a time and participants respond by identifying the color of the typeface. Findings from the EST demonstrate that emotional stimuli cause more interference than do neutral stimuli (Williams et al., 1996); however the EST is limited in its ability to assess the type of, or direction in which, interference is occurring. Interference only demonstrates that certain classes of stimuli are affecting the perceiver in some manner. Demonstrations of interference provide limited information as to whether stimuli are causing avoidance or approach (de Ruiter & Brosschot, 1994). This is because approach and avoidance processes both involve exaggerated attentional processing (Posner, 1980), and may look similar in the EST because the participant must attend to a central point.

Thus, the DP has its benefits, but we here note five considerations that can enhance future DP research, and then discuss tasks that could enhance knowledge if they were examined contemporaneously with cognitive biases (see also Price et al., 2014). First, the addition of a separate neutral-neutral (e.g., Donaldson et al., 2007; Koster, Crombez, Verschuere, & De Houwer, 2004) or negative-positive (e.g., Pineles & Mineka, 2005; Stirling, Eley, & Clark, 2006) stimulus pair condition is recommended, as this will allow for a better evaluation of comparative biases. For example, Donaldson et al. (2007) were able to compare negative-neutral, positive-neutral and neutral-neutral scores to assess positive and negative biases. This affords the researcher an extra level of interpretability beyond what is available when using only negative-neutral and positive-neutral conditions.

Second, increased diversity of stimulus onset asynchronies (SOAs) would help determine at what time attenuated responding to positive stimuli occurs. Extant evidence suggests that depressed individuals are biased away from positive words presented for 500 ms durations or above. However, few studies have investigated how stimulus durations between 500 and 1,000 ms might alter this effect.

Third, it would be beneficial for raw latency data to be presented in results sections. In most dot-probe studies, an attentional bias score is calculated by subtracting all congruent (i.e., emotional stimulus left/subsequent dot-probe left; emotional stimulus right/subsequent dot-probe right) probe-stimulus combinations from all incongruent (i.e., emotional stimulus right/subsequent dot-probe left; emotional stimulus left/subsequent dot-probe right) combinations. This is somewhat problematic, as there is reason to believe that avoidance may be more likely to occur from the left (or, possibly, the top, or upper-left, depending on modification of design) stimulus than the right (e.g., Fox, 2002). Distinctions between left and right visual fields are difficult to determine via most DP results sections, however, because tables are presented with means and standard deviations of attention bias scores, but not of raw latency. Thus, raw latency data would allow for a more precise examination of the differences in emotional processing that occur due to visual fields and more discrete analyses of how these differences relate to avoidance.

Our fourth point is a suggestion likely made obvious by our p-curve meta-analytic results. We suggest using either clinical samples or extreme pre-selection techniques to allow for sufficient power to find attentional bias effects.

We lastly suggest the inclusion of multiple cognitive tasks during assessments, if time permits. These can include implicit accuracy measures (e.g., Mogg et al., 1993; Snodgrass & Shevrin, 2006), which allow for a regression of identification accuracy onto attentional bias scores to assess the relationship between accuracy and attention (Greenwald, Klinger, & Schuh, 1995) and tasks indexing working memory (i.e., the affective N-back; Pe, Koval, & Kuppens, 2013). Differential findings on working memory or cognitive control tasks (Gotlib & Joormann, 2010; Joormann & Gotlib, 2010; Kashdan, Weeks, & Savostyanova, 2011; Snyder, 2013) can further discriminate between reward devaluation and even-handedness. For example, difficulty inhibiting irrelevant negative material and/or failure to update positive information, as evidenced by poor performance on those aspects of affective N-back tasks (Gotlib & Joormann, 2010; Pe et al., 2013) likely are associated with subjective distressed states (D’Avanzato, Joormann, Siemer, & Gotlib, 2013). However, working memory deficits may operate independently from reward devaluation as measured by avoidance of positive information on the DP. For example, difficulty inhibiting negative information may involve separate underlying cognitive/affective mechanisms, because it indicates overvaluing of negative information, not devaluing of positive information. Alternatively, poor updating of positive information in working memory may indeed result in reward devaluation (e.g., Veling et al., s2009). Future research comparing results on multiple tasks in the same participants can help answer these questions.

Conclusion

This meta-analytic review examined positive attentional bias findings in depressed and anxious persons resulting from the dot-probe paradigm. We found strong support that depressed individuals exhibit attentional biases away from positive information, and that symptomatic individuals exhibit attentional biases toward negative information. We discussed three overarching frameworks, including threat-processing structures, even-handedness/self-regulation, and reward devaluation, which might explain these findings. Even-handed models, which posit a lack of bias toward positive information in depressed individuals, received some support from our findings. Reward devaluation, which posits an inhibition or avoidance of positive information, also received strong empirical support. In light of these findings, we have provided a further-developed theoretical treatment regarding reward devaluation in depression. The reward processing structure we posited – operating either in parallel with or serially after a threat processing structure – builds a synergistic model to accommodate both DP findings of avoidance of positive information as well as vigilance toward negative information.

Eventually, the theory of reward devaluation and accompanying clear evidence that depressed individuals avoid positive information may have implications regarding the manner in which depression is conceptualized, assessed, and treated, some of the future directions of which we summarized here. Future theoretical and empirical investigation can continue to focus on why and how highly depressed individuals come to devalue reward. These investigations will benefit from further incorporating theoretical advances from social psychology (Koole, 2009; Loersch & Payne, 2011; Swann, 1997), personality and clinical science (Borsboom & Cramer, 2013; Cervone, 2004), cognitive science (MacLeod, 2007, Snodgrass, Bernat, & Shevrin, 2004), and neurodevelopment (Frewen et al., 2008; Pluess & Belsky, 2013), with the goal of producing interactive models of contextualized persons in distress.

Appendix A

Procedure for computing bias scores and variances from RT tables

First, we computed bias scores for each group by subtracting the mean RT for congruent trials (probe replaces the emotional stimulus) from the mean RT for incongruent trials (probe replaces the neutral stimulus). Next, we estimated the SDs of those bias scores using the procedure recommended by Borenstein and colleagues for computing the standard deviation of the mean difference between repeated measures when the correlation between those measures is unknown (2009). Specifically, we used the following formula to compute the SD of a bias score derived from RT table data:

$${SD}_{\mathit{diff}}=\sqrt{{SD}_1^2+{SD}_2^2-2r({SD}_1)({SD}_2)},$$

where SD₁ refers to the standard deviation from the mean RT on incongruent trials, SD₂ refers to the standard deviation from the mean RT on congruent trials, and r refers to the incongruent-congruent correlation. We used an estimate of r = .9 in all cases. To reach this estimate, we examined the average bias scores and standard deviations from DP studies wherein the original authors reported this information for all tested conditions, regardless of whether or not the biases were significant. We then tested different estimates for r by plugging each r value into the formula above and computing bias scores and SD_diff from RT tables. We found that using a value of r = .9 produced SDs that were similar to those reported by original authors, which suggests that .9 is a reasonable estimate of the within-group incongruent-congruent RT correlation for the DP task.

If RT tables reported mean incongruent and congruent RTs separately for emotional stimuli presented in each screen location or for each half of the DP task (for example, Bradley, Mogg, Falla, & Hamilton, 1998), separate bias scores and SD estimates were computed and then averaged to produce a single effect size for each group. Similarly, if the original authors reported separate bias scores or significance tests for emotional stimuli presented in each screen location or for each half of the DP task, the results were averaged to create a single mean effect size for each group.

Formulas used in computing Cohen’s d and Hedge’s g

For between-subjects findings, the following formulas were used in computing Cohen’s d:

$$\begin{array}{c}d=\frac{\mathit{mean}\mathit{bias}\mathit{score}}{{SD}_{\mathit{within}}}\\ {SD}_{\mathit{within}}=\sqrt{\frac{(n_1-1){SD}_1^2+(n_2-1){SD}_2^2}{n_1+n_2-2}}\end{array}$$

where n₁ and SD₁ are, respectively, the sample size and standard deviation of the bias score for the high symptom group and n₂ and SD₂ are, respectively, the sample size and standard deviation of the bias score for the control group. The variance and standard error of Cohen’s d were computed as follows:

$$\begin{array}{c}{V}_d=\frac{n_1+n_2}{n_1{n}_2}+\frac{d^2}{2(n_1+n_2)}\\ {SE}_d=\sqrt{V_d}\end{array}$$

For within-subjects findings, the same formula was used for computing Cohen’s d as was used for between-subjects findings, except that in this case the following formula was used to compute the within-groups standard deviation:

$${SD}_{\mathit{within}}=\frac{{SD}_{\mathit{diff}}}{\sqrt{2(1-r)}}$$

where SD_diff is the standard deviation of the bias score and r is the estimated correlation between RTs for incongruent and congruent trials. We used an estimate of r = .9 in all cases.

The variance and standard error of Cohen’s d were computed as follows:

$$\begin{array}{c}{V}_d=\left(\frac{1}{n}+\frac{d^2}{2n}\right)2(1-r)\\ {SE}_d=\sqrt{V_d}\end{array}$$

To compute Hedge’s g, each effect size Cohen’s d was multiplied by a correction factor, J:

$$\begin{array}{c}J=1-\frac{3}{4df-1}\\ g=J\times d\end{array}$$

where df is the degrees of freedom within, which is n₁ + n₂ − 2 for between-subjects effects and n − 1 for within-subjects effects. The variance and standard error of Hedge’s g were computed as follows:

$$\begin{array}{c}{V}_g=J^2\times V_d\\ {SE}_g=\sqrt{V_g}\end{array}$$

Inclusion of p-values from attenuated and reversing interactions in p-curves

P-curve guidelines state that simple effects should not be entered into p-curves when the original authors hypothesized an attenuated interaction (Simonsohn et al., 2014). This is because publication bias incentivizes statistically significant results on the analyses testing the authors’ hypotheses (Borenstein et al., 2009; Rothstein, Sutton, & Borenstein, 2005). Therefore, if the original authors hypothesized an attenuated interaction, then the analysis along with associated simple effects might go unpublished unless the attenuated interaction is significant, and the published distribution of simple effects in such cases may retain the bias of selective reporting. However, the effects we examined either (a) were the result of hypotheses examining simple, i.e., non-attenuated effects, or (b) were part of multiple and/or exploratory hypotheses. Moreover, (c) p-curve guidelines state that subsets of simple effect p-values from reversing interactions may be included in p-curves, because they are distributed uniformly under the null. The interaction p-curve rules result in some ambiguity, however, due to the fact that many of our positive avoidance p-values do not result from hypotheses. That is, they are often reversal interactive results, but only simple effects are hypothesized or findings are not relevant to stated hypotheses. Thus, we have included simple effect p-values, as noted in our selection rules, and we have created separate p-curves prioritizing between-subject and within-subject findings, to ensure that no subset of simple effects is unduly biased to conclude evidential value is present even when it is not (Simonsohn et al., 2014). In addition, to verify (a), (b), and (c), we examined any prospectively relevant pattern of interactions and simple effects for each study as noted in the column Why p-value(s) selected do not violate p-curve guidelines regarding attenuated or reversing interactions in our disclosure table in Appendix A. Thus, the overall pattern of p-curve findings is shielded from retaining the impact of selective reporting (as further evidenced by the differential pattern of p-curve findings that emerged for positive and negative biases).

Appendix B

Table B1.

Dot probe studies excluded from traditional meta-analysis and reasons for exclusion

Study	Reason for exclusion
Barnes, Coombes, Armstrong, Higgins, & Janelle, 2010	Results not reported separately for each valence category
Brune, Nadolny, Gunturkun, & Wolf, 2013	Relationship between relevant symptoms and bias not measured or not reported
Chen, Ehlers, Clark, & Mansell, 2002	Faces paired with images of household objects
Fani et al., 2013	Relationship between relevant symptoms and bias not measured or not reported
Faunce, Mapledoram, & Job, 2004	Did not include an unambiguously positive category of stimuli
Garner, Mogg, & Bradley, 2006	Dot probe data not reported
Harmer, Charles, McTavish, Favaron, & Cowen, 2012	Results not reported separately for each valence category
Holmes, Bradley, Nielsen, & Mogg, 2009	Relationship between relevant symptoms and bias not measured or not reported
Huang, Berenbaum, & Chow, 2013	Results not reported separately for each valence category
Johnson, Gibb, & McGeary, 2010	Relationship between relevant symptoms and bias not measured or not reported
Johnson, Joormann, & Gotlib, 2007	Data from these participants reported in Gotlib, Kasch, Traill, Joormann, Arnow, & Johnson, 2004, which was included in our meta-analysis
Joormann, Dkane, & Gotlib, 2006	Data from these participants reported in Gotlib, Krasnoperova, Yue, & Joormann, 2004, which was included in our meta-analysis
Krujit, Putman, & Van der Does, 2013	Relationship between relevant symptoms and bias not reported at baseline
Lacreuse, Schatz, Strazzullo, King, & Ready, 2013	Volunteers with elevated clinical symptoms excluded from study
Lautenbacher et al., 2009	Study included children and/or adolescents
Livermore, Sharpe, & McKenzie, 2007	DP results only reported as difference in bias among three groups
Lorenz et al., 2013	Relationship between relevant symptoms and bias not measured or not reported
Mansell, Clark, Ehlers, & Chen, 1999	Faces paired with images of household objects
Pishyar, Harris, & Menzies, 2004	Results not reported separately for each valence category
Schofield, Johnson, Inhoff, & Coles, 2012	Dot probe data not reported
Shapiro & Burchell, 2012	Relationship between relevant symptoms and bias not measured or not reported
Sposari & Rapee, 2007	Faces paired with images of household objects
Rock, Goodwin, & Harmer, 2010	Relationship between relevant symptoms and bias not measured or not reported
Rohner, 2002	Dot probe data not reported; “bias index” computed using eye tracking data
Taylor, Bomyea, Amir, 2011	Relationship between relevant symptoms and bias not reported at baseline (i.e., before attentional training)
Taylor & John, 2004	Dot probe task differed from usual procedure

Appendix C

Table C1.

P-curve Disclosure Table for Avoidance of Positive Information

Study	Quoted Text from Original Paper Stating Hypotheses	Study Design	Finding(s) to be P-curved	Quoted Text from Original Paper Describing Within-Subjects Results	Quoted Text from Original Paper Describing Between-Subjects Results	Quoted Text from Original Paper Describing Correlational Results	Why p-value(s) selected do not violate p-curve guidelines regarding attenuated or reversing interactions
Bradley, Mogg, Falla, & Hamilton (1998)	The main aims of the study were to examine: (1) whether high trait anxious individuals show an initial attentional bias towards threat faces (i.e. in 500msec condition); and (2) whether this predicted attentional bias will be maintained at the longer interval of 1250msec, or whether avoidant strategies may emerge in the latter condition.	2 × 2 × 2 ANOVA of bias scores with trait anxiety (high, low) as a between-subjects variable and exposure duration (500, 1250 ms) and emotional face type (threat, happy) as within-subjects variables	Difference of means (happy bias score for high anxious vs. low anxious participants)	*Non-significant	In the 500msec condition, the high trait anxious group were significantly more vigilant for threat [t(36) = 2.14, P < .05], and avoidant of happy faces [t(36) = 2.58, P < .05], compared with the low trait anxious group.	*N/A	Original authors’ stated hypotheses were exploratory/comparative in nature.
Bradley, Mogg, & Millar (2000)	1. High levels of anxiety (state or trait) will be associated with an attentional bias for threat faces, relative to neutral faces. 2. According to the “general negativity” hypothesis, this bias will be found not only for threat faces, but also for sad faces. 3. Following the “emotionality” hypothesis, the anxiety-related attentional bias will operate for emotional faces in general, including happy faces. 4. High levels of dysphoria will be associated with: (a) vigilance for sad faces, and (b) avoidance of happy faces, as expected from schema and network models	3 × 3 ANOVA of bias scores with BDI group (high, medium, low) as a between-subjects variable and emotional face type as a within-subjects variable	Correlation between happy bias score and symptom severity	*Not reported	*Not reported	BDI also correlated significantly with avoidance of happy faces relative to neutral faces (r = −.28, p < .05).	In hypothesis 4, original authors predict simple effect of dysphoria on attention to happy faces as part of a reversing interaction.
Brosschot, de Ruiter, & Kindt (1999)	We expected that interference by threat words would be higher in REP subjects than in HA subjects and higher in the latter than in TLA subjects. Secondly, we expected that REP subjects would shift their attention away from social threat words in the VPT in comparison to physical threat words and, additionally, to positive words, while HA subjects were expected to shift their attention toward social threat words. Thirdly, we thought TLA subjects would show no selective bias in either task … As an additional check, we also used positively valenced social words in both tasks, in addition to general positive words, in order to investigate whether the expected biases for social words pertain to the semantic category of general social information, instead of to the specific threatening content	4 × 2 × 2 × 4 ANOVA of bias scores with group (repressors, high defensive/high anxious, high anxious, truly low anxious), state anxiety (high, low), and order of tasks (dot-probe first, Stroop first) as between-subjects variables and emotional word category (social negative, social positive, physical negative, general negative) as a within-subjects variable	Difference of means (social positive bias score for high trait anxious vs. truly low trait anxious participants)	*Non-significant	*Calculated based on bias scores and standard deviations reported in a table: t(19) = 2.60	*Not reported	Original authors did not make predictions about attentional bias to positive stimuli in HA group, with the exception of predictions for the REP group; stated hypotheses regarding attentional bias to positive stimuli were exploratory in nature.
Donaldson, Lam, & Mathews (2007)	First, we predicted that at baseline, depressed patients would show an attentional bias for negative words presented for 1000 ms but not for 500 ms; that these biases would not occur for positive or neutral words and that control participants would not show any biases at all regardless of exposure duration and word valence. Second, we predicted that the rumination induction would produce an increase in the attentional bias for negative words in depressed patients at 1000 ms, the distraction induction would produce a decrease and there would be no change in attentional responses for control participants. Third, we predicted that the strength of the baseline attentional bias shown by depressed patients at 1000 ms would be proportionally related to levels of trait rumination and inversely related to levels of trait distraction.	2 × 2 × 2 × 3 × 2 ANOVA of bias scores with group (controls, patients) and induction (distraction, rumination) as between-subjects variables and exposure duration (500 ms, 1000 ms), word type (negative, positive, neutral), and time (pre- and post-induction) as within-subjects variables	Difference of means (depressed group mean positive bias score vs. hypothetical mean of zero)	Means and standard deviations were as follows: … positive words M = −1 (−.6) *(p-value calculated based on bias score and standard deviation reported in text)	*Not reported	*Not reported	The original authors hypothesized an attenuated interaction, but one including duration and induction, which was unlikely to result in selective reporting of simple effects (as the DP literature allows for the publication of simple effects with or without an interaction with duration even when multiple durations are part of the design). We thus examined the relevant simple effect, which is solely within-subjects and distinct from the irrelevant hypothesized interaction.
Fritzsche et al. (2010)	In the present study we examined whether formerly depressed individuals and patients with asthma exhibit cognitive biases similar to those observed in currently depressed individuals. In addition, we compared these groups with healthy participants.	4 × 2 ANOVA of bias scores with group (asthma, depressed, remitted depressed, controls) as a between-subjects variable and emotional face type (sad, happy) as a within-subjects variable	Difference of means (happy bias score for MDD vs. NC group)	*Non-significant	Post-hoc tests for happy faces indicated that the NC group showed a higher bias towards happy faces than the MDD group, RMD group and the asthma group (all p < 0.05), which did not differ from each other. *Calculated based on bias scores and standard deviations reported in a table: t(38) = 2.23	*Not reported	Original authors’ stated hypotheses were exploratory in nature.
Fritzsche et al. (2013)	We hypothesized that compared with healthy controls, COPD patients with comorbid depression would show depression-specific biases in information processing, which are comparable to those observed in lung-healthy patients with depression … we hypothesized that depression-like biases already occur in patients with COPD before comorbid depression manifests … we additionally explored the associations between cognitive biases, smoking behaviour and depressive symptoms in patients with COPD.	4 × 2 ANOVA of bias scores with group (MDD, COPD + MDD, COPD, controls) as a between-subjects variable and emotional face type (sad, happy) as a within-subjects variable; One-sample t-tests comparing bias to a hypothetical mean of zero	Difference of means (happy bias score for MDD vs. NC group)	*Non-significant	*Calculated based on mean bias score and standard deviation reported in a table, t(37) = 3.42	*Not reported	Original authors did not make specific predictions about the nature of attentional biases in lung-healthy MDD group; analyses within MDD group and between MDD group and Controls were exploratory in nature.
Gotlib, Kasch et al. (2004)	The present study was designed to address two specific questions. First, do clinically anxious and clinically depressed individuals exhibit specific (and different) schema-congruent patterns of bias in information processing? And second, do information processing biases cohere across different tasks assessing attention and memory functioning? … First, we predicted that MDD, GSP, and NC participants would differ with respect to biases in their processing of emotional stimuli. More specifically, we expected that, compared with the NC participants, both the MDD and the GSP participants would exhibit a negative bias on all tasks. Second, we predicted that we would find evidence of stimulus specificity, such that the MDD and GSP participants would demonstrate differential responses to the different categories of stimuli, with the GSP participants exhibiting stronger biases for socially threatening content stimuli and the MDD participants demonstrating stronger biases for sad-content stimuli.	3 × 3 ANOVA of bias scores with group (depressed, generalized anxiety, controls) as a between-subjects variable and emotional face type (sad, happy, angry) as a within-subjects variable	Correlation between happy bias score and depressive symptom severity	*Non-significant	*Non-significant	As shown in Table 5, bias scores for the happy faces on the emotion face dot-probe task were significantly correlated with severity of current depression: More severely depressed individuals demonstrated a greater bias away from happy faces. *Correlations reported in a table, correlation between happy bias and symptoms of depression, r = −.36, n = 87	Original authors did not make specific predictions about attention to positive stimuli; stated hypothesis focused primarily on between-subjects effect of negativity and specificity of negative emotional stimuli.
Ioannou, Mogg, & Bradley (2004)	(1) High trait anxious individuals with low levels of defensiveness (HA) should show vigilance for threat. Moreover, they were expected to show a higher level of vigilance for threat when compared with either the low anxious or repressor groups (Bradley et al., 1998). (2) Repressors (REP) should show attentional avoidance of threat. (3) Defensive high anxious participants (DHA) should show an attentional bias for threat to at least the same degree as the HA group i.e., vigilance for threat. (4) Low anxious participants (LA) should not show an attentional bias. In addition, the task included trials in which a happy face was paired with a neutral face, in order to examine whether any attentional bias for threat faces was specific to negative stimuli or would also operate for emotional stimuli in general (i.e., for both threat and happy faces).	2 × 2 × 2 ANOVA of bias scores with trait anxiety (high, low) and defensiveness (high, low) as between-subjects variables and exposure duration (500, 1250 ms) and emotional face type (threat, happy) as within-subjects variables	Difference of means (HA group mean happy bias score vs. hypothetical mean of zero)	The results showed that the HA group’s happy bias scores were significantly below zero, (t(10) = 2.65, p < 0.05), indicating that the HA group had an attentional preference for neutral faces relative to happy faces.	*Not reported	*Not reported	Original authors’ stated hypotheses did not include specific predictions about positive stimuli for the HA group; rather, analyses involving positive stimuli for this group were exploratory in nature.
Joormann & Gotlib (2007)	The present study was designed to address this issue by using the dot-probe task to examine whether formerly depressed individuals, as well as currently depressed persons, are characterized by selective attention to sad faces. If negative cognitive biases are not merely symptoms of depression, we expected that both currently and formerly depressed individuals, compared to never-disordered controls, would demonstrate an attentional bias for sad faces on the dot-probe task.	3 × 2 ANOVA of bias scores with group (depressed, remitted depressed, never-depressed controls) as a between-subjects variable and emotional face type (sad, happy) as a within-subjects variable	Difference of means (happy bias score for MDD vs. NC group)	*Non-significant	With respect to the happy faces, the NC participants demonstrated significantly greater vigilance than did the MDD participants, t(43) = 2.07, p < .05	*Not reported	Original authors’ stated hypotheses did not include predictions about positive stimuli; rather, analyses involving positive stimuli were exploratory in nature.
Keogh, Dillon, Georgiou, & Hunt (2001)	It was predicted that those high in anxiety sensitivity would exhibit specific selective attentional bias towards the location of physically threatening words relative to neutral controls. It was also expected that such individuals would not exhibit any such bias towards social threat-related or positive words. Finally, based on previous evidence with trait anxiety (MacLeod & Mathews, 1988) and anxiety sensitive pain patients (Asmundson et al., 1997), it was expected that those low in physical anxiety sensitivity would selectively avoid threatening material.	2 × 3 ANOVA of bias scores with physical anxiety sensitivity group (high, low) as a between-subjects variable and emotional word valence (physical threat, social threat, positive) as a within-subjects variable	Difference of means (positive bias score for high AS vs. low AS participants)	*Not reported	Furthermore, those high in physical anxiety sensitivity exhibited a selective avoidance of positive material, whereas those low in physical anxiety sensitivity seemed to attend towards the location of such material [F(1, 49) = 4.08, p < .05].	*Non-significant	Original authors’ only stated hypothesis regarding positive stimuli was that high AS participants would not show vigilance toward positive stimuli.
Keogh, Ellery, Hunt, & Hannent (2001)	It was specifically predicted that: 1. Individuals high in the fear of pain would selectively attend towards the location of pain-related material, whereas those low in such fears would avoid the location of such material. 2. Such biases would only be found for pain-related material, rather than generally for any negative or emotional material. 3. These effects would be related more to anxiety than depression.	3 × 3 ANOVA of bias scores with fear of pain group (high, medium, low) as the between-subjects variable and emotional word type (physical threat, social threat, positive) as a within-subjects variable	Correlation between positive word bias scores and anxiety subscale of the Depression Anxiety Stress Scale (DASS)	*N/A; participants not grouped by DASS anxiety	*N/A; participants not grouped by DASS anxiety	Both the general stress (r = −0.24, P < 0.05) and the specific anxiety scales of the DASS (r = −0.40, P < 0.001) were found to negatively correlate with the positive bias index.	Original authors did not predict relationship between anxiety and processing of positive stimuli; this analysis was exploratory in nature.
Lindstrom et al. (2011)	In adults exposed to trauma, research reveals positive associations between PTSD severity and attention bias towards threats (Bar-Haim, Lamy et al. 2007). Therefore, we hypothesized that parents with high, relative to low, levels of 9/11-related trauma would show greater attention bias towards threat.	Independent-sample t-tests of bias scores for angry faces and for happy faces, comparing parents with PTSD and parents without PTSD	Difference of means (happy bias score for parents with PTSD vs. parents without PTSD)	*Non-significant	There was a group difference in happy bias based on parents’ PTSD and MDD diagnoses (PTSD: t(43) = 2.42, p = 0.02; happy bias in PTSD cases = −22.99 ms ± 42.66, Non-PTSD cases = 15.27ms ± 31.50 …) *Independent samples t-test recalculated using bias scores, t(43) = 3.03	*Non-significant	Original authors’ stated hypotheses did not include predictions about positive stimuli; rather, analyses involving positive stimuli were exploratory in nature.
Mingtian, Xiongzhao, Jinyao, Shuqiao, & Atchley (2011)	Research questions were as follows: (1) Do depressed patients show negative attentional bias in the dot-probe task? (2) Is attentional bias related to cue presentation time? Based on previous MDD research it is clear that we would predict that the answer to the first of these questions should be “yes, depressed patients should show a negative attentional bias.” … this prediction should be seen via an attentional bias towards negative pictures (i.e. MDD showing faster RTs, greater accuracy, or positive bias scores for negative pictures) or via a lack of attention for positive pictures (i.e., MDD showing slower RTs, lower accuracy, or a lack of positive attention bias score for positive pictures). … though we could see differences in attention allocated to positive vs. negative pictures in the shortened stimulus condition, we expected to see stronger evidence of attentional bias towards negative information and away from positive information in the MDD group at the longer, 500 ms picture presentation condition, than in the shorter presentation condition.	2 × 2 × 2 ANOVA of bias scores for each presentation duration separately, with group (MDD, ND) as a between-subjects variable and emotion word type (positive, negative) as a within-subjects variable	Difference of means (MDD group mean bias score vs. hypothetical mean of zero); difference of means (positive bias score for MDD vs. ND group)	The bias scores on POS–NEU trials were significantly smaller than zero for the MDD group, t(23) = −2.29, p = 0.031, which means that the MDD group failed to attend to positive pictures …	Results, which are illustrated in Fig. 3, show that ND group’s bias scores for POS–NEU trials (9.30 ± 10.33 ms) were significantly larger than those of MDD group’s (−5.50 ± 11.77 ms), t(47) = −4.69, p < 0.0005, which means ND group demonstrated significantly greater attention for positive pictures than did MDD group.	*Not reported	Original authors’ stated research questions do not include attentional biases to positive stimuli. Then, potential hypotheses regarding attentional biases to positive stimuli were phrased broadly and included simple effects (i.e., “MDD showing slower RTs, lower accuracy, or a lack of positive attention bias score for positive pictures”).
Mueller et al. (2009)	If individuals with SAD are hypervigilant to threat (Bar-Haim et al. 2007), we predicted that they would react faster to probes replacing angry faces as opposed to probes replacing neutral faces. Conversely, if socially anxious individuals avoid angry and/or happy faces (Mansell et al. 1999), they should have longer RTs for probes preceded by emotional as opposed to neutral faces.	2 × 2 ANOVA of bias scores with group (SAD, controls) as a between-subjects variable and emotional face valence (angry, happy) as a within-subjects variable	Difference of means (SAD group mean happy bias score vs. hypothetical mean of zero)	Finally, following happy-neutral face-pairs SAD participants reacted faster to probes replacing neutral vs. happy faces [t(11) = 3.30, p < 0.008].	*Non-significant	*Not reported	Original authors’ stated hypotheses were suggestive of a reversing interaction. Thus, according to p-curve guidelines, inclusion of the simple effect (i.e., within-subjects avoidance of positive stimuli in the SAD group) is allowable.
Pishyar, Harris, & Menzies (Cognitive Behavioral Group Therapy condition at Time 1; 2008)	It was predicted that there would be no difference between the CBGT and WLC groups on self-report and attentional bias measures on the first measurement occasion, but that after therapy the CBGT group only would have reduced anxiety and depression scores and altered responding on the measures of attentional bias. Consistent with Musa et al. (2003), Mogg et al. (2004) and Pishyar et al. (2004) it was predicted that a tendency to preferentially direct attention towards threatening words and faces would be evident on the dot-probe task at Time 1 for both the CBGT and WLC groups. This vigilance towards threatening words and faces would be diminished at Time 2 for the CBGT group only. … It would be expected that both groups would preferentially attend to words that suggested body sensations at Time 1, but at Time 2, only the WLC would preferentially attend to body-sensation words.	2 × 2 × 2 ANOVA of bias scores with group (CBGT, WLC) as a between-subjects factor and emotional facial expression (threatening, happy) and time of test (pre-treatment, post-treatment) as within-subjects factors	Difference of means (CBGT group pre-treatment mean happy bias score vs. hypothetical mean of zero)	*Calculated based on bias score and standard deviation reported in a table: CBGT pre-treatment bias away from happy faces, t(15) = 3.77	*Not conducted because this study did not include a healthy/low-anxious group	*Not reported	Original authors’ stated hypotheses did not include predictions about positive stimuli and instead focused on time differences that are irrelevant to the present analysis; analyses involving positive stimuli were exploratory in nature.
Pishyar, Harris, & Menzies (Wait List Control condition at Time 1; 2008)	*See above	*See above	Difference of means (WLC group pre-treatment mean happy bias score at baseline vs. hypothetical mean of zero)	*Calculated based on bias score and standard deviation reported in a table: WLC pre-treatment bias away from happy faces, t(15) = 4.93	*Not conducted because this study did not include a healthy/low-anxious group	*Not reported	Original authors’ stated hypotheses did not include positive stimuli; analyses involving positive stimuli were exploratory in nature.
Shane & Peterson (Study 1; 2007)	Thus, the present set of studies was designed to further investigate the extent to which dysphoria is characterized by hypervigilance to negative information, hypovigilance to positive information, or a combination of both processing biases. … First, we sought to determine whether these biases constitute global biases in attentional processing … or conversely are specific to either early or late stages of processing. … Second, we wanted to evaluate the possibility that the positive and negative biases may exist as independent markers of dysphoria. … First, and most directly, the correlation between the positive and negative biases could be calculated. A low correlation could suggest independence between attention allocated toward positive stimuli and attention allocated toward negative stimuli. In addition, we were particularly interested in the possibility that dysphoric individuals may manifest both positive and negative biases, but that each bias may show a distinct temporal pattern. … A final aim of the present research was to evaluate the predictive value of bias differentials (BD), which were calculated by subtracting participants’ biases to negative information from their biases to positive information.	2 × 2 × 2 ANOVA of bias scores with dysphoria (high, low) as a between-subjects variable and emotional stimulus valence (positive, negative) and exposure duration (500 ms, 1500 ms) as within-subjects variables	Difference of means (dysphoric group mean positive bias score vs. hypothetical mean of zero); difference of means (positive bias score for dysphoric vs. non-dysphoric group)	On short-duration trials, the dysphoric group showed pronounced avoidance of the positive pictures that differed from zero, t(29) = −2.35, d =.42, p = .03	As can be seen, non-dysphoric individuals showed a slight vigilance toward positive stimuli, whereas dysphoric individuals showed a more pronounced avoidance of positive stimuli. Planned comparisons confirmed that these attentional biases differed significantly from each other, t(70) = 2.13, d =/.50, p = .04.	*N/A	Original authors’ stated hypotheses were exploratory in nature though suggestive of a reversing interaction.
Shane & Peterson (Study 2; 2007)	Our predictions were similar to study one, in that we predicted the dysphoric group to manifest increased processing of the depression-specific words. In addition, based on the results of study one, we anticipated that the dysphoric group may also show increased avoidance of positive stimuli.	2 × 2 × 2 ANOVA of bias scores with dysphoria (high, low) as a between-subjects variable and emotional stimulus valence (positive, negative) and exposure duration (200 ms, 1500 ms) as within-subjects variables	Difference of means (dysphoric group mean positive bias score vs. hypothetical mean of zero); difference of means (positive bias score for dysphoric vs. non-dysphoric group)	Non-dysphoric individuals showed a slight, although nonsignificant, vigilance toward the positive stimuli … while dysphoric individuals showed a more pronounced avoidance of the positive stimuli, t(25) = −2.51, d = .55, p = .02.	Planned comparisons confirmed these attentional patterns to differ significantly from one another, t(63) = 3.03, d = .77, p = .004.	*N/A	Original authors’ stated hypotheses included a simple effect, that the dysphoric group would show avoidance of positive stimuli, as part of a reversing interaction. Thus, according to p-curve guidelines, inclusion of the simple effect is allowable.
Taylor, Bomyea, & Amir (2010)	Several lines of evidence led us to hypothesize that diminished attentional allocation toward positive social cues would mediate the link between level of social anxiety and anxiety reactivity to a social stressor. … To the extent that social anxiety is associated with a diminished tendency to orient attention toward positive cues, we predicted that such individuals would be particularly vulnerable to experiencing elevated states of anxiety during a stressful social task.	Mediation analyses via bootstrapping methods	Correlation between positive bias score and symptom severity	*Not reported	*Not reported	*Bivariate correlations reported in table; correlation between social anxiety symptoms and positive attentional bias, r = −.41	Original authors stated hypotheses involved mediational processes, not attenuated or reversing interactions.

Note. Where hypotheses were not explicitly stated in an original paper, the clearest available statement of the purpose of the study was quoted from that paper. Results included in p- curves appear in bold. In columns labeled “Quoted text from original paper,” asterisk denotes comments by the current authors. Results are listed as “not reported” if original authors did not conduct or did not report results of relevant analyses; results are listed as “N/A” if relevant analyses were conducted and reported by original authors, but no p-values from these analyses were included in p-curves due to selection rules of the present meta-analysis

Appendix D

Table D1.

PP-values for p-curve of positive bias findings; within-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Bradley et al. (1998)	t(36)=2.58	.01411	.28215	.71785	.48564
Donaldson et al. (2007)	t(35)=10.00	.00000	.01000	.99000	.99000
Keogh, Ellery et al. (2001)	r(73)=.40	.00038	.00754	.99246	.94256
Lindstrom et al. (2011)	t(43)=3.03	.00413	.08256	.91744	.74750
Mingtian et al. (2011)	t(23)=2.29	.03152	.63049	.36951	.21114
Mueller et al. (2009)	t(11)=3.30	.00708	.14156	.85844	.69839
Pishyar et al. (Cognitive Behavioral Group Therapy condition at Time 1; 2008)	t(15)=3.77	.00185	.03707	.96293	.87317
Pishyar et al. (Waitlist Control condition at Time 1; 2008)	t(15)=4.93	.00018	.00363	.99637	.97839
Shane & Peterson (Study 1; 2007)	t(29)=2.35	.02579	.51581	.48419	.28815
Shane & Peterson (Study 2; 2007)	t(25)=2.51	.01891	.37827	.62173	.40161
Taylor et al. (2010)	r(42)=.41	.00571	.11419	.88581	.69338
Bradley et al. (2000)	r(53)=.28	.03841	.76820	.23180	.12080
Brosschot et al. (1999)	t(19)=2.60	.01759	.35170	.64830	.43261
Fritzsche et al. (2010)	t(38)=2.23	.03173	.63464	.36536	.20336
Fritzsche et al. (2013)	t(37)=3.42	.00154	.03080	.96920	.86643
Gotlib, Kasch et al. (2004)	r(87)=.36	.00053	.01060	.98940	.92698
Ioannou et al. (2004)	t(10)=2.65	.02431	.48626	.51374	.33816
Joormann & Gotlib (2007)	t(43)=2.07	.04449	.88985	.11015	.05523
Keogh, Dillon et al. (2001)	F(1,49)=4.08	.04888	.97758	.02242	.01088

Table D2.

PP-values for p-curve of positive bias findings; between-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Bradley et al. (1998)	t(36)=2.58	.01411	.28215	.71785	.48564
Donaldson et al. (2007)	t(35)=10.00	.00000	.01000	.99000	.99000
Keogh, Ellery et al. (2001)	r(73)=.40	.00038	.00754	.99246	.94256
Lindstrom et al. (2001)	t(43)=3.03	.00413	.08256	.91744	.74750
Mingtian et al. (2011)	t(47)=4.69	.00002	.00048	.99952	.99247
Mueller et al. (2009)	t(11)=3.30	.00708	.14156	.85844	.69839
Pishyar et al. (Cognitive Behavioral Group Therapy condition at Time 1; 2008)	t(15)=3.77	.00185	.03707	.96293	.87317
Pishyar et al. (Waitlist Control condition at Time 1; 2008)	t(15)=4.93	.00018	.00363	.99637	.97839
Shane & Peterson (Study 1; 2007)	t(70)=2.13	.03669	.73377	.26623	.13969
Shane & Peterson (Study 2; 2007)	t(63)=3.03	.00354	.07089	.92911	.76437
Taylor et al. (2010)	r(42)=.41	.00571	.11419	.88581	.69338
Bradley et al. (2000)	r(53)=.28	.03841	.76820	.23180	.12080
Brosschot et al. (1999)	t(19)=2.60	.01759	.35170	.64830	.43261
Fritzsche et al. (2010)	t(38)=2.23	.03173	.63464	.36536	.20336
Fritzsche et al. (2013)	t(37)=3.42	.00154	.03080	.96920	.86643
Gotlib, Kasch et al. (2004)	r(87)=.36	.00053	.01060	.98940	.92698
Ioannou et al. (2004)	t(10)=2.65	.02431	.48626	.51374	.33816
Joormann & Gotlib (2007)	t(43)=2.07	.04449	.88985	.11015	.05523
Keogh, Dillon et al. (2001)	F(1,49)=4.08	.04888	.97758	.02242	.01088

Table D3.

PP-values for p-curve of negative bias findings from studies with significant positive bias findings; within-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Bradley et al. (1998)	t(18)=2.20	.04111	.82217	.17783	.09572
Shane & Peterson (Study 2; 2007)	t(25)=2.37	.02582	.51632	.48368	.28999
Fritzsche et al. (2010)	t(19)=2.37	.02853	.57054	.42946	.25531
Fritzsche et al. (2013)	t(19)=4.44	.00028	.00562	.99438	.96587
Gotlib, Kasch et al. (2004)	t(87)=2.78	.00666	.13323	.86677	.65450
Ioannou et al. (2004)	t(10)=2.25	.04818	.96359	.03641	.01985
Joormann & Gotlib (2007)	t(25)=2.79	.00994	.19880	.80120	.58726
Keogh, Dillon et al. (2001)	F(1,49)=4.61	.03676	.73522	.26478	.14033
Pishyar et al. (Cognitive Behavioral Group Therapy condition at Time 1; 2008)	t(15)=3.79	.00178	.03558	.96442	.87684
Pishyar et al. (Waitlist Control condition at Time 1; 2008)	t(15)=2.95	.00993	.19867	.80133	.60552

Table D4.

PP-values for p-curve of negative bias findings from studies with significant positive bias findings; between-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Bradley et al. (1998)	F(1,36)=5.99	.01940	.38793	.61207	.38706
Shane & Peterson (Study 2; 2007)	t(63)=2.36	.02139	.42770	.57230	.34754
Fritzsche et al. (2010)	t(38)=3.14	.00326	.06528	.93472	.78367
Fritzsche et al. (2013)	t(37)=4.70	.00004	.00071	.99929	.99065
Gotlib, Kasch et al. (2004)	t(141)=2.18	.03092	.61834	.38166	.20806
Ioannou et al. (2004)	t(19)=2.43	.02518	.50361	.49639	.30472
Joormann & Gotlib (2007)	t(43)=3.74	.00054	.01079	.98921	.93200
Keogh, Dillon et al. (2001)	F(1,49)=4.61	.03676	.73522	.26478	.14033
Pishyar et al. (Cognitive Behavioral Group Therapy condition at Time 1; 2008)	t(15)=3.79	.00178	.03558	.96442	.87684
Pishyar et al. (Waitlist Control condition at Time 1; 2008)	t(15)=2.95	.00993	.19867	.80133	.60552

Table D5.

PP-values for p-curve of negative bias findings from studies without significant positive bias findings; within-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Bradley et al. (1997)	F(1,69)=4.29	.04208	.84158	.15842	.07993
Mogg & Bradley (Experiment 3; 1999b)	t(10)=2.37	.03928	.78554	.21446	.12438
Mogg & Bradley (2002)	t(10)=2.68	.02309	.46185	.53815	.35852
Mogg et al. (1994)	t(35)=2.04	.04895	.97906	.02094	.01026
Mogg et al. (1995)	t(16)=2.99	.00866	.17316	.82684	.63564
Mogg et al. (2004)	t(13)=2.29	.03938	.78765	.21235	.11928
Oehlberg et al. (Study 1; 2012)	r(63)=.27	.02962	.59239	.40761	.22774
Oehlberg et al. (Study 2; 2012)	r(165)=.28	.00025	.00495	.99505	.95356
Reinecke et al. (2011)	t(22)=2.72	.01250	.25008	.74992	.53111
Reinecke et al. (2013)	t(13)=2.75	.01654	.33076	.66924	.46621
Schrooten et al. (2012)	t(29)=2.09	.04550	.90994	.09006	.04558
Donaldson et al. (2007)	t(35)=3.70	.00074	.01473	.98527	.91895
Stevens et al. (2009)	F(1,19)=15.10	.00099	.01989	.98011	.91329
Tran et al. (2013)	t(49)=2.15	.03652	.73035	.26965	.14296
Eldar et al. (2010)	t(22)=2.93	.00775	.15504	.84496	.64848
Fox (Experiment 2; 2002)	t(17)=5.7	.00003	.00052	.99948	.99521
Gotlib, Krasnoperova et al. (2004)	t(18)=3.02	.00736	.14716	.85284	.66674
Hunt et al. (Masked condition; 2006)	F(1,103)=8.02	.00557	.11131	.88869	.68739
Klumpp & Amir (2009)	t(38)=2.05	.04731	.94621	.05379	.02657
Miskovic & Schmidt (2012)	t(16)=2.43	.02724	.54481	.45519	.27758
Mogg & Bradley (1999a)	t(18)=2.45	.02474	.49487	.50513	.31266

Table D6.

PP-values for p-curve of negative bias findings from studies without significant positive bias findings; between-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Bradley et al. (1997)	F(1,69)=4.29	.04208	.84158	.15842	.07993
Mogg & Bradley (Experiment 3; 1999b)	t(10)=2.37	.03928	.78554	.21446	.12438
Mogg & Bradley (2002)	F(1,25)=14.17	.00091	.01810	.98190	.91291
Mogg et al. (1994)	F(1,64)=7.34	.00864	.17286	.82714	.60298
Mogg et al. (1995)	F(1,30)=9.40	.00456	.09126	.90874	.73952
Mogg et al. (2004)	t(13)=2.29	.03938	.78765	.21235	.11928
Oehlberg et al. (Study 1; 2012)	r(63)=.27	.02962	.59239	.40761	.22774
Oehlberg et al. (Study 2; 2012)	r(165)=.28	.00025	.00495	.99505	.95356
Reinecke et al. (2011)	t(43)=2.26	.02894	.57884	.42116	.23933
Reinecke et al. (2013)	t(13)=2.75	.01654	.33076	.66924	.46621
Schrooten et al. (2012)	t(59)=2.44	.01771	.35422	.64578	.41029
Donaldson et al. (2007)	t(35)=3.70	.00074	.01473	.98527	.91895
Stevens et al. (2009)	F(1,39)=6.81	.01279	.25582	.74418	.51224
Tran et al. (2013)	t(119)=2.49	.01415	.28309	.71691	.47227
Eldar et al. (2010)	t(22)=2.93	.00775	.15504	.84496	.64848
Fox (Experiment 2; 2002)	t(34)=4.6	.00006	.00113	.99887	.98719
Gotlib, Krasnoperova et al. (2004)	t(33)=2.27	.02987	.59732	.40268	.22920
Hunt et al. (Masked condition; 2006)	F(1,103)=8.02	.00557	.11131	.88869	.68739
Klumpp & Amir (2009)	t(74)=2.14	.03565	.71302	.28698	.15167
Miskovic & Schmidt (2012)	F(1,31)=4.63	.03932	.78639	.21361	.11290
Mogg & Bradley (1999a)	t(36)=2.12	.04097	.81934	.18066	.09354

Table D7.

PP-values for p-curve of negative bias findings in studies using clinical samples; within-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Donaldson et al. (2007)	t(35)=3.70	.00074	.01473	.98527	.91895
Pishyar et al. (Waitlist Control condition at Time 1; 2008)	t(15)=2.95	.00993	.19867	.80133	.60552
Reinecke et al. (2011)	t(22)=2.72	.01250	.25008	.74992	.53111
Reinecke et al. (2013)	t(13)=2.75	.01654	.33076	.66924	.46621
Stevens et al. (2009)	F(1,19)=15.10	.00099	.01989	.98011	.91329
Fritzsche et al. (2010)	t(19)=2.37	.02853	.57054	.42946	.25531
Fritzsche et al. (2013)	t(19)=4.44	.00028	.00562	.99438	.96587
Gotlib, Kasch et al. (2004)	t(87)=2.78	.00666	.13323	.86677	.65450
Gotlib, Krasnoperova et al. (2004)	t(18)=3.02	.00736	.14716	.85284	.66674
Joormann & Gotlib (2007)	t(25)=2.79	.00994	.19880	.80120	.58726
Mogg et al. (1995)	t(16)=2.99	.00866	.17316	.82684	.63564
Mogg et al. (2004)	t(13)=2.29	.03938	.78765	.21235	.11928
Pishyar et al. (Cognitive Behavioral Group Therapy condition at Time 1; 2008)	t(15)=3.79	.00178	.03558	.96442	.87684

Table D8.

PP-values for p-curve of negative bias findings in studies using clinical samples; between-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Donaldson et al. (2007)	t(35)=3.70	.00074	.01473	.98527	.91895
Pishyar et al. (Waitlist Control condition at Time 1; 2008)	t(15)=2.95	.00993	.19867	.80133	.60552
Reinecke et al. (2011)	t(43)=2.26	.02894	.57884	.42116	.23933
Reinecke et al. (2013)	t(13)=2.75	.01654	.33076	.66924	.46621
Stevens et al. (2009)	F(1,39)=6.81	.01279	.25582	.74418	.51224
Fritzsche et al. (2010)	t(38)=3.14	.00326	.06528	.93472	.78367
Fritzsche et al. (2013)	t(37)=4.70	.00004	.00071	.99929	.99065
Gotlib, Kasch et al. (2004)	t(141)=2.18	.03092	.61834	.38166	.20806
Gotlib, Krasnoperova et al. (2004)	t(33)=2.27	.02987	.59732	.40268	.22920
Joormann & Gotlib (2007)	t(43)=3.74	.00054	.01079	.98921	.93200
Mogg et al. (1995)	F(1,30)=9.40	.00456	.09126	.90874	.73952
Mogg et al. (2004)	t(13)=2.29	.03938	.78765	.21235	.11928
Pishyar et al. (Cognitive Behavioral Group Therapy condition at Time 1; 2008)	t(15)=3.79	.00178	.03558	.96442	.87684

Table D9.

PP-values for p-curve of negative bias findings in studies using non-clinical samples; within-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Bradley et al. (1998)	t(18)=2.20	.04111	.82217	.17783	.09572
Mogg & Bradley (1999a)	t(18)=2.45	.02474	.49487	.50513	.31266
Mogg & Bradley (Experiment 3; 1999b)	t(10)=2.37	.03928	.78554	.21446	.12438
Mogg & Bradley (2002)	t(10)=2.68	.02309	.46185	.53815	.35852
Mogg et al. (1994)	t(35)=2.04	.04895	.97906	.02094	.01026
Oehlberg et al. (Study 1; 2012)	r(63)=.27	.02962	.59239	.40761	.22774
Oehlberg et al. (Study 2; 2012)	r(165)=.28	.00025	.00495	.99505	.95356
Schrooten et al. (2012)	t(29)=2.09	.04550	.90994	.09006	.04558
Shane & Peterson (Study 2; 2007)	t(25)=2.37	.02582	.51632	.48368	.28999
Tran et al. (2013)	t(49)=2.15	.03652	.73035	.26965	.14296
Eldar et al. (2010)	t(22)=2.93	.00775	.15504	.84496	.64848
Bradley et al. (Experiments 1 & 2; 1997)	F(1,69)=4.29	.04208	.84158	.15842	.07993
Fox (Experiment 2; 2002)	t(17)=5.7	.00003	.00052	.99948	.99521
Hunt et al. (Masked condition; 2006)	F(1,103)=8.02	.00557	.11131	.88869	.68739
Ioannou et al. (2004)	t(10)=2.25	.04818	.96359	.03641	.01985
Keogh, Dillon et al. (2001)	F(1,49)=4.61	.03676	.73522	.26478	.14033
Klumpp & Amir (2009)	t(38)=2.05	.04731	.94621	.05379	.02657
Miskovic & Schmidt (2012)	t(16)=2.43	.02724	.54481	.45519	.27758

Table D10.

PP-values for p-curve of negative bias findings in studies using non-clinical samples; between-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Bradley et al. (1998)	F(1,36)=5.99	.01940	.38793	.61207	.38706
Mogg & Bradley (1999a)	t(36)=2.12	.04097	.81934	.18066	.09354
Mogg & Bradley (Experiment 3; 1999b)	t(10)=2.37	.03928	.78554	.21446	.12438
Mogg & Bradley (2002)	F(1,25)=14.17	.00091	.01810	.98190	.91291
Mogg et al. (1994)	F(1,64)=7.34	.00864	.17286	.82714	.60298
Oehlberg et al. (Study 1; 2012)	r(63)=.27	.02962	.59239	.40761	.22774
Oehlberg et al. (Study 2; 2012)	r(165)=.28	.00025	.00495	.99505	.95356
Schrooten et al. (2012)	t(59)=2.44	.01771	.35422	.64578	.41029
Shane & Peterson (Study 2; 2007)	t(63)=2.36	.02139	.42770	.57230	.34754
Tran et al. (2013)	t(119)=2.49	.01415	.28309	.71691	.47227
Eldar et al. (2010)	t(22)=2.93	.00775	.15504	.84496	.64848
Bradley et al. (1997)	F(1,69)=4.29	.04208	.84158	.15842	.07993
Fox (2002)	t(34)=4.6	.00006	.00113	.99887	.98719
Hunt et al. (Masked condition; 2006)	F(1,103)=8.02	.00557	.11131	.88869	.68739
Ioannou et al. (2004)	t(19)=2.43	.02518	.50361	.49639	.30472
Keogh, Dillon et al. (2001)	F(1,49)=4.61	.03676	.73522	.26478	.14033
Klumpp & Amir (2009)	t(74)=2.14	.03565	.71302	.28698	.15167
Miskovic & Schmidt (2012)	F(1,31)=4.63	.03932	.78639	.21361	.11290

Table D11.

PP-values for p-curve of positive bias findings in studies using clinical samples; within-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Donaldson et al. (2007)	t(35)=10.00	.00000	.01000	.99000	.99000
Fritzsche et al. (2010)	t(38)=2.23	.03173	.63464	.36536	.20336
Fritzsche et al. (2013)	t(37)=3.42	.00154	.03080	.96920	.86643
Gotlib, Kasch et al. (2004)	r(87)=.36	.00053	.01060	.98940	.92698
Joormann & Gotlib (2007)	t(43)=2.07	.04449	.88985	.11015	.05523
Mingtian et al. (2011)	t(23)=2.29	.03152	.63049	.36951	.21114
Lindstrom et al. (2011)	t(43)=3.03	.00413	.08256	.91744	.74750
Mueller et al. (2009)	t(11)=3.30	.00708	.14156	.85844	.69839
Pishyar et al. (Cognitive Behavioral Group Therapy condition at Time 1; 2008)	t(15)=3.77	.00185	.03707	.96293	.87317
Pishyar et al. (Waitlist Control condition at Time 1; 2008)	t(15)=4.93	.00018	.00363	.99637	.97839

Table D12.

PP-values for p-curve of positive bias findings in studies using clinical samples; between-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Donaldson et al. (2007)	t(35)=10.00	.00000	.01000	.99000	.99000
Fritzsche et al. (2010)	t(38)=2.23	.03173	.63464	.36536	.20336
Fritzsche et al. (2013)	t(37)=3.42	.00154	.03080	.96920	.86643
Gotlib, Kasch et al. (2004)	r(87)=.36	.00053	.01060	.98940	.92698
Joormann & Gotlib (2007)	t(43)=2.07	.04449	.88985	.11015	.05523
Mingtian et al. (2011)	t(47)=4.69	.00002	.00048	.99952	.99247
Lindstrom et al. (2011)	t(43)=3.03	.00413	.08256	.91744	.74750
Mueller et al. (2009)	t(11)=3.30	.00708	.14156	.85844	.69839
Pishyar et al. (Cognitive Behavioral Group Therapy condition at Time 1; 2008)	t(15)=3.77	.00185	.03707	.96293	.87317
Pishyar et al. (Waitlist Control condition at Time 1; 2008)	t(15)=4.93	.00018	.00363	.99637	.97839

Table D13.

PP-values for p-curve of positive bias findings in studies using non-clinical samples; within-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Bradley et al. (1998)	t(36)=2.58	.01411	.28215	.71785	.48564
Bradley et al. (2000)	r(53)=.28	.03841	.76820	.23180	.12080
Brosschot et al. (1999)	t(19)=2.60	.01759	.35170	.64830	.43261
Ioannou et al. (2004)	t(10)=2.65	.02431	.48626	.51374	.33816
Keogh, Dillon et al. (2001)	F(1,49)=4.08	.04888	.97758	.02242	.01088
Keogh, Ellery et al. (2001)	r(73)=.40	.00038	.00754	.99246	.94256
Shane & Peterson (Study 1; 2007)	t(29)=2.35	.02579	.51581	.48419	.28815
Shane & Peterson (Study 2; 2007)	t(25)=2.51	.01891	.37827	.62173	.40161
Taylor et al. (2010)	r(42)=.41	.00571	.11419	.88581	.69338

Table D14.

PP-values for p-curve of positive bias findings in studies using non-clinical samples; between-subjects findings prioritized

Study	Test entered by user	Recalculated p-value	pp-values
			Right skew	Left skew	Power of 33%
Bradley et al. (1998)	t(36)=2.58	.01411	.28215	.71785	.48564
Bradley et al. (2000)	r(53)=.28	.03841	.76820	.23180	.12080
Brosschot et al. (1999)	t(19)=2.60	.01759	.35170	.64830	.43261
Ioannou et al. (2004)	t(10)=2.65	.02431	.48626	.51374	.33816
Keogh, Dillon et al. (2001)	F(1,49)=4.08	.04888	.97758	.02242	.01088
Keogh, Ellery et al. (2001)	r(73)=.40	.00038	.00754	.99246	.94256
Shane & Peterson (Study 1; 2007)	t(70)=2.13	.03669	.73377	.26623	.13969
Shane & Peterson (Study 2; 2007)	t(63)=3.03	.00354	.07089	.92911	.76437
Taylor et al. (2010)	r(42)=.41	.00571	.11419	.88581	.69338