The Value of Suppressor Effects in Explicating the Construct Validity of Symptom Measures

Abstract

Suppressor effects are operating when the addition of a predictor increases the predictive power of another variable. We argue that suppressor effects can play a valuable role in explicating the construct validity of symptom measures by bringing into clearer focus opposing elements that are inherent—but largely hidden—in the measure’s overall score. We illustrate this point using theoretically grounded, replicated suppressor effects that have emerged in analyses of the original Inventory of Depression and Anxiety Symptoms (IDAS; Watson et al., 2007) and its expanded second version (IDAS-II; Watson et al., 2012). In Study 1, we demonstrate that the IDAS-II Appetite Gain and Appetite Loss scales contain both (a) a shared distress component that creates a positive correlation between them and (b) a specific symptom component that produces a natural negative association between them (i.e., people who recently have experienced decreased interest in food/loss of appetite are less likely to report a concomitant increase in appetite/weight). In Study 2, we establish that mania scales also contain two distinct elements—namely, high energy/positive emotionality and general distress/dysfunction—that oppose each another in many instances. In both studies, we obtained evidence of suppression effects that were highly robust across different types of respondents (e.g., clinical outpatients, community adults, college students) and using both self-report and interview-based measures. These replicable suppressor effects establish that many homogeneous, unidimensional symptom scales actually contain distinguishable components with distinct—at times, even antagonistic—properties.

Suppressor effects have had a long and uneven history in psychological assessment. The concept of a suppressor variable was first introduced into the psychological literature by Horst (1941). Horst discussed cases in which a variable that was uncorrelated with the criterion—but was significantly correlated with another predictor—boosted the overall predictive power of a model in multiple regression. Horst’s explanation for this somewhat puzzling phenomenon was that the suppressor (the variable that is uncorrelated with the criterion) controls for criterion-irrelevant variance in the other predictor, thereby boosting its predictive power.

Horst (1966; cited in Paulhus, Robins, Trzesniewski, & Tracy, 2004) provides a good example from a study of pilot selection in World War II. Applicants were given a battery of written tests to evaluate their potential; spatial ability predicted successful training, but verbal ability did not. However, verbal ability was significantly correlated with spatial ability, reflecting the common underlying influence of general mental ability (g). As a result, when verbal ability (the suppressor) was put into the regression equation, the overall R² increased significantly.

The concept of suppression quickly had a major impact in the assessment literature. One prominent application involved the creation of validity scales to control for the effects of response biases such as social desirability and acquiescence (for a discussion, see McGrath, Mitchell, Kim, & Hough, 2010; Paulhus et al., 2004). For example, only five years after the publication of Horst’s seminal article, Meehl and Hathaway (1946) introduced the K scale as a suppressor variable that could be used to enhance the validity of the standard clinical scales in the Minnesota Multiphasic Personality Inventory (MMPI). The K scale was developed following closely on the logic of Horst’s formulation: It “was not assumed to be measuring anything which in itself is of psychiatric significance” (p. 544), but simply was “intended to be the correction of the other scores” (p. 544) by eliminating criterion-irrelevant variance—specifically, defensive responding that presumably masked “true variance.”

Subsequent authors expanded the concept of suppression, and suppressor effects now are defined as cases in which the inclusion of a second predictor increases the predictive power of one or both predictors (Cohen & Cohen, 1975; Conger, 1974; Gaylord-Harden, Cunningham, Holmbeck, & Grant, 2010; Lynam, Hoyle, & Newman, 2006; McNemar, 1945; Paulhus et al., 2004). Three basic types of suppressor effects have come to be recognized in the literature (see Cohen & Cohen, 1975; Conger, 1974; Gaylord-Harden et al., 2010; Paulhus et al., 2004). The type identified by Horst (1941)— in which the suppressor is uncorrelated (or only weakly correlated) with the criterion—now is termed classical suppression. Reciprocal or cooperative suppression involves cases in which two predictors either (a) correlate oppositely with the criterion, but are positively related to one another or (b) both are correlated positively with the criterion but negatively with one another; in these cases, including both predictors in the regression equation increases both of their beta weights. Finally, net or cross-over suppression describes cases in which all three variables (i.e., both predictors and the criterion) are correlated positively with one another; including both predictors in the regression equation increases the weight for the stronger predictor and changes the sign of the weaker predictor (i.e., a positive zero-order correlation becomes a negative beta weight).¹

The Dark Age of Suppression

The initial enthusiasm for suppressors eventually gave way to pronounced disappointment (see Wiggins, 1973; Paulhus et al., 2004), which can be traced to three basic problems that emerged in this literature. First, validity scales have not proven to be particularly effective in boosting the predictive power of other measures (e.g., Barthlow, Graham, Ben-Porath, Tellegen, & McNulty, 2002; Borkenau & Ostendorf, 1992; McGrath et al., 2010; Piedmont, McCrae, Riemann, & Angleitner, 2000; Paulhus et al., 2004; Wiggins, 1973), in many cases because they did not have the properties required. The MMPI K scale, for instance, correlates strongly negatively with measures of neuroticism and negative affectivity (Barthlow et al., 2002; Watson & Clark, 1984), which is inconsistent with the rationale for its creation and made its use as a correction factor problematic.

Second, suppressor effects frequently fail to replicate (Ghiselli, 1972; Paulhus et al., 2004; Wiggins, 1973). Ghiselli (1972) dismissed suppressors as “fragile and elusive,” likening them to a “will-o-the-wisp.” (p. 270) Similarly, in an influential review of the personality assessment literature, Wiggins (1973) noted that the published results to date were unimpressive and that “The number of unpublished failures to find suppressor variables cannot be estimated” (p. 38).

Finally, suppressor effects were judged to be of little practical value. Wiggins (1973), for example, argued that “In those instances in which suppressor variables have been developed successfully…the net predictive gain does not appear to be greater than that which would have been achieved by employing a more conventional variable as an additional predictor” (p. 38). He further added that “It seems reasonable to conclude that the utility of suppressor variables remains to be demonstrated” (p. 38); we return to this issue in the General Discussion. Despite these concerns, researchers have continued to publish suppressor effects that are difficult to interpret, are of uncertain practical value, and that fail to replicate across samples (see Lynam et al., 2006). As a result, suppressor effects have acquired a somewhat unsavory reputation.

Renewed Interest in Suppressor Effects

Nevertheless, several authors have demonstrated the value of well-conducted suppressor analyses (e.g., Blonigen et al., 2010; Harmon-Jones & Harmon-Jones, 2010; Hicks & Patrick, 2006; Hopwood & Morey, 2008; McNulty & Harkness, 2002). For example, in a study in which personality traits and other individual-differences variables were used to differentiate offenders from non-offenders, Collins and Schmidt (1997) identified four replicable suppressor variables that boosted the overall predictive power of a regression model by 11% in an independent cross-validation sample. Paulhus et al. (2004) discussed two replicable mutual suppression effects—one involving shame and guilt, and the other related to self-esteem and narcissism—that were well grounded in theory and, therefore, psychologically meaningful and readily interpretable. Similarly, Gaylord-Harden et al. (2010) demonstrated replicable and theoretically meaningful suppressor effects involving various coping strategies.

We extend this work by illustrating how suppressor effects can play a valuable role in explicating the construct validity of symptom measures. Consistent with the original formulation of Horst (1941; see also McNemar, 1945), they do so by bringing into clearer focus components that are hidden—either partly or completely—in the original measure. More specifically, by eliminating variance in a predictor shared with the suppressor variable, we are able to see another component within a scale—one that does not overlap with the suppressor—more clearly. We illustrate these points using theoretically grounded, replicable suppressor effects that have emerged in analyses of the original Inventory of Depression and Anxiety Symptoms (IDAS; Watson et al., 2007) and its expanded second version (IDAS-II; Watson et al., 2012).

We must emphasize that the partialled components that emerge in regression do not represent the “true” construct that is being assessed in the measure, a point of some confusion in the literature (see Lynam et al., 2006, for a related discussion); rather, all aspects of the original measure are relevant in establishing its construct validity. Indeed, one important implication of suppressor effects is that they demonstrate that even homogeneous, unidimensional measures can contain multiple components that may oppose one another, thereby obscuring their existence.

Study 1

Our first case illustrates a situation that may be relatively common in the psychopathology literature. It involves two IDAS symptom scales—one assessing appetite loss, the other measuring appetite gain—that logically should be negatively correlated with one another. That is, one would expect that people who recently have experienced decreased interest in food/loss of appetite would be less likely to report a concomitant increase in appetite. Indeed, to a considerable extent, these two symptoms seem to be incompatible with one another.

The situation is complicated, however, by the fact that both scales are indicators of psychopathology (see Watson et al., 2007). As such, they both contain a component of general distress/negative affect/demoralization (Ben-Porath & Tellegen, 2008; Lahey et al., 2012; Watson, 2005, 2009; Watson et al., 2007) that produces a positive correlation between them. Thus, the weak observed correlation between these scales represents the dual effects of these antagonistic elements, which—to some extent—cancel each other out.

We examine the nature and robustness of these effects in four large samples of respondents: psychiatric patients, community adults, college students, and postpartum women. Various aspects of these samples—including demographic characteristics and methods of recruitment—are described in greater detail elsewhere (Watson et al., 2007, 2008, 2012). In addition, we illustrate the generality of these effects by reporting analyses using non-IDAS indicators of general distress/depression.

Method

Participants

Psychiatric patients

Outpatients (N = 1,914) were recruited from a variety of clinical sites in Iowa City, Iowa; Long Island, New York; and the greater South Bend metropolitan area (see Watson et al., 2008, 2012). The sample consisted of 677 men and 1,227 (64%) women (10 participants did not specify sex), with a mean age of 41.1 years (range = 18–83 years).

Community adults

We report data on 1,837 adults living in either eastern Iowa or the greater South Bend metropolitan area (see Watson et al., 2007, 2012). This sample included 529 men and 1,303 (71%) women (five unspecified), with a mean age of 37.4 years (range = 18–85 years).

College Students

This sample included 3,484 students enrolled in psychology courses at the Universities of Iowa, Buffalo, North Texas, and Notre Dame (see Watson et al., 2007, 2008, 2012). It consisted of 1,218 men and 2,226 (64%) women (40 unspecified), with a mean age of 19.7 years (range = 18–57 years).

Postpartum women

Postpartum women (N = 1,070) who had delivered within the previous 4 months were identified through public birth records in several eastern Iowa counties (this is an expanded version of the sample that is described in Watson et al., 2007). These participants had a mean age of 27.9 years (range = 18–45 years).

Measures

IDAS

All participants completed the original 64-item IDAS (Watson et al., 2007); we report here on the Appetite Loss, Appetite Gain, and Dysphoria scales. Respondents indicated the extent to which they experienced each symptom “during the past two weeks, including today” on a 5-point scale ranging from not at all to extremely. Appetite Loss (e.g., “I did not have much of an appetite”) and Appetite Gain (e.g., “I ate more than usual”) both are 3-item scales that assess a narrow range of symptom content and, therefore, have high average interitem correlations (AICs); Watson et al. (2007) report AICs ranging from .51 to .76 across various samples (see their Table 3). Watson et al. (2007, 2008) report extensive reliability and validity data on these scales. Coefficient alphas in the current samples ranged from .78 to .88, with AICs ranging from .54 to .70.

Table 3.

Regression Analyses Predicting Appetite Gain from Appetite Loss and Various Measures of Depressive Symptom Severity

Predictor/Effect	BDI-II	CES-D	EPDS	IMAS
Appetite Loss
β Alone	−.14*	−.09	−.22*	−.28*
β Joint	−.35*	−.39*	−.37*	−.49*
Sobel Test (z)	21.19*	8.42*	9.33*	6.14*
Depression Measure
β Alone	.29*	.28*	.29*	.12
β Joint	.46*	.50*	.42*	.38*
Sobel Test (z)	−17.49*	−7.55*	−8.81*	−7.33*
Model Predictive Power
Sum of Individual R2s	.107	.085	.132	.091
Joint Regression R2	.185	.175	.201	.181
Suppressor R2 Increment	.078	.090	.070	.090

Note. N = 3,445 (BDI-II), 686 (CES-D), 1,070 (EPDS), 390 (IMAS). BDI-II = Beck Depression Inventory-II. CES-D = Center for Epidemiological Studies Depression Scale. EPDS = Edinburgh Postnatal Depression Scale. IMAS = Interview for Mood and Anxiety Symptoms.

^*p < .01

In contrast, Dysphoria is a broad and relatively nonspecific scale that assesses the general distress/negative affectivity dimension that lies at the core of the mood and anxiety disorders (see Watson, 2009; Watson et al., 2007, 2008). It contains 10 items assessing depressed mood, anhedonia, worry, worthlessness, guilt, hopelessness, psychomotor difficulties, and cognitive problems. Coefficient alphas for this scale ranged from .88 to .90 (with AICs ranging from .43 to .48) in these samples.

Beck Depression Inventory-II (BDI-II)

The 21-item BDI-II (Beck, Steer, & Brown, 1996) is one of the most widely used and best validated self-report measures of depressive symptom severity (see Joiner, Walker, Pettit, Perez, & Cukrowicz, 2005). BDI-II items are each rated on a 4-point scale ranging from 0 to 3. Respondents choose the option that best characterizes how they have been feeling “during the past two weeks, including today.” BDI-II data were available on 971 patients, 743 adults, 675 students, and 1,056 postpartum women. Coefficient alphas ranged from .87 to .95 in these samples.

Center for Epidemiological Studies Depression Scale (CES-D)

The 20-item CES-D (Radloff, 1977) is another widely used measure of depression (Joiner et al., 2005). Items are scored on a scale of 0 (rarely or hardly ever) to 3 (most or all of the time). The item content assesses depressed mood and other symptoms of mood disorder, in addition to positive affect and interpersonal difficulties. Data were available from 302 patients and 384 adults. Coefficient alphas were .93 (patients) and .94 (adults).

Edinburgh Postnatal Depression Scale (EPDS)

The postpartum women completed the 10-item EPDS (Cox, Holden, & Sagovsky, 1987). For each item, participants choose the response (from four options) that best describes their experience over the previous week. Substantial evidence has established the validity of the EPDS as a tool to screen for depression in the postpartum period (Cox & Holden, 2003; Eberhard-Gran, Eskild, Tambs, Opjordsmoen, & Samuelson, 2001). The EPDS had a coefficient alpha of .87 in this sample.

The Interview for Mood and Anxiety Symptoms (IMAS)

We used the IMAS (Gamez, Kotov, & Watson, 2010; Watson et al., 2007, 2012) to obtain symptom-level clinical ratings; these data were available on 390 patients. The IMAS assesses current (i.e., past month) symptoms without the use of skip-outs; that is, every question is asked of every participant. Items were derived from the mood and anxiety disorder modules of the Composite International Diagnostic Interview (CIDI; Kessler & Üstün, 2004) and were designed to cover all DSM-IV mood and anxiety disorder symptom criteria. Based on data from prior studies (Watson et al., 2007; Gamez et al., 2010), the IMAS was revised to improve its symptom coverage. We used the revised version, which consists of 10 scales corresponding to major DSM-IV anxiety and mood syndromes, plus irritability. We report data in Study 1 on the 49-item IMAS Depression scale (α = .95); the 20-item Mania scale is discussed in Study 2. For information on the other IMAS scales, see Watson et al. (2012).

Extensively trained lay interviewers administered the IMAS. Individual items are scored on a 3-point rating scale (absent, subthreshold, above threshold). All interviews were recorded; a randomly selected interviewer rescored 34 tapes (8.7%). Interrater reliability consistently was excellent, with ICCs ranging from .97 to .99 across the various scales (see Watson et al., 2012).

Results and Discussion

IDAS-II Analyses

Scale correlations

Table 1 presents the correlations among the IDAS Dysphoria, Appetite Loss, and Appetite Gain scales separately in each sample. In addition, we report combined correlations collapsed across all of the samples (N = 8,305). To eliminate mean-level differences in symptom levels across these four populations, we standardized the scores on a within-sample basis and then combined them for a single overall analysis.

Table 1.

Simple Correlations Between the IDAS Dysphoria, Appetite Loss and Appetite Gain Scales

Sample	Dysphoria-Appetite Loss	Dysphoria-Appetite Gain	Appetite Loss-Appetite Gain
Clinical Patients	.44*	.26*	−.26*
Adults	.48*	.40*	−.05
College Students	.48*	.46*	.03
Postpartum	.40*	.35*	−.22*
Overall	.46*	.39*	−.09*

Note. N = 1,914 (Patients), 1,837 (Adults), 3,484 (College Students), 1,070 (Postpartum), 8,305 (Overall). IDAS = Inventory of Depression and Anxiety Symptoms.

^*p < .01

Table 1 indicates that the appetite scales both correlated moderately positively with Dysphoria in all four samples, with coefficients ranging from .26 (Appetite Gain in the patients) to .48 (Appetite Loss in the student and the community adult samples). The overall mean correlations (.46 for Appetite Loss, .39 for Appetite Gain) demonstrate that both appetite scales are indicators of psychopathology and contain a common component of general distress/negative affect. Under most circumstances, the fact that these symptom scales both correlate moderately positively with a third variable would lead one to expect that they also would be associated positively with one another. As discussed earlier, however, the situation here is complicated by the fact that indicators of appetite gain and appetite loss logically should relate negatively to one another. This natural inverse relation, in turn, helps to explain the fact that the appetite scales actually are weakly negatively related to one another (overall mean r = −.09, range = .03 to −.26). As suggested earlier, these observed correlations appear to represent the joint effect of two opposing elements—namely, the positively correlated general distress component and the negatively correlated specific appetite components—that largely cancel each other out. As such, this represents a very promising context for the emergence of suppressor effects.

Suppression effects on the association between Appetite Gain and Appetite Loss

Suppression effects can be examined formally by investigating how standardized beta coefficients change as one moves from a one- to a two-predictor regression model (e.g., Gaylord-Harden et al., 2010; Paulhus et al., 2004). We first examine the hypothesized suppressor effect on the association between the two appetite scales. The upper portion of Table 2 shows how the addition of Dysphoria as a predictor influences the explanatory power of Appetite Loss in predicting Appetite Gain. These results demonstrate the existence of a substantial and—as can be seen from the results in the different samples—highly replicable suppressor effect. By itself, Appetite Loss had an overall standardized weight of only −.09 in predicting Appetite Gain. The inclusion of Dysphoria in the regression model, however, increased the size of this negative weight in all four samples, with coefficients ranging from −.24 (students) to −.46 (patients); across the four samples, the overall mean coefficient increased to −.33, a moderate inverse association. Thus, controlling for the general distress component allowed the natural negative correlation between the two appetite scales to emerge more clearly.

Table 2.

Regression Results Predicting Appetite Gain from Appetite Loss and Dysphoria

Predictor/Effect	Patients	Adults	Students	Postpartum	Overall
Appetite Loss
β Alone	−.26*	−.05	.03	−.22*	−.09*
β Joint	−.46*	−.32*	−.24*	−.43*	−.33*
Sobel Test (z)	15.01*	16.67*	23.37*	11.19*	34.20*
Dysphoria
β Alone	.26*	.40*	.46*	.35*	.39*
β Joint	.47*	.55*	.57*	.52*	.54*
Sobel Test (z)	−14.93*	−11.77*	−13.30*	−10.30*	−25.82*
Model Predictive Power
Sum of Individual R2s	.135	.166	.209	.170	.157
Joint Regression R2	.241	.240	.254	.275	.237
Suppressor R2 Increment	.106	.074	.045	.105	.080

Note. N = 1,914 (Patients), 1,837 (Adults), 3,484 (Students), 1,070 (Postpartum), 8,305 (Overall).

^*p < .01

We computed the Sobel test (MacKinnon, Krull, & Lockwood, 2000)—which has been shown to work well in large samples—to test the significance of these suppression effects (we used a calculator for the Sobel test that is available online at http://www.quantpsy.org/sobel/sobel.htm). As is displayed in Table 2, all of the tests were significant, establishing the existence of a significant suppressor effect in every sample.

Suppression effects on the association between Appetite Gain and Dysphoria

Next, we investigated potential suppression effects on the positive correlation between Appetite Gain and Dysphoria. The middle portion of Table 2 shows how the inclusion of Appetite Loss as a second predictor influences the explanatory power of Dysphoria in predicting Appetite Gain. By itself, Dysphoria had an overall standardized weight of .39 with Appetite Gain; the inclusion of Appetite Loss in the regression model significantly increased the magnitude of the regression weight (overall β = .54) in all four samples. Consequently, this is a case of reciprocal or cooperative suppression: The positive association between these two symptom scales emerges more strongly in the two-predictor models.

Quantifying the overall gain in predictive power

One striking aspect of suppressor effects is that they can lead to surprisingly large increases in overall model predictive power. We quantify the impressive power of these cooperative suppressor effects in the bottom portion of Table 2. The first row simply sums the independent explanatory power of the two predictors. For example, in the patient results, the individual predictive contributions of Appetite Loss (−.256² = .065) and Dysphoria (.263² = .069) sum to an R² value of .135. When both predictors are included in the regression model simultaneously, however, its overall explanatory power surges to .241 (second row). Thus, these cooperative suppressor effects jointly provide an incremental R² boost of .106 (from .135 to .241) in this sample. Across the four samples, the two suppressors jointly augment the predictive power of the model by an additional 4.5% to 10.6%, with an overall value of 8.0%.

Using Appetite Loss as the criterion

We also can test these suppressor effects by conducting parallel analyses in which Appetite Gain and Dysphoria are used as predictors of Appetite Loss. These results are available online and are reported in supplemental Table S1. Again, we obtained evidence of significant cooperative suppression in all four samples.

Analyses Using Other Measures of Depression

Next, we tested the generality of these suppression effects by using other measures of depression—the BDI-II, CES-D, EPDS, and IMAS—to model the general distress component in the appetite scales. Because data on the CES-D (patients and adults) and BDI-II (all four samples) were collected from different populations, we again standardized the scale scores on a within-population basis to eliminate mean-level differences, and then combined them for an overall analysis for each measure.

Paralleling the Table 2 analyses, Table 3 reports regression results using Appetite Loss and depression to predict Appetite Gain. Significant cooperative suppression effects emerged for all four measures. As in Table 2, the combined predictive power of the two suppressor effects was quite large: Across the four analyses, they jointly contributed an additional 7.0% (EPDS) to 9.0% (CES-D and IMAS) of the variance in the prediction of Appetite Gain.

Supplemental Table S2 presents parallel regression results using Appetite Gain and depression to predict Appetite Loss. Significant cooperative suppression effects again emerged for all four measures.

Summary

We found evidence of significant cooperative suppression effects in every regression analysis. Thus, these effects clearly are highly robust: We obtained significant evidence of cooperative suppression across four different types of respondents and using both self-report and interview-based measures of distress/depression. Moreover, the magnitude of these effects was substantial: Across the analyses reported in Tables 2 and 3, the two suppressors contributed an additional 4.5% to 10.6% of the variance beyond the separate linear effects of the individual predictors.

These results clearly establish that the IDAS appetite scales contain two distinct components that have opposing effects. On the one hand, each scale taps a distinct, specific type of appetite disturbance that naturally is negatively correlated with the other. On the other hand, both scales also are indicators of psychopathology and, consequently, contain a shared component of general distress/negative affect (Ben-Porath & Tellegen, 2008; Lahey et al., 2012; Watson, 2005, 2009) that produces a positive correlation between them. Thus, the observed zero-order correlation between these scales represents the joint effect of these antagonistic elements. Suppressor analyses separate these two components and allow them to be observed, whereas they are obscured when only the simple, zero-order correlations are considered.

Study 2

Virtually all major forms of psychopathology are associated with elevated levels of negative affectivity, although the magnitude of this general distress component varies substantially across disorders (see Kotov, Gamez, Schmidt, & Watson, 2010; Mineka, Watson, & Clark, 1998; Watson, 2009). In addition, many disorders—including depression, social anxiety/social phobia, and schizophrenia/schizotypy—are characterized by reduced levels of positive affect (for a review, see Watson & Naragon-Gainey, 2010). In other words, it generally is the case that elevated levels of positive affect are adaptive and are associated with greater health, well-being, and superior psychosocial functioning. Overall, therefore, we generally expect symptom measures to be associated with higher levels of negative mood and, to a lesser extent, lower levels of positive mood.

Because elated, expansive mood is a hallmark of many manic episodes (American Psychiatric Association, 2000; Gruber, Mauss, & Tamir, 2011; Watson et al., 2012), however, bipolar symptom measures run counter to this general trend and may contain a substantial elevated positive affect component. This unusual combination of distress and elevated positive mood makes these scales very promising candidates for suppressor effects. We examine these effects in Study 2, focusing in particular on two new bipolar symptom scales—IDAS-II Mania and Euphoria (Watson et al., 2012).

Watson et al. (2012) presented extensive data establishing that Euphoria assesses a pathological form of positive affect: Because it has a significant pathological component, it shares a general distress component with measures such as Dysphoria, which yields a positive correlation between these scales. However, unlike Dysphoria—which is associated with lower levels of positive mood (see Watson et al., 2007)—Euphoria is associated with elevated positive affect (Watson et al., 2012), thereby containing a second component that should correlate negatively with scales assessing general distress/negative affect. Thus, the pathological and positive mood components of Euphoria should correlate in opposite directions—positively and negatively, respectively—with Dysphoria. As in the case of the appetite scales, these opposing forces largely should cancel each other out, such that Euphoria should be weakly related to Dysphoria at the simple zero-order level.

We examine suppressor effects in mania measures using subsets of three of the Study 1 samples: psychiatric patients, community adults, and college students. Although we focus primarily on the two IDAS-II bipolar scales, we also test the generality of these effects by reporting analyses using non-IDAS indicators of mania.

Method

Participants

As noted, these results are based on subsets of the Study 1 patient, adult and student participants who were assessed using the complete IDAS-II (Watson et al., 2012). We present data based on 908 patients, 1,091 community adults, and 2,504 college students.

Measures

Well-Being

In addition to the Dysphoria scale described in Study 1, all participants completed the IDAS-II Well-Being, Mania and Euphoria scales. The 8-item Well-Being scale assesses a healthy, adaptive form of positive affect (e.g., “I felt cheerful,” “I looked forward to things with enjoyment,” “I felt hopeful about the future”). It shows impressive specificity by displaying substantially stronger negative associations with measures of depression versus anxiety (Watson, 2009; Watson & Naragon-Gainey, 2010; Watson et al., 2007, 2008). Coefficient alphas ranged from .88 to .90 in the current samples (AICs: .47 to .52).

Mania

The 5-item Mania scale assesses such bipolar symptoms as talkativeness/pressure of speech, flight of ideas, and restlessness/agitation (e.g., “It felt like my mind was moving ‘a mile a minute’,” “I kept racing from one activity to the next”). Watson et al. (2012) reported that the scale showed substantial convergent validity with other self-reported mania scales (e.g., r = .56 with the Hypomania scale of the General Behavior Inventory [GBI], Depue, Krauss, Spoont, & Arbisi, 1989; Depue et al., 1981). It also exhibited significant criterion validity, correlating particularly strongly (r = .63) with the IMAS Mania scale and significantly with current manic-episode diagnoses (polychoric r = .33) on the Structured Clinical Interview for DSM-IV (SCID; First, Spitzer, Gibbon, & Williams, 1997). Coefficient alphas ranged from .82 to .86 in our samples (AICs: .47 to .56).

Euphoria

As noted, the 5-item Euphoria scale assesses a dysfunctional form of positive affect. It taps content reflecting an elated and expansive mood, heightened energy, and grandiosity/excessive self-esteem (e.g., “ I felt like I was ‘on top of the world’,” “ I felt that I could do things that other people couldn’t”). Watson et al. (2012) reported that Euphoria displayed significant convergent validity in relation to self-reported mania measures and demonstrated strong criterion validity. Among the IDAS-II scales, Euphoria had the strongest individual associations with both IMAS Mania (r = .64) and SCID diagnoses of current manic episodes (polychoric r = .47). In the current samples, its coefficient alphas ranged from .72 to .79 (AICs: .34 to .43).

GBI Hypomania

The GBI was designed to assess individuals’ risk for bipolar disorder (Depue et al., 1981). The 21-item Hypomania scale taps a broad range of bipolar symptoms (e.g., distractibility, reduced need for sleep, pressure of speech). Respondents rate how often they experience each item on a 4-point scale ranging from never or hardly to very often or almost constantly. GBI data were available on 713 adults and 993 students. Coefficient alphas were .92 (adults) and .91 (students).

Hypomanic Personality Scale (HPS)

The 48-item HPS (Eckblad & Chapman, 1986) measures long-term manic/hypomanic tendencies, rather than current symptoms, using a true-false format. It taps a diverse array of content, including hyperactive, goal-directed and exhibitionistic behaviors, as well as feelings of euphoria and flight of ideas. We collected HPS responses from 363 adults and 1,069 students. Coefficient alphas were .89 (adults) and .88 (students).

IMAS Mania

We collected interview-based ratings on the IMAS Mania scale (20 items; α = .88) in the Study 1 patient sample (N = 389 for this measure). As discussed in Study 1, the IMAS was administered by extensively trained lay interviewers, who displayed excellent interrater reliability (ICCs ranged from .97 to .99 across the various scales).

Results and Discussion

IDAS-II Scale Correlations

Table 4 presents correlations among the IDAS-II Dysphoria, Mania, Well-Being and Euphoria scales. To simplify the presentation, we again standardized the scale scores on a within-sample basis to eliminate mean-level differences and then combined them in a single overall analysis (these data are adapted from Watson et al., 2012, Table 3).

Table 4.

Overall Correlations between the IDAS-II Dysphoria, Mania, Well-Being and Euphoria Scales (Standardized Combined Sample)

Scale	1	2	3	4
1. Dysphoria	.—
2. Mania	.56*	.—
3. Well-Being	−.40*	.02	.—
4. Euphoria	.08*	.45*	.51*	.—

Note. N = 4,503. Correlations ≥ |.40| are in bold. IDAS-II = Expanded version of the Inventory of Depression and Anxiety Symptoms.

^*p < .01

It is noteworthy that Euphoria related positively to both Well-Being (r = .51) and Mania (r = .45), consistent with the argument that Euphoria taps a maladaptive form of positive affect that is specifically related to mania (see also Watson et al., 2012). Nevertheless, this correlational pattern is somewhat unusual for two reasons: (1) As indicated in Table 4, Well-Being and Mania are completely unrelated to one another (r = .02). (2) Euphoria has a substantial positive correlation with both scales, despite the fact that Mania (r = .56) and Well-Being (r = −.40) have opposite associations with Dysphoria. As discussed earlier, this unusual pattern is helpful in understanding the correlation between Euphoria and Dysphoria, which can be viewed as the joint effect of two opposing forces. That is, the pathological and positive mood components of Euphoria are positively and negatively related to Dysphoria, respectively. These antagonistic components largely cancel each other out, producing a very weak zero-order correlation (r = .08) between Euphoria and Dysphoria. We examine this suppressor effect directly in a series of regression analyses.

The very low (.02) correlation between Mania and Well-Being is another odd element in Table 4. This weak coefficient also reflects the opposing effects of two antagonistic components. On the one hand, Mania (r = .45) and Well-Being (r = .51) both are positively correlated with Euphoria. These coefficients presumably reflect a common component of high energy/positive emotionality that builds in a positive correlation between Mania and Well-Being. On the other hand, Mania (r = .56) and Well-Being (r = −.40) correlate oppositely with Dysphoria, reflecting the fact that Mania and Well-Being represent maladaptive and adaptive states, respectively. Thus, this element should create a natural negative correlation between the scales. These opposing forces can be expected to cancel each other out, thereby producing the near-zero correlation observed in Table 4. We examine this suppressor effect directly in a second series of regression analyses.

Analysis 1: Euphoria, Dysphoria and Well-Being

Suppression effects on the association between Euphoria and Dysphoria

We first examine the hypothesized suppressor effect on the association between Euphoria and Dysphoria. The upper portion of Table 5 shows how the inclusion of Well-Being as a second predictor of Euphoria influences the explanatory power of Dysphoria, demonstrating the existence of another substantial—and highly replicable—suppressor effect. By itself, Dysphoria had an overall standardized weight of only .08 as a predictor of Euphoria, whereas including Well-Being in the regression model significantly increased the size of this weight in all three samples, with coefficients now ranging from .32 (adults and students) to .40 (patients); across the three samples, the overall mean coefficient increased to .34, which represents a moderate association. Thus, controlling for the overlapping positive mood component—which builds in a negative association between Dysphoria and Euphoria—allows their shared pathological component to emerge.

Table 5.

Regression Analyses Predicting Euphoria from Dysphoria and Well-Being

Predictor/Effect	Patients	Adults	Students	Overall
Dysphoria
β Alone	.12*	.03	.09*	.08*
β Joint	.40*	.32*	.32*	.34*
Sobel Test (z)	−11.96*	−13.64*	−17.11*	−25.01*
Well-Being
β Alone	.48*	.49*	.54*	.51*
β Joint	.65*	.64*	.65*	.65*
Sobel Test (z)	−9.78*	−9.55*	−13.50*	−19.33*
Model Predictive Power
Sum of Individual R2s	.243	.239	.297	.270
Joint Regression R2	.357	.321	.380	.360
Suppressor R2 Increment	.114	.082	.083	.090

Note. N = 908 (Patients), 1,091 (Adults), 2,504 (Students), 4,503 (Overall).

^*p < .01

Suppression effects on the association between Euphoria and Well-Being

Next, we investigated potential suppression effects on the observed correlation between Euphoria and Well-Being. The middle portion of Table 5 shows how the inclusion of Dysphoria as a second predictor of Euphoria influences the explanatory power of Well-Being. By itself, Well-Being had an overall standardized weight of .51 with Euphoria, and the inclusion of Dysphoria in the regression model significantly increased the magnitude of this regression weight (overall β = .65) in all three samples. Consequently, this again is a case of cooperative suppression: By controlling for the pathological component common to Well-Being and Dysphoria, the positive association between Euphoria and Well-Being (reflecting their shared positive mood component) emerges more strongly in the two-predictor model.

Quantifying the overall gain in predictive power

The striking power of these cooperative suppressor effects is demonstrated in the bottom portion of Table 5. Overall, these effects are as impressive as those observed in Study 1. Table 5 indicates that the two suppressors jointly augmented the predictive power of the model by an additional 8.2% (adults) to 11.4% (patients), with an overall value of 9.0%.

Using Dysphoria as the criterion

We also can test this suppressor effect by conducting parallel analyses in which Euphoria and Well-Being are used to predict Dysphoria; these results are presented in supplemental Table S3. Once again, these results established the existence of significant cooperative suppression in all three samples.

Analysis 2: Mania, Dysphoria and Well-Being

Suppression effects on the association between Mania and Well-Being

We first examine the hypothesized suppressor effect on the association between Mania and Well-Being. The upper portion of Table 6 examines how the inclusion of Dysphoria as a second predictor of Mania influences the explanatory power of Well-Being, again demonstrating the existence of a substantial and highly replicable suppressor effect. By itself, Well-Being had an overall standardized weight of only .02 as a predictor of Mania, whereas the inclusion of Dysphoria in the regression model significantly increased the size of this weight in all three samples; coefficients now ranged from .25 (adults) to .30 (students), with an overall weight increase to .29, a moderate association. Thus, controlling for the overlapping pathological component—which produces a negative correlation between Well-Being (an adaptive state) and Mania (a maladaptive state)—allowed the positive correlation between the scales to emerge.

Table 6.

Regression Analyses Predicting IDAS-II Mania from Dysphoria and Well-Being

Predictor/Effect	Patients	Adults	Students	Overall
Well-Being
β Alone	−.01	−.05	.07*	.02
β Joint	.29*	.25*	.30*	.29*
Sobel Test (z)	−12.30*	−14.01*	−17.19*	−25.35*
Dysphoria
β Alone	.57*	.55*	.56*	.56*
β Joint	.70*	.67*	.66*	.67*
Sobel Test (z)	−8.23*	−8.17*	−13.13*	−17.81*
Model Predictive Power
Sum of Individual R2s	.328	.309	.313	.312
Joint Regression R2	.395	.357	.389	.382
Suppressor R2 Increment	.068	.048	.076	.070

Note. N = 908 (Patients), 1,091 (Adults), 2,504 (Students), Overall (4,503).

^*p < .01

Suppression effects on the association between Mania and Dysphoria

Next, we investigated potential suppression effects on the observed correlation between Mania and Dysphoria. The middle portion of Table 6 shows how the inclusion of Well-Being as a second predictor of Mania influences the explanatory power of Dysphoria. By itself, Dysphoria had an overall standardized weight of .56 with Mania, whereas the inclusion of Well-Being in the regression model significantly increased the magnitude of this regression weight (overall β = .67) in all three samples. Consequently, this is another case of cooperative suppression: By controlling for the overlapping positive mood component—on which Dysphoria and Mania are inversely related—the positive association between the scales (reflecting their shared pathological component) is accentuated.

Quantifying the overall gain in predictive power

We quantify the incremental power of these cooperative suppressor effects in the bottom portion of Table 6. Once again, the magnitude of these effects is impressive: Table 6 indicates that the two suppressors jointly augmented the predictive power of the model by an additional 4.8% (adults) to 7.6% (students) across samples, with an overall mean of 7.0%.

Using Well-Being as the criterion

We also can test this suppressor effect by conducting parallel analyses in which Mania and Dysphoria are used to predict Well-Being; these results are presented in supplemental Table S4. Again, we found evidence of significant cooperative suppression in all three samples.

Analyses Using Other Measures of Mania

Next, we tested the generality of these suppression effects using two different self-report scales (GBI Hypomania and the HPS) and one interview-based measure (IMAS Mania) of mania. Because data on the GBI and HPS were collected on both adults and students, we again standardized the scale scores on a within-population basis to eliminate mean-level differences across samples, and combined them for a single overall analysis for each measure.

As discussed previously, IDAS-II Euphoria correlated strongly with Well-Being (overall r = .51) and weakly with Dysphoria (r = .08), whereas Mania displayed the opposite pattern (rs = .02 and .56, respectively; see Table 4). As shown in Table 7, the correlational pattern of the other mania scales resembled the pattern observed earlier for IDAS-II Mania: They were moderately positively related to Dysphoria (rs ranged from .25 to .47) and more weakly associated with Well-Being (rs ranged from −.03 to .27). We therefore conducted three new series of analyses following the logic of those presented in Table 6.

Table 7.

Regression Analyses Predicting Various Mania Measures from Well-Being and Dysphoria

Predictor/Effect	GBI Hypomania	HPS	IMAS
Well-Being
β Alone	−.03	.11*	.27*
β Joint	.19*	.22*	.47*
Sobel Test (z)	−14.27*	−9.10*	−6.69*
Dysphoria
β Alone	.47*	.25*	.34*
β Joint	.54*	.32*	.52*
Sobel Test (z)	−7.38*	−7.03*	−6.44*
Model Predictive Power
Sum of Individual R2s	.221	.072	.189
Joint Regression R2	.249	.102	.306
Suppressor R2 Increment	.028	.030	.117

Note. N = 1,706 (GBI Hypomania), 1,432 (HPS), 389 (IMAS). GBI = General Behavior Inventory. HPS = Hypomanic Personality Scale. IMAS = Interview for Mood and Anxiety Symptoms.

^*p < .01

Paralleling the Table 6 analyses, Table 7 reports regression results using Well-Being and Dysphoria to predict mania. Significant cooperative suppression effects emerged in all three analyses. The combined power of the two suppressor effects was relatively modest in the prediction of both GBI Hypomania (an incremental 2.8% of the variance) and the HPS (an additional 3.0%). In contrast, the magnitude of these suppression effects was much more striking in the analyses of the interview-based IMAS Mania scale. Adding Dysphoria as a second predictor increased the weight for Well-Being from .27 to .47; conversely, including Well-Being in the regression boosted Dysphoria’s beta coefficient from .34 to .52. Together, the two suppressor effects jointly contributed an additional 11.7% of the variance in this analysis.

Supplemental Table S5 presents parallel regression results using Dysphoria and mania to predict scores on Well-Being. Significant cooperative suppression effects again emerged in every analysis.

Summary

We obtained evidence of significant cooperative suppression effects in every regression analysis. Replicating the findings of Study 1, these suppressor effects clearly are robust. Once again, it is noteworthy that we obtained significant evidence of cooperative suppression across different types of respondents and using both self-report and interview-based measures of mania. Moreover, with the exception of the analyses involving GBI Hypomania and the HPS, the magnitude of these effects was substantial. Across all of the other analyses, the two suppressors contributed an additional 4.8% to 11.7% of the variance beyond the separate linear effects of the individual predictors (see Tables 5 through 7).

These results establish that these mania scales contain two distinct, opposing elements: Each scale contains a significant component of high energy/positive emotionality; however, they also are indicators of psychopathology and, consequently, contain a component of general distress/dysfunction (Watson, 2009). These antagonistic elements serve to attenuate their observed zero-order correlations with both Well-Being (a clear marker of positive emotionality) and Dysphoria (a strong indicator of general distress/dysfunction). The suppressor analyses separate these two components and, therefore, allow them to be observed, whereas they are masked in the zero-order correlations.

Note, however, that various mania scales weight these two elements quite differently. Our results demonstrate that IDAS-II Mania, GBI Hypomania, the HPS, and the IMAS Mania scale all contain a prominent element of general distress and a smaller component related to energy/positive emotionality; consequently, they displayed moderate to strong zero-order correlations with Dysphoria (rs ranged from .25 to .56; see Tables 4 and 7) and somewhat weaker associations with Well-Being (rs ranged from −.03 to .27). In contrast, IDAS-II Euphoria clearly contains a larger element of energy/positive emotionality and a smaller distress component (see Table 4); accordingly, it correlated substantially with Well-Being (r = .51) and only weakly with Dysphoria (r = .08). More generally, other mania measures can be expected to display a wide range of correlational patterns, depending on how they weight these (and potentially other) elements.

General Discussion

Demonstration of Replicable Suppressor Effects

Starting in the 1990s, several studies have demonstrated the value of well-conducted suppressor analyses (e.g., Blonigen et al., 2010; Collins & Schmidt, 1997; Gaylord-Harden et al., 2010; Harmon-Jones & Harmon-Jones, 2010; Hicks & Patrick, 2006; Paulhus et al., 2004). In this paper, we augmented this growing body of evidence by demonstrating two sets of highly robust and theoretically meaningful cooperative suppressor effects. In Study 1, we showed that the two IDAS appetite scales—Appetite Gain and Appetite Loss—contain both (a) a shared distress component that generates a positive correlation between them and (b) a specific symptom component that produces a negative association between them (i.e., people who recently have experienced decreased interest in food/loss of appetite are less likely to report a concomitant increase in appetite/weight). These antagonistic elements largely cancel each other out, thereby yielding a weak zero-order correlation between the scales (overall r = −.09 across the four samples).

In Study 2, we demonstrated that the two IDAS-II bipolar symptom scales—Mania and Euphoria—include components of both general distress/dysfunction and energy/positive emotionality (see Table 7). These opposing elements counteract each other to yield low zero-order correlations between (a) Euphoria and Dysphoria and (b) Mania and Well-Being. Controlling for one of these elements in regression analyses allows the other to be seen, whereas they mask each other in the zero-order correlations.

Explicating the Construct Validity of Symptom Scales

Mania

Our suppressor analyses have helped to explicate the complex nature of the IDAS-II bipolar symptom scales. As noted, IDAS-II Mania and Euphoria contain elements of both general distress/dysfunction and energy/positive emotionality (see Table 7). However, Mania contains a much stronger component of general distress, which was readily apparent in our analyses. It had simple zero-order correlations ranging from .55 to .57 with Dysphoria across the three samples (overall r = .56; see Tables 4 & 6); this association became even stronger after Well-Being was included in the regression equation (overall β = .67). Nevertheless, Mania also contains some specific variance that produces positive associations with indicators of positive emotionality, such as IDAS-II Euphoria and Well-Being. Because Euphoria also is a measure of pathological positive affect, these two elements work in synchrony, so that Mania and Euphoria are substantially related at the simple bivariate level (r = .45 overall; see Table 4). In contrast, Well-Being taps a healthy, functional form of positive affect, whose adaptive nature is antagonistic to the general distress component of Mania, thereby attenuating the zero-order correlation between them (r = .02 overall). By controlling for the influence of the dominant pathological element, the suppressor analyses allow the specific, positive emotional component of Mania to emerge (overall β with Well-Being = .29 in Table 6), thereby helping to explicate the true nature of the scale.

Euphoria

Compared to Mania, Euphoria contains a much stronger element of energy/positive emotionality and a weaker general distress component. Its dominant positive affective component emerged very clearly in our analyses. Euphoria had simple zero-order correlations ranging from .48 to .54 with Well-Being across the three samples (overall r = .51; see Tables 4 & 5), and including Dysphoria in the regression equation augmented this association even further (overall β = .65). In contrast, its weaker component of general distress/dysfunction was obscured in the zero-order analyses, due to the fact that its positive affective element yields a negative correlation with Dysphoria. Thus, the pathological and positive mood components of Euphoria are positively and negatively related to Dysphoria, respectively. These antagonistic components largely cancel each other out, producing a very weak zero-order correlation (r = .08) between them. By controlling for the influence of its dominant positive mood component, the suppressor analyses allow Euphoria’s nonspecific distress element to emerge (overall β with Dysphoria = .34; see Table 5). Thus, these suppressor effects again help to clarify the construct validity of this symptom scale.

Appetite

Whereas the Study 2 suppressor effects largely stem from the fact that the mania scales tap a pathological variant of a normally adaptive process (i.e., positive affect), the Study 1 suppressor analyses involving the IDAS-II appetite scales illustrate a different type of situation that likely occurs with some frequency across different types of psychopathology. Most symptom scales contain variance reflecting general distress/dysfunction (Mineka et al., 1998; Watson, 2009). This shared distress component produces a positive association between indicators of psychopathology whose magnitude varies as a function of the size of the indicators’ nonspecific elements. In certain instances, however, the specific components of particular symptom scales have a natural negative correlation. In the current case, for instance, few people experience both appetite/weight gain and appetite/weight loss simultaneously.

As another example, some forms of psychopathology are characterized by social aloofness and withdrawal, whereas others reflect interpersonally oriented processes such as attention seeking, dependency and separation anxiety (see Krueger et al., 2011); the zero-order correlations between these different types of measures should be attenuated by the antagonistic properties of their nonspecific (general psychopathology) and specific (the nature of their interpersonal variance) elements. It will be interesting to explicate the nature and scope of this type of suppressor effect in future research.

Theoretical and Assessment Implications

The hierarchical nature of psychopathology

We believe that most symptom measures possess multiple components due, in part, to the fact that psychopathology is structured hierarchically (Lahey et al., 2012), such that most scales minimally contain both a nonspecific higher order component and a specific lower order component (Mineka et al., 1998; Watson, 2005, 2009). With regard to the former, most symptom scales contain variance reflecting general distress/dysfunction (Lahey et al., 2012; Watson, 2009). In other words, regardless of the specific nature of the complaint, symptom measures tap into significant, ongoing problems in people’s lives. Consequently, as McNulty and Harkness (2002) put it, “They intrinsically involve detecting a problem in oneself” (p. 443). The intrinsically problematic nature of this symptom content builds in a shared component of nonspecific distress.

This certainly is true of the scales examined here, as they all correlated significantly with IDAS-II Dysphoria and other markers of general distress (e.g., the BDI-II and CES-D) in our analyses. It is noteworthy, however, that the general distress/pathology component in Euphoria was largely obscured at the simple bivariate level and only became visible in the multivariate analyses (see Table 5). The nonspecific component in the two appetite scales also was relatively modest (see also Watson, 2009).

Viewed in this context, it is striking that we were able to demonstrate such strong and robust suppressor effects using various IDAS-II scales. The IDAS was created following the same basic logic that was used in the development of the Minnesota Multiphasic Personality Inventory—2 Restructured Form (MMPI-2-RF; Ben Porath & Tellegen, 2008). In comparison to older instruments, these measures were designed to enhance the discriminant validity of symptom scales, and they did so by adopting a two-part assessment strategy of (a) minimizing the size of the general distress/pathology component in most scales and (b) localizing this nonspecific variance within a single broad measure that was explicitly created to capture the general higher order factor (Demoralization in the MMPI-2-RF, Dysphoria in the IDAS-II).

Our results underscore the fact that although these efforts were successful in reducing the size of this nonspecific component, they did not eliminate it entirely: Specific symptom scales still contain some element of general distress/demoralization. As we suggested earlier, this likely represents a fundamental aspect of symptom measures: The intrinsically problematic nature of these scales’ content creates a common nonspecific component of general distress that builds in a positive correlation between them.

Multidimensional items in unifactorial scales

In recent years, several authors have argued for the importance of using factor analysis (especially confirmatory factor analysis [CFA]) to ensure the creation of homogeneous, unifactorial scales (for a discussion of this issue, see Smith, McCarthy, & Zapolski, 2009). In this regard, it is noteworthy that the IDAS-II scales examined in these suppressor analyses all are homogeneous measures that were developed using factor analyses. For example, the IDAS-II Euphoria scale had AICs ranging from .34 to .43 across the three Study 2 samples. Moreover, Watson et al. (2012) reported that all five items loaded .45 or greater on the same underlying dimension in an item-level factor analysis. The AICs for the other scales were even higher in the current samples, ranging from .47 to .56 for Mania, and from .54 to .70 for the two appetite scales. Nevertheless, our replicable suppressor effects clearly establish that these homogeneous scales contain distinguishable components with distinct—even antagonistic—properties.

Our results, therefore, create an apparent paradox: How can homogeneous scales contain multiple components with distinctive properties? The answer is that commonly used scale construction/validation methods—such as CFA—simply model the covariations among the items without fully explicating the nature of the items themselves. That is, techniques such as CFA simply identify the number of underlying latent variables that are needed to account for the correlations among a given set of indicators, and do not examine (indeed, are not designed to examine) the potentially complex, multilevel nature of these indicators.

In fact, one can use overlapping sets of items to create fully unifactorial scales at varying levels of specificity versus generality within complex hierarchical models. Consider, for example, the IDAS-II item I woke up early and could not get back to sleep. At the narrowest, most specific level, this item can be viewed as an indicator of terminal insomnia or early morning awakening; it could be combined with items of similar content to create a very narrow, unifactorial measure of early morning awakening. However, it also correlates moderately to strongly with items assessing other types of insomnia and other symptoms of major depression, and even with a wide range of internalizing and externalizing symptoms. It therefore also could be used in creating unidimensional measures of insomnia, depression, or even general distress/dysfunction, respectively. Note, however, that although each of these scales would be unifactorial in the conventional sense, they all would be composed of complex, multidimensional items. Thus, a series of CFAs based on different sets of indicators actually could be used to establish that the item I woke up early and could not get back to sleep is an indicator of (a) terminal insomnia, (b) general insomnia, (c) depression, and (d) general distress/dysfunction. Note, moreover, that this complexity is inherent in the item itself (for a more complete discussion and demonstration of this point, see Watson & Clark, 2013).

Conventional factor-analytic methods provide valuable information about the underlying sources of the covariations among items. Because a single factor analysis is restricted in the range of indicators it examines, it cannot be expected to explicate fully the complex nature of the items themselves; as our example in the preceding paragraph illustrates, however, a series of targeted structural analyses may be able to do this effectively. Related to this, our suppressor analyses demonstrate the importance of context in clarifying the complex nature of homogeneous symptom scales: By carefully selecting theoretically meaningful combinations of measures (e.g., by examining Euphoria in relation to both Well-Being and Dysphoria), suppressor analyses allow their individual components to emerge more clearly.

The practical predictive power of suppressor effects

Throughout this paper, we have emphasized the conceptual value of suppressor effects in clarifying the complex nature of symptom scales. Our focus therefore differs substantially from several seminal papers on this topic, which instead emphasized the practical importance of suppressor variables in enhancing one’s power to predict important outcomes. This, then, leads to an important question: How important are suppressors in enhancing the predictive power of symptom scales?

A complete examination of this issue is beyond the scope of this paper. However, based on analyses of our own data, we are inclined to agree with the classic viewpoint offered by Wiggins (1973), who concluded that the apparent predictive gain provided by suppressors typically can be eliminated by adding another well-chosen predictor or two to the regression model.

We illustrate this important point using the subsample of patients who were interviewed on the IMAS (N = 363 in these analyses). Table 8 presents results from analyses in which three IDAS-II scales—Euphoria, Well-Being and Dysphoria—were used to predict scores on four IMAS scales: Depression, Generalized Anxiety Disorder (GAD; 12 items, α = .89), Posttraumatic Stress Disorder (PTSD; 25 items, α = .89), and Panic (23 items, α = .87). The top portion of Table 8 displays the simple bivariate relations between these variables. Despite the fact that Euphoria and Well-Being are substantially positively correlated (r = .54 in this subsample), they again showed opposite relations with the IMAS: Euphoria had weakly positive relations (β values ranged from .13 to .22), whereas Well-Being had low to moderate negative associations (βs ranged from −.15 to −.36) with these scales. As we have seen, this pattern signifies fertile ground for the emergence of suppressor effects.

Table 8.

Predicting IMAS Symptom Scores from the IDAS-II Well-Being, Euphoria, and Dysphoria Scales

Model/Predictor	Depression	GAD	PTSD	Panic
One Predictor Models
Dysphoria	.74*	.72*	.63*	.53*
Euphoria	.13	.16*	.18*	.22*
Well-Being	−.36*	−.26*	−.22*	−.15*
Two Predictor Model
Euphoria	.45*	.43*	.42*	.43*
Well-Being	−.60*	−.49*	−.45*	−.38*
Sum of Individual R2s	.144	.095	.081	.073
Joint Regression R2	.274	.199	.175	.154
Three Predictor Model
Dysphoria	.65*	.69*	.58*	.48*
Euphoria	.16*	.12	.16*	.21*
Well-Being	−.18*	−.05	−.07	−.07
Sum of Individual R2s	.694	.619	.478	.357
Joint Regression R2	.569	.533	.413	.316

Note. N = 363. Values shown are standardized β weights. GAD = Generalized Anxiety Disorder. PTSD = Posttraumatic Stress Disorder.

^*p < .01

We tested for suppression in analyses reported in the middle portion of Table 8; here, Euphoria and Well-Being both were used as predictors of the IMAS scales. As expected, these results revealed the existence of substantial cooperative suppressor effects in all four analyses: The β weights for Euphoria now range from .42 to .45, whereas those for Well-Being range from −.38 to −.60. Moreover, replicating the results of previous analyses, including both predictors in the regression increased the power of the model dramatically in comparison to the simple bivariate relations. Specifically, these cooperative suppressor effects jointly provided an incremental R² boost ranging from .081 (Panic) to .130 (Depression) over the separate linear effects of the two predictors. Once again, these suppressor analyses have uncovered important variance in these scales that is hidden at the bivariate level.

As is shown in the top portion of Table 8, Dysphoria has strong, positive associations with all four IMAS criteria (β values range from .53 to. 74). The bottom portion of the table illustrates the effects of adding it as a third predictor in the regression model. As can be seen, its inclusion greatly enhances the predictive power of the overall model (R²s now range from .316 to .569) while eliminating the suppressor effects entirely. The β weights for Euphoria now range from only .12 to .21, whereas those for Well-Being range from only −.05 to −.18. Thus, based on analyses of our own data, we have reached the same basic conclusion as Wiggins (1973): Suppressor effects typically can be eliminated by the inclusion of additional predictors, which greatly limits their practical value.

Limitations and Future Directions

The current studies have several strengths. Most notably, we establish the robustness of suppressor effects across different types of respondents—clinical outpatients, community adults, college students in both studies, plus postpartum women in Study 1—and across different self-report and interview measures of mania and distress/depression. Nevertheless, our findings were limited to a relatively small group of symptom scales. Furthermore, our appetite-scale analyses were limited to measures of self-reported symptoms. It therefore will be important to examine the generalizability of these findings across different methods (e.g., informant ratings) and a broader range of measures.

Conclusion

Our findings demonstrate that suppressor effects can play a valuable role in explicating the construct validity of symptom measures. They do so by bringing into focus components that are obscured in the overall measure. More fundamentally, they remind us that even highly homogeneous psychopathology scales contain both specific and nonspecific components that may operate in tension with one another. We hope that our results will help to stimulate further research into the nature, scope, and implications of suppressor effects in the assessment of psychopathology.