Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Jul 24.
Published in final edited form as: Cognit Ther Res. 2011 Jun;35(3):266–275. doi: 10.1007/s10608-010-9295-1

Shared Variance among Self-Report and Behavioral Measures of Distress Intolerance

R Kathryn McHugh a, Stacey B Daughters b, Carl W Lejuez c, Heather W Murray a, Bridget A Hearon a, Stephanie M Gorka b,c, Michael W Otto a
PMCID: PMC3721199  NIHMSID: NIHMS483370  PMID: 23894216

Abstract

Distress intolerance may be an important individual difference variable in understanding maladaptive coping responses across diagnostic categories. However, the measurement of distress intolerance remains inconsistent across studies and little evidence for convergent validity among existing measures is available. This study evaluated the overlap among self-report and behavioral measures of distress intolerance in four samples, including an unselected sample, a sample of patients with drug dependence, and two samples of cigarette smokers. Results suggested that the self-report measures were highly correlated, as were the behavioral measures; however, behavioral and self-report measures did not exhibit significant associations with each other. There was some evidence of domain specificity, with anxiety sensitivity demonstrating strong associations with somatic distress intolerance, and a lack of association between behavioral measures that elicit affective distress and those that elicit somatic distress. These findings highlight a potential divergence in the literature relative to the conceptualization of distress intolerance as either sensitivity to distress or as the inability to persist at a task when distressed. Further research is needed to elucidate the conceptualization and measurement of distress intolerance to facilitate future clinical and research applications of this construct.

Keywords: Distress Intolerance, Measurement, Task Persistence, Anxiety Sensitivity

Introduction

Distress intolerance (DI) is an individual difference variable reflecting the inability to tolerate distressing states or the inability to persist at goal-directed tasks when distressed; this inability is hypothesized to result in amplification of distress and motivate maladaptive or inflexible coping strategies (see Zvolsensky & Otto, 2007). In recent years, distress intolerance has emerged as a clinically-relevant variable with transdiagnostic applicability. DI appears to be relevant to a wide variety of pathological conditions (e.g., Corstorphine, Mountford, Tomlinson, Waller, & Meyer, 2007; Daughters, Sargeant, Bornovalova, Gratz, & Lejuez, 2008; Gratz, Rosenthal, Tull, Lejuez, & Gunderson, 2006), and is associated with harmful or risky behaviors such as smoking (Quinn, Brandon, & Copeland, 1996), binging/purging symptoms (Anestis, Selby, Fink, & Joiner, 2007), and relapse to substance use (e.g., Brown, Lejuez, Kahler, & Strong, 2002; Daughters, Lejuez, Kahler, Strong, & Brown, 2005). Furthermore, the reduction of DI is a core component of several cognitive-behavioral treatments (Barlow, Allen, & Choate, 2004; Brown et al., 2008; Hayes, Strosahl, & Wilson, 1999; Linehan, 1993; Otto, Powers, & Fischmann, 2005; Smits, Berry, Tart, & Powers, 2008; Tull, Schulzinger, Schmidt, Zvolensky, & Lejuez, 2007).

Given the increased attention to DI in both research and treatment applications, the validity and reliability of its measurement is of particular importance. To facilitate continued advances in this promising area, it is important to achieve greater consensus on the “gold standard” measurement of DI. However, despite the development of psychometrically-sound measures, there is substantial inconsistency in the measurement of DI across studies. For example, several self-report measures are utilized as measures of DI, including both those designed for this purpose (e.g., Distress Tolerance Scale; Simons & Gaher, 2005) and measures of related constructs used as indices of DI (e.g., Anxiety Sensitivity Index; Reiss, Peterson, Gursky, & McNally, 1986). In addition, several behavioral measures have been used, which assess different types of distress (e.g., frustration, pain), and thus may be domain specific. Moreover, the degree to which these measures are capturing the same latent construct (i.e., share variance) is unclear.

At this time, there is a need for further studies to validate methods for assessing DI. A first step toward this goal is the evaluation of existing measures to determine whether they are measuring the same construct. This study addresses this need by utilizing data from three studies comprised of four samples to evaluate the degree of overlap among self-report and behavioral measures of DI.1

Measuring Distress Intolerance

Self-report measures specifically developed for the measurement of DI include the Distress Tolerance Scale (DTS; Simons & Gaher, 2005), the Discomfort Intolerance Scale (DIS; Schmidt et al., 2006), and the Frustration Discomfort Scale (FDS; Harrington, 2005). Although these measures were designed specifically for the measurement of DI, they each were derived from distinct models of the construct. The DTS is a 15 item self-report measure based on a conceptualization of responses to distress in terms of (1) tolerability, (2) acceptability, (3) functional interference, and (4) emotion regulation. The DTS has demonstrated strong psychometric properties with high internal consistency for each factor (α = 0.72–0.82) and good test-retest reliability over a 6-month interval (r = 0.61). The DIS is a 7 item self-report measure of the ability to tolerate somatic or physical distress on two factors, the ability to tolerate pain and avoidance of physical discomfort. Thus, this measure aims to capture DI as it relates to a specific subtype of distress (uncomfortable physical sensations). The DIS has demonstrated good internal consistency for both factors (α = 0.91 and 0.72 respectively) and good test-retest reliability over a 12 week period (r = 0.63 and 0.66). Finally, the FDS is a 47 item self-report measure of intolerance of frustration based on the rational-emotive behavior therapy model, which posits that dysfunctional beliefs can be categorized by frustration intolerance and low self-worth (see Harrington, 2005). This measure has demonstrated both strong internal consistency (α = 0.95) as well as evidence for discriminant validity.

Three self-report measures developed as measures of other constructs have also been used as indices of DI. Each of these measure constructs that capture either sensitivity to distressing states or behaviors consistent with a lack of persistence during distressing states and thus have been conceptualized as reflective of DI. The Anxiety Sensitivity Index (ASI; Peterson & Reiss, 1992) is a 16 item self-report measure of anxiety sensitivity, or fear of somatic symptoms characteristic of anxious states. The ASI demonstrates high levels of internal consistency in nonclinical populations (α = 0.90) and good test-retest reliability (r = 0.75 for a two week time period). Although early application of the ASI primarily focused on the anxiety disorders, this construct has demonstrated relevance to heterogeneous syndromes (e.g., substance use disorders; Lejuez, Paulson, Daughters, Bornovalova, & Zvolensky, 2006). Thus, anxiety sensitivity has been conceptualized as a construct related to DI as it reflects a perceived intolerance of anxiety-like symptoms. The Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004) is a multi-faceted measure of emotion regulation that specifically evaluates deficits in the ability to regulate emotion. The factor of the DERS most conceptually relevant to DI is the subscale that assesses the ability to persist at goal-driven behavior when feeling distressed. The DERS has been shown to have good test-retest reliability (r = 0.88, p < .01 over a period of 4 to 8 weeks) and high internal consistency (α = 0.93; Gratz & Roemer, 2004). Finally, the Acceptance and Action Questionnaire (AAQ; Hayes et al., 2004) is a 9 item self-report measure of experiential avoidance, or the tendency to avoid distressing internal experiences (e.g., emotions, somatic symptoms). The AAQ has demonstrated adequate internal consistency (α = 0.70; Hayes et al., 2004).

Behavioral measures have used the latency to discontinue distressing tasks as an index of DI. Two such measures use task failure plus negative feedback to induce negative affect. The computerized Paced Auditory Serial Addition Task (PASAT-C; Lejuez, Kahler, & Brown, 2003) asks participants to sum numbers sequentially as they appear on a computer screen. The speed of the task increases over time and, with each error, negative feedback is provided. Although not used for two of the studies presented here, a newer version is now available that utilizes a titration level where participant skill level is measured during the first round which then determines the speed for the more difficult rounds. Although skill level can be covaried out in the original version, this newer version provides a more level playing field across participants.

Similarly the computerized Mirror-Tracing Persistence Task (MTPT-C; Strong et al., 2003) consists of images on the screen that the participant is asked to trace using the computer mouse. This task is particularly challenging because the movement of the cursor is opposite to the movement of the mouse, such that the task is similar to tracing a mirror-image of the object, and with each error negative feedback (a computerized “explosion” sound) is provided. Similar to the PASAT-C, a newer titrated version is available with the thickness of the line varying as a function of skill level. In both tasks, to increase motivation participants are instructed that their performance on the task will determine their reward at the end of the session, yet they can quit the final level at any time. Task persistence is used as the measure of DI. Internal consistency cannot be computed as done for self report measures, however, it is notable that self-reports of dysphoria (often composed of anxiety, frustration, and difficulty concentrating) significantly increase following task administration for both the PASAT-C and MTPT-C indicating the tasks do indeed produce distress. Furthermore the magnitude of self-reported distress on the task is uncorrelated with DI, providing support for these measures as capturing inability to tolerate distress and not just the experience of distress itself (e.g., Daughters, Lejuez, Kahler, et al., 2005).

Behavioral measures that induce physical distress include breath holding and the cold pressor task. Breath holding (BH) tasks assess the latency to taking a breath after breath holding, and often involve several trials, with the longest duration of breath holding used as the relevant measure. The cold pressor task (CP) is used widely to assess pain tolerance and thus may provide an index of distress tolerance relative to physical pain. In this task, participants are instructed to submerge their non-dominant hand up to the wrist in a bath of cold water (e.g., 0–2 degrees Celsius). Similar to the PASAT-C and MTPT-C, participants are informed that they can discontinue the task at any time but that their reward at the end of the session is dependent on their performance; time to discontinuation is used as an index of pain tolerance. As compared to the PASAT-C and MTPT-C, these measures of somatic persistence are not hypothesized to induce an enduring mood; a fleeting sense of emotional relief is common following completion of the pain and breath holding tasks.

These measures represent the current state of the field regarding the assessment of DI, and reflect the heterogeneity in its conceptualization at this time. The two broad components that these assessment strategies capture are: sensitivity to distress (e.g., the degree to which it is bothersome or uncomfortable) and behavioral persistence during states of distress (e.g., the ability to continue at a task despite distress). In addition, variability is seen in the type of distress that is measured. For example, the FDS captures distress specific to frustration, whereas the ASI captures distress specific to anxiety sensations. The degree to which each of these measures is similar or may represent distinction components of DI remains unclear at this time.

Relatively little information is available regarding the construct validity of these DI-specific self-report and behavioral measures. In addition, the degree to which these measures of DI share variance, and thus reflect indices of the same construct, has not been systematically examined. Below, we use data from four studies of diverse populations -- an unselected sample, individuals with an illicit drug dependence disorder, and two samples of cigarette smokers -- that included measures of DI to evaluate the degree of shared variance among these measures. Given the lack of an available gold standard measure or a clear criterion with which to compare these measures, the current exploratory study was conducted to evaluate the degree to which the self-report and behavioral measures share variance. For each study we calculated the correlations: (a) among self-report measures, (b) among behavioral measures, and (c) between self-report and behavioral measures.

Study 1

Study 1 included data from a study evaluating the effect of stress on attentional bias toward threat in an unselected sample (McHugh, Behar, Geem, Gutner, & Otto, 2009). Participants completed a battery of self-report questionnaires and either the PASAT-C or CP.

Participants

Participants age 18 or older recruited from the local community in the Greater Boston area (n=33) and from psychology courses at Boston University (n=38) were enrolled after screening to rule-out hormonal dysfunction and current use of steroidal medication.2 Seventy-two participants provided informed consent and were enrolled in the study; one participant dropped out of the study prior to completing any study procedures. Of the 71 participants (36 women, 35 men) included in the analysis the mean age was 22.3 years (SD = 5.7, range = 18–45). The sample was mostly Caucasian (73%) and highly educated (96% with at least some college).

Measures

Participants completed two self-report (ASI and DERS) and one of two behavioral measures (PASAT-C or CP) of DI; 35 participants completed the PASAT-C and 36 completed the CP. In order to ensure that the behavioral measures elicited distress (and thus were able to assess intolerance of distress), we evaluated change in measures of distress pre- to post-tasks. As part of the PASAT-C program, ratings of distress are collected immediately prior to and following the task. We calculated a dysphoria rating to reflect general distress from these items, consisting of self-reported anxiety, difficulty concentrating, frustration and irritability from the PASAT-C. This dysphoria rating has been used in previous studies as a measure of general distress (e.g., Daughters, Lejuez, Bornovalova et al., 2005). Using a paired-samples t-test, there was a significant increase in dysphoria (t(34) = 6.49, p < .001), suggesting that this manipulation was successful for inducing distress. Internal consistency reliability for this measure was strong (α= .87). Given the reliability of the CP for inducing pain, we did not collect information to test the efficacy of this manipulation.

Results

All participants were included in the analyses. The results of Study 1 are summarized in Table 1. Evaluation of the self-report measures identified a significant correlation between the ASI and the DERS persistence subscale, which reflected a large effect size (r = 0.41, p < .001), according to Cohen’s standards (Cohen, 1988).

Table 1.

Study 1 Correlations Between Distress Intolerance Measures and Sample Descriptives

Measure ASI DERS CP mean SD range
Self-Report
 ASI -- 18.55 9.64 2–50
 DERS 0.41** -- 2.82 0.99 1.2–5
Behavioral
 CP 0.38* 0.31 -- 47.42 17.91 13–60
Open in a new tab

Note.

*

p<.05,

**

p<.01; ASI = Anxiety Sensitivity Index, DERS = Difficulties in Emotion Regulation Scale – persistence at goal-directed activities when distressed subscale, CP = Cold Pressor.

In examining the association between self-report and behavioral measures, the ASI was significantly associated with time to discontinuation in the CP task (r = 0.38, p < .05) reflecting a large effect size, but the association between DERS persistence scale and the CP did not reach significance (r = 0.31, ns). Evaluation of the PASAT-C time to discontinuation data revealed a substantial ceiling effect (91% of participants completed the maximum duration) which precluded evaluation of this outcome measure.

As some authors have recently suggested that the dichotomous evaluation is more ecologically valid for the measurement of DI (e.g., Daughters et al., 2009), because it reflects the behavior of negative reinforcement (i.e., persisting or discontinuing). However, continuous evaluation of persistence may be meaningful relative to amount of time that one is willing to persist despite distress. Given the lack of data at this time indicating the superiority of one approach over the other, we also evaluated whether participants, split into two groups based on whether they discontinued the CP task early differed in their scores on the ASI or DERS persistence scale. Independent samples t-tests evaluating the difference between mean ASI and DERS scores and persistence on the CP measured dichotomously (early discontinuation or no discontinuation; 36% discontinued) revealed a significant difference between groups on the ASI (t(34) = 2.30, p < .05, r = .37), but not on the DERS (t(34) = 1.89, ns, r = .31).

Study 1 Discussion

Despite significant association between the ASI and DERS persistence scale as self-report indices of DI, only the ASI was significantly associated with the behavioral measure of DI (time to discontinuation in the CP task). The association between ASI and pain specifically was expected given the well-established association between ASI and pain indices (see Stewart and Asmundson, 2006). Due to the ceiling effect on the PASAT-C, more challenging or distressing tasks, or tasks of longer duration, may be needed for well-educated unselected samples to adequately capture the range of DI scores. Indeed, a newer version of the PASAT-C is available that adjusts for participant skill level.

Study 2

Given the importance of both somatic and emotional distress in substance dependence (Kreek & Koob, 1998), DI has been widely examined in this area. In particular, DI has been associated with shorter time to relapse in both cigarette smokers (Brandon et al., 2003; Brown, Lejuez, Kahler, & Strong, 2002; Hajek, 1991; Zvolensky, Feldner, Eifert, & Brown, 2001) and illicit drug users (Daughters, Lejuez, Kahler, et al., 2005). DI has also been associated with early treatment drop-out among illicit drug abusers (Daughters, Lejuez, Bornovalova, et al., 2005), highlighting this as a group for whom DI may be of particular clinical relevance.

Participants

Study participants included individuals entering a residential drug treatment center in Northeast Washington D.C. Treatment at this center involves a mix of strategies adopted from Alcoholics and Narcotics Anonymous as well as group sessions focused on relapse prevention. Complete abstinence from drugs and alcohol (verified by a clean urine drug test) is required upon entry into the center and through the duration of the program, with the exception of nicotine; regular drug testing is provided and any drug or alcohol use results in immediate dismissal from the center. When needed, detoxification from an outside source is required prior to entry into the center; therefore, acute drug effects likely did not influence the current findings.

All participants were approached on the Monday of their first week of treatment. Participants providing informed consent (N = 129, 23 women) reported a mean age of 42 years (SD = 10.3, range = 20–66). The sample was primarily African American (88%) and educational level was heterogeneous (28% less than high school, 36% high school or equivalent, 22% some college, 12% technical or college degree or higher; 2 participants data unavailable).

Measures

Study 2 used two self-report measures (DTS and DIS) and two behavioral measures (MTPT-C and PASAT-C). Dysphoria ratings were calculated based on the scale described in Study 1 for both the MTPT-C and the PASAT-C from ratings taken prior to and following the tasks. Results of a manipulation check suggested that ratings of dysphoria significantly increased for both the PASAT-C (t(129) = 7.04, p < .001) and the MTPT-C (t(124) = 6.55, p < .001). Internal consistency reliability was strong from both the MTPT-C (α = .76) and the PASAT-C (α = .81). In addition, to evaluate whether DI was distinct from distress itself, the correlation between post-task dysphoria and persistence, controlling for baseline dysphoria, was evaluated. The partial correlation was not significant for the MTPT-C (r = 0.06, ns), but was significant for the PASAT-C (r = −0.31, p < .001).

Results

The results of Study 2 are summarized in Table 2. Evaluation of the correlation between self-report measures indicated that the DTS and DIS were significantly negatively correlated (r = −0.25, p < .01): lower DTS scores and higher DIS scores indicate higher DI. Additionally, time to discontinuation on the MTPT-C and PASAT-C were significantly correlated with a medium effect size (r = 0.29, p < .01). Evaluating the association between the MTPT-C and PASAT-C based on dichotomous definition of persistence also identified a significant association (χ2 (1,129) = 8.57, p < .01) also in magnitude of a medium effect size (r = .26).

Table 2.

Study 2 Correlations Between Distress Intolerance Measures and Sample Descriptives

Measure DIS DTS MTPT-C PASAT-C mean SD range
Self-Report
 DIS -- 2.84 0.87 1.08–5
 DTS −0.25** -- 3.29 1.13 0.4–6
Behavioral
MTPT-C −0.12 0.07 -- 201.89 107.23 2.92–300
PASAT-C −0.08 0.11 0.29** -- 203.81 121.84 2–300
Open in a new tab

Note.

*

p<.05,

**

p<.01; DIS = Discomfort Intolerance Scale, DTS = Distress Tolerance Scale, MTPT-C = Mirror Tracing Persistence Task, PASAT-C = Paced Auditory Serial Addition Task.

In the tests of the association between the self-report and behavioral measures, DTS was not significantly associated with the PASAT-C time to discontinuation (r = 0.11, ns) or the MTPT-C time to discontinuation (r = 0.07, ns). Similar results were noted for the relationship between the DIS and PASAT-C (r = −0.08, ns) and MTPT-C (r = −0.12, ns). Independent samples t-tests evaluating the difference between mean DIS and DTS and persistence on behavioral tasks measured dichotomously (early discontinuation or no discontinuation) revealed no significant difference between groups (t(128) = 1.37, ns, r = .12) for the MTPT-C (63.8% discontinued) and for the PASAT-C (t(128) = 0.16, ns, r = .02; 44.6% discontinued).

Study 2 Discussion

Study 2 provides evidence consistent with method variance. Self-report measures correlated significantly with other self-report measures, and behavioral measures correlated significantly with other behavioral measures; however, self-report and behavioral measures did not correlate with each other. This was the case for both continuous and dichotomous definitions of the behavioral measure outcomes.

Study 3

Cigarette smokers comprise another group in which DI has been examined extensively. For example, elevated DI has been linked to smoking status (Quinn, Brandon, & Copeland, 1996) and shortened duration of abstinence following quit attempt (Brandon et al., 2003; Brown et al., 2002; Hajek, 1991; Zvolensky, Feldner, Eifert, & Brown, 2001). Moreover, treatment focused on reducing DI has demonstrated efficacy for smoking cessation (Brown et al., 2008). Data from two studies including current cigarette smokers were analyzed. In the first study (Study 3a), individuals age 18 years or older who smoked a minimum of 10 cigarettes on average per day and had been smoking for at least 1 year were eligible for a study of an attentional training intervention for reducing attentional bias toward smoking cues (McHugh, Murray, Hearon, & Otto, 2009). In Study 3b early lapsing nicotine dependent, non-treatment seeking adults were recruited and randomized to one of the three 30-minute video interventions.

Participants

Current smokers were recruited from the Greater Boston area using internet and newspaper advertisements for 2 separate studies. In Study 3a, participants completed a battery of self-report questionnaires and the MTPT-C task prior to evaluation of attentional bias and the attentional training intervention. Fifty-four participants (14 women), with a mean age of 41 years (SD = 12.8, range = 19–66) were enrolled in the study. The sample was diverse with respect to race, with 63% Caucasian, 32% African American, 2% Asian, and 2% identifying as “other.” There was also variability in educational attainment (32% with a high school degree or less, 48% with some college, and 20% completed college or a graduate degree).

In Study 3b, 44 participants (15 women) provided informed consent and were enrolled in the study. Participants completed a battery of self-report questionnaires and three behavioral measures including the PASAT-C, the MTPT-C, and BH (the order of administration was counterbalanced). The mean age of the sample was 40.39 years (SD = 11.97, range = 18–59). Sixty one percent of the sample identified their race as Caucasian, 32% as African American, 5% as Hispanic, and 2% as other. The sample was also diverse relative to educational attainment (46% with a high school degree of less, 36% receiving some college, 16% with a college degree, and 2% with a graduate degree). There were no differences between groups in scores on any DI measures and no effect of the intervention on these measures, and thus all data are collapsed for the purpose of this analysis.

Measures

In Study 3a, participants were administered the ASI and the MTPT-C task. In addition, craving for cigarettes was evaluated as a potential clinical index of DI, as a factor that reflects desire to engage in a risky behavior that may alleviate distress. Craving was measured using a single item self-report question rated from 0–100. Results of a paired-samples t-test evaluating change in dysphoria rating before and after the task confirmed successful elicitation of distress in the MTPT-C (t(45) = 7.14, p < .001). Post-task dysphoria was not associated with persistence when controlling for pre-task dysphoria (r = 0.20, ns). Internal consistency reliability was strong from both the MTPT-C (α = .88).

Study 3b used the ASI, AAQ, and DERS in addition to three behavioral measures – the MTPT-C, PASAT-C, and breath holding. The change in dysphoria ratings was significant for both the MTPT-C (t(38) = 3.95, p < .001) and the PASAT-C (t(40) = 6.57, p < .001). As noted, no manipulation check is relevant for the BH task. In addition, post-task dysphoria was not significantly associated with persistence when controlling for pre-task dysphoria for either the MTPT-C (r = −0.14, ns) or the PASAT-C (r = −0.30, ns). Internal consistency reliability was strong from both the MTPT-C (α = .90) and the PASAT-C (α = .91).

Results

The mean scores in the first sample were 23.41 for the ASI (SD = 11.65, range = 5–62) and 159.92 for the MTPT-C (SD = 106.73, range = 0.32–300). In the first sample, the ASI was not associated with time to discontinuation in the MTPT-C (r = 0.04, ns) or with change in urge to smoke following the MTPT-C (r = 0.12, ns). Time to discontinuation in the MTPT-C was associated with the single item change in craving in the magnitude of a large effect size (r = 0.43, p < .01). An independent samples t-test evaluating the difference between the ASI and persistence on the MTPT-C measured dichotomously (76% discontinued) revealed no significant difference between groups (t(44) = 0.49, ns).

In the second sample, the self-report measures were significantly correlated with each other (see Table 3). Additionally, the PASAT-C and MTPT-C were significantly correlated in the magnitude of a large effect size (r = 0.39, p < .05), but not with BH (rs = .09 and .05 for PASAT-C and MTPT-C, respectively, both ns). Evaluation of the association between the PASAT-C and MTPT-C based on dichotomous categorization did not however result in a significant association (χ2 (1,44) = 0.49, ns; r = .11). BH could not be examined dichotomously as there was no fixed endpoint for this measurement.

Table 3.

Study 3b Correlations Between Distress Intolerance Measures and Sample Descriptive Statistics

Measure AAQ ASI DERS BH MTPT-C PASAT-C mean SD range
Self-Report
 AAQ -- 34.14 7.83 20–58
 ASI 0.46** -- 18.84 11.29 1–54
 DERS 0.57** 0.52** -- 2.37 0.86 1–5
Behavioral
 BH −0.09 0.29 0.2 -- 48.88 21.88 10–103
 MTPT-C 0.21 −0.05 0.19 0.05 -- 237.18 141.51 0.17–390
 PASAT-C 0.2 −0.12 0.09 0.09 0.39* -- 372.83 258.53 2–600
Open in a new tab

Note.

*

p<.05,

**

p<.01; AAQ = Acceptance and Action Questionnaire, ASI = Anxiety Sensitivity Index, DERS = Difficulties in Emotion Regulation Scale – persistence at goal-directed activities when distressed subscale, BH = Breath Holding, MTPT-C = Mirror Tracing Persistence Task, PASAT-C = Paced Auditory Serial Addition Task.

Correlations between self-report and behavioral measures also yielded mixed results, with the association between ASI and BH approaching but not reaching significance (r = 0.29, p = .055), but no significant association between ASI and PASAT-C (r = −0.12, ns) or MTPT-C (r = −0.05, ns). No significant associations with the behavioral measures were obtained for the DERS persistence scale and AAQ. Examining the behavioral measures dichotomously, no significant differences in self-report measure scores emerged for the PASAT-C or the MTPT-C for any of the 3 self-report measures.

Study 3 Discussion

Study 3 provides further evidence for the lack of shared variance between self-report and behavioral measures of DI in a sample of current smokers. In two independent evaluations, similar patterns were noted, with strong overlap within the self-report and behavioral measures, and poor overlap between these groups of measures.

General Discussion

Using data from four independent samples (n = 298), we examined the shared variance among self-report and behavioral measures of DI. Several self-report and behavioral measures were evaluated, reflecting the current state of the literature in this area. Results indicated that self-report measures were consistently correlated with each other across studies with effect sizes ranging from medium to large according to Cohen’s standards (rs range from .25–.57). In addition, the PASAT-C and MTPT-C, behavioral measures that involve the elicitation of frustration were consistently correlated with effect sizes ranging from medium to large (rs range from .29–.39). However, the self-report measures did not exhibit significant associations with the MTPT-C or the PASAT-C in any of the three studies in which they were administered.

A different pattern emerged for the association between the ASI and behavioral measures of DI involving somatic distress. The ASI was significantly associated with time to discontinuation in CP with a large effect size (r = .38) and moderate effect size for the BH task (r = .29). A similar relationship was not seen between the ASI and the emotional behavioral DI measures (PASAT-C and MTPT-C), or the other self-report measures and the somatic behavioral measures.

Our findings address a number of important challenges for the understanding and assessment of DI. First, our study addresses the issue of method variance – specifically whether behavioral and self-report measures share sufficient variance. Traditionally, behavioral measures have the strength of evaluating the desired construct in a more ecologically valid manner with less bias relative to participant self-awareness and efforts to please the experimenter (i.e., social desirability). Self-report measures are easy to administer, can be used efficiently, and are more flexible relative to the concepts that can be evaluated (e.g., a measure can inquire about a range of behaviors that may be difficult to measure individually). Both methods may be influenced by participant motivation – in the case of behavioral measures, relative to the motivation to engage in the task, and for self-report measures, relative to the desire to present oneself in a particular manner. In the literature evaluating the association between DI and clinical outcomes, both self-report and behavioral measures have been associated with clinical status and maladaptive coping responses, providing support for their use. However, the lack of association across methods (with the exception of anxiety sensitivity and somatic behavioral measures), suggests that additional research is needed to determine whether these measures are capturing the same construct.

One potential explanation for this inconsistency is the issue of distress domain specificity. Little research has focused on whether DI varies as a function of the type of distress elicited (e.g., pain, sadness, frustration), which may be particularly important given the domain-specificity of both behavioral and some self-report measures. Indeed, research studies often utilize DI in relation to one type of distress (e.g., frustration) as a proxy for DI more broadly; however, if DI varies across types of distress, it may be important to evaluate the domains of distress most relevant for the particular clinical/study aims. We found some evidence to support the concept of distress domain specificity. In particular, there was a lack of association between affective (e.g., PASAT-C, MTPT-C) and somatic (e.g., BH) behavioral measures. The lack of overlap among measures is consistent with past investigations in this area (e.g., Brown et al., 2002). In addition, anxiety sensitivity, which places a large focus on physical symptoms associated with anxiety, was found to share variance with somatic and not affective behavioral measures. The association between anxiety sensitivity and breath holding is consistent with conceptual models of anxiety sensitivity; however, the association with pain suggests that there may be a broader conceptual similarity relative to somatic distress. The evaluation of the importance of domain specificity in the conceptualization of DI, particularly as it may relate to different behaviors or clinical states will be an important future research direction.

In our study, we did not include an external, gold-standard referent to determine which of these disparate measures provided the most functional prediction (e.g., degree of use of maladaptive coping strategies in response to emotional distress). Hence, the current state of the literature reflects a divergence between behavioral and self-report measures. This divergence may be reflective of basic differences in the core conceptualization of DI: whether it is a construct defined primarily by distress sensitivity or task persistence while experiencing distress. Either distress sensitivity or task persistence during distress may be important for accounting for the maladaptive coping strategies that have come to define the clinical importance of assessing distress sensitivity (e.g., Anestis et al., 2007; Brown et al., 2002; Daughters, Lejuez, Kahler, et al., 2005; Quinn et al., 1996). Accordingly, until additional research shows that one of these methods outperforms the other, we recommend inclusion of both behavioral and self-report measures in studies of this construct, in particular when examining their relationship to behavioral outcomes, with particular consideration of inclusion of the ASI given its relatively strong performance in sharing variance across assessment strategies.

Given these findings, the testing of conceptual models of DI to better determine the similarities and distinctions among these constructs (e.g., anxiety sensitivity, pain tolerance, emotional tolerance) is of particular importance. It is unclear whether the failure of existing measures to converge is reflective of distinctions between these models or whether the existing assessment strategies are unable to capture variability across types of distress. The development and testing of conceptual models will be crucial to the improvement of measurement strategies in this area. Indeed, the measures described above may reflect components of a broader model of DI, such as perceived access to strategies to regulate distress, sensitivity to distress, and behavioral persistence.

Our study is limited by the failure to include all measures of DI in all the studies we evaluated. Due to this fact, we relied on correlational analyses to understand the pattern of relationships among DI measures. More robust statistical techniques (e.g., factor analysis) will be important with larger samples to assess whether the behavioral (persistence in response to affective or somatic challenges) and self-report measures are capturing the same latent construct. Furthermore, our study involved secondary analyses of existing datasets. Studies designed specifically to test conceptual models of DI and to evaluate the relative strengths of existing and novel measures will be of particular importance to elucidating the nature of this construct and determining a gold standard measurement strategy. Nonetheless, the consistency of findings with different measures in heterogeneous samples with respect to clinical features and sociodemographic characteristics enhances the confidence in our results to date.

Given the demonstrated importance of DI in both the research and treatment of a range of psychological disorders, improving its measurement is an important future direction for the field. Our results call attention to the importance of evaluating domain specificity in the assessment of DI. In addition, our results encourage further evaluation of the validity and correlates of the self-report and behavioral measures of DI currently in use. For example, studies designed to determine the relative benefits of dichotomous and continuous scoring of behavioral measures is needed to identify the superior strategy. In addition, the development of self-report measures that more directly assess behavior in response to distress, and may be less vulnerable to bias, may be an important future direction. The development and identification of consensus in the assessment of DI will facilitate ongoing research and clinical agendas in this important area.

Footnotes

1

The correlations presented in this study represent secondary analyses of datasets from four studies. The sample descriptive statistics have been reported elsewhere; however, these particular correlational analyses have not been reported in previous studies.

2

Due to the collection of salivary cortisol in this study, individuals with hormonal dysfunction or currently using steroidal medications were excluded from the study.

References

  1. Anestis MD, Selby EA, Fink EL, Joiner TE. The multifaceted role of distress tolerance in dysregulated eating behaviors. International Journal of Eating Disorders. 2007;40:718–726. doi: 10.1002/eat.20471. [DOI] [PubMed] [Google Scholar]
  2. Barlow DH, Allen LB, Choate M. Toward a unified treatment for emotional disorders. Behavior Therapy. 2004;35:205–230. doi: 10.1016/j.beth.2016.11.005. [DOI] [PubMed] [Google Scholar]
  3. Brandon TH, Herzog TA, Juliano LM, Irvin JE, Lazev AB, Simmons VN. Pretreatment task persistence predicts smoking cessation outcome. Journal of Abnormal Psychology. 2003;112:448–456. doi: 10.1037/0021-843x.112.3.448. [DOI] [PubMed] [Google Scholar]
  4. Brown RA, Lejuez CW, Kahler CW, Strong DR. Distress tolerance and duration of past smoking cessation attempts. Journal of Abnormal Psychology. 2002;111:180–185. [PubMed] [Google Scholar]
  5. Brown RA, Palm KM, Strong DR, Lejuez CW, Kahler CW, Zvolensky MJ, et al. Distress tolerance treatment for early-lapse smokers: Rationale, program description, and preliminary findings. Behavior Modification. 2008;32:302–332. doi: 10.1177/0145445507309024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Cohen J. Statistical power analysis for the behavioral sciences. 2. Lawrence Earlbaum Associates; Hillsdale, NJ: 1988. [Google Scholar]
  7. Corstorphine E, Mountford V, Tomlinson S, Waller G, Meyer C. Distress tolerance in the eating disorders. Eating Behaviors. 2007;8:91–97. doi: 10.1016/j.eatbeh.2006.02.003. [DOI] [PubMed] [Google Scholar]
  8. Daughters SB, Lejuez CW, Bornovalova MA, Kahler CW, Strong DR, Brown RA. Distress tolerance as a predictor of early treatment dropout in a residential substance abuse treatment facility. Journal of Abnormal Psychology. 2005;114:729–734. doi: 10.1037/0021-843X.114.4.729. [DOI] [PubMed] [Google Scholar]
  9. Daughters SB, Lejuez CW, Kahler CW, Strong DR, Brown RA. Psychological distress tolerance and duration of most recent abstinence attempt among residential treatment-seeking substance abusers. Psychology of Addictive Behavior. 2005;19:208–211. doi: 10.1037/0893-164X.19.2.208. [DOI] [PubMed] [Google Scholar]
  10. Daughters SB, Reynolds EK, MacPherson L, Kahler CW, Danielson CK, Zvolensky M, Lejeuz CW. Distress tolerance and early adolescent externalizing and internalizing symptoms: The moderating role of gender and ethnicity. Behaviour Research and Therapy. 2009;47:198–205. doi: 10.1016/j.brat.2008.12.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Daughters SB, Sargeant MN, Bornovalova MA, Gratz KL, Lejuez CW. The relationship between distress tolerance and antisocial personality disorder among male inner-city treatment seeking substance users. Journal of Personality Disorders. 2008;22:509–524. doi: 10.1521/pedi.2008.22.5.509. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Gratz KL, Roemer L. Multidimensional assessment of emotion regulation and dysregulation: Development, factor structure, and initial validation of the difficulties in emotion regulation scale. Journal of Psychopathology and Behavioral Assessment. 2004;26:41–54. [Google Scholar]
  13. Gratz KL, Rosenthal MZ, Tull MT, Lejuez CW, Gunderson JG. An experimental investigation of emotion dysregulation in borderline personality disorder. Journal of Abnormal Psychology. 2006;115:850–855. doi: 10.1037/0021-843X.115.4.850. [DOI] [PubMed] [Google Scholar]
  14. Hajek P. Individual differences in difficulty quitting smoking. British Journal of Addiction. 1991;86:555–558. doi: 10.1111/j.1360-0443.1991.tb01807.x. [DOI] [PubMed] [Google Scholar]
  15. Harrington N. The Frustration Discomfort Scale: Development and psychometric properties. Clinical Psychology and Psychotherapy. 2005;12:374–387. [Google Scholar]
  16. Hayes SC, Strosahl KD, Wilson KG. Acceptance and commitment therapy: An experiential approach to behavior change. New York, NY, US: Guilford Press; 1999. [Google Scholar]
  17. Hayes SC, Strosahl K, Wilson KG, Bissett RT, Pistorello J, Toarmino D, et al. Measuring experiential avoidance: A preliminary test of a working model. Psychological Record. 2004;54:553–578. [Google Scholar]
  18. Kreek MJ, Koob G. Drug dependence: Stress and dysregulation of brain reward pathways. Drug and Alcohol Dependence. 1998;51:1149–1159. doi: 10.1016/s0376-8716(98)00064-7. [DOI] [PubMed] [Google Scholar]
  19. Lejuez CW, Kahler CW, Brown RA. A modified computer version of the Paced Auditory Serial Addition Task (PASAT) as a laboratory-based stressor. Behavior Therapy. 2003;26:290–293. [Google Scholar]
  20. Lejuez CW, Paulson A, Daughters SB, Bornovalova MA, Zvolensky MJ. The association between heroin use and anxiety sensitivity among inner-city individuals in residential drug use treatment. Behaviour Research and Therapy. 2006;44:667–677. doi: 10.1016/j.brat.2005.04.006. [DOI] [PubMed] [Google Scholar]
  21. Linehan MM. Cognitive-behavioral treatment of borderline personality disorder. New York, NY, US: Guilford Press; 1993. [Google Scholar]
  22. McHugh RK, Behar E, Geem D, Gutner CA, Otto MW. Cortisol and attentional bias toward threat. 2009 doi: 10.1080/10615801003596969. Unpublished manuscript. [DOI] [PubMed] [Google Scholar]
  23. McHugh RK, Murray HW, Hearon BA, Otto MW. Attentional bias and craving in smokers: The impact of an attentional training manipulation. 2009 doi: 10.1093/ntr/ntq171. Unpublished manuscript. [DOI] [PubMed] [Google Scholar]
  24. Otto MW, Powers MB, Fischmann D. Emotional exposure in the treatment of substance use disorders: Conceptual model, evidence, and future directions. Clinical Psychology Review. 2005;25:824–839. doi: 10.1016/j.cpr.2005.05.002. [DOI] [PubMed] [Google Scholar]
  25. Peterson RA, Reiss S. Anxiety Sensitivity Index Revised manual. Worthington, OH: International Diagnostic Systems Publishing Corportation; 1992. [Google Scholar]
  26. Quinn EP, Brandon TH, Copeland AL. Is task persistence related to smoking and substance abuse? The application of learned industriousness theory to addictive behaviors. Experimental and Clinical Psychopharmacology. 1996;4:186–190. [Google Scholar]
  27. Reiss S, Peterson RA, Gursky DM, McNally RJ. Anxiety sensitivity, anxiety frequency and the predictions of fearfulness. Behaviour Research and Therapy. 1986;24:1–8. doi: 10.1016/0005-7967(86)90143-9. [DOI] [PubMed] [Google Scholar]
  28. Schmidt NB, Richey JA, Fitzpatrick KK. Discomfort intolerance: Development of a construct and measure relevant to panic disorder. Journal of Anxiety Disorders. 2006;20:263–280. doi: 10.1016/j.janxdis.2005.02.002. [DOI] [PubMed] [Google Scholar]
  29. Simons JS, Gaher RM. The Distress Tolerance Scale: Development and validation of a self-report measure. Motivation and Emotion. 2005;29:83–102. [Google Scholar]
  30. Smits JAJ, Berry AC, Tart CD, Powers MB. The efficacy of cognitive-behavioral interventions for reducing anxiety sensitivity: a meta-analytic review. Behaviour Research and Therapy. 2008;46:1047–1054. doi: 10.1016/j.brat.2008.06.010. [DOI] [PubMed] [Google Scholar]
  31. Stewart SH, Asmundson GJ. Anxiety sensitivity and its impact on pain experiences and conditions: a state of the art. Cogn Behav Ther. 2006;35(4):185–8. doi: 10.1080/16506070601090457. Review. [DOI] [PubMed] [Google Scholar]
  32. Strong DR, Lejuez CW, Daughters SB, Marinello M, Kahler CW, Brown RA. The Computerized Mirror Tracing Task Version 1. 2003 Unpublished manuscript. [Google Scholar]
  33. Tull MT, Schulzinger D, Schmidt NB, Zvolensky MJ, Lejuez CW. Development and initial examination of a brief intervention for heightened anxiety sensitivity among heroin users. Behavior Modification. 2007;31:220–242. doi: 10.1177/0145445506297020. [DOI] [PubMed] [Google Scholar]
  34. Zvolensky MJ, Feldner MT, Eifert GH, Brown RA. Affective style among smokers: Understanding anxiety sensitivity, emotional reactivity, and distress tolerance using biological challenge. Addictive Behavior. 2001;26:901–915. doi: 10.1016/s0306-4603(01)00242-8. [DOI] [PubMed] [Google Scholar]
  35. Zvolensky MJ, Otto MW. Affective intolerance, sensitivity, and processing: Advances in clinical science introduction. Behavior Therapy. 2007;38:228–233. [Google Scholar]

RESOURCES