Relations Among Behavioral and Questionnaire Measures of Impulsivity in a Sample of Suicide Attempters

Abstract

Despite the focus on impulsivity within suicide research, it remains unclear the extent to which impulsivity assessments, that purportedly tap similar constructs, show significant overlap in samples of individuals with suicidal behaviors. In a sample of 69 suicide attempters, we took a multitrait, multimethod approach to examine the relation among various questionnaire and behavioral assessments of impulsivity facets. With the exception of urgency and go-stop performance there was little evidence of concordance between questionnaire and behavioral measures. These findings suggest researchers cannot presume that measures of “impulsivity” assess similar psychological processes and that more nuanced terminology is needed.

Impulsivity is a risk factor for suicidal outcomes (e.g., suicidal ideation, suicide attempts, and death by suicide; for a review, see Brezo, Paris, & Turecki, 2006). However, impulsivity is a heterogeneous construct, believed to reflect divergent processes. Given the multifacted nature of impulsivity, and the multiple approaches used to assess this construct and its subcomponents, it is essential to test the overlap among various methods that are purportedly tapping the same psychological processes. This focus can facilitate more refined scientific communication, aid attempts to develop more precise measures of impulsivity facets, and ultimately improve the theoretical and clinical utility of research linking impulsivity with suicidal behaviors.

More specifically, impulsivity is a broad construct that has been assessed using multiple methods (e.g., self-report questionnaires, behavioral tasks) when studying maladaptive outcomes such as suicidal behaviors (e.g., Dougherty et al., 2004; Swann et al., 2005). Although “impulsivity” is a common term, there is no general consensus regarding strict definitions of this construct or its possible components. There are a multitude of definitions of impulsivity (e.g., Moeller, Barratt, Dougherty, Schmitz, & Swann, 2001), which leads to inconsistent use of the term across and within research domains and disciplines. As noted by several researchers (e.g., Cyders & Coskunpinar, 2011; Smith et al., 2007), this indistinctness hampers scientific progress on several fronts, including impeding translational approaches, obscuring theory, and hindering the development of clinical interventions.

Given the disagreements in how to best define the construct of impulsivity and its facets, it is unsurprising that researchers have developed a plethora of different assessment methods and approaches. For example, numerous studies have focused their attention on developing self-reported assessments of impulsivity. Consolidating various self-report measures through the use of exploratory and confirmatory factor analysis, Whiteside and Lynam (2001) suggested a four-factor solution for impulsivity: sensation seeking, lack of planning, lack of persistence, and urgency (acting rashly when distressed). However, a separate area of research has focused on the development of behavioral assessments of impulsivity. Numerous behavioral tasks have been developed, including tasks thought to measure inhibitory control (e.g., the Go Stop Task; Dougherty, Mathias, Marsh, & Jagar, 2005), persistence (e.g., the Mirror Tracing Persistence Task; Strong et al., 2003) risk taking (e.g., the Balloon Analogue Risk Task; Lejuez et al., 2002), and delayed discounting (e.g., the Single Key Impulsivity Paradigm; Dougherty, Mathias, Marsh, & Jagar, 2005; see Cyders & Coskupinar, 2011; Strauss & Smith, 2009 for a detailed discussion of construct validity for questionnaire and behavioral assessments).

There have been increasing efforts to organize various facets of impulsivity as measured by both questionnaire and behavioral approaches. In order to facilitate increased consilience among research groups that tend to utilize either questionnaire or behavioral measures of impulsivity, some researchers have suggested conceptual overlap among some impulsivity facets assessed by self-report measures (e.g., the UPPS facets) and behavioral indicators of impulsivity. Specifically, as noted by Dick et al. (2010) and Bechara and Van der Linden (2005), urgency may relate to measures of prepotent response inhibition (e.g., go/no go tasks). Indeed, some initial evidence suggested that performance on go/no-go tasks correlated with measures of urgency (Gay, Rochat, Billieux, d’Acremont, & Van der Linden, 2008).

However, more recent evidence suggests that consilience may be difficult between questionnaire measures and behavioral tasks thought to assess similar impulsivity facets. In a recent meta-analysis, Cyders and Coskunpinar (2011) found significant but small (significant rs ranged from .10-.13) overlap between laboratory tasks and questionnaire-based measures of impulsivity. Based on these findings, Cyders and Coskunpinar concluded that the various measures of “impulsivity” should not be referred to as measuring the same construct and researchers should carefully specify the constructs they are aiming to operationalize when using both behavioral and questionnaire-based measures of impulsivity.

The goal of the current paper is to examine the relations among various questionnaire and behavioral assessments of impulsivity in a sample of recent suicide attempters. Specifically, this multitrait, multimethod approach does not have any bearing on the “impulsivity” of the recent suicide attempt but rather illuminates the extent to which multiple measures (e.g., self-report urgency and behavioral prepotent response measures) thought to assess the same construct (e.g., inhibitory control) overlap in a high-risk sample of interest to both researchers and clinicians (i.e., suicide attempters). This information can be used to improve future research and clinical efforts involving suicidal behaviors by allowing for more informed decisions when assessing traits related to impulsivity. Further, these results can help guide attempts to design measures of impulsivity in specific clinical samples by showing strengths and shortcomings of existing approaches for assessing impulsivity.

Method

Participants and Procedure

We recruited 69 patients who presented to a Level 1 trauma hospital within 24 hours after a suicide attempt (i.e., a self-inflicted behavior with some intent to die; Silverman, Berman, Sanddal, O’Carroll, & Joiner, 2007) by overdose (81.66%), sharp-instrument (16.67%), and/or by more violent methods (6.66%), such as by gun or hanging. Exclusion criteria included the following: current acute psychotic disorder, bipolar affective disorder with active manic episode, profound psychomotor retardation, dementia, head injury, neuropathy, and pregnancy. In addition, participants could not use any alcohol or drugs within 4 days of participation (assessed via self-report and urine drug screen) given that substance use (and withdrawal from substances) can impact reaction time. Patients were recruited close to discharge and completed their participation in a private room on the psychiatric inpatient ward. The two session assessment was comprised of a comprehensive self-report based assessment and a laboratory assessment where participants completed a number of behavioral tasks of impulsivity (the tasks were counterbalanced to control for possible order effects). The laboratory assessment occurred within two days of the initial assessment. Participants were paid $90 for their time and also received additional compensation based on their overall performance on the behavioral tasks. The sample was predominately female (54%) and Caucasian (57.97%). A smaller percentage identified as African-American (33.33%), Native American (2.90%), and mixed race/ethnicity (5.80%). The average age was 32.19 (SD = 11.49, range = 18 to 57).

Measures

The UPPS Impulsive Behavior Scale-Revised (UPPS-R; Whiteside & Lynam, 2001) is a 45-item questionnaire that was used to assess four facets of impulsivity (i.e., urgency, lack of perseverance, sensation seeking, and lack of premeditation). Summed subscale scores were coded such that higher scores reflect higher levels of impulsivity. Alphas were .87-.90 for the four subscales.

The Immediate Memory Task (IMT; Dougherty, Marsh, & Mathias, 2002) measures deficits in behavioral initiation (rapid responding that occurs prior to complete processing). Commission errors are believed to reflect response initiation aspects of impulsivity because they result from incomplete stimuli processing, leading to rapid, but incorrect responses. Commission errors are found more frequently among those with psychiatric disorders characterized by impulsivity (e.g., Dougherty et al., 2003). During this 10.5 minute testing session, participants are shown a series of 5 digit numbers on a computer screen and are asked to respond if a number is the same as the previous set. Each number is presented for 500 ms, at 500 ms intervals. Non-matching stimuli include a catch stimulus (a 5 digit number which differs from the previous set by only one digit) and a filler stimulus (a novel 5 digit number). The dependent variable included the IMT ratio (commission errors proportion/correct detection proportion; Swann et al.2005) to reflect errors in behavioral initiation.

The Go Stop Task (Dougherty et al., 2005) measures behavioral response inhibition (inability to inhibit an already initiated response) and has been shown to be related to disorders characterized by impulsivity (e.g., borderline personality disorder; Coffey, Schumacher, Baschnagel, Hawk, & Holloman, 2011). In the current study, participants were presented with 3 blocks of trials, each block comprising a series of 100 randomly generated 3-digit numbers presented on a computer screen for 500 ms, at 1500 ms intervals. Participants were instructed to respond to matched number pairs, but to offer no response if an unmatched number is presented. Additionally, participants were instructed that if the second set of digits in a matched number pair changes color from black to red, they are to inhibit their response (i.e., stop trial). Consistent with previous research (e.g., Marsh, Dougherty, Mathias, Moeller, & Hicks, 2002), the dependent variable included the percentage of stop trials in which the participant was unable to successfully inhibit an already initiated response on the 150 ms delay.

The Balloon Analogue Risk Task (BART; Lejuez et al., 2002) was developed to model risk taking in the laboratory and has shown strong relations to measures of risky behaviors (e.g., problematic substance use and risky sexual behaviors; Lejuez et al., 2002). The task provides a computer model of inflating a balloon where each “breath” the subject adds to the balloon accrues money for the subject. Participants were to pump the balloon to earn as much money as possible, taking into consideration that the balloon could pop at any time (resulting in money lost). All balloons had the same probability of exploding and there were 20 balloon trials. When a balloon is pumped past it’s individual explosion point, all points in the temporary bank are lost and the next uninflated balloon is shown. At any point during each balloon, the subject can stop pumping the balloon and click the “Collect” button and the next uninflated balloon is shown. Clicking this button transfers all points from the temporary bank to the permanent bank. The dependent measure included the mean number of balloon pumps that did not explode.

The Mirror Tracing Persistence Task (MTT; Strong et al., 2003) is a task developed to measure persistence which requires participants to trace a geometric figure while observing their hand movements in a mirror. Behavioral persistence correlates with cognitive persistence measures (e.g., solving anagrams; Brandon et al., 2003) and deficits in behavioral persistence have been linked with having a substance abuse history (e.g., Quinn, Brandon, & Copeland, 1996). There were two trials: the first trial consists of a practice trial, and the second trial consists of tracing the same geometric figure as the first trial, but the width of the figure is titrated based on the participant’s performance on the practice trial (to equate level of difficulty). Participants are instructed to complete the second trial or to push the space bar to discontinue the task. Participants are allowed only seven minutes to trace the drawing. The dependent measure consists of the average duration spent on the second trial (in ms), reverse-coded, to indicate higher levels of impulsivity (less persistence).

The Single Key Impulsivity Paradigm (SKIP; Dougherty et al., 2005) measures how much a given participant values delayed rewards. Previous research has shown that individuals with disorders characterized by high levels of impulsivity (e.g., disruptive behavior disorders) value delayed rewards more than controls (e.g., Dougherty, Bjork, Harper, Marsh et al., 2003). The 20-minute SKIP session measures the rate and pattern of operant responses to receive rewards. Participants are free to click a mouse whenever they desire to receive a reward. However, the reward will be in direct proportion to the length of time since the participant’s last response (i.e., participants receive greater rewards the longer they delay their response). The primary dependent measure included the average delay to a response. This variable was log-transformed and reverse-coded to indicate higher levels of impulsivity (preference for immediate rewards). We also examined total number of responses and the longest delay, as dependent variables, and the pattern of results was similar and thus not presented.

Proposed multi-trait, multi-measure matrix. Each of the behavioral measures were linked a priori to one of the four UPPS facets. More specifically, the immediate memory task and the go-stop measure (broadly categorized as measures of prepotent response inhibition; e.g., Cyders & Coskunpinar, 2011) were linked to UPPS urgency, the mirror-tracing task (which presumably assesses lack of persistence) was linked to UPPS lack of perseverance, the BART (which is considered a measure of risk-taking propensity) was linked to UPPS sensation seeking, and the SKIP (which is a measure used to assess the rate and pattern of free operant responses for reward), a task associated with nonplanning (Marsh et al., 2002; Swann, Bjork, Moeller, & Dougherty, 2002), was linked to UPPS lack of premeditation. Notably, these measures could ostensibly be matched in various ways that suggest different multitrait, multimethod arrangements than is currently proposed. This arrangement was chosen based on previous research (i.e., go-stop performance correlates with UPPS urgency; Gay et al., 2008) as well as matching measures that descriptively assess similar constructs (e.g., the mirror-tracing task and UPPS lack of perseverance are thought to assess persistence/perseverance).

Results

Examination of the distribution of the IMT revealed one statistical outlier (i.e., a value above the upper bound of the interquartile range); this participant was removed from analyses involving the IMT. Correlations among the various measures of impulsivity (as well as means and standard deviations of the measures) are shown in Table 1. In Table 1, the bottom left corner displays convergent validity correlations (monotrait-heteromethod), the top right corner displays discriminant validity correlations (heterotrait-heteromethod), and the upper left and lower right corners display correlations by method (i.e., behavioral task or self-report; heterotrait-monomethod). With the exception of the significant correlation between UPPS urgency and Go-Stop, none of the proposed monotrait heteromethod correlations were significant. In fact, the only other significant correlation across tasks was between IMT and MTT. Notably, though some measures are broadly thought to measures similar constructs (e.g., IMT and go-stop), only IMT and MTT (thought to assess distinct facets of impulsivity) showed a significant correlation among the behavioral measures of impulsivity.

Table 1.

Multitrait multimethod matrix of self-report and behavioral assessments of impulsivity

	Self-Report				Behavioral Task
	Heterotrait-Monomethod				Heterotrait-Heteromethod
	URGa	PERb	SSc	PREd	IMTa	GOSTa	MTTb	BARTc	SKIP
URGa							−.10	.09	.05
PERb	.12				.15	−.15		−.09	.00
SSc	.26*	−.30*			.01	.22	−.05		.01
PREd	.28*	.51***	−.13		.19	−.05	−.24	−.05
	Monotrait-Heteromethod				Heterotrait-Monomethod
IMTa	−.06								.14
GOSTa	.31*				.12				−.08
MTTb		−.17			.29*	−.13			−.01
BARTc			.12		.01	.08	−.06		−.08
SKIP				.13
Mean	32.04	22.87	25.28	28.05	4.85	38.93	127604.92	36.87	3.09
SD	8.84	7.22	9.85	7.27	2.07	23.47	148425.01	14.33	1.72

Note. Self-report and behavioral tasks hypothesized to assess the same facet of impulsivity share the same subscript. Bottom left corner displays convergent validity correlations (monotrait-heteromethod), top right corner displays discriminant validity correlations (heterotrait-heteromethod). SD = Standard Deviation. URG = UPPS - negative urgency; PER = UPPS - lack of perseverance; SS = UPPS - sensation seeking; PRE = UPPS - lack of premediation; IMT = immediate memory task; GOST = go stop impulsivity paradigm; MTT = mirror tracing task; BART = balloon analogue risk task. SKIP = single key impulsivity paradigm.

^*p < .05

^**p < .01

^***p < .001. N = 58-69.

Discussion

Although “impulsivity” is often referenced in clinical and research settings and has been indicated as an important risk factor contributing to suicidal behaviors, there is a lack of agreement regarding precise definitions, or ideal measurement approaches, of this construct. Further, it remains unclear the extent to which a relatively broad battery of these measures correlate within specific clinical samples, such as recent suicide attempters. With the exception of performance on the Go-Stop and UPPS urgency, our multitrait, multimethod table showed little evidence of significant overlap among measures that used different methods (i.e., questionnaire vs. behavioral tasks) to ostensibly assess similar underlying constructs.

Consistent with the findings from a community sample (Gay et al., 2008), we found evidence that the percent of errors of stop trials within the Go-Stop task significantly correlated with scores from the UPPS urgency scale. Taken together, these findings are supportive of Bechara and Van der Linden (2005) suggestion that urgency may relate to the ability to inhibit prepotent responses. Further, Cyders and Coskunpinar (2011) meta-analytic study showed the only significant correlation between (negative) urgency and behavioral task performance involved measures of prepotent response inhibition, including performance on Go-Stop tasks. However, this relation was also based on studies that used continuing performance tasks, such as the IMT. In this sample the correlation between the IMT and UPPS urgency was small (r = -.06) and statistically non-significant. Thus the support for Bechara and Van der Linden contention that urgency relates to measures of prepotent response inhibition may vary by the specific measure used to assess this form of inhibition.

Sensation seeking is typically positively or non-significantly correlated with other UPPS measures (e.g., Smith et al., 2007); however UPPS sensation seeking was negatively and significantly correlated with UPPS lack of perseverance in this sample. One plausible explanation for this exampled proffered by Gay et al. (2008) is the somewhat nebulous interpretation of sensation seeking scores. More specifically, identical scores on sensation seeking may suggest a healthy openness to novel experiences or risk-taking tendencies. Therefore, correlations between sensation seeking and other UPPS measures may vary depending on sample characteristics where sensation seeking may reflect different dispositions.

In addition to a general lack of overlap between questionnaire and behavioral measures of impulsivity, there was also one significant correlation among the behavioral measures of impulsivity (i.e., the MTT and the IMT). This correlation is surprising given these measures are thought to assess different impulsivity facets (persistence vs. prepotent response inhibition); future studies are needed to determine the robustness of this finding. However, the Go-Stop and IMT, which exhibited a small (r=.10) and statistically non-significant correlation, have both been considered to be measures of prepotent response inhibition. Thus, researchers seeking to assess the construct of prepotent response inhibition should use multiple instruments as well as multiple methods in order to maximize construct validity.

Several researchers have questioned whether monothetic versus construct representation approaches to assessing impulsivity should be expected to show significant overlap (e.g., Cyders & Coskunpinar, 2011), even in instances when these measures are thought to tap similar psychological processes. A common perception is that questionnaire-based measures are thought to assess more “trait” levels of impulsivity and its related components whereas behavioral tasks are thought to be more relevant to “state” impulsivity (see Cyders & Coskunpinar, 2011; Dougherty et al., 2009). Regardless of the validity of this perspective, the issue remains that the distinct processes assessed by these various measures are often referenced with similar terms (e.g., MTT lack of persistence/UPPS lack of perseverance). Echoing the sentiments of Cyders and Coskunpinar, we believe these data strongly suggest that more nuanced and distinct terms should be used in research focusing on the link between “impulsivity” and suicidal behaviors. For example, rather using the broad term “impulsivity” proponents of the UPPS refer to urgency, lack of precontemplation, lack of persistence, and sensation seeking; similar precision should be used when referring to behavioral measures of “impulsivity”.

There are several limitations of the current study. The modest sample size precluded other analytic approaches to determine the concurrent/discriminant validity, such as exploratory/ confirmatory factor analysis. However, given the limited overlap among these measures it is unclear whether a coherent factor structure could be fit to these measures, regardless of sample size. Further, though the modest sample size may have resulted in reduced statistical power to detect significant relations among the measures, the proposed monotrait-heteromethod correlations were generally small in magnitude. Thus, with the exception of performance on the Go-Stop and UPPS urgency, the current findings cast doubt on the proposition that these measures are tapping similar impulsivity facets, apart from statistical power. Although some of the measures could be matched in the multitrait, multimethod table based on description (e.g., the MTT [thought to measure lack of persistence] with UPPS lack of perseverance), other measures had less definitive links (e.g., UPPS sensation seeking and the BART [an accepted measure of risk taking]). Regardless of organization there was little evidence of overlap among the questionnaire and behavioral tasks.

We want to explicitly state that the focus of this work is on the overlap among indices of impulsivity in a sample of recent suicide attempters; the current findings do not address, in anyway, the extent to which the suicide attempt (the act itself) was “impulsive”. Indeed, neither retrospective nor prospective studies (see Brezo et al., 2006) linking traits related to impulsivity to suicide attempter status will inform, or suggest, that any particular attempt was necessarily “impulsive”. Rather, these studies suggest that heightened levels of impulsivity are, at the very least, a distal risk factor for suicidal outcomes (see Brezo et al., 2006).

In sum, these findings suggest researchers interested in assessing impulsivity among individuals with suicidal behaviors should not assume that various available measures of impulsivity assess similar psychological processes. As a corollary, increased precision should be used when referring to impulsivity-related constructs to highlight the apparent multifaceted nature of this broad construct.