Evaluation of the Psychometric Properties of the Multidimensional Revenge Attitudes Inventory-21

Abstract

Despite a few hypothesized associations between revenge and suicide in the suicide literature, the potential of revenge as a multidimensional construct related to suicide has remained unexplored. Using data from undergraduate samples across 2 studies, we examined support for the psychometric properties and nomological network of scores on the Multidimensional Revenge Attitudes Inventory-21 (MRAI-21), a new self-report instrument composed of 3 dimensions: craving for revenge, revenge rumination, and suicide-related revenge. Results from Study 1 (N = 510), suggested that a 3-factor oblique solution obtained through contemporary factor analytic methods provided the best fit for the sample data. Estimates of internal consistency reliability for the MRAI-21 scale scores were above .90. In Study 2 (N = 380), we examined internal consistency reliability estimates for 6 concurrent self-report measures and conducted convergent validity analyses using latent variable modeling with scores on the MRAI-21 and concurrent measures. Results showed that scores on all instruments had adequate estimates of reliability and revealed a unique network of correlates for each of the MRAI-21 scale scores. Findings suggest that revenge can be measured as a multidimensional construct within the context of suicide; future directions and clinical implications are discussed.

An individual’s desire for revenge stems from the perception of having been wronged and wanting to harm the person who has harmed them; what constitutes a perceived wrongdoing is subject to social norms and power imbalances between the victim and transgressor (Stillwell, Baumeister, & Del Priore, 2008; Strelan, Weick, & Vasiljevic, 2014; Stuckless & Goranson, 1992). As such, an act of revenge can be considered a form of punishment that either communicates societal norms or imposes physical or emotional suffering upon a transgressor (e.g., suicide and self-harm; Crockett, Özdemir, & Fehr, 2014). The desire for revenge, where punishment is directed toward the self (e.g., suicide and nonsuicidal self-injury [NSSI]) as a means to “get back” at a transgressor is the focus of this paper. This desire can either manifest overtly (harming oneself physically) and/or covertly (thinking of harming myself physically). Here, we focus on the desire for revenge as a form of covert behavior to the extent that the individual victim engages in a range of cognitive and emotional processing.

Notably, in an early conceptualization of revenge-related suicides, Furst and Ostow (1965) posited that suicide can serve as a method to release aggression without physically harming a hated or disappointing transgressor by redirecting anger to the self. In previous investigations, a study of undergraduates found that those who engaged in nonsuicidal self-injury (NSSI) endorsed desires for revenge and autonomy as interpersonal functions of their self-injurious behaviors (Klonsky & Glenn, 2009). Interviews with adolescent psychiatric inpatients exposed individual and relationship dimensions as the two general themes of suicide attempts, with the relationship dimension containing a subtheme of revenge. The purpose of suicide attempts in those investigations was to convey a message of contempt, suffering, and guilt to those who had caused emotional distress, in hopes of inflicting permanent remorse on the transgressor (Orri et al., 2014). Suicide, like revenge, has also been linked to feelings of powerlessness. As an example, in some Pacific Islander societies, a subordinate or otherwise powerless individual may kill oneself as a final display of power (Booth, 1999; Counts, 1984). Regarding the relationship between anger and suicide, Goldston et al. (1996) found that youth who made previous but no recent suicide attempts possessed more trait anger than first-time attempters, repeat attempters, and nonsuicidal youth.

Further, anger directed inward as a motive for suicide has been supported in a qualitative examination of notes written before both completed and attempted suicides, controlling for sex and age (Lester, 1994). Additionally, anger has been strongly associated with revenge and suicide, respectively, in other studies (Bies, Tripp, & Kramer, 1997; Buss, 1961; Eisenberger, Lynch, Aselage, & Rohdieck, 2004; Stenstrom, Lickel, Denson, & Miller, 2008). Unlike externalized anger, revenge can be conceptualized as an emotional state as opposed exclusively to an overt behavioral response to perceived wrongdoing (Schumann & Ross, 2010). These studies imply that there is a considerable link, rather than a causal relationship, between internalized anger (i.e., thoughts–cognitions and emotions–affect), suicide, and revenge that is worth examining.

Though the desire for revenge-related suicide has been indirectly discussed in these studies, it has not received an extensive empirical examination. However, a recent study by Abbas, Mohanna, Diab, Chikoore, and Wang (2018) introduced the Ideal Typical Meanings Questionnaire (ITMQ) to enhance our understanding of the motives for suicide attempts and self-injury. Specifically, the construction of this new instrument was based on Baechler’s (1979) 11 types of meanings and motives for suicide and suicidal behaviors, one of which is vengeance. Three items in the ITMQ explore vengeance/retaliation-based motivation for suicide (e.g., “My aim in killing/harming myself was to have another person blamed for my death”). The ITMQ is the first known empirical measure designed to explore further our understanding of the relationship between revenge and self-harm behaviors. While other self-report measures of revenge exist in the literature, they do not tap into the construct of suicide-related revenge. For example, a commonly used measure of revenge, the Vengeance Scale (Stuckless & Goranson, 1992), conceptualizes revenge as a unidimensional construct; the items are not designed to assess a range of suicide-related behaviors.

Moreover, the advent of the American TV drama series 13 Reasons Why has recently brought public attention to suicide-related behaviors. Specifically, the series highlights the relevancy of revenge in popular media, particularly in young adult populations. Some researchers posit that the TV series glamorizes suicide, as it portrays events leading up to a revenge-motivated death by suicide that successfully induces guilt and remorse among members of a high school community (Polanczyk, 2017). A recent study found that in the month following the show’s debut in early 2017, adolescent suicides spiked nearly 30% more than in any other month recorded over the previous 5 years (Bridge et al., 2020). This finding highlights the relevance and urgency of studying revenge-motivated suicide.

Rising concerns about the link between suicidal behaviors and revenge call for a closer examination of these constructs. To address this gap in the literature, Osman, Gutierrez, Bagge, Freedenthal, and Wong (2015) constructed the Multidimensional Revenge Attitudes Inventory–21 (MRAI-21), a 21-item instrument to examine three dimensions of the revenge and suicide-related construct. All items are scaled from 1 (strongly disagree) to 5 (strongly agree). Briefly, the researchers conceptualized three dimensions for the MRAI-21, including revenge rumination or reflection, craving for revenge, and suicide-related revenge.¹

Revenge Rumination or Reflection

This dimension draws on the construct of rumination to highlight the excessive or repetitive expression of the revenge construct. Indeed, there is extensive support for the construct validity of the rumination construct in the extant literature. As a cognitive process, rumination is generally conceptualized as repetitive self-focused thinking that tends to be directed at a specific life event or condition such as depression and perfectionism. Several reviews and studies have found a relationship between rumination and suicide-related behaviors (see, e.g., Morrison & O’Connor, 2008; Smith, Alloy, & Abramson, 2006). Within the MRAI-21, the seven-item Revenge Rumination dimension taps several emotional pain-related responses that are experienced in interpersonal interactions and can, therefore, be conceptualized as a risk factor for suicide (Gäbler & Maercker, 2011). Example items are: “When someone hurts my feelings, I spend a lot of time thinking about it” and “When someone humiliates or shames me, it takes quite a bit of time for me to get over it.”

Craving for Revenge

The Craving for Revenge dimension is conceptualized as the intense and compelling desire to get back at someone, in particular, because of an overt interpersonal transgression (i.e., basic equity theory; see, Walster, Walster, & Berscheid, 1978). As an example, Gabriel and Monaco (1994) explored several manifestations of the craving for revenge construct to include both adaptive and maladaptive components within the general affect theory. Maladaptive revenge, unlike adaptive revenge, is seen as a compulsion to get even with someone, generally resulting in at-risk behaviors. Several clinical case analyses and research studies have also highlighted the dimension of desire for revenge in interpersonal situations (see, e.g., Arlow & Baudry, 2002). Thus, within the MRAI-21, the seven-item Craving for Revenge dimension serves as an interpersonal risk factor for suicide. Sample items include: “I am not satisfied unless I can find a way to get back at the person who hurts my feelings” and “Finding a way to get back at someone who hurts my feelings will keep me from feeling humiliated.”

Suicide-Related Revenge

Clinical researchers have paid relatively little attention to developing instruments that could enhance our understanding of the link between intense desire for revenge and specific suicide-related behaviors, including repeated suicide attempts and threats. The only empirical measure of this link is the newly developed ITMQ (Abbas et al., 2018). Distinct from this instrument, the seven-item Suicide-Related Revenge dimension of the MRAI-21 draws extensively on research that has focused on suicide as vengeance within interpersonal relationships (see, e.g., Davis, Callanan, Lester, & Haines, 2009; Hjelmeland et al., 2002). Sample items from the Suicide-Related Revenge dimension read, “I have thought about ways to kill myself in order to make someone feel intense guilt” and “When someone harms me on purpose, I experience a strong desire to kill myself as the only way to get even.”

Overview of the Construction of the Multidimensional Revenge Attitudes Inventory

Osman et al. (2015) conducted several pilot studies in the development of the MRAI-21. A team of graduate students, undergraduate students, and lab members generated an initial pool of 45 items after the three dimensions were specified. The researchers also reviewed items from several self-report measures of psychopathology, suicidal behavior, and interpersonal relationships. In the first data reduction phase (N = 682, males = 287 and females = 395), items were rated for relevancy and representativeness and subjected to incremental content validity analysis (regression analysis), internal consistency reliability tests, and principal components analysis. Item–response theory modeling was used further to determine the final 21-item version of the instrument. Also, content analyses demonstrated a Flesch-Kincaid grade level of 9.6. In a subsequent exploratory principal-axis factor analytic study (N = 432, males = 156 and females = 276), parallel analysis (Promax oblimin rotation) and minimum average partial indicated a three-factor solution, with factor loadings above .60 and low to moderate factor intercorrelations (.24 to .42). Together, the three factors accounted for 61.08% of the variance in scores. Using exploratory structural equation modeling (ESEM; Asparouhov & Muthén, 2009) with the same data, a three-factor solution provided strong fit estimates (CFI; comparative fit index = .991, TLI; Tucker-Lewis Index = .998, RMSEA; root mean square error of approximation = .044; 90% CI [.036, .052]) and demonstrated low factor intercorrelations (range = .18 to .31).

To evaluate the extent to which items are specific to the identified MRAI-21 dimensions, Osman et al. (2015) examined the fit of an exploratory bifactor model as an alternative to the original oblique three-factor solution. Specifically, the typical bifactor model includes a general (trait, “g”) factor on which all the items within the target instrument are constrained to load, and a set of orthogonal group factors (“s”) that represent the specific dimensions of the target construct. Results of their analyses further showed that the variance accounted for by each group factor was independent of the variance from the general factor (explained common variance = 0.35), providing support for the specificity of the MRAI-21 scale scores. Given the strong psychometric properties reported for the instrument development samples, we expected that similar estimates would be obtained for scores on the MRAI-21 scales for the current samples.

Overview of Present Research

The MRAI-21 is the first multidimensional instrument that allows for the empirical evaluation of the relationship between revenge attitudes and suicide-related thoughts and behaviors. As noted previously, the dimensions of the instrument were conceptualized globally as behavioral processes, not limited to overt behaviors. The present research sought to replicate and expand upon findings from the instrument development studies with the MRAI-21 (Osman et al., 2015). Specifically, the overarching goals of the current analyses with the MRAI-21 were to analyze the factor structure, report on the preliminary psychometric properties, and explore a network of potential correlates for scale scores for the MRAI-21.

Study 1 aims to examine further the factor structure of the MRAI-21 as obtained from ESEM, in addition to reporting estimates of internal consistency reliability for the MRAI-21 scale scores in an undergraduate sample. Concurrent measures included in Study 2 allowed for an exploratory examination of the evidence for convergent validity of MRAI-21 scale scores using bivariate correlations and multiple linear regression models in an independent undergraduate sample.

Study 1 (N = 510)

Method

Objectives.

The objectives for Study 1 were to reexamine the factor structure of the MRAI-21 using ESEM and to obtain estimates of scale score internal consistency reliability.

Participants and procedure.

A sample of 510 college students was recruited through an undergraduate psychology subject pool at a large university in the southwestern United States. Participants provided informed consent and completed a randomly ordered battery of self-report instruments, including the MRAI-21 and a demographics questionnaire. During each session, approximately two–three trained graduate and undergraduate students administered all the questionnaires in paper–pencil format. During the data cleaning sessions, we excluded questionnaire packets with three or more items left unanswered for any of the instruments. Mean age for the sample was 21.85 years (SD = 5.75 years, range 18 to 61 years). Self-identified females and males respectively constituted 63.7% (n = 325) and 36.3% (n = 185) of the sample (see Table 1). The university’s Institutional Review Board approved all the study procedures.

Table 1.

Demographics for Study 1 and Study 2

Variable	Study 1 (N = 510)	Study 2 (N = 380)
Age
Mean	21.86	19.89
SD	5.75	3.22
Minimum	18	18
Maximum	61	50

	N	%	N	%
Sex
Male	185	36.3	131	34.5
Female	325	63.7	249	65.5
Ethnicity
White/European	140	27.5	101	26.6
African American/Black	43	8.4	33	8.7
Asian American	34	6.7	30	7.9
Hispanic/Latino American	252	49.4	183	48.2
Middle Eastern	9	1.8	3	0.8
Biracial/Multiethnic	26	5.1	28	7.4
Other	6	1.2	2	0.5
Sexual orientation
Gay	8	1.6	4	1.1
Lesbian	12	2.4	7	1.8
Bisexual	19	3.7	22	5.8
Heterosexual	450	88.2	340	89.5
Other/Unsure	21	4.1	7	1.8
Year in college
First year	160	31.4	218	57.4
Second year	80	15.7	80	21.1
Third year	106	20.8	50	13.2
Fourth year/Graduated	164	32.2	32	8.4

Plan of data analysis.

As aforementioned, analyses from the pilot studies provided evidence that a three-factor oblique solution was the best fit for the 21 items included in the MRAI-21 (Osman et al., 2015). Thus, we elected to explore and compare a one-factor, a two-factor, and a three-factor ESEM model for the sample data in the Mplus 7.2 (Muthén & Muthén, 1998 –2015) program. ESEM was chosen over traditional factor analytic methods such as confirmatory factor analysis (CFA), because of the limitations of these traditional methods. For instance, CFA models restrict items to just one factor, which can result in inflated and biased parameter estimates that reduce discriminant validity (Asparouhov & Muthén, 2009; Marsh et al., 2009; Marsh, Morin, Parker, & Kaur, 2014; Schmitt & Sass, 2011). Further, a major advantage of employing ESEM for factor analysis is that this method includes elements of both exploratory factor analysis (EFA) and CFA, allowing for a more robust analysis. Specifically, ESEM adopts a rigorous EFA procedure in which all of the MRAI-21 items loaded onto the related factors within an SEM framework. In contrast to CFA, interfactor correlations obtained from ESEM are less prone to inflation; thus, the factors extracted within ESEM tend to be distinct. Also, items are not constrained to zero loadings on nontarget factors as in CFA, making it possible to assess complex items that cross load onto more than one factor (Asparouhov & Muthén, 2009; Marsh et al., 2014). Thus, we applied ESEM to the sample data.²

We first examined the univariate and multivariate distributions of the individual item responses. Results from both the Kolmogorov-Smirnov and Shapiro-Wilk tests of univariate normality indicated that the distribution of scores on all 21 items was non-normal. Additionally, the normalized estimate of Mardia’s coefficient was found to be significantly different from zero, suggesting non-normality at the multivariate level (χ² = 181.04, p < .0001). To account for the non-normal distribution of our data, we conducted ESEM with maximum likelihood estimator with robust standard errors (MLR).³ We specified an oblique rotation due to the expected low to moderate correlations between each of the MRAI-21 dimensions. The fit of the models was evaluated using multiple goodness-of-fit indices. As recommended in the literature, these indices include the comparative fit index (CFI values > .90), the Tucker-Lewis Index (TLI values > .90), and the root mean square error of approximation (RMSEA, values ≤.06; see Browne & Cudeck, 1993; Lance, Butts, & Michels, 2006; Tucker & Lewis, 1973).

We computed means, standard deviations, and internal consistency reliability estimates with coefficient alpha using latent variable modeling (α; Raykov & Marcoulides, 2015) and coefficient omega (ω, McDonald, 1999). These estimators were chosen over Cronbach’s alpha (Cronbach, 1951) in light of findings that the latter tends to be an inaccurate measure of internal consistency (Sijtsma, 2009). Cronbach’s alpha estimates are inflated because the composition of true score, which includes true score and measurement error, is ignored and the assumption of tauequivalence (e.g., all unidimensional items have the same factor loading) is almost always violated. The alternate reliability estimates correct for these issues (Sijtsma, 2009). Both reliability estimates were computed with bootstrapping with 2,000 iterations to obtain 95% confidence intervals (CI). Additionally, the Akaike information criterion (AIC) and the range of correlations for each item with all other scale items (item-total r) were computed for scores on each MRAI-21 dimension.

Results

Descriptive statistics and internal consistency reliability.

Estimates for internal consistency reliability (coefficient-α and coefficient-ω), Akaike information criterion (AIC), and a range of item-total correlations (item-total r) are reported in Table 2. All of the MRAI-21 scale scores exhibited high estimates of internal consistency, with coefficient-α and coefficient-ω values well above the .70 cutoff recommended for research utility (Cicchetti, 1994). Specifically, coefficient-α = .92, 95% CI [.90, .93] and coefficient-ω = .93, 95% CI [.91, .94] were computed for the Craving for Revenge scale scores, coefficient-α and ω = .92, 95% CI [.91, .93] were computed for the Revenge Rumination scale scores, and coefficient-α = .90, 95% CI [.88, .92] and coefficient-ω = .92, 95% CI [.91, .94] were computed for the Suicide-Related Revenge scale scores. The Akaike information criterion (AIC) values for all scale scores ranged from .62 to .65. The item-total correlations ranged from r = .67 to .78 for Revenge Rumination, from r = .68 to .78 for Craving for Revenge and from r = .60 to .82 for Suicide-Related Revenge.

Table 2.

Means, Standard Deviations, and Three Alternative Estimates of Internal Consistency Reliability of Scale Scores for Study 1 and Study 2

Scale	M	SD	α [95% CI]	ω [95% CI]	AIC	Interitem r
Study 1 (N = 510)
MRAI-21 CR	10.56	4.95	.92 [.90, .93]	.93 [.91, .94]	0.65	[.68, .82]
MRAI-21 RR	18.38	7.33	.92 [.91, .93]	.92 [.91, .93]	0.62	[.67, .78]
MRAI-21 SRR	8.53	3.73	.90 [.88, .92]	.92 [.91, .94]	0.62	[.60, .82]
Study 2 (N = 380)
MRAI-21 CR	10.54	4.79	.92 [.89, .93]	.93 [.91, .94]
MRAI-21 RR	19.26	7.48	.92 [.91, .94]	.93 [.92, .94]
MRAI-21 SRR	8.67	3.98	.95 [.93, .97]	.96 [.95, .97]
MSRI-21 MBT	19.29	7.84	.94 [.93, .95]
MSRI-21 FSC	13.03	5.10	.85 [.83, .87]
MSRI-21 NSE	9.72	5.49	.96 [.95,.97]
PID-5-BF NAF	8.17	4.70	.79 [.75, .82]
PID-5-BF DTC	4.99	3.93	.70 [.64, .74]
PID-5-BF ANT	3.56	3.42	.68 [.62, .73]
PID-5-BF DIS	4.82	4.12	.76 [.72, .80]
PID-5-BF PSY	6.07	4.60	.79 [.75, 82]
SA-45 ANX	8.53	4.03	.80 [.77, .83]
SA-45 DEP	10.73	5.45	.91 [.89, .92]
SA-45 HOS	7.14	3.26	.82 [.79, .85]
SA-45 INT	10.03	5.20	.88 [.86, .90]
SA-45 OCD	11.58	5.13	.85 [.82, .87]
SA-45 PAR	9.49	4.35	.79 [.75, .82]
SA-45 PHO	7.52	3.99	.85 [.83, .87]
SA-45 PSY	7.08	3.05	.71 [.66, .75]
SA-45 SOM	8.43	3.90	.81 [.78, .84]
TRIM Rev	2.55	0.89	.90 [.88,.90]
TRIM Avo	2.54	0.89	.91 [.90,.92]
VS-20 Total	57.83	19.16	.91 [.90, .93]
SBQR Total	5.14	2.70	.81 [.77,.84]

Note. α = coefficient α; ω = coefficient ω; CI = confidence interval; AIC = average interitem correlation. MRAI-21 = Multidimensional Revenge Attitudes Inventory–21; MRAI-21 CR = Craving for Revenge; MRAI-21 RR = Revenge Rumination; MRAI-21 SRR = Suicide-Related Revenge; MSRI = Multidimensional Shame Related Inventory; MSRI-21 MBT = Maladaptive Behavior Tendencies; MSRI-21 FSC = Fear of Social Consequences; MSRI-21 NSE = Negative Self-Evaluation; PID-5B-F = Personality Inventory for the DSM–5 Brief Form; PID-5-BF NAF = Negative Affectivity; PID-5-BF DTC = Detachment; PID-5-BF ANT = Antagonism; PID-5-BF DIS = Disinhibition; PID-5-BF PSY = Psychoticism; SA-45 = Symptom Assessment-45; SA-45 ANX = Anxiety; SA-45 DEP = Depression; SA-45 HOS = Hostility; SA-45 INT = Interpersonal Sensitivity; SA-45 OCD = Obsessive Compulsive Disorder; SA-45 PAR = Paranoid Ideation; SA-45 PHO = Phobic Anxiety; SA-45 PSY = Psychoticism; SA-45 SOM = Somatization. TRIM = Transgression-Related Interpersonal Motivations; TRIM Rev = Revenge Motivations; TRIM Avo = TRIM Avoidance Motivations; VS-20 Total Score = Vengeance Scale-20 Total Score; SBQR Total = Suicidal Behaviors Questionnaire Revised Total Score.

Exploratory structural equation modeling.

Estimates for the one-factor ESEM model yielded a poor fit to the sample data. While the two-factor ESEM model outperformed the one-factor model in terms of model fit, the two-factor model still yielded poor model fit (see Table 3). In contrast, estimates for the oblique three-factor ESEM model provided an excellent fit for the sample data, where χ² = 333.07 (df = 150), CFI = .95, TLI = .93, and RMSEA = .05, 90% CI [.04, .06]. The standardized factor loadings from the ESEM for the three-factor model for the Revenge Rumination factor ranged from .66 to .83, for the Suicide-Related Revenge factor ranged from .58 to .89, and for the Craving for Revenge ranged from .64 to .83. Finally, standardized factor intercorrelations were low to moderate: Revenge Rumination–Craving for Revenge = .27, Revenge Rumination–Suicide-Related Revenge = .20, and Craving for Revenge–Suicide-Related Revenge = .14.

Table 3.

Goodness of Fit Indices for ESEM for the Multidimensional Revenge Attitudes Inventory–21

						90% CI
Model	χ2	df	CFI	TLI	RMSEA	LL	UL
1-Factor	2214.72	189	0.44	0.38	0.15	[0.14,	0.15]
2-Factor	1338.43	169	0.68	0.60	0.12	[0.11,	0.12]
3-Factor	333.07	150	0.95	0.93	0.05	[0.04,	0.06]

Note. N = 510. CFA = confirmatory factor analysis; ESEM = exploratory structural equation modeling; df = degrees of freedom; CFI = comparative fit index; TLI = Tucker-Lewis index; RMSEA = root mean square error of approximation; LL = lower limit; UL = upper limit.

A table is included for Study 1 in the online supplemental materials. Table 1 displays the standardized factor loadings from the retained correlated three-factor ESEM model, including factor intercorrelations.

Study 2 (N = 380)

Method

Objectives.

Study 2 explored a preliminary nomological network for the MRAI-21 scale scores as demonstrated through bivariate correlations with scores on empirically and rationally related instruments and through multiple linear regression modeling.

Participants and procedure.

A sample of 380 college students was recruited through an undergraduate psychology subject pool at a large university in the southern United States. Participants provided informed consent and anonymously completed a randomly ordered battery of instruments, including the Multidimensional Revenge Attitudes Inventory-21 and a demographics questionnaire, for course credit. All responses were obtained via the online survey software Qualtrics. The mean age for the sample was 19.89 years (SD = 3.22 years, range 18 to 50 years). Self-identified females and males respectively constituted 65.5% (n = 249) and 34.5% (n = 131) of the sample (see Table 1). The university’s Institutional Review Board approved all procedures.

Measures.

Multidimensional Shame Response Inventory-21 (MSRI-21; Garcia, Acosta, Pirani, Edwards, & Osman, 2017).

The MSRI-21 is a 21-item measure of affective and behavioral responses to shame scaled from 1 (Strongly Disagree) to 5 (Strongly Agree). The measure assesses shame-related responses to Negative Self-Evaluation (NSE; e.g., “When I experience feelings of shame I often feel worse about myself”), Maladaptive Behavior Tendency (MBT; e.g., “When I experience feelings of shame, I react intensely by harming myself physically and on purpose …”), and Fear of Social Consequences (FSC; e.g., “Because of fear of being ashamed, I do not see myself as ever relying on another person for help or support”). Each of the three scales is scored as the sum of its items. In recent studies, the MSRI-21 scale scores have been shown to demonstrate good internal consistency, a clear three-factor structure, and strong evidence of convergent and discriminant validity (Osman, Freedenthal, Bagge, Gutierrez, & Wong, 2017; Pirani, Garcia, Acosta, & Osman, 2016). For the current study, coefficient-α = .96, 95% CI [.95, .97] for the Negative Self-Evaluation, coefficient-α .85, 95% CI [.83, .87] for the Fear of Social Consequences, and coefficient-α = .94, 95% CI [.93, .95] for the Maladaptive Behavior Tendency scale scores.

Personality Inventory for the DSM–5 Brief Form (PID-5-BF; Krueger, Derringer, Markon, Watson, & Skodol, 2013).

The PID-5-BF is a brief 25-item self-report measure of DSM–5 (American Psychiatric Association, 2013) pathological personality traits scaled from 0 (very false or often false) to 3 (very true or often true). The PID-5-BF measures the following pathological personality traits: detachment (DTC; e.g., “I often feel like nothing I do really matters”), disinhibition (DIS; e.g., “I feel like I act totally on impulse”), psychoticism (PSY; e.g., “My thoughts often don’t make sense to others”), antagonism (ANT; e.g., “It’s no big deal if I hurt other people’s feelings”), and negative affectivity (NAF; e.g., “I worry about almost everything”). Each of the scales is scored as the sum of its items. In a recent review of the original PID-5, Al-Dajani, Gralnick, and Bagby (2016) discussed that the PID-5-BF scale scores demonstrated adequate reliability estimates across multiple studies. For the current study, coefficient-α = .79, 95% CI [.75, .82] for PID-5-BF NAF; coefficient-α = .70, 95% CI [.64, .74] for PID-5-BF DTC; coefficient-α = .68, 95% CI [.62, .73] for PID-5-BF ANT; coefficient-α = .76, 95% CI [.72, .80] for PID-5-BF DIS; and coefficient-α = .79, 95% CI [.75, .82] for PID-5-BF PSY.

Suicidal Behaviors Questionnaire–Revised (SBQ-R; Osman et al., 2001).

The SBQ-R is a four-item self-report instrument designed to assess lifetime suicidal ideation or attempts, frequency of suicide ideation, suicide threats, and future likelihood of suicide-related behavior. In previous research studies, the SBQ-R has demonstrated good internal consistency reliability and validity estimates in clinical, nonclinical, and college samples (Gutierrez, Osman, Barrios, & Kopper, 2001; Osman, Kopper, Barrios, Gutierrez, & Bagge, 2004). For the current study, coefficient-α for the SBQ-R total scale score was .81, 95% CI [.77, .84].

Symptom Assessment-45 Questionnaire (SA-45; Strategic Advantage, 2000).

The SA-45 is made up of nine scales designed to measure psychiatric symptomology: anxiety, depression, hostility, interpersonal sensitivity, obsessive–compulsive, paranoid ideation, phobic anxiety, psychoticism, and somatization. Participants rate each item on a 5-point Likert-type scale to indicate the extent to which the item has bothered or distressed them in the past 7 days, ranging from 1 (Not at all) to 5 (Extremely). Scale scores on this instrument demonstrate strong internal consistency reliability (coefficient-α > .75) in clinical and nonclinical populations. Further, results from validation studies suggest that scores on this instrument correlate significantly with related measures of psychopathology (i.e., depression, anxiety; Davison et al., 1997; Maruish, Bershadsky, & Goldstein, 1998; McConnell, Pargament, Ellison, & Flannelly, 2006). Reliability estimates for all scale scores for the current study sample were adequate (coefficient-α ≥ .80) and are presented in Table 2.

The Vengeance Scale-20 (VS-20; Stuckless & Goranson, 1992).

The VS-20 is a unidimensional self-report measure of revenge attitudes and responses to a perceived transgression. The measure includes 20 items scaled from 1 (Disagree strongly) to 7 (Agree strongly), where 10 items are coded regularly (e.g., “If someone causes me trouble, I’ll find a way to make them regret it”) and 10 items are reverse coded (e.g., “It is always better not to seek vengeance”). In previous studies, the global VS-20 total scale scores have demonstrated adequate reliability estimates, unidimensionality, and strong test–retest validity estimates in multiple populations (Hutt, Iverson, Bass, & Gayton, 1997; Ruggi, Gilli, Stuckless, & Oasi, 2012; Siu, 2002). For the current study, a total score was calculated by the sum of all of its items, coefficient-α = .91, 95% CI [.90, .93].

Transgression-Related Interpersonal Motivations Scale–12 (TRIM-12; McCullough et al., 1998).

The TRIM-12 scale is made up of 12 items measuring two motivational systems that govern interpersonal defenses. Five items are devoted to measuring motivation to seek revenge against the offender, and the remaining items measure the motivation to avoid the offender. Participants are asked to indicate their current thoughts and feelings about an individual who hurt them and rate the items from 1 (Strongly Disagree) to 5 (Strongly Agree). The psychometric properties of this measure have held up in diverse populations with various targets (i.e., in samples of spouses and teachers) as offenders. Particularly, test–retest reliability, construct validity, convergent and divergent validity estimates have held up in college student populations across the world (e.g., Lampton, Oliver, Worthington, & Berry, 2005; Orcutt, 2006; Safaria, 2014). For the current study, scale scores were computed by the sum of the items. Coefficient-α for TRIM-12 revenge motivations scale score was .90, 95% CI [.88, .90] and for TRIM-12 avoidance motivations scale score was .91, 95% CI [.90, .92].

Plan of data analysis.

We computed means and standard deviations for all scale scores. Internal consistency reliability was estimated in the Mplus 7.2 (Muthén & Muthén, 1998–2015) program with coefficient alpha using latent variable modeling (α; Raykov & Marcoulides, 2015) for all scale scores. Coefficient omega (ω, McDonald, 1999) estimates were only computed for scores on the MRAI-21 scales, given the concerns with Cronbach’s alpha addressed in Study 1 (Cronbach, 1951; Sijtsma, 2009). Both reliability estimates were computed with bootstrapping with 2,000 iterations to obtain 95% confidence intervals (CI).

We first elected to explore the potential nomological network for the MRAI-21 scale scores by computing bivariate correlations. Pearson’s product–moment correlation coefficients (r) were computed in the SYSTAT 13.0 for windows (SYSTAT Software, 2009) program with a Bonferroni correction applied to control the familywise error rate. To determine the relationships between scores on the specific MRAI-21 scales and scores on the SBQ-R, MSRI-21, VS-20, PID-5-BF, SA-45, and TRIM-12 scales, scores on each MRAI-21 scale and the set of concurrent measures were included in each (familywise) analysis. Concurrent measures were grouped into clusters using a combination of empirical (i.e., scale-level exploratory factor analysis) and rational methods (i.e., extant literature and expert experiences of the researchers) to simplify interpretations, yielding the following clusters: revenge, suicide, interpersonal distress, internal distress, and anxiety. Coefficients of .00 to .29 were interpreted as slight/negligible correlates; coefficients of .30 to .49 as low positive correlates; coefficients of .50 to .69 as moderate positive correlates; and coefficients of .70 or higher as high positive correlates (Hinkle, Wiersma, & Jurs, 2003).

In addition to bivariate correlations, multiple linear regression analyses were conducted to explore associations of the MRAI-21 scale scores with select sets of scale scores from the concurrent measures. The following scales were chosen from all available concurrent measures and modeled as outcome variables: the global VS-20 total and the TRIM-12 revenge motivations scale scores were chosen as the only two explicitly revenge-oriented concurrent measures in our study, SBQ-R total and MSRI-21 maladaptive behavior tendency were chosen as measures of suicidality, the PID-5-BF scale antagonism and the SA-45 hostility scale were chosen as trait-based measures of antagonism, impulsivity, and anger, respectively, and finally MSRI-21 negative self-evaluation was included as a measure of negative rumination. All items were modeled using the recommended scoring procedures and applied as outcome variables. The MRAI-21 scale scores (craving for revenge, revenge rumination, and suicide-related rumination) served as predictors for each outcome variable. Inspection of standard diagnostic plots, such as Q-Q plots for residuals and residuals by predictor plots, for each regression model were consistent with assumptions of normality of residuals, correct model specification, and homoscedasticity of residuals. The Durbin-Watson statistics were 1.9 (approximately 2) for all models, satisfying the assumption of independence of residuals. Finally, the Variance Inflation Factor (VIF) test yielded estimates <5.0 for all models, indicating that there were no problems with multicollinearity. Each multiple regression model was cross-validated using the k-fold validation method to address issues with overfitting in R using the caret package (Koul, Becchio, & Cavallo, 2018; Kuhan et al., 2016). Given the small sample size of the current study, a k value of 5 was chosen for each model to balance the bias-variance ratio (Morin & Davis, 2017). The standard deviation of the R² value ranged from 0.03–0.07 for all models suggesting fair reproducibility of results. Results from the final model extracted from R for each regression analysis were reported.

For the bivariate correlation analyses, correlations between the MRAI-21 dimensions and the dimensions in the revenge cluster (global VS-20 and TRIM-12 revenge motivations) and dimensions in the suicide cluster (SBQ-R total and MSRI-21 maladaptive behavior tendencies) and in the linear regression analyses, the effects of the MRAI–21 dimensions predicting for the revenge based measures (global VS-20 total and the TRIM-12 revenge motivations) and the suicide based measures (SBQ-R total and MSRI-21 maladaptive behavior tendency scales) were of particular interest to demonstrate convergent validity.

Results

Descriptive statistics and internal consistency reliability.

The means, standard deviations, and the internal consistency reliability estimates for scores on the MRAI-21 and all concurrent measures are displayed in Table 2. All the MRAI-21 scale scores exhibited adequate estimates of internal consistency. Coefficient-α and coefficient-ω estimates for all scales scores except for the PID-5-BF antagonism scale were above the .70 cutoff recommended for research utility (Cicchetti, 1994). Despite this short-coming, we elected to retain this scale in our analyses, as the upper bound for its coefficient-α reliability estimate was .73.

Bivariate correlations.

All zero-order correlations are presented in Table 4. For the revenge cluster, scores on MRAI-21 craving for revenge dimension had positive and significant correlations with scores on the TRIM revenge motivations, r = .31, p < .05 and the global VS-20 total measure (r = .55, p < .05; see Table 4). While the global VS-20 total measure was not a significant correlate of MRAI-21 revenge rumination’s dimension (r = .05, p = .32), it is a significant correlate of MRAI-21 suicide-related revenge, r = .22, p < .05. Similarly, within the suicide cluster, low to moderate and significant correlations were observed between scores on majority of the MRAI-21 dimensions and the suicide cluster dimensions. The association between scores on the MRAI-21 craving for revenge dimensions and the SBQ-R total dimensions was not significant (r = .12, p = .22).

Table 4.

Differential Correlates for MRAI Scale Scores

Concurrent measure clusters	Revenge rumination	Craving for revenge	Suicide-related revenge
Revenge
TRIM-12 REV	0.22	0.31	0.15
Global VS-20 total	0.05	0.55	0.22
Suicide
SBQ-R TOTAL	0.41	0.12	0.31
MSRI-21 MBT	0.27	0.21	0.43
Interpersonal distress
SA-45 PAR	0.45	0.34	0.32
S-A45 HOS	0.29	0.22	0.28
SA-45 INT	0.57	0.23	0.34
SA-45 DEP	0.52	0.21	0.30
PID-5-BF ANT	0.04	0.33	0.17
MSRI-21 FSC	0.39	0.27	0.31
TRIM AVO	0.22	0.32	0.13
Internal distress
PID-5-BF NAF	0.57	0.21	0.22
PID-5-BF PSY	0.37	0.20	0.24
MSRI-21 NSE	0.68	0.20	0.27
PID-5-BF DTC	0.28	0.18	0.27
PID-5-BF DIS	0.11	0.22	0.16
Anxiety
SA-45 ANX	0.45	0.19	0.25
SA-45 OCD	0.44	0.11	0.20
SA-45 SOM	0.23	0.10	0.16
SA-45 PHO	0.32	0.19	0.27

Note. N = 380. Bolded values indicate significance at p = .05 with Bonferroni correction applied. Italicized values meet or exceed the criteria for low positive correlations based on effect size (>.30). MRAI = Multidimensional Revenge Attitudes Inventory; MSRI = Multidimensional Shame Related Inventory; MSRI-21 MBT-21 = Maladaptive Behavior Tendencies; MSRI-21 FSC = Fear of Social Consequences; MSRI-21 NSE = Negative Self-Evaluation. PID-5B-F = Personality Inventory for the DSM–5 Brief Form; PID-5-BF NAF = Negative Affectivity; PID-5-BF DTC = Detachment; PID-5-BF ANT = Antagonism; PID-5-BF DIS = Disinhibition; PID-5-BF PSY = Psychoticism. SA-45 = Symptom Assessment-45; SA-45 ANX = Anxiety; SA-45 DEP = Depression; SA-45 HOS = Hostility; SA-45 INT = Interpersonal Sensitivity; SA-45 OCD = Obsessive Compulsive Disorder; SA-45 PAR = Paranoid Ideation; SA-45 PHO = Phobic Anxiety; SA-45 PSY = Psychoticism; SA-45 SOM = Somatization. TRIM-12 = Transgression-Related Interpersonal Motivations; TRIM-12 Rev = Revenge Motivations; TRIM AVO = TRIM Avoidance Motivations; VS-20 Total Score = Vengeance Scale-20 Total Score; SBQR Total = Suicidal Behaviors Questionnaire Revised Total Score.

Within the interpersonal distress cluster, scores on SA-45 paranoid ideation, hostility, interpersonal distress, depression and MSRI-21 fear social consequences dimensions were significant correlates of all the MRAI-21 dimensions. In the internal distress cluster, all MRAI-21 dimensions were significant correlates of PID-5-BF negative affect, psychoticism, detachment, and MSRI negative self-evaluation. However, scores on the MRAI-21 revenge rumination and suicide-related revenge dimensions were not significantly related to scores on the PID-5-BF disinhibition dimension. Lastly, in the anxiety cluster, all MRAI-21 dimensions had positive and significant correlations with dimensions of SA-45 anxiety and phobic anxiety, ranging from r = .19 to r = .45, p < .05.

Regression.

Results from the multiple linear regression analyses supported the convergent validity of the MRAI-21 scales (see Table 5). For all models, R² values ranged from 0.10 to 0.47, indicating that for each outcome, the MRAI-21 scale scores accounted for a sufficient amount of variance.

Table 5.

Results From Multiple Regression Analyses Predicting Scores on Concurrent Measures, Study 2

Outcome	F	R 2	Predictor	β	p	Zero-order r	sr2
VS-20 Total	63.48	0.33	CR	0.67	.00	0.55	0.29
			RR	−0.13	.00	0.05	0.01
			SRR	−0.14	.00	0.22	0.01
TRIM Rev	16.61	0.11	CR	0.31	.00	0.31	0.06
			RR	0.14	.01	0.22	0.02
			SRR	−0.08	.21	0.15	0.00
PID-5-BF ANT	15.70	0.10	CR	0.36	.00	0.33	0.08
			RR	−0.08	.14	0.04	0.01
			SRR	−0.01	.89	0.17	0.00
SA-45 HOS	18.32	0.13	CR	0.04	.49	0.22	0.00
			RR	0.22	.00	0.29	0.04
			SRR	0.19	.00	0.28	0.02
SBQR Total	37.26	0.23	CR	−0.18	.00	0.12	0.02
			RR	0.38	.00	0.41	0.13
			SRR	0.30	.00	0.31	0.06
MSRI-21 MBT	33.21	0.20	CR	−0.10	.08	0.21	0.01
			RR	0.17	.00	0.27	0.02
			SRR	0.43	.00	0.43	0.12
MSRI-21 NSE	109.61	0.47	CR	−0.08	.08	0.20	0.00
			RR	0.67	.00	0.68	0.39
			SRR	0.12	.01	0.27	0.02

Note. N = 380. sr² = Squared Semipartial Correlation; CR = Craving for Revenge; RR = Revenge Rumination; SRR = Suicide-Related Revenge; VS-20 Total = The Vengeance Scale Total Score; TRIM Rev = Transgression-Related Interpersonal Motivations Scale Revenge Motivations; PID-5-BF ANT = Personality Inventory for the DSM–5 Antagonism; PID-5-BF DIS = Disinhibition; SA-45 HOS = Symptom Assessment-45 Hostility; SBQR Total = Suicide Behaviors Questionnaire Total Score; MSRI-21 MBT = Multidimensional Shame Response Inventory Maladaptive Behavior Tendency; MSRI-21 NSE = Negative Self-Evaluation.

Notably, scores on the craving for revenge scale were strongly associated with scores on the global VS-20 total and the TRIM revenge motivations scales (β = .31, β= .67, p < .001, respectively). Scores on the MRAI-21 revenge rumination and MRAI-21 suicide-related revenge scale scores were positive and significant predictors for scores on both the SBQ-R Total (β = .38, β = .30, p < .001, respectively) and the MSRI-21 maladaptive behavior tendency scale scores (β = .17, β = .43, p < .001, respectively).

The trait-based measure of antagonism closely related to scores on craving for revenge scale score (β = .36, p < .001), controlling for the MRAI-21 suicide-related revenge and the MRAI-21 revenge rumination scale scores. Further, the MRAI-21 revenge rumination and the MRAI-21 suicide-related revenge scale scores were significant and positive predictors of the SA-45 hostility scale score (β = .22, β = .19, p < .001, respectively). Finally, scores on the MRAI-21 revenge rumination scale evidenced a strong association with scores on the MSRI-21 negative self-evaluation scale, controlling for scores on the MRAI-21 suicide-related revenge and MRAI-21 craving for revenge scales (β = .67, p < .001).

Discussion

Study 1 examined the validity of the correlated three-factor MRAI-21 model using ESEM in a college student sample. Results are consistent with preliminary findings (Osman et al., 2015), in that the three dimensions reported by Osman et al. were recovered: Craving for Revenge, Revenge Rumination, and Suicide-Related Revenge. Additionally, good reliability estimates, range of interitem correlations, and AIC values provided strong evidence for the research utility of the MRAI-21 scales. Together, findings from the first study confirmed that the MRAI-21 is a valid and reliable measure of revenge attitudes in the context of suicide.

Study 2 explored a nomological network of the MRAI-21 scale scores using Pearson’s product–moment correlations and multiple linear regression analyses. Overall, scale scores on the MRAI-21 revenge rumination, craving for revenge, and suicide-related revenge had positive and significant correlations with clusters of scores on measures of interpersonal distress, internal distress, anxiety, revenge, and suicide. Further, the scores on the MRAI-21 dimensions had low to moderate significant and positive associations with scores on other measures of revenge and suicide, indicating strong evidence for convergent validity.

Positive low to moderate correlations observed between scores for MRAI-21 craving for revenge dimension and scores for vengeance, paranoid ideation, and revenge motivations indicate that this dimension taps into a metaconstruct of emotional disturbance caused by revenge fantasies. Parallel to the items comprising the VS-Total and TRIM-12 revenge motivations, items in the craving for revenge dimension are crafted to evaluate thoughts, feelings, behaviors, and emotions related to acting on revenge fantasies in response to a perceived transgression. These conclusions are further supported by results of the multiple regression analyses, where the VS-Total and TRIM-12 revenge motivations scale scores are outcome variables predicted by the MRAI-21 dimensions. High responses on MRAI-21 craving for revenge predict an increase in responses on VS-Total and TRIM-12 revenge motivations. These results clearly differentiate the craving for revenge dimension from the other MRAI-21 dimensions and showcase strong evidence of convergent validity, given that the scale taps into the construct of externalizing revenge, parallel to other measures of revenge. Notably, craving for revenge served as the only significant predictor for antagonism and disinhibition, relative to the other MRAI-21 dimensions. These traits may be indicative of strong desires for revenge. Trait-based aggression and antagonism should be further explored as associates of revenge attitudes.

The MRAI-21 revenge rumination scale scores were found to have positive associations with scores on measures of negative self-evaluation, interpersonal sensitivity, and negative affect. Scores on this dimension were also significant and positive predictors of hostility, suicidal behavioral tendencies, and negative self-evaluation. Further, scores on the revenge rumination dimension discriminated between the constructs noted above and the other MRAI-21 dimensions. These results suggest that the MRAI-21 revenge rumination dimension is relevant to research findings where revenge cognitions and revenge fantasies are evidenced to perpetuate negative affect and hostility. For instance, Bies et al. (1997) have suggested that revenge can manifest itself in a mostly cognitive manner through obsessive rumination and fantasies. Moreover, rumination about adverse events can perpetuate pessimistic attitudes (Lyubomirsky & Nolen-Hoeksema, 1993) and increase the stability of one’s attitudes toward the event (Wilson & Kraft, 1993). Additionally, revenge rumination scale scores were found to correlate positively with scores on measures of behavioral inhibition and reactive aggression, suggesting that revenge is a delayed, cognitive process fueled by interpersonal frustrations (Bjørnebekk & Howard, 2012).

The suicide-related revenge scale was of particular interest due to its novelty. Results from Study 2 found strong associations between scores on this dimension of the instrument and scores on the other measures of suicidality, including maladaptive behavior tendencies and suicide-related behaviors, all of which are established risk factors for suicide. Study results provided further evidence that this dimension is designed to tap into the construct of suicide in the context of revenge. Specifically, it is noteworthy that this is one of the few studies providing direct empirical evidence for a link between suicidal behaviors and a content-specific measure of the revenge construct. This relationship has been conceptualized in many different contexts (e.g., Douglas, 2015; Vijayakumar, 2015), but never thoroughly examined empirically.

Finally, it is essential to note that MRAI-21 revenge rumination and suicide-related revenge both were negative and statistically significant predictors of scores on the global VS-20 total score. However, the suicide-related revenge scale score was not a significant predictor of the TRIM revenge motivations scale score, relative to the other MRAI-21 scale scores. These associations indicate that the dimensions of the MRAI-21, particularly the revenge rumination and suicide-related revenge dimensions, are distinct from most traditional measures of revenge. These proposed dimensions expand the areas of revenge research and tap into other risk factors, as evidenced by strong associations of scores on the suicide-related revenge and revenge rumination with scores on measures of suicide, hostility, maladaptive behavioral tendencies, and negative self-evaluation. Together, not only do the correlates and regression models highlight the distinctness of each of the MRAI-21 dimensions, but they can also help establish and identify potential traits, risk factors, and behaviors that increase the risk of revenge-based attitudes.

Given that the recruited sample was comprised of college students, the generalizability of our results is limited. Further, it is important to recognize that the individuals in these samples were not at high risk of suicide. Future research should test whether the factor structure will hold in diverse community-based, clinical, and incarcerated populations. It might also be fruitful to examine cultural differences in scores on the MRAI-21, given that suicide is influenced by various political, social, and religious factors across cultures (Canetto, 2008). Further, although previous research has found that there are no significant gender differences in self-reported thoughts of revenge (Barber, Maltby, & Macaskill, 2005), exploring potential gender differences in responses to the MRAI-21 scales would be a logical next step. Additionally, future work should attempt to expand the known nomological network of the MRAI-21 scales with diverse correlates, including measures of protective factors.

Despite these limitations, the current studies empirically support the psychometric properties of a new measure of suicide-related revenge attitudes in a low-risk college student sample. The Multidimensional Revenge Attitudes Inventory-21 confirms that revenge is indeed a multidimensional construct that can be empirically validated and measured within the context of suicide, with an extensive nomological network and potential for research and clinical utility.

Public Significance Statement.

While links between revenge and suicide are hypothesized in the current literature, a multidimensional measure on revenge-motivated suicide is missing. The present studies introduce and provide empirical support for the psychometric properties of a new measure, the Multidimensional Revenge Attitudes Inventory-21, which assesses craving for revenge, revenge rumination, and suicide-related revenge.