Development and validation of the Recognizing Addictive Disorders scale: A transdiagnostic measure of substance-related and other addictive disorders.

Abstract

Objective:

Substance use disorders and behavioral addictions commonly co-occur. However, few available self-report measures reliably and validly assess the full range of addictive conditions. The development and initial validation of a new measure—Recognizing Addictive Disorders scale (RAD) addresses a significant gap in the literature.

Method:

After items were generated and evaluated in Study 1, Study 2 (N = 300), applied exploratory factor analysis to the item pool using an online-based community sample. In Study 3 (N = 427), the factor structure was validated using an independent online-based community sample and confirmatory factor analysis.

Results:

The scale demonstrated good internal consistency (a = .91) and construct validity, including replication of the factor structure (χ² (553) = 760.83, p < .001, CFI = .997, TLI = .997, RMSEA = .030) and correlation with a related transdiagnostic measure of addiction (r = .72).

Discussion:

Overall, results support preliminary validity of a brief transdiagnostic measure of addiction that considers a diverse range of behaviors. For patients presenting to substance abuse treatment, this tool may be useful in identifying the frequency of other types of non- substance problems, which could ultimately aid in treatment planning.

Introduction

Growing evidence is available to direct clinical decision making for patients presenting with one addictive disorder. However, patients often present with complex comorbidity, and these comorbidities may be underdiagnosed and ultimately affect treatment (Flynn & Brown, 2008). There are a number of valid and reliable assessment tools for screening for alcohol, drug, tobacco, and gambling disorder, but few instruments assess multiple conditions within the same measure or consider addictive behaviors broadly (e.g., binge eating disorder). The current work aimed to address this gap with the development and psychometric testing of the Recognizing Addictive Disorders (RAD) scale.

Several transdiagnostic theories of addiction have been posited, including Orford’s addiction as excessive appetite (Orford, 2001), Jacob’s general theory of addictions (Jacobs, 1986), Shaffer’s addiction syndrome model (Shaffer et al., 2004), and Griffiths’ component model of addictions (Griffiths, 2005), all of which aim to expand and explain the many diverse forms that addiction can take. These models have strengths and weaknesses as well as varying levels of support. Shaffer’s addiction syndrome model is notable for its consideration of shared and distinct symptoms associated with the various forms of addiction, and it includes a comprehensive description of the stages of addiction. The component model (Griffiths, 2005) has also emerged as potentially useful in describing the wide diversity of addictive behaviors and transdiagnostic treatment approaches aligned with this theory are currently in development (Kim & Hodgins, 2018). The development of the RAD scale is consistent with the major tenants of both models.

No consensus exists delineating what behaviors do or do not represent substance-related and other addictive disorder (SRAD). The current work considers seven types of SRAD: alcohol use disorders, drug use disorders, tobacco use disorders, gambling disorder, binge eating disorder, hypersexual disorder, and video gaming disorder. While some conditions—such as alcohol use disorders, drug use disorder, and gambling disorder—have strong support within the literature (Rehm et al., 2006), other behaviors are more controversial (e.g., binge eating disorder is most often conceptualized as a feeding and eating disorder). Additionally, each condition has not been explicitly operationalized with an agreed upon definition (e.g., hypersexual disorder). For the purposes of the current work, hypersexual behavior includes sexual acts and viewing pornography with or without concurrent masturbation. This inclusion is supported by empirical research showing that individuals who endorse problematic sexual behaviors or identify as “sex addicts” report excessive pornography use with significant frequency (Kafka, 2010). In addition to those conditions already formally recognized as addictive disorders, (alcohol, drug, and tobacco use disorders as well as gambling disorder), hypersexual behavior and excessive video-game playing were included because they were specifically considered for inclusion as other addictive disorders during the development of DSM-5 (King & Delfabbro, 2013; Reid et al., 2012). Binge eating was included due to strong preclinical and clinical evidence of neurobiological similarities to substance use disorders (Davis & Carter, 2009), while the evidence was less strong for other behaviors related to eating (e.g. restriction or exercise as addictions (Szabo, Griffiths, Marcos, Mervó, & Demetrovics, 2015). The behaviors examined here do not represent an exhaustive list of clinically relevant behaviors with significant similarities to other addictive disorders (e.g., internet addiction; Weinstein & Lejoyeux, 2010). However, these subtypes were chosen due to the best available evidence indicating similar phenomenology to established addictive disorders. Namely, all behaviors considered are associated with evidence of 1) using a larger amount or over a longer period of time than was intended, 2) physical or psychological problems made worse by use or engagement in the behavior, 3) social or interpersonal problems, 4) neglecting major roles, and 5) craving. Data from the National Epidemiologic Study on Alcohol and Related Conditions (NESARC) study show that these criteria possess strong discriminative properties (Hasin et al., 2013).

It is critical to understand if patients are presenting with symptoms of more than one SRAD. Though the available data are limited, the evidence that does exist suggests important differences for comorbid SRAD. For example, researchers found that individuals with binge eating disorder undergoing a tobacco cessation treatment were less likely to quit as compared with individuals who did not have binge eating disorder (White, Peters, & Toll, 2010). More optimistically, some research shows that treatment that considers two or more SRAD simultaneously can effectively reduce both treatment targets. Hartman and colleagues (2012), found that treatment addressing both substance use and hypersexual behavior effectively reduced both behaviors at 6-months posttreatment. Concurrent treatment may be able to more effectively intervene on event-level interactions. For example, ecological momentary assessment supports that engagement in one SRAD-behavior can increase odds of engagement in other SRAD behaviors. Specifically, alcohol can increase betting behavior (Hing, Russell, Thomas, & Jenkinson, 2019), and substance use can increase risky sexual behavior (Rendina, Moody, Ventuneac, Grov, & Parsons, 2015). Better understanding of what types of comorbidity patients are presenting with can help clinicians better tailor the treatment to increase positive outcomes for a range of SRADs.

Currently, a handful of transdiagnostic measures exist: The Shorter PROMIS Questionnaire (SPQ; Christo et al., 2003), Composite Measure of Problem Behaviors (CMPB; Kingston, Clarke, Ritchie, & Remington, 2011), the Risky, Impulsive, and Self-Destructive Behaviors Questionnaire (RISQ; Sadeh & Baskin-Sommers, 2017), and the Screener for Substance and Behavioural Addiction (Schluter, Hodgins, Wolfe, & Wild, 2018). However, some limitations related to existing measures are notable. The SPQ is prohibitively long for some settings, with over 150 items. The CMPB has been psychometrically validated but does not include assessment of gambling disorder, which commonly co-occurs with other types of addiction. The RISQ is not intended to screen for the symptoms of addiction and instead assess for the frequency of externalizing behaviors. Addiction is known to have behavioral and psychological components (Reid, Carpenter, & Lloyd, 2009). Therefore, any transdiagnostic measure intended to help identify possible comorbidity should include assessment of both behavioral and psychological aspects of addiction. The SSBA (Schluter et al., 2018) was validated using a large community sample broadly representative of the Canadian English-speaking adult population and has notable psychometric strengths. However, the proposed measure differs in the symptoms assessed. Specifically, the SSBA includes two items referring to loss of control, an item related to dependence (e.g., I had to do it to function), and an item that broadly covers consequences related to use. In contrast, the RAD scale, considers separately 1) physical or psychological problems made worse; 2) social or interpersonal problems and 3) neglecting of major roles. The scale also assesses craving. Assessment of these symptom domains represents a strength of the RAD scale as they were identified as having strong ability to classify individuals with and without SUDs within epidemiologic research in the United States (see Hasin et al., 2013).

Across available measures, the current study aimed to address these limitations by developing a psychometrically valid transdiagnostic scale that is brief enough to allow for administration in situations in which participant or patient burden is a concern. It was hypothesized that all seven behaviors would load onto a latent factor of addiction.

Methods

Of the 11 DSM SRAD criteria, five were chosen as the basis for developing items. Item development was limited to five criteria in the interest of limiting the length of the total scale, and the specific criteria were chosen based on epidemiologic research demonstrating that these criteria had strong discriminative properties along with a range of difficulty level, based on NESARC data (Hasin et al., 2013). In Study 1, a large pool of items (N = 259) was developed by PhD level psychologists or clinical psychology doctoral students with expertise in addiction. Items were subjected to reading level analysis. Items were tested in three independent studies. As a part of Study 1, nine experts reviewed a sampling of the proposed items and rated the behavior and DSM criterion they believed was being assessed. All items were reviewed by at least four experts. Items with lower than 75% agreement on DSM criterion (n = 23) were dropped. Additionally, non-experts rated each item’s clarity using a rating scale of 1—very unclear to 7—very clear. Items were ranked according to average clarity rating, and items in the bottom 20% were eliminated. In Study 2, participants completed 189 items, and the data were subjected to exploratory factor analysis. In Study 3, participants completed the newly developed RAD scale, which included 38 items. To reduce participant burden, participants were randomly assigned to complete one or two additional measures of the eight additional measures included for the purposes of exploring construct validity.

Participants.

Participants for Studies 2 and 3 were recruited through Amazon’s Mechanical Turk, which is an online-based recruitment platform that has been shown to produce reliable, high-quality data, including addiction related research and measurement development for addictions research (Kim & Hodgins, 2017; Mellis & Bickel, 2020; Strickland & Stoops, 2019). Data also demonstrate that MTurk participants are reasonably representative as compared with nationally representative data (McCredie & Morey, 2018). Although most data support the reliability and validity of MTurk samples, some data suggest both fraudulent and careless responding (Chandler & Paolacci, 2017; Chandler, Sisso, & Shapiro, 2020). In the current study, significant attempts were made to minimize careless responding, including use of questions to ascertain attention and review of unusual response times. Initially, identification of unusual response time was attempted using z-scores; however, floor effects significantly limited the utility of this approach. For study 2, initial review of the data revealed a high number of cases who stopped responding partway through the survey. For study 2 only, the decision was made to exclude individuals who missed more than 10% of potential RAD items (n = 66). An additional 52 participants were excluded in study 2 for responding in an unusually short (i.e., less than 10 minutes) or unusually long period of time (more than one hour; n = 38) or failing to respond accurately to questions assessing attention (n = 14). Subsequent review revealed significant overlap in individuals who missed more than 10% of RAD items and unusual response time. In study 3, only response time and embedded attention checks were used to identify valid cases, and a total of 162 participants were excluded for responding in an unusually short (i.e., less than 5 minutes) or unusually long period of time (more than one hour; n = 151) or failing to respond accurately to questions assessing attention (n = 11).

Participants responded to ads to complete an online, cross-sectional survey about health behaviors. Participants were considered eligible if they were over the age of 18, lived in the United States, missed fewer than 10% of items, and did not show evidence of careless responding. All participants provided informed consent, and this study received approval from the approval from Eastern Michigan University's Institutional Review Board. For Study 2 and 3, participants were recruited without further eligibility considerations until a minimum N = 300 was achieved. At that time, recruitment was closed to individuals who identified as White non-Hispanic to increase the proportion of non-white respondents. Table 1 includes a summary of demographic information for Study 2 and Study 3. There were no significant demographic differences with respect to gender, income, and education. In Study 3, there were significantly fewer individuals who identified as White and greater racial and ethnic variability, including a higher percentage of individuals who identified as Black, Biracial, and Asian.

Table 1.

Sample Characteristics Study 2 and 3

	Study 2 (N = 300)	Study 3 (N = 427)
Gender, n (%)
Male	137 (45.5)	163 (38.2)
Female	163 (54.2)	259 (60.7)
Transgender/Other	1 (0.3)	3 (0.7)
Race/Ethnicity, n (%)
White	214 (71.3)	267 (63.3)
Black	18 (6.0)	58 (13.7)
Hispanic	22 (7.3)	24 (5.7)
Asian	25 (8.3)	42 (10.0)
Bi/Multiracial	12 (4.0)	22 (5.2)
Other	9 (3.1)	9 (2.1)
Education, n (%)
HS or less	62 (20.6)	105 (24.6)
Some college	71 (23.6)	96 (22.5)
College	168 (55.8)	226 (52.9)
Employment, n (%)
Employed full-time	180 (59.8)	263 (61.6)
Employed part-time	56 (18.6)	79 (18.5)
Unemployed	39 (13.0)	54 (12.6)
Other	26 (8.6)	31 (7.3)
Age, M(SD)	37.7 (12.3)	37.3 (11.9)

Measures

Recognizing Addictive Disorders scale (RAD)

The RAD was initially developed using a potential pool of 259 items. The final scale is 35-items assessing symptoms of 7 potential substance or other related addictive disorders within the last 3 months. Participants indicate the degree to which an item describes them on a scale of 0 to 6, with three anchors includes (0—Does not describe me at all, 3—Describes me somewhat, 6—Describes me very well). Total scores ranged from 0–210. Note that average subscale scores as well as total score can be computed.

Adverse Childhood Experience Scale (ACES)

ACES (Felitti et al., 1998) is a 10-item self-report measure assessing for the presence or absence of adverse or traumatic events that occurred before the age of 18. Large-scale studies have demonstrated that ACE scores are significantly related to mental health and physical morbidity, including substance misuse and obesity (Anda et al., 2006; Dube, Williamson, Thompson, Felitti, & Anda, 2004). The measure demonstrated acceptable reliability in the current sample Cronbach α = .76.

Patient Health Questionnaire (PHQ)

The PHQ (Spitzer, Kroenke, & Williams, 1999) includes the PHQ-9 and GAD-7-item subscale assessing depression and anxiety, within the last 2 weeks and 6 months respectively. Good to excellent reliability and construct validity have been widely established (Martin, Rief, Klaiberg, & Braehler, 2006; Spitzer, Kroenke, Williams, & Löwe, 2006). The PHQ-9 and GAD-7 demonstrated good to excellent reliability in the current sample, with Cronbach α = .89 and .93 respectively.

Shortened UPPS-P

The Shortened UPPS-P (Cyders, Littlefield, Coffey, & Karyadi, 2014) is a 20-item self-report measure assessing multidimensional aspects of impulsivity, and it represents a shortened version of the original 59-item questionnaire (Cyders et al., 2007). In the validation paper, the CFA demonstrated the same factor structure as the original measure; a model including five correlated latent facets of impulsivity fit the data well. The total scale demonstrated good reliability in the current sample Cronbach α = .87.

Composite Measure of Problem Behaviors (CMPB)

The CMPB (Kingston et al., 2011) is a 60-item self-report measure assessing ten different types of risky behaviors, and items were initially derived from existing well-validated inventories that assessed the problem behavior in isolation. In the validation paper, the subscales showed good internal consistency and the overall test-retest showed good to excellent reliability. The measure demonstrated good reliability in the current sample, with Cronbach α = .88.

Brief Difficulties in Emotion Regulation (DERS-16)

The DERS-16 (Bjureberg et al., 2016) is a 16-item self-report measure assessing problems related to emotion regulation, which includes several related skills: being aware of emotions, accurately labeling emotions, making decisions based on emotional information, and modulating emotions. There is broad support for construct validity, including data demonstrating that the DERS mediates the relationship between early childhood adverse events and maladaptive behaviors such as alcohol misuse, risky sex, physical aggression, and impulsive spending (Espeleta, Brett, Ridings, Leavens, & Mullins, 2018). The measure demonstrated excellent reliability in the current sample, with Cronbach α = .95.

Statistical Analyses

Study 1:

Potential items were subjected to reading-level analysis using the Flesch-Kincaid formula. Proportion of agreement was utilized to eliminate items with poor content validity. Mean item rating for clarity and comprehensibility was utilized to eliminate ambiguous or unclear items.

Study 2:

Sample descriptive and variable distributions were first examined using IBM SPSS Version 24. Next exploratory factor analysis (EFA) was conducted using the ‘psych’ package (Revelle, 2018) within R (R Core Team, 2015). EFA was used to evaluate the dimensionality of a set of multiple indicators by uncovering the smallest number of interpretable factors that explain the covariances among indicators (Brown, 2006) and eliminate poorly performing items. Brown (2006) describes five steps for EFA: factor extraction, factor rotation, factor selection, interpretation, and completing CFA on the same sample. Due to non-normality in the data, robust-weighted least squares estimation was first attempted, but there were problems with convergence. Instead, weighted least squares estimation was used for factor extraction at the EFA stage. Review of eigen values, parallel analysis, and Velicer’s minimum average partial method (MAP) criterion, and evidence from the larger literature base within the field of addiction were used to determine the number of factors to retain, and oblimin rotations were utilized to aid in interpretation.

Study 3:

Confirmatory factor analysis (CFA) with robust-weighted least squares estimation was completed using MPlus Version 7.0. χ², comparative fit index (CFI; Bentler, 1990), Tucker-Lewis index (TLI; Tucker & Lewis, 1973), and Root Mean Square Error of Approximation (RMSEA; Browne & Cudeck, 1993) were used. Based on the recommendations of Hu and Bentler (1999), a value of .95 or higher was considered indicative of good fit for CFI and TLI, whereas a value of 0.06 or lower was considered good fit for RMSEA. Psychometric properties were explored, including internal consistency, and construct validity based on correlations with measures of associated features. Mean group differences for demographic groups were explored using non-parametric methods (Mann-Whitney, Kruskall Wallace) and for significant differences effect sizes were calculated a transformation of the z-score $r=\frac{z}{\sqrt{N}}$ (Rosenthal, Cooper, & Hedges, 1994).

Results

A total of 259 items were developed. Reading analysis indicated an average reading level of grade 5.7. Based on expert review, 23 items were eliminated due to poor agreement, and an additional 47 items were eliminated due to lower ratings of comprehensibility and clarity by non-experts. Initial attempts at EFA revealed problems with convergence, likely due to a high ratio of items to participants. Fifty-seven items with a very low level of endorsement, based on fewer than 5% of the sample rating the item 5 or higher on the response scale (1—Does not describe me at all to 7—Describes me very well) were eliminated. The resulting item pool included 132 items.

For the total item pool, parallel analysis and the MAP criterion suggested 8 factors, while the eigenvalues suggested 12 factors. Most rotations revealed a pattern of hypersexuality items loading onto 2 different factors, although cross-loading was still apparent. Proportionally, the 8^th factor explained only 2% of the variance. The decision was made to retain a 7-factor structure for item reduction. Given the support for discrete substance/behavioral factors, EFA was performed in order to retain the items that loaded the most highly onto their specific factor, while minimizing any variance in the item explained by a factor other than the primary factor. Across all seven behaviors, the suggested number of factors to be retained consistently differed based on the method used. In all cases, a parsimonious solution involving a one-factor solution was broadly an appropriate fit to the data using EFA.

Next, based on the recommendations by Brown (2006), CFA was performed on the same sample to check the appropriateness of the hypothesized model before collecting an additional sample. The data demonstrated some problems with fit. EFA results were reexamined. While some data suggested a single factor for each behavior, results also showed a pattern of two clusters of symptoms: 1) taking the substance (or engaging in the behavior) in larger amounts or over a longer period of time than was intended, physical or psychological problems made worse, and craving; and 2) social or interpersonal problems and neglecting major roles, with items from cluster 1 showing stronger loadings. These clusters were conceptualized as internalizing and externalizing symptoms, and a model using only the internalizing symptoms was tested using CFA for all behaviors. Based on fit statistics, the internalizing model was a better fit for the smoking, gambling, and hypersexual items. However, the role of externalizing symptoms in gambling disorder is well-established, and as such the decision was made to retain items that would allow for the model using all 5 DSM criteria as well as the internalizing model, which only included 3 DSM symptoms. The resulting scale included 38-items—each subscale included 5 items with the exception of gambling, which included 8 items in order to accommodate both conceptual approaches to retaining items. These extra three items were later dropped for the final scale total of 35 items. Table 2 includes the fit statistics for items retained under the conceptual model for alcohol, drug, tobacco, binge eating, and video-gaming as well as the items retained under the internalizing model for the gambling and hypersexuality subscales.

Table 2.

Fit Statistics for Confirmatory Factor Analysis Study 2

Subscale	Fit Indices
	χ2 (df)	CFI	TLI	RMSEA (90% CI)	SRMR
Alcohol	21.846(5), p = .001	.987	.974	.109 (.065–.158)	.019
Drug	35.506(5), p < .001	.975	.950	.147 (.104–.194)	.025
Smoking (internalizing model)	9.524(5), p ≤ .001	.998	.995	.056 (.000–.110)	.006
Gambling	55.336(5), p < .001	.967	.935	.187 (.144–.233)	.024
Gambling (internalizing model)	8.067(5), p = .153	.997	.993	.046 (.000–.101)	.020
Binge Eating	21.427(5), p = .001	.984	.968	.106 (.063–.154)	.028
Hypersexual (internalizing model)	16.385(5), p = .033	.985	.970	089 (.043–.138)	.025
Video Gaming	28.311(5), p < .001	.977	.955	.127 (.084–.174)	.024

Note. CFA = Confirmatory Factor Analysis, CFI = Comparative Fit Index, TLI = Tucker-Lewis Index, RMSEA = Root Mean Square Error of Approximation, SRMR =Standardized Root Mean Residual, DSM = Diagnostic and Statistical Manual for Psychiatric Disorders.

With Study 3 data, an EFA supported a unidimensional factor structure using all 5 DSM criteria for the gambling items. After eliminating the three extra gambling items, a measurement model was specified to include 1 higher order latent factor of addiction and 7 latent factors defined by 5 observed indicators. The data showed an excellent fit with the specified model, χ² (553) = 760.83, p < .001, CFI = .997, TLI = .997, RMSEA = .030. Figure 1 presents the standardized factor loadings, with the factor variance set equal to 1. All subscales loaded on the latent factor at .40 or above. Drug use symptoms loaded the most strongly onto the latent factor and binge eating symptoms loaded the least strongly. For the total scale, Cronbach’s α = .92 and Cronbach’s α for the subscales raged from .90 to .98.

Figure 1.

Measurement model of the full Recognizing Addictive Disorders scale. This figure illustrates the factor loadings for the items as well as latent factors assessed within the Recognizing Addictive Disorders scale. The e refers to error, which is any variance not explained by the model. The 7 lower order latent factors include: alcohol use, drug use, smoking, gambling, binge eating, hypersexuality/pornography, and excessive video game playing.

As the measure is internally consistent, the total score can reliably be interpreted. Analyses demonstrated some demographic differences with respect to total score. Men (Mdn = 23.50) reported significantly more symptoms than women (Mdn = 14.50), U = 15656.50, p < .001, r = −.20. Individuals with a high school education (Mdn = 25.00) reported significantly more symptoms than individuals with higher education (Mdn = 16.00), U = 13388.50, p = .009, r = −.13. Age was significantly, negatively correlated with addiction symptoms, r_tau = −.13, p < .001. There were no differences across racial groups. Table 3 includes the bivariate correlations, with shading where darker colors represent a strong relationship. RAD total score was strongly, positive correlated with scores on CMPB. The RAD total score as well as subscales demonstrated largely significant relationships with emotion dysregulation, impulsivity, and depression. The relationship with adverse childhood events and anxiety was more variable. In particular, drug use, binge eating, hypersexuality, and excessive video-game playing were unrelated to adverse childhood events. Anxiety was unrelated to drug use, smoking, and gambling. Table 4 includes a list of the final items along with the means and standard deviations for each item in Study 3.

Table 3.

Bivariate Correlations RAD and Related Measures

		1	2	3	4	5	6	7	8	9	10	11
1	RAD Total
2	RAD Alcohol	.65**
3	RAD Drug	.62**	.35**
4	RAD Smoking	.61**	.35**	.29**
5	RAD Gambling	.57**	.29**	.36**	.24**
6	RAD Binge Eating	.52**	.17**	.24**	.13**	.14**
7	RAD Hypersexual	.58**	.22**	.33**	.10*	.25**	.21**
8	RAD Video Game	.58**	.30**	.30**	.10*	.31**	.21**	.28**
9	ACES Total	.28**	.28**	0.14	.31**	.18*	0.09	0.09	0.07
10	PHQ-9	.50**	.30**	.34**	.19*	.30**	.41**	.33**	.42**	.31**
11	GAD-7	.31**	.31**	0.06	0.15	0.16	.24**	.19*	.23**	.35**	.74**
12	S-UPPS Total	.43**	.36**	.22*	0.08	0.12	.34**	.32**	.18*	─	─	─
13	CMPB Total	.72**	.56**	.56**	.46**	.34**	.42**	.28**	.38**	─	─	─
14	DERS-16_Total	.44**	.32**	.27**	.23**	.28**	.31**	.32**	.22**	.31**	.77**	.65**

Notes. RAD = Recognizing Addictive Disorders Scale, ACES = Adverse Childhood Experiences Scale, S-UPPS = Shortened UPPS Scale, CMPC = Composite Measure of Problem Behaviors, DERS = Difficulties in Emotion Regulation.

^†EM Dashes indicate that no correlation was computed to due missing data as a part of randomized, block survey administration

^**p < .01 level (2-tailed).

^*p < .05 (2-tailed).

Table 4.

Descriptive Statistics for Final RAD items Study 3

		Mean	SD
1	Drinking is like a slippery slope, I end up drinking more than I wanted to.	0.93	1.70
2	My drinking has caused a disagreement or two.	0.78	1.51
3	Because I was drinking, I wasn’t able to get as many things done at home, work, or school.	0.71	1.54
4	I probably think about drinking more than most people do.	0.62	1.41
5	It’s hard to cut down, even though I know drinking isn’t good for my health.	0.69	1.56
6	I worry about my health because of my drug use.	0.20	0.80
7	With drugs, I can get carried away and use a lot more than I wanted to.	0.27	0.97
8	My drug use prevents me from getting too close to people.	0.17	0.74
9	My responsibilities can fall through the cracks because of my drug use.	0.20	0.81
10	There are times that I feel a strong urge to use drugs.	0.39	1.23
11	At certain times of the day, I find myself really wanting a cigarette.	1.12	2.06
12	Sometimes my cravings for cigarettes are powerful.	1.03	1.95
13	When I get stressed, I can smoke a lot more than I planned	1.07	2.07
14	Sometimes I feel driven to smoke.	1.02	1.93
15	I still smoke even though people tell me it’s bad for my health.	1.10	2.10
16	Time gets away from me when I am gambling.	0.52	1.36
17	If I lose a lot of money, I can feel down for several days	0.63	1.51
18	I get distracted by thoughts of gambling.	0.32	1.02
19	I may skip out on certain things so I can go gamble.	0.27	1.00
20	Gambling has caused problems for me with my friends/family.	0.25	0.96
21	Sometimes my cravings for certain foods are overpowering.	1.60	2.01
22	Being unable to control my eating can cause some stress in my relationships.	0.56	1.40
23	I think I am less happy because of my binge eating.	0.85	1.76
24	I can feel so upset from binge eating that I don’t get to things that I said I would.	0.64	1.53
25	I tend to lose control when I eat, despite my good intentions.	1.00	1.73
26	I can feel a strong desire to engage in sexual activity.	1.92	2.12
27	When it comes to sex/porn, I almost always want more.	1.08	1.68
28	I find myself thinking about how good it would feel to look at pornography or engage in other sexual behaviors.	1.22	1.82
29	Porn and sex can make me feel better, but it can also make me feel worse.	1.01	1.70
30	I am very preoccupied by sexual thoughts and/or desires.	0.86	1.50
31	Other people think my video-gaming is excessive	0.56	1.35
32	I lose track of time when I am playing video games.	1.25	1.93
33	Video games can make me less motivated to get other things done.	0.88	1.63
34	I neglect my physical health such as being active because of video games.	0.50	1.28
35	Sometimes it’s hard to get video games off of my mind.	0.61	1.36

Discussion

We aimed to develop a brief questionnaire that measures symptoms related to seven putative addictive behaviors: alcohol use, drug use, tobacco use, gambling, binge eating, hypersexual behavior, and video game playing. Few instruments adopt a transdiagnostic approach, which limits understanding of comorbidity among these behaviors. We collected data from three independent samples to: develop items; establish factor structure and reduce the total item pool to 35 items; confirm factor structure and examine support for reliability and construct validity.

Study 2 initially demonstrated problems with convergence, likely due to too many items. The decision was made to exclude items with very low levels of endorsement. Similar procedures have been used in other measurement development work (Cicero, Neis, Klaunig, & Trask, 2017). Study 2 also included independent EFAs for each behavior. Across all seven behaviors, the suggested number of factors to be retained consistently differed based on the method used. These discrepancies served to further underscore experts’ description of EFA as an art form, or at minimum subjective (Brown, 2014; Streiner, 2017; Worthington & Whittaker, 2016). In all cases, a single factor solution provided at least one item for each DSM symptom that loaded strongly onto a single factor solution. The findings aligned with the theoretical impetus, and in the absence of strong consensus between methodologies for retaining factors, was considered an appropriate approach.

In line with the iterative procedures for scale development, an independent CFA was also conducted on the Study 2 sample using the total items retained through EFA. Estimates revealed significant problems with fit. One model that offered modest improvements with respect to the RMSEA, TLI, and SRMR included symptoms that could be largely characterized as internalizing. Internalizing symptoms are distressing to the individual who experiences them, but the symptoms do not include overtly negative social behavior (O’Neil, Conner, & Kendall, 2011). Generally, mood and anxiety disorders are understood as internalizing disorders (Kotov, Gamez, Schmidt, & Watson, 2010),whereas SUDs and conduct disorders are considered externalizing disorders (Kotov et al., 2010). However, internalizing symptoms often occur in individuals with SUDs (O’Neil et al., 2011). In the current work, internalizing symptoms were thought to include: 1) using a larger amount or over a longer period of time than was intended, 2) making a physical or psychological problem worse and 3) craving, while externalizing symptoms included relationship problems and neglecting major roles.

For the smoking subscale, fit indices descriptively indicated that the internalizing model was the best fit. This is largely consistent with available literature, which critiques the utility and predictive validity of many of the DSM criteria such as neglect of major roles in tobacco use disorders (Baker, Breslau, Covey, & Shiffman, 2012). For the gambling items, fit indices supported the internalizing model. However, this finding is inconsistent with the available literature. The decision was made to retain items that would allow for either the internalizing or theoretical model, and Study 3 data was used to finalize the items for the gambling subscale. For the hypersexual items, the extant literature is largely lacking. Several small studies have documented neglect of major roles (Coleman, Miner, Ohlerking, & Raymond, 2001; Muench et al., 2007), but the rates of this particular symptom appear low. While more data are needed, the available research does support the importance of internalizing symptoms in hypersexual disorder (e.g., Forbes, Baillie, & Schniering, 2016).

As an initial step towards measure validation, Study 3 included responses to RAD items as well as a subsample of associated measures. It also included fewer individuals who identified as White and more individuals who identified as Black. As individuals from minority groups are often underrepresented in measurement development work (Redding, Maddock, & Rossi, 2006), this was consistent with the aim of the sampling strategy. In contrast to earlier analyses, robust least squares estimation was used to calculate fit and parameters, and the accuracy of factor loadings likely improved as a result of this estimation method. The total model included one higher order factor of addiction and seven latent factors defined by five observed indicators reflecting a range of DSM criteria. All subscales loaded onto the latent factor, though differences were observed in the relative strengths of the loadings. The binge eating subscale loaded the least strongly. Future research is needed to replicate this finding. However, it is possible that individuals do not identify with the term “binge eating”, but still experience some of the symptoms assessed by RAD, which were developed to be consistent with DSM-5 SUD criteria rather than feeding and eating disorder criteria. (American Psychiatric Association, 2013). Evidence in support of and challenging an addictions perspective of specific eating problems abounds (Finlayson, 2017; Gordon, Ariel-Donges, Bauman, & Merlo, 2018; Hebebrand et al., 2014). Some of the strongest evidence supports food addiction, a related but distinct type of eating pathology, as closely related to addiction (Davis, 2013). Future research should explore the relationship between the RAD binge eating subscale and other validated measures of food addiction and binge eating. Researchers might also consider omitting the term “binge eating” in favor of more neutral terminology.

Fit indices indicated excellent fit for the total model, thus providing empirical support for a 7-factor measure. The data demonstrate support for utilizing the measure to briefly assess, based on continuous measurement, for symptoms of discrete conditions as well as using a total score. The measure is consistent with major transdiagnostic theories of addiction, including Shaffer’s addiction syndrome model (Shaffer et al., 2004), and Griffiths’ component model of addictions (Griffiths, 2005). The measure is unique as compared with other available transdiagnostic assessment tools because it was developed based on major findings from NESARC-II (Hasin et al., 2013). Specifically, the NESARC-II study, which was the largest epidemiologic study completed to date, published data on the ability of various DSM-5 indicators to classify people with and without a substance use disorder. Items for this measure were developed to align with DSM-5 indicators that demonstrated strong discrimination. The measure is also brief and considers a wide range of behaviors.

There was overall support for construct validity based on associations with related measures. The strongest linear relationships were observed for RAD total score or subscale scores and levels of depression, impulsivity, and emotion dysregulation. Emerging evidence supports emotion regulation as a transdiagnostic feature of anxiety disorders, depressive disorders, eating disorders, and SUDs (Sloan et al., 2017). Sloan and colleagues (2017) also found a significant positive effect for changes in emotion regulation following effective treatment for a range of disorders, including SUD treatment. The RAD scale could be useful in these contexts; if initial data demonstrate that a patient has problems with a range of addictive disorders, a clinician could work to identify the process underlying most of the behaviors as compared with more targeted interventions (e.g., reducing exposure to triggering situations for a specific drug).

These preliminary data also demonstrated demographic differences. Men endorsed greater symptoms, which is consistent with the literature that supports higher rates of alcohol use disorder (Bridget F. Grant et al., 2015), drug use disorder (Bridget F Grant et al., 2016), gambling disorder (Black & Shaw, 2019), hypersexuality (Kafka, 2010), and excessive video game playing (Gentile, 2009) among men. More research is needed to understand variables mediating this relationship, including possible neurobiological mediators. Research has demonstrated that after controlling for disorder severity, male gamers as compared with female gamers show significantly higher activation in the brain regions associated with craving, when exposed to cues (Dong, Wang, Du, & Potenza, 2018). The data also demonstrated that individuals who were younger and had lower education reported greater symptoms which is consistent with epidemiologic data for alcohol and drug use disorders. Replication is needed in both non-clinical and clinical samples to confirm these finding.

These results must be interpreted within the context of several important limitations.

First, there was relatively low endorsement of most behaviors. This is consistent with the broader population for whom this measure was intended, but it impacts the items that were retained. The relationship between the indicators and the latent factor may be unduly influenced by those who have only low levels of the behavior. Importantly, some of these issues represent important future directions, which could include sensitivity and specificity analyses. The second major limitation associated with low endorsement rates involves the distribution of data and accuracy of estimation when skew or kurtosis is present. Extreme skew or kurtosis can cause less reliable estimation in SEM (Li, 2016). This limitation is especially salient for Study 2, where attempts to use more robust estimators were unsuccessful. However, the confirmation of the proposed factor structure using a more appropriate estimator in Study 3 alleviates concern related to this limitation. A final consideration includes the limited oversight and verification of self-reported data on online platforms, particularly for inclusion criteria. A series of experiments by Chandler and colleagues (2017) showed that prescreening questions for eligibility purposes may be associated with fraudulent responding. In the current study, the only prescreening question included self-identified race for a subset of participants in Studies 2 and 3, which aimed to recruit individuals who identify as racial/ethnic minorities. It is possible that individuals could have learned the eligibility criteria and reattempted the survey by fraudulently identifying their race. As a consequence, findings related to race, which were all non-significant in the current study, should be interpreted cautiously. Moreover, since the time that data analysis was completed on this study, protocols have been published which allow for systematic assessment of Mturk data validity (e.g., see Kennedy, Clifford, Burleigh, Jewell, & Waggoner, 2018 for details). Data could not be analyzed in this manner retrospectively as studies were sequentially dependent. That is, data from retained cases in study 2 determined which questions individuals were exposed to in study 3.

Other limitations of the current study are largely consistent with most similar measurement development studies and represent future directions as much as limitations. For example, the current study investigated limited forms of reliability and validity. Future research should investigate additional psychometric properties such as test-retest reliability and criterion-related validity, which would also allow for the development of cut-scores. Additionally, while data support the seven identified behaviors, growing literature supports others potentially addictive behaviors that were not considered here, including cannabis, internet addiction, and compulsive buying. Finally, investigations related to further psychometric properties of the RAD scale within more heterogeneous samples including clinical samples also represent important future directions.

The implications for this work also extend beyond assessment to include treatment. There is growing interest in understanding the utility of treatments proven effective for other similar conditions, rather than disorder-specific treatment. For example, there is new evidence supporting the efficacy of a combination of bupropion and naltrexone for binge eating disorder (Guerdjikova et al., 2017), which was traditionally marketed for alcohol use disorder or cocaine use disorder (Garbutt et al., 2005; Schmitz, Stotts, Rhoades, & Grabowski, 2001). This transdiagnostic approach is also evident in psychotherapy research, including transdiagnostic manualized treatments (e.g., Unified Protocol; Farchione et al., 2012) as well as the adaptation of treatments to address commonly co-occurring problems (e.g., DBT for concurrent eating disorder and SUD; Courbasson, Nishikawa, & Dixon, 2012). Ultimately, such research and treatment in naturalistic settings may become more feasible if identifying comorbid conditions is easier, which can be accomplished through use of the RAD scale.