A Psychometric Evaluation of the Marijuana Problems Index Among College Students: Confirmatory Factor Analysis and Measurement Invariance by Gender

Abstract

Although the Marijuana Problems Index (MPI) is widely used in studies with college student samples to reflect a unidimensional measure of cannabis-related problems, no studies have assessed the psychometric properties of the MPI in a college student population. The present study sought to resolve this gap in a sample of 879 college students reporting past-year cannabis use. Confirmatory factor analyses were used to test the factor structure of the unidimensional 23- and 18-item MPI and measurement invariance across gender. Bivariate correlations between the final factors, cannabis use history/frequency, and other substance use outcomes were used to examine concurrent and discriminant validities (i.e., vs. noncannabis outcomes). The 18-item (but not the 23-item) MPI demonstrated good model fit, measurement invariance across gender, adequate internal reliability, as well as concurrent and discriminant validities. Results support the use of the 18-item MPI over the 23-item MPI for conceptualizing problematic cannabis use, including the testing of gender-specific differences, among college students. Findings also reinforce the importance of evaluating the psychometric properties of widely used measures across samples.

The Marijuana Problem Index (MPI)¹ is a measure adapted from the Rutgers Alcohol Problem Index (RAPI) to index problem severity specific to cannabis use among adolescents and college students/young adults (Johnson & White, 1989; White & Labouvie, 1989). Both the original 23-item and slightly shortened 18-item versions of the MPI have been widely used as criterion markers to validate other measures, intervention outcomes, and conceptual/theoretical outcomes (Blevins et al., 2016; Elliott et al., 2014; Simons et al., 1998; Vandrey et al., 2005). The 23-item MPI has even been included as the primary indicator of cannabis problems in the major National Institutes of Health’s (NIH) Adolescent Brain Cognitive Development (ABCD) Study evaluating trajectories of substance use and related consequences across adolescence (Lisdahl et al., 2018). Despite widespread use, there has been fairly limited work evaluating the psychometric properties of either version of the MPI alone and none contrasting the utility of one version over the other.

The primary evidence that the MPI is psychometrically sound is generally based upon studies conducted on the RAPI (e.g., α > .80; Simons & Carey, 2006; Simons et al., 1998; Vandrey et al., 2005). However, psychometric studies on the RAPI have yielded conflicting findings in terms of the number of factors or items that can be reliably detected, particularly when also considering potential gender biases (Earleywine et al., 2008; Martens et al., 2007; Neal et al., 2006; White & Labouvie, 1989). For example, while the initial RAPI study reported that the 23 items fit a three-factor model best, the high correlations between factors (rs ranging from .52 to .68), overlapping content, and goal of creating a simple, clinically useful tool led to the recommendation of using a one-factor model (White & Labouvie, 1989). The original findings appear to be reinforced by subsequent psychometric studies of the 23-item RAPI identifying one-, two-, and three-factor models as best fitting, with one-factor models being reported after dropping poorer performing items (Earleywine et al., 2008; Martens et al., 2007; Neal et al., 2006). The initial RAPI study also was referenced when White et al. (2005) introduced an 18-item version which dropped Items 14 (“Tried to cut down”), 17 (“Had a fight, argument, or bad feelings with a friend”), 18 (“Had a fight, argument, or bad feelings with a family member”), 20 (“Felt you were going crazy”), and 21 (“Had a bad time”). That there is fairly limited empirical consensus across which of the 23 items should be dropped, may help explain why the original 23- and 18-item RAPIs have continued to be most widely used over the other variations.

There are at least three important concerns regarding the lack of psychometric validation data of the MPI in college students. First, studies have largely relied on the initial validation work of the RAPI which included participants aged 12–21 years (Johnson & White, 1989; Simons et al., 1998; White & Labouvie, 1989). The only psychometric study of the MPI was also conducted with adolescents (Knapp et al., 2018). Confirmatory factor analysis (CFA) suggested a unidimensional structure of the 23-item MPI and positive correlations of the final factor with cannabis use frequency measures and Diagnostic and Statistical Manual of Mental Disorders 4th ed. (DSM-IV) diagnoses evidenced external validity (Knapp et al., 2018). However, cannabis patterns and consequences are quite distinct from that of alcohol and patterns of use often shift between adolescence and young adulthood alongside changes in autonomy, living arrangements, and availability (Miech et al., 2017; Simons et al., 2012; Staff et al., 2010). At the same time, demonstrating whether the measure is similarly reliable and valid in a young adult population would indicate the MPI may be suitable for longitudinal studies across the adolescent-emerging adulthood transition or drawing meaningful comparisons in MPI scores across these age groups. Second, although some studies additionally reference Cronbach’s α of prior studies, this is only one indicator of reliability (internal), that is predicated on the assumption that the scale is unidimensional and the items hold equal weighting (Dunn et al., 2014). Third, there are several versions of the MPI that vary the number of items (18–23), producing different possible ranges of scores (18–90 vs. 23–115) that make it difficult to draw meaningful comparisons of scores across studies (Elliott et al., 2014; Gaher & Simons, 2007). Yet, absent psychometric data on any of the MPI versions (e.g., 23 item vs. 18 item) or tests directly comparing their structures, future researchers are challenged to select versions based upon precedent or data collected on the RAPI or among adolescents which may not be appropriate. The absent psychometric data also limits the extent to which researchers can make an informed decision to select any version of the MPI over other existing cannabis problem measures such as the Cannabis Use Disorder Identification Test (CUDIT), Cannabis Problems Questionnaire (CPQ; Copeland et al., 2005), and the Marijuana Consequences Questionnaire (MACQ; Simons et al., 2012).

Ideally, tests of factor structure, reliability, and validity are conducted among each unique sample and configuration of tests prior to being used for more conceptually focused research questions (Slaney et al., 2009; Thompson, 2003, 2004). Then, before examining group-based differences on the measure (e.g., determining gender differences on the number or severity of cannabis-related problems), the psychometric models are determined to be equivalent across the groups of interest (Jöreskog & Sorbom, 1979; Marsh et al., 2018; Meredith, 1993). Despite increased interest in potential gender differences in cannabis use problems, no published studies have evaluated whether the MPI factor structure as an index of cannabis problems can be interpreted similarly across gender (Kerridge et al., 2018). Thus, it is unclear the extent to which men and women evidence true differences in the number or severity of cannabis-related problems or, alternatively, are differentially interpreting the MPI items.

The aims of the present study were to examine the psychometric properties of both the original 23-item and the briefer 18-item MPI. Based on prior work and theory, we expected the CFAs of both the 23- and 18-item MPIs to fit a unidimensional structure and demonstrate comparable fit to each other (Knapp et al., 2018; White & Labouvie, 1989). As prior studies have not evaluated measurement invariance of the MPI and there have been some inconsistent findings on the RAPI for alcohol, we tentatively expected the final structures to be invariant across gender (Earleywine et al., 2008; Neal et al., 2006). Then, internal reliability, concurrent validity, and discriminant validity were examined for the final factors across gender identity. For concurrent validity, the MPI was expected to be negatively correlated with age of first cannabis use (indicating those who started using at a younger age have more current problems) and positively correlated with use frequency. For discriminant validity, the correlations between the MPI and other substance use frequency/problem variables were expected to be smaller than that between the MPI and cannabis use frequency variables.

Method

Participants

We report how we determined our sample size, all data exclusions, all manipulations (not applicable), and all measures in the study. Participants were drawn from a larger, ongoing online survey study (N = 2,107) focused on polysubstance use and psychological well-being among college students. Sample size for the present study was determined by all available cases who met inclusion/exclusion criteria. No manipulations were employed and this study was not pre-registered. Participants were recruited from a general undergraduate subject pool (49.2% psychology majors) at the University of North Texas, in which cannabis use is primarily illegal for medical and recreational purposes (some medical exceptions at the time of data collection included low-tetrahydrocannabinol [THC] cannabidiol for patients with intractable epilepsy). Data were collected every semester from November 2016 through February 2018. Participants were included in the final sample (N = 879) if they consented and met the following eligibility criteria: (a) age 18–25 years, (b) reported use of cannabis at least once in the past year, and (c) identified as cisgender men or women.² All procedures were approved by the University of North Texas Institutional Review Board prior to participant contact. See Table 1 for additional demographic and cannabis use descriptives of the full sample. For additional data and study materials, please contact the first or third author.

Table 1.

Descriptive Statistics for Total Sample, Validation Sample, and Confirmatory Sample

Variable	Possible range	Total sample (N = 879)	Ciswomen (n = 633)	Cismen (n = 246)
Age (M, SD)	18–25	20.13 (1.74)	20.03 (1.68)	20.40 (1.88) **
Sexual orientation
Heterosexual		686 (78.0%)	486 (76.8%)	200 (81.3%)
Gay		24 (2.7%)	3 (0.5%)	21 (8.5%) *
Lesbian		11 (1.3%)	11 (1.7%)	0
Bisexual/pansexual		143 (16.3%)	120 (19.0%)	23 (9.3%) *
Asexual		8 (0.9%)	8 (1.3%)	0 *
Other not listed		7 (0.8%)	5 (0.8%)	2 (0.8%)
Race/ethnicity (n, %)
Asian		43 (4.9%)	27 (4.3%)	16 (6.5%)
Black		124 (14.1%)	86 (13.6%)	38 (15.4%)
White		407 (46.3%)	300 (47.4%)	107 (43.5%)
Hispanic		188 (21.4%)	132 (20.9%)	56 (22.8%)
Native American		2 (0.2%)	2 (0.3%)	0
Other		14 (1.6%)	10 (1.6%)	4 (1.6%)
Multiracial		101 (11.5%)	76 (12.0%)	25 (10.2%)
Cannabis use (M, SD)
Age first used	5–25	16.99 (6.47)	17.03 (7.50)	16.87 (2.20)
Past-year use frequency	1–7	4.61 (1.96)	4.53 (1.93)	4.82 (2.03) *
Past 30-day use frequency	1–7	3.15 (2.29)	3.04 (2.23)	3.42 (2.41) *
Ever co-administered (n, %)	0–1	543 (61.8%)	386 (61.0%)	157 (63.8%)
MPI sum (M, SD)
23 item	23–115	32.36 (13.12)	31.56 (12.31)	34.53 (14.87) ***
18 item	18–90	25.42 (10.71)	24.73 (10.03)	27.28 (12.17) ***
AUDIT sum (M, SD)		6.4 (4.74)	6.29 (4.72)	6.69 (4.78)
Other substance use frequencya (Mdn, IQR)
Past-year alcohol use	1–7	5.00 (1.59)	5.00 (1.58)	4.98 (1.62)
Past-year LSD use	1–7	1.99 (1.00)	1.93 (0.95)	2.08 (1.09)
Past-year psilocybin use	1–7	1.77 (0.90)	1.65 (0.65)	1.93 (1.13)
Past-year MDMA use	1–7	1.99 (1.14)	1.93 (0.98)	2.18 (1.52)
Past-year cocaine use	1–7	2.68 (1.53)	2.80 (1.57)	2.43 (1.42)
Past-year Ritalin misuse	1–7	1.84 (1.36)	1.84 (1.42)	1.83 (1.27)
Past-year Adderall misuse	1–7	2.79 (2.01)	2.74 (1.98)	2.92 (2.10)
Past-year anxiolytic misuse	1–7	2.14 (1.44)	2.10 (1.37)	2.24 (1.61)
Past-year painkiller misuse	1–7	1.89 (1.15)	2.01 (1.23)	1.54 (0.83)

Note. MPI = Marijuana Problems Index; AUDIT = Alcohol Use Disorders Identification Test; IQR = interquartile range; LSD = lysergic acid diethylamide; MDMA = 3,4-Methylenedioxymethamphetamine. Bolded values indicate statistically significant differences in means or proportions by gender, tested via one-way ANOVA and chi-squared analyses, respectively.

^aOther substance use frequency coded such that 1 = “0 occasions,” 2 = “1–2 occasions,” 3 = “3–5 occasions,” 4 = “6–9 occasions,” 5 = “10–19 occasions,” 6 = “20–39 occasions,” 7 = “40 or more occasions.”

^*p < .05.

^**p < .01.

^***p < .001.

Measures

Cannabis Problems

Participants completed the 23-item Marijuana Problems Index (MPI), assessing the severity of problems associated with cannabis use in the past year on a scale of 1 = Never to 5 = More than 10 times (Johnson & White, 1995; Simons & Carey, 2006; White & Labouvie, 1989). The shortened, 18-item version excludes five items from the original version as reported in prior literature (Anderson et al., 2015; Davis et al., 2014; White et al., 2005). Scores could range from 23 to 115 on the 23-item scale and 18 to 90 on the 18-item scale, with higher scores indicated greater use related problems for both.

Cannabis Use and Co-Administration History

A 4-item self-report measure assessed cannabis use experiences for descriptive and validity analyses. Specifically, these questions asked about age at first use (responses ranging from 10 to 25 years), use frequency, and lifetime co-administration with other substances. For past-year and past-month marijuana use frequency, participants were asked two single face valid questions (“In the last [year/30 days], how many times have you used cannabis/marijuana?”) with responses ranging on a 7-point Likert-type scale: 1 = None, 2 = 1–2, 3 = 2–3, 4 = 4–5, 5 = 6–9, 6 = 10–19, 7 = 20+. Lifetime co-administration was assessed with a single face valid item (“In your lifetime, have you ever co-administered cannabis/marijuana with any other substance?”) in which participants could respond “Yes” or “No.” The wording of these items and associated responses are similar to those used in the Monitoring the Future Survey, which is regularly administered to nationally representative samples of young adults and has been used in similar studies assessing college student substance use (e.g., Bachman et al., 2015; Cloutier, Kearns, et al., 2019; Kearns et al., 2019).

Other Substance Use History

Participants were asked face valid questions about their frequency of past-year alcohol, lysergic acid diethylamide (LSD), psilocybin, 3,4-Methylenedioxymethamphetamine (MDMA), cocaine, Ritalin, Adderall, prescription anxiolytic medication, and prescription pain medication misuse. As with cannabis, responses ranged on a 7-point, Likert-type scale: 1 = None, 2 = 1–2, 3 = 2–3, 4 = 4–5, 5 = 6–9, 6 = 10–19, 7 = 20+. Participants also reported on their past-year alcohol problems via the Alcohol Use Disorders Identification Test (AUDIT; Babor et al., 2001).

Other Demographics/Relevant Covariates

Participants were asked to report on age, assigned sex (male, female, other [please specify]), gender identity (man, woman, nonbinary, other [please specify], and race/ethnicity).

Analytic Approach

Preliminary

First, descriptive statistics were computed. Chi-square and analyses of variance were used to identify whether demographic and cannabis-related characteristics differed across gender.

Primary Analytic Overview

The primary analyses occurred in three steps for both the 23-item and 18-item MPI: (a) evaluate the unidimensional factor structure(s) MPI with CFA; (b) test measurement invariance across gender with CFA; and (c) evaluate additional markers of validity across gender by correlating the factors with related variables. Because the MPI items are rated on a 5-point scale, the data were treated as ordinal and the factor structure was estimated using weighted least squares means and variances adjusted estimation (WLSMV; Hirschfeld & Von Brachel, 2014). Model fit indices were examined to evaluate model fit. Although cutoffs are somewhat arbitrary (Marsh et al., 2014), current conventions suggest that good model fit is indicated by comparative fit index (CFI) values ≥ .95, root mean square error of approximation (RMSEA) values < .06, and standardized root mean square residual (SRMR) values < .08 (Hu & Bentler, 1999) while acceptable fit can be indicated by CFI ≥ .90, RMSEA ≤ .10, and SRMR ≤ .10 (MacCallum et al., 2006; Vandenberg & Lance, 2000). Chi-squared values are also reported for comparative purposes, with lower values indicating better fit (Yuan & Bentler, 2004). Poor fit can be indicated by CFI < .90, RMSEA > .10, and SRMR > .10. Modification indices were considered when the standardized residuals were greater than .4 and made substantive sense because this indicated a potential relationship that was not explained by the model (Kline, 2016).

Measurement Invariance Across Gender

Measurement invariance tests whether identified groups (i.e., men and women) interpret the measure in a conceptually similar way and whether there are any items biased toward one group (Bialosiewicz et al., 2013; Jöreskog & Sorbom, 1979; Meredith, 1993). There are multiple levels reflecting increasing “strength” of invariance across groups that are tested in a sequence of models that increasingly restrain different parameters. The lowest level of measurement invariance (configural) tests whether the same factor structure fits the groups well. Next, the metric or factorial invariance model tests whether the factor loadings are the same across groups. Next, scalar invariance tests whether item intercepts are equivalent across groups and is needed in order to make mean comparisons on the factors across gender. Finally, the highest measurement invariance model (error/residual) tests whether the error variances for each item are equivalent across groups (Chen, 2007; Cheung & Rensvold, 2002; Meredith, 1993). According to Chen (2007), metric invariance is established if the difference between the CFI of the configural invariance model and metric invariance model is ≤.01, and the difference in the RMSEA is ≤.015 or the difference in SRMR is ≤.03. For the remaining invariance models, retention is based on whether the difference in CFI is ≤.01 and the difference in the RMSEA is ≤.015.

Reliability and Validity

The full sample was used to assess internal consistency as well as demonstrate concurrent and discriminant validities of the MPI. Each analysis was conducted on the total sample then separately by gender. For concurrent validity, Pearson bivariate correlations were used to assess associations between the MPI factor and age of first cannabis use, past-year cannabis use, and past-month cannabis use. For discriminant validity, partial correlations (i.e., controlling for cannabis use frequency) were computed to compare the association between the MPI and the frequency of using other substances and experiencing alcohol-related problems. To test whether the association between the MPI and each outcome was moderated by gender, linear and logistic regressions were conducted on the variables with continuous (i.e., age of first cannabis use, AUDIT sum scores, all other substance use frequency scores) and dichotomous (i.e., ever co-administered other substances with cannabis) outcomes, respectively. Only participants who endorsed lifetime use of the noncannabis substances were included for each analysis resulting in slightly different sample sizes reported.

Software and Data Management

The CFA and measurement invariance tests were conducted using RStudio (packages “lavaan,” “psych,” “semTools”; Jorgensenet al., 2018; Revelle, 2019; Rosseel, 2012). All the model fit indices reported in this study are based on robust versions of these model fit indexes (i.e., computed with adjusted standard errors). Descriptive statistics, reliability, and validity analyses in the full sample were conducted using SPSS (Version 26.0) and RStudio. Zero-order (r) and partial correlation coefficients ( pr) and p values were interpreted for statistical significance and effect size in the convergent and discriminant validity analyses. Assumptions for each statistical analysis were checked and met. Less than .03% of the data were missing with fewer than four unique cases missing data on any given MPI item. Given the lack of systematic missing patterns, pairwise deletion was used.

Results

Preliminary

As indicated in Table 1, there were some baseline gender differences in the present sample. Specifically, cismen were slightly older than ciswomen as well as more likely to endorse gay as a sexual orientation. Cismen also reported more frequent past-year and 30-day cannabis use frequency as well as higher scores on the MPI. No gender differences were observed in terms of race/ethnicity, alcohol-related problems, nor other substance use frequency.

Confirmatory Factor Analysis for Marijuana Problems Index

The fit indices for all of the models tested are shown in Table 2, and the factor pattern coefficients/loadings are shown in Table 3 along with item descriptions. Overall, the 23-item model evidenced poor-to-good fit across indices and the 18-item model evidenced acceptable-to-good fit. Given the proximity of the fit indices to the cutoffs, relatively large sample, and the primary purpose of evaluating the psychometric properties of the measure as it has been used (cf. identifying the best-fitting structure), we initially retained the unidimensional structure, items, and uncorrelated errors.

Table 2.

Model Fit Indices for Each Model Modification and Measurement Invariance Test

Model description	χ2	df	CFI	RMSEA [95% CI]	SRMR
23 items (uncorrelated errors)
Full sample	1875.76	230	.923	.078 [.075, .082]	.078
18 items (uncorrelated errors)
Full sample	1087.97	135	.946	.077 [.073, .082]	.077
18 items (correlated errors)
Full sample	590.65	134	.994	.054 [.049, .058]	.065
Measurement invariance fit indices for the 18-item MPI across gender
Label	χ2	df	CFI	RMSEA	SRMR
Ciswomen only (n = 633)	787.88	135	.941	.075 [.070, .081]	.084
Cismen only (n = 246)	311.85	135	.974	.064 [.055, .074]	.078
Configural	926.43	270	.946	.082	.079
Metric	1000.42	287	.962	.067	.081
Scalar	1010.15	340	.955	.067	.079
Means/structural	1047.93	358	.956	.066	.079
Measurement invariance fit indices for the 18-item MPI with Items 1 and 6 correlated across gender
Ciswomen only (n = 633)	463.31	134	.992	.055 [.049, .060]	.074
Cismen only (n = 246)	256.96	134	.997	.051 [.041, .060]	.067
Configural	720.27	268	.962	.07	.072
Metric	828.74	285	.971	.059	.076
Scalar	789.45	338	.968	.057	.072
Means/structural	831.00	339	.968	.057	.072

Note. N = 879. χ² are reported for comparative purposes, with lower values indicating better fit (Yuan & Bentler, 2004). CFI = comparative fit index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual. Good model fit is indicated by CFI ≥ .95, RMSEA ≤ .06, and SRMR ≤ .08 (Hu & Bentler, 1999). Acceptable fit is indicated by CFI ≥ .90, RMSEA ≤ .10, and SRMR ≤ .10 (MacCallum et al., 2006; Vandenberg & Lance, 2000). Metric invariance is indicated by a ΔCFI ≤ .01 and ΔRMSEA ≤ .015 or ΔSRMR ≤ .03. Scalar and means/structural invariance is indicated by ΔCFI ≤ .01 and ΔRMSEA ≤ .015 or ΔSRMR ≤ .01.

Table 3.

Factor Pattern/Structure Coefficients for 18-Item, Unidimensional MPI

Item	Question	f/p
1	Not able to do your homework or study for a test	0.743
2	Got into fights, acted bad, or did mean things	0.84
3	Missed out on other things because you spent too much money on marijuana	0.874
4	Went to work or school high or stoned	0.707
5	Caused shame or embarrassment to someone	0.826
6	Neglected your responsibilities	0.812
7	Relatives avoided you	0.903
8	Felt that you needed more marijuana than you used to use in order to get the same effect	0.807
9	Tried to control your marijuana use by trying to smoke marijuana only certain times of day or certain places	0.769
10	Had withdrawal symptoms, that is, felt sick because you stopped or cut down on smoking marijuana	0.855
11	Noticed a change in your personality	0.729
12	Felt that you had a problem with school	0.915
13	Missed a day (or part of a day) of school or work	0.855
15	Suddenly found yourself in a place that you could not remember getting to	0.793
16	Passed out or fainted suddenly	0.796
19	Kept smoking marijuana when you promised yourself not to	0.789
22	Felt physically or physiologically dependent on marijuana	0.865
23	Was told by a friend or neighbor to stop or cut down your marijuana use	0.891
14	Tried to cut down on smoking marijuana
17	Had a fight, argument, or bad feelings with a friend
18	Had a fight, argument, or bad feelings with a family member
20	Felt you were going crazy
21	Had a bad time

Note. n = 879. f/p = standardized factor pattern coefficients. Standardized factor structure coefficients can be computed by multiplying factor pattern coefficients with the factor correlations given in Table 4.

Measurement Invariance Across Gender

The configural invariance test for the 23-item MPI failed to converge because Item 2 received too few endorsements across response categories in relation to all 23 items. We explored dropping Item 2 and correlating error terms that made substantive sense (Items 1 and 6 described below); because these did not improve model fit nor meet criteria for configural invariance, they are not reported in detail to focus on results for the 18-item model.

As shown in Table 2, all three fit indices for the 18-item model with uncorrelated errors were acceptable for the configural invariance test, mixed for the metric invariance test (RMSEA and SRMR indicated acceptable fit, but ΔCFI was .016 above cutoffs of .01), and acceptable for the scalar and error invariance tests (Chen, 2007).³ Reevaluation of the modification indices for models among both men and women suggested correlating the error terms of Items 1 and 6; review of the item content suggested this could be theoretically justified as Item 1 is a more explicit example of Item 6. Incorporating this error substantially improved model fit to good fit across all metrics and satisfied all levels of measurement invariance. Meeting configural, metric, scalar, and error/residual variance means that the unidimensional structure, pattern coefficients, item means, and error terms of the 18-item MPI are measured and interpreted in a comparable way among cismen and ciswomen. As shown in Supplemental Analyses Table 2, similar results were obtained when participants who identified as gender minorities were included in the initial CFA models.

Reliability/Validity

The polychoric α for the final factor model was excellent (.97). The Cronbach’s α was also excellent in the total sample (α = .93), as well as among men (α = .93) and women (α = .92) separately. Please see Table 4 for concurrent and discriminant validity analyses for the 18-item MPI. As expected, in the total sample, the MPI was positively associated with past-year and past-month cannabis use frequency. However, age of first use was only negatively associated with MPI factor scores for cismen. When analyses were separated out by gender, the strength of the associations were fairly consistent among cismen and ciswomen. Also as expected, the partial correlations (i.e., controlling for use frequency) between both MPIs and all other substance use frequency variables were smaller and generally not statistically significant relative to comparable cannabis items, with the exception of modest, positive associates with alcohol problems. As shown in Supplemental Analyses Table 3, among participants identifying as gender minorities, higher MPI scores were associated with earlier first use of cannabis (similar to cismen) and positively associated with past-year cannabis (similar to cismen and ciswomen). However, the MPI was not associated with past-month cannabis use nor co-administration history (unlike responses from cismen and ciswomen).

Table 4.

Correlation Among the 18-Item MPI, Cannabis Use Frequency, and Other Substance Use Frequency in Total Sample

Variable	Total (N = 879)	Ciswomen (n = 633)	Cismen (n = 246)
Concurrent validity
Age of first cannabis use	−.03	.01	−.26**
Past-year cannabis use	.55**	.57**	.52**
Past-month cannabis use	.48**	.50**	.45**
Ever co-administered?	.33**	.34**	.31**
Discriminant validity
Alcohol problems (n)	.20** (779)	.20** (566)	.19** (213)
Past-year alcohol use (n)	<.01 (832)	.01 (602)	.01 (230)
Past-year LSD use (n)	.01 (112)	−.11 (69)	.13 (43)
Past-year psilocybin use (n)	.11 (76)	−.02 (41)	.16 (35)
Past-year MDMA use (n)	.13 (91)	.14 (70)	.03 (21)
Past-year cocaine use (n)	.02 (131)	.08 (91)	−.05 (40)
Past-year Ritalin misuse (n)	−.27 (18)	−.46** (13)	.22 (5)
Past-year Adderall misuse (n)	−.01 (118)	−.07 (84)	.17 (34)
Past-year anxiolytic misuse (n)	.15 (85)	.22* (61)	.07 (24)
Past-year painkiller misuse (n)	−.03 (46)	−.04 (37)	.19 (9)

Note. All cannabis use frequency coefficients are Pearson bivariate correlations; all other substance use frequency coefficients are partial correlations, controlling for past-year cannabis use frequency. Only participants reporting lifetime endorsements of noncannabis substances were included for the partial correlations resulting in different sample sizes for each analysis (see ns in parentheses below each correlation coefficient). LSD = lysergic acid diethylamide; MDMA = 3,4-Methylenedioxymethamphetamine.

^*p < .05.

^**p < .01.

As demonstrated in Table 5, the linear regression analyses revealed statistically significant main effects of the MPI and moderation effects of MPI × Gender on all four concurrent validity outcomes. For the nine discriminant validity tests, none indicated a statistically significant moderation effect, so detailed analyses were omitted.

Table 5.

Linear and Logistic Regressions Testing Moderating Effects of Gender and MPI on Cannabis Outcomes

	DV: age of first cannabis use			DV: past-year cannabis use			DV: past-month cannabis use frequency			DV: ever co-administered
Variable	ΔR2	b	p	ΔR2	b	p	ΔR2	b	p	ΔR2	b	p
Level 1 (main effects)	0.019			0.217			0.177			0.098
Past-year cannabis use frequency		−0.55	<.001		—	—		—	—		—	—
MPI		0.10	.002		0.11	<.001		0.11	<.001		0.10	<.001
Gendera		−0.14	.779		0.08	.544		0.17	.284		−0.15	.39
Level 2 (interaction)	0.005			0.007			0.006
Gender × MPI		−0.09	.041		−0.03	.006		−0.04	.015	0.007	−0.05	.04
Total model R2	0.024			0.224			0.183			0.11
Total model F/χ2 statistics	F(4, 845) = 5.205			F(3, 849) = 81.64			F(3, 849) = 63.28			χ2(3) = 68.32

Note. DV = dependent variable; MPI = Marijuana Problems Index.

^aGender is coded such that 0 = ciswomen and 1 = cismen; therefore, negative associations indicate a stronger effect for ciswomen and positive associations indicate a stronger effect for cismen.

Discussion

The MPI is widely used in studies with college student samples to assess cannabis problems (e.g., Lisdahl et al., 2018). Like the RAPI for alcohol, the 23 items of the MPI are generally summed together to reflect a unidimensional measure of problems. A recent validation study with adolescents from school and outpatient settings supported this structure (Knapp et al., 2018); however, this study did not examine whether the structure was invariant across gender and no studies have been conducted with college students. The present study sought to fill this gap by examining the psychometric properties of the 23-item and 18-item MPI among a college student population. Both versions fit a unidimensional structure, however, because the 23-item MPI model failed to converge, only the 18-item MPI was investigated further. The 18-item MPI without correlated terms had acceptable fit in the total model, but did not meet cutoffs for configural invariance; when Items 1 and 6 were correlated, the 18-item MPI and all subsequent measurement invariance tests met criteria for good model fit. Correlations of the 18-item MPI with cannabis use frequency and statistically nonsignificant associations with other substance use frequency also demonstrated concurrent and discriminant validities.

Findings generally support the stability and equivalence of the MPI across cismen and ciswomen. Specifically, findings indicated that these gender identity subgroups (a) conceptualized the MPI construct in a similar manner (i.e., equivalent factorial structure; configural invariance); (b) interpreted the MPI items in a similar manner (i.e., equivalent factor loadings; metric invariance); and (c) demonstrated uniformity in response styles on the MPI with no systematic variations in item scores (i.e., equivalent intercept thresholds; scalar invariance). Establishing configural, metric, and scalar invariance indicates that the MPI met the most stringent level of invariance expected in social science psychometric measurement (Little, 2013). With consideration given to the composition of the current sample (e.g., undergraduate and nonclinical sample), findings suggest that it may be reasonable for researchers and clinicians to administer, score, and compare the MPI score across cismen and ciswomen without measure modifications (Van de Schoot et al., 2012). Indeed, employing the 18-item MPI and other measures that meet this level of invariance is a foundational step for future conceptual work studying cannabis problems among emerging adult college students broadly and disentangling the effects of gendered experiences that may contribute to greater cannabis-related problems more specifically. In the present study, we were able to identify associations central to measures of cannabis problems that were consistent across gender while also revealing gender-specific discrepancies. For example, the medium–large correlations between past-year and past-month cannabis use frequency and the 18-item MPI were consistent across gender. However, negative correlations between the MPI and age of onset were observed among cismen only. This suggests that cismen may develop more problems at a faster rate relative to ciswomen who initiate cannabis at the same time and use as frequently or that cismen are more likely to have persisting problems relative to ciswomen. While these particular associations will need to be replicated in a longitudinal design, it nevertheless reinforces the need for invariant measures to identify and better understand cannabis etiologies that are stable across gender (e.g., more frequent cannabis use) and those that are gender-specific (e.g., sensitive windows for developing problems) for targeted intervention efforts (Cooper & Haney, 2014; Pearson, 2019; Sherman et al., 2016).

Importantly, findings from the present study should only be generalized to nonmedicinal cannabis use among cisgender men and women enrolled in college settings. The present study focused on cisgender men and women, in part, because the small sample of diverse gender minority students (n = 34 reporting 6+ different identities) limited statistical power for conducting analyses in a way that fully recognized the diversity of their identities. As shown in Supplemental Analyses, when participants who identified as gender minorities were included in the initial CFA models, overall fit indices fell from “good” to “acceptable” suggesting possible poorer fit. The validity analyses also indicated a stronger pattern of associations between the MPI and age of first cannabis use (r = −.43, p < .05) and past-year cannabis use (r = .53, p < .05). While this pattern could serve as preliminary evidence of the validity of the measure in this population, it could also simply be an artifact of running analyses on a small sample of heterogeneous gender identity groups. In contrast to cismen and ciswomen, the association between the MPI and past-month cannabis use frequency was not statistically significant (r = .25, p > .05). There is a growing body of work highlighting the unique problems gender, racial, and ethnic minorities face as well as the need to meaningfully evaluate the extent to which measures have been psychometrically validated across identity characteristics (Cloutier et al., 2021; Mayer et al., 2008; Meyer, 2003; Reisner et al., 2014, 2015). The present findings reinforce the need for future research explicitly designed to evaluate the psychometric properties of the MPI and other substance use problem measures among diverse populations to ensure valid and culturally relevant assessments. As the present study includes a majority of participants identifying as ciswomen, replication of the present findings is warranted. The present study also attempts to contribute to a larger effort to collect and report gender identity data more inclusively in the substance use literature, even when projects are not exclusively focused on minoritized populations, as this information can provide estimates of effect size for other research, including meta-analyses (Flentje et al., 2015). Although research from Monitoring the Future (Schulenberg et al., 2020) indicates similar past-year prevalence rates of cannabis use between college and noncollege young adults, future research may consider replication in noncollege populations. Replication efforts in states with different cannabis legalization statuses with samples reporting higher frequency cannabis use is also needed.

In contrast to prior theoretical and empirical work, the 23-item MPI model did not fit a unidimensional factor structure (e.g., Johnson & White, 1989; Knapp et al., 2018; Simons et al., 1998; White et al., 2005). First, there have been several variations in the MPI, ranging in item numbers (e.g., 17–22; Anderson et al., 2015; Maisto et al., 2011; White et al., 2005), time frames (e.g., past 30 days to lifetime; Elliott et al., 2014; Gaher & Simons, 2007; Simons et al., 1998), and response options (e.g., 3–5 response options; Knapp et al., 2018; Lee et al., 2010; Phillips et al., 2015). The measure used in the present study was most similar to that used by Knapp et al. (2018) in terms of items and rating scales, but varied in terms of time frame. Knapp et al. (2018) also conducted their study among treatment-seeking adolescents, suggesting differences as a function of population age, severity of use (i.e., clinical vs. nonclinical populations), or the time frame used in the instructions. It is possible our specific set of decisions resulted in poorer fit with the 23-item model that would not have occurred using a different time frame, response scale, or if a different population had been sampled (e.g., clinical adolescent sample; selecting only heavy cannabis users). Nevertheless, the findings shed light on this issue and the need to test the psychometric validity of these various combinations.

Additional limitations of the present study design offer several future research directions. First, the present study examined the psychometric properties of the MPI using a CFA framework. While necessary, it is not a sufficient way to establish all of the important psychometric properties or rule out all potential gender biases. For example, item response theory (IRT) and differential item functioning (DIF) analyses have been used to identify age and gender-biased items in the RAPI (e.g., Cohn et al., 2011; Earleywine et al., 2008; Neal et al., 2006) and similar work is warranted for the MPI. As several MPI instructions and items refer to “smoking,” future work should consider revising to be more inclusive of all products and consider the extent to which items might be biased as a function of mode/route of administration. The present cross-sectional study also did not include other tests of psychometric properties (e.g., test–retest reliability; predictive validity; concurrent validity with DSM diagnoses). Future research should employ longitudinal methods with multiple assessments and modes (e.g., clinical interviews) of cannabis-related problems, as well as among clinical populations, to replicate and extend the current findings. Second, the correlation of the errors for Items 1 and 6 could indicate that the 18-item MPI could be abbreviated even further. We retained the item in the present study to provide psychometric data for the closest approximation of the MPI as it is presently being used. Future research focused on abbreviating the MPI even further should consider these items in particular. Third, although the 18-item MPI generally indicated small, nonsignificant associations with the majority of other substance use outcomes, discriminant validity findings may be partially limited by moderate, statistically significant associations with alcohol. Notably, the nonzero association between cannabis and alcohol problem measures is consistent with both theoretical and empirical accounts of co-use; most cannabis users also report using alcohol (e.g., 92% of present sample used both in the past year) and many report problems across substances (Kearns et al., 2019; Knapp et al., 2019; Stinson et al., 2005; Yurasek et al., 2017). As polysubstance use is increasingly recognized as the norm, future research may consider more comprehensive assessments of both alcohol and cannabis use problems such as the MPI, RAPI, and Semistructured Clinical Interview for DSM-5 (SCID-5; First et al., 2015). Fourth, in place of measures adapted from alcohol, future researchers should consider developing cannabis measures “from the ground up” to identify cannabis-specific items/constructs (e.g., Metrik et al., 2017) and those relevant to poly- and co-use patterns (e.g., alcohol; tobacco; Fairman, 2015; Yurasek et al., 2017).

Finally, as data were collected in a state that had extremely limited allowances for medicinal use (e.g., epilepsy), the study was not designed to distinguish use for these purposes nor evaluate product potency. Medicinal cannabis patients may use products with low potency at a high frequency which may have reduced the strength of the associations between MPI scores and use frequency. Recent research also indicates the possible role of route/mode of administration and product potency on cannabis use-related consequences and emergence of long-term problems (e.g., cannabis use disorder; Cloutier et al., 2021; Prince & Conner, 2019; Spindle et al., 2019). Future psychometric research on cannabis-related problems/consequences that considers participants primary motive (e.g., medicinal vs. nonmedicinal/recreational use), route/mode of administration, and product potency is critically needed, particularly as legislative changes make more products more easily accessible (National Academies of Sciences Engineering and Medicine, 2017).

The present study sought to test the psychometric properties of the MPI—a widely used measure of cannabis problems modified from the RAPI. The findings provided support for the 18-item, but not the 23-item MPI. This revised structure had good model fit and satisfied the highest level of measurement invariance across gender. The 18-item MPI further evidenced adequate internal reliability as well as concurrent and discriminant validities. Overall, findings offer psychometric validation data on the 18-item MPI among college students and highlight the importance of establishing the psychometric properties for widely used measures across samples.

Public Health Significance.

With changing views on cannabis (e.g., legalization), there is a need to evaluate the psychometric properties of existing cannabis problem measures. Findings provide support for the psychometric properties of the 18-item but not 23-item Marijuana Problems Index (MPI). The revised MPI is a reliable, gender invariant, and psychometrically valid tool for researchers seeking to conceptualize cannabis problems among college students.