An Item Response Theory Analysis of DSM-IV Cannabis Abuse and Dependence Criteria in Adolescents

Abstract

Objective

To examine three aspects of adolescent cannabis problems: 1) do DSM-IV cannabis abuse and dependence criteria represent two different levels of severity of substance involvement, 2) to what degree do each of the 11 abuse and dependence criteria assess adolescent cannabis problems, and 3) do the DSM-IV items function similarly across different adolescent populations?

Method

We examined 5587 adolescents aged 11–19, including 615 youth in treatment for substance use disorders, 179 adjudicated youth, and 4793 youth from the community. All subjects were assessed with a structured diagnostic interview. Item response theory was utilized to analyze symptom endorsement patterns.

Results

Abuse and dependence criteria were not found to represent different levels of severity of problem cannabis use in any of the samples. Among the 11 abuse and dependence criteria, Problems cutting down and Legal problems were the least informative for distinguishing problem users. Two dependence criteria and three of the four abuse criteria indicated different severities of cannabis problems across samples.

Conclusions

We found little evidence to support the idea that abuse and dependence are separate constructs for adolescent cannabis problems. Furthermore, certain abuse criteria may indicate severe substance problems while specific dependence items may indicate less severe problems. The abuse items in particular need further study. These results have implications for the refinement of the current substance use disorder criteria for DSM-V.

INTRODUCTION

In anticipation of the Diagnostic and Statistical Manual of Mental Disorders, 5^th Edition (DSM-V), researchers are examining the utility of the DSM-IV criteria for substance use disorders ^{1, 2}. Many of these studies have examined the abuse and dependence constructs, and the individual criteria that comprise them. Much of the work that led to the distinction between abuse and dependence derives from studies of adults with alcohol use disorders ^{3, 4}. Few studies have examined DSM-IV criteria for cannabis use disorders, and in particular, whether diagnostic distinctions such as abuse or dependence are meaningful for these disorders. Furthermore, studies of adolescents are even less common.

One reason to examine DSM-IV criteria for cannabis abuse and dependence in adolescents is that it is the most commonly used illicit substance among U.S. adolescents ⁵. According to the Monitoring the Future study, in 2005 49% of 12^th graders reported having tried cannabis at some point in their lifetime, 30% used within the past month, and 5% smoked cannabis daily ⁵. Cannabis dependence develops in approximately 10–14% of adolescent-onset users versus in about 8% of adult-onset users ^6–8. Adolescents and young adults represent the majority of patients admitted to publicly funded treatment facilities for cannabis use disorders. Youths aged 12 to 17 constitute about two thirds of such admissions, with young adults aged 18 to 25 constituting most of the remaining third ⁹. Thus, cannabis use is frequent among adolescents, develops into dependence among a substantial proportion of users, and represents a significant public health burden.

Very few studies have examined the DSM-IV criteria for cannabis use disorders, in contrast to the extensive literature on alcohol disorders. An important point of discussion in the adult alcohol use disorder literature is the separation of alcohol abuse and dependence into separate constructs, where abuse is a less severe construct reflecting the social or legal consequences of drinking versus the more severe dependence construct reflecting the physiological, cognitive, and psychosocial aspects of dependence ³. Subsequent factor analytic ^10–13 and longitudinal ^{4, 14–16} studies examining alcohol as well as illicit drugs support this conceptualization. However, others examining the distinction at the item level have questioned it, showing weak support for the abuse construct ^{10, 11, 17, 18}, for the separation of abuse and dependence ^19–22, and for dependence being more severe than abuse ^{21, 22}.

To examine whether cannabis abuse and dependence criteria represent different measures of problem cannabis use, Teesson et al ²³ utilized factor analytic methods to examine the structure of cannabis abuse and dependence in an Australian adult population sample. Cannabis abuse and dependence reflected a common liability, with abuse and dependence factors correlating at 0.99. On the other hand, Winters et al. ¹⁶ found evidence for the abuse-dependence distinction with adolescent cannabis use. Other studies found good support for the validity of cannabis dependence, but less support for abuse ^{10, 11, 24, 25}, consistent with studies examining alcohol and other drugs ^{17, 18}. However, more studies examining the properties of individual abuse and dependence items need to be conducted before any firm conclusions can be drawn.

Item Response Theory (IRT) is a method that allows researchers to examine not only the abuse and dependence constructs, but the properties of the criteria that comprise them. Typically, researchers who study abuse and dependence either examine these categories as defined by DSM-IV, or utilize symptom counts as a continuous measure of substance involvement. When developing constructs such as abuse or dependence, factor analysis is a commonly used method to test whether individual criteria cluster together in a single “factor” ¹². IRT has a few advantages over factor analysis; when examining a series of items such as the DSM-IV substance use disorder criteria, rather than simply adding the number of criteria endorsed by an individual, IRT examines which particular criteria are endorsed by each individual, allowing that criteria may not have equal “weight” in predicting substance use problems. Thus, IRT estimates how informative a particular item is as well as how useful it is in predicting the severity of pathology in those who endorse it. Heuristically, it is helpful to think of IRT through the example of a test of mathematical ability – easier questions will be answered correctly by more of the test takers and harder questions by fewer. Thus, instead of simply “counting” how many questions one answered correctly, IRT examines which questions were answered correctly and judges the “mathematical ability” of the test taker by this pattern. Similarly, each question would be rated as “harder” or “easier” based upon the patterns of endorsement. In the context of substance problems, certain criteria may be endorsed only by more severely affected individuals who endorse many symptoms, whereas other criteria may be endorsed by many, including those who endorse only one or two symptoms.

A few studies have used IRT to examine the DSM-IV cannabis abuse and dependence criteria. Langenbucher et al. ²² examined alcohol, cannabis, and cocaine criteria in 372 adults in treatment for substance abuse and found no evidence supporting the separation of abuse and dependence into separate diagnoses. Teesson et al. ²³ performed an IRT analysis of the DSM-IV cannabis items in a general population sample of adults and found that three of the four cannabis abuse items showed high severity levels, suggesting that abuse may indicate greater severity of substance use problems than previously thought.

Very few studies have examined the DSM-IV criteria for cannabis use in adolescents, and it is important to do so because adolescents may show different abuse or dependence patterns than adults. For example, alcohol abuse symptoms are more common in adolescents and young adults than in adults ²⁶. Winters et al. ¹⁶ reported that DSM-IV criteria resulted in more alcohol and cannabis abuse diagnoses and fewer dependence diagnoses compared to theDSM-III-R in an adolescent clinical sample. Martin et al. ²⁷ utilized IRT methods to examine the DSM-IV criteria for both alcohol and cannabis in an adolescent clinical sample. They found no evidence supporting the separation of abuse and dependence items for adolescent alcohol or cannabis disorders, and also found no evidence supporting the greater severity of dependence over abuse. Their results have not been replicated in other adolescent samples, and it is particularly important to conduct these analyses in community samples, as highly selected samples may be less useful for estimating the characteristics of individual items in a way that will be informative in the development of diagnostic definitions.

Adolescents may also show different criteria endorsement patterns than adults. For example, in a review of studies examining alcohol use disorders in adolescents, Martin & Winters ²⁶ reported that adolescents don’t often experience dependence symptoms such as Withdrawal or Use despite health problems, or abuse symptoms such as Use in hazardous situations (e.g. driving drunk) or Legal problems, to the extent that adults do. Also, adolescents tend to highly endorse the Tolerance criterion; however, Tolerance has shown limited utility in differentiating adolescents with mild substance use problems from those with more severe ones, for both alcohol ²⁷ and cannabis ²⁸. Utilizing factor analysis, Teesson et al. ²³ found that Use in hazardous situations and Legal problems displayed weak loadings when examining the DSM-IV items for cannabis in an adult population sample, and Langenbucher et al. ²² found that Tolerance and Legal problems displayed weak loadings in their adult clinical sample, suggesting that these symptoms may not be very informative for adolescent substance use problems. However, these latter two results need replication in adolescent samples.

Finally, it is important to examine whether the DSM-IV substance use criteria are adequate for use with different populations. Adolescents in treatment may report different criteria than adolescents in the community, and a good diagnostic system should be useful for those with milder substance abuse problems as well as the more severe ones. Most of the literature examining DSM-IV substance abuse and dependence criteria has focused on alcohol, and the few studies focusing on cannabis criteria have primarily examined adult populations.

We utilized Item Response Theory to address the following questions: One, do the cannabis abuse and dependence criteria in the DSM-IV reflect two non-overlapping levels of severity in adolescents, in which dependence is more severe than abuse? Two, to what degree do each of the 11 items assess adolescent cannabis problems? And three, do the DSM-IV items function similarly across different adolescent populations? We examined these questions in three samples of adolescents: a clinical (treatment) sample, an adjudicated sample, and a community sample. Our sample is larger and provides greater diversity of substance use severity than seen in the above-cited studies.

METHOD

Sample

We examined 5587 adolescents aged 11–19 from three different samples (Table 1): a clinical sample of youth recruited from a substance abuse treatment center, a sample of adjudicated youth, and a community sample. The data examined in this project come from studies that have IRB approval and federal certificates of confidentiality. All data were de-identified. Consent/assent forms were obtained from each subject.

Table 1.

Sample demographics

Sample	N	Mean Age	Gender (%)	Ethnicity (%)
Combined	5587	15.3 (2.1)	M = 58 F = 42	Cauc: 78 Hisp: 14 AA: 3 Other: 5
Clinical	615	15.8 (1.2)	M = 92 F = 8	Cauc: 50 Hisp: 37 AA: 9 Other: 4
Adjudicated	179	16.3 (1.6)	M = 77 F = 23	Cauc: 63 Hisp: 14 AA: 10 Other: 13
Community	4793	15.2 (2.2)	M = 53 F = 47	Cauc: 82 Hisp: 11 AA: 2 Other: 5

Note. Standard deviation in parentheses; Cauc = Caucasian, Hisp = Hispanic, AA = African American, Other = other ethnicity or biracial.

Clinical sample

Adolescents from this sample come from the Colorado Family Study, a component of the Center for Antisocial Drug Dependence (CADD; DA 11015). Over 600 adolescent probands were recruited from 1993 to 2003 from an adolescent substance abuse treatment center affiliated with the University of Colorado. Initial ascertainment and interviewing was carried out when probands were entering residential or intensive day treatment for adolescent substance dependence. Details regarding this sample have been reported elsewhere ²⁹.

Adjudicated sample

Every year, millions of adolescents are arrested, a proportion of whom are adjudicated (convicted and placed on probation). Those youth who were adjudicated in the Denver metropolitan area were contacted by phone and mail and invited to participate in a study of the family transmission and comorbidity of adolescent substance use and conduct disorder (DA 015522). Those who participated (n = 202) were interviewed between 2001 and 2006 using the same core instruments utilized in the clinical and community samples to allow for comparability across samples.

Community sample

This sample includes adolescents from the Colorado Twin Registry, Colorado Adoption Project, and Family Control Samples, which are all components of the CADD. The twin sample consists of 1400 general population adolescent twin pairs, and 400 siblings of twins. The adoption sample consists of 673 adoptees, matched controls and their biological siblings, followed since birth. A control sample collected to match the youth from substance abuse treatment programs includes approximately 400 families. This sample has been described elsewhere ²⁹.

Sample used for analysis

For clinical and adjudicated samples, only probands were analyzed (i.e. youth identified through treatment for substance use disorders or identified as being adjudicated). For the community sample, the analysis was conducted by weighting subjects within the same family to account for their relatedness. For example, data from two siblings within a family were each weighted .5 and .5, data from three siblings were weighted .33, .33, .34, etc. This weighting procedure resulted in 2060 total observations for the community sample. Basic demographic information for the combined and individual samples is presented in Table 1.

Measures

Subjects in all three samples were assessed for substance use with the Composite International Diagnostic Interview – Substance Abuse Module (CIDI-SAM) ³⁰, a structured, face-to-face interview designed to be administered by trained lay interviewers, which provides DSM-IV symptom and diagnostic information for ten drug classes. The CIDI’s reliability and validity ³¹ made a version of it the main assessment for DSM-IV Substance Field Trials and for the National Comorbidity Study ³². Its validity for use with substance dependent adolescents has been demonstrated ^{33, 34}. We currently use a version that allows us to compute symptom counts as well as generate DSM-IV abuse and dependence diagnoses.

Analyses

Factor Analyses

We conducted an exploratory factor analysis (EFA) of the 11 cannabis abuse and dependence items using Mplus software ³⁵. The factor analysis was conducted using tetrachoric correlations, which are more appropriate for binary data than Pearson correlations ³⁶. Large ratios of first to second eigenvalues and a better fit of single versus multiple factor models were utilized as evidence for unidimensionality^{22, 37}.

IRT analyses

For simplicity, each of the 11 DSM-IV substance abuse and dependence symptoms will be referred to as “items” throughout this paper. PARSCALE ³⁸ was utilized to perform IRT analyses of the 11 items. A two-parameter IRT model was utilized, in which both item discrimination (a) and item severity (β) parameters were estimated. The item discrimination (a) parameter represents the ability of that item to discriminate among those with latent trait levels (i.e. cannabis problems) above and below the item’s severity threshold. A higher estimate indicates better discrimination and that the item is a “good” one. The severity (β) parameter (or threshold) represents the latent trait z-score where the probability of endorsing that item is 0.5. Thus, an item with a severity estimate of 1.5 suggests that the item is endorsed 50% of the time by subjects who are at a severity level of 1.5 standard deviations above the mean on the latent cannabis problems trait. Item severity is a function of the percentage of endorsement as well as the pattern of endorsement. In other words, IRT examines not only how many subjects endorse a particular item, but which subjects endorse it. A high severity estimate for a particular item suggests that the item is endorsed less frequently and is endorsed primarily by those endorsing many other items as well, thus indicating greater severity of the latent trait in an individual. A useful measurement scale includes items with a range of severities across the latent trait, as well as high discrimination values.

In order to examine whether the DSM-IV cannabis items functioned similarly in clinical, adjudicated, and community samples, we performed a Differential Item Functioning (DIF) analysis for the items. This analysis examines whether symptom severity (β) parameters for each item differ significantly among the samples. Significant DIF suggests that an item indicates different levels of pathology in different samples ³⁹. DIF for the discrimination (a) parameters were not performed due to power limitations; while β is estimated from both item prevalence and item endorsement patterns, a is estimated from item endorsement only, and thus lacks the power necessary for DIF analyses.

RESULTS

DSM-IV cannabis symptoms and diagnoses

Table 2 displays the mean symptom counts as well as the percentage of DSM-IV cannabis abuse and dependence diagnoses in the individual and combined samples. Cannabis symptoms and diagnoses were highest in the clinical sample and lowest in the community sample, with the adjudicated sample falling between the two.

Table 2.

DSM-IV Cannabis symptoms and diagnoses for all samples

Sample	Mean Symptom Count	Abuse		Dependence
		%	freq.	%	freq.
Combined	.99 (2.2)	8.0	448	10.4	583
Clinical	5.1 (2.9)	28.3	174	58.6	360
Adjudicated	2.2 (2.7)	24.6	44	20.7	37
Community	.41 (1.3)	4.8	230	3.9	186

Note. Standard deviation in parentheses; maximum possible symptom count is 11; freq. = frequency

Item endorsement

The percentage of subjects in each sample who endorsed each DSM-IV cannabis abuse and dependence item is presented in Figure 1. For each item, endorsement was highest for the clinical sample and lowest for the community sample. The most highly endorsed items for all three samples included dependence items D1 (Tolerance) and D7 (Use despite health problems), and abuse item A4 (Social/interpersonal problems).

Figure 1.

Percent endorsement of DSM-IV cannabis abuse and dependence items for each sample

Exploratory Factor Analyses

EFA results showed a high eigenvalue ratio of first factor (eigenvalue = 7.09) to second factor (eigenvalue = 0.90) for the 11 DSM-IV items, and model fit was no worse for the more parsimonious single factor model compared with the fit of the two factor model, suggesting that the items collectively represent a single latent dimension.

IRT analys

Individual item parameter estimates are shown with the item characteristic curves (ICCs) in Figure 2. ICCs are a graphical display of both the discrimination (a) and severity (β) parameters for each symptom, and allow for comparison across symptoms. Item discrimination (a) estimates, which represent the item’s ability to discriminate among those with cannabis problems above and below the item’s severity threshold, varied from 1.31 to 2.48. Items D6 (Activities given up) and D2 (Withdrawal) showed the highest discrimination ability and items D4 (Problems cutting down) and A3 (Legal problems) showed the lowest discrimination.

Figure 2.

Item Characteristic Curves for DSM-IV abuse and dependence criteria

Item severity (β) parameters, which represent the cannabis problems z-score where there’s a 50% probability of endorsing that item, were scaled to represent z-scores in the community sample – in other words, the severity estimates represent severity z-scores for the general population. These estimates ranged from 1.87 to 2.87 standard deviations above the mean (Figure 2). In addition, when comparing the severity parameters across samples, the abuse estimates and dependence estimates were intermixed for all three samples, with both abuse and dependence items on the lower and upper ends of the severity spectrum (Figure 3). For example, those items indicating the greatest severity levels included abuse items A2 (Use in hazardous situations) and A3 (Legal problems), as well as dependence item D4 (Problems cutting down). Conversely, dependence items D1 (Tolerance) and D7 (Use despite health problems) indicated low severity levels across all three samples, as did abuse item A4 (Social/Interpersonal problems). These results suggest that the abuse items are not necessarily indicative of less severe pathology, and that some items may be more useful than others in identifying adolescents with more serious cannabis problems.

Figure 3.

Cannabis item severity estimate comparison across samples, in standard deviations

The Differential Item Functioning (DIF) analyses examined whether the severity parameters for each item differed significantly among the three samples. Omnibus results for these analyses showed that only two dependence items functioned differently across the three samples (Figure 4): D1 (Tolerance) and D3 (Larger amounts/longer period). However, there were significant differences among samples for three of the four abuse items (Use in hazardous situations, Legal problems, and Social/interpersonal problems). These results suggest that these five items indicate different levels of severity of cannabis problems in different samples.

Figure 4.

Differential Item Functioning (DIF) analysis results for DSM-IV cannabis abuse and dependence items

DISCUSSION

We utilized Item Response Theory (IRT) to examine the utility of DSM-IV cannabis abuse and dependence items in three adolescent samples: a substance abuse treatment sample, an adjudicated sample, and a community sample. This study is unique in that it is a large sample of both community and clinically ascertained adolescents, which provides information about the DSM-IV items at the population level as well as at more severe levels of cannabis use.

Abuse-dependence distinction

When we examined the DSM-IV cannabis item severity (β) parameters, we found that the abuse and dependence symptoms overlapped one another and varied from low to high severity. In other words, some abuse items indicated high severity of cannabis problems and some dependence itemindicated low severity of cannabis problems. For example, the dependence items D7 (Use despite health problems) and D1 (Tolerance) indicated lower severity levels in all three samples. These items were two of the more highly endorsed items (Figure 1), and thus were likely endorsed by those with low levels of cannabis problems as well as high levels. On the other hand, abuse items A2 (Use in hazardous situations) and A3 (Legal problems) tended to indicate a higher level of severity. This is similar to the results of Teesson et al. ²³, who reported that three of the four cannabis abuse items showed higher severity estimates than most of the dependence criteria. Thus, the cannabis abuse criteria may, in fact, indicate serious cannabis problems.

Overall, these results cast doubt upon the assumption in the literature that cannabis dependence is more severe than abuse, and add to a growing number of studies that have examined alcohol, cannabis, and other drugs in adolescents and adults and come to similar conclusions. And while the DSM-IV doesn’t explicitly state that a substance dependence diagnosis is more severe than an abuse diagnosis, it is implied by the fact that an abuse diagnosis cannot be assigned once criteria are met for dependence, and it is commonly stated outright in the literature. And while some have suggested that individuals with a dependence diagnosis fare worse than those with an abuse diagnosis ¹⁶, others have made it clear that the individual abuse items indicate high levels of severity of substance problems ²³, suggesting that some other factor, perhaps the number of symptoms required for diagnosis, may need more examination. Thus, the DSM-V might consider removing the requirement that an abuse diagnosis cannot be assigned if there is a dependence diagnosis, and should reconsider the number of symptoms required for an abuse or dependence diagnosis.

DSM-IV cannabis abuse and dependence item

In order to evaluate the individual functioning of the 11 DSM-IV items, we examined both the discrimination and severity parameters for each item. The symptoms D4 (Problems cutting down) and A3 (Legal problems) showed the lowest discrimination values, consistent with the study by Martin et al. ²³ that examined cannabis and alcohol symptoms in adolescents. Legal problems also did not load well in two other factor analyses of cannabis items in adults ^{5, 20}, suggesting that this symptom may be problematic in some way. As far as cannabis use disorders go, Legal problems may not have the same meaning or discriminatory ability that it does for alcohol use disorders. Cannabis is an illicit substance, and even though adolescents show greater levels of alcohol use than cannabis use, cannabis is currently the top reason for adolescent admissions to treatment ⁴⁰. Alcohol is also illegal for adolescents (in the U.S.), but perhaps the illicitness of cannabis (for all ages) makes all types of users (casual and problematic) more likely to encounter legal problems. Overall, the item Legal problems may need further study. The symptoms D6 (Activities given up) and D2 (Withdrawal) displayed the highest discrimination values, suggesting that they are particularly good at identifying adolescents with significant cannabis problems.

Examination of item severity estimates showed that the level of severity indicated for each DSM-IV item ranged from 1.87 to 2.87 SD above the population mean (Figure 2), suggesting that the items are more useful for identifying adolescents with severe cannabis problems. The dependence items D6 (Activities given up) and D2 (Withdrawal) displayed high severity; along with their high discrimination values, these items are likely very useful in identifying adolescents who have the most severe cannabis problems. Several items – dependence items D5 (Time spent obtaining) and D3 (Larger/longer), and abuse items A2 (Use in hazardous situations) and A4 (Social/interpersonal problems) – displayed similar parameter estimates and are clustered together in Figure 2, suggesting that they are somewhat redundant. However, these items provide a range of ways to identify adolescents with significant cannabis problems. Overall, the addition of items that identify adolescents with less severe cannabis problems would be useful.

Comparing items across samples

When we examined whether the item severity estimates differed significantly among the three samples, we found that they did for two of the cannabis dependence items (Tolerance and Larger amounts/longer period) and for three of the four cannabis abuse items (Figure 4). This suggests that some items indicate greater severity of cannabis problems in some populations more than others. An IRT analysis of the DSM-IV alcohol items examined this sample and also found that the same two items, Tolerance and Larger/longer, showed significant DIF (Gelhorn et al., unpublished data). After accounting for mean differences, both of these items were endorsed less often in the clinical sample. These two items tend to be more highly endorsed among adolescents ²⁸, who may naturally experience tolerance or find themselves consuming more of the substance than planned by virtue of simply being inexperienced users. In addition, it is interesting that the abuse items generally showed greater population differences, which may help explain why the abuse construct has less statistical support than does dependence. One study suggested that the DSM-IV abuse items may be difficult to operationalize, thus rendering them less reliable ²; however, no other studies have addressed this issue. Overall, more research is needed to determine the reason for these population differences, and whether they are specific to adolescents. Ideally, the DSM items would not show these differences.

The findings from this study should be interpreted while considering the following limitations: first, despite the large treatment and community samples, the adjudicated sample was small (n = 172) in terms of the power necessary to perform IRT analyses. Thus, the results for that sample should be interpreted with caution. Second, while the large community sample was nearly 50% female, the adjudicated and especially the treatment sample had fewer females. We did not have adequate power to test for sex differences across these samples. Finally, while the results for this study are generally consistent with several other studies examining alcohol and other drugs, they are limited to cannabis use in adolescents.

In summary, our study examined the DSM-IV cannabis items in three different samples of adolescents, representing the largest and most clinically diverse sample to examine these questions thus far. Consistent with several other studies examining both alcohol and other drugs, we found little evidence to support the abuse-dependence distinction and found that abuse items can indicate severe substance problems just as dependence items can indicate mild ones. The abuse items in particular need further study. These results have implications for the refinement of the current substance use disorder criteria for DSM-V.