The Adolescent Substance Abuse Goal Commitment (ASAGC) Questionnaire: An Examination of Clinical Utility and Psychometric Properties

Abstract

Commitment to change is an innovative potential mediator or mechanism of behavior change that has not been examined in adolescents with substance use disorders (SUD). The Adolescent Substance Abuse Goal Commitment questionnaire (ASAGC) is a 16-item measure developed to assess an individual’s commitment to his/her stated treatment goal. The objectives of this study are to explore the research and clinical utility of the commitment construct as measured by the ASAGC. During Sessions 3 and 9 of a 10-week SUD treatment, therapists completed the ASAGC for 170 13–18 year-old adolescents. An exploratory factor analysis was conducted on the ATAGC items. Concurrent validity with related constructs, self-efficacy and motivation for change, was examined as well. At both sessions, the factor analysis resulted in two scales – Commitment to Recovery and Commitment to Harm Reduction. The ASAGC scales were found to demonstrate a high level of internal consistency (alpha coefficients ranged from .92 to .96 over time). In contrast to the Commitment to Harm Reduction scale, the Commitment to Recovery scale consistently correlated with scales from the Situational Confidence Questionnaire assessing self-efficacy, evidencing concurrent validity. Similarly, the Commitment to Recovery scale was related to the Problem Recognition Questionnaire, providing further evidence of the validity of the ASAGC. The ASAGC is a reliable and valid clinical research instrument for the assessment of adolescents’ commitment to their substance abuse treatment goal. Clinical researchers may take advantage of the clinical utility of the ASAGC including its ability to differentiate between commitment to abstinence versus commitment to harm reduction.

1. Introduction

Significant progress has been made over the past 20 years in the development of evidence-based-practice treatment protocols for youth with substance use disorders (SUD)¹. Most interventions have been provided in outpatient settings where more than 80% of youth are treated ². The focus of outpatient treatment for youth has been on several therapeutic approaches and modalities including family/community therapies, cognitive behavioral therapy, motivational interviewing, and 12-step/fellowship meetings as reviewed in recent meta-analyses ^3–4, as well as integrated interventions reported in the benchmark cannabis Youth Treatment (CYT) study ⁵.

Despite prominent differences in theory, and design and methodology, studies employing various treatment modalities in youth with SUD have reported remarkably similar outcomes ³. Rates of adolescent relapse of substance involvement are comparable to those of adults during the first year post treatment completion ^{6, 7}. Research has shown that about sixty percent of adolescents continue to vacillate in and out of recovery after discharge from 3-month treatment programs ^{5, 8}. At this point, relatively little is known about mechanisms of behavior change (MBC) in adolescents receiving these interventions, which highlights the need to study the underlying processes involved ^9,10. Most evidence-based treatments are “theory-driven,” at least to some degree. However, meta-analyses examining the hypothesized mechanisms of action on which the interventions are based have not yielded clear results ^{11, 12} on how adults engaged in Alcoholic-Anonymous ¹³, Cognitive Behavioral Therapy ¹⁴ (CBT), and Motivational Interviewing ¹⁵ (MI) actually change by regularly scheduled treatment sessions over a prolonged period of time ^{16, 17}. Changes in self-efficacy ^18,19, coping skills ¹⁰, and motivation or readiness to change ²⁰, appear to account for some portion of treatment effects in the adolescent research literature.

It has been proposed that a higher order construct of motivation to change may reflect commitment to change by adhering to identified treatment goals²¹. Kelly and Greene recently developed a five-item commitment to sobriety scale for emerging adults 18–25 years of age ²¹. They argued that “in contrast to being motivated to change, being committed to change implies the presence of a stronger desire that is more compelling and forceful, and that may be less susceptible to the undulating future circumstances and contingencies that so often weaken resolve and make motivation fluctuating.” Hall and colleagues developed a single-item commitment to abstinence questionnaire for adults, which uses six response categories to differentiate the participant’s goals surrounding abstinence ²². This measure was validated by subsequent research ^{23, 24}.

At this time, we are not aware of an instrument measuring commitment to change tailored to specific treatment objectives in adolescents with SUD. We have developed a 16-item questionnaire, the Adolescent Substance Abuse Goal Commitment Questionnaire (ASAGC), to assess the adolescent’s commitment to his/her stated goal of substance abuse treatment (see appendix for the ASAGC). Although the ultimate goal of treatment is Recovery (i.e., abstinence/relapse prevention), some youth might choose a Harm Reduction goal (i.e., decrease only in frequency and /or severity of use), or might drift between the two goals at different points in the continuity of care ²⁵ from assessment and through treatment, aftercare or follow-up. Therefore, the instrument was designed to assess commitment to both of these two goals. The items included in the instrument were the result of a selective review process of multiple relevant items from the abstinence and harm reduction oriented literature generated by the authors before the onset of the study. The objectives of this study are to 1) introduce the construct of and the rationale for measuring commitment to treatment goals; and 2) examine the clinical utility and initial psychometric properties of the ASAGC.

2. Methodology

2.1. Participants and Procedures

A total of 294 adolescents were screened for the study. Of those screened, 235 met the eligibility criteria for participation. Of those eligible, 179 completed intake and signed consent forms. This study included individuals from intake who had complete data.

Specifically, the sample included 170 13–18 year-old adolescents (67% male). Most of the adolescents were Caucasian (79%); 13% were Latino, 4% were African American, and 4% were biracial/other. The mean age of the adolescents was 15.90 (SD=1.20). All of the adolescents were receiving treatment for a current DSM-IV diagnosis of an alcohol use disorder (29% met the criteria for alcohol abuse and 26% met the criteria for alcohol dependence) or consistent with eligibility criteria, had reported at least three days of drinking within the last 90 days. In addition, 77% of the adolescents met the criteria for cannabis use disorder (25% met the criteria for cannabis abuse and 52% met the criteria for cannabis dependence). Additional inclusion criteria included the ability to read and comprehend English at a fifth-grade level, not planning to move out of state for the next 12 months, and willingness to accept aftercare. Exclusion criteria included meeting substance dependence criteria for any substance other than alcohol, nicotine, or marijuana, a lifetime diagnosis of schizophrenia, suicidal ideation with a plan, suicidal behavior or self-injurious behavior in the past 30 days, or any current medical condition that would compromise their ability to participate in the study.

This study was a prospective, intent to treat study. The treatment phase consisted of ten weekly cognitive behavioral therapy sessions. For additional information relating to the design and outcomes, please refer to Kaminer et al. ²⁶. The protocol and informed assent and consent procedures from subjects and their guardians respectively were approved by the University of Connecticut Health Center’s Institutional Review Board.

2.2. Measures

Adolescent Substance Abuse Goal Commitment (ASAGC)

Therapists completed the ASAGC questionnaire for the participants during sessions 3 and 9 of treatment. The ASAGC assesses an individual’s commitment to his/her stated treatment goal. The ASAGC includes 16 items that are completed on a response scale ranging from 0=definitely not to 4=definitely committed. A representative item is “Does the adolescent express commitment to recovery (abstinence/relapse prevention) as a goal?”

Situational Confidence Questionnaire (SCQ)

The adolescents also completed the revised 39-item Situational Confidence Questionnaire ²⁷ at session 8. The SCQ was designed to assess perceived confidence to resist alcohol or substance use in high-risk situations. A sample SCQ item is “I would be able to resist the urge to use heavily if I had an argument with a friend.” The response scale ranges from 0 to 10, with higher scores reflecting more confidence in resisting substance use. The SCQ includes the following subscales: Unpleasant Emotions/Frustrations, Physical Discomfort, Social Problems at Work, Social Tension, Pleasant Emotions, Positive Social Situations, Urges and Temptations, and Testing Personal Control. The SCQ has been shown to be a valid and reliable instrument for use with adolescents ^{18, 19}.

Problem Recognition Questionnaire (PRQ)

The adolescents completed the 25-item Problem Recognition Questionnaire ²⁸ at session 5. The PRQ assesses both adolescent problem recognition and willingness to change drug use and seek treatment. A representative PRQ item is “Using alcohol or drugs is a real problem in my life.” The response scale is a 4-point Likert-type scale ranging from 1 = strongly disagree to 4 = strongly agree. The PRQ has been shown to be both a reliable and valid measure to assess motivation and readiness for treatment ²⁸. PRQ scores are trichotomized as follows: low recognition for treatment (PRQ score = 21–39), moderate recognition for treatment (PRQ score = 40–59), and high recognition for treatment (PRQ score = 60 or greater).

2.3. Data Analysis

An exploratory factor analysis was conducted on the ASAGC assessed during Session 3 and Session 9 of treatment. For the factor analysis, an oblique promax rotation was specified to allow for factors to be correlated. Conceptually, the decision was made to extract and compare 2, 3, and 4 factor solutions at each time of assessment. Cronbach’s alpha coefficients were calculated to measure homogeneity and to assess the internal consistency of the ASAGC factors. Concurrent validity was examined by correlating the ASAGC factors with scales from the SCQ. Analysis of variance (ANOVA) models also were conducted to examine the relationship between the ASAGC factors and the PRQ.

3. Results

3.1. Factor Analysis

At both sessions 3 and 9, the two-factor model provided the best statistical fit to the data. It also was the most parsimonious and made the most sense conceptually. The values for the Kaiser-Meyer-Olkin (KMO) test ²⁹, which measures the strength of the associations between variables, were .84 and .83 at Sessions 3 and 9, respectively. Of note, KMO values greater than .8 are considered good and indicate that the factor analysis adequately captures the variables ²⁹. Bartlett’s test of sphericity ³⁰ also was significant at both times of assessment (X²(120) = 1886.80, p<.001, X²(120) = 2101.15, p<.001, respectively), indicating that the correlation matrix was not an identity matrix.

The two factors that emerged reflect commitment to “Harm Reduction” and commitment to “Recovery.” Both factors had eigenvalues (EVs) greater than 1.0 (EVs for Harm Reduction were 7.11 and 8.10, and EVs for Recovery were 2.93 and 2.82, at Session 3 and Session 9, respectively). Although two additional factors had EVs of 1, they did not make sense conceptually and contained two-item factors. The Harm Reduction factor explained 44% of the variance at Session 3 and 51% of the variance at Session 9 and the Recovery factor explained 18% of the variance at both times of assessment. Importantly, the two factors were moderately correlated with one another (r = .39, p<.001, and r = .48, p<.001, at Sessions 3 and 9, respectively).

The factors and their respective loadings are shown in Table 1. An a priori decision was made to retain only items with a loading over .30. However, all of the items had factor loadings greater than .30. Therefore, none of the items were deleted. Of note, item #13 “Has the adolescent been active in planning for an alternative drug-free life style?” had a factor loading greater than .30 on both factors. However, conceptually, it was related more to the items in the Recovery factor and therefore was retained with that factor. Items from the Harm Reduction and Recovery factors subsequently were summed to create Harm Reduction and Recovery scale scores at each time of assessment. For Harm Reduction, the scale mean was 14.39 (SD=6.45) at Session 3 and 15.07 (SD=7.90) at Session 9. The respective scale means for Recovery were 10.78 (SD=7.82) and 11.05 (SD=7.98).

Table 1.

Pattern Matrices for the Adolescent Treatment/Aftercare Goal Commitment Measure

	Session 3		Session 9
Items	HR	R	HR	R
Has the adolescent been planning for continuing harm reduction?	0.93	-	0.93	-
Is the adolescent engaged/active in ongoing harm reduction?	0.90	-	0.92	-
Does the adolescent realize that recovery is an ongoing process requiring personal accountability?	0.85	-	0.80	-
Does the adolescent demonstrate commitment for continued harm reduction only?	0.85	-	0.90	-
Does the adolescent realize that harm reduction is an ongoing process requiring personal accountability?	0.85	-	0.93	-
Does the adolescent express commitment only to harm reduction?	0.84	-	0.93	-
Are the adolescent's expectations for harm reduction insightful of the high-risk conditions exposing him/her to relapse?	0.84	-	0.93	-
Are the adolescent's expectations for recovery insightful of the high-risk conditions exposing him/her to relapse?	0.82	-	0.81	-
Is the adolescent engaged/active in ongoing recovery?	-	0.94	-	0.92
Has the adolescent been planning for continuing recovery?	-	0.89	-	0.87
Does the participant demonstrate commitment to recovery?	-	0.81	-	0.96
Does the adolescent express commitment to recovery (abstinence/relapse prevention) as a goal?	-	0.65	-	0.86
Has the adolescent been engaged/active in alternative drug-free style?	-	0.63	-	0.66
Has the adolescent been active in planning for an alternative drug-free style?	0.42	0.48	0.52	0.46
Has the adolescent connected with self-help groups?	-	0.46	-	0.41
Has the adolescent been engaged/active with self-help groups?	-	0.48	-	0.40

Note. Only factor loadings greater than .30 are shown.

HR = Harm Reduction, R = Recovery.

3.2. Reliability

Cronbach’s alpha coefficients were calculated to examine the internal consistency of the factors. The alpha coefficients for both Harm Reduction (α=.94 and α=.96 at Sessions 3 and 9, respectively) and Recovery (α=.93 and α=.92 at Sessions 3 and 9, respectively) were excellent. It also is important to note that the factors were stable over time. The same items loaded on the Harm Reduction and Recovery factors at both times of assessment.

Validity

To examine the concurrent validity of the Harm Reduction and Recovery scales, Pearson-product moment correlations were calculated between these scales at Session 9 and the SCQ scales at Session 8.

As shown in Table 2, none of the correlations between the Harm Reduction scale and the SCQ scales were significant. In contrast, the Recovery scale consistently was associated with the SCQ scales. More specifically, commitment to recovery was positively associated with confidence to resist alcohol use when experiencing negative affect situations including unpleasant emotions (r = .17, p<.05), physical discomfort (r = .20, p<.05), and social tensions (r = .18, p<.05). Commitment to recovery also was positively related to confidence in resisting alcohol use when experiencing pleasant emotions (r = .23, p<.01) and positive social situations (r = .24, p<.01). Similarly, commitment to recovery was significantly related to confidence in resisting alcohol use when experiencing urges (r = .33, p<.001) and when testing personal control (r = .20, p<.05).

Table 2.

Pearson Product-Moment Correlations Between the ATAGC (Harm Reduction and Recovery) Scales and the SCQ Scales

Variable	1	2	3	4	5	6	7	8	9	10	Mean	SD
ATAGC Scales
1. Harm Reduction	--										15.07	7.90
2. Recovery	.48***	--									11.05	7.98
SCQ Scaless
3. Unpleasant Emotions	.03	.17*	--								47.85	23.22
4. Physical Discomfort	.12	.20*	.76***	--							27.55	11.94
5. Social Problems at Work	.00	.16	.82***	.73***	--						20.55	9.14
6. Social Tension	.13	.18*	.82***	.78***	.82***	--					32.68	14.39
7. Pleasant Emotions	.09	.23**	.56***	.57***	.52***	.51***	--				19.64	9.25
8. Positive Social Situations	.03	.24**	.53***	.38***	.35***	.40***	.58***	--			36.11	24.16
9. Urges and Temptations	.14	.33***	.65***	.55***	.51***	.58***	.60***	.83***	--		20.89	11.45
10. Testing Personal Control	.04	.20*	.72***	.63***	.68***	.67***	.57***	.67***	.81***	--	24.20	11.66

^*p<.05;

^**p<.01;

^***p<.001.

To further examine validity, the relations between the Harm Reduction and Recovery scales and the PRQ were examined. As noted, the PRQ score was trichotomized into three groups: Low, moderate and high problem recognition for treatment. Analysis of Variance (ANOVA) models subsequently were conducted to examine whether the PRQ predicted later commitment to Harm Reduction and to Recovery. The between-subjects factor was the PRQ assessed at session 5. The dependent variables were the Harm Reduction and Recovery scales assessed at session 9. Separate models were conducted for Harm Reduction and Recovery.

The model predicting Harm Reduction from the PRQ was not significant, F(2,97) = 3.09, p = n.s., η² = .06. In contrast, the model predicting Recovery from the PRQ was significant, F(2,138) = 5.96, p<.01, η² = .08. Post-hoc Bonferroni multiple comparisons tests indicated that individuals in the moderate recognition for treatment group had significantly lower (commitment to) Recovery scores than those in the high recognition for treatment group (mean difference = −4.66, p<.01).

4. Discussion

The primary objectives of this study were to explore the clinical utility, as well as the psychometric properties, of the ASAGC. Factors that emerged reflect commitment to Harm Reduction and commitment to Recovery (that is, abstinence/sobriety). These factors adhere to the conceptual framework utilized in the field of SUD ^31–34. Furthermore, the stability of the factors over time in treatment was established (at both Sessions 3 and 9, the two-factor model provided the best statistical fit to the data).

None of the correlations between the Harm Reduction scale and the SCQ scales that reflect self-efficacy were significant. In contrast, the Recovery scale consistently was associated with the SCQ scales. These results indicate that adolescents who are highly committed to a harm reduction goal are not as confident in resisting the urge to drink in high-risk situations as are those who are committed to abstinence. Consistent with the self-efficacy findings measured by the SCQ, the Harm Reduction scale was not related to recognition that there is a problem as assessed with the PRQ. However, a significant relationship was found between the Recovery scale and the PRQ. More specifically, adolescents with a high recognition for treatment had significantly higher (commitment to) Recovery scores than those with a moderate recognition for treatment. Taken together, the analyses examining the relationships between the ASAGC scales and the SCQ and the PRQ support the validity of the ASAGC. Furthermore, the differential pattern of relations observed for the Harm Reduction and the Recovery scales provides additional support for the distinct differences between these two constructs and the potential limitation of Harm Reduction as a treatment goal for youth.

The present study developed and validated an instrument to assess commitment to substance abuse treatment (the ASAGC) in a large sample of adolescents receiving treatment. Importantly, assessment occurred over time and data were collected from both adolescents and their therapists. Nevertheless, limitations of the study should be noted. Of note, the sample included adolescents receiving substance abuse treatment in the northeastern United States. Therefore, the results may not be generalized to adolescents living outside of this area. In addition, because the sample was largely male, gender differences could not be addressed. Given the homogeneity of the sample, it would be important for future research to replicate the ASAGC factors with samples of adolescents with different characteristics (e.g., in regard to race/ethnicity, gender, geographic residence). The present study also was exploratory in nature. An important next step for future research would be to confirm the factor structure found in this study by conducting a confirmatory factor analysis (CFA). Despite these limitations, this study makes a significant and innovative contribution to the research of MBCs in adolescent SUD treatment. Importantly, the ASAGC was shown to demonstrate clinical utility and to be a reliable and valid instrument for the assessment of adolescents’ commitment to their substance abuse treatment goal. Future studies should examine the predictive value and mediating aspects of the construct of commitment to treatment goal to treatment outcomes. In addition, it would be important to examine the stability of the commitment construct through aftercare (continued care) and follow-up.

4.1. Conclusion

The ASAGC is a reliable and valid clinical research instrument for the assessment of adolescents’ commitment to their substance abuse treatment goal. Clinical researchers may take advantage of the clinical utility of the ASAGC including its ability to differentiate between commitment to abstinence versus commitment to harm reduction. Further examination from a developmentally informed approach is necessary to study the potential difference between adolescent and adult Harm Reduction oriented therapeutic approaches.

Highlights.

• The Adolescent Substance Abuse Goal Commitment (ASAGC) questionnaire was designed to investigate two clinical constructs commonly used in the adult literature, Abstinence/Sobriety and Harm Reduction

• The ASAGC questionnaire shows promising clinical utility and psychometric properties

• The findings question the clinical utility of the harm reduction construct for youth with substance use disorders as compared to adults