Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 May 4.
Published in final edited form as: J Child Adolesc Subst Abuse. 2016 May 4;25(5):417–427. doi: 10.1080/1067828X.2015.1056866

Informed Assent Recall Among Adolescents in Substance Use Disorder Treatment Research

Jordan P Davis 1, Emily A Lux 1, Douglas C Smith 1, Leah Cleeland 1
PMCID: PMC5048739  NIHMSID: NIHMS745711  PMID: 27721646

Abstract

Evidence suggests that vulnerable populations such as substance users, those involved in the criminal justice system, and those with cognitive deficits often fail to recall information regarding the informed assent process. This study investigated correlates of assent quiz errors (AQE) among adolescents enrolling in a substance use disorder treatment study.

Method

Adolescents (age 13-19) entering substance use treatment completed a standard informed consent procedure to participate in a longitudinal research study, followed by a brief 6-item assent quiz.

Results

Informed assent quiz errors were lower in this study relative to those observed in the adult literature. Being male and having lower treatment resistance was associated with making an AQE. Both days of marijuana use and days spent in the criminal justice system were associated with AQE. Hyperactivity-impulsivity approached significance (p = .057) but was not a correlate of AQE.

Conclusion

Those collecting assent should be aware that heavily-using males involved in the criminal justice system are prone to make more errors, and should use procedures such as assent quizzes to enhance their understanding of study procedures. Future research should investigate the impact of motivation and consider using experimental designs to test adolescent recall of study procedures under various assenting conditions.

Keywords: Capacity to consent, informed consent, assent, adolescent, substance use treatment, research ethics

Introduction

Although reviews on ethical issues surrounding adolescents’ participation in substance use disorder (SUD) treatment trials exist (Brody & Waldron, 2000), there is no empirical research on their capacity to assent to such participation. This lack of information affects Institutional Review Boards’ (IRB) decision-making processes in manners that may affect the generalizability of this important body of research. It is critical to establish an empirical basis for making decisions about when adolescents’ capacity to assent to participation in SUD treatment trials may be compromised (Anderson & DuBois, 2007).

Adolescents who use substances suffer from problems that may affect the assenting process (e.g. withdrawal symptoms, cognition difficulties; (Munro, Saxton, & Butters, 2000; Tapert & Brown, 2000). In particular, comorbid psychopathology (i.e. impulsivity and alcohol use; (de Wit, 2009; Potenza, 2007), cognitive impairment, and educational attainment (McCrady & Bux Jr, 1999) may inhibit comprehension of study procedures among substance users (Kiluk, Nich, & Carroll, 2010). It should be noted that 67% of adolescents entering SUD treatment reported criteria for one or more psychiatric disorders, such as conduct disorder, attention deficit hyperactivity disorder, depression, and generalized anxiety disorder (Dennis, White, & Ives, 2009). Further, studies have shown that a large proportion of substance users are involved with both treatment agencies and the criminal justice system (Weisner & Schmidt, 1995), with nearly 45% of treatment referrals coming from a criminal justice department (SAMHSA, 2014). Some studies have found low consent recall rates among study participants who are involved in the criminal justice system (Festinger et al., 2007; Rounsaville, Hunkele, Easton, Nich, & Carroll, 2008). This represents a particularly vulnerable population that may be pressured into treatment by the criminal justice system, potentially hindering their ability to comprehend and understand research procedures (Brody & Waldron, 2000; DuVal & Salmon, 2004; Moser et al., 2004). To our knowledge, no study has examined assent quiz errors (AQE) among adolescents entering SUD treatment. Thus, the current study is the first to test associations of AQE among adolescents in SUD treatment.

Informed Assent: Definitions and Barriers to Informed Decision Making

In research where adolescents are not legally allowed to consent to their study participation without parental consent, they are asked to provide informed assent (Smith, Boel-Studt, & Cleeland, 2009). Under the Common Rule (Code of Federal Regulations Title 45, Part 46 & subpart D, 1994) the same principles apply to both informed consent and informed assent. For consistency, we use the term informed assent in this study as participants are mostly under age 18. We note, however, that some literature below refers to collection of informed consent among adult populations.

Assent is considered fully informed only to the extent to which participants understand the risks, benefits, and procedures associated with a study (O'Neill, 2003). That is, informed assent should consist of participants having full information about the study, understanding the voluntary nature of research, and possessing the capacity to make a decision (Appelbaum & Roth, 1982; Forehand Jr & Ciccone, 2004).

Barriers to informed assent

Participants entering research studies have been found to be unaware that they were part of a research study, were unable to recall risks related to the study, lacked an understanding of randomization procedures, and were unaware that they could withdraw from the study at any time (Cassileth, Zupkis, Sutton-Smith, & March, 1980; Edwards, Lilford, Thornton, & Hewison, 1998; Festinger et al., 2007; Festinger et al., 2009; Verheggen & van Wijmen, 1996). This may be partially due to participants receiving large amounts of information at once such as written forms, verbal explanations and study brochures (McCrady & Bux Jr, 1999; Roberts & Indermaur, 2003; Rounsaville et al., 2008). Other studies have found that participants’ understanding of study procedures is positively correlated to education, vocabulary level, and intelligence testing (Taub, Kline, & Baker, 1981; Taub & Baker, 1983; Taub, Baker, & Sturr, 1986; Taub, Baker, Kline, & Sturr, 1987; Young, Hooker, & Freeberg, 1990). As mentioned previously, individuals with substance use problems often present with myriad forms of psychopathology that can inhibit their ability to provide informed assent (Festinger et al., 2007; McCrady & Bux Jr, 1999). Recent neuroimaging research has found that the effects of alcohol and drug use may be associated with cognitive impairments, changes, and/or deficits (Bolla et al., 2003; Goldstein & Volkow, 2002), as well as other psychological impairments (e.g. information processing, memory; (Ersche, Clark, London, Robbins, & Sahakian, 2006; Ersche & Sahakian, 2007; Fals-Stewart, O'Farrell, Freitas, McFarlin, & Rutigliano, 2000). In fact, nearly 75% of chronic substance users have some form of cognitive impairment (Bates & Convit, 1999; Rourke & Grant, 2009), raising questions about substance using participants’ abilities to adequately recall information about the study.

Assent Recall Research

To address impairment problems or deficits, researchers sometimes use assent quizzes where they ask potential participants to respond to questions that gauge their understanding of study procedures. These quizzes are typically administered shortly after reviewing assent documents. One review found most adult studies reported assent quiz recall scores below 60%, with only a small percentage (21%) of studies reporting recall scores over 75% (Flory & Emanuel, 2004). This is concerning as many of these studies measured recall scores immediately after the informed assent documents are reviewed with potential participants. These low scores may reflect low comprehension of research procedures, rights, risks, and one's ability to withdrawal from a study, hindering participants’ capability to make truly informed decisions about study participation.

Rounsaville et al. (2008) found that only 55% of young adult marijuana users were able to correctly answer all assent quiz questions. Further, 20% incorrectly answered a question regarding their right to refuse participation. The authors attributed this to participants’ involvement in the criminal justice system, and recommended additional research on how criminal justice system involvement influences treatment assenting procedures. Further, marijuana use was significantly correlated with quiz scores and participants’ understanding of their right to withdrawal from the study. Our study extends findings by examining whether criminal justice involvement is associated with AQE among adolescents entering SUD treatment.

Festinger and colleagues (2007) investigated assent recall among young adult drug court clients entering a research study. At the 2 week follow up participants were unable to recall nearly 70% of the study-related information. For example, participants were unable to recall the purpose of the study, risks or benefits, participant rights, and information about who they should contact regarding questions or harm during the study. One of the key predictors of assent recall was drug use severity, with those using more frequently making more AQE. One limitation of this study, however, was that it included a large proportion of participants (65%) not meeting criteria for substance dependence. Our study extends findings to a sample of adolescents where a higher proportion of participants have past year dependence (43%) and abuse (36%) diagnoses.

Festinger and colleagues’ (2007) study was also limited by not including measures of motivation, which they alleged would be correlated with AQE. That is, it is possible that those with low treatment motivation may enter longitudinal research because they feel coerced to participate in treatment, and may solely be seeking research incentives from study participation. If that is the case, they may be less invested during research appointments, and more prone to making informed assent quiz errors. For example, Rounsaville and colleagues (2008) mentioned, but did not report, correlations between assent quiz scores and treatment motivation among adult marijuana users. This study was limited by only completing bivariate tests of association between motivation and treatment resistance with assent quiz scores. To our knowledge, our study is the first to report the multivariate testing on the associations between treatment motivation and assent quiz errors. We hypothesize a negative association between motivation and AQE. Conversely, we expect a positive association between treatment resistance and AQE, indicating that more treatment resistant youth will make more assent quiz errors.

Kiluk et al., (2010) investigated neurocognitive predictors of AQE among adults seeking outpatient substance use treatment. Results showed that 85% of participants made at least one AQE, with AQE significantly associated with intelligence, attention, and frequency of substance use. Interestingly, authors did not find a significant association between impulsivity and AQE. This was counterintuitive as impulsivity impacts cognitive processes (Potenza, 2007) and is a risk factor for addiction, especially for adolescents (Bechara, 2003; Lejuez, Aklin, Zvolensky, & Pedulla, 2003). Whereas Kiluk et al., (2010) only tested bivariate correlations, our study rules out third variable explanations by using a multivariate approach. This represents a substantial methodological improvement and contribution to the research on assent quiz errors.

Summary and Hypotheses

Ethical research practices are especially crucial as adolescents participating in SUD treatment trials are assumed to possess diminished capacity to assent. Informed assent is a key part of federal regulations governing research participation requirement of federal regulations (Festinger et al., 2007). It is vital that we have a better understanding of adolescent characteristics associated with poorer comprehension of study procedures, as no empirical research exists. Further, it is critical to extend assent quiz research to adolescent populations, who are assumed to have even lower capacity to assent versus adults with substance use disorders. Research has shown that neurocognitive factors impact assent recall, especially for those who use substances (Festinger et al., 2007; McCrady & Bux Jr, 1999; Munro et al., 2000; Tapert & Brown, 2000), experience comorbid psychological disorders (McCrady & Bux Jr, 1999), or are involved in the criminal justice system (Festinger et al., 2007; Rounsaville et al., 2008).

All of the studies investigating AQE have been with adults or young adult subjects, and we cannot assume that capacity to assent will be similar across development. For example, adolescents state higher interpersonal reasons for change (Smith, Cleeland, & Dennis, 2010) and experience better treatment outcomes compared to when compared to young adults (Smith, Godley, Godley, & Dennis, 2011). Based on prior literature, we hypothesized that days of substance use, criminal justice system involvement, cognitive impairment, treatment resistance, and hyperactivity-impulsivity will be positively correlated with AQE. Treatment motivation was hypothesized to be negatively associated with AQE.

Methods

Participants and Procedures

Adolescents (age 13-19) were referred to SUD treatment in Chicago, IL as part of a 12-month, SAMHSA-funded treatment evaluation project. In total, 94 adolescents were assessed for eligibility for treatment and assented to participate in the study. Adolescents were eligible for this study if they 1) were referred to outpatient treatment; 2) spoke English fluently; 3) were not actively psychotic; 4) provided assent and had their parent or legal guardian consent to their participation; 5) were not incarcerated; and 6) did not have plans of moving or knowledge of impending incarceration.

On average, participants were 15.9 (SD = 1.26) years old, with females comprising 21% of the sample (see Table 1). The majority of participants were African-American (38%), Hispanic (23%), or Biracial/Multiracial (21%). Regarding substance use, 40% of participants had past year substance dependence, and participants used marijuana an average of 32 (SD = 30) out of the past 90 days and spent 3.4 (SD = 14.52) days in substance use treatment prior to intake. Further, during the 90 days prior to treatment intake, participants spent an average of 32 (SD = 42.2) days on probation and reported 40 (SD = 47.7) days of involvement with the criminal justice system.

Table 1.

Baseline Characteristics of Adolescents by Error Status

Error (n =30) M (SD) or n (%) No Error (n =64) M (SD) or n (%) Total (n = 94)
Age (in years) 15.9 (1.39) 16.0 (1.20) 15.9 (1.26)
% Female* 3 (3.19) 18 (19.2) 21(22.3)
Racial Background n(%)
    African-American 10 (10.6) 28 (29.8) 38 (40.4)
    Hispanic 7 (7.45) 16 (17.02) 23 (24.5)
    Biracial/multiracial 7 (7.45) 14 (14.9) 21 (22.3)
    White 4 (4.26) 6 (6.38) 10 (10.6)
    Other 2 (2.13) 0 (0.0) 2 (2.13)
Substance Use/Mental Health
    Past Year DSM IV Dependence n(%) 13 (13.8) 27 (28.7) 40 (42.6)
    Past Year DSM IV Abuse n(%) 12 (12.9) 21 (22.6) 33 (35.5)
    Days of marijuana use (past 90)* 41.0 (31.9) 27.8 (28.4) 32.0 (30.0)
    Days of alcohol use (past 90) 4.37 (4.78) 5.36 (11.4) 5.04 (9.75)
    Years of Substance Use 3.36 (2.59) 3.34 (2.73) 3.35 (2.34)
    Age of First Use less than 15a n(%) 25 (26.6) 49 (52.1) 74 (78.72)
    Substance use treatment indexb 3.29 (11.1) 3.47 (15.8) 3.44 (14.5)
    ADHD scale 8.23 (5.37) 7.28(5.62) 7.59 (5.53)
    Mean conduct disorder symptoms 3.8 (2.34) 3.31(2.82) 3.47 (2.67)
    General anxiety disorderc n(%) 4 (4.26) 3 (3.19) 7 (7.45)
    Hyperactivity-Impulsivity Scale 3.17 (2.83) 2.53 (2.71) 2.73 (2.75)
    Traumatic stress scaled ** 3.7 (2.13) 2.27(1.39) 2.72 (3.77)
Cognitive Impairment
    Cognitive Impairment Screene 3.46 (2.90) 3.12 (2.95) 3.23 (2.93)
Criminal Justice Involvement
    CJSIf* 55.7 (48.8) 33.26 (44.9) 40.4 (47.1)
    Days on probation* 47.7 (30.6) 24.9 (14.9) 32.3 (42.2)
    Days in jail 7.83 (15.4) 3.02 (12.7) 4.62 (13.8)
Readiness to Change Scales
    Treatment Motivation Index 2.08 (1.28) 1.64 (1.23) 1.78 (1.25)
    Treatment Resistance Index** .882 (.596) 1.24 (1.00) 1.13 (.916)
    Self-Efficacy Scale 4.30 (.952) 4.34 (1.29) 4.32 (1.19)
Open in a new tab
a

Percent of participants that used substances before the age of 15;

b

Number of days in substance use treatment out of the past 90;

c

Percent diagnosed with general anxiety disorder;

d

count of memories or symptoms of trauma in past year (e.g. PTSD)

e

Measures person's ability to place themselves in space and time;

f

Count of days involved in criminal justice system out of past 90.

ADHD= Attention Deficit Hyperactivity Disorder; CJSI= Criminal Justice System Index

*

p<.05

**

p < .10

Participants were assessed at baseline using the Global Appraisal of Individual Needs (Dennis, Titus, White, Unsicker, & Hodgkins, 2003) and had scheduled follow up sessions at 3, 6, and 12 months. Adolescents initially received the Adolescent Community Reinforcement Approach (A-CRA), a 12-14 session cost-efficient, empirically-supported treatment (Godley et al., 2001) for adolescents suffering from SUD. For those that needed on-going support to stay drug free after the initial 12 sessions, participants were offered A-CRA's companion aftercare model, called Assertive Continuing Care. This study was approved and monitored by the Institutional Review Board at the lead author's institution.

All study participants underwent a standard informed assent procedure where the staff explained the assent procedure to the participant (i.e. why assent is necessary, need for parent/guardian signature), and read and discussed each section of the informed assent document with participants. Afterwards, the staff answered any participant questions. Participants were then asked to complete an assent quiz consisting of 6 true/false items (See measures section below). The research staff member then reviewed any aspects of the study that the adolescent appeared to misunderstand and completed study enrollment procedures.

Measures

Assent quiz error (AQE)

A study-specific, six-item (true/false) assent quiz (α = 0.54) was developed based on study procedures. Responses were coded as true (true=1) or false (false=0), with five of the six items being reversed scored. Scale scores could vary from 0-6, with higher scores indicating a better understanding of the study risks, procedures, and protections against risks. Based on the scale distribution, we created a dichotomized AQE measure that indicated the presence of any errors (AQE = 1) vs. no errors (AQE = 0).

Treatment motivation index (TMI)

The TMI is a count of items referring to the participant's perception, pressure, and support to stay in or enter treatment and his/her own need for treatment. Higher scores on this scale suggest more motivation to enter or remain in treatment. Example questions included, “There is a lot of pressure for you to be in alcohol or drug treatment,” and “You need support from friends and relatives to deal with your alcohol or drug use.” This scale was adapted from reliable treatment motivation indices that predicted substance use outcomes (Knight, Holcom, & Simpson, 1994; Simpson & Joe, 1993).

Treatment resistance index (TRI)

The treatment resistance index is a count of items endorsed regarding difficulties entering or staying in treatment. Example questions included, “Your old friends may try to get you to drink or use drugs again,” and “You have too many other responsibilities now to be in a treatment program.” Higher scores indicate a larger number of factors preventing the participant from entering or staying in treatment. The instrument on which it is based is reliable (α = 0.70-.90; (Knight et al., 1994; Stephens, Curtin, Simpson, & Roffman, 1994) and predicts SUD treatment outcomes.

Criminal justice system index (CJSI)

The CJSI is the average percent of days a participant has spent in the criminal justice system through detention, monitoring, or incarceration. Values for this index range from 0 to 90 with higher scores indicating a higher proportion of days of various types of involvement in the criminal justice system in the past 90 days. The CJSI is a strong predictor of substance use outcomes (Morral, McCaffrey, & Ridgeway, 2004).

Marijuana Use

Marijuana use was measured with a single item derived from the larger Substance Frequency Scale. The substance frequency scale uses time line follow back (Sobell & Sobell, 1992) procedures and is a reliable and valid instrument (Dennis, Funk, Godley, Godley, & Waldron, 2004) for measuring substance use among adults and adolescents. Specifically, participants were asked “in the past 90 days, on how many days have you used any type of marijuana, blunts or THC?”

Hyperactivity and impulsivity scale (HIS)

The HIS is a count of past year DSM-IV hyperactivity and impulsivity symptoms. Higher scores on this scale indicate greater hyperactivity and impulsivity problems. Example items included, “During the past 12 months have you been unable to stay in a seat or where you were supposed to stay?,” or “...felt like you were always on the go or driven by a motor?” This scale was adapted from the (American Psychiatric Association, 2000) ADHD criteria and is a valid and reliable measure (α =0.70; (Dennis, Chan, & Funk, 2006)

Data Analyses

Prior to analyses we tested bivariate associations between variables that would potentially diminish capacity to assent (e.g., gender, anxiety disorders, days of marijuana use etc.) and our AQE measure. (A table of correlations between independent variables by AQE status is available in the supplemental material for this article.) If variables were significantly associated with AQE (error vs. no error) at the significance level of p < .10 (Hosmer, Lemeshow, & Sturdivant, 2000), they were included in our multivariate regression model. Interestingly, some of the variables we expected to retain in our final model based on prior literature were not significantly associated with AQE in these initial bivariate tests. These variables included the cognitive impairment scale (t = −.53, p = .600), the substance frequency scale (t = −1.04, p = .300), the substance problem scale (t = −.53, p = .600), the general mental health distress scale (t = 1.39, p = .279), the attention deficit hyperactivity disorder scale (t = −.78, p = .440), the conduct disorder scale (t = −.82, p = .413), the behavior complexity scale (t = −.87, p = .384), the inattentive-disorder scale (t = −.41, p = .682), and a dichotomous measure indicating the presence of any co-occurring disorder (χ2 = .559, p = .455). Based the above-stated criteria, the predictors left for our model were female (1 = yes, 0 = no), the treatment motivation index (TMI), the treatment resistance index (TRI), days of marijuana use at baseline, the hyperactivity-impulsivity scale (HIS), the generalized anxiety scale (GAD), the traumatic stress disorder scale (TSS), and the criminal justice system involvement scale (CJSI).

As model fitting using logistic regression is sensitive to multicollinearity among the independent variables (Hosmer Jr & Lemeshow, 2004), we reviewed a number of multicollinearity diagnostics such as the variance inflation factor (VIF), condition number (CN), and variance-decomposition proportions (VDP). Variables with VIF of > 2, CN > 5 and VDP > 0.50 were eliminated from the model (P. Allison, 1999; Belsley, Kuh, & Welsch, 2005; Belsley, 2006). Two variables were eliminated, including GAD (VDP = .632, t = 1.25, p = .216) and TSS (VD P= .807, t = 1.11, p = .270). Thus, our final model regressed AQE on female, TMI, TRI, CJSI, baseline days of marijuana use (out of past 90) and HIS. Analyses were completed using SAS software (SAS Institute Inc. 2011.Version 9.3, 2011).

To address missing data we used multiple imputation (k = 20) and the EM algorithm in SAS (SAS Institute Inc. 2011.Version 9.3, 2011) based on (Schafer & Graham, 2002) recommendations for logistic regression models. Missing data for key variables maximized at 8.5%. Given the assumption that data is missing at random, the expectation maximization algorithm gives unbiased estimates of missing data (P. D. Allison, 2002; McLachlan, Krishnan, & Ng, 2008; Schafer & Graham, 2002).

Results

AQE Frequencies

Overall, 68.1% (n = 64) of participants were able to recall all six items on the assent quiz (see Table 2), with a mean score of 5.52 (SD= .877). The most commonly missed items on the AQE dealt with intent-to-treat procedures (13.6%; i.e. “If you stop coming to treatment, you have to quit the study, too.”), reporting results only in non-identifying, aggregate form (8.05%; i.e. “If the study is described in a paper people will know who you are and what you said.”), and benefits of the study (6.98%; “If you do this study, researchers can help you get out of jail, or out of trouble with the courts.”).

Table 2.

Consent Quiz Performance

Question % correct % incorrect
1. If you do this study, researchers can help you get out of jail, or out of trouble with the courts. 93.02 6.98
2. If the study is described in a paper or talked about people will know who you are and what you said 91.95 8.05
3. If you tell the researcher you are being abused, or thinking about killing yourself, the researchers cannot say anything about this to others 94.32 5.68
4. If you stop coming to treatment, you have to quit the study, too 86.36 13.64
5. When your answers are put into the computer, everyone who uses that computer later sees your answers 95.45 4.55
6. we will tape record your sessions so we know that you are getting good treatment 95.45 4.55
Open in a new tab

*Overall 68% of individuals were able to recall all 6 questions correctly.

Multivariate Results

Table 3 shows results from the logistic regression model. Collectively, the variables in the model accounted for 34.3% of the variance in AQE. Results indicated that TRI (OR = .362, p = .005 and gender (OR = .186, p = .026) were negative predictors of AQE. Thus, higher treatment resistance and being male were associated with lower odds of AQE. Further, marijuana use at baseline (OR = 1.02, p = .033) and days involved with the criminal justice system (OR = 1.02, p = .008) were positive predictors of AQE. That is, more days of marijuana use at baseline and more days involved with the criminal justice system were associated with AQE. Finally, though not statistically significant at α = 0.05, HIS (OR = 1.21, p =.057) was positively associated with AQE. Thus, higher hyperactivity and impulsivity were associated with more assent quiz errors.

Table 3.

Logistic Regression Results

OR 95% CI
 p
LL UL
Model fit: R2 =.343, χ2= 26.4,df=6; p=.0002
Intercept .000
Femalea .186 .042 .820 .026*
Treatment Motivation 1.44 .927 2.23 .105
Treatment Resistance .362 .178 .735 .005*
Days of marijuana use 1.02 1.00 1.04 .033*
CJSI 1.02 1.00 1.03 .008*
HIS 1.21 .994 1.46 .057**
Open in a new tab
a

Female coded as female=1.

OR= Odds ratio; LL=Lower Limit; UB= Upper Limit.

CJSI= Criminal Justice System Index; HIS= Hyperactivity-Impulsivity Scale

*

p < .05

**

p < .10

Discussion

This is the first study that identifies correlates of assent quiz errors (AQE) among adolescents participating in a longitudinal SUD treatment outcomes study. Results indicated that only 32% of participants made one or more AQE, with the average adolescent answering 92% of the questions correctly. This recall rate of informed assent information is higher than that found in one review of adult assent quiz performance, where less than 60% of participants answer all assent quiz items correctly (Flory & Emanuel, 2004). Thus, notwithstanding limitations, one major contribution of this study is that it challenges the notion that at-risk adolescents invited into longitudinal substance use disorder treatment studies have highly diminished capacity to assent. Relative to adult research on capacity to consent, adolescents appear to have slightly better recall. This is important to know, as diminished capacity to consent may be used as a rationale for asking adolescent researchers for extra protections for adolescents. For example, parental consent is collected in the vast majority of adolescent substance use disorder treatment studies, even though adolescents are often legally able to consent to their own treatment (Smith et al., 2009). Thus, these findings implore IRB members to carefully examine whether they have implicit biases regarding capacity to assent for adolescents with heavy substance use and comorbid conditions (See Table 1).

Implications from Descriptive Analysis of Types of Errors

Important research implications arose from the analysis of types of errors that adolescents made on this assent recall quiz. In summary, nearly all participants understood procedures associated with duty to warn/mandated reporting of child abuse (94.3%), the concept of restricted access of data within computer networks (95.4%), and the purpose of taping treatment sessions (95.5%). The most commonly missed item pertained to the intent to treat concept, specifically that youth should still complete research follow up assessments even when they drop out of treatment. This is consistent with anecdotal reports that research participants in SUD treatment trials often confuse the roles of research and clinical staff members (Scott & White, 2005). These findings suggest the need for researchers to carefully differentiate the roles of research and clinical staff members for adolescent participants. Finally, approximately 7% of youth erroneously indicated that research participation would help them get out of legal trouble. This underscores the need to emphasize that treatment participation, not research participation, may improve legal standing. Furthermore, adolescent participants involved in criminal justice proceedings need to be told about the availability of non-study treatment alternatives.

Potential Explanations for Associations with AQE

Correlations between adolescent demographic and clinical characteristics and assent quiz errors were mostly in the expected direction and consistent with prior literature. Specifically, we found that males, those with higher criminal justice involvement, and those using marijuana more heavily were at risk for making AQE. Our findings on the associations between marijuana use, criminal justice involvement, hyperactivity, and AQE are consistent with other research involving emerging and older adults (Festinger et al., 2007; Festinger et al., 2009; Kiluk et al., 2010; Rounsaville et al., 2008). Comprehension checks on the core features of substance use disorder treatment studies are needed for adolescents who use marijuana more frequently, are involved in the criminal justice system and present with impulsivity and hyperactive symptoms. For example, research assistants could do a debriefing after administrating an assent quiz, reviewing any items that participants answered incorrectly on assent quizzes.

Deficiencies in assent recall may be magnified for those involved with the criminal justice system (Festinger, Dugosh, Croft, Arabia, & Marlowe, 2011), and our study extends these findings to adolescents. It may be that criminally-referred adolescents, like their counterparts in adult studies, are prone to consent errors due to perceived coercion to participate in studies (Dugosh, Festinger, Croft, & Marlowe, 2010; Festinger et al., 2011; Moser et al., 2004). That is, those involved with the criminal justice system may not feel entirely autonomous in their decision making when approached by research personnel, whom they may equate with authorities. These study findings may point to the need to use third party “facilitators” (Benson, Roth, Appelbaum, Lidz, & Winslade, 1988; DeRenzo, 1994; Reiser & Knudson, 1993; Stiles, Poythress, Hall, Falkenbach, & Williams, 2001) for criminally-referred adolescents invited to participate in SUD treatment studies. A third party facilitator is a ‘neutral educator’ (Benson et al., 1988) or a ‘research intermediary’ (Stiles et al., 2001) not involved in the research project who is responsible for collecting informed consent. Festinger and colleagues (2011) found that individuals assigned to a third part facilitator had a better understanding that involvement in the research would not improve their judicial standing.

Another implication of these findings is that it may be especially important to use procedures to enhance the comprehension of the assent process with heavy marijuana using males (i.e., greater than 40/90 days) involved in the criminal justice system. Incentivizing assent higher assent recall by adolescents may improve their comprehension of study information. Emerging adults assigned to an incentivized corrected feedback condition (e.g. participants received feedback for wrong answers and a monetary incentive for each correct answer) recalled more information about study procedures during follow up assessments (Festinger, Dugosh, Marlowe, & Clements, 2014).

Counterintuitively, treatment resistance was negatively associated with AQE, even when criminal justice involvement was accounted for in the model. It is perplexing why treatment resistance was negatively associated with AQE, and we offer some possible explanations here that should be evaluated in future research. First, adolescents in this study may have been part of the criminal justice system for an extended period of time and could have encountered multiple treatment episodes. Such adolescents may view it as socially desirable to say they are not resistant to receiving treatment and may be simultaneously less likely to make assent quiz errors due to general experience with treatment consenting procedures. Second, it is possible that some adolescents gave socially desirable responses during initial assessments to treatment resistance questions (i.e., faking lower resistance), and decided to enter the study for other reasons such as seeking research participation incentives. Thus, such adolescents could be considered to have low investment in the actual study, and in turn, may have been less attentive during consenting procedures. One implication from this is that assent quiz errors may be a proxy variable for low commitment to the actual research study, and future studies may consider using assent quiz scores as predictors of follow up retention in longitudinal treatment research. Finally, it is possible that this is simply a spurious correlation. Although there have been prior recommendations to look at the association between motivation with assent recall (Festinger et al., 2007; Festinger et al., 2009; Rounsaville et al., 2008), a strong theoretical model for the mechanism through which resistance affects recall has yet to be specified. Our study is among the first to look at the association between treatment resistance and assent quiz errors and should be replicated in future research.

It is unclear why some variables were not significantly associated with AQE. For example, we found a non-significant positive association between hyperactivity-impulsivity and AQE (p = .057). This is consistent with other studies regarding hyperactivity-impulsivity and assent quiz recall (Festinger et al., 2007; Kiluk et al., 2010). It may be that that hyperactivity does not affect adolescents’ ability to recall study procedures, or just that performance on this particular assent quiz (e.g. true, false) was not affected by hyperactivity-impulsivity. Surprisingly, we also did not find a significant association between the cognitive impairment screening and AQE status. We are unsure why this occurred, as several studies have shown correlations between cognitive impairment and consent quiz scores. It may be that our participants had much lower cognitive impairment, as the adult studies reviewed included severely impaired participants (i.e., thought disordered participants). Additionally, adolescents who present with SUDs may have fluctuating cognitive capacities due to intoxication or attention problems (Pagliaro & Pagliaro, 1996; Tarter, Mezzich, Hsieh, & Parks, 1995). Thus, their cognitive impairments may be transitory, and occur mostly when they are actively intoxicated versus when they are completing an initial assessment. Furthermore, several studies have shown that AQE is associated with intelligence scores (Kiluk et al., 2010). Perhaps, if other proxies for impairment such as intelligence were included in our cognitive impairment screen we may have found a significant association with AQE status. Also, it could be that the cognitive impairment screen scores were sometimes influenced by the research assessment staff member's speed of administration. That is, the cognitive impairment screen is administered very early in the GAIN interview, which last roughly 90 minutes (Titus et al., 2012). If some interviewers rapidly administered the items when the adolescent was getting oriented to the structure of the interview, it could have influenced scores and the findings presented here. Finally, it may be that the restricted variability in cognitive impairment scores aided in our null finding.

Limitations

Although our study makes a significant contribution by extending findings on assent quiz errors to adolescents, the findings should be interpreted in light of study limitations. The primary limitation of this study was the cross-sectional design, which limits our ability to make causal inferences. Future research could consider using experimental designs to test adolescents’ recall under various assenting conditions. Second, our measure included study-specific true-false questions, and we can’t be certain of the assessment's difficulty level. This hinders our ability to know for sure if adolescents’ better performance on this assent quiz relative to that observed in the adult literature (Flory & Emanuel, 2004) is due to this potential measurement artifact. However, we do note that despite the low error rate, we found evidence for the measure's construct validity. That is, assent quiz errors were positively associated with marijuana use and hyperactivity/impulsivity symptoms. Thus, it is plausible that adolescents do, in fact, have better recall of study features compared to adults. Second, it is possible that other unmeasured participant characteristics (i.e., intelligence) may have influenced associations between study variables and AQE. Third, we did not have any observational measures on how the assenting was handled by the research staff. Research staff member behavior may have had a strong influence on which youth made assent quiz errors. In research on data quality (n =15,858), interviewer training and experience were stronger predictors of data quality than participants’ psychiatric and substance use severity (Titus et al., 2012). This may underscore the need for staff training to ensure that particular staff members are not associated with more AQE. Future research may use observational procedures to determine how research staff member competency influences adolescent AQE. Fourth, it may be that we had limited power to detect potentially meaningful associations between some variables and AQE. Finally, findings may not may not generalize to all adolescent programs given the setting and racial diversity among participants. Nevertheless, as there are no studies on the association between AQE and characteristics of adolescent substance users, this study provides important preliminary data that may be replicated in future studies that address these limitations.

Conclusion

It is essential that adolescents understand and assent voluntarily prior to enrolling in a longitudinal SUD treatment study. Our study showed that, relative to adults in SUD treatment trials, adolescents appear to make fewer assent quiz errors. This beckons IRB's to consider that at-risk adolescent samples should not be automatically assumed to have diminished capacity to assent. Nevertheless, those collecting assent should be aware that heavy marijuana-using adolescents, male adolescents, those presenting with hyperactivity/impulsivity symptoms, and those that are involved in the criminal justice system are prone to making more errors. With these adolescents, special strategies may be warranted to enhance their understanding of study procedures. Additional research on capacity to assent is needed with adolescents suffering from substance use problems.

Acknowledgments

The development of this article was supported by NIAAA grant # K23AA017702 (Smith-PI). The views, however, are those of the authors and do not reflect official positions of the US government.

References

  1. Allison P. Logistic regression using SAS®: Theory and application. SAS Publishing; 1999. [Google Scholar]
  2. Allison PD. Missing data: Quantitative applications in the social sciences. British Journal of Mathematical and Statistical Psychology. 2002;55(1):193–196. [Google Scholar]
  3. American Psychiatric Association . Diagnostic and statistical manual of mental disorders: DSM-IV-TR®. American Psychiatric Pub; Washington D.C.: 2000. [Google Scholar]
  4. Anderson EE, DuBois JM. The need for evidence-based research ethics: A review of the substance abuse literature. Drug and Alcohol Dependence. 2007;86(2):95–105. doi: 10.1016/j.drugalcdep.2006.06.011. [DOI] [PubMed] [Google Scholar]
  5. Appelbaum PS, Roth LH. Competency to consent to research: A psychiatric overview. Archives of General Psychiatry. 1982;39(8):951–958. doi: 10.1001/archpsyc.1982.04290080061009. [DOI] [PubMed] [Google Scholar]
  6. Bates ME, Convit A. Assessment of neuropsychological functions in psychiatric disorders. In: Avraham C, editor. Neuropsychology and neuroimaging of alcohol and illicit drug abuse. American Psychiatric Association; Arlington, VA: 1999. pp. 373–445. [Google Scholar]
  7. Bechara A. Risky business: Emotion, decision-making, and addiction. Journal of Gambling Studies. 2003;19(1):23–51. doi: 10.1023/a:1021223113233. [DOI] [PubMed] [Google Scholar]
  8. Belsley DA. Conditioning diagnostics. encyclopedia of statistical sciences. 2nd ed. Wiley Online Library; 2006. [Google Scholar]
  9. Belsley DA, Kuh E, Welsch RE. Regression diagnostics: Identifying influential data and sources of collinearity. John Wiley & Sons.; 2005. [Google Scholar]
  10. Benson PR, Roth LH, Appelbaum PS, Lidz CW, Winslade WJ. Information disclosure, subject understanding, and informed consent in psychiatric research. Law and Human Behavior. 1988;12(4):455–475. doi: 10.1007/BF01044628. [DOI] [PubMed] [Google Scholar]
  11. Bolla K, Eldreth D, London E, Kiehl K, Mouratidis M, Contoreggi C, Kimes A. Orbitofrontal cortex dysfunction in abstinent cocaine abusers performing a decision-making task. NeuroImage. 2003;19(3):1085–1094. doi: 10.1016/s1053-8119(03)00113-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Brody JL, Waldron HB. Ethical issues in research on the treatment of adolescent substance abuse disorders. Addictive Behaviors. 2000;25(2):217–228. doi: 10.1016/s0306-4603(99)00041-6. [DOI] [PubMed] [Google Scholar]
  13. Cassileth BR, Zupkis RV, Sutton-Smith K, March V. Informed consent -- why are its goals imperfectly realized? The New England Journal of Medicine. 1980;302(16):896–900. doi: 10.1056/NEJM198004173021605. doi:10.1056/NEJM198004173021605 [doi] [DOI] [PubMed] [Google Scholar]
  14. Code of federal regulations title 45, part 46;subpart D. protection of human subjects. (OPRR reports) U.S. government printing office; washington, DC: 1994. [Google Scholar]
  15. de Wit H. Impulsivity as a determinant and consequence of drug use: A review of underlying processes. Addiction Biology. 2009;14(1):22–31. doi: 10.1111/j.1369-1600.2008.00129.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Dennis ML, Chan Y, Funk RR. Development and validation of the GAIN short screener (GSS) for internalizing, externalizing and substance use disorders and crime/violence problems among adolescents and adults. The American Journal on Addictions. 2006;15(s1):s80–s91. doi: 10.1080/10550490601006055. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Dennis ML, Titus JC, White MK, Unsicker JI, Hodgkins D. Global appraisal of individual needs: Administration guide for the GAIN and related measures. Chestnut Health Systems; Bloomington, IL: 2003. [Google Scholar]
  18. Dennis ML, White MK, Ives ML. Adolescent substance abuse. Springer; 2009. Individual characteristics and needs associated with substance misuse of adolescents and young adults in addiction treatment. pp. 45–72. [Google Scholar]
  19. Dennis ML, Funk R, Godley SH, Godley MD, Waldron H. Cross-validation of the alcohol and cannabis use measures in the global appraisal of individual needs (GAIN) and timeline followback (TLFB; form 90) among adolescents in substance abuse treatment. Addiction. 2004;99:120–128. doi: 10.1111/j.1360-0443.2004.00859.x. doi:10.1111/j.1360-0443.2004.00859.x. [DOI] [PubMed] [Google Scholar]
  20. DeRenzo EG. The ethics of involving psychiatrically impaired persons in research. IRB: Ethics and Human Research. 1994;16(6):7–11. [PubMed] [Google Scholar]
  21. Dugosh KL, Festinger DS, Croft JR, Marlowe DB. Measuring coercion to participate in research within a doubly vulnerable population: Initial development of the coercion assessment scale. Journal of Empirical Research on Human Research Ethics. 2010;5(1):93–102. doi: 10.1525/jer.2010.5.1.93. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. DuVal G, Salmon C. Research note: Ethics of drug treatment research with court-supervised subjects. Journal of Drug Issues. 2004;34(4):991–1006. doi: 10.1177/002204260403400414. [DOI] [PubMed] [Google Scholar]
  23. Edwards SJ, Lilford RJ, Thornton J, Hewison J. Informed consent for clinical trials: In search of the “best” method. Social Science & Medicine. 1998;47(11):1825–1840. doi: 10.1016/s0277-9536(98)00235-4. [DOI] [PubMed] [Google Scholar]
  24. Ersche KD, Clark L, London M, Robbins TW, Sahakian BJ. Profile of executive and memory function associated with amphetamine and opiate dependence. Neuropsychopharmacology. 2006;31(5):1036–1047. doi: 10.1038/sj.npp.1300889. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Ersche KD, Sahakian BJ. The neuropsychology of amphetamine and opiate dependence: Implications for treatment. Neuropsychology Review. 2007;17(3):317–336. doi: 10.1007/s11065-007-9033-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Fals-Stewart W, O'Farrell TJ, Freitas TT, McFarlin SK, Rutigliano P. The timeline followback reports of psychoactive substance use by drug-abusing patients: Psychometric properties. Journal of Consulting and Clinical Psychology. 2000;68(1):134. doi: 10.1037//0022-006x.68.1.134. [DOI] [PubMed] [Google Scholar]
  27. Festinger DS, Dugosh KL, Croft JR, Arabia PL, Marlowe DB. Do research intermediaries reduce perceived coercion to enter research trials among criminally involved substance abusers? Ethics & Behavior. 2011;21(3):252–259. doi: 10.1080/10508422.2011.570167. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Festinger DS, Marlowe DB, Croft JR, Dugosh KL, Arabia PL, Benasutti KM. Monetary incentives improve recall of research consent information: It pays to remember. Experimental and Clinical Psychopharmacology. 2009;17(2):99. doi: 10.1037/a0015421. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Festinger DS, Ratanadilok K, Marlowe DB, Dugosh KL, Patapis NS, DeMatteo DS. Neuropsychological functioning and recall of research consent information among drug court clients. Ethics & Behavior. 2007;17(2):163–186. [Google Scholar]
  30. Festinger DS, Dugosh KL, Marlowe DB, Clements NT. Achieving new levels of recall in consent to research by combining remedial and motivational techniques. Journal of Medical Ethics. 2014;40(4):264–268. doi: 10.1136/medethics-2012-101124. doi:10.1136/medethics-2012-101124 [doi] [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Flory J, Emanuel E. Interventions to improve research participants' understanding in informed consent for research: A systematic review. Jama. 2004;292(13):1593–1601. doi: 10.1001/jama.292.13.1593. [DOI] [PubMed] [Google Scholar]
  32. Forehand L, Jr, Ciccone JR. The competence of adolescents to consent to treatment. Adolescent Psychiatry: Developmental and Clinical Studies. 2004;28:5–27. [Google Scholar]
  33. Godley SH, Meyers RJ, Smith JE, Karvinen T, Titus JC, Godley MD, Kelberg P. The adolescent community reinforcement approach for adolescent cannabis users, cannabis youth treatment (CYT) series. 2001;4 [Google Scholar]
  34. Goldstein RZ, Volkow ND. Drug addiction and its underlying neurobiological basis: Neuroimaging evidence for the involvement of the frontal cortex. American Journal of Psychiatry. 2002;159(10):1642–1652. doi: 10.1176/appi.ajp.159.10.1642. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Hosmer DW, Jr, Lemeshow S. Applied logistic regression. John Wiley & Sons; 2004. [Google Scholar]
  36. Hosmer DW, Lemeshow S, Sturdivant RX. Assessing the fit of the model. Applied Logistic Regression. (Third Edition) 2000:153–225. [Google Scholar]
  37. Kiluk BD, Nich C, Carroll KM. Neurocognitive indicators predict results of an informed-consent quiz among substance-dependent treatment seekers entering a randomized clinical trial. Journal of Studies on Alcohol and Drugs. 2010;71(5):704–712. doi: 10.15288/jsad.2010.71.704. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Knight K, Holcom M, Simpson DD. TCU psychosocial functioning and motivation scales: Manual on psychometric properties. Texas Christian University, Institute of Behavioral Research; Fort Worth: 1994. [Google Scholar]
  39. Lejuez C, Aklin WM, Zvolensky MJ, Pedulla CM. Evaluation of the balloon analogue risk task (BART) as a predictor of adolescent real-world risk-taking behaviours. Journal of Adolescence. 2003;26(4):475–479. doi: 10.1016/s0140-1971(03)00036-8. [DOI] [PubMed] [Google Scholar]
  40. McCrady BS, Bux DA., Jr Ethical issues in informed consent with substance abusers. Journal of Consulting and Clinical Psychology. 1999;67(2):186. doi: 10.1037//0022-006x.67.2.186. [DOI] [PubMed] [Google Scholar]
  41. McLachlan GJ, Krishnan T, Ng SK. The EM algorithm and extensions. John Wiley & Sons; Hoboken, New Jersey: 2008. [Google Scholar]
  42. Morral AR, McCaffrey DF, Ridgeway G. Effectiveness of community-based treatment for substance-abusing adolescents: 12-month outcomes of youths entering phoenix academy or alternative probation dispositions. Psychology of Addictive Behaviors. 2004;18(3):257. doi: 10.1037/0893-164X.18.3.257. [DOI] [PubMed] [Google Scholar]
  43. Moser DJ, Arndt S, Kanz JE, Benjamin ML, Bayless JD, Reese RL, Flaum MA. Coercion and informed consent in research involving prisoners. Comprehensive Psychiatry. 2004;45(1):1–9. doi: 10.1016/j.comppsych.2003.09.009. [DOI] [PubMed] [Google Scholar]
  44. Munro CA, Saxton J, Butters MA. The neuropsychological consequences of abstinence among older alcoholics: A Cross-Sectional study. Alcoholism: Clinical and Experimental Research. 2000;24(10):1510–1516. [PubMed] [Google Scholar]
  45. O'Neill O. Some limits of informed consent. Journal of Medical Ethics. 2003;29(1):4. doi: 10.1136/jme.29.1.4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Pagliaro AM, Pagliaro LA. Substance use among children and adolescents: Its nature, extent, and effects from conception to adulthood. John Wiley; 1996. [Google Scholar]
  47. Potenza M. To do or not to do? the complexities of addiction, motivation, self-control, and impulsivity. American Journal of Psychiatry. 2007;164(1):4–6. doi: 10.1176/ajp.2007.164.1.4. [DOI] [PubMed] [Google Scholar]
  48. Reiser SJ, Knudson P. Protecting research subjects after consent: The case for the“research intermediary.”. IRB: A Review of Human Subjects Research. 1993;15(2):10–11. [PubMed] [Google Scholar]
  49. Roberts L, Indermaur D. Signed consent forms in criminological research: Protection for researchers and ethics committees but a threat to research participants? Psychiatry, Psychology and Law. 2003;10(2):289–299. [Google Scholar]
  50. Rounsaville DB, Hunkele K, Easton CJ, Nich C, Carroll KM. Making consent more informed: Preliminary results from a multiple-choice test among probation-referred marijuana users entering a randomized clinical trial. The Journal of the American Academy of Psychiatry and the Law. 2008;36(3):354–359. doi:36/3/354 [pii] [PMC free article] [PubMed] [Google Scholar]
  51. Rourke SB, Grant I. Neuropsychological assessment of neuropsychiatric and neuromedical disorders. 3rd ed. Oxford University Press; New York: 2009. The neurobehavioral correlates of alcoholism. pp. 398–454. [Google Scholar]
  52. SAMHSA. Center for behavioral health statistics and quality.treatment episode data set (TEDS): 2002-2012. national admissions to substance abuse treatment services. Substance abuse and mental health services administration; rockville, MD: 2014. BHSISSeries S-71, HHS publication no. (SMA) 14-4850. [Google Scholar]
  53. SAS Institute Inc. 2011.Version 9.3. Base SAS® 9.3 procedures guide. SAS Institute Inc.; Cary, NC: 2011. [Google Scholar]
  54. Schafer JL, Graham JW. Missing data: Our view of the state of the art. Psychological Methods. 2002;7(2):147. [PubMed] [Google Scholar]
  55. Scott CK, White WL. Ethical issues in the conduct of longitudinal studies of addiction treatment. Journal of Substance Abuse Treatment. 2005;28(2):S91–S101. doi: 10.1016/j.jsat.2004.10.012. [DOI] [PubMed] [Google Scholar]
  56. Simpson DD, Joe GW. Motivation as a predictor of early dropout from drug abuse treatment. Psychotherapy: Theory, Research, Practice, Training. 1993;30(2):357. [Google Scholar]
  57. Smith DC, Boel-Studt S, Cleeland L. Parental consent in adolescent substance abuse treatment outcome studies. Journal of Substance Abuse Treatment. 2009;37(3):298–306. doi: 10.1016/j.jsat.2009.03.007. [DOI] [PubMed] [Google Scholar]
  58. Smith DC, Cleeland L, Dennis ML. Reasons for quitting among emerging adults and adolescents in substance-use-disorder treatment. Journal of Studies on Alcohol and Drugs. 2010;71(3):400–409. doi: 10.15288/jsad.2010.71.400. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Smith DC, Godley SH, Godley MD, Dennis ML. Adolescent community reinforcement approach outcomes differ among emerging adults and adolescents. Journal of Substance Abuse Treatment. 2011;41(4):422–430. doi: 10.1016/j.jsat.2011.06.003. doi: http://dx.doi.org/10.1016/j.jsat.2011.06.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Sobell LC, Sobell MB. Measuring alcohol consumption. Springer; 1992. Timeline follow-back. pp. 41–72. [Google Scholar]
  61. Stephens RS, Curtin L, Simpson EE, Roffman RA. Testing the abstinence violation effect construct with marijuana cessation. Addictive Behaviors. 1994;19(1):23–32. doi: 10.1016/0306-4603(94)90048-5. [DOI] [PubMed] [Google Scholar]
  62. Stiles PG, Poythress NG, Hall A, Falkenbach D, Williams R. Improving understanding of research consent disclosures among persons with mental illness. Psychiatric Services. 2001;52(6):780–785. doi: 10.1176/appi.ps.52.6.780. [DOI] [PubMed] [Google Scholar]
  63. Tapert SF, Brown SA. Substance dependence, family history of alcohol dependence and neuropsychological functioning in adolescence. Addiction. 2000;95(7):1043–1053. doi: 10.1046/j.1360-0443.2000.95710436.x. [DOI] [PubMed] [Google Scholar]
  64. Tarter RE, Mezzich AC, Hsieh Y, Parks SM. Cognitive capacity in female adolescent substance abusers. Drug and Alcohol Dependence. 1995;39(1):15–21. doi: 10.1016/0376-8716(95)01129-m. [DOI] [PubMed] [Google Scholar]
  65. Taub HA, Baker MT. The effect of repeated testing upon comprehension of informed consent materials by elderly volunteers. Experimental Aging Research. 1983;9(3):135–138. doi: 10.1080/03610738308258441. [DOI] [PubMed] [Google Scholar]
  66. Taub HA, Baker MT, Kline GE, Sturr JF. Comprehension of informed consent information by young-old through old-old volunteers. Experimental Aging Research. 1987;13(4):173–178. doi: 10.1080/03610738708259321. [DOI] [PubMed] [Google Scholar]
  67. Taub HA, Baker MT, Sturr JF. Informed consent for research: Effects of readability, patient age, and education. Journal of the American Geriatrics Society. 1986 doi: 10.1111/j.1532-5415.1986.tb05766.x. [DOI] [PubMed] [Google Scholar]
  68. Taub HA, Kline GE, Baker MT. The elderly and informed consent: Effects of vocabulary level and corrected feedback. Experimental Aging Research. 1981;7(2):137–146. doi: 10.1080/03610738108259796. [DOI] [PubMed] [Google Scholar]
  69. Titus JC, Smith DC, Dennis ML, Ives M, Twanow L, White MK. Impact of a training and certification program on the quality of interviewer-collected self-report assessment data. Journal of Substance Abuse Treatment. 2012;42(2):201–212. doi: 10.1016/j.jsat.2011.10.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Verheggen FW, van Wijmen FC. Informed consent in clinical trials. Health Policy. 1996;36(2):131–153. doi: 10.1016/0168-8510(95)00805-5. [DOI] [PubMed] [Google Scholar]
  71. Weisner C, Schmidt LA. Expanding the frame of health services research in the drug abuse field. Health Services Research. 1995;30(5):707–726. [PMC free article] [PubMed] [Google Scholar]
  72. Young DR, Hooker DT, Freeberg FE. Informed consent documents: Increasing comprehension by reducing reading level. IRB: Ethics and Human Research. 1990;12(3):1–5. [PubMed] [Google Scholar]

RESOURCES