Abstract
Objective:
To examine whether reduced substance-free enjoyable activity (i.e., ‘alternative reinforcers’) is a mediating mechanism linking lower socioeconomic status and adolescent substance use risk.
Method:
High school students in Los Angeles, CA (N = 2,553, 2013—2014, M age baseline = 14.1) were administered three semiannual surveys. Socioeconomic status was measured by highest parental education reported at Wave 1 (the beginning of 9th grade). Three elements of alternative reinforcement at Wave 2 (six-month follow-up) were assessed as mediators: ratings of frequency of engagement, level of enjoyment, and frequency × enjoyment product scores of substance-free typically pleasant activities (like participation in sports teams or school clubs). Study outcomes included prior six-month alcohol, marijuana, tobacco, and other substance use at Wave 3 (twelve-month follow-up). Logistic regression models adjusting for alternative reinforcers and substance use from the preceding wave as well as other cofactors were used to examine the association of Wave 1 parental education with Wave 3 substance use and mediation by Wave 2 alternative reinforcement.
Results:
Lower parental education at Wave 1 was associated with a greater likelihood of reporting alcohol ((β = −0.122, 95% CI = −0.234, −0.009) and marijuana (β = −0.168, 95% CI = −0.302, −0.034) use at Wave 3. The inverse association between parental education and substance use was statistically mediated by each element of diminished alternative reinforcement at Wave 2. Lower parental education at Wave 1 was associated with lower alternative reinforcement at Wave 2, which in turn was associated with greater likelihood of alcohol (range of βindirect effects : −0.007 [95% CI = −0.016, −0.001] to −0.01 [95% CI = −0.018, −0.004]) and marijuana (βs: −0.011 [95% CI = −0.022,−0.002] to −0.018 [95% CI = −0.035, −0.005]) use at Wave 3. Parental education was not associated with use of combustible tobacco products or other drugs at Wave 3 adjusting for Wave 1 combustible tobacco and other drug use, respectively (ps ≥ 0.061).
Conclusion:
Diminished access to and engagement in substance-free enjoyable activity may in part underlie socioeconomic disparities in adolescent alcohol and marijuana use risk. Increasing substance-free enjoyable activities may be useful in substance abuse prevention in socioeconomically disadvantaged youth.
Keywords: Health disparities, Socioeconomic status, Alternative reinforcers, Adolescents, Substance use
1. Introduction
Substance use is one of the greatest sources of preventable morbidity and mortality (Lozano et al., 2013). From a lifespan perspective, adolescence is a crucial period when substance use typically onsets and confers vulnerability to persistent use problems throughout adulthood (Bonomo et al., 2004; King and Chassin, 2007; Mathers et al., 2006; Riggs et al, 2007; Windle et al, 2008). Furthermore, the onset of substance use during earlier stages of adolescence (i.e., the ages of 13—15) can interfere with normative neural and social development occurring during this period, potentially having long-term negative effects (Casey and Jones, 2010; Lubman et al., 2007b; Squeglia et al., 2009). Despite the importance and impact of adolescent substance use, preventive interventions often have modest effects (Lubman et al., 2007a). As such, understanding risk pathways to substance use uptake during adolescence is vital for developing preventive interventions that may reduce the overall public health burden associated with substance use.
A critical issue in the consideration of adolescent substance use is the substantial inequity in risk of use uptake across different segments of society (Pampel et al., 2010). Socioeconomic disparities in the prevalence of substance use, abuse, and dependence across a wide range of substances exist (Barbeau et al., 2004; Chuang et al., 2005; Pampel et al., 2010; Van Lenthe et al., 2007; Williams et al., 2010) and appear to emerge as early as adolescence (Bachman et al., 2011; Edwards et al., 2007; Lemstra et al., 2008; Reijneveld et al., 2005; Unger et al., 2007). Indeed, markers of socioeconomic status (SES), such as level of parental education, are inversely associated with substance use in adolescents (Bachman et al., 2011; Edwards et al., 2007; Unger et al., 2007). Expanding lines of inquiry to the identification of modifiable mechanisms that underlie the association between SES and adolescent substance use is essential. These mechanisms can be targeted in intervention programs to prevent the onset of harmful patterns of use that disproportionately affect the socioeconomically disadvantaged. Addressing modifiable mechanisms can ultimately reduce socioeconomic disparities in substance use that begin in adolescence and potentially extend across the lifespan.
One key risk factor for substance use that may be over-represented amongst teens of lower SES is the absence of engagement in healthy pleasant activities that protect against substance use uptake (Leventhal et al., 2015); these activities are referred to in the behavioral economic literature and throughout this paper as alternative reinforcers. Individuals have an inherent drive to experience pleasure or reinforcement that is especially prominent during adolescence, a phenomenon that has implications for substance use risk (Correia et al., 2005; Hardin and Ernst, 2009; Steinberg, 2008). Based on a behavioral economic framework, substance use can be conceptualized as a choice (Bickel and Vuchinich, 2000; Green and Fisher, 2000) dependent on the availability of alternative reinforcers (i.e. peer organizations, sports teams, art classes) that represent alternate healthy means of obtaining pleasure outside of substance use (Audrain-McGovern et al., 2004b). If there is an alternative means of deriving enjoyment, the motivation to use substances decreases; if alternative reinforcement is less available or difficult to access, the motivation to use substances will increase. Literature supports this supposition, documenting inverse associations between engagement in alternative reinforcing activities and substance use among youth (Audrain-McGovern et al., 2004a; Audrain-McGovern et al., 2011; Correia et al., 2005; Leventhal et al., 2015).
Adolescents of lower SES may have less access to substance-free pleasant activities due to financial restrictions (unable to pay for music lessons, sports teams, etc.), neighborhood deprivation (i.e. low-SES youth may be surrounded by fewer recreational outlets), or other constraints (Control and Prevention, 2003; Diez Roux and Mair, 2010; Hanson and Chen, 2007; Moore et al., 2008; Powell et al., 2006). Because adolescents of lower SES may have fewer available substance-free alternative reinforcers, they may be more likely to choose substances — which are often abundantly available in low SES communities (LaVeist and Wallace, 2000) — as a means of deriving pleasure. If the absence of alternative reinforcers is indeed overrepresented amongst low-SES teens and explains the socioeconomic disparity in adolescent substance use, the application of interventions that aim to increase substance-free alternative reinforcers may be ideal for preventing substance use among low-SES teens and reducing socioeconomic disparities.
A previous analysis of a sample of ninth-grade students in Los Angeles found that alternative reinforcers mediated the cross-sectional relation between lower SES and substance use (Leventhal et al., 2015). That is, lower SES was associated with lower alternative reinforcement, which in turn was associated with greater susceptibility to substance use experimentation and greater likelihood of sustained use. However, two points require further clarification. First, given the cross-sectional design, the directionality of the association remained unclear. Consequently, it is imperative to empirically test this mediational pathway using a longitudinal design prior to concluding that alternative reinforcers are a promising intervention to offset socioeconomic disparities in adolescent substance use risk. Second, alternative reinforcement can be disaggregated into: (1) frequency of engagement in a diversity of substance-free activities; (2) enjoyment derived from engaging in substance-free activities; and (3) their combination, meaning the synergistic impact of engaging in substance-free activities that are high in enjoyment on a frequent basis (frequency × enjoyment product score). The prior study focused solely on the product of frequency and engagement, leaving unclear which element of alternative reinforcement may underlie the observed socioeconomic disparities in substance use.
To address these questions, the present report examines diminished alternative reinforcement as a mechanism underlying socioeconomic disparities in adolescent substance use by examining a longitudinal cohort that spanned a critical developmental window of risk. This report includes measures assessed at baseline (Wave 1; fall 9th grade), a six-month follow-up (Wave 2; spring 9th grade), and a twelve-month follow-up (Wave 3; fall 10th grade). This study used mediational analysis to test the hypothesis that three elements of diminished alternative reinforcement (i.e. frequency of engagement, level of enjoyment, and frequency × enjoyment product scores of substance-free typically pleasant activities) at Wave 2 would mediate the inverse relation between SES at Wave 1 and use of alcohol, marijuana, tobacco and other drugs at Wave 3.
2. Methods
2.1. Participants and procedures
Data were collected as part of the Happiness and Health Study, a longitudinal survey of substance use and mental health among students from ten participating high schools in the Los Angeles metropolitan area. These schools were selected based on their representation of diverse sociodemographic characteristics (see Table 1); using school level data, the percentage of students eligible for free lunch within each school (student’s parental income ≤ 185% of the national poverty level) on average across the ten schools was 31.1% (SD = 19.7, range across school: 8.0% — 68.2%). Students who were not enrolled in special education (i.e., students with severe learning disabilities) or English as a Second Language Programs were eligible (N = 4100). Among 4100 eligible students, 3874 (94.5%) provided active written or verbal assent; of this group, 3396 (82.8%) provided active written or verbal parental consent. Each participating school was compensated through their general activity fund. Prior to the start of the study, students were given $5 gift cards to local retailers (e.g., Starbucks) for returning a consent form, regardless of whether or not they chose to consent to study participation. Students were not individually given monetary compensation for completion of the survey; however, on the day of data collection, students were offered small incentives (e.g., pencils, pens and keychains) for their participation.
Table 1.
Variable | Statistics |
---|---|
Sociodemographics | |
Age, M(SD) | 14.1 (0.4) |
Gender, % | |
Female | 54.8% |
Male | 45.2% |
Race/Ethnicity, % | |
American Indian or Alaskan Native | 0.8% |
Asian | 16.8% |
Black or African American | 4.7% |
Hispanic or Latino | 45.7% |
Native Hawaiian or Pacific Islander | 3.6% |
White | 16.7% |
Multiracial | 5.9% |
Other | 5.8% |
Highest parental education, % | |
8th grade or less | 4.0% |
Some high school | 9.0% |
High school graduate | 16.2% |
Some college | 19.4% |
College graduate | 32.4% |
Advanced degree | 19.0% |
Living situation, % | |
Both parents | 65.3% |
Other | 34.7% |
Alternative reinforcersa at Wave 1 / Wave 2, M(SD) | |
Product | 73.4 (27.7) / 69.0 (31.0)b |
Frequency | 46.9 (14.4) / 45.6 (15.6)b |
Pleasure | 56.3 (15.8) / 53.9 (18.3)b |
Past six-month use at Wave 1 / Wave 2 / Wave 3, % | |
Alcohol | 14.5% / 22.2%b / 23.3% |
Marijuana | 8.1% / 13.8%b / 15.3% |
Tobacco | 11.3% / 18.7%b / 16.9%c |
Other drugs | 5.1% / 11.8%b / 9.4%c |
Note. Students from Los Angeles, CA area high schools participating in the Happiness & Health Study in 2013 and 2014 who completed survey measures across Waves 1–3 on parental education, alternative reinforcement, and past six-month any use (N = 2633). Wave 1 = Cohort in fall of 9th grade year (2013; baseline); Wave 2 = spring 9th grade (2014; 6-month follow-up); Wave 3 = fall 10th grade (2014; 12-month follow-up).
Modified Pleasant Events Schedule.
Significant difference in mean score or prevalence in comparison to respective Wave 1 mean score or prevalence (p < .05).
Significant difference in prevalence from Wave 2 respective prevalence (p < .05).
Data collection involved 3 assessment waves that took place approximately six months apart: Wave 1 (baseline; 9th grade, fall 2013, number of students surveyed = 3383), Wave 2 (6-month follow-up; 9th grade, spring 2013; n surveyed = 3293), and Wave 3 (12-month follow-up; 10th grade, fall 2014; n surveyed = 3283). At each wave, paper-and-pencil surveys were administered onsite in students’ classrooms. Students who were not in class during data collections completed telephone, postal mail, or online surveys; these students were offered $10 gift cards to two retailers, Starbucks or Amazon, for completing a survey outside of school. Data were missing for students who did not complete the entire survey within the allotted time or were absent on one of the assessment days and unable to complete surveys through alternate methods. Among participants who completed surveys at all three waves (N = 3221), those who did not provide complete data on SES, alternative reinforcement, or substance use outcomes at all waves due to item-level non-response (range of ns: 265–342 depending on substance-specific outcome; see below) and those who selected the response “Don’t know” for both parents’ education level (n = 403) were not included in the analyses. Available data varies across substance specific outcomes (alcohol: N = 2476; marijuana: N = 2536; tobacco: N = 2551; other drugs: N = 2553). The University of Southern California Institutional Review Board approved the study.
2.2. Measures
2.2.1. Parental education
Highest level of parental education completed, as reported at Wave 1, was assessed using ordinal forced choice item for each parent (1 = 8th grade or less, 2 = some high school, 3 = high school graduate, 4 = some college, 5 = college graduate, 6 = advanced degree). Highest level of parental education completed was the key SES marker in a prior cross-sectional report demonstrating interrelations between SES, alternative reinforcers, and substance use (Leventhal et al., 2015). As in prior work using parental education as a marker of adolescent SES (Unger et al., 2007), the highest education level across the two parents was used in analyses; if data was available for only one parent (n = 31), that response was used.
2.2.2. Past six-month substance use
Substance use was assessed at each of the three time points using standard validated items used in epidemiologic surveys of adolescents (Johnston et al., 2015). Students were asked whether they had used any of the following substances for recreational purposes or to get “high” in the past six-months (yes/no): few puffs of a cigarette, whole cigarette, electronic cigarettes, smokeless tobacco, big cigars, little cigars or cigarillos, hookah water pipes, other forms of tobacco products, marijuana, blunts, one full drink of alcohol, inhalants, cocaine, methamphetamines, ecstasy, LSD/mushrooms/psychedelics, salvia, heroin, prescription pain killers, tranquilizers or sedatives, diet pills, and prescription stimulant pills. These responses were used to code the primary past six-month use outcomes (yes/no) for: (1) alcohol, (2) marijuana (use of marijuana or blunts [marijuana rolled in a tobacco leaf]), (3) combustible tobacco (use of any of the tobacco products other than e-cigarettes), and (4) other drug use (use of any other substance besides those included in the alcohol, marijuana and tobacco categories). See Table 1 for past six-month prevalence for each of the four drug categories by assessment wave.
2.2.3. Alternative reinforcement
This study utilized a modified version of the Pleasant Events Schedule (PES) (MacPhillamy and Lewinsohn, 1976) for adolescents, which has also been used in other prior work (Audrain-McGovern et al., 2011; Leventhal et al., 2015). The PES was administered at all three time points. Participants rated 44 different typically pleasant social and non-social/solitary activities (e.g., going out to eat at a restaurant, playing musical instruments, visiting/hanging out with friends, participating in clubs or community organizations) for both frequency of engagement (coded as 0 = never; 1 =1–6 times; 2 = 7or more times) and pleasure experienced (0 = not pleasurable; 1 = somewhat pleasurable; 2 = very pleasurable) in the past 30 days. Additionally, participants were asked to indicate (yes/no) whether they associated the pleasant activity with alcohol, smoking, or drug use (Bickel and Vuchinich, 2000). For activities reported as ‘not associated with substance use,’ three alternative reinforcement outcomes were calculated: (1) sum of frequency of engagement ratings of each activity, (2) sum of enjoyment ratings of each activity, and (3) the sum of the frequency × enjoyment product scores for each activity.
2.2.4. Covariates
Gender, race/ethnicity (coded as a nominal variable to reflect the 8 categories listed in Table 1) and living situation (i.e., who do you live with most of the time?; coded as 1 = both parents, 0 = other type of living situation [e.g., single parent household or other relative]) were measured as covariates. These factors may be associated with substance use and SES and therefore may confound key associations (Duncan et al., 2002; Ellickson et al., 2001; Jefferis et al., 2004).
2.3. Analytic approach
Preliminary analysis involved reporting descriptive statistics and testing for within-participant changes in mean scores and prevalence rates across waves for alternative reinforcers and substance use variables. All primary analyses utilized generalized estimating equations (GEEs) accounting for nesting of data within school or logistic binary or linear continuous regression modeling, which were used for substance use statuses and alternative reinforcement score outcomes, respectively (Zeger et al.,1988). For each substance outcome, the relation of Wave 1 SES with Wave 3 substance use, controlling for Wave 1 substance use status (total effect), was first tested. For outcomes with significant total effects, mediation was then tested through computing effects from two component paths: (l)The A path constituted the relation of Wave 1 SES with alternative reinforcement at Wave 2 adjusting for Wave 1 alternative reinforcement. (2) The B path constituted the relation of alternative reinforcement at Wave 2 with substance use at Wave 3 when adjusting for Wave 1 SES and substance use. The product of the coefficients from the A path and B path models indicate the strength of the indirect (“mediated”) effect. Using the PRODCLIN approach (MacKinnon et al., 2007), the significance of the mediational pathway was determined by computing asymmetric 95% confidence intervals (CIs) around the indirect effect, whereby 95% CIs that do not overlap with zero are considered statistically significant. For each outcome, separate mediational pathways were tested for the cross product, frequency, and enjoyment mediator variables. Also reported are: (1) the remaining direct effects of parental education on substance use at Wave 3 adjusting for the mediator (alternative reinforcement at Wave 2), which indicates the extent of the total effect that is not explained by the mediational pathway; and (2) the proportion of the total effect accounted for via the mediational pathway (indirect effect divided by total effect).
GEEs were tested in an initial model and adjusted model that added study covariates (gender, ethnicity [8 categories: see Table 1], and living situation [lives with both parents vs. other]). Analyses were conducted in SAS via PROC GENMOD using an exchangeable correlation matrix, modeling parental education as a continuous variable, as in Leventhal et al. (2015).
Missing data on covariates were accounted for using dummy variable adjustment (Cohen et al., 2013), in order to allow inclusion of the entire analytic sample in analyses; thus, a dummy variable codes the missingness of each covariate with missing data. Continuous variables were standardized (M = 0, SD = 1) prior to GEE modeling to facilitate interpretation. Results are reported as regression coefficients (βs±95% CIs). Significance was set to 0.05.
2.3.1. Exploratory analysis of sex and ethnic differences
To determine the role of sex and ethnicity in the findings, this study conducted exploratory analyses of interaction terms between sex and parental education and between ethnicity and parental education in the prediction of Wave 3 substance use; this was used to determine if total effect estimates varied by sex or ethnicity. To determine the potential for moderated mediation by sex or ethnicity, this study further examined the interaction terms of these variables with the respective predictors in the component A and B path equations as in the primary mediational analysis described above.
3. Results
3.1. Preliminary analyses
Descriptive statistics for demographics and study variables are depicted in Table 1. In the overall sample, prevalence of past six-month substance use significantly increased between Wave 1 and Wave 2 across each substance (p < 0.05). Tobacco (p = 0.02) and other drug use (p = 0.0005) prevalence significantly increased between Wave 2 and Wave 3; the prevalence of alcohol (p = 0.52) and marijuana (p = 0.12) use did not significantly change from Wave 2 to Wave 3. Alternative reinforcement variables decreased from Wave 1 to Wave 2 (ps ≤ 0.0001).
3.2. Primary analyses
3.2.1. Alcohol
As illustrated in Table 2, lower parental education at Wave 1 was associated with a greater likelihood of reporting past six-month alcohol use at Wave 3 in the initial model, adjusting for Wave 1 alcohol use status (total effect: (β = −0.184, p = 0.001). Component paths in modeling mediation by PES cross-product showed significant A (β = 0.061, p < 0.0001) and B (β = −0.219, p = 0.007) paths. Lower SES at Wave 1 was associated with: (1) decreased levels of alternative reinforcement cross-product scores at Wave 2 (adjusting for Wave 1 cross-product scores); and (2) decreased alternative reinforcement cross-product scores at Wave 2 were associated with greater likelihood of alcohol use at Wave 3 (adjusting for Wave 1 cross-product scores and alcohol use). The indirect effect was significant (β = −0.013, p < 0.05), such that 7.1% of association between lower Wave 1 SES and greater likelihood of alcohol use at Wave 3 was mediated by diminished alternative reinforcement at the intermediate Wave 2 timepoint. A direct effect remained (β = −0.166, p = 0.005), indicating that a statistically significant portion of the association of Wave 1 SES to Wave 3 alcohol use was not accounted for by the mediational pathway.
Table 2.
Total Effect | Component Paths | Mediation: SES → Alt. Reinf. → Alcohol Use |
||||
---|---|---|---|---|---|---|
SES → Alcohol Usea | SES → Alt. Reinf.b | Alt. Reinf. → Outcome Controlling for SESc | Indirect effect | Direct effect | Percentage of total effect mediated | |
Models | β (95% CI) | β(95% CI) | β(95% CI) | β(95% CI) | β(95% CI) | |
Mediator: Pleasant Events Schedule Product Score | ||||||
Initial model | −0.184 (−0.293, −0.074)** | 0.061 (0.029, 0.092)† | −0.219 (−0.373, −0.065)** | −0.013 (−0.027, −0.003)* | −0.163 (−0.277, −0.048)** | 7.1% |
Adjustedd | −0.122 (−0.234, −0.009)* | 0.057 (0.016, 0.097)** | −0.226 (−0.337, −0.116)** | −0.013 (−0.026, −0.003)* | −0.098 (−0.211, −0.015)* | 10.7% |
Mediator: Pleasant Events Schedule Frequency Rating | ||||||
Initial model | −0.184 (−0.293, −0.074)** | 0.065 (0.036, 0.094) † | −0.128 (−0.257, 0.002)e | −0.008 (−0.019, 0.0)f | −0.170 (−0.284, −0.056)** | 4.3% |
Adjustedd | −0.122 (−0.234, −0.009)* | 0.056 (0.020, 0.092)** | −0.128 (−0.230, 0.026)* | −0.007 (−0.016, −0.001)* | −0.109 (−0.221, −0.004)* | 5.7% |
Mediator: Pleasant Events Schedule Enjoyment Rating | ||||||
Initial model | −0.184 (−0.293, −0.074)*** | 0.050 (0.030, 0.069) † | −0.208 (−0.351, −0.064)** | −0.010 (−0.019, −0.003)* | −0.167 (−0.275, −0.059)** | 5.4% |
Adjustedd | −0.122 (−0.234, −0.009)* | 0.044 (0.020, 0.068)*** | −0.222 (−0.335, −0.108)** | −0.010 (−0.018, −0.004)* | −0.101 (−0.214, 0.012)e | 8.2% |
Note. Students from Los Angeles, CA area high schools participating in the Happiness & Health Study in 2013 and 2014. SES = Socioeconomic status as indicated by highest parental education. Alt. Reinf. = Alternative Reinforcers as assessed by a Modified Pleasant Events Schedule. Alcohol Use = Consumption of a full drink of alcohol in the prior six months. β(95% CI) = Standardized parameter estimate with 95% confidence interval for regressor estimate from Generalized Estimating Equation model accounting for nesting of data within school. Each model uses the overall sample with alcohol data, N = 2476.
p ≤ 0.08.
p = 0.05.
p < 0.05.
p < 0.01.
p < 0.001.
Association of Wave 1 SES with Wave 3 alcohol use controlling for Wave 1 alcohol use.
Association of Wave 1 SES with alternative reinforcement at Wave 2 adjusting for Wave 1 alternative reinforcement (“a” path).
Association of Wave 2 alternative reinforcement with Wave 3 alcohol use controlling for Wave 1 SES and alcohol use (“b” path).
Models are adjusted for gender, ethnicity/race, and living situation
When each of the abovementioned models were re-tested by substituting PES measures for frequency of engagement and enjoyment as mediators and after adjusting for gender, race/ethnicity, and living situation, the results were largely unchanged and remained either statistically significant (p < 0.05) or on a non-significant trend (ps: 0.054 to 0.08; see Table 2.for details).
3.2.2. Marijuana
Table 3 highlights that lower parental education at Wave 1 was associated with a greater likelihood of reporting past six-month marijuana use at Wave 3 in the initial model with adjustment for Wave 1 substance use status (total effect: β = −0.192, p = 0.002). Both the A path (β = 0.056, p < 0.0001) and B path (β = −0.360, p < 0.0001) were also significant when modeling mediation by PES cross-product. The indirect effect was significant (β = −0.020, p < 0.05), such that the 10.4% of association between lower Wave 1 SES and greater likelihood of marijuana use at Wave 3 was mediated by diminished alternative reinforcement at the intermediate Wave 2 timepoint. A direct effect remained (β = −0.167, p = 0.009), indicating that a statistically significant portion of the association of Wave 1 SES to Wave 3 marijuana use was not accounted for by the mediational pathway.
Table 3.
Total Effect | Component Paths | Mediation: SES → Alt. Reinf. → Marijuana Use |
||||
---|---|---|---|---|---|---|
SES → Marijuana Usea | SES → Alt. Reinf.b | Alt. Reinf. → Outcome Controlling for SESc | Indirect effect | Direct effect | Percentage of total effect mediated | |
β(95% CI) | β(95% CI) | β(95% CI) | β(95% CI) | β(95% CI) | ||
Mediator: Pleasant Events Schedule Product Score | ||||||
Initial model | −0.192 (−0.323, −0.062)** | 0.056 (0.027, 0.086)† | −0.360 (−0.510, −0.210)† | −0.020 (−0.035, −0.008)* | −0.167 (−0.299, −0.034)** | 10.4% |
Adjustedd | −0.168 (−0.302, −0.034)* | 0.054 (0.015, 0.092)** | −0.342 (−0.482, −0.202)† | −0.018 (−0.035, −0.005)* | −0.139 (−0.276, −0.002)* | 10.7% |
Mediator: Pleasant Events Schedule Frequency Rating | ||||||
Initial model | −0.192 (−0.323, −0.062)** | 0.061 (0.031, 0.091)† | −0.218 (−0.347, −0.088)** | −0.013 (−0.025, −0.004)* | −0.174 (−0.303, −0.044)** | 6.8% |
Adjustedd | −0.168 (−0.302, −0.034)* | 0.054 (0.020, 0.088)** | −0.198 (−0.325, −0.071)** | −0.011 (−0.022, −0.002)* | −0.150 (−0.285, −0.014)* | 6.5% |
Mediator: Pleasant Events Schedule Enjoyment Rating | ||||||
Initial model | −0.192 (−0.323, −0.062)** | 0.045 (0.028, 0.061)† | −0.399 (−0.521, −0.276)† | −0.018 (−0.027, −0.010)* | −0.168 (−0.300, −0.036)* | 9.4% |
Adjustedd | −0.168 (−0.302, −0.034)* | 0.039 (0.018,0.061)*** | −0.386 (−0.506, −0.266)† | −0.016 (−0.027, −0.006)* | −0.137 (−0.274, −0.001)* | 9.5% |
Note. Students from Los Angeles, CA area high schools participating in the Happiness & Health Study in 2013 and 2014. SES = Socioeconomic status as indicated by highest parental education. Alt Reinf. = Alternative Reinforcers as assessed by a Modified Pleasant Events Schedule. Marijuana Use = Use of any marijuana product in the prior six months. β(95% CI) = Standardized parameter estimate with 95% confidence interval for regressor estimate from Generalized Estimating Equation model accounting for nesting of data within school. Each model uses the overall sample with marijuana data, N = 2536.
p < 0.05.
p < 0.01.
p < 0.001.
p < 0.0001.
Association of Wave 1 SES with Wave 3 marijuana use controlling for Wave 1 marijuana use.
Association of Wave 1 SES with alternative reinforcement at Wave 2 adjusting for Wave 1 alternative reinforcement (“a” path).
Association of Wave 2 alternative reinforcement with Wave 3 marijuana use controlling for Wave 1 SES and marijuana use (“b” path).
Models are adjusted for gender, ethnicity/race, and living situation.
Re-testing the above models using PES components for frequency of engagement and enjoyment, as well as re-testing after adjusting for gender, race/ethnicity, and living situation yielded largely unchanged results (see Table 3.for details).
3.2.3. Tobacco
Lower parental education at Wave 1 was not significantly associated with a greater likelihood of reporting past six-month tobacco use at Wave 3 after adjusting for Wave 1 tobacco use in the initial model (total effect: β = −0.075, p = 0.26), which precluded mediational analysis.
3.2.4. Other drugs
SES did not have a significant relationship with other drug use at Wave 3 controlling for Wave 1 use (total effect: β = −0.127, p = 0.06), which precluded mediational analysis.
3.2.5. Exploratory analysis of sex and ethnic differences
Interaction terms between (1) sex and parental education and (2) ethnicity and parental education in the prediction of Wave 3 substance use were not significant. Also nonsignificant were the interaction terms of ethnicity and sex with the respective predictors in the component A and B path equations as in the primary mediational analysis described above (results not shown). Consequently, no evidence was found for sex or ethnic differences in the total or mediated effects.
4. Discussion
This study reports new longitudinal evidence of alternative reinforcement as a mechanism underlying socioeconomic disparities in youth substance use. Given the longitudinal design, the findings support directionally-specific inferences regarding how SES, alternative reinforcers, and specific substance use may be interrelated. Results were largely consistent across marijuana and alcohol use outcomes, multiple elements of alternative reinforcement (i.e., frequency, enjoyment, and their cross-product), and with and without adjustment for relevant cofactors, including ethnicity/race and family living situation. Furthermore, there was no evidence of sex or ethnic differences in socioeconomic disparities in adolescent alcohol and marijuana use risk or their mediation by alternative reinforcement in this study. Accordingly, it is unlikely that these results reflect ethnic-specific cultural processes or that SES is simply a marker of a different family structure that contributes to alternative reinforcement and substance use. As such, this finding adds to an emerging body of research that alternative reinforcers may be protective factors against teen substance use uptake (Bickel et al., 2014; Correia et al., 2002, 2003, 2005). This study also extends prior work to note that the absence of this protective factor may be a salient underpinning of the heightened substance use risk faced by a socioeconomically disadvantaged segment of the population.
These results provide particular insight on the relative components of alternative reinforcement in socioeconomic disparities in teen substance use by disaggregating frequency of behavioral engagement and level of enjoyment from alternative substance-free activities. Both elements of alternative reinforcement mediated socioeconomic disparities in adolescent alcohol and marijuana use. Thus, the results may reflect any and all of the following mechanisms: (1) disadvantaged teens may have less access to a wider variety of substance-free activities, irrespective of the enjoyment such activities produced; (2) low SES may be associated with endogenous insensitivity to rewards (i.e., anhedonia); and/or (3) adolescents of lower SES may have poor or limited access to high quality substance-free activities that yield robust enjoyment to the typical teen. Thus, multiple facets of diminished alternative reinforcement are worthy of consideration in research and prevention approaches that target substance-free activities in low SES youth.
Teens of lower (versus higher) SES may be more likely to live in communities in which substances are particularly accessible. Increased accessibility may occur through several mechanisms, including an increased density of alcohol and tobacco retail outlets (Henriksen et al., 2008; Romley et al., 2007) and visible substance use activity that introduces an opportunity to use and may increase environmental cues to partake in substance use (Campbell et al., 2015; Linton et al., 2014). At the same time, the built environment surrounding low-SES adolescents has been shown to have disproportionately diminished density and quality of parks, recreational facilities, and community centers when compared to high-SES neighborhoods (Estabrooks et al., 2003). Such parks and recreational facilities can serve as a means to obtain alternative reinforcement to prevent substance use uptake. Thus, it may be beneficial when developing policies to recognize that the environments surrounding low-SES adolescents may be biased towards accessing substance-related reinforcement and obstructing opportunities for alternative reinforcement.
Recent life course perspective research has found that among young adults who were unemployed, those who spent their childhood and adolescence in a lower SES household saw an exacerbated detrimental impact on their heavy episodic drinking and cigarette use over time (Lee et al., 2015). Thus, to the extent that employment provides a source of substance-free gratification, insufficient alternative reinforcement from recreational activities in adolescence for individuals from lower SES backgrounds could presumably translate into a deficiency of work-related alternative reinforcement. These factors may contribute to adulthood substance use risk and intensified socioeconomic disparity.
The present report did not find significant total effects of low SES at Wave 1 predicting increased tobacco or other drug use at Wave 3 controlling for Wave 1 substance use status. As illustrated in Table 1, prevalence rates of combustible tobacco and other drug use decreased significantly across Wave 2 and Wave 3, which may reflect a tendency toward temporary experimentation with substances occurring in the transition to the first year of high school (which overlaps with Wave 2) that then dissipates as youth move into 10th grade (Wave 3). Because of this backdrop, there may have been insufficient changes in prevalence rates to detect such predictors of uptake of tobacco and other drug use. Therefore, longer follow-up periods may be necessary to model patterns of disparities for certain substances and hence permit investigation of underlying mechanisms such as alternative reinforcement. Another possibility is that socioeconomic disparities for these two drugs may onset at an earlier age (prior to high school) and therefore require longitudinal follow-up during earlier periods of development like middle school. Indeed, prior analyses of this sample found cross-sectional associations of cigarette smoking and lower parental education at the beginning of 9th grade (Leventhal et al., 2015).
4.1. Limitations
To our knowledge, this investigation is the first to examine the mediational role of alternative reinforcers in the relationship between SES and substance use in a longitudinal sample. Along with several study strengths (such as large sample, demographically diverse, and high retention rate), limitations must be noted. Participants were sampled from a restricted geographic region and data on key measures were unavailable for some students enrolled in the study, which raises limitations on generalizability. Additionally, the measure of substance use status, while highly sensitive, does not differentiate frequency of use. Therefore, it does not facilitate inferences about progression towards dependence. Although parental education is a key SES indicator (Galobardes et al., 2006), investigating multiple indicators of SES would be valuable in capturing teens who are not aware of their parents’ education level in addition to establishing the multidimensionality of the construct (Galobardes et al., 2006). The measure of PES used to assess alternative reinforcement in this report neither accounts for an expanding array of pleasant activities in diverse youth nor provides specific conceptually-homogenous subscales. It would be helpful to isolate population-specific dimensions of the PES to elucidate which activities are the most salient reinforcers in specific socioeconomic subgroups, so as to precisely inform intervention efforts for lower-SES adolescent substance use. Finally, in order to reduce the burden on educators and students, this was a self-report survey with fairly brief assessments; additional work utilizing alternate methods (e.g., biomarkers of substance exposures or clinical interviews) may be fruitful.
4.2. Conclusion
The findings of this study highlight the potential importance of alternative reinforcement as a target for both prevention and intervention efforts for socioeconomically disadvantaged youth. Regarding prevention, changes in the environment, including disparities in distribution of recreation, park access and related services across low-SES communities, may have broad implications for understanding and preventing the progression of substance use across adolescence. Regarding intervention, recent psychosocial interventions that involve teaching college students to identify and engage in healthy prosocial activities (i.e., alternative reinforcement) have been shown to be effective in reducing drinking in the general population of young adults, and should continue to be explored in teens (Correia et al., 2005; Murphy et al., 2012a, 2012b). Additionally, low SES adolescents with diminished access to alternative reinforcers may benefit from mindfulness-based interventions (Burke, 2010; Kahler et al., 2015) that teach teens to recognize and enhance their enjoyment from the limited pleasant activities available. Doing so may potentially increase the intensity and length of the enjoyment derived from the limited alternative reinforcers available to certain low-SES youth populations. Given that engagement in pleasant substance-free activities is modifiable via intervention (Correia et al., 2005; Murphy et al., 2012a), this study’s work may inform prevention programming and policy that provide adolescent alternative reinforcers in low SES communities.
Acknowledgements
Funding: This research was supported by National Institutes of Health Grants R01-DA033296 and F31039708.
Role of Funder: The funding agency had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Footnotes
Disclosures: The authors report no potential conflicts of interests.
References
- Audrain-McGovern J, Rodriguez D, Tercyak KP, Cuevas J, Rodgers K, Patterson F, 2004a. Identifying and characterizing adolescent smoking trajectories. Cancer Epidemiol. Biomark. Prev 13, 2023–2034. [PubMed] [Google Scholar]
- Audrain-McGovern J, Rodriguez D, Tercyak KP, Epstein LH, Goldman P, Wileyto EP, 2004b. Applying a behavioral economic framework to understanding adolescent smoking. Psychol. Addict. Behav 18, 64. [DOI] [PubMed] [Google Scholar]
- Audrain-McGovern J, Rodriguez D, Rodgers K, Cuevas j., 2011. Declining alternative reinforcers link depression to young adult smoking. Addiction 106, 178–187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bachman JG, O’Malley PM, Johnston LD, Schulenberg JE, Wallace JM Jr., 2011. Racial/ethnic differences in the relationship between parental education and substance use among US 8th-, 10th-, and 12th-grade students: findings from the monitoring the future project. J. Stud. Alcohol Drugs 72, 279–285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barbeau EM, Krieger N, Soobader MJ, 2004. Working class matters: socioeconomic disadvantage, race/ethnicity, gender, and smoking in NHIS 2000. Am. j. public health 94, 269–278. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickel WK, Johnson MW, Koffarnus MN, MacKillop J, Murphy JG, 2014. The behavioral economics of substance use disorders: reinforcement pathologies and their repair. Annu. Rev. Clin. Psychol 10, 641. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickel WK, Vuchinich RE, 2000. Reframing Health Behavior Change with Behavioral Economics. Psychology Press. [Google Scholar]
- Bonomo YA, Bowes G, Coffey C, Carlin JB, Patton GC, 2004. Teenage drinking and the onset of alcohol dependence: a cohort study over seven years. Addiction 99,1520–1528. [DOI] [PubMed] [Google Scholar]
- Burke CA, 2010. Mindfulness-based approaches with children and adolescents: a preliminary review of current research in an emergent field. J. Child Fam Stud 19,133–144. [Google Scholar]
- Campbell LF, Wilmoth K, Mason M, 2015. Association of exposure to neigh borhood drug activity, neurobehavioral traits, and marijuana use among at-risk African American females. Addict Behav. 50, 45–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Casey B, Jones RM., 2010. Neurobiology of the adolescent brain and behavior: implications for substance use disorders. J. Am. Acad. Child Adolesc. Psychiatr 49,1189–1201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chuang Y-C, Cubbin C, Ahn D, Winkleby MA, 2005. Effects of neighbourhood socioeconomic status and convenience store concentration on individual level smoking. J. Epidemiol. Community Health 59, 568–573. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cohen J, Cohen P, West SG, Aiken LS, 2013. Applied Multiple Regression/correlation Analysis for the Behavioral Sciences. Routledge. [Google Scholar]
- Control, C.f.D., Prevention, 2003. Physical activity levels among children aged 9–13 years-United States, 2002. MMWR. Morb. Mortal. Wkly. Rep 52, 785. [PubMed] [Google Scholar]
- Correia CJ, Benson TA, Carey KB, 2005. Decreased substance use following increases in alternative behaviors: a preliminary investigation. Addict. Behav 30, 19–27. [DOI] [PubMed] [Google Scholar]
- Correia CJ, Carey KB, Borsari B, 2002. Measuring substance-free and substance-related reinforcement in the natural environment. Psychol. Addict. Behav 16, 28. [PMC free article] [PubMed] [Google Scholar]
- Correia CJ, Carey KB, Simons J, Borsari BE, 2003. Relationships between binge drinking and substance-free reinforcement in a sample of college students: a preliminary investigation. Addict Behav. 28, 361–368. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Diez Roux AV, Mair C, 2010. Neighborhoods and health. Ann. N. Y. Acad. Sci 1186, 125–145. [DOI] [PubMed] [Google Scholar]
- Duncan SC, Duncan TE, Strycker LA, Chaumeton NR, 2002. Neighborhood physical activity opportunity: a multilevel contextual model. Res. Q. Exerc. Sport 73, 457–463. [DOI] [PubMed] [Google Scholar]
- Edwards J, Hemphill SA, Tresidder J, Humphreys C, Toumbourou J, Murray D, 2007. Mental health promotion and socioeconomic disadvantage: lessons from substance abuse, violence and crime prevention and child health. Health Pro-mot J. Aust. off. J. Aust Assoc. Health Promot. Prof 18,184. [DOI] [PubMed] [Google Scholar]
- Ellickson PL, Mcguigan KA, Klein DJ, 2001. Predictors of late-onset smoking and cessation over 10 years. J. Adolesc. Health 29,101–108. [DOI] [PubMed] [Google Scholar]
- Estabrooks PA, Lee RE., Gyurcsik NC, 2003. Resources for physical activity participation: does availability and accessibility differ by neighborhood socioeconomic status? Ann. Behav. Med 25, 100–104. [DOI] [PubMed] [Google Scholar]
- Galobardes B, Shaw M, Lawlor DA, Lynch JW, Smith GD, 2006. Indicators of socioeconomic position (part 1). J. Epidemiol. Community Health 60, 7–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Green L, Fisher E, 2000. Economic substitutability: some implications for health behavior. Reframing health Behav. Change Behav. Econ 115–144. [Google Scholar]
- Hanson MD, Chen E, 2007. Socioeconomic status and health behaviors in adolescence: a review of the literature. J. Behav. Med 30, 263–285. [DOI] [PubMed] [Google Scholar]
- Hardin MG, Ernst M, 2009. Functional brain imaging of development-related risk and vulnerability for substance use in adolescents. J. Addict. Med 3, 47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Henriksen L, Feighery EC., Schleicher NC, Cowling DW, Kline RS, Fortmann SP, 2008. Is adolescent smoking related to the density and proximity of tobacco outlets and retail cigarette advertising near schools? Prev. Med 47, 210–214. [DOI] [PubMed] [Google Scholar]
- Jefferis BJ, Power C, Graham H, Manor O, 2004. Effects of childhood socioeconomic circumstances on persistent smoking. Am. J. public health 94, 279–285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnston L, O’Malley P, Miech R, Bachman J, Schulenberg J, 2015. Monitoring the Future National Survey Results on Drug Use: 1975–2014: Overview, Key Findings on Adolescent Drug Use. Ann Arbor: Institute for Social Research, The University of Michigan. [Google Scholar]
- Kahler CW, Spillane NS, Day AM, Cioe PA, Parks A, Leventhal AM, et al. , 2015. Positive psychotherapy for smoking cessation: a pilot randomized controlled trial. Nicotine Tob. Res ntv011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- King KM, Chassin L, 2007. A prospective study of the effects of age of initiation of alcohol and drug use on young adult substance dependence. J. Stud. Alcohol Drugs 68, 256–265. [DOI] [PubMed] [Google Scholar]
- LaVeist TA, Wallace JM, 2000. Health risk and inequitable distribution of liquor stores in African American neighborhood. Soc. Sci. Med 51, 613–617. [DOI] [PubMed] [Google Scholar]
- Lee JO, Hill KG, Hartigan LA, Boden JM, Guttmannova K, Kosterman R, et al. , 2015. Unemployment and substance use problems among young adults: does childhood low socioeconomic status exacerbate the effect? Soc. Sci. Med 143, 36–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lemstra M, Bennett NR, Neudorf C, Kunst A, Nannapaneni U, Warren LM., et al. , 2008. A meta-analysis of marijuana and alcohol use by socio-economic status in adolescents aged 10–15 years. Can. J. Public Health/Revue Can. de Sante’e Publique 172–177. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leventhal AM, Bello MS, Unger JB, Strong DR, Kirkpatrick MG, Audrain-McGovern J, 2015. Diminished alternative reinforcement as a mechanism underlying socioeconomic disparities in adolescent substance use. Prev. Med 80, 75–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Linton SL, Jennings JM, Latkin CA, Gomez MB, Mehta SH, 2014. Application of space-time scan statistics to describe geographic and temporal clustering of visible drug activity. J. Urb. Health 91, 940–956. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, et al. , 2013. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 380, 2095–2128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lubman DI, Hides L, Yucel M, Toumbourou JW, Lubman D, Hides L, et al. , 2007a. Intervening early to reduce developmentally harmful substance use among youth populations. Med. J. Aust 187, S22–S25. [DOI] [PubMed] [Google Scholar]
- Lubman DI, Yucel M, Hall WD, 2007b. Substance use and the adolescent brain: a toxic combination? J. Psychopharmacol 21, 792–794. [DOI] [PubMed] [Google Scholar]
- MacKinnon DP, Fritz MS, Williams J, Lockwood CM, 2007. Distribution of the product confidence limits for the indirect effect: program PRODCLIN. Behav. Res. methods 39, 384–389. [DOI] [PMC free article] [PubMed] [Google Scholar]
- MacPhillamy DJ, Lewinsohn PM, 1976. Manual for the Pleasant Events Schedule: DJ MacPhillamy & PM Lewinsohn.
- Mathers M, Toumbourou J, Catalano R, Williams J, Patton G, 2006. Conse quences of youth tobacco use: a review of prospective behavioural studies. Addiction 101, 948–958. [DOI] [PubMed] [Google Scholar]
- Moore LV., Roux AVD, Evenson KR, McGinn AP, Brines SJ, 2008. Availability of recreational resources in minority and low socioeconomic status areas. Am. J. Prev. Med 34,16–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Murphy JG, Dennhardt AA, Skidmore JR, Borsari B, Barnett NP, Colby SM, et al. , 2012a. A randomized controlled trial of a behavioral economic supplement to brief motivational interventions for college drinking. J. Consult. Clin. Psychol 80, 876. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Murphy JG, Skidmore JR, Dennhardt AA, Martens MP, Borsari B, Barnett NP, et al. , 2012b. A behavioral economic supplement to brief motivational interventions for college drinking. Addict Res. Theory 20, 456–465. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pampel FC, Krueger PM, Denney JT, 2010. Socioeconomic disparities in health behaviors. Annu. Rev. Sociol 36, 349. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Powell LM, Slater S, Chaloupka FJ, Harper D, 2006. Availability of physical activity-related facilities and neighborhood demographic and socioeconomic characteristics: a national study. Am. J. Public Health 96,1676–1680. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reijneveld SA, Brugman E, Verhulst FC, Verloove-Vanhorick SP, 2005. Area deprivation and child psychosocial problems. Soc. Psychiatry Psychiatr. Epidemiol 40, 18–23. [DOI] [PubMed] [Google Scholar]
- Riggs NR, Chou C-P, Li C, Pentz MA, 2007. Adolescent to emerging adulthood smoking trajectories: when do smoking trajectories diverge, and do they predict early adulthood nicotine dependence? Nicotine Tob. Res 9, 1147–1154. [DOI] [PubMed] [Google Scholar]
- Romley JA, Cohen D, Ringel j., Sturm R, 2007. Alcohol and environmental justice: the density of liquor stores and bars in urban neighborhoods in the United States. J. Stud. Alcohol Drugs 68, 48–55. [DOI] [PubMed] [Google Scholar]
- Squeglia LM, Jacobus J, Tapert SF, 2009. The influence of substance use on adolescent brain development Clin. EEG Neurosci. 40, 31–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Steinberg L, 2008. A social neuroscience perspective on adolescent risk-taking. Dev. Rev 28, 78–106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Unger JB, Sun P, Johnson CA, 2007. Socioeconomic correlates of smoking among an ethnically diverse sample of 8th grade adolescents in Southern California. Prev. Med 44, 323–327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Van Lenthe FJ, Martikainen P, Mackenbach JP, 2007. Neighbourhood inequalities in health and health-related behaviour: results of selective migration? Health & Place 13,123–137. [DOI] [PubMed] [Google Scholar]
- Williams DR, Mohammed SA, Leavell J, Collins C, 2010. Race, socioeconomic status, and health: complexities, ongoing challenges, and research opportunities. Ann. N. Y. Acad. Sci 1186, 69–101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Windle M, Spear LP., Fuligni AJ, Angold A, Brown JD, Pine D, et al. , 2008. Transitions into underage and problem drinking: developmental processes and mechanisms between 10 and 15 years of age. Pediatrics 121, S273–S289. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zeger SL, Liang K-Y, Albert PS, 1988. Models for longitudinal data: a generalized estimating equation approach. Biometrics 1049–1060. [PubMed] [Google Scholar]