Detecting Initiation or Risk for Initiation of Substance Use before High School during Pediatric Well-Child Check-Ups

Abstract

Background

Youth substance use (SU) is prevalent and costly, affecting mental and physical health. American Academy of Pediatrics and Affordable Care Act call for SU screening and prevention. The Youth Risk Index^© (YRI) was tested as a screening tool for having initiated and propensity to initiate SU before high school (which forecasts SU disorder). YRI was hypothesized to have good to excellent psychometrics, feasibility and stakeholder acceptability for use during well-child check-ups.

Design

A high-risk longitudinal design with two cross-sectional replication samples, ages 9–13 was used. Analyses included receiver operating characteristics and regression analyses.

Participants

A one-year longitudinal sample (N=640) was used for YRI derivation. Replication samples were a cross-sectional sample (N=345) and well-child check-up patients (N=105) for testing feasibility, validity and acceptability as a screening tool.

Results

YRI has excellent test-retest reliability and good sensitivity and specificity for concurrent and one-year-later SU (odds ratio=7.44 CI=4.3–13.0) and conduct problems (odds ratios=7.33 CI=3.9–13.7). Results were replicated in both cross-sectional samples. Well-child patients, parents and pediatric staff rated YRI screening as important, acceptable, and a needed service.

Conclusions

Identifying at-risk youth prior to age 13 could reap years of opportunity to intervene before onset of SU disorder. Most results pertained to YRI’s association with concurrent or recent past risky behaviors; further replication ought to specify its predictive validity, especially adolescent-onset risky behaviors. YRI well identifies youth at risk for SU and conduct problems prior to high school, is feasible and valid for screening during well-child check-ups, and is acceptable to stakeholders.

1. INTRODUCTION

Substance use (SU) and consequent disorder (SUD) harms large proportions of youth, society and healthcare systems. U.S. prevalence of lifetime SUD consequent to using tobacco, alcohol and illegal drugs, respectively, is 24%, 20% and 10% (Compton et al., 2007; Hasin et al., 2007). SU consequences cost the U.S. over $600 billion annually (e.g., due to SUD treatment, crime, medical and psychiatric disorders, and sexually transmitted infection; NDIC, 2011). Globally, the 4^th, 5^th, and 15^th leading contributors to disease burden are smoking, alcoholism and illegal drug use, respectively (Lopez et al., 2006).

SU before high school confers multifold increased propensity for SUD and is not rare (Hingson et al., 2006; Ystrom et al., in press). Of U.S. 8^th graders, 33.1% drank alcohol, 18.4% smoked tobacco, and 16.4% smoked marijuana (Johnston et al., 2012). Such high-risk youth need intervention before high school (about age 14) when access to addictive substances increases (Andrews et al., 2003; Windle et al., 2008). Etiology research documents numerous factors that bias child development toward SU/SUD (e.g., stress, disinhibition, peers, family characteristics; Caspi et al., 1996; Garner et al., 2012; Kirisci et al., 2009; Tarter et al., 2003). Screening for propensity of SU before high school (when SU incidence sharply increases) provides long-term opportunity to intervene before SUD onset, a propitious strategy, given the high rate of SUD relapse.

SU initiation almost always occurs during adolescence. SU/SUD prevention traditionally has occurred via schools, policy, and policing, although pediatricians are increasingly sought for assistance (NIAAA, 2011). As purveyors of youth healthcare, pediatricians are in a critical position to improve such efforts. Affirming American Academy of Pediatrics long-standing position, recent policy states that “Pediatricians should … recognize risk factors for … alcohol and other substance abuse among youth, screen for use, provide appropriate brief interventions, and refer to treatment. … prevention programs … from elementary school through college should be promoted by pediatricians and the health care community” (AAP, 2010). The Affordable Care Act mandated that youth SU be assessed without copayment and insurance cover preventive services (DHHS, 2012). However, the U.S. Preventive Services Task Force (2013) deemed that evidence is insufficient to recommend for or against screening and counseling for youth SU.

Pediatrician use of standardized screening tools for developmental delays has recently increased (Radecki et al., 2011) whereas using standardized tools for risk of behavior problems has lagged (Jee et al., 2011; Kelleher and Stevens, 2009). Reasons for their reluctance include practice burden (time, staffing cost), unfamiliarity with treatment resources for referral, high false-positive rates of screens, and fear of alienating patients or parents with queries about sensitive behaviors (Van Hook, et al., 2007). Given the Affordable Care Act opportunities for expanded medical home models, including co-located mental health and SUD prevention services, a screening tool for early identification of SU risk (per the Bright Futures guidelines) is the final component needed for healthcare-based SU prevention. Reinforcing the strategy of implementing prevention prior to high school, well-child check-up compliance is nearly 1/3 more prevalent in U.S. 6- to 12-year-olds than older youth (Selden, 2006).

As mentioned, one barrier to pediatricians using behavioral screening tools is fear of alienating patients and parents with use of sensitive questions, such as child SU. Also, many young adolescents who have high propensity for SUD have not initiated SU (e.g., due to availability). Thus the objective of this study was to develop the Youth Risk Index^© (YRI^©) screening tool to measure propensity for SU/SUD rather than SU per se. This study tested the YRI for identifying 9- to 13-year-olds who have or are at risk for initiating SU before high school using a rigorous approach of replicating results in two samples and for a second outcome, conduct disorder behaviors, which frequently precedes SU/SUD. YRI items were from the Assessment of Liability and EXposure to Substance use and Antisocial behavior^© (ALEXSA^©) system which is more developmentally appropriate than paper-and-pencil alternatives for pre-teens (Ridenour and Feinberg, 2007; Ridenour et al., 2009; 2011; 2012). Youth are the best informants for many of their own characteristics (e.g., affect and behaviors they conceal) and their perspective provides unique information not available from parents or teachers (de los Reyes and Kazdin, 2005). ALEXSA’s reliability and validity spans ages 6 – 16, different races/ethnicities, genders, psychiatric functioning, academic abilities, residential settings, and in utero drug exposure (Ridenour and Feinberg, 2007; Ridenour et al., 2009, 2011, 2012; Min et al., 2014). ALEXSA is computerized, cartoon-based (Figure 1), and appealing to children; its audios read questions and response options; and it can output electronic records. It can be completed by illiterate children, a critical feature considering 38% of U.S. 4^th graders (51% in urban settings) cannot read at basic levels (Lutkus et al., 2005). To avoid querying sensitive behaviors, YRI items were drawn from ALEXSA “risk factor” subscales, each of which has a well-developed theoretical foundation and documented statistical prediction of behavior problems (Table 1).

Figure 1.

Illustration of the Youth Risk Index Item Format.

Note: The electronic version of the picture depicting “No way!” is animated with Alex or Alexis shaking his/her head in a “no” motion. The electronic version of the picture depicting “Yes - definitely!” is animated with Alex or Alexis nodding his/her head.

Table 1.

ALEXSA Domains of Risk

Risk Domain Factor	Reliability21,23		Brief Description	Risk Factor Subscales (number of items included in the YRI)
	Test-retest (ICC)	Internal ConsistencyA
		Internal Sources of Risk
Disinhibition	.81	.92	Volatility and regulation of emotional and behavioral impulses	Anger Coping (6), Distractibility (4), Impulsivity (4), Irritability
Acceptance of Deviant Behavior	.76B	.80	Perceived permissibility of socially unsanctioned behaviors	Safety of Drugs (1), Tolerance of Deviance (3)
Self Management	.76	.82	Learned skills that reduce the probability of and consequences from mistakes or disinhibition	Planning and Concentration, Problem Solving
School Protection	.74	.72	Confidence in, enjoyment of, and motivation for good academic performance	Academic Competency, School Bonding, School Commitment, School Atmosphere: Adults
Sensation Seeking	.82	.84	Zuckerman sensation seeking constructs that correlate with behavior problems in youth	Social Disinhibition (2), Thrill Seeking (2), Gambling
		External Sources of Risk
Social Contagion	.80	.91	Exposure to and vulnerability to imitating peer problem behaviors	Friends’ Conduct Disorder Criteria (7), Peer Pressure Susceptibility (4), Violence Exposure (1)
Social Support	.80C	.83	Emotional bonding, communication, and support for coping with problems that is provided by others	Peer Attachment, Social Support: Adults, Social Support: Youth, Number of Friends
Neighborhood Risks	.76	.81	Neighborhood characteristics associated with crime and SUD as well as personal stressors	Neighborhood Atmosphere, School Atmosphere: Students, Street Gang Exposure, Last Year Stressors
Family Discord	.71	.74	Chaotic family life in terms of dysfunctional interactions, SU, and problems with the law	Family Conflict (2), Family Behavior Problems, Parental Permissiveness
Parent Fortification	.74	.90	Caretaker nurturance, parent knowledge of youth behavior away from the family, affect and behavior between the youth and caretaker	Parental Attachment, Parental Monitoring, Parental Nurturance
		Clinical Measures (Subscales)
Conduct Disorder Criteria	.69	.78	Items query DSM-IV conduct disorder behaviors (conduct problems consists of 2 or more behaviors)	n/a
Depression (7)	.60	.87	Items query youth symptoms of depression.	n/a
Alcohol Risk Index (2)	.78	.81	Alcohol use and risks that are highly proximal to alcohol use	n/a
Tobacco Risk Index (2)	.72	.73	Tobacco use and risks that are highly proximal to tobacco use	n/a

Note: ICC = intraclass correlation; ICCs of 0.40 or less are low; 0.40 to 0.59 are fair; 0.60 to 0.74 are good; 0.74 or greater are excellent.

^A= Cronbach’s alpha (Shrout and Fleiss, 1979).

^B= one subscale, Safety of Drugs, was not included in test-retest analysis.

^C= one subscale, Peer Attachment, was not included in test-retest analysis. Underlined subscales contain items that are included in the YRI; parenthetical numbers indicate how many subscale items are included in the YRI.

YRI was hypothesized to have sensitivity, specificity, reliability and validity for measuring propensity for SU and conduct problems prior to high school. A brief YRI, consisting of the items that correlate most with SU, was hypothesized to be feasible, acceptable to stakeholders, and valid for screening during pediatric well-child check-ups. Purposes of well-child check-ups include monitoring a youth’s health (e.g., recording vitals or monitoring medications), advising parents on childcare and illness prevention, and confirming that a patient’s health is adequate to participate in activities (e.g., sports). Thus, assessing patients, introducing primary prevention, and advising parents are expected and commonplace procedures during pediatric well-child visits.

2. METHODS

2.1 Participants

Two samples representing greater-than-average-risk for behavior problems and a sample of well-child check-up patients were recruited in the Northeastern U.S. IRB approval was obtained from universities and collaborating organizations before initiating recruitment. Table 2 presents demographics for youth in the three samples.

Table 2.

Demographics when Entered Study

	Chronic Stress Sample (N=640)	Psychometric Sample (N=345)	Well-child Sample (N=105)
Gender (% female)	47.3%	47.9%	50.7%
Age in Years as Mean (SD)*	11.0 (1.6)	10.5 (1.1)	10.5 (1.0)
Receives Free School Meals*	60.3%	45.9%	21.1%
Race/Ethnicity * A
Caucasian	68.1%	44.2%	74.3%
African-American	16.7	17.3	21.1
Latino	5.2	16.3	1.3
Other	10.0	21.2	2.6
Consumed Alcohol or Tobacco
Prior to YRI	28.4%	31.3%	24.6%
By 1 year after YRI	32.5	- -	- -
Had 2+ Conduct Disorder Criteria
Prior to YRI	35.3%	31.3%	12.5%
By 1 year after YRI	37.0	- -	- -
YRI Screen Scores
Mean when first recruited (SD)*	1.03 (.44)	.86 (.53)	.79 (.37)
Range	.23 – 2.74	.0 – 2.84	.25 – 1.82
Mean one year after recruited (SD)	1.05 (.43)	- -	- -

Note:

^*Chronic Stress Sample differs from the other samples differ at the p<.05 level.

^A=may not sum to 100% due to rounding error.

2.1.1 Chronic Stress Sample

Between June, 2004 and August, 2010, 9- to 13-year-olds attended a summer camp designed for youth experiencing chronic stress, who are at heightened risk for SUD (McMahon et al., 2003). Staff procedures to determine camp eligibility appear elsewhere (Ridenour et al., 2011a, 2011b, 2012). They required: an adult to sponsor and apply on a youth’s behalf; the sponsor to (a) describe the youth’s stressors and (b) rate their impact on the youth’s functioning; and staff to corroborate the sponsor’s report with a professional at the youth’s school. About 60% of applications resulted in a youth attending the camp. About 70% of campers attended for two or more contiguous years, providing data for longitudinal analyses. Study dropout resembled missing-at-random patterns across demographics, child risk factors and family characteristics (Ridenour et al., 2012).

Sources of chronic stress were categorized by staff as: family poverty, serious family problems (e.g., incarcerated parent), social problems (e.g., severe peer rejection), poor academic performance, or emotional problems (e.g., mood disorder; diagnoses were not made during the study). Participants had a mean 2.4 (SD=1.3) stressors and were from urban, suburban and rural residences (residence was not quantified). Participants completed ALEXSA subscales during program evaluations. Analyses included only data from a participant’s 1^st–2^nd years of camp attendance to avoid biases due to repeated observations of only some participants.

2.1.2 Psychometric Sample

From August, 2003 to January, 2005, 9- to 12-year-olds were recruited from three settings (average, above average, and high risk) for psychometric testing of the full ALEXSA (Ridenour and Feinberg, 2007; Ridenour et al., 2009). Intentionally recruiting three subsamples of differing levels of SU/SUD risk ensured that the full range of SU/SUD risk was represented. The ALEXSA was administered twice (in the same settings), two weeks apart on average.

Exclusion criteria consisted of academic handicaps, active psychosis, and active suicidal or homicidal tendencies. Youth who met criteria were given a description of the study, a cover letter and two consent forms. Participation required that one consent form be returned with parent signature to a teacher or nurse (according to recruitment setting). The IRB-approved consent prohibited ascertainment of non-participant data (precluding comparison to participants). Even so, the final sample was useful, varying on gender, ethnicity, residential setting, economic status, educational achievement and psychiatric diagnosis (Ridenour et al., 2009).

The average risk subsample was recruited from regular education classrooms of a large rural school. The above average risk subsample was recruited from remedial education classrooms of rural and urban schools. Remedial program placements were based on poor performance in reading, mathematics or both, which correlate with behavior problems (Bachman et al., 2008; Martin et al., 2000) as do learning disabilities (Karacostas and Fisher, 1993; Magg et al., 1994). The rural program was for Mexican, migrant, English-as-Second-Language students. The urban program was for English-speaking students of any ethnicity or race.

The high risk subsample was psychiatric inpatients with a mood or disruptive behavior diagnosis (Armstrong and Costello, 2002; Molina and Pelham, 2003); patients with suicidal or homicidal tendencies were eligible to participate in this subsample. Inpatient recruitment differed from students only in that psychiatric nurses presented the study and obtained consent. Study interviewers were available to answer parent or participant questions. One hospital provided acute care, rarely lasting six or more days, typically for homicidal or suicidal behavior. The other hospital treated severe, chronic illness and required patients to stay at least eight months. All study declinations were by parents of acute care patients.

The demographic composition of the respective three subsamples differed (p<.05) in terms of the prevalence of girls (57.8%, 55.8%, 30.4%), receipt of free school meals (45.4%, 66.9%, 46.0%) and the mean and SD for age (10.4 and .97, 10.5 and 1.24, 10.9 and 1.14). The respective subsamples also differed in prevalence of Caucasian (93.3%, 1.7%, 51.7%), African-American (1.0%, 45.5%, 20.7%), Latino (1.0%, 41.3%, 13.8%) and mixed or other races (4.8%, 11.6%, 13.88%) (p<.05).

2.1.3 Well-child Check-up Patients

From October, 2012 to July, 2013, parents and patients ages 9–12 (N=152) seen at a suburban private practice (72.2%) and a rural hospital office (27.8%) were invited to participate in a feasibility study, consented, and screened by nurses while waiting for physicians in an examination room. Inclusion criteria were: being accompanied by a parent, English speaking, and not mentally handicapped (per patient charts). The parent demographics were 52% Caucasian, 43% African-American, 3% Latino; mean age =39.8 years (SD=7.1 years); 52% married to the child’s biological parent, 5.2% married to the child’s stepparent, 26% single, 12% living with a partner; and 96.6% were the child’s biological mother (the remaining were biological fathers). Household size was a mean 4.5 members (SD=1.2) and the average household monthly income was $3,001–4,000 (SD was about $2,000). Child and parent ratings of the screening protocol and validity data were collected at a university research facility approximately 2 weeks after screening occurred.

The eight pediatric staff who conducted screenings (75% pediatricians, 25% head nurses, ½ from an urban practice servicing low SES families and ½ from a suburban practice servicing middle to high SES families) were queried regarding protocol feasibility and acceptability after the screening study was completed. The brief questionnaire (available on request from the first author) was based on previously-developed surveys that evaluated stakeholder acceptability of screening for alcohol use disorder and other risky behaviors (Gordon et al., 2011; Olson et al., 2009).

Brief 16-item YRIs were used to mimic the time that is expected for the envisioned computer adaptive testing version of the YRI for clinical uses (see Discussion). Different items were administered per age, thereby testing patient acceptability of the range of YRI items. The 16-item YRIs were completed by 105 (69.1%) of eligible patients (53.1% of refusing parents could not commit to the follow-up validity assessments, 40.6% stated that they participate in no research or offered no reason). Of those screened, within one month, 58.1% of patient-parent dyads completed validity assessments (adequate for validity studies). Compared to drop-outs, patients who completed the validity assessments had greater YRI scores (mean=.70, SD=.40 vs mean=.49, SD=.30, Welch’s F=9.4, p<.005), thus representing patients more likely to screen “at risk.”

The 16-item YRIs correlated .56 with the full YRI and required a mean 4.9 minutes (SD=1.9), 2 minutes shorter than the planned 7-minute screening. Thus, for validity analyses in well-child patients, a Brief YRI score comprised of the 23 statistically best items was tested. The 23-item YRI and full YRI correlated .96 in well-child patients, .91 in the Chronic Stress Sample and .96 in the Psychometric Sample (all p<.001).

2.2 Youth Risk Index (YRI) Development

YRI items were selected from the pool of 350 ALEXSA items, which assess 40 youth risk factors for SU/SUD and antisocial behavior (detailed elsewhere; Ridenour and Feinberg, 2007; Ridenour et al., 2009, 2011, 2012). The ALEXSA’s construct validity was documented using factor analyses in which subscales statistically aggregated into 10 factors (risk domains), consistent with decades of SU/SUD etiology literature (Table 1). In summer of 2002, ten focus groups were conducted with 9- to 10-year-old children who were attending summer school (mean n=3 per group) at urban, low-income schools (10 Caucasians, 10 African-Americans, 10 Latinos) to preliminarily test and refine ALEXSA items and cartoon illustrations. Their comprehension of questions, response options, and illustrations was evaluated by asking them what item wording and pictures meant, how they could be made better, and whether anything (and what) was wrong with them. In summer of 2009, 10 additional cognitive interviews (50% girls, 50% each African-American and Caucasian) were conducted with 9- to 12-year-olds from a South Atlantic state who experienced reading difficulties (per parent report) using different items. ALEXSA items used in focus groups and cognitive interviews systematically varied in length, complexity, and construct being measured. ALEXSA criterion validity includes correlations with SU, conduct disorder criteria and depression. Concurrent and predictive validity was demonstrated compared to parent and teacher reports (e.g., Child Behavior Checklist) in Caucasians, African-Americans, and Latinos; and across ages 5 to 16.

Four steps derived YRI scores. First, within the Chronic Stress sample (2.1.1), logistic regression identified candidate items that predicted lifetime alcohol or tobacco use by one year later for each age of participants (ages 9 – 13; item results are available from the first author). Second, candidate items were reduced to those which predicted SU at two or more ages (to eliminate chance associations; 47 remained, see Table 1). Third, YRI scores equaled the mean of item scores, ranging from 0 to 3. Fourth, three competing thresholds were tested (results presented herein): low-risk (corresponding to sensitivity=80%), moderate-risk (balanced sensitivity and specificity), and high-risk (corresponding to specificity=80%).

2.2.1 Validity Criteria

Validity tests compared the YRI to SU and conduct problems. SU (alcohol or tobacco) was queried using two ALEXSA items that were based on U.S. surveillance surveys. “Have you ever drunk alcohol, even just a sip?” (test-retest ICC=.83) was chosen over a “full drink” to match epidemiology studies of the sample age (Donovan et al., 2004). Although parents may allow a child to sip alcohol, in this age group such permissiveness appears to reflect propensity for subsequent problematic alcohol use (Komro et al., 2007; Velleman et al., 2005). The second item was “Have you ever used tobacco, even just to try it?” (test-retest ICC =.65). Conduct problems frequently predate or co-occur with SU/SUD. Thus, two or more conduct disorder behaviors (test-retest ICC=.81) served as a secondary criterion (which better forecasts chronic antisocial behavior than conduct disorder diagnosis; Robins and Price, 1991). SU and conduct problems are qualitatively different; nonetheless, their incidences each increase exponentially during adolescence (Andrews et al., 2003; Johnston et al., 2012) and many programs have efficacy for preventing or reducing both behaviors (Diamond and Lee, 2011; Dishion and Stormshak, 2007; Embry and Bigland, 2008).

In well-child check-up patients, validity of YRI scores was tested against previously validated tools (cf. Table 7). The Semi-Structured Assessment of the Genetics of Alcoholism was used to obtain parent reports of their child’s psychiatric diagnoses; its psychometric properties have been reported in multiple studies (Bucholz et al., 1994; Hesselbrock et al., 1999). The child-report Dominic-R uses a cartoon-based, computer format that resembles the YRI to assess the most common childhood psychiatric diagnoses; it has demonstrated validity in several psychometric studies (Arseneault et al., 2005; Scott et al., 2006; Valla et al, 2000). The YRI also was correlated with subscales used in the Family Check-Up prevention program as part of its ecologically-based case conceptualization (Dishion et al., 2007). These subscales included the Conduct Problems, Hyperactivity, and Prosocial Behavior subscales of the well-validated Strengths and Difficulties Questionnaire (Stone et al., 2010) and the Deviant Peer Associations, Parental Monitoring, and School Success subscales that were created and validated by the developers of the Family Check-Up (Dishion et al., 1991, 2003). Finally, YRI’s convergent validity was tested using the Transmissible Liability Index, a validated measure of the level of a child’s familial risk for lifetime substance use disorder that is about 80% heritable (Hicks et al., 2012; Kirisci et al., 2009; Ridenour et al., 2011; Vanyukov et al., 2009).

Table 7.

Convergent and Discriminant Validity Tests

Validity Criterion	Correlation with 23-item YRI
Conduct Disorder Symptom CountA	.43; .59
ODD Symptom CountA	.66; .46
SDQ Conduct Problems	.39
SDQ Hyperactivity	.47
FCU Deviant Peer Associations	.45
SDQ Prosocial BehaviorB	−.37
FCU School SuccessB	−.57
FCU Parental MonitoringB	−.31

^AKendall’s tau correlations were used based on non-normal distributions of psychiatric symptoms, which were from parent-reports using the Semi-Structured Assessment of the Genetics of Alcoholism; and child-reports using the Dominic-R, respectively.

^BDiscriminant validity; a negative correlation between the YRI and this “protective factor” is expected. FCU=Family Check-Up. All correlations p<.01.

SDQ=Strengths and Difficulties Questionnaire. FCU=Family Check-Up.

2.3 Analyses

To test how well YRI scores quantify SU risk, receiver operating characteristics analysis identified and compared the aforementioned three competing thresholds. Odds ratios (ORs) quantified change in SU risk per one-point increment in YRI (range=0–3). Intraclass correlation (ICC) quantified two-week, test-retest reliability. ICC corrects for chance agreement, requires consistent scores (not only consistent rank order) and is more conservative than correlations such as Pearson r (Cicchetti, 1994). Subgroup differences were tested using χ² or ANOVA. Pearson r and Kendalls’ tau tested convergent and discriminant validity.

3. RESULTS

Table 2 presents demographics. The samples are distinct and thus well-suited for replication testing. They differ (p<.001) on age, receiving free school meals, race/ethnicity and mean YRI scores. Participation rate also differs (p<.001) between the Chronic Stress (99.9%), Psychometric (57.8%), and Well-Child (58.1%) samples. Of the Psychometric subsamples, participation is lowest for regular education students (127 of 303, 41.9%), followed by remedial education students (145 of 205, 70.7%) and psychiatric inpatients (76 of 94, 80.9%); retest retention rates are 99.2%, 89.7% and 76.0%, respectively (all inpatient drop-outs were acute care patients who were released before retesting occurred). One-year retention rate for Chronic Stress participants is 81.1% (N=519 of 640).

3.1 Predictive Accuracy and Validity

Table 3 presents YRI detection of SU risk. Because items were selected based on their prediction of lifetime SU up to one year later, the sizable OR for predicting this outcome (7.44, CI=4.3–13.0) is not surprising. Replicating this result, large ORs also occur for concurrent SU and incidence of SU initiation over the subsequent year and in the replication samples. Area under the ROC also is strong for SU one year later (.72) and is larger in replication analyses. Moreover, results predicting conduct problems equal or improve upon SU.

Table 3. Youth Risk Index.

Associations with Substance Use, Conduct Problems

	Odds Ratio (CI)	AROC (CI)
Substance Use
Chronic Stress Sample
Lifetime concurrent	12.33 (7.1–21.5)	.77 (.03)
Lifetime by one year later	7.44 (4.3–13.0)	.72 (.03)
Initiated during subsequent year	2.79 (1.2–6.3)	.62 (.04)
Psychometric Sample
Concurrent	4.76 (2.8–8.1)	.74 (.03)
Well-Child Sample
Concurrent*	5.29 (1.1–26.6)	.75 (.07)
Conduct Problems
Chronic Stress Sample
Lifetime concurrent	33.42 (15.8–70.6)	.83 (.02)
Lifetime by one year later	7.33 (3.9–13.7)	.72 (.03)
Initiated during subsequent year	5.47 (2.4–12.6)	.68 (.04)
Psychometric Sample
Concurrent	4.76 (2.8–8.1)	.88 (.02)
Well-Child Sample
Concurrent*	30.55 (3.8–243.1)	.82 (.06)

Note: CI=95% confidence interval (an odds ratio meets p<.05 if the CI crosses over 1.0). AROC=area under the receiver operating characteristics curve.

^*Confidence intervals are large due to sample size.

YRI scores and SU prevalence in Psychometric sample subgroups are consistent with their a-priori expected differences (YRI F=27.7, p<.001 and SU χ²=6.2, p<.05). YRI means (SD) and SU prevalence for average-, above average-, and high-risk subsamples were, respectively: .63 (.42) and 24.6%, .91 (.53) and 31.9%, 1.18 (.51) and 41.3%.

3.2 Clinical Thresholds

Comparisons among the competing thresholds appear in Table 4. As expected, the low and high thresholds respectively provided the best sensitivity and specificity and even better discriminated rates of conduct problems. However, their respective specificity and sensitivity were poor. To capitalize on the strengths of each threshold and eliminate their weaknesses, a two-threshold classification was tested. Table 5 demonstrates that the two-threshold classification distinguishes levels of risk both in terms of SU and conduct problems.

Table 4.

Competing Youth Risk Index Thresholds for Detecting Propensity for Substance Use before High School

YRI Threshold*	Substance Use		Conduct Problems
	Sensitivity	Specificity	Sensitivity	Specificity
High-risk Threshold
Chronic Stress Sample
Lifetime concurrent	.60	.77	.64	.85
Lifetime by 1 year later	.52	.80*	.48	.80*
Initiated during subsequent year	.22	.87	.22	.87
Psychometric Sample
Lifetime concurrent	.40	.84	.30	.91
Moderate-risk Threshold
Chronic Stress Sample
Lifetime concurrent	.79	.57	.85	.65
Lifetime by 1 year later	.71	.60	.71	.62
Initiated during subsequent year	.57	.62	.63	.66
Psychometric Sample
Lifetime concurrent	.64	.72	.84	.80
Low-risk Threshold
Chronic Stress Sample
Lifetime concurrent	.90	.45	.91	.52
Lifetime by 1 year later	.80*	.45	.80*	.49
Initiated during subsequent year	.60	.58	.60	.58
Psychometric Sample
Lifetime concurrent	.77	.67	.89	.72

Note:

^*The low-risk threshold (YRI=.8049) corresponds to sensitivity=80% for SU, moderate-risk threshold (YRI=.9245) has balanced sensitivity and specificity, and the high-risk threshold (YRI=1.1797) corresponds to specificity=80% for SU. SU=initiation of alcohol or tobacco use.

^APositive predictive value. Results are not reported for the Well-Child Sample due to sample size.

Table 5.

Risk of Substance Use and Conduct Problems Among Three Strata of Risk, Based on Youth Risk Index Low and High Thresholds

	Odds Ratio per Risk Group Increment
Chronic Stress Sample
Lifetime SU concurrent	3.27 (2.5–4.3)
Lifetime SU by 1 year later	2.47 (1.9–3.3)
SU initiated during subsequent year	1.59 (1.1–2.3)
Lifetime conduct problems concurrent	4.87 (3.5–6.8)
Lifetime conduct problems by 1 year later	2.44 (1.8–3.3)
Conduct problems initiated during subsequent year	1.63 (1.0–2.5)
Psychometric Sample
Lifetime SU concurrent	2.76 (2.0–3.8)
Lifetime conduct problems concurrent	6.07 (4.1–9.0)
Well-Child Sample
Lifetime SU concurrent	2.55 (1.1–5.8)
Lifetime conduct problems concurrent	3.69 (1.6–8.6)

Note: All results p<.001, tested using χ². Parenthetical values present 95% confidence intervals (an odds ratio meets p<.05 if the CI fails to cross 1.0).

3.3 Reliability

YRI test-retest reliability (in the Psychometric Sample) is excellent (ICC=.85). YRI one-year stability (in the Chronic Stress Sample) is large (ICC=.60).

3.4 Screening Feasibility, Acceptability, and Validity

YRI screening was considered important, easily completed, confidential and needed for well-child check-ups by patients, parents and pediatric staff (Table 6). When asked how to improve the protocol, two pediatric staff cited concerns with potential glitches using the technology (the others offered no suggestions). The 23-item Brief YRI demonstrated good convergent and discriminant validity compared to psychiatric behavior problems (parent- and child-reports), transmissible risk for SUD (parent-reports), and Family CheckUp measures used for case conceptualization (parent-reports; Table 7).

Table 6.

Stakeholders Acceptability of YRI Screening

Characteristic of Screening Protocol	Parents	Patients	Pediatric Staff
Happy with/did not mind screening	100.0%	91.2%	100.0%
Pediatrician helping kids behave safer is important	100.0%	94.8%	100.0%
Had no or little trouble completing screening	100.0%	98.2%	- -
Child had no or little trouble completing screening	100.0%	- -	- -
Easy or not hard to answer honestly	98.3%	93.0%	- -
Concerned about confidentiality	0.0%	7.0%	- -
Gave a wrong answer on purpose	1.7%	5.3%	- -
Preferred paper form over computer	0.0%	5.3%	- -
Preferred reception room over exam room	5.0%	5.3%	12.5%
Preferred doctor administer screening over nurse	3.4%	14.0%	- -
Happy if pediatrician screens patients	93.2%	- -	100.0%
If child was ‘at risk’ would seek help (probably)	83.3% (10.0%)	- -	- -
If child was ‘at risk’ AND doctor knew who could help, would seek help (probably)	83.3% (13.3%)	- -	- -
Patient ‘flow’ was impeded	- -	- -	0.0%
Screening provided important clinical information	- -	- -	75.0%

4. DISCUSSION

SU/SUD prevention occurs primarily outside of healthcare in spite of the impacts of SU/SUD on chronic and costly illnesses (Kelleher and Stevens, 2009; McLellan et al., 2000). Toward meeting the pediatrician Bright Futures standards and progressing toward expanding SU/SUD screening and prevention services in medicine, this study demonstrated the YRI’s psychometrics for screening older children at risk for SU before high school (who are thus at risk for SUD). Moreover, YRI screening was feasible, acceptable to stakeholders, and provided clinical utility (e.g., by detecting patients with risk characteristics that are targeted by the Family Check-Up prevention program).

Compared to using one threshold, a two-threshold structure better classifies level of propensity for SU. Rather than “at risk” vs. “not at risk” categories, reporting to parents that a child has low, moderate, or high risk can be used by physicians to more effectively communicate to parents the continuum of SU propensity, uncertainty in prediction, dynamic nature of SU/SUD propensity, and benefits of preventive intervention and monitoring. Two thresholds also offer flexibility in clinical decision-making (Kraemer et al., 1999). For example, considerations in patient referral decisions include local prevalence of SU problems and available intervention resources. Where SU is widespread, the likelihood of exposure to and opportunity for SU may justify referring youth in the moderate risk range. Where SU prevalence is low and intervention resources are scant, the higher threshold might be used. A third consideration is whether false negatives or false positives are worse (e.g., consequences of potential SUD vs. potential for labeling), which also varies among locations.

4.1 Potential of Pediatrician-initiated SUD Prevention

Compared to schools, the pediatric medical home provides important advantages for SU/SUD prevention in terms of comprehensive health and development promotion, screening, and sustainable dissemination of prevention services (DHHS, 2010; Elliott and Mihalic, 2004; Greenberg, 2004). Specific advantages that pediatricians offer include parent and patient trust in their long-term relationship, confidentiality and expertise; their role in health screening and surveillance to promote healthy development; and their role in prevention counseling and guiding parents about the challenges of each developmental stage (Levy and Kokotailo, 2011; Moseley et al., 2011). Certain evidence suggests that screening within the medical home as part of well-child care encourages families to engage in prevention. For instance, physician screening of behaviors such as exercise and diet and referral for brief treatment is efficacious for improving healthy behaviors (Olson et al., 2009).

SU/SUD prevention screening can initiate brief family intervention using motivational interviewing to address “moderate” or “high” YRI scores, thereby disrupting developmental pathways to SU/SUD. An ongoing study funded by the U.S. National Institute on Drug Abuse (Ridenour, Reynolds and Shaw, Co-PIs) is testing this approach using a screening, brief intervention, and referral to treatment protocol. In this study, treatment consists of the Family Check-Up (Dishion and Stormshak, 2007) and the 23-item YRI reported herein serves as the screening tool. Work is in progress to develop a computer adaptive testing version to permit full YRI administration within 7½ minutes (Kirisci et al., 2012).

Well-child check-up screening could amplify demand for intervention tools for community professionals, including psychologists and family counselors. Evidence-based programs for late childhood and middle school ages span family-based individualized interventions, such as the Family Check-Up, to universal interventions targeting specific risks (e.g., disinhibition; Diamond and Lee, 2011; Dishion and Stormshak, 2007; Embry and Bigland, 2008). Important progress is ongoing for screening of adolescent alcohol use disorder using the CRAFFT, thereby facilitating early treatment (Knight et al., 2002), although efficacy evidence for this approach to treat SUD is mixed (Roy-Byrne et al., 2014; Saitz et al., 2014). In response to underage drinking rates, alcoholism, and their consequences, the National Institute on Alcohol Abuse and Alcoholism and American Academy of Pediatrics recently collaborated to fund studies of screening and brief intervention protocols targeting underage alcohol abuse (NIAAA, 2011). The YRI complements these strategies by detecting propensity for SUD years before onset of disorder to yield long-term efforts to deflect a youth’s development from SUD (Ridenour et al., 2012).

4.2 Limitations

This study provided considerable strengths; nonetheless, three limitations should be noted. First, 18.9% of Chronic Stress participants were lost to follow-up. Evidence from two sources suggests attrition bias did not occur: (a) results for one-year outcomes were replicated using outcomes prior to the YRI in two samples and (b) previous extensive analyses of the sample’s demographics and risk factors demonstrated no differences between longitudinal participants and drop-outs (Ridenour et al., 2012). Second, research specifically on larger and more representative samples of well-child patients could refine YRI thresholds specifically for this population, including distinct thresholds for genders, races/ethnicities or regions (e.g., marijuana use is less prevalent in the Northeast than elsewhere in the U.S.). Third, many high-risk youth do not initiate SU before high school (e.g., due to unavailability of substances); longer-term follow-up may provide better outcomes data to refine YRI thresholds. Likewise, a study with longer-term follow-up would provide better replication testing regarding the ability of the YRI to predict subsequent SU and conduct problems, especially in light of the sharp rise in incidence of initiating SU and conduct problems during middle adolescence.

4.3 Next Steps

Study results support several next steps. As mentioned, a computer adaptive testing version of the full YRI would be optimal. Also needed is coupling the YRI with one or more SU/SUD prevention programs (work that is ongoing, as described earlier). Testing for separate thresholds for subpopulations (e.g., ages, genders, geographic regions) may improve prediction accuracy.

The aforementioned ongoing study that couples YRI screening with the Family Check-Up is testing a protocol for pediatrician nurses to (a) administer the YRI while patients wait for the physician (e.g., while taking vitals), (b) immediately score the results, and (c) provide results to physicians to make referrals or other recommendations. For that study, at-risk scores qualify a parent-child dyad to participate in a clinical trial of the Family Check-Up prevention program. A significant focus of that study is to delineate a screening protocol, incorporating input from the participating physicians and nurses, to recommend to future users of the YRI. Until then, and used within the care-as-usual comparison condition, families may seek (and physicians can recommend) intervention by family therapists or other mental health providers.

To summarize, results support using the YRI to screen for risk of SU before high school (and subsequent SUD). Such early identification allows years of preventive intervention. Given that the YRI overcomes historical barriers to pediatrician use of behavioral screening tools, this investigation represents a critical step toward an evidence-based protocol to detect propensity of SU prior to high school. Hopefully, this individualized prevention strategy can complement parallel approaches to screen for and treat SUD, which at best has generated mixed results and weak efficacy (Roy-Byrne et al., 2014; Saitz et al., 2014). One potential beneficiary of the YRI is pediatricians because it fulfills important policy edicts. When implemented, this strategy could drive referrals, resource development and revenue for pediatricians, as well as health care policies to disrupt development of adolescent SU/SUD.

• Proposed is a tool for screening propensity of risky behavior in youth ages 9–13

• Youth Risk Index (YRI) items are non-sensitive and youth with illiteracy can take it

• YRI is computerized, takes about 7.5 minutes, and is designed for well-child visits

• YRI psychometrics were replicated in three distinct samples, using 3 levels of risk

• Pediatricians and nurses, their patients and parents found YRI screening acceptable