Psychometrics and Cross-Cultural Comparisons of the Illustration-Based Assessment of Liability and Exposure to Substance Use and Antisocial Behavior© for Children

Abstract

Elementary school-age child report instruments that do not require reading or interviews are lacking. In four samples, psychometric estimates for 5- to 9-year-olds were obtained for the Assessment of Liability and Exposure to Substance use and Antisocial behavior© (ALEXSA©), a child-report instrument that can be completed even by illiterate children. Invariance between minority groups vs Caucasians also was tested. Samples were: high-risk, low SES African-Americans (n=337), youth of varied ethnicities experiencing chronic stress (n=209), Mexican migrants in a reading remediation program (n=45), and U.S. twins (42 pairs) who were nearly all Caucasian. Validity criteria consisted of child-, parent-, teacher- and research evaluator-ratings on previously developed research and clinical instruments. Replicating results with older samples, ALEXSA factors had adequate or better reliabilities and demonstrated validity in all four studies. Ethnic invariance was found except for differences that were expected due to migrant's after-school program. In sum, psychometrics of the ALEXSA were supported for 5- to 9-year-olds of varied races/ethnicities, risk levels and academic skills.

Key objectives of behavioral problem prevention programs are detection and curtailing of risk, optimally during childhood [1]. These objectives require measuring overall risk and specific sources of risk in children and can be especially useful in family programs for selective/indicated prevention [2]. The best informants for many early manifestations of behavior problems, risk factors, and intervention outcomes are the children [3, 4]. However, obstructions to reliable, valid and pragmatic child report tools include ubiquitous features of extant assessments such as reading requirements, potential biases in interviews due to social desirability or child embarrassment and boredom with these tasks. To fill this need, the present four studies of 5- to 9-year-olds obtained psychometrics of the Assessment of Liability and EXposure to Substance use and Antisocial behavior© (ALEXSA©) [5], a computerized child report assessment of early manifestations of, and risk factors for, behavior problems.

The ALEXSA utilizes assessment technologies, such as illustrations and the audio computer-assisted self-interview (ACASI), to enhance youth comprehension of items compared to written formats. Carefully choreographed, illustrated ACASIs permit even illiterate youth to complete the ALEXSA, which is critical for child reports. About 38% of U.S. 4th graders (51% in urban regions) read below basic skill levels [6]. Poor reading ability forecasts behavior problems [6, 7], rendering data from poor readers critical for surveillance, need assessments and testing program outcomes. Adolescents perceive that computer surveys provide greater confidentiality than paper mediums [8], one reason for the generally greater disclosure of risky and problem behaviors on computer surveys [9]. Computer assessments can collect, store and score data in one step, thereby reducing human effort, expense and potential for errors compared to paper instruments.

Previously, ALEXSA data from 9- to 12-year-old students demonstrated good to excellent test-retest reliability and construct validity in all nine factor scores, 34 of 39 subscales, and 11 of 18 items from the alcohol or tobacco risk indexes [10, 11]. Moreover, as hypothesized, only those subscales that putatively measure characteristics related to psychiatric illness and the Family Conflict subscale discriminated psychiatric inpatients from school students [12]. In a prospective study of 8- to 16-year olds with chronic stress, 20 of 21 ALEXSA subscales represented age-related increasing levels of risk for behavior problems on average [13].

In addition to improving developmental appropriateness, instruments to measure risk for behavior problems in U.S. children need to be validated in culturally diverse populations. To illustrate, Hispanic children are the largest and fastest-growing subpopulation in the U.S. [14] and are at increased risk for substance abuse (e.g., they initiate alcohol use at younger ages on average than other children) [15]. They also have lower reading and math achievement on average than Whites and Asian Americans [16].

This investigation tested ALEXSA scores in four distinct samples younger than 10 years old. One challenge in psychometric evaluation of the ALEXSA is the lack of alternative, comparable measures for testing validity. The first study tested ALEXSA subscales for which validity criteria readily exist (Conduct Disorder Criteria and Depression) and are traditionally used as outcomes. The remaining studies test psychometrics of ALEXSA factor scores against these subscales and other criteria with which ALEXSA factors putatively are/are not associated. Invariance between races/ethnicities also was evaluated. Specific hypotheses appear in the context of each study. Tests were conducted under rigorous conditions including the restricted range of high-risk youth, cognitive immaturity, settings with multisensory distracters, use of adult's reports as validity criteria, and English-as-Second-Language students with poor reading skills.

METHODS

For all studies, IRB approvals were obtained from collaborating institutions prior to subject recruitment. Parent consent and child assent were obtained prior to data collection. The ALEXSA is described next, as it is the focus of all four studies.

Instrumentation

Assessment of Liability and EXposure to Antisocial Behavior and Substance Use© (ALEXSA©)

The ALEXSA system was developed to quantify specific predictors and overall level of risk for antisocial behavior and substance use prior to high school [10]. Its theoretical bases are the multifactorial, liability-threshold and ecological theories, each of which conceptualizes an individual's overall risk as an aggregate of manifold specific risk factors. Each risk factor is measured by a subscale and has its own theoretical orientation that specifies its link(s) to behavior problems [10].

Derivation of subscales was based on extensive literature review, their putative links to existing preventive interventions, developmental appropriateness, and pragmatic utility such as being able to measure a construct using few items so that many subscales can be administered with minimal respondent burden. The ALEXSA includes 350 items organized into 39 subscales (although additional subscales have been created). Each subscale and factor is being developed as a stand-alone measure to permit an ALEXSA user to administer only those subscales or factors in which s(he) is most interested.

Detailed descriptions of the ALEXSA, including questions, appear elsewhere [10, 11, 13]. Response options for every item include “Don't Know” and “Refuse to Answer,” which were scored as missing because research on scoring these options for children is sparse [17]. Subscale names describe the construct they are designed to measure as well as its directionality; high scores reflect a high level of the construct.

ALEXSA questions and response options are portrayed by cartoon sibling protagonists, Alexis and Alex (names were based on international usage and correspondence to the ALEXSA acronym). Girls view Alexis response options; boys view Alex response options. Protagonist features were designed to be able to occur in any race (dark hair, dark eyes, tan complexion), but most closely resemble Hispanic or Mediterranean ethnicities.

ALEXSA Factor Scores

In a previous study of 9- to 12-year-old students in regular and remedial education classes, factor analyses derived nine factors from ALEXSA subscales [10]. They are described, below, with internal consistencies (α) and test-retest reliabilities (intraclass correlations or ICC) from the students. ICC of .39 or less = low, .40 to .59 = fair, .60 to .74 = good and excellent = .74 or greater [18]. Certain factors theoretically measure characteristics that are primarily internal (e.g., environmental influences on them notwithstanding). Disinhibition (α=.92, ICC=.81) putatively measures emotional and behavioral volatility and poor regulation of impulses [19] using subscales of Irritability, Anger Coping, Distractibility and Impulsivity. Sensation Seeking (α=.84, ICC=.82) is based on Zuckerman sensation seeking scales that correlate with behavior problems in youth [20], including Thrill Seeking, Social Disinhibition and Gambling. Self Management (α=.82, ICC=.76) consists of learned skills that reduce the probability of and consequences from mistakes or disinhibition; its subscales are Planning & Concentration and Problem Solving [19].

Other factors in theory measure predictors of behavior problems that are primarily external to youth (e.g., perceptual differences notwithstanding). Family Discord (α=.74, ICC=.71) includes subscales to measure harmful behaviors and interaction patterns among family members: Family Conflict [21], Family Behavior Problems (substance abuse, legal problems) and Parental Permissiveness. Parent Fortification (α=.90, ICC=.74) putatively measures caretaker nurturance, knowledge of a youth's life outside of the family and caring affect and behavior toward the youth, including Parental Monitoring, Attachment and Nurturance [22]. Social Contagion (α=.91, ICC=.80) measures are designed to quantify characteristics related to social contexts, primarily with peers [23]: Friends Conduct Disorder Criteria, Violence Exposure and Peer Pressure Susceptibility (although Peer Pressure Susceptibility loaded on the Social Contagion factor, it is considered an internal measure). Social Support (α=.67, ICC=.80) gauges help available from and emotional attachment to others using Social Support: Adults, Social Support: Youth and Peer Attachment subscales [24]. School Protection (α=.72, ICC=.74) subscales measure confidence in, liking of and desire for good academic performance using Academic Competency, School Atmosphere: Adults, School Commitment and School Bonding subscales [25]. Neighborhood Risks (α=.81, ICC=.76) queries environmental factors outside of school and family functioning with Neighborhood Atmosphere, School Atmosphere: Students, Last Year Stressors, and Gang Exposure.

Two additional ALEXSA subscales were investigated. Conduct Disorder Criteria queries 12 DSM-IV criteria using dichotomous items; α=.78 and ICC=.69. Content of Depression items is loosely based on the well-established self-report measures Child Depression Inventory and Center for Epidemiological Studies – Depression [26,27] using four-point Likert items. Its α=.87 and ICC=.60.

Analyses

Several statistical techniques were utilized in multiple studies; techniques used only for one study are described separately in that section. Reliability estimates were Cronbach's and intraclass correlation coefficient (ICC). α is ubiquitous, because of convenience, in spite of its well-documented shortcomings [28,29]. Several aspects of ALEXSA scores that are advantageous for child reports (e.g., using as few items as possible which measure distinct aspects of a construct) are penalized by α. Hence, α=.70 was considered adequate. ICC test-retest reliability is more conservative than Pearson correlation and estimates the proportion of variance in scores due to between-person differences [18]. Concurrent validities and twin pair correlations were tested using Pearson correlations. Race/ethnic invariance for means and variances was tested using ANOVA and homogeneity of variance.

STUDY 1: VALIDITY OF ALEXSA OUTCOMES SUBSCALES IN A HOMOGENOUS HIGH-RISK SAMPLE

Among ALEXSA subscales, Conduct Disorder Criteria and Depression are most similar to other instruments that could serve as validity criteria. Thus, the first study tested their convergent, discriminant and predictive validities. Hypotheses were that (1) Conduct Disorder Criteria correlates significantly with measures of externalizing behavior, (2) Depression correlates significantly with measures of internalizing problems, and (3) each correlates to a lesser degree with the dissimilar validity criteria (because internalizing and externalizing problems correlate). ALEXSA scores were hypothesized to correlate less with adult ratings than other child reports [3].

Sample

Data were from a case-control study of children prenatally exposed to either (a) addictive drugs including cocaine (cases) or (b) addictive drugs excluding cocaine (controls) [30,31]. This homogenous high-risk sample provided a rigorous test of validity due to its range restriction (e.g., correlations ought to be larger in a community sample). Moreover, ALEXSA subscales were administered at the end of an assessment protocol lasting about three hours.

Participants were born between 1994 and 1996 in a large urban Midwestern teaching hospital, recruited through their biological mothers. Children were primarily African-American (80%) and received public assistance at birth (98%); 53% were female. Maternal eligibility criteria were: received urine toxicology screens because they did not receive prenatal care, exhibiting behavior suggesting intoxication, admitted to previous involvement with Child Protective Services or admitting drug use to hospital or research staff. Exclusion criteria were (a) mothers having lifetime schizophrenia, bipolar disorder, mental retardation, age <19 or HIV or (b) children with fetal alcohol syndrome or other serious birth defects. In utero cocaine exposure (CE) was found for 213 participants (197 were not exposed (NCE)). A total of 11 infants (8 CE, 3 NCE) died and 7.5% of subjects were lost to follow-up or withdrew from the study (10 CE, 5 NCE). Age 9 and 10 assessments were completed by 93% and 92%, respectively, of the original sample. Data from 337 participants were available for analyses (N=314 for one-year predictions).

Validity Criteria

Previously developed measures of child externalizing and internalizing problems served as validity criteria. The computer, illustration-based Dominic-R (African-American version) provides child self-reports of DSM-IV symptoms (scored 0, 1 or 2 indicating “no problem,” “maybe a problem,” or “likely a problem.”); Dominic-R scores correlated with clinician diagnoses (kappas ranged from .64-.88) [32]. Subscale test-retest ICCs ranged from .71-.81; α=.89 for Internalizing Problems and Externalizing Problems. Primary caregivers rated children's behavior using the Child Behavior Checklist (CBCL) [33,34] Externalizing and Internalizing T-scores. Teachers and research evaluators completed the Connors’ Teacher Rating Scale [35] Conduct Problems, Hyperactivity Index and Inattention-Passive subscales. At age 10, CBCL and research evaluator Connors’ ratings scales were collected, providing one-year predictive validity criteria.

Results

Table 1 presents sample demographics. Conduct Disorder Criteria α=.72; Depression α=.83. Results were consistent with hypotheses (Table 2). Correlations between Dominic-R and adult's ratings also appear for comparison to ALEXSA results. As expected, correlations are greatest between the two child report instruments. However, whereas Dominic-R subscales r=.67 (p<.001), the ALEXSA Conduct Disorder Criteria and Depression subscales evidence greater discriminant validity (r=.21, p<.001). Compared to Dominic-R results, similar correlations were observed between ALEXSA subscales and adult ratings. ALEXSA results also resembled or were better than correlations between adults ratings (r=.02 between CBCL Externalizing and examiner Conduct Problems, .36 between CBCL Externalizing and teacher Conduct Problems and .14 between examiner and teacher Conduct Problems).

Table 1.

Demographics and Descriptive Statistics of Samples

	Subsample 1	Subsample 2	Subsample 3	Subsample 4; Twins A	Subsample 4; Twins B
Sample Size	337	209	45	42	42
Gender (% boys)	48.1%	39.8%	66.7%	50.0%	40.5%
Age Range	9	8 to 9	5 to 9	6 to 9	6 to 9
Age Mean (SD)	9.00	8.61 (.49)	8.47 (1.20)	7.69 (1.08)	7.71 (1.19)
Low Economic StatusA	98.0%	63.1%	42.9%	N/A	N/A
Racial/Ethnic Composition	80.0% African-American	55.9% Caucasian	100% Hispanic	93.3% Caucasian	93.3% Caucasian
Sensation Seeking	N/A	1.2 (.65)	1.1 (.76)	1.1 (.60)	1.2 (.63)
Social Contagion	N/A	.6 (.80)	.6 (1.08)	.6 (.80)	.7 (.89)
Disinhibition	N/A	1.2 (.56)	1.0 (.51)	1.1 (.40)	1.1 (.42)
Self Management	N/A	1.8 (.52)	2.0 (.75)	2.0 (.45)	1.9 (.63)
Parent Fortification	N/A	2.1 (.87)*	2.1 (.71)	2.3 (.61)	2.3 (.47)
Social Support	N/A	N/A	2.1 (.83)	2.0 (.52)	1.8 (.53)
School Protection	N/A	2.2 (.66)	2.8 (.57)	2.6 (.48)	2.5 (.58)

Note:

^ABased on reports of receiving free lunch at school. Socioeconomic data were not available as part of the twins protocol. Parenthetical values are standard deviations. N/A = not assessed.

^*based on a Parent Fortification subscale, Parental Monitoring (which has a factor loading of .78)

Table 2.

Convergent, Discriminant and Predictive Validities of the ALEXSA Conduct Disorder and Depression Subscales in High-Risk African-Americans at Age 9

		ALEXSAA		Dominic-R
		Conduct Disorder	Depression	Externalizing	Internalizing
Dominic-R	Externalizing	.44**	.39**	-	.67**
	Internalizing	.19**	.49**	.67**	-
CBCL (Parent Ratings)	Externalizing	.09	.14*	.13	.08
	Externalizing (age 10)	.16*	.12	.18*	.11
	Internalizing	.01	.14*	.07	.08
	Internalizing (age 10)	.04	.13	.13	.14
Conners Examiner Ratings	Conduct Problems	.18**	.20**	.16*	.13*
	Conduct Problems (age 10)	.25**	.12	.19**	.08
	Hyperactivity Index	.32**	.21**	.24**	.19**
	Hyperactivity Index (age 10)	.23**	.12	.18**	.05
	Inattention-Passive	.30**	.19**	.14*	.17**
	Inattention-Passive (age 10)	.18**	.10	.14*	.02
Conners Teacher Ratings	Conduct Problems	.12	.09	.16*	.15
	Hyperactivity Index	.28**	.13	.25**	.15
	Inattention-Passive	.36**	.16*	.23**	.16*

Note: N=337.

^**p<.001.

^*p<.01.

Dominic-R scores are from the Problem subscale scores. CBCL= Child Behavior Checklist, completed by participant mothers.

^AConduct Disorder Criteria and Depression subscales correlated .21**. For comparison, r=.02 between CBCL Externalizing and examiner Conduct Problems, .36 between CBCL Externalizing and teacher Conduct Problems and .14 between examiner and teacher Conduct Problems.

STUDY 2: VALIDITY OF FACTOR SCORES IN YOUTH WITH CHRONIC STRESS

For testing ALEXSA validity, an ideal sample would be at elevated risk for psychopathology yet heterogeneous because ALEXSA scores putatively measure levels ranging from healthy and beneficial to harmful [10,11,13]. The ALEXSA's aforementioned underlying theories suggest that different risk factors occur among youth whereas a homogeneous sample might exhibit only certain risk factors. One population fitting these parameters is youth experiencing chronic stress.

Chronic stress is well-documented to be associated with externalizing and internalizing problems, substance abuse and other psychiatric disorders [36]. Stressors tax adaptive capacity; chronic stress putatively delays development of cognitive and emotional abilities including coping, problem-solving and decision-making [37]. Mediators between chronic stress and adolescent psychopathology include competencies and coping skills, family and peer environments, social support, positive/beneficial activities, depressogenic cognitions and exposure to violence [36], which overlap considerably with ALEXSA scales. Evidence is lacking on whether mediators of stress and adolescent psychopathology also fill a mediation role in children.

ALEXSA scores in 8- to 9-year-olds with chronic stress were hypothesized to correlate with Conduct Disorder Criteria concurrently, one year later (predictive validity), and to a greater degree than with Depression (discrimnant validity). These correlations with Conduct Disorder Criteria and Depression also explored potential mediation of ALEXSA risk factors between these outcomes and chronic stress in children [38].

As described earlier, tests for differences between racial/ethnic groups are needed to understand if such differences ought to be accounted for when interpreting ALEXSA scores. Accordingly, another hypothesis was that no mean or variance differences occur between Caucasians vs minority groups.

Sample

The sample of 209 8- to 9-year-olds attended a summer camp for youth experiencing chronic stress between 2004 and 2009. IRB approval for analysis of de-identified data precluded the research team from inspecting /coding participant's stress (camp records of participant stressors were applicant responses to open-ended questions). Even so, camp funding was contingent upon attendees experiencing significant long-term stressors, so several steps were taken by camp staff to meet this requirement. An adult applied for camp attendance on the youth's behalf and described source(s), duration and negative impact on functioning of the stressor(s). Youths’ school personnel (e.g., teacher, psychologist) were contacted to corroborate the application and obtain additional information (e.g., academic performance, social functioning). Exclusion criteria were physical and mental handicaps, history of arson and physical aggression. Annually, about 60% of applications result in a youth attending camp. About 70% of campers from one year attend camp the next year; previous analyses demonstrate that returning campers do not differ from other campers in terms of demographics, behavior problems or risk factors [13].

Sources of chronic stress were categorized by program staff as: low family income (i.e., poverty), serious family problems (e.g., an incarcerated or drug-addicted parent), social problems (e.g., severe peer rejection), chronic poor academic performance or emotional problems (e.g., a mood disorder, although diagnoses were not made as part of this study). To illustrate the scoring of the camp category of social problems, a 1 to 7 scale was based on research literature [39] with the worst score being “few, if any, friendships and child is actively disliked by peer group.” Participants were from urban, suburban and rural settings (although not measured per se) of a Northeastern U.S. state. All campers that qualified for Study 2 participated (participation rate=100%).

ALEXSA

Seven of the previously described nine ALEXSA factors were represented in the subscales selected by camp staff for their program evaluation. These factors were Disinhibition, Sensation Seeking, Self Management, Parent Fortification (using the Parental Monitoring subscale which loads .78 on the factor), Family Discord (using the Family Conflict subscale which loads .72 on the factor), Social Contagion and School Protection. Only 35 minutes was allotted during camp for ALEXSA administration. To ensure that all subscales could be completed by all participants, some items of certain subscales were omitted (detailed in [13]).

Results

Demographics appear in Table 1. ALEXSA factors α appear in Table 3; α=.70 for Family Conflict, .73 for Parental Monitoring, .77 for Conduct Disorder Criteria, and .84 for Depression.

Table 3.

Reliabilities of ALEXSA Factors

	Subscample 2: Chronic Stressα	Subsample 3: Migrantsα	Subsample 3: MigrantsT1-T2	Subsample 4: Twins Aα	Subsample 4: Twins Bα
Disinhibition	.88	.80 (18)	.71 (19)	.80 (28)	.74 (24)
Sensation Seeking	.67	.83 (19)	.86 (18)	.80 (28)	.74 (24)
Social Contagion	.82	.78 (31)	.84 (34)	.87 (27)	.71 (31)
Self Management	.66	.80 (18)	.77 (19)	.90 (30)	.58 (31)
Parent Fortification	-	.83 (15)	.50 (38)	.74 (28)	.90 (26)
Social Support	-	.77 (17)	.82 (17)	.73 (29)	.73 (29)
School Protection	.71	.78 (23)	.73 (22)	.68 (28)	.75 (27)

Note: α = Cronbach's alpha. T1-T2 = test-retest reliability using intraclass correlation coefficient. Chronic stress sample N=209. Parenthetical values present N for smaller samples due to α requiring that no data are missing.

Overall, correlations between ALEXSA scales and Conduct Disorder Criteria or Depression are consistent with hypotheses, supporting their concurrent, discriminant and predictive validity (Table 4). Concurrent correlations with Conduct Disorder Criteria were all significant in 9-year-olds (except Parental Monitoring) and nearly all were sizable in 8-year-olds (small sample size precluded many from reaching statistical significance). On average, one-year predictive correlations were slightly lower than concurrent correlations. Correlations with Depression equaled or were lower than results with Conduct Disorder Criteria, except for the family variables, consistent with past evidence regarding familial factors and depression [38].

Table 4.

Convergent and Predictive Validity of ALEXSA Factors in 8- and 9-year-olds Experiencing Chronic Stress

	Age	Conduct Disorder Criteria			Depression
		Age 8	Age 9	Age 10	Age 8	Age 9	Age 10
Disinhibition	8	.76**(29)	.78**(21)	-	.14 (30)	.22 (29)	-
	9	-	.24* (59)	.20 (57)	-	.27* (69)	.03 (64)
Sensation Seeking	8	.36 (20)	.54 (13)	-	.13 (21)	.09 (21)	-
	9	-	.43**(50)	.36*(48)	-	.20 (59)	.15 (55)
Self Management	8	-.34 (29)	-.25 (21)	-	-.13 (30)	.14 (29)	-
	9	-	-.28*(59)	-.14 (57)	-	-.19 (69)	.04 (64)
Family Conflict	8	.41 (23)	.41 (19)	-	.30 (30)	.49* (19)	-
	9	-	.42**(55)	.07 (51)	-	.32* (63)	.31* (58)
Parental Monitoring	8	-.31 (28)	-.17 (20)	-	-.07 (28)	-.05 (27)	-
	9	-	-.15 (58)	.04 (55)	-	-.31*(67)	-.04 (62)
Social Contagion	8	.40* (29)	.25 (21)	-	-.24 (30)	.27 (29)	-
	9	-	.34* (59)	.08 (57)	-	.20 (69)	.33* (64)
School Protection	8	-.08 (29)	.02 (21)	-	.01 (30)	.09 (29)	-
	9	-	-.29* (59)	-.23 (57)	-	-.29* (69)	-.05 (64)

Note: Parenthetical terms present the “n” for the analysis.

^**p>.001.

^*p>.01.

Table 5 presents tests of racial/ethnic equivalences between Caucasians and minority groups. As hypothesized, no statistical differences occur in terms of means or variances, with one exception. Minority children have greater variance in Conduct Disorder Criteria than their Caucasian peers.

Table 5.

Equivalencies between Caucasians and Minorities in 8- to 9-year-olds with Chronic Stress

	Means		Standard Deviations
	Caucasians	Minorities	Caucasians	Minorities
Conduct Disorder Criteria	.8	1.2	1.05*	1.85*
Depression	1.1	1.1	.64	.62
Disinhibition	1.2	1.2	.52	.59
Sensation Seeking	1.1	1.2	.62	.65
Self Management	1.8	1.8	.46	.56
Family Conflict	.2	.3	.27	.29
Parental Monitoring	2.0	2.1	.89	.86
Social Contagion	.6	.6	.82	.80
School Protection	2.1	2.2	.55	.65

STUDY 3: VALIDITY IN MEXICAN MIGRANTS

As mentioned, other than Depression and Conduct Disorder Criteria subscales, few validity criteria are available for ALEXSA scales. One exception is the Social Health Profile (SHP) [40], a teacher rating scale of student's social competence and behavior in school contexts. SHP has been used primarily for testing the efficacy of prevention programs. Several ALEXSA subscales (listed in Table 6) putatively measure factors associated with social or behavioral functioning in school. Thus, although the SHP does not provide a validity test in the traditional sense that it measures the same characteristics as ALEXSA subscales (cf. validity criteria of Study 1), the two sets of scores are expected to correlate. This study also permitted the elucidation of 1.5-week, test-retest reliabilities of ALEXSA factors, which were hypothesized to be good or better (ICC>.59).

Table 6.

Correlations Between ALEXSA Subscales and Social Health Profile Teacher Ratings

ALEXSA Subscale	Social Health Profile Factor
	Regular Teacher	Remedial Teacher
Tolerance of Deviance	-.12	.42*
School Atmosphere: Adults	-.40*	-.26
Distractibility	.32	.07
Planning and Concentration	-.22	-.33
Problem Solving	-.12	-.43*
Social Support	-.26	-.24
Irritability	.46*	.17
Peer Attachment	-.57*	-.65*
Peer Pressure Susceptibility	.49*	.85*
Friend's Conduct Disorder Criteria	.36*	.66*

Note: N=40.

^*p<.01.

Cell entries present Pearson correlations for ALEXSA subscales with ratings by regular teacher/remedial program teacher. Pearson correlation between teacher SHP factor scores = .63, p<.01. Greater SHP scores indicate problematic behavioral and social adjustment in school.

Sample

The ALEXSA format was designed in part to enhance comprehension of questions and items in children with reading deficits. Estimating psychometrics with English-as-Second-Language students tests whether the ALEXSA format accomplishes this goal. This test was even more stringent in the present sample because participants were from a reading remediation program. Also, participants were ages five to nine. Adding to these arduous testing circumstances, validity criteria consisted of adult ratings [3,4].

In the Mexican migrant subsample of the aforementioned study of remedial students, SHP was completed by remedial and regular education teachers as part of the migrants’ program evaluation three to five months after the ALEXSA. The 45 5- to 9-year-old Mexican migrants from rural Pennsylvania [41] completed the English version of the ALEXSA. Remunerations of $10 were paid for time and effort. Participation rate was 100% due to excellent rapport between program staff and students. The 9-year-olds data were previously analyzed within a larger sample of ages 9-12 in aggregate with several races/ethnicities [10]. Participants re-took the ALEXSA after a mean 11.75 days (SD = 8.88).

Validity Criterion

SHP scores served only as a convergent validity criterion (e.g., not for discriminant validity) because of the small subsample size and to avoid a prohibitive statistical correction for number of analyses (e.g., Bonferroni). SHP items composing the Cognitive Concentration and Authority Acceptance subscales are from the Teacher Observation of Child Adaptation, Revised [42]. SHP items composing the Prosocial and Emotional Regulation subscales are from the Social Competence Scale [43]. Greater scores indicate problems in functioning. Corrigan [40] reported that SHP subscales ranged from .80 to .96 for teacher ratings of normative (n=387) and high risk (n=155) second grade students and differentiated the two samples at p<.001.

Factor analyses of SHP subscales in the present sample indicated a single factor summarizes them. In remedial program teachers, one factor accounted for 83% of inter-subscale variance and subscale loadings were .81 or larger. In regular education teachers, one factor accounted for 77% of inter-subscale variance and subscale loadings were .78 or larger. Agreement between teacher ratings was .63 for the SHP factor, but lower for its subscales: .38 (p<.05) for Authority Acceptance, .59 (p<.01) for Cognitive Concentration, .08 (NS) for Emotional Regulation and .10 (NS) for Prosocial. Thus, only the SHP factor was used in validity tests. Reasons why teacher ratings might differ include class settings, student populations used for comparison to rate a participant, and the nature of relationship between the teacher and participant.

Results

Demographics appear in Table 1. Each ALEXSA factor is at least .78 (Table 3). ICCs are greater than .70 (good to excellent), except Parent Fortification = .50 (fair) (Table 3).

Table 6 presents correlations between ALEXSA and SHP. Overall, results were consistent with hypotheses with a few exceptions partly due to the small sample. Every subscale except three correlated significantly with one type of teacher's SHP and those three would reach significance in a slightly larger sample. Replication is needed in a larger sample using a research driven protocol (e.g., all measures being completed in closer time proximity than three months).

STUDY 4: TWIN STUDY OF HERITABILITY, ASSOCIATIONS WITH TRANSMISSIBLE RISK

In a previous prospective study of ALEXSA score trajectories from ages 8 to 16, subscales varied greatly in their amenability over time to naturally-occurring environmental influences [13]. One hypothesis generated from the results was that less amenable constructs were more affected by family transmission, perhaps heritable, sources. The approximate order of factors’ amenability (based subscale results), from low to high, was: Social Contagion (largely due to Peer Pressure Susceptibility); then Disinhibition, Sensation Seeking and Self-Monitoring; then Parent Fortification; and finally School Protection and Family Discord.

Presently, it was hypothesized that about the same rank order of ALEXSA factors would occur in terms of heritability and transmissible risk for drug abuse. Consistent with the trajectory analysis evidence, Disinhibition and Sensation Seeking have sizable heritability in older samples. Peer pressure susceptibility is internal and trait-like [44] and evocative genotype-environment correlations are putatively captured by Social Contagion's other subscales (Friends Conduct Disorder Criteria, Violence Exposure) [45]. Heritabilities and transmissible risk were hypothesized to be small or zero for Self Management (learned skills), Parent Fortification (putatively a measure of shared environment) and Social Support (external, non-shared environment factor).

This study also made possible ethnicity comparison because study 4 participants were in the same age range as the Study 3 sample. It was hypothesized that no mean or variance differences occur between native U.S. Caucasian twins and Mexican migrants during mid to late childhood.

Sample

The study setting tested a goal of the ALEXSA format: to hold and lengthen children's attention span compared to other multiple-response option instruments. A long-standing clinical recommendation is to administer psychological tests where few distractions occur. The WISC-III states, “...conduct the test in a quiet, adequately lit, well-ventilated room...no one other than you and the child should be in the room during testing” [46, p. 34]. To test whether the ALEXSA can overcome distractors, it was administered during a festival for twins. The open tent where ALEXSAs were completed was characterized by crowds of passer-bys, additional visual and auditory distracters from carnival rides and live bands; olfactory distracters occurred in the form of carnival food aromas. It was hypothesized that adequate or better reliability and validity estimates would be found in spite of the distractions.

Six- to nine-year-olds completed ALEXSAs with privacy screen overlays. Festival attendees were from the Midwest or Northeast U.S. and, reflecting twin prevalence, largely Caucasian. A parent from each family (95.2% were mothers), 18 monozygotic and 24 dizygotic twin pairs participated. All participants were remunerated $10 for time and effort. For analyses, the elder twin of each pair was randomly assigned to subsample A or B with the younger twin assigned to the alternate subsample. Analyzing subsamples separately prevented potential biases from the nonrandom nature of twin pairs and their birth order. How many 6- to 9-year-olds attended the fair was unknown; so, a recruitment rate could not be determined. Economic status was not assessed.

Validity Criteria

Parents completed the Transmissible Liability Index (TLI) which consists of items from well-established instruments that distinguish sons of fathers with a life time illegal drug use disorder from sons of fathers with no major psychiatric disorder [47-49]. TLI variance is almost entirely due to additive heritability (h²=.8) [49,50]. In a longitudinal study of drug abuse etiology in boys, TLI scores at about age 11 predicted substance use disorder by age 19 with 68% overall accuracy (p< .01) and a 70% annual hazard ratio.

Analyses

Because the small sample of twins precluded full behavior genetic analyses, monozygotic and dizygotic twin correlations were quantified using Fisher's r-to-z transformation and z-scores to estimate heritability (Z = 0.0 for nonsignificant twin pair r) [51]. For ethnic comparisons, the mean age difference between subsamples 3 and 4 (Table 1) was statistically controlled using linear regression. χ² tests in AMOS software compared the fit of two models to the data: one with variances constrained to be equal between ethnicities versus one with the variances freed.

Results

. Results are consistent with hypotheses. ALEXSA factors α range from .68 to .90, except that Self Management = .58 in subsample B. With one exception, the TLI only correlates with Sensation Seeking (r=.39, p<.01) and Disinhibition (r=.34, p<.01) in twin subsample A. The exception was a -.28 (p<.05) correlation between the TLI and School Protection (which may be indirectly effected by highly heritable traits such as IQ). Monozygotic twin correlations (p<.05) occur for Disinhibition (.33), Sensation Seeking (.45), Social Contagion (.48) and Parent Fortification (.65); one dizygotic twin correlation occurs, for Parent Fortification (.63). Thus, Z_diff estimates of heritability are 89% for Social Contagion, 53% for Disinhibition, 36% for Sensation Seeking, 4% for Parent Fortification (suggesting a large shared environment influence) and nil for the other factors because neither coefficient is statistically significant.

The only mean difference between Hispanics and Caucasians (Twins B) is on School Protection (p<0.05) (Table 1) (attributable to Hispanic's remedial program). Moreover, the only variances on which they differ statistically are Parent Fortification and Social Support (p<0.05).

DISCUSSION

Overall, results of these studies supported ALEXSA psychometrics as well as its cross-cultural equivalence. Study 1 demonstrated the validity of two clinical outcome subscales, Conduct Disorder Criteria and Depression. Studies 2, 3 and 4 documented the reliabilities of ALEXSA factor scores, each under unique arduous circumstances thereby providing psychometric properties under “worst-case scenarios” and replication. Study 2 documented concurrent and predictive validities of the ALEXSA factors compared to externalizing and internalizing. Study 3 results demonstrated the ALEXSA's ability to detect a range of specific factors related to social competence and behavior of children. Study 4 preliminarily demonstrated that the putative range of genetic and familial influences among the specific ALEXSA factors do occur. Collectively, these findings support using ALEXSA factors not only for research purposes but also for clinical assessment and program evaluation.

The fact that psychometric support for the ALEXSA was found in samples as young as 5 demonstrates its potential for extending researchers’ ability to efficiently garner self-report data from children, including poor readers and non-native minorities. Moreover, correlation coefficients involving ALEXSA scores were not consistently biased in positive or negative directions, suggesting that no systematic method variance (e.g., error variance due to computer format) occurred.

LIMITATIONS

The most important impediment was the sample sizes of Studies 3 and 4 which can be overcome in the future by using larger samples; preliminary results herein support such research. Another impediment is the limited validity criteria for comparing ALEXSA scores. It is hoped that the ALEXSA and its format will encourage additional developmentally apropos instruments for acquiring child-report data. Whereas a strength of the psychometric studies was that reliability and validity was demonstrated under stringent conditions, psychometric data also need to be collected under standard research protocols for complete evaluation of the ALEXSA as well as comparison to other instruments. Finally, research is needed to determine how to best score the “Don't Know” and “Refuse to Answer” response options.

STRENGTHS

Study limitations notwithstanding, a number of strengths were noted. As mentioned, ALEXSA reliability and validity was supported in spite of being conducted under stringent conditions. A variety of validity criteria and methods were used. Subsamples spanned a range of ethnicities, developmental levels and average risk for psychopathology. The replicated evidence found under such varied conditions further supports ALEXSA psychometrics and ethnic invariance. The innovative ALEXSA format also was a strength.

IMPLICATIONS FOR RESEARCH & PREVENTION

Overall, results suggest the ALEXSA could be useful for childhood screening and needs assessment related to behavior problems. To illustrate, subscales of Social Contagion forecast risky and antisocial behavior in adolescence [44]. If social contagion risk could be screened in childhood, subsequent selective intervention may improve an at-risk child's resistance to peer pressure [52]. The ALEXSA provides a tool for evaluating profiles of risk factors and it may be useful for determining if sets of risk factors cluster within persons so that intervention could address each factor in a cluster. Thus, by virtue of the range of risk factors that are measured, the ALEXSA could be useful for tailored interventions designed around a child's or family's profile of risk factors [2].

Results demonstrate the feasibility of engaging poor readers (who are at heightened risk for behavior problems) as well as racially/ethnically diverse children to complete a computer assessment of risk factors. The ALEXSA could provide a developmentally- and culturally-sensitive screener, needs assessment and outcomes measure under many circumstances. To illustrate, the only difference between subsamples 3 and 4 reflected Hispanic's exposure to intervention, suggesting that ALEXSA scales may detect prevention program efficacies as well as impacts on specific subpopulations [53].

To summarize, this study was the first to document psychometric properties of ALEXSA factors in 5- to 9-year-olds and to test its cross-cultural invariance. Progress among developmental researchers, pediatricians, epidemiologists, preventionists and educators researching etiology and prevention of behavior problems in children and young adolescents could be facilitated by a user-friendly instrument that does not require reading. Accordingly, this investigation represents an important step toward advancing etiology of behavior problems, its translation to applied settings and evidence-based prevention.