Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Sep 1.
Published in final edited form as: School Ment Health. 2015 Mar 4;7(3):161–173. doi: 10.1007/s12310-015-9146-z

Mental Health Service Use in Schools and Non-School-Based Outpatient Settings: Comparing Predictors of Service Use

David A Langer 1, Jeffrey J Wood 2, Patricia A Wood 3, Ann F Garland 4, John Landsverk 5, Richard L Hough 6
PMCID: PMC4591548  NIHMSID: NIHMS669396  PMID: 26442131

Abstract

Researchers have consistently documented a gap between the large number of US youth meeting criteria for a mental health disorder with significant associated impairment, and the comparatively few youth receiving services. School-based mental health care may address the need–services gap by offering services more equitably to youth in need, irrespective of family economic resources, availability of transportation, and other factors that can impede access to community clinics. However, diagnoses alone do not fully capture the severity of an individual's mental health status and need for services. Studying service use only in relation to diagnoses may restrict our understanding of the degree to which service use is reflective of service need, and inhibit our ability to compare school and non-school-based outpatient settings on their responsiveness to service need. The present study evaluated predictors of mental health service use in school- and community-based settings for youth who had had an active case in one of two public sectors of care, comparing empirically-derived dimensional measurements of youth mental health service need and impairment ratings against non-need variables (e.g., ethnicity, income). Three dimensions of youth mental health service need were identified. Mental health service need and non-need variables each played a significant predictive role. Parent-rated impairment was the strongest need-based predictor of service use across settings. The impact of non-need variables varied by service setting, with parental income having a particularly noticeable effect on school-based services. Across time, preceding service use and impairment each significantly predicted future service use.

Keywords: Mental Health, Service Use, Psychopathology, Youth, School

Researchers have consistently documented a gap between the large number of US youth meeting criteria for a mental health disorder with significant associated impairment (Merikangas et al. 2010), and the comparatively few youth receiving services (Merikangas et al. 2011; Kataoka et al. 2002). School-based mental health care may address the need–services gap by offering services more equitably to youth in need, irrespective of family economic resources, availability of transportation, and other factors that can impede access to community clinics (e.g., Owens et al. 2002) and increase health disparities. However, the diagnostic indicators that are most often used as a measure of need may not fully capture the severity of an individual's mental health status (e.g., Stiffman and Cunningham 1990), preventing a comprehensive study of the service need–service use relationship in school and non-school-based outpatient settings.

Measuring Mental Health Service Need

A substantial proportion of US youth demonstrate significant mental health service need, yet how to characterize need remains a topic of debate. Brewin et al. (1987) define need as “functional impairment that is due to some potentially remediable or preventable cause.” This definition is frequently operationalized as the presence of a psychiatric diagnosis with specified symptoms, and significant associated impairment (e.g., Flisher et al. 1997). Epidemiological studies estimate that between 3% and 18% of children have at least one psychiatric disorder that causes significant functional impairment (Costello et al. 2005; Flisher et al. 1997). Prevalence rates of youth with at least one psychiatric disorder, regardless of impairment, range as high as 45% (see review by Costello et al. 2003). Despite the strengths of the categorical approach (e.g., specific disorder criteria facilitate communication among professionals and provide clear cut-points for making treatment decisions), the DSM categorical system does not clearly differentiate severity among diagnosed youth, obfuscating the identification of those with the highest levels of need. Categorical systems may also miss the substantial number of youth in need of mental health care who do not meet criteria for a specific disorder. Angold et al. (1999), using DSM-III-R (APA 1987), reported that youth with symptomatic impairment yet no formal diagnosis were as disturbed as youth with a diagnosis.

Dimensional models (i.e., continuum-based conceptualizations) offer an alternative to the categorical conceptualization of youth psychopathology. Clark, Watson, and Reynolds (1995) argue that a dimensional approach will reduce the number of categories into a smaller number of dimensions. This has been demonstrated in dimensional measures such as the Child Behavior Checklist (Achenbach 2009; Achenbach and Rescorla 2001) and the Behavior Assessment System for Children (BASC; Reynolds and Kamphaus 1992). However, poor differentiation between and lack of homogeneity within CBCL syndrome scales has prompted researchers to advise against using the CBCL as a treatment selection tool (Hartman et al. 1999), and use of the CBCL and BASC as an indicator of mental health need is complicated further by its treatment of subscale symptoms equally (e.g., suicidal intent is given the same weight as low mood) and its ambiguous inclusion of functional impairment when assigning dimension scores. The present study evaluated dimensional measures of youth psychopathology simultaneously with parent and youth impairment ratings in the prediction of mental health service use, comparing need-based against “non-need-based” predictors. Using the Flisher et al. (1997) operationalization of need as the presence of a psychiatric diagnosis with specified symptoms and significant associated impairment, predictors are considered to be “non-need-based” if they are not a measure of symptoms or impairment, and are thus not what would be targeted in mental health services (e.g., ethnicity, family income).

School-Based and Community-Based Mental Health Services

Often, youth with the greatest mental health need face multiple barriers to service use (Angold et al. 1998; Kataoka et al. 2002; Fox et al. 2002), and these non-need-based factors largely impact service use across multiple service settings. In Zahner and Duskalakis's research (1997), being of a male gender, single mother household, or lower socioeconomic status (SES) increased the likelihood of service use to a statistically similar degree as the CBCL total score. Costello and Janiszewski (1990) reported that children with similar psychopathology differed in their chances of obtaining professional mental health care as a function of sex, race, and possibly age. Elster et al. (2003) reviewed existing studies on the role of ethnicity in service utilization and found that in six out of nine studies, African American adolescents received fewer mental health services than their Caucasian counterparts. The extent to which service use is predicted by non-need-based factors (e.g., family income, ethnicity) represents inequalities in the service system and “health disparities.” The Institute of Medicine, in its 2002 report, Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care, defines disparities as, “racial or ethnic differences in the quality of health care that are not due to access-related factors or clinical needs, preferences, and appropriateness of intervention” (Institute of Medicine 2002; McGuire et al. 2006). Ideally, service use will be most strongly related to need-based predictors (e.g., symptomatology, impairment) that directly relate to the nature of the services offered.

Findings over the previous two decades indicate that service use is also related to mental health diagnoses (Leaf et al. 1996; Offord et al. 1987), an indicator of need, such that diagnosed youth were more likely to receive services. Hazen et al. (2004), however, reported that diagnostic profiles were related to lifetime service use but accounted for little variance in the type and intensity of treatment received, possibly reflecting the difficulty of distinguishing need among youth with identical diagnoses. Although each of these studies is limited by standard diagnostic indicators of service need, they demonstrate that components of mental health need have a clear role in the prediction of service use while also highlighting the significant variability in service use that need-based predictors do not explain.

Although sociodemographic variables have been shown to relate to service use in school and community settings (Zahner and Daskalakis 1997; Koot and Verhulst 1992), existing research suggests that such factors exert less impact on school-based services (e.g., Garland et al. 2001a). Slade (2002) found that access to school-based mental health services did not differ based on race in a nationally representative survey of students in Grades 7 through 12. School-based service use is not dependent on insurance status as is community-based mental health care (Kataoka et al. 2002). Farmer et al. (1999) reported that children in families living below the poverty line received more school-based services but no more (or fewer) clinic-based services than other children. Glied et al. (1997) reported that higher household incomes predicted greater clinic-based service use but had no significant effect on school-based service use. In short, school-based services offer the hope of being more responsive to lower income families. The present study examined the role of mental health service need versus non-need predictors in school-based versus community-based, outpatient service settings in a novel way, highlighting disparities in care (Institute of Medicine 2002) by incorporating empirically-derived measurements of youth mental health need into a larger model of service use predictors.

Lastly, youth service use is characterized by inconsistencies over time in service use persistence and service sector (Farmer et al. 2003). Some previous research showed that the majority of youth who began using services in any given three-month span did not use services in the next three-month span (Farmer et al. 1999). Few existing studies have focused on the pathways into and through mental health service systems, and no known studies have compared community- and school-based settings on the degree to which their use is predicted by need versus non-need factors. Farmer et al. (2003), in the Great Smoky Mountains Study, found that school-based services were a primary resource for identifying and initiating treatment for youth in need, yet the degree to which school-based service settings maintain treatment with high-need youth (or continued to provide services to youth with diminished mental health need) was not addressed. Using a sample of youth who were active in at least one of two public sectors of care in a major urban area (the mental health or public school special education services sectors), the present study assesses service use over a span of 18 months to model changes in service use in school-based and community-based outpatient settings across time based on impairment ratings and previous service use, compared to non-need variables.

Method

Study Design

Subjects were 933 youth aged 11-18 years who had active cases in at least one of two public sectors of care – mental health and public school services for youths with serious emotional disturbance (SED) – in a large metropolitan area during the latter half of the 1996-1997 fiscal year. Youths’ parents were also interviewed. This is a subsample of the Patterns of Care (POC) study (Garland et al. 2001b), which drew from a stratified sample of 1,715 youth who had active cases in one or more of the following five public sectors of care: alcohol/drug, child welfare, juvenile justice, mental health, and SED. After providing informed consent and assent, parents and youth were interviewed, typically in their home. Assessments were performed at baseline (N=933), and then six months (N=905), one year (N=845), and 18 months (N=840) after baseline. Baseline interviews occurred between October, 1997 and March, 1999. At baseline, the sample was 66.1% male and 33.9% female. The ethnic composition was 38.9% Caucasian, 19.7% African American, 24.9% Hispanic, 7.6% Asian/Pacific Islander American (hereafter termed “Asian”), and 8.9% “other.” Several groups were intentionally oversampled (e.g., Asian youth) to provide an adequate sample size for subgroup analyses. A poststratification weighting procedure adjusted for probability of sampling, stratification procedures of the larger study, and oversampling of subgroups (Henry 1990).

Measures

Diagnostic Interview Schedule for Children Version IV (DISC-IV; Shaffer et al. 2000)

Symptomatology and diagnostic profiles were obtained using the DISC 4.0, which is based on DSM-IV criteria (APA 1994). Lay interviewers were trained on the instrument by a member of the DISC Editorial Board. Previous versions of the DISC have shown strong reliability (Schwab-Stone et al. 1996), and preliminary data demonstrate good test-retest reliability for the DISC-IV in a clinical sample (Shaffer et al. 2000). Youth aged 11-18 years completed modules assessing a wide range of symptoms. Parent-report was not obtained for the complete diagnostic interview due to time constraints.

Service Assessment for Children and Adolescents (SACA; Horwitz et al. 2001)

The SACA is a structured interview designed to assess lifetime and past year use of a variety of mental health services. Parents and youths each reported on service use and service use was considered present if either reporter endorsed it. SACA reliability and validity data are well-documented (Horwitz et al. 2001; Stiffman et al. 2000). Analyses divided service use into school-based services and community-based, outpatient services. Under the umbrella term “school-based services” are four categories: (1) school-based counseling or therapy services, (2) special help in regular classroom, (3) special classroom in regular school, (4) special school for youth with emotional and behavioral disorders. Under the umbrella term “community-based, outpatient services” are five categories: (1) specialty mental health outpatient services (e.g., community mental health center), (2) non-specialty mental health outpatient services (e.g., family preservation), (3) other outpatient services (e.g., self-help group), (4) other mental health related services (e.g., court counselor), and (5) outpatient alcohol or drug treatment.

Columbia Impairment Scale (CIS; Bird et al. 1993)

The CIS has strong psychometric properties (Bird et al. 1993) and assesses both parent- and youth-reported functional impairment in the realms of interpersonal relations, broad psychopathology, school/job, and leisure time. The measure consists of 13 items, each rated on a 5-point Likert scale (0=no problem, 4=a very big problem). Scores of 15 or higher indicate clinically significant impairment. The internal consistency reliability estimate for the scale is excellent (alpha for parent report = .88; alpha for youth report = .83). The CIS also possesses excellent test-retest reliability and evidence for construct and predictive validity (Bird et al. 1996).

Family income

Parents reported on their total family income on an incremental scale of annual incomes from ≤$1,000 to ≥$200,000. The current analyses interpreted category midpoints as a continuous variable (Glied et al. 1997), and used a logarithmic transformation to linearize the relationship between income and other variables (J. Cohen et al. 2003).

Family demography

Parents reported on race/ethnicity and educational background. Ethnicity was dummy-coded into three dichotomous variables — Hispanic, Black, Asian — with Caucasians being the reference group (J. Cohen et al. 2003). Parental education was specified as an ordinal variable with higher numbers indicating higher levels of education.

Statistical Analyses

All analyses were conducted with the MPlus 5.1 software (Muthén and Muthén 2007).

Factor analysis

Dimensional measures of youth mental health service need were derived through factor analysis for use in predicting service use in each service setting. Youth-rated symptoms were collected by the DISC for 11 major DSM-IV diagnostic categories: Attention deficit hyperactivity disorder (ADHD), conduct disorder (CD), oppositional defiant disorder (ODD), posttraumatic stress disorder, generalized anxiety disorder, separation anxiety disorder, obsessive-compulsive disorder, social phobia, panic disorder, major depressive disorder (MDD), and hypomania. Highly correlated (Φ > .80), redundant items lead to unreliable factor analysis results (Fabrigar et al. 1999) and were thus eliminated using an algorithm to minimize the number of items removed. Exploratory factor analysis (EFA) of tetrachoric correlations among the symptoms (e.g., Williamson et al. 2002) was used to evaluate possible factor structures. Factors were selected based on goodness-of-fit indexes (Hu and Bentler 1998) including the Likelihood Ratio Test (Lawley 1956), the Comparative Fit Index (CFI) (Bentler 1990), the Root Mean Square Error of Approximation (RMSEA) fit index (Steiger and Lind 1980), and the “scree test” (Cattell 1966). Factor scores were computed with each item contributing to a factor score for a participant relative to its loading on that factor (i.e., refined factor scores) (Grice 2001).

Predicting service use

Service use was divided into school-based service use and community-based outpatient service use, each measured over the first six months of the study. Youth who did not identify as belonging to an ethnic category (n = 83) were excluded from service use analyses. Potential non-need based predictors were: ethnicity, parental education, parental income, and youth gender. These factors have been commonly noted to be among the most significant predictors of service use (John et al. 1995; Farmer et al. 1999; P. Cohen and Hesselbart 1993; Jensen et al. 1990; Offord et al. 1987). Gender was dummy-coded so that males were coded as 0 and females were coded as 1.

To compare need and non-need predictors of service use, service use was specified as a dichotomous outcome and a series of nested, logistic models were compared using an adjusted log-likelihood ratio test (J. Cohen et al. 2003). To compute the contribution of a set of predictors (or an individual predictor), the Deviance of the full model (with the set of predictors) is subtracted from the Deviance of the model without that set. The difference is compared against a chi-square distribution with the degrees of freedom equaling the number of predictors in the full model minus the number of predictors in the model without the set of predictors that are under investigation. Due to MPlus's scaling of the chi-square and log-likelihood values, an adjusted log-likelihood ratio test was used (Muthén 2008; Satorra 2000). To test the relative influence of need and non-need predictors on service use in school versus community service settings, a structural equation model with a logistic outcome variable was used (Loehlin 1992). Mental health need was entered simultaneously with the non-need-based factors to predict two dichotomous outcomes: school-based service use and community-based service use.

Predicting service use across time

Hierarchical Linear Modeling (Bryk and Raudenbush 1992) was used to evaluate the predictive impact of preceding impairment and service use on future service use over the course of 18 months. Separate models were run for school-based and community-based outpatient service use. Preceding impairment and service use were treated as time varying covariates in the models incorporating all available waves of longitudinal data for each subject, controlling for non-need variables by entering them as Level 2 predictors. During follow-up assessments, only one reporter (most often the parent) completed the Columbia Impairment Scale, so only parent-rated impairment was used in the analyses for consistency.

Results

Factor Analysis

The initial item pool of 108 DISC symptoms was reduced to 53 symptoms after eliminating redundancies. EFA was performed on the set of the 53 non-redundant symptoms using the WLSMV estimator (Ivanova et al. 2007) with an oblique rotation method, Geomax (Hendrickson and White 1964). Fit indices for one through six-factor models are detailed in Table 1. The lack of comparable improvements in model fit indices beyond three factors, coupled with the theoretical appeal of the three factors, led to a three factor solution being chosen. The first factor of the three factor solution comprises all nonredundant ADHD and ODD symptoms, the loadings of which are significant at the p <.05 level, plus other symptoms of inattention, hyperactivity, oppositionality, restlessness, talkativeness, and agitation. This factor is referred to as the ADHD/ODD factor. The second factor is composed of all of the nonredundant CD symptoms, with the addition of other symptoms of oppositionality, irritability, and disregard for painful consequences of actions. The second factor is thus the CD factor. The third factor comprises nearly all of the anxiety disorder, MDD, and hypomania symptoms (plus “frequently fidgets or squirms”), and is named the anxiety/mood factor.

Table 1.

Factor Analysis Fit Indices

# of Factors Chi-Square df p-value CFI RMSEA Eigenvalue
Baseline Model 953.782 67 .0000
1 253.755 69 .0000 .792 .047 15.373
2 172.962 80 .0000 .895 .031 6.048
3 130.140 81 .0004 .945 .022 3.846
4 123.876 91 .0125 .963 .017 3.327
5 127.929 99 .0268 .967 .016 2.175
6 120.445 99 .0704 .976 .013 1.991
Open in a new tab

Note. CFI = Comparative Fit Index; RMSEA = Root Mean Square Error of Approximation.

Table 2 displays the relationships among psychopathology factors, CIS impairment ratings, and the non-need variables. Female youth had higher scores on the internalizing factor, whereas males had higher scores on the ADHD/ODD and CD factors. Parents rated female youth as less impaired. Parents with more education rated their youth as more impaired, as did parents with higher incomes. Ethnic minority parents tended to rate their youth as less impaired, with Asian parents rating their children as least impaired. Ethnic minority youth tended to rate themselves lower on the ADHD/ODD and anxiety/mood factors and higher on the CD factor, with the exception of Asian youth, who rated themselves higher on the anxiety/mood factor.

Table 2.

Correlations Between Factors, Impairment Ratings, and Non-Need Variables

ADHD/ODD factor CD factor Anxiety/Mood factor CISP CISC
Gender −0.033 −0.153** 0.13** −0.106** 0.028
Parental Education 0.106** −0.078* 0.106** 0.157** 0
Black −0.121** 0.046 −0.097** −0.227** 0.03
Hispanic −0.074* 0.194** −0.07* −0.117** 0.015
Asian −0.038 0.092** 0.112** −0.483** −0.062
Parental Income 0.014 0.036 −0.026 0.086** 0.029
Open in a new tab

Note.

*

p < .05

**

p < .01.

Gender = 0 for males, 1 for females; CISP = Columbia Impairment Scale, Parent-Rated; CISC = Columbia Impairment Scale, Child-Rated.

Service Use Prediction

Reports of service use over the first six months of the study were high for outpatient services (n = 539, 57.78%) and school-based services (n = 494, 52.95%), with 334 youths (35.80%) having reported service use in both settings, and 234 youths not reporting any service during that time period (25.08%). Rates of service use for services falling under the umbrella term “school-based services” were: (1) school-based counseling or therapy services (18.15%), (2) special help in regular classroom (7.80%), (3) special classroom in regular school (28.13%), and (4) special school for youth with emotional and behavioral disorders (22.92%). Rates of service use for services falling under the umbrella term “community-based, outpatient services” were: (1) specialty mental health outpatient services (38.27%), (2) non-specialty mental health outpatient services (10.28%), (3) other outpatient services (13.5%), (4) other mental health related services (9.51%), and (5) outpatient alcohol or drug treatment (3.34%).

Model fit comparisons for models containing only need, only non-need, and need and non-need sets of variables suggest that both need and non-need variables played a significant role in the prediction of service use. Adding the set of need variables to the set of non-need variables provided a significantly better fit to models predicting any service use (χ2 = 87.070, df = 5, p < .001), outpatient service use (χ2 = 110.807, df = 5, p < .001), and school-based service use (χ2 = 90.295, df = 5, p < .001). Alternatively, adding the set of non-need variables to the set of need variables also provided a significantly better fit to models predicting any service use (χ2 = 30.2735, df = 6, p < .001), outpatient service use (χ2 = 28.596, df = 6, p < .001), and school-based service use (χ2 = 43.299, df = 6, p < .001).

The odds ratios, coefficient estimates, and significance levels for need and non-need predictors of outpatient and school service use, respectively, are shown in Tables 3 and 4. Odds ratios (ORs) serve as measures of effect size; in the current context, an OR represents the odds of receiving services if a certain criterion is met (e.g., membership in an ethnic group) relative to the odds of receiving services if that same criterion is not met. When the predictor is continuous (e.g., factors and impairment ratings), the OR represents the change in odds for each unit increase of the predictor variable. Parent- and youth-rated impairment were significantly related to outpatient service use when entered in the model containing only need variables and the model containing need and non-need variables, with higher parent-rated impairment predicting greater outpatient service use (OR = 1.049 in the full model) and higher youth-rated impairment predicting lower outpatient service use (OR = 0.959 in the full model). This means that for each unit increase in parent-rated impairment, the odds of having received outpatient services increase by approximately 5%, whereas for youth-rated impairment, a unit increase leads to the odds of receiving outpatient services decreasing by approximately 4%. For the parent-rated impairment scale, which in the current sample had a standard deviation of 11, an increase of one standard deviation of parent-rated impairment would result in a notable increase of odds of outpatient service use of approximately 55%. The youth-rated impairment scale had a standard deviation of nine in the current sample, so an increase of one standard deviation in youth impairment would result in a decrease of odds of outpatient service use of approximately 37%. A suppression effect seems unlikely, however, as youth-rated impairment predicted marginally lower outpatient service use in the full model excluding parent-rated impairment (OR = .977, p = .067). In the model containing only non-need variables, higher parental education predicted significantly greater outpatient service use (OR = 1.274; for each additional level of education a parent received, the odds of service use increased by 27%), yet this difference was no longer significant in the full model, indicating that this variability may be explained by their impairment ratings. Parent-rated impairment remained significant when predicting school service use (OR = 1.026 in the full model), yet youth-rated impairment was not significant. A one-unit increase in parental impairment ratings led to an approximately 3% increase in the likelihood of school services being used. In the model with only non-need variables, higher parental income predicted greater school service use, and Hispanics and Asians used fewer school services, yet only parental income remained a significant non-need predictor of school service use in the full model (OR = 1.835). Given the logarithmic transformation of parental income to linearize the relationship between income and other variables (Cohen et al., 2003), this OR is more difficult to interpret, but as an example, going from an income one standard deviation below the mean to the mean income would result in an approximately 80% increase in the likelihood of receiving school-based services.

Table 3.

Odds Ratios and Logistic Regression Coefficients for Psychopathology Factors, Impairment Ratings, and Non-Need Variables Predicting Community-Based Outpatient Service Use

Set(s) Included in Equation Variable Odds Ratio Coefficient Estimate S.E. Est/S.E. p-value
Need Variables ADHD/ODD Factor 1.097 .092 .168 .551 .582
CD Factor 1.109 .104 .155 .670 .503
Anxiety/Mood Factor 1.255 .227 .176 1.286 .198
CISP 1.053 .052 .012 4.388 .000
CISC .960 −.040 .013 −3.008 .003
Non-need Variables Youth Gender .815 −.204 .200 −1.021 .307
Parental Education Level 1.274 .242 .112 2.157 .031
Parental Income 1.199 .182 .251 .725 .469
Youth Ethnicity – Black 1.052 .050 .249 .203 .839
Youth Ethnicity – Hispanic .942 −.059 .264 −.225 .822
Youth Ethnicity – Asian .539 −.619 .333 −1.860 .063
Need and Non-need Variables ADHD/ODD Factor 1.069 .067 .176 .381 .703
CD Factor 1.126 .119 .167 .711 .477
Anxiety/Mood Factor 1.311 .271 .186 1.457 .145
CISP 1.049 .047 .013 3.747 .000
CISC .959 −.042 .014 −3.086 .002
Youth Gender .839 −.176 .227 −.775 .438
Parental Education Level 1.198 .181 .129 1.397 .162
Parental Income 1.214 .194 .266 .730 .622
Youth Ethnicity – Black 1.290 .254 .263 .968 .333
Youth Ethnicity – Hispanic 1.192 .175 .306 .574 .566
Youth Ethnicity – Asian .827 −.190 .386 −.492 .622
Open in a new tab

Note. CISP = Columbia Impairment Scale, Parent-Rated; CISC = Columbia Impairment Scale, Child-Rated.

Table 4.

Odds Ratios and Logistic Regression Coefficients for Psychopathology Factors, Impairment Ratings, and Non-Need Variables Predicting School-Based Service Use

Set(s) Included in Equation Variable Odds Ratio Coefficient Estimate S.E. Est/S.E. p-value
Need Variables ADHD/ODD Factor 1.291 .255 .163 1.567 .117
CD Factor .841 −.173 .168 −1.034 .301
Anxiety/Mood Factor .852 −.160 .174 −.922 .357
CISP 1.033 .033 .011 3.102 .002
CISC .988 −.012 .012 −.897 .370
Non-need Variables Youth Gender .919 −.089 .198 −.425 .671
Parental Education Level 1.218 .193 .115 1.715 .086
Parental Income 1.797 .586 .254 2.305 .021
Youth Ethnicity – Black .727 −.318 .249 −1.280 .201
Youth Ethnicity – Hispanic .535 −.626 .256 −2.445 .014
Youth Ethnicity – Asian .413 −.883 .360 −2.445 .014
Need and Non-need Variables ADHD/ODD Factor 1.219 .198 .162 1.223 .221
CD Factor .903 −.102 .172 −.592 .554
Anxiety/Mood Factor .848 −.165 .165 −.999 .318
CISP 1.026 .026 .011 2.301 .021
CISC .994 −.006 .013 −.444 .657
Youth Gender 1.022 .022 .217 .100 .920
Parental Education Level 1.190 .174 .130 1.335 .182
Parental Income 1.835 .607 .273 2.223 .026
Youth Ethnicity – Black .956 .045 .276 −.162 .871
Youth Ethnicity – Hispanic .627 −.467 .275 −1.698 .090
Youth Ethnicity – Asian .586 −.535 .387 −1.381 .167
Open in a new tab

Note. CISP = Columbia Impairment Scale, Parent-Rated; CISC = Columbia Impairment Scale, Child-Rated.

For outpatient settings, preceding impairment (OR = 1.044) and outpatient service use (OR = 5.442) each significantly predicted an increase in a youth's likelihood of outpatient service use (Table 5). For each impairment score unit increase, the odds of receiving outpatient services increased by approximately 4%; given the variability of impairment scores (baseline impairment scores having a standard deviation of 11), one standard deviation change in impairment score would result in a 44% change in odds of receiving service use. Receiving outpatient services in the previous time period, however, increased the odds of receiving outpatient services in the current time period by approximately 540%. This same relationship was found for school settings (Table 5), though the loading for preceding impairment was relatively weaker (OR = 1.013; a ~1% increase in service use likelihood for each increase of impairment rating, so a 14% change in odds for one standard deviation change in impairment), and the impact of preceding service use was more prominent (OR = 12.465), with the odds of receiving school services being increased by 1,247% for those who were receiving school-based services in the preceding time period. These findings show that even though preceding impairment has a significant impact on service use, preceding service use is a much stronger predictor, especially for school-based services.

Table 5.

Prediction of Service Use by Preceding Impairment and Preceding Service Use, Controlling for Youth/Family Characteristics

Community-based Service Use School-based Service Use
β SE p-value OR β SE p-value OR
Level 1 Predictors
    Preceding Impairment .043 .007 .000 1.044 .012 .006 .041 1.013
    Preceding Service Use 1.694 .176 .000 5.442 2.523 .167 .000 12.465
Level 2 Predictors
    Gender −.464 .155 .003 −.340 .146 .020
    Parental Education .170 .089 .057 .094 .082 .256
    Parental Income .256 .186 .168 .485 .193 .012
    Youth Ethnicity – Black −.128 .202 .525 .071 .180 .695
    Youth Ethnicity – Hispanic −.040 .191 .832 −.215 .182 .237
    Youth Ethnicity – Asian −.379 .186 .168 −.763 .266 .004
Open in a new tab

Note: Gender = 0 for males, 1 for females.

Discussion

The present study, using an empirically-based approach to measure mental health service need, found that both mental health service need and non-need factors contributed to the prediction of school- and community-based service use in a high-risk sample of youth who had recently had an active case in the mental health sector of care, the SED sector of care, or both. Of the symptom-based factor scores and impairment ratings, parent ratings of youth impairment were the strongest predictor of service use in either setting, even when controlling for non-need-based factors such as income and ethnicity, suggesting that perceptions of impairment play a more prominent role in the prediction of mental health service use than type of symptomatology. The only consistent non-need-based predictor of service use was parental income, which had a positive relationship with school-based service use.

Measuring Mental Health Service Need

When assessing the dimensional structure of the 53 nonredundant symptoms culled from the larger set of DSM-IV symptoms, a three factor solution fit the data best. Although there is a large body of research examining the dimensional structure of youth psychopathology, studies using a wide range of symptomatology (as opposed to disorder-specific scales) are more limited, with even fewer focusing on the collection of DSM-IV symptoms. Some of the most influential work on the broad range of youth psychopathology has been the work of Achenbach et al. (Achenbach 2009; Achenbach and Rescorla 2001), along with work resulting in other influential scales such as the Behavior Assessment System for Children (BASC; Reynolds and Kamphaus 1992). The present results differ from the CBCL and BASC findings by delineating two primary factors belonging to the broader “Externalizing” category, while also finding a single factor for anxiety and mood disorder symptoms. Rather than representing a conflict between present and past research, this discrepancy likely reflects the differences in the relevant comorbidities for youth in clinical, referred samples, and the present sample is one of high-risk youth who were at one point active in a public sector of care whereas the CBCL and BASC research focused on community samples. Hartman and colleagues (2001), however, used factor analytic methods to analyze a set of DSM-IV symptoms across a sample of multi-national youth and found that a three factor model consisting of ADHD, aggressive behavior, and internalizing problem dimensions fit their data better than a two factor model divided by internalizing and externalizing symptoms, similar to the findings of the present study.

Predicting Mental Health Service Use

Even though youth in the present sample were all, at the time of enumeration, active in either (or both) the SED or MH public sectors of care, only a portion of youth in the present study received services in schools or community-based outpatient settings throughout the course of the study. This may partially reflect the periodic nature of mental illness (see Cicchetti and Rogosch 2002), in that youth who had reason to be active in a mental health public sector of care may have no longer needed services at the time of the study assessments, yet it may also reflect inefficient service allocation and health disparities (Institute of Medicine, 2002). In model comparisons predicting service use in either setting, mental health need variables and non-need variables each had a significant predictive role. The significance of individual predictors varied depending on whether the outcome was community- or school-based service use.

Need-based predictors of service use

Parent- and youth-rated impairment were the only significant need-based predictors of outpatient service use in models with and without non-need predictors. Interestingly, higher parent-rated impairment predicted greater outpatient service use, whereas higher youth-rated impairment predicted lower outpatient service use (even in a model without parent-rated impairment). Parent- and youth-rated impairment scores were significantly positively correlated, yet shared a relatively small proportion of variability in common (i.e., less than 15%) and it is unclear which components of youth-rated impairment have a negative relationship with community-based outpatient service use. Previous research has highlighted the importance of parent perceptions in the prediction of service use (Zahner and Daskalakis 1997), possibly because parents often decide when to pursue services (Armbruster and Kazdin 1994). As an example, the parent-therapist relationship significantly relates to the rate of cancelled or missed therapy sessions, though the youth-therapist relationship has no impact on session attendance (Hawley and Weisz 2005). It is also possible that youth receiving services may have been biased to report less impairment to meet the perceived expectations of the assessor, or that youth who see themselves as more impaired are also less motivated to engage in treatment.

Parent-rated impairment was the only significant need-based predictor of school-based service use. The provision of school-based services is often determined by the assessments and perceptions of school-based faculty and staff, including teacher observations and reports. The present study did not include teacher reports of youth behavior. Previous research suggests that parent ratings of psychopathology and functioning, compared to youth ratings, may better approximate the views of teachers (Mattison et al. 2007; Stanger and Lewis 1993). Zahner and Daskalakis (1997) found that parental belief that a youth needs help strongly predicted service use in both community- and school-based settings. Parents who perceive higher youth impairment may also be more likely to advocate for school-based services.

Although disruptive youth are more common in school-based SED programs than anxious or depressed youth (Mattison et al. 1992), neither the ADHD/ODD nor the CD factor was significant in the presence of impairment ratings. In light of these findings, the importance of specific symptom endorsement, at least youth-reported symptoms, relative to impairment ratings is brought into question. The present findings suggest that the type of symptom presentation may not be as important as concurrent impairment. Previous POC research (Garland et al. 2005; Hazen et al. 2004) found the presence of a diagnosis predicted outpatient service use more strongly than elevated impairment ratings, and other services research has reported similar findings (Leaf et al. 1996; Offord et al. 1987), though of these studies, only POC specifically sampled high-risk youth who had recently had active cases in mental health and/or SED sectors of care. It is possible that the present study's youth-reported symptoms do not adequately capture the entire range of a youth's symptomatology (Jensen et al. 1999) and the strength of parent-rated impairment reflects unassessed symptoms, necessitating additional research.

Non-need-based predictors of service use

Parental income was the only non-need-based predictor that remained significant in the model with need-based variables, and only predicted school-based service use. No non-need-based predictors were consistently significant predictors of community-based outpatient service use. Youth with higher parental incomes (likely a proxy for community-level SES) were more likely to receive school-based mental health services. This finding is in contrast to previous findings that schools were more responsive to those with lower incomes (Farmer et al. 1999) or less affected by household income (Glied et al. 1997). This divergence may reflect the region-specific nature of mental health care provision, whereby schools serve as the default provider for low-SES children in some regions, and in other regions, resource-limited schools may refer children to publicly-funded community settings. Given limited funding for school-based mental health services (Evans et al. 2003; Weist et al. 2003), some schools have been able to access fee-for-service funding mechanisms (e.g., Medicaid), yet others have reported that creating an adequate billing structure to recoup fee-for-service reimbursements, as community clinics do, is infeasible (Robinson et al. 2000). A federal survey of public school districts found that 49% of respondents relied on local funds to finance mental health interventions, making local funding the third most prevalent funding source behind IDEA and state special education funds (Foster et al. 2005), though the latter two funding sources place more limitations on youth eligibility for services. The present results suggest that the current funding structure of school-based services, at least in the region in which the study was conducted, translates into fewer services for low-SES youth, regardless of mental health need.

The finding that ethnicity was not a consistently significant predictor of mental health service use is in contrast to previous research on the larger POC sample (Garland et al. 2005) and research on community samples (Costello and Janiszewski 1990; Elster et al. 2003; Kataoka et al. 2002; Wu et al. 2001). Lau et al. (2004) reported little variability by ethnicity in youth-reported YSR problem scores, suggesting that the youth-reported symptomatology in the present study may not be as biased by ethnicity, yet parent-rated impairment was shown in the present study to strongly relate to ethnicity, and may explain the service use variation found in previous POC studies. Several of these studies included measures of impairment (e.g., Garland et al. 2005), yet these were mostly dichotomized versions of continuous impairment scales (CIS, CGAS) that considered youth impaired if either parent-, child-, or interviewer-reported impairment ratings surpassed a specified cutpoint, potentially attenuating the strong relationship between parent-rated impairment and ethnicity.

Prediction of service use across time

Previous service use and previous impairment each predicted current service use in school- and community-based settings, suggesting that both settings are responsive (service use is more likely for youth with higher impairment ratings) and persistent (service use is more likely for youth with preceding service use). That preceding service use significantly predicted current service use when accounting for preceding mental health need may indicate inefficient services allocation – ideally, service use is provided for youth in need, regardless of whether they have already received services – yet the degree to which the significance of preceding service use reflects continuing services that are needed to maintain treatment gains is unclear. This finding is also qualified by the limited measurement of mental health service need across time (restricted to adult-rated impairment) that may not capture aspects of a youth's psychopathology and impairment.

This pattern of findings has precedent in previous research. Farmer and colleagues (1999) reported that 30% of youth using any form of mental health services in a three-month period continue to use mental health services in the subsequent three-month period. Youth who used services in at least two three-month periods were also more likely to have higher symptom levels. Impairment ratings were not significant predictors of persistent service use, yet impairment was assessed in relation to specific diagnostic categories, making it difficult to compare to the present study. Ortega et al. (2007) found that impairment did not significantly predict persistent service use, yet they found that family burden, as reported by parents, was a significant predictor. It is possible that parent-rated impairment in the present study reflects family burden in addition to youth impairment, and the significance of parent-rated impairment is due to its measurement of related constructs. Although limited in the range of predictors included, the present study represents an advance in youth services research, measuring service use as a dynamic variable that changes over time, and testing the predictive weight of impairment and preceding service use in the context of non-need predictors.

Despite the many strengths of the present study, several limitations should be noted. Using a sample of youth who belonged to a mental health public sector of care at the time of sample selection resulted in higher symptom rates and greater impairment than would be found in a community sample, in addition to higher rates of service use. The intent of the sampling procedure was to provide greater clarity on the psychopathology and service use patterns for “high-risk” youth, though this means the findings may not generalize to community samples. In terms of the psychopathology findings, the type of psychopathology may also be different; results from the larger POC sample found that ADHD and disruptive behavior disorders had a 50% prevalence rate, higher than anxiety (10%) and mood (7%) disorders (Garland et al. 2001b). Therefore, the structure of psychopathology presented in the current study likely differs from that of the general population. Higher rates of service use in this high-risk sample that, at the time of the study, was currently (or had previously been) active in a public sector of care may have impacted the service use findings by limiting service use variability or showing different patterns of service use than would be found in community samples. However, these results remain relevant to other clinical samples of youth, and still provide a more nuanced understanding of the youth in mental health public sectors of care.

Another limitation of the present study, noted above, is that only youth reported on their symptoms. Given the consistently low reliability between parent and youth reports of psychopathology symptoms (Kraemer et al. 2003) this is a clear limitation, yet other research suggests that the impact of including only youth-reported symptoms may be buffered by the older mean age of the sample, since parent–child agreement increases as youth become adolescents (Jensen et al. 1999). Lastly, service use patterns may have changed since the time these data were collected in the late 1990s, and the rates of service use found in the locality of the study may not generalize to other communities.

A methodological strength of the present study was its use of impairment ratings that were not tied to specific diagnoses. Impairment ratings were significant in the prediction of service use across settings. The impairment measure used in the present study assesses four major areas of functioning — interpersonal relations, broad psychopathology domains, functioning in job or schoolwork, and use of leisure time — and future research on the predictive power of these circumscribed domains may shed more light on the importance (or lack of importance) of understanding the components of impairment.

Lastly, the present study, as is common in the majority of services research, approached the services data globally, aggregating many different types of services based on the settings in which the services were provided. Future research may wish to explore services within a given setting separately, allowing one to distinguish brief, minimal interventions from regular, intensive interventions. Services could also be studied by funding source. Schools often contract with non-school-based resources to provide mental health services (Evans et al. 2003), and these services may be considered a joint effort between schools and community-based settings.

In conclusion, the current study assessed the relative importance of mental health service need and non-need factors in the prediction of service use in school- and community-based outpatient settings for youth with recently active cases in the mental health or SED public sectors of care, addressing limitations of previous research that conceptualized need in purely diagnostic or impairment terms. Findings suggest that both need and non-need variables impact service use in both settings, with higher parent-rated impairment predicting greater service use overall, and higher parental income predicting more school-based service use.

Acknowledgments

This research was supported in part by National Institute of Mental Health grants F31-MH079631 (DAL) and U01MH 55282 (RLH).

Contributor Information

David A. Langer, Department of Psychology, Boston University 648 Beacon St. 6th Floor Boston, MA 02215 dalanger@bu.edu, 617-353-9610 (phone) 617-353-9609 (fax)

Jeffrey J. Wood, University of California, Los Angeles Moore Hall 3132A 405 Hilgard Avenue Los Angeles, CA 90095-1521

Patricia A. Wood, Child and Adolescent Services Research Center 3020 Children's Way MC 5033 San Diego, CA 92123

Ann F. Garland, Department of School, Family, and Mental Health Professions, University of San Diego, San Diego, USA

John Landsverk, Child and Adolescent Services Research Center 3020 Children's Way MC 5033 San Diego, CA 92123

Richard L. Hough, Child and Adolescent Services Research Center 3020 Children's Way MC 5033 San Diego, CA 92123

Literature Cited

  1. Achenbach TM. The Achenbach System of Empirically Based Assessment (ASEBA): Development, findings, theory, and applications. University of Vermont Research Center for Children, Youth, & Families; Burlington, VT: 2009. [Google Scholar]
  2. Achenbach TM, Rescorla LA. Manual for ASEBA School-Age Forms & Profiles. University of Vermont, Research Center for Children, Youth, & Families; Burlington, VT: 2001. [Google Scholar]
  3. Angold A, Costello EJ, Farmer EMZ, Burns BJ, Erkanli A. Impaired but undiagnosed. Journal of the American Academy of Child and Adolescent Psychiatry. 1999;38(2):129–137. doi: 10.1097/00004583-199902000-00011. [DOI] [PubMed] [Google Scholar]
  4. Angold A, Messer SC, Stangl D, Farmer EM, Costello EJ, Burns BJ. Perceived parental burden and service use for child and adolescent psychiatric disorders. American Journal of Public Health. 1998;88(1):75–80. doi: 10.2105/ajph.88.1.75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. APA . Diagnostic and Statistical Manual of Mental Disorders. 3rd edition (DSM-III) American Psychiatric Association; Washington, DC: 1987. [Google Scholar]
  6. APA . Diagnostic and Statistical Manual of Mental Disorders. 4th edition (DSM-IV) American Psychiatric Association; Washington, DC: 1994. [Google Scholar]
  7. Armbruster P, Kazdin AE. Attrition in child psychotherapy. Advances in Clinical Child Psychology. 1994;16:81–108. [Google Scholar]
  8. Bentler PM. Comparative fit indexes in structural models. Psychological Bulletin. 1990;107(2):238–246. doi: 10.1037/0033-2909.107.2.238. [DOI] [PubMed] [Google Scholar]
  9. Bird HR, Andrews H, Schwab-Stone M, Goodman S, Dulcan M, Richters J, et al. Global measures of impairment for epidemiologic and clinical use with children and adolescents. International Journal of Methods in Psychiatric Research. 1996;6(4):295–307. [Google Scholar]
  10. Bird HR, Shaffer D, Fisher P, Gould MS, Staghezza B, Chen JY, et al. The Columbia Impairment Scale (CIS): Pilot findings on a measure of global impairment for children and adolescents. International Journal of Methods in Psychiatric Research. 1993;3(3):167–176. [Google Scholar]
  11. Brewin CR, Wing JK, Mangen SP, Brugha TS, MacCarthy B. Principles and practice of measuring needs in the long-term mentally ill: the MRC needs for care assessment. Psychological Medicine. 1987;17(4):971–981. doi: 10.1017/s0033291700000787. [DOI] [PubMed] [Google Scholar]
  12. Bryk AS, Raudenbush SW. Hierarchical linear models in social and behavioral research: Applications and data analysis methods (First ed., Advanced qualitative techniques in the social sciences. Vol. 265. Sage Publications; Newbury Park, CA: 1992. [Google Scholar]
  13. Cattell RB. The scree test for the number of factors. Multivariate Behavioral Research. 1966;1:245–276. doi: 10.1207/s15327906mbr0102_10. [DOI] [PubMed] [Google Scholar]
  14. Cicchetti D, Rogosch FA. A developmental psychopathology perspective on adolescence. Journal of Consulting and Clinical Psychology. 2002;70(1):6–20. doi: 10.1037//0022-006x.70.1.6. [DOI] [PubMed] [Google Scholar]
  15. Clark LA, Watson D, Reynolds S. Diagnosis and Classification of Psychopathology: Challenges to the Current System and Future Directions. Annual Review of Psychology. 1995;46:121–153. doi: 10.1146/annurev.ps.46.020195.001005. [DOI] [PubMed] [Google Scholar]
  16. Cohen J, Cohen P, West SG, Aiken LS. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Third ed. Lawrence Erlbaum Associates; Mahwah, New Jersey: 2003. [Google Scholar]
  17. Cohen P, Hesselbart CS. Demographic factors in the use of children's mental health services. American Journal of Public Health. 1993;83:49–52. doi: 10.2105/ajph.83.1.49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Costello EJ, Egger H, Angold A. 10-year research update review: the epidemiology of child and adolescent psychiatric disorders: I. Methods and public health burden. Journal of the American Academy of Child and Adolescent Psychiatry. 2005;44(10):972–986. doi: 10.1097/01.chi.0000172552.41596.6f. [DOI] [PubMed] [Google Scholar]
  19. Costello EJ, Janiszewski S. Who gets treated? Factors associated with referral in children with psychiatric disorders. Acta Psychiatrica Scandinavica. 1990;81(6):523–529. doi: 10.1111/j.1600-0447.1990.tb05492.x. [DOI] [PubMed] [Google Scholar]
  20. Costello EJ, Mustillo S, Erkanli A, Keeler G, Angold A. Prevalence and Development of Psychiatric Disorders in Childhood and Adolescence. Archives of General Psychiatry. 2003;60:837–844. doi: 10.1001/archpsyc.60.8.837. [DOI] [PubMed] [Google Scholar]
  21. Elster A, Jarosik J, VanGeest J, Fleming M. Racial and Ethnic Disparities in Health Care for Adolescents: A Systematic Review of the Literature. Archives of Pediatrics and Adolescent Medicine. 2003;157(9):867–874. doi: 10.1001/archpedi.157.9.867. [DOI] [PubMed] [Google Scholar]
  22. Evans SW, Glass-Siegel M, Frank A, Van Treuren R, Lever NA, Weist MD. Overcoming the challenges of funding school mental health programs. In: Weist MD, editor. Handbook of school mental health: Advancing practice and research. Issues in clinical child psychology. Kluwer Academic/Plenum Publishers; New York, NY: 2003. pp. 73–86. [Google Scholar]
  23. Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods. 1999;4(3):272–299. [Google Scholar]
  24. Farmer EMZ, Burns BJ, Phillips SD, Angold A, Psych MRC, Costello EJ. Pathways Into and Through Mental Health Services for Children and Adolescents. Psychiatric Services. 2003;54:60–66. doi: 10.1176/appi.ps.54.1.60. [DOI] [PubMed] [Google Scholar]
  25. Farmer EMZ, Stangl DK, Burns BJ, Costello EJ, Angold A. Use, Persistence, and Intensity: Patterns of Care for Children's Mental Health Across One Year. Community Mental Health Journal. 1999;35(1):31–46. doi: 10.1023/a:1018743908617. [DOI] [PubMed] [Google Scholar]
  26. Flisher AJ, Kramer RA, Grosser RC, Alegria M, Bird HR, Bourdon K, et al. Correlates of unmet need for mental health services by children and adolescents. Psychological Medicine. 1997;27:1145–1154. doi: 10.1017/s0033291797005412. [DOI] [PubMed] [Google Scholar]
  27. Foster S, Rollefson M, Doksum T, Noonan D, Robinson G. School Mental Health Services in the United States, 2002–2003. Center for Mental Health Services, Substance Abuse and Mental Health Services Administration; Rockville, MD: 2005. [Google Scholar]
  28. Fox H, McManus MA, Reichman MB. The strengths and weaknesses of private health insurance coverage for children with special health care needs. MCH Policy Research Center; Washington, DC: 2002. [Google Scholar]
  29. Garland AF, Hough RL, Landsverk JA, Brown SA. Multi-sector complexity of systems of care for youth with mental health needs. Children’s Services: Social Policy, Research and Practice. 2001a;4(3):123–140. [Google Scholar]
  30. Garland AF, Hough RL, McCabe KM, Yeh M, Wood PA, Aarons GA. Prevalence of psychiatric disorders in youths across five sectors of care. Journal of the American Academy of Child and Adolescent Psychiatry. 2001b;40(4):409–418. doi: 10.1097/00004583-200104000-00009. [DOI] [PubMed] [Google Scholar]
  31. Garland AF, Lau AS, Yeh M, McCabe KM, Hough RL, Landsverk JA. Racial and Ethnic Differences in Utilization of Mental Health Services Among High-Risk Youths. American Journal of Psychiatry. 2005;162(7):1336–1343. doi: 10.1176/appi.ajp.162.7.1336. [DOI] [PubMed] [Google Scholar]
  32. Glied S, Hoven CW, Garrett AB, Moore RE, Leaf P, Bird HR, et al. Measuring Child Mental Health Status for Services Research. Journal of Child and Family Studies. 1997;6(2):177–190. [Google Scholar]
  33. Grice JW. Computing and evaluating factor scores. Psychological Methods. 2001;6(4):430–450. [PubMed] [Google Scholar]
  34. Hartman CA, Hox J, Auerbach J, Erol N, Fonseca AC, Mellenbergh GJ, et al. Syndrome Dimensions of the Child Behavior Checklist and the Teacher Report Form: A Critical Empirical Evaluation. Journal of Child Psychology and Psychiatry. 1999;40(07):1095–1116. [PubMed] [Google Scholar]
  35. Hartman CA, Hox J, Mellenbergh GJ, Boyle MH, Offord DR, Racine Y, et al. DSM-IV Internal Construct Validity: When a Taxonomy Meets Data. Journal of Child Psychology and Psychiatry. 2001;42(6):817–836. doi: 10.1111/1469-7610.00778. [DOI] [PubMed] [Google Scholar]
  36. Hawley KM, Weisz JR. Youth versus parent working alliance in usual clinical care: Distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Adolescent Psychology. 2005;34(1):117–128. doi: 10.1207/s15374424jccp3401_11. [DOI] [PubMed] [Google Scholar]
  37. Hazen AL, Hough RL, Landsverk JA, Wood PA. Use of Mental Health Services by Youths in Public Sectors of Care. Mental Health Services Research. 2004;6(4):213–226. doi: 10.1023/b:mhsr.0000044747.54525.36. [DOI] [PubMed] [Google Scholar]
  38. Hendrickson AE, White PO. Promax: A quick method for rotation to oblique simple structure. British Journal of Statistical Psychology. 1964;17(1):65–70. doi: 10.1111/j.2044-8317.1966.tb00358.x. [DOI] [PubMed] [Google Scholar]
  39. Henry GT. Practical Sampling. Sage Publications Inc; 1990. [Google Scholar]
  40. Horwitz SM, Hoagwood K, Stiffman AR, Summerfeld T, Weisz JR, Costello EJ, et al. Reliability of the Services Assessment for Children and Adolescents. Psychiatric Services. 2001;52(8):1088–1094. doi: 10.1176/appi.ps.52.8.1088. doi:10.1176/appi.ps.52.8.1088. [DOI] [PubMed] [Google Scholar]
  41. Hu LT, Bentler PM. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods. 1998;3(4):424–453. [Google Scholar]
  42. Institute of Medicine . Unequal treatment: Confronting racial and ethnic disparities in health care. National Academies Press.; Washington, DC: 2002. [PubMed] [Google Scholar]
  43. Ivanova M, Achenbach TM, Rescorla L, Dumenci L, Almqvist F, Bilenberg N, et al. The Generalizability of the Youth Self-Report Syndrome Structure in 23 Societies. Journal of Consulting and Clinical Psychology. 2007;75(5):729–738. doi: 10.1037/0022-006X.75.5.729. [DOI] [PubMed] [Google Scholar]
  44. Jensen PS, Bloedau L, Davis H. Children at risk: II. Risk factors and clinic utilization. Journal of the American Academy of Child and Adolescent Psychiatry. 1990;29(5):804–812. doi: 10.1097/00004583-199009000-00021. [DOI] [PubMed] [Google Scholar]
  45. Jensen PS, Rubio-Stipec M, Canino G, Bird HR, Dulcan MK, Schwab-Stone ME, et al. Parent and child contributions to diagnosis of mental disorder: Are both informants always necessary? Journal of the American Academy of Child & Adolescent Psychiatry. 1999;38(12):1569–1579. doi: 10.1097/00004583-199912000-00019. [DOI] [PubMed] [Google Scholar]
  46. John LH, Offord DR, Boyle MH, Racine YA. Factors predicting use of mental health and social services by children 6-16 years old: findings from the Ontario Child Health Study. American Journal of Orthopsychiatry. 1995;65(1):76–86. doi: 10.1037/h0079597. [DOI] [PubMed] [Google Scholar]
  47. Kataoka SH, Zhang L, Wells KB. Unmet Need for Mental Health Care Among US Children: Variation by Ethnicity and Insurance Status. American Journal of Psychiatry. 2002;159(9):1548–1555. doi: 10.1176/appi.ajp.159.9.1548. [DOI] [PubMed] [Google Scholar]
  48. Koot HM, Verhulst FC. Prediction of children's referral to mental health and special education services from earlier adjustment. Journal of Child Psychology and Psychiatry. 1992;33(4):717–729. doi: 10.1111/j.1469-7610.1992.tb00907.x. [DOI] [PubMed] [Google Scholar]
  49. Kraemer HC, Measelle JR, Ablow JC, Essex MJ, Boyce WT, Kupfer DJ. A New Approach to Integrating Data From Multiple Informants in Psychiatric Assessment and Research: Mixing and Matching Contexts and Perspectives. American Journal of Psychiatry. 2003;160:1566–1577. doi: 10.1176/appi.ajp.160.9.1566. [DOI] [PubMed] [Google Scholar]
  50. Lau AS, Garland AF, Yeh M, McCabe KM, Wood PA, Hough RL. Race/Ethnicity and Inter-Informant Agreement in Assessing Adolescent Psychopathology. Journal of Emotional and Behavioral Disorders. 2004;12(3):145–156. [Google Scholar]
  51. Lawley DN. A General Method for Approximating to the Distribution of Likelihood Ratio Criteria. Biometrika. 1956;43(3/4):295–303. [Google Scholar]
  52. Leaf PJ, Alegria M, Cohen P, Goodman SH, Horwitz SM, Hoven CW, et al. Mental health service use in the community and schools: results from the four-community MECA study. Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35(7):889–897. doi: 10.1097/00004583-199607000-00014. [DOI] [PubMed] [Google Scholar]
  53. Loehlin JC. Latent Variable Models: An Introduction to Factor, Path, and Structural Analysis. Lawrence Erlbaum Associates; 1992. [Google Scholar]
  54. Mattison RE, Carlson GA, Cantwell DP, Asarnow JR. Teacher and parent ratings of children with depressive disorders. Journal of Emotional and Behavioral Disorders. 2007;15(3):184–192. [Google Scholar]
  55. Mattison RE, Morales J, Bauer MA. Distinguishing characteristics of elementary schoolboys recommended for SED placement. Behavioral Disorders. 1992;17(2):107–114. [Google Scholar]
  56. McGuire TG, Alegria M, Cook BL, Wells KB, Zaslavsky AM. Implementing the Institute of Medicine definition of disparities: An application to mental health care. Health Services Research. 2006;41(5):1979–2005. doi: 10.1111/j.1475-6773.2006.00583.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication--Adolescent Supplement (NCS-A). Journal of the American Academy of Child and Adolescent Psychiatry. 2010;49(10):980–989. doi: 10.1016/j.jaac.2010.05.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Merikangas KR, He JP, Burstein M, Swendsen J, Avenevoli S, Case B, et al. Service utilization for lifetime mental disorders in U.S. adolescents: results of the National Comorbidity Survey-Adolescent Supplement (NCS-A). Journal of the American Academy of Child and Adolescent Psychiatry. 2011;50(1):32–45. doi: 10.1016/j.jaac.2010.10.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Muthen B. [October 5 2008];Chi-square difference testing using the Satorra-Bentler scaled chi-square. 2008 http://www.statmodel.com/chidiff.shtml.
  60. Muthén LK, Muthén BO. MPlus user's guide. 5th edition Muthén & Muthén.; Los Angeles, CA: 2007. [Google Scholar]
  61. Offord DR, Boyle MH, Szatmari P, Rae-Grant NI, Links PS, Cadman DT, et al. Ontario Child Health Study. II. Six-month prevalence of disorder and rates of service utilization. Archives of General Psychiatry. 1987;44(9):832–836. doi: 10.1001/archpsyc.1987.01800210084013. [DOI] [PubMed] [Google Scholar]
  62. Owens PL, Hoagwood K, Horwitz SM, Leaf PJ, Poduska JM, Kellam SG, et al. Barriers to children’s mental health services. Journal of the American Academy of Child and Adolescent Psychiatry. 2002;41(6):731–738. doi: 10.1097/00004583-200206000-00013. [DOI] [PubMed] [Google Scholar]
  63. Ortega AN, Chavez L, Inkelas M, Canino G. Persistence of mental health service use among Latino children: A clinical and community study. Administration and Policy in Mental Health and Mental Health Services Research. 2007;34:353–362. doi: 10.1007/s10488-007-0114-2. [DOI] [PubMed] [Google Scholar]
  64. Reynolds CR, Kamphaus RW. Behavior Assessment System for Children (BASC) American Guidance Service; Circle Pines, MN: 1992. [Google Scholar]
  65. Robinson GK, Barrett M, Tunkelrott T, Kim J. School-based mental health services under Medicaid managed care. Center for Mental Health Services, Substance Abuse and Mental Health Services Administration; Rockville, MD: 2000. [Google Scholar]
  66. Satorra A. Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In: Heijmans RDH, Pollock DSG, Satorra A, editors. Innovations in multivariate statistical analysis. A Festschrift for Heinz Neudecker. Kluwer Academic Publishers; London: 2000. pp. 233–247. [Google Scholar]
  67. Schwab-Stone ME, Shaffer D, Dulcan MK, Jensen PS, Fisher P, Bird HR, et al. Criterion validity of the NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC-2.3). Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35(7):878–888. doi: 10.1097/00004583-199607000-00013. [DOI] [PubMed] [Google Scholar]
  68. Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME. NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): Description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry. 2000;39(1):28–38. doi: 10.1097/00004583-200001000-00014. [DOI] [PubMed] [Google Scholar]
  69. Slade EP. Effects of School-Based Mental Health Programs on Mental Health Service use by Adolescents at School and in the Community. Mental Health Services Research. 2002;4(3):151–166. doi: 10.1023/a:1019711113312. [DOI] [PubMed] [Google Scholar]
  70. Stanger C, Lewis M. Agreement among parents, teachers, and children on internalizing and externalizing behavior problems. Journal of Clinical Child Psychology. 1993;22(1):107–115. [Google Scholar]
  71. Steiger JH, Lind JC. Statistically based tests for the number of common factors.. Paper presented at the Annual Meeting of the Psychometric Society; Iowa City, IA. 1980. [Google Scholar]
  72. Stiffman AR, Cunningham R. The epidemiology of child and adolescent mental health disorders. In: Gibbs JT, editor. Child and Adolescent Mental Health: Challenges for Social Work Education and Practice. NASW Press Berkeley; Berkeley, CA: 1990. pp. 9–32. [Google Scholar]
  73. Stiffman AR, Horwitz SM, Hoagwood K, Compton W, Cottler L, Bean DL, et al. The Service Assessment for Children and Adolescents (SACA): Adult and child reports. Journal of the American Academy of Child and Adolescent Psychiatry. 2000;39(8):1032–1039. doi: 10.1097/00004583-200008000-00019. [DOI] [PubMed] [Google Scholar]
  74. Weist MD, Goldstein J, Evans SW, Lever NA, Axelrod J, Schreters R, et al. Funding a full continuum of mental health promotion and intervention programs in the school. Journal of Adolescent Health. 2003;32(Suppl6):70–78. doi: 10.1016/s1054-139x(03)00067-3. [DOI] [PubMed] [Google Scholar]
  75. Williamson DA, Womble LG, Smeets MAM, Netemeyer RG, Thaw JM, Kutlesic V, et al. Latent structure of eating disorder symptoms: A factor analytic and taxometric investigation. American Journal of Psychiatry. 2002;159(3):412–418. doi: 10.1176/appi.ajp.159.3.412. [DOI] [PubMed] [Google Scholar]
  76. Wu P, Hoven CW, Cohen P, Liu X, Moore RE, Tiet Q, et al. Factors associated with use of mental health services for depression by children and adolescents. Psychiatric Services. 2001;52(2):189–195. doi: 10.1176/appi.ps.52.2.189. [DOI] [PubMed] [Google Scholar]
  77. Zahner GE, Daskalakis C. Factors associated with mental health, general health, and school-based service use for child psychopathology. American Journal of Public Health. 1997;87(9):1440–1448. doi: 10.2105/ajph.87.9.1440. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES