Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Sep 1.
Published in final edited form as: J Psychopathol Behav Assess. 2014 Oct 7;37(3):422–431. doi: 10.1007/s10862-014-9464-8

Measurement of Behavioral and Emotional Outcomes of Youth in Foster Care: Investigation of the Roles of Age and Placement Type

Joy Gabrielli 1, Yo Jackson 1, Shaquanna Brown 1
PMCID: PMC4606882  NIHMSID: NIHMS633648  PMID: 26478653

Abstract

The Behavioral Assessment System for Children–2 (BASC-2) is used to assess behavioral and emotional outcomes for youth. Research providing evidence for use of the BASC-2 parent-report form historically has included biological parents reporting on their children (Reynolds and Kamphaus 2004). For youth residing in out-of-home placements through enrollment in foster care, caregivers reporting on their functioning may include foster parents or residential staff. Given the significant adverse mental health outcomes for youth in foster care and the need to adequately assess adjustment in foster care, the purpose of the study was to evaluate the measurement properties of caregivers’ report on the parent report form (PRS) of the BASC-2 in foster care youth. Using 479 respondents, a measurement model was fit to the data demonstrating adequate fit across Internalizing Problems, Externalizing Problems, and Adaptive Skills. Further, a comparison of measurement properties across child and adolescent groups and groups of youth residing in residential facilities versus foster homes was conducted. Factorial invariance and latent means also were assessed. The BASC-2 PRS was found to be an adequate assessment of psychological outcomes for youth in foster care when completed by foster parents or residential facility staff.

Keywords: Behavioral and emotional assessment, Foster youth, Factorial invariance, Psychometric assessment, Residential and foster home placement

Research indicates that abuse is strongly associated with a variety of negative outcomes including school failure (Kendall-Tackett and Eckenrode 1996), poor mental health outcomes (Cicchetti and Toth 2005), and poor behavioral outcomes (Kaplan, Pelcovitz, and Labruna 1999; Petrenko, Friend, Garrido, Taussig, and Culhane 2012). Moreover, for the 20.8 % of child victims of maltreatment placed in foster care due to substantiated child maltreatment, evidence suggests significant deficits in typical development and psychological health. Clearly, the field needs to provide youth in foster care with valid and useful tools for capturing their behavioral adjustment. The purpose of this study is to provide first time evidence of the factor structure of the Behavioral Assessment System for Children-2 (BASC-2) for a sample of youth in the foster care system.

Broadband behavior rating scales are commonly administered in lieu of or in conjunction with methods of direct behavior assessment (e.g., observations) due to their brevity and demonstrated success capturing psychological and behavioral differences among diverse populations (Robins, Schoff, Glutting, and Abelkop 2003). One of the most widely used behavior rating scales, the BASC-2 PRS, assesses emotional symptoms, behavioral problems, and adjustment patterns in youth across three major domains of functioning: internalizing, externalizing, and adaptive behavior (Reynolds and Kamphaus 2004). Psychometric properties of the original BASC system have been well-established across a number of diverse populations (e.g., McNamara, Hollmann, and Riegel 1994; Nail and Evans 1997; Robins et al. 2003), and have contributed to the measure's increasing popularity in research and clinical settings. Five years after its initial publication, the BASC was ranked the fourteenth most frequently used assessment in a survey of school psychologists (Boyle, Matthews, and Saklofske 2008). Although retaining the key features of the original BASC, the latest version of the measure, the BASC-2, offers improved psychometric properties and newly created content scales (e.g., Functional Communication and Activities of Daily Living; Reynolds and Kamphaus 2004). Similar to the original BASC, the BASC-2, has experienced widespread popularity, becoming the most widely used behavior rating scale among school psychologists aiming to identify students with deficits such as emotional disturbances, with nearly 98 % using both the parent and teacher scales in these evaluations (Hanchon and Allen 2013). A particular strength of the BASC-2 is that it, unlike other broadband measures of emotional and behavioral functioning (e.g., Child Behavior Checklist), generates individual subscales for Anxiety and Depression. This is an especially important distinction given that when compared to the general population, foster care youth are at increased risk for high levels of anxiety and depression (Pilowsky and Wu 2006). Moreover, when teased apart, literature suggests that youth involved with foster care may be more likely to develop symptoms of depression than symptoms of anxiety (Havalchak, White, O'Brien, and Pecora 2007). Unfortunately, most of the parent report data collected using this popular measure, have been gathered from clinical samples or large normative groups using ratings by biological parents making the measure's utility and validity in foster populations indeterminate (Reynolds and Kamphaus 2004).

For example, for children in foster care, the parent of the child is not always clearly defined or consistent as caretakers may include foster parents, or residential facility staff. Moreover, foster children may be moved several times from foster home to foster home or in and out of residential facilities throughout their duration in foster care making the parent reporter for a given child highly variable (Oosterman, Schuengel, Slot, Bullens, and Dorleijers 2007). Children in foster care often have significant risk for poor adjustment and mental illness (English et al. 2005), and, while often used as a clinical tool, the measurement model of the BASC-2 has not yet been assessed in a foster care population. To better understand and address the mental health needs of youth in care, the field requires tools to accurately assess the psychological functioning of children in the foster care system. The purpose of the present study was to empirically examine the measurement properties of the parent report version (PRS) of the BASC-2 in a sample of foster caregivers to determine if surrogate parents or foster parents respond to the items on this tool like other parents included in the norming samples.

Given that caregiver reporters for youth in foster care differ from the original norm sample and include foster parents as well as residential facility staff, factorial invariance testing was conducted across residential and foster placement groups. Residential placements differ significantly from foster home placements due to caregiving provided by paid staff, a higher number of youth residing in one facility, and a higher level of service coordination present on-site (e.g., often therapists and case-managers work at the residential facility where children reside; James et al. 2006; Park, Epstein, Jordan, Mandell, and Lyons 2009). Further, placement in a residential facility is intended to serve as intervention for youth behaviors too severe to be managed in a foster home setting (Barth 2002). Research has demonstrated differences in emotional and behavioral outcomes between youth residing in foster homes versus youth residing in residential facilities. Specifically, when compared to youth residing in foster care, youth residing in residential facilities report more internalizing and externalizing problems, higher levels of substance abuse problems, and are more likely to be diagnosed with a behavioral or mental health disorder (Baker, Kurland, Curtis, Alexander, and Papa-Lentini 2007; McCrae, Lee, Barth, and Rauktis 2010). Identification of true differences across groups is difficult to determine without first ensuring the measurement model is similar across groups. Therefore, factorial invariance testing was an essential preliminary step for exploring outcomes in youth in foster care through use of the BASC-2 PRS.

Moreover, research has shown age trends indicating that caregivers perceive younger children as exhibiting more problem behaviors than older children, particularly following exposure to a negative life event (Graham-Berman and Perkins 2010). These findings may be due to older children's increasing demands for independence and parents’ decreased ability to monitor their child's behavior. Alternatively, these findings may also highlight age differences in symptom expression. For example, researchers have found evidence for a developmental decline in hyperactivity, an externalizing problem behavior, in youth transitioning to middle school, specifically among youth diagnosed with ADHD (Langberg et al. 2008). Age is a particularly salient construct among foster care youth given research indicating that youth who are maltreated during later childhood are more likely to experience a host of negative outcomes including posttraumatic stress disorder and emotional distress than those youth who are maltreated earlier in life (English et al. 2005). Similarly, older age at time of placement in the foster care system is associated with greater placement disruptions, an identified risk factor for a range of maladaptive outcomes including problem behaviors (Oosterman et al. 2007). Taken together, these findings suggest that older youth experience more difficulty adjusting after exposure to negative life events and that this difficulty may manifest in a number of different ways including behaviorally, with increases in both externalizing and internalizing problem behaviors. The current study accounted for these age-related differences by conducting factorial invariance across the age groups.

Evidence on the psychometric properties of the BASC-2 PRS in a foster care sample would prove useful, particularly because this subset of youth commonly exhibit higher levels of maladaptive behaviors and are in need of more specialized treatment interventions. Discrepancies about youths’ problem behaviors are often found between biological parents and youth who have never experienced out-of-home placements, and prior research has either explained discrepancies as measurement error or as influenced by reporter characteristics (De Los Reyes 2011; De Los Reyes and Kazdin 2005). Given that youth residing in foster care and residential settings often experience multiple caregiver transitions and different care-givers, it is especially important that scientific tools are consistent and reliable in detection of maladaptive behaviors in this unique population.

The purpose of this study is to provide first time evidence of the factor structure of the BASC-2 PRS for a sample of children and adolescents in the foster care system. Psychometric properties of the BASC-2 PRS were assessed through model fit using structural equation modeling techniques, and caregiver reports on children and adolescents are compared for factorial invariance across age groups (i.e., child versus adolescent) and placement type (i.e., foster home versus residential facility). It was hypothesized that the factor structure of the BASC-2 PRS model of Internalizing Problems, Externalizing Problems, and Adaptive Skills scores would adequately fit the data with minimal transformation, and differences across child and adolescent groups and residential versus foster placement groups would be found at the mean level once factorial invariance was established. Specifically, it was hypothesized that older youth would demonstrate higher levels of Externalizing and Internalizing Problems scores and lower scores on Adaptive Skills than younger youth. Similarly, it was expected that youth residing in residential placements would demonstrate higher scores on Externalizing and Internalizing Problems and lower scores on Adaptive Skills than youth residing in foster homes.

Method

Participants

The current study's sample included 479 adult reporters for children with a mean age of 12.9 (SD=3.1) residing in a large, Midwestern, metropolitan city. Of the adult reports, 303 reports were on adolescents (youth ages 12 years and older) and 176 reports were on children (youth ages 8 to 11 years). Regarding youth demographics, 37 % resided in residential facilities, and the remaining 63 % lived in foster homes. Approximately half of the foster youth who were reported on by caregivers were female (47 %). Additionally, the majority of the youth were Black or African American (50 %), followed by White or Caucasian (35 %), Multiracial (10 %), Hispanic or Latino (3 %), and Other (2 %). These youth demographics match national and regional estimates of youth residing in foster care. The reporters were primarily foster parents (50 %), followed by staff or caseworkers (37 %), and biological relatives (i.e., grandparents) (13 %). Finally, roughly 59 % of the youth have received a mental health diagnosis and 65 % have been treated for an emotional or psychological problem. Of those with diagnoses reported, 41 % reported youth with disorders related to externalizing behavior problems (e.g., Attention-Deficit Hyperactivity Disorder (ADHD), Conduct Disorder), 68 % reported on youth with disorders related to internalizing behavior problems (e.g., Major Depressive Disorder), and, 16 % reported on youth who had comorbid disorders (e.g., ADHD and Anxiety Disorder).

Measures

The BASC-2, parent-report form (BASC-2 PRS), provides an assessment of the child's behavior at home and in the community across 150 (adolescent version) - 160 (child version) items measured on a Likert scale (Reynolds and Kamphaus 2004). The measure includes composite scales of Internalizing Problems, Externalizing Problems, and Adaptive Skills. The Internalizing composite consists of the anxiety, depression, and somatization subscales. The Externalizing composite includes the hyperactivity, aggression, and conduct problems subscales. The Adaptive Skills composite has five subscales: adaptability, social skills, leadership, activities of daily living, and functional communication. Reliability estimates across composite indices range from alphas of .89 to .95 for parent reports of kids aged 8 and older (Reynolds & Kamphaus), and the BASC-2 PRS has consistently performed well as a measure of behavioral outcome when compared to the Achenbach System of Empirically Based Assessment Child Behavior Checklist (Achenbach and Rescorla 2000), the Conners’ Parent Rating Scale-Revised (Conners 1997), and the Behavior Rating Inventory of Executive Functioning (Gioia, Isquith, Guy, and Kenworthy 2000; Reynolds and Kamphaus 2004).

Procedures

Children and caregivers participated in the SPARK (Studying Pathways to Adjustment and Resilience in Kids) Project, a federally funded research project investigating risk and protective factors of children in foster care. Specifically, the project seeks to identify individual, family, and community variables that promote psychological well-being and adaptation.

Data were collected from foster children and an adult caregiver who could report on the child's environment and behavior (such as a foster parent or residential facility staff person). Children had to be 8 years or older, living in their current setting for at least 1 month, and have IQs of 70 or above to be enrolled in the study. Additionally, the caregiver must have known and cared for the child for at least 30 days prior to study enrollment. In the case of residential facilities, staff who best knew the child participant were selected to complete the caregiver report. Caregivers completed an informed consent process, and children provided informed assent.

Children and their caregivers were invited to complete questionnaires administered through an audio-assisted computerized survey. As part of the project, the child's care-giver completed the BASC-2 PRS. Data were collected at three time points with 3-month intervals; however, the current study included data from only the time 1 assessment. See Jackson et al. (2012), for additional details on the project methodology including recruitment strategy and data collection procedures.

Missing Data

Because the data was collected using a computerized survey that requires a response from each participant before continuing to the next question, there was very little missing data (less than 1 % missing). The BASC-2 PRS has a specific protocol for missing data that involves multiplying the number of missing values by a constant value (either “0”, “1”, or “2” depending upon the subscale). This formula was established by the BASC-2 developers to provide relative weight to omitted answers on the BASC-2 based on previous data of average item responses for the specific subscales (Reynolds and Kamphaus 2004). Missing data points were rare, with three reporters, out of 479, each missing one subscale and covariance coverage from .988 to 1.00.

Analytic Procedure

To evaluate the measurement properties of the BASC-2 PRS, a multi-group structural equation modeling (SEM) approach was used, which enabled the structure and model fit of the BASC-2 PRS to be compared across child and adolescent groups. To account for the nestedness of the data given several caregiver reporters reported on multiple youth (e.g., siblings in the same foster home had the same caregiver reporter; some residential staff reported on multiple youth), a clustering factor was added to the model. In the child versus adolescent comparison, there were 133 independent caregiver reports on 176 children and 198 independent caregiver reports on 303 adolescents. To identify the constructs being measured, the scale of each construct must be set as an initial step to provide a reference point for measurement model process. This project utilized the marker variable method of scale setting, in which the latent variance of each construct is freely estimated, and the loading of the first indicator for each construct was fixed at 1.0. Since each construct had three or more indicators, the model was overidentified, meaning there were more observed variance and covariance values available than the number of parameters that were to be estimated (Little, Slegers, and Card 2006).

The BASC-2 PRS includes questions that are grouped to formulate subscales that load onto higher order composite constructs. For the purposes of this measurement analysis, the item-level data was averaged to create subscale indicators comprising the observed data; subsequently, the subscales are parceled item-level data. Parceling can be considered an improvement on simple item-level data as it reduces the likelihood of correlated residuals and dual indicator loadings, it provides a higher ratio of common-to-unique factor variance, and it provides a more parsimonious model with reduced sampling error (Little, Cunningham, Shahar, and Widaman 2002). Parceling was particularly appropriate for this project as the composite constructs and subscale indicators were of greater interest than the item-level data.

The measurement properties of the two group models (child versus adolescent; residential versus foster home placement) were compared via tests identifying the best fit of the indicators and the constructs. Further examination of the invariance of measurement parameters such as the factor loadings, intercepts, and freed and fixed parameters is appropriate in such comparative analyses. To ensure the same constructs were being assessed in both groups, testing of factorial invariance was performed. The equivalence of the models was assessed through equating the indicator loadings then the means (Little, Card, Slegers, and Ledford 2007).

Results

Measurement Model Fit

The standard BASC-2 PRS model was implemented with the caregiver report on the foster youth. All indicators were significantly correlated at p<.05 level for the Adaptive Skills, Internalizing Problems, and Externalizing Problems scales. The nonsignificant loadings were pruned (i.e., Adaptability from the Adaptive Skills composite), and modification indices were employed to free correlated residuals (i.e., Aggression with Depression and Conduct Problems subscales; Social Skills with Anxiety and Leadership subscales; Communication with Leadership subscales). The primary freely estimated confirmatory factor analysis (CFA) using maximum likelihood rotation for the BASC-2 PRS demonstrated adequate fit in the overall sample (χ2 (54, n = 479)= 177.304, p<0.001, RMSEA (0.082–0.114)=.098, TLI=.925, CFI=.955).

The final model for both child and adolescent groups consisted of three latent constructs (i.e., Internalizing Problems, Externalizing Problems, and Adaptive Skills). The Internalizing Problems composite had factor loadings from the Somatization, Anxiety, and Depression subscales. The Externalizing Problems composite included loadings from the Hyperactivity, Aggression, and Conduct Problems sub-scales. Finally, the Adaptive Skills composite was indicated by loadings from the Social Skills, Leadership, Functional Communication, and Activities of Daily Living. See Fig. 1 for the final measurement model depiction. Mean T-scores and standard deviations for the subscales across each group analyzed can be found in Table 1.

Fig. 1.

Fig. 1

Open in a new tab

Final structural model for the child and adolescent groups on the BASC-2 PRS

Table 1.

Mean T-scores of BASC-2 subscales across four study groups

Adolescent (SD) Child (SD) Residential (SD) Foster(SD)
Externalizinga
    Conduct problems 70.89 (15.57) 70.37 (14.77) 75.22 (14.43) 68.16 (15.26)
    Aggression 66.16 (15.43) 67.35 (15.05) 72.61 (14.50) 63.26 (14.73)
    Hyperactivity 66.91 (13.32) 64.68 (10.71) 69.70 (11.50) 64.03 (12.59)
Internalizinga
    Depression 63.62 (13.49) 62.84 (13.24) 67.79 (13.24) 60.73 (12.84)
    Anxiety 60.09 (10.49) 57.50 (10.75) 61.09 (10.37) 58.05 (10.75)
    Somatization 58.54 (13.56) 56.38 (13.18) 58.62 (12.15) 57.23 (14.26)
Adaptive skillsb
    Functional communication 41.52 (8.89) 41.37 (9.21) 39.80 (8.14) 42.31 (9.32)
    Leadership 47.88 (7.44) 48.46 (6.97) 46.68 (7.09) 48.90 (7.29)
    Acts of daily living 46.01 (9.13) 43.71 (8.67) 44.40 (8.81) 45.59 (9.19)
    Social skills 47.80 (7.90) 48.53 (8.10) 46.30 (7.55) 49.04 (8.06)
Open in a new tab
a

T-scores 60–65 considered to be in the “at-risk” range, scores 70 < considered to be in the “clinical” range

b

T-scores 31–40 considered to be in the “at-risk” range, scores <30 considered to be in the “clinical” range

Child and Adolescent Group Invariance Testing

The first step of invariance testing was to test for configural invariance. This was done by running the same model separately in each group. The BASC-2 PRS demonstrated adequate fit in the child dataset (χ2(27, n = 176) =74.702, p<0.001, RMSEA (0.074–0.128)=.100, TLI=.915, CFI=.949) and ade quate fit in the adolescent dataset (χ2(27, n = 303) =101.299, p<0.001, RMSEA (0.076–0.115)=.095, TLI=.930, CFI=.958). See Table 2 for more information on factor loadings, residuals, and R2 values. Once it was determined that the child and adolescent groups had the same composition of fixed and freed parameters, it was necessary to verify if the matching factor loadings were equivalent. This is accomplished through equating the loadings between the two groups, while allowing intercepts to differ, and comparing this nested model to the configural model using the RMSEA ‘Reasonableness’ Test and the Chi-Square Difference test. The fit for the full metric invariance model was significantly worse than the configural invariance model, so we explored the possibility of partial invariance. Use of model data, modification indices, and theoretical reasons led us to allow the Hyperactivity and Activities of Daily Living to differ in loadings across age groups, thus allowing us to achieve partial metric invariance (χ2(59, n=479) =188.355; RMSEA=.096(.080–.111); CFI=.953; TLI=.928; See Table 3). Tests for scalar invariance were completed, however, due to misfit models would not converge. Thus, only partial metric invariance was obtained for the child and adolescent group comparison. Further, correlations between the latent constructs across both the child and adolescent groups were in the expected direction and suggest strong relations between constructs (See Table 4). For additional information on correlations of subscales on this measure within this sample, please refer to Jackson et al. (2014).

Table 2.

Factor loadings for child and adolescent groups

Indicator Equated estimatesa: Loading (SE) Freed adolescent estimates: Loading (SE)
Externalizing: Latent variance (Adolescent = 158.693; Child = 171.105)
    Conduct problems 1.000 (0.000)
    Aggression 0.998 (0.036)
    Hyperactivityb 0.676 (0.054) 1.023 (0.065)
Internalizing Latent variance (Adolescent = 171.031; Child = 152.273)
    Depression 1.000 (0.000)
    Anxiety 0.537 (0.041)
    Somatization 0.523 (0.052)
Adaptive Skills: Latent variance (Adolescent = 57.364; Child = 46.526)
    Functional communication 1.000 (0.000)
    Leadership 0.734 (0.043)
    Acts of daily livingb 1.192 (0.099) 1.023 (0.054)
    Social skills 0.750 (0.055)
    Depression with aggression 19.404 (3.734)
    Social skills with anxiety 12.221 (2.318)
    Leadership with social skills 9.378 (1.585)
    Aggression with conduct problemsb 8.967 (7.951) 36.493 (8.112)
    Functional communication with leadershipb 13.004 (3.047) 4.815 (1.889)
Open in a new tab
a

For loadings allowed to vary across groups, this column contains the values for the Child group

b

These means were freed to obtain Partial Strong Invariance

Table 3.

Fit indices for the nested sequence for invariance testing across child and adolescent groups

Model χ 2 df p Δ χ2 Δ df p RMSEA RMSEA 90 % CI TLI CFI Constraint tenable
Configural invariance 177.304 54 <.001 – – – 0.098 0.082–0.114 0.925 0.955 Yes
Loading invariance1 188.335 62 <.001 11.031 8 .200 0.092 0.077–0.107 0.933 0.954 Partial
Intercept invariance1 No convergence – – – – – – – – No
Open in a new tab
1

Evaluated with the RMSEA Model Test and Chi Square χ2 Difference Test

Each nested model contains its constraints, plus the constraints of all previous, tenable models

Table 4.

Correlations between latent constructs for adolescent and child groups

Externalizing Internalizing Adaptive Skills
Adolescent: externalizing 1.000
Child: externalizing 1.000
Adolescent: internalizing 0.706 1.000
Child: internalizing 0.701 1.000
Adolescent: adaptive skills –0.724 –0.578 1.000
Child: adaptive skills –0.754 –0.559 1.000
Open in a new tab

Domains of Externalizing Problems, Internalizing Problems, and Adaptive Skills were assessed on the BASC-2 PRS. All correlations are within age group and relations were statistically significant at the p<.001 level

Residential and Foster Home Placement Group Invariance Testing

Given the failure to obtain full measurement invariance across age groups, measurement invariance across foster placement type was examined within each age dataset (child and adolescent) separately. Due to sample size issues, invariance tests were not tenable within the child dataset (i.e., only 39 residential children and 137 foster placement children). Therefore, invariance testing across placement type was conducted only using the adolescent dataset (i.e., 136 residential adolescents and 167 foster placement adolescents). Further, given the significant chi-square (24.797, p<.001) across age and placement type in this sample, it was determined that the best assessment of true invariance across groups would have to be conducted within age datasets to account for the relation between age and placement. Within the adolescent youth, configural invariance testing revealed adequate fit across both residential and foster home groups (χ2(54, n=479)=134.561; RMSEA=.105(.083–.127); CFI=.952; TLI=.921). For the weak invariance test, loadings were equated, and results revealed this did not result in significantly worse model fit (Δχ2(12)= 16.817, p=.157; See Table 5). Once weak invariance was attained, intercepts across groups were equated to test for strong invariance. While this final test was significant at the p<.05 level, suggesting loss in model fit under the constraint of equal intercepts, other model estimates (i.e., RMSEA, CFI) did not indicate significant change in model fit. While it could be stated that factorial invariance was obtained across residential and foster placement groups within the adolescent participants based on minimal changes to RMSEA and CFI, a more conservative approach using the chi-square difference test suggested freeing an additional intercept. In our model, by freeing the Aggression intercept we obtained partial strong invariance (Δχ2(6)=12.388, p=.053; See Table 5). Finally, a comparison of latent means across residential and foster placement youth was conducted (See Table 6). Using youth in foster placement as the reference, findings revealed youth in residential placements had significantly higher mean levels of Externalizing (7.425; p<.001) and Internalizing Problems (6.572; p<.001) and significantly lower levels of Adaptive Skills (−2.709; p<.05).

Table 5.

Fit indices for the nested sequence for invariance testing across residential and foster home groups within the adolescent group

Model χ 2 df p Δ χ2 Δ df p RMSEA RMSEA 90 % CI TLI CFI Constraint tenable
Configural invariance 134.561 54 <.001 – – – 0.105 0.083–0.127 0.921 0.952 Yes
Loading invariance1, 2 151.378 66 <.001 16.817 12 .157 0.097 0.077–0.118 0.931 0.949 Yes
Intercept invariance1 167.322 73 <.001 15.944 7 .026 0.097 0.078–0.117 0.931 0.944 Yes
Intercept invariance1, 2 151.378 72 <.001 12.388 6 .053 0.097 0.077–0.116 0.932 0.946 Partial
Open in a new tab
1

Evaluated with the RMSEA Model Test

2

Evaluated with the χ2 Difference Test

Each nested model contains its constraints, plus the constraints of all previous, tenable models

Table 6.

Results of latent mean comparisons across residential versus foster home placements

Construct Latent mean difference Wald statistic p Equivalent across groups
Internalizing 6.572 3.584 <.001 No
Externalizing 7.425 3.493 <.001 No
Adaptive skills –2.709 1.990 <.001 No
Open in a new tab

Latent means in the foster placement group were used as the reference group and standardized to have a mean of zero. Non-equivalence suggests significant latent mean differences

Discussion

The purpose of this study was to evaluate the measurement model of the BASC-2, PRS identifying Internalizing Problems, Externalizing Problems, and Adaptive Skills using reports from non-biological caretakers for youth in foster care. Analyses revealed a model of the BASC-2 PRS with adequate fit. A second aim of this study was to assess measurement invariance across caregivers’ report of behavior for child and adolescent groups and across youth residing in foster homes versus youth residing in residential facilities. Findings revealed that, in the age comparison, partial weak factorial invariance was attained after freeing two subscale loadings, Hyperactivity and Activities of Daily Living; strong invariance was not achieved across groups, which suggests corresponding subscale means were not similar in the child and adolescent groups. Moreover, it appears that the latent constructs are all associated as indicated by the correlation matrix presented in Table 4. The directionality of the correlations between constructs was also as expected in both groups (i.e., positive correlations between Internalizing Problems, Externalizing Problems, and negative correlations between Adaptive Skills and the prior two constructs).

These findings are not surprising given extant literature indicating that there are differences in the presentation of symptoms across developmental stages (Hersen and Turner 2003). For example, one study found that while depressed adolescents experienced more severe symptoms of depression, depressed children were more irritable than depressed adolescents (Birmaher et al. 2009). As youth age and develop more sophisticated cognitive, emotional, social, and biological processes, the complex nature and magnitude of these changes may contribute to the expression of behavioral and emotional symptoms, thereby supporting our finding of age differences in the measurement model. Moreover, for behavioral problems such as hyperactivity (a key factor difference in the child and adolescent measurement models in the present study) symptom expression is likely to change over the course of child development (Biederman, Mick, and Faraone 2000).

Because invariance testing across the age groups suggested different structural models for the child and adolescent groups, invariance testing across residential and foster placements within each age group was conducted. Sample size limitations within the child group would not allow for further examination of the measure model; however, within the adolescent group findings revealed factorial invariance across placement type. This finding provides evidence that the data for the models operated in similar fashions across the placement groups. The results suggest that in other samples of foster caretakers, it would likely be appropriate to collapse the two placement groups into one data set. Any differences between group means can be accounted for by an inclusion of a covariate to adjust for these differences in future studies.

These findings support the use of this tool in foster care populations, and the high scores across composites received by these participants suggest a significant need for clinical services. These findings corroborate prior research identifying youth in foster care as at-risk for demonstrating behavioral and emotional problems (Carbone, Sawyer, Searle, and Robinson 2007). Further comparison of the latent means at the composite score level revealed a significant difference between placement groups across all three composite scales. Caregivers reporting on youth in residential facilities reported higher scores across the Externalizing Problems and Internalizing Problems composites and lower scores for the Adaptive Skills composite. These findings suggest that problems displayed by youth in foster care may differ in degree or severity depending on placement type. While these results corroborate prior findings indicating that youth residing in residential facilities may demonstrate more significant mental health and behavioral problems than youth placed in foster home settings (James et al. 2006; McCrae et al. 2010), these results may also highlight differences in reporter sensitivity. Residential facility personnel may be more sensitive to identifying behavioral and emotional problems than foster parents because they may undergo extensive and ongoing training in behavioral and emotional needs of youth residing in residential facilities. Moreover, residential facilities often offer individual and group psychotherapy, which staff, at times, may attend. Finally, residential workers are often required to document behavioral problems in youth as part of their job requirements, whereas foster parents are not always aware of or required to implement such documentation or behavior tracking mechanisms with the youth they have residing in their homes. Further research is needed to identify if differences in scores are due to true differences in symptom expression or differences in placement type.

Limitations of the Study

One limitation of the present study was failure to obtain a model with excellent fit using the BASC-2 PRS in the current sample. It should be noted, however, that the model fit presented here is better than the model fit provided by the full model reported in the BASC-2 manual (Reynolds and Kamphaus 2004) and may be just as good as other measurement models within this population. Unfortunately, little research of this nature has been done within a foster care sample. This study provides evidence of reasonable model fit and highlights the need for additional research. Despite the fact that there are few agreed upon guidelines for determination of needed sample size for an SEM analysis, it is possible with a larger number of participants in each group that model fit may have improved and confidence in the findings around group latent mean differences would be higher (Browne and Cudeck 1993). Additionally, power to detect differences is influenced by sample size as well as degrees of freedom (MacCallum, Browne, and Sugawara 1996). Further, it is unclear if non-optimal model fit for the child and adolescent models could have been remediated at the subscale and item-level CFA. Parcels of the items were created by averaging the item-level data to compose the subscale indicators. The subscale-level constructs have strong support in the literature; however it is possible these subscales could have been adjusted at the item level using SEM techniques to identify a model with better fit. Given the variability and complexity of placements (e.g., some children leave and re-enter care; some children reside with one family and others reside in many different placements) we were unable to control for how long the caregivers knew the youth. This may be an important factor to consider in future studies utilizing caregiver report on foster youth. Finally, it should be noted that the identification of youth having “residential” or “foster home” placements is somewhat arbitrary given how frequently youth move across placement settings (Farmer, Wagner, Burns, and Richards 2003). In this project, foster placement was identified at the time of assessment, but youth may have had prior placements across setting types. Results should be interpreted with caution when youth are categorized based on a cross-sectional look at their placement status.

Implications for Future Research

This study sought to provide evidence for the use of the BASC-2 PRS measure with foster care youth. Data from these analyses provide support for differences in measurement models for child and adolescent foster care youth; however, further research is needed to identify the best factor structure of this measure for this population if the two groups are compared without freeing the measurement models. Next steps for this process would involve an examination of the item-level factor structure at the subscale level to determine if closer fit can be attained. Moreover, results from this study highlight the importance of examining the measurement models of tools being used within subpopulations, a finding that has been noted in research on other subgroups of youth (Little 1997; Steele et al. 2006). Youth in foster care represent such a unique and under-researched group that special care must be given to ensure appropriate measurement of the constructs under investigation.

Future research examining the association between foster youths'self-report and caregivers’ report would also be useful. Given the diverse ethnic representation of foster children and that the majority of children in foster care are placed with Caucasian caregivers, studies examining the measurement properties of the BASC-2 PRS accounting for the relationship between ethnicity of reporter and ethnicity of the child would also be recommended as prior research has noted discrepancies may be related to child ethnicity (De Los Reyes and Kazdin 2005). Variables that have been associated with behavioral outcomes, such as abuse experiences or other salient life experiences, will offer additional information on how behavioral and emotional outcomes function for youth in foster care as well.

Additionally, further research examining the differences between youth in residential placements versus foster home placements on composite scales of Internalizing Problems, Externalizing Problems and Adaptive Skills is warranted. Based on the results of this analysis, further research on the unique clinical needs of this understudied population are indicated, and evidence suggests that children and adolescents in different placements may have differential symptom expression identified by caregivers. It is unclear from this study whether the significant difference identified in these latent means on the BASC-2 PRS was a true difference between groups on these symptom expressions or an artifact of other relationships that were not identified. Given the small residential placement sample within the child subgroup, further research into these relationships with larger groups would be useful.

Finally, the promising findings posited by the study suggest that the BASC-2 PRS is an adequate measure of behavioral functions as reported by non-biological caregivers in the lives of foster youth. Identification of a proper measure for these important constructs is only the first step in further understanding the difficulties faced by these children. Continued research is needed to investigate the many influences related to behavioral outcomes for these foster youth, as well as to further the knowledge base on appropriate measures for use in this population.

Acknowledgments

The current project was made possible by funding from the National Institutes of Mental Health, RO1 grant MH079252-03 and the National Institute on Drug Abuse, F31 grant DA034423.

Footnotes

Conflict of Interest Joy Gabrielli declares that there is no conflict of interest, Yo Jackson declares that there is no conflict of interest, Shaquanna Brown declares that there is no conflict of interest.

Experiment Participants This research was approved by the University of Kansas institutional review board as well as district county judges who serve as legal guardians for the youth in foster care surveyed through the present study.

References

  1. Achenbach TM, Rescorla L. Manual for the ASEBA pre-school forms & profiles: An integrated system of multi-informant assessment. University of Vermont, Research Center for Children, Youth, & Families; Burlington: 2000. [Google Scholar]
  2. Baker AJ, Kurland D, Curtis P, Alexander G, Papa-Lentini C. Mental health and behavioral problems of youth in the child welfare system: Residential treatment centers compared to therapeutic foster care in the Odyssey Project population. Child Welfare. 2007;86(3):97–123. [PubMed] [Google Scholar]
  3. Barth RP. Institutions vs. foster homes: The empirical base for the second century of debate. Annie E. Casey Foundation, University of North Carolina, School of Social Work, Jordan Institute for Families; Chapel Hill: 2002. [Google Scholar]
  4. Biederman J, Mick E, Faraone SV. Age-dependent decline of symptoms of attention deficit hyperactivity disorder: impact of remission definition and symptom type. American Journal of Psychiatry. 2000;157(5):816–818. doi: 10.1176/appi.ajp.157.5.816. [DOI] [PubMed] [Google Scholar]
  5. Birmaher B, Axelson D, Strober M, Gill MK, Yang M, Ryan N, Leonard H. Comparison of manic and depressive symptoms between children and adolescents with bipolar spectrum disorders. Bipolar Disorders. 2009;11(1):52–62. doi: 10.1111/j.1399-5618.2008.00659.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Boyle GJ, Matthews G, Saklofske DH. The SAGE handbook of personality theory and assessment. Sage; Thousand Oaks: 2008. [Google Scholar]
  7. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors. Testing structural equation models. Sage; Newbury Park: 1993. pp. 136–162. [Google Scholar]
  8. Carbone JA, Sawyer MG, Searle AK, Robinson PJ. The health-related quality of life of children and adolescents in home-based foster care. Quality of Life Research. 2007;16(7):1157–1166. doi: 10.1007/s11136-007-9227-z. [DOI] [PubMed] [Google Scholar]
  9. Cicchetti D, Toth SL. Child maltreatment. Annual Review of Clinical Psychology. 2005;1:409–438. doi: 10.1146/annurev.clinpsy.1.102803.144029. [DOI] [PubMed] [Google Scholar]
  10. Conners CK. Conners’ Rating Scales-Revised. Multi-Health Systems; North Tonawanda: 1997. [Google Scholar]
  11. De Los Reyes A. Introduction to the special section: More than measurement error: Discovering meaning behind informant discrepancies in clinical assessments of children and adolescents. Journal of Clinical Child & Adolescent Psychology. 2011;40(1):1–9. doi: 10.1080/15374416.2011.533405. [DOI] [PubMed] [Google Scholar]
  12. De Los Reyes A, Kazdin AE. Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin. 2005;131(4):483. doi: 10.1037/0033-2909.131.4.483. [DOI] [PubMed] [Google Scholar]
  13. English DJ, Upadhyaya MP, Litrownik AJ, Marshall JM, Runyan DK, Graham JC, Dubowitz H. Maltreatment's wake: The relationship of maltreatment dimensions to child outcomes. Child Abuse & Neglect. 2005;29(5):597–619. doi: 10.1016/j.chiabu.2004.12.008. [DOI] [PubMed] [Google Scholar]
  14. Farmer EM, Wagner HR, Burns BJ, Richards JT. Treatment foster care in a system of care: Sequences and correlates of residential placements. Journal of Child and Family Studies. 2003;12(1):11–25. [Google Scholar]
  15. Gioia GA, Isquith PK, Guy SC, Kenworthy L. Test review behavior rating inventory of executive function. Child Neuropsychology. 2000;6(3):235–238. doi: 10.1076/chin.6.3.235.3152. [DOI] [PubMed] [Google Scholar]
  16. Graham-Berman SA, Perkins S. Effects of early exposure and lifetime exposure to intimate partner violence (IPV) on child adjustment. Violence and Victims. 2010;25(4):427–439. doi: 10.1891/0886-6708.25.4.427. [DOI] [PubMed] [Google Scholar]
  17. Hanchon TA, Allen RA. Identifying students with emotional disturbance: School psychologists’ practices and perceptions. Psychology in the Schools. 2013;50(2):193–208. [Google Scholar]
  18. Havalchak A, White CR, O'Brien K, Pecora PJ. Casey family programs young adult survey 2006: Examining outcomes for young adults served in out-of-home care. Casey Family Programs; Seattle: 2007. [Google Scholar]
  19. Hersen M, Turner SM. Diagnostic interviewing. New York: Kluwer Academic Publishers. 2003 [Google Scholar]
  20. Jackson Y, Gabrielli J, Fleming K, Tunno AM, Makanui PK. Untangling the relative contribution of maltreatment severity and frequency to type of behavioral outcome in foster youth. Child Abuse & Neglect. 2014;38(7):1147–1159. doi: 10.1016/j.chiabu.2014.01.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Jackson Y, Gabrielli J, Tunno AM, Hambrick EP. Strategies for longitudinal research with youth in foster care: A demonstration of methods, barriers, and innovations. Children and Youth Services Review. 2012;34(7):1208–1213. doi: 10.1016/j.childyouth.2012.02.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. James S, Leslie LK, Hurlburt MS, Slymen DJ, Landsverk J, Davis I, Zhang J. Children in out-of-home care: Entry into intensive or restrictive mental health and residential care placements. Journal of Emotional and Behavioral Disorders. 2006;14(4):196–208. [Google Scholar]
  23. Kaplan S, Pelcovitz D, Labruna V. Child and adolescent abuse and neglect research: A review of the past 10 years. Part 1: Physical and emotional abuse and neglect. Journal of American Academy of Child and Adolescent Psychiatry. 1999;38(10):1214–1221. doi: 10.1097/00004583-199910000-00009. [DOI] [PubMed] [Google Scholar]
  24. Kendall-Tackett KA, Eckenrode J. The effects of neglect on academic achievement and disciplinary problems: A developmental perspective. Child Abuse & Neglect. 1996;20:161–169. doi: 10.1016/s0145-2134(95)00139-5. [DOI] [PubMed] [Google Scholar]
  25. Langberg JM, Epstein JN, Altaye M, Molina BS, Arnold LE, Vitiello B. The transition to middle school is associated with changes in the developmental trajectory of ADHD symptom-atology in young adolescents with ADHD. Journal of Clinical Child & Adolescent Psychology. 2008;37(3):651–663. doi: 10.1080/15374410802148095. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Little TD. Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research. 1997;32(1):53–76. doi: 10.1207/s15327906mbr3201_3. [DOI] [PubMed] [Google Scholar]
  27. Little TD, Cunningham WA, Shahar G, Widaman KF. To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling. 2002;9(2):151–173. [Google Scholar]
  28. Little TD, Slegers DW, Card NA. A non-arbitrary method of identifying and scaling latent variables in SEM and MACS models. Structural Equation Modeling. 2006;13(1):59–72. [Google Scholar]
  29. Little TD, Card NA, Slegers DW, Ledford EC. Representing contextual effects in multiple-group MACS models. In: Little TD, Bovaird JA, Card NA, editors. Modeling ecological and contextual effects in longitudinal studies of human development. Lawrence Erlbaum; Mahwah: 2007. [Google Scholar]
  30. Mac Callum RC, Browne MW, Sugawara HM. Power analysis and determination of sample size for covariance structure modeling. Psychological Methods. 1996;1(2):130–149. [Google Scholar]
  31. McCrae JS, Lee BR, Barth RP, Rauktis ME. Comparing three years of well-being outcomes for youth in group care and nonkinship foster care. Child Welfare. 2010;89:229–250. [PubMed] [Google Scholar]
  32. McNamara JR, Hollmann C, Riegel T. A preliminary study of the usefulness of the behavior assessment systems for children in the evaluation of mental health needs in a head start population. Psychological Reports. 1994;75(3):1195–1201. doi: 10.2466/pr0.1994.75.3.1195. [DOI] [PubMed] [Google Scholar]
  33. Nail JM, Evans JG. The emotional adjustment of gifted adolescents: A view of global functioning. Roeper Review. 1997;20(1):18–21. [Google Scholar]
  34. Oosterman M, Schuengel N, Slot NW, Bullens RAR, Dorleijers TAH. Disruptions in foster care: A review and meta-analysis. Children and Youth Services Review. 2007;29(1):53–76. [Google Scholar]
  35. Park JM, Epstein R, Jordan N, Mandell DS, Lyons JS. Predictors of residential placement following a psychiatric crisis episode among children and youth in state custody. American Journal of Orthopsychiatry. 2009;79(2):228–235. doi: 10.1037/a0015939. [DOI] [PubMed] [Google Scholar]
  36. Petrenko CLM, Friend A, Garrido EF, Taussig HN, Culhane SE. Does subtype matter? Assessing the effects of mal-treatment on functioning in preadolescent youth in out-of-home care. Child Abuse & Neglect. 2012;36(9):633–644. doi: 10.1016/j.chiabu.2012.07.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Pilowsky DJ, Wu LT. Psychiatric symptoms and substance use disorders in a nationally representative sample of American adolescents involved with foster care. Journal of Adolescent Health. 2006;38(4):351–358. doi: 10.1016/j.jadohealth.2005.06.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Reynolds CR, Kamphaus RW. Behavior assessment system for children. 2nd ed. AGS Publishing; Manual. Circle Pines: 2004. [Google Scholar]
  39. Robins PM, Schoff KM, Glutting JJ, Abelkop AS. Discriminative validity of the behavior assessment system for children-parent rating scales in children with recurrent abdominal pain and matched controls. Psychology in the Schools. 2003;40(2):145–154. [Google Scholar]
  40. Steele RG, Little TD, Ilardi SS, Forehand R, Brody GH, Hunter HL. A confirmatory comparison of the factor structure of the children's depression inventory between European American and African American youth. Journal of Child and Family Studies. 2006;15(6):773–788. [Google Scholar]

RESOURCES