Abstract
Objective
To assess the structural overlap between the Behavior Rating Inventory of Executive Function (BRIEF) and Achenbach Child Behavior Check List (CBCL) among children in Uganda.
Methods
Caregiver ratings for the BRIEF and CBCL were obtained for two independent samples of school-aged children: 106 children (5-12 years old, 50% males) with a history of severe malaria, and on 144 HIV-infected children (5-12 years old, 58% males) in Uganda. Exploratory factor analysis was used to evaluate the factor structure of the 8 sub-scales for the BRIEF and the 8 scales of the CBCL to determine correlation.
Results
Overall, children in the severe malaria group had higher (increased symptom) BRIEF and CBCL scores than children in the HIV-infected group. We identified 3 factors that provided a reasonable fit to the data and could be characterized as 3 specific domains: 1) Metacognition, which consisted of the scales in the BRIEF Metacognition domain; 2) Behavioral Adjustment, which was comprised of the scales in the BRIEF Behavioral Regulation domain and the Externalizing Symptoms scales in the CBCL; and 3) Emotional Adjustment, which mainly consisted of the Internalizing Symptoms scales in the CBCL. The BRIEF Behavior Regulation and CBCL Externalizing Symptoms scales, however, did overlap in terms of assessing similar behavior symptoms. These findings were consistent across the severe malaria and HIV-infected samples of children.
Conclusion
The BRIEF and CBCL instruments offer distinct, yet complementary, assessments of behavior in clinical pediatric populations in the Ugandan context, supporting the use of these measures for similar research settings.
Keywords: BRIEF, CBCL, psychological assessment, behavior, children, Sub-Saharan Africa
Increased survival of infectious diseases affecting the central nervous system (CNS) is often associated with corresponding increases in developmental delays, behavior problems, and learning disabilities 1. More specific to the African context, the two major epidemics affecting the pediatric population, HIV/AIDS and malaria, affect a significant proportion of infants, with chronic neurobehavioral deficits resulting in impairments in cognition, and executive and adaptive functioning even when controlling for confounders such as weight for age, and nutritional status 2,3,4
Recent research provides evidence of behavioral problems in children surviving malaria or with HIV. A study looking at several hundred children in Blantyre Malawi suggests that cerebral malaria in children is a significant risk factor for the development of disruptive behavior disorders and disabilities characterized by motor, sensory, or language deficiencies 5,6. HIV-infected school-age children frequently exhibit behavioral problems associated with attention-deficit/hyperactivity disorder (i.e. such as impulsivity, hyperactivity and difficulties concentrating) 1 and affective disorders (i.e. depression and anxiety) 7. Additionally, compromised executive function and slowed information processing have also been reported 8. Such findings underscore the importance of examining behavior and neurocognitive functions, in addition to global cognition in these conditions.
Given the well-documented role of behavioral problems and cognitive impairment on social and academic achievement, it is critical to have specific psychometric tools to evaluate behavioral areas that are known to be affected. For example, Bangirana and colleagues 9 conclude that a Lugandan version of the parental-report version of the CBCL in school-age children is a reliable measure of behavioral problems in Ugandan children surviving cerebral malaria. The CBCL seemed especially sensitive in documenting depressive symptoms and thought problems, as likely behavioral outcomes of cerebral malaria in surviving children 9. The CBCL was also useful in documenting the emotional benefits of computerized cognitive rehabilitation training (CCRT) in Ugandan school-age children surviving cerebral malaria 9.
The Behavior Rating Inventory of Executive Function (BRIEF) 10 and the Achenbach Child Behavior Check List (CBCL) 11 are useful measures to assess behavioral, emotional, social, and functional problems in school age children. The BRIEF was designed to screen for emotional and behavioral aspects of a child's executive functioning as assessed by the responses of the principal caregiver or parent. The Achenbach CBCL was designed to more broadly screen for emotional and psychiatric symptoms in children.
Although the two instruments are often used together in that they are designed to assess essentially different behavioral domains (executive function versus psychiatric symptoms), the extent of overlap between the BRIEF and CBCL is unclear. Extensive use of these assessments through translated versions represents testing opportunities with children of diverse backgrounds, but it remains critical to empirically determine the structure of the scales in the clinical and research settings where they are commonly used.
One way to address the structure of the association of these two scales is by assessing the patterns of correlations with exploratory factor analysis 12. Exploratory factor analysis 12 can be used to empirically test theoretical questions about the underlying structure of psychological phenomena. Factor analyses describe a collection of observed variables in terms of a smaller collection of (unobservable) latent variables or factors. These factors are constructed to explain variation that is common to the observed variables. When EFA is applied to correlations among ratings of scale items, we can obtain patterns of cooccurring items that can be interpreted as dimensions or domains of that scale.
The objective of this study was to assess the structural overlap of the CBCL and BRIEF scales using EFA in two independent clinical samples of Ugandan school-age children: HIV-infected and with history of a severe malaria episode. We hypothesized that the sub-scales comprising the two global domains for each of the two instruments will load into separate factors, in accordance with the general dimensions of emotional versus behavioral adjustment problems.
METHODS
This secondary analysis included baseline data from two samples of children aged 5 to 12 years. The use of two samples was intended to provide an opportunity to see if the overlap of CBCL and BRIEF scales was similar across two independent groups of children.
The fist sample of children in this analysis included the baseline data of 106 children with a history of severe malaria that were enrolled in a randomized-controlled trial exploring the effects of a Computerized Cognitive Rehabilitation Training (CCRT) program. Children surviving a severe malaria episode (either severe malaria anemia or cerebral malaria) were recruited between 2010 and 2013 from a cohort of severe malaria study children admitted at Mulago Hospital in Kampala, Uganda. Clinical characteristics of the children are described fully by Bangirana and colleagues 13 and were enrolled in the CCRT study after completing a two-year post-illness follow-up and if they were 5 years of age or older. Cerebral malaria was defined as: 1) coma (Blantyre Coma Score ≤ 2 or Glasgow coma scale score ≤8) 2) Plasmodium falciparum on blood smear; and 3) no other known cause of coma (e.g. hypoglycemia-associated coma reversed by glucose infusion, meningitis, or prolonged postictal state. Severe malaria anemia was defined as presence of Plasmodium falciparum on blood smear in children with hemoglobin level <5mg/dL. Exclusion criteria included known chronic illness requiring medical care, known developmental delay, evidence of central nervous system disease at screening physical examination or history of coma, head trauma, malnutrition, or cerebral palsy.
The second sample included 144 HIV-infected children enrolled for a feasibility study of a caregiver training program to enhance child development (Mediational Intervention for Sensitizing Caregivers - MISC) carried out in Kayunga district (80km northeast of Kampala). Children who were perinatally-infected and confirmed as HIV-positive with Western Blot and ELISA tests were evaluated and enrolled from 2010 to 2013. Children were excluded from the MISC study if they had a medical history of serious birth complications, severe malnutrition, bacterial meningitis, encephalitis, cerebral malaria or other known brain injury or disorder requiring hospitalization or which could overshadow the developmental benefits of the parenting intervention. At the time of the MISC study, 63 children (44%) were on triple-combination anti-retroviral therapy (Trimune: d4T/3TC/nevirapine). Details on the immunological profile of the sample can be found elsewhere26.
Procedures
Written consent was obtained from the parent/guardian and assent from children seven years and older. After administering informed consent, child testing and caregiver questionnaires were done in Luganda, the local language spoken in Kampala and Kayunga districts in a private, quiet setting in the project's office. The Institutional Review Boards of Michigan State University, University of Michigan, and the School of Medicine Research Ethics Committee at Makerere University, and the Ugandan National Council for Science and Technology approved the severe malaria and pediatric HIV CCRT studies.
Measures
Child Behavior Checklist (CBCL)
The CBCL is a paper-pencil parent/caregiver report on child behavior consisting of 120 items scored on a three-point Likert scale (0=absent, 1=occurs sometimes, 2=occurs often). The time frame for the item responses is the past six months. The instrument is organized in 8 syndrome scales (Anxious/Depressed, Depressed, Somatic Complaints, Social Problems, Attention Problems, Thought Problems, Rule-breaking Behavior, Aggressive Behavior) that group into two higher order factors- Internalizing and Externalizing Problems, or into one summary score; Total Problems (summation of all items). The 2014 revision also added 6 DSM-V oriented scales consistent with DSM diagnostic categories: Depressive Problems, Anxiety Problems, Somatic Problems, ADHD, Oppositional Defiant Problems, and Conduct Problems. It has been widely used as rating scale in different contexts 14 including Ugandan children. The Luganda version of the CBCL was previously translated and adapted by Bangirana and colleagues (2009) in a two-step process. First the CBCL was translated and back-translated by two different research assistants fluent in Luganda and English. Second, a psychiatrist fluent in both Luganda and English compared the two English versions and resolved any discrepancies by editing the translated version to match the original English version. The test-retest and internal reliability of the CBCL in Ugandan children was assessed and reported between 0.64 and 0.83, respectively 9.
Behavior Rating Inventory of Executive Function (BRIEF)
Caregivers were also interviewed with the Behavior Rating Inventory of Executive Function (BRIEF) for school-age children (5-12 years). This instrument is specifically designed to measure the range of executive function behaviors in children and adolescents with 86-items in 8 non-overlapping clinical scales (Inhibit, Shift, Emotional Control, Initiate, Working Memory, Plan/Organize, Organization of Materials, and Monitor). The BRIEF is divided into two broad indexes, each of which is derived from specific clinical scales: Meta-Cognition (Monitor, Organization of Materials, Plan/Organize, Working Memory, Initiate) and Behavioral Regulation (Emotional Control, Shift, Inhibit), and a Global Executive Composite score which takes into account all clinical scales and represents the child's overall executive function 10. Publisher copyright permission was obtained for the BRIEF and was translated into Luganda, the local language (forward and backward translation) as specified by the publisher (PAR, Inc.), and the final version was approved by one of the test authors (Peter Isquith).
The BRIEF was added as measure for these studies because it should prove sensitive to the more pervasive executive function and higher-order neurocognitive disabilities that typify the encephalopathy associated with even clinically stable pediatric HIV disease15,16 Further, this test should be well suited to detect more subtle brain injury and executive function deficits if they occur among ARV-exposed infants.
Statistical analyses
Demographic characteristics and symptom endorsement frequencies were calculated for both samples and comparisons were done contrasting means and proportions using t-tests and χ2 tests, respectively. Internal consistency of the BRIEF and CBCL scales were evaluated using Cronbach's α coefficient 17.
We used EFA with principal factor estimation to explore the factor structure of the BRIEF with 8 sub-scales included the two global domains (Behavioral Regulation Index, Metacognition Index), with either: a) the 8 sub-scales comprising the two principal problem (syndrome) domains of the CBCL (Internalizing and Externalizing symptoms), or b) the 6 DSM-V oriented sub-scales (Depressive, Anxiety, Somatic, ADHD, Oppositional/Defiant, and Conduct Problems) of the CBCL in the HIV-infected and malaria survivor samples, separately.
Because outcomes of EFAs depend heavily on the number of factors retained and there are no established criteria to determine this procedure, a number of decision rules were applied examining several solutions before coming to a final conclusion 18. The decision rules we used included assessing the eigenvalues (selecting factors with a value close to 1), graphically representing the eigenvalues to visually analyze the relative importance of the factors, where a sharp drop in the plot signals that subsequent factors are ignorable (scree plot), and a parallel analysis (comparing the number of real eigenvalues that are greater than the corresponding expected values from random data). We performed orthogonal transformations of the common factors and associated factor loadings to facilitate the interpretation of the factor analytical solution 19. All analyses were performed in STATA version 12 20.
RESULTS
Table 1 summarizes the distributions of the demographic characteristics of children included in the study and their neuropsychological scores from the BRIEF and CBCL. Children in the severe malaria group were younger (p<0.01) and had a slightly larger proportion of males (61%) compared to the HIV-infected children (p=0.08). Overall, children in the severe malaria group had higher (increased symptom) BRIEF and CBCL scores than children in the HIV-infected group.
Table 1.
Demographic characteristics, BRIEF and CBCL scales scores across HIV+ (N=144) and survivors of severe malaria (N=106) children, Uganda.
| Characteristic | HIV+ (N=144) | Severe malaria (N=106) | p-value* |
|---|---|---|---|
| Gender, n (%) | |||
| Males | 72 (50) | 65 (61) | 0.08 |
| Females | 72 (50) | 41 (39) | |
| Age, mean (range), yrs | 8.9 (6-12.7) | 7 (5-12.8) | <0.01 |
| BRIEF T score, M (SD) | |||
| Behavioral Regulation Index | 46 (9.3) | 58.1 (12.3) | < 0.01 |
| Metacognition Index | 48.8 (10.1) | 56.3 (12) | < 0.01 |
| Global Executive composite | 47.9 (10.3) | 58 (12.5) | < 0.01 |
| CBCL | |||
| Syndrome scales T score, M (SD) | |||
| Internalizing | 60.9 (9.2) | 62.8 (7.7) | 0.08 |
| Externalizing | 58.9 (8.1) | 62.8 (9.3) | < 0.01 |
| DSM-Vderived scales T score, M (SD) | |||
| Depressive Problems | 61.5 (5.1) | 61 (7.3) | 0.52 |
| Anxiety Problems | 52.4 (3.9) | 55.3 (6.5) | < 0.01 |
| Somatic Problems | 61.1 (8.3) | 58.1 (6.6) | < 0.01 |
| ADHD Problems | 55.2 (6.7) | 57.2 (7.2) | 0.02 |
| Oppositional defiant Problems | 56.1 (7.2) | 56.3 (10.7) | 0.86 |
| Conduct Problems | 58 (8.6) | 63.1 (9.9) | < 0.01 |
| Total problems T score, M (SD) | 46.1 (25) | 47.2 (27) | 0.74 |
p-value for the difference in estimate comparing HIV+ with severe malaria children
Bold type indicates significance <0.05
The reliability coefficient (Cronbach's alpha) of the BRIEF was high in both the severe malaria group (α=0.90) and HIV+ group (α=0.88). Internal consistency analyses of the CBCL showed similar Cronbach's alpha values for children surviving severe malaria (0.93) and children infected with HIV (0.90).
EFA combining the 8 BRIEF sub-scales and 8 CBCL sub-scales in each clinical sample separately resulted in 3 factors composed of CBCL and BRIEF sub-scales that could be characterized as: Metacognition, Behavioral Adjustment, and Emotional Adjustment. In the severe malaria group, these 3 factors jointly accounted for 97% of the variance in the analysis combining the BRIEF with the CBCL syndrome scales, and 93% of the variance when analyzing the BRIEF together with the CBCL DSM-IV scales. Among children HIV+, total variance explained by the 3 factors was lower in the model with the CBCL syndrome scales (63%), but comparable when using the CBCL DSM-IV scales (95%).
The relationship of each of the CBCL and BRIEF sub-scales to the underlying factors is expressed by the factor loading. As shown in Tables 2 and 3, factor loadings of a sub-scale assigned to a specific factor generally exceeded 0.40 (except for Withdrawn/Depressed, 0.38, and Somatic Complaints, 0.30, of the CBCL). Since factor loadings can be interpreted as standardized regression coefficients, another interpretation would be that most of the scales from the BRIEF and the CBCL displayed a strong correlation (>0.40) with their corresponding factor. The composition of these 3 factors was comparable across the two samples. The BRIEF Metacognition scales (i.e. Initiate, Working Memory, Plan/Organize, Organization of Materials, and Monitor) and the CBCL scales pertaining to Internalizing Symptoms (Emotional Adjustment- i.e. Anxious/Depressed, Withdrawn/Depressed, Somatic Complaints, Social Problems) were correlated with different factors (Metacognition and Emotional Adjustment, respectively) across both the HIV-infected and severe malaria clinical samples. The BRIEF Behavior Regulation Index sub-scales (i.e. Inhibit and Emotional Control) loaded with the CBCL Externalizing sub-scales (i.e. Rule-Breaking and Aggressive Behavior) into the Behavioral Adjustment factor in the HIV-infected group. For the severe malaria group, the BRIEF Behavior Regulation Index sub-scales (i.e. Shift and Emotional Control) loaded together with CBCL Internalizing Problems (Anxious/Depressed and Somatic Complaints) in the Emotional Adjustment factor.
Table 2.
Factor loadings of the BRIEF and CBCL (syndromes scales) across HIV+ (N=144) and severe malaria (N=106) school-aged children in Uganda.
| Metacognition | Behavioral adjustment | Emotional adjustment | ||||
|---|---|---|---|---|---|---|
| HIV+ | Malaria | HIV+ | Malaria | HIV+ | Malaria | |
| BRIEF Scales | ||||||
| Behavioral Regulation Index | ||||||
| Inhibit | 0.36 | 0.57 | 0.72 | 0.44 | 0.20 | 0.38 |
| Shift | 0.30 | 0.28 | 0.19 | 0.20 | 0.53 | 0.74 |
| Emotional Control | 0.35 | 0.36 | 0.50 | 0.24 | 0.49 | 0.69 |
| Metacognition Index | ||||||
| Initiate | 0.57 | 0.63 | 0.39 | 0.38 | 0.09 | 0.22 |
| Working Memory | 0.66 | 0.74 | 0.26 | 0.34 | 0.22 | 0.22 |
| Plan/Organize | 0.75 | 0.75 | 0.21 | 0.09 | 0.26 | 0.32 |
| Organize Materials | 0.65 | 0.59 | 0.30 | 0.20 | 0.10 | 0.25 |
| Monitor | 0.73 | 0.77 | 0.30 | 0.17 | 0.07 | 0.14 |
| CBCL Problem/Syndrome Scales | ||||||
| Internalizing problems | ||||||
| Anxious/Depressed | 0.09 | 0.20 | 0.18 | 0.49 | 0.73 | 0.68 |
| Withdrawn/Depressed | 0.17 | 0.38 | 0.17 | 0.34 | 0.63 | 0.25 |
| Somatic Complaints | 0.26 | 0.13 | 0.21 | 0.27 | 0.56 | 0.30 |
| Externalizing problems | ||||||
| Rule-breaking | 0.27 | 0.25 | 0.63 | 0.80 | 0.12 | 0.09 |
| Aggressive Behavior | 0.23 | 0.29 | 0.77 | 0.70 | 0.27 | 0.45 |
| Social | 0.13 | 0.42 | 0.41 | 0.54 | 0.60 | 0.46 |
| Thought | 0.37 | 0.01 | 0.46 | 0.73 | 0.20 | 0.39 |
| Attention | 0.44 | 0.48 | 0.56 | 0.70 | 0.24 | 0.18 |
*Bold type shows loading with which scale was assigned to a specific factor
Table 3.
Factor loadings of the BRIEF and CBCL (DSM-IV scales) across HIV+ (N=144) and severe malaria (N=106) school-aged children in Uganda.
| Metacognition | Behavioral adjustment | Emotional adjustment | ||||
|---|---|---|---|---|---|---|
| HIV+ | Malaria | HIV+ | Malaria | HIV+ | Malaria | |
| BRIEF Scales | ||||||
| Behavioral Regulation Index | ||||||
| Inhibit | 0.34 | 0.54 | 0.74 | 0.49 | 0.21 | 0.38 |
| Shift | 0.25 | 0.28 | 0.28 | 0.23 | 0.54 | 0.69 |
| Emotional Control | 0.31 | 0.32 | 0.56 | 0.29 | 0.48 | 0.70 |
| Metacognition Index | ||||||
| Initiate | 0.54 | 0.58 | 0.44 | 0.45 | 0.07 | 0.24 |
| Working Memory | 0.65 | 0.71 | 0.25 | 0.36 | 0.24 | 0.27 |
| Plan/Organize | 0.76 | 0.73 | 0.20 | 0.15 | 0.24 | 0.34 |
| Organize Materials | 0.66 | 0.57 | 0.30 | 0.34 | 0.14 | 0.24 |
| Monitor | 0.73 | 0.74 | 0.32 | 0.24 | 0.08 | 0.13 |
| CBCL DSM-IV Scales | ||||||
| Affective Problems | 0.32 | 0.29 | 0.21 | 0.53 | 0.63 | 0.36 |
| Anxiety Problems | 0.01 | 0.30 | 0.36 | 0.54 | 0.50 | 0.60 |
| Somatic Problems | 0.18 | 0.08 | 0.06 | 0.13 | 0.49 | 0.34 |
| ADHD Problems | 0.38 | 0.48 | 0.55 | 0.67 | 0.18 | 0.32 |
| Oppositional Defiant Problems | 0.17 | 0.24 | 0.74 | 0.71 | 0.16 | 0.21 |
| Conduct Problems | 0.30 | 0.21 | 0.73 | 0.77 | 0.11 | 0.18 |
In both HIV-infected and severe malaria groups, the DSM-oriented scales from the CBCL correlated with two factors; the Behavioral adjustment factor (i.e. ADHD Problems, Oppositional Defiant Problems, Conduct Problems), and the Emotional adjustment factor (i.e. Affective, Anxiety, Somatic Problems).
DISCUSSION
The BRIEF and the CBCL are parental-report measures of child behavior frequently used in research settings. Because the BRIEF was designed to screen for emotional and behavioral symptoms related to a child's executive functioning and the Achenbach CBCL assesses more general emotional and behavioral psychiatric symptoms, these measures are frequently used together. Our results suggest that both these instruments assess generally distinct dimensions of children's behavioral adaptation in the home and community in a consistent manner between two distinct clinical groups in the Sub-Saharan African context (HIV-infected and severe malaria survivors).
Overall, children in the severe malaria group had higher (increased symptom) BRIEF and CBCL scores than children in the HIV-infected group. Other studies have reported the emergence of behavior problems in children surviving cerebral malaria 6. More recently, the severity of CBCL behavioral symptoms in Malawian cerebral malaria survivors has been related to severity of clinical indicators of illness during the acute phase 21.
Furthermore, ADHD-type symptoms have been documented with computerized continuous performance task measures in Ugandan cerebral malaria survivors at 6-month and two-year post-illness follow-up 2,22. However, recent reviews of children with HIV have noted that as they emerge from middle childhood into adolescence, they are at higher risk for behavioral and psychiatric problems 4.
Across both severe malaria and HIV-infected samples, item loadings for each factor were substantial and each item had a substantial correlation with its factor (i.e. essentially above 0.4), suggesting that items were relevant in defining the factor's dimensionality. Lowest item loadings corresponded to the Withdrawn/Depressed (0.38) and Somatic Complaints (0.30) of the CBCL syndrome scales, and Somatic Problems (0.35) of the CBCL with DSM-V scales. In a review of the factor structure of the CBCL across 31 cultural settings, item loadings were more different for internalizing problems (i.e. Withdrawn/Depressed) and somatic complaints than on loadings for externalizing problems 23.
The 3 dimensions derived form factor analyses make theoretical sense based on previous findings on the factorial structure of the BRIEF 24 and the CBCL 23; all factors could be reasonably named, and we were able to describe the items as indicative of their underlying factor. The model structure included Behavioral Regulation, Emotional Regulation, and Metacognition factors, supporting a 3-dimesional model as the most appropriate structure for the 9-scale BRIEF. It is possible that item redundancy could be one of the reasons why the factor structure holds together; the strength of the relationship could be due in part to the multiple scales correlating closely because they are responded in a similar way.
Our results are in support of a distinction between emotional and behavioral regulation domains of the BRIEF, as others have previously described 24,25. In this view, our findings coincide with the notion that executive functions can be conceptualized as a collection of related, although distinct, abilities that direct and control goal-oriented tasks with behavioral, cognitive and emotional components. With these results, we are the first to report on the dimensional structure of the BRIEF in a Sub-Saharan setting and in two clinical populations.
Concerns have been raised over the ability of the DSM-Oriented Scales of the CBCL to adequately measure DSM-oriented constructs, especially because of the few items included in the measure 26. Findings from our study indicate that the DSM-oriented scales assessing overt behavioral problems (i.e. ADHD problems, Oppositional Defiant Problems) have a common underlying factor pertaining to Behavioral Adjustment, while those assessing emotional problems (i.e. Affective Problems, Anxiety Problems) group together under the Emotional Adjustment factor, providing preliminary evidence towards the psychometric properties of the DSM Scales. Additionally, the variance explained was comparable between analysis with the CBCL syndrome scales and the CBCL DSM-oriented scales. Further empirical scrutiny of the psychometric properties of the CBCL should be sought in future research.
In our EFA analyses, the Metacognition factor was entirely defined by BRIEF scales (Initiate, Working Memory, Plan/Organize, Organization of Materials, and Monitor), while the Behavioral Adjustment factor was composed of CBCL scales (i.e. Externalizing Problems from the syndrome scale and ADHD, Oppositional Defiant, and Conduct Problems from the DSM-IV derived scales). The BRIEF Behavior Regulation Index domain had scales that overlap with both the emotional and behavioral psychiatric symptom domains of the CBCL, represented in the Behavioral and Emotional Adjustment factors.
However, the overall picture from EFA suggest that the BRIEF and CBCL assess distinct dimensions of behavioral and emotional problems in school age children with infectious diseases that can have profound neurocognitive and psychiatric implications. This suggests that when used together, the BRIEF and the CBCL can provide for a distinct yet complementary emotional and behavioral profile of the adjustment of these at-risk children.
While the present findings provide insight to the extent of overlap between the BRIEF and CBCL, several limitations should be acknowledged. An inherent limitation of self-report measures is the tendency of participants to respond the same way to similarly worded items, also termed method variance. This may influence how items aggregate in construct factors. However, this effect is likely to have been comparable to other studies since we found similar results in the factor structure. In a recent report submitted by our group for publication, we have also documented that maternal depression can be significantly related to higher reported levels of behavior problems on the BRIEF scale for preschool-age Ugandan children with HIV. This same relationship was not significant in the HIV-exposed/non-infected cohort of children and caregivers 27. Likewise, Boivin and colleagues 28 have reported higher levels of CBCL internalizing symptoms in Ugandan preschool children with HIV, reported by mothers with higher levels of depression on the Hopkins Symptoms Checklist. The relationship between caregiver depression and elevated symptoms reported on the BRIEF and on the CBCL in HIV-infected children could seriously bias the sensitivity and validity of these measures when used to compare individual scores. Because factor analysis’ emphasis is on the patterns of correlations among items, this analytical technique is less influenced by a potential response bias related to maternal psychopathology.
Despite these limitations, findings from the analysis presented here demonstrating the cross-cultural consistency of the factor structure, suggest that the BRIEF and the CBCL can be adapted for use to assess behavioral and emotional problems in children in urban and rural areas of Uganda. What partly motivated this analysis were comments from caregivers remarking the redundancy of questions when assessed with the BRIEF followed by the CBCL. Since results suggest that both these instruments can be beneficial in a research context because they assess distinct dimensions of children's emotional and behavioral adaptation in a consistent manner, we recommend that they not be applied one after the other to reduce the possibility of acquiescence or uniform response bias.
We conclude that both the BRIEF and CBCL are useful as multi-dimensional measures of executive function and behavior problems, respectively, among clinical samples of school-age children in Uganda. Based on their structure and internal consistency, the dimensional models presented here coincide with theoretical models of executive function and behavioral problems in children and support the use of these measures in research contexts. Future studies in children with psychiatric diagnoses are needed to document the clinical validity of the BRIEF and CBCL in Uganda.
Acknowledgments
We would like to thank all our team members at the Kayunga (Sylvia, Agatha, Janet, Madiina, Agatha, Betty, Peter, Ahmed, Godfrey) and Mulago (Monica, Titus, Stella Butalla, Stella Kibuuka, Mariah, Michael Seviiri, Michael Sengendo) research sites as well as the children and their caregivers for participating in these studies.
Funding
This work was supported by the National Institute of Child Health and Development and the National Institute of Mental Health (grant numbers R01HD064416, R34MH084782). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funding source had no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.
Footnotes
I (we) certify that there is no conflict of interest with any financial organization regarding the material discussed in the manuscript.
Contributor Information
Familiar Itziar, Department of Psychiatry, Michigan State University.
Ruisenor-Escudero Horacio, Department of Psychiatry, Michigan State University.
Giordani Bruno, Departments of Psychiatry Neurology, and Psychology, University of Michigan.
Bangirana Paul, Department of Psychiatry, Makerere University College of Health Sciences.
Nakasujja Noeline, Department of Psychiatry, Makerere University College of Health Sciences.
Opoka Robert, Department of Pediatrics and Child Health, Makerere University College of Health Sciences, Makerere University.
Boivin Michael, Department of Psychiatry, Michigan State University.
References
- 1.Mellins CA, Elkington KS, Leu C-S, et al. Prevalence and change in psychiatric disorders among perinatally HIV-infected and HIV-exposed youth. AIDS care. 2012;24(8):953–962. doi: 10.1080/09540121.2012.668174. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Boivin MJ, Bangirana P, Byarugaba J, et al. Cognitive impairment after cerebral malaria in children: a prospective study. Pediatrics. 2007 Feb;119(2):e360–366. doi: 10.1542/peds.2006-2027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Grantham-McGregor S, Cheung YB, Cueto S, Glewwe P, Richter L, Strupp B. Developmental potential in the first 5 years for children in developing countries. Lancet. 2007 Jan 6;369(9555):60–70. doi: 10.1016/S0140-6736(07)60032-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Laughton B, Cornell M, Boivin M, Van Rie A. Neurodevelopment in perinatally HIV-infected children: a concern for adolescence. Journal of the International AIDS Society. 2013;16:18603. doi: 10.7448/IAS.16.1.18603. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Idro R, Kakooza-Mwesige A, Balyejjussa S, et al. Severe neurological sequelae and behaviour problems after cerebral malaria in Ugandan children. BMC research notes. 2010;3:104. doi: 10.1186/1756-0500-3-104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Birbeck GL, Molyneux ME, Kaplan PW, et al. Blantyre Malaria Project Epilepsy Study (BMPES) of neurological outcomes in retinopathy-positive paediatric cerebral malaria survivors: a prospective cohort study. Lancet neurology. 2010 Dec;9(12):1173–1181. doi: 10.1016/S1474-4422(10)70270-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Musisi S, Kinyanda E. Emotional and behavioural disorders in HIV seropositive adolescents in urban Uganda. East African medical journal. 2009 Jan;86(1):16–24. doi: 10.4314/eamj.v86i1.46923. [DOI] [PubMed] [Google Scholar]
- 8.Wachsler-Felder JL, Golden CJ. Neuropsychological consequences of HIV in children: a review of current literature. Clinical psychology review. 2002 Apr;22(3):443–464. doi: 10.1016/s0272-7358(01)00108-8. [DOI] [PubMed] [Google Scholar]
- 9.Bangirana P, Nakasujja N, Giordani B, Opoka RO, John CC, Boivin MJ. Reliability of the Luganda version of the Child Behaviour Checklist in measuring behavioural problems after cerebral malaria. Child and adolescent psychiatry and mental health. 2009;3:38. doi: 10.1186/1753-2000-3-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Gioia GA, Isquith PK, Guy SC, Kenworthy L. Test review behavior rating inventory of executive function. Child Neuropsychology. 2000;6(3):235–238. doi: 10.1076/chin.6.3.235.3152. [DOI] [PubMed] [Google Scholar]
- 11.Achenbach TM. Integrative Guide to the 1991 CBCL/4-18, YSR, and TRF Profiles. University of Vermont, Department of Psychology; Burlington, VT: 1991. [Google Scholar]
- 12.Bonniface M, Nguefack S, Zeh O, et al. Computed tomography findings in cerebral palsy in Yaoundé–Cameroon. Journal Africain d'Imagerie Médicale. 2014;5(3) [Google Scholar]
- 13.Bangirana P, Opoka RO, Boivin MJ, et al. Severe Malarial Anemia is Associated With Long-term Neurocognitive Impairment. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America. 2014 Aug 1;59(3):336–344. doi: 10.1093/cid/ciu293. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Ivanova MY, Dobrean A, Dopfner M, et al. Testing the 8-syndrome structure of the child behavior checklist in 30 societies. Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53. 2007 Jul-Sep;36(3):405–417. doi: 10.1080/15374410701444363. [DOI] [PubMed] [Google Scholar]
- 15.Boivin MJ, Giordani B. Neuropsychological assessment of African children: Evidence for a universal basis to cognitive ability. In: Chiao JY, editor. Cultural Neuroscience: Cultural Influences on Brain Function. Vol. 178. Elsevier Publications; New York, NY: 2009. pp. 113–135. [DOI] [PubMed] [Google Scholar]
- 16.Boivin MJ, Green SD, Davies AG, Giordani B, Mokili JK, Cutting WA. A preliminary evaluation of the cognitive and motor effects of pediatric HIV infection in Zairian children. Health Psychol. 1995 Jan;14(1):13–21. doi: 10.1037//0278-6133.14.1.13. [DOI] [PubMed] [Google Scholar]
- 17.Cronbach LJ. Coefficient alpha and the internal structure of tests. psychometrika. 1951;16(3):297–334. [Google Scholar]
- 18.Ford JK, MacCallum RC, Tait M. The application of exploratory factor analysis in applied psychology: A critical review and analysis. Personnel Psychology. 1986;39(2):291–314. [Google Scholar]
- 19.Thurstone LL. The Vectors of Mind: Multiple-Factor Analysis for the Isolation of Primary Traits. University of Chicago Press; Chicago, Il: 1935. [Google Scholar]
- 20.Stata [computer program]. Version 12. College Station, TX: 2012. [Google Scholar]
- 21.Boivin MJ, Vokhiwa M, Sikorskii A, Magen JG, Beare NA. Cerebral Malaria Retinopathy Predictors of Persisting Neurocognitive Outcomes in Malawian Children. The Pediatric infectious disease journal. 2014 Apr 22; doi: 10.1097/INF.0000000000000296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.John CC, Bangirana P, Byarugaba J, et al. Cerebral malaria in children is associated with long-term cognitive impairment. Pediatrics. 2008 Jul;122(1):e92–99. doi: 10.1542/peds.2007-3709. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Rescorla L, Achenbach T, Ivanova MY, et al. Behavioral and emotional problems reported by parents of children ages 6 to 16 in 31 societies. Journal of Emotional and behavioral Disorders. 2007;15(3):130–142. [Google Scholar]
- 24.Gioia GA, Isquith PK, Retzlaff PD, Espy KA. Confirmatory factor analysis of the Behavior Rating Inventory of Executive Function (BRIEF) in a clinical sample. Child neuropsychology : a journal on normal and abnormal development in childhood and adolescence. 2002 Dec;8(4):249–257. doi: 10.1076/chin.8.4.249.13513. [DOI] [PubMed] [Google Scholar]
- 25.Egeland J, Fallmyr O. Confirmatory Factor Analysis of the Behavior Rating Inventory of Executive Function (BRIEF): support for a distinction between emotional and behavioral regulation. Child neuropsychology : a journal on normal and abnormal development in childhood and adolescence. 2010;16(4):326–337. doi: 10.1080/09297041003601462. [DOI] [PubMed] [Google Scholar]
- 26.Kendall PC, Puliafico AC, Barmish AJ, Choudhury MS, Henin A, Treadwell KS. Assessing anxiety with the Child Behavior Checklist and the Teacher Report Form. Journal of anxiety disorders. 2007;21(8):1004–1015. doi: 10.1016/j.janxdis.2006.10.012. [DOI] [PubMed] [Google Scholar]
- 27.Familiar IN,N, Bass J, Sikorskii A, Murray S, Ruisenor-Escudero H, Bangirana P, Opoka RO, Boivin MJ. Caregivers’ depressive symptoms and parent-report of child executive function among young children in Uganda. 2014 doi: 10.1016/j.lindif.2015.01.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Boivin MJ, Bangirana P, Nakasujja N, et al. A year-long caregiver training program improves cognition in preschool Ugandan children with human immunodeficiency virus. The Journal of pediatrics. 2013 Nov;163(5):1409–1416. e1401–1405. doi: 10.1016/j.jpeds.2013.06.055. [DOI] [PMC free article] [PubMed] [Google Scholar]
