Screening Child Social-emotional and Behavioral Functioning in Low-Income African Country Contexts

Abstract

Background

children in low-income countries (LICs). Currently, there is little information available on the use of brief screening instruments Increased attention is being paid to identifying and responding to the social-emotional and behavioral needs of in LICs. The lack of psychometrically sound brief assessment tools creates a challenge in determining the population prevalence of child social-emotional and behavioral risk burden in Sub-Saharan African (SSA) country contexts. This study sought to determine the reliability and validity of three brief parent-rated screening tools-the Social Competence Scale (SCS), Pictorial Pediatric Symptom Checklist (PPSC), and the Strengths and Difficulties Questionnaire (SDQ)-in Uganda. These tools consider both strength- and pathology-based dimensions of child outcomes.

Methods

Parents of 154 Ugandan 5–9 year-old children who were enrolled in Nursery to Primary 3 in Kampala (the capital city of Uganda) and part of a school-based mental health intervention trial were recruited and interviewed. About 54% of parents had educational attainment of primary school level or less. One hundred and one of these parents were interviewed a second time, about 5 months after the first/baseline assessment. Data from both time points were utilized to assess reliability and validity.

Results

Inspection of psychometric properties supports the utility of these three brief screening measures to assess children’s social-emotional and behavioral functioning as demonstrated by adequate internal consistency, temporal stability, discriminant validity, concurrent validity, and predictive validity. Subscales from three screening measures were inter-related and associated with family characteristics, such as parental depression and food insecurity, in the expected directions.

Conclusion

This study provides evidence supporting the appropriateness of using three tools and applying the developmental and behavioral constructs measured in each assessment in a low-income African setting.

Introduction

Children’s early social-emotional and behavioral functioning are important predictors of later social adjustment and psychopathology [1–4]. Greater social competence, emotion regulation, and behavioral functioning promotes better child development in a number of domains, including better academic achievement [5,6], social adjustment [7–9], and interpersonal relationships [10–12]. Therefore, systematic routine behavioral screening for early detection of social-emotional dysfunction in school, community, and pediatric primary care settings will allow for early intervention and improved child outcomes [13,14].

In low-income countries (LICs), there is increasing awareness of the importance of social-emotional development to adjustment and school performance during early childhood [15–17]. There is an increased need for measures of early social-emotional and behavioral functioning that are brief, easy-to-use, and have utility in assessing these constructs among children. However, the use of screening for assessing behavioral adjustment in LICs faces several challenges. On one hand, communities in LICs may lack linguistically appropriate tools and adequately trained professionals to carry out screening assessments [18]. On the other hand, whether behavior problems derived from high-income countries is relevant to LIC contexts remains questionable [19]. Among measurement validation studies, most tools that have been validated in LICs are focused on older children (e.g., middle childhood, adolescents) [18,20], or long forms of standardized assessment tools (e.g., Child Behavior Checklist/CBCL), which tend to take long and may not be cost-efficient for population screening [21]. Therefore, linguistically appropriate early childhood screening tools that capture culturally relevant social-emotional and behavioral constructs and are easily implemented in LIC settings are immensely needed.

The goal of this study was to evaluate the utility of three brief parent-rated measures-Social Competence Scale [22,23], Pictorial Pediatric Symptom Checklist [14,24–26], and Strengths and Difficulties Questionnaire [3,27]-in assessing preschool- and school-aged children’s social-emotional and behavioral functioning. These scales were chosen because they have been applied in diverse populations (including parents with low literacy rates) [23,25,26,28,29], and are considerably shorter than other standardized behavioral measures. This makes them especially attractive for use as screening tools in educational, clinical, or research settings in LICs. In addition, these measures capture important skills and problem behaviors in children, which have been shown to predict later adjustment [23,30,31].

The Social Competence Scale (SCS) has been demonstrated to be a psychometrically sound measure of social competence for children in preschool through second grade [22,23,32]. The SCS assesses two major domains of social competence-prosocial skills/communication and emotion regulation [22,23,32]. It has previously been used in large prevention studies with Head Start children [33] and with community samples in the United States (US) [23].

The Pediatric Symptom Checklist (PSC) has been shown to be a reliable and valid measure for assessing children’s externalizing, internalizing, and attention problems [14,24]. The scale has been validated for use in in primary care settings for children ages 4 to 16 years old from diverse backgrounds in primary care settings [14,24,34–36]. A pictorial version of the PSC (PPSC) was developed in recent years that contains pictorial descriptions in addition to written text [25,26]. The add-on visual feature increases detection and understanding of the questions without a need to change the content, which is highly relevant for low-income communities/countries because of the low-literacy rates in these settings [37]. The PPSC has been validated in English [38], Spanish [25,26] and Filipino versions [39] with preschool and school-aged children, but has not been used in LICs or in Africa.

The Strengths and Difficulties Questionnaire-Parent version (SDQ) [3,27] is a brief screening questionnaire for child behavioral functioning. SDQ is a widely used and reliable self-administered psychopathology screening for children aged 3–17 years old. The SDQ contains five subscales: emotional problems, hyperactivity and inattention, conduct problems, peer relationship difficulties, and prosocial behaviors. It has been translated in more than 80 languages, and is widely used by researchers, clinicians, and educators in the US and other high- and middle-income countries [40,41]. Only a handful of studies have assessed the psychometric properties of the instrument in low-income countries [28,29], and only one in sub-Saharan Africa [42].

This paper builds on this existing literature to test the utility of these three screening measures in the Ugandan context. We assessed the factor structure, reliability, and validity of the three brief social-emotional and behavioral screening measures. For validation purpose, family functioning measures such as parental depression, social support, and family food insecurity (defined as any food resource or hunger issue that occurs in families) were selected as validity criteria. These variables were chosen because previous research has documented that high parental negative affect (e.g., psychological well-being, stress), low family social support, and living in poverty (e.g., food insecurity) are associated with ineffective parenting, poor quality of parent-child relationship, and higher level of problem behavior in children [43–47]. In addition, caregivers’ depression has been found to be related to poor child social competence and high adjustment problems [48–53].

Our study contributes to psychometric property testing for identifying appropriate tools to evaluate African children’s social- emotional and behavioral functioning. The findings also contribute to a better understanding of the application of developmental and behavioral constructs developed in high-income countries to African settings. Given cultural differences in adult expectations regarding developmental tasks for children, we cannot assume that such constructs would manifest the same way cross-culturally. Our study adds new evidence for studying child development in African settings.

Methods

Participants

Study participants were 154 Ugandan parents of children attending primary schools. The study sample was recruited as part of a school-based mental health intervention trial that aimed to improve teachers’ utilization of evidence-based behavioral management strategies in classrooms to promote child mental health [54]. Ten schools were recruited (5 randomly assigned to intervention and the other 5 schools to control). Teachers (not parents) were the target of intervention. The intervention last 4 months (including a 5-day training and 13 weekly coaching sessions for teachers). As part of the evaluation, 10–15 parents were randomly selected/recruited from each school to participate in an assessment to provide data on child outcomes. Parents in this study were defined as biological birth parents or non-birth adult primary caregivers who lived with the target children and played a major role in caregiving. Non-birth adult primary caregivers were surveyed if they were the primary caregivers because biological parents might not be actively involved in the children’s lives (20%). Approximately one third of the families (35%) reported experiencing food insecurity. Most parents were female (79%). About one third of parents (29%) were single, and 54% had educational attainment of primary school (7 years of education) or less. Parents’ mean age was 34.7 years (SD=9.9, range=18–79). Children’s mean age was 6.6 years (SD=1.0 years, range=5–9), 49% were boys, and all were enrolled in Nursery to Primary 3 classes in Kampala, the capital city of Uganda.

For measurement validation purposes, data from 154 families who participated in the first round of data collection (pre-intervention) and 101 of the families who participated in the follow-up data collection were utilized. The follow-up data (post-intervention) were collected about 5–6 months after the first assessment. The 53 parents who did not participate in the follow-up data collection (27 intervention and 26 control) were either unable to be reached or had personal/family reasons for not being able to participate. The followed and non-followed families did not differ on family demographic characteristics (i.e., parental education, food insecurity status, household size, child gender, and child age), parental depression, and social support characteristics.

Procedure

Participating parents were randomly selected from Ugandan schools and identified through a regional school list. No family that was approached declined participation in this study. Parents who participated were asked whether they would prefer to be interviewed in English (the official language in Ugandan schools) or Luganda (the primary local language). For parents who were literate, a written informed consent was obtained, and a signed consent form was documented. For parents who were illiterate, an oral consent was given, and a literate witness (e.g., research staff, community guide) signed the consent form on behalf of the participant. After the informed consent process, parents were scheduled for the first interview administered either at home or in their child’s school. All data collection was conducted in a one-on-one interview format by trained bachelor or master-level social science researchers. Based on the parents’ preference, either the English or Luganda version of assessment package was used. The Luganda version was translated based on the recommended method suggested in the literature (i.e., applying translation and back-translation, and using a team review approach to resolve any discrepancies between the versions and to determine whether the translated material is appropriate and meaningful for English and Luganda speakers) [55,56]. About 5 months after the first assessment, parents were re-contacted for the second interview. The majority of the informed consents and interviews (59%) were conducted in Luganda, and the rest were conducted in English. The study protocol was approved by the Internal Review Board of Makerere University College of Health Sciences and the Uganda National Council of Science and Technology, the country’s human research regulatory body.

Measures

Child Social-Emotion and Behavioral Screening Measures

The Social Competence Scale (SCS) [22,23] assesses children’s positive social behaviors. Parents rate how well 12 statements describe their child on a five-point scale from 0 (not at all) to 4 (very well). In studies with preschool-age and elementary school-age children in the U.S., the measure has been shown to yield two reliable subscales: emotion regulation and prosocial/communication skills as well as a total scale [22,23,32,57]. Internal consistency, assessed using Cronbach’s alpha, based on a normative and a high-risk sample was from .76 to .82 for emotion regulation; from .74 to .84 for prosocial/communication skills; and from .84 to .89 for the total SCS [22,57]. The measure was shown to discriminate between normative and high-risk samples of children, with higher total social competence ratings in the normative samples relative to those in the high-risk samples [22,23,57].

The Pictorial Pediatric Symptom Checklist (PPSC) [25,26,38,39] assesses early symptoms of behavioral problems. The measure has been shown to yield three reliable subscales in other samples: internalizing, externalizing, and attention problems. Eight items from the original PPSC checklist (included 3 internalizing, 3 externalizing and 2 attention symptom items) were included in this study. We did not apply the full 17-item version because of assessment time constraint. Based on consultation with local professionals, we included the items that were more relevant to young children in Ugandan contexts. Parents rated how well each statement describes their child on a three-point scale from 0 (never) to 2 (often). Internal consistency based on low-income community samples for children aged 4 to 16 ranges from 0.58 to 0.89, with high reliability for the total scale, and lower reliability for the attention subscale. The measure has been shown to have concurrent validity. The subscales were related to the original PSC (in written format) and standardized child behavior measure (e.g., Child Behavioral Checklist/CBCL) in expected ways [34,39]. The scale has also been shown to be feasibility administrated by local community health workers in Mexico and Philippines [25,39].

The Strengths and Difficulties Questionnaire (SDQ) [3,27] consists of 25 statements relating to specific strengths and difficulties faced by children based on a 3-point scale (0=not true, 1=somewhat true, 2=certainly true). Scores on the four problem behavior subscales (i.e., Emotional Problems, Conduct Problems, Hyperactivity, Peer Problems) are summed to provide a Total Difficulties score, ranging from 0 to 40. A score of 16–40 is considered abnormal based on the US standardization, and a score of 17–40 is considered abnormal based on the British standard [13,40]. Internal consistency, measured by the Cronbach alpha, has been shown to vary with sub-scales. The internal consistency for the American version of the parent SDQ (based on a norm sample of 10,367) [13] is comparable with the British normative study (based on a norm sample of 18,415) [19], with excellent reliability for the Total Difficulties scale (α =.71–.78), good to excellent reliability for four subscales (α =.56–.65 for Emotion Problems; α =.60 for Conduct Problems; α =.61–.67 for Hyperactivity; α =.60 to .66 for Prosocial Behaviors), and fair reliability for the Peer Problems scale (α =.30 to .41) [19]). Scale validity has also been supported in previous studies. High parent rated SDQ difficulties have been found to be associated with higher rates of learning disabilities, a higher likelihood of living in poverty, and higher levels of child social-emotional problems using standardized behavioral assessments [3,40].

Measures of Validity Criteria

Food insecurity (3 items, α =.85) was assessed using the Household Hunger Scale [58]. It evaluates family food resources and members’ hunger status in the past 4 weeks (e.g., no food to eat because of lack of resources to get food; any household member goes to sleep at night hungry, any household member goes a whole day and night without eating anything at all because there was not enough food). If any food resource or hunger issue occurs in families, it would be defined as a food insecure family.

Parental Depression

The Patient Health Questionnaire (PHQ-9; 10 items, internal consistency α =.86–.89) [59,60] is a brief screening measure used to assess parents’ depressive symptoms. Parents rate each symptom item on a 4-point scale (0=not at all; 3=nearly every day). A total score was created for 9 symptom items. PHQ-9 has been validated previously based on Ugandan samples [61,62]. Using the Mini-International Neuropsychiatric Interview (MINI), a widely used short structured diagnostic interview for DSM-IV and ICD-10 psychiatric disorders [63], as the gold standard, the clinical cut-off score of 10 has been suggested for the Ugandan population (with sensitivity of 0.91 and specificity 0.81). Individuals with a score of 10 or above would suggest a high likelihood of having a depressive disorder using MINI [61]. For the purpose of this study, the continuous scale score was used for analyses.

Social Support (4 items, α =.85) was evaluated using items adapted from the Multidimensional Scale of Perceived Social Support [64,65]. It evaluates perceived support for comfort, emotion sharing, and help needed on a 5-point scale (1=strongly disagree, 5= strongly agree). The original scale asks social support from family and friend separately (e.g., there is a family member who is around when I am in need; there is a friend who is around when I am in need). For this study, we did not distinguish family from friend support, and collapse questions into one (e.g., there is a special person who is around when I am in need). The scale has shown to be a valid scale in our work (e.g., higher social support was associated with low level of depression) [66].

Analysis Plan

To test whether the factor structures described by developers are consistent with data from the Ugandan sample, we conducted a series of maximum-likelihood confirmatory factor analyses (CFA), using MPLUS 6.0 (67), based on the theorized structure of each scale. To confirm factor structures consistency across different time points, we carried out CFA separately using data from baseline (N= 154; 86 intervention and 68 control) and follow-up assessments (N=101; 59 intervention and 42 control). Three indices were used to evaluate the fit of the CFA models, including the chi-square goodness-of-fit (χ²), the goodness of fit index (GFI), and the root mean square error of approximation (RMSEA). As a general guideline, GFI values of .90 or above and RMSEA values of .08 or less are considered as support for adequate fit of models [68,69].

To assess the reliability of the screening measures, we evaluated internal consistency (measured by Cronbach alphas) of the items based on factors derived from the CFAs using Time 1 and Time 2 data. To assess the stability of the measures, we used the two time points of ratings from the control sample (N=42) and conducted a series of product-moment correlation analyses.

Finally, we assessed three types of measurement validity. Discriminant validity was evaluated by comparing mean score differences on the scales between high-risk and low-risk samples (i.e., food security vs. food insecurity; abnormal-level vs. normal-level of problem behaviors using the SDQ cut-off 17 based on the US standard) and between male and female children using independent t-tests. Time 1 data were utilized for this purpose. Significant high- and low-risk sample and/or gender differences on the measures would support the measures’ discriminant validity, with the expectation that the high-risk sample and boys would exhibit higher behavioral problems compared with the low-risk sample and girls, respectively. To evaluate concurrent validity, we utilized all social-emotional and behavioral measures, as well as select validity measures (i.e., parental depression, social support) collected at Time 1 (prior to intervention). Pearson correlations were conducted to examine general patterns of concurrent associations. Finally, to study the predictive validity, we assessed the magnitude of association between the Time 1 scales and the Time 2 measures. The evaluation of predictive validity was limited to the control sample to ensure that if the intervention changed the factor structure of the measures, this would not confound the results.

Results

Factor Structure

To understand the underlying constructs for the SCS, we examined whether a one- or two-factor model fit well with the data using CFAs based on previous research that has identified two reliable subscales of emotion regulation and prosocial/communication skills. Results indicated that both one- and two factor models fit the data well (χ² (50) = 56.70/84.20, CFI=.98/.92, RMSEA=.03/.08 for Time1/Time2 for the one-factor model; and χ² (49) = 52.65/84.18, CFI = .99/.93, RMSEA = .02/.08 for Time1/Time2 for the two-factor model), with slightly better fit for the 2-factor model than the 1-factor model at Time 1 (χ²Δ (1–2 factor model) = 4.05, p <.05). Factor loadings for items on the one-factor or two-factor model were all above .30 at Time 1 and above .45 at Time 2. The correlation between the two factors was moderate to high (r = .56/.76 at Time 1/Time 2), suggesting that the two factors are distinct constructs but have overlapping variation.

For the PPSC, we assessed the model fit for the a priori hypothesized three-factor (internalizing, externalizing, and attention behavior problems) or two-factor model (internalizing and externalizing composites) given the shortened version of the scale used. Overall, the three-factor model fit better than the two-factor model with the Time 1 data, χ² (17/16) = 17.55/21.52, CFI = 1.00/.97, RMSEA = .01/.05 for the three-factor/two-factor model; χ²Δ (2–3 factor model) = −3.97, p <.05. However, at Time 2, the two-factor model fit better than the three-factor model, χ² (17/16) = 29.15/19.87, CFI = .91/.97, RMSEA = .08/.05 for the three-factor/two-factor model; χ²Δ (2–3 factor model) = −9.28, p <.001. Factor loadings for items on the three-factor model were all above .35 at both time points, and items on the two-factor model were all above .26, with one exception loaded at .12. Both CFA and factor loading results indicated applying the developers’ three-factor model might be acceptable. The correlations among the three factors were low (r = .20–.21/.06–.24 at Time 1/Time 2), suggesting that the factors measure distinct underlying constructs that are not correlated.

Finally, to assess the factor structure for the SDQ, we tested a 1-factor CFA (i.e., the 20 Total Difficulties items), as well as a 4-factor solution (i.e., the four subscales of emotion symptom, conduct problem, hyperactivity, and peer problem) to assess if the four subscales are distinct constructs. The CFAs indicated a poor fit of the 4-factor model, (χ² (164) = 215.67/233.47, CFI=.75/.68, RMSEA=.05/.06 for Time1/Time2), as well as poor fit of the 1-factor model, (χ² (170) = 249.58/299.96, CFI=.61/.41, RMSEA=.06/.09 for Time1/Time2). The 4-factor model appeared to provide a better fit for the data than the 1-factor model (χ²Δ (1–4 factor model) = 33.91/66.49 for Time1/Time2, ps <.001). Factor loadings for the SCS, PPSC, and SDQ constructs/scales are provided in the Appendix.

Reliability

Internal Consistency

We evaluated internal consistency by calculating Cronbach’s alphas for each assessment time point (see Table 1). Internal consistency was assessed based on the factors derived from the CFAs (i.e., for the SCS and PPSC) and the developer scales (i.e., for the SDQ). Table 1 summarizes the reliability results. Overall, alpha coefficients were high for the SCS total scale (all > .80) and adequate for the PPSC internalizing, externalizing, and attention problems and the SDQ Total Difficulties and Emotion Symptom scales (all > .60) at Time 1 and Time 2. However, alpha coefficients for the SDQ Conduct Problem, Hyperactivity, Peer Problems and Prosocial Scales were low (<.60 at one time or both times). Therefore, for the SCS, we computed the Total Social Competence score as well as two subscale scores (prosocial/communication and emotional regulation) for each assessment time point by averaging the items included in the subscales. For the PPSC, we computed three subscale scores-Internalizing, Externalizing, and Attention Problems-by summing the scale items; and for the SDQ, we only computed the Total Difficulties and the Emotion Symptom sum scores because only these had adequate internal consistency.

Table 1.

Reliability for the Study Measures.

A. Pediatric Screening Measures	Number of Items	Time 1	Time 2	Test-Retest
Pictorial Pediatric Symptom Checklist (PPSC)
Internalizing problems	3	.68	.67	.15
Externalizing problems	3	.59	.62	.17
Attention problems	2	.72	.65	.28+
Social Competence Scale (SCS)
Total scale	12	.81	.87	.36*
Emotion Regulation	6	.66	.77	.22
Prosocial/Communication	6	.75	.79	.46**
Strengths and Difficulties Questionnaire (SDQ)
Total Difficulties Scale	20	.63	.61	.47**
Emotion Symptom Scale	5	.60	.62	.37*
Conduct Problem	5	.39	.60	.33*
Hyperactivity	5	.25	.37	.41**
Peer Problem	5	.42	.15	.35*
Prosocial Scale	5	.48	.52	.31*
B. Family Validation Measures
Household Food Insecurity	3	.85	.86	.25
PHQ-9 Depression	9	.83	.89	.52***
MSPSS-Social Support	4	.80	.73	.40**

Note. SDQ total problem scale includes all items except items from the prosocial scale. Test-retest reliability are based on the Control Sample who have both baseline and follow-up data (n=42).

^***p <.001,

^**p<.01,

^*p<.05.,

⁺p<.10

Temporal Stability

To examine the temporal stability/test-retest reliability of the scales over a 5-month period, Pearson correlations were calculated using the control sample only.

Validity

As shown in Table 1, stability was low to moderate for the SCS scales (r = .22 – .46) and the SDQ scales (r = .31 – .47). The stability was low for the PPSC scales (r = .15 – .28). Although some of the SDQ scales (e.g., conduct problem, hyperactivity, peer problem, prosocial skills) did not have adequate internal consistency, the stability for these scales was moderate (r = .31 – .41).

Discriminant Validity

We assessed whether there were mean score differences on the SCS, PPSC, and SDQ scales between girls and boys, and between high- and low-risk samples, defined as food security vs. food insecurity and high behavioral problems vs. low behavioral problems (SDQ ≥ 17 vs. SDQ < 17 as defined above). We found no gender difference for any social-emotional measures. However, we found support of discriminant validity between high- and low-risk populations. Specifically, children in the high-behavioral risk group showed significantly higher problem behaviors (as rated by the PPSC and SDQ) and lower social competence (as rated by the SCS) than the children in the low-behavioral risk group. Similarly, children from food insecure families showed significantly lower social competence and higher externalizing, internalizing and emotional problems than children from food secure families. (See Table 2 for the mean scores for all subgroups and the full sample).

Table 2.

Discriminating Validity (comparisons between High and Low Risk Samples)

	Problem Behavior Risk		Food Security Status		Total Sample
	High Risk (N=41)	Low Risk (N=113)	Insecure (N=54)	Secure (N=100)	Total (N=154)
	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
PPSC-Internalizing problems	1.68 (1.63)***	.68 (.98)	1.35 (1.60)*	.73 (.98)	.95 (1.27)
PPSC-Externalizing problems	1.80 (1.74)**	.91 (1.12)	1.44 (1.74)*	.99 (1.09)	1.15 (1.37)
PPSC-Attention problems	2.00 (1.57)**	1.14 (1.16)	1.48 (1.48)	1.31 (1.25)	1.37 (1.33)
SCS-Total scale	2.08 (.71)**	2.44 (.59)	2.21 (.63)*	2.42 (.64)	2.34 (.65)
SCS-Emotion Regulation	1.99 (.78)*	2.27 (.70)	2.17 (.77)	2.21 (.71)	2.20 (.73)
SCS-Prosocial/Communication	2.17 (.85)**	2.60 (.65)	2.24 (.70)**	2.62 (.72)	2.49 (.74)
SDQ-Total Problem Scale	20.59 (2.59)***	10.77 (3.30)	15.22 (5.52)**	12.39 (5.02)	13.38 (5.36)
SDQ-Emotion Symptom Scale	6.34 (1.96)***	2.73 (2.00)	4.50 (2.64)**	3.26 (2.40)	3.69 (2.55)

Note. Analyses based on Time 1 data. Problem behavior high-risk and low-risk groups were defined as SDQ Problem Scale score ≥ 17 and < 17.

^***p <.001,

^**p<.01,

^*p<.05. Based on parent-report of SDQ, the mean (standard deviation) of the Total Difficulties for American 4–17 years olds was 7.1 (5.7) and for the emotional symptoms was 1.6 (1.8). The mean (standard deviation) of the Total Difficulties for Australian 7–17 years olds was 8.2 (6.1) and for the emotional symptoms was 2.1 (2.0) (40). For the SCS, the means of the total social competence for American preschooler to 2nd grade children were ranged from 2.22 to 2.68 (23). Mean scores comparisons on SDQ and SCS may suggest that Ugandan children have higher burden of child mental health problems.

Concurrent Validity

As shown in Table 3, the PPSC, SCS and SDQ measures were associated with each other in the expected directions, such that adaptive behaviors were positively correlated and adaptive and maladaptive behaviors were negatively correlated. For example, the PPSC internalizing and externalizing problem scales were positively correlated with the SDQ Total Difficulties and Emotion Symptom scales.

Table 3.

Concurrent Validity

Behavioral Measures	1	2	3	4	5	6	7	8	9
1. PPSC-Internalizing problems	1.00
2. PPSC-Externalizing problems	.21**	1.00
3. PPSC-Attention problems	.21*	.20*	1.00
4. SCS-Emotion Regulation	−.16*	−.30***	−.23**	1.00
5. SCS-Prosocial	−.07	−.26**	−.31***	.56***	1.00
6. SCS-Total Competence	−.13	−.32***	−.31***	.88***	.88***	1.00
7. SDQ-Emotion Symptom	.41***	.16*	.08	−.11	−.08	−.11	1.00
8. SDQ- Total Difficulties	.39***	.45***	.32***	−.23**	−.22**	−.26**	.71***	1.00
Family Validation Measures
9. Parental Depression	.41***	.10	.01	−.07	−.13	−.11	.41***	.37**	1.00
10. Social Support	−.11	.04	−.01	.13	.28***	.23**	−.23**	−.19*	−.30***

Note. Correlations reported are based on Time 1 data (N=154).

^***p <.001,

^**p<.01,

^*p<.05,

⁺p<.10

The Emotion Regulation and Prosocial/Communication scores measured by the SCS were both negatively associated with PPSC-externalizing problems and SDQ-Total Difficulties and positively correlated to each other.

Moreover, in examining whether the three social-emotional screening measure scores were associated with family characteristics, we found results in the expected direction.

Specifically, higher parental depressive symptoms were associated with higher child internalizing problems (as measured by the PPSC), higher emotion symptoms and higher behavioral difficulties (as measured by the SDQ). Parents who reported having more social support had children with higher scores of social competence, and lower emotional symptoms and behavioral difficulties (measured by the SDQ).

Predictive Validity

We assessed how scores at Time 1 were associated with scores at Time 2 using data from the control group. As shown in Table 4, the PPSC internalizing, attention, and externalizing scores were not significantly correlated across time points. However, PPSC internalizing problems at Time 1 were positively and significantly associated with SDQ Total Difficulties at Time 2. In addition, social competency at Time 1 was negatively correlated with SDQ Total Difficulties at Time 2; and SDQ behavioral difficulties at Time 1 were negatively correlated with social competency at Time 2.

Table 4.

Predictive Validity.

Time 1 Social-emotional Measures	Time 2 Social-emotional Measures
	1	2	3	4	5	6	7	8
1. PPSC-Internalizing problems	.15	.04	.00	−.01	−.21	−.11	.07	.30*
2. PPSC-Externalizing problems	−.08	.17	−.05	−.13	−.03	−.08	−.07	−.01
3. PPSC-Attention problems	−.14	.25	.28+	−.22	−.23	−.24	−.05	.37*
4. SCS-Emotion Regulation	−.06	−.24	−.05	.22	.13	.19	−.05	−.39*
5. SCS-Prosocial	−.21	−.12	−.10	.42**	.46**	.46**	−.14	−.42
6. SCS-Total Competence	−.15	−.20	−.08	.35*	.33*	.36*	−.11	−.45**
7. SDQ-Emotion Symptom	.24	−.20	.03	−.08	−.15	−.12	.37*	.30*
8. SDQ- Total Difficulties	.07	−.08	.22	−.20	−.33*	−.27+	.26+	.47**

Note. Predictive Correlations reported are based on the Control Sample who have both baseline and follow-up data (n=42).

Discussion

The primary objective of the present study was to evaluate the potential utility of three brief parent-rated measures. These instruments were originally designed for preschool and elementary school-age children and have been validated in high-income countries.

This study assessed the appropriateness of parent-reported children’s social-emotional functioning in one SSA country. Overall, the CFA findings support most of the developmental and behavioral constructs that were developed in high-income countries. Reliability and validity assessments also indicate that the three tools have adequate psychometric properties (with the exception of some SDQ subscales) when used with Ugandan children (5–9 years old), and are useful for assessing emotion regulation, prosocial skills (two major skills for young children), internalizing and attention problems. These tools have the potential to be used as screening assessments or intervention evaluation tools for monitoring children’s behavioral difficulties.

A significant contribution of this study is the examination of both strength-based and pathology-based behavioral constructs, and testing the psychometric properties of measures that have implications for use in population research and in educational, clinical, or research settings but have not been used with Ugandan children. Previous studies have reported the utility of applying these measures in primary care, educational, and child welfare service settings in other contexts [3,23,25–27,38,39]. Our findings indicate that these measures can potentially be useful in low-income countries.

The SDQ has been translated into more than 80 languages, but there is insufficient validation evidence regarding applicability in LICs (despite its widespread use). Our study tests the underlying factor structures and concurrent and predictive validity of the SDQ, and adds new validation evidence from LIC settings. Although the 4-factor solution (emotion symptoms, conduct problems, hyperactivity, peer problems) and the 1-factor solution (the Total Difficulties) did not fit the data well in the Ugandan sample, it should be noted that all subscales had moderate stability (r =.31–.47) across time. Previous American and British normative studies (based on over 10,000 sample size) found excellent reliability for Total Difficulties (alphas > .70); good to excellent for the subscales Emotion Problems, Conduct Problems, Hyperactivity, and Prosocial Behavior; and fair for peer problems [13,19]. Our study found some support of these patterns in Uganda. For example, we found acceptable reliability for the Total Difficulties and Emotion Symptom scales, but low reliability for the Peer Problem and Hyperactivity scales. Given the relatively small sample size (N=154), future research should utilize a larger representative sample to re-evaluate the underlying factor structures in this population.

Previous validation studies for the SDQ and PPSC have also included standardized or clinical mental health diagnostic assessments (e.g., Child Behavioral Checklist, Kids’ version of Schedule for Affective Disorders and Schizophrenia/K-SADS) to identify clinical cut-off scores and evaluate sensitivity and specificity of the cut-off [13,19]. Identifying meaningful cut-off scores have implications for detecting children at-risk or with abnormalities. Such features are needed for planning early and preventative interventions. Therefore, future research should utilize large clinical and community-based samples to further validate the SDQ for clinical usage in LICs.

In testing discriminant validity, contrary to expectations, we did not find gender differences in any social-emotional or problem behavior factors. These findings are consistent with some studies that are based on samples of U.S. minority children [70,71]. However, the lack of gender differences is inconsistent with findings from the United States DQ norm studies or studies based on White samples from high-income countries generally find significantly higher levels of problem behaviors and lower social competence for boys [8,13,72]. Another U.S. study found that parent-reported symptomatology increased as children aged, from 8–14 years for boys, and 11–17 years for girls [13]. It is unclear whether the lack of gender differences in our study can be attributed to cultural differences in Uganda compared to the US, or if the screening measures are not sensitive to real gender differences in this cultural context. This issue will need to be examined further in future research and studies with larger sample sizes.

Although the overall findings support the use of the SCS, PPSC, and SDQ with young children, the present study has several limitations. First, analyses were limited to children ages 5 to 9 years old, and thus the full spectrum of childhood was not assessed. In addition, conclusions about predictive validity was based on a 5-month time period. The utility of these assessment tools for older children (age 10–17), or in predicting long-term outcomes, requires further investigation. Second, this study did not apply the full 17-item version of the PPSC. Utilizing a smaller number of subscale items may contribute to low temporal stability for one of the subscales (i.e., 3 of the original 5 internalizing problem subscale items were included). Future research should utilize the full scale for more comprehensive screening and better cross-study comparisons. Third, the lack of gender differences in measured behaviors was contrary to expectations, and may be related to the unique characteristics of the SSA or LIC settings. Given the data, we are not able to investigate this issue further. It would be useful to examine this issue in a larger sample. Finally, the data collected in this study was based on single informant reports (i.e., parents). Future research should include objective diagnostic categories or consider multiple informants to gain a nuanced understanding of scale validity in diverse contexts.

Conclusion

Social-emotional and behavioral problems are common among children and adolescents but screening among children in LICs is uncommon. The use of screening or brief assessment instruments to improve the identification and assessment of pediatric dysfunction is an important step in addressing pediatric population’s social-emotional and behavioral health needs. Although the overall findings support the utility of the SCS, PPSC, and SDQ to be used in samples of children in Uganda, caution is warranted when using and interpreting the subscales of the SDQ, given the low internal consistency for some subscales. More studies are needed to validate PPSC and SDQ against standardized measures and clinical diagnoses. Another important next step is to identify meaningful cut-off scores for clinical usage using larger representative populations.

Abbreviations

SSA

Sub-Saharan African

SCS

Social Competence Scale

PPSC

Pictorial Pediatric Symptom Checklist

SDQ

Strengths and Difficulties Questionnaire

LICs

Low-Income Countries

APPENDIX. ITEM FACTOR LOADINGS FOR THE THREE SCREENING MEASURES

Appendix Table A.

Item Factor Loadings for the Social Competence Scale

	Time 1		Time 2
	1-factor model	2-factor model	1-factor model	2-factor model
Emotion Regulation Subscale
Can accept things not going his/her way(1)	.428	.449	.527	.490
Copes well with failure (1)	.310	.305	.665	.662
Thinks before acting	.554	.595	.564	.568
Can calm down by himself/herself when excited or “all wound up”	.490	.480	.584	.603
Does what he/she is told to do	.606	.615	.633	.642
Controls his/her temper when there is a disagreement	.563	.613	.672	.679
Prosocial/Communication Scale
Works out problems with friends or brothers and sisters on his/her own	.456	.484	.500	.503
Is very good at understanding other people’s feelings (2)	.563	.587	.470	.489
Shares things with others (4)	.693	.715	.744	.685
Is helpful to others (3)(4)	.602	.540	.800	.740
Listens to others’ points of view (3)	.575	.530	.649	.661
Can give suggestions and opinions without being bossy (2)	.485	.469	.591	.574
Confirmatory Factor Analysis Model Fit	1-factor model:		1-factor model:
	χ2 (50)= 56.698,		χ2 (50) = 84.196,
	p=.239, CFI=.984,		p<.001, CFI=.928,
	RMSEA=.029;		RMSEA=.081;
	2-factor model:		2-factor model:
	χ2 (49) = 52.65,		χ2 (49)= 84.18,
	CFI=.99,		CFI=.93,
	RMSEA=.02		RMSEA=.08

Note. In both factor models, we allowed some items to be correlated (based on modification indices in MPLUS) to have adequate fit. Numbers (1) (2) (3) and (4) indicate item-pairs that we allowed to be correlated (e.g., we allowed “Can accept things not going his/her way (1)” to be related to “Copes well with failure (1)”).

Appendix Table B.1.

Item Factor Loadings for the Pictorial Pediatric Symptom Checklist (3-Factor Model)

	Time 1			Time 2
	Internalizing	Externalizing	Attention	Internalizing	Externalizing	Attention
Sad, Unhappy	.613			.739
Hopeless	.896			.831
Worries a lot	.430			.382
Fidgety, unable to sit still			.741			.743
Distracted easily			.748			.644
Fights with other children		.687			.756
Teases others		.558			.641
Takes things that do not belong to him/her		.459			.448
Confirmatory Factor Analysis Model Fit	χ2 (17) = 17.545, p=.418, CFI=.997, RMSEA=.014			χ2 (17) = 29.148, p=.033, CFI=.911, RMSEA=.083

Note. Eight of the 17 original items were included (3/5 internalizing, 3/7 externalizing, and 2/5 attention items were included). Items not included were Internalizing items “Is down on self” & “seems to be having less fun”; Attention items “daydreams too much”, “has trouble concentrating”, & “acts as if driven by a motor”; and Externalizing items “Refuses to share”, “does not understand other people’s feelings”, “blames others for his/her troubles” and “does not listen to rules.”

Appendix Table B.2.

Item Factor Loadings for the Pictorial Pediatric Symptom Checklist (2-Factor Model)

	Time 1		Time 2
	Internalizing	Externalizing + Attention	Internalizing	Externalizing + Attention
Sad, Unhappy	.627		.745
Hopeless	.874		.825
Worries a lot	.442		.384
Fidgety, unable to sit still (1)		.287		.261
Distracted easily (1)		.346		.493
Fights with other children (2)(3)		.572		.117
Teases others (2)		.527		.365
Takes things that do not belong to him/her (3)		.414		.626
Confirmatory Factor Analysis Model Fit	χ2 (16) = 21.523, p=.159, CFI=.972, RMSEA=.047		χ2 (16) = 19.870, p=.226, CFI=.970, RMSEA=.048

Note. In the factor model, we allowed some items to be correlated (based on modification indices in MPLUS). Numbers (1) (2) and (3) indicate item-pairs that we allowed to be correlated.

Appendix Table C.

Item Factor Loadings for the Strength and Difficulty Questionnaire

	Time 1		Time 2
	1-factor model	4-factor model	1-factor model	3-factor model
Emotion Symptom Subscale
Often complains of headaches, stomach-aches or sickness	.435	.390	.414	.399
Many worries, often seems worried	.451	.561	.519	.590
Often unhappy, down-hearted or tearful	.442	.552	.572	.765
Nervous or clingy in new situations, easily loses confidence	.481	.510	.295	.267
Many fears, easily scared	.342	.393	.312	.443
Conduct Problem Subscale
Often has temper tantrums or hot tempers	.325	.368	.526	.449
Generally obedient, usually does what adults request (R)	.167	.217	.166	.323
Often fights with other children or bullies them	.249	.449	.341	.632
Often lies or cheats	.263	.500	.367	.613
Steals from home, school or elsewhere	.092	.255	.281	.527
Hyperactivity Subscale
Restless, overactive. cannot stay still for long	.253	.508	.173	.470
Constantly fidgeting or squirming	.292	.596	.336	.909
Easily distracted, concentration wanders	.187	.192	.395	.162
Thinks things out before acting (R)	−.076	−.081	−.101	.094
Sees tasks through to the end. good attention span (R)	−.055	−.023	.186	.079
Peer Problems
Rather solitary, tends to play alone	.397	.417	.050	–
Has at least one good friend (R)	.202	.239	−.103	–
Generally liked by other children (R)	.076	.121	−.004	–
Picked on or bullied by other children	.513	.537	.405	–
Gets on better with adults than with other children	.331	.344	.026	–
Confirmatory Factor Analysis Model Fit	1-factor model:		1-factor model:
	χ2 (170) = 249.583,		χ2 (170) = 299.959,
	p=.170, CFI=.613,		p<.001, CFI=.406,
	RMSEA=.055;		RMSEA=.086;
	4-factor model:		3-factor model:
	χ2 (164) = 215.674,		χ2 (87) = 123.968,
	p=.004, CFI=.749,		p=.006, CFI=.782,
	RMSEA=.045		RMSEA=.064

Note. (R) indicates item was reverse coded. Results indicate poor one-factor model fit. The 4-factor model did not converge using the time 2 data, which might be due to the smaller sample or low correlations among peer problems items. We refit the model by excluding items from Peer Problem subscale and examining the 3-factor model (reported in Table -Time 2).