Content of this journal is licensed under a Abstract
Background:
Children in low-resource countries like Georgia often have limited access to assessment measures for mental health care services. This study adapts and validates the mental health screening tool the Strength and Difficulties Questionnaire for use in Georgian children.
Methods:
A total of 16 654 children were assessed by a parent and/or teacher using Georgian-adapted Strength and Difficulties Questionnaire. Receiver operating characteristic analyses were performed to assess the discriminative validity of the Strength and Difficulties Questionnaires and to establish an optimal cutoff score.
Results:
Data from 15 738 parents- and 13 560 teachers-administered Strength and Difficulties Questionnaire were analyzed. The internal consistency analysis showed Cronbach’s alpha to be 0.625 and 0.621 for parent- and teacher-administered Strength and Difficulties Questionnaire, respectively. The area under the curve (95% CI) shows that the Strength and Difficulties Questionnaire can differentiate risk group children from typically developing peers: parent-administered questionnaires—0.629 (0.556-0.702) and teacher-administered questionnaires—0.680 (0.611-0.789). Parent-administered Strength and Difficulties Questionnaire has a cutoff value of 16 or more with 92.5% of sensitivity and teacher-administered Strength and Difficulties Questionnaire—14 or more with a sensitivity of 85.6%.
Conclusion:
The study finds that the Strength and Difficulties Questionnaire is a valid screening instrument and does not depend on the informant—parent or teacher. It suggests that with appropriate cultural adaptation, the SDQ can be used in the Republic of Georgia to identify children at risk for mental disorders and help guide resource allocation.
Keywords: Strength and Difficulties Questionnaire, screening, population study
Main Points
The Strength and Difficulties Questionnaire (SDQ) is an effective screening tool for the identification of children at risk for mental disorders.
Validity of the SDQ does not depend on the informant—parent or teacher but cutoff scores are different.
Culturally adapted SDQ can be used to guide resource allocation in low-resource countries like the Republic of Georgia.
Introduction
Approximately 85% of the global population live in low- and middle-income countries (LMICs) constituting over 80% of people with mental health and substance use disorders.1 The majority of these problems started before the age of 18 years.2,3 It is also estimated that more than 10% of children and adolescents live with a diagnosed mental disorder impacting the child, family, and community.4
Early detection and treatment have an important role in preventing psychosocial problems; may benefit the child’s development, well-being, and future life, adulthood physical and mental health; and reduce the negative impact on the cost of health-care system and society.5,6,7
Additionally, children and adolescents from LMICs often have limited access to mental health-care services, usually due to limited resources and limited availability of qualified staff.8 Furthermore, an additional barrier to service provision is the restricted availability of translated assessment measures that are reliable, valid, and easy to use among non-mental health professionals.9,10,11
The Strengths and Difficulties Questionnaire (SDQ) is a brief behavioral screening questionnaire for 2- to 16-year-old children, which is one of the most widely used for the purpose in both clinical practice and research.12 It was originally published in English, has been subsequently translated into over 80 languages, and is freely available from the Internet (www.sdqinfo.com). The psychometric properties of the SDQ have been studied in different countries and in children with different abilities.6,7,9,10,11,13,14,15,16,17,18,19,20,21,22,23,24
In the Republic of Georgia, many cases of mental disorders are undiagnosed and underestimated. By the UNICEF 2020 report,25 which is based on statistics from the Ministry of Health of Georgia, the total number of children with behavioral and emotional problems under 15 years is 8842 (out of total 764.9 thousand), which is only about 1.15% of this population.26 Real number of prevalence is unknown. Recently the government of Georgia approved a strategic plan for mental health for 2022-2030 years and outlined the importance of raising public awareness, improvement of professional and technical resources, and improvement of availability of community-based mental health services.27
To provide surveillance and prevalence studies at the population level and to help ensure referrals to child and adolescent mental health services in low-resource countries like Georgia28 is adaptation and validation of a widely accepted screening tool, administration of which does not need highly trained personnel.
The aim of this study was the adaptation and validation of the Georgian SDQ for use in elementary schools. The target population was third-grade students, as 3 years in a row children learn different subjects from the same teacher and the teacher’s answers to the questionnaire could be more reliable.
Material and Methods
The education system in Georgia allows everyone in the country to receive at least general education. The target population of the study was all third-grade students of 5 main cities of Georgia (Tbilisi, Kutaisi, Batumi, Zugdidi, and Telavi), where live 47% (n = 47 997) of all children. Randomly selected institutions from all 264 schools were considered as clusters from where we identified individual participants. Informative lectures on mental health, research materials, and screening questionnaires were provided to school principals and teachers by the study team. Georgian-speaking teachers of third-grade children from whom informed consent was obtained were invited for participation in the study. They distributed materials to all Georgian-speaking parents and obtained parents’ informed consent.
Two-stage model was used for the study. In the first phase (2019-2020 years), SDQ screening and sampling of screen positive and negative students were performed; in the second phase (2020-2021 years), diagnostic assessments and best-estimate diagnoses were defined. For the evaluation of children in second phase, standardized diagnostic assessments were used including the Test of Nonverbal Intelligence, fourth edition (TONI-4), Vineland Adaptive Behavior Scales, second edition (Vineland II), the Schedule for Affective Disorders and Schizophrenia for School-Age Children—Present and Lifetime Version (K-SADS-PL), the Autism Diagnostic Interview-Revised (ADI-R), the Autism Diagnostic Schedule-II (ADOS-II), and speech, language, and communication assessment tests. During the COVID-19 lockdown period, telemedicine was used for the assessment with instruments not requiring direct contact with a child. At least 2 independent, experienced, Georgian clinicians made consensus best estimate diagnosis based on the revision of all medical documents and considering diagnostic criteria of International Statistical Classification of Diseases and Related Health Problems, 10th edition.29
The study was approved by the Ethics Committee and Institutional Review Board of the National Disease Control Center of Georgia (#2019-032). Official permission for the study was obtained from the principals of all participating schools. Following verbal explanation, written informed consent was obtained from each parent and teacher.
Screening
The SDQ, a short screening questionnaire, consists of 25 items. Each item requires the respondent to answer “not true” (0 points), “somewhat true” (1 point), or “certainly true” (2 points). A total score is the sum of all item scores (maximum of 50 points). It takes approximately 10 minutes to complete and does not need any training for the use of the SDQ. We used the Georgian version of SDQ which is freely available online.30
The 25 items were divided between 5 scales: (1) emotional symptoms (5 items); (2) conduct problems (5 items); (3) hyperactivity/inattention (5 items); (4) peer relationship problems (5 items); and (5) prosocial behavior (5 items);
The same 25 items are included in questionnaires for completion by the parents or teachers of 4- to 16-year old children.12 There are multiple versions of the measure: teacher report for ages 4-11 and 11-17, parent report for ages 4-17, and child self report for 11- to 17-year-old adolescents. Additionally, there is an early childhood SDQ for 2-4-year-old children.
From overall 22 553 students, 16 654 (74%) were screened—questionnaire were completed by parents and/or teachers (Table 1). From participated 16 654 children 8233 (49.7%) were females and 8378 (50.3%) were males. After final cleaning (removing duplicates and incomplete questionnaires) parents’ dataset included 15 738 records and teachers’ 13 560.
Table 1.
Descriptive Statistics—Frequency of Children in the Parent- and Teacher-Administered Datasets and Participation Rate
|
City |
Number of Children | ||||||
|---|---|---|---|---|---|---|---|
| Registered n | Total n (%) in Parent-Dataset | Total n (%) in Teacher-Dataset | n (%) in Both Datasets | n (%) in Parent-Dataset Only | n (%) in Teacher-Dataset Only | n (%) Participation Rate | |
| Tbilisi | 16 247 | 10 652 (65.56) | 8501 (52.32) | 7915 (48.72) | 2737 (16.85) | 586 (3.61) | 11 264 (69.33) |
| Kutaisi | 2638 | 2043 (77.45) | 1949 (73.88) | 1833 (69.48) | 210 (7.96) | 116 (4.4) | 2162 (81.96) |
| Batumi | 2529 | 2150 (85.01) | 2165 (85.61) | 2048 (80.98) | 102 (4.03) | 117 (4.63) | 2283 (90.27) |
| Zugdidi | 730 | 545 (74.66) | 545 (74.66) | 545 (74.66) | — | — | 545 (74.66) |
| Telavi | 409 | 348 (85.09) | 400 (97.8) | 348 (85.09) | — | 52 (12.71) | 400 (97.8) |
| Total | 22 553 | 15 738 (69.78) | 13 560 (60.13) | 12 689 (56.26) | 3049 (13.52) | 871 (3.86) | 16 654 (73.84) |
According to SDQ scores screened children were allocated to the top first, second, third, fourth, fifth, and tenth percentiles (Table 2).
Table 2.
Descriptive Statistics—Frequency of Children with Strength and Difficulties Questionnaire Scores by Percentile Groups and Places of Residency
| Percentile (%) |
SDQs Dataset | Batumi | Kutaisi | Tbilisi | Zugdidi | Telavi | Total |
|---|---|---|---|---|---|---|---|
| n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | ||
| 90 | Parent | 300 (13.91) | 193 (9.89) | 1041 (12.2) | 34 (6.21) | 30 (7.52) | 1598 (11.76) |
| Teacher | 352 (16.31) | 186 (9.52) | 1080 (12.68) | 67 (12.32) | 24 (6.04) | 1709 (12.2) | |
| 95 | Parent | 190 (8.82) | 116 (6.01) | 637 (7.51) | 21 (3.86) | 17 (4.33) | 981 (7.21) |
| Teacher | 165 (7.61) | 93 (4.76) | 522 (6.14) | 23 (4.23) | 10 (2.47) | 813 (6.03) | |
| 96 | Parent | 159 (7.33) | 90 (4.55) | 515 (6.13) | 19 (3.52) | 12 (3.02) | 795 (5.86) |
| Teacher | 118 (5.51) | 71 (3.61) | 412 (4.79) | 16 (2.91) | 8 (2,01) | 625 (4.62) | |
| 97 | Parent | 111 (5.12) | 73 (3.74) | 376 (4.43) | 11 (2.03) | 9 (2.34) | 580 (4.31) |
| Teacher | 98 (4.54) | 56 (2.84) | 321 (3.82) | 10 (1.82) | 5 (1.33) | 490 (3.57) | |
| 98 | Parent | 78 (3.64) | 62 (3.24) | 282 (3.31) | 6 (1.13) | 7 (1.76) | 435 (3.21) |
| Teacher | 56 (2.62) | 38 (1.92) | 197 (2.33) | 5 (0.87) | 2 (0.51) | 298 (2.24) | |
| 99 | Parent | 39 (1.81) | 34 (1.71) | 140 (1.64) | 4 (0.73) | 3 (0.79) | 220 (1.58) |
| Teacher | 19 (0.93) | 12 (0.62) | 72 (0.78) | 2 (0.42) | 1 (0.32) | 106 (0.78) |
Diagnostic Assessment
For diagnostic assessments of children, additional written informed consent from parents was obtained. During the COVID-19 lockdowns, telemedicine was used for assessment by the tools: ADI-R, K-SADS-PL, and Vineland-II. In periods of milder restrictions tools, that need direct observation of a child: the ADOS-2, communication, speech and language assessment, and TONI-4 were completed at the local rehabilitation centers where children lived. All these tools were prepared in advance for use in the current study—after translation and back translation they were piloted; however, validation and establishment of sensitivity and specificity were not performed.
For the final diagnosis, criteria of the International Classification of Diseases and Related Health Problems tenth revision (ICD-10) and Diagnostic and Statistical Manual of Mental Disorders (DSM-5) were used.29,31
Cultural Consideration
In case identification, the study team discussed cultural specifics, and all potential sources of bias carefully addressed. Two bilingual translators, whose mother language was Georgian translated the SDQ from English to Georgian. Additional bilingual translators, not involved in translation, discussed and resolved discrepancies between the 2 translators. Georgian speaker with advanced mastery in the English language, blinded to the original English version of SDQ, and back-translated from Georgian to English. The original and translated questionnaires were compared. All items of the questionnaire were reviewed and discussed by the group of mental health team members and parents. They carefully checked the idiomatic, semantic, conceptual, and experiential equivalence of the translated and original versions. To resolve discrepancies, consensus was reached on all items. A focus group of 22 persons including teachers and parents were pilot tested the final version of the Georgian SDQ. They were asked using open-ended questions if translated each item retained the original meaning.
For minimizing possible cultural biases in diagnoses, all members of the diagnostic team were with a minimum of 5 years of clinical practice and 2 years of research diagnostic experience with local professionals. Additionally, a randomly chosen diagnosed sample was validated by US mental health experts (10%).
Statistical Analysis
Descriptive statistics of the data are presented with n (%) and, for non-normalized variables were given as median (min, max; IQR). To find the correlation between categorical variables, Pearson chi-square test was used. To check the normality of the distribution of continuous variables, the Kolmogorov–Smirnov and the Shapiro–Wilk tests were applied. To assess differences between independent non-parametric variables, a Mann–Whitney U test was used. To test the correlation between paired samples, the Wilcoxon test was applied. The Cronbach’s α was used to check the internal consistency of the Georgian versions of SDQ. The test–retest reliability was assessed as the Guttman split-half coefficient. A principal components and factor analysis were provided to study the construct of SDQ 5 subscales in the Georgian version. A Kaiser–Meyer–Olkin Measure of sampling adequacy was calculated. Separate scree-plots were constructed to determine the number of factors for Georgian versions of parent- and teacher-administered SDQs. When the probability value was less than .05, we concluded that a statistically significant difference does exist.
To evaluate the accuracy of the screening questionnaire, the Receiver Operating Characteristic (ROC) analysis was used. A 95% CI of the area under the curve (AUC) was calculated. Specificity, sensitivity, and positive and negative predictive values (PPV and NPV) were evaluated for parent-administered and teacher-administered SDQ scores. To determine the best balance between sensitivity and specificity for SDQs, we used the calculation of the Youden index as the sum of Specificity and Sensitivity subtracted by 100%.32 Data was analyzed by using Statistical Package for the Social Sciences (SPSS) Statistics for Windows, version 23.0 (IBM SPSS Corp.; Armonk, NY, USA).
Results
From a total of 16 654 students screened with SDQ, 67.64% (n = 11 264 ) were from Tbilisi (the biggest city), 13.71% (n = 2283) from Batumi, 12.98% (n = 2162) from Kutaisi, 3.27% (n = 545) from Zugdidi, and 2.4% (n = 400) from Telavi (Table 1), reflecting the ratio of third-grade students population in these cities. The highest value of participation was in Telavi—98.68%, then in Batumi, Kutaisi, Zugdidi, and Tbilisi (90.01%, 82.43%, 75.22%, and 69.27%, respectively).
From parents-administered 15 738 SDQ questionnaires 7847 (50.14%) were for females and 7827 (49.86%) for males. From teacher-administered 13 560, 6664 (49.24%) were for females and 6879 (50.83%) were for males.
Analyzing data of SDQ questionnaires completed by parents showed acceptable internal consistency—the Cronbach’s α was 0.625.33,34 The Guttman split-half coefficient was 0.600 which indicates acceptable test–retest reliability. A Kaiser–Meyer–Olkin Measure was 0.828 (P < .001), which indicates good sampling adequacy. Figure 1 shows the scree plot for parent-administered SDQ (Figure 1).
Figure 1.
Scree plot of optimal number of components for parent-administered strength and difficulties questionnaire.
The SDQ questionnaires completed by teachers indicated a reasonable internal consistency with Cronbach’s α 0.621. The Guttman split-half coefficient for teacher-administered SDQ was 0.614. A Kaiser–Meyer–Olkin Measure was 0.867 (P < .001), which indicates good sampling adequacy of the teacher-administered SDQ. Figure 2 shows the scree-plot for teacher-administered SDQ.
Figure 2.
Scree plot of optimal number of components for teacher-administered strength and difficulties questionnaire.
Both scree-plots for parent- and teacher-administered questionnaires show that 5-6 components are adequate for data reduction for the Georgian version of SDQ that is in line with the construct of the original version.
Descriptive statistics shoved median value for parent-administered SDQ was 16 (min. 0, max. 42; IQR 7) and for teacher-administered questionnaire 14 (min. 4, max. 39; IQR 6); this difference is statistically significant (P < .001).
Descriptive statistics revealed that the median value for male students was 17 (min. 0, max. 40; IQR 6) which was higher than those for females—16 (min. 2, max. 42; IQR 6) (from parents dataset), the difference was statistically significant (P < .001); as for teachers dataset, the median value for male students was 15 (min. 4, max. 39; IQR 6), the same parameter for female students was 14 (min. 5, max. 39; IQR 9), observed difference was statistically significant (P < .001).
The SDQ total score of all 981 students of the upper 95th percentile from questionnaires completed by parents was 26. The total scores of all 813 students of the same upper 95th percentile but from questionnaires completed by teachers was 24. Table 1 shows descriptive statistics by cities and datasets.
Three hundred individuals (298 children from parent administered and 271 from teacher administered group) were diagnostically assessed (Table 3). Among them, 210 (70.00%) children were diagnosed with neurodevelopmental disorders, in particular: 53 (25.2%) children were diagnosed with autism spectrum disorders; 57 (24.14%) with attention deficit hyperactivity disorder; 21 (10.00%) with a learning disability; 20 (9.52%) with conduct disorder; 19 (9.05%) with anxiety disorder; another 19 (9.05%) with speech and language disorders; and remaining 14 (6.66%) with tics, enuresis, or unspecified developmental disorders. The remaining 90 (30.00%) individuals weren’t diagnosed with any mental health disorders. Case identification is shown on the flow chart (Figure 3).
Table 3.
Descriptive Statistics—Strength and Difficulties Questionnaire Percentile Groups Distributed by Informant and Diagnosis
| Informant | Diagnosis | 0 to 89th% n (%) |
90th to 95th% n (%) |
95h to 96th% n (%) |
96th to 97th% n (%) |
97th to 98th% n (%) |
98th to 99th% n (%) |
99th to 100% n (%) |
Total n (%) |
|---|---|---|---|---|---|---|---|---|---|
| Parent | Typical development | 51 (58.01) | 7 (8.02) | 7 (8.02) | 6 (6.81) | 4 (4.54) | 9 (10.24) | 4 (4.51) | 88 (100.00) |
| Mental disorders | 81 (38.63) | 27 (12.91) | 19 (9.04) | 10 (4.82) | 20 (9.53) | 28 (13.33) | 25 (11.92) | 210 (100.00) | |
| Total | 132 (44.32) | 34 (11.42) | 26 (8.72) | 16 (5.42) | 24 (8.12) | 37 (12.41) | 29 (9.72) | 298 (100.00) | |
| Teacher | Typical development | 62 (75.61) | 5 (6.14) | 2 (2.41) | 3 (3.74) | 2 (2.43) | 7 (8.53) | 1 (1.23) | 82 (100.00) |
| Mental disorders | 85 (45.01) | 34 (18.03) | 13 (6.91) | 8 (4.22) | 17 (9.01) | 19 (10.12) | 13 (6.91) | 189 (100.00) | |
| Total | 147 (54.21) | 39 (14.42) | 15 (5.54) | 11 (4.11) | 19 (7.03) | 26 (9.64) | 14 (5.21) | 271 (100) |
Figure 3.
Case identification chart.
Nine hundred eighty-one children from the parent-administered questionnaire scored above 95th percentile. Among them, 132 individuals were undergone final diagnosis, and 102 (77.27%) were diagnosed with developmental disorders. On the other hand, 14 140 individuals scored below 90th percentile and among them, 132 individuals were assessed for final diagnosis, where 81 (61.36%) individuals showed developmental disorders. In summary, more individuals from the upper 95th percentile were diagnosed with neurodevelopmental disorders compared to those, from those, scored below 90th percentile; this difference is statistically significant (P = .005).
Eight hundred thirteen children scored above 95th percentile according to teacher-administered questionnaire; 85 of them were assessed for final diagnosis and 70 (82.35%) were diagnosed with developmental disorders. 11 851 individuals scored below 90th percentile, 147 of them were further assessed for final diagnosis, and in 85 (57.82%) cases some neurodevelopmental disorders were revealed. More children from upper 95th percentile were diagnosed with developmental disorders compared to those from lower 90th percentile (P < .001).
According to the parent-administered questionnaire, out of 210 individuals, diagnosed with some neurodevelopmental disorders, 102 (48.57%) were from the upper 95th percentile. On the other hand, among 88 children with typical development, 30 (34.09%) were from the upper 95th percentile. As for the teacher-administered questionnaire, from 189 children diagnosed with any neurodevelopmental disorders, 70 (37.03%) of them were from the upper 95th percentile, whereas among 82 children with typical development, 15 (18.29%) were scored 95th percentile and more. For more detailed information, see Table 3.
For assessment of discriminative validity of the SDQ in recognizing mental disorders, the ROC analyses were used. Figure 4 shows ROC curves of parent-administered, teacher-administered, and the maximum SDQ.
Figure 4.
Receiver operational characteristic curve of strength and difficulties questionnaire.
The AUC indicates good discriminant ability of the SDQ: 0.629 (95% CI 0.556-0.702) for parent-dataset and 0.680 (95% CI 0.611-0.789) for teacher-dataset. Table 4 displays the specificity, sensitivity, NPV, and PPV for the optimal cut-off considering the ROC analyses. The cutoff score for parent-dataset SDQ was 17 or more, giving the highest Youden index 17.0%, sensitivity 0.898, and specificity 0.268. The cutoff score for teacher-dataset SDQ was 15 or more with Youden index 19%, sensitivity 0.840, and specificity 0.354.
Table 4.
Gender-Specific Parameters of Sensitivity, Specificity, Positive Predictive Value, and Negative Predictive Value of the Parent- and Teacher-Administered Strength and Difficulties Questionnaire
| Informant | ROC Area Under Curve (95% CI) | P | Cutoff Value | Sensitivity | Specificity | PPV | NPV | Youden Index | |
|---|---|---|---|---|---|---|---|---|---|
| Total | Parent SDQ | 0.629 (0.556-0.702) | <.001 | ≥17 | 0.898 | 0.268 | 0.735 | 0.537 | 0.17 |
| Teacher SDQ | 0.680 (0.611-0.789) | <.001 | ≥15 | 0.840 | 0.354 | 0.741 | 0.492 | 0.19 | |
| Males | Parent SDQ (n = 204) | 0.651 (0.561-0.740) | <.001 | ≥17 | 0.905 | 0.308 | 0.777 | 0.552 | 0.21 |
| Teacher SDQ (n = 191) | 0.665 (0.578-0.751) | <.001 | ≥16 | 0.832 | 0.385 | 0.783 | 0.453 | 0.22 | |
| Females | Parent SDQ (n = 94) | 0.594 (0.465-0.724)* | .213 | — | — | — | — | — | — |
| Teacher SDQ (n = 84) | 0.686 (0.567-0.805) | .023 | ≥15 | 0.780 | 0.467 | 0.709 | 0.560 | 0.25 |
NPV, negative predictive value; PPV, positive predictive value; SDQ, Strength and Difficulties Questionnaire.
*Non-significant.
We did not find statistically significant difference (Table 4) indicating gender-specific discriminative ability of the parent-administered SDQ (P = .213). Details are provided in Table 4.
Discussion
For validation of Georgian SDQ, population-based study was used covering 47% of third-grade children of elementary public schools in the Republic of Georgia. A total of 16 654 children were assessed by a parent and/or teacher completing the SDQ. Screen positive children then underwent standardized diagnostic assessment.
The SDQ is widely used and validated for use in different countries and different languages for identification of children at risk for mental disorders.5,6,7,9,10,11,13,14,16 These studies suggested new cut-offs for the SDQ scales based on the assessed population and suggested new norms to standardize the SDQ.
To the best of our knowledge, this is the first study to validate Georgian SDQ in a representative population sample of third-grade schoolchildren considering parents’ and teachers’ reports. In particular, we assessed the ability of total score in identifying children with emotional and behavioral disorders–based comprehensive assessment using standardized diagnostic procedures. Furthermore, we compared the degree of agreement between parent and teacher rating, and child gender. Overall, the current research confirms that Georgian version of SDQ with a five-factor solution suggested by Goodman is a reliable and valid instrument to identify mental health problems from both informant’s perspective. Internal consistency of the Georgian version of the SDQ showed comparable results to the Goodman study, where the mean Cronbach’s α was 0.73.35
Our study revealed difference between parents’ and teachers’ cutoff scores, which is in concordance with studies of Murray and colleagues provided in UK,18 confirming clinical observations that problems can differ in different settings. The same was mentioned in Goodman’s 2001 study.35 Despite the fact that the sensitivity of cutoff scores of both informants was high, predictive value of parents’ information was better and they scoring higher than teachers. The reason for this difference may be parent’s more close relationship with their children and spending more hours with them, than teachers. Also, parents react more sensitively to their children’s externalizing behaviors.36 Also, Georgian parents and teachers may consider child behavior as atypical when a child behaves in a culturally inappropriate manner. So, cultural context should be considered in the validation of any instruments.37
In this study, we found gender differences in cut-offs from either informant, which is in line with Espanol-Martin, Gray’s, Levantini’s, Muratori’s, and Murray’s studies.6,7,16,17,18
Considering recent report of UNICEF,4 showing low awareness and limited resources in Georgia, like other LMICs, use of the SDQ can improve early identification of mental health problems and promote development of much-needed services. The study was part of a larger research project on children’s mental health and validation of the Autism Spectrum Screening Questionnaire was already published.38
This study has some limitations. Part of the instruments we used in the study were not validated for use in Georgian children yet. Further, the final best estimate diagnoses were not based on the scores and algorithms of the instruments, they were based on judgment of clinical experts’ team. Our sample included only children of limited age range. Future studies should explore the validity of SDQ for full age range. We did not find reliable discriminative ability of parent-administered questionnaire to identify girls at risk for mental disorders. Additional efforts are required to find reasons and ways to improve Georgian SDQ for female children.
Culturally adapted Georgian SDQ is an effective screening tool for identification of children at risk for mental disorders in Georgian elementary school setting and does not depend on informant. It appears that both parent- and teacher-completed SDQ are equally valid, but with different cutoff scores.
The results of our study may help policymakers in priority setting, resource allocation, planning of service development for the promotion of mental health, and prevention of child mental disorders considering the country context.
Funding Statement
This work was supported by Shota Rustaveli National Science Foundation of Georgia (SRNSFG) [grant number FR-18-304], Caritas Czech Republic in Georgia [grant number GE39].
Footnotes
Ethics Committee Approval: This study was approved by Ethics Committee and Institutional Review Board of the National Desease Control Center of Georgia (Approval No: 2019-032, Date: 1.07.2019).
Informed Consent: Written informed consent was obtained from the parents of each child participated in the study.
Peer-review: Externally peer-reviewed.
Author Contributions: Concept – M.G., B.L.L., Y.S.K.; Design – M.G., M.Z., N.M., B.L.L., Y.S.K.; Supervision – B.L.L., Y.S.K.; Resources – M.Z., G.L.; Materials – M.G., G.L., B.L.L., Y.S.K.; Data Collection and/or Processing – M.G., M.Z., N.M., T.M.; Analysis and/or Interpretation – M.G., M.Z., G.L., T.M., B.L.L., Y.S.K.; Literature Search – M.G., M.Z., T.M., N.M.; Writing – M.G., M.Z., T.M., N.M., G.L.; Critical Review – B.L.L., Y.S.K.
Acknowledgments: We are grateful to the children, parents, and teachers participating in the study and to the other members of the project group for making the study possible. Special thanks to Inga Zarandia, Lika Mkhatvari, Irma Nodia, Nino Paatashvili and their teams for help at screening stage.
Declaration of Interests: The authors have no conflict of interest to declare.
References
- 1. Rathod S, Pinninti N, Irfan M, et al. Mental health service provision in low- and middle-income countries. Health Serv Insights. 2017;10:1178632917694350. ( 10.1177/1178632917694350) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Davies SC, Lemer C, Strelitz J, Weil L. Our children deserve better: prevention pays. Lancet. 2013;382(9902):1383 1384.( 10.1016/S0140-6736(13)62004-8) [DOI] [PubMed] [Google Scholar]
- 3. McGorry PD, Mei C. Early intervention in youth mental health: progress and future directions. Evid Based Ment Health. 2018;21(4):182 184. ( 10.1136/ebmental-2018-300060) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. united nations children’s fund, United Nations Children’s Fund. The state of the World's children. Indian J Pediatr. 1988;55(2):294 294. ( 10.1007/BF02722201) [DOI] [Google Scholar]
- 5. Mieloo CL, Bevaart F, Donker MC, van Oort FV, Raat H, Jansen W. Validation of the SDQ in a multi-ethnic population of young children. Eur J Public Health. 2014;24(1):26 32. ( 10.1093/eurpub/ckt100) [DOI] [PubMed] [Google Scholar]
- 6. Español-Martín G, Pagerols M, Prat R, et al. Strengths and difficulties questionnaire: psychometric properties and normative data for Spanish 5- to 17-year-olds. Assessment. 2021;28(5):1445 1458. ( 10.1177/1073191120918929) [DOI] [PubMed] [Google Scholar]
- 7. Gray EJ, Scott JG, Lawrence DM, Thomas HJ. Concordance between adolescents and parents on the Strengths and Difficulties Questionnaire: analysis of an Australian nationally representative sample. Aust N Z J Psychiatry. 2021;55(11):1058 1070. ( 10.1177/00048674211009610) [DOI] [PubMed] [Google Scholar]
- 8. Lu C, Li Z, Patel V. Global child and adolescent mental health: the orphan of development assistance for health. PLOS Med. 2018;15(3):e1002524. ( 10.1371/journal.pmed.1002524) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Alyahri A, Goodman R. Validation of the Arabic strengths and difficulties questionnaire and the development and well-being assessment. East Mediterr Health J. 2006;12(suppl 2):S138 S146. [PubMed] [Google Scholar]
- 10. Gomez R, Motti-Stefanidi F, Jordan S, Stavropoulos V. Greek validation of the factor structure and longitudinal measurement invariance of the strengths and difficulties questionnaire-self report (SDQ-SR): exploratory structural equation modelling. Child Psychiatry Hum Dev. 2021;52(5):880 890. ( 10.1007/s10578-020-01065-7) [DOI] [PubMed] [Google Scholar]
- 11. Santiago PHR, Manzini Macedo D, Haag D, et al. Exploratory Graph Analysis of the Strengths and Difficulties Questionnaire for Aboriginal and/or Torres Strait Islander Children. Front Psychol. 2021;12:573825. ( 10.3389/fpsyg.2021.573825) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Goodman R. The extended version of the Strengths and Difficulties Questionnaire as a guide to child psychiatric caseness and consequent burden. J Child Psychol Psychiatry. 1999;40(5):791 799. ( 10.1111/1469-7610.00494) [DOI] [PubMed] [Google Scholar]
- 13. Silva TB, Osório FL, Loureiro SR. SDQ: discriminative validity and diagnostic potential. Front Psychol. 2015;6:811. ( 10.3389/fpsyg.2015.00811) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Aoki A, Ganchimeg T, Naranbaatar N, et al. Validation of the parent version of the Strengths and Difficulties Questionnaire (SDQ) to screen mental health problems among school-age children in Mongolia. BMC Psychiatry. 2021;21(1):218. ( 10.1186/s12888-021-03218-x) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Black L, Mansfield R, Panayiotou M. Age appropriateness of the self-report strengths and difficulties questionnaire. Assessment. 2021;28(6):1556 1569. ( 10.1177/1073191120903382) [DOI] [PubMed] [Google Scholar]
- 16. Levantini V, Cei E, Pirri G, Muratori P. Psychometric properties of the strengths and difficulties questionnaire-dysregulation profile in Italian early adolescents. Children (Basel). 2021;8(12):1123. ( 10.3390/children8121123) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Muratori P, Paciello M, Castro E, et al. At-risk early adolescents profiles in the community: a cluster analysis using the strengths and difficulties questionnaire. Psychiatry Res. 2021;305:114209. ( 10.1016/j.psychres.2021.114209) [DOI] [PubMed] [Google Scholar]
- 18. Murray AL, Speyer LG, Hall HA, Valdebenito S, Hughes C. Teacher versus parent informant measurement invariance of the strengths and difficulties questionnaire. J Pediatr Psychol. 2021;46(10):1249 1257. ( 10.1093/jpepsy/jsab062) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Aarø LE, Davids EL, Mathews C, Wubs AG, Smith ORF, de Vries PJ. Internalizing problems, externalizing problems, and prosocial behavior - three dimensions of the strengths and difficulties questionnaire (SDQ): A study among South African adolescents. Scand J Psychol. 2022;63(4):415 425. ( 10.1111/sjop.12815) [DOI] [PubMed] [Google Scholar]
- 20. Bunting L, McCartan C, Davidson G, et al. Rationale and methods of the 'Northern Ireland Youth wellbeing Survey' and initial findings from the Strengths and Difficulties Questionnaire. Clin Child Psychol Psychiatry. 2022;27(3):670 685. ( 10.1177/13591045221075525) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Kaiser S, Halvorsen MB. The Strengths and Difficulties Questionnaire self-report-, parent-, and teacher version in children with intellectual and developmental disabilities. Res Dev Disabil. 2022;123:104194. ( 10.1016/j.ridd.2022.104194) [DOI] [PubMed] [Google Scholar]
- 22. Green AE, Weinberger SJ, Harder VS. The strengths and difficulties questionnaire as a mental health screening tool for newly arrived pediatric refugees. J Immigr Minor Health. 2021;23(3):494 501. ( 10.1007/s10903-020-01082-7) [DOI] [PubMed] [Google Scholar]
- 23. Murray AL, Speyer LG, Hall HA, Valdebenito S, Hughes C. A longitudinal and gender invariance analysis of the strengths and difficulties questionnaire across ages 3, 5, 7, 11, 14, and 17 in a Large U.K.-Representative Sample. Assessment. 2022;29(6):1248 1261. ( 10.1177/10731911211009312) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Ribeiro Santiago PH, Manzini D, Haag D, Roberts R, Smithers LG, Jamieson L. Exploratory graph analysis of the strengths and difficulties questionnaire in the longitudinal study of Australian children. Assessment. 2022;29(8):1622 1640. ( 10.1177/10731911211024338) [DOI] [PubMed] [Google Scholar]
- 25. united nations children’s fund, United Nations Children’s Fund. Children and youth in Georgia; 2020. Available at: https://www.unicef.org/georgia/media/5916/file/Childrens%20and%20Youth%20in%20georgia%20-%20web.pdf [Google Scholar]
- 26. GeoStat. Georgian population; 2021. Available at: https://www.geostat.ge/en/modules/categories/41/population [Google Scholar]
- 27. Georgian Government. Strategic plan of mental health of Georgia; 2022. Available at: https://matsne.gov.ge/ka/document/view/5357283?publication=0 [Google Scholar]
- 28. Georgia. Data. n.d. Accessed January 2, 2023 https://data.worldbank.org/country/georgia [Google Scholar]
- 29. WHO - World Health Organization. International Statistical Classification of Diseases and Related Health Problems. 10th ed. Geneva, Switzerland: World Health Organization; 2010. [Google Scholar]
- 30. SDQ. The Strengths and Difficulties Questionnaire, Georgian version. Available at: https://www.sdqinfo.org/py/sdqinfo/b0.py [Google Scholar]
- 31. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Washington, D.C.: American Psychiatric Publishing; 2013. [Google Scholar]
- 32. Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3(1):32 35. ( 10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3) [DOI] [PubMed] [Google Scholar]
- 33. Daud KA, Khidzir NZ, Ismail AR, Abdullah FA. Validity and reliability of instrument to measure social media skills among small and medium entrepreneurs at Pengkalan Datu River. Int J Dev Sustain. 2018;7(3):1026 1037. [Google Scholar]
- 34. Taber KS. The use of Cronbach’s alpha when developing and reporting research instruments in science education. Res Sci Educ. 2018;48(6):1273 1296. ( 10.1007/s11165-016-9602-2) [DOI] [Google Scholar]
- 35. Goodman R. Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry. 2001;40(11):1337 1345. ( 10.1097/00004583-200111000-00015) [DOI] [PubMed] [Google Scholar]
- 36. Cheng S, Keyes KM, Bitfoi A, et al. Understanding parent-teacher agreement of the Strengths and Difficulties Questionnaire (SDQ): comparison across seven European countries. Int J Methods Psychiatr Res. 2018;27(1):e1589. ( 10.1002/mpr.1589) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Zirakashvili M, Gabunia M, Tatishvili N, et al. Cultural adaptation and psychometric validation of the ages and stages questionnaires for use in Georgia. J Child Fam Stud. 2018;27(3):739 749. ( 10.1007/s10826-017-0917-z) [DOI] [Google Scholar]
- 38. Zirakashvili M, Gabunia M, Mebonia N, et al. Adaptation of autism spectrum screening questionnaire (ASSQ) for use in Georgian school settings. J Public Ment Health. 2022;21(4):309 322. ( 10.1108/JPMH-03-2022-0028) [DOI] [Google Scholar]