Mental health literacy and intervention program adaptation in the internationalization of school psychology for Vietnam

Abstract

This paper discusses how school psychology technology developed in Western countries can be adapted for global contexts and “internationalized.” The article reports results of two studies, providing examples of: (a) our school psychology internationalization experiences in Vietnam, as lessons hopefully useful for other professionals interested in international development; and (b) how Western researchers can learn through internalization experiences. Because mental health literacy is foundational for mental health development, Study 1 focused on assessment of mental health literacy among 353 Vietnamese teachers, with findings suggesting overall low mental health literacy among these participants. Study 2 focused on our Vietnam ACES ProS high-school problem-solving therapy program. We discuss the Mental Health Capacity Development Model guiding development of ACES ProS and report positive results of an evaluation of ACES ProS involving 100 Vietnamese high-school students. Program cultural adaptation (e.g., deciding whether teacher classroom praise should be excluded from classroom behavior management due to Vietnamese students’ tendency to react to praise with increased competitive behavior) is reviewed as an example of challenges faced in school psychology internationalization. This program of research shows that school psychology internationalization can be successful but requires careful attention and close collaboration.

General Introduction

School and educational psychology have been important factors in supporting students’ educational outcomes, including their academic, socio-emotional, and behavioral functioning. For the most part, school psychology has been developed and utilized in high-income countries (HIC) in North America and Europe, and less in low- and middle-income countries (LMIC) around the world. However, there has been increasing interest in the internationalization of school psychology (Begeny, 2018). Goals of school psychology internationalization are not only to adapt and transfer technology and knowledge across different parts of the world (e.g., from HIC to LMIC), but also for all countries to benefit through a broadening of the perspective of school psychology, promoting diversity, and making school psychology more truly universal by understanding how it varies across international cultures (Begeny, 2018). The current paper focuses on the Southeast Asian LMIC of Vietnam, on two studies involving child mental health and life functioning. The paper also discusses how school psychology concepts and technology can be, or have been, adapted for international settings beyond those in which they were developed.

Vietnam and School-based Mental Health

Vietnam is the thirteenth most populous country in the world, with 40% of its population from 0–24 years old (Central Intelligence Agency, 2017). The prevalence of significant mental health problems among Vietnamese children aged 6–16 years is around 13% (Weiss et al., 2014). Unfortunately, development of psychological and mental health services in Vietnam is still in its early stages, particularly in schools (Le, Hagans, Powers, & Hass, 2011). Despite the high prevalence of child mental problems in Vietnam and growing evidence for the effectiveness of school-based mental health (SBMH) programs in LMIC, actual implementation has been limited. In Vietnam, the International Classification of Diseases (ICD) and Diagnostic and Statistical Manual for Mental Disorders (DSM) systems are used within the medical system (Vietnam Military Medical College, 2005), but the extent to which these Western-based formal knowledge bases are applied outside of the medical system (e.g., within education) is limited.

Child mental health is a global challenge, with the prevalence of mental health disorders in children and adolescents high and increasing in both HIC and LMIC (World Health Organization, 2012). Addressing child mental health needs is particularly challenging in LMIC. In most LMIC there is a significant lack of both personnel and physical mental health resources; this lack is particularly pronounced in regards to children (Patel, Kieling, Maulik, & Divan, 2013; Weiss, Dang, Ngo, Pollack, Sang, Lam, et al., 2011). Further, LMIC tend to have lower levels of mental health literacy (e.g., understanding what mental health problems are, and are not) and higher levels of mental health-related stigma, which contributes to decreased treatment seeking (Wei, Hayden, Kutcher, Zygmunt, & McGrath, 2013).

SBMH is an efficient approach to address these challenges. Advantages of SBMH for LMIC include: (a) use of pre-existing physical infrastructure (i.e., schools); (b) reduced stigma due to service provision within educational rather than a “mental” health context; (c) increased treatment access due to children attending school; and (d) task-shifting of service provision to child-focused education personnel (e.g. teachers); (Dang, Weiss, Nguyen, Tran, & Pollack, 2017; Fazel, Patel, Thomas, & Tol, 2014). Consequently, there has been increasing interest in SBMH in LMIC. In a recent review, Fazel et al. (2014) identified 22 studies of SBMH programs in LMIC, with 11 evaluated using a randomized design. However, none involved “pure treatment interventions” (p. 391), 36% did not show positive outcomes, and most focused on special populations (e.g., children in war zones). Results from this and other reviews (Barry, Clarke, Jenk, & V, 2013; Dang, Weiss, et al., 2017) suggest: (a) SBMH programs have significant potential but programs must be developed carefully; (b) it is critical that programs target and evaluate the main outcomes of interest (e.g., emotional and behavioral mental health problems); and (c) although programs targeting special populations are important (e.g., children in war zones), programs also should be adapted and evaluated for the more general LMIC context.

In science, the value of guiding models has long been recognized (Hegselmann, Müller, & Troitzsch, 1996). One model potentially useful for internationalization of school psychology is the Mental Health Capacity Development Model (Weiss et al., 2012), designed for more efficient and effective global child mental health development (Weiss et al., 2011). This model focuses on key factors believed to underlie ultimate development success: Capacity Development Goals of (1a) demonstrated program efficacy and (1b) program sustainability, with (b) Capacity Development Targets (e.g., research capacity development to evaluate program efficacy; development of local supervision capacity to ensure program sustainability) used to achieve the two goals. Below we discuss application of this model when developing school psychology services in Vietnam.

Study 1: Assessing Teachers’ Mental Health Literacy in Vietnam

Regardless of the country or context, a central foundation to child—and all—mental health development is mental health literacy, first defined by Jorm (2000) as “knowledge and beliefs about mental disorders and their treatment, which aid their recognition, management or prevention” (p. 396). Mental health literacy is conceptualized as having several components, including: (a) ability to recognize mental health and related problems; (b) knowledge and beliefs about risk factors and causes; (c) self-help awareness and skills; (d) understanding of professional help; and (e) attitudes facilitating recognition and appropriate help-seeking. In school settings, educators play a central role in mental health literacy. For instance, teachers can promote mental health for students by providing psychologically supportive classrooms. Further, teachers are often the first adult to identify children’s mental health needs and make referrals for services, and they are a key source of implementing classroom mental health programs. For all of these roles, teachers’ mental health literacy is crucial (Reinke, Stormont, Herman, Puri, & Goel, 2011). The goal of Study 1 was to assess levels of mental health literacy among Vietnamese teachers, which also helped to determine the extent to which mental health literacy training needed to be emphasized in our school psychology mental health programs.

Method

Participants.

Participants were recruited at the annual four-day national workshops for teachers of students in grades 6–12 conducted by the Vietnamese Ministry of Education and Training (it should be noted that these workshops, at least to date, do not provide training or discussion of mental health literacy). Of 421 eligible teachers, 353 (84%) from 33 of the 58 provinces of Vietnam participated. The average age of the teachers was 35.4 years (SD=6.36, range=22 to 58 years), with 65% male, and years of teaching experience ranging from 1 to 36 years (M=12.83, SD=6.39). Twenty-five percent of teachers had a master’s degree, 72% had a bachelor’s degree, and the remaining 3% had either a community college or high-school degree.

Measures.

The Mental Health Literacy Scale (MHLS; O’Connor & Casey, 2015) was used to assess teachers’ knowledge and attitudes regarding mental health and related constructs. The English language MHLS includes 35 items from six domains of mental health literacy: (a) ability to recognize mental health disorders; (b) knowledge of risk factors and causes; (c) knowledge of where to seek information; (d) knowledge of self-help treatment; (e) knowledge of professional help available; and (f) attitudes promoting recognition and help-seeking behavior. The first eight items are rated on a four point Likert scale, ranging from Very Unlikely True (e.g., regarding the stated mental health fact) to Very Likely True. The remaining items are rated on a five point Likert scale, ranging from Strongly Disagree (for stigma and other items; e.g., “People with a mental illness are dangerous”) to Strongly Agree. The MHLS produces a Total Score (with the stigma items reverse scored) representing overall level of mental health literacy (O’Connor & Casey, 2015). The six domains of the MHLS are not conceptualized as factors or subscales of the MHLS, but simply as areas within mental health literacy covered by the MHLS. O’Connor and Casey’s (2015) report suggests that the English version MHLS has acceptable validity and reliability (e.g., Cronbach’s alpha for the Total Score = .87; test-retest reliability = .80).

A key part of internationalization of school psychology technology often involves cross-cultural adaptation of assessment instruments (Dang, Weiss, Pollack, & Nguyen, 2011). For the MHLS adaptation, our bilingual team reviewed item appropriateness for Vietnam, and ultimately excluded (a) two items relating to gender as a risk factor for mental health problems—omitted because there is no evidence regarding their validity in Vietnam, and (b) two items regarding conditions under which mental health professionals may break confidentiality—excluded because Vietnam does not yet have a formal code of ethics defining professional confidentiality. The measure was translated and back-translated following standard procedures (Van de Vijver & Poortinga, 2004).

The MHLS Vietnamese version (MHLS-VN) includes 31 items covering four domains of mental health literacy: (a) ability to recognize mental health disorders (8 items); (b) knowledge of where to seek mental health-related information (4 items); (c) knowledge about mental health treatments (3 items); and (d) attitudes (e.g., stigma) that promote or inhibit mental health help-seeking (16 items). Cronbach’s alpha for the MHLS-VN Total Score was .72. The maximum Total Score for the MHLS-VN is 144 and minimum 31. To account for the item deletions and allow for comparison between the MHLS-VN and the original MHLS norms, the MHLS-VN Total Score was adjusted by taking the mean of the items within the MHLS-VN and multiplying by the number of items in the original English language MHLS to equate the two total scores.

Procedures.

Teachers were recruited at the annual training workshop provided in the three regions of the country by the Vietnamese Ministry of Education and Training. In each of the workshops, middle- and high-school teachers were asked on the second day of the workshop to participate in a survey regarding their understanding of children’s mental health. A project coordinator uninvolved in the workshop described the study to the teachers including its voluntary nature, answered questions about the study, and then obtained signed consent from interested teachers. Questionnaires were given to participants to complete during their free time and returned to the researchers. Data were collected from January 2016 to March 2016. The study was approved by the USA FWA IRB (#00018223) at Vietnam National University.

Analyses.

Analyses first focused on comparisons of the MHLS-VN Total Score to MHLS Australian norms, to assess levels of Vietnamese teachers’ mental health literacy relative to Western norms. We also assessed the extent to which Vietnamese teachers’ mental health literacy correlated with their age, number of years teaching, and level of education to understand how mental health literacy might vary across teacher-related characteristics. Second, we assessed rates of endorsement of individual MHLS-VN items in regards to participants’ stigma-related beliefs. If the participant responded as “Agree” or “Very strongly agree” to an item (e.g., “People with a mental illness could snap out if it if they wanted”) then they were considered to show stigma in regards to that item; if they responded “Strongly Disagree”, “Disagree” or “Neither agree or disagree” then they were considered to not show stigma for the item.

Results

The mean MHLS-VN adjusted Total Score was 106.15 (SD=10.37), approximately 1.7 standard deviations below the Australian norm group mean of 127.23 (SD=12.63; O’Connor & Casey, 2015), indicating relatively lower levels of mental health literacy among the Vietnamese teachers. Male and female teachers did not differ significantly on MHLS-VN Total Score (M=105.9 vs. 106.6, respectively, t[350] = .58, ns). Correlations with the MHLS-VN Total Score were: Age, r = −.10 (ns); Years Teaching, r = −.08 (ns), Level of Teacher Education, r = −.06 (ns). The Kolmogorov-Smirnov Test (Stephens, 1974) was used to determine whether analyses met parametric assumptions. For all three analyses, p > .15 indicating the residuals did not deviate significantly from a normal distribution and that correlational analyses were thus appropriate.

With regard to mental illness-related attitudes, 77% of teachers endorsed one or more of the nine stigma items, with 39% indicating they believed people with mental illness are dangerous, and 35% indicating that seeing a mental health professional means you are not strong enough to manage your own difficulties (see Table 1). Ninety-three percent of teachers failed to endorse one or more positive attitude items. Small minorities reported accepting attitudes towards individuals with mental illness, with 18% being willing to employ someone with mental illness, 20% willing to vote for a politician with mental illness, and 23% willing to have someone with mental illness marry into their family (see Table 1). Teachers were somewhat more open to less committed relationships, with 56% willing to spend an evening socializing with someone with mental illness, and 42% willing to work with closely with someone with mental illness.

Table 1.

Rates of endorsement of stigma attitudes, and mental health supportive attitudes on the Mental Health Literacy Scale-Vietnamese versions (MHLS-VN)

% endorsing	Item
20%	20. People with a mental illness could snap out if it if they wanted
18%	21. A mental illness is a sign of personal weakness
25%	22. A mental illness is not a real medical illness
39%	23. People with a mental illness are dangerous
7%	24. It is best to avoid people with a mental illness so that you don’t develop this problem
11%	25. If I had a mental illness I would not tell anyone
35%	26. Seeing a mental health professional means you are not strong enough to manage your own difficulties
11%	27. If I had a mental illness, I would not seek help from a mental health professional
10%	28. I believe treatment for a mental illness, provided by a mental health professional, would not be effective
30%	29. How willing would you be to move next door to someone with a mental illness?
56%	30. How willing would you be to spend an evening socializing with someone with a mental illness?
40%	31. How willing would you be to make friends with someone with a mental illness?
42%	32. How willing would you be to have someone with a mental illness start working closely with you on a job?
23%	33. How willing would you be to have someone with a mental illness marry into your family?
20%	34. How willing would you be to vote for a politician if you knew they had suffered a mental illness?
18%	35. How willing would you be to employ someone if you knew they had a mental illness?

Notes. Items 20 to 28 are stigma items, items 29 to 35 are mental health supportive items

Discussion

Although it is important to consider limitations to Study 1 (e.g., survey data may not necessarily reflect respondents’ actual beliefs or behavior; the findings may not generalize to other educators, such as those in primary school; it is unknown how the overall results compare to educators from other HIC and LMIC countries outside of Australia), this study is the first national survey of teacher mental health literacy in Vietnam. Results indicated generally low levels of mental health literacy, with the average Vietnamese teacher at the 5^th percentile of the Australian norms. Relations between mental health literacy, and teacher age, length of time teaching, and level of education were non-significant, differing from some prior studies in HIC and LMIC. For instance, some studies in the USA and Norway have found that teaching experience may increase understanding and awareness of students’ mental health problems, likely because of working with such students in their class (Ekornes, Hauge, & Lund, 2012). In contrast, other studies in the USA, Ethiopia, and India have found that teachers with less teaching experience had better mental health literacy compared to their senior colleagues (Kerebih, Abrha, Frank, & Abera, 2016; Koller, Osterlind, Paris, & Weston, 2004; Mendonsa & Shibabuddeen, 2013), perhaps because more recently trained teachers received more updated pre-service training. The lack of relations between mental health literacy and teaching experience or teachers’ age may reflect that in Vietnam, child mental health is not a significant part of teacher training, in the past or present. Consequently, teacher mental health literacy is a central part our development of SBMH programs in Vietnam, as discussed below.

Study 2: School-Based Mental Health in Vietnam

Background

We are not aware of any structured school-based mental health programs in Vietnam prior to 2011, and our programs discussed below were among the first formal school-based mental health programs implemented in the country. Our team has focused on two programs, the Reaching Educators, Children and Parents in Vietnam (RECAP-VN; Dang et al., 2017) Program for elementary school students, in Hanoi, and our problem solving therapy program (ACES ProS) for high-school students, in Danang. Study 2 focuses on the development and evaluation of ACES ProS, but we start by discussing RECAP-VN because technical development of ACES ProS was derived from our experiences with RECAP-VN and information about RECAP-VN should offer useful details regarding internationalization processes in school psychology.

RECAP-VN: Development, Adaptation, and the Collaboration Process

RECAP-VN was the first SBMH program formally research-evaluated in Vietnam (Dang et al., 2017). Its foundation is the RECAP program developed and evaluated in the USA (Han, Catron, Weiss, & Marciel, 2005; Weiss, Harris, Catron, & Han, 2003). RECAP is a school-based training program for elementary school children with emotional and behavioral problems. The Western intervention theories upon which RECAP is based include a strong emphasis on positive reinforcement of desired behavior, cognitive behavior therapy for developing students’ inter- and intra-personal skills, and development of social classroom norms to support adaptive functioning. In the USA, RECAP involves (a) classroom groups, (b) classroom teachers, (c) small-group sessions with RECAP participants, (d) individual sessions with RECAP participants, and (e) parents. In Vietnam, to support sustainability (the second Capacity Development Goal),program complexity was reduced with RECAP-VN so that it focused only on two components: the universal classroom program and teacher classroom behavior management. In short, RECAP-VN supports teachers’ use of effective classroom strategies (e.g., emphasizing reinforcement of desired behaviors over punishment of undesired behaviors) and provides classroom training to students in social and emotional health skills. The program was first implemented in Hanoi, Vietnam by clinical psychology graduate students from the College of Education, Vietnam National University. After three years of supported program implementation, local teachers have taken responsibility for the program (e.g., they continue to implement and support it), which indicates at least initial success in achieving sustainability.

Mental health literacy is a foundation for mental health intervention, but as results of Study 1 indicate, Vietnamese teachers’ overall level of mental health literacy appears low. Thus, a key adaptation for RECAP-VN was an increased emphasis on mental health literacy, with development and provision of a culturally-specific teacher mental health literacy component. In the teacher program training, a session on recognition of common mental health problems (e.g. anxiety, depression, ADHD) as mental health symptoms rather than as “bad behavior” or “being lazy” was added. Another important adaptation issue was how positive reinforcement should be utilized. In many Western societies, positive reinforcement is a central component of most social skills, behavioral, and cognitive mental health interventions for students and young people, based on the fundamental principle of increasing desired behavior through the use of positive reinforcement. In Vietnam, one complexity we have encountered when working with students – in particular high-school and older students – is how they respond to public verbal praise of desired behavior. Our experience is that in addition to increasing targeted behavior (e.g., asking the teacher a question), because Vietnamese society is collectivistic and hierarchical (Feng, Qiang, Yu, & Ti, 2014), public praise from someone high in the hierarchy (i.e., a teacher) may result in a student feeling superior towards other students not praised and lead to increased competitive behavior within the student classroom hierarchy. We consequently had extended discussions with elementary school teachers to determine how classroom verbal praise might need to be modified or potentially even dropped. However, it was concluded that this issue only became problematic later in middle-school and beyond, so teacher classroom verbal praise was retained in RECAP-VN. Why Vietnamese middle or high school students react with feelings of superiority towards peers when receiving teacher positive reinforcement, whereas Western students generally do not, is not clear. This may be an important area for future research, and is another example of how internationalization processes in school psychology may be useful for extending our understanding of human behavior more broadly across various cultures (e.g., potential limitations to classroom praise within and outside of LMIC contexts).

After RECAP-VN had been implemented in Vietnam for two years, its efficacy was evaluated in an experimental study among 443 second grade students, with the study approved by the USA FWA IRB (#00018223) at Vietnam National University. The outcome evaluation included an assessment of the effect of risk status, to determine whether students with (high risk) and without (low risk) current mental health problems benefited equally from the program (Dang et al., 2017). Significant treatment main effects were found on both social skills and mental health functioning. However, the interaction between Risk Status and Treatment was significant for social skills but not for mental health outcomes, suggesting that different mechanisms may underlie program effects for high versus low risk students.

ACES ProS Development and Adaptation

RECAP-VN is an elementary school SBMH program, successfully adapted for the LMIC context of Vietnam. However, it is important that SBMH cover the grade and age range of school-aged students, including adolescents. In young people from 15–25 years old, mental health disorders are the single most frequent cause of disability (World Health Organization, 2012), and adolescent years are a critical window during which mental health can be promoted and mental health problems effectively addressed. The second study upon which this paper focuses is an evaluation of our adolescent-focused ACES ProS high-school mental health program. The foundation of ACES ProS is Problem Solving Therapy (Nezu & Nezu, 2013), an evidenced-based intervention based on research indicating (a) non-adaptive problem-solving is as a major risk-factor for a variety of mental health problems, and (b) people’s natural problem-solving potential can be compromised by non-adaptive problem-solving (e.g., impulsive instead of thoughtful) and/or a failure in emotion regulation leading to increased non-adaptive coping (Nezu & Nezu, 2013). We continue this section by discussing the ACES ProS adaptation process, challenges we faced, internationalization concepts we considered, and then report methods and findings from the first formal evaluation of the ACES ProS program.

General development context and considerations.

The impetus for ACES ProS came from a meeting between our research team and the Danang Provincial Department of Education where we were discussing a USA NIH-funded mental health risk-factor study we were conducting in Danang high-schools (e.g., Weiss, Nguyen, Trung, Ngo, & Lau, 2018). Upon hearing study results and of his high school students’ strong interest in mental health issues, the superintendent offered that if we would provide support, he would hire Bachelor’s-level psychologists from Vietnam and place them in Danang high-schools to support students. This led to developing the ACES ProS SBMH program, which has been overseen by the Psychology Department at the Danang Psychiatric Hospital.

ACES ProS has been developed over the past three years with relatively little HIC funding, highlighting the importance and value of the second Capacity Development Goal, focusing early on long-term sustainability (Weiss et al., 2012). This also highlights a key issue related to internationalization work in school psychology: the more development objectives come from values and practices reflecting genuine reciprocity and respect (Begeny, 2018), with team members strongly committed to the location where the work will be used (as opposed to objectives developing from “helicopter” researchers with a primary interest in career advancement; Ferreira & Gendron, 2011), the stronger the likelihood for team consensus and global health development success and sustainability. This conceptualization of internationalization was also informed by a recent study (Weiss & Pollack, 2017) of 268 global health researchers regarding barriers to global health development and factors related to development success and sustainability. This study found that the more Academic Global Health Achievement-oriented a global health researcher was (e.g., a greater number of global health publications), the less LMIC-Oriented the researcher was (e.g., the less time the researcher spent in LMIC). Level of LMIC-Oriented was positively correlated with perceptions of the level seriousness of (a) Corruption, Lack of Competence global health barriers, and (b) Priority Selection barriers. Also, Academic Global Health Achievement was negatively correlated with overall perceptions of global health barrier seriousness. These findings suggest conflicting perceptions of barriers to global health development across researchers who are more LMIC-oriented (e.g., those living in or spending considerable time in LMIC) versus those less LMIC-oriented (e.g., HIC helicopter researchers), and between LMIC versus HIC global health development professionals. Again, the findings highlight the importance of genuine collaboration so barriers, including those related to sustainability, are viewed similarly and most effectively and efficiently addressed.

ACES ProS adaptation process, challenges, and related research.

Although Problem Solving Therapy (PST) has been adapted for use in a number of countries (Nezu & Nezu, 2013), it was necessary that ACES ProS be structured for the specific cultural context of Vietnamese high-schools. One specific challenge was that relationships in Vietnam between students and adults (e.g., teachers and parents) tend to be hierarchical, which conflicts with the fundamental PST principle of the individual being an active, autonomous problem-solver. For example, even after the initial training, facilitators (e.g., high school staff) often would tell students what to do during and outside the PST sessions, rather than supporting students in their own problem solving. ACES ProS requires students be actively involved in training sessions in order to share their thoughts and reactions (positive and negative) as well as questions and suggestions for the group. However, much of education in Vietnam still involves passive learning, with respect for teachers communicated by silent attention (Tran, 2013). As such, ACES ProS students at least initially had difficulty in actively participating in the program (e.g., sharing their thoughts, challenging potential solutions).

These and other challenges were shared as a joint problem with the students, and approached as an opportunity to model the ACES ProS problem solving steps. After discussion with students and school staff, it was decided to have a “contest” where teachers and students would write short solutions to problems generated by the Danang Psychiatric Hospital Psychology Department (which has overseen ACES ProS) summarizing challenges raised by students and teachers. Solutions were blind rated by students for their utility and reviewed by the Psychology Department, with about 75% of the student-generated solutions and 35% of the teacher solutions rated as likely to be effective. This was discussed with the schools that students had a better understanding of their lives “in these modern times” (a standard Vietnamese phrase indicating how rapidly Vietnamese society has been changing). This was accepted by school staff and students as evidence of the importance of students learning to solve their own problems, with some but not excessive adult guidance by adults.

PST has a strong evidence based within and outside of LMIC, with meta-analyses and systematic reviews (e.g., Bell & D’Zurilla, 2009) summarizing supportive studies across a wide range of countries and settings. For instance, Sorsdahl et al. (2015) evaluated the effectiveness of a PST intervention for adult patients presenting at emergency rooms in South Africa for substance abuse problems. At a three-month follow-up, study participants assigned to the PST intervention showed lower scores for substance abuse-related problems than a control group provided with an information brochure. In their review of child and adolescent mental health programs in LMIC, Barry et al. (2013) identified two SBMH programs based in part on problem-solving therapy, one in India (Srikala & Kumar, 2010) and one in South Africa (De Villiers & Van den Berg, 2012). However, both programs were multi-component (e.g., the program in South Africa included training in emotion regulation, stress management, interpersonal skills as well as problem-solving), and to our knowledge the present study is the first reported evaluation of PST in LMIC schools. Thus, to establish the efficacy of ACES ProS within the LMIC context of Vietnam (i.e., achieving the first Capacity Development Goal of demonstrated program efficacy), the goal of Study 2 was to conduct a randomized trial of ACES ProS.

Method

Participants.

Four high-schools were invited by the school superintendent’s office to participate in the evaluation project; all four schools choose to participate. The screening sample contained 628 10^th and 11^th grade students from the schools. The outcome study sample contained 100 students balanced across the schools who were at or above the borderline clinical range on the Strengths and Difficulties Questionnaire (SDQ) Behavioral Problems and/or Emotional Problems scales (see Procedures, below). The mean age of these participants was 15.73 years (SD=.72), with 60% female.

Measures.

Students were assessed at baseline and one week after treatment end (3 months later) with the Vietnamese version of the SDQ (Dang, Nguyen, & Weiss, 2017). The SDQ contains 20 problem items (e.g., I worry a lot) and 5 strength items (e.g., I usually share with others) rated on a 0–2 Likert scale. Based on the ACES ProS clinical targets, the present study used the SDQ Emotional Problems and SDQ Behavioral Problems subscales, with internal consistency for the subscales equal to .70 and .67, respectively, in the present sample. Complete data at both time points was obtained from 95% of students (5 of the 100 students did not complete the measures due to being absent at the T2 assessment).

Procedures.

After schools indicated their interest in the project, a research assistant introduced the program to 10th and 11th grade students in their homerooms. Interested students were given a packet to take home to parents. Students with parental and self-consent were screened using the SDQ, with 100 students balanced across schools and grades randomly selected from those at or above the borderline clinical range on the SDQ Behavioral Problems and/or Emotional Problems scales. Students were randomly assigned at the grade level to receive ACES ProS or a no-treatment control condition, with the no-treatment condition representing services as usual. Conditions were balanced, with 10^th graders receiving ACES ProS in two schools, and 11^th graders receiving ACES ProS in the other two schools. The study was approved by the USA FWA IRB (#00011251) at the Danang Psychiatric Hospital. A power analysis using SAS Proc Power for our main analyses (Group effects on the SDQ subscales) indicated that for (a) α=.05, (b) a predictor effect size of d = .65 (based on the various PST meta-analyses; e.g., Bell & D’Zurilla, 2009), and (c) total N=100, power was .90.

Results

To evaluate the effects of Group, an ANCOVA with Time 1 (T1) scores as the covariate, propensity weighting (AUC=.82), and Group as a fixed effect was used. An ANCOVA rather than change scores or repeated measures analysis (for which two time points would be algebraically equivalent to change scores) was used due to the well-established problems with the reliability of changes scores (e.g., Cronbach & Furby, 1970; Raykov, 1999). First, we tested for T1 differences using general linear model analyses with the propensity covariate, with Age: F(1,92) = 0.00, p > .99; T1 SDQ Emotional Problems: (1,92) = 0.00, p > .99; T1 SDQ Behavior Problems: F(1,92) = 0.06, p > .81; Contrast of T1 SDQ Emotional Problems vs. Behavioral Problems: F(1,92) = 0.03, p > .87; to assess whether T1 Group differences on Sex, we conducted a generalized linear model analysis in SAS Proc Glimmix with a logit link function and binary distribution, which produced Sex: F(1,92) = 0.00, p > .99. In the main analysis of the effects of ACES ProS on T2 functioning, the effect of Group was significant on both Emotional Problems (F[1,91]=7.39, p<.01) and Behavioral Problems (F[1,91]=13.40, p<.0005), with ACES ProS students showing lower Time 2 scores than control condition participants on both dimensions (see Table 2). Omega-squared (ω²) effect sizes, which represent an unbiased population estimate of the frequently used η² or R² effect size, were partial ω²=.06 for emotional problems, and partial ω²=.12 for behavior problems. We tested whether these effects differed for males and females, including Sex as a moderator in the above analyses. The Sex × Treatment interaction was non-significant for both Emotional Problems (F[1,89]=0.02) and Behavioral Problems (F[1,89]=0.23). To determine whether these various analyses met parametric assumptions, we computed the Kolmogorov-Smirnov Test (Stephens, 1974) on the residuals; for all analyses, p > .10, indicating that residuals did not deviate significantly from normality.

Table 2.

Strengths and Difficulties Questionnaire (SDQ) means and standard deviations in ACES ProS study

SDQ scale	Control Mean (SD)	Treatment Mean (SD)
T1 Emotional Problems	6.58 (1.85)	6.58 (1.64)
T2 Emotional Problems*	5.18 (2.09)	3.79 (2.02)
T1 Behavioral Problems	3.15 (1.87)	3.27 (2.17)
T2 Behavioral Problems**	3.26 (2.11)	1.40 (1.88)

Notes. Means and standard deviations (SD) are adjusted using the propensity covariate analyses.

^{* =}p <.01 for the Control vs. Treatment comparison;

^{** =}p <.0005 for the Control vs. Treatment comparison.

Discussion

To the best of our knowledge, this is the first evaluation of a focused PST program in LMIC schools. Previous assessments of PST programs evaluated in LMIC schools (De Villers & Van den Berg, 2012; Srikala & Kumar, 2010) have involved programs where PST was one of several different components. In both studies, although some effects on intermediate outcomes were significant, effects on mental health outcomes were not. In contrast, effects in our study on both ultimate outcome domains (emotional problems and behavioral problems) were significant, with effect sizes in the medium to large range (partial ω²=.06, and partial ω²=.12, respectively, equivalent of Cohen’s d = .57 and d = .77). Given limited research in this area, it is hard to determine with certainty why mental health outcomes in our study differed from these other studies. However, one possible explanation is that our ACES ProsS program achieved stronger effects because it focuses on one specific skill set, problem-solving, rather than on a range of skills sets. This reasoning is speculative, of course, but does parallel the decision within RECAP-VN to reduce the number of program components and complexity (as we described previously).

Although some researchers have found PST to be similarly effective on adolescents’ emotional and behavior problems (e.g., Merrill, Smith, Cumming, & Daunic, 2017), most research in this area has been conducted in Western countries. Our study found that the effect on behavior problems was twice as large as the effect on emotional problems. A possible explanation for these findings is that behavior problems, in general, have higher social stigmatization in Southeast Asia (e.g., Weiss, Tram, Weisz, Rescorla, & Achenbach, 2009) than emotional problems, and there is less support for adolescents to overcome their behavior problems. With relatively less support, the ACES ProS program may be able to have a larger impact by offering the needed support. Our study is of course only a single study and additional research is needed to address some key limitations, which include: (a) a focus on a single school system, and one of the most progressive in the country; (b) a relatively small sample size (N=100); and (c) a lack of follow-up data.

General Discussion

The purpose of this paper has been to review and discuss our school-based mental health projects in Vietnam with a general purpose of providing examples of how school psychology technology can be internationalized. The content is presented with the hope that it might potentially provide some small measure of guidance for other child mental health professionals interested in developing support for children and adolescents in LMIC, but also as an example of how HIC researchers can learn in the process. Our presentation of the adaptation process for RECAP-VN and ACES ProS programs hopefully shows that adapting intervention programs can be complex and requires careful and strategic collaboration. As described earlier, 36% of the SBMH program evaluations reviewed by Fazel et al. (2014) failed to produce significant effects, suggesting unsuccessful adaptation.

Another goal of this paper was to discuss the various challenges we faced during this work, as well as how challenges were at least preliminarily addressed so that others in the field might benefit from these experiences. In addition to these specific program adaptation challenges, there have been other challenges faced more general to the Vietnam context and likely in similar countries. These challenges likely are not unique to LMIC and probably also occur in HIC, such as those in North America and Europe (Langley, Nadeem, Kataoka, Stein, & Jaycox, 2010), but they may be particularly challenging in LMIC contexts. For example, at least initially, primary and secondary school educators in Vietnam have not seen mental health as part of their work responsibilities. This may be in part due to relatively low mental health literacy in Vietnam or to a lack of understanding about the connection between academic achievement and mental health in children. Therefore, for the effective implementation of SBMH, there needs to be an enhanced understanding of the roles of education and schools in society. School staff may be encouraged and supported to learn more about the potential direct and indirect benefits of mental health programs for schools, students, and society (e.g., program developers or researchers can provide interesting and relevant articles). It is also essential that mental health practitioners and researchers—including those working within or outside of their respective country or culture—recognize that collaboration must be genuine, with a respectful understanding of the reality of the school system’s primary focus on education (Tran, 2013; World Bank, 2013).

Another general challenge has been physical space and time requirements. Because of a shortage of buildings, some schools in Vietnam run from early morning until late afternoon with two separate shifts (one in the morning, one in the afternoon) of different students and different teachers in each shift. Classrooms thus are shared across teachers, so it can be difficult for a teacher to place and leave SBMH program materials on classroom boards or walls. In addition, because there is a national educational curriculum with lesson structure and textbooks fixed and monitored by the Ministry of Education and Training, the ability to integrate mental health into the more general curriculum is somewhat limited. Students have few “study halls” and most rooms are in use constantly, making scheduling of mental health service-delivery programs difficult. Also, beginning in middle-school, classes run six days per week (i.e., students attend school on Saturday), and many students take extra “học thêm” classes (i.e., private tutoring programs provided by school teachers outside of school hours, theoretically optional but in reality mandatory for students). Again, these challenges are not unique to LMIC (Langley et al., 2010) but may be particularly challenging there.

To overcome these challenges, strong support from principals, teachers, and school administrators is essential from the beginning. If these stakeholders understand the importance and benefits of SBMH programs for their students, they will likely begin to find practical ways to address the challenges. For example, in our work, classroom teachers became willing to share classroom time and resources (e.g., classroom bulletin boards) across shifts once they more fully understood the potential benefits of ACES ProS and RECAP-VN for their students. A central factor underlying the success of these various projects is that they developed out of a genuine collaborative process, with no “helicopter researchers” (Ferreira & Gendron, 2011).

One general limitation of the program of research discussed here has been that our SBMH work has focused on two cities in Vietnam (Hanoi, Danang), which are among the most developed in the country; there will likely be more challenges in other cities, particularly ones less developed. Similarly, although our team also works in Cambodia, in this report we have focused on a single country, which is a limitation as there likely will be substantial differences between countries. One important direction for future research will be to better understand how cultural factors influence and affect the development, implementation, and sustainability of school psychology programs across the globe.

Conclusions

Although often given less attention than adult mental health, child mental health is an essential component of society (Patel et al., 2013). Internationalization of school psychology has great potential to support areas such as child mental health development in LMIC (Begeny, 2018). It is, however, a complex task, given cross-national differences in social roles and cultural norms. Maintaining a focus on the twin Capacity Development Goals of demonstrated efficacy and sustainability is important (e.g., not assuming that programs developed in HIC are effective in LMIC, or that they will be maintained after external support is withdrawn). For program efficacy, it is important to target and assess program effects on the ultimate outcomes of interest (in most cases, emotional and behavioral health), not just on intermediate outcome targets (e.g., social skills). The focus on sustainability highlights the importance of community participatory research and development, and true collaboration among partners committed to the same broad goals (e.g., enhancement of child mental health functioning).