Skip to main content
NCBI home page
Internet Interventions logoLink to Internet Interventions
. 2019 Jan 14;15:76–86. doi: 10.1016/j.invent.2018.11.005

Adapting an internet-delivered intervention for depression for a Colombian college student population: An illustration of an integrative empirical approach

Alicia Salamanca-Sanabria a,b,⁎, Derek Richards a,b, Ladislav Timulak a
PMCID: PMC6356089  PMID: 30740313

Abstract

Background

Culturally adapted psychotherapy (CAP) studies are limited and until now there are few published examples that illustrate the process of cultural adaptation with internet-delivered treatments.

Aim

This paper aims to illustrate an integrative approach to the cultural adaptation of an evidence-based internet-delivered cognitive-behavioural therapy intervention for depression (Space from Depression programme).

Method

Mixed method approach utilising quantitative and qualitative methods to assist in the cultural adaptation of the Space from Depression programme was used. The adaptation involved a framework for cultural sensitivity (CSF), alongside an ecological validity framework (EVF) and principles from cross-cultural assessment research. The method included the development of a theory-informed measure, the Cultural Relevance Questionnaire (CRQ), designed specifically for this research.

Results

The adaptation included an establishment of CSF, which included the incorporation of Colombian cultural expressions. College students' (n = 5) and experts' (n = 7) evaluated the EVF based on cross-cultural assessment principles of a preliminary adapted version through the CRQ, showing reliability in the sample (Cronbach's Alpha 0.744). Qualitative analysis supported the culturally sensitive changes or incorporations made to the programme, such as: personal stories and textual translations from English and these were considered ecologically valid and representative.

Conclusions

The research provided support for the idea that CAP can be conducted systematically for internet-delivered interventions.

Keywords: Culturally adapted psychotherapy, Internet-delivered treatment, Cultural sensitivity framework, Ecological validity framework, Cultural equivalence, Depression

1. Introduction

High prevalence rates of depression have been reported across different cultural groups (Bromet et al., 2011; Ferrari et al., 2013; Kessler and Bromet, 2013; R. Kessler et al., 2009; Kirmayer et al., 2017) with an upward trend in prevalence in low-and middle-income countries (LMICs) (World Health Organisation, 2017), such as Colombia. This country has high prevalence rates reported, which are not far removed from prevalence rates found in high-income-countries (HICs). For instance, a study showed that 12-month prevalence rates of depression reported in Colombia were estimated to be between 6.6 and 10% (Gómez-Restrepo et al., 2011), while 12-month prevalence in the United States and Europe were estimated to be 6.6% (Ayuso-Mateos et al., 2001) and 8.5% (Kessler et al., 2010) respectively.

Recent studies on prevalence of major depression in Latin America place Colombia as having the second highest rates of major depression after Brazil (Bromet et al., 2011; Kessler et al., 2010; Kessler and Bromet, 2013). In line with the previous, a national survey from the Colombian Ministry of Health, (2015) estimated point prevalence of mild to moderate depressive symptoms at 15.6% and severe depressive symptoms at 4.2% of adults. Additionally, depressive disorders have been found to be more frequent among college students in comparison to the general population (Cuijpers et al., 2016). A recent systematic review shows a weighted mean prevalence of depression symptoms in college students of 30.6% (Ibrahim et al., 2013). These rates are comparable to reports conducted in other countries in Latin America (Cova Solar et al., 2007; Melo-Carrillo et al., 2012; Morales et al., 2013; Nogueira-Martins et al., 2004). Two studies carried out in Colombia have reported high prevalence of depression symptoms in college students, the first in 30% (Arrivillaga-Quintero et al., 2004)and the second in 36.2% (Richards and Salamanca-Sanabria, 2014) of the student population sample.

The consistent high prevalence of depression reported in the literature justifies the implementation of interventions for depression in Colombia. However, there are several barriers to access to mental health interventions in that country. In Colombia, the majority (85–95%) of individuals with mental health problems do not access or cannot access services they require or need (Ministry of Social Protection, 2012). Additionally, a National Survey from the Colombian Ministry of Health (2015) shows that about 50% of the population reports that personal stigma is one of the principal causes of not accessing a mental health service in Colombia, followed by geographical location and limited service availability (Ministry of Health, 2015). Furthermore, most of the treatments in LMICs are implemented without considering cultural context; very little research has investigated culturally adapted treatments (Vally and Maggott, 2015), while there is even less research in internet-delivered treatments (Arjadi et al., 2015; Martínez et al., 2018).

Consequently, it is relevant to establish effective treatments for depression and investigate the intervention effects across diverse populations (Kalibatseva and Leong, 2014; Vally and Maggott, 2015) such as LMICs. Given that most of the treatments for depression are developed in Western HICs (e.g., Europe, the US) attention has to be paid to the cultural sensitivity and ecological validity of treatments, so as to maximise their potential effectiveness and aid their effective dissemination in various countries (e.g., Latin America, Asia) (Bernal et al., 2009; Hwang, 2009).

Historically, several psychotherapeutic approaches for the treatment of depression have been developed over the last century, including cognitive-behavioural therapy (Beck and Alford, 2009; Cuijpers et al., 2013; Renner et al., 2013). Nevertheless, consideration of the need to culturally adapt these evidence-based interventions (EBIs) has received limited attention (Rathod, 2016). Over the past decade, studies have examined the potential of Culturally Adapted Psychotherapy (CAP) for depression in different cultural groups (Kalibatseva and Leong, 2014). However this field has experienced difficulties in achieving a convergence of methodologies to be used in the cultural adaptation of a treatment (Bernal and Domenech-Rodríguez, 2012), and empirical studies are limited (Bernal and Domenech-Rodríguez, 2012; Chowdhary et al., 2014; Chu and Leino, 2017; Helms, 2015).

There are several reasons for considering the cultural adaptation of treatments. First, evidence suggests that cultural influences affect the diagnostic and treatment processes (Canino and Alegría, 2008; Givens et al., 2007; Hall, 2001). Second, research supports the relevance of developing treatments based on diverse cultural groups' needs and contexts (Bernal, Jiménez-Chafey, & Domenech-Rodríguez, 2009; Chowdhary et al., 2014; Miranda et al., 2005). There exists cultural differences in the expression and understanding of depressive symptoms across cultures (Haroz et al., 2017; Karasz, 2005; Muñoz et al., 2005; Richards and Salamanca-Sanabria, 2014; Yusim et al., 2010). For example people from South America and Asia trend to express depression through somatic symptoms (e.g., pain, digestive problems), while Caucasian (Europeans) express depression using cognitive factors (e.g., beliefs, thoughts) (Parker et al., 2001; Richards and Salamanca-Sanabria, 2014; Yusim et al., 2010). Third, most outcome research of Evidence-Base Treatments are with Caucasian samples (Bernal and Scharró-del-Río, 2001; Chu and Leino, 2017; Griner and Smith, 2006; La Roche and Christopher, 2008), therefore the treatments are not necessarily generalisable to other ethnics groups (Chu and Leino, 2017; Lau, 2006). Fourth, studies have failed to analyse how cultural groups respond differently to interventions and there is evidence to support lower levels of engagement in treatments when they are delivered to minorities or diverse populations (Lau, 2006). Also, studies have failed to put efforts in the inclusion of a representative sample of minories (van Loon et al., 2013) and there are limited studies of CAP in the country of origin (Cuijpers et al., 2018; Vally and Maggott, 2015). And fifth, ethically, delivering relevant interventions has to consider optimising the potential benefits to the persons and communities; ethical research must be culturally sensitive to the manner in which clinical interventions are introduced and conducted with culturally diverse communities (Trimble et al., 2012).

Considering the previous, culturally adapting treatments implicates an understanding of both culture and psychotherapeutic interventions, which have a complex relationship and several factors need to be considered. Culture can be seen to involve several principal characteristics: It shapes behaviours (e.g., customs, habits); categorises perceptions (e.g., beliefs about mental illness); it involves aspects of experiences related to the context (e.g., family upbringing, socio-economic status); it is deeply shared by members of a particular society or social group; it provides an orientating guide to organise behaviour and it transfers values across generations (Rathod, 2016).

The American Psychological Association (2003) defines culture as a set of belief that influence values, customs and psychological process such as language and education. Castro et al. (2010) describes culture as the embodiment of a worldview through learned and transmitted beliefs, values, and practices, including religious and spiritual traditions. Some authors have considered culture as a structural framework that includes identity that consists of a unique combination of elements (e.g., gender, race, ethnicity), geographic (e.g., urban-rural, country), and associative (e.g., family, religion). It includes ethic group that consists of a collective of people who share common elements (e.g., beliefs, heritage, and status).

Culturally-adapted psychotherapy (CAP) is defined as a systematic change of an intervention protocols through which consideration of culture and context modifies the treatment in accordance with clients' values, contexts, and has cultural relevance to the target population (Bernal et al., 2009). For psychological interventions, adaptation involves the modification of key characteristics, elements, and methods of delivery, while maintaining the core theoretical components of the intervention (McKleroy et al., 2006; Nicolas et al., 2009). Until now there is no golden standard to adapt an intervention culturally. Therefore there are several methods available with a variety of possible adaptations. The following section outlines some of the main approaches.

1.1. Theoretical approaches to culturally-adapted psychotherapy

Over the last three decades, psychologists have considered research approaches for the cultural adaptation of EBTs for common mental health problems (Bernal and Domenech-Rodríguez, 2012; Chu and Leino, 2017). Some examples include the work of Griner and Smith (2006) and van Loon et al. (2013) who proposed that adaptations should consider (a) Assesment/discussions of cultural background (b) Ethnic/language matching between the client and the therapist; (c) Translations or adaptations of language; (d) Incorporating cultural values and beliefs in treatment; (e) Collaborating with the population group and adapting the treatment (e.g., focus groups).

The ecological validity framework (EVF) proposed by Bernal et al. (1995) involves eight dimensions that must be incorporated into treatment to increase both the ecological validity and the overall external validity of a treatment. These dimensions are: (a) language, which must be culturally appropriate and syntonic (oral and written); (b) persons refers to the client–therapist relationship; (c) metaphors, refers to the symbols and concepts; (d) content, refers to cultural knowledge about values, customs, and traditions (e) concepts, refer to the constructs of the theoretical model to be used in treatment; (f) goals, implies the establishment of an agreement between the therapist and client as to the goals of treatment; (g) methods, refers to the procedures to follow for the achievement of the treatment goals and (h) context indicates the consideration of the client's social, economic, and political context (Bernal and Sáez-Santiago, 2006).

Cultural Sensitivity Framework (CSF) proposed by Resnicow et al. (2000), is the most common method used in cultural adaptations (Hall et al., 2016). This framework involves either a top down approach (surface structure), where the research team includes cultural modifications to the programme based on observable social and behavioural characteristics (e.g. expressions, images or cultural metaphors about mental illness) (Resnicow et al., 2000; Wilson and Miller, 2003) or a bottom up approach (deep structure), where, for instance, through focus groups the research team analyses what part of the treatment is relevant based on deeper cultural, social, historical, environmental, and psychological factors.

Helms (2015) proposed principles from cross-cultural assessment in the cultural adaptation of an intervention. These are three types of cultural equivalence proposed by Lonner (1985), which have been conceptualised and extended by Helms (2015). Helms concluded that research in culturally-adapted psychotherapy did not include considerations for assessing ethnicity that is defined as traditions transmitted from generation to generation, which are internalised by the person as member of the ethnic cultural group. As a result of this the authors proposed that the process for culturally adapting an intervention should include functional equivalence, conceptual equivalence and linguistic equivalence to achieve an effective cultural adaptation Functional equivalence is defined as the extent to which the same ostensible behaviours (e.g., crying) are interpreted similarly in different cultural or racial groups, occur with equal frequency within these groups, and elicit similar reactions from other members of the groups. Conceptual equivalence, refers to the extent to which different concepts are analogous for the cultural group that is targeted for the treatment; and linguistic equivalence, indicates the language or dialect used during the process has been adjusted so that it has meaning to the person(s) being assessed. In addition, language must be considered in a process and outcome evaluations (Helms, 2015).

The current study attempted to integrate cultural sensitivity framework (CSF) (Resnicow et al., 2000; Wilson and Miller, 2003), cross-cultural principles (Helms, 2015) and ecological validity framework (EVF) (Bernal and Sáez-Santiago, 2006). This integration of theoretical approaches contributed to a systematic cultural adaptations of the internet-delivered intervention. The CSF facilitated modifications to the internet-delivered programme based on top down approach, while maintaining the core theoretical components of the intervention. EVF and cross-cultural assessment principles supported the programme's evaluation by members of the community and also experts.

1.2. Culturally Adapted Psychotherapy (CAP) for depression in Latin America

Previous reviews of CAP (Huey and Polo, 2008; Miranda et al., 2005) and meta-analyses (Benish et al., 2011; Griner and Smith, 2006; Smith and Trimble, 2016) have examined the effectiveness of cultural adaptations in numerous treatments targeting, such as family therapy, substance abuse, psychotic disorders, anxiety and depression. Culturally adapted treatments seem more effective than no treatment (d = 0.58), treatment as usual (d = 0.22) or non-adapted treatment (d = 0.32) (Benish et al., 2011; Huey and Polo, 2008) and show moderately strong benefit from pre- to post- intervention (d = 0.45) (Griner and Smith, 2006). A recent meta-analysis found the overall affect size was d = 0.67, which indicates that CAP has better outcomes than other conditions (non-adapted treatment or control group)(Smith and Trimble, 2016).

Recently, a systematic review of depression psychotherapies among Latinos in the U.S, found that studies incorporated between zero and six cultural adaptations, often a mix of deep and surface structure changes (e.g. material translated into Spanish) and nine types of deep structure changes (e.g. piloting of adapted therapy, inclusion of appropriate cultural metaphors and values) (Collado et al., 2016). However, more studies are needed on culturally adapted treatments for Latinos with depression to improve the intervention engagement and outcomes (Kanter et al., 2015).

On the other hand, studies of CAP for a majority population of a particular country, as opposed for a minority in a Western country, are limited. In Latin America Kalibatseva and Leong (2014) report a study in Puerto Rico, which documented a cultural adaptation of CBT with parents of Latino adolescents with depression. The intervention was refined and culturally adapted using the ecological validity framework (EVF), which outlines eight dimensions described by Bernal and Sáez-Santiago (2006). Qualitative analysis reported the intervention was successfully implemented (Sáez-Santiago et al., 2012). The adapted programme has demonstrated significant reductions in depression post treatment and also in comparison to waiting list group (Rosselló and Bernal, 1999, Rosselló and Bernal, 2005; Rosselló et al., 2012).

A recent meta-analysis found eleven studies in Low Middle Income Countries (LMICs) and only two in Latin America using CAP for depression treatment (Vally and Maggott, 2015). In Colombia, a CBT treatment was culturally adapted to target the mental health of dementia caregivers. The programme used was an adaptation of an intervention for Hispanic/Latino population in the U.S. (Gallagher-Thompson, 1992; Gallagher-Thompson et al., 2001). The results showed statistically significant post-treatment effects between the intervention group and control group on depression symptoms as measured by PHQ-9 and positive effects on a measure of satisfaction with life (Arango-Lasprilla et al., 2014). However, the adaptation process was not documented. Similarly, an intervention and material were culturally sensitised for women of limited means with depressive symptoms in Mexico, based on a CBT protocol developed in California (Munoz and Ying, 2002). The adaptation was made by researchers and hospital personnel. The outcomes showed a decrease in depressive symptoms for participants across the modules and treatment components as measured by the Centre for Epidemiological Studies Depression Scale (CES-D) (Lara et al., 2003).

These studies show that culturally-adapted treatments demonstrated efficacy but such treatments with Latin-Americans in their country of origin are limited. Vally and Maggott (2015) concluded in their meta-analysis that the evaluations and a systematic process of cultural adaptations have been absent or lacking in rigour. Therefore their quality remains undetermined.

1.3. Culturally adapted internet-delivered treatment for depression

Studies of culturally adapted low-intensity internet-delivered treatments have been reported. For instance, Kayrouz et al. (2015a) examined the efficacy and acceptability of a culturally modified therapist-guided CBT for Arab Australians with anxiety and depression. The cultural adaptation involved examples and educational stories relevant to the population based on feedback from members of the Arab community through focus groups and online survey combined with a literature review on acculturation conducted by the author (Kayrouz et al., 2015b). The results showed improvements significantly across all outcome measures.

Another study showed the efficacy of a culturally attuned internet CBT for Chinese Australians with weekly telephone support. The cultural adaptation included redrawing illustrations, incorporation of cultural expressions according to Chinese values, reframing skills and the revision of the programme by Chinese health professionals and consumers. The experimental group reported significantly reduced depressive symptoms compared to a control group (Choi et al., 2012). Furthermore, Ünlü Ince et al. (2013) examined an internet-based, culturally sensitivity self-help problem-solving intervention for Turkish migrants with depressive symptoms. The cultural adaptation involved migration stories and examples recognised by the target population, also Turkish people evaluated the culturally adapted intervention. The results showed no significant differences on depression between the experimental and control group. However, significant differences post-treatment were found. None of these studies alluded to the previous frameworks for culturally adapted psychotherapy.

Our literature search revealed that in Latin America there are no studies of culturally adapted internet-delivered treatment for depression conducted in the country of origin. However, there are some studies investigating internet interventions for depression developed in Latin America. For instance, naturalistic study monitoring an internet CBT treatment produced in Mexico demonstrated to be a valuable tool for Mexicans with depression in terms of accessibility and mental health support for women (Lara et al., 2014). Moreover, Tiburcio et al. (2016) in a usability study reported on the participants experience of an internet intervention for substance abuse and depressive symptoms in Mexico. They concluded that the intervention realised reductions in drug use and depressive symptoms among users and therefore could be used as an alternative to treatment-as-usual. Furthermore, Espinosa et al. (2016) evaluated the feasibility and acceptability of the internet-delivered CBT programme developed in Chile based on SUMMIT (Supportive Monitoring and Disease Management over the Internet) (German intervention). The programme provided support to clients after they finished their treatment for depression. Although the results show high rates of acceptance and satisfaction, the authors do not detail any cultural adaptions to the treatment, despite the programme being originally made in Germany.

A recent systematic review found only three studies on e-mental health interventions in Low and Middle-Income Countries (LMICs), which were in China and Romania (Arjadi et al., 2015). Likewise, Harper-Shehadeh et al. (2016) found in a recent review and meta-analysis on cultural adaptation of online interventions and bibliotherapy with minimally guided treatment that the details on the CAP methodologies were unreported. However, these studies were effective reducing depression and anxiety significantly. Despite the outcomes showing positive post-treatment results, more research is required to develop interventions that incorporate frameworks for cultural adaptation, especially as adaptation processes were not clearly or fully described in previous research (Harper-Shehadeh et al., 2016).

1.4. Space from depression programme

Space from Depression is the name of seven module internet-based cognitive-behavioural (iCBT) intervention for the treatment of depression (See Table 1) developed by SilverCloud Health. Space from Depression programme has shown to be efficacious in comparison to waiting list control group in a community sample in Ireland (Richards et al., 2015). The treatment was evaluated by users as acceptable and satisfactory (Richards et al., 2016).

Table 1.

Space from Depression programme description intervention (Richards et al., 2015).

Module Brief description
Welcome to SilverCloud This module introduces to the platform. The module explains all of the functions, privacy, icons and buttons on the platform.
Getting started Outlines the basic premise of CBT, provide information about depression, and introduce some of the key ideas of Space from Depression. Users are encouraged to begin to chart their own current difficulties with depression.
Understanding feelings The focus in this module is on mood monitoring and emotional literacy. Users can explore different aspects of emotions, physical reactions, action and inaction, and how they are related.
Boosting behaviour This module focuses on behavioural change to improve mood. Ideas about behavioural activation are included, and users can plan and record activities, and chart their relationship with their mood.
Spotting thoughts This module focuses on noting and tracking thoughts. Users can explore the connection between their cognitions and their mood, and record them graphically.
Challenging your thoughts This module supports users to challenge distorted or overly negative thinking patterns, with thought records, as well as helpful coping thoughts.
Bringing It all together In this final module, users are encouraged to bring together all the skills and ideas they have gathered so far, note their personal warning signs, and plan for staying well.
Open in a new tab

The treatment includes self-monitoring, behavioural activation, cognitive restructuring, and challenging core beliefs (see Table 1). All modules have the same structure and format, which consist of quizzes, videos, educational content, activities with homework suggestions and a review (Salamanca-Sanabria et al., 2018). Also, users have a supporter, who gives feedback and monitor users' treatment each week (Richards et al., 2015). The supporters in the current study were Colombian postgraduate student in clinical psychology with supervised experience in CBT with adults (Salamanca-Sanabria et al., 2018) who also evaluated the preliminary cultural adaptation version of the Space from depression programme.

This study presents a description of the cultural adaptation of the Space from Depression programme and empirical assessment of its cultural relevance. The aim of this study was to investigate whether the initial top-down (researcher led) cultural adaptation was assessed as culturally relevant by the target population (college students) and by experts with experience in psychological intervention research and internet-delivered intervention research. To achieve this the paper illustrates an integrative approach for the tailoring of an evidence-based internet-delivered treatment for depression to the specific needs of Colombians. We offer an illustration of how we attempted to integrate cultural sensitivity framework (CSF), ecological validity framework (EVF) and cross-cultural assessment research in a systematic way. Furthermore, the idea was to add transparency to the process of culturally adapting an internet-delivered intervention.

2. Method

This adaptation consisted of a mixed method approach utilising quantitative and qualitative methods to assist in the cultural adaptation of the Space from Depression programme for Colombian population.

2.1. Procedure

The programme Space from Depression was culturally adapted for Colombian college student population. The method involved three distinct phases of adaptation, the first considering cultural sensitivity in the adaptation, the second one establishing ecological validity of the adaptation and the third one consisted of the cultural incorporations into the programme (See Table 2).

Table 2.

Cultural adaptation methodology description.

Phases Brief description
Phase a. Cultural sensitivity: Initial adaptation utilising principles of Cultural sensitivity (Research team, professional translators and video company in Colombia). Programme translations and cultural incorporations: Translation from English to Spanish and incorporation of the cultural expressions in: general content, psychoeducation information, activities, tools, examples, personals stories, videos, quizzes and quotes.
Phase b. Ecological Validity:
(Community and specialist evaluation).		 Assessment of a culturally adapted treatment by users and experts: Involved the development of a Cultural Relevance Questionnaire (CRQ) measure to capture data on ecological validity of the adapted intervention and its use by users and experts.
Phase c. Cultural incorporations
(Research team).		 The integration of the feedback for further adjustments to already adapted intervention.
Open in a new tab

2.1.1. Phase a: cultural sensitivity

The process for the cultural sensitivity approach is described in the framework by Resnicow et al. (2000). This phase included the preliminary cultural adaptation of “Space from Depression” programme (see Table 2).

2.1.2. Phase b: ecological validity

The process used to establish ecological validity was developed using approaches proposed by Bernal and Sáez-Santiago (2006) and Helms (2015) that involved an evaluation of the adapted programme for cultural sensitivity (Phase 1) by Colombian college students and clinical experts using the Cultural Relevance Questionnaire (CRQ) that was developed specifically for this study, see Table 3 for description.

Table 3.

Experts' evaluators' information.

Experts
(N = 7)		 Ph.D. CAP experience Internet-delivered treatment experience Nationality
Expert 1 Clinical Psychology Yes No Colombian
Expert 2 Psychology Yes No Colombian
Expert 3 Public Health No No Colombian
Expert 4 Clinical Psychology Yes No Colombian
Expert 5 Clinical and Health Psychology Yes No Colombian
Expert 6 Positive Psychology Yes Yes Argentinian
Expert 7 Psychology Yes Yes Spanish
Open in a new tab

2.1.3. Phase c: cultural incorporations

Once the college students and experts completed the CRQ (Phase c), the information was analysed and incorporated in the programme by the researcher. The analysis was carried out through a quantitative and a qualitative information, see Table 2 for description.

2.2. Sample of reviewers

Five psychology master students volunteers from a university in Colombia participated as volunteers in this research, they underwent the adapted Space from Depression programme from the position of a user (college student). Five experts in clinical psychology from Colombia evaluated the initial adaptation. In addition, two clinical psychologist experts from Argentina and Spain, who have experience with internet-delivered interventions and also with their translation and adaption for use in Latin America, evaluated the already adjusted programme (See phase 1) using the CRQ. These criteria aimed to guarantee a diversity of experience and skills to support a rigorous approach to assess and establish the ecological validity of the culturally adapted intervention. Experts were invited to participate in this research and they were selected on the basis of following criteria: a) native Spanish speaker from Colombia and/or other Spanish speaking countries; b) PhD in clinical psychology or in psychology; and c) experience in: CBT, depression, research, and low-intensity internet-delivered treatment (See Table 3).

2.3. Recruitment & procedure

A research group in a Colombian university sent an email to all of master in clinical psychology students (seventeen students) with a summary of the research study, its aims and a description of the task that was review the programme and completed the CRQ. Five participants signed the informed consent and after that they reviewed the programme and complete the CRQ after engaging with all of the treatment modules (See the appendix A). After 2 weeks students' volunteers completed the CRQ from the SilverCloud platform.

Nine experts were initially invited to participate in this research. The experts were researchers and clinical psychologists in Colombia, Argentina and Spain. Four Colombian experts were selected from two Colombian universities, where the Space from Depression programme were applied. Two Colombian Experts were selected based on recommendations from a research collaborator (LCC) in Colombia. Two Spanish and one Argentian investigators were recommended by the second author (DR), who were selected based on the background experience of internet-delivered interventions and cultural adaptations. The email contained information about the study including the aims of the research and informed consent, which was signed and returned by email. Seven experts reviewed the programme and completed the CRQ. The experts completed the questionnaire after 4 weeks, returning it by email. College students' volunteers and experts completed the first section of the CRQ. The second section of the CRQ was only completed by experts.

Qualitative information was analysed using a descriptive-qualitative analysis (Dey, 2003) to identify categories that could be later used to incorporate further changes (in addition to Phase 1 changes) to the intervention.

2.4. Assessments

The study consisted of three phases: (a) an initial cultural adaptation programme by the researcher and SilverCloud team (working with the digital content manager (RB) and the chief technical officer (JB) (b) assessed by experts and users using the Cultural Relevance Questionnaire (CRQ) and (c) the further feedback from the experts and users was used in the final version of the Space from Depression in Study 1 (See Table 4).

Table 4.

Study 1 measures used.

Phase Measure Assessed variable Evaluators team
a) Culturally sensitivity None Space from depression programme (English version) Researcher and the SilverCloud team
b) Cultural Validity Cultural Relevance Questionnaire (CRQ) Culturally sensitivity programme Completed by experts and users
c) Cultural modifications None Culturally adapted Space from depression programme Researcher in discussion with the supervisors
Open in a new tab

2.5. Measure

2.5.1. Cultural Relevance Questionnaire (CRQ)

The Cultural Relevance Questionnaire was designed based on a literature review regarding culturally adapted psychotherapy processes and methods. The CRQ has two sections: (1) a general assessment of the programme and (2) an assessment of each of the modules. The first section, the general assessment of the programme in its entirety, it is composed of five items assessed on a 5 point Likert scale (1 represents that functional, conceptual or linguistic equivalence are not reflected within the programme and 5 means that all of these equivalences are reflected within the programme). Each question also includes an open request to elicit qualitative comments from the evaluators' detailing the reasons for their quantitative evaluation. This general assessment section is based on Helms (2015) and Bernal and Sáez-Santiago (2006). The first section of CRQ therefore examined:

2.5.2. Functional equivalence

The raters are asked to evaluated the extent to which the same apparent behaviours are interpreted similarly in different cultural or racial groups, occur with equal frequency within these groups, and elicit similar reactions from other members of the groups. Therefore, treatment components could be interpreted similarly by the target cultural group (e.g., personal stories). The category is composed of three items: (1) the programme involves familiar behavioural or emotional expressions for the cultural group being targeted, (2) the treatment reflects the people and cultural context (e.g., social, political, economic, ethnic, and historical), (3) the treatment goals are tailored to work with clients from this cultural context (e.g., examples, personal stories, tools). Each question also includes an open request to elicit qualitative comments from the evaluators' detailing the reasons for their quantitative evaluation.

2.5.3. Conceptual equivalence

This part of the questionnaire consisted of 1 item that evaluated the extent to which different concepts are the same or analogous between groups (i.e., adjectives used to describe depression). The conceptual equivalence item is: The treatment includes symbols and concepts shared by the cultural group; for instance cultural expressions of depression, ideas or analogies about mental illness are included in the programme. Again it is supplemented by an open-ended space allowing for comments in this area.

2.5.4. Linguistic equivalence

This part of the questionnaire consisted of 1 item that evaluated the level of oral and written language adjustments made for the programme (e.g. regionalism, slang). Also, it is supplemented by an open-ended space allowing for comments in this area.

The second section of the CRQ evaluates each module in the programme Space from Depression The programme is composed by seven modules, which correspond to seven items in the CRQ. Each module was divided into four components that were: Content, examples, personal stories, exercises. Each component collects quantitative judgment assessed on a 5 point Likert scale, based again on the functional equivalence, Conceptual equivalence and Linguistic equivalence principles. Likewise, it is supplemented by a space allowing for comments in each component on each component. Finally, there is an open-ended space allowing for comments for each of the module. The internal consistency (Cronbach's α) of the CRQ scale reached the value of α = 0.744. (See the CRQ into appendix A).

2.6. Data analysis

Descriptive statistics were reported based on results from the quantitative questions on the CRQ, which were analysed in SPSS. The CRQ data collected from the open questions were analysed qualitatively (Dey, 2003). Once all the questionnaires were collected, the information was analysed in Atlas Ti. This phase involved a descriptive-qualitative analysis, which consisted of categories generated by evaluators (users and experts) based on comments at the open-ended spaces in the CRQ (general and specific per module). The process of data analysis was carried out by one of the authors (AS) and it was reviewed by the authors (DR)(LT).

3. Results

The results are described in three distinct phases considering (a) an initial adaptation taking into account cultural sensitivity and (b) assessment of initial adaptation as to its ecological validity and gathering of further suggestions for changes and (c) the incorporations or modifications based on the feedback from the CRQ by the evaluators.

3.1. Phase a: initial adaptation of space from depression based on principles of cultural sensitivity

The cultural sensitivity approach to adaptation as described in the framework by Resnicow et al. (2000) was employed. This phase was carried out by the principal researcher (AS) in conjunction with collaborators from SilverCloud Health (developers), professional translators and video production teams from Colombia. The process resulted in an initial adapted version of the intervention. Firstly the programme was translated from English to Spanish by professional translators, and secondly the principal researcher reviewed the translated content in all sections. The revision include the incorporation of cultural expressions and examples in the programme. At the same time, the preliminary adaptation involved the production of new videos and the modification of personal stories to describe Latin American college student scenarios (e.g. economic problems, pregnancy, relationships) and stereotypes for Colombian college students, including the use of Colombian actors (e.g., overall age, clothes, hair style match and accent).

The outcomes in this phase involved major adjustments to several programme components including: (a) translations from English to Spanish; (b) rewriting of the personal stories; (c) remaking the audio recordings and videos; (d) revisions to the quizzes, quotes, and (e) development of the supporter training (manual). All sections were modified to incorporate culturally appropriate expressions and language in to make it more understandable for Colombian college students.

The principal researcher (AS), originally from Bogota, Colombia, made the adaptation based on observable characteristics, experiences, norms, values, behavioural patterns and beliefs of the target population (Resnicow et al., 2000). AS is a psychologist, she has worked as a therapist, researcher and lecturer with different populations in Colombia, including college students, clinical population from low, middle and high socioeconomic status people in different regions of Colombia. The adaptation process carried out by AS consisted on:

  • (a)

    Language – Translations revisions: The translators' versions were reviewed several times in order to refine the grammar, making the content more understandable for the target population. For instance, informal language style aimed at college students was used (i.e. tú the informal you), known as “Tuteo” in Spanish. The Colombian iCBT programme (I can feel better programme/Yo puedo sentirme bien in Spanish) was a modified version of the clinically efficacious Space from depression iCBT Programme (Richards et al., 2015). The name I can feel better was selected as a culturally suitable descriptor for improving mood in order to avoid the stigma related to depression in the Colombian culture. Furthermore, the concept “Personal guide” (“Guía personal” in Spanish) was suggested and selected instead of “Supporter”, which does not have a direct meaningful translation in Spanish.

  • (b)

    Personal Stories: College students' personal stories in Colombia were adapted based on AS' professional experience with potentially depressed student population. The topics in the stories included: (a) a pregnant student, (b) economic problems, (c) difficulties in romantic or peer relationships and (d) high self-criticism/perfectionism. Likewise, these personal stories were reviewed by a counselor (RG) in Colombia. Then, appropriate images were chosen to represent the Colombian characters portrayed in each personal story.

  • (c)

    Audio recordings and videos: Four mindfulness exercises were re-recorded featuring a standard Colombian accent. Likewise, the videos in the programme were overlaid with a Colombian voiceover of the script. Additionally, eight introductory videos were made by Colombian actors.

  • (d)

    Quizzes, quotes: Quizzes and quotes were revised the language and appropriate cultural topics and expressions. For instance a question was added to the introductory quiz “Do Latin Americans suffer from depression?” Moreover, quotes from Latin-American writers were incorporated (e.g. Gabriel Garcia Marquez, Isabel Allende among others) into the programme.

  • (e)

    Supporter training: The training manual was translated into Spanish.

    This phase consisted of the initial adaptation of the programme, based on cultural CSF (Top down adaptation). The end product of Phase 1 of the cultural adaptation of Space from Depression was an initial version of the programme adaptation, specifically tailored to a Colombian student population that was further adapted by utilising processes described below in Phase b.

3.2. Phase b: assessment of ecological validity of the initial adaptation (with suggestions for further adaptations)

This section is the assessment of initial adaptation as to its ecological validity [Agreement between users and experts respective perception of the intervention (Bernal et al., 2009)] and gathering of further suggestions for changes is presented here. Cross-cultural principles – Functional, Conceptual and Linguistic - assessment of the programme were evaluated by users and experts (See Table 5).

Table 5.

General evaluation by users and experts.

Cross-cultural principles Rangea Users (n = 5)
 Experts (n = 7)
 Combined
M (SD) M (SD) M (SD)
Functional equivalence (1–5) 4.53 (0.380) 4.38 (0.756) 4.44 (0.609)
Conceptual equivalence (1–5) 4.60 (0.548) 4.29 (1.25) 4.42 (0.996)
Linguistic equivalence (1–5) 4.20 (0.447) 3.86 (1.06) 4.00 (0.853)
Open in a new tab
a

1 = Functional, conceptual or linguistic equivalence are not reflected within the programme. 5 = All of the equivalences are reflected within the programme.

Evaluators provided positive feedback about the initial version of the culturally adapted programme. College students and experts assessment that the programme was mostly suitable in their evaluation of the intervention as a whole (Table 5). Experts evaluated its component modules as (Table 6).

Table 6.

Functional, conceptual and linguistic equivalence per module.

Module Rangea Functional equivalence
 Conceptual equivalence
 Linguistic equivalence
M (SD) M (SD) M (SD)
Welcome to SilverCloud (1–5) 4.64 (0.4532) 4.17 (1.1060) 4.17 (1.0677)
Getting started (1–5) 4.78 (0.3037) 4.28 (1.0842) 4.17 (1.0649)
Understanding feelings (1–5) 4.71 (0.3660) 4.28 (1.1273) 4.14 (1.0870)
Boosting behaviour (1–5) 4.75 (0.4190) 4.22 (1.0746) 4.12 (1.0394)
Spotting thoughts (1–5) 4.78 (0.3037) 4.35 (1.0880) 4.24 (1.0842)
Challenging your thoughts (1–5) 4.82 (0.3134) 4.42 (1.1060) 4.29 (1.113)
Bringing It all together (1–5) 4.74 (0.3819) 4.35 (1.0979) 4.24 (1.0897)
Open in a new tab
a

1 = Functional, conceptual or linguistic equivalence are not reflected within the programme. 5 = All of the equivalences are reflected within the programme.

3.2.1. Qualitative analysis

The results of the qualitative component to the CRQ supported five overarching categories: (a) “language's suggestions or changes” a category, which is understood as instances of language use that could be improved upon within the intervention; (b) “concept coherence” which is defined as the conceptual consistency and understanding of the cognitive behavioural therapy elements in the programme, (c) “quality of delivering” is defined as the manner in which the content is delivered; and (d) “positive impressions” relates to positive comments regarding the strengths of the intervention.

In qualitative comments, college students and experts highlighted some aspects of the intervention that required further attention. Their comments centred on the language used (clarification, colloquial language), relevance of examples and personal stories, conceptual clarity and the engagement promoting technical features of the programme. Specifically, college students suggested refining personal stories, which were evaluated as not so natural for them. Experts, for instance suggested renaming modules. For example, the title “Boosting behaviour” was not considered a suitable title for a module. “It has no meaning in the everyday context of the population” Language suggestions also extended to its more colloquial use (language's suggestions or changes category) including, for instance, Colombian expressions such as, “guayabo” (hung over), “ciclovía” (bike rides on Sundays). Furthermore, experts suggested changing some of the examples, such as: “Failing an exam” instead of “Losing a driver licence”, as the latter is not a common situation for students in Colombia.

The category conceptual coherence was only evaluated by experts, for instance, with a Mindfulness exercise “CBT module is teaching challenging thoughts the session teaches how to challenge and change thoughts but the Mindfulness training is more about the acceptance of thoughts”. The quality of delivery was typically commented by users and experts on in the context of videos: “Videos are not convincing due to low quality” “Consider animations instead of videos”, “The videos presented for each module are not very convincing”, “videos can be improved” In this category the college students also suggested including a quick introduction in the Mindfulness component before to start the exercise, in order to improve the instruction and experience for the user.

Encouragingly, the college students and experts also stated that “In most parts of the programme the language is suitable and understandable” For example, some quotes were: “The information is presented in a didactically and an understandable manner for the users”, “Elements of the programme are familiar to the targeted cultural group”, “Personal stories are close to our reality”, “The cultural context can be analysed through personal histories”, “The intervention presents clearly and simply the CBT basic aspects”, “The emotions' synonyms are well translated and help to understand the meaning of each feeling”, among others.

3.3. Phase c. Incorporation and modification of the programme based on the CRQ feedback

The end product of phase 2 was the incorporation and modification of the programme based on the feedback. Qualitative data collected using the CRQ showed that while in principle college students and experts were happy with the cultural adaptation, they also made several suggestions as to the language, examples used and personal stories used. They also commented on other features of the programme not necessarily related to cultural adaptation, such as the quality of the videos or conceptual match between different parts of the programme. The feedback provided was used in the final adaptation of the programme. Based on this type of feedback the research team made modifications to the titles used in the programme. Comments supported the revision of the language in some parts of the programme, such as the inclusion of modules' title. For instance, “How to start to motivate myself?” was used instead of “boosting behaviour” and “How to question my thoughts?” was used instead of “challenging thoughts”. Also, the term “Beating behavioural traps” was not used as a title in the programme Therefore, in the Spanish version, a more informal title was employed: “How to overcome the vicious cycles of depression?” Likewise, personal stories were adjusted and the details were extended to include colloquial jargon for college students to attune the personal stories making that more natural.

Furthermore, mindfulness activities were reviewed and included a quick introduction. The mindfulness activity in the “challenging thoughts” module was changed with more emphasis on acceptance of thoughts. Furthermore, the analysis contributed to increasing the quality of the presentation of the programme (e.g., videos replaced by animation). The animations included images of a typical college student in Colombia. For example, animations involved a male student with a traditional Colombian bag called a “mochila” that is originally made by Colombian indigenous groups. The image of an average Colombian girl with her hairstyle and clothes matching current styles was taken as the model to inspire the animations. Also, animations included the public bus transport in Bogotá called “Transmilenio”.

Adapting an intervention programme is essential to accommodate diversity of experiences and traditions (Domenech Rodríguez et al., 2011). Those examples illustrate features incorporated into the programme. Most of the modifications in this illustration might be obvious (e.g. renaming modules), however the modification implicates a flexibility approach. The culturally adapted Space from depression programme was tested in Colombia for college students (Salamanca-Sanabria et al., 2018).

4. Discussion

This study developed a robust and theoretically-informed methodology to adapt an internet-delivered intervention for depression in another culture. Limited studies about cultural adaptation in psychotherapy have been conducted, despite twenty years of efforts to conceptualise methods, frameworks and models in this field (Bernal and Adames, 2017; Chu and Leino, 2017; Helms, 2015), and even more limited research available in culturally adapted internet-delivered treatments (Arjadi et al., 2015; Harper-Shehadeh et al., 2016).

Evidence suggests that culturally adapted face-to-face treatments can lead to positive outcomes (Benish et al., 2011; Hall, 2001; Hall et al., 2016; Huey and Polo, 2008; Kalibatseva and Leong, 2014; Moodley et al., 2013). However, there is limited knowledge about any unique criteria that could contribute to an optimal adaptation process (Kalibatseva and Leong, 2014). Likewise, culturally adapted treatments have been critiqued for their lack of theoretical grounding, due to the absence of a systematic adaptation (Chu and Leino, 2017; Helms, 2015). This study is a contribution to expand this field and attempt a systematic modification of an internet-delivered intervention using an integrative organising theoretical framework.

The method was composed of three phases. The first phase facilitated the tailoring of the internet-delivered programme (e.g., language), while maintaining the original treatment components (fidelity). The second phase involved cross-cultural principles assessment research proposed by Lonner (1985) and extend by Helms (2015), and the elements of the ecological validity framework proposed by Bernal and Sáez-Santiago (2006), which were evaluated by users and experts through the Cultural Relevance Questionnaire (CRQ). Finally, the third phase consisted of the incorporations into the programme Space from depression based on the feedback in the CRQ.

In general, users and experts were positive about the culturally adapted programme, while they suggested further changes. The quantitative analysis of the CRQ showed that the initial version of the programme was assessed positively by users and experts regarding the cultural validity of the treatment [Functional equivalence, Conceptual equivalence, Linguistic equivalence]. Functional equivalence scores suggest that the programme included components (e.g. examples, personal stories), that are relevant for the cultural group (Regnault and Herdman, 2015). Furthermore, conceptual equivalence scores suggest that the online intervention used constructs that should be understood by the target population (e.g. adjectives that describe depression) (Helms, 2015). Finally, linguistic equivalence scores suggests that the language used in the programme could be improved (e.g. inclusion of cultural expressions) within the population. Expert's ratings were slightly lower than ratings of users in the general section in the CRQ, which might support a more rigorous analysis of items from the experts, who also provided several comments to the programme components compared to users.

The methodology documented in this paper is a first experience that illustrates an attempt to systematically modify an evidence-based internet-delivered intervention for depression, which is replicable in future studies. This experience supports the inclusion of community members in the process of adapting or developing a treatment as an essential component to increase its validity (Bernal and Domenech-Rodríguez, 2012). Furthermore, studies have demonstrated participation of the community in cultural adaptations is of utmost important to attract and retain participants (Kalibatseva and Leong, 2014).

Studies have argued the importance of developing measures and refining existing measures of cultural validity as the next step in this field (Castro et al., 2010; Helms, 2015). The integrative approach adopted for the current study with the development of the associated CRQ measure is a first contribution to integrate a paradigm for developing measures to incorporate ethnic and racial experiences, previously discussed by Helms (2015). The CRQ can be further refined and elaborated upon in future studies. The authors recommends in the culturally adapted assessment process based on the CRQ to include the original version of the programme and the adapted version to verify the cultural equivalences' content in the programmes for future studies as is suggested by (Helms, 2015). In this study, not all of our evaluators could check the English version, due to limited knowledge of the language. Therefore, the assessment was based on the observations from the culturally adapted Space from depression programme and thus can be rather seen as assessing cultural relevance than equivalence.

The inclusion of diverse populations are generally underrepresented in scientific and clinical analyses and are known to be a hard-to-reach population for research purposes (Arjadi et al., 2015; Cuijpers et al., 2018). The current study supports the inclusion of diverse population in research and the relevant necessity to use culturally adapted internet-delivered interventions for depression.

4.1. Limitations

Our approach was characterised mostly as top-down (the initial adaptation was prepared by an expert and the researcher) and therefore likely misses some of the richness that could be gained from a more “balanced” approach that would incorporate more fully the views, interests, opinions of users and clinicians on the ground, such as opinions from a focus groups. It is relevant that Colombia is a diverse cultural country, where socioeconomic status also present cultural differences (Culture Ministry of Colombia, 2013). There are challenges and risks to adapting an intervention, such as presuming global understanding from knowledge of any one culture or subculture (Rathod and Kingdon, 2014).

Another limitation is that the college students' sample who evaluated the preliminary cultural adaptation of the programme did not include college students with depressive symptomatology. The Cultural Relevance Questionnaire (CRQ) showing reliability in the sample (Cronbach's Alpha 0.744). However the study not included a representative sample that allows identify factors loadings analysis and therefore a validation of the CRQ. Also, the second section of the CRQ was completed by only the experts, leaving out comments and suggestions for college students.

4.2. Future directions

Internet-delivered interventions for depression have shown significant effects post treatment (Richards and Richardson, 2012), specifically in Western high income countries. Low and middle income countries around the world have scarce access to mental health services (World Health Organization, 2008). A strategy to utilise the internet to provide more widely-available and low cost mental health care has potential. Additionally, internet-delivered treatments could contribute to the globalization of mental health services and psychological interventions, for which cultural adaptation is key (Arjadi et al., 2015). Therefore, establishing a standardised procedure for cultural adaptation of treatments is relevant.

The authors recommend the empirical examination of the process for cultural adaptation and to incorporate multiple methods of assessment, including measures such as the CRQ, and indeed improving upon this initial CRQ, which can assess the ecological validity of a culturally adapted treatment. Furthermore, community evaluations are a relevant part of the culturally adapted programme; therefore we suggest to include their evaluations before, during and at the end of the intervention, which is recommended by others authors (Castro et al., 2010).

Cultural adaptations methodologies appear to be critical for engaging the users and delivering services effectively (Domenech Rodríguez et al., 2011). Research in the area needs more work to empirically document the impact of culturally adapting existing evidence based internet-delivered interventions. The information presented in this paper may be of interest to the community of researchers, who are integrating knowledge and procedures with the realities of working in ethnically, culturally, socio-economically, and otherwise diverse communities.

4.3. Implications

The culturally adapted programme (Space from Depression) described in this paper is currently being researched in a randomised control trial (RCT) to examine the efficacy and feasibility of internet-delivered treatment for Colombians with depression (Salamanca-Sanabria et al., 2018). The authors recommend conducting RCTs of culturally adapted interventions, and to continue to assess the adaptations, in order to achieve efficacy and engagement with the treatment.

Conflicts of interest

Alicia Salamanca is the recipient of an Irish Research Council (IRC) scholarship award [EPSPG/2014/98] under the Enterprise/Academia collaboration scheme. The scholarship is funded by the IRC and SilverCloud Health. Since March 2018 Alicia Salamanca is a part time employee in SilverCloud Health. Dr. Derek Richards is Director of Clinical Research at SilverCloud Health. Dr. Ladislav Timulak provides a research consultancy for SilverCloud Health.

Acknowledgements

Alicia Salamanca-Sanabria would like to acknowledge the Irish Research Council (IRC) and SilverCloud Health for the scholarship granted under the Enterprise/ Academia collaboration scheme[EPSPG/2014/98].

As well as all the evaluators who collaborated in this study.

Footnotes

Appendix A

Supplementary data to this article can be found online at https://doi.org/10.1016/j.invent.2018.11.005.

Appendix A. Supplementary data

Supplementary material

mmc1.docx (56.4KB, docx)

References

  1. American Psychological Association Guidelines on multicultural education, training, research, practice, and organizational change for psychologists. Am. Psychol. 2003;58(5):377–402. doi: 10.1037/0003-066x.58.5.377. [DOI] [PubMed] [Google Scholar]
  2. Arango-Lasprilla J.C., Panyavin I., Merchán E.J.H., Perrin P.B., Arroyo-Anlló E.M., Snipes D.J., Arabia J. Evaluation of a group cognitive–behavioral dementia caregiver intervention in Latin America. Am. J. Alzheimers Dis. Other Demen. 2014;29(6):548–555. doi: 10.1177/1533317514523668. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Arjadi R., Nauta M., Chowdhary N., Bockting C. A systematic review of online interventions for mental health in low and middle income countries: a neglected field. Glob. Ment. Health. 2015;2 doi: 10.1017/gmh.2015.10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Arrivillaga-Quintero M., Cortés-Garcia C., Goicochea-Jiménez V.L., Lozano-Ortiz T.M. Caracterización de la Depresión en Jóvenes Universitarios. Characterization of Depression in University Young People. Universitas Psychologica. 2004;3(1):17–26. [Google Scholar]
  5. Ayuso-Mateos J.L., Vázquez-Barquero J.L., Dowrick C., Lehtinen V., Dalgard O.S., Casey P.…Wilkinson G. Depressive disorders in Europe: prevalence figures from the ODIN study. Br. J. Psychiatry. 2001;179(4):308–316. doi: 10.1192/bjp.179.4.308. [DOI] [PubMed] [Google Scholar]
  6. Beck A., Alford B.A. University of Pennsylvania Press; 2009. Depression: Causes and Treatment. [Google Scholar]
  7. Benish S.G., Quintana S., Wampold B.E. Culturally adapted psychotherapy and the legitimacy of myth: a direct-comparison meta-analysis. J. Couns. Psychol. 2011;58(3):279–289. doi: 10.1037/a0023626. [DOI] [PubMed] [Google Scholar]
  8. Bernal G., Adames C. Cultural adaptations: conceptual, ethical, contextual, and methodological issues for working with ethnocultural and majority-world populations. Prev. Sci. 2017;18(6):681–688. doi: 10.1007/s11121-017-0806-0. [DOI] [PubMed] [Google Scholar]
  9. Bernal G., Domenech-Rodríguez M. American Psychological Association; 2012. Cultural Adaptations: Tools for Evidence-based Practice with Diverse Populations. [Google Scholar]
  10. Bernal G., Sáez-Santiago E. Culturally centered psychosocial interventions. J. Community Psychol. 2006;34(2):121. [Google Scholar]
  11. Bernal G., Scharró-del-Río M.R. Are empirically supported treatments valid for ethnic minorities? Toward an alternative approach for treatment research. Cult. Divers. Ethn. Minor. Psychol. 2001;7(4):328. doi: 10.1037/1099-9809.7.4.328. [DOI] [PubMed] [Google Scholar]
  12. Bernal G., Bonilla J., Bellido C. Ecological validity and cultural sensitivity for outcome research: issues for the cultural adaptation and development of psychosocial treatments with Hispanics. J. Abnorm. Child Psychol. 1995;23(1):67–82. doi: 10.1007/BF01447045. [DOI] [PubMed] [Google Scholar]
  13. Bernal G., Jiménez-Chafey M.I., Domenech Rodríguez M.M. Cultural adaptation of treatments: a resource for considering culture in evidence-based practice. Prof. Psychol. Res. Pract. 2009;40(4):361–368. [Google Scholar]
  14. Bromet, Andrade L.H., Hwang I., Sampson N.A., Alonso J., De Girolamo G.…Iwata N. Cross-national epidemiology of DSM-IV major depressive episode. BMC Med. 2011;9(1):90. doi: 10.1186/1741-7015-9-90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Canino G., Alegría M. Psychiatric diagnosis–is it universal or relative to culture? J. Child Psychol. Psychiatry. 2008;49(3):237–250. doi: 10.1111/j.1469-7610.2007.01854.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Castro F.G., Barrera M., Holleran Steiker L.K. Issues and challenges in the design of culturally adapted evidence-based interventions. Annu. Rev. Clin. Psychol. 2010;Vol. 6:213–239. doi: 10.1146/annurev-clinpsy-033109-132032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Choi I., Zou J., Titov N., Dear B.F., Li S., Johnston L., Hunt C. Culturally attuned internet treatment for depression amongst Chinese Australians: a randomised controlled trial. J. Affect. Disord. 2012;136(3):459–468. doi: 10.1016/j.jad.2011.11.003. [DOI] [PubMed] [Google Scholar]
  18. Chowdhary N., Jotheeswaran A., Nadkarni A., Hollon S., King M., Jordans M.…Patel V. The methods and outcomes of cultural adaptations of psychological treatments for depressive disorders: a systematic review. Psychol. Med. 2014;44(06):1131–1146. doi: 10.1017/S0033291713001785. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Chu J., Leino A. Advancement in the maturing science of cultural adaptations of evidence-based interventions. J. Consult. Clin. Psychol. 2017;85(1):45–57. doi: 10.1037/ccp0000145. (10.1037/ccp0000145.supp (Supplemental)) [DOI] [PubMed] [Google Scholar]
  20. Collado A., Lim A.C., MacPherson L. A systematic review of depression psychotherapies among Latinos. Clin. Psychol. Rev. 2016;45:193–209. doi: 10.1016/j.cpr.2016.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Cova Solar F., Alvial S.W., Aro D.M., Bonifetti D.A., Hernández M.M., Rodríguez C.C. Problemas de salud mental en estudiantes de la Universidad de Concepción = Mental health problems in students from the University of Concepción. Ter. Psicol. 2007;25(2):105–112. [Google Scholar]
  22. Cuijpers P., Berking M., Andersson G., Quigley L., Kleiboer A., Dobson K.S. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can. J. Psychiatr. 2013;58(7):376–385. doi: 10.1177/070674371305800702. [DOI] [PubMed] [Google Scholar]
  23. Cuijpers P., Cristea I.A., Ebert D.D., Koot H.M., Auerbach R.P., Bruffaerts R., Kessler R. Depressive symptoms are commonly reported in general populations and also among college students (Rotenstein et al., 2016) Depress. Anxiety. 2016;33(5):400–414. doi: 10.1002/da.22461. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Cuijpers P., Karyotaki E., Reijnders M., Purgato M., Barbui C. Psychotherapies for depression in low-and middle-income countries: a meta-analysis. World Psychiatry. 2018;17(1):90–101. doi: 10.1002/wps.20493. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Culture Ministry of Colombia Cultural diversity. 2013. http://www.mincultura.gov.co/areas/poblaciones/publicaciones/Documents/Cartilla%20Diversidad%20Cultural.pdf Retrieved from.
  26. Dey I. Routledge; 2003. Qualitative Data Analysis: a User Friendly Guide for Social Scientists. [Google Scholar]
  27. Domenech Rodríguez M., Baumann A.A., Schwartz A.L. Cultural adaptation of an evidence based intervention: from theory to practice in a Latino/a community context. Am. J. Community Psychol. 2011;47(1–2):170–186. doi: 10.1007/s10464-010-9371-4. [DOI] [PubMed] [Google Scholar]
  28. Espinosa H.D., Carrasco Á., Moessner M., Cáceres C., Gloger S., Rojas G.…Krause M. Acceptability study of “Ascenso”: an online program for monitoring and supporting patients with depression in Chile. Telemed. E-Health. 2016;22(7):577–583. doi: 10.1089/tmj.2015.0124. [DOI] [PubMed] [Google Scholar]
  29. Ferrari A., Charlson F.J., Norman R.E., Patten S.B., Freedman G., Murray C.J.…Whiteford H.A. Burden of depressive disorders by country, sex, age, and year: findings from the global burden of disease study 2010. PLoS Med. 2013;10(11) doi: 10.1371/journal.pmed.1001547. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Gallagher-Thompson D. Department of Veterans Affairs Medical Center; 1992. Controlling your Frustration: a Class for Caregivers. [Google Scholar]
  31. Gallagher-Thompson D., Arean P., Rivera P., Thompson L.W. A psychoeducational intervention to reduce distress in Hispanic family caregivers: results of a pilot study. Clin. Gerontol. 2001;23(1–2):17–32. [Google Scholar]
  32. Givens J.L., Houston T.K., Van Voorhees B.W., Ford D.E., Cooper L.A. Ethnicity and preferences for depression treatment. Gen. Hosp. Psychiatry. 2007;29(3):182–191. doi: 10.1016/j.genhosppsych.2006.11.002. [DOI] [PubMed] [Google Scholar]
  33. Gómez-Restrepo C., Ospina L., Castro-Díaz S., Gil F., Villegas C.A.A. Prevalencia del trastorno depresivo en pacientes que consultan un servicio de urgencias en Bogotá. = Prevalence of depressive disorder among emergency department patients in Bogotá. Rev. Colomb. Psiquiatr. 2011;40(1):22–37. [Google Scholar]
  34. Griner D., Smith T.B. Culturally adapted mental health intervention: a meta-analytic review. Psychother. Theory Res. Pract. Train. 2006;43(4):531–548. doi: 10.1037/0033-3204.43.4.531. [DOI] [PubMed] [Google Scholar]
  35. Hall Psychotherapy research with ethnic minorities: empirical, ethical, and conceptual issues. J. Consult. Clin. Psychol. 2001;69(3):502–510. doi: 10.1037//0022-006x.69.3.502. [DOI] [PubMed] [Google Scholar]
  36. Hall, Ibaraki A.Y., Huang E.R., Marti C.N., Stice E. A meta-analysis of cultural adaptations of psychological interventions. Behav. Ther. 2016;47(6):993–1014. doi: 10.1016/j.beth.2016.09.005. [DOI] [PubMed] [Google Scholar]
  37. Haroz E., Ritchey M., Bass J., Kohrt B., Augustinavicius J., Michalopoulos L.…Bolton P. How is depression experienced around the world? A systematic review of qualitative literature. Soc. Sci. Med. 2017;183:151–162. doi: 10.1016/j.socscimed.2016.12.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Harper-Shehadeh M., Heim E., Chowdhary N., Maercker A., Albanese E. Cultural adaptation of minimally guided interventions for common mental disorders: a systematic review and meta-analysis. JMIR Ment. Health. 2016;3(3) doi: 10.2196/mental.5776. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Helms J.E. An examination of the evidence in culturally adapted evidence-based or empirically supported interventions. Transcult. Psychiatry. 2015;52(2):174. doi: 10.1177/1363461514563642. [DOI] [PubMed] [Google Scholar]
  40. Huey S.J., Polo A.J. Evidence-based psychosocial treatments for ethnic minority youth. J. Clin. Child Adolesc. Psychol. 2008;37(1):262–301. doi: 10.1080/15374410701820174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Hwang W.-C. The formative method for adapting psychotherapy (FMAP): a community-based developmental approach to culturally adapting therapy. Prof. Psychol. Res. Pract. 2009;40(4):369–377. doi: 10.1037/a0016240. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Ibrahim A.K., Kelly S.J., Adams C.E., Glazebrook C. A systematic review of studies of depression prevalence in university students. J. Psychiatr. Res. 2013;47(3):391–400. doi: 10.1016/j.jpsychires.2012.11.015. [DOI] [PubMed] [Google Scholar]
  43. Ince B.Ü., Cuijpers P., van 't Hof E., van Ballegooijen W., Christensen H., Riper H. Internet-based, culturally sensitive, problem-solving therapy for Turkish migrants with depression: randomized controlled trial. J. Med. Internet Res. 2013;15(10):248–262. doi: 10.2196/jmir.2853. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Kalibatseva Z., Leong F.T.L. A critical review of culturally sensitive treatments for depression: recommendations for intervention and research. Psychol. Serv. 2014;11(4):433–450. doi: 10.1037/a0036047. [DOI] [PubMed] [Google Scholar]
  45. Kanter J.W., Santiago-Rivera A.L., Santos M.M., Nagy G., López M., Hurtado G.D., West P. A randomized hybrid efficacy and effectiveness trial of behavioral activation for Latinos with depression. Behav. Ther. 2015;46(2):177–192. doi: 10.1016/j.beth.2014.09.011. [DOI] [PubMed] [Google Scholar]
  46. Karasz A. Cultural differences in conceptual models of depression. Soc. Sci. Med. 2005;60(7):1625–1635. doi: 10.1016/j.socscimed.2004.08.011. [DOI] [PubMed] [Google Scholar]
  47. Kayrouz R., Dear B.F., Johnston L., Gandy M., Fogliati V.J., Sheehan J., Titov N. A feasibility open trial of guided internet-delivered cognitive behavioural therapy for anxiety and depression amongst Arab Australians. Internet Interv. 2015;2:32–38. [Google Scholar]
  48. Kayrouz R., Dear B.F., Johnston L., Keyrouz L., Nehme E., Laube R., Titov N. Intergenerational and cross-cultural differences in emotional wellbeing, mental health service utilisation, treatment-seeking preferences and acceptability of psychological treatments for Arab Australians. Int. J. Soc. Psychiatry. 2015;61(5):484–491. doi: 10.1177/0020764014553004. [DOI] [PubMed] [Google Scholar]
  49. Kessler, Bromet E. The epidemiology of depression across cultures. Annu. Rev. Public Health. 2013;34(1):119–138. doi: 10.1146/annurev-publhealth-031912-114409. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Kessler R., Aguilar-Gaxiola S., Alonso J., Chatterji S., Lee S., Ormel J.…Wang P.S. The global burden of mental disorders: an update from the WHO World Mental Health (WMH) surveys. Epidemiol. Psichiatr. Soc. 2009;18(01):23–33. doi: 10.1017/s1121189x00001421. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Kessler, Birnbaum H.G., Shahly V., Bromet E., Hwang I., McLaughlin K.A.…Demyttenaere K. Age differences in the prevalence and co-morbidity of DSM-IV major depressive episodes: results from the WHO World Mental Health survey initiative. Depress. Anxiety. 2010;27(4):351–364. doi: 10.1002/da.20634. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Kirmayer L.J., Gomez-Carrillo A., Veissière S. Culture and depression in global mental health: an ecosocial approach to the phenomenology of psychiatric disorders. Soc. Sci. Med. 2017;183:163–168. doi: 10.1016/j.socscimed.2017.04.034. [DOI] [PubMed] [Google Scholar]
  53. La Roche M., Christopher M.S. Culture and empirically supported treatments: on the road to a collision? Cult. Psychol. 2008;14(3):333–356. [Google Scholar]
  54. Lara M., Navarro C., Rubí N., Mondragón L. Outcome results of two levels of intervention in low-income women with depressive symptoms. Am. J. Orthop. 2003;73(1):35–43. doi: 10.1037/0002-9432.73.1.35. [DOI] [PubMed] [Google Scholar]
  55. Lara M., Tiburcio M., Aguilar-Abrego A., Sánchez-Solís A. A four-year experience with a web-based self-help intervention for depressive symptoms in Mexico. Rev. Panam. Salud Publica. 2014;35(5–6):399–406. [PubMed] [Google Scholar]
  56. Lau A.S. Making the case for selective and directed cultural adaptations of evidence-based treatments: examples from parent training. Clin. Psychol. Sci. Pract. 2006;13(4):295–310. [Google Scholar]
  57. Lonner W.J. Issues in testing and assessment in cross-cultural counseling. Couns. Psychol. 1985;13(4):599–614. [Google Scholar]
  58. Martínez P., Rojas G., Martínez V., Lara M.A., Pérez C. Internet-based interventions for the prevention and treatment of depression in people living in developing countries: a systematic review. J. Affect. Disord. 2018;234:193–200. doi: 10.1016/j.jad.2018.02.079. [DOI] [PubMed] [Google Scholar]
  59. McKleroy V.S., Galbraith J.S., Cummings B., Jones P., Harshbarger C., Collins C.…Team A. Adapting evidence-based behavioral interventions for new settings and target populations. AIDS Educ. Prev. 2006;18:59–73. doi: 10.1521/aeap.2006.18.supp.59. (supp) [DOI] [PubMed] [Google Scholar]
  60. Melo-Carrillo A., Van Oudenhove L., Lopez-Avila A. Depressive symptoms among Mexican medical students: high prevalence and the effect of a group psychoeducation intervention. J. Affect. Disord. 2012;136(3):1098–1103. doi: 10.1016/j.jad.2011.10.040. [DOI] [PubMed] [Google Scholar]
  61. Ministry of Health . 2015. National Mental Health survey (978-958-8903-19-4) (retrieved from Bogota, Colombia) [Google Scholar]
  62. Ministry of Social Protection . 2012. Report on the national panorama of health and disease. (Retrieved from Bogotá, Colombia) [Google Scholar]
  63. Miranda J., Bernal, Lau A., Kohn L., Hwang W.-C., LaFromboise T. State of the science on psychosocial interventions for ethnic minorities. Annu. Rev. Clin. Psychol. 2005;1(1):113–142. doi: 10.1146/annurev.clinpsy.1.102803.143822. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Moodley R., Gielen U.P., Wu R. Routledge/Taylor & Francis Group; New York, NY, US: 2013. Handbook of Counseling and Psychotherapy in an International Context. [Google Scholar]
  65. Morales C.F., Pérez V.B., León S.O., Medrano A.J., Urrutia Aguilar M.E., Guevara-Guzmán R. Síntomas depresivos y rendimiento escolar en estudiantes de Medicina. = Depressive symptoms and academic performance in medical students. Salud Mental. 2013;36(1):59–65. [Google Scholar]
  66. Munoz R.F., Ying Y.-W. JHU Press; 2002. The Prevention of Depression: Research and Practice. [Google Scholar]
  67. Muñoz R.A., McBride M.E., Brnabic A.J.M., López C.J., Hetem L.A.B., Secin R., Dueñas H.J. Major depressive disorder in Latin America: the relationship between depression severity, painful somatic symptoms, and quality of life. J. Affect. Disord. 2005;86(1):93–98. doi: 10.1016/j.jad.2004.12.012. [DOI] [PubMed] [Google Scholar]
  68. Nicolas G., Arntz D.L., Hirsch B., Schmiedigen A. Cultural adaptation of a group treatment for Haitian American adolescents. Prof. Psychol. Res. Pract. 2009;40(4):378–384. [Google Scholar]
  69. Nogueira-Martins L.A., Neto R.F., Macedo P.C.M., Cítero V.A., Mari J.J. The mental health of graduate students at the Federal University of São Paulo: a preliminary report. Braz. J. Med. Biol. Res. 2004;37(10):1519–1524. doi: 10.1590/s0100-879x2004001000011. [DOI] [PubMed] [Google Scholar]
  70. Parker G., Gladstone G., Chee K.T. Depression in the planet's largest ethnic group: the Chinese. Am. J. Psychiatr. 2001;158(6):857–864. doi: 10.1176/appi.ajp.158.6.857. [DOI] [PubMed] [Google Scholar]
  71. Rathod S. Contemporary psychotherapy and cultural adaptations. J. Contemp. Psychother. 2016:1–3. [Google Scholar]
  72. Rathod S., Kingdon D. Case for cultural adaptation of psychological interventions for mental healthcare in low and middle income countries. BMJ. 2014;349:g7636. doi: 10.1136/bmj.g7636. [DOI] [PubMed] [Google Scholar]
  73. Regnault A., Herdman M. Using quantitative methods within the universalist model framework to explore the cross-cultural equivalence of patient-reported outcome instruments. Qual. Life Res. Int. J. Qual. Life Asp. Treat. Care Rehab. 2015;24(1):115–124. doi: 10.1007/s11136-014-0722-8. [DOI] [PubMed] [Google Scholar]
  74. Renner F., Arntz A., Leeuw I., Huibers M. Treatment for chronic depression using schema therapy. Clin. Psychol. Sci. Pract. 2013;20(2):166–180. [Google Scholar]
  75. Resnicow K., Soler R., Braithwaite R.L., Ahluwalia J.S., Butler J. Cultural sensitivity in substance use prevention. J. Community Psychol. 2000;28(3):271–290. [Google Scholar]
  76. Richards, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin. Psychol. Rev. 2012;32(4):329–342. doi: 10.1016/j.cpr.2012.02.004. [DOI] [PubMed] [Google Scholar]
  77. Richards D., Salamanca-Sanabria Point-prevalence of depression and associated risk factors. Aust. J. Psychol. 2014;148(3):305–326. doi: 10.1080/00223980.2013.800831. [DOI] [PubMed] [Google Scholar]
  78. Richards D., Timulak L., O'Brien E., Hayes C., Vigano N., Sharry J., Doherty G. A randomized controlled trial of an internet-delivered treatment: its potential as a low-intensity community intervention for adults with symptoms of depression. Behav. Res. Ther. 2015;75:20–31. doi: 10.1016/j.brat.2015.10.005. [DOI] [PubMed] [Google Scholar]
  79. Richards D., Murphy T., Viganó N., Timulak L., Doherty G., Sharry J., Hayes C. Acceptability, satisfaction and perceived efficacy of “space from depression” an internet-delivered treatment for depression. Internet Interv. 2016;5:12–22. doi: 10.1016/j.invent.2016.06.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  80. Rosselló J., Bernal The efficacy of cognitive-behavioral and interpersonal treatments for depression in Puerto Rican adolescents. J. Consult. Clin. Psychol. 1999;67(5):734–745. doi: 10.1037//0022-006x.67.5.734. [DOI] [PubMed] [Google Scholar]
  81. Rosselló J., Bernal . New developments in cognitive-behavioral and interpersonal treatments for depressed Puerto Rican adolescents. In: Hibbs E.D., Jensen P.S., Hibbs E.D., Jensen P.S., editors. Psychosocial Treatments for Child and Adolescent Disorders: Empirically Based Strategies for Clinical Practice. 2nd ed. American Psychological Association; Washington, DC, US: 2005. pp. 187–217. [Google Scholar]
  82. Rosselló J., Bernal G., Rivera-Medina C. Individual and group CBT and IPT for Puerto Rican adolescents with depressive symptoms. Cultur. Divers. Ethnic Minor. Psychol. 2012;1(S):36–51. doi: 10.1037/1099-9809.14.3.234. [DOI] [PubMed] [Google Scholar]
  83. Sáez-Santiago E., Bernal G., Reyes-Rodríguez M.L., Bonilla-Silva K. Development and cultural adaptation of the Taller de Educación Para Padres y Madres (TEPSI): Psychoeducation for parents of Latino/a adolescents with depression. In: Bernal G., Domenech Rodríguez M., editors. Cultural adaptations: Tools for evidence-based practice with diverse populations. 2012. pp. 91–112. [Google Scholar]
  84. Salamanca-Sanabria A., Richards D., Timulak L., Castro-Camacho L., Mojica-Perilla M., Parra-Villa Y. Assessing the efficacy of a culturally adapted cognitive behavioural internet-delivered treatment for depression: protocol for a randomised controlled trial. BMC Psychiatry. 2018;18(1):53. doi: 10.1186/s12888-018-1634-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  85. Smith T., Trimble J.E. Foundations of Multicultural Psychology: Research to Inform Effective Practice. American Psychological Association; Washington, DC, US: 2016. Culturally adapted mental health services: an updated meta-analysis of client outcomes; pp. 129–144. [Google Scholar]
  86. Tiburcio M., Lara M.A., Abrego A.A., Fernández M., Vélez N.M., Sánchez A. Web-based intervention to reduce substance abuse and depressive symptoms in Mexico: development and usability test. JMIR Ment. Health. 2016;3(3):e47. doi: 10.2196/mental.6001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  87. Trimble J.E., Scharrón-del-Río M.R., Hill J.S. Cultural Adaptations: Tools for Evidence-based Practice with Diverse Populations. American Psychological Association; Washington, DC: 2012. Ethical considerations in the application of cultural adaptation models with ethnocultural populations; p. 45Á67. [Google Scholar]
  88. Vally Z., Maggott C. Evaluating the outcome of cultural adaptations of cognitive-behavioural therapy for adult depression: a meta-analysis of treatment studies in developing countries. Int. J. Adv. Couns. 2015;37(4):293–304. [Google Scholar]
  89. van Loon A., van Schaik A., Dekker J., Beekman A. Bridging the gap for ethnic minority adult outpatients with depression and anxiety disorders by culturally adapted treatments. J. Affect. Disord. 2013;147(1–3):9–16. doi: 10.1016/j.jad.2012.12.014. [DOI] [PubMed] [Google Scholar]
  90. Wilson B.D., Miller R.L. Examining strategies for culturally grounded HIV prevention: a review. AIDS Educ. Prev. 2003;15(2):184–202. doi: 10.1521/aeap.15.3.184.23838. [DOI] [PubMed] [Google Scholar]
  91. World Health Organization . 2008. mhGAP: Mental Health Gap Action Program: scaling up care for mental, neurological, and substance use disorders.http://whqlibdoc.who.int/publications/2008/9789241596206_eng.pdf?ua=1 retrieved from. [PubMed] [Google Scholar]
  92. World Health Organisation, W . World Health Organisation. 2017. Depression and other common mental disorders: global health estimates. [Google Scholar]
  93. Yusim A., Anbarasan D., Hall B., Goetz R., Neugebauer R., Stewart T.…Ruiz P. Sociocultural domains of depression among indigenous populations in Latin America. Int. Rev. Psychiatry. 2010;22(4):370–377. doi: 10.3109/09540261.2010.500870. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary material

mmc1.docx (56.4KB, docx)

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES