Cross-cultural application of Achenbach System of Empirically Based Assessment: instrument translation in Chinese, challenges, and future directions

Introduction

Early childhood behavioral problems are significant public health issues and have been associated with many negative outcomes, including poor academic performance, low self-esteem, juvenile delinquency, adult criminal behavior, psychiatric diagnosis later in life, aggression, and violence.^[1] The Achenbach System of Empirically Based Assessment (ASEBA) is one of the most widely used and highly validated measures for assessing child behavioral problems, including externalizing problems (e.g., aggressive and rule-breaking behavior), internalizing problems (e.g., depression and anxiety), and attention problems (e.g., inattention and hyperactivity). The ASEBA for school-age children consists of three parallel questionnaires: the Child Behavior Checklist (CBCL) with parents as informants, the Teacher's Report Form (TRF), and the Youth Self-Report (YSR).

Assessing healthcare issues including mental health diagnosis and treatment in different ethnic groups is an important factor for providing culturally competent care. It is essential to develop outcome measures in non-English languages that are culturally sensitive and appropriate for the target group. Additionally, applying the same standardized approach to developing instruments in other languages is necessary to compare one cultural group with another.

To date, ASEBA instrumentation has been translated into 80 languages, with thousands of published reports from over 60 cultures.^[2] For school-age children and adolescents, the taxonomic construct validity of ASEBA syndromes has been supported by confirmatory factor analyses of the CBCL from 30 countries (n=58 043), the TRF from 20 countries (n=30 030), and the YSR from 23 societies (n=30 243).^[3-5] More recent studies^[6,7] have reported using the ASEBA as an international comparison for behavioral problems in preschool children across 24 and 23 societies, respectively. Despite the number and success of current translations, more are needed. This article aims to demonstrate the need for a Chinese version of the ASEBA for preschool and school-age children, and to illustrate the challenges of such a translation.

The ASEBA for Chinese populations

There is a great need for accurate and reliable behavioral assessments of Chinese populations. China's modernization and industrial growth have produced many factors affecting Chinese children negatively, resulting in an increase in behavior problems.^[8] First, the influence of the Chinese media continues to grow and affect the population. In a study from Shanghai the presence of media in school-aged children's bedrooms resulted in negative effects on their sleep/wake patterns, duration of sleep, and sleep disorders.^[9] This may in turn affect how children behave during the day and also may have an influence on their physiological, psychological, emotional, behavioral, and cognitive development. Second, as China undergoes rapid economic development and reform, psychosocial stress stemming from the educational environment, peer and interpersonal relationships, unemployment, and family conflict may be a risk factor for emotional and behavioral problems among the population, particularly the youth in China.^[1,10] Third, environmental factors (such as air pollution and lead exposure) can raise the risks for neurocognitive and behavior disorders. China's rapid industrial development has brought about serious environmental concerns.^[11] Research on the association between lead exposure and aggression has received increased attention with accumulating evidence from experimental research in animals^[12,13] and epidemiological studies of children and adults.^[14-16] Increased environmental pollution in China will further raise the risks for behavior problems in children.

The development of the Chinese version of the ASEBA is intended to provide a culturally sensitive version of a well-supported, widely used instrument to measure children's functioning. Additionally, having a standardized tool that remains as close to the original ASEBA as possible ensures validity and reliability when comparing English-speaking populations with Chinese-speaking populations. However, translation of the ASEBA into Chinese is somewhat hindered by the fact that Chinese is a complex, highly nuanced language containing many dialects that vary widely by geographic regions (e.g., Mandarin, Cantonese, Hsiang, Wu, etc.). Therefore, a translation based on only one dialect might not be generalizable to all Chinese-speaking regions, suggesting the need for region-specific translations. We encountered very similar challenges in our translation of all three ASEBA questionnaires; therefore the following account will focus only on our experiences in translating the CBCL and Caregiver-Teacher Report Form/1-5 (CTRF/1-5), which can be extended to encompass our experience in translating the other two questionnaires as well. We hope that by presenting our reflections on this process, we can provide some insights into future instrument translations for cross-cultural studies.

While translation is a vital part of cross-cultural research, the mere establishment of a linguistic equivalence between measures is insufficient to determine the cross-cultural validation of an assessment tool. That is, beyond translation, further studies of validation and standardization are needed to demonstrate the applicability of a measure to other cultures. However, the focus of this article is limited to that of the translation phase. Recognizing translation as one component of cross-cultural research, we believe it is one of the most important initial steps in establishing true cross-cultural validity of a measure, without which accurate validation or standardization of a measure in another population is highly problematic.

Our efforts to translate ASEBA instruments into a culturally appropriated amalgamation of Mainland and Hong Kong Chinese fit into the wider arena of instrument translation and adaptation outlined by Geisinger's^[17] review of the Psychological Association's (1974-1992) Psychological Abstracts on foreign language translation and testing. Briefly, Geisinger pointed out that recently the term test adaptation frequently has replaced test translation. This shift in terminology documents the adaptations in references to culture, in content, and in wording that are needed in addition to simple translation in revising a test. According to Geisinger, there are very few guidelines regarding test adaptations that have been developed, and those that do exist have not been widely circulated. We hope that through sharing our experience in our own translation process, we can help in addressing this present gap in the literature.

The translation process

Chinese Hong Kong version of the ASEBA

The Hong Kong Chinese versions of the school-age CBCL, TRF, and YSR were translated by a group of psychology professors and graduate students at the Chinese Hong Kong University in the mid 1990s. Translation from Achenbach's original 1991 used the standard procedure of both translating and back-translating. The 2001 English school-age versions of the CBCL, TRF, and YSR were translated into the Hong Kong version shortly after their publication by the same group. This also used the standard procedure of translation and back-translation. The final versions were reviewed and approved by committee consensus, including a panel of clinical psychologists and child psychiatrists. Test-retest reliability and criterion validity of the Chinese versions of the CBCL, TRF, and YSR were given by Leung et al.^[18]

Chinese Mandarin version of the ASEBA

Translation of the Chinese Mandarin version of the ASEBA was a two-stage process. The first stage emphasized translation of the preschool version. This took place initially during the early 2000s when the instrument was used in the China Jintan Project.^[11] The translation focused on maximally retaining the original wording with appropriate Chinese. We followed the standardized translation process using doctoral-level, bilingual researchers. We also involved a focus group that included preschool principals and teachers, elementary school teachers, parents of the cohort subjects, school nurses, local researchers, and pediatricians. Several meetings were held until consensus on content validity was reached. The experiences and feedback we received from this data collection process aided our second stage translation.

The second stage was focused on the school-age ASEBA forms. A formal translation process took place between 2008 and 2010 and included an interdisciplinary group of psychologists, public health professionals, nursing educators, and their graduate students. These individuals were not only bilingual in Chinese and English but also bicultural in Mandarin and Cantonese. The same focus group from the first stage of translation participated in this second stage. The focus group reviewed the ASEBA forms (CBCL, TRF, and YSR) item-by-item in order to provide detailed feedback from the community on each item's wording and meanings. This feedback was then incorporated in the ASEBA forms to produce a finalized Chinese version. The translation was conducted using the standard procedure of translation and back-translation. The newly revised Chinese version has been thoroughly modified to fit both Mandarin and Cantonese speaking populations, while retaining the original wording and meaning. Unlike the Hong Kong Chinese version, the Mandarin Chinese version has not yet been validated; however, a systematic, valid translation is the first step in this process. We recently reported the factor structure of the preschool CBCL and CTRF using the newly revised Mandarin Chinese version in Jintan Cohort Project.^[11] This process is illustrated in the Fig.

Fig.

Development of the New Mandarin Chinese version of the ASEBA.

Developing the Chinese CBCL/CTRF/1.5-5: challenges and future plans

Numerous studies support the use of the Chinese ASEBA across many age groups of children and adolescents,^[18-24] and the construct validity of the Chinese ASEBA for ages 6-18 years (CBCL/6-18) has been established.^[18,22] However, less research has been conducted on the Chinese version of the CBCL/CTRF for preschool-age groups (CBCL/CTRF/1.5-5). Assessment of early childhood mental health problems is important for designing early intervention programs and for identifying possible vulnerability factors. Creation of effective prevention programs depends in part on accurate risk assessment. Examination of behavioral and affective outcomes can help healthcare providers understand and prevent the development of future maladaptive behaviors. Within a broader context, this also carries public health implications in terms of possibly preventing violent crime, reducing education drop-out rates, and lowering reliance on government financial assistance programs as maladaptive behaviors have been associated with an array of negative psychosocial, educational, and socioeconomic factors.^[25]

To our knowledge, the only published study examining the Chinese CBCL/1.5-5 for mainland China populations prior to 2000 comes from Tan et al.^[20] However, this study assessed Chinese girls adopted into North American households and thus does not enhance our understanding of the use of the Chinese CBCL in the child's culture of origin. The cultural context in which the child is raised may affect the interpretation of the child's behavior in terms of cultural norms when considering what falls outside the range of “normal” or “acceptable”. Therefore, extrapolation of results from Tan et al to children residing in China would be difficult. Other uses of the Chinese CBCL have either been with school-age children or in translations for dialects in regions such as Hong Kong and Taiwan.^[20]

Use of the Hong Kong translation of the CBCL/1.5-5 had both strengths and weaknesses. Despite the fact that Hong Kong has adapted many grammatical features of modern written Chinese since the 1900s, Hong Kong's Cantonese dialect nevertheless still deviates to some extent from Mainland Chinese. These deviations between written Chinese used in Hong Kong and that used in Mainland China can in part be attributed to the influence of English on the Hong Kong culture.^[26] As noted above, the use of a Chinese version of the CBCL translated from one single dialect of Chinese would not necessarily be sufficient for all Chinese-speaking populations. For instance, for a translation to be usable throughout China, it needs to combine Cantonese and Mandarin syntax, diction, and semantics.

To address this gap in the literature, we developed the Chinese version of the CBCL/CTRF/1.5-5 for Mainland China populations,^[19] based on the existing Hong Kong version of the CBCL/1.5-5 as a template. The procedure for translation included forward translation of the original instrument into the target language by a bilingual translator, review of the Chinese version by a monolingual reviewer for incomprehensible or ambiguous words, backward translation into English, and a comparison of the back-translated version to the original English CBCL/1.5-5 to assess discrepancies. Errors in either the forward or backward-translation process required repeating the entire process. Recently, we published a report on the application of the preschool child behavior checklist and the caregiver-teacher report form to mainland Chinese children using data from the China Jintan Cohort Study.^[11]

The challenges we encountered in translating the Chinese CBCL/CTRF/1.5-5 were similar to the problems echoed in a review of translation studies by Weeks et al.^[27] Overall, the main challenges in our endeavor to translate the ASEBA into Mandarin speaking Chinese included 1) maintaining as much of original wording as possible, 2) maintaining connotations of idiomatic expressions, and 3) ensuring phrasing as concise as possible while still addressing the first two challenges. In the specific case of Chinese, there are further differences between widely different dialects that needed to be addressed, which required us to ensure the questions matched the cultural setting of the local population.

The first type of challenge in our translation process arises from the idiomatic nature of the original English phrases. For example, “gets into everything” is an idiomatic expression that has no literal translation into Chinese. In converting the item to Chinese, there are a variety of word choices that might convey similar ideas. The initial translation of the phrase in Hong Kong Chinese is 样样事都理, which literally means “want to sort everything”. However, this implies extraneous ideas of management and organizational skills that are not implied in the English phrase. Therefore, the phrase is changed to 样样事都要参加, meaning “want to participate in everything”, which is closer to the meaning of the English phrase.

The second type of challenge in our translation process arises from the fact that though phrases sometimes do have literal translations, they do not necessarily possess the same connotations. For example, the phrase “screams a lot” was originally translated to “经常尖叫” which literally means “often screams”. However, the connotation of pain is strongly present in the Chinese word for “scream” than it is in the English word. This translation might therefore imply too strongly that the child is in physical pain rather than just being fussy Therefore, the translation was changed to “经常大喊大叫” which literally means “often yells loudly” and is more appropriate for the context.

Challenges such as these can be effectively addressed by appropriate planning and using translation strategies, such as employing popular idioms rather than relying on local syntax; utilizing teams of translators rather than individuals; recruiting bilingual individuals to back-translate instruments to ensure content is accurate and conveyed clearly; and using focus groups and reviewers from target populations for validation studies.

Translation followed by backward-translation has long been recommended as the standard procedure for adapting an assessment instrument for cross-cultural research.^[28-31] It is considered a rigorous method because it serves to validate and ensure the accuracy of the translated version. Even though backward-translation is a helpful means of testing equivalence between the source language version and the translated language version, it is not necessarily flawless. Inaccuracies in the process arise because comparisons are only made between the original version and the back-translated version in the same language, in this case, English. Incorrect wording in the Chinese version may therefore go undetected because the individual performing the back-translation may intuitively understand the Chinese and translate it back into English, masking any discrepancy between versions. For instance, if the sentence structure in the Chinese version is rarely used and not understood by the local population, a back-translator may still be able to understand and successfully translate it because of his or her familiarity with the structure of English. Therefore, although the literal word-by-word meaning may be correct, someone who only understands Chinese may not understand the intended meaning.

In a review of different translation studies, Maneesriwongul and Dixon^[32] discussed the advantages and disadvantages of different methods for performing translation and verified their respective validities. In addition to backward-translation, administering the translated versions to monolingual subjects can be used to test the cultural as well as linguistic accuracies of the translated version.^[32] However, this method is still not ideal since there is no way of knowing if the monolingual testers are interpreting the wording correctly. According to Maneesriwongul and Dixon,^[32] the most preferred method is to perform back-translation and then to test with both monolingual and bilingual testers. The bilingual testers would complete both the source language version and the translated language version. Overall, this method would test equivalence between the source and translated language versions, examine the appropriateness of both versions for monolingual and bilingual subjects, and detect discrepancies between the two versions. We believe that this method is the best choice when both monolingual and bilingual test subjects are available.

We made use of this method in our own translation process because it represents the most thorough method of ensuring cross-cultural validity. In order to accomplish this, we used a large number of both monolingual and bilingual testers. This group of testers included graduate students, psychologists, professors, parents, and community members. They discussed the translation of the measure item-by-item in order to verify the accuracy and equivalence of the translated version. In addition, we aimed to account for variations in language between the different regions of China. For example, even within the Mainland, there are slight discrepancies between the northern and southern areas. Factors such as education level, lifestyle, and regional practices may vary in these regions and create challenges in comprehension, which could impact responses. In Hong Kong, where there is more of a western influence, certain behaviors are perceived differently than they are in Mainland China. A behavior with a neutral connotation in Hong Kong may have negative connotation in Mainland China. Though such differences are not inherently tied to the language itself, we feel that they should be considered in the translation. Accurate understanding of each item of the ASEBA scale directly influences the instrumental consistency. As such, it is important that we choose common and easily understood expressions during translation. We therefore endeavored to word our translation of ASEBA scale items to be easily understood by respondents with relatively low education levels. To address regional language differences, we used people from the northern and southern regions of China to judge whether the translated CBCL was appropriate for both areas. An example of such a discrepancy is related to the earlier mentioned phrase “gets into everything” which had the original translation of 样样事都理. Here the word 理 literally means “to sort”. This was considered to be inappropriate for northern Mandarin speaking populations, to whom the word would imply management and organizational skills. However, the phrase would be considered more appropriate for southern Cantonese speakers, in whose dialect the word carries less of an organizational implication and more of a “getting one's hands on a matter” connotation. In the end, it was decided that the phrase should be changed to 样样事都要参加, meaning “want to participate in everything”, as it is less regionally charged in addition to being closer to the original meaning of the phrase in English. This is an example of our effort to produce a unified Chinese version of the ASEBA which can be used by both Mandarin and Cantonese speaking populations.

Summary

Children's behavior is complex and may involve both genetic and environmental influences,^[33-37] but it is important to note that assessments of children's behavior should take into account the cultural environment and social content. There is a clear need to establish the use of the ASEBA for Mainland China populations, particularly in order to assess externalizing and internalizing problems among preschool-age children. Our process of translating the Chinese CBCL/1.5-5 shed light on the nuances of establishing cross-cultural validity of an instrument. Other researchers who attempt to translate validated measures from one language into another language should be cognizant that variations within the target language may limit its generalizability. Furthermore, without following proper translation procedures, validation and use of the measure cannot be empirically established. The Chinese language is highly complex, and translation studies to evaluate Chinese translations can benefit from reviewing the challenges faced in our undertaking and considering the solutions provided herein.