Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 Apr 20.
Published in final edited form as: J Int Neuropsychol Soc. 2007 Sep;13(5):781–790. doi: 10.1017/S1355617707071007

Neurobehavioral effects of HIV-1 infection in China and the United States: A pilot study

Lucette A Cysique 1, Hua Jin 1,2, Donald R Franklin Jr 1, Erin E Morgan 1, Chuan Shi 3, Xin Yu 3, Zunyou Wu 4, Michael J Taylor 1, Thomas D Marcotte 1, Scott Letendre 1, Christopher Ake 1, Igor Grant 1, Robert K Heaton 1,2; The HNRC Group
PMCID: PMC2857379  NIHMSID: NIHMS189036  PMID: 17697409

Abstract

The HIV epidemic in China has been increasing exponentially, yet there have been no studies of the neurobehavioral effects of HIV infection in that country. Most neuroAIDS research has been conducted in Western countries using Western neuropsychological (NP) methods, and it is unclear whether these testing methods are appropriate for use in China. Twenty-eight HIV seropositive (HIV+) and twenty-three HIV seronegative (HIV−) individuals with comparable gender, age, and education distributions were recruited in Beijing and the rural Anhui province in China. Thirty-nine HIV+ and thirty-one HIV− individuals were selected from a larger U.S. cohort recruited at the HIV Neurobehavioral Research Center, in San Diego, to be matched to the Chinese sample for age, disease status, and treatment variables. The NP test battery used with the U.S. and China cohorts included instruments widely used to study HIV infection in the United States. It consisted of 14 individual test measures, each assigned to one of seven ability areas thought to be especially vulnerable to effects of HIV on the brain (i.e., verbal fluency, abstraction/executive function, speed of information processing, working memory, learning, delayed recall, and motor function). To explore the cross-cultural equivalence and validity of the NP measures, we compared our Chinese and U.S. samples on the individual tests, as well as mean scaled scores for the total battery and seven ability domains. On each NP test measure, the mean of the Chinese HIV+ group was worse than that of the HIV−group. A series of 2 × 2 analyses of variance involving HIV+ and HIV− groups from both countries revealed highly significant HIV effects on the Global and all Domain mean scaled scores. Country effects appeared on two of the individual ability areas, at least partly due to education differences between the two countries. Importantly, the absence of HIV-by-Country interactions suggests that the NP effects of HIV are similar in the two countries. The NP test battery that was chosen and adapted for use in this study of HIV in China appears to have good cross-cultural equivalence, but appropriate Chinese norms will be needed to identify disease-related impairment in individual Chinese people. To inform the development of such norms, a much larger study of demographic effects will be needed, especially considering the wide range of education in that country.

Keywords: HIV/AIDS, Neuropsychological functioning, Cognition, China, Cross-cultural assessment, Everyday functioning

INTRODUCTION

According to a recent (December 2005) report by the Chinese Ministry of Health, the United Nations Program on HIV/AIDS (UNAIDS) and the World Health Organization (WHO), the HIV epidemic in China has been growing exponentially over the past several years. The reported estimates were that 650,000 people are living with HIV infection in China, with 70,000 new cases of infection occurring in 2005. Also, although the HIV epidemic in China began among injection drug users and commercial sex workers, it has been spreading from these high risk groups to the general population (China Ministry of Health and UN Group report, 2003). Because China is the most populous country on earth, with more than 1.3 billion people, the public health consequences of the Chinese HIV epidemic are of substantial international concern.

HIV enters the central nervous system (CNS) early after infection, and frequently is associated with structural and functional brain abnormalities (Grant et al., 1987; Masliah et al., 1992; Navia et al., 1986). Risks for significant brain involvement are highest in the more advanced stages of HIV disease (approximately 50%), but such changes also occur in approximately a third of early, medically asymptomatic carriers (Heaton et al., 1995; White et al., 1995).

Much of the research concerning HIV-related brain involvement has used standardized neuropsychological (NP) tests as outcome measures (Grant & Martin, 1994; White et al., 1995). The sensitivity of NP testing to brain disorders of diverse etiologies is well established (Lezak et al., 2004). Within the context of HIV infection, results of NP testing have shown robust associations with structural and functional brain imaging (Jernigan et al., 1993; Stout et al., 1998), as well as with postmortem neuropathology findings (Cherner et al., 2002; Everall et al., 1999; Masliah et al., 1997; Moore et al., 2006). Moreover, NP impairment in HIV-infected persons has been shown to be an independent predictor of early mortality (Ellis et al., 1997; Mayeux et al., 1993) and to be strongly predictive of a wide variety of difficulties in activities of daily living (Heaton et al., 2004a).

With a few notable exceptions (Maj et al., 1993, 1994; Yang et al., 1999; Yepthomi et al., 2006), most NP research with HIV+ populations has been conducted in Western countries, usually with NP tests that were standardized in English. It is unclear whether their results will generalize to other parts of the world, where the HIV epidemic has reached massive proportions, and where the specific manifestations of HIV-associated brain disease are largely unknown.

Chan et al. (2003) recently reviewed 123 NP studies (none of which examined neuroAIDS) that were conducted in Asian countries between 1981 and 2002. Studies in mainland China and Hong Kong primarily used measures that were originally standardized in the United States and then translated, sometimes in modified form, for use with Chinese populations. These included well-known instruments, such as the Wechsler Adult Intelligence Scale-Revised (WAIS), the Wechsler Memory Scale (WMS), Chinese adaptations of the Halstead Reitan Battery and Luria Nebraska Battery, the Wisconsin Card Sorting Test, the Category Fluency Test, Color Trails, the Hiscock Forced Choice Digit Memory Test (for assessing effort), and various aphasia batteries and dementia screening instruments. In an effort to identify tests that might have immediate applicability in clinical and research applications, Chan et al. determined whether measures (1) had evidence of accurate translation (by independent back-translation into English), (2) used local norms that included Asian samples of more than 50 subjects, and (3) provided evidence of convergent or discriminant validity, and cross-cultural equivalence. The authors reported that only two tests used in mainland China and three in Hong Kong met these criteria. These included two dementia screening tests, two clinical memory tests, and a category fluency test.

The authors concluded that, although many Western NP tests have been used in China, adequate local/national norms and evidence of cross-cultural equivalence of the tests are frequently lacking. This finding may have more to do with a dearth of research than limitations of the tests. Nevertheless, the relative lack of proven instruments and local norms is a major challenge in designing neurobehavioral research in China.

Starting in early 2003, the HIV Neurobehavioral Research Center (HNRC) group, from the University of California at San Diego (UCSD), began discussing with investigators from the China Comprehensive International Program for Research on AIDS (CIPRA) the possibility of establishing a collaboration to conduct neuroAIDS studies in China. To determine the feasibility of such a study, a pilot project was initiated to examine the adaptability and validity of neuro-cognitive assessments used at HNRC, in HIV-infected persons (HIV+) in China. In this article, we report our findings on these neurocognitive assessments in HIV+ and demographically matched HIV-uninfected persons (HIV−) in China, and compare their results with those from HIV+ and demographically matched HIV− persons in the United States. As such, this study is the first to examine effects of HIV on neurocognitive status in a Chinese sample, and also the first to compare results between China and the United States to explore the cross-cultural validity of the NP instruments used.

METHODS

The China HIV Neurocognitive Feasibility Study was launched in early 2004 after more than 1 year of preparatory work that included face to face meetings by investigators from both countries, agreement on research objectives, discussion of feasibility and sampling design, selection of research instruments, obtaining permission from U.S. test publishers to translate and adapt Western tests for use in the research, then translation and back-translation of the selected English language instruments, revision and modification of culturally inappropriate items, and the training of examiners in China. This research study was approved by the Institutional Review Boards from China Center for Disease Control (CDC) and National Center for AIDS (NCAIDS), and UCSD. Written informed consent was obtained from all participants after the research procedure had been fully explained to them.

Participants

The Chinese participants were HIV+ men and women 18 to 50 years of age, and demographically comparable HIV− controls. Individuals with a history of non–HIV-related confounding neuromedical factors that might potentially cause impairment of neurocognitive function were excluded from the study. These exclusion criteria included head injury with unconsciousness greater than 30 min, non–HIV-related neurological disorders (e.g., epilepsy, stroke), psychotic disorders (schizophrenia and bipolar disorder), and potentially significant levels of current substance use, defined as more than two alcoholic drinks per day over the past 30 days, or use of any illegal drugs three times or more per week in the past 30 days. One HIV+ participant was excluded due to illiteracy in Mandarin (which was not his first language), and one HIV− participant was excluded due to visual disability.

To sample from both urban and rural populations in China, this study was conducted in two different areas: the city of Beijing and rural Anhui province. Participants in Beijing were enrolled from the HIV clinic at You An Hospital, and those in Anhui were recruited from a local HIV screening station sponsored by the China Comprehensive AIDS Response (CARES) program. Recruitment for this study was conducted via word-of-mouth, posted announcements, and flyers distributed by the local CDC clinics, and nurse recruiters at HIV clinical sites. All participants signed the study consent before they were formally enrolled in this study. The Beijing site was picked because two investigators (Drs. Wu and Yu) were physically located in Beijing and able to provide immediate supervision during this feasibility study. After the initial group of 21 HIV+ and 16 HIV− participants were recruited and tested in Beijing, and this experience was deemed to be satisfactory, the protocol was field-tested with 7 more HIV+ and 7 more HIV− participants in the rural Anhui province because Anhui province is the chosen site for a larger follow-up investigation. Altogether, 28 HIV-infected participants and 23 HIV− controls completed the feasibility study. All HIV− participants received the HIV Quick Test (OraSure Technologies, Inc., Bethlehem, PA) to confirm their HIV status before being enrolled.

To explore the generalizability of HIV effects on Western test instruments across cultures, U.S. HIV+ and HIV− participants also were included. These participants were selected from larger samples from the HNRC to be as comparable as possible to the China samples with respect to demographic characteristics, stage of HIV disease, treatment status, and absence of non–HIV-related risk factors for neurocognitive impairment (see previously described criteria of exclusion).

The demographic and clinical characteristics of all groups are presented in Table 1. Chinese and U.S. groups were comparable for age and gender. Despite our effort to find education-comparable U.S. samples, education levels in China tend to be lower and we were unable to match across countries on this variable. However, there were no differences between HIV+ and HIV− groups within either country.

Table 1.

Demographic and clinical characteristics of HIV+ and HIV− groups in China and in the United States

China
U.S.
HIV− (n = 23) HIV+ (n = 28) HIV− (n = 31) HIV + (n = 39) p value*
Age (years) 35.3 (6.7) 35.6 (6.7) 34.2 (8.45) 34.9 (5.0) ns
Education (years) 10.4 (3.6) 10.6 (3.3) 12.3 (1.46) 12.0 (2.1) .03
Gender male/female 17/6 21/7 22/9 33/6 ns
Current CD4 count 262.0 (219.5) 131.5 (169.6) .005
% AIDS 79 79 ns
Duration of known seropositivity (months) 28.2 (25.6) 99.4 (60.8) .0001
BDI total 6.6 (7.8) 22.4 (12.7) 5.1 (7.1) 20.57 (18.4) .0001
BDI (somatic items excluded) 4.1 (5.3) 14.6 (8.6) 1.9 (2.9) 10.6 (14.9) .0001
PAOFI total 1.09 (2.09) 7.6 (8.2) 3.1 (5.2) 9.3 (7.9) .0001
Study site Beijing (%) 70 75 ns
Employment status: Currently employed (%) 82.6 50 .02
On ARV drugs (%) 75 67 ns

Note. Data are presented as Mean (SD) unless designated otherwise. Chinese HIV+ and HIV− groups had significantly lower education levels (p< .05) than did the U.S. HIV+ and HIV− groups. The Chinese HIV+ group obtained significantly higher BDI scores (p< .0001) compared with both Chinese and U.S. HIV− groups, but not compared with the U.S. HIV+ group. The U.S. HIV+ group obtained significantly higher BDI scores (p< .0001) compared to both the U.S. and Chinese HIV− groups, but not compared to the Chinese HIV+ group. The Chinese HIV+ group obtained significantly higher PAOFI scores (p< .0001) compared with both Chinese and U.S. HIV− groups, but not compared with the U.S. HIV+ group. The U.S. HIV+ group obtained significantly higher PAOFI scores (p< .0001) compared with both U.S. and Chinese HIV− groups, but not compared with the Chinese HIV+ group. ns = nonsignificant; BDI = Beck Depression Inventory; PAOFI = Patient’s Assessment of Own Functioning Inventory, an instrumental of activity of daily living; ARV = antiretroviral.

*

The p value is of one-way analysis of variance across all groups.

Most HIV+ individuals in both countries had advanced disease and met U.S. CDC criteria for AIDS (CDC, 1993). Also, most HIV+ participants were being prescribed anti-retroviral treatment, although the U.S. sample was more immunosuppressed (lower CD4 count, see Table 1). However, in both the Chinese and U.S. samples, current CD4 cell counts were not correlated with a summary measure of NP performance (The Global Deficit Score, GDS defined below). Lastly, results on the Beck Depression Inventory (BDI; Beck, 1976) and the Patient’s Assessment of Own Functioning Inventory (PAOFI, Chelune et al., 1986) indicate that both HIV+ groups, relative to their respective HIV− controls, had comparably increased depressive symptoms and self-reported cognitive difficulties in their everyday lives (Table 1).

Procedure

In addition to providing demographic information, every participant underwent a comprehensive neurocognitive evaluation, structured psychiatric examination, as well as assessment of daily functioning. In HIV+ participants, HIV staging and recent CD4 counts were also recorded.

Neuropsychological Evaluation

Test selection and adaptation

Together, the U.S. and Chinese investigators evaluated the entire HNRC neurobehavioral protocol to gauge each test’s appropriateness for use in China. Some of the tests (or very similar instruments) had been used in China for some time and were known to be acceptable. Most of the other instruments in the HNRC battery also were judged to be acceptable with few or no modifications (other than translating the tests into Mandarin). It was anticipated that some participants would have little or no education; only one of the tests requires participants to read (and just a few simple words). A few tests require some simple counting or addition of single-digit numbers; the Chinese investigators thought all participants could do this, because even those with minimal education perform simple math and counting in their daily lives. Once the instruments were agreed upon, we contacted the publishing companies and obtained permission to translate these instruments and instructions into Mandarin. The Chinese investigators translated these instruments from English to Mandarin. Back-translation was performed independently by a professional translator with no prior knowledge of these instruments.

The test battery selected for the current research taps multiple cognitive–motor ability domains that repeatedly have been found to be affected by HIV-associated brain disease in the United States (listed in Table 2). The battery was carefully reviewed by Chinese mental health professionals, who considered the U.S. tests to be culturally appropriate for the study populations in China. With the permission of the test publishers, minor adaptations of certain items were made to improve their familiarity/understandability by Chinese subjects (e.g., a few items from the word list of the Hopkins Verbal Learning Test-Revised). Because Mandarin writing uses characters instead of letters, in the Chinese battery, we substituted three tests of the same ability domain that do not use English letters: WMS-III Spatial Span for WAIS-III Letter-Number Sequencing; Color Trails II for Trails B; and Action Fluency for Letter Fluency (see test references in Table 2).

Table 2.

Neuropsychological test battery

Training and data quality assurance

To implement the feasibility study, the HNRC team traveled to Beijing in January 2004 to train five Mandarin bilingual examiners from the Institute of Mental Health at Peking University. Before this training session, training videotapes, test manuals, and test equipment and forms were sent to Beijing for review and practice. During the 1-week training session in Beijing, each test was demonstrated, and its purpose and administration nuances were discussed. Several rounds of “mock testing” were conducted. Certification sessions subsequently took place using staff volunteers from the hospital as test subjects. All certifications were done in Mandarin and were monitored by bilingual investigators from both countries. All five trainees were certified during the training week. The examiners continued to practice test administration before initiation of the feasibility study, and these test protocols were checked and double-scored by the U.S. team.

Assessment of Mood and Subjective Complaints

One of the purposes of this study was to obtain preliminary information on the relationships among mood disturbances (see detailed protocol in Jin et al., 2006), subjective complaints of cognitive difficulties in everyday life, and objective test evidence of neurocognitive impairment. Mood symptoms were assessed using the Mandarin version of the BDI-I, a 21-item self-report scale (Beck, 1976; Ping, 1993; Zheng, 1987). As in the English version, each item of BDI has four response options (0 to 3) of graded severity. Subjective neurocognitive complaints were assessed using the PAOFI, which was translated into Mandarin and back-translated to check accuracy (Chelune et al., 1986). The PAOFI includes 33 items on which participants rate themselves as having neurobehavioral difficulties in their everyday lives, using a six-point scale: almost never, very infrequently, once in a while, fairly often, very often, and almost always, in domains of memory, language and communication, sensory–perceptual and motor skills, and higher level cognitive functions. The score used is the sum of items on which the participants reported experiencing difficulties as either “fairly often,” “very often,” or “almost always” (Chelune et al., 1986). Employment status was derived from question 35 of the PAOFI questionnaire.

Quality Control of Data

To ensure data quality, we conducted weekly phone conferences immediately after the field study started, to discuss and answer any questions generated from the field testing. Daily e-mail communications also were used to answer and solve any testing-related questions. Completed test booklets were copied and sent by overnight mail to the U.S. weekly for review by bilingual HNRC personnel. If any illogical answers or unclear responses were found, those cases were reviewed with testers in China during the weekly phone conference. Further verification and corrections were made before the data were coded and entered into the HNRC database in the United States.

Data Analysis

To create domain and overall neuropsychological summary measures, scores on the individual tests were placed on a common metric. For this, we chose to use demographically uncorrected scaled scores, which in large U.S. normative samples have a mean of 10 and a standard deviation of 3 (e.g., Heaton et al., 2004b). Although the scaled scores on tests from the Wechsler Intelligence and Memory scales normally are age-corrected, we avoided this by using the mean age for the study group (i.e., 35) instead of the participants’ actual ages. Average scaled scores for the tests within each ability domain and across the entire test battery were then analyzed using two-way analyses of variance (ANOVAs), to explore effects of HIV status, country, and possible interactions between these factors. Of most interest within these analyses are the effects of HIV status, and the Country-by-HIV interactions. If the latter interactions were found to be significant, this finding would suggest that any NP effects of HIV were different in the two countries. Given the exploratory nature of these analyses, the alpha level was designated at 5%, two-tailed.

RESULTS

Table 3 summarizes the results of all four groups on the individual NP test raw scores.

Table 3.

Raw scores (mean and SD) on individual NP test measures for Chinese and U.S. HIV− and HIV+ groups

China
U.S.
HIV− (n = 23)
Mean (SD)
HIV+ (n = 28)
Mean (SD)
HIV− (n = 31)
Mean (SD)
HIV+ (n = 39)
Mean (SD)
Action fluency 14.04 (3.68) 11.75 (4.64)
Letter fluency 44.00 (10.49) 36.66 (12.31)
PASAT-50 40.21 (7.54) 37.07 (11.30) 38.61 (8.81) 34.75 (9.71)
WMS-III Spatial Span 16.30 (3.25) 14.92 (2.90)
WAIS-III L-N Sequencing - - 11.35 (2.73) 10.30 (2.53)
WAIS-III Digit Symbol 69.04 (15.89) 60.92 (20.89) 74.90 (15.41) 61.58 (17.31)
WAIS-III Symbol Search 28.21 (10.06) 26.46 (11.56) 36.10 (7.36) 28.58 (9.84)
Trail Making Test A 33.43 (10.14) 39.78 (12.34) 22.80 (5.47) 34.28 (24.74)
WCST Preservative errors 11.39 (7.05) 13.42 (7.76) 8.74 (5.85) 14.20 (13.63)
Trail Making Test B 60.26 (18.28) 81.81 (41.29)
Color Trails II 72.52 (18.92) 88.00 (26.86)
HVLT-R Learning 25.39 (4.07) 22.14 (6.37) 28.09 (4.44) 25.53 (5.01)
BVMT-R Learning 27.04 (5.62) 23.35 (7.54) 25.70 (5.84) 22.20 (8.08)
HVLT-R Delayed Recall 9.47 (1.80) 8.60 (2.75) 9.83 (2.39) 8.46 (3.05)
BVMT-R Delayed Recall 10.56 (1.47) 9.78 (2.60) 9.90 (1.71) 8.30 (3.03)
Grooved Pegboard DH 65.69 (9.51) 68.85 (10.91) 64.13 (7.42) 80.33 (27.14)
Grooved Pegboard NDH 72.82 (11.51) 73.96 (12.62) 72.33 (9.58) 87.66 (31.38)

Note. PASAT = Paced Auditory Serial Addition Task; WMS-III = Wechsler Memory Scale-Third Edition; WAIS-III = Wechsler Adult Intelligence Scale-Third Edition; WCST = Wisconsin Card Sorting Test; HVLT-R = Hopkins Verbal Learning Test-Revised; BVMT-R = Brief Visuospatial Memory Test-Revised; DH = dominant hand; NDH = nondominant hand.

HIV Effects in the China Groups

On every individual NP test measure, the mean of the Chinese HIV+ group was worse than that of the Chinese HIV− group, regardless of whether raw scores or scaled scores were used. Medium HIV effect sizes (range, .42 to .65) were noted for both raw scores and scaled scores on 7 of the 14 individual test measures (Verbal fluency for Action, WAIS-III Digit Symbol, Trails A, WMS-III Spatial Span, Color Trails-II, Hopkins Verbal Learning Test-Revised, and Hopkins Verbal Learning Test-Revised Total Learning; see Figure 1). The Global mean scaled score also yielded a medium HIV effect size (0.55), and HIV effect sizes for domain scaled scores ranged from 0.22 to 0.60. Moreover, across all 14 measures, the effect sizes for raw scores and scaled scores were highly correlated (r = .97; p < .0001; Figure 1).

Fig. 1.

Fig. 1

Effect sizes between HIV+ and HIV− Chinese individuals computed from scaled scores and raw scores. ESs = effect sizes; WCST Persev = Wisconsin Card Sorting Test Perseverative Errors; Digit Symbol = WAIS-III Digit Symbols; Sym Search = WAIS-III Symbol Search; PASAT = Paced Auditory Serial Addition Task; BVMT-R = Brief Visuospatial Memory Test-Revised; HVLT-R = Hopkins Verbal Learning Test-Revised; Learn = Learning; Recall 5 Recognition; Dom = dominant; NonDom = non-dominant.

Effects of HIV and Country on NP Mean Scaled Scores

Table 4 summarizes the Global and Domain mean scaled scores of the four groups, as well as the results of the 2 × 2 (HIV status by country) ANOVAs on these measures. The ANOVAs revealed robust HIV effects for the Global score and all Domain scores. Significant country effects were found for two ability domains: Verbal Fluency and Speed of Information Processing. The U.S. group obtained higher scores on these domains, but there were no significant country effects on the global NP summary score or on the remaining five NP Domain scores. Finally, there were no significant HIV-by-Country interaction effects in any of these ANOVAs.

Table 4.

Scaled mean scores for Chinese and U.S. HIV− and HIV+ groups

China
U.S.
HIV− (n = 23) HIV+ (n = 28) HIV− (n = 31) HIV+ (n = 39) Group and interaction effects
Mean (SD) Mean (SD) Mean (SD) Mean (SD) HIV Country Interaction
Executive Function 10.52 (2.31) 9.21 (2.27) 10.71 (2.10) 9.13 (2.69) .02 ns ns
Verbal Fluency 7.56 (2.01) 6.25 (2.47) 11.16 (2.52) 9.30 (3.05) .002 <.0001a ns
Learning 9.13 (2.43) 7.55 (2.70) 9.66 (2.55) 8.25 (2.73) .003 ns ns
Memory 9.69 (2.29) 8.76 (2.83) 9.46 (2.51) 7.75 (2.82) .009 ns ns
Motor 9.85 (2.16) 9.39 (2.03) 10.03 (1.90) 7.98 (2.86) .005 ns ns
SIP 8.51 (2.14) 7.70 (2.82) 10.76 (2.16) 8.45 (2.52) .0009 .002a ns
Attention/WM 10.39 (2.09) 9.39 (2.28) 10.09 (2.65) 9.00 (2.64) .03 ns ns
Global (all tests) 9.45 (1.66) 8.42 (2.01) 10.23 (1.62) 8.39 (2.27) .0001 ns ns
Global (partial)b 9.29 (1.96) 8.38 (2.13) 10.03 (1.74) 8.20 (2.30) .0004 ns ns

Note. ns = nonsignificant; SIP = Speed of Information Processing; WM = Working Memory.

a

U.S. individuals obtained better performance compared to Chinese individuals.

b

Excluding tests that differ across countries (Trail Making Test B vs. Color Trail II; Action Fluency vs. Letter Fluency; Letter-Number Sequencing vs. Spatial Span).

To explore the effect of slightly different NP test measures between the U.S. groups and the Chinese (See Table 2 and Table 3 for details), we re-computed the Global mean scaled scores deleting these tests (see prorated Global score in Table 4). We found similar results on the 2 × 2 ANOVA: a robust HIV effect, and no significant country effect or Country-by-HIV interaction.

Self-Report of Depressed Mood and Functional Difficulties

Our Chinese HIV+ group demonstrated considerably more depressed mood than their HIV− controls, but not compared with their HIV+ U.S. counterpart (see Table 1). On the PAOFI, similar findings were observed: there were significantly more cognitive complaints in both the HIV+ samples compared with both the HIV− samples, while the Chinese and U.S. HIV+ groups did not differ (see Table 1).

NP performances (Global mean Scaled Score) were relatively independent of depressed mood (i.e., BDI) in both HIV+ samples (r = −.19; p = .32 for Chinese HIV+, and r = −.22; p = .20 for U.S. HIV+). On the other hand, cognitive complaints on the PAOFI were very strongly associated with depressive symptoms as reported on the BDI, in both HIV+ samples (r =.55; p < .003, for Chinese HIV+; and r = .70; p < .0001 for U.S. HIV+). Cognitive complaints (PAOFI) were significantly related to NP performance (Global mean scaled score) in the China group (r = −.47; p = .02; for Chinese HIV+) and showed a trend in the U.S. group (r = −.30; p = .09; for U.S. HIV+). Lastly, employment status in the Chinese HIV+ group showed a modest correlation with NP performance (r = .35; p = .06, between employment status and Global mean scaled score).

DISCUSSION

The above results, while admittedly tentative, are encouraging in several respects. First, the NP test battery that was chosen and adapted for use in this study was understood and accepted by the Chinese HIV+ and HIV− participants, from both urban and rural areas of the country, whose education levels ranged widely (4 to 18 years). Second, the battery, which was translated into Mandarin, appears to hold considerable promise for identifying and characterizing behavioral effects of HIV brain involvement in China. Finally, the consistent HIV effects across countries and the modest country effects in the current study (Table 4) suggest that these tests may have reasonable cross-cultural equivalence. Further psychometric analyses with much larger samples of neurologically normal and abnormal people are needed to confirm this.

The NP test batteries between the two countries were identical except for three NP tests that were substituted for the Chinese individuals to increase the cross-cultural suitability (Spatial Span for Letter Number-Sequencing; Color-Trails II for Trail Making Test B; and Action Fluency for Letter Fluency). The first test substitution was chosen to respect findings of factor analyses of the WMS showing that Spatial Span and Letter Number-Sequencing grouped with the same factor (WMS-III, 1997). In addition, Color-Trails II was designed to tap similar sequencing and processing requirements as Trail Making B (Maj et al., 1993). Finally, Action and Letter Fluency are both tests of word generation that tap into frontal neural systems (Woods et al., 2005). Despite these similarities, slight differences in tests may have influenced the HIV effects observed in the China and U.S. cohorts. To check for this possibility, we re-computed the overall mean scaled scores without these tests and found identical results (Table 4).

Country effects on two NP Domain scores are difficult to interpret for two reasons: the significant education differences between the U.S. and China groups probably influenced there results to some degree, and different tests were used for the Verbal Fluency domain in the two countries (Letter Fluency for U.S. and Action Fluency for China). Nevertheless, the most important findings in these ANO-VAs were the consistently significant HIV effects for the total test battery and all seven ability domains, and the absence of any significant interaction effects. These results suggest comparable, significant effects of HIV on NP functioning in both countries.

Closer inspection of the pattern of HIV-related deficits reveals that NP impairment was dominated by deficits in abstraction/executive function, information processing speed, as well as learning in Chinese HIV+ individuals (Figure 1). This pattern of deficits is generally consistent with what has been shown in Western studies (Cysique et al., 2006; Grant & Martin, 1994; Reger et al., 2002).

High rates of depressed mood in HIV-infected persons were not significantly associated with NP performance in the United States or China. These preliminary results are consistent with a large body of literature showing that depressive symptoms do not account for neurocognitive impairment in HIV-infected individuals (Cysique et al., 2007; Goggin et al., 1997). In contrast, depressive complaints were strongly associated with self-report of neurobehavioral problems in both countries. This finding also is consistent with what has been reported in North American HIV cohorts (Carter et al., 2003; Rourke et al., 1999).

Consistent with U.S. findings on the impact of HIV-associated neurocognitive impairment on instrumental activities of daily living (Heaton et al., 2004a) and employment (Albert et al., 1995; Heaton et al., 1994), we found, in the Chinese HIV+ group, a correlation between NP functioning and complaints of cognitive difficulties in everyday life (i.e., PAOFI), and also a modest association between unemployment status and overall level of NP performance. However, given the major cultural and lifestyle differences in these two countries, much more research is needed to understand the functional consequences of general and specific neurocognitive deficits in China. In particular, because NP impairment has been shown to predict poor medication adherence among HIV+ individuals in the United States (Hinkin et al., 2004), it would be important to determine whether NP-impaired HIV+ people in China are at similar risk for poor medication management and possibly worse antiretroviral treatment outcome (d’Arminio Monforte et al., 1998).

A major limitation of this pilot study is the absence of established normative standards for Chinese people on the NP test battery. Although NP norms with appropriate demographic corrections were available for the U.S. participants, it is considered likely that demographic effects are different in China. For example, very low levels of education and illiteracy are not represented in the U.S. normative samples, but such backgrounds still are relatively common in China (especially in the rural areas). As a consequence, larger education effects might be expected within our China study groups. Unfortunately, our sample sizes were much too small to address this problem in the current study. Future large scale studies of healthy Chinese with diverse backgrounds will be needed to develop norms that can be used confidently to classify disease-related “impairments” in individual cases. However, the robust HIV effect sizes in our demographically matched Chinese samples (Figure 1) strongly suggest that these tests have potential for identifying individuals within the Chinese population who are suffering from CNS complications of HIV. Thus, our results would support continued work to better understand the nature and causes of NP performance differences in the normal Chinese population, as well as in people affected by HIV infection and other diseases of the CNS.

ACKNOWLEDGMENTS

This study was supported by the NIMH grant 5 P30 MH62512 (Dr. Grant, P.I.). The HIV Neurobehavioral Research Center (HNRC) is supported by Center award MH 62512 from NIMH. * The San Diego HIV Neurobehavioral Research Center (HNRC) group is affiliated with the University of California, San Diego, the Naval Hospital, San Diego, and the Veterans Affairs San Diego Healthcare System, and includes: Director: Igor Grant, M.D.; Co-Directors: J. Hampton Atkinson, M.D., Ronald J. Ellis, M.D., Ph.D., and J. Allen McCutchan, M.D.; Center Manager: Thomas D. Marcotte, Ph.D.; Naval Hospital, San Diego: Braden R. Hale, M.D., M.P.H. (P.I.); Neuromedical Component: Ronald J. Ellis, M.D., Ph.D. (P.I.), J. Allen McCutchan, M.D., Scott Letendre, M.D., Edmund Capparelli, Pharm.D., Rachel Schrier, Ph.D.; Neurobehavioral Component: Robert K. Heaton, Ph.D. (P.I.), Mariana Cherner, Ph.D., Steven Paul Woods, Psy.D.; Neuroimaging Component: Terry Jernigan, Ph.D. (P.I.), Christine Fennema-Notestine, Ph.D., Sarah L., Archibald, M.A., John Hesselink, M.D., Jacopo Annese, Ph.D., Michael J. Taylor, Ph.D., Brian Schweinsburg, Ph.D.; Neurobiology Component: Eliezer Masliah, M.D. (P.I.), Ian Everall, FRCPsych., FRCPath., Ph.D., T. Dianne Langford, Ph.D.; Neurovirology Component: Douglas Richman, M.D., (P.I.), David M. Smith, M.D.; International Component: J. Allen McCutchan, M.D., (P.I.); Developmental Component: Ian Everall, FRCPsych., FRCPath., Ph.D. (P.I.), Stuart Lipton, M.D., Ph.D.; Clinical Trials Component: J. Allen McCutchan, M.D., J. Hampton Atkinson, M.D., Ronald J. Ellis, M.D., Ph.D., Scott Letendre, M.D.; Participant Accrual and Retention Unit: J. Hampton Atkinson, M.D. (P.I.), Rodney von Jaeger, M.P.H.; Data Management Unit: Anthony C. Gamst, Ph.D. (P.I.), Clint Cushman, B.A., (Data Systems Manager), Michelle Frybarger, B.A., Daniel R. Masys, M.D. (Senior Consultant); Statistics Unit: Ian Abramson, Ph.D. (P.I.), Christopher Ake, Ph.D., Deborah Lazzaretto, M.S. The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of the Navy, Department of Defense, nor the United States Government.

REFERENCES

  1. Albert SM, Marder K, Dooneief G, Bell K, Sano M, Todak G, Stern Y. Neuropsychologic impairment in early HIV infection. A risk factor for work disability. Archives of Neurology. 1995;52:525–530. doi: 10.1001/archneur.1995.00540290115027. [DOI] [PubMed] [Google Scholar]
  2. Beck A. Cognitive therapy and the emotional disorders. New York: Hoeber; 1976. [Google Scholar]
  3. Benedict RH. Brief Visuospatial Memory Test-Revised. Odessa, FL: Psychological Assessment Resources, Inc; 1997. [Google Scholar]
  4. Benton A, Hamsher K, Sivan A. Multilingual aphasia examination. Iowa City, IA: AJA Associates; 1994. [Google Scholar]
  5. Brandt J, Benedict RH. Hopkins Verbal Learning Test-Revised. Professional manual. Lutz, FL: Psychological Assessment Resources, Inc; 2001. [Google Scholar]
  6. Carter SL, Rourke SB, Murji S, Shore D, Rourke BP. Cognitive complaints, depression, medical symptoms, and their association with neuropsychological functioning in HIV infection: A structural equation model analysis. Neuropsychology. 2003;17:410–419. doi: 10.1037/0894-4105.17.3.410. [DOI] [PubMed] [Google Scholar]
  7. Chan AS, Shum D, Cheung RW. Recent development of cognitive and neuropsychological assessment in Asian countries. Psychological Assessment. 2003;15:257–267. doi: 10.1037/1040-3590.15.3.257. [DOI] [PubMed] [Google Scholar]
  8. Chelune G, Heaton R, Lehman R. Neuropsychological and personality correlates of patient’s complaints of disability. In: Tarter R, Goldstein G, editors. Advances in clinical neuropsychology. New York: New York Plenum Press; 1986. pp. 95–126. [Google Scholar]
  9. Cherner M, Masliah E, Ellis RJ, Marcotte TD, Moore DJ, Grant I, Heaton RK. Neurocognitive dysfunction predicts postmortem findings of HIV encephalitis. Neurology. 2002;59:1563–1567. doi: 10.1212/01.wnl.0000034175.11956.79. [DOI] [PubMed] [Google Scholar]
  10. China Ministry of Health, & the United Nations Theme Group on HIV/AIDS in China. A joint assessment of HIV/AIDS prevention, treatment, and care in China. Beijing. 2003 December 1st; [Google Scholar]
  11. Chinese Ministry of Health, UNAIDS, & WHO. Progress on implementing UNGASS declaration of commitment in China 2005. 2005 December; Available at: http://data.Unaids.Org/pub/report/2006/2006_country_progress_report_china_en.Pdf.
  12. Cysique LAJ, Deutsch R, Atkinson HJ, Young C, Marcotte TD, Dawson L, Grant I, Heaton RK. Incident major depression does not affect neuropsychological functioning in HIV infected men. Journal of the International Neuropsychological Society. 2007;13:1–11. doi: 10.1017/S1355617707070026. [DOI] [PubMed] [Google Scholar]
  13. Cysique LAJ, Maruff P, Brew BJ. The neuropsychological profile of symptomatic AIDS and ADC patients in the pre–highly active antiretroviral therapy era: A meta-analysis. Journal of the International Neuropsychological Society. 2006;12:368–382. doi: 10.1017/s1355617706060401. [DOI] [PubMed] [Google Scholar]
  14. d’Arminio Monforte A, Testa L, Adorni F, Chiesa E, Bini T, Moscatelli GC, Abeli C, Rusconi S, Sollima S, Balotta C, Musicco M, Galli M, Moroni M. Clinical outcome and predictive factors of failure of highly active antiretroviral therapy in antiretroviral-experienced patients in advanced stages of HIV-1 infection. AIDS. 1998;12:1631–1637. doi: 10.1097/00002030-199813000-00010. [DOI] [PubMed] [Google Scholar]
  15. Diehr MC, Heaton RK, Miller W. The paced auditory serial addition task (PASAT): Norms for age, education, and ethnicity. Assessment. 1998;5:375–387. doi: 10.1177/107319119800500407. [DOI] [PubMed] [Google Scholar]
  16. Ellis RJ, Deutsch R, Heaton RK, Marcotte TD, McCutchan JA, Nelson JA, Abramson I, Thal LJ, Atkinson JH, Wallace MR, Grant I. Neurocognitive impairment is an independent risk factor for death in HIV infection. San Diego HIV Neurobehavioral Research Center group. Archives of Neurology. 1997;54:416–424. doi: 10.1001/archneur.1997.00550160054016. [DOI] [PubMed] [Google Scholar]
  17. Everall IP, Heaton RK, Marcotte TD, Ellis RJ, McCutchan JA, Atkinson JH, Grant I, Mallory M, Masliah E. Cortical synaptic density is reduced in mild to moderate human immunodeficiency virus neurocognitive disorder. HNRC group. HIV Neurobehavioral Research Center. Brain Pathology. 1999;9:209–217. doi: 10.1111/j.1750-3639.1999.tb00219.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Gladsjo JA, Schuman CC, Evans JD. Norms for letter and category fluency: Demographic corrections for age, education, and ethnicity. Assessment. 1999;6:147–178. doi: 10.1177/107319119900600204. [DOI] [PubMed] [Google Scholar]
  19. Goggin KJ, Zisook S, Heaton RK, Atkinson HJ, Marshall S, McCuthchan JA, Chandler JL, Grant I. Neuropsychological performance of HIV-1 infected men with major depression. Journal of the International Neuropsychological Society. 1997;3:457–464. [PubMed] [Google Scholar]
  20. Grant I, Atkinson HJ, Hesselink JR, Kennedy CJ, Richman DD, Spector SA, McCuthchan JA. Evidence for early central nervous system involvement in the acquired immunodefieciency syndrome (AIDS) and other human immunodeficiency virus (HIV) infections. Studies with neuropsychological testing and magnetic resonance imaging. Annals of Internal Medicine. 1987;107:828–836. doi: 10.7326/0003-4819-107-6-828. [DOI] [PubMed] [Google Scholar]
  21. Grant I, Martin A, editors. Neuropsychology of HIV infection. New York: Oxford University Press; 1994. [Google Scholar]
  22. Gronwall DM. Paced auditory serial-addition task: A measure of recovery from concussion. Perceptual and Motor Skills. 1977;44:367–373. doi: 10.2466/pms.1977.44.2.367. [DOI] [PubMed] [Google Scholar]
  23. Heaton R, Grant I, Butters N, White DA, Kirson D, Atkinson HJ, McCuthchan JA, Taylor M, Kelly MD, Ellis RJ, Wolfson T, Velin RA, Marcotte TD, Hesselink JR, Jernigan TL, Chandler J, Wallace M, Abramason I. The HNRC 500-neuropsychology of HIV infection at different disease stages. Journal of the International Neuropsychological Society. 1995;1:231–251. doi: 10.1017/s1355617700000230. [DOI] [PubMed] [Google Scholar]
  24. Heaton RK, Grant I, Matthews CG. Comprehensive norms for an expanded Halstead-Reitan battery: Demographic corrections research findings, and clinical applications. Odessa, FL: Psychological Assessment Resources; 1991. [Google Scholar]
  25. Heaton R, Marcotte TD, Rivera Mindt M, Sadek J, Moore D, Bentley H, McCutchan AJ, Reicks C, Grant I. The impact of HIV associated neuropsychological impairment on everyday functioning. Journal of the International Neuropsychological Society. 2004a;10:317–331. doi: 10.1017/S1355617704102130. [DOI] [PubMed] [Google Scholar]
  26. Heaton R, Miller W, Taylor MJ, Grant I. Revised comprehensive norms for an expanded Halstead Reitan Battery: Demographically adjusted neuropsychological norms for African American and Caucasian adults. Lutz, FL: Psychological Assessment Resources; 2004b. [Google Scholar]
  27. Heaton R, Taylor M, Manly J. Demographic effects and use of demographically corrected norms with the WAI-III and WMS-III. In: Tulsky D, Saklofske D, Heaton RK, editors. Clinical interpretation of the WAI-III and WMS-III. San Diego, CA: Academic Press; 2003. [Google Scholar]
  28. Heaton RK, Velin RA, McCutchan JA, Gulevich SJ, Atkinson JH, Wallace MR, Godfrey HP, Kirson DA, Grant I. Neuropsychological impairment in human immunodeficiency virus-infection: Implications for employment. HNRC group. HIV Neurobehavioral Research Center. Psychosomatic Medicine. 1994;56:8–17. doi: 10.1097/00006842-199401000-00001. [DOI] [PubMed] [Google Scholar]
  29. Hinkin CH, Hardy DJ, Mason KI, Castellon SA, Durvasula RS, Lam MN, Stefaniak M. Medication adherence in HIV infected adults: Effect of patient age, cognitive status and substance. AIDS. 2004;18 Suppl. 1:S19–S25. doi: 10.1097/00002030-200418001-00004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Jernigan TL, Archibald S, Hesselink JR, Atkinson JH, Velin RA, McCutchan JA, Chandler JL, Grant I. Magnetic resonance imaging morphometric analysis of cerebral volume loss in human immunodeficiency virus infection. The HNRC group. Archives of Neurology. 1993;50:250–255. doi: 10.1001/archneur.1993.00540030016007. [DOI] [PubMed] [Google Scholar]
  31. Jin H, Atkinson HJ, Yu X, Heaton RK, Shi C, Marcotte TP, Young C, Sadek J, Wu Z, Grant I. Depression and suicidality in HIV/AIDS in China. Journal of Affective Disorders. 2006;94:269–275. doi: 10.1016/j.jad.2006.04.013. [DOI] [PubMed] [Google Scholar]
  32. Klove H. Clinical neuropsychology. In: Forster FM, editor. The medical clinics of North America. New York: Saunders; 1963. [PubMed] [Google Scholar]
  33. Kongs SK, Thompson LL, Iverson GL, Heaton RK. WCST-64: Wisconsin Card Sorting Test-64 card version professional manual. Odessa, FL: Psychological Assessment Resources, Inc; 2000. [Google Scholar]
  34. Lezak M, Howieson D, Loring D, Hannay J, Fischer J. Neuropsychological assessment. 4th ed. Oxford: Oxford University Press; 2004. [Google Scholar]
  35. Maj M, D’Elia L, Satz P, Janssen R, Zaudig M, Uchiyama C, Starace F, Galderisi S, Chervinsky A. Evaluation of two new neuropsychological tests designed to minimize cultural bias in the assessment of HIV-1 seropositive persons: A WHO study. Archives of Clinical Neuropsychology. 1993;8:123–135. [PubMed] [Google Scholar]
  36. Maj M, Satz P, Janssen R, Zaudig R, Starace F, D’Elia L, Sughondhabirom B, Mussa M, Naber D, Ndetei D, Schulte G, Sartorius N. WHO neuropsychiatric AIDS study, cross-sectional phase II. Archives of General Psychiatry. 1994;51:51–61. doi: 10.1001/archpsyc.1994.03950010051007. [DOI] [PubMed] [Google Scholar]
  37. Masliah E, Achim CL, Ge N, DeTeresa R, Terry RD, Wiley CA. Spectrum of human immunodeficiency virus-associated neocortical damage. Annals of Neurology. 1992;32:321–329. doi: 10.1002/ana.410320304. [DOI] [PubMed] [Google Scholar]
  38. Masliah E, Heaton RK, Marcotte TD, Ellis RJ, Wiley CA, Mallory M, Achim CL, McCutchan AJ, Nelson JA, Atkinson HJ, Grant I. Dendritic injury is a pathological substrate for human immunodeficiency virus-related cognitive disorders. Annals of Neurology. 1997;42:963–972. doi: 10.1002/ana.410420618. [DOI] [PubMed] [Google Scholar]
  39. Mayeux R, Stern Y, Tang MX, Todak G, Marder K, Sano M, Richards M, Stein Z, Ehrhardt A, Gorman J. Mortality risks in gay men with human immunodeficiency virus infection and cognitive impairment. Neurology. 1993;43:176–182. doi: 10.1212/wnl.43.1_part_1.176. [DOI] [PubMed] [Google Scholar]
  40. Moore DJ, Masliah E, Rippeth J, Gonzalez R, Carey CL, Cherner M, Ellis RJ, Achim CL, Marcotte TD, Heaton RK, Grant I. Cortical and subcortical neurodegeneration is associated with HIV neurocognitive impairment. AIDS. 2006;20:879–887. doi: 10.1097/01.aids.0000218552.69834.00. [DOI] [PubMed] [Google Scholar]
  41. Navia BA, Cho ES, Petito CK, Price RW. The AIDS Dementia Complex: II neuropathology. Annals of Neurology. 1986;19:525–535. doi: 10.1002/ana.410190603. [DOI] [PubMed] [Google Scholar]
  42. Piatt AL, Fields JA, Paolo AM, Koller WC, Tröster AI. Lexical, semantic, and action verbal fluency in Parkinson’s disease with and without dementia. Journal of Clinical and Experimental Neuropsychology. 1999;21:435–443. doi: 10.1076/jcen.21.4.435.885. [DOI] [PubMed] [Google Scholar]
  43. Ping L. Chinese version of BDI. Supplement issue of Mental Health Rating Scale, Publisher of Chinese Journal of Psychology. 1993:191–194. [Google Scholar]
  44. Psychological Corporation. Wechsler Adult Intelligence Scale-Third Edition (WAIS-III) San Antonio, TX: Psychological Corporation; 1997a. [Google Scholar]
  45. Psychological Corporation. Wechsler Memory Scale-Third Edition Manual (WMS-III) San Antonio: The Psychological Corporation; 1997b. [Google Scholar]
  46. Reger M, Wesh R, Razani J, Martin DJ, Boone KB. A meta-analysis of the neuropsychological sequelae of HIV infection. Journal of the International Neuropsychological Society. 2002;8:410–424. doi: 10.1017/s1355617702813212. [DOI] [PubMed] [Google Scholar]
  47. Rourke SB, Halman MH, Bassel C. Neurocognitive complaints in HIV-infection and their relationship to depressive symptoms and neuropsychological functioning. Journal of Clinical and Experimental Neuropsychology. 1999;21:737–756. doi: 10.1076/jcen.21.6.737.863. [DOI] [PubMed] [Google Scholar]
  48. Stout JC, Ellis RJ, Jernigan TL, Archibald SL, Abramson I, Wolfson T, McCutchan JA, Wallace MR, Atkinson JH, Grant I. Progressive cerebral volume loss in human immunodeficiency virus infection: A longitudinal volumetric magnetic resonance imaging study. Archives of Neurology. 1998;55:161–168. doi: 10.1001/archneur.55.2.161. [DOI] [PubMed] [Google Scholar]
  49. U.S. War Department. Army individual test battery-manual of directions and scoring. Washington, DC: War Department, Adjutant General’s Office; 1944. [Google Scholar]
  50. White DA, Heaton R, Monsch AU. Neuropsychological studies of asymptomatic human immunodeficiency virus-type 1 infected individuals. Journal of the International Neuropsychological Society. 1995;1:304–315. doi: 10.1017/s1355617700000308. [DOI] [PubMed] [Google Scholar]
  51. Woods SP, Scott JC, Sires DA, Grant I, Heaton RK, Tröster AI. Action (verb) fluency: Test-retest reliability, normative standards, and construct validity. Journal of the International Neuropsychological Society. 2005;11:408–415. [PubMed] [Google Scholar]
  52. Yang Y, Zhang G, Xu Y. A seroepidemiological study of HCV and HVB infection among intravenous drug abusers. Zhonghua Shi Yan He Lin Chuang Bing Du Xue Za Zhi. 1999;30:247–250. [PubMed] [Google Scholar]
  53. Yepthomi T, Paul R, Vallabhaneni S, Kumarasamy N, Tate DF, Solomon S, Flanigan T. Neurocognitive consequences of HIV in southern India: A preliminary study of clade c virus. Journal of the International Neuropsychological Society. 2006;12:424–430. doi: 10.1017/s1355617706060516. [DOI] [PubMed] [Google Scholar]
  54. Zheng YP. Application of BDI in depressive neurosis. Chinese Journal of Neurology and Psychiatry. 1987;13:236–237. [Google Scholar]

RESOURCES