What is the optimal neuropsychological test battery for schizophrenia in China?

Abstract

Background:

The MATRICS consensus cognitive battery (MCCB) has been widely used to evaluate cognitive deficits in schizophrenia (SCZ), however, no study has formally examined the validity of the MCCB in Chinese SCZ. This study compared Chinese SCZ patients with healthy Chinese controls on the MCCB and some additional neurocognitive tests to determine if the Chinese MCCB is an optimal battery to assess the cognitive deficits in Chinese SCZ patients.

Method:

The study enrolled and examined 230 patients met DSM-IV criteria for SCZ and 656 healthy controls matched for gender, age and education. Besides the MCCB, we also included some additional neurocognitive tests that have been widely used in patients with schizophrenia. We selected MCCB and non-MCCB tests with large effect size, to assemble a new “optimal battery” and compared its performance with that of the standard MCCB.

Results:

Comparing the putative “optimal” battery with the original MCCB, more patients with SCZ were identified as cognitively impaired according to the criteria of GDS ≥ 0.50 for the optimal battery (166 vs 135, or 72.2% vs 58.7%). The rate of cognitive impairment according to MCCB GDS in patients with SCZ who were currently working, ever worked and never worked are 45.5%, 61.6% and 70.8% (p = 0.051), whereas the optimal battery GDS showed 56.4%, 74.8%, 91.7% (p = 0.003), respectively.

Conclusions:

Our study needs validation with independent samples but suggests that the current “optimal” cognitive battery could be more sensitive than the widely used MCCB in detecting SCZ related cognitive impairment in China.

1. Introduction

Cognitive impairment is a core symptom in schizophrenia, which should be precisely evaluated and considered clinically, due to its significant effects on functional outcome of this disorder (Shamsi et al., 2011; Talarowska et al., 2015). A growing number of cognitive tests have been were validated and applied in Asia, the mean number of published studies increasing from 1.8 per year in the 1980s to 16.0 in 2001 (Chan et al., 2003). A search by the keywords “China” and “Cognitive test” in PubMed as of 2017, identifies the total number of published studies from China as over 700. However, in the field of schizophrenia, tests like the Wisconsin Card Sorting Test (WCST), the Stroop task, N-back and verbal fluency (Liang et al., 2016; Yang et al., 2015; Yu et al., 2016), typically have been used in isolation in China, and do not constitute a comprehensive, sensitive assessment. Similarly, sometimes the researchers have used only limited subtests of Wechsler Adult Intelligence Scale (WAIS) to evaluate schizophrenia in China (Lin et al., 2012).

Zhang and colleagues translated and validated the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) to evaluate cognitive function in Chinese individuals with schizophrenia. However, they did not establish normative standards for China (Zhang et al., 2008). The Cambridge Neuropsychological Test Automated Battery (CANTAB) was not fully introduced into China: Only one of the CANTAB subtests, the Cambridge Face Memory Test (CFMT), was presented with Chinese norms. This test is for prosopagnosia (McKone et al., 2017), and even the full CANTAB mainly focuses narrowly on visual–spatial cognitive deficits in schizophrenia (Levaux et al., 2007).

The MATRICS consensus cognitive battery (MCCB) probably is the most widely used and standardized neuropsychological test battery for assessing cognition in schizophrenia worldwide. The MCCB was developed by Nuechterlein et al (Nuechterlein et al., 2008). These authors employed five criteria and judgements from an expert panel to select neuropsychological tests which could be combined as a standard neurocognitive assessment for cognitive impairment in schizophrenia. We translated and established norms for the MCCB with healthy adults in China from 2008 to 2014 (Shi et al., 2015). Compared with the MCCB performance patterns reported in the United States, we found an increased age effect on verbal learning in China. We also found that performance on the Emotional Intelligence Test (MSCEIT) was better in younger people than in older people in China, whereas the opposite has been found in United States. Regarding gender differences, we found women performed better on the measures of processing speed and social cognition, whereas women in the United States (but not in China) performed better on the MCCB verbal learning measure. These differences across countries suggest the importance of cultural and race effects that may be variable in international contexts. Another example is the Letter-Number Sequencing (LNS) test in MCCB, which is totally inappropriate in the Chinese setting due to the fact that there is no corresponding alphabet order in Chinese. Studies have found that this phenomenon of different international patterns of performance occurs not only in normal people, but also in patients with schizophrenia (Holmen et al., 2010; Lystad et al., 2014; Rodriguez-Jimenez et al., 2015). For example, Rodriguez-Jimenez et al. (2015) found that the most impaired cognitive domains in Spanish patients with schizophrenia were verbal learning and visual learning, whereas, in the United State, the most impaired cognitive domains were working memory and processing speed (Kern et al., 2011). In summary, we assume that the most suitable neuropsychological test battery for schizophrenia may be different in China and the United States.

In order to explore what is the optimal neuropsychological test battery for schizophrenia in China, we have considered the individual MCCB tests, but have included in our research some additional neuropsychological tests which also have been widely used in schizophrenia (Gulec et al., 2013; Heaton et al., 2004; Laurenson et al., 2015; Shan et al., 2008; Townsend et al., 2002; Woods et al., 2005). All of the tests in our battery have been co-normed in a large Chinese standardization sample, and here we consider their relative sensitivity in identifying cognitive impairment in adult Chinese patients with schizophrenia.

2. Methods

The study was conducted in six different sites in China by six institutions: Beijing (the capital of China, located in Huabei of China) by the Peking University Institution of Mental Health, (PUIMH), Changsha (Central south of China) by the Second Xiangya Hospital, Central South University; Shanghai (south east of China) by Shanghai Mental Health Center, Shanghai Jiao Tong University; Xian (north west of China) by Xian Mental Health Center; Harbin (North east of China) by the First Harbin Psychiatric Hospital; and Kunming (south west of China) by the First Affiliated Hospital of Kunming Medical University. The six sites were chosen to have multiple, widespread geographic regions of China represented, and the healthy control sample was stratified by age, gender and education approximate to the report of China Census Bureau in 2005 (China Census Bureau, 2006). The study was centrally approved by the IRB of PUIMH.

2.1. Recruitment procedures

a. Healthy controls

Details of the healthy control sample recruitment and stratification for the MCCB Chinese norm analysis have been reported elsewhere (Shi et al., 2015). In brief, the healthy control participants living in the community were contacted by the recruiter in each site. Potential subjects were identified by informant self-referral via flyers in hospital, and were enrolled after a face to face personal interview, in which the subjects were screened with the inclusion and exclusion criteria. The major exclusion criteria consisted of any history of schizophrenia or other major psychiatric disorders, developmental disorder, active substance abuse, clinically significant neurological disease or head injury with loss of consciousness, as well as inability to understand Mandarin sufficiently to comprehend testing instructions, and inability to comprehend the consent form appropriately. A total of 656 healthy volunteers were assessed using the MCCB for China and additional neuropsychological tests included in this study.

b. Patients with schizophrenia

The recruitment procedure was similar in all six sites. Outpatients with schizophrenia were introduced by their psychiatrists to the recruiter in each site, and were enrolled after a face to face personal interview, in which the patients were screened for the inclusion and exclusion criteria. The inclusion criteria were: diagnosis of schizophrenia based on diagnostic interview conducted for the study or on record at the site according to DSM-IV-R criteria; outpatient status; age 18–60; ability to understand and read spoken Mandarin sufficiently to comprehend testing procedures; ability to comprehend the consent form appropriately; no significant psychotropic medication changes in past 2 months and none anticipated for the next month; evidence of stable symptomatology ≥3 months (e.g., no hospitalizations for schizophrenia; no increase in level of psychiatric care due to worsening of symptoms of schizophrenia); PANSS score no more than 5 (moderately severe) on P1 (delusions), P3 (hallucinatory behavior), P5 (grandiosity), P6 (suspiciousness/persecution) and G9 (unusual thought content); PANSS score no more than 4 (moderate) on P2 (conceptual disorganization). If potential subjects met the inclusion criteria, the study was explained and subjects signed the informed consent document.

Exclusion criteria for the schizophrenia group were: alcohol or substance dependence in past six months; alcohol or substance abuse in past three months; clinically significant neurological disease; head injury (loss of consciousness over 1 h) as determined by medical history; current medical condition that would interfere with valid assessment; dystonia or parkinsonism that would affect validity of assessment based on clinical assessment; females who have a known pregnancy or are nursing; currently taking medications like methylphenidate, anti-dementia medications, amphetamine, lithium, monoamine oxidase inhibitors and tricyclic antidepressants; benzodiazepines, sedatives, or anticholinergic medications administered within 12 hours of assessment. A total sample of 230 patients with chronic schizophrenia was recruited and assessed using the MCCB for China and additional neuropsychological tests included in the study. After 4–8 weeks, we retested 188 subjects to measure test–retest reliability of the different instruments with this population. We assessed the control sample and patient sample in the same timeframe (from 07/2009 to 12/2010) in all six study cites. The participants were reimbursed 50 RMB for each visit of the study.

2.2. Assessment tools

a. Positive And Negative Syndrome Scale (PANSS):

PANSS has 30 items, which are combined to create three scales measuring positive (7 items) and negative (7 items) syndromes, and general psychopathology (16 items). Each item has a definition and specific 7-level operational criteria: 1-absent, 2-minimal, 3-mild, 4-moderate, 5-moderate–severe, 6-severe, and 7-extreme (Kay et al., 1987)

b. MATRICS Consensus Cognitive Battery (MCCB):

The Chinese version of MCCB includes nine tests that were administrated in the following order: Trail Making Test (TMT) Part A; Brief Assessment of Cognition in Schizophrenia (BACS) Symbol Coding; Form 1 of Hopkins Verbal Learning Test-Revised (HVLT-R), learning trials 1,2,3 and delay recall; Wechsler Memory Scale - Third Edition (WMS-III) Spatial Span; Form 1 of Neuropsychological Assessment Battery (NAB) Mazes; Form 1 of Brief Visuospatial Memory Test-Revised (BVMT-R), learning trials 1,2,3 and delay recall; Category fluency (animal names); Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) Managing Emotions; Continuous Performance Test-identical pairs version (CPT-IP). The MCCB covers seven cognitive domains including attention, information processing speed, verbal learning and memory, visual learning and memory, working memory, reasoning, problem solving, and social cognition. This set of tests was translated and revised by Chuan Shi and colleagues, standardized in the Chinese culture, and has good reliability and validity (Kern et al., 2008; Shi et al., 2015). In the current study, we administered not only the three learning trials of HVLT-R and BVMT-R, but also the delayed recall trial in both tests; we used same forms of HVLT-R and BVMT-R in the retest session.

c. Additional neuropsychological tests:

We have included some additional neuropsychological tests in this study, which have been used in prior studies of neurocognitive disorders in China (Heaton et al., 2008). These are action/verb fluency (Woods et al., 2005), Wisconsin Card Sorting Test-64 item version (WCST-64) (Shan et al., 2008), Grooved Pegboard (Heaton et al., 2004), Color Trails test I and II (Gulec et al., 2013), Stroop Color-Word Test (Laurenson et al., 2015) and Paced Auditory Serial Addition Task (PASAT-50, 2.4 seconds version) (Townsend et al., 2002). All these tests were co-normed with the MCCB using the current healthy comparison group. We included the PASAT of working memory as a substitute for the WAIS-III Letter-number Sequencing subtest, which was deleted due to the cultural difference mentioned above (Latin alphabet is not used in China). We also included action fluency because its reported sensitivity to frontal systems dysfunction may make it more relevant to schizophrenia than other fluency tests. Also, as Chan et al (Chan et al., 2008) discussed, executive function is a multifaceted term, include planning (Maze), working memory (Spatial Span and PASAT), inhibition (Stroop Task), mental flexibility (Color Trails II), reasoning as well as the ability to adjust the strategy based upon feedback (WCST),. Therefore we added these tests to cover other subdomains of executive function. Finally, it has been reported that speeded, fine motor coordination (Grooved Pegboard) is also a very important subdomain in schizophrenia (Hu et al., 2011), so this subtest was included as well.

2.3. Data analysis

The study samples are described using summary statistics. Means and standard deviations are presented for continuous variables, whereas counts and percentages are presented for categorical and binary variables. Comparisons between the schizophrenia and normal control groups were performed with independent t-tests for continuous variables and X² tests for categorical variables. Effect sizes (Cohen’s d) were calculated for neuropsychological group differences on neurocognitive variables, and Intraclass correlation coefficient (ICC) were used to examine test–retest reliability. Comparisons among three work status groups were performed by Analysis of Variance (ANOVA). All comparisons of neuropsychological tests employed demographically corrected T-scores that were generated from raw scores based on normative data in China, which were developed with the same healthy comparison group as in the current study, and are described in detail in our previous report (Shi et al., 2015). Also, the Global Deficit Score (GDS) method was used for classification of overall impairment status on the MCCB battery plus a modified battery that included only those MCCB and added tests that were most sensitive to differences between patients with schizophrenia and healthy controls (Carey et al., 2004; Heaton et al., 2004). Briefly, demographically corrected T-scores were converted to deficit scores according to the following criteria: T > 39 = 0 (normal), 39 ≥ T ≥ 35 = 1 (mild impairment), 34 ≥ T ≥ 30 = 2 (mild to moderate impairment), 29 ≥ T ≥ 25 = 3 (moderate impairment), 24 ≥ T ≥ 20 = 4 (moderate to severe impairment), T < 20 = 5 (severe impairment). Deficit scores were summed across the test battery and then divided by the number of individual measures to compute the GDS. The GDS can be analyzed as a continuous variable indicating number and severity of neurobehavioral deficits across the entire test battery, or as a cut-off of ≥0.50 that can be used to classify overall NP impairment (Carey et al., 2004; Heaton et al., 2004; Taylor and Heaton, 2001).

3. Results

Demographic and clinical data for the schizophrenia and normal control groups are summarized in Table 1. The data of normal control group comes from our previous study (Shi et al., 2015). Finally 230 patients with chronic schizophrenia were recruited, 31 from Beijing, 37 from Changsha, 33 from Shanghai, 43 from Xian, 48 from Harbin and 38 from Kunming. Compared with the normal control group, the patients with schizophrenia were less likely to be married (33.9% vs 77.4%), employed (23.9% vs 81.3%), had less lifetime duration of employment (12.6 vs 18.3 years) and less work time in the last 12 months (3.1 vs 9.6 months). For the patients who were currently working, they earned less income than did healthy controls (1086.4 vs 2067.1 RMB) as well. There were no significant differences in age, gender and years of education. The demographics of the retest sample of 188 did not differ from the total sample of 230 (P > 0.5).

Table 1.

Demographic and clinical characteristics

Variable	Schizophrenia group (n = 230)	Healthy controls (n = 656)	t/chi square	P
Age (yrs)	38.7(11.5)	39.3(11.4)	0.64	0.525
Gender (men%)	49.5	50.3	0.04	0.847
Education (yrs)	10.9(2.9)	10.8(3.2)	0.22	0.828
Marriage (percent)			169.44	<0.001
Married	33.9	77.4
Widowed	1.3	0.9
Separated	0.4	0
Divorced	20.9	2.6
Single	43.5	19.1
Employment status (percent)			255.17	<0.001
Current work	23.9	81.3
Ever work	65.7	14.6
Never work	10.4	4.1
Work time in lifelong (yrs)	12.6(10.4)	18.3(11.9)	6.88	<0.001
Work in last 12 months (months)	3.1(4.6)	9.6(4.4)	18.54	<0.001
Individual income per month (RMB)	1086.4(1281.7)	2067.1(1611.9)	6.36	<0.001
Family history (percent)
Negative	52.6
Positive	15.2
Missing	32.2
Onset pattern (percent)
Acute	26.1
Insidious	41.7
Missing	32.2
Relapse time	2.7(3.0)
Inhospitalization time	2.7(2.7)
PANSS total#	50.5(14.0)
PANSS Positive	10.5(4.5)
PANSS negative	14.4(5.8)
PANSS General Psychopathology	25.5(6.6)

^#PANSS: positive and negative syndrome scale

The patients with schizophrenia performed significantly worse than healthy controls on all neurocognitive tests (P < 0.001, except for WCST, p = 0.004). The effect sizes generally were from medium to large (see Table 2). Comparing the effect sizes of MCCB and added tests, the MCCB tests ranged from 0.42 to 1.24 (median 0.69) and the added tests ranged from 0.22 to 1.28 (median 0.79). If we only include the neuropsychological tests with large effect sizes of 0.8 and over, and combine those to make a “new” cognitive battery for patients with schizophrenia in China (NBSC), this new battery contains 4 measures from MCCB and 5 new measures (Trial making A, BACS, HVLT-R learning and recall, CPT-IP, dominant hand Grooved Pegboard, Color Trails I and II, PASAT). The test–retest reliabilities (ICCs) of all subtests range from 0.64 to 0.94; the range for the nine NBSC measures is 0.71–0.94, with a median of 0.80 (see Table 2).

Table 2.

Comparison between patients with schizophrenia and healthy controls in 9 MCCB and added tests, and test–retest reliability in schizophrenia

Neuropsychological tests*	Schizophrenia group (n = 230)	Healthy Controls (n = 656)	t	P	Effect size (Cohen’s d)	ICC
MCCB tests
Trail making A	41.2(10.4)	50.0(10.0)	11.38	<0.001	0.87	0.86
BACS symbol coding	37.4(10.5)	50.0(10.0)	16.22	<0.001	1.24	0.94
HVLT-R learning	40.8(10.5)	50.0(10.0)	11.85	<0.001	0.91	0.83
WMS-III spatial span	44.6(12.0)	50.0(10.0)	6.05	<0.001	0.51	0.81
NAB mazes	43.5(11.5)	50.0(10.0)	7.59	<0.001	0.62	0.84
BVMT-R learning	42.8(11.5)	50.0(10.0)	8.42	<0.001	0.69	0.83
Animal fluency	44.0(10.3)	50.0(10.0)	7.76	<0.001	0.59	0.72
MSCEIT managing emotions	45.6(11.9)	50.0(10.0)	5.07	<0.001	0.42	0.81
CPT-IP	41.3(10.2)	50.0(10.0)	11.33	<0.001	0.87	0.90
Added tests
HVLT-R recall	40.2(11.0)	50.0(10.0)	12.36	<0.001	0.96	0.72
BVMT-R recall	40.2(11.0)	50.0(10.0)	7.15	<0.001	0.61	0.64
Action fluency	44.8(10.6)	50.0(10.0)	6.73	<0.001	0.52	0.74
WCST (total errors)	47.8(10.5)	50.0(10.0)	2.90	0.004	0.22	0.76
Grooved pegboard (dominant hand)	36.9(10.9)	50.0(10.0)	16.70	<0.001	1.28	0.80
Color trails I	40.9(11.3)	50.0(10.0)	11.49	<0.001	0.88	0.71
Color trails II	38.0(13.0)	50.0(10.0)	12.69	<0.001	1.10	0.73
Stroop word	42.4(10.3)	50.0(10.0)	9.85	<0.001	0.76	0.81
Stroop color	42.1(9.8)	50.0(10.0)	10.34	<0.001	0.79	0.90
Stroop interference	42.5(10.1)	50.0(10.0)	9.72	<0.001	0.75	0.89
PASAT	41.8(10.4)	50.0(10.0)	10.58	<0.001	0.81	0.75
MCCB_GDS
GDS < 0.5	41.3%	83.5%
1 > GDS ≥ 0.50	27.4%	13.6%
2 > GDS ≥1	23.0%	2.6%
3 > GDS ≥ 2	7.9%	0.3%
4 > GDS ≥ 3	0.4%	0%
5 > GDS ≥4	0%	0%
GDS ≥5	0%	0%
NBSC_GDS
GDS < 0.5	27.8%	82.1%
1 > GDS ≥ 0.50	30.5%	13.7%
2 > GDS ≥1	26.0%	3.9%
3 > GDS ≥ 2	11.8%	0.3%
4 > GDS ≥ 3	3.5%	0%
5 > GDS ≥4	0.4%	0%
GDS ≥5	0%	0%

^*tests abbreviations: BACS: Brief Assessment of Cognition in Schizophrenia; HVLT-R: Hopkins Verbal Learning Test-Revised version; WMS-III: Wechsler Memory Scale - Third Edition; NAB: Neuropsychological Assessment Battery-Mazes; BVMT-R: Brief Visuospatial Memory Test-Revised; MSCEIT: Mayer-Salovey-Caruso Emotional Intelligence; CPT-IP: Continuous Performance Test – identical pairs version; WCST: Wisconsin Card Sorting Test; PASAT: Paced Auditory Serial Addition Task; MCCB:MATRICS consensus cognitive battery; NBSC: “new” neurocognitive battery for schizophrenia in China; GDS: Global Deficit Score; ICC: Intraclass Correlation Coefficient.

Comparing the new battery with the original MCCB, more patients with schizophrenia were identified as cognitively impaired according to the criterion of GDS ≥ 0.50 for the new battery (166 vs 135, or 72.2% vs 58.7% for the MCCB, chi square 9.237, p = 0.002). If we calculate the GDS of the whole battery (both original and new tests), 142 patients (61.7%) with schizophrenia were identified as cognitively impaired. By contrast, for the normal controls, specificity rates were comparable for both batteries (83.5% for MCCB, 82.1% for the NBSC, chi square 0.459, p = 0.498), and consistent with the desired impairment classification cutoff of one standard deviation below the mean for healthy controls. The distributions of GDS-defined impairment rates for the original MCCB and the new battery (NBSC) are presented in Table 2 for both the normative sample and the schizophrenia sample.

The NBSC based upon this sample of schizophrenia patients in China covers six subdomains, including attention, working memory, speed of information processing, verbal episodic memory (learning and delayed recall), executive function and fine motor function; what are missing in the NBSC from the original MCCB are tests of visual episodic memory, Spatial Span, NAB mazes, animal fluency and social cognition. Five subdomains in NBSC overlap with MCCB. Considering the six subdomains in NBSC and the additional subdomains from the MCCB (visual learning and social cognition), all 8 subdomains have mean T scores that were significantly lower in patients with schizophrenia compared with normal healthy controls (Table 2). The subdomain T scores of working memory (PASAT) and executive function (Color Trials II) of NBSC were significantly lower in patients with schizophrenia than were corresponding subdomains of MCCB; this is also true for the composite T score as well (see Fig. 1).

Fig. 1.

Comparisons of cognitive subdomain and global mean T-scores among normal controls, schizophrenic patients evaluated with MCCB and NBSC. a: Significant group effect, p < 0.001, with schizophrenia patients scoring worse than controls. b: Significant group effect, p < 0.001, between schizophrenic patients evaluated with MCCB and NBSC.

When we compared neurocognition among patients with schizophrenia who were currently working, versus ever worked and never worked, we found there were significant group differences in BACS (p = 0.004), HVLT-R learning (p = 0.035), BVMT-R learning (p = 0.023), BVMT-R Recall (p = 0.032), Grooved Pegboard (p < 0.001), Stroop word reading (p = 0.044) as well as the MCCB composite (p = 0.013), NBSC composite (P = 0.006), MCCB_GDS (p = 0.038) and NBSC GDS (0.020) (see Table 3). Post hoc tests with Bonferroni correction showed that patients with schizophrenia who were currently working performed significantly better on BACS (p = 0.003), HVLT-R (p = 0.03), BVMT-R (p = 0.024), Grooved Pegboard (p = 0.001), MCCB (0.035) and NBSC (0.024) than patients with schizophrenia who never worked. Based on LSD test, patients with schizophrenia who were currently working had better NBSC GDS (0.024) than patients with schizophrenia who ever worked.

Table 3.

Neurocognition difference among different employment status

	Currently working (n = 55)	Ever worked (n = 151)	Never worked (n = 24)	F-value*	P
MCCB tests
Trail making A	43.1 (10.1)	41.0 (10.8)	38.3 (7.9)	1.87	0.157
BACS symbol coding	40.6 (10.6)	37.1 (10.4)	32.3 (8.5)	5.66	0.004
HVLT-R learning	42.9 (10.7)	40.7 (10.1)	36.3 (11.0)	3.39	0.035
WMS-III spatial span	47.6 (12.1)	43.4 (11.8)	45.2 (12.2)	2.54	0.081
NAB mazes	45.2 (11.6)	42.8 (11.2)	43.9 (13.2)	0.92	0.401
BVMT-R learning	45.8 (11.2)	42.4 (11.3)	38.4 (12.4)	3.86	0.023
Animal fluency	43.8 (10.8)	44.5 (10.3)	40.8 (9.1)	1.42	0.245
MSCEIT managing emotions	48.5 (10.4)	44.9 (12.3)	42.9 (12.0)	2.49	0.086
CPT-IP	43.3 (12.2)	40.5 (9.5)	41.4 (8.9)	1.53	0.219
Added tests
HVLT-R recall	42.6(11.4)	39.9(11.0)	36.6 (9.2)	2.72	0.068
BVMT-R recall	47.3 (12.6)	42.3 (12.2)	41.9 (12.9)	3.49	0.032
Action fluency	45.8 (10.2)	44.0 (10.9)	46.9 (10.0)	1.11	0.333
WCST	49.4 (10.7)	46.9 (10.7)	49.6 (9.0)	1.55	0.214
Grooved pegboard	42.6 (11.11)	35.4 (10.4)	33.1 (9.2)	11.41	<0.001
Color trail I	43.3 (11.0)	40.2 (11.6)	39.5 (9.7)	1.77	0.173
Color trail II	39.7 (13.4)	37.8 (13.0)	35.4 (11.9)	0.93	0.397
Stroop word	43.9 (9.8)	41.2 (10.4)	46.0 (9.6)	3.17	0.044
Stroop color	44.1 (8.7)	41.1 (10.5)	42.0 (7.6)	1.57	0.210
Stroop interference	42.5 (10.1)	50.0 (10.0)	42.4 (8.9)	0.72	0.490
PASAT	44.0 (10.0)	42.0 (10.4)	41.5 (12.3)	2.28	0.105
Battery
MCCB	45.12(7.21)	42.28(7.01)	40.75(6.67)	4.44	0.013
NBSC	42.51(8.26)	39.09(7.58)	37.44(6.17)	5.19	0.006
MCCB_GDS	0.67(0.72)	0.87(0.73)	1.11(0.74)	3.33	0.038
NBSC_GDS	0.82(0.85)	1.14(0.91)	1.38(0.84)	3.99	0.020

^*one-way ANOVA.

The rates of cognitive impairment according to MCCB GDS and NBSC GDS in patients with schizophrenia who were currently working, ever worked and never worked are 45.5%, 61.6%, 70.8% (MCCB) and 56.4%, 74.8%, 91.7% (NBSC). Chi square test showed that the rates of cognitive impairment were significantly different by NBSC GDS (p = 0.003), whereas the difference was not significant by MCCB GDS (p = 0.051).

4. Discussion

In this study, we investigated in China a new (or revised) neurocognitive test battery for schizophrenia named NBSC, and found that it appears to be more sensitive in distinguishing Chinese participants with schizophrenia from normal people than is the standard MCCB, and also stronger associations with functional status (employment). The NBSC includes nine subtests and six subdomains, and was able to identify more patients with cognitive impairment than MCCB as well as a total (combined) neurocognitive test battery. However, its selection was based upon the current sample, so its apparent superiority needs to be validated in independent samples.

Cognitive impairments are core features of schizophrenia, which are demonstrated in prodrome stage, first episode as well as chronic stage of the illness (Bliksted et al., 2014; Pietrzak et al., 2009; Simon et al., 2012). In recent years, the MATICS Consensus Cognitive Battery (MCCB), consisting of 10 subtests grouped by cognitive domains, has been widely used in many countries as a standardized measure of cognitive impairment in patients with schizophrenia (Fonseca et al., 2017; Jedrasik-Styla et al., 2015; Kaneda et al., 2013; Mohn et al., 2017; Noh et al., 2010; Rodriguez-Jimenez et al., 2012). However, studies in many countries such as Spain, Poland, Norway, Brazil, Korea and Japan found differences in patterns of MCCB performance of patients with schizophrenia when comparing with the United States setting (Holmen et al., 2010; Lystad et al., 2014; Rodriguez-Jimenez et al., 2015). In the United States, the most impaired subdomains of MCCB were working memory and processing speed, in Spain they were verbal learning and visual learning, in Norway they were verbal learning and working memory, and in China we see greatest schizophrenia-related impairments in verbal learning and processing speed (Holmen et al., 2010; Kern et al., 2011; Rodriguez-Jimenez et al., 2015; Shi et al., 2015). These differences are unlikely to be explained fully by sample differences, and cultural and race differences may also contribute.

Racial and cultural differences across settings not only have been found in MCCB, but also for other neuropsychological tests. Studies have shown that older African Americans performed worse than Caucasians in single neuropsychological test like Clock Drawing and Digital Symbol Substitution, as well as global cognitive function assessed by Mini-Mental State Examination (MMSE). The differences are still significant even on scores adjusted by age, education and socioeconomic status (Mehta et al., 2004; Schillerstrom et al., 2007). Heaton et al. (2004) found similar race/ethnicity differences on the expanded Halstead-Reitan Neuropsychological Battery, including the Wechsler Intelligence Scales. It has been well documented that East Asians differ from Westerners in conscious perception and attention, as well as unconscious learning (Fu et al., 2013; Nisbett and Miyamoto, 2005; Nisbett et al., 2001). In terms of the above evidence, we think it is reasonable to prioritize the culture and race validated neuropsychological tests to patients with schizophrenia in China, which appears to be better (more sensitive) than applying the complete, original MCCB.

Preliminary evidence presented here suggests that our new battery could identify more impaired individuals with chronic schizophrenia than MCCB (72% vs 59%), whereas the two batteries had comparable specificity in correctly classifying healthy controls (82.1% vs 83.5%). The new battery included MCCB measures from three cognitive domains, (processing speed, attention and verbal learning) with MCCB tests in the domains of social cognition, executive function, visual learning and working memory not included. Our prior norm study indicated the subtest Mayer–Salovey–Caruso Emotional Intelligence Test (MSCEIT): Managing Emotions did not appear to be very sensitive measure of social cognition within the Chinese context (Shi et al., 2015), and it was the least sensitive MCCB measure to disease related impairment in the current study. Another study conducted in China showed there were no significant differences in visual learning and social cognition subdomains of MCCB between first episode patients with schizophrenia and normal controls (Cao et al., 2017). This suggests that a more culturally relevant social cognition instrument may need to be developed for China. A new subdomain included in our current battery is complex motor skill, evaluated by the Grooved Pegboard. This is a test of perceptual-motor speed that has been found in factor analytic work to load with other measures of processing speed (Patt et al., 2018). Here it had the largest schizophrenia versus healthy control effect size of all tests considered (both MCCB and added tests; Cohen’s d = 1.28). Although medication side effects of the extrapyramidal system could contribute to this impairment, another study showed the Chinese drug-naïve first-episode patients with schizophrenia also had significant motor skill impairment (Hu et al., 2011).

Previous studies showed that the working memory, attention, and processing speed could predict work ability in schizophrenia (August et al., 2012; Bowie et al., 2008; Shamsi et al., 2011). Our study found that in addition to processing speed, fine motor skill, verbal and visual learning also appear to influence work ability for Chinese patients with schizophrenia. It demonstrated repeatedly the new subdomain beyond MCCB, i.e. speeded fine motor skills, was an important subdomain, which should be assessed in the patients with schizophrenia in China. In addition, our finding that the BVMT-R measures of visual episodic memory were sensitive to work status group differences suggests that this test might be added to the NBSC when questions about everyday functioning are of concern. The added tests may or may not be more “difficult” in general but, at least in China, they appear to provide increased sensitivity to cognitive impairment in schizophrenia. Including all the MCCB tests for research certainly would have an advantage in comparing results to those of other international studies. Also, including all the MCCB tests may increase the comprehensiveness of the battery, which could improve its characterization of cognitive status in some patients, but it would also involve a longer assessment time. Also it is unknown whether some tests that have been omitted from the NBSC could be more sensitive to some treatments; if so, this may make the NBSC less sensitive to such treatment effects.

In conclusion, though the MATRICS Consensus Cognitive Battery (MCCB) has been widely used in many countries as a standardized measure of cognitive impairment in schizophrenia, MCCB tests of some cognitive domains may not be optimal for detecting cognitive impairment in schizophrenia internationally, due to cultural, language or other background differences in different countries. Our study suggested that a new battery could be more sensitive to detect the cognitive impairment in patients with schizophrenia in China. However, these results need to be further validated with new samples. Both MCCB and our new battery demonstrated a significant association of cognitive impairment with employment difficulties in this Chinese patient group.

Given that the globalization phenomenon is increasingly prominent, an important question is whether the future development and the use of neuropsychological testing in mainland China should adopt a culture-specific approach or whether we should develop an integration and cross-cultural approach for clinical and research-related work. We intend to test, and validate in the future a cognitive assessment tool like the NIH Toolbox Cognitive Battery, which was specifically developed as a cross-cultural battery, and compare it the current MBSC and possibly some new, culturally-specific tests developed by ourselves to see what is optimal.

There are significant limitations to the current study that should be noted. First, the new battery (NBSC) was selected based upon its optimal performance with the present sample of clinically stable individuals with chronic schizophrenia. As such, any firm conclusions about its sensitivity must await replication with at least one independent sample. Second, although our comparison of the NBSC in the current sample suggests that it is more sensitive to schizophrenia-associated cognitive impairment in China than is the standard MCCB, there is no independent “gold standard” to verify presence of impairment in individual participants with schizophrenia. However, the fact that the two batteries both showed comparable, good specificity in correctly classifying healthy controls in China provides some confidence that both are accurately distinguishing impaired from unimpaired cognition in this country. Also the fact that associations between GDS-defined cognitive impairment and employment history in schizophrenia were superior for the NBSC versus the MCCB supports the validity of the GDS-defined classifications by the NBSC. Another limitation is that we used only one subtest to evaluate social cognition included in this study and it was not sensitive to distinguish schizophrenia from normal controls. In the future we will try to include more subtests and make needed cultural modifications to measure the theory of mind, social knowledge, social setting awareness, emotional management, attribution bias, etc, to determine the most sensitive subtests for this important cognitive domain in China. Finally, this study only validated the NBSC in stable chronic patients with schizophrenia, and future work must examine it in first episode patients with schizophrenia as well as other stable chronic patients with schizophrenia.