Screening for cognitive impairment in schizophrenia: Psychometric properties of the German version of the Screen for Cognitive Impairment in Psychiatry (SCIP-G)

Abstract

Background

The Screen for Cognitive Impairment in Psychiatry (SCIP) is a brief scale designed for detecting cognitive deficits in patients with psychiatric disorders including schizophrenia. In this preliminary study the psychometric properties of the German version of the SCIP are examined in a sample of patients with schizophrenia and schizoaffective psychosis (DSM-IV) as well as in healthy controls.

Methods

Thirty patients and thirty matched controls were asked to complete two versions of the SCIP separated by two-week intervals in addition to psychiatric and neurocognitive instruments including assessments to measure psychosocial functioning. Feasibility, reliability and validity of the SCIP were examined in order to determine parallel reliability. The convergent validity was assessed by the BACS (Brief Assessment of Cognition in Schizophrenia) and the MMSE (Mini-Mental-State-Examination).

Results

Significant differences in cognitive performance between patients and healthy controls were detected in both versions of the SCIP. The SCIP effectively discriminated between patients and the control sample. The reliability of the parallel versions of the SCIP was supported by high correlations between the alternate forms, and by the high internal consistency of SCIP subtests within the patient sample. Construct validity of the SCIP was supported by high correlations between the SCIP and the BACS total scores, and by high correlations with common cognitive domain scores from the two tests.

Conclusions

Our data show that the German version of the SCIP (SCIP-G) is a brief, valid and reliable assessment tool for the detection of cognitive impairment in patients with schizophrenia or schizoaffective psychosis.

1. Introduction

Cognitive impairment is considered a core feature of schizophrenia and is of particular importance for the diagnostic, therapeutic and rehabilitative process (Sachs et al., 2004, Keefe, 2014, Nuechterlein et al., 2014). Neurocognition is correlated with therapeutic adherence, functional outcome and quality of life (Harvey, 2014). While a wide range of different assessment batteries has been used to measure cognitive function (Keefe et al., 2011), and while neurocognition in psychiatric patients has been well studied herewith, a detailed assessment of cognitive function, unfortunately, is often not included in routine clinical practice.

In general, evaluating cognitive impairment as well as measuring the effects of various treatment strategies requires reliable, valid and efficient assessment procedures. The Neurocognitive Committee for the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Project recommends the use of a tailored cognitive battery for research in this patient population that takes into account the special cognitive disability of this group; it is suggested to administer this battery of standardized neuropsychological instruments in approximately 60 to 120 min and by specially trained staff (Green et al., 2004; Nuechterlein et al., 2008). In contrast, the Brief Assessment of Cognition in Schizophrenia (BACS, Keefe et al., 2004, Keefe et al., 2008) is a battery which was designed to be less comprehensive and to require only approximately 35 min of testing time. The original version of the Brief Assessment of Cognition in Schizophrenia (BACS) has already been translated and standardized in a German version (Sachs et al., 2011).

The present study deals with the Screen for Cognitive Impairment in Psychiatry (SCIP; Purdon, 2005), a brief instrument which is applicable with minimal training and which requires only approximately 15 min to administer. The SCIP has been developed to test cognitive deficits in patients with all psychiatric disorders. The original version is in English and has good reliability in healthy controls (Purdon, 2005). The test has been shown to be valid and reliable in different language versions, for instance in Japanese (Hirabayashi et al., 2006), Spanish (Pino et al., 2008), Danish (Jensen et al., 2015), French (Tourjman et al., 2016) and Italian (Murri et al., 2020). The alternate forms of the Spanish version (SCIP-S) were equivalent when tested in a large sample of patients suffering from schizophrenia, with intra-correlation coefficients ranging from 0.74–0.81, and internal consistency and validity that were sufficient to confirm that the SCIP is an adequate tool to measure cognitive functioning in patients with schizophrenia (Pino et al., 2008). In further studies the reliability, validity and internal consistency of the SCIP was confirmed in patients with bipolar disorder (Guilera et al., 2009), in patients diagnosed with schizophrenia and bipolar disorder (Rojo et al., 2010) and in unipolar depressive disorder (Ott et al., 2016).

The aim of the present study was to assess the psychometric properties of the German version of the SCIP (SCIP-G). The objective of this study was to provide the first clinical normative data for the SCIP-G, to test the discriminant validity by comparing patients with schizophrenia to healthy controls, as well as assessing the sensitivity and specificity, the test-retest reliability, the internal consistency and validity. In addition, the association of neurocognition (as measured by SCIP-G) with quality of life and functional outcome was investigated.

2. Methods and materials

2.1. Subjects

Thirty patients with schizophrenia or schizoaffective psychosis and thirty healthy controls matched to patients by gender, age and education participated in the study. All were aged between 18 and 50 years. Data were collected at the Medical University of Vienna, Department of Psychiatry and Psychotherapy. The participants gave their informed consent and the study was approved by the local ethics committee. The study was carried out in accordance with the Declaration of Helsinki. All procedures were applied to patients and healthy controls by experienced psychiatrists and neuropsychologists. Psychiatrists were trained with the structured clinical interview for DSM-IV (American Psychiatric Association, 1994), patient edition and non-patient edition (SCID-P, SCID-NP, First et al., 1995a, First et al., 1995b), the Positive and Negative Syndrome Scale (PANSS, Kay et al., 1987) and the inclusion and exclusion criteria of the study. For the investigation of inclusion and exclusion criteria, social and demographic characteristics were assessed. Patients were interviewed by the SCID-P, healthy controls by the SCID-NP. At the baseline examination, the Mehrfachwortschatztest Form B (MWT-B, Lehrl, 1997) was completed.

Patients with a primary diagnosis other than schizophrenia or schizoaffective disorder, assessed according to the DSM-IV criteria, were not included in the study. Patients were also excluded if they had exhibited comorbid substance abuse (alcohol, drugs) active within the previous 6 months, or had experienced a serious somatic disorder, any neurological disorder at the time of the study, or any serious lifetime disorder that could have had an influence on the study. Other exclusion criteria were extant participation in a clinical trial, depot neuroleptic treatment in the past, long-term treatment with or abuse of tranquilizers, or difficulty reading and/or writing. 22 in-patients and 8 out-patients were included in the study, the prerequisite being clinical stabilisation defined by the fact that medication had not been changed for a two-week period. No changes in drug regimen or dose were permitted during the study.

2.2. The Screen for Cognitive Impairment in Psychiatry

The Screen of Cognitive Impairment in Psychiatry (SCIP, Purdon, 2005) was designed for detecting cognitive impairment in patients with psychiatric disorders. It requires nearly 15 min and may be administered without the need of additional equipment (only pencil and paper). Three alternative forms of the scale are available for repeated testing to avoid learning effects. The SCIP includes a Working Memory Test (WMT), a Verbal Learning Test – Immediate (VLT-I), a Verbal Fluency Test (VFT), a Verbal Learning Test – Delayed (VLT-D), and a Processing Speed Test (PST). The original SCIP version is in English.

2.3. Adaptation of the SCIP and assessment procedure

Two German forms of the SCIP were created using the standard translation/back-translation procedure verified by Scot E. Purdon, the author of the original English version. The psychologists were experienced on neuropsychological assessments of psychiatric samples and were trained on the standardized administration of the SCIP-G. Participants were examined at two occasions separated by two weeks (±2 days) intervals corresponding to visit 1 (baseline) and visit 2. The parallel reliability of the SCIP-G was evaluated using a cross-over counterbalanced design. The randomized half of the participants received form 1 of the SCIP-G at visit 1 and form 2 at visit 2. The second half of the participants received form 2 at visit 1 and form 1 at visit 2.

Furthermore, on each visit clinical ratings were carried out using the Brief Assessment of Cognition in Schizophrenia (BACS, German version, Sachs et al., 2011), the Mini-Mental-State-Examination (MMSE, Cockrell and Folstein, 1988), the PANSS (Kay et al., 1987), the Clinical Global Impression Scale (CGI-G, Guy, 1976), the Beck Depression Scale (BDI, Beck et al., 1961), the State-Trait Anxiety Inventory (STAI, Laux et al., 1981), the Personal and Social Performance Scale (PSP, Juckel et al., 2008), the Symptom Check List (SCL-90-R, Franke, 1995) and the World Health Organisation Quality of Life Assessment Instrument, Brief Form (WHOQOL-BREF, WHOQOL Group, 1998).

2.4. Statistical analysis

The analyses were undertaken using the SPSS statistical package version 27.0 and the significance level was set at Alpha 0.05. The reliability of the SCIP-G was examined with multivariate comparisons of baseline scores and by multivariate analyses between the baseline scores and visit 1 (MANOVA). The practice effect was calculated with Cohen's d, representing the difference between scores at visit 2 and visit 1 divided by the standard deviation of visit 1. Sensitivity and specificity of the SCIP-G for cognitive impairment were investigated by the Receiver – Operating – Characteristic (ROC). To test the differences between the two alternative forms, t-tests were carried out. Test – retest reliability was assessed with Pearson correlations between the SCIP-G alternative forms. Internal consistency of the SCIP-G was examined with Cronbach's Alpha coefficient. The construct validity of the SCIP-G was assessed with Pearson's correlation coefficients and linear regression analysis between the SCIP-G subtests and the scores from the standard neuropsychological tests.

3. Results

3.1. Sample description

Thirty patients diagnosed with schizophrenia (93.3%) or schizoaffective disorder (6.7%) participated in the study. Demographic data and characteristics of the participants are shown in Table 1. In both the patient group and the control group 12 females and 18 males were included. The mean age of the patients was 30.43 (SD = 9.35) years, the mean age of the controls 30.23 (SD = 9.49). The years of education were 13.23 (SD = 2.45) and 13.27 (SD = 2.31), respectively. As for the premorbid intelligence, the mean IQ scores were 105.93 (SD = 14.31) in the patient group and 112.83 (SD = 14.44) in the healthy control group. No significant differences were observed at baseline between the patient sample and the controls.

Table 1.

Characteristics of the study sample at baseline.

Variable	Schizophrenia and schizoaffective disorder		Controls
	(N = 30)		(n = 30)
	N	%	N	%
Gender, female	12	40	12	40
	M (SD)		M (SD)
Age	30,43 (9,35)		30,23 (9,49)
Years of education	13.23 (2.45)		13.27 (2.31)
Premorbid intelligence IQ	105,93 (14,31)		112,83 (14,44)
Duration of illness years	7,70 (6,60)
Prior hospitalization	4,90 (3,67)
CGI	4,57 (0,73)
PANSS
Positive score	16,17 (4,56)
Negative score	21,43 (4,72)
Total score	80,00 (13,68)
Medication
Atypical antipsychotics	79,4
Antidepressants	59,7
Benzodiazepines	56,7
Typical antipsychotics	30
Mood stabilizers	3,3

M mean, SD standard deviation.

CGI Clinical Global Impression Scale.

PANSS Positive and Negative Syndrome Scale.

In the patient group the mean age at the first manifestation of the disorder was 22.60 (SD = 5.72) years, the average duration of illness was 7.70 years (SD = 6.60) and the average number of prior hospitalizations was 4.90 (SD = 3.67). All patients received antipsychotic medication. Sixty percent were treated with a second generation antipsychotic as only antipsychotic, 33.3% received a combination of two antipsychotics, 6.7% received three antipsychotic drugs. In addition to antipsychotic treatment, 53.0% received antidepressants, and 56.7% were receiving benzodiazepines. At baseline, the CGI score was 4.57 (SD = 0.73), the PANSS negative score was 21.43 (SD = 4.72), the positive score was 16.17 (SD = 4.56), the PANSS total score was 80.00 (SD = 13.68).

3.2. SCIP scores

3.2.1. Comparisons and effect sizes

All 60 participants completed the SCIP-G at both visits. The results were compared by multivariate analyses (MANOVA); significant differences between patients and healthy probands were observed in all subtests and in the total score of the two forms (SCIP-G form 1 (F(5,52) = 32.23, p < .001) and SCIP-G form 2 (F(5,52) = 21.29, p < .001)).

The univariate comparisons for all five subscales and the total score were also significant and are shown on Table 2, Table 3. No significant differences were detected with regard to the order of the forms and no interaction effect was detected between groups and order of the forms. The effect sizes (Cohen's d) of the differences between groups ranged from 1.47 (VFT SCIP-G form 1) to 3.04 (total score SCIP-G form 1). The magnitude of the difference was substantial.

Table 2.

Univariate comparison of cognitive performance in patients and healthy controls - form 1.

SCIP-G	Patients		Controls		F(1,56)	p	Cohen's d
Form 1	M	SD	M	SD
VLT-I	21.47	3.66	26.87	1.46	54.77	<.001	1.94
WMT	16.03	2.97	22.40	1.59	106,14	<.001	2.68
VFT	10.57	2.70	15.80	4.24	32.14	<.001	1.47
VLT-D	6.20	2.16	8.83	1.29	31.94	<.001	1.48
PST	8.30	1.92	13.67	2.17	103.33	<.001	2.62
Total	62.57	9.51	87.57	6.71	134.72	<.001	3.04

Univariate comparison of cognitive performance between patients and healthy controls.

P values for t-tests are reported along with Cohen's d as a measure of effect size.

M mean, SD standard deviation.

SCIP-G Screen for Cognitive Impairment in Psychiatry - German version.

VLT-I verbal learning test – immediate; WMT working memory test, VFT verbal fluency test, VLT-D verbal learning test-delayed, PST psychomotor speed test.

Table 3.

Comparison of cognitive performance in patients and healthy controls - form 2.

SCIP-G	Patients		Controls		F(1,56)	p	Cohen's d
Form 2	M	SD	M	SD
VLT-I	22.23	3.61	26.50	1.33	36.34	<.001	1.57
MWT	17.23	3.80	22.93	1.31	58.33	<.001	2.01
VFT	12.57	4.40	22.73	4.10	52.89	<.001	2.34
VLT-D	5.80	2.67	8.93	1.31	32.28	<.001	1.49
PST	8.70	2.25	13.30	2.26	60.68	<.001	2.04
Total	66.53	11.81	92.60	7.02	107.85	<.001	2.68

Comparison of cognitive performance between patients and healthy controls.

P values for t-tests are reported along with Cohen's d as a measure of effect size.

M mean, SD standard deviation.

SCIP-G Screen for Cognitive Impairment in Psychiatry – German version.

VLT-I verbal learning test – immediate; WMT working memory test, VFT verbal fluency test, VLT-D verbal learning test-delayed, PST psychomotor speed test.

3.2.2. Sensitivity and specificity

To assess the utility of the SCIP-G (form 1) to differentiate between patients and controls, a receiver operating characteristics (ROC) curve analysis for the total SCIP-G score was performed. Fig. 1 shows the ROC curve. The sensitivity was 0.93 and the specificity was 0.97 with an AUC score of 98.9%. The Chi 2 – test (Chi 2(1) = 48.65 (p < .001)) was significant. For the total score of the SCIP-G test (form 2) the ROC curve is shown in Fig. 2: the sensitivity was 0.90, the specificity was 0.97 and the AUC score was 97.2%. The Chi 2 – test (Chi 2(1) = 45.27 (p < .001)) was significant.

Fig. 1.

ROC curve for the SCIP-G - Form 1:

The receiver operating characteristics (ROC) curve for the predicted probability for being either identified as a patient or a healthy control.

Fig. 2.

ROC curve for the SCIP-G - Form 2:

The receiver operating characteristics (ROC) curve for the predicted probability for being either identified as a patient or a healthy control.

Misclassification occurred in 3 out of 60 persons with SCIP form 1 and 4 out of 60 persons with SCIP form 2.

3.2.3. Test-retest reliability

The parallel reliability of the SCIP was evaluated with a cross-over counterbalanced design (as described in Section 2.3). Parallel test reliability was supported by relatively high correlations between the total SCIP-G scores from form 1 and form 2. Pearson correlation coefficients were above r = 0.69 for the patient sample and the healthy control sample, regardless of the order in which the forms were administered. For the SCIP-G total scores, the reliability coefficients of the sequence “form 1 - form 2” were higher than of the sequence “form 2 - form 1” in both the patient and the control group. The data are shown in Table 4.

Table 4.

The parallel test reliability of the SCIP-G total scores in patients and controls.

Cross over design		M	SD	r	p
Patients
Form 1 - form 2	SCIP form 1 total score	61.93	9.93	0.760	.001
	SCIP form 2 total score	66.80	14.77
Form 2 - form 1	SCIP form 1 total score	63.20	9.37	0.694	.004
	SCIP form 2 total score	66.27	14.77
Controls
Form 1 - form 2	SCIP form 1 total score	86.80	7.41	0.762	.001
	SCIP form 2 total score	90.13	5.96
Form 2 - form 1	SCIP form 1 total score	88.33	6.09	0.694	.004
	SCIP form 2 total score	95.07	7.31

r = Pearson's correlation, M mean, SD standard deviation.

SCIP-G Screen for Cognitive Impairment in Psychiatry - German version.

In order to quantify any possible practice effects from repeat testing, the results from the two test forms were compared. In patients no difference was found (SCIP-G “form 1 - form 2” t(14) = −1.947, p = .072; SCIP-G “form 2 - form 1” t(14) = −1.693, p = .113). In the control group, a difference was found (SCIP-G “form 1 - form 2” t(14) = −2.686, p = .018 and “form 2 - form 1” t(14) = −4.861, p < .001). This result does not reveal a practice effect, but rather suggests that form 2 was easier to perform for healthy controls than form 1.

3.2.4. Internal consistency

The five subtests of the SCIP-G reached a Cronbach's Alpha of 0.728 for the form 1 in the patient group and of 0.445 for the healthy controls. The internal consistency of the form 2 of the SCIP-G showed an Alpha of 0.722 in patients and of 0.421 in healthy probands. These are acceptable values only for the patient group. For the control group a low internal consistency of the subtests of the SCIP-G must be recognized.

3.2.5. Construct validity

Pearson's correlation coefficients between the total scores of the SCIP-G and test scores from the BACS were calculated. All scores were statistically significant (patients: SCIP-G form 1 with BACS A 0.617, SCIP-G form 2 with BACS B 0.757; healthy probands: SCIP-G form 1 with BACS A 0.719, SCIP-G form 2 with BACS B 0.600). Fig. 3, Fig. 4 show the correlations between the SCIP-G and the German version of the BACS in healthy probands and patients, respectively.

Fig. 3.

Correlations between SCIP-G form 1 and 2 and BACS A and B in healthy controls.

Fig. 4.

Correlations between SCIP-G form 1 and 2 and BACS A and B in patients.

The convergent validity of the SCIP-G was also assessed for each cognitive domain deemed to be similar between the SCIP-G and the BACS; VLT of the SCIP-G was correlated with the BACS verbal memory test, the WMT of the SCIP-G with the Digit Sequencing of the BACS, the VFT of the SCIP-G with the Word fluency of the BACS, and the PST of the SCIP-G with the Symbol Coding of the BACS. The correlation coefficients between the subtests of the SCIP-G and the BACS were between 0.363 (VFT of the SCIP-G form 1 and Word Fluency of the BACS A) and 0.766 (PST of the SCIP-G form 1 and the Symbol Coding of the BACS A) in the patient group. The correlations between subtests are included in Table 5.

Table 5.

Results of the assessment of correlations between subtests of the SCIP-G and the German version of the BACS.

Pearson's correlations between SCIP-G form 1 and BACS A
SCIP-G form 1 and BACS A	Patients		Controls
	r	p	r	p
VLT-I (SCIP-G) and verbal memory (BACS)	0.619	<.001	0.445	.014
WMT (SCIP-G) and digit sequencing (BACS)	0.450	.013	0.152	.424
VFT (SCIP-G) and verbal fluency (BACS)	0.363	.048	0.539	.002
PST (SCIP-G) and symbol coding (BACS)	0.766	<.001	0.407	.025

Pearson's correlations between SCIP-G form 2 and BACS B
SCIP-G form 2 and BACS B	Patients		Controls
	r	p	r	p
VLT-I (SCIP-G) and verbal memory (BACS)	0.649	<.001	0.513	.004
WMT (SCIP-G) and digit sequencing (BACS)	0.544	.002	0.151	.426
VFT (SCIP-G) and verbal fluency (BACS)	0.553	.002	0.627	<.001
PST (SCIP-G) and symbol coding (BACS)	0.755	<.001	0.168	.374

r Pearson's correlations.

SCIP-G Screen for Cognitive Impairment in Psychiatry – German version.

VLT-I Verbal Learning Test - Immediate, WMT Working Memory Test, VFT Verbal Fluency Test, PST Psychomotor Speed Test.

BACS Brief Assessment of Cognition in Schizophrenia.

Moreover, correlations were calculated between the SCIP-G forms and the MMSE Scores in the patient group (SCIP-G form 1 and MMSE baseline r = 0.516; SCIP-G form 1 and MMSE visit 2 r = 0.662. SCIP-G form 2 and MMSE baseline r = 0.439; SCIP-G form 2 and MMSE visit 2 r = 0.652). Due to a ceiling effect, in our sample the MMSE did not adequately map differences in cognitive ability in healthy controls.

3.3. Correlations between cognitive impairment and psychopathology

Pearson's correlations showed significant negative correlations between the Positive Syndrome Scale of the PANSS and the SCIP-G form 1 at both visit 1 (r = −0.59) and visit 2 (r = −0.62). Otherwise, as shown in Table 6 no significant correlations were found between the two forms of the SCIP-G and PANSS ratings.

Table 6.

Pearson's correlations between the forms of the SCIP-G and the Positive and Negative Syndrome Scale.

	Visit 1		Visit 1		Visit 2		Visit 2
	SCIP-G form 1		SCIP-G form 2		SCIP-G form 1		SCIP-G form 2
	r	p	r	p	r	p	r	p
PANSS
Positive symptoms	−0.591	.020	0.175	.534	−0.624	.013	−0.209	.455
Negative symptoms	−0.247	.376	−0.109	.700	−0.434	.106	−0.250	.368
Total score	−0.354	.195	0.257	.355	−0.429	.111	−0.109	.699

r Pearson's correlation coefficients.

SCIP-G Screen for Cognitive Impairment in Psychiatry – German version.

PANSS Positive and Negative Syndrome Scale.

As to the SCL-90-R, the linear logistic regression analysis detected no significance between the symptom scores of the SCL-90-R and the SCIP-G (patients: F(9,20) = 0.347 and controls: F(9,20) = 0.447). Looking at specific predictors within the SCL-90-R (global severity index, positive symptom total, positive symptom distress index) and cognitive function as measured by the SCIP-G, no significant correlations (linear regression) in the patient and healthy control group were detected.

3.4. Correlations between cognitive impairment, quality of life and functional outcome

In patients, Pearson's correlations were calculated between the SCIP-G total scores and the WHOQOL-BREF total scores and subscales at visit 1 and visit 2. No significant correlations were detected as shown in Table 7.

Table 7.

SCIP-G in patients: Pearson's correlations with the WHOQOL-BREF and the PSP.

	Visit 1		Visit 2
WHOQOL-BREF	r	p	r	p
Physical health	−0.123	0.518	0.109	0.565
Psychological health	−0.179	0.345	0.182	0.335
Social relationships	−0.208	0.269	−0.172	0.365
Environment	−0.225	0.232	0.184	0.332
Total score	−0.211	0.263	0.132	0.488
PSP
Total score	−0.144	0.447	0.266	0.156

r Pearson's correlation coefficients.

SCIP-G Screen for Cognitive Impairment in Psychiatry – German version.

WHOQOL-BREF The World Health Organisation Quality of Life Assessment Instrument – Brief Form.

PSP Personal and Social Performance Scale.

Pearson's correlations were also assessed in patients between SCIP-G total scores and functional outcome measured by the PSP. No significant correlations were found between cognitive impairment and social functioning as shown in Table 7.

3.5. Depression and anxiety in patients and healthy controls

To evaluate the influence of anxiety on the SCIP-G, the State Trait Anxiety Inventory was used. The results of patients and healthy controls revealed no significant impact on cognitive functioning in the MANCOVA (State Anxiety: visit 1, SCIP-G form 1 F(1,55) = 0,36, p = .550; SCIP-G form 2 F(1,55) = 0.71, p = .402; visit 2, SCIP-G form 1 F(1,55) = 1.09, p = .299; SCIP-G form 2 F(1,55) = 0.715, p = .402. Trait Anxiety: visit 1, SCIP-G form 1 F(1,55) = 0.031, p = .580; SCIP-G form 2 F(1,55) = 0.038, p = .169; visit 2, SCIP-G form 1 F(1,55) = 1.940, p = .169 and SCIP-G form 2 F(1,55) = 0.466, p = .498).

To test the effect of depressive symptoms, the Beck Depression Scale was performed. The covariance analysis (MANCOVA) detected no significant results (visit 1, SCIP-G form 1 F(1,55) = 0.891, p = .349; SCIP-G form 2 F(1,55) = 0.07, p = .790; visit 2, SCIP-G form 1 F(1,55) = 0.028, p = .868; SCIP-G form 2 F(1,55) = 0.237, p = .628).

4. Discussion

The present study assessed the psychometric properties of the German version of the SCIP. The findings demonstrate the feasibility, reliability and validity of the SCIP-G applied to patients with schizophrenia and healthy controls. The alternative forms of the SCIP showed a satisfactory test-retest reliability. The internal consistency was also good with a Cronbach's Alpha of 0.73 for SCIP-G form 1 and 0.72 for SCIP form 2. Furthermore, the correlation of the SCIP-G with the German version of the BACS demonstrated that the two screening instruments measure similar cognitive domains.

In our study that compared patients with schizophrenia and schizoaffective disorders with healthy controls, the results of all five subscales and the total score were significantly different, showing that the SCIP–G is able to appropriately distinguish cognitively impaired patients from healthy controls. These findings stand in line with the results of the Spanish version of the SCIP (SCIP-S; Pino et al., 2008) in a sample of patients diagnosed with schizophrenia-spectrum disorder. The results and effect sizes (Cohen's d form 1 = 3.04 and Cohen's d form 2 = 2.68) in the German version showed stronger differences than in the Spanish version (Pino et al., 2008: Cohen's d = 1.41 and Rojo et al., 2010: Cohen's d = 2.02). The ROC-curves showed a good proportion between specificity and sensitivity of the SCIP (98,9% total score version 1, 97,2% total score version 2). These results are very similar to those of Rojo et al. (2010: 92,7%). Therefore, the SCIP-G is a useful tool for the screening of cognitive impairment in patients with schizophrenia. We consider the tool to be useful for screening, as the presence of a cognitive deficit is reliably revealed in a reasonably short time (15 min) and thus important decisions can be made with rational support in everyday in- and out-patient care: Should the hospital stay be prolonged because the cognitive dysfunction is so severe? Should specific cognitive remediation measures be initiated? Is a change in medication necessary?

The SCIP in its Spanish version also showed a differentiation between bipolar I patient and healthy controls. In patients with schizophrenia spectrum disorder, bipolar disorder and healthy controls, Rojo et al. (2010) calculated a sensitivity of 0.88 and a specificity of 0.89. These results indicate that the SCIP is able to differentiate between patients and individuals with specific impairments and those without cognitive impairment. The study by Gómez-Benito et al. (2013) reported diagnostic-specific standardization data for the SCIP for functional psychosis (schizophrenia and bipolar I disorder) and studied the impact of age and educational level on the SCIP. Tourjman et al. (2019) found that cognitive deficits assessed with the SCIP in its French version are associated with disease severity in major depressive disorder (MDD) patients, underlying that the SCIP can be easily used in routine clinical evaluation (Ott et al., 2021).

As to the parallel test-reliability of the total score, our data suggest a satisfactory test – retest reliability of the SCIP-G. Interestingly, for healthy probands, form 2 of the SCIP-G was easier to perform than form 1. A closer look to the raw data showed that the VFT of the SCIP-G was responsible for this. In form 1, words were searched for with C and L as first alphabetic character, in form 2 with P and W. These four letters were transferred without change from the original English version. As in German, words that begin with the character C are less frequent than words that begin with L, P or W, for healthy probands form 1 was more difficult than form 2. Consequently, in future, a different character should be chosen for form 1 of the SCIP-G in order to provide similar requirements for both forms. In contrast to the English version (Purdon, 2005), in the German version no practice effect could be detected. The choice of the letter C made it difficult or impossible to show a possible practice effect in the SCIP-G.

In the German version, the internal consistency Cronbach's Alpha = 0.73 for the SCIP form 1 in patients was exactly the same as the internal consistency reported by Pino et al. (2008). The internal consistency was similar for the SCIP-G form 2 (Alpha =. 72). With respect to the convergent validity, the correlations between the SCIP-G and the German version of the BACS demonstrated that the SCIP-G measures similar cognitive domains; the highest correlation was detected between the subtests in the patient group. The correlation between fluency in the SCIP-G form 1 and Word fluency form BACS is notably lower than the correlation for SCIP-G form 2 and Word fluency from BACS B. This finding can be explained by the fact that the use of the initial letter C in the SCIP-G form 1 generated insufficient responses. The low correlations between SCIP-G and the MMSE suggest that the MMSE is not suited to measure cognitive deficits in patients with schizophrenia (Cullen et al., 2007).

The SCIP-G did not show relevant correlations with the psychopathology assessed with the PANSS and the SCL-90-R in patients with schizophrenia and schizoaffective disorder. A detailed examination indicates that form 1 but not form 2 showed an unexpected correlation with positive symptoms. A repetition of this study with a higher number of subjects may reveal the random nature of this result.

No correlation was seen with regard to functional outcome and quality of life. This stands in contrast to the findings by Ueoka et al. (2011) and the metaanalysis by Tolman and Kurtz (2012) showing that cognitive impairment affects quality of life. As opposed to the correlations between cognitive deficits and functional outcome described by Green (2016), in the present study, patients with schizophrenia and schizoaffective disorder were functionally impaired, but no correlations were detected between cognitive deficits assessed by the SCIP-G and functional outcome as measured by the PSP. Further studies using the SCIP and different measures of functional outcome are suggested to illuminate this question.

In conclusion, the SCIP -including the SCIP-G- can easily be used for screening purposes in psychiatric patients and might play an increasing role in clinical practice. Future research should include the development of guidelines for the interpretation of scoring in relation to treatment results (Gómez-Benito et al., 2013); the SCIP might herewith be suitable for inclusion in multiple research protocols to assess the effectiveness of treatments in clinical trials.

5. Limitations and future research

The SCIP-G is a brief tool for the detection of cognitive impairment in psychiatry and was used in this study in patients with schizophrenia and schizoaffective disorder as compared to healthy controls. A satisfactory test – retest reliability of the SCIP-G total score was found. For the subtests the values were less consistent. A future study with a higher sample size and using a different character for the VFT of form 1 should address this limitation. In addition, an extension with a longer follow up is suggested to evaluate the stability and course of cognitive functions as measured with the SCIP-G. Furthermore, retest-reliability scores should be determined for the SCIP-G, which we did not calculate due to the cross over design of this study. The convergent validity was tested with the BACS and high correlations were found between the subtests except for the subtest “delayed verbal learning” due to the fact that a counterpart for this subtest in the BACS is not available.

In our study, patients were matched to controls on personal education. In a future, larger study it would seem more appropriate to match the samples on parental education to avoid a “super control” sample.

In summary, this is a preliminary study: in light of the small samples, the findings must be interpreted carefully.

CRediT authorship contribution statement

The authors wish to thank the participants for contributing their time and effort.

Declaration of competing interest

The authors declare no conflict of interest.