Validation of Chinese Version of Positive and Negative Syndrome Scale-6 in Clinical Setting: A Preliminary Study

Abstract

Background:

The Positive and Negative Syndrome Scale-6 (PANSS-6) is a brief measure to assess the core symptoms of schizophrenia. Psychometric characteristics of PANSS-6 in clinical settings are to be determined. We conducted this study among Chinese inpatients of schizophrenia in clinical settings, to determine psychometric characteristics of PANSS-6, and its’ accuracy for identifying antipsychotic efficacy.

Methods:

Two hundred sixteen inpatients of schizophrenia were interviewed at baseline, week 4 and week 8 by experienced psychiatrists to collect information for rating PANSS-30 and PANSS-6. Internal consistency was estimated by the Cronbach's α; criterion validity was determined by Spearman’s correlations between sum scores of PANSS-6 and PANSS-30; factorial validity was determined by confirmatory factor analysis (CFA); the sensitivity, specificity, positive predictive value (PPV); and negative predictive value (NPV) of PANSS-6 for identifying responders and remitters were calculated.

Results:

The Cronbach’s α coefficients of PANSS-6 were 0.72 (95% CI: 0.66-0.78) at baseline. Sum scores of PANSS-6 were significantly correlated with that of PANSS-30. Confirmatory factor analysis confirmed this 2-factor model fit well: χ²/df = 1.331, P = .223; CFI = 0.994; TLI = 0.988, RMSEA = 0.037 (90% CI: 0.000-0.090); SRMR = 0.033. Standard factor loadings of each item ranged from 0.60 to 0.89. At week 8, 92 (48.42%) and 63 (33.16%) inpatients were classified as responders and remitters. The sensitivity of PANSS-6 for identifying responders and remitters was 0.77 and 1.0, specificities were 0.84 and 0.86.

Conclusion:

Positive and Negative Syndrome Scale-6 is a sound scale for measuring psychotic severity and monitoring treatment outcomes of schizophrenia in clinical settings.

Main Points

• Positive and Negative Syndrome Scale-6 is a reliable and valid scale for psychotic symptom assessment in clinical settings.

• Positive and Negative Syndrome Scale-6 had a 2-factor structure that match the positive and negative symptom clusters of schizophrenia.

• Positive and Negative Syndrome Scale-6 can help to measure outcomes of antipsychotic treatment.

Introduction

Schizophrenia is a chronic, severe mental disorder. Well-established quantitative measures were recommended for the assessment of symptom severity and treatment outcomes of schizophrenia in clinical trials and practices.^1,2 Among rating scales for schizophrenia assessment, the 30 items Positive and Negative Syndrome Scale (PANSS-30) published in the 1980s is the most widely used tool for measuring psychotic severity.³

Due to the burdensome items, PANSS-30 is a time-consuming tool, which takes more than 1 hour to complete a full clinical interview for rating all 30 items. This limited the wide use of PANSS-30 in routine clinical practice. Several shorter and valid scales were developed based on PANSS-30, for example, PANSS-8, PANSS-14, and PANSS-19.^4-6 Another critical utility of PANSS-30 was monitoring antipsychotic outcomes. The previous study demonstrated improvements of unspecific symptoms (e.g., anxiety, depression, and other items in the general psychopathology subscale) could lead to a 20% decrease in the PANSS sum score.⁷ These unspecific symptoms are associated but do not represent the severity of schizophrenia. Therefore, the improvements of these symptoms could not reflect changes in core psychopathology after appropriate antipsychotic treatment. Outcome monitoring should mainly focus on core psychotic symptoms of schizophrenia.

Recently, Østergaard et al. has published a short version of PANSS-6 and validated its’ psychometric characteristics among acute and chronic hospitalized patients of schizophrenia and proved its’ accuracy in identifying treatment efficacy.^8-10 Collectively, accumulating evidence suggested that PANSS-6 is a brief and valid instrument for measuring “core” positive and negative symptoms of schizophrenia in clinical trials. It was also recommended as a quantitative measure for measurement-based care of schizophrenia in clinical settings.¹

However, the forementioned PANSS-6 studies were based on pooled data from different randomized controlled trials (RCTs).^9-11 Up to 80% of patients were excluded from RCTs.¹² Patients in the real-world have more symptomatology heterogeneity than participants in RCTs. Although recommended for clinical measuring, psychometric characteristics of PANSS-6 in “real-world” clinical practice remain unknown. Moreover, the factorial validity is still to be determined. We thus conducted this study among Chinese inpatients of schizophrenia in the clinical setting, to test psychometric characteristics of PANSS-6, and its’ accuracy of identifying responders and remitters after 8-week antipsychotics treatments.

Methods

Participants and Procedure

This study was reviewed and approved by the Medical Ethical Committee of Fourth People’s Hospital of Lianyungang (Approval no. 2019LYGSYYXLL-P02). The procedures were in accordance with the Helsinki declaration. In the present study, 216 inpatients were recruited from the forth people’s hospital of Liangyungang, Jiangsu, China through convenience sampling. The including criteria were as follow: (1) age ≥16 years; (2) newly hospitalized patients with a primary diagnosis of schizophrenia according to the criteria of International Classification of Diseases, the Tenth Revision (ICD-10). Inpatients having difficulty communicating with psychiatrists for any reason were excluded from the present study. All participants received an introductory description of this study and signed written informed consent before they participated in the study. Types and dosage of antipsychotics were determined by psychiatrists according to induvial characteristics.

Positive and Negative Syndrome Scale-6 Rating

The PANSS-6 was adapted from PANSS-30 by Østergaard et al.⁹ using an item response theory analysis. Positive and Negative Syndrome Scale-6 is a 7-point Likert scale including the following 6 items: delusions (P1), conceptual disorganization (P2), hallucinations (P3), blunted affect (N1), social withdrawal (N4), lack of spontaneity, and flow of conversation (N6). Each item has 7 response categories as follows: “1 Absent”, “2 Minimal”, “3 Mild”, “4 Moderate”, “5 Moderate severe”, “6 Severe”, “7 Extreme”. In this study, experienced psychiatrists completed clinical interviews at baseline, week 4 and week 8 to collect information for rating PANSS-30 and PANSS-6. The scores of PANSS-6 were extracted from that of PANSS-30.^9,10

Statistical Analysis

All statistical analyses were performed using psych (Revelle W., Evanston, Illinois, USA, https://CRAN.R-project.org/package=psych), lavaan (Rosseel Y., Ghent, Belgium, http://CRAN.R-project.org/package=lavaan), and semPlot (Epskamp S., Amsterdam, Netherlands, https://cran.r-project.org/web/packages/semPlot) packages of R 4.0.2, a free software environment for statistical computing and graphics (R Core Team, Vienna, Austria, https://www.Rproject.org/). For all statistical analyses, differences were considered significant at P < .05.

Firstly, mean, standard deviation (SD), number of patients with each response category, and corrected item-total correlation coefficients (CITCs) were computed as descriptive statistics for each item of PANSS-6. Corrected item-total correlation coefficients was suggested to be greater than 0.30.¹³

Secondly, the internal consistency with 95% CI of the PANSS-6 was estimated by the Cronbach's α coefficients at baseline, week 4 and week 8.

Thirdly, the criterion validity was determined by the correlation between PANSS-6 sum scores and PANSS-30 sum scores at baseline, week 4 and week 8, using Spearman's method. The correlations between PANSS-6 reductive rates and PANSS-30 reductive rates were also computed. The PANSS-30 was used as “golden standard” as this validated scale is the most widely used tool for evaluating psychotic symptoms.

Then, the factorial validity of PANSS-6 was determined using a confirmatory factor analysis (CFA). Robust maximum likelihood (MLR) was chosen as a model estimator in the analysis of ordinal data. The χ²/df, comparative fit index (CFI), Tucker-Lewis index (TLI), and root mean square error of approximation (RMSEA) with a 90% CI and standardized root mean residual (SRMR) were used to test the fit of the model. A good global fit is indicated for χ²/df < 3 (with P ≥ .05), CFI > 0.95, TLI > 0.95, RMSEA < 0.06 and SRMR < 0.08.¹⁴

Lastly, we tested the accuracy of PANSS-6 for identifying responders and remitters, after 8 weeks of antipsychotics treatments. Treatment response was defined as a reductive rate of PANSS at week 8 ≥ 50%.¹⁵ As response categories of PANSS-30 and PANSS-6 range from 1 to 7, a value of sum score subtracts item numbers of the scale was used for calculating the reductive rate. Reductive rate = (Sum score _baseline – Sum score _endpoint)/(Sum score _baseline – N).¹⁵ N represents the number of scale items. Symptomatic remission was defined as all of the following 8 items of PANSS were rated ≤ 3 at week 8: P1, P2, P3, N1, N4, N6, mannerisms or posturing (G5), and unusual thought content (G9)⁵. Positive and Negative Syndrome Scale-6 response criteria were defined as reductive rate ≥ 50% and PANSS-6 remission criteria were defined as all 6 items scored ≤ 3. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated.

Results

Demographics of Participants and Item Characteristics of PANSS-6

The mean age of participants was 41.76±12.83. One hundred eighteen inpatients were male (54.6%). Fifty-nine (27.31%) inpatients had educational experiences of less than 6 years, 117 (54.67%) had 6-12 years of educational experience and 40 (18.52%) had more than 12 years. The mean sum scores of PANSS-30 and PANSS-6 at baseline, week 4 and week 8 were demonstrated in Table 1. Item characteristics (Mean, SD, response numbers, skewness, kurtosis, and corrected item-total correlation) of PANSS-6 were demonstrated in Table 2. The item means at baseline ranged from 3.06 to 5.50, and CITCs ranged from 0.37 to 0.59.

Table 1.

Demographic Characteristics of Participants (n = 216)

Characteristics	Values
Gender, n (%)
Males	118 (54.6%)
Females	98 (45.4%)
Age, Mean (SD) in years	41.76 (12.83)
Education, n (%)
≤6 years	59 (27.31%)
6-12 years	117 (54.67%)
>12 years	40 (18.52%)
Treatment, mg, Median (Range)
Aripiprazole (n = 11)	20 (15-30)
Olanzapine (n = 107)	20 (10-30)
Quetiapine (n = 27)	700 (300-750)
Risperidone (n = 36)	5 (4-6)
Clozapine (n = 35)	400 (250-400)
Sum scores of PANSS-30, Mean (SD)
Baseline	94.82±10.93
Week 4	74.07±16.53
Week 8	64.19±13.89
Sum scores of PANSS-6, Mean (SD)
Baseline	23.96±3.72
Week 4	18.65±4.58
Week 8	15.55±4.07

Table 2.

Item Scores of PANSS-6 at Baseline of this Study

Items	Response Categories (n)							M (SD)	Skewness	Kurtosis	CITC
	1	2	3	4	5	6	7
P1	0	1	7	31	56	85	36	5.50 (1.06)	-0.53	2.85	0.59
P2	1	7	57	88	54	9	0	3.99 (0.92)	-0.05	2.86	0.44
P3	0	2	45	46	88	27	8	4.54 (1.09)	0.02	2.47	0.40
N1	4	35	90	77	9	1	0	3.25 (0.86)	-0.12	3.03	0.49
N4	3	18	83	79	26	7	0	3.59 (0.96)	0.17	3.24	0.45
N6	4	48	98	62	3	1	0	3.06 (0.82)	0.02	3.02	0.37

CITC, corrected item-total^a correlation. ^atotal refers to sum score of PANSS-6.

Internal Consistency of PANSS-6

The Cronbach’s α coefficients of PANSS-6 were 0.72 (95% CI: 0.66-0.78) at baseline, 0.77 (95% CI: 0.72-0.82) at week 4, and 0.77 (95% CI: 0.72-0.82) at week 8. While the Cronbach's α coefficients of PANSS-30 were 0.72 (95% CI: 0.67-0.77) at baseline, 0.89 (95% CI: 0.87-0.91), and 0.88 (95% CI: 0.86-0.90).

Criterion Validity of PANSS-6

Positive and Negative Syndrome Scale-6 sum scores were significantly correlated with and PANSS-30 sum scores at baseline, week 4, and week 8, with correlation coefficients as 0.72, 0.83, and 0.87 (all P < .001). The reductive rate of PANSS-6 at week 4 and week 8 was also significantly correlated with that of PANSS-30 (r_week4 = 0.84, r_week8 = 0.82, all P < .001).

Factorial Validity of PANSS-6

The theoretical structure of PANSS-6 consists of 2 factors: the first factor includes all items measuring positive items (P1, P2, and P3), while the second factor includes all negative symptoms (N1, N4, and N6) in PANSS-6. Confirmatory factor analysis result suggested this 2-factor structure model of PANSS-6 could be acceptable, with fit indices as follow: χ²/df = 1.331, P = .223; CFI = 0.994; TLI = 0.988, RMSEA = 0.037 (90% CI: 0.000-0.090); SRMR = 0.033. Standard factor loadings of each item ranged from 0.60 to 0.89 (Figure 1).

Figure 1.

Path diagram of the 2-factor structure of the PANSS-6. The model was estimated using MLR. χ²/df = 1.331, P = .223; CFI = 0.994; TLI = 0.988, RMSEA = 0.037 (90% CI: 0.000–0.090); SRMR = 0.033. Pos = positive subscale; Neg = negative subscale. P1 = delusions, P2 = conceptual disorganization, P3 = hallucinations, N1 = blunted affect, N4 = social withdrawal, N6 = lack of spontaneity and flow of conversation.

Accuracy of Using PANSS-6 for Identifying Responders and Remitters

One hundred ninety inpatients (87.96%) had PANSS scores at week 8 for this analysis, 92 (48.42%) inpatients were identified as responders and 63 (33.16%) were remitters. The sensitivity, specificity, PPV, and NPV using PANSS-6 to identify responders and remitters are listed in Table 4. The sensitivity and NPV of remission were 1.0 because the symptom remission criteria include 6 items of PANSS-6.

Table 4.

Sensitivity, Specificity, Positive Predictive Value (PPV), and Negative Predictive Value (NPV) of PANSS-6 for Identifying Response and Remission at Week 8

	Sensitivity	Specificity	PPV	NPV
Response	0.77	0.84	0.82	0.80
Remission	1.0	0.86	0.78	1.0

Discussions

The primary aim of this study was to validate the psychometric characteristics of PANSS-6 in “real world” clinical settings. The major finding of this study is that PANSS-6 had good internal consistency, criterion validity, and factorial validity among Chinese schizophrenia in patients in clinical settings. The present study also found that the reduced rate of PANSS-6 at different time points was highly correlated with that of PANSS-30, and the PANSS-6 could accurately identify responders and remitters after 8 weeks of antipsychotic treatment, in clinical management of schizophrenia.

Previous studies validated PANSS-6 among participants included in RCTs.^9-11 Commonly, RCTs are recommended as the highest level of evidence in the hierarchy of research designs. Numerous RTCs have demonstrated the efficacy and safety of antipsychotics.^16-18 Like in other medical fields, data from RTCs could not directly or completely answer all questions in the clinical management of schizophrenia.¹⁹ Because of selection bias, participants included in RCTs could not represent the majority of the patient population.¹² This might increase the difficulty of translating RCTs evidence into clinical practice. Data extracted from real-world sources could therefore provide important complementary of RCTs.²⁰ Accumulating real-world evidence (RWE) helps to further understand the outcomes of treatments in routine, daily psychiatric practice.^21-24

For now, quantitative measures are still the major indicator of symptomatologic severity in psychiatric practice.²⁵ Psychometric properties, especially parameters estimated by classic test theory, depending on the population included in individual study.²⁶ Although validated in RCTs, the psychometric properties of PANSS-6 are still to be determined among real-world patients of schizophrenia. In this sense, the psychometric properties of PANSS-6 in the clinical population of schizophrenia make it a sound tool for RWE studies and clinical practice.

The present study also demonstrated the 2-factor structural of PANSS-6 for the first time. Three items in positive subscales cover 3 of 4 “core” symptoms of schizophrenia in ICD-10 and all 3 essential symptoms in DSM-5. Meanwhile, although negative symptoms are not “essential” in diagnostic criteria, it is still viewed as a core domain in the concept of schizophrenia.²⁷ The 2-factor structure of PANSS-6 is consistent with symptom dichotomy in schizophrenia, which makes it more suitable for measuring the core severity of schizophrenia in comparison with the drawn-out PANSS-30.

PANSS-6 is a quantitative measure according to the conceptional changes of schizophrenia. In PANSS-30, delusion is measured by 4 items (P1, P5, P6, and G9) taking the delusion types into accounting. Unlike that, PANSS-6 uses only 1 item (P1) for measuring delusion, regardless of specific types. In the latest diagnostic criteria of schizophrenia in the DSM-5 and ICD-11, one critical revision is that different types of hallucinations and delusions have the same diagnostic values. This change is based on the low specificity of first-rank hallucinatory voices and bizarre delusions for identifying schizophrenia.²⁸

Several major limitations should be noted in this study. First and foremost, no specific interview for PANSS-6 rating was involved in this study. PANSS-6 was rated according to information collected from a clinical interview, which was used for the PANSS-30 rating. The influence of unspecific items in this interview on PANSS-6 rating was unclear now. Second, inter-rater reliability of PANSS-6 has not been evaluated in this study, this was mainly due to the lack of specific structure interview for PANSS-6 rating. Recently, the Simplified Negative and Positive Symptoms Interview (SNAPSI) for rating.⁸ The study has indicated a good level of inter-rater reliability, after minimal training in the use of the SNAPSI for PANSS-6 rating⁸. We have acquired authorization from the authors to translate SNAPSI into Chinese and will validate it among Chinese patients of schizophrenia. Third, PANSS-6 is focused on core psychotic symptoms of schizophrenia. However, the comprehensive status of patients should be assessed in clinical settings. DSM-5, therefore, recommended a multiple-domain method of schizophrenia severity measuring, which consists of hallucinations, delusions, disorganized speech, abnormal psychomotor behavior, negative symptoms, cognitive impairment, depression and mania, and function. This could help to figure out the psychopathological panorama of schizophrenia patients and enable monitoring treatment efficacies in different aspects of the disorder. Therefore, scales especially measuring cognitive, emotional, and functional symptoms should be involved in the comprehensive assessment of schizophrenia in clinical settings. Finally, this study involved both acute and chronic schizophrenia inpatients and did not differentiate them in statistical analyses. Future studies should determine psychometric characteristics among acute and chronic inpatients separately.

In conclusion, this psychometric study demonstrated PANSS has good reliability and validity for measuring core symptomatic severity among Chinese patients of schizophrenia in clinical practice. Using PANSS-6, psychiatrists could conveniently assess treatment efficacy after antipsychotic treatment. In the future, the psychometric properties of PANSS-6 should be reverified among Chinese patients using a Chinese version of SNAPSI for information collecting.

Data Availability Statement:

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Table 3.

Correlation Matrix of PANSS-6

	P1	P2	P3	N1	N4	N6
P1	1	-	-	-	-	-
P2	0.601	1	-	-	-	-
P3	0.532	0.436	1	-	-	-
N1	0.245	0.161	0.085	1	-	-
N4	0.261	0.126	0.152	0.613	1	-
N6	0.189	0.049	0.077	0.578	0.427	1

Funding Statement

This study was supported by the Jiangsu Hospital Association, China (JSYGY-3-2019-169); and the Health Commission of Lianyungang City, Jiangsu Province, China (201928).