Which Rating Scales are Regarded as ‘The Standard’ in Clinical Trials for Schizophrenia? A Critical Review

Abstract

Background

This paper reviewed which rating scales past studies adopted as an outcome measure in clinical trials for schizophrenia, for which a consensus has been lacking.

Methods

A PubMed search was conducted using keywords ‘outcome’, ‘rating scales’, and ‘schizophrenia’. Studies published in 1999, 2004 and 2009 were examined to globally see if a trend has changed over the last decade.

Results

One-hundred fifty articles were inspected. As for psychopathology, the positive and negative syndrome scale (PANSS) has been by far the most frequently utilized scale (46%, 79%, and 78% in the respective years), followed by the brief psychiatric rating scale. Affective/anxiety symptoms have been only rarely recorded. Extrapyramidal symptoms have been assessed mostly with the Simpson Angus scale (SAS), more frequently in combination with the abnormal involuntary movement scale (AIMS) and Barnes akathisia scale (BARS) recently. Non-motor adverse effects have been typically reported without a usage of formal rating scales. Depending on the interest of investigation, other critical domains of the illness including functioning, cognition and subjective perspectives have been sporadically reported through the rating scales. The assessment scales were similarly utilized across the years, except for a numerical rise in scale utilization to rate the latter three domains in 2009.

Conclusions

The PANSS and set of AIMS, BARS and SAS, which are expected to take about 60 minutes to complete, are frequently utilized and may be regarded as a ‘standard’ in clinical trials for schizophrenia. Clinical implication of the findings and practical challenges with the existing scales are discussed.

Introduction

In an effort to trace the effect of interventions and facilitate measurement-based treatment, it has been a convention to quantify symptoms in schizophrenia with the rating scales. In fact, these assessment scales have constituted the primary outcome in many studies for schizophrenia. However, symptoms and problems of the patients in the real world are widespread and are not confined to classical positive/negative symptoms, but extend well to affective/anxiety symptoms, adverse effects of medications, functional restrictions, and cognitive difficulties and to limitations in subjective domains including well-being and quality of life.¹

As such, there has been a huge list of assessment scales² and it has not been straightforward on which rating scales make up an adequate but a practical assessment in a time-limited encounter.³ Further, a possible change in the trend for the assessment scales adopted has not been reported to the best of the author’s knowledge. In this manuscript, past clinical studies for schizophrenia were reviewed, first in order to have a sense on the rating scales that have constituted ‘the standard’ over the last decade, then to critically discuss challenges with them.

Methods

A PubMed search (July 2010) was conducted using keywords ‘outcome’, ‘rating scales’, and ‘schizophrenia’. Three years, each five years apart (1999, 2004 and 2009), were examined to globally see a trend in the assessment scales used over the last decade. The search date was restricted to the respective years (e.g., from 01 Jan. 2009 to 31 Dec. 2009 for the year 2009). Also, limits were set with ‘Clinical Trial’ and ‘English’. Reviews and meta-analyses, and studies without data on the rating scales or with a main focus on other diagnoses than schizophrenia and related psychotic disorders, were excluded.

Each study was investigated for which rating scales it used as an outcome measure and the frequency of rating scales utilized was examined. The rating scales recorded in ≥10% of the studies were targeted. The following situations were counted as positive for scale utilization—different versions of the scale was used (e.g., 18- versus 24-item version of the brief psychiatric rating scale: BPRS⁴), only a part of the scale was utilized (e.g., positive subscale of the positive and negative syndrome scale: PANSS⁵), or only a subset of the subject was evaluated with the scales.

Outcomes were arbitrarily classified to nine categories as follows: Global evaluation, Classical psychopathology (i.e., positive and negative symptoms), Affective/Anxiety symptoms, Extrapyramidal symptoms (EPS), Non-motor adverse effects, Functioning, Subjective perspectives (e.g., quality of life: QOL, well-being, preference of medications and awareness of the illness), Cognition, and Others. Since cognitive assessments usually consist of multiple tests (and as a computerized battery at times), they are dichotomized as either yes or no for the purpose of this paper. Fisher’s exact test was used to assess the differences in frequency of scale utilization across the years, and a p-value of <0.05 was considered significant (two-tailed).

Results

Overview

A PubMed search yielded 36, 42 and 84 hits in the years 1999, 2004 and 2009, respectively. After excluding ineligible papers, 150 studies (93%) (35, 38 and 77 articles, respectively) that described the results with the rating scales in clinical trials for schizophrenia were examined. Tables 1–3 describe the frequency of rating scales utilized in ≥10% of the occasions for each domain in the respective years.

Table 1. Rating Scales Used in Clinical Trials for Schizophrenia in 1999 (35 Studies).

DOMAINS	SCALES	PERCENTAGE UTILIZED
Global evaluation	ANY scales	31%
	CGI	31%
Classical psychopathology	ANY scales	97%
	PANSS	46%
	BPRS	54%
	SANS	34%
	SAPS	17%
Affective/Anxiety symptoms	ANY scales	20%
	HRSD	11%
EPS	ANY scales	57%
	SAS	40%
	AIMS	26%
	BARS	23%
Non-motor A/Es	ANY scales	3%
Functioning	ANY scales	29%
	GAF/GAS	26%
Subjective perspectives	ANY scales	29%
Cognition	ANY scales	11%
Others	ANY scales	20%

Abbreviations: A/Es adverse effects; AIMS abnormal involuntary movement scale; BARS Barnes akathisia rating scale, BPRS brief psychiatric rating scale; CGI clinical global impression; EPS extrapyramidal symptoms; GAF global assessment of functioning; GAS global assessment scale; HRSD Hamilton rating scale for depression; PANSS positive and negative syndrome scale; SANS scale for the assessment of negative symptoms; SAPS; scale for the assessment of positive symptoms; SAS Simpson-Angus scale.

Table 3. Rating Scales Used in Clinical Trials for Schizophrenia in 2009 (77 Studies).

DOMAINS	SCALES	PERCENTAGE UTILIZED
Global evaluation	ANY scales	42%
	CGI	42%
Classical psychopathology	ANY scales	94%
	PANSS	78%
	BPRS	17%
Affective/Anxiety symptoms	ANY scales	27%
	CDSS	18%
EPS	ANY scales	39%
	SAS	25%
	AIMS	19%
	BARS	18%
Non-motor A/Es	ANY scales	12%
Functioning	ANY scales	38%
	GAF/GAS	27%
Subjective perspectives	ANY scales	38%
	QLS	12%
Cognition	ANY scales	30%
Others	ANY scales	29%

Abbreviations (also see Tables 1 and 2): CDSS Calgary depression scale for schizophrenia; QLS quality of life scale.

Overall, the frequency in these assessment scales was similar across the years, except for a numerical rise in usage of the scales for cognition, functioning and subjective perspectives in 2009. Specifically, functioning was more frequently recorded in 2009 compared with 2004 (p < 0.001), and cognitive assessments were more likely reported in 2009 (p < 0.05 in comparison with 2004).

Global Evaluation

The clinical global impression: CGI⁶ has been the sole scale used as a global measure. It simply evaluates the severity of illness (normal:1 to moderate:4 to most ill:7) as well as change (very much improvement: 1 to no change:4 to very much worsening:7) with a score of 1–7. No other global evaluation scales for severity and change have been utilized.

Classical Psychopathology (Positive and Negative Symptoms)

As expected, almost all of the studies reported on this aspect with a usage of the rating scales. The PANSS (30-item—7 for positive, 7 for negative and 16 for general psychopathology subscales) has been by far the most frequently utilized scale for this purpose. It was followed by the BPRS (typically 18-item version), which outnumbered the PANSS in 1999, and was sometimes extracted from the PANSS (as 18-item version).

The next common scale was the scale for the assessment of negative symptoms: SANS⁷ (20 symptom items plus five global items) and the scale for the assessment of positive symptoms: SAPS⁸ (30 symptom items plus four global items). They were rated altogether at times (9 of 150 studies). Sometimes, the SANS was assessed together with the PANSS (8 of 150 studies).

Affective/Anxiety Symptoms

Although the frequency in usage was rather low, the Hamilton rating scale for depression: HRSD⁹ (typically 17 items), and more recently the Calgary depression rating scale for schizophrenia: CDSS¹⁰ (nine items) have been the most frequently recorded scale. In contrast, subjective scales for depression have very rarely been utilized. Rating scales for anxiety symptoms, both objective and subjective, have been barely used.

Extrapyramidal Symptoms

It has been typical to assess parkinsonism with the Simpson–Angus scale: SAS¹¹ (10 items), tardive movement disorders with the abnormal involuntary movement scale: AIMS¹² (10 items plus two dental status items), akathisia with the Barnes akathisia rating scale: BARS¹³ (four items). These three scales were frequently rated altogether. In fact, if any one of these scales was assessed, both of the rest were also evaluated in 40% of the cases overall (and as high as 74% in 2009). The extrapyramidal symptom rating scale: ESRS¹⁴ (41 items plus 4 CGIs’ for akathisia, dyskinesia, dystonia and parkinsonism) has been used much less frequently.

Non-Motor Adverse Effects

The Udvalg for Kliniske Undersogelser: UKU side effect rating scale¹⁵ (48 items plus interference and action items, eight of which evaluates neurologic adverse effects) has been the only scale that was utilized in 13% of the studies in 2004. In 1999, merely one of the 35 studies evaluated non-motor adverse effects with the rating scale (UKU). On the other hand, the frequency of treatment-emergent (motor and non-motor) adverse effects has been occasionally described in tables (spontaneously reported or observed, but without a usage of the formal scales).

Functioning

The global assessment of functioning: GAF¹⁶ and its precedent global assessment scale: GAS¹⁷ have been the most frequently utilized scale. They simply rate the global status with a score of 0–100. Performance-based functional scales have been very rarely utilized.

Subjective Perspectives

While some of the studies recorded this domain with a usage of the rating scales, only the quality of life scale: QLS¹⁸ (21 items) has been used in 12% of the 2009 studies.

Cognition

While some of the cognitive assessments were performed in only 11% in the years 1999 and 2004, they were rated in 30% of the studies in 2009. And the assessments used showed much more variety in 2009 expanding from classical paper-pencil test to computerized facial emotion recognition test to multiple tests that are expressed in the context of a composite cognitive score.

Others

In some of the studies, assessments were extended to premorbid adjustment, disability, comorbid substance use, prognostic evaluation, caregivers’ perspectives and aggression and so forth. However, none of the scales have been utilized for ≥10% of the studies in any of the respective years.

Discussion

It was found that clinical trials in schizophrenia are likely to utilize the PANSS for psychopathology as well as the set of AIMS, BARS and SAS for EPS assessment. Overall frequency in the assessment scales for schizophrenia in an effort to evaluate multiple domains within the illness appeared to be similar across years, except for a more recent attention on cognition, functioning and subjective perspectives. The PANSS together with the set of AIMS, BARS and SAS may be regarded as ‘the standard’ in clinical trials for schizophrenia. This ‘standard’ set of assessment scales is expected to take about 60 minutes (30–40/5–10/10/10 minutes for the PANSS/AIMS/BARS/SAS, respectively).² Such a time requirement obviously represents an obstacle for the real-world practice.

Studies have utilized different scales for their different interests and we can not be entirely certain about which scales are adequate in a specific study. It is important to acknowledge that all assessment scales do have some pertinence across multiple illness domains. Furthermore, contrary to the naming, the QLS for example was designed to assess deficit symptoms and can well be regarded as a functional outcome measure in schizophrenia. On the other hand, an interpretation of clinical relevance on improvements in a part of the scales, or in subscales within the scale, remains somewhat complex although such a data presentation is sometimes found to focus on statistically significant differences.

Limitations of this paper include a limited number of years and studies investigated, and an arbitrary classification of outcomes into domains. Outcome measures may be in part guided by the nature of the study (e.g., pharmacologic versus psychosocial) or the setting (e.g., real-world versus research). The year of publication (or tradition) of each rating scale is also an important factor, since rating scales would require some time to become familiarized with (and years that accompany with the citations serve to have a sense on the ‘age’ of the scales). Challenges with the existing rating scales are discussed below.

Symptoms

The PANSS has been ‘the standard’ scale and frequently adopted as the primary outcome measure in clinical studies for schizophrenia. It is reasonable to assume the more number of items (and the wider the potential score distribution) in a scale, the more likely one would be able to detect a difference but at the cost of time. In order to discern any difference, it might be better to rate as many scales as possible (e.g., all of the PANSS, BPRS, SAPS, and SANS), which nonetheless would be unrealistically time-consuming and in fact has been a case for none of the 150 studies investigated herein.

Redundancy within/across the scales is also of concern. For instance, factor analyses of the PANSS have identified several components¹⁹ and as such, rating these extracted factors instead of all 30 items might even be sufficient. In line with this view, efforts are ongoing to make simpler rating scales. For instance, The Clinical Global Impression-Schizophrenia scale: CGI-SCH²⁰ is akin to the CGI but it consists of four common symptomatic aspects (i.e., positive, negative, depressive and cognitive symptoms) in addition to global severity/change. This scale has been used in a series of naturalistic, observational investigations (The European Schizophrenia Outpatient Health Outcomes (SOHO) study).²¹

Affective and anxiety symptoms have not been usually assessed with the rating scale in spite of a relatively high prevalence reported in the literature,²² although it might still be possible to capture these problems with such items as depression, guilt feelings, anxiety, and tension in the PANSS for instance. The Montgomery- Åsberg depression rating scale: MADRS²³ (10 items) has not been so frequently utilized in this population, and none of the anxiety scales have been commonly used. Another important issue is to evaluate usefulness of subjective scales for mood and anxiety in patients with schizophrenia.

Adverse Effects

This domain mainly consists of two parts—EPS and non-EPS adverse effects. As for the former, the combination of the AIMS (tardive involuntary movement), BARS (akathisia) and SAS (parkinsonism) appears to have been ‘a standard’. The ESRS has been much less commonly used although comprehensive, well presumably due to a lengthy process to complete. This topic has recently been reviewed in detail elsewhere.²⁴ A major challenge here is how to interpret different scales with different item numbers as ‘an overall EPS burden’ within the subject in question. The drug-induced extrapyramidal symptoms scale: DIEPSS²⁵ (nine items) might represent a useful alternative.

Non-motor adverse effects, including anticholinergic, metabolic, autonomic and sexual problems, have been usually described without a usage of the scales and sometimes reported in table, for which standardization is warranted.²⁶ Evaluations that depend on spontaneous report may result in underestimation. Although the UKU side effect rating scale has been occasionally used, it still lacks some crucial elements such as metabolic parameters, for which a significant impact in this population is noted²⁷ and a regular monitoring is recommended.²⁸ This may be due to the fact that it was published before newer antipsychotic medications, which are relatively more problematic in terms of metabolic disturbances in general, have become widely available in the market.

An obvious difficulty for the scales with multiple items is an inherent non-comparability of the same total score for differently endorsed problems (e.g., possibly non-uniform implication of moderate severity in metabolic versus sexual adverse effects) as well as a validity and pragmatic usefulness in severity differentials (e.g., 0–3 (none, mild, moderate, and severe) versus more detailed 0–5 (none, equivocal, mild, moderate, marked and severe)).

Functioning

The GAS and GAF have been ‘a standard’ to assess global functioning. The social and occupational functioning scale: SOFAS,¹⁶ which is a functional derivative of the GAF, has been described only sporadically. They are indeed quite simple and user-friendly but might be too simple to capture functional status. Ongoing efforts in this respect include the personal and social performance scale: PSP²⁹ that modeled the SOFAS, and the functional assessment for comprehensive treatment in schizophrenia: FACT-Sz³⁰ that is similar to the GAF but more detailed and more widely differentiates patients.

It has however been infrequent that functional scales, in contrast to symptomatic rating scales, have constituted the primary outcome measure in studies for schizophrenia,^31-34 although global functioning appears to serve as a heuristic outcome that may represent ‘the net effect of everything’ in patients. Some studies on child and adolescent schizophrenia used the Children’s version of the GAS.³⁵ On the other hand, global functioning scales specific for geriatric patients have not been reported, which would be all the more pertinent in light of a recent aging society. However, a complexity is how to define a ‘norm (or normal trajectory)’ with which any abnormality or deviance is compared in an aged population (Suzuki et al., submitted).

Performance-based scales, such as the UCSD Performance-Based Skills Assessment: UPSA,³⁶ have not gathered much popularity thus far. This might be a result from unfamiliarity of the performance-based scales despite a claim for their potential usefulness,³⁷ but another important consideration is ‘what patients are actually doing’ versus ‘what patients can potentially do’. Performance-based assessment scales may more closely reflect the latter under a probable impact of cognitive capability, while other aspects such as motivation and mood may be more implicated in the former.^38,39

Subjective Perspectives

This domain consists of multiple elements, ranging from well-being and quality of life to preference of medications to awareness of the illness. Representative rating scales would include the subjective well-being under neuroleptics: SWN⁴⁰ (short version of 20 items), drug attitude inventory: DAI⁴¹ (short version of 10 items), MOS 36-item short-form health survey: SF-36⁴² or its shorter 12-item version: SF-12,⁴³ the WHO Quality of Life-BREF: WHOQOL-BREF⁴⁴ (26 items), and the Scale to Assess Unawareness of Mental Disorder: SUMD⁴⁵ (20 items), although they have not been commonly utilized (only in <10% of the studies).

In practice, it is possible that patients with substantial symptoms and impairments may regard their lives as ‘good’.⁴⁶ In contrast, a report indicating that psychosocial functioning was negatively associated with self-esteem and satisfaction with life⁴⁷ might not argue against a possibility that patients with relatively higher functioning could face to a reality, to recognize a hardship in life. As such, a difficulty in measuring QOL in this population makes some standardized evaluations complicated (e.g., QALY).⁴⁸ It is also important to be aware of strength and weakness of subjectively completed versus objectively assessed rating scales regarding this issue.⁴⁹ A QOL in patients is a complex construct⁵⁰ and therefore, these scales may have a room for improvement to more nicely represent subjectivity on the side of patients.⁵¹

Cognition

Different studies have utilized different assessment measures. The mini-mental state examination: MMSE⁵² (30 points from seven categories), while time-friendly, has not been widely utilized and might be somewhat rough to evaluate cognition in schizophrenia. This topic is extensively reviewed elsewhere (e.g., the National Institute of Mental Health’s Measurement and Treatment Research to Improve Cognition in Schizophrenia: MATRICS⁵³). The MATRICS battery consists of 10 tests that represent seven cognitive domains and the time for completion is estimated to be about 65 minutes.

Other briefer scales include the Repeatable Battery for the Assessment of Neuropsychological Status: RBANS⁵⁴ that is reported to take <30 minutes, and the brief assessment of cognition in schizophrenia: BACS⁵⁵ that needs <35 minutes to evaluate. Important issues, apart from time burden, are whether stability (as an intermediate or endophenotype) versus changeability in cognition is to be assumed, and how a change in cognitive test scores translates into actual outcome in other domains of the illness. Also, it would be useful to have a concept of ‘responder’ (or treatment-resistance) as has been defined with a ≥20% decrease in the PANSS score and so forth (Suzuki et al., revision submitted).

General Note

Caveats in rating the PANSS are commented on since it has been the standard scale amongst others. Double-blind studies have offered the most solid evidence, whereby independent raters assess the patients at baseline and typically the same raters follow the same patients throughout. If one wishes to maintain true blindness, every assessment can be performed by the different rater, which obviously poses two major problems—feasibility (to assure adequate number of raters) and reliability among raters.

Therefore, two possibilities in a typical study should be noted as confounding factors in quantification with the scales. First, the result of baseline assessment will have a significant impact for later assessments. As for a rater effect at the very baseline, it is reported that a psychiatrist who saw a patient for the first time underrated the PANSS scores by 10%, compared with the ones obtained by the psychiatrist in charge who has known that patient very well.⁵⁶ Second, if a better psychological interaction between patients and assessors happens with more encounters, patients may feel less guarded to express themselves more frankly (for instance for their hidden delusions).

Contrarily, another possibility is assessors get psychologically accustomed to patients, which might not necessarily result in more severity in scoring (in lieu of a possible increase in identifiable symptoms). These issues are expected to affect rater drift within the rater across longitudinal assessments. Use of performance-based, objective rating scales could overcome these issues but they are mostly applicable to cognitive measurements in general and a part of functional scales. As such, although rater effect and rater drift issues have rarely been the target of studies, more work is clearly indicated for the purpose of better ‘quantification’ with the rating scales.

Finally, given various needs in patients with schizophrenia, it might be appropriate to make use of the scales that are miscellaneous in nature. Examples are the targeted inventory on problems in schizophrenia: TIP-Sz³⁰ (10 items) and the Investigator’s assessment questionnaire: IAQ⁵⁷ (10 items). On the other hand, apart from more time requirement and a possibility that patients may not tolerate lengthy assessments, use of multiple scales renders summarizing the data more challenging. In this context, separate reporting of the parent study is common, although tracing the studies is sometimes complicating.

The author recommends that global functioning should always be reported with a simple scale since it could represent the most proximal effects of various distal elements in the illness. More work is necessary on ‘subjectivity’ regarding the subjective assessment scales in patients with schizophrenia. Further, it would be useful to have the scale that is comprehensive for both motor plus non-motor adverse effects.

Conclusion

Clinical trials in schizophrenia thus far have frequently been using the PANSS for psychopathology as well as the set of AIMS, BARS and SAS for EPS assessment, all of which may take an hour to complete. While this assessment set could be considered as ‘a gold standard’, more efforts are obviously indicated on how best to successfully and quantitatively assess patients with the rating scales, taking into account various issues patients face in reality together with study interests and realistic feasibility.

Table 2. Rating Scales Used in Clinical Trials for Schizophrenia in 2004 (38 Studies).

DOMAINS	SCALES	PERCENTAGE UTILIZED
Global evaluation	ANY scales	50%
	CGI	50%
Classical psychopathology	ANY scales	95%
	PANSS	79%
	BPRS	29%
Affective/Anxiety symptoms	ANY scales	32%
	HRSD	11%
EPS	ANY scales	50%
	SAS	29%
	AIMS	29%
	BARS	16%
	ESRS	13%
Non-motor A/Es	ANY scales	13%
	UKU	13%
Functioning	ANY scales	13%
	GAF/GAS	11%
Subjective perspectives	ANY scales	24%
Cognition	ANY scales	11%
Others	ANY scales	21%

Abbreviations (also see Table 1): ESRS extrapyramidal symptom rating scale; UKU Udvalg for Kliniske Undersogelser side effect rating scale.