Skip to main content
NCBI home page
International Journal of Methods in Psychiatric Research logoLink to International Journal of Methods in Psychiatric Research
. 2015 Jul 14;25(1):3–11. doi: 10.1002/mpr.1476

Challenging the understanding of significant improvement and outcome in schizophrenia – the concept of reliable and clinically significant change methods

Rebecca Schennach 1,2,✉, Hans‐Jürgen Möller 1, Michael Obermeier 1, Florian Seemüller 1, Markus Jäger 1, Max Schmauss 3, Gerd Laux 4, Herbert Pfeiffer 5, Dieter Naber 6, Lutz G Schmidt 7, Wolfgang Gaebel 8, Joachim Klosterkötter 9, Isabella Heuser 10, Wolfgang Maier 11, Matthias R Lemke 12, Eckart Rüther 13, Stefan Klingberg 14, Markus Gastpar 15, Richard Musil 1, Ilja Spellmann 1, Michael Riedel 1,16
PMCID: PMC6877263  PMID: 26178421

Abstract

Significant changes of schizophrenia patients during inpatient treatment were evalutaed and compared to established outcome criteria. The concept of reliable and clinically significant change methods was applied to three hundred and ninety‐six patients suffering from a schizophrenia spectrum disorder. First, information on whether or not the change of the patient's condition is sufficient in order to declare that it is beyond a measurement error or random effect (= reliable change) was evaluated and in a second step it was observed if the reliable change was clinically meaningful (= clinically significant change). Different Positive and Negative Syndrome Scale for Schizophrenia (PANSS) thresholds were applied to define the clinically significant change (40, 45 and 50 points). These changes were then compared to established outcome criteria such as response and remission. Seventy‐nine of the 396 patients (20%) showed a reliable improvement of symptoms, whereas 70% improved without achieving a reliable change of their condition. Of the 79 patients achieving a reliable change during treatment 8‐15% concurrently showed a clinically significant change depending on the respective PANSS threshold. In contrast, 56% of the patients achieved response and 60% were in remission at discharge when applying established outcome criteria. Our results showed that a rather small number of schizophrenia patients were found to reliably change during inpatient treatment, with even less patients achieving a clinically significant change. The concept of reliable and clinically significant changes revealed to be a lot more stringent than today's established outcome criteria and should be critically evaluated regarding its use in schizophrenia patients. Copyright © 2015 John Wiley & Sons, Ltd.

Keywords: schizophrenia, reliable change, clinically significant change, outcome

Short abstract

Significant changes of schizophrenia patients during inpatient treatment were evalutaed and compared to established outcome criteria. The concept of reliable and clinically significant change methods was applied to three hundred and ninety‐six patients suffering from a schizophrenia spectrum disorder. First, information on whether or not the change of the patient's condition is sufficient in order to declare that it is beyond a measurement error or random effect (= reliable change) was evaluated and in a second step it was observed if the reliable change was clinically meaningful (= clinically significant change). Different Positive and Negative Syndrome Scale for Schizophrenia (PANSS) thresholds were applied to define the clinically significant change (40, 45 and 50 points). These changes were then compared to established outcome criteria such as response and remission. Seventy‐nine of the 396 patients (20%) showed a reliable improvement of symptoms, whereas 70% improved without achieving a reliable change of their condition. Of the 79 patients achieving a reliable change during treatment 8‐15% concurrently showed a clinically significant change depending on the respective PANSS threshold. In contrast, 56% of the patients achieved response and 60% were in remission at discharge when applying established outcome criteria. Our results showed that a rather small number of schizophrenia patients were found to reliably change during inpatient treatment, with even less patients achieving a clinically significant change. The concept of reliable and clinically significant changes revealed to be a lot more stringent than today's established outcome criteria and should be critically evaluated regarding its use in schizophrenia patients.

Introduction

Traditionally, researchers in psychiatry have evaluated treatment effects on the basis of statistical comparisons between mean changes of rating scales with growing recognition that such methods are problematic (Guyatt et al., 1989; Yeaton and Sechrest, 1981). Also, clinical outcome measures such as response bear disadvantages and uncertainties regarding the validity of the results. As these methods often use arbitrary thresholds to evaluate treatment outcome, the respective results might be biased challenging their usability in clinical studies and practice (Evans, 1998; Leucht et al., 2005, 2007). This is especially important when considering that such methods are the gold standard of outcome in approval studies questioning the reliability and relevance of study results which might have led to the approval of drugs (Lipkovich et al., 2009).

Therefore, in order to be able to reliably measure treatment efficacy concurrently evaluating its clinical relevance, Jacobson and Truax (1991) proposed the method of determining a reliable and clinically significant change of the individual in the context of observed changes for the whole patient sample. Until today, this concept has been primarily used in psychotherapy and quality of life research (Crosby et al., 2003; Sloan et al., 2002) and to our knowledge it has not been evaluated in schizophrenia patients which is surprising given the importance of valid and adequate measuring methods of treatment efficacy. Aim of the present study was therefore to apply the methods of reliable and clinically significant changes in schizophrenia patients during inpatient treatment and to evaluate their use in this patient sample and to compare the concept to established outcome criteria.

Methods

Subjects

Data were collected in a multicenter follow‐up programme (German Research Network on Schizophrenia) (Wolwer et al., 2003) at 11 psychiatric university hospitals and three psychiatric district hospitals. All patients admitted to one of the earlier mentioned hospitals between January 2001 and December 2004 with the diagnosis of schizophrenia, schizophreniform disorder, delusional disorder and schizoaffective disorder according to DSM‐IV criteria were included. Subjects were aged between 18 and 65 years. Exclusion criteria were a head injury, a history of major medical illness and alcohol or drug dependency. An informed written consent had to be provided to participate in the study. The study protocol was approved by the local ethics committees (Jager et al., 2007).

Assessments

DSM‐IV diagnoses were established by clinical researchers on the basis of the German version of the Structured Clinical Interview for DSM‐IV (American Psychiatric Association, 1994). Socio‐demographic and course‐related variables were collected using a standardized documentation system (Cording, 1998) during interviews with patients, relatives and care providers. To assess symptom severity the Positive and Negative Syndrome Scale for Schizophrenia (PANSS) (Kay et al., 1988) was used. The PANSS is a widely used 30‐item scale for the assessment of schizophrenic symptoms composed of three subscales: positive symptoms (items P1–P7), negative symptoms (items N1–N7) and general psychopathology (G1–G16). The Global Assessment of Functioning (GAF) scale (American Psychiatric Association, 1994), which comprises from the axis V of the DSM‐IV, was introduced as a measure of the patient's psychological, social and occupational functioning. All raters had been trained using the applied scales and ratings were performed within the first three days after and biweekly during admission as well as at discharge. A high inter‐rater reliability was achieved (ANOVA‐ICC > 0.8). The ratings were not blinded, however, independent of the clinical routine and treatment applied performed by study investigators.

Statistical analysis

The analysis of determining reliable and clinically significant changes was performed as proposed by Jacobson and Truax (1991) incorporating two different aspects:

  1. Reliable change. A reliable change refers to the extent to which the change observed falls beyond the range which could be attributed to the measurement variability itself (Evans, 1998). The measurement variability is called Reliable Change Index (Jacobson and Truax, 1991). This index assesses and standardizes the difference between two measures considering the scale's test–retest reliability and its variance in the current sample. In the article at hand a significance level of 0.05 was used yielding to a cutoff from at least ±1.96 for the reliable change index as threshold for significance. In order to have a conservative estimation of the test–retest reliability of the PANSS, the test–retest reliability of 0.6 was chosen as published by Kay et al. (1988). Combined with the variation of the current sample the reliable change index can be transferred into a score which determines if the observed change is a reliable change. In the study at hand the index resulted in a scale difference of 33 or more points which was found to be a reliable change.

  2. Clinically significant change. A clinically significant change is defined as a change that is clinically meaningful, e.g. that the patient has changed from a “symptomatic” to a “non‐symptomatic/healthy” condition. In order to determine the patient's current state and to examine whether or not the patient is “ill” or “healthy” a score of a healthy comparison group is used. As there are no data on healthy controls for the PANSS or other schizophrenia rating scales different PANSS scores were evaluated. The examined PANSS scores range from 40 points (42 points referring to a condition without any significant psychiatric symptoms as revealed by linking analyses [Leucht et al., 2006; Levine et al., 2008]) up to 50 points mirroring a condition where a patient is mildly impaired by psychopathological symptoms.

In order to compare the detected clinically significant change of the patient to traditional outcome domains the following established outcome criteria were chosen: Response was defined as a ≥ 50% improvement of the PANSS total score from admission to discharge as proposed by Leucht et al. (2007), and remission using the symptom‐severity component of the standardized remission criteria (Andreasen et al., 2005). We followed the recommendations of Obermeier et al. (2009) concerning the calculation of percent changes in the PANSS subtracting 30 points from every PANSS before calculating the percent change. All statistical analyses were performed using the statistical software environment R 2.11.1 (R Development Core Team, 2008).

Results

Patients

In the entire multicenter study 474 patients were enrolled. Forty‐six patients had to drop out for different reasons (e.g. retrospective violation of inclusion criteria, withdrawal of informed consent). Another 32 patients were excluded from analysis: 28 patients because they were discharged from the hospital within seven days after admission and four patients due to incomplete PANSS ratings. Therefore, the sample available for analysis comprised 396 subjects (227 male/169 female). The mean age was 35.54 years (±10.98) and the mean duration of illness was 7.73 years (±9.04). Of these 30% of the patients suffered from their first illness episode. The mean PANSS total score at admission was 70.52 (±18.79) and 49.85 (±15.08) points at discharge showing a highly statistical significant improvement (p < 0.0001). During the study patients were treated under naturalistic conditions as follows: 80% of the patients received first‐generation antipsychotic, 79% of patients second‐generation antipsychotic treatment and 63% of the patients were treated with first – as well as second –generation antipsychotics. Tranquilizers were administered in 78% of patients and mood stabilizers in 16%. Of these 26% of the patients were also treated with antidepressants.

Assessments

Reliable change of psychopathology in schizophrenia patients

Frequency of schizophrenia patients achieving a reliable change during inpatient treatment

The proportion of patients with a reliable and non‐reliable change is shown in Figure 1. In 79 patients (20%) a reliable improvement of symptoms could be observed. No patient worsened reliably. Ten patients (3%) did not change at all; 279 patients (70%) improved during inpatient treatment, yet without achieving a reliable change of their condition; 28 patients (7%) worsened despite receiving treatment, yet the level of worsening was not found to be reliable.

Figure 1.

Figure 1

Open in a new tab

Reliable change of the PANSS in schizophrenia inpatients.

Dependency of the reliability of an observed change on the standard error of measurement

The reliable change index indicating how many patients achieve a reliable change during treatment is assessed using a variation on the standard error of measurement. This standard error takes into account two aspects, namely the standard deviation of the measure applied to the patient sample and the test–retest reliability of the measure. This means that the reliable change index varies depending on the patient sample and the test–retest reliability of the applied measure. Figure 2 shows how much the reliable change index would vary if a different reliability and standard deviation would be assumed.

Figure 2.

Figure 2

Open in a new tab

The reliable change index (including the cutoff of clinical significant changes) in Cut in dependence of the scale's test–retest reliability and the study populations PANSS standard deviation.

Clinically significant change in schizophrenia patients

PANSS threshold of 40 points

Applying the threshold of a PANSS total score of 40 points we found that out of the 79 patients (20%) improving reliably, 31 patients (8%) showed a clinically significant change meaning that these patients achieved a reliable improvement concurrently achieving the threshold for a clinically significant change. Forty‐eight patients (12%) improving reliably did not reach the defined threshold for a clinically significant improvement, Figure 3.

Figure 3.

Figure 3

Open in a new tab

Reliable and clinically significant change in schizophrenia inpatients applying a PANSS threshold of 40 points to define clinical significance.

No significant difference was found between the antipsychotic treatment applied (atypical antipsychotics/typical antipsychotics/combination of atypical and typical antipsychotics) between patients achieving/not achieving a clinically significant change (p = 0.2676).

PANSS threshold of 45 points

When calculating the clinically significant change with the PANSS threshold of 45 points, 45 patients (11%) were able to improve clinically significant, whereas 34 patients (8.5%) improved reliably, but did not meat the defined threshold (see Figure 4).

Figure 4.

Figure 4

Open in a new tab

Reliable and clinically significant change in schizophrenia inpatients applying a PANSS threshold of 45 points to define clinical significance.

Again, no significant difference could be observed when comparing patients with/without a clinically significant change in terms of the antipsychotic treatment applied (p = 0.4967).

PANSS threshold of 50 points

Applying a threshold of a PANSS score of 50 to examine a clinically significant change, 60 patients (15%) were found to improve clinically significant, 19 patients (5%) did improve reliably yet did not achieve a clinically significant change (see Figure 5).

Figure 5.

Figure 5

Open in a new tab

Reliable and clinically significant change in schizophrenia inpatients applying a PANSS threshold of 50 points to define clinical significance.

Also, when evaluating the threshold of 50 points on the PANSS to defined a clinically significant change, no significant difference could be found between the patients achieving/not achieving a clinically significant change regarding the antipsychotic treatment applied (p = 0.1958).

Clinical significance as additional potential outcome domain in comparison to response and remission as established outcome criteria

In order to compare the concept of a clinically significant change to currently applied outcome criteria, responders and symptomatic remitters were analysed. Of the 396 patients, 220 patients (56%) were responders at discharge and 239 patients (60%) were symptomatic remitters. For the comparison of patients with a clinically significant change and those achieving response and symptomatic remission see Table 1. A highly significant association was observed between response and remission at discharge and the likelihood of concurrently achieving a clinically significant change during treatment independent of the PANSS threshold applied.

Table 1.

Association between achieving response and remission at discharge and a clinically significant improvement during inpatient treatment considering different PANSS thresholds evaluated

PANSS threshold of 40 p‐Value PANSS threshold of 45 p‐Value PANSS threshold of 50 p‐Value
yes/no yes/no yes/no
Responder 31/189 0.00 45/175 0.00 60/160 0.00
Non‐responder 0/176 0/176 0/176
Remitter 31/208 0.00 43/196 0.00 54/185 0.00
Non‐remitter 0/157 2/155 6/151
Open in a new tab

Discussion

The concept of reliable and clinically significant changes in psychiatry and schizophrenia – clinical and statistical aspects

Generally, it should to be critically discussed whether or not the concept of Jacobson and Truax (1991) on reliable and clinically significant changes is applicable in psychiatric patients. Within this concept a patient can only achieve a clinically significant change when his status changes from “symptomatic” to “non‐symptomatic/healthy” referring to a general change of his condition. However, in psychiatric patients already a change in one symptom, e.g. suicidality, can be understood to be a clinically relevant or significant improvement which would not be detected within the concept of Jacobson and Truax (1991). Therefore, this concept might not be precise enough for psychiatric patients where the presence and improvement of certain symptoms might have a specific impact on the general condition.

However, on the other side the two‐fold outcome criterion of assessing reliable and clinically significant changes bears several advantages. First, this method provides information on what is traditionally understood to be response and remission at the same time. The reliable change of the patient can be understood to be synonymous to response, whereas the evaluation on the clinical relevance of the reliable change mirrors remission. The major advantage of the assessment of the reliability of change compared to response is that for the evaluation of a reliable change no threshold is needed which makes this method less arbitrary and considerably more valid. Besides, the fact that the index to measure the reliable change is calculated using the respective scale's reliability and the study population's mean standard deviation underlines the individuality of the concept. Figure 2 shows how on the one side the index depends on the psychometric properties of the respective psychopathology rating scale and study sample. But also, this method enhances comparability of study results for every researcher would have the exact same conditions given that the rating scale's reliability as a standardized measure cannot be arbitrarily chosen. Therefore, it can be concluded that the evaluation of a reliable change is a less susceptible and more valid method to measure a relevant change of the patient's condition compared to response where different and arbitrary definitions are often applied (Leucht et al., 2007, 2009b; Leucht and Kane, 2006). These advantages have also been pointed out by other authors. When, for example, examining postnatal depression Matthey (2004) found the concept of a clinically significant change to add to our understanding of outcome by including the observation of an average improvement in patients recovering, those improving but still in the distressed range, those with no change at all and patients who deteriorate. Also, Voderholzer et al. (2014) applied this concept in a naturalistic study on depressed patients and psychotherapeutic treatment finding the calculation of a clinically significant change to allow a very differentiated evaluation of treatment effects. A potential limitation might be that the evaluation of the clinical significance requires a cutoff score in order to separate between symptomatic/non‐symptomatic. However, in schizophrenia patients data on normative schizophrenia psychopathology rating scales are rarely available so that the originally proposed concept cannot be followed.

There has also been criticism of this concept from a statistical point of view with several refinements and modifications to the reliable change methodology in the literature (Hageman and Arrindell, 1993; Hsu, 1999; Speer, 1992). Amongst others, it has been criticized that the more extreme the scores at admission, the greater the chance of the scores regressing to the mean in turn influencing the reliable change index (Evans, 1998) limiting the external validity of the results. This has led to methodological adaptations of the formula proposed by Jacobson and Truax (1991) to calculate the reliable change index accounting for regression to the mean. However, in the present study we held on to the original concept of the calculation of the reliable change index based on several other literature reports stating that even though regression to the mean is always a possibility it is not nearly as frequent in clinical data sets as believed (Speer, 1998) and that its presence is not necessarily “of a problematic kind” (Hageman and Arrindell, 1993). Also, it has been stated that there is no consensus among methodologists about the ubiquitousness of regression to the mean (Speer, 1999). McGlinchey et al. (2002) furthermore concluded that current research data support “the Jacobson Truax method as a ‘null’ method that has yet to be rejected by an alternative method of superior performance”.

Reliable and clinically significant changes of psychopathology during inpatient treatment in schizophrenia patients

To our knowledge, this is the first study in schizophrenia patients applying the concept of reliable and clinically significant change methods. We found 20% of the examined patients to show a reliable improvement of their symptoms, and depending on the different PANSS thresholds between 8% and 15% of the patients achieved a clinically significant change during treatment. Keeping results of effectiveness studies in schizophrenia in mind reporting response rates up to almost 70% (Boter et al., 2009) and remission rates between 40 and 45% (Helldin et al., 2006; San et al., 2007) the present results suggest that the reliable and clinically significant changes method might be too strict and unrealistic to be applied in schizophrenia patients on first glance.

However, reports of studies using growth mixture modeling in schizophrenia patients indicate that present results using reliable and clinically significant methods might not be inadequate to be applied in schizophrenia patients (Case et al., 2011; Levine and Leucht, 2010). Stauffer et al. (2011) for example evaluated 1990 schizophrenia patients within a 24‐week‐trial and found only 2.4% of the examined patients to show a sustained, rapid and dramatic improvement compared to 90.6% of the patients with slower‐onset and partial response and a considerably lower overall improvement. In a study on 420 schizophrenia patients Marques et al. (2010) found 10% of the patients to be dramatic responders with a 74% improvement in the Brief Psychotic Rating Scale during treatment. Very similar results are reported by Levine and Rabinowitz (2010) stating that 17% of patients who were severely ill at baseline showed much improvement (76% improvement). No significant difference in the patients’ clinically significant change was found regarding the antipsychotic treatment applied which is in line with studies reporting similar efficacy rates for typical as well as atypical antipsychotics (Leucht et al., 2009a).

In the study at hand, all patients achieving a clinically significant change were concurrently treatment responder independent of the PANSS threshold applied (Table 1). Interestingly, this was not the case in terms of remission. The less stringent the cutoff, the more patients achieved symptomatic remission without concurrently achieving a clinically significant change of their psychopathological symptoms as defined by Jacobson and Truax (1991). This discrepancy is in line with other study reports finding the consensus remission criteria to be defined less restrictively in order to euphemistically increase the percentage of remitters in schizophrenia (Schennach‐Wolff et al., 2010). The consensus remission criteria were also found to “reflect the reality of clinical trials, in which more stringent definitions of remission would probably rarely be achieved” as stated in a selective review (Leucht et al., 2009b). The method of evaluating reliable and clinically significant changes seems to separate between patients with a considerable and a lagged and impaired improvement. This suggests that this method might be suitable for research and drug trials differentiating better between patients with/without a satisfying treatment effect compared to the established outcome criteria. However, future trials are warranted to re‐evaluate present study results, the applicability of the reliable and clinically significant change method as well as the chosen thresholds of clinical significance in schizophrenia patients.

Strengths and limitations

To our knowledge, this is the first study in schizophrenia patients evaluating the method of reliable and clinically significant changes during inpatient treatment. The naturalistic study design is both, a strength and a limitation. As mentioned earlier, a potential limitation is the calculation of the cutoff determining the amount of patients achieving a clinically significant improvement. Given that there are no data on normative schizophrenia psychopathology rating scales the original proposal of Jacobson and Truax (1991) might not be that easy to transfer to an illness like schizophrenia possibly complicating its use in these patients. Also, no subjective patient data have been included limiting the complexity of outcome in schizophrenia. Besides, serum drug levels were not evaluated in all patients and the patients’ adherence was not strictly monitored, e.g. by pill counts which should be kept in mind when discussing present results on response to antipsychotic treatment.

Conclusion

According to the concept of reliable and clinically significant change methods 20% of schizophrenia inpatients showed a reliable improvement of their condition, between 8% and 15% of these patients concurrently improved from a symptomatic state at admission to a non‐symptomatic state at discharge. Compared to traditional outcome criteria such as response or remission this concept seems to be too strict at first glance. However, when considering comparative literature on different response trajectories or cluster analyses present results seem to adequately mirror the proportion of patients with a fast and clinically relevant improvement. Future trials are needed to re‐evaluate present study results and the applicability of the reliable and clinically significant change methods in schizophrenia patients.

Financial support

The study was performed within the framework of the German Research Network on Schizophrenia, which is funded by the German Federal Ministry for Education and Research BMBF (grant 01 GI 0233).

Declaration of interest statement

The authors have no competing interests.

Schennach, R. , Möller, H.‐J. , Obermeier, M. , Seemüller, F. , Jäger, M. , Schmauss, M. , Laux, G. , Pfeiffer, H. , Naber, D. , Schmidt, L. G. , Gaebel, W. , Klosterkötter, J. , Heuser, I. , Maier, W. , Lemke, M. R. , Rüther, E. , Klingberg, S. , Gastpar, M. , Musil, R. , Spellmann, I. , and Riedel, M. (2016) Challenging the understanding of significant improvement and outcome in schizophrenia – the concept of reliable and clinically significant change methods. Int. J. Methods Psychiatr. Res., 25: 3–11. doi: 10.1002/mpr.1476.

References

  1. American Psychiatric Association . (1994) Diagnostic and Statistical Manual of Mental Disorders, Washington, DC, American Psychiatric Association. [Google Scholar]
  2. Andreasen N.C., Carpenter W.T. Jr, Kane J.M., Lasser R.A., Marder S.R., Weinberger D.R. (2005) Remission in schizophrenia: proposed criteria and rationale for consensus. American Journal of Psychiatry, 162(3), 441–449. [DOI] [PubMed] [Google Scholar]
  3. Boter H., Peuskens J., Libiger J., Fleischhacker W.W., Davidson M., Galderisi S., Kahn R.S. (2009) Effectiveness of antipsychotics in first‐episode schizophrenia and schizophreniform disorder on response and remission: an open randomized clinical trial (EUFEST). Schizophrenia Research, 115(2–3), 97–103. [DOI] [PubMed] [Google Scholar]
  4. Case M., Stauffer V.L., scher‐Svanum H., Conley R., Kapur S., Kane J.M., Kollack‐Walker S., Jacob J., Kinon B.J. (2011) The heterogeneity of antipsychotic response in the treatment of schizophrenia. Psychological Medicine, 41(6), 1291–300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Cording C. (1998) [Conceptual aspects in development and implementation of basic psychiatric documentation]. Psychiatrische Praxis, 25(4), 175–178. [PubMed] [Google Scholar]
  6. Crosby R.D., Kolotkin R.L., Williams G.R. (2003) Defining clinically meaningful change in health‐related quality of life. Journal of Clinical Epidemiology, 56(5), 395–407. [DOI] [PubMed] [Google Scholar]
  7. Evans C. (1998) The contribution of reliable and clinically significant change methods to evidence‐based mental health. Evidence‐Based Mental Health, 1(3), 70–72. [Google Scholar]
  8. Guyatt G.H., Deyo R.A., Charlson M., Levine M.N., Mitchell A. (1989) Responsiveness and validity in health status measurement: a clarification. Journal of Clinical Epidemiology, 42(5), 403–408. [DOI] [PubMed] [Google Scholar]
  9. Hageman W.J., Arrindell W.A. (1993) A further refinement of the reliable change (RC) index by improving the pre‐post difference score: introducing RCID. Behaviour Research and Therapy, 31(7), 693–700. [DOI] [PubMed] [Google Scholar]
  10. Helldin L., Kane J.M., Karilampi U., Norlander T., Archer T. (2006) Remission and cognitive ability in a cohort of patients with schizophrenia. Journal of Psychiatric Research, 40(8), 738–745. [DOI] [PubMed] [Google Scholar]
  11. Hiller W., von Zerssen D., Mombour W., Wittchen H.U. (1986) Die IPMS, Weinheim, Beltz. [Google Scholar]
  12. Hsu L. M. (1999) A comparison of three methods of identifying reliable and clinically significant client changes: commentary on Hageman and Arrindell. Behaviour Research and Therapy, 37(12), 1195–1202. [DOI] [PubMed] [Google Scholar]
  13. Jacobson N.S., Truax P. (1991) Clinical significance: a statistical approach to defining meaningful changes in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. [DOI] [PubMed] [Google Scholar]
  14. Jager M., Riedel M., Messer T., Laux G., Pfeiffer H., Naber D., Schmidt L.G., Gaebel W., Huff W., Heuser I., Kuhn K.U., Lemke M.R., Ruther E., Buchkremer G., Gastpar M., Bottlender R., Strauss A., Moller H.J. (2007) Psychopathological characteristics and treatment response of first episode compared with multiple episode schizophrenic disorders. European Archives of Psychiatry and Clinical Neuroscience, 257(1), 47–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Kay S.R., Opler L.A., Lindenmayer J.P. (1988) Reliability and validity of the positive and negative syndrome scale for schizophrenics. Psychiatry Research, 23(1), 99–110. [DOI] [PubMed] [Google Scholar]
  16. Leucht S., Corves C., Arbter D., Engel R.R., Li C., Davis J.M., (2009. a) Second‐generation versus first-generation antipsychotic drugs for schizophrenia: a meta‐analysis. Lancet, 373(9657), 31–41. [DOI] [PubMed] [Google Scholar]
  17. Leucht S., Davis J.M., Engel R.R., Kissling W., Kane J.M. (2009. b) Definitions of response and remission in schizophrenia: recommendations for their use and their presentation. Acta Psychiatrica Scandinavica. Supplementum, 438, 7–14. [DOI] [PubMed] [Google Scholar]
  18. Leucht S., Davis J.M., Engel R.R., Kane J.M., Wagenpfeil S. (2007) Defining ‘response’ in antipsychotic drug trials: recommendations for the use of scale‐derived cutoffs. Neuropsychopharmacology, 32(9), 1903–1910. [DOI] [PubMed] [Google Scholar]
  19. Leucht S., Kane J.M. (2006) Measurement‐based psychiatry: definitions of response, remission, stability, and relapse in schizophrenia. Journal of Clinical Psychiatry, 67(11), 1813–1814. [DOI] [PubMed] [Google Scholar]
  20. Leucht S., Kane J.M., Etschel E., Kissling W., Hamann J., Engel R.R. (2006) Linking the PANSS, BPRS, and CGI: clinical implications. Neuropsychopharmacology, 31(10), 2318–2325. [DOI] [PubMed] [Google Scholar]
  21. Leucht S., Kane J.M., Kissling W., Hamann J., Etschel E., Engel R.R. (2005) What does the PANSS mean? Schizophrenia Research, 79(2–3), 231–238. [DOI] [PubMed] [Google Scholar]
  22. Levine S.Z., Leucht S. (2010) Elaboration on the early‐onset hypothesis of antipsychotic drug action: treatment response trajectories. Biological Psychiatry, 68(1), 86–92. [DOI] [PubMed] [Google Scholar]
  23. Levine S.Z., Rabinowitz J. (2010) Trajectories and antecedents of treatment response over time in early‐episode psychosis. Schizophrenia Bulletin, 36(3), 624–632. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Levine S.Z., Rabinowitz J., Engel R., Etschel E., Leucht S. (2008) Extrapolation between measures of symptom severity and change: an examination of the PANSS and CGI. Schizophrenia Research, 98(1–3), 318–322. [DOI] [PubMed] [Google Scholar]
  25. Lipkovich I.A., Deberdt W., Csernansky J.G., Buckley P., Peuskens J., Kollack‐Walker S., Rotelli M., Houston J.P. (2009) Defining “good” and “poor” outcomes in patients with schizophrenia or schizoaffective disorder: a multidimensional data‐driven approach. Psychiatry Research, 170(2–3), 161–167. [DOI] [PubMed] [Google Scholar]
  26. Marques T.R., Arenovich T., Agid O., Sajeev G., Muthen B., Chen L., Kinon B.J., Kapur S. (2010) The different trajectories of antipsychotic response: antipsychotics versus placebo. Psychological Medicine, 20, 1–8. [DOI] [PubMed] [Google Scholar]
  27. Matthey S. (2004) Calculating clinically significant change in postnatal depression studies using the Edinburgh Postnatal Depression Scale. Journal of Affective Disorders, 78(3), 269–272. [DOI] [PubMed] [Google Scholar]
  28. McGlinchey J.B., Atkins D.C., Jacobson N.S. (2002) Clinical significant methods: which one to use and how useful are they? Behavior Therapy, 33, 529–550. [Google Scholar]
  29. Obermeier M., Mayr A., Schennach‐Wolff R., Seemuller F., Moller H.J., Riedel M. (2009) Should the PANSS be rescaled? Schizophrenia Bulletin, 36(3), 455–460. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. R Development Core Team (2008) A language and environment for statistical computing. Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org [Google Scholar]
  31. San L., Ciudad A., Alvarez E., Bobes J., Gilaberte I. (2007) Symptomatic remission and social/vocational functioning in outpatients with schizophrenia: prevalence and associations in a cross‐sectional study. European Psychiatry, 22(8), 490–498. [DOI] [PubMed] [Google Scholar]
  32. Schennach‐Wolff R., Moller H.J., Jager M., Seemuller F., Obermeier M., Messer T., Laux G., Pfeiffer H., Naber D., Schmidt L.G., Gaebel W., Klosterkotter J., Heuser I., Maier W., Lemke M.R., Ruther E., Klingberg S., Gastpar M., Riedel M. (2010) A critical analysis and discussion of the appropriateness of the schizophrenia consensus remission criteria in clinical pharmaceutical trials. Pharmacopsychiatry, 43(7), 245–251. [DOI] [PubMed] [Google Scholar]
  33. Sloan J.A., Cella D., Frost M., Guyatt G.H., Sprangers M., Symonds T. (2002) Assessing clinical significance in measuring oncology patient quality of life: introduction to the symposium, content overview, and definition of terms. Mayo Clinical Proceedings, 77(4), 367–370. [DOI] [PubMed] [Google Scholar]
  34. Speer D.C. (1992) Clinically significant change: Jacobson and Truax (1991) revisited. Journal of Consulting and Clinical Psychology, 60(3), 402–408. [DOI] [PubMed] [Google Scholar]
  35. Speer D.C. (1998) Mental Health Outcome Evaluation, San Diego, CA, Academic Press. [Google Scholar]
  36. Speer D.C. (1999) What is the role of two‐wave designs in clinical research? Comment on Hageman and Arrindell. Behaviour Research and Therapy, 37(12), 1203–1210. [DOI] [PubMed] [Google Scholar]
  37. Stauffer V., Case M., Kollack‐Walker S., Scher‐Svanum H., Ball T., Kapur S., Kinon B.J. (2011) Trajectories of response to treatment with atypical antipsychotic medication in patients with schizophrenia pooled from 6 double‐blind, randomized clinical trials. Schizophrenia Research, 130(1–3), 11–19. [DOI] [PubMed] [Google Scholar]
  38. Voderholzer U., Koch S., Hillert A., Schlegl S. (2014) Response und Non‐Response in der stationären Psychotherapie depressiver Patienten, German. Psychotherapeut, 57, 410–416. [Google Scholar]
  39. Wolwer W., Buchkremer G., Hafner H., Klosterkotter J., Maier W., Moller H.J., Gaebel W. (2003) German research network on schizophrenia‐bridging the gap between research and care. European Archives of Psychiatry and Clinical Neuroscience, 253(6), 321–329. [DOI] [PubMed] [Google Scholar]
  40. Yeaton W.H., Sechrest L. (1981) Critical dimensions in the choice and maintenance of successful treatments: strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology, 49(2), 156–167. [DOI] [PubMed] [Google Scholar]

Articles from International Journal of Methods in Psychiatric Research are provided here courtesy of Wiley

RESOURCES