Use of Clinical Global Impressions-Severity (CGI-S) to Assess Response to Antidepressant Treatment in Patients with Treatment-Resistant Depression

Abstract

Background

This post-hoc analysis evaluated the agreement between Clinical Global Impressions-Severity (CGI-S) score- and Montgomery–Åsberg Depression Rating Scale (MADRS) total score-based assessment of response in patients with treatment-resistant depression (TRD) treated with esketamine nasal spray plus a newly initiated oral antidepressant (ESK-NS + AD).

Methods

Data were analyzed from a phase 3, randomized, double-blind study (TRANSFORM-2) of flexibly dosed esketamine or placebo nasal spray plus a newly initiated oral-AD in adults with moderate-to-severe TRD. Patients with ≥50% reduction in MADRS from baseline at the end of the 4-week acute treatment phase were defined as responders. For the CGI-S-based assessment of response, patients with ≥2 points decrease from baseline or a CGI-S score of ≤3 (mildly depressed to normal) were considered responders. Cohen’s kappa coefficient was calculated to assess level of agreement between MADRS and CGI-S-based assessments.

Results

At the end of 4-week treatment, the proportion of responders among all study patients (n=201) was similar when assessed using the MADRS (61%) and CGI-S (62%) methods, with substantial agreement (Cohen’s kappa=0.76; sensitivity=92%; specificity=84%) between both methods. When restricting analysis to ESK-NS + AD-treated patients (n=101) who had a higher response rate (on MADRS: 69%; on CGI-S: 68%), the agreement remained substantial (Cohen’s kappa=0.75; sensitivity=91%; specificity=84%).

Conclusion

The CGI-S may be a practical and reliable alternative to the MADRS to assess response to ESK-NS + AD in patients with TRD and can be used in real-world practice to support informed treatment decisions.

Introduction

Measurement-based care (MBC) is a critical component of an evidence-based decision support framework that guides patient treatment.¹ In mental health clinical practice, MBC approaches have been designed to provide systematic, quantitative assessments to track treatment progress, measure symptom severity and reflect on important aspects of health-related quality of life, satisfaction and functioning.^2,3 Thus, MBC can be implemented in various ways to enhance clinical practice in mental healthcare settings. Several clinician-rated instruments that rely on interviews and questionnaires for assessments of symptoms have been used to measure treatment outcomes in clinical studies of patients with major depressive disorder (MDD).^4,5 The Montgomery–Åsberg Depression Rating Scale (MADRS), a 10-item questionnaire (each item yields a score of 0 to 6), is a validated, clinician-rated measure of depression designed to measure depression severity and to detect changes due to antidepressant treatment in patients with MDD.⁶ It is one of the most frequently used scales for assessment of depression in clinical trials of antidepressant treatments submitted to the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA).^5,7 However, the MADRS as a research instrument usually takes approximately 20–40 minutes to administer and therefore may not be feasible for clinicians to use in their routine practice.^6,8

By contrast, the Clinical Global Impressions-Severity (CGI-S) is a single-item, 7-point scale designed to assess global impression of severity rather than rating individual depressive symptoms. The CGI-S requires the clinician to rate the severity of patient’s illness from a global perspective, relative to the clinician’s experience with patients who have MDD. The brevity and simplicity of CGI-S makes it a more practical measurement tool that can easily be used by a clinician in a busy real-world clinical practice setting.⁹ The CGI-S directly reflects a clinician’s judgment and captures clinical impressions of depression severity based on observed and reported symptoms, behavior, and function in the past 7 days.^9,10 The scale has also shown good inter-rater reliability among clinicians.^7,9,11 Several studies have evaluated the association between change on the MADRS and a 1-point change on the CGI-S and report a range of results. A 1-point reduction in CGI-S score translated into an 8–9-point change in MADRS total score, in a study evaluating the relationship of MADRS and CGI-S scales in patients with MDD.⁷ Another study concluded that based on a 1-point improvement in the CGI-S anchor, a 10-point improvement on the MADRS is an appropriate meaningful change threshold for within-patient change.¹² In a recent post-hoc analysis of phase 3 studies of esketamine nasal spray in patients with treatment-resistant depression (TRD), a 2-point reduction in CGI-S score corresponded to a 12-point reduction in MADRS, thus establishing thresholds for clinically meaningful within-patient changes and better interpretability.¹³ Phase 3 clinical studies of esketamine nasal spray in patients with TRD have used MADRS as the primary assessment of treatment response and remission.^14–18 Both MADRS and CGI-S scales have demonstrated comparable ability to measure treatment outcomes with a high correlation in patients with MDD.⁴

Evaluating rating agreement between MADRS-based and CGI-S-based outcomes using patient-level data is important to translate clinical trial results into clinical practice. The purpose of this post-hoc analysis was to evaluate the agreement with the pre-specified MADRS-based assessment and a CGI-S score-based definition of response to acute antidepressant treatment in patients with TRD.

Methods

Study Design and Patients

Data for this post-hoc analysis were derived from a phase 3, randomized, double-blind, active-controlled study (TRANSFORM-2; NCT02418585)¹⁷ of flexibly dosed esketamine nasal spray in patients with TRD. Details of study design and population have been described elsewhere.¹⁷ Briefly, patients with MDD aged between 18 and 64 years with moderate-to-severe depression (30-item Inventory of Depressive Symptoms-Clinician-rated total score ≥34) were included. Patients also had to meet the study definition of TRD: failed to respond to ≥2 antidepressant treatments of adequate dose and duration in the current episode, with one antidepressant treatment failure assessed prospectively at study entry. The study had three phases: a 4-week screening/prospective observation phase, a 4-week double-blind treatment phase, and a posttreatment follow-up phase of up to 24 weeks. In the 4-week acute treatment phase, patients were randomly assigned (1:1) to treatment with flexibly dosed esketamine nasal spray (56 or 84 mg) twice a week plus a newly initiated daily oral antidepressant (ESK-NS + AD) or oral antidepressant plus placebo nasal spray (AD + PBO-NS).

Independent Ethics Committees or Institutional Review Boards (listed in Supplementary Material) for each country approved the study protocol. The study was conducted in accordance with ethical principles of the Declaration of Helsinki, Good Clinical Practices and applicable regulatory requirements. All patients provided written informed consent before study enrollment.

Assessments

The primary efficacy endpoint of the TRANSFORM-2 study was change in MADRS total score from baseline (day-1) to endpoint (day-28) of the double-blind treatment phase. Since transient dissociative and sedative effects of esketamine nasal spray could potentially bias the rater, all MADRS assessments were performed predose by independent, remote (by telephone) raters who were blind to the protocol details. Response (secondary endpoint) was pre-specified as patients with ≥50% reduction from baseline in MADRS total scores. Investigators at the site rated the severity of depression using the CGI-S. CGI-S assessments were also conducted predose.

For the alternative CGI-S-based assessment of response in this post-hoc analysis, patients with ≥2 points decrease from baseline or a CGI-S score of ≤3 (mildly depressed to normal) were considered responders based on the cut-off described previously in the literature.^9,23 As per CGI-S guidelines, based on severity of illness of the patient, score of 1 to 7 was assigned, wherein, 1 = Normal – not at all ill; 2 = Borderline ill; 3 = Mildly ill; 4 = Moderately ill; 5 = Markedly ill; 6 = Severely ill; and 7 = Among the most extremely ill patients. The proposed cut-off criterion was based on a best fit model of optimal definitions of response from the literature.

Statistical methods

Cohen’s kappa coefficient was calculated to assess the level of agreement between MADRS- and CGI-S-based assessments of response. Sensitivity and specificity of the CGI-S-determined response were estimated, using MADRS-based classification as reference. As part of additional sensitivity analyses, the level of agreement between MADRS- and other definitions of CGI-S-based response (2-point improvement or a CGI-S score of ≤3) was also performed.

Results

Patients

Details of patient demographics, baseline characteristics and disposition have been provided in the primary study publication.¹⁷ The dataset for the efficacy analyses of the TRANSFORM-2 study included 223 patients (ESK-NS + AD: n=114; AD + PBO-NS: n=109) who had a mean (SD) age of 45.7 (11.89) years and were mostly women (61.9%) and white (93.3%). At baseline, the mean (SD) MADRS total score was 37.1 (5.67), CGI-S total score was 5.1 (0.67) and all patients with moderate-to-severe TRD had a CGI-S score ranging from 4 to 7.

Assessment of Response

A total of 201 patients had a day-28 MADRS assessment; however, 5 patients did not have a day-28 CGI-S assessment. Also, of the 198 patients who had a day-28 CGI-S assessment, two did not have a day-28 MADRS assessment. Thus, a total of 196 patients with both MADRS and CGI-S at day-28 were included in the analysis. At the end of the 4-week double-blind phase, the proportion of responders among all patients (n=201) was similar when assessed using the MADRS (61%) and CGI-S (63%) methods (Table 1).

Table 1.

Assessment of Response Using MADRS and CGI-S

	All Patientsa (N=196)		Esketamine Nasal Spray Plus Oral Antidepressant (N=100)
	MADRSb	CGI-Sc	MADRSb	CGI-Sc
Responders, n (%)	120 (61.2)	124 (63.3)	69 (69.0)	68 (68.0)
Non-responders, n (%)	76 (38.8)	72 (36.7)	31 (31.0)	32 (32.0)
Sensitivity, %	92.5		91.3
Specificity, %	82.9		83.9
Cohen’s kappa (95% CI)	0.76 (0.67–0.86), p<0.0001		0.75 (0.60–0.89), p<0.0001

Notes: ^aIncludes patients who received esketamine nasal spray plus oral antidepressant and patients who received oral antidepressant plus placebo nasal spray; ^bpatients with ≥50% reduction in MADRS from baseline at the end of the 4-week acute treatment phase were defined as responders; ^cpatients with ≥2 points decrease from baseline or a CGI-S score of ≤3 (mildly depressed to normal) were defined responders.

Abbreviations: CGI-S, Clinical Global Impressions-Severity; CI, confidence interval; MADRS, Montgomery–Åsberg Depression Rating Scale.

The Cohen’s kappa was 0.76 (95% CI: 0.67–0.86), suggesting substantial agreement between the two methods. The CGI-S-based assessment demonstrated 92.5% sensitivity and 82.9% specificity versus the MADRS-based assessment. These findings were further supported by additional sensitivity analysis, wherein CGI-S-based response using both definitions showed comparable agreement with the MADRS-based response assessment. In the overall population, the composite definition showed better sensitivity and slightly higher agreement with MADRS than the one based on 2-point improvement (Cohen’s kappa [95% CI] = 0.73 [0.63–0.83]), while the one based on CGI-S ≤3 showed marginally higher Cohen’s kappa (0.80 [0.71–0.88]) (Supplementary Table 1).

Similar results were observed in patients treated with ESK-NS + AD (Table 1). The proportion of responders was 69% using the MADRS scale and 68% using the CGI-S scale. There was substantial agreement (Cohen’s kappa=0.75, 95% CI: 0.60–0.89) between the two scales and the CGI-S assessment showed 91.3% sensitivity and 83.9% specificity. Additional analyses of CGI-S-based response using the other two definitions also showed comparable agreement with MADRS (Supplementary Table 1).

Discussion

This post-hoc analysis confirmed that the pre-specified MADRS criterion for response, ie, ≥50% reduction from baseline in MADRS total scores, used to evaluate acute treatment effect of ESK-NS + AD substantially corresponded with the CGI-S definition of ≥2 points reduction from baseline or cut-off score of ≤3. At the end of the 4-week treatment phase, the proportions of responders and non-responders among all patients and patients treated with ESK-NS + AD assessed using the MADRS response criterion were similar to those assessed using the CGI-S criterion. There was substantial agreement (Cohen’s kappa coefficient: 0.75–0.76) between the two scales.

Standardized scales provide tangible evidence of treatment response in clinical research. However, the implementation of scales such as MADRS in real-world psychiatry practice is often burdensome for the clinicians and patients. Given the variety of rating scales, using clinically and practically relevant measures that could potentially lower the burden of time associated with assessment is of great importance.^19,20 The MADRS has been used in pivotal clinical studies of antidepressants; however, its adoption in clinical practice is uncommon. The purpose of this study was to evaluate an alternative approach to the MADRS using the CGI-S that has similar rigor for assessment of response albeit with lesser burden and can be used by clinicians in real-world practice. During routine clinical visits, the use of a semi-structured MADRS tool may not be practical to the physician and cumbersome for patients who have to formally answer to 10 standardized questions and can potentially restrict patient-physician interaction to an assessment bound within the 10 items of the scale. Findings from the present analysis underscore the agreement between depression rating scales and measures of global illness severity and indicate a high agreement between the MADRS and CGI-S for assessment of response. Thus, the simpler and readily understood CGI-S mimics the assessment of response in the clinical trial and can be used to measure treatment response in real-world clinical practice.⁹ By utilizing a clinician-determined summary, the CGI-S is more intuitive and improvement on the CGI-S reflects a more global account of the patient’s disease status and overall well-being, which is clinically meaningful for the clinician, patients and payer in contrast to assessments based on standardized rating scales for treatment effectiveness.⁹

This post-hoc analysis was based on data from over 200 adult patients with the same diagnosis (and definition) of TRD participating in a phase 3 clinical study from the registration program of esketamine nasal spray, representing a relatively homogenous population for assessment.¹⁷ Furthermore, the analysis included data from both active control (AD + PBO-NS)- and study drug (ESK-NS + AD)-treated patients, providing wider range scores that make the findings conceivably more relevant and robust.

Limitations

Inherent limitations of a post-hoc analysis should be considered. The primary study enrolled patients with limited medical and psychiatric co-morbidities and substance use disorder history. Patients with current or recent history (within 6 months) of homicidal ideation/intent or suicidal ideation with intent to act or suicidal behavior (within the past year); diagnosis of psychotic disorder, MDD with psychotic features, bipolar or related disorders, borderline, antisocial, histrionic, or narcissistic personality disorder, obsessive-compulsive disorder, intellectual disability, autism spectrum disorder; uncontrolled hypertension; seizures were excluded; thus, the sample may not be representative of all the patients with TRD. However, a majority of the study population had either moderate or serious anxiety at baseline, similar to the broader population with TRD. The use of CGI-S also has some inherent limitations. Standardization across clinics typically does not occur; so, neither interrater reliability nor a consensus of how much each rating relies on symptoms or daily function or satisfaction has been agreed. Furthermore, given the concerns of clinician bias, a brief patient-reported scale would be a suitable alternative for regular in-clinic assessments. Nevertheless, the CGI-S approach reflects in many ways the current assessment process in real-world practice and in this case allows the definition of clinically meaningful changes. Finally, it is acknowledged that the most widely used approach of considering a reduction of severity of symptoms of ≥50% from baseline to be a clinical responder is to a large extent arbitrary.¹³ Moreover, a report based on data from patients with TRD found that a 35% reduction in the MADRS score was associated with a clinically meaningful degree of improvement in health-related quality-of-life²¹ as assessed by the Quality-of-Life Enjoyment and Satisfaction Questionnaire.²²

Conclusions

To the best of our knowledge, this analysis is the first to report comparable assessment of response using MADRS- and CGI-S-based criteria following acute treatment with esketamine nasal spray plus an oral antidepressant or oral antidepressant plus placebo nasal spray in patients with TRD. CGI-S is a practical and reliable alternative for MADRS to assess treatment response and therefore the clinicians can use the CGI-S-based definition to evaluate response to an antidepressant treatment, including esketamine nasal spray in patients with TRD in real-world practice to support informed treatment decisions.

Funding Statement

This study was supported by Janssen Research & Development, LLC.

Data Sharing Statement

The data sharing policy of Janssen Pharmaceutical Companies of Johnson & Johnson is available at https://www.janssen.com/clinical-trials/transparency. Requests for access to the study data can be submitted through Yale Open Data Access (YODA) Project site at http://yoda.yale.edu.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

All authors are employees of Janssen and may hold company stocks or stock options. Joachim Morrens was an employee of Johnson & Johnson during the conduct of the study. The authors report no other conflicts of interest in this work.