Lack of early improvement with antipsychotics is a marker for subsequent non-response in behavioral and psychological symptoms of dementia: Analysis of CATIE-AD data

Abstract

Objective

Prediction of response/non-response to antipsychotics is especially important in patients with behavioral and psychological symptoms of dementia (BPSD) in whom antipsychotic exposure increases risks of death. We aimed to examine whether presence/absence of early improvement of BPSD with antipsychotics is associated with subsequent response/non-response.

Design

Post-hoc analysis of the Clinical Antipsychotic Trials in Intervention Effectiveness with Alzheimer’s Disease (CATIE-AD) study (2001–2004) (trial registration: NCT00015548).

Setting

45 sites in the United States.

Participants

245 subjects (olanzapine, n=90; quetiapine, n=81; risperidone, n=74) with a DSM-IV diagnosis of dementia of the Alzheimer’s type who presented with a score of 1 or more in the Brief Psychiatric Rating Scale (BPRS) at baseline (Phase 1 of CATIE-AD).

Intervention

Subjects were randomly assigned to treatment with olanzapine, quetiapine, risperidone, or placebo in a double-blind manner.

Measurements

We examined associations between response at week 8, and demographic and clinical characteristics, including BPRS total score reduction at week 2, using logistic regression analyses. Prediction performance of binary classification (presence/absence) of improvement/no improvement at week 2 for response at week 8 was examined.

Results

BPRS total score reduction at week 2 (mean percentage score reduction, 12.6%) was significantly associated with response at week 8 (odds ratio, 1.18; 95% CI, 1.11–1.26). The 5% score reduction cut-off at week 2 showed the highest accuracy (0.71) with sensitivity, specificity, PPV, and NPV of 0.76, 0.65, 0.69, and 0.72, respectively.

Conclusion

Lack of even a very small early improvement with antipsychotic treatment may be a marker of subsequent non-response in BPSD.

Introduction

Behavioral and psychological symptoms such as delusions, hallucinations, agitation and aggression are difficult to manage in patients with dementia.¹ While non-psychopharmacological interventions are the first option to consider, drug treatments are widely used.^2,3 Antipsychotic drugs have the best evidence for effectiveness in the management of behavioral and psychological symptoms with dementia (BPSD).⁴ However, use of antipsychotic medication continues to be controversial and subject to scrutiny and international policy oversight, as substantial morbidity and increased mortality associated with their use^5,6 led to a US Food and Drug Administration (FDA) black box warning against the use of atypical antipsychotics in patients with dementia.^7,8 More recently, a 2014 update to the American Psychiatric Association’s Practice Guidelines recommends that antipsychotics must be used with caution and at the lowest effective dosage because they are associated with severe adverse events.⁹ Further, the frequency, severity and potential consequences of the adverse effects of antipsychotics are greater in older patients due to age-related changes in pharmacokinetic and pharmacodynamics parameters.^10,11 Therefore, it would be clinically important to identify potential responders and non-responders to antipsychotic treatment as early as possible after treatment is initiated to inform benefit-risk considerations in individual patients.^12,13 If such response prediction is valid, those who are unlikely to respond to a particular drug could be switched to another treatment option, hence reducing exposure to antipsychotics that offer little clinical gain.^12–14

In patients with schizophrenia, a number of previous studies have shown that early improvement following antipsychotic drug use is associated with subsequent favorable treatment outcomes.^12,13,15 Likewise, lack of early improvement with antipsychotics predicts unfavorable outcomes at endpoint and this has already been incorporated into treatment guidelines.¹⁶ The same holds for treatment of depression with antidepressants.^14,17 However, no studies have investigated the ability of early symptom improvement to predict later response with antipsychotics in patients with BPSD.

To investigate this, we conducted a post-hoc analysis of the data from the Clinical Antipsychotic Trial of Intervention Effectiveness-Alzheimer’s disease (CATIE-AD)^18,19 to examine whether presence/absence of improvement with antipsychotics (olanzapine, quetiapine, and risperidone) after 2 weeks treatment would be associated with treatment response/non-response at week 8 in patients with BPSD.

Methods

Study design

The CATIE-AD was funded by the National Institute of Mental Health to compare the effectiveness of antipsychotic drugs in patients with Alzheimer’s disease and psychosis or agitated/aggressive behavior. The study has been described in detail elsewhere.^18,19 Briefly, it was conducted between April 2001 and November 2004 at 45 clinical sites in the United States. Four hundred and twenty-one patients with a diagnosis of dementia of the Alzheimer’s type based on the Structured Clinical Interview of the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV)²⁰ or probable Alzheimer’s disease based on the National Institute of Neurological and Communicative Disorders Association (NINCDA-ADRDA),²¹ participated in the trial. Patients were initially randomized to olanzapine, quetiapine, risperidone, or placebo under double-blind conditions, and received treatment for up to 36 weeks or until treatment was discontinued for any reason (Phase 1). Medications were prepared in low-dose and high-dose capsules (olanzapine: 2.5 mg or 5.0 mg, quetiapine: 25 mg or 50 mg and risperidone: 0.5 mg or 1.0 mg, respectively). Study physicians adjusted medication dosage based on their clinical judgment and patient response.

Data used in this analysis were derived from the patients who were receiving olanzapine, quetiapine, or risperidone and received assessments with the Brief Psychiatric Rating Scale (BPRS)²² or the Neuropsychiatric Inventory (NPI)²³ at both baseline and week 2 in Phase 1 of CATIE-AD. The protocols were approved by the local institutional review boards, and all patients gave written informed consent to participate in this trial. Ethical approval was not sought for this specific analysis that used completely anonymous data.

Clinical Subtypes

Based on the data in Phase 1 of CATIE-AD, patients were classified by age group (i.e. ages of ≤69 years or ≥70), sex, race (i.e. white vs. other), and dementia psychosis subtype (i.e. paranoid, misidentification, mixed, and non-psychotic). This categorization was based on factorial analysis of NPI delusions and hallucinations domains,^24,25 which identified two factors: a ‘paranoid’ subtype (delusions of persecution and/or abandonment); and a ‘misidentification’ subtype (misidentification phenomena and/or hallucinations). Patients who were experiencing both types of symptoms were described as ‘mixed’.

Statistical analysis

First, to examine factors associated with response at week 8, binary logistic regression analyses were conducted with antipsychotic medication used, gender, age group, race, dementia psychosis subtype (only for NPI analysis), total score in the BPRS or NPI at baseline, and reduction in the BPRS or NPI total scores from baseline to week 2. A multivariate model was used for the last 2 variables (i.e. total score in the BPRS or NPI at baseline, and reduction in the BPRS or NPI total scores from baseline to week 2) and univariate model for the other variables. With regard to the definition of response, a score reduction of ≥ one minimal clinically important difference (MCID),^26,27 defined as a half of the standard deviation (SD) of change from baseline at week 8 in the BPRS or NPI was adopted; MCIDs were 6.4 to 7.6 for BPRS and 8.3 to 10.5 for NPI, depending on the dataset generated with multiple imputations²⁸ as described below.

Next, the prediction performance of binary classification of early improvement at week 2 (present or absent) for response at week 8 was examined. To this end, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the consecutive cut-off points in 5% increments between 5% and 25% in the BPRS total or NPI total scores at week 2 were calculated. To seek the optimum cut-off point, accuracy, defined as (True Positive + True Negative) / Total N, was calculated. Accuracy depends on the number of observations, which may render it inferior to the careful and balanced consideration of sensitivity and specificity. To address this potential pitfall, cut-off points that demonstrated a level of ≥0.5 in both sensitivity and specificity with the highest degree of accuracy were examined.²⁹ In addition, the area under the curve (AUC) of the receiver operating characteristic (ROC) was also calculated.

Multiple imputation of the outcome and predictors was performed to deal with missing values, using SAS, version 9.4 (SAS Institute Inc., Cary, NC, USA.). To account for variability in imputed values, 100 imputed data sets were created using Proc MI (a procedure within SAS) using Markov chain Monte Carlo (MCMC) imputation method. Multiple imputation is a method in which missing values are replaced with predicted values from a regression model, but in order to reincorporate variance that is lost by using a simple prediction, a residual term is added to each value based on a normal distribution with mean zero and variance equal to the residual variance from the regression model. In the case of this study the imputation was single-chain done with 200 burn-in iterations, as are the default settings. The imputation was done 100 times (as mentioned above), the resulting datasets were then analyzed and the results were pooled using Proc MIANALYZE. Other statistical analyses, including additional available case analysis, were performed, using SPSS version 22.0 (IBM, New York). A p-value of <0.05 was considered statistically significant (two-tailed).

Results

Subject characteristics

All two-hundred forty-five patients (olanzapine, n=90; quetiapine, n=81; risperidone, n=74) and 242 patients (olanzapine, n=90; quetiapine, n=80; risperidone, n=72) in the intention-to-treat (ITT) samples were included in the analyses for the BPRS and NPI, respectively. Demographic and clinical characteristics are summarized in Table 1.

Table 1.

Demographic and Clinical Characteristics of the Patients

Rating scales used	BPRS (N=245)	NPI (N=242)
Characteristics
Age, years	78.1±7.4 (51–103)	78.3 ± 7.3 (51–103)
Education, years	12.3±3.4 (0–21)a	12.3±3.4 (0–21)b
MMSE at baseline	15.2±5.8 (4–28)c	15.2±5.7 (4–28)d
Total score at baseline	27.0±12.0 (3–66)	35.6±18.0 (3–104)
Sex
Male	109 (44.5)	107 (44.2)
Race
White	196 (80.0)	193 (79.8)
Black or African-American	39 (15.9)	39 (16.1)
American Indian or Alaska Native	1 (0.4)	1 (0.4)
Asian	8 (3.3)	8 (3.3)
Two or more races	1 (0.4)	1 (0.4)
Married	148 (60.4)	147 (60.7)
Medication
Olanzapine	90 (36.7)	90 (37.2)
Quetiapine	81 (33.1)	80 (33.1)
Risperidone	74 (30.2)	72 (29.8)
Dementia subtypes
Paranoid	n.a.	69 (28.5)
Misidentification	n.a.	13 (5.4)
Mixed	n.a.	117 (48.3)
Non-psychotic	n.a.	43 (17.8)

Values are shown as mean±SD (range) or n (%).

^aThe data were available in 236 patients.

^bThe data were available in 233 patients.

^cThe data were available in 244 patients.

^dThe data were available in 241 patients.

Abbreviations: BPRS, Brief Psychiatric Rating Scale; MMSE, Mini-Mental State Examination; NPI, Neuropsychiatric Inventory; SD, standard deviation

Factors associated with response to antipsychotic drugs at week 8

The missing proportions (out of 245) for the variables included in the BPRS imputation model were 28.9%, 52.2%, and 62% respectively for the variable BPRS at weeks 4, 8, and 12. Baseline BPRS values and week 2 values were complete. The missing proportions (out of 242) for the variables included in the NPI imputation model were 28.1%, 51.6%, and 61.5% respectively for the variable NPI at weeks 4, 8, and 12. Baseline NPI values and week 2 values were complete. There were no missing values for other variables (i.e. age, sex, race, antipsychotics, or subtype). The total score reduction in the BPRS or NPI at week 2 was significantly associated with subsequent response to antipsychotic treatment at week 8 (Tables 2 and 3). In contrast, factors other than the total score in the BPRS or NPI at baseline failed to show any association with subsequent response. Results obtained with an available case analysis were similar to these findings (Supplementary Tables 1 and 2).

Table 2.

Association Between Demographic and Clinical Characteristics and Antipsychotic Response in BPRS at Week 8

Variables	Odds Ratio	95%CI	t-value#	df#	p-value#
Multivariate Model
BPRS total score at baseline	1.05	(1.01–1.09)	2.59	506	0.001
BPRS total score reduction at week 2	1.17	(1.10–1.25)	4.51	618	<0.001
Univariate Models
Age (years)
≤69	1 (reference)
≥70	0.92	(0.37–2.29)	0.19	1394	0.852
Sex
Male	1 (reference)
Female	1.55	(0.86–2.80)	1.46	1520	0.143
Race
White	1 (reference)
Others	1.56	(0.73–3.31)	1.16	1276	0.247
Antipsychotics
Olanzapine	1 (reference)
Quetiapine	1.46	(0.73–2.95)	1.07	1549	0.287
Risperidone	1.33	(0.63–2.78)	0.75	1116	0.453

^#t-value and d.f. were estimated in the MI applied for analysis of missing data. The SE and d.f. were estimated with Rubin’s rule in the MI.

Values including odds ratios and p-values are derived from binary logistic regression analysis.

Response was defined as ≥ one minimal clinically important difference (i.e. 4.6 to 5.8).

P-values of <0.05 were shown in bold.

Abbreviations: BPRS, Brief Psychiatric Rating Scale; CI, confidence interval; MI, multiple imputation; SE, standard error

Table 3.

Association Between Demographic and Clinical Characteristics and Antipsychotic Response in NPI at Week 8

Variables	Odds Ratio	95%CI	t-value#	df#	p-value#
Multivariate Model
NPI total score at baseline	1.06	(1.03–1.09)	3.82	530	<0.001
NPI total score reduction at week 2	1.09	(1.04–1.14)	3.54	338	<0.001
Univariate Models
Age (years)
≤69	1 (reference)
≥70	1.31	(0.54–3.14)	0.60	2415	0.552
Sex
Male	1 (reference)
Female	0.97	(0.53–1.77)	0.11	1193	0.909
Race
White	1 (reference)
Others	1.23	(0.58–2.57)	0.54	1450	0.591
Antipsychotics
Olanzapine	1 (reference)
Quetiapine	1.60	(0.77–3.35)	1.59	1023	0.207
Risperidone	1.77	(0.85–3.71)	1.26	1334	0.130
Subtype
Paranoid	1 (reference)
Misidentification	0.96	(0.24–3.79)	0.06	2247	0.954
Mixed	0.89	(0.43–1.84)	0.31	1127	0.757
Non-psychotic	0.35	(0.13–0.94)	2.07	734	0.038

^#t-value and d.f. were estimated in the MI applied for analysis of missing data. The SE and d.f. were estimated with Rubin’s rule in the MI.

Values including odds ratios and p-values are derived from binary logistic regression analysis.

Response was defined as ≥ one minimal clinically important difference (i.e. 8.3 to 10.3).

P-values of <0.05 were shown in bold.

Abbreviations: CI, confidence interval; MI, multiple imputation; NPI, the Neuropsychiatric Inventory; SE, standard error

Prediction performance of presence/absence of improvement at week 2 for response at week 8

The prediction performance of binary classification of early improvement at week 2 for response at week 8 is shown in Table 4; sensitivity and NPV were slightly higher than specificity and PPV. The 5% cut-offs in the BPRS and the NPI at week 2 showed the highest degree of accuracy for the prediction of response at week 8. The ROC analysis demonstrated high values for the use of BPRS and NPI total score reductions for the prediction of response at week 8 with 0.76 and 0.75, respectively. The 5% and 10% cut-offs in BPRS and NPI at week 2 showed the highest degree of accuracy for the prediction of response at week 8, respectively, when available case analysis was employed (Supplementary Table 3).

Table 4.

Prediction Performance of Score Reduction at Week 2 for Response at Week 8

Rating scales used	Percentage score reduction at week 2	Sensitivity	Specificity	PPV	NPV	Accuracy	AUC
BPRS (N=245)	5%	0.76	0.65	0.69	0.72	0.705	0.76
	10%	0.68	0.70	0.70	0.68	0.690
	15%	0.59	0.76	0.71	0.64	0.672
	20%	0.51	0.80	0.72	0.61	0.651
	25%	0.45	0.83	0.73	0.59	0.636
NPI (N=242)	5%	0.80	0.58	0.69	0.71	0.700	0.75
	10%	0.75	0.64	0.71	0.68	0.697
	15%	0.68	0.69	0.72	0.64	0.682
	20%	0.64	0.74	0.74	0.63	0.683
	25%	0.56	0.77	0.74	0.59	0.651

Response was defined as ≥one minimal clinically important difference (i.e. 4.6 to 5.8 for the BPRS and 8.3 to 10.3 for the NPI).

Multiple imputation was used to deal with missing values.

Abbreviations: AUC, area under the curve; BPRS, Brief Psychiatric Rating Scale; NPI, Neuropsychiatric Inventory; NPV, negative predictive value; PPV, positive predictive value

Discussion

As the proportion of aging individuals within society increases, the management of BPSD represents an urgent unresolved clinical issue. To our knowledge, this is the first study to investigate the impact of presence/absence of early improvement with antipsychotic drugs on subsequent treatment outcomes in patients with BPSD. We found that the reduction in total score at week 2 was significantly associated with subsequent clinically important response at week 8 although the modest magnitude of the association should be taken into account. Furthermore, score reductions of 5% in the BPRS and NPI total scores at week 2 appeared to perform well as clinically relevant cut-offs, with the highest degree of accuracy for the prediction of response at week 8. Given the fact that NPVs were higher than PPVs, these findings suggest that, if there is no improvement in the early stage of treatment, continuation of the antipsychotic in question is likely to be futile.

Previous studies focusing on patients with schizophrenia or major depressive disorders (MDD) have shown that presence/absence of early improvement with antipsychotics or antidepressants can be a robust predictor of subsequent response/non-response^{12,13,15,17,30–32} although the conditions of psychosis and mood symptoms may substantially differ among patients with schizophrenia, MDD, and AD. In patients with schizophrenia, improvements such as a ≥25% reduction in the BPRS or a ≥20% reduction in the Positive and Negative Syndrome Scale (PANSS)³³ total score at week 2 predict response at 4, 8, and 12 weeks, while lack of such initial improvement at week 2 is associated with poor outcomes thereafter.^12,15,30 Such associations have also been previously identified in relation to antidepressant treatment.^17,31,32 Since there has been no prior report of the degree of change that should be used to define early improvement with antipsychotic treatment in BPSD, we tested consecutive cut-off points to explore the optimum threshold. In contrast to those previous studies, optimally performing cut-offs were relatively low (5% for BPRS and NPI, respectively) in the current study. This discrepancy is likely attributable to differences in symptom trajectories over time in people with dementia compared to other illnesses and the heterogeneous nature of symptoms contained within the nonspecific treatment target that BPSD represents. In the present study, the mean percentage score reduction in the BPRS total score at week 2 was as low as 12.6% (from 27.0 to 23.6), for which floor effects should be taken into account. This reduction is much lower than seen in schizophrenia trials. For example, one double-blind randomized controlled trial data of schizophrenia patients demonstrated that the mean percentage score reduction in the PANSS at week 2 was 29.2% (from 95.0 to 67.3) for risperidone and 21.1% (from 97.3 to 76.8) for quetiapine,³⁴ which roughly corresponds to a percentage BPRS improvement of 30%.³⁵ Thus, the symptom improvement from baseline to week 2 reported in schizophrenia seems greater than that in BPSD with modest severity. These low cut-off values (i.e. 5%), with high NPVs, seen in our study reinforce the observation that patients with no improvement at the early stage of antipsychotic treatment in BPSD are unlikely to derive any further clinical benefit thereafter. Prediction of non-response is especially important in patients with BPSD in whom the exposure to antipsychotic drugs has been reported to increase risks of serious side effects, including death.^7,36 Those potential non-responders may benefit from a switch from antipsychotic treatment that will unlikely work to another treatment option at the earliest opportunity; this will also minimize the exposure to antipsychotic drugs and hence reduce such lethal adverse events.

Prediction performance in the present study was high and comparable to that in previous studies that have included patients with schizophrenia; for example, lack of early improvement at 2 weeks predicted subsequent non-response at week 8 or 12 with NPVs of 0.73–0.84.^12,15 Thus, early improvement with antipsychotic treatment could serve as a robust predictor of subsequent treatment response, irrespective of diagnoses. While the prediction performance in the present study seems high, it should be noted that 20–30% of the patients were still judged as false positives or false negatives. Therefore, further investigations are clearly needed to improve the prediction performance to reduce the risk of misclassifications.

The association between lack of early improvement with antipsychotics and subsequent non-response could provide a clinically relevant opportunity to discontinue medications that carry significant risk of harm in people with dementia and explore alternative treatment options (where available) at an early stage. This is critically important since the use of antipsychotics can result in a variety of side effects,⁵ including increased mortality.^5,7 Indeed, longer use of antipsychotics is associated with increased mortality^5,7 and there is also evidence that this association is dose-dependent.⁷ In the light of these findings, the use of antipsychotics is not recommended as a first-line treatment for BPSD.³⁷ Despite these safety concerns, prescribing surveys have consistently shown the continuing and frequent use of antipsychotics for patients with severe BPSD, which clearly underscores the importance of the topic addressed by our study.³⁸ On the other hand, while the results of this study suggest clinical utility of discontinuing the medication that does not seem to provide any further benefit and trying a next treatment option, there is not any better evidence-supported therapy, which is a dilemma in the treatment of BPSD.

The results of our study must be interpreted in the light of some limitations. First, CATIE-AD was not originally designed to examine whether presence/absence of early improvement with antipsychotics could predict subsequent treatment outcomes. The association between early improvement and subsequent response was derived from a post-hoc analysis; therefore, appropriate caution is required in interpretation of the results. Second, only patients treated with olanzapine, quetiapine, or risperidone, were included, which limits any extrapolation of our findings to other antipsychotics. Third, the potential influence of medication dose was not taken into consideration since flexible dosing was employed in this study. Fourth, the choice of weeks 2 and 8 for the timing of assessments was based on previous studies that have examined prediction performance in patients with schizophrenia and MDD,^15,17 but it may still be considered arbitrary. Fifth, other factors such as adverse events, which may work as predictors of poor subsequent response, were not taken into consideration in the present study since they were not evaluated in a systematic manner, using assessment scales. Further investigations focusing on the potential roles of adverse events in predicting subsequent outcomes are warranted. Sixth, the primary outcomes for this analysis were BPRS and NPI total scores. However, the total scores on these instruments include a broad range of symptoms and therefore may not always reflect treatment targets. Although we included dementia psychosis subtype as an independent variable in the logistic regression analysis for NPI and found no significant relationship in this regard, further investigations focusing on specific symptoms are clearly needed. Seventh, although the odds ratios that predicted subsequent response were statistically significant, they were relatively small. Moreover, while accuracy of the prediction performance was found to be generally good, there still were many inaccuracies in the model. These results suggest that the response to antipsychotic treatment may not be easy to accurately predict solely based on early symptom improvement. In fact, treatment response has been reported to be associated with a number of factors, including genetic background.³⁹ Further investigations such as genetic studies to identify more detailed predictors for good treatment response in BPSD are warranted. Thus, the results of this study should be interpreted with caution in the clinical settings. Finally, there was a large amount of missing data for the BPRS and NPI scores at week 8 (i.e. 52.2% and 51.6%, respectively), which we addressed through the use of multiple imputation.

Although this method is a valid approach to missing data,⁴⁰ and one which produced similar results for our main finding compared to available case analysis, we cannot be certain that the imputed data are completely representative of the original data. This is perhaps most relevant when considering the potential influence of medication dose and adverse events on drop-out and subsequent outcome. Furthermore, since the CATIE design allowed participants to be transitioned to other treatments (i.e. switching from Phase 1 to Phase 2), clinical reasons for exit from Phase 1 were not randomly related with insufficient efficacy or adverse effects. Thus, those remaining in the study phase 1 may not be entirely representative of the group initially treated, which limits the generalizability of the findings in the present study. For these reasons, our observations should be viewed as preliminary and need to be confirmed in a prospective clinical trial.

In conclusion, presence/absence of early improvement at week 2 with antipsychotic treatment may be a predictor of subsequent response or non-response at week 8 in the treatment of BPSD, as has been shown to be the case for depression and schizophrenia. This finding indicates that, especially in light of higher NPVs, evaluating patients early in the course of treatment with antipsychotic drugs help identify non-responders who are unlikely to benefit from continuation of the current antipsychotic. Although future prospective studies are needed to confirm those preliminary findings, the results of this study underscore the relevance of focusing on symptom trajectories in guiding antipsychotic treatment on an individual basis to minimize unwanted adverse effects in the treatment of BPSD.