Regression to the Mean: Implications for Clinical Trials of Psychotropic Agents in Dementia

Abstract

Effective drug development depends on understanding and optimizing results from controlled clinical trials. A recent double-blind, randomized, controlled trial of the treatment of agitation in patients with Alzheimer’s disease (AD) found no difference among the four arms of the study: haloperidol, trazodone, behavioral therapy, placebo. The current analysis was undertaken to further investigate the issues bearing on this outcome and to identify better means of detecting psychotropic effects in trials involving patients with AD.

This was post hoc analysis of a clinical trial data set. Patients in the placebo group were divided into responders (25% reduction in symptoms), worseners (25% worsening in baseline agitation scores), and those without a change in symptoms.

Analysis of the trial outcomes demonstrated that the reduction observed in the placebo group was of the same magnitude as predicted by regression to the mean. Patients exhibiting greater improvement had more severe baseline behavioral disturbances. The relatively modest severity of agitation and the low medication doses achieved in the study may have further contributed to the failure to distinguish among treatment groups.

Research design adjustments such as collection of both screening and baseline measures to determine eligibility may limit the effects of regression to the mean on trial outcomes and reduce this challenge to clinical trials.

REGRESSION TO THE MEAN: IMPLICATIONS FOR CLINICAL TRIALS OF PSYCHOTROPIC AGENTS IN DEMENTIA

Randomized controlled trials (RCT) are necessary to establish the efficacy of potentially useful pharmacologic agents. In the past ten years, RCT have been employed in assessing the efficacy of a variety of treatments for Alzheimer’s disease (AD). Cholinesterase inhibitors, vitamin E, prednisone, estrogen, ginkgo biloba, and other agents have all been studied in RCTs [1]. In addition, psychotropic agents potentially useful in the treatment of common behavioral disturbances of AD have been evaluated [2–5].

In clinical trials addressing neuropsychiatric symptoms in AD, robust changes have been observed in patients in the placebo group [2, 3]. In contrast to other disorders where such responses have been intensively studied, there have been no previous investigations of the placebo-group changes in RCTs involving patients with AD. True placebo effects, regression to the mean, and treatment benefit from participating in a clinical trial may all contribute to placebo group responses. A recent randomized clinical trial⁵ found no difference between placebo, pharmacotherapy, and behavioral therapy in the treatment of agitation in AD; all groups improved. This trial is of particular importance since it is the only multicenter trial conducted comparing pharmacologic and nonpharmacologic instruments in AD. To investigate the observations made in this trial, we conducted a post hoc analysis of the response in the placebo group. We investigated patient characteristics (dementia severity, dementia duration and characteristics of the behavioral abnormalities), caregiver characteristics (gender, age, education), and trial characteristics (site and time of entry of the patient into the trial) to identify potential predictors of the response observed in the placebo group. We also assessed the timing of the response in the different trial groups and examined the profile of adverse events occurring in the placebo group. Finally, we estimated the expected regression to the mean to determine if this statistical effect could explain the observed response. Better understanding of placebo group behavior in RCTs may aid in the construction of future trials of psychotropic agents in AD.

Methods

Patient Selection and Trial Design

The data for this study were derived from an investigation of the efficacy of behavioral management, haloperidol and trazodone compared to placebo in reducing agitation in patients with AD. The trial utilized of a 17-week parallel group design. 148 patients meeting National Institute of Neurological and Communicative Disorders and Stroke (NINCDS) -Alzheimer’s Disease and Related Disorders Association (ADRDA) criteria [6] for probable or possible AD were randomly assigned to behavioral management (n=41), haloperidol (n=34), trazodone (n=37) or placebo (n=36). Outcomes and measures relevant to the current analysis included the Cohen-Mansfield Agitation Inventory (CMAI) [7], the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) Behavior Rating Scale for Dementia (BRSD) [8], the Revised Memory and Behavior Problem Checklist (RMBPC) [9], the Screen for Caregiver Burden (SCB) [10], and the Mini-Mental State Examination (MMSE) [11]. The CMAI assesses the frequency of 36 agitated behaviors using a 7-point Likert-type scale for each behavior. The tool has established inter-rater reliability [12]. The BRSD is a 46 item caregiver-based instrument that provides information on six factors: psychosis, behavioral dysregulation, depression, inertia, irritability/aggression, and vegetative symptoms. Good inter-rater reliability for this instrument has been demonstrated [8]. The RBMPC is a caregiver-based instrument that rates 24 problems on two 5 point scales: 1) frequency of problems and 2) caregiver distress associated with each problem. Memory-related, depressive and disruptive subscale scores are derived from the instrument. Good internal consistency, reliability, and validity have been described [9]. Caregiver burden was assessed with the reaction score of the RBMPC and the SCB [10]. The 25 item SCB, created specifically for spouse caregivers of AD patients, provides scores for objective burden (OB; number of potentially negative experiences) and subjective burden (SB; appraised distress in response to the experiences). The internal consistency, test-retest reliability, and validity (content, divergent, convergent, and criterion) of the scales have been established [10]. The MMSE is a 30 point questionnaire providing a global measure of cognitive function based on questions assessing orientation, attention, recent memory, calculation, naming, comprehension, repetition and construction [11]. Adverse symptoms occurring in the course of the trial were based on caregiver queries and an adverse symptom checklist (ASC).

Study Design

The primary interest in this study was to determine characteristics of agitated patients who were assigned to the placebo arm of the study and whose agitation scores declined (improved). The variable of interest for defining the response was the CMAI. For this analysis, a “responder” was defined as a patient in the placebo group with a reduction in total CMAI score from baseline to week 17 of at least 25%. We compared this group to “non-responders” comprised of all other placebo-arm subjects. In addition, we compared responders to placebo-arm patients who worsened at least 25% during the course of the trial (“worseners”) (Table 1). Using these definitions, all subjects with available responses were included in the first analysis whereas, in the analysis comparing those who improved with those who worsened, there was a group of individuals who were excluded as they neither improved nor worsened. RMBPC, BRSD and MMSE scores at baseline, and demographic variables of the patients who received placebo were analyzed to identify predictors of responder status. Patient and caregiver gender, age and education were reviewed for their potential predictive value. ASC endorsements and adverse event reporting in the study groups also were analyzed. Two trial factors---time of entry to study (after date of study initiation) and study site---were assessed as possible predictor variables as each of these has been suggested to influence response rates. Timing of improvement also may be important in identifying patients who respond to trial conditions with reduced symptoms and we analyzed this variable in the three groups.

Table 1.

Demographic Characteristics of the Study Population

Variable	Responders B (n=10)	Nonresponders (n=24)	Worseners BB (n=9)	Responders vs nonresponders P value:	Responders vs worseners P value:
PT AGE	75.4±3.8	75.4±6.9	75.6±6.0	0.994	0.946
PT GENDER (% female)	60.0%	70.8%	66.7%	0.824	0.692
PT EDUC (yr)	13.5±2.2	11.92±3.6	11.8±3.3	0.204	0.190
PT MMSE	13.1±10.3	13.5±7.8	16.7±8.9	0.902	0.433
Duration of AD (yr)	5.0±2.9	5.1±3.1	5.0±2.6	0.926	0.965
CG AGE *	66.1±14.5	69.8±8.2	71.2±9.8	0.355	0.385
CG GENDER (% female) *	50.0%	54.2%	44.4%	0.824	0.742
CG EDUC * (yr)	13.4±2.5	12.6±2.8	12.2±2.6	0.458	0.328
Relationship of CG to PT*	70% spouse	70.8% spouse	77.8% spouse	0.227	0.453

B Responders are patients whose scores improved by at least 25%. BB Worseners are those patients who not only did not respond, but who were classified as having worsened by at least 25%. Nonresponders includes Worseners.

^*Patients were classified and the characteristics of their caregivers were calculated.

(AD – Alzheimer’s disease; EDUC – education; CG – caregiver; PT – patients)

Instrument Protocol Comparison Group

To determine if the results could be explained on the basis of regression to the mean, the expected changes in this population were estimated using an external validation sample assessed with the same instruments [12–14]. The comparison data were collected by the ADCS Instrument Protocol designed to develop new measures for use in clinical trials. The Instrument Protocol had all the characteristics and recruitment strategies of a clinical trial but no treatments were administered. The BRSD and CMAI were included in the protocol that included 137 patients with probable AD reexamined serially over a 12 month period. MMSE scores of patients participating in the protocol were above 10, comparable to the scores of patients included in the agitation study. Sampling weights necessary to produce the same expected mean and variance for age, education, baseline MMSE, baseline BRSD and expected proportion of women were developed using the model data set. Using these estimated sampling weights, the expected level of regression to the mean was computed. To ensure that differences between the two populations in terms of sampling schedule did not affect the results of the analysis, exchangeability of observations within subjects in the validation sample was tested using a resampling procedure [15]. The estimation of sampling weights employed a metric on multinomial probabilities and minimized the distance between the estimated probabilities and the uniform weighting. This approach produces relatively conservative estimates of the likely effect of regression to the mean and is based on work in constrained density estimation [16].

Statistical Analysis

Change in CMAI total score (end – baseline) was calculated for the 29 placebo subjects who completed the trial (of the original 36) and the five additional subjects who dropped out of the study but for whom discontinuation visit data were collected (n=34).

The null hypothesis for the current study was that there were no differences across groups (responder/non-responder, responders/worseners) on any demographic variable, BRSD scores, RMBPC scores, MMSE scores, patient or caregiver gender, time to enter into the trial, or study site. Kolmogorov-Smirnov tests found no evidence of significant non-normality, therefore, simple t-tests were used in the analyses of all continuous variables. The Mann-Whitney U test (non-parametric) was used to assess differences between time to entry by response status. Chi square analysis was used to assess the presence of any association between site and response status. We examined the rates of endorsement of adverse symptoms on the ASC via t-test for independent samples.

Results

Table 1 shows the demographic characteristics of each group. Table 2 shows the scores for the behavior rating scales administered in the study.

Table 2.

Baseline Behavioral Domain-Specific Contributions to Response as Assessed by the Behavior Rating Scale for Dementia (BRSD) and its Revised Memory and Behavior Problem Checklist (RMBPC)

Variable	Responders (N = 10)	Nonresponders (N = 24)	Worseners (N = 9)	Responders vs nonresponders P value:	Responders vs worseners P value:
CMAIT	48.1±30.8	37.5±24.6	23.3±15.8	0.293	0.045**
BRSD Total	58.0±21.6	49.4±25.2	33.2±18.1	0.351	0.016**
BRSD-P	7.0±5.7	6.6±7.9	3.7±4.9	0.897	0.202
BRSD-V	1.8±1.1	2.0±1.4	1.4±1.3	0.620	0.570
BRSD-BD	6.3±3.6	7.2±3.6	5.2±3.3	0.510	0.507
BRSD-I/A	12.3±5.5	7.7±5.5	6.0±5.3	0.030 **	0.021 **
BRSD-IN	2.0±0.94	1.8±0.92	1.8±0.67	0.635	0.565
BRSD-D	1.02±4.8	8.8±8.2	3.6±4.6	0.537	0.007 **
RMBPC Total Freq.	1.8±0.45	1.6±0.70	1.0±0.67	0.423	0.013 **
RMBPC-F-De	1.5±0.66	1.3±1.1	0.51±0.56	0.490	0.004 **
RMBPC-F-M	3.0±0.16	2.8±1.1	2.2±1.4	0.598	0.101
RMBPC-F-Di	1.2±0.77	0.88±0.79	0.70±0.51	0.375	0.180
RMBPC-De*	2.0±1.1	1.8±1.3	1.3±1.4	0.582	0.224
RMBPC-M*	2.0±0.78	1.6±1.3	1.1±1.3	0.359	0.093
RMBPC-Di*	2.4±1.2	1.7±1.3	1.9±1.3	0.151	0.406

^*Patients were classified as responders/nonresponders/worseners and the characteristics of the caregivers of such patients are shown here.

^**these p values are unadjusted for multiple comparisons.

P-psycliosis, BD - behavior dysregulation, I/A- irritibility/aggression, IN - inertia, D-digression, V-vegetative symptoms. De - depression, M- memory related, Di-disruptive. F- frequency of symptoms.

Responders vs. Worseners

In the analysis comparing placebo arm patients who responded by improving by at least 25% (N = 10) to those who worsened by at least 25% (N = 9), there were no statistically significant differences in patient age, patient gender, patient education, baseline MMSE scores, duration of AD, caregiver age, caregiver gender, caregiver education, or relationship of caregiver to patient. None of the demographic variables investigated distinguished these groups of patients or the caregivers of patients in these groups.

In terms of frequency of symptoms on each instrument at baseline, responders had higher total RMBPC frequency scores (t(16) = 2.79, p<.05) and higher depression frequency subscores (t(16) = 3.42, p<.01) than did worseners. Responders had higher total BRSD scores, and higher irritability/aggression (t(17)= 2.53, p<.05) scores but lower BRSD depression total scores than worseners (t(17)=3.09, p<.01) (Table 2).

Table 3 shows the caregiver ratings for burden (SCB) and reaction to the RMBPC symptoms. Burden tended to be higher at baseline for caregivers of responders relative to ratings for non-responders and worseners. After correction for multiple comparisons, these differences failed to reach significance.

Table 3.

Baseline Caregiver Contributions to Response

Variable	Responders (N = 10)	Nonresponders (N = 24)	Worseners (N = 9)	Responders vs nonresponders P value:	Responders vs worseners P value:
SCB - Subjective	18.2±9.5	10.6±6.8	8.8±5.7	0.013 **	0.019 **
SCB - Objective	11.6±3.7	9.6±4.6	7.4±4.8	0.231	0.048 **
RMBPC Total Reaction*	2.1±0.79	1.9±0.99	1.6±0.83	0.555	0.246
RMBPC-R-De*	2.0±1.1	1.8±1.3	1.3±1.4	0.582	0.224
RMBPC-R-M*	2.0±0.78	1.6±1.3	1.1±1.3	0.359	0.093
RMBPC-R-Di*	2.4±1.2	1.7±1.3	1.9±1.3	0.151	0.406

^*Patients were classified as responders/nonresponders/worseners and the characteristics of the caregivers of such patients are shown here. ** these p values are unadjusted for multiple comparisons.

SCB - Scale for Caregiver Burden

RMBPC- Revised Memory and Behavior Problem Checklist; De – depression, M – memory related, Di – disruptive, R – reaction to symptoms.

Responders vs. Nonresponders

There were no significant differences in demographe characteristics of placebo group responders compared to non-responders.

When symptoms at baseline were compared, responders had higher BRSD irritability/aggression scores (p=0.030); no other differences approached statistical significance.

Time Course of Improvement in Responders

Placebo responders (at least 25% improvement on CMAI) were compared to responders in each of the three treatment arms to determine if the time course of response differed (i.e., if responders were identifiable at either or both of the 9- and 17- week visits). Among all subjects with data at the week 9 (midpoint) visit, placebo responses were evident early in the trial (by week 9) with 50% of the placebo group showing at least 25% improvement at the midpoint of the study. This compares with 28.1% of the BMT arm, 27.3% of the trazodone arm, and 32.1% of the haloperidol arm showing this degree of improvement by week 9. At week 17, however, 29.4% of placebo subjects (with data) had at least 25% improvement in CMAI, while 26.5%, 25.7%, and 48.4% of the BMT, trazodone and haloperidol arm subjects showed at least 25% improvement at week 17. Although there was a tendency for placebo patients to respond earlier in the trial, these differences were not statistically significant.

No differences were found in any of the placebo subgroups according to center study site or days to study entry; responders tended to enter the study later (389.9 days after recruitment began) than non-responders (305.7 days), but the difference was not statistically significant.

Adverse Events in the Placebo Group

We calculated the proportion of the placebo group who experienced any of the items on the ASC at least once, or any time after baseline. Responders endorsed fewer of the 11 ASC items than non-responders (t (29.28) = 2.91, unadjusted p<.01). There were no differences between placebo arm subgroups in total adverse event reporting (i.e., the number of adverse events reported for each patient over the course of the trial).

At least 10% of patients receiving placebo evidenced dry mouth, akathisia, rigidity, dyskinesia, drowsiness, brady-kinesia, tremor and fatigue; 8.3% experienced drooling. These adverse events were equally common among placebo responders and non-responders and their incidence at week 17 did not distinguish the treatment groups [6].

Expected Effects of Regression to the Mean

Using the procedures described above for calculating the expected regression to the mean, we found that the expected change in the CMAI score (from baseline to study end) was 5.88 compared to the observed change score of 6.0 indicating that most of the change observed in the study could be attributed to regression to the mean. Model data were not found to be in conflict with the hypothesis of exchangeability, establishing the validity of the modeling procedures.

There were no significant differences among the groups at baseline or in magnitude of response on the CMAI indicating that regression to the mean may account for a substantial amount of the measured change from baseline in all groups.

Discussion

In this study, the presence of more severe behavioral disturbances at time of entry to study was a predictor of improvement in the placebo group when compared to individuals who worsened in that group. This finding was consistent whether the RMBPC or BRSD was used to assess behavior. Caregivers of the responder group tended to have high burden scores at study entry but this result did not reach significance after adjustment for multiple testing. There was a lower rate of ASC endorsement among placebo responders compared to placebo non-responders. None of the other demographic or trial variables included in the analysis had outcome prediction utility. Thus, more severe behavior abnormalities and a low rate of ASC endorsement were associated with improvement in the placebo group in this clinical trial.

This analysis suggests that the observed changes can be attributed largely to regression to the mean. Using the values from the external validation sample to model the expected regression predicted nearly the exact amount of change observed in the “responder” group. Statistical regression describes the tendency for extreme values to move closer to the mean when measures are repeated over time [17]. Regression to the mean likely accounts for some changes commonly attributed to placebo effects but is independent of any investigative influence and tends to characterize all measures, including laboratory values [17]. The most effective safeguard against regression to the mean is to require that the patient meet entry criteria with respect to values on psychopathology rating scales or several occasions prior to randomization or on averaged scores. Addition of multiple pretreatment measures will lessen vulnerability to this effect [17]. Awareness of these influences will help strengthen designs of RCT addressing behavioral changes in patients with AD and other disorders.

An alternative interpretation of the data from the study is that none of the agents or interventions was truly effective in reducing agitation. Previous studies of haloperidol [18] and trazodone [19] in similar populations, however, suggest that study design issues may have influenced the ability to observe treatment outcomes.

Other factors contributing to the failure to differentiate active treatments from placebo in this study include a relatively undefined target behavior (“agitation”), a relatively moderate severity of agitation in the study population, and relatively low doses of therapeutic agents in the pharmacologic treatment arms of the study. Agitation levels in this population were higher than in clinical trials of cognitive agents in dementia patients but lower than found in institutionalized patients [2, 4]. Attention to these issues in future trials will likely enhance the ability to detect a treatment effect.