Hippocampus and basal forebrain volumes modulate effects of anticholinergic treatment on delayed recall in healthy older adults

Abstract

Introduction

Volumes of hippocampus and cholinergic basal forebrain are associated with delayed recall performance and may modulate the effect of a muscarinic receptor antagonist on delayed recall in healthy volunteers.

Methods

We studied 15 older adults before and after the oral administration of a single dose of 1 or 2 mg of the preferential M1 muscarinic receptor antagonist trihexyphenidyl (Artane™) or placebo in a double-blind randomized cross-over design. Hippocampus and basal forebrain volumes were measured using magnetic resonance imaging.

Results

We found a significant interaction between treatment and hippocampus volume and a trend level effect between treatment and anterior basal forebrain volume on task performance, with an attenuation of the association between volume size and performance with trihexyphenidyl.

Discussion

These findings suggest a reduction of delayed recall performance with increasing doses of the muscarinic antagonist that is related to an uncoupling of the association of task performance with cholinergic basal forebrain and hippocampus volumes.

1. Introduction

Administration of trihexyphenidyl hydrochloride (Artane™) and other muscarinic receptor antagonists has been reported to decrease delayed recall in healthy volunteers [1], [2], [3]. These findings are consistent with cholinergic transmission playing a key role in attention and memory tasks that require effort and concentration [4]. The main cholinergic input to the human cerebral cortex arises from the basal forebrain cholinergic nuclei [5]. The hippocampus is involved in the coherent representation of a memory, a requirement for successful retrieval [6]. The main cholinergic input to the hippocampus arises from the most anterior subnuclei of the cholinergic basal forebrain [7], termed Ch1 and Ch2 according to Mesulam's nomenclature [8].

In this study, we examined whether the volumes of the anterior basal forebrain and hippocampus, as measured from structural magnetic resonance imaging (MRI) scans, modulate the effect of anticholinergic treatment with trihexyphenidyl on delayed recall performance in a group of 15 cognitively and physically healthy older adults. We expected that a higher volume would be associated with a higher delayed recall performance and that this association would be reduced with higher doses of trihexyphenidyl.

2. Participants and methods

The study included 15 healthy elderly individuals (eight women), mean age was 66.9 (standard deviation [SD] 3.7) years, ranging between 62 and 74 years, and mean education was 16.7 (SD 2.3) years. Individuals did not take medications known to affect cognitive functioning, such as neuroleptics or antidepressants, at least 2 weeks before beginning the study and had a negative urine toxicology screen. Further details of recruitment have been described before [9]. All subjects were only examined if they gave their written informed consent. The study has been carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki).

2.1. Neuropsychological testing and study design

Participants were recruited as part of a study on the effects of APOE variants in response to trihexyphenidyl [9]. Delayed recall (after 15 minutes) was tested using the Buschke Selective Reminding Test [10], administered before and 1, 2.5, and 5 hours after drug/placebo administration. Each subject participated in the 3-week, double-blind, randomized, placebo-controlled study, with sessions taking place 1 week apart: treatment conditions were placebo, 1.0 mg, and 2.0 mg of trihexyphenidyl. To exclude training effects within and across treatment sessions, 12 parallel word lists were used, in a randomized rotational basis across sessions. The same list was used across subjects in each of the 12 assessment periods (treatment (3) X test time (4)).

2.2. MRI data acquisition and analysis

The acquisition was performed on a 1.5 T Siemens Vision system (Erlangen, Germany) at the Nathan S. Kline Institute for Psychiatric Research, NY, USA. Images were acquired using a sagittal magnetization prepared rapid gradient-echo sequence (repetition time/echo time = 11.4/11.9 ms, 1 excitation, matrix = 256 × 256, field of view = 307 mm, 1.2 mm³ isotropic voxel, 172 slices, no gap).

MRI data processing followed procedures described previously for hippocampus [11] and basal forebrain [12] volumetry, implemented in SPM8 and the VBM8-toolbox in Matlab. Basal forebrain subregions [8] were determined according to a map from an in cranio post-mortem MRI scan and histology of a single individual's brain, as previously described [12]. The total intracranial volume was used in the statistical model to account for differences in head size, and was calculated as the sum of the total segmented gray matter, white matter, and cerebrospinal fluid volumes in native space. We selected volumes of left and right hippocampus and volumes of most anterior basal forebrain nuclei, Ch1, and Ch2 according to Mesulam's nomenclature [8] that provide the main cholinergic innervation of the hippocampus [7].

2.3. Statistical analysis

We determined the effect of treatment on delayed recall across subjects using a mixed effects model with subject-related random effects, controlling for age and sex. For time after drug intake (0–5 hours), we compared a linear with a second order polynomial term. The model fit was compared between the two nested models (first vs. second order polynomial term for time) using Akaike's information criterion (AIC) [13].

To test for an interaction between volumes of basal forebrain and hippocampus, respectively, and the drug effect, we selected the performance at the time of peak drug effect (1–2.5 hours postingestion [14]); we also tested for main effects of volume and drug, and controlled for total intracranial volume, sex, and age in all analyses. The significance of parameters was determined using t-statistics with degrees of freedom determined according to the Satterthwaite approximation. For depiction of effects, we used the “effects” library in R. We computed the fitted values and standard errors for delayed recall under the model for the interaction term of treatment by (mean centered) volume with the values of the other predictors being fixed at typical values, i.e., for an interval scaled covariate at its mean, and for a factor at its proportional distribution in the data, as described in [15].

Analyses were performed with R, version 3.1.1, including the libraries “lme4” and “lmerTest,” available at http://cran.r-project.org/web/packages.

3. Results

In the basic model across all time points, we found a significant linear effect of time on delayed recall performance (t = −4.6, df = 162, P < .001). There was no significant main effect of treatment (t = −1.68, df = 162, P = .09). The second order polynomial model improved the fit over the linear model (AIC = 837.7 for the linear model and AIC = 813.1 for the polynomial model).

For hippocampus, considering the peak drug effect time points only, we detected a significant volume by treatment interaction (left and right hippocampus: t = −2.1, df = 72, P = .04; Fig. 1) and a significant main effect of treatment (t = −3.2, df = 72, P = .002). With anterior basal forebrain volume, we observed a trend level significant interaction effect (t = −1.8, df = 72, P = .074; Fig. 2), and a significant main effect of treatment (t = −3.2, df = 72, P = .002). AIC of nested models indicated moderately improved fit for the complex models compared with the basic model: basic model AIC = 423.4; full model with Ch1/2, AIC = 421.3; full model with left hippocampus, AIC = 422.3; full model with right hippocampus, AIC = 422.2.

Fig. 1.

Association of delayed recall performance with left hippocampus volume at different levels of treatment. Treatment 1 mg = 1 mg trihexyphenidyl. Treatment 2 mg = 2 mg trihexyphenidyl.

Fig. 2.

Association of delayed recall performance with anterior basal forebrain (Ch1/2) volume at different levels of treatment. For legend see Fig. 1.

4. Discussion

We found a decline in delayed recall performance in response to the muscarinic antagonist trihexyphenidyl, consistent with previous studies [1], [2], [3] and predictions. We used a mixed effects model to take interindividual variation in performance levels into account [16]. The time activity curve of the drug was best modeled by a second order polynomial consistent with the previously reported central pharmacodynamics of trihexyphenidyl [14]. Under placebo, hippocampal and anterior basal forebrain volumes were positively associated with delayed recall performance. The treatment-induced decline of performance was associated with a reduced or even reverse association between regional volume and task performance with a higher dose of the muscarinic receptor antagonist (Fig. 1, Fig. 2). The treatment-induced reductions of performance, compared with placebo, were more pronounced at higher volumes. This finding suggests the presence of a possible floor effect for performance at smaller volumes whereby performance levels only decrease slightly as a consequence of medication because cholinergic blocking is less effective when the system and its input areas in the hippocampus are already impaired.

The numerically similar effects for the hippocampus and the cholinergic system would indicate that cholinergic input toward the hippocampus from the anterior basal forebrain (the main source of cholinergic projections to the hippocampus [7]) only partially regulates hippocampus-related determinants of delayed recall performance [17]. Because of the limited number of participants in our sample, however, we did not formally test for such a potential mediation effect.

Contrary to our observations, one might have expected that the effect of anticholinergic treatment on task performance should be less pronounced with larger volumes rather than with smaller volumes, if the former indicate a higher number of neurons with viable M1 muscarinic receptors that could compensate for a partial block of receptors. This assumption would hold if the variation in volume size was not linked as a state marker to performance but mainly represented a trait marker of reserve capacity, and if the level of anticholinergic effect was just above the threshold necessary to elicit functional effects. Here, however, we examined a sample of older adults where a low dose of the drug already induced significant decline in performance. This suggests that variation in cholinergic system integrity serves at least partially as a state marker for functional performance, and at smaller volumes the blockade of muscarinic receptors occurs in an already impaired cholinergic system. To better resolve this question, one would need to study the interaction between anticholinergic treatment and hippocampus and basal forebrain volumes in healthy young adults where variation in volume is expected not to be rate limiting for task performance.

Increased atrophy of the cholinergic basal forebrain and decline of cholinergic functioning over and above the effects of normal aging have been associated with cognitive decline and the development of dementia in neurodegenerative conditions such as Alzheimer's disease and Lewy body disorders [11]. Controlled anticholinergic stress tests have been proposed as prognostic markers for the development of dementia in the elderly [18], [19]. MRI-based measurements of the structural integrity of the cholinergic basal forebrain and its functionally relevant target areas may contribute important neurobiological information to such pharmacological stress tests.

In summary, our findings, which should be replicated in an independent sample, suggest that anticholinergic treatment leads to a partial uncoupling of hippocampus and basal forebrain atrophy from delayed recall task performance, providing in vivo evidence that both structures functionally subserve delayed recall mediated by cholinergic input to the hippocampus.

Research in context.

• 1.Systematic review: We reviewed the literature using medical databases, such as PubMed, abstracts from conferences, and cited work in review papers. Many studies investigated behavioral effects of cholinergic system dysfunction in rodents and monkeys, some studies also reported effects of long-term and short-term anticholinergic treatment in humans, but very few studies investigated the interaction between the integrity of key structures of the cholinergic system and the effect of anticholinergic treatment in humans.
• 2.Interpretation: Our findings suggest that cholinergic system structural integrity modulates the effect of anticholinergic treatment on delayed recall performance in humans. These data provide in vivo evidence on the structural underpinnings of cholinergic function in the human brain.
• 3.Future directions: If confirmed in an independent sample our data motivate the study of the potential role of cholinergic system structural and functional integrity to predict imminent cognitive decline in cognitively healthy older people.