Manipulating Decisiveness in Decision Making: Effects of Clonidine on Hippocampal Search Strategies

Abstract

Decisiveness is the ability to commit to a decision quickly and efficiently; in contrast, indecision entails the repeated consideration of multiple alternative possibilities. In humans, the α2-adrenergic receptor agonist clonidine increases decisiveness in tasks that require planning through unknown neural mechanisms. In rats, indecision is manifested as reorienting behaviors at choice points (vicarious trial and error [VTE]), during which hippocampal representations alternate between prospective options. To determine whether the increase in decisiveness driven by clonidine also entails changes in hippocampal search processes, we compared the effect of clonidine on spatial representations in hippocampal neural ensembles as rats passed through a T-shaped decision point. Consistent with previous experiments, hippocampal representations reflected both chosen and unchosen paths during VTE events under saline control conditions. Also, consistent with previous experiments, hippocampal representations reflected the chosen path more than the unchosen path when the animal did not show VTE at the choice point. Injection of clonidine suppressed the spatial representation of the unchosen path at the choice point on VTE laps and hastened the differentiation of spatial representations of the chosen path from the unchosen path on non-VTE laps to appear before reaching the choice point. These results suggest that the decisiveness seen under clonidine is due to limited exploration of potential options in hippocampus, and suggest novel roles for noradrenaline as a modulator of the hippocampal search processes.

SIGNIFICANCE STATEMENT Clonidine, an α2-adrenergic receptor agonist, which decreases the level of noradrenaline in vivo, has an interesting effect in humans and other animals: it makes them more decisive. However, the mechanisms by which clonidine makes them more decisive remain unknown. Researchers have speculated that clonidine limits the amount of mental search that subjects do when planning options. We test this hypothesis by measuring the mental search strategy in rats through hippocampal recordings. We find that clonidine limits the options searched by rats, suggesting that noradrenaline also plays a role in balancing exploration and exploitation in internally simulated behaviors, similar to its role in balancing exploration and exploitation in external behaviors.

Introduction

Decisiveness is the ability to commit to an option quickly and efficiently. It can be contrasted with indecision, which consists of the repeated consideration of multiple potential options. In humans, decisiveness can be measured on planning tasks, which require subjects to preplan or simulate future actions before their execution. For example, decisiveness can be measured behaviorally in the Tower of London task, which requires subjects to move discs from one stack to another under a complex set of rules (Shallice, 1982). Subjects solve the Tower of London task by mentally preplanning sequences of moves. Previous studies have reported that the administration of clonidine, an α2-adrenergic receptor agonist, reduced the time spent by subjects before moving, even while not changing the number of excess moves executed (Coull et al., 1995; Jäkälä et al., 1999). Researchers have suggested that clonidine increased decisiveness on these tasks by limiting mental exploration; however, it is currently technologically impossible to directly measure the mental search process of these tasks in humans, so this hypothesis has been impossible to test.

When rats come to a difficult choice, such as at a choice point on a T-maze, they sometimes pause and look back and forth, orienting serially toward the possible goals, as if reflecting an indecision between possible options. When it was first observed, this behavior was called “vicarious trial and error” (VTE) and was hypothesized to reflect a mental exploration of options (Muenzinger, 1938; Tolman, 1939; 1948). Consistent with this hypothesis, VTE increases during initial learning and on difficult choices with uncertainty, when animals would be expected to be indecisive, but vanishes as behavior automates with further experiences and animals become more sure of the correct choices (Blumenthal et al., 2011; Papale et al., 2012; Gardner et al., 2013; Schmidt et al., 2013).

Recent electrophysiological studies have found that VTE is accompanied by a prospective process in the hippocampus (Johnson and Redish, 2007). Hippocampal cells fire spikes at specific locations within the environment (the “place field” of each cell) (O'Keefe and Dostrovsky, 1971), and it is possible to decode the location represented by the population activity from a neural ensemble of simultaneously recorded place cells (Wilson and McNaughton, 1993; Zhang et al., 1998). During navigation and attentive pauses, the hippocampus shows a characteristic local field potential (LFP) oscillating in the 4–12 Hz range, called theta (Vanderwolf, 1969; O'Keefe and Nadel, 1978). During VTE events, hippocampal spatial representations sweep serially ahead of the animal within each theta cycle, on some cycles down one option, on other cycles down the other (Johnson and Redish, 2007). These representations run sequences to the potential goal of the animal (Gupta et al., 2012; Wikenheiser and Redish, 2015). Thus, VTE reflects rats' indecisive mental search processes, including planning and evaluating potential actions (Johnson et al., 2007; van der Meer et al., 2010).

Recently, it has been reported that VTE is suppressed by systemic injection of clonidine (Amemiya et al., 2014); however, that study did not examine neural activity associating with the effects of clonidine. Given that clonidine suppresses VTE in rats (Amemiya et al., 2014) much as it reduces planning in humans (Coull et al., 1995; Jäkälä et al., 1999), the ability to measure the neural correlates of mental search processes in rats provides an opportunity to determine whether clonidine is affecting decisiveness by limiting search processes. To determine whether the increase in decisiveness driven by systemic injection of clonidine entails changes in search processes, we recorded from hippocampal neural ensembles on a simple, well-characterized decision task known to show VTE, and measured the change in prospective representations.

Materials and Methods

Subjects

Male Fisher-Brown Norway hybrid rats (FBNF-1), age 11–17 months at time of implantation (n = 6), were maintained on a 12:12 h light/dark cycle. Rats were food restricted to no less than 80% of their body weight during behavioral training, and water was freely available in the home cage at all times. All procedures were conducted in accordance with National Institutes of Health guidelines for animal care and approved by the Institutional Animal Care and Use Committee at the University of Minnesota. Care was taken to minimize the number of animals used in these experiments and to minimize suffering.

Task

Rats were trained to perform a modified version of a Hebb-Williams maze (HWM; Hebb and Williams, 1946), similar to the multiple-T left, right, alternate task (Blumenthal et al., 2011; Steiner and Redish, 2012; Powell and Redish, 2014). The maze was a wooden rectangle box with carpeted floor and DUPLO brick walls that could be altered to change the internal maze portion (Fig. 1). At the end of a changeable center path, rats came to a high-cost choice point (CP) and had to make a left or right turn. If they made the correct choice, they received a food reward (two unflavored food pellets, 45 mg each, Research Diets) at the corresponding side feeder location (Fs) and at the center feeder location (Fc). The pellets were delivered using automatic pellet dispensers (Med Associates). If the rat made the wrong choice, no pellet was delivered at either the side feeder location or the center feeder location. Three different reward contingencies were used: turn left at CP (L contingency), turn right at CP (R contingency), or alternate between sides (A contingency). The alternation contingency required the rat to run to the side opposite to the most recently rewarded side. During training sessions, the reward contingency was held constant through an entire session, but changed randomly from session to session. Rats ran the task continuously for 30 min on each daily training session. Probe sessions lasted 40 min. During the probe test sessions, each session began with one reward contingency, but the reward contingency changed after ∼20 min, which was approximately the halfway point of the session. (We will refer to this change in reward contingency as the reward contingency “switch.”) No external cues were provided to signal the change in reward contingency (beyond lack of delivery of expected reward). Rats ran a subset of the six possible combinations of left (L), right (R), and alternation (A) contingencies (LR, LA, RL, RA, AL, AR), pseudorandomly. The central path remained constant throughout a given session, including both reward contingencies of a probe session, but changed from session to session.

Figure 1.

The left-right-alternate Hebb-Williams maze (HWM). A, General layout of the modified HWM used in these experiments. The maze consisted of a changeable central path, choice point (CP), and return rails with food reward sites (Fs, Side feeder; Fc, center feeder) leading back to the start (S) of the loop. Green dotted lines including CP enclose the (unmarked) choice area. Rats were trained to run through the central path and to turn left or right for reward in a continuous loop. Three reward contingencies were used: turn left (L), turn right (R), or alternate (A) for reward. During the probe test sessions, the reward contingency changed approximately halfway through the session. B, Right panels, Four examples of potential shapes of the center path. Red lines indicate movable walls that formed the changeable center path.

Surgery

After pretraining on the HWM, rats were chronically implanted with 14-tetrode hyperdrives (12 electrodes for recording, 2 for references) targeting the right CA1 region of dorsal hippocampus (3.8 mm posterior and 3.0 mm right-lateral from bregma). The details of the surgery have been described previously (Regier et al., 2015). Following the surgery, tetrodes and references were slowly advanced toward the pyramidal cell layer over ∼2 weeks. The hippocampal pyramidal layer was identified by the size and reversal of sharp-wave ripples, as well as by burst firing of cells in synchrony with the ripple portions of the sharp-wave ripple complexes. One reference was lowered to the hippocampal fissure for theta detection, and the other was left in corpus callosum or a quiet region of cortex to be used as a superficial reference.

Data collection

Neural data were recorded by a 64-channel Neuralynx Cheetah system and an overhead camera tracked subject position via LEDs on the headstage. LFPs were sampled at 2 kHz and bandpass filtered from 1 to 475 Hz. Spike trains were identified and recorded online using built-in filters, filtered from 600 to 6000 Hz, and then spikes were clustered offline. Neurons were separated into putative cells on the basis of specific waveform properties using KlustaKwik (K.D. Harris) and MClust 3.5 or MClust 4.0 (A.D.R.). Only clusters well separated from others and/or with L_ratio < 0.15 were used for analysis. Recording sessions consisted of 2 or 3 nonswitch sessions per rat and 6–9 switch sessions per rat under pharmacological manipulation (clonidine) or vehicle control (saline).

Pharmacological manipulation

Clonidine hydrochloride (Sigma-Aldrich), an α2 autoreceptor agonist, was dissolved in saline and injected at a dosage of 30 μg/kg. Saline vehicle was used as a control. For the recording sessions, clonidine or saline vehicle was intraperitoneally injected 30 min before the rat was placed on the track. The rat rested in its home cage for those 30 min. The doses and waiting time after the injection were chosen based on previous studies (Sara et al., 1995; Amemiya et al., 2014). The order of injection of either clonidine or saline was randomly assigned in each rat, and the investigator was blind to which was injected.

Histology

After the experiment was completed, tetrode locations were marked with small lesions by passing a small amount of anodal current (5 μA for 10 s) through each tetrode. After at least 2 d had passed, rats were anesthetized and perfused transcardially with saline followed by 10% formalin. Brains were stored in formalin followed by 30% sucrose formalin until slicing. Coronal slices were made through the area of the implantation and stained with Cresyl Violet to visualize tetrode tracks.

Data analysis

Choice behavior.

All behavioral measurements were calculated from the position tracking data. We calculated proportion of correct choices, time spent at the choice point (choice time, measured in seconds), and averaged running speed (cm/s) and acceleration (cm/s²).

zIdPhi measurement.

VTE was quantified by calculating zIdPhi, the z-scored integrated angular velocity across a pass through the choice point (Papale et al., 2012; Steiner and Redish, 2012). Briefly, a pass through the choice point was defined by the sequence of <x,y> coordinates as the rat ran through the choice area of the maze covering the choice point. The orientation of motion was found by applying the Janabi–Sharifi algorithm (Janabi–Sharifi et al., 2000) to the <x,y> sequence to determine <dx,dy> and taking the arctangent of <dx,dy> to determine the orientation of motion <φ>. The change in orientation of motion <dphi> was calculated by applying the Janabi–Sharifi algorithm to the <φ> sequence. The absolute value of <dphi> was calculated and integrated across the entire pass to produce an IdPhi for a given pass. Left and right passes were separately z-scored within each session to produce a zIdPhi measure for each pass through the choice point because the path through the choice point can have a different baseline of integrated absolute angular velocity depending on the specific path required by the changeable central path (maze-portion) used in a given day. Consistent with previous experiments (Steiner and Redish, 2012; Schmidt et al., 2013), zIdPhi distributed as a peak with a long tail skewed to the right (skewness = 4.3; see Fig. 2). VTE events were identified as samples in the long, right-skewed tail. zIdPhi values >0.5 were defined as VTE events.

Figure 2.

The rats expressed VTE in the choice area. A, VTE was quantified with zIdPhi in the choice area. Four sample passes through the choice point are displayed. Examples ➀ and ➁ were classified as non-VTE laps; examples ➂ and ➃ were classified as VTE laps. Gray-colored dots indicate rats' positions in the choice area for all passes in the session. Blue lines indicate the path of the animal during a single pass through the choice point. The title of each example shows the zIdPhi score for that pass. B, Distribution of zIdPhi from all passes through the choice point for all rats. Because zIdPhi distributes as a peak with a long tail, the mode of the distribution is less than zero, even though it is a z-scored measure. Laps with zIdPhi > 0.5 (dotted line) are defined as VTE. zIdPhi scores for the examples in A are marked on the distribution.

Theta LFP analyses.

To identify theta characteristics, the LFP data recorded from the hippocampal fissure were bandpass filtered between 4 and 12 Hz. To calculate theta frequency and power, we estimated instantaneous frequency and amplitude by applying the Hilbert transform to the filtered LFP. Average theta frequency was calculated by averaging the instantaneous frequency from all time during passes through the choice point or across the entire session. Theta power was calculated by averaging the instantaneous amplitude during passes across the choice point or during runs along the side paths (from the side feeder to the center feeder). Theta cycles were determined based on waveform-based theta as previously reported (Belluscio et al., 2012). To detect theta cycles, local minima and maxima in the theta were determined as peaks and trough of theta, respectively. Theta cycles were defined as the time between the troughs.

Place fields analyses.

Place fields were calculated from spikes recorded when the animal's running speed was >5 cm/s. Units with a mean firing rate <0.04 Hz (non–task-responsive cells) or >4 Hz (putative interneurons) after the exclusion of spikes were excluded from further analyses. To calculate the spatial tuning curves of cells, the maze was linearized into 65 bins, and the average firing rate (normalized by time at that given bin) was calculated for the left and right side paths separately (Gupta et al., 2012). This gave a tuning curve of 65 × 2 bins. Place fields were then identified as contiguous bins in which the firing rate was ≥10% the cell's session maximum firing rate. Fields separated by <3.5 cm were merged. Firing rate of place cells in a place field was calculated as averaged firing rate across the place field. To examine remapping effects, the correlation between the spatial tuning curves of place cells before and after the contingency switch was calculated.

Spatial decoding.

To examine the spatial representation in hippocampus, we used a one-step Bayesian spatial decoding algorithm (Zhang et al., 1998). We decoded spiking in each theta cycle to obtain a probability distribution across those 65 × 2 spatial bins. Because the sequences are known to occur within each theta cycle (Foster and Wilson, 2007; Gupta et al., 2012; Wikenheiser and Redish, 2015), we used the theta cycles as decoding time bins. Representation of chosen versus unchosen side was determined by the proportion of the decoded posterior ahead of the animal on the left or right side path. Sessions with at least 11 simultaneously recorded cells were used for decoding analyses.

To determine whether there was a change in the overall ability to represent position under the influence of clonidine, we measured confusion matrices. The confusion matrix was defined as the average posterior decoded probability over locations in the maze as a function of the animal's position in the maze. The entropy as spatial information content in each theta cycle was calculated as the entropy of the posterior decoded probability.

Statistics

To assess changes of behavioral data over a session, each session was divided into four phases, early laps (1–15) and late laps (16-) before the contingency switch and early laps (1–15) and late laps (16-) after the contingency switch. To compare effects of injection of clonidine across phases, we calculated a two-way ANOVA [drug condition as a between-subject factor × task phase as a repeated measure]. To evaluate how VTE and non-VTE laps differed during these four phases, we calculated a two-way ANOVA [drug condition × lap type (non-VTE/VTE) as between-subject factors]. Because some data (e.g., zIdPhi) are not normally distributed (Fig. 2B), which can compromise the reliability of ANOVA analyses, we also calculated a complimentary analysis, in which we applied Wilcoxon rank sum tests with Bonferroni correction for multiple comparisons measuring changes across the task phases in each drug condition, between saline and clonidine in each task phase, and the difference between non-VTE and VTE laps. To compare differences of decoding probability data, we used a three-way ANOVA [drug condition as a between-subject factor × rat condition as a between-subject factor × task phase as a repeated measure] and a two sample t test comparing non-VTE and VTE laps, and a paired t test was applied for between unchosen and chosen side. T tests were also applied to evaluate differences of decoding probability from zero. Box plots were used to show quartiles and distribution of data. The bottom and top edges of each box are the 25th and 75th percentiles, respectively; the thick line in the box is the median; the error bars show the minimum and maximum data points within 1.5 × IQR; and data points outside 1.5 × IQR are plotted individually as outliers.

Results

All rats ran at least 2 or 3 nonswitch sessions and 6–9 switch sessions under clonidine or saline injection; 28 saline sessions and 27 clonidine sessions were recorded and are analyzed here.

Rats learned both the initial and switch contingencies reliably (Fig. 3A). No differences were seen in the number of correct choices made between the saline and clonidine conditions (F_(1,10) = 0.28, p = 0.61 drug; F_(1,10) = 100.27, p < 0.0001 task phase: F_(1,30) = 0.83, p = 0.49 interaction). To determine statistical significance without an assumption of normality, Bonferroni-corrected Wilcoxon rank sum tests were applied. The Bonferroni-corrected rank sum test found an improvement in the proportion of correct laps taken as the session progressed both before and after the contingency switch but found no significant difference between the saline and clonidine conditions (Fig. 3A).

Figure 3.

Clonidine suppressed zIdPhi in the spatial decision-making task without affecting task performance. A, Proportion of correct laps. There was no difference in the proportion of correct laps under saline and clonidine conditions during the first 15 laps or during later laps either before or after the switch in reward contingency. The switch was done ∼20 min into the 40 min session, so rats could run a variable number of laps both before and after the switch. B, Average zIdPhi before and after the switch, separated by early and late laps. Note the significant decrease in zIdPhi under clonidine after the switch. C, Proportion of VTE laps. There were fewer VTE laps during the first 15 laps after the switch under clonidine, but this was not significant (p = 0.31). A–C, Center points indicate the median of the data; error bars indicate the range of the highest and lowest data points. Blue lines indicate saline condition. Red lines indicate clonidine condition. Blue dotted lines indicate significant difference between phases in saline condition. Red dotted lines indicate significant difference between phases in clonidine condition. Black dotted line indicates the significant difference between saline and clonidine condition in the early phase after the switch. *p < 0.0087, Bonferroni-corrected Wilcoxon rank sum test.

Consistent with previous observations (Blumenthal et al., 2011; Steiner and Redish, 2012; Amemiya et al., 2014), there was increased zIdPhi in the early phase of the task, which decreased as rats automated their behavior (Fig. 3B). zIdPhi also increased following the switch, indicating a significant increase in VTE after the switch in reward contingency (Fig. 3B). Clonidine inhibited the increase in zIdPhi after the contingency switch. A two-way ANOVA found significant effects of drug, task phase, and an interaction on zIdPhi (F_(1,10) = 6.93, p = 0.025 drug; F_(1,10) = 16.06, p < 0.0001 task phase: F_(1,30) = 4.10, p = 0.016 interaction). To determine statistical significance without an assumption of normality, Bonferroni-corrected Wilcoxon rank sum tests were applied. The Bonferroni-corrected Wilcoxon rank sum tests found that clonidine suppressed the increase of VTE after the contingency switch seen in the saline condition (p = 0.0087). In addition, we calculated the ratio of VTE laps in each phase (Fig. 3C). A two-way ANOVA found a significant effect of task phase, but not of drug or interaction (F_(1,10) = 0.001 p = 0.98 drug; F_(1,10) = 5.06; p = 0.0059 task phase: F_(1,30) = 0.58, p = 0.63 interaction). We also applied Bonferroni-corrected Wilcoxon rank sum tests on the proportion of VTE laps and found that the proportion of VTE laps increased significantly after contingency switch in the saline condition (p = 0.0022 late phase of before switch vs early phase of after switch in saline) but did not reach significance after Bonferroni correction in the clonidine condition (p = 0.026 late phase of before switch vs early phase of after switch in clonidine).

To investigate the characteristics of VTE, we compared behavior between VTE and non-VTE laps. We collected 2503 non-VTE laps and 295 VTE laps in the saline condition, and 1730 non-VTE and 245 VTE laps in the clonidine condition from the 6 rats. We compared the likelihood of taking the correct choice between non-VTE and VTE laps (Fig. 4A). A two-way ANOVA found that the likelihood of taking the correct choice was lower in VTE laps, but that there was no effect of drug or any interaction (lap type: F_(1,20) = 37.98, p < 0.001; drug: F_(1,20) = 0.24, p = 0.63; interaction: F_(1,20) = 0.001, p = 0.97). We also applied a rank sum test, which confirmed that the proportion of correct choices was lower on VTE laps for both saline and clonidine conditions (p < 0.0083); there was no significant effect of clonidine on the same lap type (non-VTE or VTE) between saline and clonidine conditions (p = 0.09 for non-VTE, p = 0.70 for VTE). There were significant effects of clonidine against VTE under saline and non-VTE under clonidine conditions (p < 0.0001), and between non-VTE under saline and VTE under clonidine (p < 0.0001).

Figure 4.

VTE relates to incorrect choice and entails taking longer to choose. A, The proportion of correct choices made on VTE laps and non-VTE laps under saline and clonidine conditions. B, Choice time on VTE laps and non-VTE laps under saline and clonidine conditions. *p < 0.0083, Bonferroni-corrected Wilcoxon rank sum test.

Not surprisingly, VTE events entailed rats spending more time at the choice point (Fig. 4B). A two-way ANOVA found that the choice time was longer in VTE laps, but found no significant effect of drug and no interaction (lap type: F_(1,20) = 88.15, p < 0.001; drug: F_(1,20) = 1.48, p = 0.24; interaction: F_(1,20) = 0.002, p = 0.97). Generally, choice time is not normally distributed; we thus also applied Bonferroni-corrected Wilcoxon rank sum tests and found that choice time was longer in VTE laps than in non-VTE laps in both saline and clonidine conditions (p < 0.0083); there was no significant effect of clonidine on when comparing the same lap type (non-VTE or VTE) between saline and clonidine conditions (p = 0.31 for non-VTE, p = 0.81 for VTE). There were significant differences between VTE under saline and non-VTE under clonidine (p < 0.0001), and between non-VTE under saline condition and VTE under clonidine (p < 0.002).

These data showed that clonidine suppressed, but did not eliminate VTE events on this task, and had no effect on the animal's ability to learn or execute this task. This is thus a particularly good task on which to examine the relationship between decisiveness and the hippocampal search processes that occur during VTE at the choice point of decision-making tasks because there will be no confounding effects of task performance. Forward spatial representations occurring during VTE can reflect the indecision during choice behavior as rats consider alternate goals (Johnson et al., 2007). For neural analyses, we used sessions with at least 11 simultaneously recorded cells (saline: n = 13 sessions, mean ensemble size = 24.2 ± 5.5 SD; clonidine: n = 14 sessions, mean ensemble size = 24.9 ± 9.0 SD).

To further ensure that task performance did not change under the influence of the drug, we measured velocity and acceleration through the choice area over the sessions (Fig. 5A,B). A two-way ANOVA found that there was a significant difference of velocity (F_(1,50) = 37.73, p < 0.0001) and acceleration (F_(1,50) = 30.5, p < 0.0001 for acceleration) between VTE and non-VTE paths, as expected, but no effect of drug (velocity: F_(1,50) = 0.18, p = 0.68; acceleration: F_(1,50) = 0.20, p = 0.66) or any interaction (velocity: F_(1,50) = 0.13, p = 0.72; acceleration: F_(1,50) = 0.24, p = 0.63). In addition, we also applied Bonferroni-corrected Wilcoxon rank sum tests on the speed and acceleration through the choice area and found equivalent results. Velocity was lower in VTE laps than in non-VTE laps in both saline and clonidine conditions (p < 0.0083), but we did not find any difference of either velocity or acceleration in same lap type (non-VTE/VTE) between saline and clonidine conditions (velocity: p = 0.94 for non-VTE, p = 0.78 for VTE, acceleration: p = 0.94 for non-VTE, p = 0.86 for VTE). As an additional control, we also calculated velocity and acceleration on the side paths between the side and center feeders. Wilcoxon rank sum tests found no effects of drug (velocity: p = 0.32, acceleration p = 0.40), indicating clonidine did not affect gross motor function.

Figure 5.

Running speed on the task. A, Averaged velocity in the choice area on non-VTE and VTE laps under saline and clonidine conditions. B, Averaged acceleration in the choice area on non-VTE and VTE laps under saline and clonidine conditions. C, Averaged velocity on the side paths from the side to center feeder under saline and clonidine conditions. D, Averaged acceleration on the side paths under both saline and clonidine conditions. *p < 0.0083, Bonferroni-corrected Wilcoxon rank sum test.

Theta power and frequency change with speed (McFarland et al., 1975; Sławińska and Kasicki, 1998). We therefore measured theta power and frequency as a function of drug and VTE (Fig. 6). Consistent with the observation that animals are running slower on VTE laps, theta frequency in the choice area was lower during VTE passes than non-VTE passes (F_(1,50) = 8.25, p = 0.006), but there was no effect of drug (F_(1,50) = 2.73, p = 0.11) or any interaction (F_(1,50) = 0.00, p = 0.99). However, there were no changes in theta power during time in the choice area (drug: F_(1,50) = 0.01, p = 0.91; lap type: F_(1,50) = 0.02, p = 0.88; interaction: F_(1,50) = 0.003, p = 0.96), consistent with previous observations that animals remain in theta throughout VTE (Johnson and Redish, 2007). In addition, we also applied Bonferroni-corrected Wilcoxon rank sum tests on the theta frequency and power through the choice area and found a significant difference only between non-VTE laps in saline and VTE laps in clonidine (p < 0.0083), but we did not find any difference of either theta frequency or power when comparing drug effects across the same lap type (non-VTE/VTE) (p = 0.15). As an additional control, we also measured theta frequency and power on the whole maze. A two-way ANOVA found no effect of drug, lap type, or any interaction between them in theta frequency (drug: F_(1,50) = 2.84, p = 0.10; lap type: F_(1,50) = 2.66, p = 0.11; interaction: F_(1,50) = 0.006, p = 0.94) or theta power (drug: F_(1,50) = 0.27, p = 0.60; lap type: F_(1,50) = 0.04, p = 0.85; interaction: F_(1,50) = 0.006, p = 0.94). In addition, we also applied Bonferroni-corrected Wilcoxon rank sum tests on the theta frequency and power on whole maze. We found no difference of theta frequency in any pairs (p > 0.16). We found no significant difference of theta power in any pairs when measured across the whole maze (p > 0.61).

Figure 6.

Characteristics of hippocampal theta. A, Averaged theta frequency in the choice area on non-VTE and VTE laps under saline and clonidine conditions. Averaged theta power was lower in VTE than non-VTE (ANOVA, F_(1,50) = 8.25, p = 0.006). Multiple-comparisons tests found that average theta power was lower in VTE in clonidine condition than non-VTE in saline. B, Averaged theta power in the choice area on non-VTE and VTE laps under saline and clonidine conditions. No significant differences were found in theta power. C, Averaged theta frequency on the whole maze as a function of lap type and pharmacological manipulation conditions. No significant differences were found. D, Averaged theta power on whole maze as a function of lap type and pharmacological manipulation conditions. No significant differences were found. *p < 0.0083, Bonferroni-corrected Wilcoxon rank sum test.

We next measured basic changes in hippocampal place cell characteristics (i.e., place field size, number of place fields, firing rate, and stability of place fields) and found no effects on any of the basic hippocampal place field characteristics as a function of drug condition (Fig. 7). We recorded 165 place cells under the saline sessions and 176 place cells under the clonidine sessions. The cells in the saline sessions had 436 place fields between them (2.64 fields/cell), and the cells in the clonidine sessions had 486 place fields between them (2.76 fields/cell). Rank sum tests did not find any significant differences in any of these characteristics as a function of drug condition (p = 0.62 for place fields size, p = 0.31 for number of place fields, p = 0.31 for firing rate, and p = 0.75 for stability of place fields).

Figure 7.

Characteristics of place fields. A, Size of place fields under saline and clonidine conditions. B, The number of place fields shown by each place cell under saline and clonidine conditions. C, Maximum firing rate in each place filed under saline and clonidine conditions. D, Correlation coefficient of spatial tuning curves before and after the contingency switch in each place cell under saline and clonidine conditions. No significant differences were found.

As defined above, indecision entails repeated search through multiple options, whereas decisiveness occurs by eliminating the repeated sampling of alternate options. Previous work has found that, during goal-directed behavior, hippocampal place cells represent paths to goal locations (Wikenheiser and Redish, 2015). Previous work has also found that, during VTE events, hippocampal representations alternate between options, as if the animal was repeatedly sampling from multiple goals (Johnson and Redish, 2007). We therefore asked whether the hippocampal representations were different under saline and clonidine conditions.

Figure 8 shows examples of decoding over different conditions as rats ran through the choice point. Typically, as animals ran through the maze, decoding was highly local, but during VTE events, decoding included maze components far ahead of the animal (consistent with Johnson and Redish, 2007). Examining decoding at the choice point, VTE laps including decoding to both forward paths of the maze in the saline condition, but non-VTE laps predominantly represented the chosen path (compare Fig. 8A,B). Although decoding during non-VTE laps under clonidine appeared similar to decoding during non-VTE laps under saline, only representing the chosen path (Fig. 8C), decoding during VTE laps under clonidine appeared quite different from under saline, only representing a single path, which was eventually chosen, even when the rat physically flirted with heading toward the unchosen path (Fig. 8D). Even when the rat's behavior included a VTE event (leaning toward one path, but choosing the other), the neurophysiological representation only reflected the chosen path.

Figure 8.

Forward spatial representation in hippocampus at the choice point. Decoding is shown in the top when the animal was at the choice point on a given lap. Red circle represents the rat's position. Bottom, Gray dots indicate the rat's position across the entire session. Blue lines indicate the path of the rat through the choice area on the given lap. Red circles represent the rat's position on the given lap. A, During non-VTE laps, under saline, hippocampal representations tended to represent only one side of the maze, which was the side eventually chosen. B, However, during VTE laps under saline, spatial representation as the rat passed through the choice point included representations of both sides of the maze. C, Decoding during non-VTE laps under clonidine appeared similar to that of saline, representing only one side of the maze, which was the side eventually chosen. D, However, under clonidine, spatial representation during VTE passes through the choice point still only represented a single direction, which tended to be the direction eventually chosen.

Previous studies have found that, during VTE, the representations are to one side or the other, serially and alternatingly, but not to both simultaneously. Directly comparing the proportion of posterior probability to each side on each theta cycle revealed that our data were consistent with these previous observations: within any given theta cycle, the representation was only to one side or the other, not both (Fig. 9).

Figure 9.

Joint distribution of decoding probability to each forward path in theta cycles at the choice point. Panels represent the joint distribution of decoding probability of forward paths between right side and left side under saline (A) and clonidine (B) conditions. Hot colors represent larger probability; cool colors represent smaller probability. There is very little decoding in the joint space, indicating that each theta cycle tended to express a particular side, showing that representations alternated between sides, consistent with the observations of Johnson and Redish (2007).

The implication of the examples in Figure 8 is that, under the influence of clonidine, even VTE laps contain only representation to one side, which was the one eventually chosen. To quantify this, we calculated the difference of decoding probability between chosen and unchosen paths at the choice point. As can be seen in Figure 10, the proportion of the posterior that represented the chosen path was high for non-VTE laps under both the saline and clonidine conditions. However, during VTE laps, under saline, the posterior was equally well divided between the two options (suggesting indecision). Interestingly, the posterior was preferentially on the chosen side during clonidine, even on VTE laps, suggesting that, even on VTE laps, the rat only considered one side, which was the one eventually chosen. A three-way ANOVA found a significant interaction between rat, drug, and representation (F_(2,1860) = 4.74, p = 0.0089). To determine whether there was increased representation of the chosen side relative to the unchosen in each condition, we applied one-sample t tests with Bonferroni correction for each condition comparing the difference against 0. Under saline, the difference between the two sides was not different from zero on VTE laps (Fig. 10A, blue bar; t₍₁₅₇₎ = 0.82, p = 0.41), consistent with previous studies that have found representations of both chosen and unchosen sides during VTE (Johnson and Redish, 2007). Also consistent with these previous experiments, decoding on non-VTE laps predominantly reflected the chosen side under saline (Fig. 10A, gray bar; t₍₁₀₂₃₎ = 7.87, p < 10⁻¹⁴). In contrast, under clonidine, decoding preferentially represented the chosen side under both non-VTE laps (Fig. 10B, gray bar; t₍₉₇₈₎ = 9.80, p < 10⁻²⁰), as well as on VTE laps (Fig. 10B, red bar; t₍₁₂₂₎ = 3.64, p < 0.001). However, a direct comparison between VTE under saline and clonidine conditions did not reach significance after correcting for multiple comparisons (t₍₂₇₉₎ = −2.2, p = 0.03).

Figure 10.

Difference of decoding probability between chosen and unchosen paths at the choice point calculated as p[chosen] − p[unchosen]. A, Difference of decoding probability between chosen and unchosen paths under saline. B, Difference of decoding probability between chosen and unchosen paths under clonidine. There was increased decoding to the chosen side on non-VTE laps under saline, but under both VTE and non-VTE laps under clonidine. Data are mean ± SE. ^#p < 0.001, probability different from zero.

To further identify when these effects occur as rats run through the choice point, we measured the amount of the posterior decoding probability assigned to each forward path (Fig. 11A, areas surrounded by green lines) after the choice point (Fig. 11A, area surrounded by magenta lines). To calculate the difference in representation between chosen and unchosen paths, we used Bonferroni-corrected paired t tests (α = 0.0025, correcting for 20 comparisons) at each position through the choice point pass (Fig. 12). Under saline, decoding on non-VTE laps showed a stronger representation of the chosen path over the unchosen path even before entry into the chosen path, but decoding on VTE laps only showed an increased representation of the chosen side after leaving the choice point (Fig. 12A,C). Under clonidine, decoding during the non-VTE laps showed increased representations of the chosen path over unchosen path even earlier than under saline. During VTE laps under clonidine, the decoding showed increased representations of the chosen path while the animal was still at the choice point, before the animal had entered the chosen path (Fig. 12B,D). Furthermore, decoding on VTE laps represented the unchosen path more than non-VTE laps, even after passing the choice point under saline, but not under clonidine (Fig. 12A,B; the dotted blue line separates from the dotted black line under saline controls, but the dotted red line does not separate from the dotted black line under clonidine).

Figure 11.

Decoding probability of chosen and unchosen paths across linearized position. A, Schematic showing location of decoded areas on the maze. Gray lines indicate rats' positions across all passes in an example session. Green regions represent the area declared as forward paths. Magenta rectangle represents the choice point for this measurement. B, Relationship between linearized position (S, Start; CP, choice point; Fs, side feeder; Fc, center feeder) and actual left/right side on the maze of the example session in A. Gray dots (lines) on each linearized position indicate corresponding 2D positions of right/left side on actual maze. Choice point (magenta-colored) was defined as a late position before the left paths and right paths separated. Green parentheses indicate the range of forward regions shown in A. Dotted rectangles represent the area zoomed in for Figure 12. C, Decoding probability of chosen and unchosen forward paths across linearized position under saline. D, Decoding probability of chosen and unchosen forward paths across linearized position under clonidine. Solid lines indicate decoding probability on the chosen path. Dotted lines indicate decoding probability on the unchosen path. Colored lines (blue/red) indicate data on VTE laps. Black lines indicate data on non-VTE laps. Green parentheses indicate forward regions. Dotted rectangles represent the area zoomed in for Figure 12. Approaching the choice point, decoding probability of the forward regions increased. After rats physically entered the chosen path, decoding probability of the chosen side was much higher than of the unchosen side. Data are mean ± SE.

Figure 12.

Decoding probability of chosen and unchosen forward regions while the rat passed through the choice point. A, Decoding probability of chosen and unchosen forward regions under saline. Decoding probability between chosen and unchosen paths differentiated before rats entered the chosen path on non-VTE laps, but only after entering the chosen path on VTE laps. B, Decoding probability of chosen and unchosen paths under clonidine. Clonidine hastened the differentiation of decoding probabilities, including moving the first significant decoding difference on VTE laps to before the rat had left the choice point. *p < 0.0025, chosen versus unchosen sides. Black represents non-VTE; blue represents VTE (saline); red represents VTE (clonidine), all indicating where the chosen sides were better represented than the unchosen side. †p < 0.0025, non–VTE-unchosen versus VTE-unchosen, indicating that, on certain VTE laps, the unchosen side contains increased representations relative to that expected from non-VTE laps. Data are mean ± SE. C, D, Difference of decoding probability was calculated as p[chosen] − p[unchosen]. C, Difference of decoding probability under saline. Decoding probability of the chosen path was larger than the unchosen path before rats entered the chosen path on non-VTE laps (left), but only afterward on VTE laps (right). D, Difference of decoding probability under clonidine. The differentiation of decoding probabilities between chosen and unchosen paths appeared earlier on both non-VTE (left) and VTE laps (right), including before the rat left the choice point. ^#p < 0.0025, probability different from zero. Data are mean ± SE.

Finally, to make sure the decoding results mentioned above were not due to effects of clonidine on decoding accuracy, we calculated the confusion matrix of decoding probability on the linearized maze and calculated the entropy of this confusion matrix. Figure 13 shows the confusion matrix of decoding probability. Overall, the spatial decoding algorithm decoded the rat's position well. No differences were seen between saline and clonidine conditions. This is verified by the fact that there was no difference of entropy of the decoded distribution between saline and clonidine across the whole maze (Wilcoxon rank sum test, p = 0.99; Fig. 14A). This lack of difference remained even when analysis was restricted to the choice area (Wilcoxon rank sum test, p = 0.75; Fig. 14B). These results suggest that the differences in forward decoding seen in Figures 9–11 between saline and clonidine are not attributable to effects of clonidine on decoding accuracy.

Figure 13.

Decoding accuracy was measured by examining the confusion matrix between decoded position and actual position under saline (A) and clonidine (B) conditions. Each plot represents the average decoding probability of each point in the linearized maze as a function of each point on the linearized maze: S, Start; CP, choice point; Fs, side feeder; Fc, center feeder. Hot colors represent larger posteriors. Overall, the algorithm accurately decoded the rats' positions under both saline and clonidine conditions.

Figure 14.

Entropy of the neural ensemble decoding posteriors. We measured the entropy of the decoded distribution at each theta cycle on whole maze (A) and restricted to the choice area (B). No differences were found in either of these measures as a function of drug condition.

Discussion

Systemic treatment with the α2-adrenergic receptor agonist clonidine makes humans more decisive. In rats, VTE has been hypothesized to reflect the indecision inherent in deliberative search processes. We found that clonidine reduced not only the amount of zIdPhi but also the spatial representation of unchosen paths as rats passed through the choice point, suggesting that clonidine makes rats more decisive by limiting mental exploration of options.

Clonidine is an α2-adrenergic receptor agonist that decreases the activity of noradrenergic neurons in locus ceruleus and decreases noradrenaline release through autoreceptors on noradrenergic neurons (L'Heureux et al., 1986; Abercrombie et al., 1988; van Veldhuizen et al., 1993; Kawahara et al., 1999; Fernández-Pastor et al., 2005). The locus ceruleus noradrenergic system has been hypothesized to control attentional processes through phasic burst activity that occurs during exploitation of a task and through tonic firing that increases exploration (Aston-Jones and Cohen, 2005). Clonidine inhibits the tonic activity of noradrenaline neurons without suppressing phasic activity (Saunier et al., 1993). Therefore, one would expect clonidine to decrease exploration. The results reported in this paper can be interpreted as a consequence of a partial decrease in the tonic levels of noradrenaline likely to decrease exploratory search processes, which would shift subjects from an exploratory behavioral mode to a more exploitative behavioral mode.

In any decision-making situation, animals must balance exploring additional choices to identify the best option against exploiting current knowledge to take the currently best known option (Sutton and Barto, 1998). Clonidine seems to shift agents from exploratory modes to exploitation modes. Experimentally, this can be observed as a decrease in indecision (exploration of multiple options) and an increased decisiveness. In situations where the state of the world is well known, that increased decisiveness can actually increase the value gained from a task (because one is no longer wasting time in indecisive exploration); however, in situations where the state of the world is unknown, that increased decisiveness will decrease the value gained from a task (because one does not explore the available options).

Previous studies have suggested that VTE behavior reflects a mental search process, alternately mentally exploring multiple options (Johnson et al., 2007). This mental exploration of alternate options can be directly observed in the hippocampal representations of those alternate options (Johnson and Redish, 2007). In the current study, we found that clonidine affected the hippocampal activity underlying mental search processes. Our results can extend the view of effects of clonidine on attentional processes from external stimuli to internal representations, suggesting that clonidine may modulate memory processes that focus on, retrieve, and maintain information necessary for making a decision.

In a previous study, clonidine reduced the number of VTE events but increased the time taken to make a decision (Amemiya et al., 2014). We did not see these effects in this experiment. The discrepancy of effects of clonidine on choice time between the previous study and current results may be due to differences in training and task design. In the previous work, rats were naive to the task at the time of testing and drug injection: they faced the choice situation at the first time; that is, it was a new experience. Therefore, rats might not have had any idea how to resolve the situation and may have needed additional effort to establish a new response/strategy to the choice situation. This may have, thus, prolonged the choice time. In contrast, in the current study, rats were well trained in the overall task, the basic task structure, and the possible contingencies before the test sessions under drug injection. Therefore, rats might already have established some response strategy knowing that they have to choose one side or the other. The more uncertain state in the previous study may have enhanced the effects of clonidine on choice time and led to differences in the number of VTE events and in the impairment of correct choice (Amemiya et al., 2014). The less uncertain state in this study allowed animals to show normal behaviors even under clonidine influence. This may have been due to the extensive experience by the animals in this task and an interaction between decision-making systems at the moment of decision on the left/right/alternate Hebb-Williams task used in these experiments.

Interestingly, although clonidine reduced the average zldPhi on this task, we actually observed a higher ratio of VTE to non-VTE laps under clonidine than under saline on the task used in this experiment (245 of 1740 = 0.14 under clonidine vs 295/2503 = 0.12 under saline, χ² test, p = 0.03). This effect is due to an interaction between decision-making systems that occurs on this task and the fact that rats ran more laps in the saline condition. On the left-right-alternate tasks, such as the one used here, rats initially use a flexible (deliberative) decision-making system, characterized by flexible behavior, VTE, and planning processes in hippocampus and ventral striatum (van der Meer et al., 2010; Blumenthal et al., 2011; Regier et al., 2015). However, within a single session, as animals continue to run additional laps, rats begin to use a more automated (habit) decision-making system, characterized by path stereotypy, a lack of VTE, and situation-action representations in dorsolateral striatum (van der Meer et al., 2010; Blumenthal et al., 2011; Regier et al., 2015). When there is a switch in reward contingency, the rats return to the flexible (deliberative) system, but that again automates as animals run more laps (Blumenthal et al., 2011; Regier et al., 2015). Because rats ran more laps under saline than under clonidine, the total lap count included more laps in the later phase, driven by the more automated (habit) decision-making system. Consistent with this explanation, we did find a lower average zIdPhi under clonidine than under saline in the early laps after the switch, but no difference in the later laps.

We observed that rats ran fewer laps under clonidine than under saline. Clonidine has antihypertensive and sedative/analgesic effects (Spyraki and Fibiger, 1982), as well as cognitive effects on attention and exploratory state (Aston-Jones and Cohen, 2005). We do not think motivational effects are likely to have caused the decrease in laps because there was no difference in running speed between conditions. Furthermore, clonidine has been reported to increase food intake (Katz et al., 1989), which would be expected to have increased the number of laps. Thus, we do not think the effects of clonidine are due to changes in motivational state.

When VTE was first observed in the 1930s, it was hypothesized to entail a mental search process (a “vicarious” exploration and test process) (Tolman, 1948). In humans, the mental imagination of future possibilities during deliberation entails an interaction between prefrontal cortices and hippocampus, with hippocampus playing a critical role in the creation of those episodic futures (Schacter and Addis, 2009; Maguire and Hassabis, 2011). In rats, hippocampal representations sweep ahead of the animal during VTE behaviors (Johnson and Redish, 2007), proceeding to the next potential goal (Gupta et al., 2012; Wikenheiser and Redish, 2015), and downstream structures such as the ventral striatum and orbitofrontal cortex encode value-related information (van der Meer and Redish, 2009; Steiner and Redish, 2012; Stott and Redish, 2014). If these hippocampal representations are reflecting mental exploration of possibilities, then a manipulation that limits that exploration (in favor of exploitation behaviors) should limit the hippocampal sequences explored. We found that clonidine, which is known to increase decisiveness in humans and has been hypothesized to limit mental exploration of alternate possibilities in humans, limited the mental exploration of possibilities seen in hippocampal representations in rats to a single option, which was the one chosen by the rat. These data strongly suggest that the hippocampal sweeps occurring during VTE behaviors are part of a mental decision-making process akin to deliberation because limiting the number of option searched also limited the options available to the rat. These data also strongly suggest that noradrenaline can affect the balance between exploration and exploitation in internal (mental) processes similarly to how it affects the balance between exploration and exploitation in external behaviors.