Differential Arc expression in the hippocampus and striatum during the transition from attentive to automatic navigation on a plus maze

Abstract

The strategies utilized to effectively perform a given task change with practice and experience. During a spatial navigation task, with relatively little training, performance is typically attentive enabling an individual to locate the position of a goal by relying on spatial landmarks. These (place) strategies require an intact hippocampus. With task repetition, performance becomes automatic; the same goal is reached using a fixed response or sequence of actions. These (response) strategies require an intact striatum. The current work aims to understand the activation patterns across these neural structures during this experience-dependent strategy transition. This was accomplished by region-specific measurement of activity-dependent immediate early gene expression among rats trained to different degrees on a dual-solution task (i.e., a task that can be solved using either place or response navigation). As expected, rats increased their reliance on response navigation with extended task experience. In addition, dorsal hippocampal expression of the immediate early gene Arc was considerably reduced in rats that used a response strategy late in training (as compared with hippocampal expression in rats that used a place strategy early in training). In line with these data, vicarious trial and error, a behavior linked to hippocampal function, also decreased with task repetition. Although Arc mRNA expression in dorsal medial or lateral striatum alone did not correlate with training stage, the ratio of expression in the medial striatum to that in the lateral striatum was relatively high among rats that used a place strategy early in training as compared with the ratio among over-trained response rats. Altogether, these results identify specific changes in the activation of dissociated neural systems that may underlie the experience-dependent emergence of response-based automatic navigation.

1. Introduction

Upon engaging in a previously unfamiliar task, attentive performance is typically required to accomplish a desired outcome. After extensive practice, performance becomes fixed and automatic. This experience-dependent transition from the use of attentive to automatic performance strategies is commonly studied in the context of spatial navigation; attentive (place) strategies rely on memory of the position of spatial landmarks to flexibly locate a goal, whereas automatic (response) strategies rely on a series of fixed movements that compose an inflexible route. This strategy transition is readily observed in numerous species, including humans (Schmitzer-Torbert & Redish, 2002) and rodents (Hicks, 1964; Packard, 1999; Packard & McGaugh, 1996), supporting the value of model systems to investigate its underlying neural mechanics.

The plus (cross) maze (e.g., Tolman, Ritchie, & Kalish, 1946) is a simple apparatus consisting of four arms built off a central square that is frequently used to study the place-to-response transition in rodents. In particular, on a dual-solution task (one that can be solved using place or response navigation; Hicks, 1964; Ritchie, Aeschllman, & Peirce, 1950), animals are trained in a room with an enriched extra-maze environment to find a reward in a static location (e.g., the west arm) from a static start position (e.g., the south arm). To identify which strategy is dominant at any particular moment, a single trial (probe) is administered by starting the animal from the opposite position to that used during training. The new position puts at odds the route associated with each navigational strategy and thus permits simple identification of the primary mode of performance.

Using this dual-solution plus maze design coupled with reversible neural inactivation techniques, Packard and McGaugh (1996) demonstrated a double dissociation between the expression of place and response strategies and their neural correlates. The expression of place navigation required the dorsal hippocampus and not the dorsolateral striatum, and the expression of response navigation required the dorsolateral striatum and not the hippocampus (see also Packard, 1999; Packard, Hirsh, & White, 1989). Further studies showed that the dorsal striatum was functionally heterogeneous, implicating the medial region in flexible spatial navigation and the lateral region in fixed cue-based response navigation (Devan & White, 1999; Yin & Knowlton, 2004).

How the activation of these distinct neural systems relates to the experience-dependent transition from place-to-response navigation, however, remains relatively unexplored. Several studies suggest that either system can control behavior at early and late time points of training (Martel et al., 2007; Packard, 1999; Packard & McGaugh, 1996; Yin & Knowlton, 2004); note that Packard and McGaugh (1996) found inactivation of the hippocampus produced arm entries during probe trials at chance levels early in dual-solution task training suggesting that the ability to engage striatal-dependent response strategies may be delayed under some conditions. If the temporal dynamics of activation across brain regions map well onto those of strategy expression (obtained from either brain-intact animals or those with site-selective dysfunction) remains an open question. Is hippocampal activation high early in training when attentive strategies dominate? Does hippocampal activity change with practice and experience? Does activity within the dorsolateral striatum rise only after extensive training, coinciding with the emergence of automatic strategies? Does the strategy transition correlate with a particular pattern of activation across neural structures?

To begin to address these questions, the current work employed a dual-solution task to elicit the strategy transition from place to response navigation. Importantly, this behavioral design was paired with post-performance assessment of Arc/Arg 3.1 (Arc), an immediate early gene (IEG) and proposed marker of neural activity (e.g., Guzowski, McNaughton, Barnes, & Worley, 1999; Pinaud & Tremere, 2006; Vazdarjanova et al., 2006). For example, Arc transcription is enhanced in response to electrical stimulation, spatial exploration, and learning and memory (Daberkow, Riedy, Kesner, & Keefe, 2007; Guzowski, Setlow, Wagner, & McGaugh, 2001; Guzowski et al., 1999; Vazdarjanova et al., 2006; for review, see Bramham et al., 2010; Pinaud & Tremere, 2006). Moreover, the proportion of hippocampal neurons expressing Arc following exploration is comparable to the proportion of neurons encoding spatial information as measured by vivo electrophysiology (see Guzowski et al., 2006 for discussion; Wilson & McNaughton, 1993). Altogether, relating Arc expression measures in the hippocampus and striatum to navigation strategy should clarify how these dissociated neural systems are engaged and interact during the experience-dependent transition from attentive to automatic performance.

Complementary to this approach, we examined training-dependent changes in vicarious trial and error (VTE) behaviors. VTE refers to the tendency for rats to pause at a choice point and look back and forth toward potential destinations (Muenzinger, 1938; Muenzinger & Gentry, 1931; Tolman, 1948). As VTE is associated with deliberation (Papale, Stott, Powell, Regier, & Redish, 2012; van der Meer, Kurth-Nelson, & Redish, 2012) and place navigation (Gardner et al., 2013; Schmidt, Papale, Redish, & Markus, 2013), and correlated with hippocampal metabolism (Hu, Xu, & Gonzalez-Lima, 2006), this behavioral measure may further clarify the degree to which attentive systems are recruited during task repetition.

2. Methods

2.1. Animals

Sixty-four male Long-Evans rats (275–500 g; 2–4 months of age) were used for experimentation. These rats were bred in-house at George Mason University (n = 47) or ordered from Charles River Laboratories (Wilmington, MA; n = 17: ~ 175 g at the time of arrival). Locally and commercially derived animals were similarly distributed across experimental groups. Prior to experimentation, animals were housed two to three per cage. All methods were carried out in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the George Mason University Institutional Animal Care and Use Committee.

2.2. Apparatus

Rats were trained to find a food reward (Froot Loop cereal; Kellogg’s) on a maze positioned in a room with a rich heterogeneous extra-maze environment (Fig. 1). We used a previously described multi-choice maze (the Opposing T’s: OpT maze) set in a plus maze configuration (see Gardner et al., 2013; minor modifications are detailed). Briefly, four arm segments (north, east, south, and west) built off a central square (choice/decision point) were utilized. The south and north arms were potential starting positions, with identical opaque start boxes affixed to the ends of each arm. Reward cups were placed at the ends of the east and west arms. On any given run, three of the four arms were accessible, which restricted the maze to a “T” shape. The south, east, and west arms were open during training; the north, east, and west arms were open during probe trials. To limit falls during maze runs without restricting visual access to the extra-maze environment, clear Plexiglas railings were attached at the ends of each of the four arm segments.

Fig. 1.

Behavioral testing and experimental time line. (A) Rats were trained from a consistent starting position (e.g., south) to find a food reward (e.g., west; filled circles indicate rewarded food cups) in a room with a heterogeneous extra-maze environment. (B) To probe the dominant strategy (place or response) at various training stages, the rat was started from the opposite (e.g., north) arm. A place strategy was identified as entry into the arm previously rewarded (e.g., west). A response strategy was identified as the use of the previously rewarded turn (e.g., left). (C) The experimental time line is illustrated. Prior to training, rats were food-restricted, habituated to the experimenter, shaped to approach and consume Froot Loop cereal from reward cups, and given two days to explore the maze. To identify the changing reliance on place and response navigation with task repetition, a probe trial was administered after task acquisition, and subsequently every seventh day. ^*Training was terminated either after the first or fifth probe trial. The brains of a subset of rats were processed for in situ hybridization experiments targeting the immediate early gene Arc (see Section 2 for full detail).

2.3. Behavioral training

Food restriction and habituation were largely implemented as previously described (Gardner et al., 2013). Rats were individually-housed and maintained at 85% of their free-feeding weight throughout the experiment. While rats were brought down to their target weight, they were handled for five minutes each day. After being held a minimum of seven days and after meeting their target weight, rats were habituated to Froot Loop (FL) cereal (Fig. 1C: shaping) and given five minutes to explore the maze for each of two days (Fig. 1C: maze habituation).

The day after maze habituation, a one-time pre-training trial was administered after which training commenced. During these trials, an animal was placed in the south start box. After ~10 s, the front door to the box was remotely raised, using a pulley, providing the rat access to the maze. If a rat did not exit the start box after 180 s, the experimenter placed the animal on the maze directly in front of the box and closed its door to restrict reentry. During all trials, the experimenter stood ~3 feet behind the south arm. The pre-training trial was unrewarded (no food was placed on the maze). On this trial, the arm first entered (with the full body excluding the tail) determined an animal’s turn preference, and the opposite arm/turn was rewarded (with half of a piece of FL cereal) on all subsequent training trials for a given subject. This procedure ensured that all animals explored both arms at least once over the course of the experiment.

Following the pre-training trial, and during all daily training sessions thereafter, six training trials were administered with an inter-trial-interval of ~45 s. Rats were started from a consistent arm (south arm) and removed once they found the food reward. For any given rat, as the locations of the food and starting position were unchanged across and within days of training, the task could be solved relying on landmarks to find the position of the food (e.g., going west) or a fixed body turn (e.g., turning left; Fig. 1). Rats were allowed to self-correct and re-trace their steps to find the food reward. The time to find the reward and the first arm entered were recorded. Correct trials were identified as those during which the rat entered the rewarded arm and reached the FL reward without entering the unrewarded arm. Training sessions proceeded daily, for a minimum of four days, and until a rat correctly navigated the maze on all six daily trials of the most recent training day and at least ten of the twelve trials over the most recent two days.

2.4. Probing strategy dominance

The day after meeting these performance criteria, a single probe trial was administered in place of training (Fig. 1). On the probe, all conditions were identical to training except that the rat was started from the north arm (the opposite of that used for training) and both the east and west arms were rewarded (as neither choice was “wrong”). If an animal attended to environmental landmarks and entered the same arm as that rewarded during training, it was documented to have used a place strategy. If the rat used the same turn rewarded during training (and thus entered the opposite arm), it was documented to have used a response strategy (Fig. 1B). The day following the probe trial, training re-commenced for an additional four weeks with additional probe trials given every seventh day (in lieu of training on those days; Fig. 1C). All training and probe trials were videotaped for analysis. Vicarious trial and error (VTE) was quantified on each probe run by summing the number of times a rat paused at the decision point of the maze and the number of times it partially entered the east and west arms prior to making a full entry (e.g., see Gardner et al., 2013).

2.5. Tissue collection

For a subset of rats (n = 37), testing was terminated following the first strategy probe; for the remaining (n = 27) animals, testing was terminated following the fifth probe trial (Fig. 1C). Unequal group samples are due to an initial cohort that was assigned to the relatively short training schedule to assess strategy use on the first probe. When possible, subsequent cohorts were pseudorandomly assigned to either the one or to the five probe condition. To assess changes in Arc mRNA expression in dorsal hippocampus and dorsal striatum during the transition from place to response navigation, brains from four rats that used a place strategy on their first probe and brains from four rats that used a response strategy on their fifth probe were processed for in situ hybridization (ISH). Specifically, five minutes after the end of the applicable probe trial, the rat was taken to a surgery room, immediately anesthetized with isoflurane, and perfused intra-cordially with ice-cold 4% paraformaldehyde mixed in diethylpyrocarbonate (DEPC)-treated phosphate-buffered saline solution. This time line was used to capture strong nuclear Arc mRNA expression among neurons active during maze exposure (Guzowski et al., 1999, 2006). Following perfusion, each brain was excised and placed in the paraformaldehyde solution overnight at 4 °C. Brains were then submerged in a 30% sucrose solution mixed in DEPC-treated water for a minimum of four days.

Brains were sectioned (30 μm) in the coronal plane at −25 °C. Sections were mounted on slides, air-dried overnight and subsequently stored at −70 °C. For each brain, four sections were collected on each of at least four slides (Colorfrost Plus Slides: Fisher Scientific). To facilitate comparison across brain regions within a given animal, each slide contained two sections of dorsal striatum (between 1.0 mm and −1.0 mm from bregma along the anterior–posterior axis; Fig. 2B; also see Fig. 3B) and two sections from dorsal hippocampus (between −2.5 mm and −4.5 mm from bregma along the anterior–posterior axis; Fig. 2A; also see Fig. 3A). These coordinates were chosen to assess regions previously implicated in attentive spatial and automatic cue-based navigation (e.g., Devan, McDonald, & White, 1999; Devan & White, 1999; Packard, 1999; Packard & McGaugh, 1996; Yin & Knowlton, 2004). Colorimetric ISH targeting the mRNA transcribed by the immediate early gene Arc/Arg 3.1 was performed using full-length digoxigenin-labeled riboprobes, detected using the APBCIP/NBT system (see Kessler, Höltke, Seibl, Burg, & Mühlegger, 1990; Wehr et al., 2009).

Fig. 2.

In situ hybridization targeting the immediate early gene Arc. Arc mRNA expression was measured from tissue sections of (A) dorsal hippocampus (between −2.5 mm and −4.5 mm from bregma along the anterior–posterior axis) and (B) dorsal striatum (between 1.0 mm and −1.0 mm from bregma along the anterior–posterior axis). Typical colorimetric ISH (relying on the AP-BCIP/NBT system; see Section 2) results are presented for antisense and sense riboprobe application and together demonstrate the specificity of the experimental technique. Neuronal sub-fields measured include CA1, CA3 and dentate gyrus (DG) of the hippocampus and medial (DMS) and lateral (DLS) striatum.

Fig. 3.

Hippocampal and striatal regions of interest. (A) Dorsal hippocampal Arc mRNA expression was measured in the pyramidal cell layer of CA1 and CA3, and in the granule cell layer of dentate gyrus (DG). (B) Dorsal striatal Arc mRNA expression was measured in medial (DMS) and lateral (DLS) sub-fields. (A–B) Exemplar regions of interest (shaded) and background regions (selected in areas that contained little to no punctate staining) are illustrated.

2.6. Probe generation

The rat Arc riboprobe DNA template (Lyford et al., 1995) was linearized through restriction digest (XhoI: sense probe; EcoRI: antisense probe, obtained from New England Biolabs). Full-length riboprobes were synthesized using T3 (sense) or T7 (antisense) RNA polymerase (Roche) in the presence of digoxigenin-labeled UTPs (DIG RNA labeling kit: Roche) and RNase inhibitor (Roche). The DNA template was degraded with DNase I (Roche) and synthesized riboprobe was extracted by spin column (OmegaBiotek), verified using gel electrophoresis, and stored at −70 °C.

2.7. In situ hybridization

For each ISH experiment, probe was diluted (1:40), denatured for ten minutes in hot (70 °C) hybridization buffer (Sigma), and applied to warmed tissue sections (250 μl/slide). A lipid ring drawn around the tissue (Pap Pen Liquid Blocker: Ted Pella), coverslipping, and inclusion of SSC/50% formamide (20 ml on a paper towel) in the Perspex incubation box prevented evaporation of the probe solution. Hybridization proceeded overnight at 62 °C. On day two, slides were washed in SSC/50% formamide at 62 °C for 15 min to remove the coverslips. Sections were washed an additional three times in SSC/50% formamide at 62 °C (30 min each). Slides were then washed in maleic acid buffer containing 0.1% Tween 20 (MABT) three times at room temperature (30 min each). Blocking solution (MABT, 20% sheep serum, 20% blocking reagent: Roche) was applied (250 μl/slide) and slides were incubated in a humidified chamber containing phosphate buffered saline (20 ml on a paper towel) for one hour. Blocking solution was removed and anti-digoxigenin antibody conjugated to alkaline phosphatase (Roche) diluted (1:1500) in blocking solution was applied (~250 μl/slide). Slides were re-placed in the humidified chamber and incubated overnight at room temperature. To begin day three, slides were washed four times in MABT (five minutes each) followed by two washes (ten minutes each) in an alkaline phosphatase staining buffer (0.1 M Sodium Chloride, 0.05 M Magnesium Chloride, 0.1 M Tris, 0.1% Tween 20, pH 9.5). Slides were incubated for four hours at 37 °C in a mixture of 10% (w/v) high molecular-weight polyvinyl alcohol, 8% Levamisole Solution (Vector Laboratories), 3.5% NBT (Roche), and 2.6% BCIP (Roche) made up in alkaline phosphatase staining buffer. The colorimetric reaction was stopped by washing slides twice in phosphate-buffered saline containing 0.1% Tween 20 (ten minutes each) and twice in deionized water (ten minutes each). Slides were dehydrated in a graded series of ethanol (70–100%), cleared in xylenes (Fisher), and permanently cover slipped (Permount). All prehybridization solutions were DEPC-treated (RPI corp) and all working surfaces and instruments were maintained RNase free with RNase Away (Fisher Scientific). Each ISH experiment included all comparative conditions and included slides that were incubated with sense riboprobe as a control. Unless stated otherwise, reagents were obtained from Sigma (St. Louis, MO).

2.8. Image analysis

Images of sections were collected using bright-field microscopy (Olympus AX70) under 2× magnification, and saved as TIFF files for analysis using ImageJ (http://imagej.nih.gov/ij). Images of sections incubated with sense riboprobe were used to confirm the specificity of the probe to bind Arc mRNA, and generally showed little staining relative to those incubated with antisense probe (Fig. 2A and B). ISH experiments for which slides incubated with sense probe displayed equivalent staining to those incubated with anti-sense probe (indicating non-selective binding) were excluded from analysis.

Arc mRNA expression, quantified as the mean grayscale intensity corrected for background, was measured from images of sections incubated with antisense riboprobe. In particular, expression was measured in the pyramidal cell layer of CA1 and CA3, in the granule cell layer of dentate gyrus (DG), and dorsolateral and dorsomedial striatum. Regions of interest are presented in Fig. 3. Background regions were selected from proximal areas containing little to no punctate staining (Fig. 3). Regional boundaries of each substructure were verified using an adult rat brain atlas (Paxinos & Watson, 2008). In addition, hippocampal regions of interest were drawn to ensure their inclusion in the applicable cell body layer. To facilitate consistent measurement, the surface area sampled from a given neural structure was maintained across sections. A researcher blind to the experimental condition with which an image was associated performed the image analyses.

2.9. Statistics

ANOVA was conducted to assess the effects of training duration on choice accuracy, reward latency, and VTE. Logistical regression with binary variables, fit with generalized estimating equations (Davis, 2002), was utilized to compare strategy use across training duration; Chi Square analysis was used to corroborate these findings. Independent-sample t-tests were conducted to determine effects of training stage on Arc expression, and strategy group on VTE. For select comparisons, Cohen’s d was computed to estimate effect size. Statistical significance was interpreted using the criterion of p < 0.05. False discovery rate correction was applied to multiple comparisons (Benjamini & Hochberg, 1995). Statistical analyses were performed using SPSS (IBM), Excel (Microsoft), and R (Dalgaard, 2008).

3. Results

3.1. Task acquisition and strategy reliance

Rats were trained daily (6 trials/day) to perform a dual-solution task on a plus maze (Fig. 1). With training, animals typically reduced arm entry errors, and reduced their latency to locate the reward (p < 0.001; Fig. 4). For example, on the first day of training, rats entered the rewarded arm first on ~60% of the trials taking ~57 s to reach the reward. In contrast, by the fourth training day, rats infrequently entered the unrewarded arm (less than 15% of the time), taking ~30 s to find the goal. Rats were trained for a minimum of four days, and until they reached 100% trial accuracy on the most recent training day and at least ~67% accuracy on the previous training day (see Section 2). About 47% of rats reached these criteria by the fourth day, ~77% by the fifth day, and ~87% by the sixth day.

Fig. 4.

Dual-solution task performance. Rats reduced their latency to find the goal (A–C) and arm entry errors (D–F) across days of training. (A–B, D–E) Equivalent learning curves between rats that used place strategies compared with those that used response strategies on the first probe suggest strategy reliance did not influence task performance. (C, F) High levels of performance were maintained with continued training. Rats that utilized place navigation on the first probe: n = 33; Rats that utilized response navigation on the first probe: n = 31. Error bars indicate ± one standard error of the mean.

To probe strategy reliance, the day after training criteria were met, a rat was started from the opposite arm to that used during training. If the rat relied on the spatial arrangement of extra-maze cues to locate the position of the reward, and thus entered the arm rewarded during training, it was classified as using a place strategy. If the rat relied on a specific motor turn, and thus made the practiced turn, entering the arm unrewarded during training, it was classified as using a response strategy (Fig. 1; see Section 2). On this first probe, ~52% of the rats used a place strategy (Fig. 5A). Moreover, the learning rate (expressed by accuracy or reward latency) was equivalent between those rats that used a place strategy and those that used a response strategy (p > 0.10), indicating strategy reliance did not influence performance level (Fig. 4). After the first probe, training re-commenced from the original start arm with subsequent probes (from the opposite start arm) given every seventh day (for a total of five probes across testing; Fig. 1C). Task performance, on average, was maintained at a high level throughout the remainder of the experiment; choice accuracy remained greater than ~95%, whereas the latency to locate the reward continued to decrease until reaching asymptotic values around the end of the fourth week of testing (Fig. 4). Moreover, with task repetition, rats gradually transitioned to almost exclusive reliance on response navigation. On the fourth probe, ~85% of animals used a response strategy, a proportion that increased to ~93% on the fifth and final probe (Fig. 5A; p < 0.01 when compared to the first probe).

Fig. 5.

Strategy reliance and vicarious trial and error across training sessions. (A) On the first probe, about half (52%) of rats relied on spatial (place) navigation. With continued training, response navigation dominated, as shown by an almost exclusive reliance on response strategies (93%) by the fifth and final probe. (B) Similar to the transition from place to response navigation, vicarious trial and error (VTE) decreased from the first to last probe trial. (B inset) However, this experience-dependent reduction in VTE may be modulated by strategy engagement. Notably, place and response strategies were equivalent relatively early in training; a modest difference in VTE between strategies increased after extended training. Error bars indicate ± one standard error of the mean.

Individual strategy analysis (see Gardner et al., 2013 for additional detail) of animals that completed all five probe trials revealed that, with repeated training, the majority of rats transitioned to the use of response strategies (56%); twenty-six percent of rats relied on response navigation throughout testing. Only one rat invariably relied on place navigation, and only two (7%) transitioned to the use of place navigation. Two animals did not fit these strategy classifications: one used a response strategy on all but the third probe, whereas the other consistently alternated between strategies.

3.2. Vicarious trial and error

On probe runs, we measured VTE, a proposed index of deliberation (Gardner et al., 2013; Papale et al., 2012; Schmidt et al., 2013; van der Meer et al., 2012). VTE was quantified by counting the number of pauses and discrete head orientations toward potential choices that occurred at the decision point of the maze (see Section 2; Gardner et al., 2013). VTE occurrence was highest on the first probe trial, and decreased with continued task experience (p < 0.05; Fig. 5B). In particular, VTE decreased twofold comparing VTE events on the fifth probe with those on the first probe. Thus, VTE was generally more frequent during place trials and less so during response trials. All conclusions on strategy recruitment and VTE across training sessions were maintained when restricting analysis to those rats that completed all five probes; specifically, upon excluding rats that were sacrificed after the first probe trial (see Section 2), with task experience, reliance on response navigation increased (probe 1: 63% vs. probe 5: 93%; p < 0.05) as VTE events decreased (probe 1: 2.0 vs. probe 5: 0.7; p < 0.05). Moreover, strategy (p > 0.07) and VTE (p > 0.10) outcomes measured on the first probe were equivalent between rats that were subsequently sacrificed after the first probe and those that continued training.

Notably, however, VTE was equally frequent during place and response trials early in testing (e.g., probe 1; Fig. 5B Inset; p > 0.10). Due to the small sample of rats that relied on place navigation on probes 4 (n = 4) and 5 (n = 2), it is difficult to make definitive conclusions upon comparing VTE across strategies late in training. Nonetheless, these preliminary data suggest that while VTE events associated with response navigation remained relatively infrequent after extensive training (i.e., probes 4–5), VTE events associated with place navigation late in training were closer in frequency to that observed early in training (see Fig. 5B Inset). Furthermore, mean VTE across all probe sessions was relatively high among rats that invariably used a place strategy (M = 2.5, n = 1) or that transitioned to the use of place navigation (M = 2.1, n = 2), as compared with VTE among rats that transitioned to response navigation (M = 1.3, SD = 0.5, n = 15) or that relied upon a response strategy throughout the experiment (M = 0.8, SD = 0.7, n = 7; p < 0.05).

3.3. Arc expression across learning stages

Colorimetric ISH using the AP-BCIP/NBT system (e.g., Kessler et al., 1990; Wehr et al., 2009) targeting Arc/Arg3.1 mRNA was performed to identify patterns of neural activation across dorsal hippocampus and dorsal striatum during the place-to-response strategy transition (see Section 2). This ISH procedure leaves a bluish-purple precipitate localized to cells that contain Arc mRNA. Basal Arc mRNA expression levels are low and increase quickly and dramatically in an activity-dependent manner (Guzowski et al., 1999; Pinaud & Tremere, 2006) selectively in alpha-CaMKII-expressing neurons within the striatum and pyramidal cell layer of the hippocampus (Vazdarjanova et al., 2006). Therefore, colored (Arc positive) cells are thought to indicate the neurons that were activated during recent experiences. Measurement of the color intensity within a particular brain region is used here as an index of regional neural activation during maze traversal (probe trials).

Hippocampal Arc mRNA expression was higher among rats that used a place strategy on the first probe than among those that used a response strategy on the fifth probe (Fig. 6A). This effect, while present in all hippocampal sub-regions, was strongest in CA1 and DG (p < 0.01; d = 3.7 and d = 2.4, respectively); the distinction in Arc expression between strategies did not reach statistical significance in CA3 (p > 0.10; d = 0.9). These data suggest that hippocampal activation is high early in training when animals are attentive, and declines with repeated practice, which coincides with the emergence of automatic navigation. In contrast, although there was a mild training stage-dependent increase in lateral (d = 0.7) and decrease (d = 0.6) in medial sub-fields, striatal Arc mRNA expression appeared relatively stable across conditions (Fig. 6B; p > 0.10). With the exception of a positive association between Arc expression in CA1 and DG (r = 0.83, p < 0.05), inter- and intra-regional correlations were non-significant. However, note the relatively small sample of subjects (n = 8) available for this type of analysis.

Fig. 6.

Distinct patterns of Arc expression within hippocampus and striatum correlate with the experience-dependent emergence of response navigation. Arc mRNA expression in the dorsal hippocampus (CA1; CA3; dentate gyrus: DG) and dorsal striatum (medial: DMS; lateral: DLS) was quantified as the mean grayscale intensity (corrected for background; see Section 2; Figs. 2 and 3). Expression among neural structures from rats that used a place strategy on the first probe and from those that used a response strategy on the fifth probe is displayed. (A) Hippocampal Arc mRNA expression is high early in training when attentive performance dominates, and declines with task repetition which coincides with the emergence of response navigation. In contrast, Arc mRNA expression in medial and lateral striatal fields remains relatively stable across testing. (C–E) However, on an individual level, the ratio of expression in hippocampus to DLS (C), but not to DMS (E), and (D) the ratio of expression in medial to lateral striatal sub-fields discriminate training stage; this latter result stems from modest (non-significant) and opposing effects on Arc of training duration/strategy engagement in medial and lateral striatum. Bars indicate mean; circles represent individual data points; ^*p < 0.05.

Upon initial assessment, the number of VTE events on probe runs was not significantly correlated with hippocampal Arc mRNA expression (p > 0.10). However, we found enhanced expression levels in CA1 and DG (p < 0.05; d = 2.5 and 1.8, respectively) from rats that demonstrated some degree of VTE (arbitrary expression units: CA1: M = 103.2, SD = 20.1; DG: M = 47.1, SD = 13.0) compared with expression in those that did not (CA1: M = 59.8, SD = 14.6; DG: M = 24.7, SD = 12.2). These data are in agreement with the role of the hippocampus in deliberative navigation as assessed by VTE (Gardner et al., 2013; Papale et al., 2012; Schmidt et al., 2013; van der Meer et al., 2012). Given that VTE and training stage were correlated, we also calculated semipartial correlations. In both CA1 and DG, training stage appeared to explain a considerably larger amount of variation in Arc expression (r = 0.64 and 0.58, respectively) than did VTE (r = 0.17 and 0.14, respectively).

To explore the possibility that relative activation across brain regions could discriminate learning stage, we also computed the ratios of Arc mRNA expression in dorsal hippocampal to dorsal striatal sub-regions, and expression in medial to lateral striatal subfields for each animal (Fig. 6C–E). As expected, the mean ratio of Arc expression in each hippocampal sub-field to that in the lateral striatum was significantly greater in rats that used a place strategy than in those that used a response strategy (p < 0.01). In contrast, the expression ratios of hippocampal sub-fields to medial striatum were equivalent between conditions (p > 0.10).

The ratio of expression in dorsal medial to that in dorsal lateral striatum was increased in rats that used a place strategy early in training compared with the expression ratio in rats that used a response strategy late in training (p < 0.01). This result stems from the summation of two non-significant effects: a decline in effect size found in medial striatum and an increase in effect size found in lateral striatum across testing. Importantly, the size of this effect (Cohen’s d = 1.7) was almost threefold larger than that found when considering either medial or lateral striatum separately. Together, these findings suggest that relative activity patterns across place and response neural systems may be good correlates of the dominant mode underlying performance across task repetition.

4. Discussion

This research begins to identify activation patterns across hippocampus and striatum that may correspond to the experience-dependent transition from attentive (place) to automatic (response) performance. This was accomplished by quantifying expression levels of the activity-dependent IEG Arc among rats that used a place strategy early in training and among those that used a response strategy after extensive training on a dual-solution plus-maze task. This approach and resulting data provide support for specific hypotheses on the temporal dynamics of neural activation that underlie the experience-dependent emergence of response navigation. Measures of vicarious trial and error provide additional markers of navigational system engagement across repetitious training.

Rats reached a high level of performance on the dual-solution task relatively quickly (~5 days). Moreover, the day after meeting performance criteria for the first probe (see Section 2), about half of the rats relied on place navigation. Conversely, after extensive practice, rats transitioned to almost exclusive reliance on response navigation with minimal further improvement in task performance. This finding replicates studies showing an experience-dependent increase in reliance on response navigation (Hicks, 1964; Packard, 1999; Packard & McGaugh, 1996). Extending these findings, as suggested by levels of Arc mRNA expression, activation of the hippocampus was relatively strong early in training (among rats that used a place strategy) and declined considerably after task repetition (coinciding with the emergence of response navigation). In contrast, Arc expression in the medial and lateral striatum, fell and rose, respectively, on a more modest scale. Interestingly, the ratio of Arc expression among structures implicated in place navigation (hippocampus and medial striatum) to expression in regions implicated in response navigation (lateral striatum) computed on an individual level significantly discriminated training stage.

These data are generally consistent with studies showing that patterns of acetylcholine release (as measured by in vivo microdialysis) within the hippocampus and lateral striatum correlate with strategy reliance across training (Chang & Gold, 2003). However, Chang and Gold (2003) found that levels of acetylcholine release in the hippocampus were maintained throughout training (independent of strategy engagement), while we report an experience-dependent reduction in hippocampal Arc expression. These discrepant findings may result from differences in the training schedule applied across experiments. In particular, Chang and Gold (2003) restricted testing to a single training day, whereas our testing duration was an average of thirty-four days. Additionally, our findings are in line with the notion that these systems compete for behavioral control; although we did not find direct neural evidence for this hypothesis (e.g., as would be indicated by an inverse correlation between activation among these functionally disparate neural structures), we suggest that further data collection will be a valuable endeavor to clarify inter-regional interactions as they relate to decision making. In summary, it is plausible that with extensive training (1) hippocampal acetylcholine release may be dissociated from neuronal firing and/or Arc expression, or (2) activity levels among hippocampal neurons appreciably decline only after task repetition across many days of training.

Declining activity of hippocampal neurons during repetitive training is likely not due to a general reduction in contextual processing as place cells have been shown to be stable across many days of exposure to the same environment (Barnes, Suster, Shen, & McNaughton, 1997; Thompson & Best, 1990; Ziv et al., 2013). Instead, reduced activity may be related to attenuation in extra place field activity associated with the replay of previous routes (Diba & Buzsáki, 2007; Foster & Wilson, 2006; Wu & Foster, 2014) and pre-play of future paths (Diba & Buzsáki, 2007; Johnson & Redish, 2007; Pfeiffer & Foster, 2013, 2015). For instance, hippocampal representation of route planning may subside as response navigation takes control of behavior. Thus, over the course of our protracted training schedule, as assessed by average neuronal firing rate, hippocampal activation may lessen. Alternatively, it may be that Arc mRNA expression becomes uncoupled from neuronal firing across training. Although this does not appear to be the case during brief yet repeated bouts of spatial exploration across several days of testing (Guzowski et al., 2006), it is unclear if uncoupling occurs during a prolonged, incentivized, dual-solution task. From this perspective, and in consideration of the role of Arc in synaptic plasticity and memory consolidation (Guzowski et al., 2000), a reduction of hippocampal Arc expression late in training may not reflect a reduction of context-specific neural representation or information processing but rather a reduction of context-specific plasticity and learning. Clearly, elucidating the relationship between Arc and neuronal activation, plasticity and learning during a protracted appetitive task will provide valuable insights into our findings.

VTE behaviors were prominent early in testing and tended to decrease with training, an effect that mirrors the strategy transition. These results complement prior studies showing that VTE is associated with place navigation and declines with repeated training (e.g., Gardner et al., 2013; Schmidt et al., 2013) and suggest, on average, a reduction of deliberation with task experience. Notably, however, early in testing, VTE was similarly observed during place and response navigation; these data corroborate previous findings that VTE is observed during either navigation style (Gardner et al., 2013; Schmidt et al., 2013), and suggest that early in learning both strategies may engage deliberative processes. This finding appears to differ from reports showing that VTE is enhanced during novel place learning when compared to VTE during novel response learning (Schmidt et al., 2013). However, in this prior report, for a given rat, place and response runs were interwoven on a trial-by-trial basis and required some degree of reversal learning of a well-learned position or turn, respectively. Our design used a dual-solution task, on which VTE was assessed at an early time point (i.e., immediately after task acquisition: probe 1), and that did not require reversal training, so some disagreement in results might be expected.

Although the number of subjects that used a place strategy after extended training was small (probe 4: n = 4, probe 5: n = 2), VTE associated with place (compared with VTE associated with response) navigation was maintained comparatively close to levels observed early in training. These findings point to two potential relationships between VTE and strategy engagement. First, it is plausible that rats that show high levels of deliberation are relatively resistant to the strategy transition. Second, as the task becomes over-trained (and automatic response-based actions become increasingly ingrained in the animal), the degree of deliberation associated with the expression of rival (i.e., spatial) strategies is enhanced. This effect may be particularly heightened when the actions associated with place and response strategies are incompatible (during probe trials); for example, Schmidt et al. (2013) found enhanced VTE during trials for which well-learned place and response strategies were associated with different actions and may compete for behavioral control.

The current data also suggest a relationship between VTE and hippocampal Arc expression. Given the modest effect (after controlling for learning stage), however, additional work on VTE in relation to immediate early gene expression would be valuable to verify its suitability as a marker of hippocampal activation during both hippocampal- and non-hippocampal-dependent tasks and during repetitious decision making.

We stress that the data presented here on Arc expression are restricted to a sample of rats that either used a place strategy early in training (n = 4) or that used a response strategy after extensive training (n = 4). Given the strong effect (Cohen’s d ~ 0.6–3.7) in Arc expression between early and late trained rats within hippocampus and striatum, the sample used for the current work appears to be sufficient; for example, plotting the individual data points representing early and late trained animals shows relatively low variability within groups in all sampled brain regions (Fig. 6A and B). Furthermore, this research design was constructed to determine activity patterns that may underlie the experience-dependent shift from place-to-response navigation. This decision restricted our ability to tease apart the potentially distinct correlations between Arc expression and strategy recruitment and Arc expression and training duration. For example, differences in hippocampal Arc expression across conditions may be due to strategy recruitment and/or training duration. Further work will be useful to determine the extent to which these factors contributed to our findings.

Additionally, this research employed a single-label colorimetric ISH design. This procedure makes quantification of activity levels in individual neurons relatively difficult. Nonetheless, our findings qualitatively replicated previous studies that quantified the number of neurons expressing Arc after novel maze exploration; these experiments showed that the proportion of hippocampal neurons positive for Arc was highest in CA1, followed by CA3, and DG, and that the proportion of dorsal striatal neurons positive for Arc was higher in medial than in lateral sub-fields (see Fig. 6; Vazdarjanova et al., 2006). This convergence between studies suggests that our measures may closely correlate with cell counts. However, we also suggest that fluorescent Arc mRNA labeling performed in conjunction with a nuclear stain that facilitates counts of cells positive for Arc would provide a distinct assessment of mRNA expression within neural sub-fields.

Yin and Knowlton (2004) reported a functional distinction along the anterior–posterior axis of the dorsal striatum. For example, lesions to posterior, but not anterior, dorsomedial striatum increased the use of response strategies on a dual-solution plusmaze task. However, also note that Devan and White (1999) found that anterior dorsomedial striatal lesions enhanced the expression of cue-based response navigation (also see Devan et al., 1999) in water tasks. The coordinates we selected for striatal Arc analysis overlap considerably with these reports (e.g., we sampled both anterior and posterior dorsal striatum; see Section 2). However, given the possibility of a functional disconnect along the rostralcaudal axis, further, more precise, measurement of striatal Arc expression as it relates to strategy recruitment appears warranted and may help clarify the functional roles of and interactions between striatal subfields in decision making.

Altogether, several notable effects of training-stage on region-specific Arc expression (which complement behavioral findings on VTE) highlight the value of our approach to study the neural mechanics of the transition from place to response navigation. More broadly, this work provides insight into changing patterns of activation among dissociated brain regions that may underlie alterations in decision-making during task repetition.