Using internal memory representations in associative learning to study hallucination-like phenomenon

Abstract

Studies of Pavlovian conditioning have enriched our understanding of how relations among events can adaptively guide behavior through the formation and use of internal mental representations. In this review, we illustrate how internal representations flexibly integrate new updated information in reinforcer revaluation to influence relationships to impact actions and outcomes. We highlight representation-mediated learning to show the similarities in properties and functions between internally generated and directly activated representations, and how normal perception of internal representations could contribute to hallucinations. Converging evidence emerges from recent behavioral and neural activation studies using animal models of schizophrenia as well as clinical studies in patients to support increased tendencies in these populations to evoke internal representations from prior associative experience that approximate hallucination-like percepts. The heightened propensity is dependent on dopaminergic activation which is known to be sensitive to hippocampal overexcitability, a condition that has been observed in patients with psychosis. This presents a network that overlaps with cognitive neural circuits and offers a fresh approach for the development of therapeutic interventions targeting psychosis.

1. Introduction

Studies of Pavlovian conditioning have contributed to understanding fundamentals of associative learning in paradigms that have wide applicability across species and behavioral systems. Delay conditioning, in which a neutral stimulus (CS) that reliably precedes an unconditioned stimulus (US) acquires the power to elicit a learned response, exhibits many similar properties in laboratory animals and humans, and contributed to a ‘model systems’ approach for studies of neural circuitry and mechanisms of neural plasticity in a productive era of research in decades past (e.g., Christian and Thompson, 2003; Fanselow and Wassum, 2016; Greco and Liberzon, 2015; Hawkins and Byrne, 2015; McLaughlin and Powell, 1999). Using many variations of Pavlovian protocols, information processing accounts of associative learning further advanced a rich understanding of how relations among events endow biological systems with adaptive behavior (Pearce and Hall, 1980; Rescorla and Wagner, 1972; Wagner, 1981). Here, too, the use of laboratory animals alongside studies in human subjects were productive settings for the study of learning processes in neurobehavioral systems.

In the context of information processing, memory representations of previously experienced events have been found to play important roles in associative learning in animals and people. For example, studies of reinforcer revaluation (described below) using comparable procedures in rodents, non-human primates, and humans revealed corresponding neurobehavioral systems. This review and commentary will further open the way for translational neuroscience in which such research has potential for rational discovery of therapeutics across preclinical to clinical drug development.

2. Internal representations in reinforcer revaluation

The establishment of associations through Pavlovian conditioning creates internal memory representations of the CS, the US, and their relationships that contain rich, detailed information that can be dynamically used to modulate learning. The importance of internal representations and its plasticity in response to adaptive change based on subsequent experience for guiding actions has been demonstrated extensively over the last several decades using reinforcer revaluation paradigms in a range of species including humans. In a common variant of a reinforcer revaluation (more specifically devaluation) task, a subject receives CS-food association, the food is subsequently devalued through satiation or pairings with an illness-inducing agent in a separate occasion in the absence of the CS, which consequently results in a decrease in the likelihood of the CS to elicit CRs. The performance of the CRs is therefore inferred to be mediated by the CS that, through its prior associative experience with the actual US, activates not only a memory representation of the US but one with a current updated value, serving as a simple yet compelling example of how internal representations have the capacity to integrate new information to guide actions adaptively.

The construct of reinforcer representation has been investigated using both Pavlovian and instrumental devaluation procedures in rats (Balleine and Dickinson, 1992), mice (Crombag et al., 2009), monkeys (Baxter et al., 2000; Málková et al., 1997; West et al., 2011), and humans (Corlett et al., 2004; Gottfried et al., 2003; Valentin et al., 2007), which attests to the excellent translation from animal to human of the concept and procedures used to study it. Additionally, converging evidence from those cross-species studies points to a neurobehavioral system that involves the interaction between the basolateral amygdala and the orbitofrontal cortex in the formation and the use of reinforcer representations (see reviews by Holland and Gallagher, 2004; Pickens and Holland; 2004). The use of reinforcer devaluation has also expanded more recently to the assessment of neurobehavioral dysfunction associated with psychiatric disorders on the integration of reinforcer representations to action-outcome associations (e.g., Barch and Dowd, 2010; Griffiths et al., 2014; Morris et al., 2015; 2018). For the rest of the article, we will further illustrate the role internal memory representations in associative learning through representation-mediated learning, and consider how irregularities in the activation of internal representations may drive hallucination-like phenomena in pathological and non-pathological populations.

3. Representation-mediated learning

Though not as extensively studied as reinforcer devaluation, representation-mediated learning has also traveled an investigatory route through studies in laboratory animals and recently in humans. These investigations found that associatively activated internal representations possess many of the features of directly activated representations of those events, and can also serve the same functions as directly activated representations in mediating and modulating learning about other cues. In a series of seminal studies starting in the early 1980s, Peter Holland established that internal representations activated from prior experiences could enter into current associations to form new learning in animals (see review by Holland & Wheeler, 2009). This was demonstrated using activation of internal representations of food to form a conditioned food aversion. In one such study (Holland, 1981), rats first received repeated pairings of a tone CS with a novel flavored food to endow the tone with the ability to activate an internal representation of the food. The presentation of the tone alone was then paired with an illness-inducing agent. Finally, a food consumption test was given in the absence of the tone to assess mediated learning of a food aversion. Rats subjected to these training procedures consumed less of the food than rats in the control groups, suggesting that the tone, through its prior associative experience with the food, activated an internal representation of the food that entered into an association with the illness-inducing agent to induce a conditioned aversion to the food. Unlike reinforcer devaluation, representation-mediated learning is sensitive to the amount of training such that learning is found only under limited CS-US pairings (e.g., tone-food), and extended training reduces the ability of the US to support new learning (e.g., an aversion), possibly due to changes in the sensory features and associability of the US representations activated by the CS (Holland, 1998; Holland, 2005).

The cognitive representation of the food from mediated learning under optimal training procedures is activated with sufficient strength and specificity at a perceptual level to serve as a potent CS to support new learning to that food. Indeed, evoked representations share sensory qualities and activate corresponding perceptual processing as directly activated representations (e.g., Delamater et al., 1986; Holland, 1990; Kerfoot et al., 2007). For instance, in Kerfoot et al. (2007), rats received repeated pairings of tone and intraoral delivery of sucrose solution. The hedonic value of sucrose was then either maintained or reduced through a pairing with an illness-inducing agent in the absence of the tone. Responding to the tone was tested days later with intraoral water infusion to assess orofacial activity for liking and disgust reactions that reflects taste palatability. Compared to those in the control group that received tone and sucrose unpaired, the rats that had the experience of tone and sucrose paired together with the palatability of the sucrose maintained showed more appetitive taste-reactivity responses to water, indicating that the rats responded in a manner consistent with drinking sucrose. Hence, the tone has seemingly evoked, through its prior associative experience, an internal representation with sufficient sensory authenticity of a real sucrose solution in the test session that the rats behaved as though they were drinking sucrose while in fact they were drinking water. Most remarkably, the rats that received tone-sucrose pairing with subsequent sucrose devaluation displayed fewer appetitive and more aversive taste-reactivity responses akin to the tasting of unpalatable flavors. In this case, the tone was able to evoke an internal representation of the sucrose solution that had been devalued at a different occasion during the absence of the tone, demonstrating the potency and dynamic nature of evoked memory representations that incorporate updated content through mediated learning in guiding behavior. In addition to mediating acquisition and extinction learning (Holland & Forbes, 1982), associatively activated internal representations can also modulate learning by facilitating or interfering with association formation; potentiation (Holland, 1983; 2006) and overshadowing (Holland, 1983) for example have both been shown with associatively activated internal representations indicating that they can functionally mimic directly activated representations.

4. Hallucination-like phenomenon

These intriguing findings with mediated learning have potential relevance for the phenomenon of hallucination with implications for normal and dysfunctional biological processing relevant for psychiatry. Commonly observed in psychiatric illnesses including schizophrenia, hallucinations occur when a stimulus is perceived in the absence of external input of the stimulus, not unlike the evocation of internal representations under normal circumstances. Under that view, hallucinations in fact can be observed in nonclinical healthy populations presumably arising from functioning in non-pathological brain circuits (Larøi et al., 2012). Increasingly, the role of internal representations elicited by prior associative experiences in accounting for hallucinatory perception has received more attention from a cognitive perspective in both animal and human research (e.g., Adams et al., 2013; Dwyer, 2018; Feeney et al., 2017; McDannald & Schoenbaum, 2009).

Using well-established animal models of schizophrenia, we and others have begun to investigate whether animals that are known to exhibit symptoms consistent with psychosis-like behavior display a greater tendency to use a representation of a prior experience to form mediated associations in learning (Busquets-Garcia et al., 2017; Kim & Koh, 2016; McDannald et al., 2011). In our studies (Koh et al., 2018a), we used mice that were exposed subchronically to ketamine during adolescence and tested them ketamine-free in young adulthood; such exposures are known to be highly effective in reproducing positive symptoms of schizophrenia (Frohlich & Van Horn, 2014; Krystal et al., 1994). Using a representation taste aversion paradigm, the mice were exposed to an odor-taste compound to endow the odor with the ability to activate a representation of the associated taste. The odor alone was subsequently paired with gastrointestinal illness in the absence of the taste. A subsequent test with the taste was given in the absence of the odor to assess the presence of a conditioned taste aversion which would signify that an internal representation of the taste activated by the odor during conditioning had entered into an association with illness. Compared to control mice, the mice with a history of ketamine exposure during adolescence showed a more pronounced taste aversion, suggesting (1) a heightened tendency to activate an internal representation from a prior associative experience (i.e., odor activates taste representation during conditioning), and/or (2) a stronger propensity to associate an internal representation with another stimulus or event (i.e., taste representation associates with illness). Recent work using mice with dominant negative expression of Disrupted-In-Schizophrenia-1 (DISC-1), a mutation that increased the risk for neuropsychiatric conditions, shed some light on these possibilities using a simple Pavlovian approach (Fry et al., 2019). The mice were first given conditioning of an auditory stimulus such as a tone paired with the delivery of sucrose solution for consumption. After extensive training, the mice were tested with water in the presence of the tone. Compared to control mice, the DISC-1 transgenic mice showed enhanced licking of water with increased palatability profile suggestive of consumption of a sucrose-like solution. Hence, the prior associative experience between tone and sucrose appeared to have endowed the tone with the ability to trigger a sucrose representation while drinking water, which provided strong support for a heightened tendency to activate internal representations of associative experiences in DISC-1 transgenic mice.

5. Neural activation of internal memory representation

Additional evidence on the nature of representations comes from studies using imaging of neural activity to capture activation in neural systems. We and others have assessed the predisposition of animals used to model schizophrenia for cues previously paired with a taste to induce a taste percept at the neural level in the absence of the taste itself, using the expression of an immediate early gene, c-fos, as a marker to detect neural activation in taste-responsive brain regions such as the insular cortex. Prior research importantly has shown the cued activation of the taste area in the insular cortex to resemble natural neural representation of taste stimuli (Saddoris et al., 2009), and stimulation of the taste cortex in the absence of an actual taste can initiate taste percepts to guide behavior (Peng et al., 2015). In our experiment using a ketamine mouse model of schizophrenia, mice received odor-taste compound exposures, and then odor exposure alone (with water to drink) aimed at activating a taste representation (Wu et al., 2020). We found that the previously taste-associated odor induced a stronger neural activation in the insular taste cortex in mice exposed to ketamine during adolescence than control mice. Using DISC-1 transgenic mice, Fry et al., (2019) similarly found those mice to display a greater c-fos expression in the insular cortex in response to an auditory cue previously paired with a taste. Taken together with the behavioral studies, the neural activation data further strengthened the assertion that animals used to model symptoms of schizophrenia, which do not exhibit any general alteration in simple associative conditioning (Johnson et al., 2013; Koh et al., 2018, Fig. 2A), have increased tendency to induce internal representations akin to hallucination-like percepts, but these data do not discount the additional possibility of an increased propensity to associate an internal representation with other stimuli or events.

6. Conditioned hallucinations in humans

The role of prior associative experience in mediating hallucinations has also been demonstrated in the laboratory setting with humans, both with and without diagnosed psychiatric illnesses. When a tone and a visual stimulus were repeatedly paired together, people reported hearing the tone subsequently in response to the visual stimulus alone (i.e., in the absence of tone); patients diagnosed with hallucinations reported hearing the tone more frequently and with higher confidence in their conditioned hallucinations than control subjects without pre-existing hallucinations (Powers et al., 2017b). The probability and confidence of the conditioned hallucinations were also strongly correlated with the severity of diagnosed hallucinations of the subjects. Furthermore, the conditioned hallucinations in those subjects involved evoked sensory representations as evidenced by increased neural activation in tone-responsive brain regions during the conditioned hallucinations.

Through such sensory conditioning, the susceptibility to hallucinate has been studied in different populations including normal individuals in the absence of any detectable illnesses, by examining the tendency to evoke internal stimulus representations based on prior experience established by associative learning (Ellson, 1941; Kot and Serper, 2002; Powers et al., 2017b). Prior knowledge is therefore critical in this top-down account of the source of hallucinations as it is used to compare to incoming sensation; if the prior expectation is strong and precise, it will dominate perception with internal representations based on the prior knowledge (Corlett et al., 2019; Teufel et al., 2015). It is worth noting again that by this account, hallucinations are maladaptive extremes along a continuum of normal functioning as opposed to mediation by some distinct pathological processes or circuitry that is extraneous to existing neurobehavioral systems (Powers et al., 2017a). Indeed, auditory hallucinations appear to elicit similar brain activation in a network of cortical and subcortical regions in both psychotic and normal individuals (e.g., Diederen et al., 2012).

7. Circuits and networks in neurocognitive function

Not unlike other manifestations of psychosis, the increased tendency to evoke internal representations in animal models of schizophrenia appears to be driven, at least in part, by dopamine hyperfunction. Adult mice exposed to ketamine sub-chronically during adolescence display hyper-responsiveness to the dopamine-releasing agent, amphetamine. The same mice when treated with the antipsychotic risperidone showed both reduced hyper-responsiveness to amphetamine and an attenuated tendency for representation-mediated learning, suggesting that conditioned hallucinations as conceptualized here may be due, at least in part, to dopamine hyperfunction (Koh et al., 2018a). Further substantiating those findings, treatment with haloperidol, which exhibits high affinity dopamine D₂ receptor antagonism, also effectively reduced the tendency to evoke internal representations in DISC-1 transgenic mice (Fry et al., 2019). A role for dopaminergic function demonstrated in those studies is consistent with the view that internal representations elicited in the context of mediated learning approximates hallucinations in patients insofar that hallucinations have also shown some sensitivity to antipsychotics that target dopamine function, and that hallucinations might arise from a dysregulation of the dopamine system causing misattribution of salience to internal representations of percepts and memories (Kapur, 2003).

Dopamine function, at the same time, is under the regulation of the hippocampus. Notably, increased metabolic activity in the hippocampus itself has been reported in individuals with schizophrenia (e.g., Allen et al., 2018; McHugo et al., 2019; Medoff et al., 2001; Sanderson et al., 2012; Schobel et al., 2013; Tregellas et al., 2014; Ziehut et al., 2010). The increased level of metabolic activity in the hippocampus of these individuals was predictive of the disease progression from a prodromal condition to the onset of psychosis, and further positively correlated with the severity of clinical symptoms of psychosis (Schobel et al., 2009). Increased hippocampal activity has also been recapitulated in well-established neurodevelopmental animal models of schizophrenia, including those using chronic ketamine exposure during adolescence (Schobel et al., 2013) and methylazoxymethanol (MAM) during gestation (Gill et al., 2011; Lodge and Grace, 2007). As noted in this recent research, pathologically increased hippocampal activation can contribute to dopamine hyperfunction that triggers or exacerbates psychosis; targeting hippocampal overactivity could thus present a route to treat psychosis by reducing dopamine hyperfunction with additional normalization of other forebrain circuitries. With respect to dopaminergic regulation, intracranial administration of a GABA_A α5 receptor agonist into the hippocampus has been shown to reduce excess dopamine activity in the ventral tegmental area (VTA) and psychosis-like symptoms in the MAM animal model of schizophrenia (Gill et al., 2011). Also, overexpression of GABA_A α5 receptors with viral-mediated gene transfer in the hippocampus of MAM-exposed rats both modulated tonic currents and firing rates in the hippocampus, normalized aberrant dopamine neuron population activity in the VTA, and attenuated cognitive symptoms associated with schizophrenia (Donegan et al., 2019). Indeed, our treatment studies in the ketamine model using the antiepileptic medication levetiracetam aimed at reducing hippocampal overactivity similarly reduced hippocampal-dependent cognitive impairment as well as dopamine-dependent amphetamine-induced hyperlocomotor activity (Koh et al., 2018b).

These findings suggest that normalizing hippocampal overactivity might be beneficial for reducing the tendency to evoke hallucination-like percepts as instantiated in representation-mediated learning that is sensitive to dopamine signaling. Interestingly, the hippocampus itself also plays an important role in mediated learning with lesion or inactivation of the hippocampus producing impairment (Iordanova et al., 2011; Wheeler et al., 2013), and deletion of cannabinoid type 1 receptors in hippocampal GABAergic neurons also disrupts mediated learning (Busquets-Garcia et al., 2018). Recent work additionally shows that the interaction between the hippocampus and the orbitofrontal cortex is important for inferring future outcomes using internal representations under sensory preconditioning, a learning paradigm that shares some procedural similarity with representation-mediated learning (Wang et al., 2020). In addition to its role in dopaminergic regulation, consideration of the hippocampus as a component of a network contributing to neurocognitive dysfunction in schizophrenia may afford the opportunity to gain efficacy for cognitive deficits that are not remediated by antipsychotics that target dopaminergic function (Koh et al., 2018a; Scott and Tamminga, 2018; Segev et al., 2018; Smucny et al., 2015). Further extensions in forebrain cortical circuitry are under study within the affected network in psychiatric illness (e.g., Donegan et al., 2019; Nelson et al., 2020). This approach is tied to the Research Domain Criteria (RDoC) initiative with greater emphasis on cognitive processes (Ford et al., 2014; Morris and Cuthbert, 2012).

8. Summary

In this brief review, we illustrated how internal representations through their flexibility are normally used to adaptively guide behavior in reinforcer devaluation. We then highlighted representation-mediated learning to further show the similarities in properties and functions between internally generated and directly activated representations, and how normal perception of internal representations could be a contributing basis for hallucinations. We discussed examples of how this has been investigated in animals and humans neurobehaviorally, and how conceptualization of hallucinations in these terms would allow for a rational development of drug therapies that may go beyond the current limitations of antipsychotics. This current research continues to leverage systems neurobiology in laboratory animals and clinical studies both grounded in a common formal theoretical framework, illustrating that core functions of circuits and more extended networks are retained in ways amenable to study across species.

Highlights.

• Mental representations in studies of learning in rodents and human research

• Internally generated mental representations could be perceived as hallucinations

• Representation-mediated learning occurs in translational studies across species

• Relevance of mediated learning to model hallucinations in preclinical research

• Normalizing hippocampal overactivity to target cognitive symptoms in schizophrenia