Novelty seeking behaviour in Parkinson’s disease

Abstract

Novelty seeking can be a positive trait leading to creativity and innovation, but it is also related to increased risk of damaging addictive behaviour. We have assessed novelty seeking with a three armed bandit task, in which novel stimuli were occasionally introduced, replacing choice options from which the participants had been choosing. This allowed us to assess whether or not they would be prone to selecting novel stimuli. We tested 25 non impulsive patients with Parkinson s disease (PD) and 27 PD patients with impulsive compulsive behaviours (ICB). Both patient groups were examined “on” and “off” dopaminergic medication in a counterbalanced order and their behaviour was compared with 24 healthy controls. We found that PD patients with ICBs were significantly more prone to choose novel options than either non impulsive PD patients or controls, regardless of medication status. Our findings suggest that attraction to novelty is a personality trait in all PD patients with ICBs which is independent of medication status.

Introduction

Humans and animals are inherently attracted to new stimuli as these can be potentially rewarding (Ennaceur and Delacour 1988; Daffner, Mesulam et al. 1998; Hughes 2007). High novelty seeking is part of adolescence and may help in normal development and the acquisition of independence (Kelley, Schochet et al. 2004): adults with novelty seeking personality traits on the other hand often have increased impulsivity, addiction, inability to delay gratification, recklessness and aggressive behaviour (Barratt 1985; Barratt 1994; Belin, Mar et al. 2008). A subgroup of patients with Parkinson s disease (PD) develop impulsive compulsive behaviours (PD+ICB) in relation to their dopamine replacement therapies. These include pathological gambling, compulsive sexual behaviour and shopping and the inappropriate, excessive overuse of dopaminergic medication (dopamine dysregulation syndrome, DDS) (Evans, Lawrence et al. 2005; Evans, Pavese et al. 2006; Voon, Thomsen et al. 2007). While self-report questionnaires have suggested that the subgroup of PD+ICB patients with DDS (Evans, Lawrence et al. 2005) and those with pathological gambling (Voon, Thomsen et al. 2007) have high levels of novelty seeking, this has not been formally studied using metric tests.

The trade-off between choosing options of known value and exploring novel options is known as exploration vs. exploitation (Daw, O’Doherty et al. 2006). Exploring novel choices and learning the value of stimuli based on reward feedback have been linked to the ventral striatum, the substantia nigra and the ventral tegmental area of the midbrain (Wittmann, Daw et al. 2008; Guitart-Masip, Bunzeck et al. 2010) as well as the hippocampus (Guitart-Masip, Bunzeck et al. 2010; Voon, Pessiglione et al. 2010). These areas either contain dopamine neurons or receive strong dopaminergic innervation. Additional studies have examined the dopamine link to learning and exploration. For example, behavioural studies in PD have shown that dopamine levels play an important role in reward learning (Seo, Beigi et al.; Cools, Clark et al. 2002; Frank, Seeberger et al. 2004; Djamshidian, Jha et al. 2010; Voon, Pessiglione et al. 2010). Complimenting this work, functional magnetic resonance imaging (fMRI) studies in healthy controls and positron emission tomography (PET) studies in PD+ICB patients have localized reward responsivity to the ventral striatum (O’Doherty, Critchley et al. 2003; Steeves, Miyasaki et al. 2009; Evans, Fleming et al. 2010; O’Sullivan, Wu et al. 2011). An important role for striatal dopamine D2 receptors in the exploration vs. exploitation trade-off has been suggested by genetic studies (Frank, Doll et al. 2009). It seems probable therefore that there is an overlap between the networks which mediate reward learning and novelty seeking, and both processes can be conceptualized as assigning value to choice options.

One of the circuits that has been proposed to mediate novelty effects includes the hippocampal projection to the ventral striatum. Specifically, the hippocampus forms a functional loop with the ventral striatum and the mid-brain dopamine neurons. The hippocampus is activated by novel information (all information that is not stored in long term memory) and regulates, via the ventral striatum, dopamine neuron firing rates (Lisman and Grace 2005). Neuropathological studies have shown that the parahippocampal gyrus is affected in later stages of PD (Braak, Ghebremedhin et al. 2004). Thus, abnormal and increased activity in the ventral striatum might be triggered by earlier neuropathological changes in the hippocampus in PD+ICB patients.

The aim of the present study was to compare novelty seeking between impulsive and non-impulsive PD patients, and also to examine the role of dopaminergic medication on novelty seeking. We hypothesized that PD+ICB as a group would be more novelty seeking than PD-ICB patients on a task which allows for exploration of novel options. We tested PD+ICB and PD patients without ICB (PD-ICB) on and off their dopaminergic medication on a modified “three armed bandit” choice task (Wittmann, Daw et al. 2008), where all participants played for real money. We compared the choices of PD-ICB and PD+ICB patients on and off their medication with a group of healthy controls who were matched for age and education to the patients group.

PATIENTS AND METHODS

PD patients were recruited from a database of attendees at the National Hospital for Neurology and Neurosurgery Queen Square, London. All patients fulfilled the Queen Square Brain Bank criteria for the diagnosis of PD (Gibb and Lees 1988) and were taking L-dopa medication. Patients with structural lesions on their brain scans were excluded from this study. Some of the patients had also had raclopride PET scanning and results of this study are presented elsewhere. (O Sullivan et al, Brain 2011). We included patients who were moderately impaired by PD. All patients showed a significant improvement (>35% improvement) after L-dopa intake which was assessed by the UPDRS (part 3) motor score. There was no significant difference in UPDRS motor scores between the 2 patient groups. L-dopa equivalent units (LEU) of patients regular daily dopamine replacement therapies were calculated as described elsewhere (Evans, Katzenschlager et al. 2004). Controls were usually recruited from amongst the patient s spouses or partners. Participants who provided written informed consent to protocols approved by the UCLH Trust local ethics committee were included. Patients who scored under 27/30 points on the Mini Mental State Examination (MMSE) (Folstein, Folstein et al. 1975) were excluded from this study.

27 PD+ICB and 25 PD-ICB patients were recruited and results were compared with 24 healthy controls. PD+ICB patients were diagnosed using proposed criteria (Lawrence, Evans et al. 2003; Evans, Katzenschlager et al. 2004; Voon, Potenza et al. 2007). Most PD+ICB patients had more than 1 ICB. The ICBs included compulsive sexual behaviour (12 patients), pathological gambling (11 patients), compulsive buying (8 patients), punding (4 patients) and kleptomania (1 patient).

Novelty Task

We performed a three-armed bandit task, modified from the “four armed bandit choice task” used previously (Wittmann, Daw et al. 2008). The task was administered on a laptop computer. Participants performed 60 trials of the task. In each trial three black and white picture post-cards were presented on the screen (Fig. 1). After presentation of the pictures, the participant was required to select one of the three pictures, and after the option was selected, they were told whether they had “won” or “lost”. We also provided auditory feedback (5 Khz for winning and 2.5 Khz for losing) to reinforce feedback learning. Following an inter-trial interval, during which the screen was blank, the participants were again presented with the 3 choice options and they could make another decision. The location of each picture was randomized from trial to trial to prevent habituation. The participants were told to pick the most often rewarded picture as many times as possible to maximize their winnings.

Fig. 1. Sequence of events in 3-armed bandit task.

After familiarization, participants were asked to choose one of the three pictures. Images were presented at randomized positions that changed on each trial. Unfamiliar and familiar pictures appeared during the test. Participants were told that each picture had some probability of winning 20p and participants should pick the rewarded picture as many times as possible. Visual and acoustic feedback was given immediately after each trial.

During the task, as the participants were making their choices and learning the reward value of the pictures, novel stimuli were introduced. This was done by replacing one of the images from which participants had been choosing with a new image, which was then a novel choice option. A novel choice option was introduced on 20% of trials, or on average every 5 trials. These novel choices were of two types -- unfamiliar and familiar. Unfamiliar stimuli were images that the patients had never seen before, whereas familiar stimuli were images that the patients had seen in pre-task training. It is important to note that both unfamiliar and familiar images refer to pictures that were introduced into the ongoing 3-armed bandit task, replacing one of the pictures that the participants had been selecting from. Familiarization was done by sending 18 black and white pictures to participant s homes prior to the experiment, and asking them to guess which country each picture was taken from. We called all participants prior to testing to ensure that participants were familiar with the set of images. On the day of testing and prior to each session we familiarized participants again. We used different sets of pictures for each session. Therefore, we re-familiarized participants with 9 of the 18 pictures prior to the first session, and the other 9 pictures prior to the second session. We also counterbalanced the pictures from the set with which the subjects were familiarized across medication conditions, so approximately half the subjects were familiarized with one half the pictures for their medicated session, and the other half of the subjects were familiarized with the other half of the pictures for their medicated session. None of the subjects knew the purpose of familiarization. There were no differences in reward values between familiar and unfamiliar pictures in the choice task. At the beginning of each of the two choice experiments, in the first trial, all participants were asked: “which picture is unfamiliar?” They all recognized the unfamiliar image among the three in the first trial.

PD patients were tested prior and after their usual anti-Parkinson medication in a counterbalanced sequence to account for order effects. All patients who were tested in their “off medication state” did not take their usual anti-Parkinson medication, including both L-dopa and any dopamine agonists, for at least 12 hours. Results were compared with 24 controls who were matched for age to the PD+ICB group. Patients who were tested first prior to their usual anti-Parkinson medication (“off medication”) performed the task between 8.00am and 9.00am. They were then retested in their “on medication” state one hour after taking their first dopaminergic medication of the day. Those patients who were tested “on medication” first performed this task usually in mid-morning when their motor symptoms were well controlled. They were re-visited on the following day prior to their medication for the second test. Controls were tested in the same way but did not take any anti-Parkinson medication. At the end of the study all participants got a modest amount of money depending on their final score (usually £5–£10).

Statistical analyses were performed using SPSS, version 18. For the demographic variables, age, gender, years of education, age of disease onset UPDRS scores, LEU dose were used as dependent variables and group (PD-ICB, PD+ICB and control) was modelled as a between subject factor. We used ANOVA, t-test or χ² test where appropriate. For the behavioural variables we first fit models to the choice data of individual participants to parameterize the value they assigned to novel stimuli, which in effect characterized the probability that they would select a novel stimulus. A higher value indicates a higher probability of selecting a novel stimulus. We then fit an ANOVA to the parameters derived from the model comparing the effect of novel stimuli in PD and ICB groups off and on medication.

Reinforcement learning model

We have fitted a reinforcement learning model to the choice behaviour of the subjects to assess whether or not they were disposed to selecting novel stimuli. This model computes the value of a novel stimulus, to the participant, before it has had any reward feedback. In general, the model updates the value, v, of the chosen option, i, based on reward feedback, r in trial t as:

$$v_i(\text{t})=v_i(\text{t}-1)+\alpha (\text{r}(\text{t})-v_i(\text{t}-1)).$$

Thus, the new value of an option is given by its old value, v_i(t−1) plus a change based on the reward prediction error (r(t)−v_i(t−1)), multiplied by the learning rate parameter, α. When a novel stimulus is introduced in trial t, there is no reward history. We can infer the value of that image to the participant, by examining how often the participant picks that image, relative to how often they pick the other options with known reward histories. Thus, we can fit, v_i(t), where t = the first trial for a novel option, i, as a free parameter. Different parameters, v_i(t), for example v_familiar(t) and v_unfamiliar(t) were fit, to allow us to examine the effects of familiarization on the initial values. Effectively the participants will have some ongoing value estimates of the options, i, and the relative propensity of the participants to pick the novel option allows us to estimate the value of that option relative to the other options. If participants tend to pick the novel option, it implies that new options are relatively more valuable than the other options, with which the participant has some experience. The model is fit by maximizing the likelihood of the choice behaviour of the participants, given the model parameters. Specifically, we can calculate the choice probability d_i(t) using:

$${\text{d}}_{\text{i}}(\text{t})=\frac{exp(v_i(\text{t}))}{{\sum }_{k=1}^{3}exp({\text{v}}_{\text{k}}(\text{t}))}$$

And then calculate the log-likelihood as

$$\text{ll}=\sum _{\text{t}=1}^{\text{T}}\mathit{log}\sum _{\text{k}=1}^{\text{a}}{\text{c}}_{\text{k}}(\text{t}){\text{d}}_{\text{k}}(\text{t}),$$

Where c_k(t)=1 when the subject chooses option k in trial t and c_k(t)=0 for all unchosen options. Thus, c_k(t) is an indicator variable which selects the choice probability d_k that corresponds to the choice the subject made in trial t, such that the log-likelihood is summed over the chosen options across trials. In other words, the model maximizes the choice probability (d_k(t)) of the actual choices the participants made. T is the total number of trials in the session for each participant. Parameters were maximized using standard techniques (Djamshidian, Jha et al. 2010).

Results

Demographic and clinical features

We found a significant effect of age between the 3 groups (F(2,73)=7.58, p=0.001). Post hoc analysis revealed that the PD-ICB group was older than the PD+ICB (p=0.001) and a trend to be older than the control group (p=0.055). There was no difference between controls and PD+ICB patients (p=0.54). We found that PD+ICB patients had a significantly younger age of disease onset (t(49)=3.39, p=0.001). There was no difference in the LEU dose, disease duration, UPDRS motor score (part 3) and years of education across the groups (Table 1). Of note, 15 out of 27 PD+ICB patients and 5 out of 25 PD-ICB patients tested report a sweet tooth, and these proportions were significantly different (χ² = 6.9, p = 0.009). All patients had an excellent response to L-dopa and improved by more than 35% on the UPDRS (part III) motor score in their on state compared to their off state. There was no significant difference in UPDRS motor scores between the 2 patient groups. There was also no difference in MMSE scores between the patient groups, (t(50)=0.56, p=0.57; mean MMSE scores in the PD+ICB group=28.7 vs PD-ICB=28.9). Further all patients were tested on a Stroop task which showed no difference between impulsive and non impulsive PD patients (Djamshidian, O’Sullivan et al. 2011). A large proportion of patients were also tested on digit forward and backward span (Djamshidian, Jha et al. 2010), which showed that the PD+ICB group had lower working memory performance. Thus, the PD+ICB and PD-ICB groups were matched for disease duration and other variables, but differed in age.

Table 1.

UPDRS = Unified Parkinson’s Disease Rating Scale; LEU = L-dopa equivalent units; DA = dopamine agonists. All values are mean ± SD. Significant differences are labeled with “*”. Controls, PD patients with (PD+ICB) and without (PD-ICB) impulsive compulsive behaviours.

	Controls	PD+ICB	PD-ICB	t value, χ 2 and F-value	p-value
Participants (no.)	24	27	25
Age (yrs)	57.8 ± 10.7	54.2 ± 9.2	64.2 ± 8.0	F = 7.6	=0.001*
Gender (male)	14	22	21	χ 2 = 5.2	=0.073
At disease onset	-	44.1 ± 8.7	52.8 ± 9.5	t = 3.2	=0.001*
Disease duration (yrs)	-	10.2 ± 5.5	11.4 ± 7.2	t = 0.68	=0.5
Education (yrs)	13.2 ± 2.9	13.4 ± 3.0	14.7 ± 3.5	F = 1.5	=0.23
LEU dose(mg/day)	-	832 ± 425	805 ± 400	t = 0.2	=0.8
DA (patients)	-	14	16	χ 2 = 3.7	=0.4
UPDRS on	-	16.6 ± 9.4	14.4 ± 5.8	t = 0.8	=0.4
UPDRS off	-	27.3 ± 9.0	26.9 ± 6.7	t = 0.19	=0.85
Improvement in UPRDS (%)	-	39.2	46.4

Novelty task

We first carried out an ANOVA on the PD and PD+ICB groups, with main effects of group, medication, and image familiarization, to assess whether or not there were differences in choice patterns for novel stimuli (Fig. 2a). An age covariate was included in all ANOVAs. There was a main effect of group (F(1, 45) = 7.03, p = 0.011), such that ICBs selected novel stimuli whether they were unfamiliar or familiar more often than PDs (Fig. 2A). The main effect of medication showed a non-significant trend (F(1, 45) = 3.2, p = 0.076) with patients on medication being less likely to select novel images. There was however no effect of unfamiliar relative to familiar new images (F(1, 45) = 1.63, p = 0.205), and there were no significant interactions. We also separately examined medication effects in PDs and ICBs by running separate ANOVAs within each group, with main effects of medication and image familiarity. There was no significant effect of medication in the PD group (F(1, 24) = 0.84, p = 0.364) or in the ICB group (F(1, 21) = 2.59, p = 0.112). Next, we compared each of the 4 clinical groups (PD and ICB off and on medication) pair-wise with the control group (Bonferroni corrected). The ICB group off (F(1, 38) = 10.75, p = 0.002) and on (F(1, 38) = 4.86, p = 0.034) medication selected novel stimuli more often than controls. The PD group did not differ significantly from controls off or on medication.

Fig. 2.

Behavioural results. A. Weight given to unfamiliar and familiar novel stimuli by each group of subjects. Off indicates off medication, on indicates on medication. Unfamiliar refers to stimuli with which the subjects had not seen prior to the choice task and familiar refers to stimuli with which subjects had seen. Inset shows residual of ANOVA model. B. Values for learning rate parameter for each group.

In the final analysis we assessed the learning rate parameter, which measures the extent to which subjects integrate feedback to update their decisions (Fig. 2B). There were, however, no significant differences after controlling for the effects of age (F(1, 45) = 1.91, p = 0.174).

Discussion

We found that the PD+ICB patients were more attracted to newly introduced pictures, than either the PD-ICB patients or normal controls, regardless of their medication status and across a group of ICBs with various diagnoses. This was true regardless of whether the novel picture came from the set with which the patient had been familiarized (familiar) or from the set with which the patient had never seen before (unfamiliar). This result is consistent with previous studies which have shown high novelty seeking personality traits in PD patients with DDS (Evans, Lawrence et al. 2005) and patients with pathological gambling (Voon, Thomsen et al. 2007) using self-rating questionnaires. Although self-rating questionnaires are helpful in diagnosis they must be interpreted with care, especially in patient groups where insight may be low, such as PD patients with ICBs (Ferrara and Stacy 2008; Lim, Evans et al. 2008), and patients with substance abuse (Goldstein, Craig et al. 2009). PD+ICB patients are also known to have significantly higher schizotypy scores than PD patients without ICBs (Housden, O’Sullivan et al.) another factor known to reduce the validity of questionnaires (Lenzenweger 2010). Thus, our results bring novelty seeking into an explicit metric framework using a task with a known neural substrate (Wittmann, Daw et al. 2008).

fMRI studies using a four option choice task have shown that activation of the ventral striatum significantly correlated with reward predication errors and exploring novel, unfamiliar stimuli (Wittmann, Daw et al. 2008). An increase in ventral striatal dopamine levels, measured with PET, has recently been demonstrated in the PD+ICB group in response to medication, gambling and reward-related cues (Gibb and Lees 1988; Evans, Lawrence et al. 2005; O’Sullivan, Wu et al. 2011). Related work has shown reduced levels of the dopamine transporter (DAT) in the ventral striatum of PD patients who had pathological gambling relative to a control group of PD patients without pathological gambling (Cilia, Ko et al. 2010). Reduced membrane DAT levels could lead to the increased synaptic dopamine levels. Thus, converging evidence suggests increased ventral-striatal dopamine levels in the PD+ICB group. In some cases, this increased dopamine signalling appears to contribute to increased sensitivity to behaviours mediated by the ventral striatum, including temporal discounting (Voon, Reynolds et al. 2009), risk taking, and feedback learning (Djamshidian, Jha et al. 2010; Voon, Pessiglione et al. 2010). Also consistent with this, many of the PD+ICB patients tested report a sweet tooth with a penchant for chocolate, and a recent study has shown an association between sweet liking, novelty seeking and addictive behaviour (Lange, Kampov-Polevoy et al. 2010).

In spite of the data which suggests that increased dopamine levels contribute to impulsivity in PD, we found no effect of acute changes in dopamine levels on novelty seeking in the current study. This suggests that the mechanism that mediates novelty, as we have operationalized it, may be unrelated to acute changes in dopamine levels brought about by withholding medication for at least 12 hours. Thus, long-term changes brought about by chronic increases in dopamine levels, rather than an acute change of dopamine level, might trigger novelty seeking behaviour in PD. This is one factor which may account for differences between our study and a previous study which found increased novelty seeking in PD-ICB patients after dopamine agonist therapy (Bodi, Keri et al. 2009). Thus, the effects seen in the Bodi et al. study may be mediated by chronic changes in levels of dopamine stimulation, as opposed to the acute changes we used. Specifically, the Bodi et al. study compared a group of never medicated patients, to a group of patients medicated for periods of several months with dopamine agonists. There are other important differences between the Bodi et al., study and ours. First, our subjects were on a combination of dopamine agonists and L-Dopa, as opposed to just dopamine agonists. When we tested our subjects off medication, we withheld both the dopamine agonists and the L-dopa, but only acutely. Second, the study of Bodi et al., found increased novelty seeking using self-report questionnaires, as opposed to a metric behavioral task. It is not clear that self-report questionnaires and metric behavioral tasks measure the same construct. Thus, the inconsistencies between the study of Bodi et al. and ours are likely due to methodological differences.

We also found that, although we pre-trained our subjects on a set of pictures, so that novel stimuli could be either unfamiliar or familiar, these manipulations reached only trend levels and were not significant, unlike previous studies (Wittmann, Daw et al. 2008). It is possible that the pre-training was not sufficient in this group of elderly participants, as we also did not see an effect in matched controls, although, participants were all able to identify the novel picture in the first trial of the task. All participants scored higher than 27 on the MMSE examination and were non-demented. In our view this makes it unlikely that they were not able to remember 9 pictures prior to each session. Additional exposure to the pictures may have been useful, however, in finding an effect of unfamiliar vs. familiar images. It is also possible that familiarity biases are smaller in elderly adults, and that more extensive training might not overcome this. In this study we have seen that impulsive PD patients are more novelty prone. However, animal studies have used outbred rats to separate novelty seeking, operationalized as an increased locomotor response in a novel environment, and impulsivity, operationalized as premature responses in a serial reaction time task (Belin, Mar et al. 2008). This study found that rats prone to novelty seeking tended to acquire cocaine self-administration more readily than their impulsive counter-parts, whereas impulsive rats tended to convert to compulsive drug use more readily than their novelty seeking counterparts. This suggests that the combination of these traits would lead individuals to be particularly prone to developing addictive behaviour. Novelty seeking could lead, for instance, to playing slot machines, which is not only the most commonly played gamble in PD but is considered to be the “crack cocaine” of gambling with the highest addictive potential (Dowling, Smith et al. 2005). Novelty seeking could lead to initiation of a potentially addictive behaviour, which then turns into addiction as a consequence of an impulsive personality trait.

In summary we have found increased novelty seeking in all PD+ICB patients in a 3 option choice task. These findings add to other studies which have shown differences in reward learning, working memory and, in PD+ICB patients with pathological gambling, differences in risk behaviour (Djamshidian, Jha et al. 2010). Interestingly, work on the Stroop task in the same patients has shown similar performance in both patient groups (Djamshidian, O Sullivan et al. 2011), which is in keeping with previous studies showing preserved frontal lobe function (Voon, Thomsen et al. 2007; Siri, Cilia et al. 2010). Overall, our results are consistent with the hypothesis, that the ventral striatum underlies novelty seeking, perhaps due to input from the hippocampus. Additional work within this setting may further clarify the role of the ventral striatum in various choice behaviours and in social processing.