Acute Effects of Mecamylamine and Varenicline on Cognitive Performance in Non-smokers With and Without Schizophrenia

Abstract

Rationale

Nicotinic acetylcholine receptors (nAChRs) have been implicated in the pathophysiology of cognitive deficits in the domains of attention and memory in schizophrenia. While nicotinic agonists and antagonists have been proposed as smoking cessation aids, few comparisons have been made of these agents on cognitive performance in individuals with schizophrenia.

Objectives

This study investigated the acute effects of a nAChR antagonist, mecamylamine, and partial agonist, varenicline, on cognitive function in non-smokers with and without schizophrenia.

Methods

Single oral doses of mecamylamine, 10 mg, varenicline, 1 mg, and placebo were administered, one week apart, in random order to adults with schizophrenia (n=30) and to healthy volunteers (n=41) in a double-blind, crossover design. The primary outcome of interest was sustained attention as assessed with hit reaction time variability (HRT-SD) on the identical pairs continuous performance test (CPT-IP).

Results

Mecamylamine worsened performance on CPT-IP HRT-SD, a measure of attention, compared to varenicline in both groups. Performance on mecamylamine was worse than performance on both placebo and varenicline on several additional measures of attention, including CPT-IP hit reaction time (HRT) and random errors at various levels of task difficulty. There was a treatment by diagnosis interaction, such that mecamylamine worsened performance on CPT-IP 2-digit HRT, 3-digit random errors, and 4-digit hit rate compared to placebo and varenicline in participants with schizophrenia, effects not observed in controls.

Conclusions

These findings support a role for nAChR’s in attention and suggest that those with schizophrenia may be particularly sensitive to nAChR blockade.

Introduction

Nicotinic acetylcholine receptors (nAChRs) are widely distributed throughout the central nervous system and have an important role in attention and memory. Nicotine and nAChR agonists improve cognitive performance, particularly attention, memory, processing speed, and response inhibition in animals (Rollema et al. 2009; Buccafusco et al. 2007; Hahn et al. 2003), healthy volunteers (Meyers 2013, Heishman et al 2010; Dunbar et al. 2007) and in people with attention deficit disorder, dementia, age associated memory impairment and schizophrenia (Potter et al. 1999; Wilens et al. 1999; Levin et al. 2001 Wilens et al. 2006; Dunbar 2007b; Levin et al. 1996; Barr et al., 2008; Jubelt et al., 2008), as reviewed in (Heishman et al., 2010). For example, sustained attention, as assessed with continuous performance task (CPT) hit reaction time variability (HRT-SD), was improved by a single dose of transdermal nicotine in non-smokers with and without schizophrenia (Barr et al. 2008), in adults with schizophrenia at varying haloperidol doses (Levin et al. 1996), and in adults with ADHD (Levin et al. 2001). Poor baseline performance on this measure has also been associated with smoking cessation failure in smokers with schizophrenia (Culhane et al. 2008). Conversely, mecamylamine, a noncompetitive nAChR antagonist that acts at α4β2, α3β4, α3β2, and α7 nAChR subtypes in the low micromolar range, impairs delayed matching-to-sample performance in non-human primates (Katner et al. 2004), impairs acquisition of new information in elderly people without affecting the retrieval of previously learned information (Newhouse et al. 1994) and worsens performance on a visuospatial working memory task in people with schizophrenia (Sacco et al. 2005).

Cognitive dysfunction is a central feature of schizophrenia that contributes to poor functional outcomes (Green 2006; Nuechterlein et al. 2004), and dysregulation of the nAChR system is implicated in its pathophysiology, as reviewed in (Winterer 2010). Polymorphisms in the neuregulin-1 gene have been associated with schizophrenia and with reduced prefrontal cortical mRNA levels of α7 nAChRs (Mathew et al. 2007). α7 nAChR subunit polymorphisms are associated with severity of P50 auditory deficits that are associated with impaired attention in people with schizophrenia (Freedman et al. 1997), an impairment that is partially reversed with α7 nAChR partial agonists (Olincy et al. 2006). Recent in vivo evidence indicates that tobacco smoking is associated with upregulation of high affinity α4β2 nAChRs in the striatum and cortex in smokers with schizophrenia compared to non-smokers with schizophrenia, such that receptor levels in smokers with schizophrenia are not different from non-smoking healthy volunteers (Esterlis et al., 2013). nAChR availability was inversely associated with negative symptoms and positively correlated with cognitive performance in smokers with schizophrenia (Esterlis, et al., 2013).

Neuronal nAChRs are a target for drug discovery efforts for treatment of schizophrenia and nicotine dependence, as well as for pain and Alzheimer’s disease (Arneric et al. 2007; Hong et al. 2011). A recent review presented converging evidence for cholinergic dysfunction in people with schizophrenia (D’Souza and Markou 2012) and postulated that the high rate of comorbidity between schizophrenia and nicotine dependence may be due to attempts to compensate for this cholinergic dysfunction.

Consequently, this study was undertaken to evaluate the effect of nAChR stimulation and blockade on the cognitive performance of adults with schizophrenia and adults without current or past history of psychiatric illness by comparing acute effects of the nAChR partial agonist, varenicline, and the nAChR noncompetitive antagonist, mecamylamine, on attention, working memory, and response inhibition performance in a laboratory study in non-smokers. Our hypotheses were that diagnostic group would moderate/interact with the effect of medication on cognitive performance; specifically, cognition (attention, working memory, and response inhibition) would be enhanced with varenicline and worsened with mecamylamine in those with and without schizophrenia and these effects would be greater in people with schizophrenia than in healthy volunteers. The primary outcome of interest was sustained attention as assessed with the CPT-IP hit reaction time variability (HRT-SD), as this was improved by a single dose of transdermal nicotine in non-smokers with and without schizophrenia (Barr et al. 2008) and in adults with schizophrenia on varying doses of haloperidol (Levin et al. 1996) and deficits in this measure are associated with failure to quit smoking in those with schizophrenia.(Culhane et al. 2008)

Methods

We conducted a randomized, double-blind, placebo-controlled, crossover study of the effects of a single dose of the nAChR partial agonist, varenicline, and the nAChR antagonist, mecamylamine, on cognitive performance in non-smokers with schizophrenia and non-smoking controls with no lifetime DSM-IV Axis I psychiatric illness. Data collection occurred from 2005–2008 in accordance with the Declaration of Helsinki with the approval and oversight from the Human Subjects Committee of the Massachusetts Department of Mental Health and the Massachusetts General Hospital (MGH). All participants demonstrated competence to consent and completed the informed consent process prior to study procedures.

Study participants in the schizophrenia group were clinically stable, outpatient, non-smokers with schizophrenia on a stable, clinically determined dose of antipsychotic medication for at least 4 weeks, which were recruited from an urban community mental health clinic in Boston. Diagnoses were confirmed by clinical interview and medical record review (CAF and AEE). Healthy volunteers were recruited through media advertisements in the greater Boston area and had no lifetime history of Axis I disorders by SCID interview and no first-degree relatives with Axis I disorders by history. Eligible participants were 18–64 years old and were never smokers or former smokers with at least 3 months abstinence at the time of enrollment. Self-reported non-smoking status was confirmed by chart review and semi-quantitative salivary cotinine <10 ng/ml (NicAlert^™, JANT Pharmacal Corp, Encino, CA, USA) and expired carbon monoxide (CO) <9 ppm as measured by Bedfont Micro Smokerlyzer III (Bedfont Scientific Ltd., Kent, UK). Exclusion criteria included drug use disorder in the prior 6 months identified by self-report or by salivary drug test at screen positive for PCP, THC, cocaine, opiates, methamphetamine, amphetamine (Accutest Saliva Test^™, JANT Pharmacal Corporation, Encino, CA, USA), or alcohol (ALCO Screen, CHEMATICS, Inc., North Webster, IN, USA). Additional exclusion criteria included pregnancy, use of investigational medicine within the prior month, diagnosis of bipolar disorder, posttraumatic stress disorder, or anorexia nervosa, cognitive impairment secondary to head injury, dementia, general medical condition, or mental retardation, recent worsening of psychiatric condition, and medical conditions contraindicated for study medications, including hypotension, orthostatic hypotension, angina, myocardial infarction, congestive heart failure, stroke, pyloric stenosis, glaucoma, renal insufficiency, or use of vasodilators.

Procedures

After enrollment, participants attended a baseline visit and three study visits, one week apart, at which single oral doses of mecamylamine, 10 mg, varenicline, 1.0 mg, and identical placebo were administered, in random order to adults with schizophrenia (n=30) and to healthy volunteers with no lifetime Axis I psychiatric illness (n=41) in a double-blind, crossover design. Participants were instructed to follow their normal morning routine prior to study visits and not to restrict usual caffeine use in order to avoid effects of caffeine withdrawal on cognitive performance.

At baseline, physiological measures and demographic information were obtained and symptom rating scales (Beck Depression Inventory (BDI), Brief Psychiatric Rating Scale (BPRS), Scale for Assessment of Negative Symptoms (SANS), the Snaith Hamilton Pleasure Scale (SHAPS)), adverse event profiles, and cognitive performance tests (continuous performance test-identical pairs (CPT-IP), 3-card Stroop, N-back, and visual spatial working memory tasks) were administered. Participants were then randomly assigned in a double blind fashion by a computer generated randomization code for the order in which they would receive study medications in the subsequent three study visits.

The three study medication visits began with assessment of blood pressure, heart rate, and testing for recent drug, alcohol and tobacco use. Participants then took either 10 mg mecamylamine (Inversine®, Targacept, Winston-Salem, NC, USA), 1.0 mg varenicline (Chantix^™, Pfizer, New York, NY, USA), or identical placebo. The mecamylamine dose was chosen based on tolerability in previous, single-dose studies (Eissenberg et al. 1996; McClernon and Rose 2005; Newhouse et al. 1994; Pickworth et al. 1997; Rose et al. 2001; Rose et al. 2003). The varenicline dose was similarly chosen based on maximum tolerable, single, acute dose in non-smokers who showed appreciable blood levels without significant side effects (Burstein et al. 2006; Faessel et al. 2006; Obach et al. 2006). Study medication was over-encapsulated in the MGH Research Pharmacy, so that the appearance of all treatments was identical. After drug administration, participants watched a movie and had a small standard meal. Adverse effects and vital signs were monitored during this period. Three hours after medication ingestion, participants completed the neuropsychological battery and clinical rating scales.

Neuropsychological measures

The Continuous Performance Test-Identical Pairs, CPT-IP, Version 4.0 (Biobehavioral Technologies, New York, NY, USA), was developed and normed for use in people with schizophrenia and normal controls (Cornblatt et al. 1989; Cornblatt et al. 1988). In this task, participants were asked to respond when two identical pairs of numbers were presented in sequence by pressing a mouse key as quickly as possible, using the dominant hand. Four blocks of 150 trials with stimulus duration of 50 milliseconds (ms) and an inter-stimulus interval (isi) of 1000 ms were given. Stimuli were presented with increasing cognitive load: 2-, 3-, and 4-digit targets. Outcome variables measured included correct hits, hit reaction time (HRT), errors of commission: false alarms and random errors, and the primary outcome, variability, or standard deviation, of hit reaction time, HRT-SD.

The three-card Stroop Task was used to assess visual attention, processing speed, and cognitive interference. Three cards (Stoelting Co., Wood Dale, IL, USA) were presented; the first contained color names printed in black ink, the second contained colored patches of ink, and the third contained color names printed in incongruously colored ink. Participants were asked to read or name as many items as possible in 45 seconds for each condition. The raw interference ratio was calculated by dividing the color-word score by the color score (Lansbergen et al. 2007). A higher interference ratio indicates better performance.

The N-back task with 1- and 2-back parametric conditions was used. During the task, a letter was displayed for 1500 ms (or until response) every 2 seconds with a 500 ms isi. Participants were asked to press the “1” key for letters that corresponded to the letter 1 back for the 1-back condition, the “2” key for the 2-back condition, and the “3” key for non-target letters. Each task consisted of 72 trials, one third of which were target trials. Outcome variables included hit rate, hit reaction time, false alarms and random errors.

The Visual spatial working memory (VSWM) task was adapted from a human analogue of a spatial working memory task (Bollini et al. 2000; Keefe et al. 1995) developed for use in primates (Goldman-Rakic 1987). In this task, programmed in E-prime version 2.0, a “+” was displayed on the computer monitor for 5 seconds in a random location (balanced to appear in each of the four monitor quadrants equally). For 16 trials, participants were asked to place the cursor where the symbol appeared immediately after its display. For 16 additional trials, participants were asked to identify the symbol location after a 30-second delay. During the delay, participants were distracted by being asked to read aloud words appearing on the screen at 2-second intervals. The 30second delay was chosen based upon significant findings with this delay in prior studies of the effects of abstinence in smokers with schizophrenia (George et al. 2002). The outcomes of interest in this task were the average distance from the target for immediate and delayed recall, as well as the 30-second visuospatial working memory deficit, defined as the difference between these two outcomes (Keefe et al. 1995).

Statistical analyses

Baseline demographic and clinical variables were compared using independent Student’s t-tests or Fisher’s exact tests. Cognitive outcomes were analyzed using crossover analyses of covariance (ANCOVA) with drug (mecamylamine vs. varenicline vs. placebo) as a within-subject factor, diagnosis (schizophrenia vs. control) as a between subject factor, age as a covariate, as well as study period and drug administration sequence as between-subject crossover design factors. Where necessary, variables were log-transformed to better meet the model assumptions of normality and homogeneity of variance. Data points more than ± 4 standard deviations from the mean of each diagnosis by drug group were flagged as outliers and were replaced with the next highest or lowest observation that was not flagged as an outlier.

The eighteen outcome variables of the CPT-IP tasks and the three outcome variables each from the N-back and VSWM tasks were expected to be highly interdependent. We used Principal Component Analyses (PCA) to reduce the many potentially correlated variables within each task (see Table 2 for the variables used) to one or two factors that explain a majority of the variation in the outcome measures. These factors were then analyzed in separate analyses of covariance to test for overall effects of study medication and diagnosis, resulting in a reduced number of statistical tests conducted, thus reducing the chance of Type I errors. We followed these analyses with individual ANCOVAs for response variables of interest and used Tukey-adjusted tests of pairwise differences to test for specific treatment effects (significant at α=0.05). All analyses were performed using SAS, version 9.2 (SAS Institute Inc., Cary, NC, USA).

Table 2.

Effects of treatment and diagnosis on cognitive performance as raw means

Task	Variable	Schizophrenia			Control			Treatment			Treatment x Diagnosis
		Mecamylamine	Varenicline	Placebo	Mecamylamine	Varenicline	Placebo	F	df	p value	F	df	p value
CPT-IP	2-digit Hit rate (number)	25.2 ± 4.5	26.2 ± 5.1	25.3 ± 5.4	28.8 ± 2.4	29.2 ± 1.7	29.3 ± 1.0	1.12	2, 125	0.3310	0.97	2, 125	0.3825
	False alarms (ln)	0.69 ± 0.71	0.76 ± 0.87	0.57 ± 0.71	0.35 ± 0.49	0.22 ± 0.39	0.34 ± 0.47	0.15	2, 125	0.8603	1.82	2, 125	0.1656
	Random errors (ln)	0.90 ± 1.03	0.66 ± 1.07	0.57 ± 0.96	0.18 ± 0.43	0.21 ± 0.45	0.12 ± 0.27	3.13	2, 125	0.0473	2.59	2, 125	0.0788
	HRT (ms)	529.6 ± 80.6	478.6 ± 76.8	474.9 ± 68.8	451.5 ± 62.9	430.3 ± 55.3	441.9 ± 61.6	14.37	2, 125	<0.0001	4.59	2, 126	0.0119
	SD HRT (ms)	141.6 ± 57.5	120.5 ± 58.5	123.1 ± 44.9	91.2 ± 31.1	83.2 ± 30.2	87.8 ± 21.7	6.04	2, 126	0.0031	1.45	2, 126	0.2382
	d prime	2.91 ± 0.96	3.07 ± 1.17	3.04 ± 1.05	3.80 ± 0.53	3.93 ± 0.48	3.87 ± 0.44	1.21	2, 124	0.3028	0.08	2, 125	0.9277
	3-digit Hit rate (number)	23.1 ± 5.8	23.7 ± 6.7	23.3 ± 7.1	26.4 ± 4.4	27.4 ± 4.0	27.0 ± 4.1	0.38	2, 124	0.6859	0.20	2, 124	0.8161
	False alarms (ln)	1.19 ± 0.96	1.04 ± 1.05	1.03 ± 0.90	0.64 ± 0.62	0.61 ± 0.63	0.51 ± 0.62	1.32	2, 125	0.2720	0.75	2, 125	0.4749
	Random errors (ln)	1.14 ± 1.14	0.80 ± 1.17	0.76 ± 1.08	0.20 ± 0.41	0.26 ± 0.44	0.19 ± 0.35	3.33	2, 125	0.0389	4.86	2, 125	0.0093
	HRT (ms)	556.5 ± 93.9	524.8 ± 102.4	519.2 ± 81.5	489.7 ± 67.5	455.9 ± 55.2	480.9 ± 69.7	7.10	2, 125	0.0012	1.56	2, 125	0.2148
	SD HRT (ms)	162.8 ± 65.0	137.3 ± 72.3	143.0 ± 72.3	106.1 ± 36.2	93.8 ± 34.4	103.7 ± 35.3	5.52	2, 125	0.0050	0.84	2, 125	0.4350
	d prime	2.26 ± 1.23	2.45 ± 1.34	2.54 ± 1.17	3.19 ± 0.82	3.38 ± 0.87	3.35 ± 0.80	1.88	2, 124	0.1566	0.02	2, 124	0.9769
	4-digit Hit rate (number)	19.0 ± 6.5	21.9 ± 5.6	21.8 ± 6.1	23.5 ± 4.8	23.9 ± 5.2	24.2 ± 4.1	7.75	2, 123	0.0007	4.17	2, 123	0.0177
	False alarms (ln)	1.69 ± 0.82	1.84 ± 0.73	1.68 ± 0.81	1.46 ± 0.68	1.57 ± 0.70	1.42 ± 0.76	1.82	2, 124	0.1666	0.07	2, 125	0.9355
	Random errors (ln)	1.31 ± 1.04	1.03 ± 1.15	1.14 ± 1.06	0.45 ± 0.54	0.24 ± 0.42	0.37 ± 0.52	4.24	2, 125	0.0166	0.07	2, 125	0.9308
	HRT (ms)	591.4 ± 113.2	542.3 ± 96.6	552.4 ± 81.8	524.8 ± 74.7	502.8 ± 76.4	517.6 ± 70.4	8.61	2, 124	0.0003	1.54	2, 125	0.2195
	SD HRT (ms)	159.3 ± 52.2	147.8 ± 81.5	152.9 ± 58.7	122.6 ± 40.1	104.9 ± 36.2	114.7 ± 38.5	2.70	2, 126	0.0708	0.21	2, 126	0.8095
	d prime	1.41 ± 0.90	1.58 ± 0.96	1.75 ± 1.04	2.11 ± 0.80	2.05 ± 0.89	2.18 ± 0.76	2.30	2, 123	0.1049	1.41	2, 123	0.2472
Stroop	Interference score	50.2 ± 6.8	52.8 ± 8.0	50.3 ± 8.3	54.4 ± 10.5	53.3 ± 9.2	53.5 ± 8.6	0.56	2, 125	0.5741	2.12	2, 125	0.1239
N-back	2-back Hit rate	0.58 ± 0.27	0.63 ± 0.26	0.64 ± 0.25	0.83 ± 0.15	0.82 ± 0.19	0.80 ± 0.19	0.48	2, 120	0.6192	3.02	2, 120	0.0525
	Error rate	0.26 ± 0.22	0.23 ± 0.22	0.20 ± 0.20	0.12 ± 0.11	0.17 ± 0.20	0.12 ± 0.11	1.75	2, 121	0.1780	1.77	2, 122	0.1739
	HRT (ms)	909.2 ± 250.9	931.2 ± 297.9	894.4 ± 282.1	864.6 ± 253.9	795.8 ± 241.7	825.7 ± 229.3	1.05	2, 120	0.3536	1.01	2, 121	0.3668
	SD HRT (ms)	477.2 ± 189.0	427.7 ± 177.9	450.3 ± 158.0	344.5 ± 130.5	342.0 ± 150.6	385.6 ± 174.1	1.89	2, 121	0.1553	1.34	2, 121	0.2651
VSWM	Mean distance, immediate (ln)	2.83 ± 0.71	2.72 ± 0.66	2.87 ± 0.76	2.29 ± 0.54	2.28 ± 0.54	2.25 ± 0.53	1.04	2, 121	0.3567	0.63	2, 121	0.5347
	Mean dist., 30sec delay (ln)	4.01 ± 0.64	4.01 ± 0.69	4.00 ± 0.76	3.58 ± 0.46	3.62 ± 0.53	3.64 ± 0.40	0.26	2, 120	0.7735	1.15	2, 120	0.3196
	Difference (30sec – imm.) (ln)	3.52 ± 0.92	3.64 ± 0.81	3.48 ± 1.00	3.24 ± 0.54	3.31 ± 0.60	3.36 ± 0.40	1.13	2, 123	0.3261	1.21	2, 123	0.3030

CPT-IP, Continuous Performance Test-Identical Pairs; ln, natural logarithm; ms, milliseconds; HRT, hit reaction time; SD, standard deviation; VSWM, Visual Spatial Working Memory. Values are shown as raw means ± SD.

Results

Participant characteristics

Eighty-nine participants were enrolled, and 71 participants (30 patients, 41 controls) completed at least one medication visit and were included in the analyses. Four participants (3 with schizophrenia, 1 control) missed one and 5 participants (3 with schizophrenia, 2 controls) missed two of the three medication visits. Participants’ demographic and clinical characteristics are presented in Table 1. Participants with schizophrenia were older (47±10 vs. 37±16, p=0.001), had a higher rate of prior nicotine dependence (52% vs. 26%, p=0.042), and had higher scores on the BDI (p=0.001), SANS (p<0.001), and BPRS (p<0.001) than controls. Of the 71 participants with at least one study visit, all reported no smoking in the past 90 days at enrollment. Self-reported smoking history was as follows: 43 reported never-smoking, 17 reported not smoking in > 3 years, 7 reported not smoking for ≥ 3 months, and 4 did not report details of their past smoking history. All participants had CO ≤ 7ppm and were verified as abstinent by semi-quantitative salivary cotinine tests at all four study visits, including baseline.

Table 1.

Demographic and baseline clinical characteristics of participants

Variable	Schizophrenia (n = 30)	Control (n = 41)	p value
Age	47 ± 10	37 ± 16	0.001
Gender
Male	20 (66.7%)	17 (42.5%)	0.056
Female	10 (33.3%)	23 (57.5%)
Race
Caucasian	24 (82.8%)	22 (57.9%)	0.070
African American	5 (17.2%)	12 (31.6%)
Asian or Pacific islander	0 (0%)	4 (10.5%)
Participant’s education (year)	14 ± 3	15 ± 2	0.111
Father’s education (year)	13 ± 5	14 ± 3	0.557
Mother’s education (year)	13 ± 4	14 ± 3	0.455
Past history of tobacco smoking	15 of 29 (51.7%)	10 of 39 (25.6%)	0.042
CO at baseline (ppm)	0.9 ± 1.2	0.6 ± 1.2	0.264
BDI	8.8 ± 9.7	1.8 ± 2.9	0.001
SANS composite	24 ± 14	5 ± 6	<0.001
BPRS	46 ± 12	27 ± 3	<0.001
SHAPS	24 ± 9	22 ± 6	0.176
Antipsychotic medications
Clozapine	10 (33.3%)
Olanzapine	8 (26.7%)
Aripiprazole	5 (16.7%)
Risperidone	4 (13.3%)
Quetiapine	3 (10.0%)
Ziprasidone	3 (10.0%)
Fluphenazine	1 (3.3%)
Loxapine	1 (3.3%)
Perphenazine	1 (3.3%)
Thioridazine	1 (3.3%)
Concomitant psychotropic medications
Fluoxetine	5 (16.7%)
Valproate	5 (16.7%)
Clonazepam	4 (13.3%)
Bupropion	2 (6.7%)
Gabapentin	2 (6.7%)
Lithium	2 (6.7%)
Lorazepam	2 (6.7%)
Trazodone	2 (6.7%)
Benztropine	1 (3.3%)
Buspirone	1 (3.3%)
Citalopram	1 (3.3%)
Clomipramine	1 (3.3%)
Doxepin	1 (3.3%)
Lamotrigine	1 (3.3%)
Modafinil	1 (3.3%)
Oxcarbazepine	1 (3.3%)
Sertraline	1 (3.3%)
Topiramate	1 (3.3%)
Venlafaxine	1 (3.3%)

CO, carbon monoxide; BDI, Beck Depression Inventory; SANS, Scale for Assessment of Negative Symptoms; BPRS, Brief Psychiatric Rating Scale; SHAPS, Snaith-Hamilton Pleasure Scale. Values are Mean ± SD or number (%). p-values based on two-sample Student’s t-tests for continuous variables and Fisher’s Exact tests (two-sided) for binary variables. There were missing data for gender (1), race (4), participant’s education (3), father’s education (16), mother’s education (13), past history of tobacco smoking (3), and CO at baseline (1). The sum of % in medications exceeds 100 as some patients take multiple medications.

Neurocognitive task performance

Principal components analyses

The first and second principal components of the CPT-IP data (accounting for 49% and 18% of the variance in the data, respectively) showed a strong overall effect of study medication (PC1_CPT-IP: F_2,122 = 10.6, p<0.0001; PC2_CPT-IP: F_2,122 = 10.3, p<0.0001) and diagnosis (PC1_CPT-IP: F_1,63.5 = 19.9, p<0.0001; PC2_CPT-IP: F_1,63.9 = 4.0, p=0.049), but no study medication by diagnosis interaction (PC1_CPT-IP: F_2,122 = 0.80, p=0.45; PC2_CPT-IP: F_2,122 = 1.75, p=0.18). In both of the components, performance on mecamylamine was significantly different from that on placebo and varenicline, while there was no difference between placebo and varenicline in either component. Neither the first (54% of variance) nor the second (31% of variance) principal component of the n-back task nor the principal component of the VSWM task (78% of variance) showed an effect of study medication (PC1_2-back: F_2,120 = 0.25, p=0.78; PC2_2-back: F_2,121 = 1.21, p=0.30; PC1_VSWM: F_2,120 = 0.33, p=0.72), or treatment by diagnosis interactions (PC1_2-back: F_2,121 = 2.99, p=0.054; PC2_2-back: F_2,121 = 0.83, p=0.44; PC1_VSWM: F_2,120 = 1.02, p=0.36). All principal components of the CPT, N-back and VSWM tasks, except for the second principal component of the 2-back task, showed a strong effect of diagnosis. Based on the results of the PCA, we focused our analyses on effects of study medication on attention as assessed with the CPT-IP in order to investigate where the effects were strongest.

Study design and age effects

An effect of visit number (i.e., period) on cognitive outcome was observed such that participants had lower interference ratios (i.e., poorer performance) in the Stroop task at the first visit than at both subsequent medication visits, likely due to practice effects. No principal component of any of the other cognitive tasks indicated any significant visit effects. Participant age was correlated with some cognitive outcomes (2-back PC2 and Stroop task) and was included as a covariate in all subsequent analyses. In the individual ANCOVAs, age significantly predicted greater numbers of false alarms in CPT-IP 2-digit, 3-digit, and 4-digit tasks, poorer performance on the Stroop task, and longer hit reaction times in the 2-back task (data not shown).

Treatment and treatment by diagnosis effects

In the CPT-IP task, mecamylamine worsened HRT-SD, HRT and random errors in all levels of task load tested (except for HRT-SD at the 4-digit level which had a trend level medication effect at p=0.07; Table 2). Mecamylamine worsened 2-digit HRT, 3-digit random errors, and 4-digit hit rate in participants with schizophrenia but had no effect in controls (Table 2). There was no effect of treatment on the Stroop interference, 2-back task, and VSWM tasks. Neurocognitive outcomes are presented as raw means by treatment and diagnosis in Table 2.

In post-hoc tests of treatment main effects on the primary outcome, CPT-IP HRT SD, participants’ performance was worse following single dose mecamylamine than following single dose varenicline. This difference was significant at p<0.05 for the 2- and 3-digit tasks, and there was a trend for significance at for the 4-digit task. Performance on neither mecamylamine nor varenicline differed from placebo on the primary outcome, nor was there a treatment by diagnosis interaction evident in post-hoc tests (Figure 1).

Figure 1.

Sustained attention as assessed with the standard deviation of the hit reaction time on the identical pairs continuous performance task was worse in both those with and without schizophrenia on mecamylamine than varenicline, while neither differed from placebo.

In post-hoc tests of the secondary measure, CPT-IP HRT, performance on mecamylamine was significantly worse than that on either varenicline or placebo on the 4-digit task and worse than that on varenicline but not placebo on the 3-digit task. At the 2-digit task difficulty, CPT-IP HRT was again worse on mecamylamine compared with varenicline and placebo, but only in participants with schizophrenia (Figure 2, bottom row).

Fig. 2.

Treatment effects of mecamylamine, varenicline, and placebo on hit rate (top row), random errors (second row), hit reaction time (third row) in the continuous performance test, identical pairs version (CPT-IP) with 2-digit (left column), 3-digit (middle), and 4-digit (right) targets. For outcome variables that showed a significant treatment by diagnosis (schizophrenia vs. control) interaction, the interaction graph is shown in place of the main effect graph. In each panel, bars or data points that do not share common letters are different at a significance level of p<0.05 (Tukey-adjusted). In all three interactions, CPT-IP performance was worse on mecamylamine in those with schizophrenia compared to performance on varenicline or placebo, while performance of controls was unaffected by either medication.

In post-hoc tests of the secondary measure, CPT-IP random errors, performance on mecamylamine was worse than that on varenicline but not on placebo in the 4-digit task, worse on mecamylamine than on varenicline or placebo in the schizophrenia group only on the 3-digit task, and worse than placebo but not varenicline on the 2-digit task (Figure 2, middle row).

For CPT-IP correct hits, performance on mecamylamine was worse than that on varenicline or placebo only in those with schizophrenia at the 4-digit task difficulty (Figure 2)

Discussion

This double-blind, placebo-controlled, crossover design study evaluated the effects of single doses of the noncompetitive nAChR antagonist, mecamylamine, and the nAChR partial agonist, varenicline, on the cognitive performance of non-smoking individuals with schizophrenia and healthy controls. Our hypothesis was that these single doses of the antagonist, mecamylamine, would worsen, and that the agonist, varenicline, would improve attention as assessed with the CPT-IP as well as other domains of attention and memory compared with placebo, and that the effect would be greater in those with schizophrenia than in controls. We observed that mecamylamine worsened performance compared with both varenicline and placebo on the first two principle components of the CPT-IP, but that there were no differences in performance on varenicline vs placebo and no treatment by diagnosis interactions.

On the primary outcome measure, sustained attention, as assessed with HRT SE, performance was worse on mecamylamine than on varenicline, but performance on neither mecamylamine nor varenicline differed from placebo. There was also no interaction effect of study drug and diagnosis on this outcome. Thus, there was an overall effect of mecamylamine to worsen attentional performance vs both placebo and varenicline on the first two components of the PCA for the CPT, and the direction of the effect was as hypothesized, and supports prior reports that nicotinic mechanisms are involved in sustained attention. In terms of the primary outcome, sustained attention (CPT-IP HRT SD), we are unable to clearly interpret this effect as a worsening with mecamylamine or an improvement with varenicline, because performance on mecamylamine was significantly worse than that on varenicline but performance on neither was different from placebo.

On other domains of the CPT-IP, attentional performance was worse on mecamylamine compared with placebo, in some instances an effect seen only in those with schizophrenia. Thus attentional performance was worse on mecamylamine than placebo on several secondary outcome measures of attention with both main effects and interaction effects, reflecting drug-associated impairment in both groups and possibly greater impairment in those with schizophrenia on some measures. Our results are consistent with prior findings that mecamylamine blocked salutary effects of smoking reinstatement on attention in those with schizophrenia (George et al. 2002; Sacco et al. 2005). Because the PCA did not reveal treatment by diagnosis interactions for either of the first two components of the CPT-IP, treatment by diagnosis interactions on individual subtests should be interpreted with caution.

Contrary to our hypothesis, we did not find evidence from effects of a single, 1.0 mg dose that varenicline may substitute at least partially for nicotine with respect to amelioration of cognitive deficits in non-smokers with schizophrenia. Performance on several attention and verbal working memory tasks on varenicline was better than on mecamylamine, but not better than placebo in participants with schizophrenia or controls. In a prior study from our group (Barr et al. 2008), a single, 7.0–14.0 mg dose of transdermal nicotine was associated with significant improvement on attention and inhibitory control, as assessed with the CPT-IP and classic Stroop task, suggesting the cognitive effect of nicotine may be greater than that of the partial agonist, varenicline, at least for a single, 1.0 mg dose. Prior studies reporting beneficial effects of varenicline on cognition in those without psychiatric illness have been repeat dosing studies (Patterson et al. 2009). A higher dose of varenicline, 2.0 mg/day, dosed daily for 8 weeks in people with schizophrenia, was associated with improved performance on CPT hit reaction time, Stroop interference, the Digital Symbol Substitution Test, and the Wisconsin Card Sorting Test (Shim et al. 2012), and in another study this dose was associated with improved verbal learning and memory, but not attention, visuo-spatial learning or memory (Smith et al. 2009). In contrast, the lower varenicline dose of 1.0 mg/day, used in the present study, when dosed for 8 weeks had no effect on sustained attention as assessed with the Conners’ CPT, spatial working memory, or processing speed, though it improved p50 sensory gating deficits, and reduced startle reactivity and anti-saccade error rate (Hong et al. 2011). Thus it appears that a 2.0 mg dose, and possibly repeated dosing, may be necessary to produce significant cognitive improvement with varenicline. Other possible explanations for the variability in findings include small treatment effect sizes and heterogeneity across studies in smoking status, degree of underlying cognitive dysfunction, disease course, and concomitant medications, any of which could impact the effect of nAChR active agents on cognitive performance. We noted no adverse effects of varenicline on cognitive performance during our study, suggesting that varenicline may be well tolerated with respect to cognitive performance when used for smoking cessation in this population. Study in acutely abstinent smokers with schizophrenia would be needed to know if varenicline may ameliorate abstinence-induced cognitive deficits in people with schizophrenia who are trying to quit smoking as it has been shown to do in smokers without psychiatric illness (Patterson et al. 2009).

This study has several strengths. Participants were non-smokers in order to avoid confounding effects of nicotine withdrawal and reinstatement on cognitive performance. The crossover design reduced the effects of inter-individual variability in cognitive performance on our ability to detect a medication effect. The study drug administration sequence was randomized to balance learning effects on assessment of cognitive performance, and we assessed a broad range of cognitive domains. We conducted a principle components analysis to reduce the dimensionality of our cognitive task datasets that showed an effect of medication on the primary dependent measure, reducing the likelihood of Type I error.

The primary limitation of the study was the single-dose design of each medication tested that may may not generalize to chronic dosing, which may have significant effects on receptor expression, subunit composition, and pharmacodynamic effects (Collins et al. 2009). Another limitation that stemmed from the single-dose design was the low dose used of both mecamylamine and varenicline, possibly reducing our ability to detect an effect of a more clinically-relevant dose. Another limitation is that we did not include a test of motor speed. Participants with schizophrenia in this study were treated with clinically determined medications, many of which have direct effects on cholinergic function that may bias the study toward the null. For example, clozapine attenuates the effects of nicotine on attentional performance in animal models of schizophrenia (Rezvani et al. 2008), fluoxetine and bupropion have been shown to have nicotinic antagonist properties in vitro (Hennings et al. 1997; Slemmer et al. 2000), and bupropion has been shown to improve variability of attention in smokers with schizophrenia independent of abstinence (Evins et al. 2005). Due to the small number of participants, it was not possible to test the additional effect of these medications on study outcomes. The pattern of effects we see in this study is for single dose 1.0 mg varenicline to have little effect on cognitive performance and for mecamylamine to have a detrimental effect on attentional performance, particularly in those with schizophrenia. Because more of the participants with schizophrenia were past smokers, and this is an issue not just of this sample but of the population of people with schizophrenia in general, who are more likely to have a lifetime diagnosis of nicotine dependence, we cannot say definitively whether the effect of mecamylamine in those with schizophrenia was due solely to the effect of the diagnosis of schizophrenia or to the comorbid diagnosis of prior nicotine dependence. A limitation to the interpretation of the results is worth highlighting. While overall tests of attentional performance revealed worsening with mecamylamine compared with placebo and varenicline, and on many of the individual subtests of the CPT, performance was consistently worse on mecamylamine than varenicline, but for the domains in which neither separated from placebo, we are unable to determine from the results whether mecamylamine worsens or varenicline improves task performance or both. Further study would be needed to clarify this result.

Conclusion

In this study, we found significant differences between the effects of mecamylamine and varenicline on the performance on the identical pairs continuous performance task. Mecamylamine worsened performance relative to placebo and varenicline on several measures of attentional performance and did so to a greater extent in study participants with schizophrenia than in healthy controls on some measures. These findings support prior work implicating nAChR in attentional performance and suggesting that those with schizophrenia may be particularly sensitive to nAChR blockade.