Effects of moderate-dose treatment with varenicline on neurobiological and cognitive biomarkers in smokers and nonsmokers with schizophrenia or schizoaffective disorder

Abstract

Context

Nicotine administration transiently improves many neurobiological and cognitive functions in schizophrenia. It is not yet clear which nAChR subtype(s) is responsible for these seemingly pervasive nicotinic effects in schizophrenia.

Objective

α4β2 is a key nAChR subtype for nicotinic actions. We investigated the effect of varenicline, a relatively specific α4β2 partial agonist/antagonist, on key biomarkers that are associated with schizophrenia and are previously shown to be responsive to nicotinic challenge in humans.

Design

double-blind, parallel, randomized, placebo controlled trial in schizophrenia patients to examine effects of varenicline on biomarkers at short-term (2 week) and long-term (8 week), using a slow titration and moderate dosing strategy for retaining α4β2 specific effect while minimizing side effects.

Setting

Outpatients.

Participants

69 smoking and nonsmoking patients randomized; 64 completed week 2; 59 completed week 8.

Intervention(s)

varenicline.

Main Outcome Measure(s)

prepulse inhibition, sensory gating, antisaccade, spatial working memory, eyetracking, processing speed, and sustained attention.

Results

Moderate dose of varenicline 1) reduced P50 sensory gating deficit after a long-term (p=0.006) but not short-term treatment; significant in nonsmokers but not in smokers; 2) reduced startle reactivity (p=0.015) regardless of baseline smoking status; and 3) improved executive function by reducing antisaccade error rate (p=0.034) regardless of smoking status. Moderate dose varenicline had no significant effect on spatial working memory, predictive and maintenance pursuit, processing speed, or sustained attention by Connor’s CPT. Clinically, there was no evidence of exacerbation of psychiatric symptoms, psychosis, depression, or suicidality using a gradual titration, 1 mg daily dose.

Conclusions

Moderate dose varenicline has a unique treatment profile on core schizophrenia related biomarkers. Further development is warranted for specific nAChR compounds and dosing/duration strategy to target subgroup of schizophrenia patients with specific biological deficits.

ClinicalTrials.gov Identifier: NCT00492349

Introduction

Smoking or nicotine challenge in humans transiently influences many biomarkers associated with schizophrenia, including prepulse inhibition^1-4, sensory gating^5;6, antisaccade^7;8, eyetracking^9-12, sustained attention^13-18, information processing speed^18-22, and spatial information processing^15;23-25, leading to the pharmaceutical effort to target nAChRs for novel CNS drug development. There are 17 known nicotinic receptor subunits²⁶. It is unclear which nAChR subtype(s) is responsible for these seemingly pervasive nicotinic effects: identifying it would be instrumental for guiding the development of biologically based drugs.

Of the nAChR subtypes, α4β2, α3β4 and α7 are the primary ones in the brain²⁶. Until recently, clinical efforts on nAChR therapeutics for schizophrenia have been focused more on α7, including neurocognitive and P50 gating improvements obtained in an initial trial of the partial α7 agonist dimethoxybenzylidene anabaseine²⁷. In the subsequent study, P50 was not reported; an improvement of negative symptoms was found²⁸. Tropisetron, a serotonin receptor antagonist with partial α7 agonist effect did not improve negative symptoms but improved visual sustained attention²⁹. Galantamine, a cholinergic compound with α7 and α4β2 allosteric modulation properties improved processing speed³⁰ although in another trial galantamine did not improve cognition³¹. Another α7 nAChR partial agonist R3487 failed to show cognitive improvement³². Overall, the findings on whether α7 compounds improve clinical symptoms or biomarkers are not consistent. The inconsistent use of endpoint measures also poses a challenge to interpret reproducibility, although the positive effects appeared less reproducible than acute nicotine effects. Alternatively, nicotine’s effects on these biomarkers might not be primarily originated from α7 but instead from α4β2. No data systematically comparing clinical α4β2 nAChR action across schizophrenia-related biomarkers are available.

At therapeutic levels, varenicline is highly selective for α4β2²⁶ and displays robust agonist and antagonist properties of nicotine³³. Varenicline is a partial agonist for α4β2, α3β2, α6 and a full agonist for α7. However, the equilibrium binding affinity is hundreds of times more for α4β2 compared with α7 and other subtypes²⁶; and the functional affinity is also 8 to 24 fold higher for α4β2 compared with α7 and α3β4³⁴. We chose a reduced dosing strategy to further separate the effect on α4β2 vs. α7 and α3β4 likely yielding a more specific α4β2 effect. We also selected biomarkers previously associated with positive response in humans during nicotine or smoking challenges as our primary endpoints: prepulse inhibition, sensory gating, antisaccade, visual spatial working memory, eyetracking, processing speed, and sustained attention. Additional rationale of biomarker selection is described in the Methods section. This design of including “nicotine-responsive biomarkers” in the same trial should facilitate cross-marker comparisons on α4β2 effects.

The study tests the hypothesis that nicotinic effect on biomarker deficits in schizophrenia is due to an α4β2 mechanism, which should inform whether nAChR CNS drug development for schizophrenia should focus on this subtype. We also planned to examine whether short-term biomarker improvement by varenicline, if present, may predict longer-term improvement in clinical outcomes. Biomarker here refers to electrophysiological, neurophysiological and cognitive measures. Varenicline provides the first relatively specific α4β2 compound for human studies; although it is not simply an agonist or antagonist so one does not necessarily expect an identical biomarker profile compared with the agonist effect of nicotine. The α4 receptor regulates sustained dopamine release in the striatum³⁵. This dopaminergic modulation of the mesolimbic pathway is considered the key mechanism of varenicline²⁶. Varenicline as an α4β2 partial agonist/antagonist for smoking cessation is thought to 1) provide sustained dopaminergic tone to limit craving by its agonist quality and 2) attenuate dopaminergic reward response to nicotine by its antagonist property²⁶, thereby breaking the reward-craving cycle leading to addiction^36-38. Schizophrenia treatment might benefit from sustained dopaminergic tone enhancement (the 1^st mechanism) and/or through modest antagonism of hyperdopaminergic activity (the 2^nd mechanism). α4β2 dysregulation is documented in schizophrenia^39-43 not secondary to smoking⁴¹, and α4β2 is involved in cognitive functions^44;45. Varenicline offers a potential alternative to treat the putative nicotinic/dopaminergic dysfunction in schizophrenia.

We recruited smoking and nonsmoking schizophrenia patients to evaluate varenicline effects with and without potential smoking-related confounds. We chose a moderate dose (1mg/day), which is half of the recommended 2mg for smoking cessation. Compared to 2mg/day, 1mg/day resulted in over 50% reduction in the primary side effect of nausea yet reduced quit rate by only a fraction⁴⁶. Therefore, a moderate dose strategy should 1) reduce risk especially in nonsmoking patients; 2) still allow testing whether sustained α4β2 modulation would influence biomarkers; and 3) further capitalize on the differential affinity of varenicline to α4β2 vs. other subunits and ensure that significant effects, if found, are likely due to α4β2 rather than α7 or α3β4 nAChR subunits.

Methods

Subjects

Participants gave informed consent approved by University of Maryland IRB. They were 18-60 years of age, with schizophrenia or schizoaffective disorder, were on antipsychotic medication and clinically stable for 4 weeks or longer. Two patients were on first-generation antipsychotics; the remainders were on second-generation antipsychotics. Patients on smoking cessation therapy were excluded. Major medical conditions, EKG atrioventricular block, and renal insufficiency were exclusionary. We randomized 69 patients (Figure 1). Age, gender, and baseline smoking status were matched (Table 1).

Figure 1.

Consort flow diagram.

Table 1.

Key baseline demographic, clinical, and smoking information.

		Varenicline Group			Placebo Group			Statistics
		ALL	NSK	SK	ALL	NSK	SK	F or X2	p values
Clinical Information	N	32	13	19	32	11	21	0.27	0.61
	Male:Female	20:12	7:6	13:6	22:10	8:3	14:7	0.28	0.60
	Age	44.03 (1.82)a	45.69 (2.64)	43.00 (2.52)	41.57 (193)	42.09 (3.15)	41.48 (2.49)	0.86	0.36
	BPRS Baseline Total	34.13 (1.44)	31.08 (1.99)	36.21 (1.90)	34.69 (1.52)	34.00 (2.49)	35.05 (1.96)	0.07	0.79
	HAM-D Baseline Total	5.16 (0.70)	5.31 (1.12)	5.05 (0.93)	6.06 (0.82)	6.64 (0.98)	5.76 (1.15)	0.70	0.40
Smoking Information	FTND			5.05 (0.60)			4.90 (0.46)	0.04	0.84
	Pack-year			24.62 (5.52)			22.50 (6.13)	0.06	0.80
	Age start smoking			17.21 (1.32)			16.52 (1.72)	0.10	0.76
	Age start regular smoking			19.37 (1.36)			19.19 (1.80)	0.01	0.94
	CPD Baseline			19.21 (3.15)			17.19 (2.84)	0.23	0.64

^aMean (standard error)

Study Design

A double-blind, parallel groups design was used. Patients were 1:1 randomized to varenicline or placebo, stratified by smoking status and gender. Smoking status was current smokers (daily smokers of any amount for over 1 year) or nonsmokers (never smokers or past-smokers who had not smoked for over 1 year). Varenicline and placebo were packaged in identical capsules placed in blister packs, dispensed in person with assessments weekly for the first 2 weeks and then bi-weekly. Patients followed a slow titration of 0.5mg daily × 1 week and then 0.5mg bid × 7 weeks. The unique α4β2 profile of varenicline could yield slow but continuous modulation seen in chronic administration of nicotine in animals⁴⁷. Therefore, key biomarkers were measured at baseline, week 2 and week 8. After the last dose, patients were monitored for 2 weeks and the study terminated at week 10. After discharge, smokers who wished to continue varenicline for smoking cessation with his/her own physician could request a disclosure of whether they were treated with varenicline or placebo. This unblinding carries a risk of biasing raters and patients, although this possibility was minimized by restricting knowledge of the treatment to one coordinator. To recruit a representative sample and avoid potential bias by patients seeking smoking cessation, desire to quit smoking was not a requirement for participation. Smoking cessation counseling was also not implemented, other than encouraging smoking cessation as routine clinical practice, in order to minimize different levels of clinical attention between smokers and nonsmokers.

Clinical and smoking related assessments

The primary measure of psychiatric symptoms was the Brief Psychiatric Rating Scale (BPRS), done at each visit. At baseline and week 8, negative symptoms were assessed with the Schedule for Assessment of Negative Symptoms (SANS), depression with the Hamilton rating scale for depression (HAM-D), and function with the Level of Function Scale (LOF) and Global Assessment of Functioning (GAF). Suicidality was assessed at every visit. Cigarettes per day (CPD) was the primary measure of smoking change. End expired CO level (not timed to the last cigarette) was collected as an approximate validation of the CPD report. To test under a relatively steady varenicline level, participants took study medication at least 2 hours before each biomarker testing. Smokers were required to refrain from smoking for 1 hour before testing. The Minnesota Nicotine Withdrawal Scale (MNWS) was given immediately after each laboratory test to evaluate potential confounding effects of withdrawal. We used the Varenicline Side Effect Checklist to rate side effects from 0 to 3 (none, mild, moderate, severe).

Biomarker laboratory assessments

1. Prepulse inhibition (PPI) and startle reactivity. PPI measures the suppression of the acoustic startle eyeblink response by a prepulse; while startle reactivity measures the amplitude of the startle eyeblink itself. PPI abnormality in schizophrenia can be reduced by smoking and nicotine^2-4;48. PPI was recorded using methods previously described^4;49. A session started with a 3-minute acclimation with 70-dB white noise. Startling pulse alone trials contained 116 dB white noise lasting 40ms. The prepulse-pulse trials contained a 20ms, 80 dB white noise prepulse. The test included 18 pulse alone trials (measuring startle reactivity) and 12 prepulse-pulse trials with 120ms inter-stimulus interval for PPI. %PPI was calculated by EMG amplitudes from [(startle alone – prepulse-pulse condition)/startle alone × 100]. Twenty-five percent of patients were classified as non-responders⁴⁹ and excluded from analysis (no difference between treatment groups).

2. P50 gating. Nicotine reduces double-click evoked P50 sensory gating deficit in schizophrenia^5;6. As previously described⁴⁹, data were collected in a sound chamber using a Neuroscan SynAmp² 64 channel system using 1 KHz sampling rate at 0.1-200 Hz. Subjects listened to 150 pairs of clicks (1-ms, 75 dB, 500-ms interclick interval, 10 second intertrial interval). Linked mastoid electrodes served as reference. Electrode impedance was kept below 5 kΩ. CZ was used for scoring^50;51. Records were filtered at 3-100 Hz (24 octave), threshold-filtered at ±75 μV, and averaged to obtain the first (S1) and second (S2) stimulus P50 waves. P50 gating was the S2/S1 P50 ratio.

3. Smooth pursuit eye movement or eyetracking. Nicotine improves performance in several eyetracking measures^9-12. We used a recently developed foveal stabilization based paradigm to examine the predictive mechanism of eyetracking⁵². Data were collected using an EyeLink II eyetracker sampling at 500 Hz, using target speeds of 18.7°/s at 24°of visual angle. We stabilized the target onto the fovea and covertly measured the predictive mechanism during eyetracking without subject’s awareness⁵². We calculated the predictive pursuit gain averaged over the 1-second stabilization period. Pursuit gain is the averaged artifact-free eye velocity divided by target velocity⁵². Maintenance pursuit gain was calculated as the eye velocity during the regular eyetracking period (without foveal stabilization) divided by target velocity.

4. Antisaccade and memory saccade: Antisaccade is an eye movement measure of disinhibition, frequently abnormal in schizophrenia⁵³. Nicotine reduces antisaccade error rate^7;8. Subjects fixated on a center target for 1.5 to 2.5 seconds. A peripheral cue was presented 5° or 10° to the right or left of center. The fixation was turned off 200 msec after the appearance of the peripheral cue⁵⁴. Subjects were instructed to make a saccade equidistant to the position of the cue but in the opposite direction. Antisaccade error rates measure the inability to inhibit the reflexive response toward the target, calculated as the number of trials where the subject looked toward the cue instead of the opposite direction, divided by the total number of valid trials.

   Nicotine increases spatial information processing^15;55 and affects spatial working memory²⁵ although some findings appear contradictory^12;15;56. We assessed spatial working memory by memory saccade, often impaired in schizophrenia⁵³. Subjects were required to fixate on a central fixation while a peripheral cue was briefly (250 msec) flashed. After another 10 seconds, subjects were signaled by the removal of the central fixation to make a saccade to the horizontal cue location. There were 8 target locations 2.5° apart, ranging from 2.5° to 10° left or right of center. Spatial working memory was measured by the positional error, calculated as the distance between the saccade position and the target position.

5. Neuropsychological measures. Deficit in sustained attention and in processing speed are known problems in schizophrenia^57;58. In the majority of subjects included in previous studies, nicotine did not affect CPT-identical pair (CPT-IP) sustained attention as measured by d’^7;12;13. However, nicotine improved Conner’s CPT performance^14-17. Processing speed measured by WAIS-III Digit Symbol Substitution Task (DSST) accounts for a disproportionate amount of the cognitive impairment seen in schizophrenia^59-62. Nicotine speeds up stimulus evaluation and information processing^18-20;22. Based on these data, we measured attention by Conner’s CPT d’ and processing speed by DSST as the primary neuropsychological endpoints. We opted to use this “nicotine-responsive biomarker battery” as the primary endpoints. The MATRICS-Consensus Cognitive Battery (MCCB)^63;64 was considered secondary because several tasks (e.g., CPT-IP) imbedded in MCCB may not be sensitive to nicotine^{7;12;16;17;23}.

All laboratory measures were processed and scored in blinded condition.

Statistical Analysis

All models were fitted using SAS® PROC MIXED. Treatment effects were analyzed using a mixed model for incomplete repeated measures ANCOVA: endpoint = baseline measurement + treatment + time + baseline smoking status + all interaction terms. Terms for smoking status control for potential confounding or moderating effects of baseline nicotine use; significant treatment × smoking interactions were followed by post hoc analyses of treatment effects in smokers versus non-smokers. The treatment main effect in this model estimates the average (across weeks) difference between treatments, while treatment × time effects lead to post hoc examination of how treatment effects changed between visits. Appropriate transformation was applied to skewed measures. If a treatment effect were found only in smoker group(s), post-hoc tests would covariate with CPD to examine potential effect secondary to smoking behavioral change. We employed restricted maximum likelihood (REML) using an unstructured covariance matrix for the correlation among observations. For measures where the unstructured covariance model did not converge, the generalized estimating equations (GEE) method was used with a compound symmetry covariance matrix. Spearman’s correlation was used in biomarker-clinical measure analyses and was limited to biomarkers that showed significant treatment effects.

Results

Prepulse inhibition and startle reactivity

There was no treatment effect (F_{(1, 41.0)}=0.65, p=0.42) or treatment by baseline smoking status interaction for PPI (Figure 2). However, there was a treatment effect for startle reactivity in which varenicline reduced startle reactivity in schizophrenia patients (F_{(1, 42.9)}=6.44, p=0.015; Figure 2). The effect was significant at week 8 (p=0.008) but not week 2 (p=0.11). The treatment by smoking status interaction was not significant. Change (baseline - week 8) of PPI and change of startle reactivity were correlated in placebo (rho=0.55, p=0.011) and varenicline (rho=0.65, p=0.001) groups. Changes in startle reactivity and BPRS total were positively correlated in the placebo (rho=0.48, p=0.032) but not the varenicline group (rho=−0.23, p=0.26). The difference between the coefficients was significant based on Fisher’s z transformation (p=0.048), suggesting that dampening of the startle reactivity by varenicline altered the relationship. Reduction in startle reactivity was also correlated with increased MCCB composite scores (r=−0.45, p=0.005); the coefficients in varenicline versus placebo were not significantly different (p=0.10).

Figure 2.

Varenicline treatment effects on PPI and P50 sensory gating. Moderate dose varenicline significantly reduced startle reactivity as measured by the acoustic startle response amplitude in schizophrenia patients compared with placebo (combined smokers and nonsmokers), significant only in week 8 (A); but did not significantly changed %PPI to the startle response (combined smokers and nonsmokers) (B). P50 gating (S2/S1 ratio) was significantly increased (reduced ratio) by moderate dose varenicline compared with placebo; the effect was significant at week 8 in the entire sample (C) and in the nonsmoker subgroup (D) but not significant in the smoker subgroup (E) although the varenicline effect followed the same pattern in both groups. Note that reduced S2/S1 ratio implies improved gating function. P50 average evoked potential amplitude was not significantly different between varenicline and placebo for response to the first click S1 in the entire sample (F) but was significantly reduced by varenicline compared with placebo for response to the second click S2 (G) (combined smokers and nonsmokers). Numbers of subjects (n) in parentheses are subjects with usable data available at baseline (same in all figures below). Numbers of subjects may vary in subsequent time points. Please refer to Figure 1. ALL: smokers and nonsmokers. NSK: nonsmokers. SK: smokers.

Sensory gating

There was a treatment by week effect (F_{(1, 56.3}=8.20, p=0.006) such that long term varenicline corrected some of the P50 gating deficit in schizophrenia patients at week 8 (t=3.07, p=0.003) but not at week 2 (p=0.67). The treatment by smoking interaction (p=0.009) indicated that the treatment effect was significant for nonsmokers (p=0.001) but not smokers (p=0.61), although the direction was consistent in both groups (Figure 2). Although an increase in S2/S1 ratio from baseline to week 8 was observable in the placebo group (Figure 2G), the change was significant in varenicline group alone (p=0.037), but not in placebo group alone (p=0.54). Exploration on P50 amplitudes showed that varenicline reduced S2 but not S1 amplitude, suggesting a gating effect not secondary to the conditioning response (Figure 2). Change of P50 gating was not correlated with change in MCCB (p=0.96) or BPRS (p=0.10).

Antisaccade and memory saccade

Compared to placebo, varenicline reduced antisaccade error rate (F_{(1, 55.3)}=4.73, p=0.034; Figure 3). There was no smoking by treatment interaction. Change score was not correlated with change scores for MCCB (p=0.34) or BPRS (p=0.51). Antisaccade was replicably correlated with MCCB (r=−0.50, p<0.001 at baseline; r=−0.49, p<0.001 at week 8); yet the changes of the two were not correlated (r=−0.14, p=0.34). No treatment or treatment-related interaction was found for memory saccade (Figure 3).

Figure 3.

Error rate (A) and positional error (B) are the primary outcome measures from the antisaccade and memory saccade tasks, respectively. Moderate dose varenicline improved antisaccade performance but not memory saccade performance compared with placebo in schizophrenia patients (combined smokers and nonsmokers). Numbers of subjects may vary in different time points. Please refer to Figure 1. ALL: smokers and nonsmokers.

Smooth pursuit eye movement

There was no treatment effect on maintenance pursuit gain (F_{(1, 55.6)}=0.04, p=0.85; Table 2) or on predictive pursuit gain (F_{(1, 55.9)}=3.82, p=0.056). The trend showed reduced performance with varenicline compared to placebo, although it was not significant (Table 2). There was no treatment by smoking interaction on either measure.

Table 2.

A summary of end point measures that showed no significant treatment effects.

		Baseline						Week 8						Statistics
		Placebo			Varenicline			Placebo			Varenicline
		ALL	NSK	SK	ALL	NSK	SK	ALL	NSK	SK	ALL	NSK	SK	F	p
Memory saccade positional error (0)	mean	1.70	1.79	1.65	1.69	1.74	1.66	1.75	1.76	1.74	1.68	1.71	1.66	0.90	0.35
	s.e.	0.11	0.17	0.15	0.11	0.12	0.16	0.12	0.20	0.16	0.10	0.13	0.14
Predictive pursuit gain	mean	0.34	0.34	0.34	0.34	0.35	0.34	0.33	0.31	0.34	0.28	0.24	0.31	3.82	0.056
	s.e.	0.02	0.03	0.03	0.03	0.05	0.04	0.03	0.04	0.04	0.02	0.04	0.03
Maintenance pursuit gain	mean	0.89	0.86	0.90	0.85	0.77	0.90	0.84	0.82	0.85	0.81	0.77	0.85	0.04	0.85
	s.e.	0.03	0.06	0.03	0.05	0.10	0.04	0.03	0.06	0.04	0.04	0.06	0.05
Connor’s CPT d’	mean	0.87	1.09	0.76	0.87	0.92	0.84	0.97	1.19	0.81	0.95	1.01	0.90	0.22	0.64
	s.e.	0.09	0.16	0.10	0.07	0.10	0.10	0.10	0.18	0.11	0.09	0.14	0.12
MCCB composite	mean	27.63	30.20	26.35	26.73	33.33	22.33	28.77	32.80	25.42	26.27	34.08	21.06	0.69	0.41
	s.e.	2.15	3.77	2.63	2.73	4.86	2.85	2.67	3.70	3.66	2.71	5.01	2.47
DSST processing speed	mean	53.18	58.82	49.53	53.84	57.31	51.47	54.32	59.09	51.24	54.23	47.08	58.74	1.69	0.20
	s.e.	3.61	4.85	4.95	3.41	5.61	4.31	4.10	7.62	4.66	3.64	5.77	4.50
LOF	mean	1.91	2.00	1.86	1.78	2.00	1.63	1.93	2.18	1.75	1.84	2.08	1.68	0.10a	0.76
	s.e.	0.14	0.13	0.20	0.11	0.16	0.14	0.13	0.12	0.19	0.11	0.14	0.15
GAF	mean	47.69	49.09	46.95	46.69	50.38	44.16	47.62	49.91	45.93	47.94	52.38	44.89	0.30	0.59
	s.e.	1.91	3.31	2.38	1.79	2.97	2.09	2.01	3.32	2.49	1.86	2.78	2.30
SANS	mean	23.06	20.73	24.29	24.47	20.77	27.00	19.85	18.60	20.63	21.40	19.38	22.94	0.17	0.68
	s.e.	2.33	3.01	3.20	2.16	3.12	2.86	2.26	2.17	3.45	2.27	3.17	3.21
HAM-D	mean	6.06	6.64	5.76	5.16	5.31	5.05	6.37	6.82	6.06	4.63	5.38	4.11	0.76	0.39
	s.e.	0.82	0.98	1.15	0.70	1.12	0.93	0.94	1.53	1.22	0.71	1.34	0.78
Cigarette per day	mean			15.00			17.53			16.81			10.68	3.33 0.042*
	s.e.			2.69			2.77			4.17			1.93
CO level (ppm)	mean			16.90			19.95			12.20			10.59	1.62	0.21
	s.e.			2.55			2.80			1.98			1.55

^aχ² value based on Mantel-Haenszel χ² test

^*Statistically significant

Cognitive performance

Processing speed as measured by DSST showed no significant effect of treatment (F_{(1, 51.5)}=1.69, p=0.20, Table 2) or treatment × smoking status interaction (F_{(1, 51.6)}=3.18, p=0.081). Treatment effects on sustained attention by Connors CPT d’ (Table 2) or hit rate and their treatment × smoking status interactions were all not significant. Treatment difference for the MCCB composite score was not significant (Table 2). Tests for variation in treatment differences among 7 MCCB domains (treatment × domain, F_{(6, 56.1)}=0.22, p=0.97), or between smokers and non-smokers, either on average across domains (treatment × smoking, F_{(1, 42.4)}=0.37, p=0.55) or by domain (treatment × smoking × domain, F_{(12, 81.4)}=0.83, p=0.62) were not significant.

Smoking outcome measures

Although enrollment was not restricted to those desiring to quit smoking, a reduction in CPD was noted in patients on varenicline compared with those on placebo (F_{(2, 64)}=3.33, p=0.042) (Figure 4). CO level was also reduced in varenicline compared with placebo although this was not significant (p=0.21) (Table 2). Changes in CO level and CPD were correlated (r=0.53, p=0.002). Two varenicline patients and one on placebo quit smoking by week 8. Change in CPD was not correlated with changes in those biomarker or clinical endpoints that showed treatment effects in the varenicline or placebo group (all p≥0.26). Dividing the varenicline group into patients who reduced CPD (n=11) vs. patients without CPD reduction (n=8) also did not find difference in endpoint measure changes (all p≥0.14), ruling out large confounds on biomarkers due to change in smoking quantity. Smokers’ CO level was not significantly correlated with any dependent measures at baseline (all r≤0.29, all p≥0.08).

Figure 4.

Varenicline effect on smoking and clinical symptoms. Cigarette per day (CPD) was significantly reduced by varenicline in schizophrenia smokers (A). Trends of improving rather than worsening total psychiatric symptoms as measured by BPRS total score (B), or psychosis as measured by BPRS psychosis subscale (C) were found (combined smokers and nonsmokers). Notably, apparent treatment differences were manifest early after the start of the treatment and remained relatively constant. Numbers of subjects may vary in different time points. Please refer to Figure 1. ALL: smokers and nonsmokers. NSK: nonsmokers. SK: smokers.

Clinical outcome and side effects

For BPRS total, there were no significant treatment or interaction effects, with a trend for reduced psychiatric symptoms by varenicline compared with placebo (F_{(1, 54.2)}=3.32, p=0.074; Figure 4). Cases of exacerbation of psychosis by varenicline has been reported⁶⁵; however, BPRS psychosis subscale showed a trend toward reduced psychosis with varenicline compared to placebo (F_{(1, 58)}=3.89, p=0.053). There were no differences in treatment effects in smokers versus nonsmokers (all p≥0.30). We found no significant effect of treatment on negative symptoms on the SANS, functions assessed by LOF or GAF, or depression on the HAM-D (Table 2). Assessments on depression, anxiety, and suicidality were further probed from other rating sources given the prominent safety concerns associated with varenicline on these areas. HAM-D item 3, suicidality, showed no treatment effect (p=0.73) and only 1 patient (on placebo) had a score>0 at week 8. There was also no treatment effect on BPRS item 13 (depression) (p=0.19; numerically higher depression rating in placebo but lower rating in varenicline from baseline to week 8). There was no treatment effect on BPRS anxiety rating (p=0.37, both groups reduced anxiety). Therefore, there was no evidence that slowly titrated varenicline at 1mg/day increased these psychiatric symptoms.

Other side effects at weeks 2 and 8 were compared to those at baseline to determine ratings that were newly present or more severe than at baseline (Table 3). Abnormal dreams (p=0.032) were reduced in varenicline relative to placebo. Non-significantly increased vomiting (15.6% versus 3.1%, p=0.20), dry mouth (34.4% versus 18.8%, p=0.26) and appetite (31.3% versus 18.8%, p=0.39) were associated with varenicline.

Table 3.

Varenicline Side Effect Checklist.

	Varenicline N=32		Placebo N=32		P-value (Fisher’s exact test)
	n	%	n	%
Nausea	10	31.3	10	31.3	1
Abdominal pain	9	28.1	11	34.4	0.79
Flatulence	9	28.1	11	34.4	0.79
Dyspepsia	3	9.4	5	15.6	0.71
Vomiting	5	15.6	1	3.1	0.20
Constipation	5	15.6	8	25	0.54
Dry mouth	11	34.4	6	18.8	0.26
Insomnia	9	28.1	11	34.4	0.79
Abnormal dream	3	9.4	11	34.4	0.032
Nightmare	6	18.8	6	18.8	1
Headache	3	9.4	10	31.3	0.06
Dizziness	8	25	8	25	1
Somnolence	14	43.8	10	31.3	0.44
Fatigue	13	40.6	10	31.3	0.60
Anxious	7	21.9	6	18.8	1
Euphoria	4	12.5	10	31.3	0.13
Increased appetite	10	31.3	6	18.8	0.39
Decreased appetite	6	18.8	6	18.8	1

Discussion

We found that 8 weeks moderate dose varenicline 1) reduced sensory gating deficit in schizophrenia patients; 2) reduced startle reactivity but did not change PPI; and 3) improved executive function measured by antisaccade error rate. There were no significant effects on spatial working memory, predictive and maintenance pursuit, processing speed, sustained attention, or MCCB. There was no evidence of exacerbation of psychiatric symptoms in schizophrenia patients in this gradual titration, moderate dose strategy; instead a trend to decrease psychosis was observed. The use of moderate dose varenicline was designed to retain varenicline’s pharmacological α4β2 actions while simultaneously minimizing effects on other nAChR subtypes (based on preclinical data) and potential side effects (based on Phase II clinical data) for schizophrenia patients.

Human data for nicotinic effects on PPI and sensory gating have been largely based on brief challenge studies. In this trial, varenicline effects on sensory gating and startle reactivity were significant only at week 8. Nicotine is a full agonist while varenicline is a 30-60% partial agonist (of the nicotinic effect on dopamine turnover) and also a partial antagonist of α4β2²⁶. This profile could modulate α4β2 and the downstream pathways through a more gradual time course different from a full agonist. One study reported no effect of single dose varenicline on P50 gating in six patients⁶⁶; another 2-week study also reported no effect on P50 gating in smokers⁶⁷. Short-term treatment designs may be appropriate for a full agonist mechanism but could miss important effects exerted by the unique partial agonist/antagonist modulation, as shown here.

Varenicline did not mimic acute nicotine effects on PPI⁴. Rollema et al reported a weak enhancement of PPI and startle reactivity in rodents under one dose, but not in other doses of varenicline and additional studies failed to show the effect⁶⁸. In humans, nicotine enhanced PPI^69-72 but opposite effects have also been observed⁷³. For startle reactivity, nicotine either does not change⁷³ or increases it^74;75. Startle reactivity is enhanced by activation of dopamine receptors^76;77; while dopamine antagonists and antipsychotic medications dampen it^68;78-81. Because long-term varenicline mimics the startle reduction aspect of antipsychotics, it may indicate a gradual down-regulation of dopaminergic function by the α4β2 antagonist aspect of varenicline. The lack of significant findings on spatial working memory may also support an antagonistic mechanism by varenicline because previous studies have shown that antagonism to high affinity nAChR receptors blocks the improvement of spatial working memory by nicotine ¹⁵. However, this may not explain the lack of effect on PPI because antipsychotics reverse PPI deficits induced by dopamine agonists¹. Varenicline 2-week treatment increases striatal D2/3 receptor availability by 11-15%⁸². Our findings encourage additional chronic exposure studies to determine its time-course on dopaminergic modulation.

Varenicline also did not mimic nicotine effects on improving maintenance pursuit and sustained attention⁷ but showed a similar effect of reducing antisaccade error^7;8;83. These findings imply that error reduction maybe more specifically influenced by α4β2, while improvement in other measures may be associated with other nAChRs. Antisaccade assesses the executive ability to inhibit distraction and attend to the instructed target. Antisaccade error in schizophrenia is thought to reflect an impaired frontostriatal pathway⁸⁴ and its striatal GABAergic dysfunction⁸⁵. Speculatively, GABAergic postsynaptic currents can be activated by the α4β2 nAChR present in presynaptic terminals of interneurons⁸⁶, a possible route by which α4β2 nAChR treatment could affect GABAergic inhibitory and thus antisaccade function.

Some aspects of schizophrenia pathology appear related to nicotinic receptor abnormalities irrespective of smoking⁴¹. Therefore, we did not expect that varenicline affects smokers but not nonsmokers or vice versa, as found in startle reactivity and antisaccade. The significant reduction in P50 gating deficit in nonsmokers but not in smokers suggests that this was not due to smoking per se. Changes in P50 gating and in CPD (r=−0.20. p=0.41) or CO (r=−0.13, p=0.63) in smokers on varenicline were not correlated. The better P50 gating in the smokers on placebo (Figure 2E) was a chance bias from randomization that could reduce the power in the smoker group.

Varenicline improved sustained attention and working memory under 3 days of mandatory abstinence in non-psychiatric subjects⁸⁷, possibly by reversing dysfunctions associated with abstinence-induced withdrawal. Under the current non-abstinence condition, 1mg varenicline did not improve sustained attention, spatial working memory and predictive pursuit (a task related to oculomotor working memory⁵²), although a higher dose could be tried. Nicotine may improve working memory in the spatial domain²⁵ although in several studies it did not improve working memory in schizophrenia^{7;12;16;17;23} and may even worsen working memory⁸⁸. The nicotinic effect on maintenance pursuit^7;12 was also not replicated by varenicline. Thus, unlike several positive findings reported by an open-labeled study⁸⁹, our study suggests that the α4β2 partial agonist/antagonist effect on cognition is modest. In fact, varenicline at the current dosage does not replicate many acute full agonist effects of nicotine. Instead, it reduces selected biomarker deficits, particularly P50 gating and antisaccade deficits. Varenicline’s long-term but not short-term effect on specific biomarkers is a novel finding, and differs from acute nicotine. It’s intriguing to see whether biomarkers that are responsive to nicotine but not varenicline would be responsive to compounds targeting non-α4β2 nAChR subtypes. This also illustrates the advantage of comparing key biomarkers in the same trial to identify plausible specific receptor - clinical biomarker relationships.

Safety, especially symptom exacerbation in mentally ill populations, has been raised in case reports and FDA box warnings. Randomized controlled trial data in non-psychiatric populations showed no major psychiatric symptom exacerbations⁹⁰ or even improvement in mood^87;91. We observed no excessive somatic or psychiatric symptoms under the current dosing, consistent with the prediction based on phase 2 data where 1mg dose reduces side effects but maintains a reasonable efficacy⁴⁶.

The study examined twenty-one biomarker, clinical and smoking related measures instead of a single primary endpoint, which raises the possibility of false positives. Our goals were to compare biomarkers previously responded to nicotine and test them simultaneously for a chance of head-to-head comparisons for α4β2 effect. None of the findings would survive corrections for the multiple comparisons, although previous nicotine effect on individual biomarker was typically tested one biomarker at a time. Nevertheless, replication studies are needed. The small N of nonsmokers on placebo at week 8 could have also reduced the power and led to false negative. All patients were on antipsychotic drugs. Because biomarker endpoints were compared to their baseline, the findings are less likely due to antipsychotic treatment although we can’t rule out potential varenicline × antipsychotics interactions.

We opted for a biomarker-based trial assuming that biomarkers should be associated with more specific biological pathways and more informative for translational follow-up studies. The study reveals a long-term effect on specific biomarkers. A longer-term treatment and/or a dose ranging design could reveal further improvements, because core neurophysiological impairments in schizophrenia are chronic and entrenched. Because biomarker improvements were seen at week 8, we could not examine whether improvement in week 2 would predict clinical improvement in week 8. However, in a still longer trial, one could examine whether improvement seen in week 8 would predict clinical outcome later. The findings suggest that previously described nicotinic effects on P50 gating and antisaccade are likely in part related to specific α4β2 nAChR modulation. Agents with specific α4β2 actions could potentially be used to target these specific biomarker deficits. Most individual patients do not have all of the cognitive or biomarker deficits that are statistically associated with schizophrenia. A given biomarker deficit is typically present in 30-50% patients, depending on cutoff criteria, and many biomarkers are not correlated⁴⁹. A novel agent targeting specific receptor subtype might correct specific biomarker(s) rather than expecting it to correct all of the heterogeneous symptom and neurobiological deficits covered under the diagnosis of schizophrenia.

In summary, we observed no evidence that moderate dose, 8-week varenicline is unsafe in stable, medicated schizophrenia patients. There is evidence of a long-term neurobiological improvement on sensory gating and antisaccade functions, and a nonsignificant reduction in psychotic symptoms, suggesting a unique efficacy profile of the presumed partial agonist/antagonist α4β2 nAChR modulation. These findings encourage further development of α4β2 nAChR modulating compounds and optimizing dosing and treatment duration that are safe and effective for treating specific neurobiological deficits, a critical unmet treatment need in schizophrenia.