Effects of Varenicline, Nicotine or Placebo on Depressive Symptoms in Postmenopausal Smokers

Abstract

Background

Varenicline carries a black box warning for neuropsychiatric adverse events.

Objective

We examined varenicline use and past history of major depressive disorder (MDD) on depressive symptoms during smoking cessation.

Method

This is a secondary analysis of two smoking cessation studies in 152 postmenopausal women who received placebo or nicotine patch, or 78 women who received varenicline with relaxation. Lifetime history of MDD (LH-MDD) was assessed at baseline and women with current MDD were excluded. Center for Epidemiologic Study Depression scale (CESD) measured depressive symptoms at baseline, 6 and 12 weeks.

Results

Baseline CESD scores were 5.3 + 4.4. Those with a LH-MDD reported higher CESD scores (p >.001). Those taking varenicline reported lower scores over all time periods compared to nicotine or placebo (p <.01). The differences between varenicline and the other treatments remained when controlling for LH-MDD, indicating an independent effect. CESD scores were associated with concurrent smoking status (p <.001), and with withdrawal symptoms (p <.001).

Conclusion

CESD score were lower in those receiving varenicline, whether this is due to an anti-depressant effect, subject selection, use of relaxation or another cause is unknown. Varenicline does not increase depressive symptoms during smoking cessation in postmenopausal women without current MDD. Subjects with a LH-MDD are susceptible to developing depressive symptoms during smoking cessation, regardless of pharmacologic aid.

Scientific Significance

Pharmacologic aids did not increase depression symptoms in this select population of postmenopausal women without current depression. Smoking cessation does increase depressive symptoms in those with LH-MDD, though the degree of increase was not clinically meaningful.

INTRODUCTION

Smoking is a major risk factor for the leading causes of morbidity and mortality in women in the United States including cancer, cardiovascular and respiratory diseases, and osteoporosis.^1,2 An estimated 16.5% of women age 45–64, and 7.9% of adults 65 or older were current smokers in 2011. Difficulty with smoking cessation in middle age and older women may be secondary to higher rates of depression, in addition to barriers to quitting such as fear of weight gain.³ In a previous study, postmenopausal smokers with a history of depression were less likely to quit smoking.⁴ Although there are many available medications and approaches to help smokers quit, there are adverse effects associated with medications. It is important to balance the risk of medication adverse effects with the health benefits of smoking cessation, and establish the extent to which adverse events occur in select populations or may negatively impact successful cessation.

Varenicline is one of the newest pharmaceutical adjuncts to smoking cessation treatment plans. The US Food and Drug Administration required that varenicline carry a black box warning⁵ for neuropsychiatric adverse events (NPAEs). Although the efficacy of varenicline as a cessation aide has been demonstrated in multiple studies,^6,7 the drug is associated with NPAEs, ranging from changes in behavior and agitation to suicidal thoughts and attempted suicide.⁵ In the official drug warning, the FDA indicates that some of the reported cases of NPAEs may have been confounded by symptoms typically seen with nicotine withdrawal. There is also the concern that medical history of psychiatric disorders may have confounded NPAE reporting. However, many of these reports were from patients who had continued to smoke with treatment and from patients without history of psychiatric illness suggesting the NPAEs are not completely attributed to withdrawal symptoms or history of psychiatric disorder. In a study by Garza et al.,⁸ the authors assessed prevalence of NPAEs with use of varenicline while controlling for nicotine withdrawal and past psychiatric history. They found no significant differences in NPAEs, specifically irritability, depressed mood, anxiety, aggression, and hostility, between the varenicline and control groups, supporting the idea that it is not varenicline on its own that causes these NPAEs.⁸ Further, a cohort from a nested General Practice Database receiving varenicline for smoking cessation did not report more depression or suicidal ideation.⁹

The goal of the present study was to evaluate symptoms of depression in postmenopausal women treated with varenicline, nicotine replacement or a placebo. Data were taken from two studies comparing treatments for smoking cessation.⁴

We hypothesized that varenicline use would lead to increased report of depressive symptoms over time. We also predicted that participants with a history of depression would have higher rates of depressive symptoms. Additionally, we hypothesized that stopping smoking, as an added stressor in the lives of our participants, would result in increased report of depressive symptoms. And, finally, we theorized that there would be a positive correlation between withdrawal symptoms and degree of depressive symptoms.

METHODS

Data for this study were drawn from two clinical trials of smoking cessation for postmenopausal smokers. Both trials had approval of the Institutional Review Board of the University of Connecticut Health Center and participants gave informed consent prior to study procedures. The nicotine replacement trial (NRT) included two treatment conditions: nicotine replacement or placebo.⁴ The Relaxation and Exercise for Smoking cessation Trial (REST) comprised two treatment conditions, both of which included prescription of varenicline, plus either exercise or relaxation as an adjunct. This trial is ongoing. Because of the possible impact of exercise on depressive symptoms, only those women who received varenicline plus relaxation are included in the present set of analyses.

Study Participants

Several key characteristics were shared between the studies (Table 1). Both studies included postmenopausal women who smoked a minimum of 10 cigarettes per day. Common exclusion criteria were the use of more two drinks per day; use of use of nicotine or other smoking cessation products (including bupropion); unstable medical or psychiatric disorder, specifically Axis I psychiatric diagnosis requiring treatment; unstable angina or recent myocardial infarction.

TABLE 1.

Baseline characteristics by treatment sample

Characteristic	All patients (N = 230)		Placebo (n = 95)		Nicotine Replacement (n = 57)		Varenicline (n = 78)		F or χ2 (df)
	M or %	SD	M or %	SD	M or %	SD	M or %	SD
Age (years)	55.6	6.7	56.6	6.9	53.9	6.9	55.2	6.8	2.94 (2,227)
Race									6.69 (6)
White %	87.3		87.4		91.2		82.1
Black %	6.3		7.4		7.0		9.0
Hispanic %	4.5		1.1		1.8		6.4
Other %	1.9		4.2		0.0		2.6
Education									4.24 (4)
<High school %	4.8		4.3		7.1		3.8
HS grad %	27.2		27.4		14.0		21.8
Post HS %	68.0		68.3		78.9		74.4
Marital									11.94* (4)
Married/cohabiting %	47.8		53.7		47.4		41.0
Divorced/widowed %	43.9		43.2		47.4		42.3
Single %	8.3		3.2		5.3		16.7
Income									3.08 (4)
Less than $20K/year	9.8		11.1		8.8		9.0
$20K to $60 K/year	48.0		47.8		56.1		42.3
More than $60 K/year	42.2		41.1		35.1		48.7
Cigarettes per day	18.3	9.9	21.4	8.4	21.6	8.0	12.0	9.8	29.08*** (2,227)
CO levels (ppm)	19.8	10.9	18.9	9.9	21.0	11.5	20.1	11.6	0.70 (2, 227)
Minn withdrawal score	5.8	5.1	5.0	5.0	5.5	5.4	6.8	4.8	2.68 (2,227)
CESD score (baseline)a	5.2	4.4	5.2	4.6	6.1	4.2	4.5	4.2	2.09 (2, 227)
Lifetime depression									14.34** (4)
Absent %	67.4		57.9		61.4		83.3
Sub-threshold %	3.0		3.2		3.5		2.6
Positive %	29.6		38.9		35.1		14.1

^aCESD raw scores shown. F-test conducted on log-transformed scores;

^*p <.05;

^**p <.01;

^***p <.001.

In the NRT study, a total of 152 women were randomized; 57 women were assigned to the nicotine patch treatment group and 95 women were assigned to the placebo group. Further exclusion criteria, to address the primary aim (osteoporosis) of the study, included disease or medications that affect bone metabolism, treatment for osteoporosis or elevated ionized calcium. Thirty-four percent of the nicotine patch participants and 33% of placebo participants were taking antidepressants.⁴

In the REST study, a total of 78 women were included in the analyses presented here. Other inclusion criteria for the entire study population were: demonstrated motivation to quit smoking and begin exercise, being ambulatory, exercising 90 minutes or less per week. Additional exclusion criteria included: systolic blood pressure greater than 165 or diastolic blood pressure greater than 100; hip fracture within the preceding 6 months; exercise-exacerbated neuromuscular disorder; treatment for depression within the last year; estimated creatinine clearance of less than 30 cc/min.

Within the NRT arm of the study, 182 women were screened, 152 were eligible and randomly assigned to receive treatment. Within the REST arm, 123 women were screened, 116 were eligible and randomized to receive treatment. As indicated above, only those women who received the relaxation intervention, along with varenicline, were included here. Of the women completing the first 12 weeks of each treatment arm, 86.0% (49/57) for nicotine patch, 84.2% (80/95) for placebo patch, and 83.1% (74/89) for varenicline were taking study medications. Within the varenicline relaxation group, medication treatment was discontinued by 24% of participants for adverse events that included symptoms of depressed mood (8); anger (2); nausea (2); palpitations (1); sadness (1); irritability (2); sleeplessness (1); and abnormal dreams.¹ In the nicotine/placebo study, adverse events that caused medication discontinuation included cardiovascular symptoms,³ skin-related conditions,⁵ psychiatric symptoms,³ or other reasons (surgery or bruising).² Subjects who stopped medication were included in the analyses if they completed the 12 week visit.

Measures and Instruments

Demographic information, along with nicotine dependence, smoking behavior, exhaled carbon monoxide (CO), and mood were assessed at each visit and comparisons were made between the treatment arms.

Recent and lifetime history of psychiatric disorder in both studies was assessed using the Structured Clinical Interview for DSM-IV.¹⁰ The SCID is a structured interview that is administered by trained interviewers. It consists of open-ended as well as closed-ended questions regarding the subject’s psychiatric symptoms, and is intended to yield DSM-IV diagnoses of psychiatric disorders.

Depressive symptoms were measured using a 10-item version of the Center for Epidemiological Studies Depression Scale (CESD), used to measure depressive symptoms in the general population.¹¹ A Likert scale is used for each item, ranging from 0 (rarely or none of the time) to 3 (most of the time). In the NRT study, the original 20-item CESD was used. However, the scale was converted to 10 items to match the short form of the CESD scale used in the REST study.

The Minnesota Nicotine Withdrawal Scale (MNWS)¹² was used to assess symptoms associated with nicotine withdrawal. This questionnaire contains questions on eight items: depressed mood, insomnia, irritability/frustration, anxiety, difficulty concentrating, restlessness, increased appetite, and craving for cigarettes. Each withdrawal symptom was rated for its severity on a scale of 0 (none) to 4 (severe).

Smoking cessation was assessed at each visit by asking the subject “have you had any cigarettes in the last 7 days.” The participant’s answer was verified by the exhaled CO level of ≤8 ppm.¹³ In order to measure the CO level, participants were asked to take a deep breath and then exhale into a CO monitor. If there was a discrepancy between the subject’s answer and the CO level, the subject was coded as a smoker for that particular visit. The subject was also coded as a smoker for any missed visits.

Procedures

The NRT study consisted of a 12-week treatment phase in which participants were randomly assigned to use either a 21 mg nicotine or placebo patch for 3 months. Basic demographic information was collected, and both the SCID and the CESD were administered at baseline and at 6 and 12 weeks. Participants completed the MNWS at baseline and 12 weeks. All women received therapeutic sessions with motivational and cognitive behavioral techniques. Participants were monitored frequently for signs of adverse effects of the medication.

In the ongoing REST study, subjects participated in behavioral counseling in conjunction with varenicline treatment for 12 weeks, while simultaneously participating in either a supervised exercise program or a supervised relaxation program to which they had been randomly assigned. During the initial visit, participants underwent depression screening by trained interviewers using the SCID to assess current and lifetime depression and completed the self-administered CESD at baseline, 6 and12 weeks (for this report), and received phone calls from nursing staff that included CESD at 2, 4, 8, and 10 weeks. Study investigators reviewed cases in which the patient reported a CESD score ≥12 (on the 20-item survey) and complained of depressed mood. After review of the patient’s symptoms, other life events, and medication use, varenicline was stopped if clinically warranted. Completion of the SCID at baseline classified participants as having either absent, sub-threshold, or threshold depression.

Data Analysis

All continuous variables were checked for normality. Correlations were run to investigate any associations among predictors and outcomes. Baseline characteristics between treatment groups were compared using analysis of variance (ANOVA) or chi-square tests. The effects of the interventions (varenicline, nicotine or placebo) on CESD scores over time were tested using a mixed model regression approach. Fixed effects included treatment group, time (baseline, 6 weeks, and 12 weeks), and the treatment × time interaction. A second model excluded those placebo and nicotine replacement patients who were taking antidepressants at the time of the study. This model also included lifetime history of depression as a covariate, along with time, treatment, and treatment × time. This model was run to control for presence of depressed patients in the placebo and nicotine replacement conditions.

A third model was run on the complete samples to evaluate the influence of withdrawal on CESD score, using MNWS at baseline and 12 weeks as time-varying covariates. Finally, smoking status and smoking intensity were used as a time-varying covariate in a fourth and fifth models to test the hypotheses that smoking severity was related to CES-D scores and that stopping smoking per se could result in increases in depressive symptoms.

RESULTS

Demographic characteristics were similar across treatment groups with the exception of marital status. The varenicline treatment group tended to have more single women than the other groups. (see Table 1). There were no significant differences in socioeconomic status among study participants. Subjects were generally smoking a pack per day at baseline except in the varenicline group, in which patients smoked closer to 12 cigarettes per day. The treatment groups did not differ significantly on the smoking related measure, CO level. Nor did they differ on withdrawal score. Baseline scores on the CESD tended to be low in all treatments, and did not differ significantly by treatment group. Based on the SCID interviews, 29.6% of participants met lifetime criteria for depression, with somewhat lower rates of lifetime depression in the varenicline group.

CESD Comparisons by Treatment

Table 2 shows the results of all mixed model analyses conducted for the present study. Results for Model 1 showed no main effect for time, or for the interaction of time × treatment. A significant main effect for treatment did emerge however. Univariate posttests indicated that the varenicline group had significantly lower CESD scores over all time points than both the placebo group (i − j difference = −2.22; se=.53; df=462.199; p <.001; 95% CI: −.95 to −3.50), and the nicotine replacement group (i − j difference = −2.62; se=.59; df =474.430; p <.001; 95% CI: 1.21–4.03).

TABLE 2.

Results of mixed model regression analyses

Model	Term entered	dfa	F
1	Treatment	2/480.138	11.92***
	Time	2/392.051	1.49
	Treatment × time	4/421.200	1.49
2	Treatment	2/366.931	5.55*
	Time	2/340.920	1.81
	Treatment × time	4/371.313	1.32
	SCID history of depression (Yes–No)	2/424.623	13.449***
3	Treatment	2/365.912	4.79**
	Time	2/324.342	3.52*
	Treatment × time	4/310.884	1.06
	Smoking Status at time point (Smoking–Non-Smoking)	1/452.391	7.56**
4	Treatment	2/372.103	4.18*
	Time	2/328.427	5.71**
	Treatment × time	4/309.813	1.21
	Cigarettes per day at time point	1/456.204	14.94***
5	Treatment	2/330.277	15.71***
	Time	1/331.978	0.18
	Treatment × time	2/329.502	2.19
	Withdrawal score at time point	1/399.903	206.93***

DV, CES-D depressive symptoms score.

^aDegrees of freedom calculated using Satterthwaite’s approximation;

^*p <.05;

^**p <.01;

^***p <.001.

CESD and History of Depression

Women with a current diagnosis of depression were excluded from both of the trials that provided data for this study. In addition, however, those taking antidepressants were excluded from the varenicline study regardless of diagnosis while those in the nicotine or placebo conditions were not. As a result of this differential exclusion procedure those with a history of depression may have been underrepresented in the varenicline sample. An analysis was therefore conducted to determine whether the differences in depressive symptoms found between treatment groups could be attributable to past lifetime depression or current use of anti-depressants. Model 2 in Table 2 shows the results of a mixed model analysis in which placebo and nicotine replacement patients were removed if they were taking any antidepressant medication at baseline. Lifetime history of depression was also entered as a covariate, along with treatment and time. As seen in the table, positive history of depression was a significant predictor of later CESD scores. Interestingly, the treatment main effect remained significant even with the inclusion of depression history in the model and with those taking antidepressants removed, suggesting that the treatment effect was independent of depression history. The results of this analysis are shown in Fig. 1. The scores shown in the figure were estimated from the linear mixed model, and then de-transformed to place them back on the original CESD scale.

FIGURE 1.

Estimated CESD scores (detransformed) by treatment sample. Results of linear mixed model analysis.

Effect of Smoking Cessation on CESD

Another possibility that was considered was that smoking cessation might in itself result in elevated depressive symptoms, thus resulting in higher CESD scores in those treatments that yielded more smoking abstinence. This possibility was tested in Model 3, using the complete samples. Smoking status at each time point was treated as a time-varying covariate. As seen in the Table, smoking status contributed to CESD scores at follow-up time points. The data revealed that participants who continued smoking following the quit date had marginally elevated CESD scores. Successful non-smokers at 6 weeks had a mean CESD of 5.61 ± 6.06 versus 7.18 ± 7.00 for smokers and 5.06 ± 5.83 at 12 weeks for non-smokers versus 6.63 ± 5.77 for smokers. Thus, we can conclude that the act of smoking cessation does not increase vulnerability to depressive symptoms, though failure to achieve cessation may. This finding did not fully account for the effect of treatment on CESD scores, however.

CESD and Smoking Severity

Another difference between treatment groups that might have accounted for CESD differences was severity of smoking, assessed by cigarettes per day at each time point. In Model 4, cigarettes per day were entered as a time-varying covariate. Results indicated that a cigarette per day was indeed predictive of CESD scores at each time point. Even with smoking severity controlled, however, the main effect for treatment persisted, such that those in the varenicline treatment reported lower CESD scores. Average CESD scores over all time points with smoking severity controlled were: Placebo mean = 6.20, SD = 3.32; Nicotine Replacement mean = 6.92, SD = 4.19; Varenicline mean = 4.72, SD = 6.41.

CESD and Withdrawal Symptoms

We assessed whether the increase in depressive symptoms seen in the placebo and nicotine groups may be attributable to withdrawal. In Model 5, MNWS scores were entered as a time-varying covariate. Results indicated that withdrawal symptoms were associated with CESD scores (p <.001). However, given that one of the questions in the MNWS assesses negative mood, it was considered that the MNWS and the CESD might be measuring similar constructs. The correlation between the CESD and the MNWS was r = .56. Therefore, we are unable to definitively conclude whether the association is an effect of negative mood or actual withdrawal. We see also in Table 2, however, that the addition of withdrawal symptoms to the model still did not explain the treatment effect, which was robust even with the inclusion of withdrawal symptoms.

Finally, there were 19 reports of symptoms of anxiety or depression in the 78 women in the varenicline study (24.3%), and five patients discontinued varenicline after reporting elevated CED scores (6.4%)per study protocol. Reports of depressed mood or sadness were also described in the nicotine/placebo study: 11% in the nicotine group and 4% in the placebo group. We evaluated baseline CESD symptoms in the 19 individuals that developed depressive/anxious symptoms and did not find them different from those of the sample overall (data not shown). We also did not find higher CESD scores in 21 individuals who left the study.

Compliance rates with nicotine and placebo patches are reported in the primary paper.⁴ In brief, patch counts demonstrated that the nicotine group wore the patch 63.3% (SD = 34%) compared to 50.4% (SD= 36%) in the placebo group. In the REST study, compliance with study medication was computed by summing the number of pills consumed over 12 week treatment period divided by the pills expected to be consumed. If the subject withdrew prior to 12 weeks, the percent was prorated for time. Compliance was 79% (SD = 24%).

Three serious adverse events were reported while on nicotine or placebo patch. Two subjects (1 nictone and 1 placebo) were hospitalized for chest pain and one woman (placebo) was hospitalized for right upper quadrant pain. There were three serious adverse events in the varenicline relaxation arm during treatment; one hospitalization for concussion, one for surgical repair of a torn glenoid labrum and one for abdominal pain evaluation.

DISCUSSION

Secondary analysis of two studies of smoking cessation in postmenopausal women assessing the effect of varenicline use on the development of depressive symptoms revealed that participants using varenicline versus nicotine patch or placebo had lower rates of depressive symptoms than did participants in the other two treatment groups.⁴ Participants with a history of depression were found to be more susceptible to developing depressive symptoms. Additionally, successful smoking cessation was found to decrease rates of depressive symptoms. Finally, development of depressive symptoms was found to positively correlate with severity of withdrawal symptoms.

The finding that depressive symptoms were less pronounced in the varenicline treatment group is consistent with a body of literature demonstrating the safety of varenicline for smoking cessation, and that varenicline might even have a slight anti-depressant effect. In a double blind, randomized, placebo-controlled study, Garza et al.⁸ demonstrated that there were no significant differences in rates of depressive symptoms, anxiety, or aggression/hostility in varenicline versus placebo groups. Another group, Rollema et al.,¹⁴ demonstrated that the partial agonist activity of varenicline at the α4β2 nicotinic acetylcholine receptor has a significant antidepressant effect in mice. The reason underlying the decreased depressive symptoms seen in the varenicline group remains unclear. It is possible that varenicline did in fact have an antidepressant effect, although this has not formally been demonstrated in human subjects. It is also possible that relaxation may have contributed in a positive way to affect.^15,16

Contrary to our original hypothesis, this analysis revealed that successful smoking cessation did not increase depressive symptoms. We had hypothesized that the act of quitting smoking is a stressful event and may result in increased incidence of depressive symptoms. However, Parrott¹⁷ showed in a 1995 paper that overall, smoking cessation leads to reduced stress. Parrott theorizes that nicotine deprivation between each cigarette causes acute episodes of stress. Therefore, quitting ultimately decreases overall stress levels. Another possible explanation for this finding is that participants in the studies included in this secondary analysis were self-enrolled. These women were motivated to quit smoking. Thus, an inability to achieve successful cessation may have resulted in increased depressive symptoms in those who were unsuccessful.

An additional finding with this analysis was the positive correlation seen between CESD scores and MNWS scores. Statistically, it would appear that depressive symptoms are attributable to nicotine withdrawal. However, it must be noted that because of the negative mood component of the MNWS, we cannot definitively come to the conclusion that depressive symptoms are secondary to withdrawal.

Although differences in CESD scores by treatment were statistically significant, these differences may be clinically irrelevant. Even with the slightly higher CESD scores seen in the nicotine group, as well as in those participants with a history of depression, none of these participants reported CESD levels indicative of clinical depression. Participants generally scored well below the screening threshold for further evaluation for depression, which is usually defined as a score of 10 or more.

It is noteworthy that several women in the varenicline group discontinued study medication due to symptoms of depressed mood; however, 11% of subjects in the nicotine or placebo groups also reported depressed mood. The increased rate of study medication discontinuation due to depressed mood in the REST study is difficult to interpret, given the frequent CESD monitoring and a lower threshold for stopping study medication required in the study design. Regardless of discontinuation rates, compliance with medication was similar across treatment conditions. A study with direct comparison between nicotine and varenicline will need to address whether discontinuation rates due to depression symptoms would differ.

This report has several strengths. Each study had strict, well-defined inclusion and exclusion criteria. Also, the combined sample size was fairly large. A limitation of the analysis was the exclusion of participants with active depression, limiting the ability to generalize results to those with current depressive symptoms. Also, given the correlation between the CESD and MNWS, the impact of nicotine withdrawal on CESD scores could not be fully examined. There is also the concern of possible selection bias with regard to the statistically significant finding of decreased incidence of depressive symptoms in the varenicline treatment group. Although active depression was an exclusion criterion for both studies included in these analyses, participants were enrolled in the NRT study even if they were actively using bupropion or other anti-depressants for depression. This was not the case for participants in the REST study. Therefore, this raises the concern that our finding of increased depressive symptoms in the placebo and nicotine groups may partly be a product of the higher incidence of baseline depression in these groups or recruiting subjects with greater vulnerability. Finally, the possible contribution of relaxation to the effect of varenicline cannot be excluded.

In summary, this secondary analysis supports the safety of varenicline use for smoking cessation in patients without a history of MDD. It also supports the previously identified finding that patients with a history of depression are more susceptible to depressive symptoms during smoking cessation.