Abstract
Rationale
We have shown that differences in the level of neural activation to stimuli associated with smoking vs. natural rewards, a biomarker related to reward sensitivity, predict treatment outcome.
Objectives
This paper examined whether this biomarker moderates the impact of bupropion or varenicline on smoking cessation.
Methods
Prior to treatment randomization, smokers (N = 180) in a placebo-controlled trial using bupropion and varenicline completed event-related potential recording (late positive potential, LPP) while viewing pleasant (P), cigarette (C)-related, and other pictures. We used Bayesian models to estimate the probability of interaction between treatment and the LPP for both efficacy and comparative effectiveness analyses.
Results
Efficacy analysis showed that smokers with more neural activation to pleasant vs. cigarette-related stimuli (P > C) had a 98–99% chance of achieving greater abstinence than placebo (OR >1.00), using either medication from the end of treatment (EOT, primary outcome) through the 3-month follow-up. Relative to placebo, smokers with higher activation to cigarette-related vs. pleasant stimuli (C > P) had a 99% chance of increased benefit from varenicline at both time points (OR >1), but only 67 and 43% with bupropion at the EOT and 3-month follow-up, respectively. Comparative effectiveness analysis found that smokers with the C > P activation pattern had a 95–98% chance of benefit from varenicline vs. bupropion, while P > C smokers had a 50–58% chance of similar improvement with varenicline at the EOT and 3 months.
Conclusions
Varenicline appears to be the treatment of choice for smokers with the C > P pattern of neural activation, while for those showing P > C, varenicline and bupropion have similar efficacy.
Keywords: Smoking cessation, Varenicline, Bupropion, Bayesian statistics, Comparative effectiveness
Introduction
Recent work in our laboratory has focused on the relationship between individual differences in brain activation to motivationally relevant stimuli and the ability to quit smoking (Versace et al. 2012, 2014). This work is based on the supposition that drug-dependent individuals demonstrate increased sensitivity to drug-related cues and reduced sensitivity to natural rewards (Koob and Volkow 2010; Volkow et al. 2010). Using the late positive potential (LPP), an event-related potential (ERP) measure of the motivational relevance attributed to emotional stimuli (Hajcak et al. 2010; Lang and Bradley 2009), we have shown that differences in the level of neural activation to stimuli associated with smoking-related vs. intrinsically pleasant stimuli predict treatment outcome (Versace et al. 2012). In our original report (Versace et al. 2012), smokers showing enhanced brain responses to cigarette (C)-related cues and blunted brain responses to intrinsically pleasant (P) stimuli (C > P; cluster 2 in our original report) at baseline are less likely to achieve smoking abstinence at the 6-month follow-up than smokers with the opposite pattern of brain reactivity (P > C; cluster 1 in our original report; see Figure S1 in the Online Resource for a plot of the original cluster solution). We subsequently reproduced these cluster findings using fMRI data obtained in an independent sample using the same stimuli from the original report. We again found that the C > P group achieved lower levels of abstinence than the P > C group, while also demonstrating lower levels of activation in the dorsal striatum and the medial and dorsolateral prefrontal cortex, areas important for processing reward and controlling executive function, respectively (Versace et al. 2014).
Previous research has found that both primary (e.g., fruit juice delivery, erotic picture presentation) and secondary (e.g., monetary gain) rewards activate overlapping brain areas (Sescousse et al. 2013), while cigarette-related cues also activate the same areas as pleasant stimuli (Versace et al. 2011a). Individual variation in the relative magnitude of activity produced in these reward-sensitive areas by different categories of stimuli (i.e., cigarette vs. intrinsically pleasant) could reflect differences in the underlying reward value (sensitivity) an individual places on stimuli in that category. We have speculated that smokers whose LPP activity is greater to cigarette vs. intrinsically pleasant stimuli (C > P) may have lower hedonic capacity (i.e., the ability to enjoy pleasurable (non-drug) activities and stimuli) and hence may be less sensitive to representations of natural rewards. Additionally, enhanced brain activity to cigarette-related cues might reflect high motivational relevance attributed to these stimuli, potentially due to nicotine-boosted, reward-related brain activity that is otherwise hypoactivated. Though speculative, this interpretation is consistent with the dual-reinforcement model of nicotine dependence: nicotine is both a primary reinforcer and a reinforcement enhancer that magnifies the incentive value of stimuli accompanying nicotine delivery (Caggiula et al. 2009). Although there may be other ways to characterize this construct, for heuristic purposes, we suggest that the relative differences in responsivity to intrinsically pleasant vs. cigarette-related cues reflects underlying sensitivity to the effects of natural rewards (i.e., reward sensitivity), which for those with C > P may be partially mitigated by smoking.
In the parent study, we demonstrated a differential benefit of varenicline vs. bupropion on abstinence (Cinciripini et al. 2013), similar to that seen in other studies (Anthenelli et al. 2016; Cahill et al. 2013), as well as differential effects on measures of depressive affect and smoking reward. Moreover, experimental findings by Brandon et al. (2011) suggest that varenicline reduces the reward value of cigarettes, while those of Rustalis et al. (2005) show that relative to placebo, bupropion does not. We speculated that C > P smokers are less sensitive to intrinsic vs. smoking-related reward and that based on these studies, varenicline could be expected to have a greater impact on abstinence among these individuals than on those demonstrating P > C. The current secondary analysis evaluates the degree to which varenicline may differentially benefit C > P smokers. While several placebo-controlled trials have established the effectiveness of both bupropion and varenicline for smoking cessation (Anthenelli et al. 2016; Cahill et al. 2013; Hughes et al. 2014), mapping differences in treatment outcome to neural markers such as the one proposed here might provide a basis for future treatment-matching strategies.
Although large for an ERP study (n = 180), the analyses described in our original report did not provide adequate power for testing a medication by reward sensitivity group (C > P vs. P > C) interaction using traditional (frequentist) methods of hypothesis testing. Here, we take a Bayesian approach to explore this interaction, which, considerations of power aside, offers the benefit of determining the probability that such an interaction exists, as opposed to making a dichotomous determination (yes/no) regarding its presence using a frequentist approach. Frequentist methods rely on a single probability threshold (e.g., p < .05) to reject the null hypothesis of no interaction (i.e., reward sensitivity does not moderate the effects of medication on abstinence). A Bayesian analysis directly estimates the probability that the alternative hypothesis is true (i.e., an interaction does, in fact, exist; Wijeysundera et al. 2009) and provides estimates for the interaction term (Simon 2002). Although exploratory in nature, this analysis can inform the design of future clinical studies and may have direct treatment implications because it estimates the probability that one drug more favorably benefits a specific subgroup. This allows for calibration of clinical decisions that weigh the risk and benefit of a treatment, given the probability of its success. In addition to the traditional efficacy question (active drug vs. placebo; Green et al. 2009), we evaluated clinical effectiveness (active drug vs. active drug) to estimate the probability that smokers with higher levels of activation to cigarette vs. naturally pleasant cues (C > P) would benefit more from varenicline than bupropion.
Methods
Participants
The original clinical sample consisted of N = 294 smokers who participated in a placebo-controlled trial involving bupropion and varenicline (clinicaltrials.gov identifier: NCT00507728), the main results of which have been previously published (Cinciripini et al. 2013). The ERPs recorded from n = 180 of these smokers provided the basis for our original report on reward sensitivity (Versace et al. 2012), and this is a follow-up analysis of that same sample. We recruited smokers interested in quitting smoking from the community. Inclusion criteria were the following: age of 18–65 years, smoking 5 or more cigarettes per day, baseline expired carbon monoxide (CO) level greater than or equal to 6 ppm, fluency in English, and having a working telephone. Exclusionary criteria included the following: taking psychotropic medication, a current psychiatric disorder (except for nicotine dependence), involvement in any smoking cessation activities, contraindications for either bupropion or varenicline, or any uncontrolled medical illness. The University of Texas MD Anderson Cancer Center Institutional Review Board approved the protocol and informed consent.
Procedures
Participants, screened for trial eligibility 1 week before the baseline ERP laboratory session, were instructed to smoke ad libitum before the baseline session. At this session, participants provided an expired CO sample and completed questionnaires, including the Fagerström Test for Nicotine Dependence (FTND; Heatherton et al. 1991). Following application of EEG electrodes, participants were instructed to keep their eyes on the screen and to minimize movement during the slideshow. Earlier reports summarize details of the screening procedure and other assessments not directly pertinent to this paper (Cinciripini et al. 2013; Robinson et al. 2013; Versace et al. 2010; Versace et al. 2012).
Picture-viewing task
During the 30-min picture-viewing task, participants viewed one of three picture sets composed of four categories, consisting of 24 pleasant (PLE), unpleasant (UNP), cigarette-related (CIG), and neutral (NEU) pictures, selected from the International Affective Picture System (IAPS) pictures (Lang et al. 2005) and from other sets developed by us (Carter et al. 2006; Versace et al. 2011b) and others (Gilbert and Rabinovich 1999; Stritzke et al. 2004). The Online Resource and previous reports (Robinson et al. 2013) provide further details of the pictures.
ERP data collection analyses and LPP response categorization
As reported previously (Versace et al. 2012), based on the amplitude of the LPP recorded at centro-parietal sites for each picture category (400–700 ms after onset), k-means clustering assigned smokers to two classes, characterized by relative differences in brain activation to PLE vs. CIG stimuli. In the current report, we refer to the original cluster 1 (N = 99) as P > C to designate a higher level of activation in the LPP to pleasant vs. cigarette pictures, while cluster 2 (N = 81), denoted C > P, indicates the opposite pattern. The Online Resource summarizes details about the EEG analyses and provides a plot of the reward sensitivity group by picture category interaction (Figure S2).
Treatment
All smokers received ten individual behavioral smoking cessation counseling sessions over the 12-week active treatment phase, six in-person (30 min each) and four by telephone (15 min each). Pharmacotherapy, initiated the day after the first treatment visit, followed the recommended dosing for a total of 12 weeks. Our main outcome paper (Cinciripini et al. 2013) provides complete details of the clinical trial.
Assessment of abstinence
Collection of abstinence data used a timeline follow-back (TLFB) procedure (Brown et al. 1998; Law et al. 2003). In this study, as in the main clinical outcome paper (Cinciripini et al. 2013), prolonged abstinence (Hughes et al. 2003) at the end of treatment (EOT) served as the primary outcome, and 3-and 6-months post-quit prolonged abstinence as secondary outcomes. The common starting point for assessing prolonged abstinence was the end of the grace period (i.e., 2 weeks following the quit date). For prolonged abstinence, relapse was defined by seven or more consecutive days of smoking or smoking at least one cigarette over two consecutive weeks from the end of the grace period to a selected future time point (Hughes et al. 2003). Verification of abstinence reports used either expired CO <10 ppm or salivary cotinine (<15 ng/ml), with participants unavailable for assessment considered non-abstinent.
Statistical approach
Analysis of abstinence data
Logistic regression (PROC GENMOD; SAS v. 9.3; SAS Institute, Inc., Carey, NC, USA) modeled smoking cessation as a function of LPP responses (P > C vs. C > P), treatment (bupropion, varenicline, placebo), and the interaction of LPP and treatment. Two types of interactions were tested: (1) an efficacy analysis, comprised of the LPP × bupropion (vs. placebo) and LPP × varenicline (vs. placebo) interactions, and (2) a comparative effectiveness analysis involving the LPP × varenicline (vs. bupropion) interaction. Follow-up tests of simple effects within the two LPP categories characterized the respective interactions.
Bayesian reasoning represents estimates and accompanying uncertainty regarding the true population parameter in the form of a probability distribution. The posterior distribution resulting from the analysis summarizes the current state of the evidence for a given hypothesis. Mathematically, this posterior distribution is a function of a prior distribution and the observed data. The current analysis takes the perspective that prior distributions are simply another model assumption which should be subject to evaluation (Gelman 2011; Spiegelhalter et al. 2004). Doing so permits assessing the degree to which the data supports the hypotheses, assuming different levels of skepticism (Parmar et al. 1996). Data are evaluated using two sets of prior distributions, each reflecting divergent prior perspectives regarding the true probability that an interaction exists between treatment and reward sensitivity. Both prior distributions are neutral and centered on the null hypothesis. Vague priors regard the existence of the interaction with considerable uncertainty; skeptical, informative priors regard the chance of an interaction as extremely unlikely (i.e., <2.5%).
Distributions of logistic regression coefficients in the log form are approximately normal (Spiegelhalter et al. 2004). Vague, neutral priors assume a distribution for each logistic regression coefficient that is approximately normal (mean = 0, variance = 1 × 106) in the log form. The prior distribution is centered on the null hypothesis of no effect (i.e., exponentiating the mean of the prior yields an OR = 1.0), with a corresponding 95% credible interval that has upper and lower limits of +1960 and −1960, respectively (i.e., OR = 1.65 × 10851 and 6.06 × 10−852). Vague, neutral priors support a broad range of plausible parameter values and characteristically provide estimated values similar to those based on frequentist, maximum likelihood estimates (for comparative purposes, we provide parameter estimates from both approaches for the efficacy analysis of prolonged abstinence in Table S1 in the Online Resource). What differs across the two approaches is the interpretation of these values? In particular, evaluation of the 95% confidence limits for the frequentist estimates of the interaction term indicates whether an interval excludes the null hypothesis (crosses 1.0). Failure to reject the null hypothesis would result in the frequentist conclusion that the study found no evidence of an interaction. The interpretive difference is best illustrated by comparing the 95% credible and confidence limits for the Bayesian and frequentist estimates, respectively, which, again, are very similar to each other in absolute magnitude. The interpretation of the frequentist 95% confidence interval is widely known: with the repetition of the same study/experiment multiple times, the frequentist 95% confidence interval is that range of values that captures the true parameter 95% of the time. Importantly, the frequentist approach remains silent regarding the relative probabilities that various values within the 95% confidence interval represent the true parameter value: a value at the center of the confidence interval is indistinguishable from one at the extremes in terms of the relative probability that it is the true, governing parameter. The Bayesian 95% credible interval based on the posterior distribution, however, does precisely this: it permits the articulation of the relative probabilities that various values in different regions of the interval constitute the true, governing parameter. By definition, the Bayesian posterior distribution is a probability density that integrates to one: the differential height of the density for the various parameter estimates it covers is an indication of the relative probability that one estimate is more or less likely than another. An investigator wishing to calculate the probability that a given parameter estimate exceeds some value (e.g., an OR = 1) may simply calculate the area under the curve for the posterior distribution that exists above the value. Therefore, even if the 95% credible interval does not exclude the null value (e.g., OR = 1), this need not bring inquiry to an end, as the Bayesian approach still permits an estimate that the true, governing parameter exceeds the null value. Since estimates from vague, neutral priors maximize the weight given to the current data and minimize the weight of prior evidence, they provide the basis for the reported Bayesian estimates.
Assessing the robustness of the conclusions based on vague, neutral priors, we present reanalyses of the same models using skeptical, informative priors. Skeptical, informative prior distributions are constructed in the log form, are centered at the null hypothesis of no effect (i.e., OR = 1.00; ln [1] = 0), and follow a normal distribution with a specified variance. Here, we took the position that an interaction is an extremely unlikely occurrence. Thus, the variance for the skeptical, informative prior was specified such that the log odds ratio for each subgroup effect, as estimated using vague, neutral priors, has only a 2.5% chance of occurring (Dixon and Simon 1991; Simon and Freedman 1997; Simon et al. 1996; Simon 2002; Spiegelhalter et al. 2004).
Results
Baseline demographics, smoking, and affective characteristics
Table 1 summarizes the baseline demographic and smoking characteristics for each of the treatment groups, according to LPP grouping. Cross tabulation and ANOVA evaluated variation on baseline measures as a function of LPP group, treatment group, and their interaction. No statistically reliable main effects for LPP group, treatment group, or the LPP by treatment group interaction on any of demographic variables emerged (see the Online Resource, “Analysis of LPP responses within treatment group” section, for additional details).
Table 1.
Variable | Varenicline (N = 58)
|
Bupropion (N = 59)
|
Placebo (N = 63)
|
||||||
---|---|---|---|---|---|---|---|---|---|
P > C, n = 25 | C > P, n = 33 | Total | P > C, n = 40 | C > P, n = 19 | Total | P > C, n = 34 | C > P, n = 29 | Total | |
Race/ethnicity/gender, % (N) | |||||||||
African-American, non-Hispanic | 20.0 (5) | 33.3 (11) | 27.6 (16) | 22.5 (9) | 36.8 (7) | 27.1 (16) | 26.5 (9) | 27.6 (8) | 27.0 (17) |
White, non-Hispanic | 52.0 (13) | 45.5 (15) | 48.3 (28) | 67.5 (27) | 47.4 (9) | 61.0 (36) | 64.7 (22) | 62.1 (18) | 63.5 (40) |
Hispanic | 16.0 (4) | 18.2 (6) | 17.2 (10) | 5.0 (2) | 10.5 (2) | 6.8 (4) | 5.9 (2) | 10.3 (3) | 7.9 (5) |
Other | 12.0 (3) | 3.0 (1) | 6.9 (4) | 5.0 (2) | 5.3 (1) | 5.1 (3) | 2.9 (1) | 0(0) | 1.6(1) |
Gender | |||||||||
Male | 60.0 (15) | 63.6 (21) | 62.1 (36) | 72.5 (29) | 68.4 (13) | 71.2 (42) | 70.6 (24) | 51.7(15) | 61.9 (39) |
Female | 40.0 (10) | 36.4 (12) | 37.9 (22) | 27.5(11) | 31.6 (6) | 28.8 (17) | 29.4 (10) | 48.3 (14) | 38.1 (24) |
Age and smoking, mean (SD) | |||||||||
Age (years) | 44.1 (11.5) | 42.8(11.1) | 43.3 (11.2) | 46.0 (8.7) | 45.9 (9.8) | 46.0 (9.01) | 45.5 (12.3) | 46.6 (9.7) | 46.0(11.1) |
Current smoking rate (cigs/day) | 19.0 (10.2) | 18.8 (7.9) | 18.9 (8.9) | 20.3 (7.5) | 17.4 (6.4) | 19.3 (7.2) | 17.5 (8.0) | 19.9 (9.3) | 18.6 (8.7) |
Age started smoking (years) | 20.6 (5.5) | 16.0 (2.6) | 18.0 (4.7) | 18.0 (4.4) | 18.1 (3.0) | 18.0 (4.0) | 18.5 (7.4) | 18.3 (6.6) | 18.4 (7.0) |
FTND total score | 4.3 (2.5) | 4.8 (2.3) | 4.6 (2.4) | 4.8 (1.8) | 4.3 (1.7) | 4.6 (1.8) | 4.3 (2.2) | 4.7 (2.1) | 4.5 (2.1) |
All frequencies are calculated within the group (column). P > C refers to smokers with higher LPP responses to pleasant than to cigarette-related stimuli; C > P refers to the smokers with higher brain responses to cigarette-related stimuli than to pleasant stimuli
Abstinence results
Figure 1 presents the prolonged abstinence rates for LPP groups (P > C vs. C > P) as a function of the treatment group and time point. Table 2 lists the posterior probabilities for the interaction between the LPP group (P > C vs. C > P) and treatment, for both the efficacy (placebo as reference) and comparative effectiveness analyses (bupropion as reference; Figures S3 and S4 show posterior distributions at the EOT). Please see Table S2 for posterior probabilities for the interaction and Tables S3 and S4 for simple effects concerning other abstinence definitions.
Table 2.
Standard efficacy parameterization (reference placebo and C > P) | Comparative effectiveness parameterization (reference bupropion and C > P) |
|||||
---|---|---|---|---|---|---|
|
|
|||||
Varenicline × LPP group interaction, p(interaction) |
Bupropion × LPP group interaction, p(interaction) |
Varenicline × LPP group interaction, p(interaction) |
||||
Outcome variable | Vague, neutral priors | Skeptical, informative priors | Vague, neutral priors | Skeptical, informative priors | Vague, neutral Priors | Skeptical, informative priors |
Prolonged abstinence at the EOT | 0.590 | 0.538 | 0.835 | 0.745 | 0.895 | 0.713 |
Prolonged abstinence at 3 months | 0.641 | 0.569 | 0.896 | 0.852 | 0.956 | 0.571 |
Prolonged abstinence at 6 months | 0.742 | 0.642 | 0.678 | 0.596 | 0.875 | 0.641 |
P > C refers to smokers with higher LPP responses to pleasant than to cigarette-related stimuli; C > P refers to the smokers with higher brain responses to cigarette-related stimuli than to pleasant stimuli
Efficacy analysis
Using vague neutral priors (Table 2), the probability of an interaction between the LPP group (P > C vs. C > P) and treatment (bupropion vs. placebo) was 84 and 90% at the EOT and 3 months and nearly 70% at 6 months, suggesting a differential effect for bupropion on abstinence rates as a function of reward sensitivity, particularly at the EOT (our primary outcome point) and 3-month follow-up. For varenicline, the chance of an LPP group × treatment (varenicline vs. placebo) interaction was 59 and 64% at the EOT and 3 months, and 74% at 6 months, suggesting that the LPP group strongly moderated the effect of bupropion but not varenicline vs. placebo, at the EOT and 3 months with the disappearance of the effect by a 6-month follow-up.
These differences are more clearly noted in the analyses of the simple effects of treatment within each of the LPP subgroups. Table 3 provides the odds ratio, 95% credible interval, and the posterior probability that the parameter estimate exceeds an OR of 1.00 at each of the selected time points (Figures S4 and S5 depict posterior densities at the EOT). Among P > C smokers, those randomized to either bupropion or varenicline have a >98% chance of achieving a higher level of abstinence relative to placebo (i.e., probability of OR >1) through the 3-month follow-up. At the 6-month follow-up, the posterior probability of achieving abstinence was 96% for varenicline and 83% for bupropion (i.e., probability of OR >1). The calculated ORs for P > C smokers, similar for each drug vs. placebo, suggest a comparable benefit from either medication through the 6-month follow-up. The results were quite different for C > P smokers. Relative to placebo, smokers with larger brain responses to cigarette-related vs. pleasant stimuli (C > P) had a 99% chance of achieving increased abstinence across all time points when treated with varenicline (i.e., probability of OR >1), but when treated with bupropion, C > P smokers had only a 67% chance of increased abstinence (i.e., OR >1) at the EOT and a 43 and 49% chance at the 3- and 6-month follow-ups. As shown in Table 3, ORs for C > P smokers for each drug vs. placebo were much higher for varenicline than bupropion (see Table S2 for other abstinence measures).
Table 3.
OR | 95% CBI LCL | 95% CBI UCL | p(odd ratio >1) | |
---|---|---|---|---|
Prolonged abstinence at the EOT | ||||
P > C | ||||
Varenicline | 2.151 | 1.069 | 9.738 | 0.981 |
Bupropion | 2.254 | 1.214 | 8.593 | 0.991 |
C > P | ||||
Varenicline | 3.784 | 1.248 | 12.776 | 0.991 |
Bupropion | 1.369 | 0.326 | 5.604 | 0.670 |
Prolonged abstinence at 3 months | ||||
P > C | ||||
Varenicline | 3.155 | 1.079 | 9.825 | 0.982 |
Bupropion | 2.864 | 1.099 | 7.733 | 0.985 |
C > P | ||||
Varenicline | 4.252 | 1.354 | 15.436 | 0.994 |
Bupropion | 0.866 | 0.150 | 4.296 | 0.430 |
Prolonged abstinence at 6 months | ||||
P > C | ||||
Varenicline | 2.630 | 0.876 | 8.161 | 0.957 |
Bupropion | 1.627 | 0.602 | 4.552 | 0.829 |
C > P | ||||
Varenicline | 4.759 | 1.248 | 24.354 | 0.990 |
Bupropion | 0.965 | 0.104 | 7.011 | 0.486 |
The reference group is placebo. The odds ratio and 95% credible interval (CBI) upper confidence limit (UCL) and lower confidence limit (LCL) are shown. The credible intervals estimate the relative probabilities that the parameter estimates fall within this range. P > C refers to smokers with higher LPP responses to pleasant than to cigarette-related stimuli; C > P refers to the smokers with higher brain responses to cigarette-related stimuli than to pleasant stimuli
Re-estimating the posterior distribution of these interactions using skeptical priors initially assumes that the interaction has only a 2.5% chance of occurring. Table 2 shows the final posterior probability estimates using skeptical informative priors for an interaction of bupropion and reward sensitivity were 75% at the EOT, 85% at 3 months, and 60% at 6 months. Comparable values for varenicline ranged from 54 to 64% over time. These results, which are more conservative because they assume only a 2.5% chance of an interaction of the observed magnitude with vague, neutral priors, show a similar pattern of an increased chance of an interaction between bupropion and LPP and a lower chance of interaction between varenicline and LPP. As in the earlier case, this relationship disappears by the 6-month measurement point.
Comparative effectiveness
Using vague neutral priors, Table 2 shows the probability of an LPP group (P > C vs. C > P) × treatment (varenicline vs. bupropion) interaction was 90% at the EOT, 96% at 3 months, and 88% at the 6-month follow-up (See Supplement Table S2 for other abstinence measures). Analysis of simple effects (Table 4, Figure S7 depicts posterior distributions at the EOT) indicated that C > P smokers benefited more from varenicline than bupropion (i.e., OR >1.00): 95% chance at the EOT and ≥98% at the 3- and 6-month follow-ups. However, P > C participants had a 50% chance of differential benefit from varenicline relative to bupropion (i.e., OR >1.00) at the EOT, 58% at 3 months, and 82% at 6 months (see supplement Table S4 for other abstinence measures).
Table 4.
OR | 95% CBI LCL | 95% CBI UCL | p(OR >1) | |
---|---|---|---|---|
Prolonged abstinence at the EOT | ||||
P > C: varenicline vs. bupropion | 1.000 | 0.363 | 2.298 | 0.498 |
C > P: varenicline vs. bupropion | 2.815 | 0.826 | 10.527 | 0.950 |
Prolonged abstinence at 3 months | ||||
P > C: varenicline vs. bupropion | 1.108 | 0.403 | 3.077 | 0.578 |
C > P: varenicline vs. bupropion | 5.089 | 1.262 | 25.562 | 0.990 |
Prolonged abstinence at 6 months | ||||
P > C: varenicline vs. bupropion | 1.615 | 0.578 | 4.571 | 0.819 |
C > P: varenicline vs. bupropion | 5.261 | 1.055 | 38.671 | 0.980 |
The odds ratio and 95% credible interval (CBI) upper confidence limit (UCL) and lower confidence limit (LCL) are shown. The credible intervals estimate the relative probabilities that the parameter estimates fall within this range. P > C refers to smokers with higher LPP responses to pleasant than to cigarette-related stimuli; C > P refers to the smokers with higher brain responses to cigarette-related stimuli than to pleasant stimuli
Using skeptical, informative priors, initially assuming only a 2.5% chance of an interaction, identified with vague priors, we found that the corresponding posterior probabilities were 71% at the EOT, 57% at the 3-month follow-up, and 64% at the 6-month follow-up. Thus, while the probability for the interaction is nearly 90% or greater at all three time points using vague neutral priors, a more conservative, skeptical approach results in posterior probability estimates that an effect exists (OR >1) of 60–70%.
Discussion
This secondary data analysis from a previous clinical trial (Cinciripini et al. 2013) evaluated the interaction of LPP response to cigarette and pleasant stimuli with pharmacotherapy (varenicline or bupropion) on smoking cessation outcome. Previously having shown that smokers with relatively higher levels of brain activation to cigarette compared to naturally pleasant stimuli quit smoking less often than those with the opposite pattern (Versace et al. 2012), we hypothesized that this divergence might reflect a lower sensitivity to natural rewards. The current paper examined the probabilities of that interaction using Bayesian statistical modeling, as an alternative to frequentist methods of hypothesis testing.
The results of our efficacy analyses (varenicline or bupropion vs. placebo), using vague neutral priors, suggest that P > C smokers demonstrate comparable likelihoods of successfully abstaining at the EOT (our primary outcome) and at the 3-month follow-up when treated with either varenicline or bupropion, whereas C > P smokers are more likely to abstain if given varenicline. By the 6-month follow-up, both groups favor varenicline relative to placebo. Importantly, however, the comparative effectiveness analysis (bupropion vs. varenicline) showed the probability of an LPP group × treatment interaction exceeded 87%, and simple effects analyses indicated this was due to a higher probability of abstinence (i.e., OR >1.00) for C > P smokers taking varenicline (99% chance of OR >1.00) vs. bupropion (43–67% of OR >1.00), across all time points. P > C smokers showed a similar probability of benefit from either varenicline (96–98% chance of OR >1.00) or bupropion (83–99% chance of OR >1.00) across time points. Using highly conservative skeptical informative priors, the posterior probabilities of the LPP group × treatment interactions were lower at all time points, although to be clear, this set of assumptions represents an extremely conservative, initial estimate of 2.5% regarding the existence of an interaction of the same magnitude as that observed with vague neutral priors. Nevertheless, the overall consistency in direction and magnitude, across time, for both sets of assumptions (vague neutral or skeptical priors), suggests that the likelihood of interaction at the EOT and 3 months, in particular, is indeed quite robust.
The results have practical implications for determining a course of pharmacotherapy and for the analysis of clinical trial data using Bayesian analytical methods. From a treatment perspective, our results suggest that a neural biomarker of reward sensitivity using the LPP may be a useful pretreatment tool in determining medication assignment. Smokers with larger brain responses to naturally rewarding than to cigarette-related stimuli (i.e., P > C) seem to demonstrate a similar probability of benefit from either bupropion or varenicline at the end of treatment and at the 3-month follow-up, though these effects diverge at 6 months. Those with larger brain responses to cigarette-related stimuli than to naturally rewarding stimuli (i.e., C > P) show a greater probability of benefit conferred by varenicline at all time points. Clinically, varenicline seems to be the better treatment choice between these two alternatives for C > P smokers, especially if we wish to maximize early treatment success. For P > C smokers, since either medication may be effective, treatment choice may depend on other factors, such as having a contraindication for one medication (i.e., seizure disorder for bupropion or significant renal impairment in the case of varenicline), or having a higher probability of certain adverse events in one or the other treatments (GI disturbance with varenicline). Cost might be another factor in determining treatment choice, as both bupropion and nicotine replacement therapy (NRT) have generic equivalents. Such factors favor the use of bupropion for P > C smokers or an equally effective low-risk treatment, such as NRT, given that large-scale clinical trials and meta-analyses have established statistical equivalence between NRT and bupropion (Mills et al. 2012; Piper et al. 2009; Stead et al. 2012).
Our findings inform and support current models of dependence that emphasize impaired reward circuitry as a primary mechanism associated with chronic drug use: chronic drug use will result in the eventual hypoactivation of the dopaminergic system and the regulatory pathways that modulate dopaminergic activity (Volkow et al. 2010), leading, in turn, to diminished sensitivity to the effects of natural rewards (e.g., food, sex) and enhanced reactivity to drug-related stimuli. Measuring neural activation in the presence of both drug-related and non-drug-related pleasant cues, we have shown that such a process is manifest in a large subgroup of smokers. We have also shown the potential of varenicline to ameliorate the adverse effects of this process on treatment outcome. Varenicline acting as a partial dopamine agonist at the α2β4 nicotinic receptor (Coe et al. 2005) may offset the diminished responsiveness of the dopaminergic system, particularly when nicotine, a potent yet often short-lived dopaminergic agonist, is withdrawn. Evidence suggests that strong stimulation of dopaminergic pathways may offset drug use in animal models of cocaine (Thanos et al. 2008) and alcohol (Thanos et al. 2004) dependence.
The current paper highlights the usefulness of Bayesian statistical approaches in the analyses of early-stage clinical trials and suggests potential applications to clinical decision making and subgroup analyses. Addressing treatment interactions using conventional (frequentist) methods requires as much as a fourfold increase in sample size over the requirement to evaluate main effects (Brookes et al. 2004). Subgroup analyses using Bayesian approaches, while valuable in their own right, might be used to justify the commitment of additional resources for a larger conventional trial. By estimating the probability of differential treatment benefit within subgroups, we can assign a probability to the alternative hypothesis, rather than relying on a dichotomous probability threshold (.05) to reject the null hypothesis. We can further increase the confidence in our findings by specifying a probability level to establish benefit. In our case, we emphasized findings where the benefit of a particular treatment for a subgroup (i.e., C > P) equaled or exceeded 70%. Doing so a priori, in a phase 2 trial, allows for a principled go-no-go decision for moving forward with a larger phase 3 trial, as recently discussed by the FDA for evaluating efficacy of new medications (Berry 2005; Goodman 2005; Lipscomb et al. 2005; O’Neill 2006; Temple 2005; US Food and Drug Administration 2004).
Limitations of the current paper relate primarily to the secondary nature of the analysis. Given that the specification of the subgroup analysis did not occur a priori, a cautious interpretation of these findings as hypotheses generating seems warranted. Moreover, as additional evidence accrues, characterizing the LPP groups in other ways should permit exploration of alternative explanations for its predictive value, including questionnaire and behavioral measures of anhedonia, reward valuation, and experience, as well as different methods for forming such subgroups. Finally, the relatively small cell sizes of the LPP groups within certain treatment condition should temper the degree to which these findings generalize to the broader population. An ongoing clinical trial in our group will provide a better opportunity to address these issues.
Supplementary Material
Acknowledgments
The authors wish to thank Krystle Bartley, Dr. Victoria L. Brown, Jennifer Canul, Janeene Frerking, Christine Jeria, Paul Longoria, Samuel W. Miller, Kevin Mulpur, Cissette Muster, Tiffany Rattler, and Susana Torres for their help in the data collection. This study was conducted in accordance with the ethical standards of the University of Texas MD Anderson Cancer Center and applicable US Department of Health and Human Services regulations.
Funding Support for this research was provided by grants from the National Institute on Drug Abuse (grant number R01DA017073) to Paul M. Cinciripini and the Cancer Center Support Grant from the National Cancer Institute (grant number P50CA70907) to MD Anderson Cancer Center. Pfizer provided the varenicline and matching placebo.
Footnotes
Electronic supplementary material The online version of this article (doi:10.1007/s00213-017-4580-2) contains supplementary material, which is available to authorized users.
Compliance with ethical standards The University of Texas MD Anderson Cancer Center Institutional Review Board approved the protocol and informed consent.
Conflict of interest Dr. Cinciripini served on the scientific advisory board of Pfizer Pharmaceuticals and conducted educational talks sponsored by Pfizer on smoking cessation (2006–2008) and has received grant support from Pfizer. The other authors declare that they have no conflict of interest.
References
- Anthenelli RM, Benowitz NL, West R, St AL, McRae T, Lawrence D, Ascher J, Russ C, Krishen A, Evins AE. Neuropsychiatric safety and efficacy of varenicline, bupropion, and nicotine patch in smokers with and without psychiatric disorders (EAGLES): a double-blind, randomised, placebo-controlled clinical trial. Lancet. 2016;387(10037):2507–2520. doi: 10.1016/S0140-6736(16)30272-0. [DOI] [PubMed] [Google Scholar]
- Berry DA. Introduction to Bayesian methods III: use and interpretation of Bayesian tools in design and analysis. Clinical Trials. 2005;2(4):295–300. doi: 10.1191/1740774505cn100oa. [DOI] [PubMed] [Google Scholar]
- Brandon TH, Drobes DJ, Unrod M, Heckman BW, Oliver JA, Roetzheim RC, Karver SB, Small BJ. Varenicline effects on craving, cue reactivity, and smoking reward. Psychopharmacology. 2011;218(2):391–403. doi: 10.1007/s00213-011-2327-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brookes ST, Whitely E, Egger M, Smith GD, Mulheran PA, Peters TJ. Subgroup analyses in randomized trials: risks of subgroup-specific analyses; power and sample size for the interaction test. J Clin Epidemiol. 2004;57(3):229–236. doi: 10.1016/j.jclinepi.2003.08.009. [DOI] [PubMed] [Google Scholar]
- Brown RA, Burgess ES, Sales SD, Whiteley JA, Evans DM, Miller IW. Reliability and validity of a smoking timeline follow-back interview. Psychol Addict Behav. 1998;12(2):101–112. doi: 10.1037/0893-164X.12.2.101. [DOI] [Google Scholar]
- Caggiula AR, Donny EC, Palmatier MI, Liu X, Chaudhri N, Sved AF. The role of nicotine in smoking: a dual-reinforcement model. In: Bevins R, Caggiula A, editors. The motivational impact of nicotine and its role in tobacco use, Nebraska symposium on motivation. Vol. 55. Springer-Verlag; New York: 2009. pp. 91–109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cahill K, Stevens S, Perera R, Lancaster T. Pharmacological interventions for smoking cessation: an overview and network meta-analysis. Cochrane Database Syst Rev. 2013;5:CD009329–CD009329. doi: 10.1002/14651858.CD009329.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carter BL, Robinson JD, Lam CY, Wetter DW, Day SX, Tsan JY, Cinciripini PM. A psychometric evaluation of cigarette stimuli used in a cue reactivity study. Nicotine Tob Res. 2006;8(3):361–369. doi: 10.1080/14622200600670215. [DOI] [PubMed] [Google Scholar]
- Cinciripini PM, Robinson JD, Karam-Hage M, Minnix JA, Lam CY, Versace F, Brown VL, Engelmann JM, Wetter DW. Effects of varenicline and bupropion sustained-release use plus intensive smoking cessation counseling on prolonged abstinence from smoking and on depression, negative affect, and other symptoms of nicotine withdrawal. JAMA Psychiatry. 2013;70(5):522–533. doi: 10.1001/jamapsychiatry.2013.678. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Coe JW, Brooks PR, Vetelino MG, Wirtz MC, Arnold EP, Huang J, Sands SB, Davis TI, Lebel LA, Fox CB, Shrikhande A, Heym JH, Schaeffer E, Rollema H, Lu Y, Mansbach RS, Chambers LK, Rovetti CC, Schulz DW, Tingley FDI, O’Neill BT. Varenicline: an alpha4beta2 nicotinic receptor partial agonist for smoking cessation. J Med Chem. 2005;48(10):3474–3477. doi: 10.1021/jm050069n. [DOI] [PubMed] [Google Scholar]
- Dixon DO, Simon R. Bayesian subset analysis. Biometrics. 1991;47(3):871–881. [PubMed] [Google Scholar]
- Gelman A. Induction and deduction in Bayesian data analysis. Rationality, Markets and Morals. 2011;2:67–78. [Google Scholar]
- Gilbert DG, Rabinovich NE. The international smoking image series (with neutral counterparts), v. 1.2. Department of Psychology, Southern Illinois University; Carbondale: 1999. [Google Scholar]
- Goodman SN. Introduction to Bayesian methods I: measuring the strength of evidence. Clinical Trials. 2005;2(4):282–290. doi: 10.1191/1740774505cn098oa. [DOI] [PubMed] [Google Scholar]
- Green CE, Moeller FG, Schmitz JM, Lucke JF, Lane SD, Swann AC, Lasky RE, Carbonari JP. Evaluation of heterogeneity in phar-macotherapy trials for drug dependence: a Bayesian approach. Am J Drug Alcohol Abuse. 2009;35(2):95–102. doi: 10.1080/00952990802647503. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hajcak G, MacNamara A, Olvet DM. Event-related potentials, emotion, and emotion regulation: an integrative review. Dev Neuropsychol. 2010;35(2):129–155. doi: 10.1080/87565640903526504. [DOI] [PubMed] [Google Scholar]
- Heatherton TF, Kozlowski LT, Frecker RC, Fagerström KO. The Fagerström test for nicotine dependence: a revision of the Fagerström Tolerance Questionnaire. Br J Addiction. 1991;86(9):1119–1127. doi: 10.1111/j.1360-0443.1991.tb01879.x. [DOI] [PubMed] [Google Scholar]
- Hughes JR, Keely JP, Niaura RS, Ossip-Klein DJ, Richmond RL, Swan GE. Measures of abstinence in clinical trials: issues and recommendations. Nicotine Tob Res. 2003;5(1):13–25. doi: 10.1080/1462220031000070552. [DOI] [PubMed] [Google Scholar]
- Hughes JR, Stead LF, Hartmann-Boyce J, Cahill K, Lancaster T. Antidepressants for smoking cessation. Cochrane Database Syst Rev. 2014;1:CD000031. doi: 10.1002/14651858.CD000031.pub4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Koob GF, Volkow ND. Neurocircuitry of addiction. Neuropsychopharmacol. 2010;35(1):217–238. doi: 10.1038/npp.2009.110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lang PJ, Bradley MM. Emotion and the motivational brain. Biol Psychol. 2009;84(3):437–450. doi: 10.1016/j.biopsycho.2009.10.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lang PJ, Bradley MM, Cuthbert BN. International Affective Picture System (IAPS): affective ratings of pictures and instruction manual. Technical report no. A-6. University of Florida; Gainesville: 2005. [Google Scholar]
- Law KL, Stroud LR, LaGasse LL, Niaura R, Liu J, Lester BM. Smoking during pregnancy and newborn neurobehavior. Pediatrics. 2003;111(6):1318–1323. doi: 10.1542/peds.111.6.1318. [DOI] [PubMed] [Google Scholar]
- Lipscomb B, Ma G, Berry DA. Bayesian predictions of final outcomes: regulatory approval of a spinal implant. Clinical Trials. 2005;2(4):325–333. doi: 10.1191/1740774505cn104oa. [DOI] [PubMed] [Google Scholar]
- Mills EJ, Wu P, Lockhart I, Thorlund K, Puhan M, Ebbert JO. Comparisons of high-dose and combination nicotine replacement therapy, varenicline, and bupropion for smoking cessation: a systematic review and multiple treatment meta-analysis. Ann Med. 2012;44(6):588–597. doi: 10.3109/07853890.2012.705016. [DOI] [PubMed] [Google Scholar]
- O’Neill RT. FDA’s critical path initiative: a perspective on contributions of biostatistics. Biom J. 2006;48(4):559–564. doi: 10.1002/bimj.200510237. [DOI] [PubMed] [Google Scholar]
- Parmar MKB, Ungerleider RS, Simon R. Assessing whether to perform a confirmatory randomized clinical trial. J Natl Cancer Inst. 1996;88(22):1645–1651. doi: 10.1093/jnci/88.22.1645. [DOI] [PubMed] [Google Scholar]
- Piper ME, Smith SS, Schlam TR, Fiore MC, Jorenby DE, Fraser D, Baker TB. A randomized placebo-controlled clinical trial of 5 smoking cessation pharmacotherapies. Arch Gen Psychiatry. 2009;66(11):1253–1262. doi: 10.1001/archgenpsychiatry.2009.142. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Robinson JD, Versace F, Lam CY, Minnix JA, Engelmann JM, Cui Y, Karam-Hage M, Shete S, Tomlinson GE, Chen TT-L, Wetter DW, Green CE, Cinciripini PM. The CHRNA3 rs578776 variant is associated with an intrinsic reward sensitivity deficit in smokers. Front Psychiatry. 2013;4(114):1–11. doi: 10.3389/fpsyt.2013.00114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rukstalis M, Jepson C, Strasser A, Lynch KG, Perkins K, Patterson F, Lerman C. Naltrexone reduces the relative reinforcing value of nicotine in a cigarette smoking choice paradigm. Psychopharmacology. 2005;180(1):41–48. doi: 10.1007/s00213-004-2136-8. [DOI] [PubMed] [Google Scholar]
- Sescousse G, Caldu X, Segura B, Dreher JC. Processing of primary and secondary rewards: a quantitative meta-analysis and review of human functional neuroimaging studies. Neurosci Biobehav Rev. 2013;37(4):681–696. doi: 10.1016/j.neubiorev.2013.02.002. [DOI] [PubMed] [Google Scholar]
- Simon R. Bayesian subset analysis: application to studying treatment-by-gender interactions. Stat Med. 2002;21(19):2909–2916. doi: 10.1002/sim.1295. [DOI] [PubMed] [Google Scholar]
- Simon R, Freedman LS. Bayesian design and analysis of two × two factorial clinical trials. Biometrics. 1997;53:456–464. doi: 10.2307/2533949. [DOI] [PubMed] [Google Scholar]
- Simon R, Dixon DO, Freidlin B. Bayesian subset analysis of a clinical trial for the treatment of HIV infections. Statist Textbooks Monogr. 1996;151:555–576. [Google Scholar]
- Spiegelhalter DJ, Abrams KR, Myles JP. Bayesian approaches to clinical trials and health-care evaluation. Vol. 13. Wiley; New York: 2004. [Google Scholar]
- Stead LF, Perera R, Bullen C, Mant D, Hartmann-Boyce J, Cahill K, Lancaster T. Nicotine replacement therapy for smoking cessation. Cochrane Database Syst Rev. 2012;11:CD000146–CD000146. doi: 10.1002/14651858.CD000146.pub4. [DOI] [PubMed] [Google Scholar]
- Stritzke WG, Breiner MJ, Curtin JJ, Lang AR. Assessment of substance cue reactivity: advances in reliability, specificity, and validity. Psychol Addict Behav. 2004;18(2):148–159. doi: 10.1037/0893-164X.18.2.148. [DOI] [PubMed] [Google Scholar]
- Temple R. How FDA currently makes decisions on clinical studies. Clinical Trials. 2005;2(4):276–281. doi: 10.1191/1740774505cn097oa. [DOI] [PubMed] [Google Scholar]
- Thanos PK, Taintor NB, Rivera SN, Umegaki H, Ikari H, Roth G, Ingram DK, Hitzemann R, Fowler JS, Gatley SJ, Wang GJ, Volkow ND. DRD2 gene transfer into the nucleus accumbens core of the alcohol preferring and nonpreferring rats attenuates alcohol drinking. Alcohol Clin ExpRes. 2004;28(5):720–728. doi: 10.1097/01.ALC.0000125270.30501.08. [DOI] [PubMed] [Google Scholar]
- Thanos PK, Michaelides M, Umegaki H, Volkow ND. D2R DNA transfer into the nucleus accumbens attenuates cocaine self-administration in rats. Synapse. 2008;62(7):481–486. doi: 10.1002/syn.20523. [DOI] [PMC free article] [PubMed] [Google Scholar]
- US Food and Drug Administration. [Accessed 27 Jan 2017];Inovation/stagnation: challenge and opportunity on the critical path to new medical products. 2004 http://www.fda.gov/downloads/ScienceResearch/SpecialTopics/CriticalPathInitiative/CriticalPathOpportunitiesReports/UCM113411.pdf.
- Versace F, Robinson JD, Lam CY, Minnix JA, Brown VL, Carter BL, Wetter DW, Cinciripini PM. Cigarette cues capture smokers’ attention: evidence from event-related potentials. Psychophysiology. 2010;47(3):435–441. doi: 10.1111/j.1469-8986.2009.00946.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Versace F, Engelmann JM, Jackson EF, Costa VD, Robinson JD, Lam CY, Minnix JA, Brown VL, Wetter DW, Cinciripini PM. Do brain responses to emotional images and cigarette cues differ? An fMRI study in smokers. Eur J Neurosci. 2011a;34(12):2054–2063. doi: 10.1111/j.1460-9568.2011.07915.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Versace F, Minnix JA, Robinson JD, Lam CY, Brown VL, Cinciripini PM. Brain reactivity to emotional, neutral and cigarette-related stimuli in smokers. Addict Biol. 2011b;16(2):296–307. doi: 10.1111/j.1369-1600.2010.00273.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Versace F, Lam CY, Engelmann JM, Robinson JD, Minnix JA, Brown VL, Cinciripini PM. Beyond cue reactivity: blunted brain responses to pleasant stimuli predict long term smoking abstinence. Addict Biol. 2012;17(6):991–1000. doi: 10.1111/j.1369-1600.2011.00372.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Versace F, Engelmann JM, Robinson JD, Jackson EF, Green CE, Lam CY, Minnix JA, Karam-Hage M, Brown VL, Wetter DW, Cinciripini PM. Pre-quit fMRI responses to pleasant and cigarette cues predict cessation outcome. Nicotine Tob Res. 2014;16(6):697–708. doi: 10.1093/ntr/ntt214. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Volkow ND, Wang GJ, Fowler JS, Tomasi D, Telang F, Baler R. Addiction: decreased reward sensitivity and increased expectation sensitivity conspire to overwhelm the brain’s control circuit. BioEssays. 2010;32(9):748–755. doi: 10.1002/bies.201000042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wijeysundera DN, Austin PC, Hux JE, Beattie WS, Laupacis A. Bayesian statistical inference enhances the interpretation of contemporary randomized controlled trials. J Clin Epidemiol. 2009;62(1):13–21. doi: 10.1016/j.jclinepi.2008.07.006. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.