Abstract
Objective
Patient expectancies are hypothesized to contribute to the efficacy and side effects of psychiatric treatments, but little research has investigated this hypothesis in the context of psychopharmacological therapies for anxiety. We prospectively investigated whether expectancies predicted efficacy and adverse events in oral therapy for Generalized Anxiety Disorder (GAD), controlling for confounding patient characteristics correlating with outcomes.
Methods
Expectancies regarding treatment efficacy and side effects were assessed at baseline of an eight week open-label phase of a trial of chamomile for Generalized Anxiety Disorder (GAD). The primary outcome was patient-reported GAD-7 scores, with clinical response and treatment-emergent side-effects as secondary outcomes. Expectancies were used to predict symptomatic and side-effect outcomes.
Results
Very few baseline patient characteristics predicted either type of expectancy. Controlling for a patient’s predicted recovery based on their baseline characteristics, higher efficacy expectancies at baseline predicted greater change on the GAD-7 (adjusted β = −0.19, p = 0.011). Efficacy expectancies also predicted a higher likelihood of attaining clinical response (adjusted odds ratio = 1.69, p = 0.002). Patients with higher side effect expectancies reported more side effects (adjusted log expected count = 0.26, p = 0.038). Efficacy expectancies were unrelated to side effect reports (log expected count = −0.05, p = 0.680), and side effect expectancies were unrelated to treatment efficacy (β = 0.08, p = 0.306).
Conclusions
Patients entering chamomile treatment for GAD with more favorable self-generated expectancies for the treatment experience greater improvement and fewer adverse events. Aligning patient expectancies with treatment selections may optimize outcomes.
Keywords: Expectancy, Placebo, Nocebo, Anxiety, Efficacy
Introduction
Patient expectancies for treatment have been identified as a key contributor to therapeutic effects and experience of side effects in both clinical practice and clinical trials (Bingel, 2014; Horing et al., 2014; Mora et al., 2011). For example, the higher the probability of being randomized to an active drug versus placebo arm of a randomized trial, the greater the observed magnitude of placebo effects in adult depression (Rutherford et al., 2009b; Rutherford et al., 2010; Rutherford et al., 2014b). Experimentally altering patients’ beliefs about whether they are taking an active medication has sometimes been found to enhance the effects of placebos (Vase et al., 2002). Similarly, side effect profiles in the placebo arms of clinical trials often resemble those of the active drug comparator (Mora et al., 2011; Rojas-Mirquez et al., 2014) (i.e., a nocebo effect), and manipulating patients’ side effect expectations affects their reports of side effects (Mondaini et al., 2007; Wise et al., 2009).
However, it is less known as to how a patient’s own positive and negative expectancies for a particular treatment shape their experiences while on that specific treatment. Across medical disciplines, prior studies have frequently measured patients’ general health optimism or pessimism rather than their expectancy that a particular treatment would be helpful for their condition or be likely to cause side effects (Barefoot et al., 2011; Enck et al., 2013; Nestoriuc et al., 2010). While these studies have been cited as providing evidence for expectancy effects in treatments, specific expectancies for treatment are psychometrically distinguishable from health optimism and pessimism (Haanstra et al., 2015).
Expectancy research in psychopharmacology has primarily concerned the treatment of depression (Krell et al., 2004; Leuchter et al., 2014; Mora et al., 2011; Rutherford et al., 2013; Rutherford and Roose, 2013; Rutherford et al., 2010; Rutherford et al., 2014b; Sotsky et al., 1991; Weimer et al., 2015), in which naturalistic and manipulated expectancies are typically found to relate to depression treatment outcomes on placebo and often on active medications (though not always; Leuchter et al., 2014). However, negative expectancies are typically not assessed (Colagiuri et al., 2013). Furthermore, no study to our knowledge has assessed positive and negative expectancies in tandem, and often little is done to disentangle expectancies from confounding patient characteristics. For example, the number of prior treatments a patient has had for a condition could relate to both a patient’s belief that they can get better on a treatment, and how treatment-resistant their illness is.
Moreover, for anxiety disorders—and generalized anxiety disorder (GAD) in particular—there has been very limited research into the role of patient-held expectancies in psychopharmacological treatment. This is unfortunate, as anxiety disorders as a class may evidence a less strong response to placebo or “common factors” interventions compared to unipolar depression (Cuijpers et al., 2012; Hofmann and Smits, 2008). Thus, it is possible that expectancy-driven responses differ in the treatment of anxiety as compared to depressive disorders, and that expectancies may have less or no relationship to outcomes for this disorder class.
On the other hand, a recent meta-analysis of psychopharmacological treatment of anxiety found that improvement on active medication was significantly greater in active-comparator studies (e.g., Drug A vs. Drug B) relative to placebo-controlled studies, replicating findings in depression (Rutherford et al., 2015). Patients have a higher expectancy for improvement in active-comparator designs relative to placebo-controlled designs (Gaudiano et al., 2013; Rutherford et al., 2009a), and these heightened expectancies are hypothesized to contribute to effects observed in active-comparator trials. Supportively, a recent randomized controlled trial treating depression reported a superiority of randomization to open-label citalopram versus placebo-controlled citalopram, and found that increases in expectancy in the open-label group mediated this superiority (Rutherford et al., 2016). Thus, it is possible that expectancy effects enhance treatment response in anxiety as they do in depression (Rutherford et al., 2009b; Rutherford et al., 2016). Ultimately, however, the relevance of the full body of depression-focused expectancy research in psychopharmacology to anxiety treatment is unclear. Observation of a relationship between patient-held expectancies and anxiety outcomes on a drug would further support an expectancy-based account of this meta-analytic finding (Rutherford et al., 2015).
Direct evidence does exist concerning the predictive value of patient expectancies in the psychotherapeutic treatment of anxiety. Early treatment expectancies have been found to correlate positively with outcomes in evidence-based psychotherapies for GAD (Borkovec and Costello, 1993; Newman and Fisher, 2010), social anxiety (Chambless et al., 1997; Safren et al., 1997), simple phobia (Price et al., 2008), and mixed anxiety disorders (Brown et al., 2014; Westra et al., 2007). Nevertheless, given that expectancies may act differently in a psychotherapy as compared to pill treatment—for example, as a motivation to engage in psychotherapeutic procedures such as exposures to feared stimuli or completing homework (Westra et al., 2007)—the transferability of this research to the psychopharmacology context is uncertain.
To help elucidate the role that particular expectancies may play in predicting symptomatic and side effect outcomes in psychopharmacological treatments for anxiety, we prospectively evaluated treatment-specific patient expectancies during a clinical trial of chamomile treatment for GAD. Expectancies for treatment efficacy and side effect emergence were assessed separately. We hypothesized that higher expectancy for treatment response would predict greater improvements in core anxiety symptoms and well-being. We also hypothesized that higher expectancy of side effect emergence would predict more reports of treatment-related side effects during treatment. Furthermore, we hypothesized that these relationships would be specific to their respective outcomes, and would not reflect general optimism or pessimism. Finally, we aimed to clarify whether any observed effects of expectancies were potentially attributable to their correlation with baseline patient characteristics that predict outcome (e.g., number of prior treatments), and hypothesized that expectancies would uniquely predict variance in outcomes even when adjusting for these baseline characteristics.
Methods
Patients
Patients were adults (≥18 years) with a DSM-IV diagnosis of GAD as a primary disorder recruited from a psychiatric clinic at a major research hospital and from primary care practices. All diagnoses were determined using the MINI-SCID/P (First et al., 2001) structured interview to assess for the presence of specific DSM-IV Axis I disorders. Discrepancies in diagnostic assessment for inclusion into the study were resolved by conferencing and consensus between the investigators of the trial. Patients diagnosed with Axis I psychosis, bipolar disorder, or substance abuse or dependence were excluded from participation.
The details of the trial design have been published previously (Mao et al., 2014). The overall study is a randomized-placebo controlled trial (RCT) to evaluate whether long-term use of chamomile will result in decreased relapse of GAD symptoms as compared to placebo. A prior RCT found a significant advantage for chamomile over placebo in acute-phase treatment of GAD (Amsterdam et al., 2009; Amsterdam et al., 2012), with a response rate comparable to that of tested anxiolytic and antidepressant therapies for GAD (Mitte et al., 2005). For this manuscript, we analyzed the data from phase I, when all participants were given an open-label administration of pharmaceutical-grade, standardized chamomile extract capsules totaling 1,500 mg/daily for 8 weeks (Mao et al., 2014).
Measurement of Expectancies
Mao Expectancy of Treatment Effects (METE)
The METE was modified from the Acupuncture Expectancy Scale developed and validated by the senior author (see online supplement for instrument) (Mao et al., 2010). The modified instrument is a 4-question patient-report questionnaire rated on a scale of 1–5 (wherein 1 is total disagreement with a statement and 5 is total agreement), which assesses a patient’s expectation that chamomile will relieve his/her primary anxiety symptoms and increase his/her coping abilities and vitality. Sample items include a patient’s relative agreement with the statements that with chamomile treatment “I will be able to cope with my anxiety better” and that “The symptoms of my anxiety will disappear.”1 The METE had good internal consistency in our sample (Cronbach’s alpha = 0.88). Patients completed the METE at baseline.
Mao Expectancy of Side Effects of Treatment (MESET)
The MESET is a 4-question patient-reported instrument rated on a scale of 1–5 (wherein 1 is total disagreement with a statement and 5 is total agreement) with three normally coded items and one reverse-coded item, which assesses a patient’s expectation that he or she will experience side effects during the course of a specific treatment (see online supplement). Sample items include a patient’s relative agreement with the statements “I am prone to the side effects of this type of therapy” and “I am not likely to experience any side-effects [of chamomile].” The MESET had adequate internal consistency in our sample (alpha = 0.72). Patients completed the MESET at baseline. Data were log-transformed to ameliorate a right skew.
Outcomes
GAD-7
The GAD-7 was the primary continuous outcome measure in the trial. The GAD-7 is a brief patient-report measure of GAD symptomatology as per DSM-IV criteria for the disorder (Herr et al., 2014). It has been shown to have good internal consistency, criterion validity, and sensitivity to treatment (Herr et al., 2014). Within this sample, the GAD-7 exhibited excellent internal consistency (alpha = 0.90). Patients reported on their symptoms using the GAD-7 at Baseline, Week 2, Week 4, and Week 8 of treatment.
Clinical Global Impression (CGI)
The CGI is a clinician-rated global measure of severity that correlates with other symptom severity outcome ratings (Guy, 1976). Clinical response was defined a priori (Mao et al., 2014) as a >50% reduction in baseline GAD-7 score and a final CGI-State score of ≤3 by Week 8. While arbitrary dichotomization of a continuous construct carries known statistical problems such as power issues (MacCallum et al., 2002; Royston et al., 2006), we included a secondary analysis of this response criterion as a perspective on the clinical importance of our findings (i.e., are expectancies associated with a patient reaching a clinically “good enough” outcome after treatment). In addition, clinical response comprised the primary outcome of the randomized phase of the parent clinical trial (Mao et al., 2014). Patients who dropped out of the trial were considered non-responders.
Hamilton Anxiety Rating Scale (HARS)
The HARS is a commonly used observer-administered measure of general psychological and somatic anxiety symptoms (Hamilton, 1959). HARS evaluations were performed by trained raters. Internal consistency for the HARS was good in this sample (alpha = 0.82).
Psychological General Well Being Index (PGWB)
The PGWB index is a self-report measure tapping into six quality of life domains: anxiety, depressed mood, positive well-being, self-control, general health, and vitality (Wiklund et al., 1991). Adequate psychometric reliability and validity are generally reported for the measure and its six subscales (Wiklund et al., 1991). Internal consistency for the PGWB was excellent in this sample (alpha = 0.95).
Treatment Emergent Symptom/Side Effect Interview
Side effects of treatment were measured at each study visit by clinicians via interview (NIMH, 1985). When possible, these side effects were confirmed by physician query and physical and laboratory findings. Information recorded included the severity of the side effect (i.e., mild, moderate, severe) and the relationship of the side effect to treatment (i.e., none, possible, probable, definite). The number of side effect reports related to treatment (possible, probable, or definite) was summed together into a count variable for a given patient.
Analyses
Within each variable, all values were standardized and centered for effect interpretability. For outcome or predictor variables with partial missing items, missing items were imputed based on available items using a random forest imputation algorithm (Stekhoven and Bühlmann, 2012).
In our primary analysis that served as our test of the expectancy-treatment outcome hypothesis, a linear mixed model was used to analyze the relationship between baseline expectancies and the slopes of GAD-7 change during chamomile treatment between Weeks 0 and 8. Time was linearly parameterized as the percentage of average change among trial completers occurring cumulatively between each time unit (e.g., between Weeks 0 and 2), ending at 1. An intention-to-treat approach was undertaken including all patients beginning treatment and providing a baseline expectancy assessment. Under the assumption that outcomes were missing at random (Rubin and Little, 2002), linear mixed models incorporated all available symptomatic measurements for a given patient to estimate person-specific slopes of symptomatic change, and fixed effects of expectancy on change slopes.
The effects of baseline GAD-7 on symptom change were included as a control covariate in the model, while the later change models controlled for both Week 2 GAD-7 and the amount of early change reported between Weeks 0 and 2. Secondary outcome analyses also used baseline scores as covariates with time.
As a secondary analysis of symptomatic outcomes, a logistic regression was run predicting the probability of being a clinical responder at Week 8 as a function of baseline expectancies, with baseline GAD-7 symptom levels and intake CGI score as covariates.
A negative binomial regression was used to analyze the relationship between baseline expectancies and counts of side effects possibly attributable to treatment a patient reported during the trial (Gardner et al., 1995). Primary side effect analyses were conducted with trial completers. Side effect expectancy (MESET) and dropout were unrelated (r = 0.01).
Potential confounding variables (i.e., confounded with expectancies) for each analysis were identified as any variable that, at the level of a statistical trend (p < 0.10), correlated with a given expectancy (see Table 1) and predicted the outcome of a given analysis. When confounding variables were identified, original models were re-run with additional controls for confounding variables. To further disentangle expectancies from general patient prognosis as predicted from demographic and clinical characteristics, we also built a model predicting GAD-7 change for each patient using a bootstrapped, AIC-based backward selection procedure (Barth et al., 2016). All baseline variables were allowed to predict GAD-7 change in the original, saturated model. A patient’s predicted GAD-7 change from the final model was tested as a covariate in the primary outcome analyses of GAD-7 change and clinical response. The final model included patient age, gender, the age at which the patient reported their first anxiety episode onset, the number of prior GAD drug treatments the patient had attempted, and the number of comorbid Axis-I psychiatric conditions the patient had.
Table 1.
Baseline Variables | No. (%) or Mean (SD) | Correlation with Efficacy Expectancy (METE) | Correlation with Side Effect Expectancy (MESET) |
---|---|---|---|
Age, y | 45·4 (15·4) | 0·27*** | −0·03 |
Gender (% Female) | 115 (66·9%) | 0·19* | −0·05 |
Race (% Caucasian) | 129 (75·0%) | −0·09 | −0·17* |
Ethnicity (% Hispanic) | 8 (4·7%) | 0·09 | −0·01 |
% Unemployed | 31 (17·3%) | 0·06 | −0·18* |
% Married | 62 (34·6%) | 0·01 | −0·04 |
Age at first GAD episode | 21.5 (15·4) | 0·12 | 0.16* |
Duration of current GAD episode (years) | 8.4 (13·9) | −0·04 | 0·5 |
% Current major depressive episode | 54 (31·4%) | 0·07 | 0·02 |
Number of psychiatric co-morbidities | 0·64 (0·84) | 0·08 | 0·07 |
Number of previous psychopharmacological treatments for GAD | 1·5 (1·7) | −0·03 | 0·02 |
GAD-7 | 15·1 (3·1) | 0·01 | 0·07 |
HARS | 14·7 (3·6) | −0·08 | 0·05 |
CGI-S | 4·16 (0·38) | −0·13 | −0·02 |
PGWB Total | 54·4 (14·0) | 0·07 | −0·19* |
METE | 12·5 (3·5) | NA | −0·04 |
MESET | 6·6 (2·7) | −0·04 | NA |
All reported correlations are Pearson correlations (for continuous variables), or point-biserial correlations coded such that positive values reflect the listed group having more of that expectancy (for categorical variables);
= p < ·05,
= p < ·001
CGI-S = Clinical Global Impression-State; GAD-7 = Generalized Anxiety Disorder 7-Item Scale; HARS = Hamilton Anxiety Rating Scale; METE = Mao Expectancy of Treatment Effects; MESET = Mao Expectancy of Side Effects of Treatment; PGWB = Psychological General Well Being
The sample size of 180 was determined by the parent study that seeks to evaluate whether long term chamomile use would prevent relapse as compared to placebo (Mao et al., 2014). We performed power estimation a priori for our expectancy investigation. With a sample size of 180 and assuming a response rate of 50%, we could detect with a power of 0.97 an effect of METE corresponding to an OR of 1.9 between patients with average METE and +1 standard deviation METE.
Results
Between March 2010 and September 2014, we enrolled a total of 179 participants. Seven did not have baseline expectancy assessments and were therefore not included in the analyses. Demographic and clinical information on the sample is reported in Table 1. The mean participant age was 45.4 (SD 15.4), and 115 (66.9%) of participants were female. The mean baseline GAD-7 score was 15.1 (SD 3.1), reflecting moderate to severe GAD symptoms. The average participant reported having been in their current GAD episode for 8.4 years, having had their first episode as a young adult (mean = 21.4), and having tried at least one prior treatment for their GAD (mean = 1.5). Approximately a third of the sample qualified for a current major depressive episode (n = 54; 31.4%).
Baseline efficacy expectancy and side-effect expectancy were unrelated to each other (r = −0.04, p = 0.620). Neither variable was associated with baseline symptom severity on the GAD-7, although some correlations were observed between expectancies and baseline data (see Table 1).
Expectancy and continuous treatment outcomes
Participants with higher efficacy expectancy (METE) scores at baseline experienced greater GAD-7 symptomatic improvement over eight weeks of treatment (β = −0.25 [95% CI: −0.39 to −0.11], t[170.1] = −3.46, p < 0.001; see Figure 1). Adjusting for age, the only potential confounder, higher efficacy expectancy remained a significant predictor (adjusted β = −0.21 [95% CI: −0.36 to −0.07], t[168.8] = −2.90, p = 0.004). Furthermore, when adjusting for predicted GAD-7 change from the prognostic model, higher efficacy expectancy remained significantly associated with increased GAD-7 improvement (adjusted β = −0.19 [95% CI: −0.33 to −0.05], t[165.8] = −2.59, p = 0.011).
Similar relationships were observed for continuous secondary outcome measures. The METE score predicted a larger reduction in anxiety symptoms by the observer-rated HARS (β = −0.20 [95% CI: −0.32 to −0.07], t[169.8] = −3.10, p = 0.002) and improved well-being as measured by the PGWB (β = 0.27 [95% CI: 0.14 to 0.40], t[157.3] = 4.06, p < 0.001).
On the contrary, baseline side effect expectancy measured by MESET did not predict GAD-7 change (β = 0.08 [95% CI: −0.07 to 0.22], t[166.8] = 1.03, p = 0.306), nor did it predict change in any secondary outcomes.
Expectancy and clinical response
Among 172 participants, 99 (58%) met clinical response criteria after eight weeks of treatment. Controlling for baseline symptom severity, patients with higher efficacy expectancy scores were more likely to be clinical responders at Week 8 (odds ratio = 1.69 [95% CI: 1.23 to 2.37], χ2(1) = 10.90, p = 0.002; illustrated in Figure 2). Adjusting additionally for predicted GAD-7 change, the relationship between expectancy and clinical response remained significant (p = 0.003).2
Conversely, there was no relationship between baseline side effect expectancy and response at Week 8 (adjusted OR = 1.08 [95% CI: 0.79 to 1.47], χ2(1) = 0.26, p = 0.613).
Expectancy and experience of treatment-related side effects
The average number of side effects potentially attributable to treatment reported during the course of eight weeks of treatment was 0.93 (SD = 1.52). Participants with higher side effect expectancy reported more side effect events that were potentially attributable to treatment (log expected count = 0.30 [95% CI: 0.05 to 0.56], SE = 0.13, p = 0.014; illustrated in Figure 3). Adjusting for patients’ employment status, the only potential confounder, higher MESET score still predicted side effect reports (adjusted log expected count = 0.26 [95% CI: 0.00 to 0.51], SE = 0.12, p = 0.038).3
By contrast, outcome expectancy measured by METE had no significant relationship to side effect reporting (log expected count = −0.05 [95% CI: −0.30 to 0.19], SE = 0.13, p = 0.684).
Discussion
In this open-label study of oral chamomile extract for generalized anxiety disorder, participants with higher expectancy for positive outcomes experienced greater reduction of anxiety symptoms and were more likely to meet criteria for clinically significant response to treatment. In addition, patients who expected more side effects reported a higher number of side effects during treatment, relative to those who expected very few side effects. Our results suggest that expectancies can be measured and predict quantifiable effects on treatment outcomes.
Moreover, the prediction of response from patients’ expectancies was not accounted for by any of the demographic, disease factor, and treatment history variables evaluated along with assessment of patients’ expectations. The predictive power of expectancies was significant even when simultaneously controlling for a patient’s predicted prognosis as estimated from a multivariate model including several baseline characteristics, suggesting that expectancies add unique value in predicting patient outcomes. Thus, expectancies may be a specific patient-level component of response to treatment that can be harnessed in clinical research and care (Rutherford et al., 2014b).
By observing a positive relationship between expectancies and anxiety outcomes, our findings converge with recent meta-analytic evidence suggesting that treatment expectancies contribute to active medication efficacy in anxiety clinical trials (Gaudiano et al., 2013; Rutherford et al., 2015; Rutherford et al., 2009a), and not only in trials treating depression (Leuchter et al., 2014; Rutherford et al., 2009b; Rutherford et al., 2016; Rutherford et al., 2014b). Expectancies may be an element of placebo and nocebo effects in active treatments for anxiety, a hypothesis which should be tested explicitly in future studies.
It is of note that our study found that response expectancies and side effect expectancies were not correlated with each other, and each predicted patients’ experiences during treatment in the relevant, but not the other, domain. Simply measuring a patient’s general “positive” versus “negative” expectancy would have missed these distinctions, which our findings suggest are clinically meaningful in predicting treatment responses. Future investigations might examine to what extent specific treatment expectancies bear differential predictions to outcomes, as compared to apparently more trait-like health optimism and pessimism, which might be less alterable in the short-term (Haanstra et al., 2015). Furthermore, treatment-specific expectancies may be further distinguished between belief that a treatment is generally effective in the population, and conviction that a treatment will be efficacious for you personally (Barth et al., 2016). This study only measured the latter type of expectancy.
Our study does not explain the mechanism(s) by which treatment-specific expectancies were reliably related to experiences on chamomile. One possibility is that outcome expectancies may influence whether and how patients progressively capitalize on the treatment they receive by making changes in their lives, such as by building on the support they receive from their clinician (Barnicot et al., 2014; Zilcha-Mano et al., 2015) or the functional gains produced by the drug. Patients with high side effect expectancy may tend to monitor their body states more vigilantly, with these patients thus being more prone to interpret bodily changes as side effects (Olatunji et al., 2007). However, more focused study is necessary to understand the specific mechanisms and time-course of different expectancy relationships.
Several limitations of the study should be noted. First, this was an observational study of the relationship between baseline expectancies and symptomatic outcomes and side effect burden. As expectancies were not directly manipulated, we cannot conclude that different baseline expectancies in any way “caused” particular experiences on the drug. On the other hand, the benefit of this design is that it suggests that naturalistic expectancies (i.e., expectancies held by the patient as they begin a treatment in a trial) may be predictive of experiences on medication. Second, this study investigated expectancies in a treatment for GAD, and future research is needed to both appropriately measure and examine the effects of expectancies in the treatment of other anxiety disorders. Third, these findings were tested in the context of a clinical trial, which may have enhanced positive expectancies and decreased negative expectancies, and provided a particularly convincing treatment frame in which expectancies could influence treatment course. External validity of these results would depend on their replication in a naturalistic clinic setting.
In addition, previous work in depression suggests that expectancies and “common factors” may be more predictive of change in placebo as compared to active psychopharmacological treatments (Leuchter et al., 2014; Zilcha-Mano et al., 2015), a pattern which cannot be assessed in the present trial due to a lack of placebo control during the acute phase. Similarly, due to the lack of a placebo or active-medication control in this phase of treatment, we cannot be sure that expectancy effects would be observed if patients were not taking chamomile specifically. Future randomized trials should examine the relationship between expectancies and outcomes on both on placebo and other types of active medication (e.g., Leuchter et al., 2014), to ascertain to what extent expectancies do or do not drive placebo and nocebo response in pure placebo and in the context of active drug effects.
Lastly, while expectancies were not confounded with any of the other characteristics we evaluated at baseline (such as number of prior drug treatments), there may have been unmeasured confounders. For example, prior exposures to medications or chamomile specifically may result in a conditioned response to future drugs or placebos (Stewart-Williams and Podd, 2004) that correlates with expectancies (i.e., “I did well on drugs previously, so I will do so again”). Patient experiences on a drug (e.g., unusual sensitivity to drug effects) could also shape a patient’s expectancies but ultimately bespeak more static, enduring characteristics of how they respond to a given medication class. Nevertheless, the potential for confounding relationships, while necessary to examine in future work, does not invalidate the pragmatic clinical and research use of expectancy measurements.
Given the observed predictive power of expectancies in both this and other investigations, researchers should consider measuring patient expectancies in clinical trials to enumerate and account for this potential specific component of treatment response and side effects (Leuchter et al., 2014). As many psychiatric trials fail to identify beneficial effects of active medication over placebo (Rutherford et al., 2014a; Rutherford and Roose, 2013; Turner et al., 2008), even with placebo responder wash-outs (Lee et al., 2004), researchers may consider controlling for expectancies in statistical analyses of treatment outcomes to clarify the unique contributions of the active drug. However, future experimental studies are necessary to help determine how best to incorporate expectancies into clinical trial design and analysis.
On the other hand, in the context of patient-centered care, patient expectancies may be an important factor to consider when a doctor and patient are choosing between treatments. In the treatment of GAD, most drugs and evidence-based psychosocial treatments have been found to have approximately equivalent efficacy on average (Mitte, 2005; Mitte et al., 2005; Westen and Morrison, 2001). Within this context, selecting a treatment based on the patient’s expectancies among different treatments may improve the patient’s probability of experiencing a clinical response, and minimize their side effect burden. Future intervention research should also investigate whether enhancing a patient’s treatment expectancies or aligning treatments with patients’ expectancy may augment routine clinical outcomes (Rutherford et al., 2013; Rutherford et al., 2016). Overall, our study suggests that patient expectancies are an easily quantifiable psychological factor that predicts both therapeutic outcomes and experience of side effects during psychopharmacological treatment for GAD.
Supplementary Material
Acknowledgments
We thank all of the clinical coordinators and research assistants for their dedication to clinical trial coordination, data collection and management. We also thank Dr. Kenneth Rockwell and the staff at the Penn Investigational Drug Service for their contrition. We thank the Swedish Herbal Institute for processing, producing, and providing the standardized oral chamomile extract product. We also thank our Data Safety and Monitoring Board members Drs. Andy Nierenberg and Sara Ratcliffe for volunteering their time and expertise. Sincere thanks go to all the patients who participated in this study.
[Funding Source]
This study is supported by grant from the National Institutes of Health/National Center for Complementary and Integrative Health R01 AT005074. This study is in part supported by an NCI P30 Cancer Center Core Grant to Craig B. Thompson. The funding agencies had no role in the design or conduct of the study. Dr. Mao has full access to all the data in the study and had final responsibility for the decision to submit for publication.
Footnotes
The measure is comparable to the Borkovec Expectancy Scale (BES) (Devilly & Borkovec, 2000). Unlike the BES, the METE’s items do not additionally inquire as to the credibility of the treatment, and the 4 expectancy items specifically inquire as to anxiety symptom success. Furthermore, as written the METE is intended to tap into cognitive expectancies for treatment, and thus does not contain the “feeling” factor of the BES.
Findings were similar when examining differentiation between response and nonresponse among only trial completers (METE p = 0.004).
One reviewer brought up the interesting possibility that prior experiences on active medications might both shape expectancies and relate to conditioned responses to pill-taking, resulting in an expectancy effect driven by past “active” drug effects (Stewart-Williams & Podd, 2004). A similar point is that past patient experiences of a drug could relate to the patient’s biological response profile to medication, which would then be correlated with an expectancy (e.g., “Since my last drug gave me side effects [partially due to my biological profile], I will expect this new drug to give me side effects”). An expectancy-outcome correlation in this instance would be (in part) an epiphenomenon of the patient’s biological responses to medication. We examined these potential confounders using information patients provided on the number of prior psychopharmacological treatments they had attempted for their GAD. Notably, neither type of expectancy significantly correlated with prior treatments. In addition, if prior drug exposures were strongly informing a patient’s chamomile expectancies, we might expect to detect heteroscedasticity in the correlation between expectancies and drug exposures (i.e., as prior treatments increases, we would see a greater frequency of relatively higher and lower expectancy values). Heteroscedasticity would be observed because patients with more past drug exposures would have more opportunities to form relatively more positive or negative expectancies, based on how successful these past drug trials were. Using the Breusch-Pagan test of heteroscedasticity, we did not find that patients with more past drug exposures had a larger spread of expectancies (p = 0.482 and p = 0.287 for METE and MESET, respectively).
We also examined whether simultaneously controlling for number of prior treatments in our three types of analyses notably attenuated the predictive value of expectancies. When additionally controlling for the relationship between prior treatments and continuous GAD-7 improvement, higher efficacy expectancy was still a significant predictor of GAD-7 improvement (p = 0.010). Similarly, adding prior treatments as a controlling covariate in the analyses of clinical response and side effect reports did not notably diminish the predictive power of expectancies (ps = 0.002 and 0.010, respectively).
Registration: Trial Number NCT01072344 at ClinicalTrials.gov
[Institutional Approval]
This study has been approved by the Institutional Review Board (Federalwide Assurance # 00004028) at the University of Pennsylvania. The protocol title is “Long-Term Chamomile Therapy of Generalized Anxiety Disorder” and the protocol number is 809780. All study participants have signed informed consent forms before participating in any study related activities.
[Conflicts of Interests]
All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that they have no conflicts of interest to report.
[Author Contributions]
JJM designed the study and managed trial and data collection processes. JRK performed data analysis and data interpretation. JA obtained the funding. QSL performed data management. IS evaluated patients. RD provided expertise in psychological assessment and statistical analyses. All authors participated in study design, writing, revision, and approving the final manuscript.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- Amsterdam JD, Li Y, Soeller I, Rockwell K, Mao JJ, Shults J. A randomized, double-blind, placebo-controlled trial of oral Matricaria recutita (chamomile) extract therapy for generalized anxiety disorder. Journal of clinical psychopharmacology. 2009;29(4):378–382. doi: 10.1097/JCP.0b013e3181ac935c. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Amsterdam JD, Shults J, Soeller I, Mao JJ, Rockwell K, Newberg AB. Chamomile (Matricaria recutita) may provide antidepressant activity in anxious, depressed humans: an exploratory study. Alternative therapies in health and medicine. 2012;18(5):44–49. [PMC free article] [PubMed] [Google Scholar]
- Barefoot JC, Brummett BH, Williams RB, Siegler IC, Helms MJ, Boyle SH, Clapp-Channing NE, Mark DB. Recovery expectations and long-term prognosis of patients with coronary heart disease. Archives of Internal Medicine. 2011;171(10):929–935. doi: 10.1001/archinternmed.2011.41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barnicot K, Wampold B, Priebe S. The effect of core clinician interpersonal behaviours on depression. Journal of Affective Disorders. 2014;167:112–117. doi: 10.1016/j.jad.2014.05.064. [DOI] [PubMed] [Google Scholar]
- Barth J, Schafroth L, Witt CM. Overlap and Differences Between Patient and Provider Expectations for Treatment Outcomes: The Case of Acupuncture. The journal of pain: official journal of the American Pain Society. 2016;17(6):685–693. doi: 10.1016/j.jpain.2016.01.477. [DOI] [PubMed] [Google Scholar]
- Bingel U. Avoiding nocebo effects to optimize treatment outcome. JAMA. 2014;312(7):693–694. doi: 10.1001/jama.2014.8342. [DOI] [PubMed] [Google Scholar]
- Borkovec TD, Costello E. Efficacy of applied relaxation and cognitive-behavioral therapy in the treatment of generalized anxiety disorder. Journal of consulting and clinical psychology. 1993;61(4):611–619. doi: 10.1037//0022-006x.61.4.611. [DOI] [PubMed] [Google Scholar]
- Brown LA, Wiley JF, Wolitzky-Taylor K, Roy-Byrne P, Sherbourne C, Stein MB, Sullivan G, Rose RD, Bystritsky A, Craske MG. Changes in self-efficacy and outcome expectancy as predictors of anxiety outcomes from the CALM study. Depress Anxiety. 2014;31(8):678–689. doi: 10.1002/da.22256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chambless DL, Tran GQ, Glass CR. Predictors of response to cognitive-behavioral group therapy for social phobia. Journal of anxiety disorders. 1997;11(3):221–240. doi: 10.1016/s0887-6185(97)00008-x. [DOI] [PubMed] [Google Scholar]
- Colagiuri B, Dhillon H, Butow PN, Jansen J, Cox K, Jacquet J. Does assessing patients’ expectancies about chemotherapy side effects influence their occurrence? Journal of pain and symptom management. 2013;46(2):275–281. doi: 10.1016/j.jpainsymman.2012.07.013. [DOI] [PubMed] [Google Scholar]
- Cuijpers P, Driessen E, Hollon SD, van Oppen P, Barth J, Andersson G. The efficacy of non-directive supportive therapy for adult depression: a meta-analysis. Clinical psychology review. 2012;32(4):280–291. doi: 10.1016/j.cpr.2012.01.003. [DOI] [PubMed] [Google Scholar]
- Enck P, Bingel U, Schedlowski M, Rief W. The placebo response in medicine: minimize, maximize or personalize? Nat Rev Drug Discov. 2013;12(3):191–204. doi: 10.1038/nrd3923. [DOI] [PubMed] [Google Scholar]
- First MS, Spitzer L, Gibbon M, Williams J. Structured Clinical Interview for DSM-IV-TR Axis I Disorders (SCID-I/P W/PSY Screen) Biometrics Research, New York State Psychiatric Institute; New York, NY: 2001. [Google Scholar]
- Gardner W, Mulvey EP, Shaw EC. Regression analyses of counts and rates: Poisson, overdispersed Poisson, and negative binomial models. Psychological Bulletin. 1995;118(3):392–404. doi: 10.1037/0033-2909.118.3.392. [DOI] [PubMed] [Google Scholar]
- Gaudiano BA, Hughes JA, Miller IW. Patients’ treatment expectancies in clinical trials of antidepressants versus psychotherapy for depression: a study using hypothetical vignettes. Comprehensive Psychiatry. 2013;54(1):28–33. doi: 10.1016/j.comppsych.2012.06.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guy W. DHEW publication ADM. 1976. Assessment manual for psychopharmacology; pp. 76–338. [Google Scholar]
- Haanstra TM, Tilbury C, Kamper SJ, Tordoir RL, Vliet Vlieland TPM, Nelissen RGHH, Cuijpers P, de Vet HCW, Dekker J, Knol DL, Ostelo RW. Can Optimism, Pessimism, Hope, Treatment Credibility and Treatment Expectancy Be Distinguished in Patients Undergoing Total Hip and Total Knee Arthroplasty? PLoS ONE. 2015;10(7):e0133730. doi: 10.1371/journal.pone.0133730. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hamilton M. THE ASSESSMENT OF ANXIETY STATES BY RATING. British Journal of Medical Psychology. 1959;32(1):50–55. doi: 10.1111/j.2044-8341.1959.tb00467.x. [DOI] [PubMed] [Google Scholar]
- Herr NR, Williams JW, Jr, Benjamin S, McDuffie J. Does this patient have generalized anxiety or panic disorder?: The Rational Clinical Examination systematic review. JAMA. 2014;312(1):78–84. doi: 10.1001/jama.2014.5950. [DOI] [PubMed] [Google Scholar]
- Hofmann SG, Smits JA. Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized placebo-controlled trials. The Journal of clinical psychiatry. 2008;69(4):621–632. doi: 10.4088/jcp.v69n0415. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Horing B, Weimer K, Muth ER, Enck P. Prediction of placebo response: A systematic review of the literature. Frontiers in Psychology. 2014;5:1079. doi: 10.3389/fpsyg.2014.01079. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Krell HV, Leuchter AF, Morgan M, Cook IA, Abrams M. Subject expectations of treatment effectiveness and outcome of treatment with an experimental antidepressant. The Journal of clinical psychiatry. 2004;65(9):1174–1179. doi: 10.4088/jcp.v65n0904. [DOI] [PubMed] [Google Scholar]
- Lee S, Walker JR, Jakul L, Sexton K. Does elimination of placebo responders in a placebo run-in increase the treatment effect in randomized clinical trials? A meta-analytic evaluation. Depress Anxiety. 2004;19(1):10–19. doi: 10.1002/da.10134. [DOI] [PubMed] [Google Scholar]
- Leuchter AF, Hunter AM, Tartter M, Cook IA. Role of pill-taking, expectation and therapeutic alliance in the placebo response in clinical trials for major depression. The British journal of psychiatry: the journal of mental science. 2014;205(6):443–449. doi: 10.1192/bjp.bp.113.140343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- MacCallum RC, Zhang S, Preacher KJ, Rucker DD. On the practice of dichotomization of quantitative variables. Psychological methods. 2002;7(1):19–40. doi: 10.1037/1082-989x.7.1.19. [DOI] [PubMed] [Google Scholar]
- Mao JJ, Li QS, Soeller I, Rockwell K, Xie SX, Amsterdam JD. Long-term chamomile therapy of generalized anxiety disorder: A study protocol for a randomized, double-blind, placebo-controlled trial. Clinical Trials. 2014;4:5. doi: 10.4172/2167-0870.1000188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mao JJ, Xie SX, Bowman MA. Uncovering the expectancy effect: the validation of the acupuncture expectancy scale. Alternative therapies in health and medicine. 2010;16(6):22–27. [PMC free article] [PubMed] [Google Scholar]
- Mitte K. Meta-analysis of cognitive-behavioral treatments for generalized anxiety disorder: a comparison with pharmacotherapy. Psychol Bull. 2005;131(5):785–795. doi: 10.1037/0033-2909.131.5.785. [DOI] [PubMed] [Google Scholar]
- Mitte K, Noack P, Steil R, Hautzinger M. A meta-analytic review of the efficacy of drug treatment in generalized anxiety disorder. Journal of clinical psychopharmacology. 2005;25(2):141–150. doi: 10.1097/01.jcp.0000155821.74832.f9. [DOI] [PubMed] [Google Scholar]
- Mondaini N, Gontero P, Giubilei G, Lombardi G, Cai T, Gavazzi A, Bartoletti R. Finasteride 5 mg and sexual side effects: how many of these are related to a nocebo phenomenon? Journal of Sexual Medicine. 2007;4(6):1708–1712. doi: 10.1111/j.1743-6109.2007.00563.x. [DOI] [PubMed] [Google Scholar]
- Mora MS, Nestoriuc Y, Rief W. Lessons learned from placebo groups in antidepressant trials. Philosophical Transactions of the Royal Society B. 2011;366:1879–1888. doi: 10.1098/rstb.2010.0394. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nestoriuc Y, Orav EJ, Liang MH, Horne R, Barsky AJ. Prediction of nonspecific side effects in rheumatoid arthritis patients by beliefs about medicines. Arthritis Care Research. 2010;62(6):791–799. doi: 10.1002/acr.20160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Newman MG, Fisher AJ. Expectancy/Credibility Change as a Mediator of Cognitive Behavioral Therapy for Generalized Anxiety Disorder: Mechanism of Action or Proxy for Symptom Change? International journal of cognitive therapy. 2010;3:245–261. doi: 10.1521/ijct.2010.3.3.245. [DOI] [PMC free article] [PubMed] [Google Scholar]
- NIMH. Treatment emergent symptoms scale. Psychopharmacological Bulletin. 1985;21:1069–1073. [Google Scholar]
- Olatunji BO, Deacon BJ, Abramowitz JS, Valentiner DP. Body Vigilance in Nonclinical and Anxiety Disorder Samples: Structure, Correlates, and Prediction of Health Concerns. Behavior Therapy. 2007;38(4):392–401. doi: 10.1016/j.beth.2006.09.002. [DOI] [PubMed] [Google Scholar]
- Price M, Anderson P, Henrich CC, Rothbaum BO. Greater expectations: using hierarchical linear modeling to examine expectancy for treatment outcome as a predictor of treatment response. Behav Ther. 2008;39(4):398–405. doi: 10.1016/j.beth.2007.12.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rojas-Mirquez JC, Rodriguez-Zuniga RJM, Bonilla-Escobar FJ, Garcia-Perdomo HA, Petkov M, Becerra L, Borsook D, Linnman C. Nocebo effect in randomized clinical trials of antidepressants in children and adolescents: Systematic review and meta-analysis. Frontiers in Behavioral Neuroscience. 2014;8:375. doi: 10.3389/fnbeh.2014.00375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006;25(1):127–141. doi: 10.1002/sim.2331. [DOI] [PubMed] [Google Scholar]
- Rubin DB, Little RJA. Statistical analysis with missing data. 2. Wiley; New York, NY: 2002. [Google Scholar]
- Rutherford BR, Bailey VS, Schneier FR, Pott E, Brown PJ, Roose SP. INFLUENCE OF STUDY DESIGN ON TREATMENT RESPONSE IN ANXIETY DISORDER CLINICAL TRIALS. Depression and Anxiety. 2015;32(12):944–957. doi: 10.1002/da.22433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rutherford BR, Marcus SM, Wang P, Sneed JR, Pelton G, Devanand D, Duan N, Roose SP. A randomized, prospective pilot study of patient expectancy and antidepressant outcome. Psychological medicine. 2013;43(5):975–982. doi: 10.1017/S0033291712001882. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rutherford BR, Pott E, Tandler JM, Wall MM, Roose SP, Lieberman JA. Placebo response in antipsychotic clinical trials: a meta-analysis. JAMA Psychiatry. 2014a;71(12):1409–1421. doi: 10.1001/jamapsychiatry.2014.1319. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rutherford BR, Roose SP. A model of placebo response in antidepressant clinical trials. American Journal of Psychiatry. 2013;170:723–733. doi: 10.1176/appi.ajp.2012.12040474. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rutherford BR, Rose SA, Sneed JR, Roose SP. Study design affects participant expectations: a survey. Journal of clinical psychopharmacology. 2009a;29(2):179–181. doi: 10.1097/JCP.0b013e31819a9181. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rutherford BR, Sneed JR, Roose SP. Does study design influence outcome?. The effects of placebo control and treatment duration in antidepressant trials. Psychotherapy and psychosomatics. 2009b;78(3):172–181. doi: 10.1159/000209348. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rutherford BR, Wager TD, Roose SP. Expectancy and treatment of depression: A review of experimental methodology and effects on patient outcome. Current Psychiatry Reviews. 2010;6(1):1–10. doi: 10.2174/157340010790596571. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rutherford BR, Wall MM, Brown PJ, Choo T-H, Wager TD, Peterson BS, Chung S, Kirsch I, Roose SP. Patient Expectancy as a Mediator of Placebo Effects in Antidepressant Clinical Trials. American Journal of Psychiatry. 2016 doi: 10.1176/appi.ajp.2016.16020225. appi.ajp.2016.16020225. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rutherford BR, Wall MM, Glass A, Stewart JW. The role of patient expectancy in placebo and nocebo effects in antidepressant trials. Journal of Clinical Psychiatry. 2014b;75(1):1040–1060. doi: 10.4088/JCP.13m08797. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Safren SA, Heimberg RG, Juster HR. Clients’ expectancies and their relationship to pretreatment symptomatology and outcome of cognitive-behavioral group treatment for social phobia. Journal of consulting and clinical psychology. 1997;65(4):694–698. doi: 10.1037//0022-006x.65.4.694. [DOI] [PubMed] [Google Scholar]
- Sotsky SM, Glass DR, Shea MT, Pilkonis PA, Collins JF, Elkin I, Watkins JT, Imber SD, Leber WR, Moyer J, et al. Patient predictors of response to psychotherapy and pharmacotherapy: findings in the NIMH Treatment of Depression Collaborative Research Program. The American journal of psychiatry. 1991;148(8):997–1008. doi: 10.1176/ajp.148.8.997. [DOI] [PubMed] [Google Scholar]
- Stekhoven DJ, Bühlmann P. MissForest—non-parametric missing value imputation for mixed-type data. Bioinformatics. 2012;28(1):112–118. doi: 10.1093/bioinformatics/btr597. [DOI] [PubMed] [Google Scholar]
- Stewart-Williams S, Podd J. The placebo effect: dissolving the expectancy versus conditioning debate. Psychol Bull. 2004;130(2):324–340. doi: 10.1037/0033-2909.130.2.324. [DOI] [PubMed] [Google Scholar]
- Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. The New England journal of medicine. 2008;358(3):252–260. doi: 10.1056/NEJMsa065779. [DOI] [PubMed] [Google Scholar]
- Vase L, Riley JL, Price DD. A comparison of placebo effects in clinical analgesic trials versus studies of placebo analgesia. Pain. 2002;99:443–452. doi: 10.1016/S0304-3959(02)00205-1. [DOI] [PubMed] [Google Scholar]
- Weimer K, Colloca L, Enck P. Placebo effects in psychiatry: mediators nnd moderators. Lancet Psychiatry. 2015;2(3):246–257. doi: 10.1016/S2215-0366(14)00092-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Westen D, Morrison K. A multidimensional meta-analysis of treatments for depression, panic, and generalized anxiety disorder: an empirical examination of the status of empirically supported therapies. Journal of consulting and clinical psychology. 2001;69(6):875–899. [PubMed] [Google Scholar]
- Westra HA, Dozois DJ, Marcus M. Expectancy, homework compliance, and initial change in cognitive-behavioral therapy for anxiety. Journal of consulting and clinical psychology. 2007;75(3):363–373. doi: 10.1037/0022-006X.75.3.363. [DOI] [PubMed] [Google Scholar]
- Wiklund I, Holst J, Karlberg J, Mattsson LA, Samsioe G, Sandin K, Uvebrant M, vonSchoultz B. A new methodological approch to the evaluation of quality of life in postmenopausal women. Maturitas. 1991;14:211–224. doi: 10.1016/0378-5122(92)90116-l. [DOI] [PubMed] [Google Scholar]
- Wise RA, Bartlett SJ, Brown ED, Castro M, Cohen R, Holbrook JT, Irvin CG, Rand CS, Sockrider MM, Sugar EA ALAACR Centers. Randomized trial of the effect of drug presentation on asthma outcomes: the American Lung Association Asthma Clinical Research Centers. Journal of Allergy and Clinical Immunology. 2009;124(3):436–444. doi: 10.1016/j.jaci.2009.05.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zilcha-Mano S, Roose SP, Barber JP, Rutherford BR. Therapeutic alliance in antidepressant treatment: cause or effect of symptomatic levels? Psychotherapy and psychosomatics. 2015;84(3):177–182. doi: 10.1159/000379756. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.