Comparison of psychological placebo and waiting list control conditions in the assessment of cognitive behavioral therapy for the treatment of generalized anxiety disorder: a meta-analysis

Abstract

Background

There is ongoing debate about the efficacy of placebos in the treatment of mental disorders. In randomized control trials (RCTs) about the treatment of generalized anxiety disorder, the administration of a psychological placebo or placement on a waiting list are the two most common control conditions. But there has never been a systematic comparison of the clinical effect of these different strategies.

Aim

Compare the change in symptom severity among individuals treated with cognitive behavioral therapy, provided a psychological placebo, or placed on a waiting list using data from RCTs on generalized anxiety disorder.

Methods

The following databases were searched for RCTs on generalized anxiety disorder: PubMed, PsycInfo, EMBASE, The Cochrane Library, CNKI, Chongqing VIP, Wanfang, Chinese Biological Medical Literature Database, and Taiwan Electronic Periodical Services. Studies were selected based on pre-defined inclusion and exclusion criteria and the quality of each included study – based on the risk of bias and the level of evidence – was formally assessed. Meta-analysis was conducted using RevMan5.3 and network meta-analyses comparing the three groups were conducted using R.

Results

Twelve studies with a combined sample size of 531 were included in the analysis. Compared to either control method (placebo or waiting list), cognitive behavioral therapy was more effective for generalized anxiety disorder. Provision of a psychological placebo was associated with a significantly greater reduction of symptoms than placement on a waiting list. Eight of the studies were classified as ‘high risk of bias’, and the overall level of evidence was classified as ‘moderate’, indicating that further research could change the overall results of the meta-analysis.

Conclusions

RCTs about the treatment of generalized anxiety disorders are generally of moderate quality; they indicate the superiority of CBT but the results cannot, as yet, be considered robust. There is evidence of a non-negligible treatment effect of psychological placebos used as control conditions in research studies. This effect should be considered when designing and interpreting the results of randomized controlled trials about the effectiveness of psychotherapeutic interventions.

1. Background

Generalized anxiety disorder (GAD) is a common anxiety disorder with an estimated lifetime prevalence between 4.3% and 5.9%.^[1] The main symptom of GAD is chronic worrying about non-specific matters that is both irrational and uncontrollable.^[2] These worries interfere with an individual’s work,interpersonal relationships,and mental and physical wellbeing.^[3] Cognitive behavioral therapy (CBT) for GAD has better sustained clinical effectiveness than medication,so it is the first choice for treatment.^[2,4]

Recently,the importance of proper controls has been emphasized in discussions about the design and conduct of clinical trials about the effectiveness of psychological therapies.^[5] However,there has been no consensus about which type of control condition should be used.^[6] A meta-analysis about the treatment of GAD by Hunot and colleagues^[7] found that CBT was superior to placebo and to treatment as usual (TAU),but they did not discuss the potential treatment effect of the placebos employed. The placebo effect is commonly ignored when reporting clinical trials,^[8] which can lead to an overestimation or underestimation of the treatment effect of the intervention under study.^[9,10] Some studies have found that a well-designed psychological placebo can achieve a similar treatment effect to that achieved by a formal psychological treatment.^[4,11] Research about the placebo effect in the treatment of mental disorders has found that placebos can produce real physiological changes that may be therapeutic.^[5,12] The presumed mechanism of action of the placebo is via the effects of anticipation,hope,faith,and (sometimes) the interaction between the provider of the placebo and the individual.^[8]

Wampold and colleagues^[6] summarized the commonly used placebos employed in studies of psychological interventions. Active placebos used as control conditions in intervention trials include alternative treatments,supportive counseling,and the ‘credible attention’ placebo.^[4,6] Non-placebo control conditions have been used in other studies,including no treatment at all,placement on a waiting list,and ‘treatment as usual’ (TAU).^[8] Furukawa and colleagues^[5] argued convincingly that the wide variation of control conditions and the variety in TAU conditions across studies seriously undermines the comparability of the results of studies. For example,applied relaxation is seen as a special type of psychotherapy^[2,13,14] so it is no longer considered an appropriate control condition for psychotherapy trials. Currently,the most widely used and widely accepted control conditions in RCTs of psychotherapy are the administration of a psychological placebo and placement on a waiting list.

Several existing meta-analyses of CBT for anxiety disorders^{[1,4,15,16,17,18,19]} report that CBT is more effective^[1,17,18] and has better long-term outcomes^[15,19]than administration of a psychological placebo or place-ment on a waiting list. However,these studies do not compare the relative effectiveness of administering a psychological placebo and placement on a waiting list.^[4,5,11] Differential effectiveness between these two commonly employed control conditions could affect the interpretation of the results of previous meta-analyses. If these control conditions have different therapeutic effects this should be considered when designing RCTs of psychotherapeutic interventions and used as a control factor when comparing or combining the results across trials.

To directly address this issue,we conducted a meta-analysis of RCTs of the treatment of GAD that simultaneously compares the therapeutic effect of CBT,administration of a psychological placebo,and placement on a waiting list.

2. Methods

2.1. Search strategy

The process of selecting articles for inclusion in the meta-analysis is shown in the flowchart in Figure 1. We adopted a four-step search procedure. First,we searched the Cochrane Library,PubMed,and other databases for systematic reviews and meta-analyses about the treatment of GAD. Second,we identified the original studies included in these systematic reviews and analyzed the title,abstract,keywords,and MeSH terms to develop our own search strategy. Third,we conducted our searches and obtained the full-text for all articles that were considered relevant. Fourth,we checked the reference lists of included studies.

Figure 1. Identification of papers for meta-analysis.

The following Boolean strategy was used to search PubMed,PsycInfo,EMBASE,the Cochrane Library,Chinese National Knowledge Infrastructure,Chongqing VIP database for Chinese Technical Periodicals,WANFANG DATA,Chinese Biological Medical Literature Database,and Taiwan Electronic Periodical Services (TEPS) for studies published prior to October 20 of 2014: (“generalized anxiety disorder” OR “GAD” OR “anxiety disorder”) AND (“cognitive behavioral treatment” OR “cognitive behavioral therapy” OR “CBT”) AND (“randomized controlled trial” OR “controlled clinical trial” OR “randomized” OR “randomly” OR “trial”) AND (“placebo psychotherapy” OR “placebo effect” OR “wait list” OR “waiting list”). References of previous reviews and meta-analyses were hand checked. We did not contact the authors to request further data or include unpublished data.

2.2. Inclusion and exclusion criteria

Inclusion criteria were that the study: (a) used a randomized controlled trial design; (b) included individuals between 18 and 70 years of age; (c) determined the GAD diagnosis based on criteria recommended by the 3^rd or 4^th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-III^[20]or DSM-IV^[21]),the 10^th edition of the International Classification of Diseases (ICD-10^[22])or the 3^rd edition of the Chinese Classification of Mental Disorders (CCMD^[23]); (d) used CBT as the main treatment; (e) conducted CBT as face-to-face therapy either individually or in a group; and (f) used a psychological placebo or placement on a waiting list as the control condition.

Potential studies were excluded if they met any of the following criteria: (a) included in-patients; (b) only involved one component of CBT (e.g.,cognitive therapy,behavioral therapy,exposure therapy,or relaxation therapy); (c) CBT was delivered via the internet or telephone or was too brief (<8 sessions); (d) did not report or control for the concurrent use of medications; (e) included individuals with schizophrenia or psychosis; or (f) did not provide sufficient information to calculate the effect size.

2.2.1. Description of CBT,psychological placebo (PP),and waiting list (WL)

The core concepts of CBT include reconceptualization,relaxation training,  exposure therapy,and stress inoculation training.^[24,25]

PP usually refers to a non-directional and neutral discussion.^[22] PP is not considered an active treatment although the recipient is actively involved. Other factors related to the administration of PP should be similar to those for the administration of CBT; these include the number of sessions,duration of sessions,and physical environment of the sessions.^[5,10] Other relevant control variables include regular contact,emergency management,and the discussion of psychological problems.^[4] The effective components of CBT including exposure exercises and cognitive correction should not be included in PP.^[5,11] PP is usually labeled ‘supportive consultation’ or ‘discussion groups’.

Individuals in the WL condition do not receive any type of treatment during the study period. Instead,they are informed that they will receive CBT after the study and they are regularly contacted by the research team who monitor their symptoms.

2.3. Outcome measures

Anxiety symptoms in the included studies were measured by commonly used clinical assessments including the Hamilton anxiety scale (HAMA),^[26] Beck Anxiety Inventory (BAI),^[27] State-Trait Anxiety Inventory (STAT-T),^[28] Penn State Worry Questionnaire (PSWQ),^[29]and Zung Self-Rating of Anxiety Scale.^[30] The primary outcome measure used in our meta-analysis is the change in scores of these scales. If multiple assessments were reported in a paper the results for HAMA were used as the primary outcome measure; if HAMA was not used,the priority was given to the scores of BAI,STAT-T,PSWQ,or SAS,respectively. The secondary outcome measure considered was the effectiveness of CBT,psychological placebo,and placement on a waiting list; that is,the proportion of individuals in each group whose symptoms ‘improved’,as defined by the investigators in each study.

2.4. Evaluation of the quality of included papers

The quality of included studies was assessed based on the 5.1.0 revision of the Cochrane Handbook for Systematic Reviews of Interventions^[31]and the Grading of Recommendations Assessment,Development and Evaluation (GRADE) tool.^[32] Two authors independently conducted the screening and quality evaluation.

2.4.1. Assessment for risk of biases

We assessed the risk of biases for each included study using the Cochrane Collaboration risk of biases assessment,based on the adequacy of the following aspects of each study: (a) randomization; (b) allocation concealment; (c) blinding of participants,treatment providers,and evaluators; (d) completeness of data; (e) risk of selective reporting; (f) other sources of biases. We also assessed five additional aspects for risks of biases in psychotherapy;^[5] these include the allegiance of researchers (conflict of interest),the use of standardized treatment protocols,the qualifications of the treatment provider,the allegiance of the treatment provider,and the use of quality control measures (e.g.,whether the treatment sessions were recorded or monitored). These thirteen items were independently assessed by two raters and any disagreements about ratings were resolved by consensus or,if necessary,referral to a third rater. If three or more of these items were rated as ‘high risk of bias’ the overall rating of risk of bias for the article was also rated as ‘high’; if less than three items were rated as ‘high’ the overall risk of bias for the article was classified as ‘low’.

2.4.2. Assessment of the level of evidence

We categorized the level of evidence into high,medium,low,and very low based on the GRADE criteria which are based on characteristics of each study including the participants,randomization,blinding,drop-out rate,treatment compliance,effectiveness,side effects,and the completeness and accuracy of reporting.

2.5. Data extraction

Two authors (ZZP and JJL) independently screened articles according to the inclusion and exclusion criteria. When there was a disagreement about whether or not a paper should be included that could not be resolved by discussion between the two screeners,a third opinion (ZL) was sought. EndNote X6 was used to manage the literature searches. Two authors (ZZP and ZL) independently extracted data using a pre-designed data-entry form. When their disagreement about an item could not be resolved by discussion,a third opinion (JJL) was sought to make the final determination about the information to be included in the analysis.

The data extraction form included: (a) general characteristics of the study (e.g.,the name of the first author,study year,title,and country); (b) demographic descriptions of the sample (e.g.,average age of the sample,duration of disease,level of education,male-to-female ratio,type of intervention,type of control,sample size,inclusion and exclusion criteria,target population,source population,and the format of CBT [i.e.,individual or group therapy]); and (c) study data (e.g.,number of people who completed the study,baseline measures,duration of treatment,measure-ment tools,primary and secondary outcome measures,number of drop outs,and duration of follow-up).

2.6. Statistical analysis

The Cochrane RevMan 5.3 and R 3.1.0 were used for data analysis. The p-value of the Cochrane Q test and I² were used to determine the level of heterogeneity. If p>0.1 and I²<50%,the heterogeneity was not considered statistically significant,so a fixed-effect model was used to generate the pooled estimate. On the other hand,if either p≤0.1 or I²≥50% the heterogeneity was considered statistically significant,so a random-effect model was used to generate the pooled estimate. Reasons for heterogeneity were explored using subgroup analysis and sensitivity analysis.

For continuous variables,the standardized mean difference (SMD) and its 95% confidence interval were generated for the pooled effect. The fixed-effect model employs the inverse variance method and the random-effect model employs the Der Simonian-Laird method. The statistical significance of the pooled effect was tested using the Z-test. Forest plots were generated to visualize the findings. If the extracted data could not be pooled using the above models,we provided a description of the results. For binary variables,the relative risk (RR) was used to generate the pooled effect. A funnel plot was used to evaluate publication bias.

There is no direct comparison between the placebo and waiting list groups,so we conducted an indirect comparison which compares PP and WL by comparing the difference between PP and CBT to the difference between WL and CBT. This network meta-analysis method uses the ‘net meta’ package (version 0.6-0) of R 3.1.0 (http://cran.r-project.org/web/packages/netmeta/index.html). Separate analyses were conducted using CBT or PP as the reference group. In the analysis with CBT as the reference group,the standard mean differences between CBT versus PP and between CBT versus WL were estimated; in the analysis with PP as the reference group,the standard mean differences between PP versus CBT and between PP versus WL were estimated.

3. Results

3.1. Search results and characteristics of included studies

The search yielded 413 publications. After reading the titles and abstracts,97 studies were left after eliminating duplicates and irrelevant studies. After reading the full-text of these 97 articles,30 were considered relevant and 12 were included in the analysis based on our inclusion and exclusion criteria.^{[14,25,38,39,40,41,42,43,44,45,46,47]} Among the 18 excluded studies,3 used medication as the control condition; 3 were for other types of anxiety disorders; 5 were about internet-delivered treatment,self-help,or brief CBT; 2 used another type of psychotherapy as the control condition; 2 were systematic reviews or meta-analyses; 1 did not randomize subjects; and 2 did not provide sufficient data for analysis.

Disagreement between the two raters in the assessment of inclusion and exclusion criteria occurred for 8 articles. The specific criteria and studies about which the raters disagreed (requiring a consensus decision or consultation with the third rater) included: (a) whether or not the treatment met the agreed definition of CBT^{[19,48,49,50]}; (b) whether or not a study employed a medication control group^[18,51]; and (c) whether or not a report provided sufficient information for the desired analysis.^[52,53]

The 12 included studies^{[14,25,38,39,40,41,42,43,44,45,46,47]} were published between 1991 and 2013 in English and all of them used DSM-III or DSM-IV diagnostic criteria. Three studies^[14,44,45] compared CBT with PP,8 studies^{[25,38,39,40,41,42,43,47]} compared CBT with WL,and 1 study^[46] compared CBT with both PP and WL. (When pooling results for the effectiveness of CBT versus control in the 12 studies,the results of CBT versus PP were used for the study that had two control groups.^[46]) Four studies were conducted in elderly individuals with GAD.^{[43,44,45,46]} Five studies used group CBT.^{[25,40,44,45,46]}The minimum duration of GAD at the time of entry into the study was 13 months.^[25,38]In most studies the number of CBT sessions rangedfrom 10 to 16 sessions. The types of PP used in these studies included nondirective therapy,^[14] minimal contact control,^[45] supportive psychotherapy,^[44] and discussion groups.^[46] The combined sample size was 531 with 250 in the CBT groups,99 in the PP groups,and 182 in the WL groups. Table 1 provides a description of the basic characteristics of included studies.

Table 1.

Characteristics of included articles

study	groups	n	age mean (sd)	months ill mean (sd)	treatment format	treatment sessions	treatment provider	outcome measures
Beasley 1992[38]	COM	12	42.5	>24	Individual	1h per week for 15 weeks	senior doctoral and staff psychologists	HAMA STAI-Trait
	WL	20	40.4		----	----
Borkovec 1993[14]	CBT	19	37.5 (13.1)	17.1 (17.2)	Individual	12 sessions 2 per week (4 for 1.5h,8 for 1h)	2 experienced therapists; 2 advance clinical graduate students	HAMA, STAI PSWQ
	AR	18
	ND	18
Butler 1991[39]	CBT	19	32.6	36.8	Individual	mean of 10.7 sessions	clinical psychologists with 8 years	HAMA BAI STAI-T
	BT	19	34.1	34.1		mean of 10.6 sessions	experience
	WL	19	38.3	37.6	----	----	----
Dugas 2003[25]	CBT	25	41.2 (9.2)	16.9 (15.2)	Group	2h per week for 14 weeks	6 licensed psychologists trained in CBT	BAI PSWQ;
	WL	27			----	telephoned every 3 weeks	----
Dugas 2010[40]	CBT	23	38.5 (12.0)	13.9 (16.7)	Group	1h per week for 12 weeks	1 licensed psychologist	PSWQ STAI-T ADIS-R
	AR	22
	WL	20			----	----	----
Ladouceur 2010[41]	CBT	14	39.7 (10.8)	15.6 (13.3)	Individual	1h per week for 16 weeks	4 licensed psychologists trained in CBT	BAI PSWQ WAQ;
	WL	12				----	----
Linden 2005[42]	CBT	36	43.3 (12.0)	NA	Individual	25 50min sessions over 14.5 weeks	12 cognitive behavior therapists	HAMA STAI
	WL/CCG	36			----	----	----
Mohlman 2013[43]	CBT	15	66.8 (4.5)	NA	Individual	NA	masters or higher graduate students	PSWQ GAD-Q
	WL	13	67.1 (5.0)		----	----	----
Stanley 1996[44]	CBT	26	68.3 (6.6)	35.5 (24.8)	Group	1.5h per week for 14 weeks	4 advanced graduate students trained in CBT	HAMA PSWQ STAI-T
	SP	20
Stanley 2003[45]	CBT	29	66.2 (5.2)	NA	Group	1.5h per week for 15 weeks	4 postdoctoral fellows and 1 graduate student	HAMA PSWQ STAI-T
	MCC	35
Wetherell 2003[46]	CBT	26	67.1 (8.2)	29.4 (28.7)	Group	1.5h per week for 12 weeks	4 advanced doctoral student	HAMA BAI PSWQ;
	DG	26
	WL	23			----	----	----
Zinbarg 2007[47]	CBT	8	41.94 (12.2)	23.4 (13.9)	Individual	12 60-75min sessions over 16 weeks	4 doctoral-level therapists	BAI CSR PSWQ
	WL	10			----	----
ADIS-IV,Anxiety Disorders Interview Schedule[33]				DG,discussion group			ND,Nondirective
AR, applied relaxation				GAD-Q, Generalized Anxiety Disorder			PSWQ, Penn State Worry
BAI, Beck Anxiety Inventory				Questionnaire for DSM-IV[35]			Questionnaire
BT, behavior therapy				HAMA, Hamilton Anxiety Rating Scale			SP, supportive psychotherapy
CCG, contact control group				HAMD, Hamilton Depression Rating Scale[36]			STAI-T, State-Trait Anxiety Inventory-
COM, relaxation plus cognitive restructuring				MCC, minimal contact control			Trait Anxiety subscale
CSR, Clinician’s Severity Rating from the Anxiety				NA, not available			WL, waiting list
Disorders Interview Schedule for DSM-IV[34]

3.2. Comparison of treatment effect

As shown in the forest plot (Figure 2),there is statistically significant heterogeneity in the results across the 12 studies (I²=60%,p≤0.01). Therefore,the random-effect model was used to generate the pooled estimate. Compared to controls,CBT is more effective in reducing anxiety symptoms (Z=5.00,p≤0.05,SMD=-0.76,95%CI=-1.06,-0.46).

Subgroup analysis based on the type of control group was conducted to identify potential sources of heterogeneity. Results are heterogeneous for the four studies that utilized PP as the control method (I²=67%,p=0.03); the random-effect model identified a better treatment effect for CBT compared to PP (SMD=-0.53,95%CI=-1.03,-0.02). Sensitivity analysis found that after excluding the most extreme result (the 1996 Stanley study^[44]) results from the remaining three studies were homogeneous (I²=49%,p=0.14); the pooled SMD from the fixed-effect model for these three studies also showed a superior effectiveness for CBT compared to PP (SMD= -0.71,95%CI=-1.04,-0.38).

Figure 2. Comparison of mean (sd) anxiety scores at the end of the intervention between persons with generalized anxiety disorder who received cognitive behavioral therapy (CBT),psychological placebo treatment (PP),or placement on a waiting list (WL).

Results from the nine studies that used WL as the control are borderline heterogeneous (I²=50%,p=0.04); the pooled SMD estimate using a random-effect model was -0.87 (95%CI=-1.20,-0.55),indicating better treatment effect of CBT. Sensitivity analysis shows that after excluding the most extreme results (the 2013 Mohlman study^[43]) results from the remaining eight studies are homogeneous (I²=9%,p=0.36); the pooled SMD in the remaining eight studies using the fixed-effect model also showed superior effectiveness of CBT compared to WL (SMD=-0.74. 95%CI=-0.97,-0.52).

3.3. Comparison of secondary outcomes

Three studies^[40,41,43] did not report the number (or proportion) of cases where treatment was not effective (i.e.,‘treatment failures’). Figure 3 shows results from the remaining nine studies. These findings are heterogeneous (I²=67%,p≤0.01). Estimates from the random-effect model shows that individuals in the CBT group were less likely to experience a treatment failure by the end of the study (i.e.,more likely to have achieved each study’s definition of ‘effective treatment’) than individuals in the control group (RR=0.68,95%CI=0.53,0.87).

Figure 3. Comparison of the number of treatment failures among persons with generalized anxiety disorder who receive cognitive behavioral therapy (CBT) compared to those who receive a psychological placebo treatment (PP) and compared to those who are placed on a waiting list (WL).

Using data from the four studies comparing CBT and PP,the results are heterogeneous (I²=73%,p≤0.01) and the pooled estimate from the random-effect model is not statistically significant (RR=0.82,95%CI=0.56,1.20),indicating no difference in the treatment failure rate (or effective treatment rate) between individuals receiving CBT and those receiving PP.

Using data from the six studies comparing CBT and WL that reported rates of achieving effective treatment,results are heterogeneous (I²=51%,p=0.05) and the pooled estimate from the random-effect model is statistically significant (RR=0.62,95%CI-0.49,0.79),indicating that a larger proportion of individuals in the CBT groups than in the WL were considered ‘improved’ at the end of the study. Sensitivity analysis found that after eliminating the most extreme result (the 2003 Dugas study^[40]) the results of the remaining five studies were homogeneous (I²=0%,p=0.54); the pooled estimate using the fixed-effect model for these five studies also indicated better treatment outcome for individuals in the CBT groups compared to that of individuals in the WL groups (RR=0.65,95%CI=0.55,0.77).

3.4. Network meta-analysis

The results of the network meta-analysis comparing the three groups of results (i.e.,those for CBT,WL,and PP) are shown in Table 2. There is statistically significant heterogeneity between the three groups of results (I²=63.7%,p=0.0014),so the random-effect model was used to estimate pooled SMD. When using CBT as the reference,PP (SMD=0.63,95%CI=0.13,1.14) and WL (SMD=1.24,95%CI=0.87,1.62) had inferior treatment effects to that of CBT. When using PP as the reference,WL had inferior treatment effect compared to PP (SMD=0.61,95%CI=0.02,1.20). In summary,CBT has the best treatment effect among the three groups and PP has a better treatment effect than WL.

Table 2.

Network meta-analysis showing the standardized mean difference (SMD) in final scores between persons with generalized anxiety disorder who receive cognitive behavioral therapy (CBT),and those who are administered a psychological placebo (PP),or placed on a wait list (WL)

	Reference intervention
	CBT	PP
PP [SMD (95% CI)]	0.63 (0.13-1.14)	----
WL [SMD (95% CI)]	1.24 (0.87-1.62)	0.61 (0.02-1.20)
Test for heterogeneity	Tau2=0.1939,I2=63.7%
Test for merging (overall) effect	Q=30.27,df=11,p=0.0014

3.5. Publication bias

As shown in Figure 4,the shape of the funnel plot of the primary outcome measure (the pooled standardized mean difference between CBT and the control group in each study) in the included studies is reasonably sym-metric. This indicates that there is little publication bias.

Figure 4. Funnel plot of results from studies that compare anxiety scores at the end of intervention between the CBT and control groups.

3.6. Risk of biases and level of evidence

The results of the risk of bias assessment are shown in Table 3. For most of the 13 items the two independent raters were in full agreement,but there were substantial disagreements about the randomization,allocation of concealment and completeness of the data,so the agreement about the overall classification of the risk of bias in each of the 12 articles was only fair (Kappa=0.64).

None of the 12 studies had blinding of the subjects or blinding of the therapists and none of them reported on potential biases related to therapist allegiance. All the studies had blinded assessors,but in six of the studies the primary outcome measure depended on self-completion rating instruments,^{[25,39,41,43,44,47]} so the final evaluation was not based on the results of the blind assessor. Overall,8 of the 12 studies were classified as having a high risk of bias.

Table 3.

Risk of different types of biases in the included studies

	Barlow 1992[38]	Borkovec 1993[14]	Butler 1991[39]	Dugas 2003[25]	Dugas 2010[40]	Ladouceur 2000[41]	Linden 2005[42]	Mohlman 2013[43]	Stanley 1996[44]	Stanley 2003[45]	Wetherell 2003[46]	Zinbarg 2007[47]	kappab
Sequence generation	low	low	low	low	low	low	low	?	low	low	low	low	0.43
Allocation concealment	?	low	?	high	low	high	low	high	?	high	low	?	0.49
Blinding therapist	high	high	high	high	high	high	high	high	high	high	high	high	1.00
Blinding assessor	low	low	low	low	low	low	low	low	low	low	low	low	1.00
Blinding participant	high	high	high	high	high	high	high	high	high	high	high	high	1.00
Incomplete outcome data	low	low	high	low	low	low	low	low	low	low	low	low	0.63
Selective reporting	low	low	low	low	low	low	low	low	low	low	low	low	1.00
Researcher allegiance	low	low	low	low	low	?	low	low	low	low	low	low	1.00
Therapist allegiance	?	?	?	?	?	?	?	?	?	?	?	?	1.00
Therapist qualification	low	low	low	low	low	low	low	high	low	low	low	low	1.00
Treatment fidelity	low	low	low	low	low	low	low	low	low	low	low	low	1.00
Treatment integrity	low	low	low	low	low	low	low	low	low	low	low	low	1.00
Other risks of biasa	high	high	low	high	low	?	?	low	high	high	low	low	0.86
OVERALL RISK OF BIAS	high	high	high	high	low	high	low	high	high	high	low	low	0.64

^a including bias from patients who are lost to follow-up,use of medication,and co-morbid conditions

^b Kappa values for inter-rater reliability of the two independent coders who assessed each of the 13 items for the 12 studies

Based on the GRADE criteria,the quality of evidence for the comparison of the primary outcome in the 12 studies that compared the SMD in scale scores between CBT and control groups was ‘moderate’. Similarly,the quality of the evidence for the comparison of the secondary outcome in the 9 studies that compared the percent of participants who had improved at the end of the trial between CBT and control groups was also rated as ‘moderate’. This indicates that further research on the issue (i.e.,on the effectiveness of CBT) is likely to have an important impact on the confidence in the estimate and may change the estimated effect.

4. Discussion

4.1. Main findings

After an extensive review of available databases we only identified 12 RCTs about treatment of GAD using CBT that met our rigorous inclusion criteria. Despite including consideration of Chinese databases no such studies were identified in China. This is a clear indicator of the lack of high-quality psychotherapy research in China and in other low- and middle-income countries.

The effectiveness of CBT in the treatment of GAD and its superiority to medication is widely accepted,^[2,4] but we found that the quality of the RCTs used to support this conventional wisdom is in doubt. Despite having blinded raters,in half of the studies the main outcome depended on the self-rating of (non-blinded) participants. The overall risk of bias was considered high in 8 of the 12 studies. And using the rigorous GRADE criteria the overall level of evidence was classified as ‘moderate’,which indicates that further research could change the widely accepted conclusion about the effectiveness of CBT. Thus the results in favor of CBT are strong,but not definitive.

Pooling results from nine studies we found that CBT has a better treatment effect than being placed on a wait list in terms of both the primary and secondary outcome measures. This finding is consistent with previous findings.^[4,7,54] However two of the studies^[40,41] — both of which used interviewer-administered assessment tools to measure anxiety symptoms - found that the differences between the CBT and WL groups were not statistically significant. A recent meta-analysis^[1] found that interviewer-administered assessments yielded smaller differences than self-report assessments,so this may have contributed to the non-significant results for these two studies.

Pooling results for the four studies that compared CBT to different types of psychological placebos,we found that CBT was superior to PP when using the continuous primary outcome measure (before vs. after reduction of anxiety scale scores),but the difference between groups was not statistically significant when using the dichotomous secondary outcome measure (i.e.,the proportion who achieved the study-specific criteria for ‘effective treatment’ by the end of the study). There are four possible factors that lead to this unexpected result: (a) the number of studies and number of cases was smaller for the CBT versus PP comparison to those for the CBT versus WL comparison,resulting in reduced power to identify differences; (b) the use of the dichotomous secondary outcome measure is much less sensitive to change than the continuous primary outcome measure (the SMD); (c) the negative effect of being put on a wait list because these individuals are less likely to seek alternative forms of help than those who receive no intervention at all;^[5,19,55,56] and (d) the positive treatment effect of PP may be larger than that of WL,decreasing the magnitude of the difference between CBT versus PP compared to the difference between CBT and WL.

Supporting this last potential explanation for the non-significant results between CBT and PP in the secondary analysis,the results of our network meta-analysis of the primary outcome measure (the continuous SMD measure) clearly demonstrated that the therapeutic effect of PP was greater than that of WL. There are several possible explanations for the non-negligible effect of the psychological placebo intervention,primarily related to the positive effects of regular contact with a concerned treatment provider.^[5] This finding of a therapeutic effect of PP is not particularly surprising. Some studies have already found that some components of CBT that may be employed as part of a psychological placebo,such as training relaxation skills,are effective by themselves.^[13,52] The main purpose of using a psychological placebo in CBT trials is to assess the ‘active component’ of CBT after controlling for other psychological factors that are inherent to any type of regular contact with a therapists (e.g.,the structure,number of sessions,and presence of the therapist). Placement on a wait list does not include these components,so it is expected that studies using this type of control group would result in larger treatment effects than studies using a psychological placebo. Thus researchers need to carefully consider the purpose of their study and the factors they wish to control in their study when deciding on which type of control group to use for RCTs about psychological interventions.

4.2. Limitations

There are several limitations that need to be considered. (a) Nine of the 12 included studies used WL as the control condition while only 4 used PP as the control condition (one study used both types of controls). Thus the statistical power to identify differences was weaker for the PP group. This may be one of the factors that resulted in the non-significant result when comparing the secondary outcome between CBT and PP. (b) Few of the studies provided follow-up results after the completion of the treatment,so we were unable to determine whether or not the placebo effect we identified in the PP groups persisted. (c) We only included studies about adults and all 12 included studies were conducted in high-income Western countries. We cannot tell whether or not the identified placebo effect would also occur with children and adolescents or in low- and middle-income countries. (d) Due to the nature of CBT,it is impossible to completely blind the participants and the therapists.^[6] In half of the 12 included RCTs the main outcome measure was based on the results of a self-report questionnaire and,thus,it was not a blind evaluation of the outcome. The failure to blind the conduct and evaluation of the studies could result in a decrease or an increase in the assessed magnitude of the placebo effect. (e) Finally,quality control of psychotherapeutic trials is still an art,not a science,so there are several uncontrolled factors (such as the quality of therapists,the ‘warmth’ of individuals providing the psychological placebo,etc.) that may have affected differences across the various studies.

4.3. Implications

The quality of the evidence supporting the effectiveness of CBT in the treatment of GAD is only fair; more rigorously designed and evaluated studies are needed to confirm (or disprove) the effectiveness of CBT. Our analysis of data from the results for the control groups used in the best RCTs available on this issue indicates that compared to placement on a waiting list,the provision of a variety of ‘psychological placebos’ has a robust treatment effect. Such non-negligible placebo effects of a control group could decrease the assessed treatment effect of any new intervention being tested in a randomized controlled trial,but it would provide a more rigorous assessment of the ‘active component’ of the intervention (i.e.,beyond the effects of regular contact with a concerned therapist). When designing studies to assess the effectiveness of psychotherapeutic interventions,researchers need to carefully consider the different interpretation of results from studies that use psychological placebos as a control condition versus results from studies that use placement on a wait list as a control condition.

Biography

Zhipei Zhu graduated from Anhui University with a degree in applied psychology. She is currently a master’s student at the Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine. Her main research interests are psychotherapy for anxiety disorders and evidence-based medicine.

Funding Statement

This study was funded by the Twelfth Five-year National Key Project (2012BAI01B04), Shanghai Shenkang Hospital Development Project (SHDC12012203) and the Shanghai Health System Leadership in Health Research Program (XBR2011005).