Placebo Response in Repetitive Transcranial Magnetic Stimulation Trials of Treatment of Auditory Hallucinations in Schizophrenia: A Meta-Analysis

Abstract

Several meta-analyses have assessed the response of patients with schizophrenia with auditory verbal hallucinations (AVH) to treatment with repetitive transcranial magnetic stimulation (rTMS); however, the placebo response has never been explored. Typically observed in a therapeutic trial, the placebo effect may have a major influence on the effectiveness of rTMS. The purpose of this meta-analysis is to evaluate the magnitude of the placebo effect observed in controlled studies of rTMS treatment of AVH, and to determine factors that can impact the magnitude of this placebo effect, such as study design considerations and the type of sham used.

The study included twenty-one articles concerning 303 patients treated by sham rTMS. A meta-analytic method was applied to obtain a combined, weighted effect size, Hedges’s g. The mean weighted effect size of the placebo effect across these 21 studies was 0.29 (P < .001). Comparison of the parallel and crossover studies revealed distinct results for each study design; placebo has a significant effect size in the 13 parallel studies (g = 0.44, P < 10⁻⁴), but not in the 8 crossover studies (g = 0.06, P = .52). In meta-analysis of the 13 parallel studies, the 45° position coil showed the highest effect size. Our results demonstrate that placebo effect should be considered a major source of bias in the assessment of rTMS efficacy. These results fundamentally inform the design of further controlled studies, particularly with respect to studies of rTMS treatment in psychiatry.

Introduction

In 1999, Hoffman et al¹ introduced a new, specific, efficient therapy for auditory verbal hallucinations (AVH), repetitive transcranial magnetic stimulation (rTMS). They confirmed the efficacy of active rTMS therapy compared with sham condition and demonstrated its harmlessness, first in a crossover controlled study² and second in a parallel controlled trial.^3,4 Since then,7 meta-analyses have been published reporting variable effect sizes. The first 4 (of the 7) meta-analyses were evaluated previously.⁵ Currently, all 7 meta-analyses can be summarized, as follows. The first meta-analysis analyzing the efficacy of rTMS was published in 2007, demonstrating an effect size of 0.76 for low frequency (1 Hz) rTMS in patients with AVH.⁶ However, this meta-analysis included both crossover and parallel studies. In 2008, an analysis of the same set of papers arrived at an effect size of 0.514.⁷ Then, in contrast, another study reported a much larger effect size of 1.04 for 1 Hz rTMS in patients with AVH.⁸ This study included 2 open-label trials in which the clinical evaluators were aware of the treatment delivered to each patient and for which the large positive effects^9,10 could bias the results and may explain the difference. Slotema et al produced 2 meta-analyses with effect sizes of 0.54¹¹ and 0.44.¹² Although the value in second study is lower, it is still significant; this analysis included 17 papers of which several were recent negative studies. A significant effect size was also found in a meta-analysis that included 17 studies of rTMS as a treatment for AVH, in which control subjects received sham stimulation.¹³ Another meta-analysis that used Bayesian approaches¹⁴ observed a significant effect size with rTMS treatment of refractory AVH and confirmed the stability of this effect, controlling both for publication bias and variable sham conditions.

It is difficult to draw a definitive conclusion regarding the efficacy of 1 Hz rTMS in the treatment of AVH based on these meta-analyses, due to the variability in the results reported. Consequently, Lefaucheur et al.¹⁵ proposed a low level (Level C) recommendation in favor of a possible efficacy of rTMS of the left temporoparietal cortex on auditory hallucinations. Several methodological problems could explain the discrepancies, including the small sample sizes included in most of the studies, the crossover design, and the high variability of the stimulation parameters used.⁵ Another parameter that has not been previously discussed is the placebo effect. This can include nonspecific effects of treatment, such as physician attention and interest; the expectations of both patients and physician; the reputation, expense, and impressiveness of TMS, particularly when neuronavigation is performed; and the way and the setting in which the clinical evaluations are processed. Generally speaking, non-pharmacological treatments can evoke a large placebo response, and this might affect the assessment of rTMS efficacy in patients with AVH.¹⁶ Moreover, the specific kind of sham treatment used in the rTMS study can also have impact. At the present time 4 types of sham procedure can be used in controlled trials: (1) a sham coil that produces no magnetic field at the scalp (shielded or distant discharge), (2 and 3) an active coil producing a real magnetic field but which is tilted away from the scalp at an angle of either 45° or 90° to reduce brain stimulation, and (4) an active coil, but with a target at a cortex location that is not known to be involved in the pathophysiology of AVH. The sham coil simulates the sound of rTMS, but it does not induce side effects such as twitch, dystonia, or skin stimulation. The active coil simulates the superficial tissue effects, but also stimulates the brain region involved in AVH, although to a lesser degree,¹⁷ or stimulates a location not involved in the pathological process. Thus, in contrast to inert controls used in pharmacological studies, the control conditions used in the context of rTMS studies are not necessarily comparable and they are not necessarily devoid of real biological effects.

Despite the existence of several meta-analyses assessing the response of patients with schizophrenia with auditory hallucinations to rTMS treatment, the placebo response has not been explored. Only 1 study has assessed the placebo effect with respect to rTMS; this study was evaluating treatment of depression, and it found a large placebo effect (Cohen’s d = 0.82).¹⁸ Consequently, the purpose of this meta-analysis was to evaluate the magnitude of the placebo effect in controlled studies of rTMS treatment of AVH in patients with schizophrenia, and to determine factors that impact this magnitude, such as study design considerations and the type of sham condition chosen. We postulated that there might be a greater effect observed with use of a 45° coil than with a sham or a 90° coil treatment, with parallel design than with crossover design, and with the number of pulses administered. Indeed, we hypothesized that 45° or 90° coils are not devoid of action on the brain, in contrast to a sham coil. We therefore expected to observe a greater effect with a 45° coil than with a sham coil because of this “active ingredient” in combination with the sham effect. We also considered that a crossover design study could attenuate the placebo effect compared to a parallel design study due to different scalp sensations and to other potential side effects (such as headache), which could allow the subject to discriminate between active and sham treatments. Finally, increasing the number of pulses either lengthens the duration of the individual sessions or increases the number of sessions, and consequently may reinforce the placebo effect by fostering a stronger patient-practitioner relationship.

Methods

Selection Criteria

We searched the MEDLINE database for articles published from 1999 to February, 2015 that contained keywords of “rTMS,” “Schizophrenia,” or “Auditory hallucinations.” Studies were eligible if: (1) the manuscript was written in English; (2) the study was a randomized, double-masked, sham-controlled (with sham coil, location of coil unrelated to AVH, 45° or 90° coil) parallel or crossover study; (3) the auditory verbal hallucinations (AVH) were assessed using a specific scale of severity; or (4) the study reported mean and SD of the AVH scale scores before and after the sham condition, or provided other statistical parameters that could be used to compute the Hedges’s g value.

Data Extraction

The following sets of variables were extracted by 3 investigators: (1) size of the sham condition group; (2) mean and SD of the AVH rating scale values taken at baseline and at the end of treatment in the sham condition group, either provided in the article or provided by the corresponding author; the pre-post correlations were estimated to be 0.8 for all the studies. When these variables were not provided, we extracted from the articles the t-value (paired t-test)^3,19 or the mean and SD of the difference within the sham condition group;^20–24 (3) study design and treatment parameters (whether parallel or crossover design, the type of coil used, location of treatment, frequency, motor threshold, number of stimuli by session, and number of sessions); and (4) demographic, clinical and treatment characteristics (eg, age, gender, use of antipsychotics, severity of AVH, and the determined efficacy of rTMS in the active group compared to the sham group).

Quantitative Analysis

As this meta-analysis was focused on within-group effect sizes, effect sizes were calculated for the mean changes in auditory hallucinations in the sham group between the pre- and post-treatment measurements. For this purpose we used the global AVH, which was evaluated in the majority of studies using the Auditory Hallucination Rating Scale (AHRS) total score^22–31 or otherwise evaluated by the AHRS self-report,³² the AHRS frequency item,^33,34 the Hallucination Change Score (HCS),^2,3,21 the Positive And Negative Syndrome Scale hallucinatory behavior,^35,36 the Haddock’s scale,²⁰ the Scale for Auditory Hallucinations,¹⁹ or the Psychotic Symptom Rating Scales.³⁷ The mean gain in each study was computed using the Comprehensive Meta Analysis (version 2) in a random effects model. After computation of the individual effect sizes for each study, meta-analytic methods were applied to obtain a combined, weighted effect size, Hedges’s g, for the entire sham group. Initial analysis calculated the homogeneity I ² in order to check whether the studies could be taken to share a common population effect size. The significance level was set at the 2-tailed 0.05 level.

We investigated several factors that could contribute to the placebo effect: the type of sham condition used, the number of pulses applied, the design of the study (parallel vs crossover), and the efficiency of active rTMS compared to sham rTMS. The type of sham condition was considered only if it was used in 2 or more studies. Consequently, 4 kinds of sham condition were included in the analysis: a sham coil, an active tilted coil on the same site (in either a 45° or 90° position, which were considered to be 2 distinct types of sham condition), and active coil on another site unrelated to AVH.

Results

The study included twenty-one articles concerning 303 patients who were treated by sham rTMS (table 1). The mean weighted effect size of the placebo effect for the 21 studies was determined to be 0.29 (P < .001) with I ² = 78.81, indicating significant heterogeneity across the studies (P < 10⁻³) (supplementary figure 1).

Table 1.

Characteristics of the 21 Studies

Author (y)	Design	Number of Patients in Sham Group (n = 303)	Type of Sham	Site of Stimulation	% MT	Total Number of Stimuli	Time of Evaluation (d)	Rating Scale	Efficacy of Active Over Sham Condition
Hoffman et al2	Crossover	12	45°	T3P32	80%	3120	4	HCS	Yes
McIntosh et al35	Crossover	16	45°	T3P3	80%	3120	7	PANSS hallucination	No
Schonfeldt et al20	Crossover	10	LU AVHa	Broca, STG	90%	4800	6	PSYRATS	No
Chibbaro et al19	Parallel	8	45°	T3P3	90%	3600	4	SAH	No
Fitzgerald et al21	Parallel	16	45°	T3P3	90%	9000	14	HCS	No
Hoffman et al3	Parallel	23	45°	T3P3	90%	7920	9	HCS	Yes
Lee et al33	Parallel	14	90°	T3P3/T4P4	100%	12000	10	AHRS frequency	No
Poulet et al22	Crossover	10	Sham coil	T3P3	90%	10000	7	AHRS	Yes
Brunelin et al25	Parallel	10	Sham coil	T3P3	90%	10000	7	AHRS	Yes
Jandl et al37	Crossover	14	45°	T3P3/T4P4	100%	4500	5	PSYRATS	No
Saba et al36	Parallel	8	Sham coil	T3P3	80%	3000	14	PANSS hallucination	No
Rosa et al34	Parallel	5	Sham coil	T3P3	90%	9600	12	AHRS frequency	No
Vercammen et al32	Parallel	12	Sham coil	T3P3, bilateral	90%	14400	13	AHRS self report	No
Loo et al26	Crossover	18	LU AVHb	Location MRI	110%	2880	4	AHRS	No
Slotema et al27	Parallel	20	90°	T3P3, fMRI	90%	18000	21	AHRS	No
De Jesus et al28	Parallel	9	LU AVc	T3P3,	80%	23040	28	AHRS	No
Blumberger et al23	Parallel	17	90°	T3P3	115%	24000	28	AHRS	No
Van Lutterveld et al29	Crossover	24	LU AVHd	T3P3, T4P4	90%	1200	1	AHRS	No
Hoffman et al24	Parallel	28	45°	Left, right Wernicke	90%	9600	21	AHRS	No
Klirova et al30	Crossover	13	90°	T3-P, left guided by PET	100%	10800	14	AHRS	Yes
Bais et al31	Parallel	16	Sham coil	T3-P3, T4-P4	90%	14400	7	AHRS	No

Note: MT, Motor threshold; T3P3, Left temporo-parietal cortex (ie, midway between the T3 and P3 sites, according to the international 10/20 system of EEG electrode placement); HCS, Hallucination Change Scale; PANSS, Positive And Negative Syndrome Scale; STG, Superior Temporal Gyrus; PSYRATS, Psychotic Symptoms Rating Scale; SAH, Severity of the Auditory Hallucination; T4P4, Right temporo-parietal cortex (ie, midway between T4 and P4); AHRS, Auditory Hallucinations Rating Scale; fMRI, functional Magnetic Resonance Imaging; LU AVH, Location unrelated (LU) to auditory verbal hallucination (AVH)

^amidline parieto-occipital region

^b Vertex, 90°

^c F3, 45°

^d Oz

There was no effect of the number of pulses applied (P = .70).

The analysis comparing the parallel and crossover studies revealed distinct results for the 2 different study designs. Sham groups showed a significant effect size (g = 0.44, P < 10⁻⁴) in the 13 parallel design studies, but not in the 8 crossover studies (g = 0.06, P = .52) (supplementary figure 2). Consequently, an additional meta-analysis was performed on the 13 parallel studies to consider the effect of individual types of sham conditions. The use of 45°, 90°, or sham coil treatments showed significant effect sizes with respect to AVH (g = 0.65, P = .01; g = 0.25, P = .005; and g = 0.36, P = .002, respectively), with the highest effect size observed with the use of the 45° position coil (figure 1).

Fig. 1.

Meta-analysis of the 13 parallel studies, taking into account the sham type (sham coil, coil positioned at 90°, or coil positioned at 45°)

A separate meta-analysis was performed to take into account the efficacy of active rTMS compared to sham rTMS. Based on the AVH rating scales and the endpoint day reported on the table 1, a significant placebo effect size was observed in the 16 studies for which there was no significant effect of active rTMS over sham rTMS (g = 0.35, P < .001), whereas the placebo effect size was not significant (g = 0.09, P = .26) in the 5 studies that reported a significant superiority of active rTMS treatment over sham rTMS (supplementary figure 3).

Discussion

This meta-analysis elucidates a significant placebo effect in patients receiving no active rTMS during controlled therapeutic studies of schizophrenia patients with AVH. However, this placebo effect is small, with an effect size of 0.29. The present study also identifies 2 factors that impact the placebo effect: the type of sham condition and the design of the study. Whereas a medium effect size (g = 0.44) was observed in the parallel studies, no placebo effect (g = 0.06) was observed in the crossover studies. Meta-analysis that included sham type and focused only on the parallel studies showed a higher effect size with a 45° coil (g = 0.65) than with a 90° coil (g = 0.25), or with a sham coil (g = 0.36). It is also notable that no placebo effect was observed (g = 0.09) in the studies that showed a significant superiority of active rTMS over the sham condition.

Placebo in Schizophrenia

The placebo phenomenon has been described for a long time in studies of schizophrenia³⁸ and meta-analyses have also described the placebo response in recent antipsychotic trials.^39,40 A review of 11 placebo-controlled, parallel group studies of patients with schizophrenia treated either by placebo or antipsychotics showed that an average of 25% of the patients in the placebo groups fulfilled criteria for a treatment response (> 50% on the Brief Psychiatric Rating Scale, or at least were much improved according to the Clinical Global Impression scale).⁴¹ Consequently, finding evidence of the placebo effect in a therapeutic trial is not unusual, even with patients with schizophrenia. However, some factors can contribute to increases or decreases in the magnitude of the placebo effect and thus cause under- or over-estimation of the efficacy of the active treatment compared to the placebo treatment. In rTMS trials, several factors specific to magnetic stimulation have to be considered during analysis of the placebo effect: the procedure used as the sham condition, the design of the study, and the procedure used to ensure investigative blindness. Moreover, administration of the sham rTMS is often combined with assessment of the motor threshold with an effective coil. Thus, the sham treatment is preceded by some real single-pulse magnetic stimulations. A 1Hz rTMS trial also involves more than 10–20 sessions over 2–4 weeks, for which the patient meets with the medical team daily, and these interactions can also contribute to a placebo effect. Moreover, an rTMS session utilizes impressive, high-tech procedures, sometimes including neuronavigation. In this case, even the sham-treated patients are exposed to their own cerebral image during the session, which can also favor a placebo effect. Finally, the beliefs of patients regarding whether they think their voices are the result of brain dysfunction or some force of external origin (eg, ghosts, other people, Gods, devils) may alternately potentiate or attenuate the placebo effect, respectively, by the self-representation of the rTMS as a therapeutic that is magnetic in nature or as a cognitive intervention. Other factors, such as the relationship between the practitioner and the patient, the subject’s expectations and desire to find relief from their hallucinations, the expectations of the patients or the investigators regarding a new treatment, and the reputation of the research center are also important factors to consider but are not specific to rTMS.

Sham Procedure

The present results confirm our hypothesis that the type of sham treatment can have an impact on the placebo effect. The observation of a greater effect with the 45° coil than with the sham or the 90° coil treatment raises the question of whether active brain stimulation contributed to add an “active” effect to the placebo effect in the case of the 45° coil treatment. Indeed, sham stimulation administered with active rTMS coils angled 45° away from the skull is not exempt from causing some active stimulation.⁴² Animal studies have confirmed this hypothesis, as 45° angled active coils was observed to reduce voltage in the brain by only 24% compared with well-positioned active coils.¹⁷ Consequently, a coil tilted away from the scalp at an angle of 45° could cause an active effect, independent of the placebo effect. In other investigations, this kind of sham condition produced clinical effects in individuals with milder AVH,³ also supporting the hypothesis that 45° coil induces brain stimulation, albeit at a weaker voltage. Another study²⁴ used the same sham condition and also reported intriguing results, when rTMS was applied in the right hemisphere. Indeed, they found that sham rTMS responses on hallucinations were correlated with nondominant motor impairment. A nearly significant correlation between the baseline attentional salience of hallucinations and an HCS improvement was also detected for the sham condition. Those results suggested that a physiological effect occurred that could not be accounted for by a placebo effect.

Additionally, the side effects related to the site of stimulation and noise should be quite striking for the participant to perceive, and could convince them of active therapy. Stimulation of the brain using rTMS inevitably causes depolarization of excitable structures located between the coil and the brain, such as muscle fibers and nerve fibers. This is particularly evident in the treatment of AVH in schizophrenia because the site of stimulation commonly used is located in the temporo-parietal region, and thus can cause a contraction of the jaw, often concomitant with stimulation of the facial nerve within its intrapetrous course. These parasitic stimulations, associated with significant noise due to the stimulation, are felt by the patients as side effects. It has been demonstrated that somatic focus increases the magnitude of placebo response,⁴³ and therefore when the subject interprets side effects of active coils as cues of therapeutic efficacy, then more side effects will be present, and the subject will be increasingly sensitive to the placebo effect. Nonetheless, even if the major differences between the placebo experience with active and passive coils reside in side effects and tactile perception, our results support the idea that the sham coil generates a similar placebo effect as that observed with an active coil positioned at 90° in patients with schizophrenia.

Design

The present study also demonstrated a medium-sized placebo effect in parallel studies but no placebo effect in crossover studies. This result confirms that parallel design is a better approach than crossover design in testing the efficacy of the treatment. In a crossover design study, the patient can detect differences between the active treatment and sham period in terms of scalp sensations, side effects, and location of the coils. Because the conditions of a crossover study gives the subject an opportunity to compare the sensations and side effects associated with both active and sham treatments, the subject is in a better position to guess what type of stimulation has been given. However, the lack of placebo effect that was observed with the crossover studies could not be explained by the coils used, because 6 of the 8 studies used active coils that would potentially induce scalp sensations. Indeed, 3 of the cross-over design studies used active stimulation of a location unrelated to AVH^20,26,29 which could induce scalp sensations similar to those produced by active stimulation of the left temporo-parietal cortex.²⁹ Three used an active coil positioned at 45°,^2,37,35 which may also produce sensations;¹⁵ however, this sham procedure might induce a lower level of somatic sensations than that produced with an active untilted coil. Another hypothesis to explain the absence of placebo effect might be that the significant effect of active rTMS when applied in the first period attenuates or even cancels the placebo effect during the second period of the trial. Moreover, among the 8 studies using crossover design, 2 studies reported an increase in AVH in the sham group,^20,22 contributing to a decrease in the overall average placebo effect size. In a complementary analysis, we computed Hedges’ g in each active group of the 21 studies to test the relationships between Hedges’ g of the active groups and those of the sham groups. First, we found a significant g value for all 21 studies (g = 0.84; P < 10⁻⁶). Second, we observed a highly significant correlation in the 13 parallel studies (r = 0.82; P = .006) (supplementary figure 4). Third, there was no significant correlation in the 8 cross-over studies (P = .09; supplementary figure 5). These results supported the notion that a similar placebo effect was observed in both groups (active and sham), but only in parallel design studies, and that the therapeutic effect in the active group included both the biological effect and the placebo effect. These results also suggested, in parallel design studies, that the same non-specific factors in both groups (active and sham), such as the reputation of the research center or a strong professional impression, might increase the response in both groups in certain laboratories. Finally, this discrepancy between parallel and cross over studies strengthens our results that the placebo effect relied on different mechanisms, depending on the study design used.

Blindness

Another effect that influences the placebo effect is blindness of the investigators. It is considered that a nonblinded experimenter, who has information about the patient’s group assignment, might show a change in practice that could affect the outcome. Numerous studies reviewed in the present meta-analysis declared that they used a double-masked procedure. However, it is difficult to consider that the experimenter is really blind when it is necessary to tilt the coil away from the scalp or to use a sham coil. In order to respect the blindness in the experimental design, it is necessary to ensure that clinical evaluations are processed by individuals who are unaware of the stimulation procedures and that the patients had never previously received active rTMS before being enrolled in the study. Although nearly all the 21 studies specified that raters and patients were unaware of the treatment assignment, only 2 studies reported that previous active rTMS was used as an exclusion criterion.^3,24 Indeed, a patient’s previous treatment with active stimulation would cause them to know the scalp sensations produced by active rTMS, and therefore could decrease the placebo effect.

It is striking that symptom scores improved after both real and sham rTMS treatment in some studies,³⁵ while others did not improve at all after sham rTMS.^2,30 It is notable that the 5 studies that reported a significant superiority of active rTMS over sham rTMS in AVH treatment are generally those for which weak or no placebo effect was observed on average. Among these 5 studies, only 1 observed a medium-sized effect in the sham group (g = 0.429).³ A small or absent placebo effect can contribute to the measurement of a large difference between drug and sham treatment and thus a superiority of the active treatment over the sham treatment in terms of efficacy. This phenomenon could not be due to the kind of coil used because among these 5 studies, 2 used an active 45° coil,^2,3 1 used an active 90° coil,³⁰ and 2 used a sham coil.^22,25 This observation underscores the necessity of checking the sham procedure, even in double-masked controlled studies. One possibility would be to ask the rater and the patient in which group they think they have been randomized; a close percentage of responses in each group would allow us to validate the sham procedure. Unfortunately, among the 20 studies, only 2 studies reported such information.^21,32

We considered that the repetition of stimulation across sessions might also enhance the placebo response. However, we did not observe any effect of the number of stimulations on the placebo response. The absence of such a link in the present study could be due to inclusion of only a small number of studies. However, this result could also suggest an absence of placebo effect association with the number of therapeutic contacts with the patient, as was recently reported in trials of antipsychotic treatments.⁴⁰

Finding a placebo effect is expected, whatever the disorder or the treatment. The predominant issue with respect to rTMS studies is to control for this placebo effect in patients receiving active and sham treatment, knowing that the nature of the treatment means that experimental blindness cannot be obtained in an rTMS study as it can in a drug treatment study. The most important recommendation to address this issue is the use of a parallel design, rather than a cross-over design, in order to avoid side effects and scalp sensations that may allow a patient to discriminate between the active treatment and sham periods. Moreover, similar procedures must be applied in both active and sham groups. We recommend the use of a sham coil to avoid application of any active stimulation during the sham period (such as that provided by a 45° coil) and the use of a sham coil with the same appearance, position, and location as that used for treatment of the active group. Some systems have been developed in which a cutaneous electrical stimulator is used in conjunction with the sham coil,⁴⁴ but the cutaneous sensation remains noticeably different from that produced by a real coil in about half of the cases, in particular with stimulation intensities higher than 80% of the motor threshold (rMT).¹⁵ Neuronavigation, if it is used, has to be applied to both the active treatment and sham groups. The same clinical investigator should follow the patient throughout the treatment and collect the psychopathological evaluations; mixing practitioners could impact the patient-practitioner relationship and thus influence the placebo effect. Assessment of rMT in both groups (sham and active), preferentially before each session, would also present an advantage, as patients receiving the sham treatment can feel physical sensations during the rMT assessment, thus inducing an equal technique-related placebo effect in both groups. We also recommend enrollment of patients who have not been previously treated by rTMS, and therefore do not have expectations related to the treatment. The practitioner who delivers rTMS must be unaware of the psychopathological evaluations and this practitioner must collect the side effects, rather than the practitioner who conducts the clinical evaluation. Indeed, the collection of side effects could cause the clinical investigator to form a judgment about whether the patient is in the treatment or sham group. Finally, at the end of the treatment, asking the patients in which group they think they are included provides a test of the blindness procedure.

This meta-analysis included 21 studies with 303 patients; however, there was considerable heterogeneity across the studies. This heterogeneity could be considered to be a limitation of this study, and it can be explained by several factors. Several different scales were used to evaluate the primary outcome. The end-point times varied widely, from 4 to 28 days. There were few studies using a location unrelated to AVH or with a 90° coil position, which limited our ability to process an analysis taking into account both the design of the study (cross-over vs parallel) and the type of the coil. The parameters of stimulation used (in terms of intensity, number of pulses, location of stimulation, and sham procedures) also varied widely from one study to another, highlighting the necessity of standardization in the rTMS procedures used for the treatment of AVH.

In conclusion, our results demonstrate that the placebo effect should be viewed as a major source of bias in the assessment of treatment efficacy. These results elucidate important considerations to guide the design of further controlled studies, particularly with respect to studies of rTMS treatment in psychiatry. Our findings give us reason to suspect that placebo effects might be at work in other treatment areas and we conclude that it is advisable to look for patterns consistent with the placebo effect in all of the domains where placebo effects cannot be entirely controlled. This particularly concerns studies of no-drug therapies, including stimulation applied as a modulatory treatment. Special attention to the design of the study, and particularly on the design of the procedure used as the sham, is necessary before any conclusion on the efficacy of such therapies can be discerned.

Supplementary Material

Supplementary material is available at http://schizophreniabulletin.oxfordjournals.org.