Clinical trial design in non‐invasive brain stimulation psychiatric research

Abstract

Major depressive disorder (MDD) trials – investigating either non‐pharmacological or pharmacological interventions – have shown mixed results. Many reasons explain this heterogeneity, but one that stands out is the trial design due to specific challenges in the field. We aimed therefore to review the methodology of non‐invasive brain stimulation (NIBS) trials and provide a framework to improve clinical trial design. We performed a systematic review for randomized, controlled MDD trials whose intervention was transcranial magnetic stimulation (rTMS) or transcranial direct current stimulation (tDCS) in MEDLINE and other databases from April 2002 to April 2008. We created an unstructured checklist based on CONSORT guidelines to extract items such as power analysis, sham method, blinding assessment, allocation concealment, operational criteria used for MDD, definition of refractory depression and primary study hypotheses. Thirty‐one studies were included. We found that the main methodological issues can be divided in to three groups: (1) issues related to phase II/small trials, (2) issues related to MDD trials and, (3) specific issues of NIBS studies. Taken together, they can threaten study validity and lead to inconclusive results. Feasible solutions include: estimating the sample size a priori; measuring the degree of refractoriness of the subjects; specifying the primary hypothesis and statistical tests; controlling predictor variables through stratification randomization methods or using strict eligibility criteria; adjusting the study design to the target population; using adaptive designs and exploring NIBS efficacy employing biological markers. In conclusion, our study summarizes the main methodological issues of NIBS trials and proposes a number of alternatives to manage them. Copyright © 2011 John Wiley & Sons, Ltd.

Introduction

Despite the significant neuropsychopharmacology advancement in the past decades, it is still needed to develop new somatic interventions for psychiatric disorders (Baghai et al., 2006; Marder, 2006). Non‐invasive brain stimulation (NIBS) is a novel therapeutic intervention facing rapid development, with preliminary studies showing significant clinical gains in several neurologic and psychiatric conditions such as major depressive disorder (MDD) (Gross et al., 2007), schizophrenia (Freitas et al., 2009), stroke (Fregni and Pascual‐Leone, 2007), and Parkinson's disease (Elahi and Chen, 2009). We use the term “NIBS” here to refer to devices applied to the scalp in order to induce electrical currents in specific brain areas. The two main examples are transcranial magnetic stimulation (TMS) and transcranial direct current stimulation (tDCS).

Besides the initial encouraging results from NIBS clinical studies, it is unclear whether these effects are related to specific properties of the technique as two MDD meta‐analyses studies (Herrmann and Ebmeier, 2006; Schutter, 2008) failed to identify predictors of response, meaning that such effects might not be reproducible in different contexts. Such mixed results might also be explained by study design issues, such as small sample sizes, lack of generalizability and insufficient blinding. This might also be true for several antidepressant studies that may fail due to methodological problems (Gelenberg et al., 2008).

Therefore, a critical overview of the methodology of NIBS trials might improve its clinical development. We aimed to review their methodology as to discuss potential pitfalls and implications for designing future trials. We focused our review on repetitive transcranial magnetic stimulation (rTMS) (a device that generates a strong magnetic field that induces an electric current to modulate a cortical focal brain area) and tDCS (a device that polarizes cortical tissue and thereby modifies neuronal excitability) trials for MDD which has a greater number of studies – although most issues can be applied for other neuropsychiatric disorders.

Methods

We performed a systematic review on all tDCS and rTMS trials published from April 2002 to April 2008. Although vagal nerve stimulation (VNS) is a neuromodulatory device used for MDD treatment, it was not included because we reckon this technique as minimally invasive (as it envolves surgery) and also because the study design methodology of VNS has been already reviewed in the past years when validating VNS intervention to epilepsy (Groves and Brown, 2005). We chose to look for articles after 2002 as Martin et al.'s meta‐analysis (2003) reviewed previous trials. The detailed literature search and selection criteria are described elsewhere (Brunoni et al., 2009); in summary, we searched for the keywords “depression”, “transcranial magnetic stimulation” and “sham” on MEDLINE, Web of Science, Cochrane and Scielo as to include only randomized, double‐blinded, sham‐controlled, parallel trials – 29 rTMS studies were found (Anderson et al., 2007; Avery et al., 2006; Bortolomasi et al., 2007; Boutros et al., 2002; Bretlau et al., 2008; Fitzgerald et al., 2006; Fitzgerald et al., 2003; Garcia‐Toro et al., 2006; Hausmann et al., 2004; Herwig et al., 2007; Herwig et al., 2003; Holtzheimer et al., 2004; Hoppner et al., 2003; Januel et al., 2006; Jorge et al., 2008; Jorge et al., 2004; Koerselman et al., 2004; Loo et al., 2003; Loo et al., 2007; Mogg et al., 2008; Mosimann et al., 2004; O'Reardon et al., 2007; Poulet et al., 2004; Rossini et al., 2005a, 2005b; Rumi et al., 2005; Stern et al., 2007; Su et al., 2005). The same criteria were used for tDCS, identifying two additional studies (Boggio et al., 2008; Fregni et al., 2006a ), for a total of 31.

We then examined each trial for potential methodological issues threatening study validity. We constructed an unstructured checklist based on the CONSORT statement and other methodological reviews analyzing pitfalls in clinical trials (Boutron et al., 2007; Glasser and Howard, 2006; Hopewell et al., 2008; Leucht et al., 2008; Zlowodzki et al., 2006). Four main aspects were addressed:

1. Statistical validity – we checked whether the studies were vulnerable to type I and type II errors by looking for their power analysis as well as for the adequacy of statistical tests and the primary hypothesis (if it was clearly defined and if it was based on response or score changes). We also performed a power analysis (using Stata 10® software) in negative studies.

2. Internal validity – we checked whether the studies were robust enough to demonstrate causality by looking for evidence of allocation concealment, potential blinding violation, sham method utilized, methods for handling missing data [intention‐to‐treat (ITT) and last‐observation‐carried‐forward (LOCF) approaches] and evidence of carryover effects.

3. Construct validity – we examined the appropriateness of the operational criteria used for major depression, depression severity and treatment resistant depression (TRD). We also checked whether the studies verified measures of effectiveness as to assess not only statistical but also clinically meaningful differences.

4. External generalizability – we classified each study in five groups to check whether their conclusions can be generalized: (1) efficacy in general conditions; (2) long‐term efficacy; (3) efficacy in specific conditions or subgroups; (4) new clinical or stereotaxic approaches and; (5) “add‐on” strategies.

Regarding the nomenclature for combination strategies, it should be underscored that the term “add‐on” is used, for some authors, to refer to subjects on antidepressant drugs (Fitzgerald et al., 2006); while others use the term “augmentation” to refer to “add‐on” strategies (Herwig et al., 2007). Here, we chose to use the terms applied in pharmacological trials (Altshuler et al., 2003) – therefore, studies whose strategy was to simultaneously combine non‐pharmacological and pharmacological interventions at the beginning of the trial are referred as “add‐on” or “accelerating” studies.

Results

General characteristics

The 31 studies included 1505 patients, with a mean age of 50.8 years, 819 (54%) of whom were female. Most of the studies had small sample sizes as 28 studies remained below the 75th percentile, with sample sizes ranging from 10 to 68 subjects, whereas three studies had sample sizes of 99 or more (Herwig et al., 2007; O'Reardon et al., 2007; Rossini et al., 2005b).

Statistical validity

Only 10 (33%) of the studies performed power analysis calculations to estimate sample size, two of them showing negative results. Seven of the 21 studies not estimating sample size showed negative findings – a power analysis of them was performed (Table 1).

Table 1.

Power analyses of the studies showing negative results, active and sham stands for the endpoint depression scores in the active and sham rTMS groups, respectively

Author	Active	Sham	Cohen's d	Sample size	Estimated power (%)
Boutros et al., 2002	27.09 ± 13.17	26.42 ± 13.35	0.05	21	5
Hausmann et al., 2004	15.8 ± 9.5	20.2 ± 10.9	0.44	38	23
Holtzheimer et al., 2004	14.6 ± 3.2	15.3 ± 3	0.23	15	7
Hoppner et al., 2003	30.3 ± 4	29 ± 11	0.18	30	6
Loo et al., 2003	32 ± 5	27 ± 5	1	19	58
Mosimann et al., 2004	23.3 ± 7.2	20.4 ± 6.6	0.44	24	10
Poulet et al., 2004	11.22 ± 6.1	18.12 ± 9.6	0.88	19	45

Note: The measure of the effect size is expressed in Cohen's d. The power analyses were performed using Stata 10® software.

Regarding statistical tests, we observed most studies performed more than five tests for efficacy, as depression was measured by at least two rating scales and outcomes were expressed as continuous (score change) as well as categorical (response/remission rates). The tests most used were: ANCOVA – generally covariating for baseline depression scores; repeated‐measures ANOVA – commonly to explore for the effects of time, treatment and their interaction and the chi square test for addressing differences in responders for categorical variables. Logistic regressions and multiple linear regressions were the tests used for identifying predictors of response.

Finally, 16 (51%) studies did not define a priori the primary depression rating scale and 18 (58%) did not define if the primary outcome assessment was either continuous (score change) or categorical (number of responders).

Internal validity

Table 2 summarizes the internal validity assessment of the studies. All studies used a “single‐blinded with external blinded rater evaluation” approach to preserve blinding and eight (25%) studies assessed blinding violation. Nineteen (61%) studies used an ITT approach – these studies handled with missing data from drop‐out patients by using the LOCF method.

Table 2.

Internal validity of the studies

Internal validity of the studies	Number of trials (%)
Blinding
Single‐blinded with external raters	31 (100%)
Blinding assessment	8 (25%)
Attrition
Intention‐to‐treat approach	19 (61%)
Allocation concealment
Not reported or unclear	19 (61%)
Sealed envelopes	4 (13%)
Random number list	8 (25%)
Sham method
Angled coil	17 (55%)
Standard coil with deceiving	9 (30%)
Other methods	5 (16%)
Wash‐out period
Unclear	6 (19%)
Less than a week	6 (19%)
One to three weeks	6 (19%)
More than three weeks	7 (22%)

Regarding sham method, nine (30%) studies used a form of sham coil (i.e. a coil that did not generate a magnetic field and was only used for blinding purposes), 17 (55%) studies used an angled coil for blinding and five studies (16%) used other methods: a shielded coil (Jorge et al., 2008), a coil generating a small magnetic field (O'Reardon et al., 2007), and a turned‐off tDCS device (Boggio et al., 2008; Fregni et al., 2006a).

Finally, although all the studies referred that patients were randomized, only 12 (39%) described the methodology used.

Construct validity

Table 3 shows the operational criteria used for construct validity. All studies based their criteira on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM‐IV): 11 (35%) studies used a structured checklist for diagnosis, while in 20 (65%) it was made by clinical interview. Nine (33%) studies presented no definition of TRD, while 17 (55%) defined it correcly.

Table 3.

Construct validity of the studies

Construct validity of the studies	Number of studies (%)
Definition of treatment resistant depression
Not defined	9 (30%)
At least one AD failed	17 (55%)
At least two AD failed	3 (10%)
One or more AD failed	2 (5%)
Diagnostic of depression
Unstructured	20 (65%)
MINI	4 (13%)
SCID	6 (19%)
SCAN	1 (3%)
Depression rating scale
HDRS	29 (93%)
MADRS	10 (33%)
BDI	17 (55%)

AD, antidepressant drug; HDRS, Hamilton Depression Rating Scale; MADRS, Montgomery–Asberg Depression Rating Scale; BDI, Beck Depression Inventory; MINI, Mini International Neuropsychiatric Interview; SCID, Structured Clinical Interview for DSM disorders; SCAN, Schedule for Clinical Assessment in Neuropsychiatry.

All studies used more than one depression rating scale, but only 14 (36%) defined the primary scale: 10 (33%) used the HDRS (Hamilton Depression Rating Scale) and four (13%), the MADRS (Montgomery–Asberg Depression Rating Scale).

Finally, the studies that performed power analyses and those that defined a priori whether the outcome assessment would be either continuous or categorical verified the clinical effectiveness of intervention; e.g. O'Reardon et al. (2007) defined a meaningful outcome as 25% reduction in depression rating scale as compared to sham group.

External generalizability

Table 4 shows that 11 (33%) studies assessed the general efficacy of NIBS and three (9%), the long‐term efficacy. Six (19%) studies assessed new NIBS approaches, such as rTMS stimulation three times a week (Anderson et al., 2007) or twice daily (Loo et al., 2007), a bilateral simultaneuous stimulation (Loo et al., 2003), a stereotaxic guided stimulation (Herwig et al., 2003), an alternated high‐frequency/low‐frequency stimulation (Garcia‐Toro et al., 2006) and a right/left sided alternated stimulation (Fitzgerald et al., 2006). Five (17%) studies assessed NIBS in specific populations, such as elderly patients (Mosimann et al., 2004) or vascular depression (Jorge et al., 2008). Finally, six studies (25%) addressed the efficacy of rTMS as an “add‐on” strategy and 20 (65%) studies addressed the efficacy in TRD patients.

Table 4.

External validity of the studies

External validity of the studies	Number of studies (%)
General topics
Refractoriness	20 (65%)
Patients in use of antidepressant drug	22 (71%)
“Accelerating” studies	6 (25%)
Augmentation studies	13 (42%)
Study hypotheses
General efficacy of intervention	11 (33%)
Long‐term efficacy	3 (10%)
Specific conditions/subgroups	5 (17%)
New clinical approaches	6 (20%)
“Add‐on” studies	6 (25%)

Discussion

We identified a number of methodological issues across the 31 NIBS trials reviewed, which can be divided in three groups: (1) issues inherent to small and exploratory studies, e.g. underpowered analysis; (2) issues common in MDD trials, for instance, diagnostic validity; (3) issues specific to NIBS trials such as imperfect blinding and generalizability of the results. We first explore the concepts of internal validity versus generalizability and thereafter discuss these issues as to propose a potential alternatives to overcome them – which are summarized in Table 5.

Table 5.

Summary of the main issues identified and possible alternatives to handle them

Topic	Issues	Alternatives
General	Nomenclature – indistinctly use of the terms “add‐on”, “augmentantion”, “acceleration”.	Use the same nomenclature of pharmacological trials: augmentantion when associating NIBS to a patient already on drugs; accelerating/potentiation/add‐on when NIBS and drug therapy start simultaneously; switch when drugs are stopped and NIBS is initiated.
Statistics	Increased probability of type I error (false positive).	Determine a priori the statistical test and only one primary hypothesis; refer other comparisons as exploratory.
Statistics	Increased probability of type II error (false negative).	Calculate sample size for an ES ≈ 0.5; use adaptive designs; force as few variables as possible in a GLM.
Statistics	Violation of assumptions of normality.	Increase the sample size; use non‐parametric tests; normalize the data, limit the number of variables imputed.
Construct validity	Diagnostic of MDD, TRD and use of rating scales.	Use structured interview of diagnosis; define refractoriness dichotomically or use a scale of degree of resistance.
Design	Randomization.	Describe methods of randomization and allocation.
Design	Blinding of a non‐pharmacological intervention.	Exclude patients non‐naïve to rTMS; avoid contact of subjects; use external raters; avoid crossover design; assess blinding.
Design	Carryover effects of NIBS are unknown.	Avoid crossover design; perform drug washout; use statistical methods to handle with carryover effects.
Generalizability	Patients with TRD have a smaller response.	Measure the degree of resistance; stratify during randomization; analyze as a covariate; perform a subgroup analysis.
Generalizability	Accelerating trials have two active groups, requiring a large sample size.	Use patients with severe MDD to decrease variability; add a third sham/placebo group; use a factorial design.
Generalizability	Augmentation trials have a wide variability of the sample.	Control TRD, gender and other predictors; use patients taking the same drug and dosage for more than 12 weeks.
Generalizability	Switch trials might not reflect clinical practice.	Enroll patients that do not want or are unable to take oral pills, e.g. pregnant women, patients with several medical conditions.

GLM, general linear model; ES, effect size; MDD, major depressive disorder; TRD, treatment‐resistant depression; NIBS, non‐invasive brain stimulation; rTMS, repetitive transcranial magnetic stimulation.

Internal validity versus generalizability in NIBS trials

There is no definite answer regarding whether a clinical trial should focus on either internal validity (i.e. to prove causality between the intervention and the outcome) or generalizability (i.e. to aim to generalize the observed results outside the experimental situation) (Glasser and Howard, 2006). Usually, small studies focus on internal validity because of strict eligibility criteria and lack of power to detect effects in a heterogeneous population, thereby grouping homogeneous groups which will only differ by the treatment applied. Such approach increases the chance of positive results if treatment is effective – in fact, small studies also perform multiple comparisons for this reason. However, these studies have low power and are prone to fail if internal validity is harmed or the treatment has a modest effect size. Besides, even underpowered trials are useful because: (1) they provide data on the best outcomes to be used in subsequent trials; (2) they provide information on parameters of stimulation and between‐subjects variability; (3) they can be pooled together in meta‐analyses. However, NIBS studies focusing on generalizability are necessary to validate the intervention as an effective treatment. The study of O'Reardon et al. (2007) is an example of an effectiveness study.

Issues related to exploratory studies

Hypothesis testing

Most trials reviewed are vulnerable to both type I and type II errors. Type I error increases when multiple statistical comparisons are performed – in fact, several studies carried out five or more statistical tests for efficacy. However, the majority of negative studies were underpowered, suggesting that these studies had false negative conclusions – in fact, this finding is common in clinical trials, as a meta‐analysis of negative findings in orthopedic trials showed that type II error was 90% (Lochner et al., 2001). Possible solutions here are: (1) to determine a priori the statistical method that will be used for the primary outcome (Hopewell et al., 2008); (2) to claim other analyses as secondary or exploratory; (3) to perform sample size estimations before the trial starts (however the primary hypothesis needs to be defined for the sample size calculation) – given the results based on NIBS meta‐analyses, the estimated effect size for sample size calculations might range from a Cohen's d of 0.4 to 0.75 (Gross et al., 2007; Schutter, 2008); (4) to use adaptive designs, as we discuss later.

Statistics

Most trials reviewed used more complex statistical approaches, such as mixed analysis of variances (ANOVAs) or multivariate ANOVAs. The advantage of using these models is to compare several dependent and independent variables simultaneously without increasing the probability of type I or II errors. However, possible caveats are that: (1) usually several variables are imputed in these general linear models, which might violate the recommended assumptions for using such models (for instance, maximal number of events or entries according to the number of independent variables) (Ottenbacher et al., 2004) and; (2) the use of parametric tests is controversial because: the sample size of most studies were not large enough to fit the Central Limit Theorem assumptions; psychometric scales are more ordinal than continuous scales (Bech, 1988), and; tests for normality were often not reported. To address these issues, we suggest: (1) to determine a priori the statistical tests; (2) to perform normality tests, such as the Kolmogorov–Smirnov test, especially when the sample size is small; (3) to limit the number of variables imputed in the covariate adjustments, usually using only baseline depression scores and, perhaps, degree of resistance; (4) to handle with known predictors of response not only through statistics, but also in the study design, i.e. at the enrollment and randomization phases (defining adequate inclusion/exclusion criteria and using stratified randomization approach, for instance), as such approach would not undermine statistical power.

Randomization

Most studies did not report the randomization methods (a process that ensure that each subject in a trial has the same probability of being assigned for any group) and allocation (the technique used to direct a subject to his/her assigned group preserving randomization and blinding) methods, therefore, we could not evaluate to which extent this issue jeopardizes the internal validity of NIBS trials.

Issues related to MDD trials

The main threat identified here refers to construct validity (i.e. how the diagnosis performed relates to the “real” diagnosis – which concerns to generalizability) and reliability (i.e. the interrater agreement when performing a diagnosis) – in fact, this is also an issue in antidepressant drug trials (Gelenberg et al., 2008). In most studies, MDD diagnosis was made through unstructured interviews, which is a threat to validity – as differential diagnoses, such as bipolar disorders and personality disorders, might have less sensitivity to diagnose these conditions (Zimmerman and Mattia, 1999) – and also to reliability, as unstructured interviews have lesser interrealibility agreeement than structured and semistructured interviews (Williams et al., 2002). Another threat is that the trials reviewed used four different diagnostic criteria to define TRD – in fact, TRD refers to a specific subgroup (15–40%) of patients failing to achieve significant clinical improvement after at least two antidepressant trials (Berlim and Turecki, 2007). Since this subgroup of patients presents lower response rates to antidepressant treatments (Rush et al., 2006) and most NIBS trials enroll TRD patients – as treating with drugs first might be more cost‐effective – future trials need to consider assessing TRD by: (1) using refractoriness criteria as defined by two failed antidepressant trials of different antidepressant classes as commonly referred in the literature (Berlim and Turecki, 2007) or; (2) grading (not only identifying) depression resistance as it is known that the degree of resistance is associated with antidepressant response (Lisanby et al., 2009; Rush et al., 2006).

Finally, it is not clear whether the primary outcome should be based on score change or depression remission: although a complete depression treatment should aim remission of symptoms (Moller, 2008), the use of score change can be useful to assess improvement in TRD patients that present low remission rates (Trivedi, 2006) as well as to increase study power.

Issues related to NIBS trials

Blinding

Blinding is a set of techniques used to keep subjects and/or study staff unaware of the intervention administered. Insufficient blinding might bias results by increasing the effectiveness of the active group (by increased staff care and/or accomplishment of patient's expectancies) and decreasing in the control group (Hrobjartsson and Gotzsche, 2003; Noseworthy et al., 1994). Also, the optimal NIBS blinding technique has not been established yet: some researchers use an angled coil in control group, therefore resulting in staff unblinding; while others prefer to use a sham coil as placebo intervention – which might be a better method of blinding, however the sensation associated with it is still not the same as that induced by the real coil. Along these lines, tDCS devices might provide a more reliable blinding method, since, accordingly to the set of parameters specified, the skin sensation associated with the use of active stimulation might be null after a couple of seconds of stimulation (Boggio et al., 2008; Gandiga et al., 2006) –– yet, at high intensities, patients might notice tingling or other symptoms associated with stimulation, compromising blinding. Therefore, alternative blinding methods – especially for rTMS – should be used, such as avoiding contact between subjects of different groups, excluding patients non‐naïve to rTMS, avoiding crossover trials and blinding raters to treatment applied (Boutron et al., 2007).

Blinding assessment

Only eight studies reported methods to address blinding, mainly by asking subjects or staff personnel to guess where patients were allocated. However, although blinding assessment can be useful to identify whether the blinding was broken (Boutron et al., 2007), this method is controversial (Fergusson et al., 2004) – e.g. correct guessing might be related to side effects and amelioration of symptoms and not to the blinding process. Therefore, blinding realiability might be obtained through other methods, such as familiarizing staff to study procedures, performing team meetings to discuss potential blinding breaking and testing the blinding technique before applying it in a efficacy trial.

Placebo effect of NIBS devices

Non‐pharmacological interventions might have a greater placebo response than pharmacological ones (Kaptchuk et al., 2000) for two theoretical reasons: one is that subjects might have greater expectancies regarding a non‐pharmacological intervention and the other is that sham interventions might have an active effect (Dincer and Linde, 2003). However, such interventions are more difficult to blind, patients might be skeptical (as with a completely different approach) and therefore have less expectancy, and NIBS devices are often used in TRD patients – these factors that might decrease placebo response. Along these lines, a recent meta‐analysis showed that the placebo sham response is similar to a placebo pill response (Brunoni et al., 2009). More studies are needed to explore the possibility of NIBS devices having a greater sham response.

Carryover effects

The carryover effect (i.e. the effect of one intervention being carried over to another) threats internal validity of NIBS trials because in both “add‐on” and “switch” strategies, non‐pharmacological and pharmacological effects might be mixed together; thereby, whether or not a difference is observed, it would not be possible to solely attribute it to the NIBS. To acknowledge this threat, the wash‐out period of antidepressant drugs should be reported, being of at least four half‐lives of the drug (Leucht et al., 2008). The issue is even more controversial in NIBS crossover designs, in which the effect duration of non‐pharmacological interventions is not yet known; therefore, such trials might need a specific analysis of potential carryover effects (comparing baselines before each intervention or including treatment effect in the model).

Treatment‐resistant depression

It is still unclear whether NIBS studies should either include or exclude refractory patients – although refractoriness can theoretically decrease NIBS efficacy as remission rates decay as refractoriness increases (Fregni et al., 2006b; Lisanby et al., 2009; Rush et al., 2006); placebo response might also decrease with refractoriness (Brunoni et al., 2009) and, therefore, NIBS effects could actually be enhanced by a lower placebo response in control group – in fact, an open tDCS study showed promising results in TRD patients (Ferrucci et al., 2009). Useful approaches might be measuring the degree of resistance of each patient with severity rating scales and perform covariate adjustments or, alternatively, classify the subject dichotomically and stratify accordingly during randomization.

Along these lines, other predictors of response, such as gender (Huang et al., 2008) and age (Fregni et al., 2006b) should also be taken into account when designing NIBS trials – these variables might be handled through stratification and/or explored through covariate and multivariate adjustments – although such analyses might increase type I error as well as require a greater sample size to avoid type II error.

Study design

Although most studies enrolled patients who were previously using antidepressant drugs, pivotal studies such as the study of O'Reardon et al. (2007) enrolled drug‐free patients and Herwig et al. (2007) enrolled patients in an accelerating study. Each approach has its strenghts and weaknesses (Figure 1).

Figure 1.

Summary of the non‐invasive brain stimulation (NIBS) studies reviewed, comparing similarities and differences of initial exploratory trials to pivotal trials.

Accelerating studies combine a NIBS and a pharmacological intervention, theoretically fastening response (Padberg and Moller, 2003), which might be useful for patients with severe and/or refractory depression. On the contrary, such design suffers from the lack of a true placebo group as at least one active intervention is conducted for each patient. Therefore, this approach might increase response rates in both groups, thereby demanding a large sample size to detect a significant difference.

Drug‐free patients designs are attractive because there is no confounding effect between NIBS and antidepressant drugs and such studies might be useful for patients who might not be taking antidepressants, as we discuss later. However, it is not known yet whether maintenance treatment with NIBS is effective because there are few follow‐up studies (Demirtas‐Tatlidede et al., 2008). In addition, this adds complexity to the study by requesting a wash‐out period.

Finally, augmentation studies do not evolve wash‐out periods and keep patients on drugs after the trial is over. Also, such approach might also have the greatest external generalizability as it would enroll patients who are still symptomatic after the pharmacological treatment – a putative target population for NIBS approaches. The drawback is that exploratory studies would enroll patients with various grades of refractoriness and different therapeutic regimens; while pivotal NIBS augmentation trials would be the most complex, needing a previous run‐in antidepressant drug trial to select eligible patients (e.g.those who did not achieve clinical response).

Improving future NIBS studies

To conclude, we discuss a few areas of improvement for future NIBS studies.

Use of adaptive designs

Adaptive designs allow modification in on‐going trials without compromising power (Chow and Chang, 2008). Such designs allow to reduce sample size and trial duration (Sagkriotis and Scholpp, 2008), drop weaker treatments (Chow and Chang, 2008) and change primary outcomes during the trial (Banerjee and Tsiatis, 2006); being increasingly used for phase II/III trials. NIBS studies could take advantage of such designs – for instance, by testing either different stimulation “doses” or stimulation sites in the scalp, dropping in advance weaker treatments; or in sample size re‐estimation to achieve statistical power.

Use of translational research

Translational research utilizes knowledge from basic sciences to improve clinical research (Woolf, 2008). Therefore, NIBS efficacy can also be explored through biological findings emerging from MDD translational studies, such as: (1) biological serum markers, such as brain‐derived neurotrophic factor (BDNF), which is related to neuroplasticity in key brain areas related to depression (Brunoni et al., 2008a) with two meta‐analyses showing that the serum levels are related to depression severity and response (Brunoni et al., 2008b; Sen et al., 2008); (2) neuroimaging studies, such as magnetic resonance imaging (MRI) spectroscopy, which measures the cerebral bioenergetic metabolism that is usually low in MDD (Iosifescu et al., 2008), in fact, a recent spectroscopty study showed that brain metabolism is altered after NIBS (Rango et al., 2008); (3) quantitative electroencephalograph (EEG) recordings, which measures brain activity and might be a predictor of antidepressant treatment (Hunter et al., 2007) – also EEG could be used to index brain activity during NIBS, as this technique can be used to index cortical activity (that is the target for NIBS treatment). This would be useful to adjust tDCS/rTMS dosage individually.

Use of NIBS in selected subgroups of patients

There are some patient groups where depression treatment is challenging and where NIBS could be an alternative treatment strategy. Pregnant women, for instance, are often hindered from drug treatment because of safety concerns (Campagne, 2007). Therefore, NIBS could be an adequate acute and maintenance treatment during pregnancy (Klirova et al., 2008). Along these lines, geriatric patients could also benefit of NIBS approaches, since they usually present co‐morbidities and polypharmacy (Rabheru, 2004) hindering optimal antidepressant treatment.

Conclusion

Conducting NIBS trials is challenging as specific methodological issues of brain stimulation are summed with the common design problems faced in depression and in pilot trials. Taken together, these pitfalls can seriously burden study validity and lead to inconclusive results. In our review, we aimed to discuss such pitfalls and, consequently, to provide further directions for future clinical trials on NIBS for MDD.