Effects of electroconvulsive therapy on cognitive functioning in patients with depression: protocol for a systematic review and meta-analysis

Abstract

Introduction

Depression is the leading cause of disability worldwide, affecting approximately 350 million people. Evidence indicates that only 60–70% of persons with major depressive disorder who tolerate antidepressants respond to first-line drug treatment; the remainder become treatment resistant. Electroconvulsive therapy (ECT) is considered an effective therapy in persons with treatment-resistant depression. The use of ECT is controversial due to concerns about temporary cognitive impairment in the acute post-treatment period. We will conduct a meta-analysis to examine the effects of ECT on cognition in persons with depression.

Methods

This systematic review and meta-analysis has been registered with PROSPERO (registration number: CRD42014009100). We developed our methods following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. We are searching MEDLINE, PsychINFO, EMBASE, CINAHL and Cochrane from the date of database inception to the end of October 2014. We are also searching the reference lists of published reviews and evidence reports for additional citations. Comparative studies (randomised controlled trials, cohort and case–control) published in English will be included in the meta-analysis. Three clinical neuropsychologists will group the cognitive tests in each included article into a set of mutually exclusive cognitive subdomains. The risk of bias of randomised controlled trials will be assessed using the Jadad scale. We will supplement the Jadad scale with additional questions based on the Cochrane risk of bias tool. The risk of bias of cohort and case–control studies will be assessed using the Newcastle-Ottawa Scale. We will employ the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) to assess the strength of evidence.

Statistical analysis

Separate meta-analyses will be conducted for each ECT treatment modality and cognitive subdomain using Comprehensive Meta-Analysis V.2.0.

Introduction

According to the WHO, depression is the leading global cause of disability. Approximately, 350 million people suffer from depression worldwide.¹ Despite the availability of numerous psychopharmacological treatments, evidence indicates that only 60–70% of persons who tolerate antidepressants will respond to first-line drug therapy for major depressive disorder (MDD).² Furthermore, at least one-third of persons with MDD who receive drug therapy will become treatment resistant.³ Various definitions have been proposed for treatment-resistant depression (TRD). The European Agency for the Evaluation of Medicinal Products has defined TRD as the failure to respond to two drugs of different classes, provided these drugs are used for a sufficient length of time and at an adequate dose.⁴ TRD has also been defined as failing four or more different therapeutic antidepressant regimens, including augmentation, combination and electroconvulsive therapy (ECT).⁵

The aetiology of TRD is unclear. Various clinical factors have been associated with treatment non-response and resistance in MDD,^{6 7} including non-adherence to treatment, poor tolerability to antidepressant medications, and medical and psychiatric comorbidity. Researchers have also identified comorbid post-traumatic stress disorder⁶ and the presence of early life adversity⁷ as important predictors of incomplete treatment response.^6–8

ECT is considered an effective acute treatment for TRD⁹ in either unipolar or bipolar depression.¹⁰ ECT is used primarily when antidepressant medications do not result in adequate response in TRD.¹¹ Approximately, 100 000 persons annually receive ECT in the USA.¹² However, the use of ECT remains controversial due to concerns about temporary cognitive impairment in persons with depression who receive acute ECT. Indeed, retrograde and anterograde memory deficits are among the more reliably reported cognitive changes due to ECT.⁹ The UK ECT Group also found that differences in ECT treatment modalities (eg, electrode placement, pulse shape, treatment frequency and treatment dosage) had a differential impact on the incidence and duration of cognitive impairment in persons with depression.⁹

Semkovska and McLoughlin¹³ examined the issue of cognitive impairment post-ECT in a recent meta-analysis. After pooling results by cognitive test, these authors found that cognitive impairment was limited to a post-treatment period of 3 days. Although Semkovska and McLoughlin¹³ did assess risk of bias, these results are not reported in the manuscript nor did they report the grading of the strength of evidence.

The purpose of the present study is to conduct a systematic review and meta-analysis of the effects of ECT on cognition in persons with depression. We seek to quantify the effect of different ECT treatment modalities on the occurrence and duration of cognitive impairment. The present review includes comparative studies only (randomised controlled trials (RCTs), cohort and case–control), which are among the highest levels of evidence. Additionally, the review only includes studies where cognitive function as an outcome is reported using standardised neuropsychological tests or self-report measures that are grouped into mutually exclusive cognitive subdomains.

In contrast to Semkovska and McLoughlin,¹³ results in the proposed review are grouped by cognitive subdomains, rather than cognitive tests. The focus on cognitive subdomains is a closer reflection of clinical and research practice. In these settings, multiple tests are available to assess performance within individual cognitive domains (eg, verbal recollective memory). The current literature reflects this heterogeneity, with multiple measures reported across studies to assess key cognitive domains that have become the focus of intense research interest. Inclusion of a wider corpus of measures within common cognitive domains reflects clinical and research practice. In further contrast to Semakovska and McLoughlin, we include studies that actively compare more conservative ECT treatments (eg, unilateral) to less conservative (eg, bilateral) ECT treatments. A primary outcome is post-treatment between-group differences in cognition for persons receiving less conservative versus more conservative ECT treatments. By contrast, Semkovska and McLoughlin¹³ compared pretreatment and post-treatment scores on cognitive tests. Although they stratified by some components of treatment modality, the resulting comparisons were within-group differences, rather than between-group (between-treatment) comparisons. From a clinical perspective, it is crucial to determine whether the impact of cognitive impairment differs between treatments. Furthermore, by including studies that measured subjective memory in addition to objective neuropsychological measures of memory, we are able to compare and contrast potential differences in these aspects of memory functioning following treatment. Finally, we provide key data concerning the risk of bias of the included studies and rate the overall strength of evidence.

Methods

This systematic review and meta-analysis was registered with PROSPERO (registration number: CRD42014009100; http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42014009100).¹⁴ We based the review methods on the Preferred Reporting Items for Systematic Reviews and Meta-Analysis¹⁵ statement.

Literature review

We are searching MEDLINE, PsychINFO, EMBASE, CINAHL and the Cochrane Central Register of Controlled Trials from database inception to the end of October 2014. The literature search mirrors the search employed by the UK ECT Review Group.⁹ We consulted a medical librarian to add specific search terms to narrow our focus to the identification of articles about cognitive side effects. The final search terms included: electroconvulsive therapy; electroshock therapy; ECT; shock therapy; convulsive therapy; mood disorders; depression; schizophrenia-and-disorders-with-psychotic-features; personality disorders; delirium-dementia, -amnesic, -cognitive-disorders; bipolar disorder; randomized-controlled-trials; random*; cohort-studies; case-control-studies; double-blind-method; single-blind-method; follow-up-studies; attention; orientation, learn*; memory; concentration; cognit*; mental-process*; executive functioning; visuospatial; language; intelligence; intellectual functioning; motor function; neuropsychology. We are also searching the references of published reviews and health technology assessments related to ECT and cognition.^{9 10 13 16–18}

Inclusion/exclusion criteria

We are including studies retrieved in the literature search that meet the following criteria:

1. Comparative studies (RCTs, cohort studies and case–control studies) assessing less versus more conservative ECT treatments;

2. Outcomes measured using standardised neuropsychological tests and self-report memory measures with established psychometric properties;

3. Diagnosis of participants with a major depressive episode (Diagnostic and Statistical Manual of Mental Disorders (DSM)-III, DSM-III-R, DSM-IV, DSM-IV-TR, Research Diagnostic Criteria (RDC), International Classification of Diseases (ICD)-9, ICD-10) or endogenous depression; and

4. Published in English.

Study selection and data extraction

Two reviewers are independently applying the inclusion and exclusion criteria to the citations retrieved in the literature search. This screening process is divided into two levels and it is guided by standardised instructions. For the first screening level, reviewers are independently evaluating the titles and abstracts. Citations that fulfil the inclusion criteria are advanced to the second screening level. Advancement also occurs if the reviewer does not find sufficient information to determine whether the citation fulfils the inclusion criteria. For the second screening level, the complete scientific paper is read to determine whether the inclusion criteria are met. At both levels, mutual agreement is required from the reviewers to advance a study. Discrepancies are resolved by consensus. When consensus is not attained, a third reviewer independently reviews the study in question and makes a final decision. We will use weighted κ to measure inter-rater agreement between reviewers at both levels of screening.

Studies that pass the second screening level advance to data extraction. A team of trained reviewers extracts data from the included studies. Standardised forms and training guide the data extraction process. The following information is extracted from each article: study design, mean age, proportion of men and women, diagnosis, co-morbidity, illness duration, illness severity, age of illness onset (in years), number of illness episodes, sample size, ECT description, total number of ECT sessions, comparator group characteristics, length of follow-up, treatment modality and cognitive outcomes. Examples of treatment modalities (less vs more conservative modalities) include bilateral versus unilateral ECT, three times versus twice weekly treatment, ultrabrief versus brief pulse, sine versus pulse, ECT versus pharmacological treatment, ECT versus no treatment and ECT versus sham. The first author of this protocol (CO) reviews the extracted data to verify the accuracy of the work. We are contacting the authors of included studies to obtain information that may be missing from the published reports.

Cognitive subdomains

Since cognitive outcomes in ECT studies are reported using a wide range of measurement instruments that increase the number of variables across and between studies, we grouped these instruments into cognitive subdomains to facilitate data extraction, reporting and analysis. Three experienced clinical neuropsychologists (BL, HM and MM) generated a list of subdomains by reviewing the included papers, identifying the cognitive instruments, and grouping these instruments into cognitive subdomains. Disagreements about domain assignment are resolved by consensus. The cognitive subdomains are: verbal memory-immediate recall, verbal memory-delayed recall, verbal memory-recognition, non-verbal memory-immediate recall, non-verbal memory-delayed recall, non-verbal memory-recognition, working memory, attention, intellectual ability, executive function, processing speed, spatial problem solving, global cognitive status, language, motor and construction/visuospatial. In addition, autobiographical memory and subjective memory as measured by standardised self-report tools are included. Notably, narrative comparison of outcomes assessed by objective and subjective measures is critical, given that patients’ subjective report of cognitive performance may differ significantly from that captured by objective measurement.

Assessment of risk of bias

Following data extraction, two reviewers will independently assess the risk of bias of each included study. Discrepancies will be resolved by consensus. If consensus is not reached, a third reviewer will decide. The risk of bias of RCTs will be assessed using the Jadad scale¹⁹ which has six questions comprising the following domains: randomisation, double blinding, tracking of withdrawals and adverse effects, appropriate use of statistics, and inclusion and exclusion criteria. We will supplement the questions on the Jadad scale with additional questions (yes/no responses) about the adequacy of allocation concealment, use of intention-to-treat analysis, justification of sample size, reporting of outliers and selective outcome reporting. Some of these additional questions are based on the Cochrane risk of bias tool²⁰; the addition of questions to existing scales has been used in other meta-analyses.²¹

The risk of bias of cohort and case–control studies will be assessed using the Newcastle-Ottawa Scale (NOS).²² The NOS is divided into two subscales, one for cohort and the other for case–control studies. Both subscales assess the following three domains: selection of study groups, comparability of study groups and ascertainment of exposure or outcome.

Using the responses to the aforementioned scales and questions, reviewers will qualitatively assess the risk of bias for each study as ‘low’, ‘unclear’ or ‘high’. According to the Cochrane Collaboration, ‘low’ means any bias is unlikely to substantively alter a study's results, ‘unclear’ means the bias causes doubts about the results, and high means the bias is likely to threaten the validity of the results.²⁰

Grading the strength of evidence

We will use the BMJ Evidence Centre guidelines for Grading of Recommendations Assessment, Development, and Evaluation (GRADE)²³ to judge the overall quality of evidence for specific subdomains. In situations where the group of studies assessing a specific subdomain has a low quality of evidence, one would hold little confidence in the validity of the results. One would also be hesitant to draw firm conclusions or make clinical recommendations based on these results. Future studies—assuming they present a higher quality of evidence—might provide a stronger basis from which to draw conclusions or make clinical recommendations.

We will use GRADE to rate the evidence separately for each cognitive subdomain. We will begin by assigning four points to each subdomain if the evidence is largely based on RCT data, or two points if the evidence is largely based on observational study data. We will then assess four other categories, that is, quality, consistency, directness and precision, and add or deduct points for each category in accordance with GRADE guidelines.²³ The additions or deductions reflect preset criteria for assessing how the components of each category contribute to the overall quality of evidence. The final point total serves as the overall GRADE²³ score: scores of 4 or more indicate high quality of evidence, a score of 3 would indicate moderate quality, 2 would suggest low quality and less than 2 would indicate very low quality. The level of confidence to make clinical recommendations based on the evidence would be stronger for higher overall scores.

GRADE's ‘quality’ category will include the risk of bias assessments. The Cochrane guidelines for ascertaining risk of bias across studies will be used to synthesise the risk of bias findings for individual studies.²⁰ These guidelines classify groups of studies according to low, unclear or high risk of bias. We will deduct points on the quality category as follows: low risk of bias (−1), unclear risk of bias (−2), high risk of bias (−3).

Statistical analysis

After all data have been extracted from the included studies, the investigators will examine the extraction tables and determine whether meta-analysis is possible. We will only conduct meta-analyses on studies that are relatively homogeneous in terms of participants (eg, age, sex, comorbidity). In the event between-study heterogeneity precludes a meta-analysis, or only permits us to conduct a meta-analysis on a subset of studies, we will undertake a narrative synthesis²⁴ of all of the included studies.

Studies that are sufficiently homogeneous in terms of participants will be meta-analysed. We will conduct separate meta-analyses for each cognitive subdomain. Within each subdomain, we will stratify the analyses by study design (RCT, observational, RCT and observational combined). The summary estimates computed in the meta-analyses will compare the differences in post-ECT cognitive impairment between groups receiving less versus more conservative ECT treatments. Initially, these comparisons will take the form of mean between-group differences in scale score. Differences in scale score are, however, difficult to interpret across disparate scales because of variations in score ranges (eg, a mean difference of 1.0 is larger on a scale that ranges from 0 to 5 relative to a scale that ranges from 1 to 100). Even standardised mean differences can be difficult to interpret clinically because no threshold exists to mark the minimum important difference in score. Therefore, we will report the study-specific and summary estimates as odds ratios (ORs) in all forest plots. ORs greater than 1.0 will indicate that persons receiving less conservative modalities have greater odds of developing cognitive impairment than persons receiving more conservative modalities. ORs less than 1.0 will show the reverse; ORs equal to 1.0 will suggest no difference between modalities.

We will record all study-specific outcomes as means and SDs or, if unavailable, as mean differences. Borenstein et al's²⁵ formulae, implemented through Comprehensive Meta-Analysis V.2.0 software,²⁶ will transform all entered data into ORs and generate forest plots. Forest plots will be computed using a fixed-effects model. We will test statistical heterogeneity for each meta-analysis using the I² statistic. If the I² value is 50% or higher, then we will recompute the forest plot using a random-effects model. Comprehensive Meta-Analysis will generate funnel plots to enable the assessment of publication bias.