Autobiographical Memory and ECT: Don’t Throw Out the Baby

Abstract

Retrograde amnesia for autobiographical information is the most critical side effect of ECT. Much, if not most, modern research demonstrating long-term autobiographical amnesia following ECT has used either the Columbia University Autobiographical Memory Interview (CUAMI) or the short form of this scale (CUAMI-SF). Semkovska and McLoughlin claimed that studies using these instruments should be dismissed and the findings ignored due to a lack of normative data, as well as concerns about the reliability and validity of these instruments. In this commentary, the development and use of these scales is reviewed. It is shown that Semkovska and McLoughlin’s critique is factually incorrect, as normative data were simultaneously collected in virtually all studies using these instruments. Furthermore, there is substantial evidence supporting the reliability and validity of these scales. Indeed, these instruments are the only neuropsychological tests repeatedly shown to covary with patient self-evaluations of ECT’s effects on memory, and have repeatedly demonstrated long-term differences in the magnitude of amnesia as a function of ECT technique. Findings with the CUAMI and CUAMI-SF provide key evidence regarding ECT’s cognitive side effect profile. It is inaccurate and inadvisable to continue to deny that ECT can exert long-term adverse effects in this domain.

Amnesia for autobiographical information is the most critical adverse cognitive effect of ECT.^1–3 Following ECT, many patients report persistent memory loss for events that occurred during the ECT course and the weeks and months before the course.^4,5 Indeed, some patients report a dense retrograde amnesia (RA), with gaps in memory of autobiographical events (e.g., vacations, illnesses, weddings, etc.) extending back several years.⁶ This phenomenon stands in contrast to the improvement in many neuropsychological domains seen shortly following the ECT course.^7,8

Janis^9–11 and Squire^12–16 pioneered the study of postECT RA for autobiographical information. Their descriptions of this phenomenon, derived from studies of small samples using limited instruments, nonetheless is largely consonant with the conclusions of modern research.^1,17 Many, if not most, modern studies of autobiographical RA following ECT have used either the Columbia University Autobiographical Memory Interview (CUAMI) or the Columbia University Autobiographical Memory Interview – Short Form (CUAMI-SF). My research team and I at the New York State Psychiatric Institute (NYSPI) and Columbia University developed these instruments,^18,19 based on an earlier measure, the Personal Memory Interview, authored by Weiner and Squire.²⁰ In a recent review in this journal, Semkovska and McLoughlin²¹ claimed that studies using the CUAMI and CUAMI-SF should be dismissed and their findings regarding RA ignored. Indeed, in their earlier meta-analysis of the objective cognitive effects of ECT they did not include autobiographical amnesia among the domains examined because of their contention that the standard measures in ECT to assess this domain, the CUAMI and CUAMI-SF, lack reliability and validity.⁸

Were the position of Semkovska and McLoughlin taken seriously (and it should not be), the implication would be that, after nearly 80 years of clinical use, the field of ECT has failed to assess properly its most critical and persistent side effect, and the one that is the most frequent source of patient complaint. In fact, Semkovska and McLoughlin go further and conclude that by rejecting the findings using these scales, there is no empirical evidence that ECT results in RA for autobiographical information at any time point.²¹

Later, we will examine the basis for Semkovska and McLoughlin’s misgivings about these scales and see that they are patently wrong, based principally on a fundamental misunderstanding about our work, or otherwise, remarkably narrow. However, before evaluating Semkovska and McLoughlin’s concerns and deciding whether or not to throw the baby out with the bath water, let’s examine the baby — what have we learned with the CUAMI and CUAMI-SF.

Studies Using the CUAMI

Four consecutive, double-blind, randomized trials were conducted at the NYSPI. The CUAMI was developed during the first of these trials, which contrasted right unilateral (RUL) and bilateral (BL) ECT, with both administered with an electrical dosage just above the seizure threshold.²² The CUAMI was then used in the next three studies.^23–25

The CUAMI is administered in a structured interview format. The baseline interview conducted prior to ECT involves queries about 281 personal events or event details. The events include illnesses and hospitalizations, work history, places of residence, travel and entertainment activities, and significant, as well as, everyday events experienced by the patients, their families, or their friends. Of these queries, 185 items require a descriptive response, such as providing an address or naming a person. Other items require a yes/no, date, or list answer. To test for a temporal gradient in RA, there are 28 queries explicitly about recent events (i.e., took place within the prior year), and 40 queries explicitly about remote events (i.e., took place more than 1 year earlier). A subset of these recent and remote event queries are matched for item content. To examine RA as a function of the affectivity of memories, inquiries also deliberately focus on negatively-charged, positively-charged, and emotionally-neutral events. Subsets of the queries about negative and positive events are also matched for content. For example, participants identify both the best and worst trip they had ever taken and are asked to provide the same details about these events.

In the 3 NYSPI studies to use the CUAMI, interviews were conducted with depressed patients while medication-free prior to ECT, medication-free within a few days of terminating the randomized ECT course, and at two-month follow-up. In the third study, the CUAMI was also administered at a six-month follow-up to further examine the persistence of deficits.

The baseline CUAMI interview in depressed patients typically took between 1–3.5 hr. In the first NYSPI study to use the CUAMI,^19,23 family members or close friends were also administered the interview (from the point of view of the patient) to identify a subset of items that could be corroborated for accuracy. This was done to guard against the possibility that depressed patients may be especially likely to be inaccurate in their recall of prior events and that inconsistency in responses over time could be due to more accurate recall at later time points. In this first study, of the 75 patients with baseline CUAMI data, a family member or close friend corroboration interview was available in 52 instances.

In this first study, a normal control sample was also interviewed on one occasion. This group, negative for lifetime psychiatric disorder, was matched to the patient sample in the distributions of age, gender, education, socioeconomic status, and verbal IQ.²⁶ The purpose of the normal control group was to estimate the extent to which productivity of memory at baseline was reduced in the patient group, thereby perhaps biasing postECT RA measures.¹⁹ In the second and third studies to use the CUAMI, a similarly selected normal control group was tested on two occasions, averaging about 4 weeks apart, corresponding to the average interval between preECT and immediate postECT testing in patients. This allowed for the determination of the extent to which CUAMI amnesia scores in patients exceeded the rates of normal forgetting or inconsistency in memory reports over time.

In all studies, CUAMI interviews after baseline (e.g., immediate postECT) only inquired about items that had a definite reply at baseline. Inquiries were not made about items that at baseline participants said they did not know or could not remember the answer or that the query did not apply to them. In this way, a subset of items was identified for each participant such that the participant had evidence of a definite memory at baseline. Thus, the CUAMI RA scores concerned only material that was previously “known” and did not attempt to quantify the extent that participants would provide responses at follow-up when they could not do so at baseline.

The CUAMI elicits multiple details about 43 discrete events (e.g., best trip = a trip to Italy). If at follow-up testing, the participant did not spontaneously recall a specific event or were inconsistent in the event reported (e.g., now a trip to California instead of Italy), they were reminded about the original description. When participants explicitly recognized the original event, subsequent queries about details pertained to the event described at baseline. When participants did not recognize the original event, there was no questioning about the details of this event.

The primary outcome measures in all three NYSPI studies using the CUAMI focused on the 185 items requiring a descriptive response (43 discrete events and 142 event details). The key measures of RA were the percentage of responses at follow-up that were consistent with responses at baseline and the number of pure memory failures. Pure memory failures were instances in which the participant could no longer provide any information about a query that was previously definitely answered. Consistency (or its inverse, inconsistency) provides a more liberal measure as it includes, as indexing RA, both pure memory failures and changes in the description of events and event details.

The first study using the CUAMI at NYSPI randomized patients to right unilateral (RUL) and bilateral (BL) electrode placements.^19,23 Patients were also randomized to low (just above seizure threshold [ST]) and moderate (2.5 x ST) dosage conditions. Patients who did not meet response criteria received a second, crossover course of BL ECT at 2.5 x ST, regardless of their original randomized treatment assignment.

The findings in this study, reported in detail by McElhiney et al.,¹⁹ demonstrated that immediately following the randomized ECT course, patients treated with BL ECT were more inconsistent in their responses and had substantially more pure memory failures than patients randomized to RUL ECT. This advantage for RUL ECT was maintained across multiple consistency measures, including separate analyses for recent and remote events, and affectively-charged and neutral events. In contrast, there was no indication that electrical dosage condition impacted on RA scores.

At two-month follow-up, patients who received a second, crossover course of BL ECT had inferior RA scores than those who received only one course of ECT. Indeed, relative to the assessment immediately after the randomized ECT course, RA scores deteriorated in the crossover patients and improved somewhat in the patients who received only one ECT course. Furthermore, there were also effects of initial electrode placement on the long-term RA measures. Patients randomized to BL ECT had inferior consistency scores at the two-month reassessment than patients randomized to RUL ECT.

Thus technical factors in the administration of ECT, specifically electrode placement and receiving 2 vs. 1 courses of ECT, but not electrical dosage, impacted on CUAMI RA scores. There is the theoretical possibility that when a subgroup demonstrates greater inconsistency over time in memory reports, increased rather than decreased accuracy in memory recall may be mediating this difference. There were three sets of evidence that indicated that this was a very unlikely explanation of the effects found with the CUAMI. Across multiple analyses, the effects obtained with inconsistency scores were tracked by those obtained with scores for pure memory failure, the “don’t remember” responses. It would be absurd to argue that the higher rate of pure memory failure with BL or crossover ECT reflected an improvement in the accuracy of recall, since patients in these groups were more commonly stating they did not remember the answer to a query that they previously provided a definite response. While the effects seen with the primary consistency measure were tracked by the measure of pure memory failure, the effect sizes tended to be greater with the consistency measure. This led to its preferential use in subsequent studies.

A second reason for doubting that improved accuracy over time substantially impacted on the CUAMI findings concerned the corroboration provided by family members and friends. In the patient group as a whole, there was greater consistency over time for corroborated vs. non-corroborated items, demonstrating perhaps a difference in the memorability or resistance to RA of events and event details known by others. Regardless, when restricting analyses to only corroborated baseline reports, the pattern of findings was unaltered. For example, the effects of treatment parameters (electrode placement, number of ECT courses) were maintained. Indeed, in light of the fact that there was marked redundancy in the effects observed with all items and only corroborated items, the use of family/friend corroboration was deemed unnecessary and dropped in future studies using the CUAMI.

The third factor impacting on the role of improved accuracy was the fact that in this study consistency scores at two-month follow-up were computed in two ways. By one method, responses at two-month follow-up were rated as consistent only if they matched the response at baseline. By a second method, responses were scored as consistent if they matched either the baseline or immediate postECT response. The second method allows for the theoretical possibility that patients provide a more accurate response at immediate postECT relative to baseline and then maintain that response at follow-up. However, intensive analyses revealed no difference between these scoring methods in the patterns revealed at two-month follow-up. Consequently, in all future work, the CUAMI and CUAMI-SF consistency scores have been computed only in relation to agreement with baseline responses.

The negative findings in this first study are also noteworthy. In line with previous research on anterograde memory and RA for public events,^1,20,27,28 there was no evidence that RA for autobiographical information was associated with clinical outcome. Some psychological theories of ECT mechanisms had argued that RA, especially for negative memories, was responsible for ECT’s therapeutic benefit.^9,10,29 In contrast, whether examined using dichotomous classification of response or using continuous measures of symptom severity, there was no indication that symptomatic improvement covaried with CUAMI RA scores. Indeed, in contrast to some theories of mood congruence,^30,31 there was also no evidence that the degree of RA varied for positively- vs. negatively-charged memories for the sample as a whole or as a function of clinical outcome.

The second NYSPI study to use the CUAMI randomized depressed patients to 3 forms of RUL ECT (1.5, 2.5 or 6.0 x ST) or a gold standard BL ECT (2.5 x ST).²⁴ CUAMI consistency and pure memory failure scores were primary outcomes measures immediately following ECT and at two-month follow-up. The principal findings were that immediately following ECT patients randomized to BL ECT had significantly more inconsistency and more pure memory failures than each of the three RUL groups, which did not differ from each other. Particularly important clinically was the fact that high-dosage RUL ECT (6 x ST) was superior in CUAMI RA measures than the BL ECT (2.5 x ST) condition. Furthermore, the advantage of a single course of RUL ECT over a single course of BL ECT remained signficant at the two-month follow-up. These findings replicated the findings of the prior study and indicated that electrode placement was more critical than dosage condition in moderating RA for autobiographical information and that electrode placement effects could be discerned two months after the completion of ECT.

The last NYSPI trial to use the CUAMI randomized depressed patients to pulse width (ultrabrief: 0.3 ms vs. standard brief pulse: 1.5 ms) and electrode placement (RUL ECT at 6 x ST vs. BL ECT at 2.5 x ST) conditions.²⁵ In this study only CUAMI consistency scores were reported, although the effects again were paralleled in pure memory failure scores (unpublished data). Immediately following ECT, there were marked effects of pulse width, electrode placement, and number of treatments. Patients who received standard brief pulse ECT (1.5 ms) were markedly less consistent in their responses than patients treated with an ultrabrief stimulus (0.3 ms.). Patients treated with BL ECT had poorer consistency scores than patients treated with RUL ECT, regardless of pulse width. Further, across the sample, a larger number of treatments was associated with poorer consistency scores.

Follow-up investigations were conducted immediately following receipt of a crossover course (BL ECT, 1.5 pulse width, 2.5 times ST) in nonresponders, and at two- and six-month follow-up in all patients. The findings again indicated that receipt of a second, crossover course of ECT resulted in further deterioration of CUAMI consistency scores. At the two-month and six-month follow-ups, patients who received ultrabrief ECT, whether RUL or BL, had markedly superior consistency scores compared to patients treated with a single course of RUL or BL brief pulse ECT or who had received a second crossover course. While there were indications that patients treated with a single course of brief pulse BL ECT or crossover ECT had inferior scores at both follow-ups than patients treated with a single course of RUL ECT, the effects of pulse width were far more substantial. Thus, it was concluded that choice of pulse width profoundly impacts on the extent of long-term RA for autobiographical information. It was also observed that the number of ECT treatments administered in the randomized phase was associated with RA scores at the two- and six-month follow-up, with more treatments linked to greater RA.

While conducting these randomized clinical trials, we also used the CUAMI to examine key issues in our understanding of ECT-induced RA for autobiographical information. Clinically, it is critical to identify predictors of the extent of long-term RA in order to identify beforehand the most vulnerable patients or to modify the administration of the treatment to lessen the probability of long-term adverse outcomes. Two possibilities had been previously suggested. There is a widely held view that individuals with pre-existing cognitive impairment or neurological insult may be more at risk for persistent negative cognitive outcomes following ECT.¹⁷ It had also been speculated that the disorientation that immediately follows ECT seizure termination is a form of rapidly shrinking RA.^32,33 It was thought that the more severe the disorientation, as measured by time to recover full orientation, the greater the RA following the treatment course.

Sobin et al.³⁴ used the sample from the first NYSPI trial to apply the CUAMI²³ to test whether modified Mini-Mental State (mMMS)³⁵ scores prior to ECT and postictal orientation recovery times during ECT predicted CUAMI consistency scores. Indeed, there were strong, linear relationships between both baseline mMMS score and postictal reorientation time in predicting the extent of RA at the CUAMI assessment during the week following ECT. These effects remained signficant when controlling for age, dosage and electrode placement condition, absolute electrical dosage, seizure duration, and total number of ECT treatments. The associations with baseline mMMS score and with postictal reorientation time were also signficant when predicting the RA consistency score assessed two months following the ECT course. Thus, this study presented the first empirical evidence that pre-existing global cognitive impairment and the duration of postictal disorientation predicted the magnitude of RA following ECT both in the short- and long-term.

Sackeim et al.³⁶ recorded resting, eyes closed, 19-lead EEG prior to ECT, before the penultimate treatment, and during the week following the ECT course in the sample that also participated in the first of the NYSPI trials to apply the CUAMI.²³ They demonstrated a specific topography of EEG changes in the theta band correlated with CUAMI consistency scores during the week following ECT. Increased theta band activity in left frontotemporal regions was associated with increased RA. The same topographic EEG pattern was also associated with the duration of postictal disorientation during the ECT course, providing further linkage between this phenomenon and RA for autobiographical information. In contrast, other EEG changes were associated with the change in mMMS scores, demonstrating specificity in neuropsychological correlates. The findings associating RA with increased left frontotemporal theta activity were consonant with the view that the altered septohippocampal function subserved the neuropsychological deficit. It has long been thought that dysfunction in medial temporal lobe structures, and the hippocampus in particular, contribute to the RA following ECT.^16,37,38

The foregoing studies demonstrated that CUAMI RA scores are sensitive to aspects of treatment administration (i.e., electrode placement, pulse width, treatment number), individual differences in global cognitive status prior to ECT, and the duration of postictal disorientation during the ECT course. These scores were also found to covary with a specific topographic pattern of altered physiological activity. Brakemeier et al.³⁹ using the sample from the third NYSPI trial to apply the CUAMI, linked CUAMI consistency scores to an entirely different domain, patient’s self-evaluation of their memory functioning.

Historically, studies of patient subjective reports of cognitive function following ECT found no relationship with objective neuropsychological measures.^1,40 Indeed, at time points when objective amnestic effects can be readily identified, ECT patients typically report improved cognitive status on subjective measures like the Squire Memory Complaint Questionnaire (SMCQ) and Cognitive Failures Questionnaire (CFQ).^40,41 Change in scores on these subjective instruments usually covary strongly with the extent of clinical improvement,⁴¹ raising doubt about their validity. In addition to administering the SMCQ and CFQ, Brakemeier et al.³⁹ used a novel instrument, the Global Self-Evaluation of Memory (GSE-My) (see also ⁴²). The GSE-My has only a single item, modeled after Clinical Global Impression (CGI) scales.⁴³ Using a Likert-scale, patients rate the extent to which they believe the course of ECT helped or hurt their memory.

Across the sample, as expected, patients had improved scores on the SMCQ and the CFQ when assessed during the week following the randomized ECT course. In contrast, as a group, the sample reported poorer memory function on the GSE-My. Also in contrast to the GSE-My, the traditional scales were insensitive to the forms of ECT administered or number of treatments. On the GSE-My, however, patients who received ultrabrief RUL ECT had the most positive evaluations and differed significantly from the brief pulse, RUL ECT group. Across the sample, a larger number of ECT treatments was associated with reports of greater impairment on the GSE-My. Associations were also examined between the three subjective measures and five measures of cognitive change, including the CUAMI. No signficant relationship emerged with the SMCQ or the CFQ with any neuropsychological measure. In contrast, GSE-My scores showed trends with a measure of spatial anterograde amnesia and a measure of RA for public events. A significant association was found for a measure of verbal anterograde amnesia. However, by far, the most robust association was found with the CUAMI consistency score. Greater subjective memory impairment following ECT was associated with more severe RA.

Studies Using the CUAMI-SF

A major drawback of the CUAMI is its time demands; baseline interviews can require more than 3 hours to administer, and considerable additional time is needed for transcribing and scoring these interviews. It was clear that this instrument could be used with severely depressed patients only in highly specialized research settings, especially given the typical time pressures to start ECT. From a measurement viewpoint, it was also clear that the CUAMI was highly reliable, as it had revealed persistent deficits months after ECT, and strong relations with treatment parameters, individual difference measures, physiological indices, and patient self-report. However, it is likely that the instrument has many extraneous items, as the principal dependent measures derived only from the subset of items that required a descriptive response, while other types of items were ignored. Furthermore, the instrument includes sampling of recent and remote events, and emotionally-charged (positive and negative) and neutral events. These items were included to examine the intrinsic characteristics of the RA, but were likely “overkill” in providing a measure of the overall severity of RA for autobiographical information. Thus, the CUAMI-SF was created to address the need for a much briefer instrument for use in multisite studies and in routine clinical practice.¹⁸

The items on the CUAMI-SF concern 6 events: last major overnight trip, last New Year’s Eve, last birthday, most recent employment, most recent medical illness, and details about an important family member or friend. For each of these six categories, five queries are made to probe memory of specific details, producing a total of 30 items. The event and detail queries were selected from the much larger set in the CUAMI on the basis of two criteria. First, we identified events and details that produced high and equivalent rates of response at baseline in the depressed patients and matched normal control sample administered the CUAMI at NYSPI. Second, among potential items, we selected those that maximized the difference between RUL and BL ECT in the randomized NYSPI studies. We presumed these selection criteria would enhance sensitivity of the instrument to RA for autobiographical information.

The CUAMI-SF is administered and scored in a manner similar to the CUAMI. In particular, at follow-up testing, inquiries are only made about items that had a definite response at baseline. Thus the collection of memories subject to assessment is individualized for each participant and memory consistency is tested only for items that at baseline provoke an identifiable memory. Scoring at follow-up allows partial credit when reports partially correspond to baseline responses. The baseline response is automatically scored 2 if a definite and identifiable memory is elicited and otherwise scored 0, corresponding to reports that participants do not know or remember the answer to the question or that the query does not apply to them. Only queries scored as 2 at baseline provide the material tested at follow-up. Follow-up responses are scored as 0 (no response or fully inconsistent), 1 (partially consistent), or 2 (fully consistent). The RA score used in research with the CUAMI-SF is the total score at a follow-up relative to the score at baseline. This reflects the percent consistency in responses and is maximally 100%, with increasing inconsistency resulting in lower scores. The CUAMI-SF usually takes 15–20 minutes to administer.

The CUAMI-SF was first applied in the multisite study examining efficacy and cognitive effects in community ECT settings, often referred to as the “Services” study.⁷ This prospective, observational study was conducted with an intent-to-treat sample of 347 depressed patients, who received ECT at 7 different hospitals in the New York City metropolitan area.^7,44 The CUAMI-SF, a primary outcome measure in this study, was administered as part of a larger neuropsychological battery prior to ECT, within a few days of ECT termination, and at six-month follow-up. Normal controls, never psychiatrically ill and matched to the patient sample in the distributions of age, gender, and education were tested at the same intervals as patients. CUAMI-SF consistency scores in patients were adjusted for the extent of inconsistency found in the normal comparison sample over time. This allowed determination of the extent to which CUAMI-SF inconsistency scores at follow-up exceeded the normal rate of inconsistency in autobiographical recall.

In the total patient sample, deficits on the CUAMI-SF were marked at postECT. While most neuropsychological measures showed improvement relative to baseline at the six-month follow-up, consistency (RA) scores remained significantly reduced compared to normal controls. In a large neuropsychological battery, the CUAMI-SF provided the measure most sensitive to short- and long-term impairment. Patients at the 7 hospitals did not differ at baseline in any neuropsychological measure. However, these hospitals differed in CUAMI-SF RA scores at both the immediate postECT and six-month time points. These differences among the hospitals were maintained after controlling for patient factors associated with neuropsychological performance. Rather, these differences in the extent of RA were attributable to differences among the hospitals in ECT technique. Once the contribution of technical factors was considered, hospital differences disappeared.

Both immediately following ECT and at the six-month follow-up, patients treated with BL ECT had greater impairment (more inconsistency) on the CUAMI-SF than patients treated with RUL ECT. Immediately following ECT, for all electrode placements, a larger number of ECT treatments was associated with poorer consistency scores. However, the slope of this decline was significantly steeper for those treated with BL than RUL or bifrontal ECT. At the six-month follow-up, a signficant relationship with treatment number was restricted to those treated with BL ECT. Increasing number of BL treatments was associated with greater inconsistency on the CUAMI-SF.

This study attempted to identify individual patients with a marked and persistent RA. To be so classified, patients had to have CUAMI-SF consistency scores at both the immediate postECT and 6-month follow-up time points that were more than 2 standard deviations below the average score of the total patient sample. Of 306 patients so classified, 38 (12.4%) met these a priori criteria for marked and persistent RA. Membership in this group was significantly greater among patients treated with BL ECT and among women.

Thus, the Services study found that psychiatric facilities differed in the severity of short- and long-term RA. These hospital differences, in turn, were attributable to variation in the practice of ECT. In particular, larger number of treatment with BL ECT was associated with greater quantitative short- and long-term RA deficits. Patient treated with BL ECT had greater representation among those with especially marked and persistent deficits.

Berman et al.⁴² examined patient subjective evaluation of memory functioning in the Services study sample. As part of the neuropsychological battery, the CFQ and GSE-My were administered at baseline, during the week following ECT, and at six-month follow-up. In the total sample there was a substantial reduction in cognitive complaints on the CFQ both immediately and six months following ECT, replicating the well established phenomenon of improved cognitive self-evaluation after ECT on most instruments.⁴⁰ In contrast, GSE-My scores decreased at the two time points. In fact, the majority of patients rated their memory as poorer both immediately following ECT (53.1%) and six-months later (64.3%).

Treatment variables had no relation to CFQ scores at either time point. Rather, the level of depressive symptomatology was strongly related to these scores at both postECT time points. GSE-My scores also had signficant covariation with concurrent Hamilton Rating Scale for Depression (HRSD) scores, but these relationships were much weaker. Instead, GSE-My scores were related to technical factors in ECT administration. In particular, six months following ECT, patients’ global self-evaluation of memory functioning was poorer with larger numbers of treatment with BL ECT, paralleling the effect observed on the CUAMI-SF.⁷

Berman et al.⁴² directly examined the relationship between change in objective neuropsychological measures (mMMS, verbal learning task, CUAMI-SF) and CFQ and GSE-My self-report scores. There were no signficant associations between neuropsychological measures and the CFQ at either time point. At both time points, GSE-My scores covaried with CUAMI-SF scores. Self-report of greater memory impairment following ECT was significantly associated with more severe RA, as indexed by the CUAMI-SF consistency scores, both immediately and six months following ECT.

Sackeim et al.⁴⁵ conducted a multisite, randomized, double-blind trial examining the effects of concurrent pharmacotherapy and electrode placement on the efficacy and safety of ECT. Patients (n=319) were randomized to treatment with nortriptyline, venlafaxine, or placebo during ECT. They were also randomized to brief pulse RUL (6 x ST) or BL (1.5 x ST) ECT. The choice of moderate dosage (1.5 x ST) BL ECT was key since the randomized NYSPI trials reviewed above largely used high dosage (2.5 x ST) BL ECT, thereby perhaps intensifying the cognitive disadvantages of this electrode placement. Patients (and matched normal controls) underwent neuropsychological testing at preECT baseline and within days of ECT termination. While a long-term clinical follow-up of this sample has been published,⁴⁶ that report focused only the determinants of relapse, and noted that missing data rates for neuropsychological measures were too high during follow-up for meaningful analyses.

In the randomized trial, findings were reported for four primary neuropsychological measures. Even after correction for the rate of inconsistency among normal controls, far and away, postECT scores on the CUAMI-SF revealed the greatest deficits. There was a signficant effect of pharmacological condition on three of the postECT neuropsychological measures, but the CUAMI-SF was the exception. Rather, there was a significant difference between high dosage (6 x ST) RUL ECT and moderate dosage (1.5 x ST) BL ECT in anterograde amnesia (verbal learning) and RA (CUAMI-SF) scores. In both cases, patients treated with BL ECT had more severe amnesia than patients treated with RUL ECT.

Summary of the Findings

This review of publications using the CUAMI and CUAMI-SF examined only those studies that I supervised. The CUAMI-SF, in particular, has been used more broadly in ECT and related research (e.g., ^47,48). Nonetheless, the research reviewed above provided a set of remarkably robust findings, in many cases replicated across studies. In each of the 5 distinct studies to use the CUAMI or CUAMI-SF, among a large number of neuropsychological measures, scores on the CUAMI or CUAMI-SF were the most sensitive to treatment group differences. Indeed, even when controlling for the normal rate of inconsistency in memory, RA scores on these instruments showed the greatest deficits immediately following ECT relative to many other neuropsychological measures. Furthermore, all 5 studies found that BL ECT resulted in greater RA immediately following ECT than RUL ECT. Three studies examining neuropsychological function at two- or six-months postECT also found greater persistent deficits in patients treated with BL ECT. Similarly, the single trial to randomize patients to different pulse widths found markedly reduced RA on the CUAMI with ultrabrief stimulation, immediately after ECT and at two-month and six-month follow-ups. Across these studies, relations with treatment number were repeatedly observed. Longer courses of ECT, especially with the BL electrode placement, were associated with greater RA both in the short- and long-term. It is also noteworthy that in each of these five studies RA scores and clinical outcome had no association.

Of special importance is the fact that RA scores on the CUAMI and CUAMI-SF covary with patients’ evaluations of the effects of ECT on their memory. In both a highly selected research sample³⁹ and a sample of patients who received ECT in the community,⁴² self-reports that ECT induced more negative effects on memory were associated with greater deficits on the CUAMI and CUAMI-SF. Until these studies, no objective neuropsychological measure had ever been found to correlate significantly with patient self-report of memory functioning following ECT.

This body of findings provides some of the most compelling evidence indicating that ECT can have persistent, adverse effects on memory for autobiographical information. These findings also provide the strongest validation to date that patients’ reports of their perceived deficits correspond to an objective reality. Yet Semkovska and McLoughlin^8,21 argue that the CUAMI and CUAMI-SF are scientifically inadequate and that the findings summarized above should be discarded and ignored. Why?

Objections to the CUAMI and CUAMI-SF

Semkovska and McLoughlin^8,21 have several reservations about the use and interpretation of these scales. The most important objection concerns a putative lack of data from healthy controls assessed over time. The primary dependent measure used with the CUAMI and CUAMI-SF has been the consistency of responses with baseline reports. All individuals show some degree of inconsistency over time in autobiographical memory recall, and Semkovska and McLoughlin⁴⁹ raise the concern that the rate of inconsistency observed following ECT in the patients in our studies was generally in the range observed in studies of normal controls assessed at the same intervals.^50,51 By failing to measure the extent of inconsistency in controls, we could not determine to what extent patient scores were, in fact, abnormal.

This objection evaporates on two grounds: (1) normal control data are not necessary to interpret differences among ECT groups and (2) Semkovska and McLoughlin apparently were not aware that the normal control data they deemed essential had, in fact, been collected and contributed to the scoring and interpretation in all but one study using the CUAMI and CUAMI-SF.

The focus of research using the CUAMI and CUAMI-SF has been on determining the aspects of ECT technique that impact on the severity and persistence of RA for autobiographical information. For example the findings to date indicate that higher numbers of ECT treatments, especially with BL ECT and/or wide electrical pulse width, are associated with more severe deficits shortly following ECT as well as months later. We presumed that showing that one treatment group differs from another (e.g., BL vs. RUL ECT) in memory scores at six months postECT demonstrates that persistence of deficit does occur and shows meaningful relationship to treatment parameters. We have always been careful to note that groups with superior consistency scores may still be manifesting a deficit, and that only comparisons to normal controls can resolve this separate issue.¹⁹ In short, the availability of normal data in no way impacts on the relative differences between ECT groups. While normal data give us a sense of the absolute magnitude of deficit (or lack thereof) in patient samples, such data has no statistical impact detecting differences among patient groups. Our focus was on testing for group differences in randomized trials and in a large observational study. Indeed, to my mind, detecting such differences and demonstrating them to be persistent over time and replicable across studies provides a stronger scientific design in establishing that ECT can have persistent effects on autobiographical memory than simply comparing consistency scores of patients to normal controls.

The foregoing argues on conceptual grounds that Semkovska and McLoughlin’s^8,21 insistence on normative data hardly undercuts the meaningfulness of the previous work. Their position would be stronger were it the case that ECT could result in enhanced consistency in autobiographical memory. For example, could it have been in our studies that BL ECT was associated with a normal rate of consistency over time and RUL ECT was associated with an improved rate? Normal data would be helpful to reveal such an unlikely pattern.

In the publication of the first NYSPI study to use the CUAMI, we noted the absence of repeat testing of normal controls as a limitation of the work.¹⁹ All the subsequent work reviewed above administered the CUAMI and CUAMI-SF to matched normal controls on multiple occasions. Patient consistency scores over time were adjusted for the rate of inconsistency in these matched controls. For example, in the Services study we stated, “The scoring of the AMI-SF necessarily results in higher scores at baseline than follow-up. The CPT and AMI-SF scores in the patient sample were adjusted for the average change seen in the [normal] comparison sample, removing the temporal effects on these two measures (p. 247).” Despite this adjustment, the deficit relative to baseline in the patient sample was profound both immediately and six months following ECT. Despite this adjustment, using standardized scoring, the deficit was greater and more persistent for the CUAMI-SF than any other neuropsychological measure.

Semkovska and McLoughlin were apparently unaware that the linchpin of their objections had already been addressed and incorporated in the published work on the CUAMI and CUAMI-SF. This may have been because the description of methods in most of our reports did not mention the existence of a matched normal control group and the collection of CUAMI or CUAMI-SF on multiple occasions in order to adjust patient consistency scores. The absence of such discussion reflected the fact that the primary focus of these publications was on group differences in consistency scores within the patient samples attributable to variation in ECT technique and individual difference factors. I apologize if this lack of detail regarding methods contributed to the gross misstatement of fact by Semkovska and McLoughlin.^8,21,48,49

Thus, it may be worthwhile to clear the record and describes the pattern of differences with normal controls in consistency scores in the second and third NYSPI studies,^24,25 the Services study,⁷ and the OPT-ECT study.⁴⁵ Immediately following ECT, every subgroup examined, but one, had inferior consistency scores compared to matched normal controls. At long-term follow-up every group described as having persistent deficits relative to another ECT group also differed at that time point from their matched controls.

I will illustrate these points with data from the last NYSPI trial to use the CUAMI. Recall that in this study patients were randomized to electrode placement (RUL 6 x ST vs. BL 2.5 x ST) and pulse width (0.3 ms vs. 1.5 ms) conditions. Figure 1, previously unpublished, presents average scores for the matched normal control group and each of the 4 ECT groups at the time point immediately following ECT relative to baseline. Pair-wise comparisons revealed that all four ECT groups differed from each other, with deficits greater with BL electrode placement and use of a brief pulse relative to an ultrabrief pulse. Also note that the ultrabrief RUL ECT group did not differ from the normal control group. In contrast, the differences between controls and all other patient groups were marked, especially for those treated with a brief pulse.

Figure 1.

Scores of healthy controls and four ECT groups on the CUAMI at immediate postECT relative to baseline. Scores at postECT (or after 4 weeks for controls) and baseline were standardized relative to the distribution in patients at baseline. The baseline normalized scores were then subtracted from the postECT normalized scores. Negative values reflect greater inconsistency. Pair-wise comparisons showed that all groups significantly differed from each other, except healthy controls and patients who received ultrabrief RUL ECT. Data, previously unpublished, are from Sackeim et al.²⁵

Figure 2 is taken from the original publication²⁵ and represents the standardized consistency scores of the ECT groups over time. A fifth group is added, specifically patients who did not respond to their randomized assignment and received a second, “crossover” course of brief pulse BL ECT (2.5 x ST). Note that the scores for four ECT groups following the randomized course should be identical to the scores in Figure 1. They are, except that a constant of 1.0431 was added to every patient’s score. This reflected the average inconsistency score for normal controls in Figure 1. In other words, the data and figure reported by us in the publication corrected patient scores for the extent of inconsistency seen in normal controls when they were tested a month apart. Thus the origin in this graph represents not the absolute amount of inconsistency, but the amount of inconsistency over and above the average seen in normal controls. In this light, it is evident that all groups had deficits relative to normal controls, except the ultrabrief RUL ECT group at all time points and the ultrabrief BL ECT group at the two-month and six-month follow-ups. Thus it is evident that Semkovska and McLoughlin’s main objection is superfluous; in all recent studies, we collected the key normal data, involving repeat testing at matched intervals, that they felt essential to interpreting findings with the CUAMI and CUAMI-SF. Furthermore, I note that while the normal data are useful in assessing the absolute size of deficits among patient groups, such data has little bearing on detecting and interpreting the differences among ECT groups in the magnitude and persistence of deficits.

Figure 2.

Standardized scores on the CUAMI for the ECT groups at multiple postECT time points. Scores were adjusted for the rate of inconsistency in healthy controls at immediate postECT. Thus, a score of zero reflects the extent of inconsistency in the healthy controls. Patients randomized to brief pulse ECT or who received crossover ECT had marked and persistent deficits relative to both controls and patients treated with ultrabrief stimuli. Figure is from Sackeim et al.²⁵

Another objection raised by Semkovska and McLoughlin is that these scales lack reliability and validity.^8,21,48 Here, these criticisms are narrowly technical and largely contradicted by the body of findings with the CUAMI and CUAMI-SF. The fact that the CUAMI and CUAMI-SF have shown robust, replicable relationships with other variables, including ECT technical parameters, individual difference predictive factors, physiological referents, and patient self-report strongly implies that these scales are reliable measuring instruments. It is true that there are no formal reports on inter-rater reliability in administering and scoring these scales or test-retest reliability of the key dependent measures. However, Semkovska et al.⁴⁸ proposed a new and more complex scoring system for the CUAMI-SF, and administered the scale twice, six months apart, to healthy controls and depressed (nonECT) patients. They reported strong reliability for each of the CUAMI-SF components they proposed. Semkovska et al.⁴⁸ had the data to score the CUAMI-SF in the traditional manner and report on its reliability. Although they did not do so, it is very likely that the results would have indicated as good or better reliability than with the new scoring methods. The larger point is that these instruments have repeatedly shown sensitivity to key phenomena in ECT, strongly underscoring their reliability in their use with the intended population. Formal report on their reliability, especially that of the consistency score in ECT samples, would be useful for archival purposes but is unlikely to prove enlightening.

The issue of validity is also obtuse. First, Semkovska and McLoughlin contend that the CUAMI and CUAMI-SF are “unvalidated” principally because they were not developed in studies of brain-injured populations.^8,21 Indeed, as opposed to these instruments, Semkovska and McLoughlin²¹ recommend use of the Autobiographical Memory Interview by Kopelman et al.^52,53 This instrument assesses RA for three time periods, childhood, early adulthood, and recent events, and separately for personal semantic memories and autobiographical incidents. Since this instrument is intended to quantify RA for autobiographical information in brain-injured individuals (e.g., dementia, traumatic head injury), there is no possibility of establishing a baseline for autobiographical memories, and thus, no possibility of examining consistency of recall. Indeed, the Kopelman AMI is scored without corroboration and as if all responses were veridical.

Semkovska and McLoughlin fail to appreciate the extraordinary advantage the ECT researcher/clinician has in evaluating RA relative to the researcher/clinician working with brain-damaged populations. There is no practical way to assess autobiographical memory prior to the onset of brain damage, whereas ECT is a scheduled procedure. In the post hoc detection of RA in brain-damaged samples, it is often adequate to make inquiries about autobiographical events that are common and ordinarily highly resistant to forgetting and to simply count the instances of “don’t remember” or “don’t know” responses. This is often adequate as RA in brain-damaged samples can be profound and, thus, gross inquiry is sufficient. In contrast, with ECT we are often dealing with more subtle manifestations of RA, and we have the luxury of being able to assess which memories are held by the patient at baseline. This allows for restricting (or titrating) the inquiries at postECT to just that material with definite memories at baseline. This practice enhances the sensitivity of instruments to ECT-induced RA. Furthermore, the availability of baseline responses in no way diminishes the capacity to quantify the number of postECT “don’t know” or “don’t remember” responses. Rather, the fact that “don’t know” responses are given for items that previously elicited a definite response justifies there designation as pure memory failures. Furthermore, the availability of a baseline allows for determination of the degree of consistency in recall in addition to the number of pure memory errors. As noted earlier, we repeatedly found the consistency measure to mirror the measure of pure memory errors in revealing effects of ECT, but with somewhat larger effect sizes.

In attacking the validity of the CUAMI and CUAMI-SF Semkovska and McLoughlin’s failed to appreciate this major of advantage of ECT: the availability of a baseline allowing titration of postECT questioning as well as quantification of consistency. In contrast, 3 studies have administered the Kopelman AMI to ECT samples.^54–56 In all instances, the results were negative and there was no evidence that ECT produced RA. Clearly, these negative results, using Semkovska and McLoughlin’s preferred method, contrast with the repeatedly replicated findings using the CUAMI and CUAMI-SF of short- and long-term deficits that were parametrically associated with ECT treatment conditions. Thus, it is ironic that Semkovska and McLoughlin recommend a measurement strategy that has proven wholly insensitive to ECT most important adverse cognitive effect, while disputing the validity of a strategy that has repeatedly demonstrated sensitivity to this effect.

In disputing the validity of the CUAMI and CUAMI-SF Semkovska and McLoughlin admitted that these instruments have considerable face validity as measures of RA for autobiographical information. Nonetheless, they ignored the substantial evidence for the construct, discriminant, and predictive validity of these instruments. These scales were developed for the express purpose of measuring ECT-induced RA.^18,19 Comparisons to pure memory errors and the use of family-member corroboration validated the use of the consistency score as a measure of RA.^19,23,24 These scales have shown the most consistent, pronounced, and persistent cognitive deficit likely in the history of ECT research.^1,8 Long-term deficits on these scales are predicted by specific neuropsychological assessments prior to or during the course of ECT.³⁴ Most critically, these scales are the only measures in the history of ECT research to show signficant covariation with patients’ self-evaluation of the effects of ECT on their memory.^39,42 Thus, it is absurd to state that these scales lack validity, as if that were an all-or-none judgment.

The emptiness of Semkovska and McLoughlin’s claim about validity is underscored by their own empirical research with the CUAMI-SF. ⁴⁸ They studied healthy controls and non-ECT depressed patients and administered a neuropsychological battery. By examining the correlations among their proposed CUAMI-SF subscales and with the neuropsychological measures, Semkovska et al.⁴⁸ claimed to have demonstrated the validity of the new CUAMI-SF scoring system. The irony here is that Semkovska et al. did not study RA at all. The healthy control and depressed groups had similar consistency scores at retesting. Their standard for claiming validity pertained to obtaining a pattern of non-hypothesized correlations in a non-relevant sample. Of note, with the same data Semkovska et al. could have claimed to establish “validity” for the original CUAMI-SF, since they tested a variant that differed only in scoring. The point here is that their standards for claiming reliability and validity were remarkably lax when applied to their own scoring variant of the CUAMI-SF. In contrast, Semkovska and McLoughlin dismissed findings with the CUAMI and CUAMI-SF despite the substantial evidence supporting their reliability and validity specifically in the assessment of ECT-induced RA.

Semkovska and McLoughlin remaining reservations center on difficulties in using the consistency measure.^8,21 Since consistency scores are expected to decrease over time, it is problematic to interpret findings as truly reflecting deficits unless there is correction for normal rates of inconsistency. I have already pointed out that such normative data was collected in almost all of our studies and informed scoring and interpretation. Semkovska and McLoughlin claim that features of the consistency score make it difficult to use in comparisons to other neuropsychological measures. Indeed, they state that consistency scores are particularly problematic when conducting meta-analyses. These concerns are misplaced since any neuropsychological measure repeatedly administered may have to be corrected for practice or other temporal effects as seen in healthy comparison participants. Further, the CUAMI and CUAMI-SF produce statistics (means, variance, effect sizes, etc.) that are just as amenable to meta-analysis as other variables. Indeed, Semkovska et al.⁴⁸ undercut their own argument since they recently reported a series of meta-analyses examining electrode placement and dosage effects on postECT cognitive functions. This work included, without difficulty, findings from studies using the CUAMI or CUAMI-SF consistency score.

Thus, it is evident that Semkovska and McLoughlin made an error in scientific judgment when stating that finding with the CUAMI and CUAMI-SF should be ignored. Their main reservation, the lack of normative data, proved to be false, although they may not have been aware that this concern was addressed in our procedures. Their claim that these instruments lack evidence of reliability and validity is also patently false. Indeed, in many respects, these scales have more substantial evidence supporting their validity as measures of postECT RA for autobiographical information than any other instrument. Finally, their technical reservations regarding the constraints of consistency scores is undercut by their own use of such scores in a recent series of meta-analyses. Taken together, these concerns are grossly insufficient as grounds for disregarding the evidence produced using these scales.

In defending the use of CUAMI and CUAMI-SF, I do not wish to imply that additional research on the reliability and validity of these instruments is unnecessary or that comparison with other methods of assessing RA for autobiographical information would not be informative. Clearly, increased scientific attention to the intrinsic characteristics, time course, and correlates of this RA is desirable, precisely because this cognitive domain shows the most persistent and severe deficits following ECT. At the same time, it is also apparent that we should not throw out the baby with the bath water. The CUAMI and CUAMI-SF have taught us a great deal about this potential adverse effect of ECT. To discard this knowledge, would, in fact, be throwing out the baby due to objections that are either erroneous or narrowly technical. Indeed, as innovations in ECT, such as the use of ultrabrief stimulation, markedly reduce the severity and persistence of this side effect, it will be even more important to develop sensitive instruments to reveal any residual deficits and/or to substantiate the claim that risk of this side effect has been eliminated.

Implications of Semkovska and McLoughlin’s Viewpoint

This is not the place to speculate on the factors that led to the gross error of judgment that propelled Semkovska and McLoughlin to reject the findings with the CUAMI and CUAMI-SF. It is noteworthy, however, that in their recent review of the measurement of RA for autobiographical information after ECT, Semkovska and McLoughlin²¹ devote considerable space to the consideration by the U.S. Food and Drug Administration (FDA) to reclassify ECT devices. Essentially, they contend that the “consequences of inadequately measuring autobiographical amnesia in the ECT literature” were “crystallized” in the FDA executive summary report (p. 130).⁵⁷ I agree that this report contained a number of factual errors, such as mistaking the Kopelman AMI for the CUAMI. However, Semkovska and McLoughlin’s main objection is that the FDA asserts in several places that ECT can result in RA for autobiographical information. In contrast, they contend that there is no adequate evidence to support such assertions. In an earlier publication they reveiwed 44 randomized trials and 40 observational studies.⁸ By their view none of these studies used a validated measure of RA or found a difference relative to sham ECT, placebo, or nonECT depressed controls. In short, they firmly support the effort to reclassify ECT devices into a less restrictive class, and believe the FDA has made a fundamental mistake in evaluating this adverse cognitive effect. Thus, by claiming that the CUAMI and CUAMI-SF are unvalidated, they argue that the consistent evidence these scales have revealed about persistent RA following ECT should be ignored by the FDA.

For decades, the predominant view in the field of ECT was that all the adverse cognitive effects of the treatment are transient, and untoward cognitive effects are not seen a few days or at most a few weeks after treatment termination.^58,59 This denial of persistent deficit was so profound that it has been proposed that patients with persistent complaints about RA for autobiographical information be considered as presenting with a somatoform disorder.⁶⁰ This denial of deficit ran counter to reports of many patients, including those who believed ECT was lifesaving, the experience of countless ECT practitioners who had patients’ in their practice who were non-litigious and appeared to have genuine RA, and the continuity of findings from the early research by Janis^9–11 and Squire^12–16 and the studies using the CUAMI and CUAMI-SF.² Thus, important progress, both clinical and scientific, was made by the field with the subsequent broad acceptance that RA for autobiographical information is a potential persistent effect. The 2001 APA Task Force Report reflected this perspective, stating in its recommended consent form that ECT can result in “permanent gaps in memory” (p. 321).

Semkovska and McLoughlin wish to undo this progress. Their biased review of the evidence, throwing out the most relevant and poignant data, leads them to conclude that there is no evidence that ECT results at all and at any time in RA for autobiographical information. Thus, they argue that the FDA is ignorant or confused when considering this serious concern. I, for one, am an adamant supporter of the effort to reclassify ECT devices, and have been working toward this end since soon after the original APA petition in 1982 to the FDA urging this agency to approve reclassification. However, I believe that we do a gross disservice to our patients, the public, and ourselves when we deny the existence of an adverse effect that has been documented in various ways for decades and is a central concern of our patients. ECT can be of extraordinary clinical value, but, like all treatments, it is not without its limitations and adverse effects.