Assessing Cognitive Function in Bipolar Disorder: Challenges and Recommendations for Clinical Trial Design

Abstract

OBJECTIVE

Neurocognitive impairment in schizophrenia has been recognized for more than a century. In contrast, only recently have significant neurocognitive deficits been recognized in bipolar disorder. Converging data suggest the importance of cognitive problems in relation to quality of life in bipolar disorder, highlighting the need for treatment and prevention efforts targeting cognition in bipolar patients. Future treatment trials targeting cognitive deficits will be met with methodological challenges due to the inherent complexity and heterogeneity of the disorder, including significant diagnostic comorbidities, the episodic nature of the illness, frequent use of polypharmacy, cognitive heterogeneity, and a lack of consensus regarding measurement of cognition and outcome in bipolar patients. Guidelines for use in designing future trials are needed.

PARTICIPANTS

The members of the consensus panel (each of the bylined authors) were selected based upon their expertise in bipolar disorder. Dr. Burdick is a neuropsychologist who has studied cognition in this illness for 15 years; Drs. Ketter, Calabrese, and Goldberg each bring considerable expertise in the treatment of bipolar disorder both within and outside of controlled clinical trials. This consensus statement was derived from work together at scientific meetings (e.g. symposium presention at the 2014 Annual meeting of the American Society of Clinical Psychopharmacology, among others) and ongoing discussions by conference call. With the exception of the public presentations on this topic, these meetings were closed to outside participants.

EVIDENCE

A literature review was undertaken by the authors to identify illness-specific challenges relevant to the design and conduct of treatment trials targeting neurocognition in bipolar disorder. Expert opinion from each of the authors guided the consensus recommendations.

CONSENSUS PROCESS

Consensus recommendations, reached by unanimous opinion of the authors, are provided here as a preliminary guide for future trial design. Recommendations comprise exclusion of certain syndromal level comorbid diagnoses and current affective instability, restrictions on numbers and types of medications, and use of pre-screening assessment to ensure enrollment of subjects with adequate objective evidence of baseline cognitive impairment.

CONCLUSIONS

Clinical trials to address cognitive deficits in bipolar disorder face distinctive design challenges. As such trials move from proof-of-concept to confirmation of clinical efficacy, it will be important to incorporate distinctive design modifications to adequately address these challenges and increase the likelihood of demonstrating cognitive remediation effects. The field is now primed to address these challenges and a comprehensive effort to formalize best practice guidelines will be a critically important next step.

The past several decades have seen considerable progress in the treatment of bipolar disorder. After a relative dearth of rigorously controlled studies during the 20 years following the approval of lithium in 1970, large multicenter controlled studies targeting mania re-emerged in 1994¹. In the following decade, attention was focused on the proportion of time depressed, as well as related morbidity and mortality^2,3. Since 2003, randomized controlled trials have supported the efficacy of several pharmacological agents for bipolar depression (i.e., olanzapine-fluoxetine combination, quetiapine, and lurasidone. In addition, naturalistic clinical trials addressed the phenomenology^4–6 and the treatment of bipolar depression, including the Systematic Treatment Enhancement Program for Bipolar Disorder (STEP-BD) funded by the National Institute of Mental Health (NIMH)⁷. Although additional studies are needed to expand options for treating affective symptoms in bipolar patients, an emerging and increasingly persuasive database now suggests that the field should extend its attention to other critical domains, such as the cognitive deficits associated with the illness. Efforts to formalize best practice recommendations for treatment trials of cognition in bipolar disorder have been extremely limited, in contrast to parallel approaches in schizophrenia. There are several challenges that arise in considering the design of such trials in bipolar patients and the time is right to shift focus and meet these challenges with a comprehensive approach.

Methods

This article briefly reviews studies of cognitive dysfunction in bipolar disorder, summarizes the challenges inherent in the design and conduct of treatment trials targeting neurocognition in patients with bipolar disorder, and provides recommendations for addressing these challenges. The review of cognition in bipolar disorder is not comprehensive, as the focus of this article was on consensus development; however, for a recent full review of this area incorporating associated neurobiological changes seen in bipolar disorder see Lim et al. 2013⁸.

The consensus process utilized for the development of this work entailed the joint effort of all four bylined authors, each with expertise in bipolar disorder, clinical trials, and neurocognitive functioning. A series of meetings, primarily by conference call resulted in the recommendations put forth in this paper. The primary limitation to this process is the fact that these were closed meetings, thereby limiting the input from a larger field of investigators also involved in this type of research. In addition, as there is a paucity of data available with regard to previous clinical trials focusing on cognition as a treatment target in bipolar disorder; recommendations are largely based upon broader expertise and opinion.

Cognitive Dysfunction in Bipolar Disorder

Recent years have seen exponential growth in studies of cognitive dysfunction in bipolar disorder (Figure 1), consistent with the current NIMH focus on implementing a dimensional approach to neuropsychiatric illnesses with the Research Domain Criteria (RDoC) initiative. Neurocognitive functioning represents one of the key RDoC constructs that crosses DSM-5 boundaries and may substantially enhance our understanding of the pathophysiology of diverse brain-based illnesses, including bipolar disorder.

Figure 1. Exponential Growth in the Literature on Cognitive Dysfunction in Bipolar Disorder (BPD).

Data were obtained from a search of the PubMed database by year from 1990 through 2012 using the terms “cognition” OR “cognitive” AND “bipolar” in the title or abstract. Date of publication for each year was defined as January 1 through December 31. Cumulative data are plotted in 2-year intervals; line represents an exponential trendline function applied to the data. BPD, bipolar disorder.

Neurocognitive deficits have long been acknowledged as a core feature in schizophrenia. Emil Kraepelin differentiated dementia praecox (schizophrenia) from manic-depressive psychosis (bipolar disorder) in the early 20^th century, believing that patients with bipolar disorder exhibited affective and cognitive symptom–free, euthymic intervals between mood episodes. More recent data indicate that a large proportion of patients with bipolar disorder experience only partial recovery from affective and cognitive symptoms between episodes. Longitudinal studies have demonstrated that many patients experience ongoing subsyndromal affective symptoms^3,9 and that neurocognitive deficits are prominent during acute depressive and manic episodes and do not entirely resolve during euthymia^10,11. In the cognitive sphere, persistent, trait-like deficits have been noted in attention, verbal learning, and executive function^12–14, with mean performance falling approximately 1 standard deviation below that of healthy controls.

Studies suggest that cognitive impairments in bipolar disorder qualitatively overlap with, but are less severe than, those in schizophrenia^10,15. One notable difference is that impairment in global cognitive performance (e.g., IQ) is characteristic of schizophrenia but generally absent in bipolar disorder^10,15. This may reflect the differences noted in premorbid IQ. Whereas future bipolar patients tend to show normal-range IQ before illness onset, many schizophrenia patients have IQ impairment well before the emergence of psychotic symptoms^10,16.

However, it is now widely recognized that deficits in more specific neurocognitive domains occur early on in bipolar disorder and escalate with an increasing number of major affective episodes¹⁷. Cognitive impairment occurs in both bipolar I and bipolar II disorder¹⁵, although some, but not all, studies indicate more impairment in bipolar I^18–20. In addition, meta-analytic data suggest that bipolar patients with a history of psychosis (which is more common in bipolar I than bipolar II disorder) perform worse than bipolar patients who have never experienced psychosis, even during affective remission²¹.

An important finding, as shown repeatedly in schizophrenia^22,23, is that persistent neurocognitive deficits in bipolar disorder contribute significantly to functional disability that prevents patients from achieving complete functional recovery^15,24,25. Although bipolar disorder treatment studies have previously focused on affective symptoms, it is important to consider the substantial disability associated with impaired cognition. Even though a recent focus has been placed on the importance of carrying out large-scale clinical trials to specifically target cognitive impairment in patients with schizophrenia²⁶, an analogous approach in bipolar disorder remains to be realized.

Challenges and Recommendations for Trial Design in Bipolar Disorder

The Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative, a collaborative effort among academia, the United States Food and Drug Administration (FDA), and the pharmaceutical industry, led to the development of guidelines for study design and a consensus cognitive battery to measure cognitive outcomes in clinical trials in patients with schizophrenia. Several recommendations were made for optimizing outcomes of cognitive trials in schizophrenia²⁷, some of which may also be applicable in bipolar disorder. However, studies of cognition in bipolar disorder patients will likely entail substantive differences from those in schizophrenia patients and necessary efforts such as the MATRICS initiative have not yet been realized in bipolar disorder. Challenges in bipolar disorder research include substantial heterogeneity in illness subtype and course (e.g., type I vs. type II, rapid cycling), fluctuating mood states over time, diverse common comorbid conditions (e.g., substance abuse, anxiety disorders), and frequent need for polypharmacy. Cognitive heterogeneity (i.e., patients ranging from unimpaired to very impaired) is also a concern in bipolar trials, but is less of an issue in schizophrenia. Table 1 presents some of these challenges and suggested preliminary recommendations, which are discussed in detail below.

TABLE 1.

Summary of Challenges and Recommendations for Cognitive Trial Design in Bipolar Disorder

Challenges/Confounds	Recommendations
Diagnostic issues
Bipolar subtype (Bipolar I/Bipolar II)	Include both subtypes, with cognitive impairment threshold
Substance use disorder comorbidity	Allow history of substance use disorder; exclude very recent history; test for illicit substances at time of assessment to rule out intoxication
Anxiety disorder comorbidity	Allow anxiety disorder diagnosis; evaluate current state and rule out subjects with current syndromal anxiety symptoms
ADHD comorbidity	Allow subjects who received ADHD diagnosis in adulthood (after bipolar onset) but exclude subjects who received diagnosis and treatment in childhood
Clinical issues
Affective symptoms (state)	Institute maximum threshold for both depression and mania symptoms; exclude acute mania/depression; subsyndromal symptoms allowable
Concomitant medications	Adjunctive design recommended, add-on to current mood-stabilizing regimen
Specific medication restrictions	Exclude subjects taking high-dose anticholinergics; topiramate; clozapine; tricyclics; benzodiazepines within 6 hours of testing; those who had ECT within past 6 months
Number of medications	Restrict inclusion to subjects taking no more than 3 psychotropic medications; dosing limitations might also be useful
Presence/severity of cognitive deficits	Include a pre-screening battery to test for objective evidence of impairment; threshold recommended at −1SD below average
Measurement issues
Neurocognitive battery	Should be sensitive enough to detect meaningful change, practical in its implementation, limited in practice effects; should be objective and not based on patient self-report. The MCCB is a reasonable core battery; additional measures may be required to enhance sensitivity and to tap into the affective processing domain.
Functional outcome measures	Should be sensitive enough to capture partial disability; should be based on objective information and an informant whenever possible.

ADHD=attention deficit hyperactivity disorder; ECT=electroconvulsive therapy; MCCB=Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Consensus Cognitive Battery.

Subject Characteristics

The updated MATRICS guidelines²⁸ recommend less restrictive exclusion criteria than originally proposed²⁷ to facilitate patient recruitment while preserving study validity. Such an approach will likely also apply when enrolling patients with bipolar disorder to optimize signal detection while permitting feasible recruitment and generalizable results. Several bipolar-specific issues should be carefully considered when designing cognitive trials.

Diagnosis

Factors relevant to the clinical course that must be accounted for in studies of bipolar disorder include the number of previous episodes, age at onset, duration of illness, and history of psychosis²⁹. The inclusion of patients with both bipolar type I and type II is preferable in most cases to allow for increased feasibility and generalizability. Although bipolar II patients may not have sufficiently severe neurocognitive impairment to be entered into a clinical trial, the use of a pre-screening deficit threshold for inclusion should address this. Balanced enrollment of bipolar I and bipolar II patients would also allow subgroup analyses by bipolar subtype, where feasible.

Patients with bipolar disorder have high rates of comorbid psychiatric disorders (DSM 5). Specifically, approximately 50% of bipolar patients have a lifetime history of comorbid substance use disorders³⁰. Exclusion of all of such cases would not only reduce feasibility, but also create problems with external validity. However, active substance abuse is a significant confounding variable in studies focused on cognition as an outcome, and subjects with a current or very recent history of abuse or dependence (i.e., in the prior 3 months) should probably be ruled out. Similarly, patients with chronic severe alcohol or substance use symptoms (e.g., with more than two legal complications, such as citations for driving under the influence, or more than two admissions to therapeutic medical detoxification and/or residential treatment programs spanning at least a decade) should probably be excluded. The current use of certain illicit substances not meeting DSM criteria for substance abuse or dependence should also be considered exclusionary, as is done in pharmacotherapy affective efficacy studies in bipolar disorder in which marijuana is allowable, but cocaine and other drugs are excluded. In this case, a toxicology screen on testing days could ensure that acute intoxication is not affecting the primary results. However, the sustained presence of markers of cannabis use in toxicology screens may make the latter challenging. Regardless of the approach used to reduce the confounding effects of substance misuse, standardized measures should be incorporated into trials to ensure adequate measurement of substance use (even within normal range), inclusive of legal substances such as alcohol and tobacco. Anxiety disorders are also highly comorbid with bipolar disorder and could have significant effects on cognitive performance³¹; therefore, the exclusion of patients with current syndromal anxiety disorders should also be considered in cognitive trials.

Finally, a diagnosis of attention deficit hyperactivity disorder (ADHD) is common in younger individuals with bipolar disorder. However, some bipolar patients are first diagnosed with ADHD in adulthood after the onset of bipolar disorder, making it difficult to differentiate between cognitive symptoms associated with bipolar illness and ADHD. For the purposes of cognitive trials, it might be useful to exclude bipolar patients who received a diagnosis of ADHD during childhood and who were treated for this condition before the onset of bipolar disorder, rather than exclude all subjects who have ever received a diagnosis of ADHD. This strategy would reduce the likelihood that individuals with true ADHD diagnoses are included in cognitive trials designed to target cognitive problems associated more specifically with bipolar disorder.

Mood State

The episodic nature of bipolar illness requires careful consideration of baseline symptom severity, particularly in light of data suggesting a significant influence of affective symptoms on cognitive performance³². If patients with current affective symptoms are included in a cognitive trial and affective improvement is reported, it may be difficult to rule out pseudospecificity. Restricting inclusion to patients who are affectively stable at the time of treatment initiation is one way to substantially reduce confounding due to affective symptoms. This can be done using strict euthymia criteria or a slightly relaxed stability definition, depending upon practicality and the specific agent being studied. If the medication being tested has possible antidepressant activity or a high risk of inducing mania, it might be preferable to use the more stringent euthymia definition to avoid effects of mood-state changes. However, given the pervasiveness of subthreshold affective symptoms, it may be necessary to permit a modest degree of such symptoms, in the interest of enrollment feasibility. The degree to which subthreshold depressive symptoms, compared to mood elevation symptoms, might affect cognition remains to be established; however, early data suggest an interactive effect of subthreshold depression and verbal memory deficits on everyday functioning in patients with bipolar disorder^33,34.

Current affective symptom severity should be assessed at screening, baseline, and intermittently (especially on cognitive testing days) throughout the trial using standardized mood ratings, such as the Hamilton Rating Scale for Depression or the Montgomery-Åsberg Depression Rating Scale and the Young Mania Rating Scale. These ratings can serve to confirm euthymic or stable state at study entry and to track any changes in affective symptoms over the course of the study. In general, it is recommended to enroll subjects who are in a relative state of remission; however, it is important to acknowledge that many patients with bipolar disorder have persistent subsyndromal affective symptoms, particularly low-grade depression, during otherwise “remitted” periods^3,9. Thus, in an effort to optimize both feasibility and generalizability, while still reducing the degree to which affective symptoms may confound cognitive analyses, cutoffs for entry may be less restrictive than those defining pure euthymia. The inclusion of cases with subsyndromal symptoms can be dealt with post hoc either by stratifying analyses based upon purely euthymic versus subsyndromal status or by including mood ratings as covariates in statistical models. More sophisticated regression or pathway-based models could eventually contribute important information on the interactions between cognition, mood, and treatment outcome in patients with bipolar illness.

In an effort to limit the confounding effect of mood cycling, the shortest possible trial duration should be considered, while still acknowledging the importance of assessing the relative persistence of any cognitive benefit that might be found. Duration will need to be adjusted based upon the specific agent of interest, its mechanism of action, the presumed onset of efficacy, and the recommended dosing strategy. Because mood-state switches or symptom exacerbations may occur during the trial, a clear threshold or specific criteria should be used for subject discontinuation to limit the degree to which mood-state changes introduce “noise” when attempting to detect a significant cognitive signal. In patients who discontinue pharmacotherapy interventions due to adverse effects (rather than due to inefficacy), off-on-off designs could be informative, although the possibility of lingering post-discontinuation cognitive effects needs to be addressed.

Another suggestion for trial design when considering mood-state confounding, as well as safety-related concerns is the required use of a concomitant mood-stabilizing medication. Although monotherapy studies might be considered optimal when testing new agents to control affective symptoms in bipolar disorder, the targeting of neurocognitive function is best done when the patient is already affectively stable, which typically requires pharmacotherapy. In addition, the inclusion of unmedicated bipolar patients is likely to significantly reduce the generalizability of the study given the widespread use of medications in this disorder. As such, the potential confounding effects of concomitant medications needs to be considered in trial design.

Concomitant Medications

The MATRICS committee recommends add-on study designs in schizophrenia^27,35 because monotherapy may be neither ethically acceptable nor representative of actual clinical practice. Although this approach might also be desirable in bipolar disorder, it may confound cognitive outcomes in bipolar patients, who commonly take multiple psychotropic medications.

Definitive data are lacking with regard to the impact on cognition of commonly used medications for bipolar disorder; however, it is clear that at least some treatments that stabilize mood can negatively affect cognitive performance^36,37. For example, lithium may be associated with distinct, although subtle, negative effects in several cognitive domains including verbal learning, short-term memory, creativity, and psychomotor performance^37,38; however, when excluding situations in which there might be some degree of lithium intoxication, several longitudinal studies have not reported such deleterious effects^{39, 40}.

Among the anticonvulsant mood stabilizers, valproate is associated with mild impairments in attention, short- and long-term memory, and motor speed; carbamazepine use is reported to result in mild memory impairments³⁷. Lamotrigine appears to be cognitively benign⁴¹, whereas topiramate, which is not indicated for bipolar disorder but may be prescribed for prevention of migraine headaches or weight control, is associated with extensive adverse cognitive effects³⁷.

Atypical antipsychotic agents vary in their propensity for sedation (diminished cognitive responsiveness) and somnolence (subjective sleepiness), with the greatest incidence noted for clozapine, olanzapine, and quetiapine⁴². Less sedating antipsychotic agents such as aripiprazole and ziprasidone can yield akathisia, anxiety, and agitation, which could also adversely affect cognition. Lurasidone, a less sedating atypical antipsychotic recently approved for bipolar I depression, although also associated with akathisia, may have a positive effect on cognition, putatively due to 5-HT₇ and 5-HT1A receptor binding within the hippocampus⁴³.

A recent review evaluated the effects of antidepressants on neurocognitive function⁴⁴ and concluded that the sedative and negative cognitive effects of tricyclic antidepressants are well established. Whereas selective serotonin reuptake inhibitors generally have a neutral effect on cognitive performance, sertraline may be somewhat better and paroxetine somewhat worse than other drugs in this class. Bupropion, reboxetine, and selective norepinephrine reuptake inhibitors may have more beneficial effects than other antidepressants. It should be noted that newer agents with novel mechanisms (e.g. vortioxetine) may show some promise in improving cognition in acutely depressed patients with major depressive disorder⁴⁵.

Although limited data are available on any specific drug class with regard to the adverse effects of pharmacology in patients with bipolar disorder, there are no studies directly addressing polypharmacy, which is very prevalent in bipolar patients, or the cumulative effects of multiple psychotropic agents on cognitive performance. Furthermore, there have been no controlled cognitive enhancement trials in bipolar disorder that have been large enough to assess the effects of concomitant medications or to stratify randomization to control for these effects. In our recently completed cognitive enhancement study of pramipexole⁴⁶, we were able to determine the effects of concomitant medications only superficially, by dividing groups based on the presence or absence of specific classes of medication (e.g., antipsychotics, anticonvulsants, lithium), and we were unable to account for polypharmacy in any meaningful way.

When designing future trials, restrictions with regard to concomitant medications might be considered somewhat arbitrary given the lack of clear evidence of deleterious cognitive effects for most bipolar pharmacotherapies; however, setting some limitations would be prudent. Given the existing evidence, medications such as topiramate, tricyclic antidepressants, clozapine, and high doses of anticholinergics should be disallowed. Moreover, subjects who have received electroconvulsive therapy (ECT) during the past 6 months should be excluded from cognitive trials. Following MATRICS recommendations, and in an effort to reduce treatment heterogeneity, subjects taking more than 3 psychotropic medications should also be excluded. These limits might be difficult to implement given the very high rates of polypharmacy in bipolar patients⁴⁷. In addition, neuroimaging studies in bipolar patients have adopted an approach to quantify concomitant medication load based upon dosage and relative burden of each medication class on brain function⁴⁸, which could be applied in clinical trial analyses as well. Sedating medications such as benzodiazepines should be restricted within 6 hours of testing, and other restrictions should be considered depending upon the agent being investigated and specific drug-drug interactions. Finally, although little is known about dose-related adverse cognitive effects, it may be prudent to limit participation to subjects who are taking recommended doses of standard mood stabilizers and/or those whose doses and serum concentrations are in the lower half of the therapeutic range, where applicable. Each of these considerations should be carefully weighed while balancing recruitment feasibility and outcome optimization.

Cognitive Heterogeneity in Bipolar Disorder

Another unique methodological issue concerns the considerable cognitive heterogeneity that is seen in bipolar disorder; in fact, a substantial proportion of patients (perhaps as high as 40%) display neuropsychological performance within the normal range^{49, 50}. Any study focusing on neurocognitive dysfunction must ensure that a sufficient level of deficit is present at baseline to be adequately targeted.

In cognitive trials of schizophrenia, it has been unnecessary to impose a specific inclusion criterion related to a minimal level of cognitive impairment at baseline; however, there is a potential need to do so in patients with bipolar disorder⁵¹. Indeed, in our controlled cognitive trial of pramipexole in bipolar patients, preliminary findings indicated that lower baseline cognitive performance was correlated with greater cognitive improvement in the patients who were taking pramipexole⁴⁶. These findings were only present in the active medication group, not in the placebo group, suggesting that this did not represent regression to the mean but rather an actual treatment effect. Moreover, the potential for ceiling effects in some patients with bipolar illness is of real concern when optimizing study design; therefore, it may be of particular importance in bipolar disorder trials to pre-screen subjects for objective evidence of baseline cognitive dysfunction and to exclude those patients with minimal or no impairment.

Several potential approaches could be implemented for pre-screening patients for sufficient cognitive dysfunction. One approach would be to use patient self-report, or subjective measures, of cognitive impairment; however, this is problematic in patients with bipolar disorder because it has been shown to be only very weakly correlated with objective performance on neurocognitive tasks^52,53. In fact, cognitive complaints among bipolar patients tend to be more highly associated with mood symptoms⁵² – the extent to which this occurs with subthreshold affective symptoms needs to be established. Objective evidence of cognitive deficit, employing a pre-screening battery different from the battery used as the primary outcome measure in a trial, may be more practical. Global screening measures commonly used in other clinical samples (e.g., the Mini-Mental State Examination; MMSE) are likely not applicable in this context due to low sensitivity. Rather, pre-screening batteries should include specific measures that are most commonly impaired in patients with bipolar disorder (e.g., verbal memory; attention). The threshold used as a cutoff can also be somewhat arbitrary; however, impairment of at least 0.5 SD below normative performance is typically considered minimally relevant from a clinical standpoint. To permit adequate room for improvement in performance, it may be preferable to require evidence of an individual having a deficit of at least −1.0 SD in one or more cognitive domains at study entry, dependent in part on the expected target specificity of the agent being tested. Although this cut off might exclude a group of patients who may show some benefit to cognitive enhancing agents it will target those with the greatest need for treatment. Recent empirical approaches to subgrouping bipolar patients based upon neurocognitive profiles suggests that up to 2/3 of bipolar patients would meet the entry criterion of −1.0 SD on more than one domain⁵⁰, supporting this as a feasible cut-off.

Sample Size

Because of the clinical heterogeneity of bipolar disorder, cognitive outcome trials will require either large representative samples or smaller samples that are more narrowly defined with regard to particular clinical characteristics⁵¹. Many clinical trials designed to assess the effects of pharmacotherapy on cognition in schizophrenia have included samples too small to detect a medium (d=0.5) effect size³⁵. Based on MATRICS recommendations for pharmacologic studies, a randomized, double-blind, placebo-controlled, add-on design is optimal²⁷. Crossover designs are not recommended for studies of cognition because of potential confounding of practice effects with treatment effects²⁸, as well as carryover effects of medications. It is worth noting, however, that the basic study design and many of the methodological strategies chosen will differ based upon the phase of study of the agent being investigated. This is particularly relevant when considering issues related to sample size and statistical power because early-phase trials designed to detect a preliminary signal differ from pivotal trials in several ways. Proof-of-concept studies and phase III trials are crucial in drug development particularly in psychiatrically vulnerable individuals, with the focus of initial studies being safety while still including efficacy measures in an effort to detect preliminary efficacy signals and to estimate effect sizes. This is especially true for agents that might be deemed high-risk. For example, the likelihood that a medication might induce mania or exacerbate psychosis should be addressed before large-scale efficacy studies are launched.

Measurement of Primary Outcomes

Neurocognitive Outcome Measures

An important first step in the MATRICS initiative was the formation of a consensus battery⁵⁴ to be uniformly applied to subsequent, large-scale clinical trials targeting cognitive dysfunction associated with schizophrenia²⁸. The MATRICS Consensus Cognitive Battery (MCCB) assesses the domains of attention/vigilance, speed of processing, working memory, verbal learning, visual learning, reasoning and problem solving, and social cognition⁵⁵. Although it was designed for use in cognitive trials of schizophrenia, the applicability of the MCCB to bipolar disorder has been tested. Burdick et al. (2011) evaluated the MCCB in patients with bipolar I disorder and found significantly greater impairment relative to a matched control group on all domains except reasoning/problem solving and social cognition²⁹. Kessler et al. (2013) reported similar results, showing significantly impaired performance on all MCCB domains in both bipolar I and bipolar II patients⁵⁶.

Unlike trials in schizophrenia, no consensus cognitive battery for bipolar disorder yet exists. Yatham et al. (2010) convened a panel of neurocognitive and bipolar experts to begin to address the question of which tests might be optimal for use in patients with bipolar disorder; however, the recommended battery was not specific to clinical trials with cognition as an outcome, but was considered by the panel for a broader application⁵⁷. Nonetheless, recommendations included the use of the MCCB as core, with additional measures added to increase sensitivity and specificity to bipolar disorder. The additional measures included the Stroop Test, the Trail Making Test-part B, the Wisconsin Card Sorting Test, and the California Verbal Learning Test (as an alternative to the Hopkins Verbal Learning Test-Revised used in the MCCB)⁵⁷. Although this is only one potential battery, it addresses some of the critical issues specific to bipolar disorder. Because the deficits in bipolar disorder are less severe than those reported in schizophrenia, higher test sensitivity is important and could be achieved by including either more difficult measures or a greater number of tests tapping each domain (e.g., several of the MCCB domain scores are derived from only a single measure). In addition, neurocognitive tests of affect, such as the affective Stroop Test or facial emotion recognition, should be considered when designing cognitive trials given their link to the pathophysiology of bipolar disorder. One existing schizophrenia battery (Brief Assessment of Cognition in Schizophrenia; BAC-S) has been modified for use in affective disorders (Brief Assessment of Cognition in Affective Disorders; BAC-A) and has been applied in a large-scale treatment trial of bipolar depression⁵⁸.

The choice regarding a primary outcome measure for cognitive trials in bipolar disorder will depend, at least in part, on the agent being studied and its purported mechanism of action; however, the use of a global cognitive measure, as is commonly used in schizophrenia trials, may be necessary if an FDA indication is being sought. Indeed, the best possible result is a drug that will improve each cognitive domain enough (or at least in the same direction) that it will improve global composite scores; however, it is crucial to highlight the lack of a global deficit in patients with bipolar disorder, which may require a different approach from that recommended in schizophrenia. Key secondary cognitive outcome measures will be important if a global composite is used as primary outcome and should be based upon a bipolar-specific profile, therefore focusing on tasks that measure attention, verbal memory, executive functions, and affective processing.

The recommended frequency of assessment will also depend upon the duration of a trial; however, testing at baseline and at least two time points thereafter is suggested to allow for potential attrition without total loss of data. In an 8-week study, for example, testing would be advisable at baseline, at 4 weeks, and again at study completion. Practice effects are nonnegligible in cognitive trials⁵⁹, and although it is unclear how large practice effects will be among patients with bipolar disorder, they are likely to be comparable to practice effects noted in healthy subjects. The MCCB was designed to minimize these practice effects. Indeed, in a phase III trial of clinically stable outpatients with schizophrenia, the practice effect was small for the MCCB composite score⁶⁰. To account for practice effects, a placebo-controlled design is optimal; however, for early-phase, open-label studies, estimated practice effects can be used if necessary based upon normative data.

Functional Capacity Outcome Measures

The selection of appropriate measure(s) of functional outcome is crucial given the FDA stipulation that cognitive enhancement trials must show evidence of benefit on measures of functional capacity as well as objective cognitive assessment²⁷. The MATRICS guidelines note that co-primary measures should be included in cognitive trials and “should assess a clinically meaningful/relevant functional outcome, but not necessarily community functioning… [which] is highly dependent on psychosocial services, patient skills, and social support—factors that are usually beyond the control of clinical trials”²⁷. The University of California at San Diego Performance-Based Skills Assessment (UPSA) is the most widely used functional capacity measure in schizophrenia and has already been adopted for several trials in schizophrenia as a co-primary measure for FDA purposes. The UPSA has been assessed with bipolar I patients, and scores were significantly associated with indicators of impairment in everyday functioning^24,61; however, given the relatively lower level of functional disability in bipolar disorder than in schizophrenia, future studies must consider possible ceiling and practice effects on the brief version of the UPSA. Other widely used self-report measures that have been used in bipolar disorder affective efficacy trials, such as the Sheehan Disability Scale, might also prove useful and be efficient and cost-effective. One measure, the Functioning Assessment Short Test (FAST) has been successful in tracking change in everyday functioning in patients with bipolar illness in non-pharmacological remediation trials^62,63 and may represent a promising tool.

Insomuch as relationships between neurocognitive performance and functional outcome are similar in schizophrenia and bipolar disorder, overlapping approaches could be considered appropriate for both patient groups¹⁵. Emerging evidence does, indeed, suggest that the associations between cognitive impairment and functional capacity in schizophrenia pertain in a similar manner to cohorts of patients with bipolar disorder^24,64. Consistent with trial design in schizophrenia, cognitive trials in patients with bipolar disorder should incorporate measures of both functional capacity (having the skills necessary to perform an activity; e.g., what a person can do) and everyday functioning in the community (e.g., what a person does do, which may be influenced by even subthreshold depressive symptoms). In shorter-duration trials, functional capacity measures (e.g., the UPSA) might be more tractable and amenable to change than are assays of how a person is functioning in the community (e.g., employment status). Although the MATRICS committee recommends a study duration of at least 6 months for phase III trials to demonstrate an enduring effect on cognition in schizophrenia²⁷, this may be less feasible in bipolar patients because of the need to control for normal cycling associated with the illness. Finally, the use of objective measures of everyday functioning and/or ratings made by an informant is generally recommended because patient self-reports of functioning are often inaccurate¹⁵.

Summary

Neurocognitive performance is significantly impaired in a substantial proportion of patients with bipolar disorder despite adequate control of affective symptoms²⁹. These deficits are strongly associated with disabilities in community function^24,25 and warrant treatment⁴⁹. In contrast to a concerted effort to target this domain in patients with schizophrenia (e.g., MATRICS), very few trials have been conducted in bipolar disorder. As the treatment of bipolar disorder evolves from the initial focus on controlling primary affective symptoms to the current consideration of secondary targets for a more complete recovery, it will be important to address the challenges inherent in this complex disorder. In this manuscript, we have highlighted several illness-specific confounders that arise when designing cognitive trials in bipolar disorder, and we have made several suggestions for handling these a priori (Table 1). While some of the considerations discussed regarding cognitive trials are unique to these types of outcome measures, several issues are more broadly relevant to the design of experimental treatment trials in bipolar disorder even when targeting the primary affective symptoms⁶⁵. Here, we propose a design that highlights the importance of both internal and external validity, but ensures the potential for improvement by requiring sufficient cognitive impairment at the time of study entry by limiting enrollment to those patients having a least −1.0 SD in one or more cognitive domains at study entry.

Although the field remains in its infancy, a consensus on approaches to be implemented will be critical for facilitating advances in this arena. It will be important to systematically address challenges as they arise during the conduct of future studies and to work as a field to optimize the chances of positive study outcomes, with the ultimate goal of optimizing the chances of positive patient outcomes. We present here some preliminary suggestions based upon the available evidence; however, the time is upon us to consider a more comprehensive, formalized consensus effort parallel to that conducted in the field for cognitive trials in schizophrenia (e.g. MATRICS).

CLINICAL POINTS.

• There has been a strong focus on cognitive deficits in schizophrenia over the past century, with concerted efforts put forth to begin targeting them directly for treatment over the past few decades. In contrast, parallel efforts in bipolar disorder are lagging, despite convergent data indicating direct and deleterious effects on functional outcome and quality of life.

• This article is aimed at researchers designing clinical trials for cognition in bipolar patients; however, lessons learned from such controlled trials can translate directly to patient care, with the ultimate goal of developing new agents to combat this disabling aspect of the illness. Moreover, a better understanding of the illness features that contribute to the neurocognitive problems in this disorder will allow for more meaningful psychoeducation for patients who hope to limit and even prevent cognitive dysfunction from occurring.