“Generalized Cognitive Deficit” in Schizophrenia: Overused or Underappreciated?

Reports of my death are greatly exaggerated—Mark Twain

Green, Horan, and Sugar provide a new look at the generalized cognitive deficit problem in schizophrenia and argue persuasively that the criticism is not appropriate in a variety of circumstances. More significantly, noting a decrease in mentions of the general deficit problem in published literature in recent years, along with mounting evidence of areas of relatively normal cognitive performance in people with this illness, Green et al suggest that the generalized deficit may come to be of lesser importance in contemporary schizophrenia research and that, increasingly, it is yielding ground to developing behavioral and neuroimaging methodologies that permit ever more precise parsing of cognitive operations and neural systems.

We suspect the reason that Green and colleagues had difficulty retrieving recent papers addressing the generalized deficit is that the evidence documenting it has simply become so overwhelming. That is, the 40-year search for focal, specific deficits, bounded by evidence of normal performance on other related measures, has been largely disappointing. Enormous effort was expended in the late 20th century trying to substantiate claims for certain differential cognitive deficits in schizophrenia, and ideas took hold then that still hold sway (“deficits in executive functions and episodic memory are particularly pronounced in this illness”). In the context of this effort, the general deficit was considered a nuisance that obscured the ability to identify critical deficits that might have provided important clues to understanding the psychological and neural mechanisms implicated in the illness.

In the clinic and the assessment lab, however, what was always most striking was the overall cognitive impairment in people with schizophrenia relative to control groups rather than any particular peaks or valleys in the performance profile. Study after study showed a reduced performance in those with the illness, generally in the vicinity of 1.0 SD, across a wide range of neuropsychological measures. Meta-analyses of neuropsychological data from thousands establish unambiguously that this is the rule in schizophrenia—at least for these sorts of measures. Thus, the lack of recent interest in the general deficit may reflect a realistic appraisal of the state of the art; clinical neuropsychological methods that have dominated the research literature over the past 25 years offer little evidence of differential deficits. To us, it is deeply puzzling that this has not become a major focus of schizophrenia research and experimental psychopathology—it is the elephant in the room for cognitive studies.

Green et al cite a number of recent papers that report areas of intact cognitive function in schizophrenia—papers that call into question the generality of the generalized deficit. Although one of us (JG) has contributed multiple experimental papers to this literature, in regard to the generalized deficit debate, we both consider these findings with caution.^1–3 As noted by Green et al, these findings often emerge from highly constrained experiments designed to isolate specific component operations, and the extent to which patient performance is fully “normal” is not always straightforward. For example, in the Posner spatial cuing paradigm, there is robust evidence that patients are able to use cues to facilitate reaction time (normal effect of selective attention), but patients are nearly invariably substantially slower overall than controls.¹ In our series of working memory experiments demonstrating intact operation of selective attention in guiding the encoding of relevant items and suppression of irrelevant items, overall working memory storage was clearly below normal levels.^2,3 Thus, our patients have shown qualitative evidence of intact operation of specific cognitive mechanisms, often coupled with an overall performance disadvantage relative to controls.

These examples are important in that they demonstrate intact function of specific cognitive mechanisms, but this is a long way short of providing a basis to discount the clear evidence for broad impairment seen in the extensive neuropsychological literature. Indeed, a simple framework roughly accommodates both themes from the literature; the more a task requires the integration of multiple cognitive operations, the more a task depends on the “general” coordinated functions of the cortex, the more likely patients are to deviate from healthy performance levels. Notably, the obverse statement is not reliably true; some highly circumscribed, simple, localized functions, such as mismatch negativity and visual perceptual organization, are clearly substantially impaired in people with schizophrenia. However, when intact performance is observed, it is likely to be a relatively “modular” function. Thus, our view is that evidence for intact performance helps to focus thinking about the nature of the generalized cognitive deficit in schizophrenia—and perhaps indicates that it would be better labeled “broad” rather than “generalized”—but does not, in the end, much diminish the challenge it poses for the field. The “elephant” may be smaller than the casual terminology suggested, but at this point in the history of cognitive research in schizophrenia, it is still the elephant.

Green et al are optimistic that new methodologies, drawing on sophisticated statistics and a maturing cognitive neuroscience, will be less vulnerable to the psychometric issues that have long challenged the field. We hope that this will be the case, facilitating progress, but we have reservations. The newer behavioral methods from the cognitive and affective neuroscience literature will face the same challenges as earlier approaches. There is still the “Chapman problem” of achieving equal discriminating power for experimental and control conditions and tasks. Measure reliability, floor and ceiling effects, standardization, and burden are always critical issues but have only recently taken center stage in development of relevant experimental paradigms.⁴ Importantly, all of these concerns are just as troublesome in neuroimaging studies as they are in behavioral studies. Neurophysiological measures of any type differ in measurement sensitivity and reliability limiting the ability to make claims that this region of interest (ROI) or this event related potentials component is impaired, while this other ROI or component is not impaired. Rather than disposing of the generalized impairment problem, the increased application of rapidly evolving, technology-intensive methodologies for cognitive studies in schizophrenia may have set the stage for a reprise of the sorts of differential deficit claims and counterclaims that we saw in decades past.

Another conundrum is whether we find ourselves faced with a no-win trade off. It may be the case that particular paradigms and analysis strategies offer some ways of corralling the generalized deficit. However, if the research strategy depends on narrower and narrower experimental constraint, we may do a better job walling off the general deficit “problem” and making interpretation of experimental findings unambiguous, but at the potential cost of learning less of importance about schizophrenia. That is, it may be that the kinds of operations that are most validly and reliably measured with these methods are rarely critical or rate-limiting for adaptive behavior. Alternatively, if schizophrenia is a network dysconnectivity disorder, more complex measurement approaches may be better suited to assay this type of disturbance than more refined and process pure measures. Clearly, the hope is newer approaches will offer more precise measures and enhance interpretation at the neural system and cognitive process level while maintaining clinically important signals (relationship with outcomes, treatment response, genetic risk status, etc), but investigation of the clinical relevance of newer methods is only beginning and the value of newer methods remains to be demonstrated by labs around the world. The older methods, despite their imperfections, have a track record in this regard—one that highlights the reality and clinical significance of broad impairment.

Statistical modeling and related correlational analyses are other tools cited by Green et al. Here too, we think a cautious stance is appropriate, particularly with regard to what variance can be considered “unique” in these analyses. The literature consists almost exclusively of analyses of data collected using measures that have psychometric limitations. Differences in discriminating power across measures, to take one example, could easily result in a spurious finding of uniqueness. We agree with the authors that not every claim of uniqueness can be dismissed on this or similar bases—but the authors suggest that problems of this sort are “unusual,” and we suspect they may be more common. We agree with Green and colleagues that causal models that span from cognition to symptoms to outcomes offer important insights. However, the ability to determine which cognitive measures/constructs play a critical role in the causal pathway will always be potentially compromised by psychometric differences in the measures evaluated in the model. The power of modeling approaches can only be enhanced with increasing measurement precision, a conclusion that we share with Green et al.

There is also a broader issue to consider. Much of the field is moving toward a focus on discrete cognitive subcomponents tied to discrete neural systems. There is great momentum in this direction and likely no turning back because the field is clearly in a hurry to more tightly integrate basic and clinical neuroscience. Of course, the ultimate goal has to be to understand how various pieces work together to form more complex biological and cognitive mechanisms and how these mechanisms get derailed in illness. The vexing challenge of accommodating “development” in this mechanistic approach is often cited. Over decades, individual development builds densely interwoven, massively interacting biological and cognitive systems. The defining characteristics of such systems may be “emergent” properties of the integrated systems that may resist this sort of dissection and reassembly strategy, even in typically developing brains. How these challenges are compounded by many years of compromised development—probably a given in schizophrenia—is almost entirely unknown. Closely related are considerations related to genetics. It is axiomatic that genes encode protein building blocks rather than integrated behaviors. Their influence on cognition may well come through low-level and widely acting mechanisms (eg, dendritic arborization, synaptic plasticity). Such processes—functioning normally or abnormally—should be expected to contribute to the emergence and refinement of a broad range of cognitive abilities over the course of development.

So, is the generalized cognitive deficit in schizophrenia a problem of the past? Green and colleagues are certainly correct that the issues are narrower than suggested by the common terminology. And they certainly deserve credit for not just touching but grabbing onto the “third rail” of experimental psychopathology. However, it would be ironic if our field began questioning whether the broad cognitive impairment documented with neuropsychological measures is a central feature of the illness, as occurred decades ago during the hunt for specific, differential cognitive deficits. At present, we would argue that the weight of evidence is fairly clear. This broad impairment is the most substantial and reliable cognitive signal in schizophrenia research. We think it deserves greater, not less, attention from researchers.

In trying to understand the genesis of this broad impairment, it will be important to consider the challenges posed to this perspective by the recent evidence showing relatively intact performance in certain aspects of cognition. There are islands of relatively intact function in the sea of broad impairment. These islands will prove to be either isolated curiosities or important clues—it is too early to tell—but it is good to have them on the map. Similarly, we suspect that the field would make more rapid progress if more researchers sampled from both traditional neuropsychological measures and cutting edge measures (eg, those developed by the CNTRACS consortium)⁴ and explored the clinical signals of both approaches in the same subjects. These are questions that can be answered with data. We appreciate the courage of Green et al to put questions about broad cognitive impairment in schizophrenia back in front of the field, out in the open, and not just hidden away in manuscript reviews and study-section meetings.

Funding

National Institute of Mental Health (MH080066, MH065034 to Dr Gold); Dr Dickinson’s work is supported by the Intramural Research Program, National Institute of Mental Helath, NIH.