The “Right Stuff” Revisited: What Have We Learned About the Determinants of Daily Functioning in Schizophrenia?

Abstract

It has been about 15 years since we published our article asking whether we are measuring the “Right Stuff” as we search for predictors and determinants of functional outcome in schizophrenia. At that time, we raised the question as to whether the neurocognitive assessments used to study outcome in schizophrenia were too narrow to capture the wide variability in factors that determine daily functioning. While the study of the determinants of functioning in schizophrenia has grown and matured, we are struck by 3 aspects of the article that evolved in different directions. First, the selection of outcome domains in the Right Stuff meta-analysis reflects a focus at that time on predictors of psychiatric rehabilitation. Second, expansion beyond traditional neurocognitive domains occurred in one suggested area (social cognition), but not another (learning potential). Third, the field has responded assertively to the recommendation to evaluate more informed and informative theoretical models.

It has been 15 years since we published the article “Neurocognitive Deficits and Functional Outcome in Schizophrenia: Are We Measuring the “Right Stuff”?”.¹ We have been invited to comment on what has changed and what we have learned since its publication. In retrospect, the article was an odd hybrid of a meta-analysis and a spirited thought piece. The unusual structure of the article was tolerated by Wayne Fenton, who was the action editor for that issue. Despite an idiosyncratic organization and a slightly technical meta-analysis, the article had strong impact. Currently, it is listed as the second most-cited article in Schizophrenia Bulletin’s history, after the publication of the Positive and Negative Syndrome Scale (PANSS) that preceded it by 13 years. Google Scholar lists 2037 citations of the Right Stuff article.

Reading the article after a long interval, we are struck by three aspects that have changed substantially: The selection of outcome domains in the meta-analysis, the recommendation to expand beyond relatively traditional neurocognitive domains, and the recommendation to move beyond bivariate correlations and evaluate more informed and informative theoretical models. We will briefly discuss each of these topics.

Selection of Outcome Domains

The outcome measures in the Right Stuff meta-analysis were divided into three categories: (1) success in psychosocial skill acquisition, (2) laboratory assessment of instrumental skills and social problem-solving ability, and (3) community functioning. Among these three only the third category, community functioning, continues to be a major focus in studies of functional outcome in schizophrenia. It includes aspects of daily living, such as work, social connections, and degree of independent living.² Although each outcome domain may have its own predictors, specific predictor-outcome linkages have not been established. Though, across studies, cognition seems to explain work outcomes better than social outcomes.

The other two categories might seem curious from today’s perspective. The first category, success in psychosocial skill acquisition, reflects the fact that our group’s initial interest in cognitive predictors of outcome grew out of questions about individual differences in response to psychiatric rehabilitation.^3,4 Essentially, we wanted to understand why some patients responded better to their rehabilitation programs than others. Structured psychiatric rehabilitation programs were well-suited for these types of questions because: (1) progress on skills training can be objectively measured by tests of content knowledge and performance-based measures of skill acquisition, and (2) progress can occur in a matter of weeks or months, as opposed to changes in community functioning that typically take longer. Not many recent studies of cognition and outcome focus on psychiatric skills training, perhaps because highly structured social skills training programs are not as common in research settings as they were previously.

A more modern way to approach this question is to try to predict individual differences in response to cognitive remediation. However, the results have been entirely mixed with some studies showing that higher baseline cognition is associated with better response to cognitive remediation,^5–8 and others showing poorer cognition at baseline is associated with better treatment response.^9–11 Given the opposing effects, it is possible that the relationships are nonlinear; for example, a predictor of good response in a low cognitively functioning group of patients might be a predictor of poor response in a high functioning group. Such a pattern was probably not noticeable in the previous studies of cognitive predictors of skills training because studies in this category in the Right Stuff article included low functioning inpatients.

Our second category of “laboratory assessments of instrumental skills and social problem-solving ability” would be called “functional capacity” in modern parlance. Functional capacity refers to tasks that assess the capacity to perform key tasks of daily living, or social interactions, by asking participants to simulate real-world activities, such as holding a conversation, selecting grocery items to prepare a meal, and planning a trip using public transportation.^12,13 Good performance on such measures indicates that a person can perform the task, but not necessarily that he/she performs it in the real world.

In the Right Stuff article, we considered functional capacity measures to reflect an outcome, but they could equally be considered as predictors of outcome. For example, in some models of outcome, functional capacity is viewed as a determinant of functioning in the community.¹⁴ The findings across studies are mixed in terms of how well functional capacity measures relate to daily functioning, with some studies finding strong connections,^14,15 and others fairly weak ones.^16,17 Overall, functional capacity shows very strong and consistent associations with neurocognitive test batteries, suggesting that some similar cognitive components are assessed in each type of task.

Functional capacity has figured prominently in a different context—it has become a major focus for clinical trials of cognition enhancement in schizophrenia. The United States Food and Drug Administration has taken the position that approval of new treatments for cognition in schizophrenia will require changes in cognitive performance, as well as a functionally meaningful co-primary measure.¹⁸ Measures of functional capacity are viewed as potentially valuable functionally meaningful co-primary measures for a few reasons: They are well tolerated by patients, they have good face validity for improved functioning, and they have high correlations with cognitive performance measures.^16,17

Expanding Beyond Neurocognitive Predictor Measures

The Right Stuff article can be viewed as a continuation of a discussion that started with an article published in the previous year in Schizophrenia Bulletin entitled “Should Schizophrenia be Treated as a Neurocognitive Disorder”.¹⁹ In that article, we suggested that social cognition might be an informative mediator acting between nonsocial cognition (neurocognition) and functioning. In the Right Stuff article we referred back to that point and expanded it by suggesting that learning potential might be another valuable mediator.

In retrospect, we were half right. Social cognition in schizophrenia has emerged as an area of energetic study and growth.²⁰ This rapid growth is partly motivated by the demonstrated links between social cognition and social disability in schizophrenia.²¹ Considerable research now supports the connection between social cognition and functional outcome. A meta-analysis²² demonstrated relationships between functional outcome and three domains of social cognition: mental state attribution = 0.48, social perception = 0.41, and emotion perception = 0.31. Further, the association between social cognition and functional outcome is stronger than that with nonsocial cognition and functional outcome.²² Consistent with that finding is evidence that social cognition provides incremental validity in terms of explained variance in functioning beyond that provided by nonsocial cognition.^23–25

In contrast, and despite our enthusiasm, the study of learning potential has nearly disappeared from the schizophrenia literature after an initial flurry of articles in the years following the Right Stuff paper. Interest in learning potential stemmed from its uniqueness as a psychological construct that differed and added to “static” measures of cognition that are generated by a single assessment. Learning potential (also called dynamic assessment) refers to the latent capacity to learn, not what knowledge the individual has already acquired. It was commonly measured with the Wisconsin Card Sorting Test (WCST) using a dynamic assessment approach in which intra-individual variability is assessed over repeat administrations in a test-train-test format. In addition to yielding dimensional measures, learning potential could be used to categorize subjects into three primary groups: learners, nonlearners, and high performers.

Published studies of learning potential since the publication of the Right Stuff paper focused on the neurocognitive profiles or psychological correlates of learning potential in schizophrenia patients.^26–29 There was also a focus on the relationship of learning potential with functional outcome, including studies of rehabilitation outcome,^30,31 functional capacity,^26,27,32 and studies examining social or community functioning.^27,33 Inconsistent findings regarding these relationships between learning potential and functional outcome might have contributed to the topic’s decline in the psychiatric literature; indeed, the findings were decidedly mixed (six positive, five negative), with relatively little support for its incremental validity beyond neurocognitive measures. In addition, the two studies that specifically examined learning potential as a mediator between neurocognition and functional outcome failed to support that role.^34,35 Hence, the diminished interest in learning potential may have been due to its lackluster performance in adding to the prediction of functional outcome, beyond less labor-intensive, static assessments of cognition.

Understanding Mechanisms and Pathways

We recognized, even back then, that simply establishing relationships between neurocognition and outcome was not enough—it was important to understand the nature of those relationships.

From this literature, we have learned about whether neurocognition is related to functional outcome. However, we have learned very little about how neurocognition is related to functional outcome. Identifying mechanisms and mediators was not a goal for these studies. Instead, these studies were testing neurocognitive-functional relationships to see if they existed, or to better understand heterogeneity in functional outcome. Once relationships have been demonstrated, it is reasonable to make a more concerted effort to identify mechanisms. (page 130 in ref.1)

This fundamental question—how do problems in cognition lead to functional outcome—has now been examined in data-based papers from several labs. Careful examinations of possible mechanisms were not available at the time of the Right Stuff article, probably because these studies require larger samples and more diverse assessments than we were seeing at the time. It is useful to divide the mechanism studies into two types: Examinations of promising mediators and elaborate multi-step models.

The most closely examined mediator is social cognition, and the evidence is strong and consistent. Over 15 separate studies have shown that social cognition has significant relationships to nonsocial cognition on the one hand, to work and social community functioning on the other, and that the direct relationship between nonsocial cognition and outcome is reduced or eliminated when social cognition is added to a model.³⁶ Hence, it behaves as a mediator. A review of this literature found that about 25% of the variance in outcome is explained by such mediation models.³⁶

Simple mediation provides strong clues as to the important variables, but it is not terribly informative about pathways. Beyond mediation, multi-step pathways with various intervening variables are being constructed and evaluated to provide insights about the key steps to daily functioning. These models have appeared in the literature more recently, and they are inherently more difficult to do—they require relatively large sample sizes to be reliable, a broad array of predictor variables to be informative, advanced statistical expertise to be defensible, and a strong a priori theory to be credible. One key challenge for elaborate models is to evaluate how both social and non-social cognitive dimensions interface with other factors that are known to impact functional outcome in schizophrenia. For example, we know that negative symptoms are related to daily functioning, and that some types of negative symptoms are more closely associated than others. It is mainly the experiential negative symptoms (eg, anhedonia and avolition), as opposed to expressive negative symptoms (eg, blunted affect), that are linked to functioning.^37,38 Experiential negative symptoms are also considered motivational symptoms in that they indicate a lack of willingness to engage in one’s community.

Given that cognitive abilities and experiential negative symptoms both are related to community functioning, it is possible that outcome is determined by two separate pathways: One based on abilities (such as cognition) and one based on motivation (such as negative symptoms). However, it is also plausible that ability and motivation are connected and reside along a single pathway. If so, the link between the two may be in the area of beliefs. For example, there is encouraging support for the importance of dysfunctional attitudes, which refers to overly generalized negative beliefs about one’s abilities.³⁹ There are several types of dysfunctional attitudes, and much of the focus in schizophrenia is on defeatist performance beliefs, which are overly generalized negative beliefs about one’s ability to successfully perform tasks.^40,41

According to a model by Beck et al^42,43 reduced ability (in our models reflected by impairments in perception, nonsocial cognition, or social cognition) leads to discouraging life events, which in turn, lead to negative attitudes and self-beliefs. These dysfunctional attitudes/beliefs contribute to the decreased motivation that we see clinically as negative symptoms. According to this single pathway model, cognition and functional outcome are related through a causal pathway involving dysfunctional attitudes.

The figure 1 shows the results of a structural equation model (SEM) evaluation of such an integrative model with key determinants of functional outcome in schizophrenia, including early visual perception, social cognition, defeatist beliefs, and experiential negative symptoms.⁴⁴ Outcome is defined as a latent trait based on work, independent living, family, and social indicators. A single pathway explained the data quite well (Chi square = 39.41, P = 0.04; comparative fit index [CFI] = 0.96; root mean square error of approximation [RMSEA] = 0.06). Further, no changes in the paths were suggested by modification indices that would have turned the single pathway into a dual pathway. For example, the SEM model was not improved by adding a direct link between social cognition and functional outcome (one separate from beliefs and motivational negative symptoms).

Fig. 1.

Adapted from Green et al. Archives of General Psychiatry, 2012.

A single pathway is the most parsimonious solution, but other groups have entertained the possibility of multi-pathway models to functioning.^15,45 Recently, a very large study (921 schizophrenia patients) was conducted with SEM using an impressive number of predictor variables (psychotic symptoms, avolition, social cognition, nonsocial cognition, functional capacity, and internalized stigma among others).¹⁴ This study found support for several pathways that start with nonsocial cognition, including one path that ran through social cognition, one that ran through functional capacity, and one that ran through stigma and resilience. Hence, the question of one versus multiple pathways to outcome in schizophrenia is not yet settled.

Conclusions

At the age of 15, the Right Stuff article reminds us of the shift in focus that occurred when cognition became a factor to be studied because of its relevance to daily functioning, beyond its informative value for underlying neural systems in schizophrenia. In the time since the publication, the field has grown, expanded, and matured. In this commentary, we discuss three aspects of the article that evolved in unexpected directions. First, the selection of outcome domains in the Right Stuff meta-analysis was heavily influenced by a focus at that time on predictors of psychiatric rehabilitation. Second, expansion beyond traditional neurocognitive domains occurred in social cognition, but not for learning potential, which was a focus of discussion in the Right Stuff article. Third, we are now finally starting to see the fruits of larger samples, more diverse measures, and clearer theories to construct testable models of outcome. These models provide insights to the pathways to daily functioning, information that probably applies to a range of neuropsychiatric conditions that are characterized by functional impairments.