Within-individual Variability in Cognitive Performance in Schizophrenia: A Narrative Review of the Key Literature and Proposed Research Agenda

Abstract

Schizophrenia is a neurodevelopmental disorder and a leading cause of disability worldwide. Deficits in cognitive function are characteristic of schizophrenia and are predictors of functional outcomes in the disorder. Within-individual variability (WIV) in cognitive performance is elevated in schizophrenia and has been suggested to provide additional insight into cognitive function over and above mean performance measures. Despite growing interest in WIV in schizophrenia, research on the clinical significance and neural correlates of WIV in the disorder remains sparse. The present narrative review summarizes the key literature linking WIV in schizophrenia to clinical, neural, and genetic correlates. Here, we aim to highlight key knowledge gaps and provide directions for future research into WIV in schizophrenia.

1. Introduction

Impaired cognitive function is a well-documented feature of people with schizophrenia (PSZ) and a major determinant of functional outcomes (Fett et al., 2011; Fioravanti, Bianchi, & Cinti, 2012). Typically, mean measures of performance on cognitive tests are used to compare cognitive function between individuals (e.g., distribution among a group) and groups (e.g., people with and without schizophrenia). Studies using such measures have contributed towards our understanding of the neurobiological underpinnings and functional consequences of impaired cognition in PSZ.

Cognitive performance may also be assessed by measuring within-individual variability (WIV) in performance on cognitive tests. WIV in cognitive functioning reflects the extent of variation in an individual’s performance relative to their mean performance across trials of a task, or across multiple tasks in a cognitive battery (MacDonald, Li, & Bäckman, 2009). In conditions such as schizophrenia where variability in performance on cognitive tests is increased, it has been argued that mean measures of performance on one or a battery of tests fail to capture some of the meaningful cognitive differences among people (MacDonald, Li, et al., 2009; Reichenberg et al., 2006; Rentrop et al., 2010). In this context, investigators have suggested that additional insight into cognitive function may be attained by measuring WIV. When performance variability increases, WIV may provide insight into the stability of cognitive processing by capturing systematic as opposed to random errors (MacDonald, Li, et al., 2009).

WIV is increased in PSZ (Cole, Weinberger, & Dickinson, 2011; Rentrop et al., 2010; Vinogradov, Poole, Willis-Shore, Ober, & Shenaut, 1998), at-risk mental state (ARMs) individuals (Shin et al., 2013) and in family members of PSZ (Roalf, Gur, et al., 2013), and there has been increasing interest in its clinical, neural, and genetic correlates. Despite this growing body of literature, many aspects of WIV in schizophrenia remain unexplored and there have been no attempts to summarize knowledge of WIV among PSZ. In this narrative review, we aim to provide a brief conceptual overview of WIV, to integrate findings from neuropsychological and neurobiological research into WIV among PSZ and to discuss the implications of these findings for future research.

2. Methods

The current literature on WIV in cognitive performance in PSZ was reviewed by searching the PubMeD, Scopus, Web of Science and PsycInfo databases (last search: 30 September 2022). We used the following search terms: “(“Within-individual variability” OR “Intra-individual variability” OR WIV OR IIV OR “Cognitive variability” OR “reaction time variability” OR “behavioural variability” OR “intraindividual differences”) AND (cognit* OR “neuropsychological tests”) AND (Schizophrenia OR Psychosis OR Psychotic disorder” OR “at-risk mental state” OR “psychotic disorder not otherwise specified” OR “schizophreniform disorder” OR “brief psychotic episode”)”. This yielded 114 records after duplicates were removed. Titles and abstracts were screened to determine eligibility and 43 articles reached inclusion criteria for consideration in this review. Cross-referencing of the relevant articles was also performed. Studies were included if they were published in English and measured within-individual variability in cognitive performance in PSZ, people with psychotic spectrum disorders, or people at risk of developing schizophrenia. Studies were excluded if the full text article was not available.

3. Within-individual Variability: Background

WIV has been widely studied in cognitive aging and is characterized by a U-shaped function across the lifespan; decreasing throughout childhood and adolescence before increasing throughout adulthood (Hultsch, MacDonald, & Dixon, 2002; MacDonald, Li, et al., 2009; Roalf et al., 2014). It is hypothesized that the characteristic changes to brain structure that occur during neurodevelopment and thereafter are responsible for the observed changes in WIV over the lifespan (MacDonald, Nyberg, & Bäckman, 2006). Additionally, sex differences in WIV have been demonstrated with males having increased WIV in cognitive performance compared to females (Roalf et al., 2014). Variability in cognitive performance is present in healthy adults (Binder, Iverson, & Brooks, 2009), however when variability exceeds a certain threshold, it is a useful indicator of impaired cognitive function (MacDonald, Li, et al., 2009). Increased WIV has been consistently associated with schizophrenia, traumatic brain injury, major neurocognitive disorders and attention deficit hyperactivity disorder (Cole et al., 2011; Leth-Steensen, Elbaz, & Douglas, 2000; Lin, Hwang-Gu, & Gau, 2015; MacDonald, Li, et al., 2009; Roalf, Ruparel, et al., 2013; Stuss, Murphy, Binns, & Alexander, 2003).

WIV may be operationalized in 3 ways: i) across-tasks in the same testing session, ii) across trials of the same task during the same testing session, and iii) across trials of the same task during different testing sessions (Cole et al., 2011; MacDonald, Li, et al., 2009). This review will focus on the first two measures of WIV, which are measured over short time periods, and are thought to better capture endogenous processes, such as abnormal neural activity (MacDonald, Li, et al., 2009). In conditions such as schizophrenia, where performance across cognitive domains is differentially affected, across-task WIV may be used to differentiate PSZ from matched controls (Cole et al., 2011; Roalf, Gur, et al., 2013; Roalf, Ruparel, et al., 2013). However, it is more common for studies of WIV to calculate across-trial WIV using reaction times for correct trials of a cognitive task (MacDonald, Li, et al., 2009). Standard measures of across-trial reaction time variability include the individual standard deviation and the individual coefficient of variation. The individual coefficient of variation is calculated as the ratio of the individual standard deviation to the individual’s mean reaction time (Stuss et al., 2003). Whereas the individual standard deviation may be higher in individuals with longer but not necessarily more variable reaction times, the individual coefficient of variation is robust to these effects and is considered a more reliable measure of WIV (Stuss et al., 2003). Standard methods of measuring reaction time variability have been criticized as reaction time distributions are typically positively skewed with a higher proportion of longer reaction times and thus, are not accurately represented by a normal distribution (see Supplementary Figure 1). A newer approach to the analysis of WIV involves applying an ex-Gaussian model to the reaction time distributions (Leth-Steensen et al., 2000; Rentrop et al., 2010). The ex-Gaussian distribution can be decomposed into two independent components: a normal component, and an exponential component, which corresponds to the right-sided tail of the distribution (Heathcote, Popiel, & Mewhort, 1991; Luce, 1986). The mean of the exponential component (tau) is used as a measure of reaction time variability with a larger tau corresponding to an increase in variability of reaction times (Heathcote et al., 1991).

4. The Functional and Clinical Significance of WIV in Schizophrenia

Variability in task performance has been documented as a feature of schizophrenia since Kraepelin’s seminal works on dementia praecox (Kraepelin, 1913). Over the past century, efforts have been made to understand the utility of WIV as a risk factor for psychosis and its relationship with symptom severity and functional outcomes. There is mounting evidence that WIV may be an early indicator of susceptibility to psychosis. In ARMS individuals, increased across-trial WIV on a reaction time task has been demonstrated when mean performance measures on cognitive tests were within normal range (Shin et al., 2013). Further, elevated WIV has been found to be predictor of psychopathology in healthy population cohorts. A large population-based longitudinal study found that elevated across-trial WIV in reaction time in adolescence was predictive of later psychotic-like experiences (Wallace & Linscott, 2018). Additionally, increased across-task WIV in adolescents with normal intelligent quotient (IQ) scores was found to be associated with an increased risk of later hospitalization for schizophrenia (Reichenberg et al., 2006). As elevated WIV has been observed in youth with ADHD and mood disorders, it is unlikely that it is specific to risk of psychosis (Doyle et al., 2018). Instead, WIV appears to represent a disruption to cognitive processes arising from neurobiological abnormalities that are common across neurodevelopmental disorders (Wallace & Linscott, 2018).

PSZ demonstrate deficits across multiple cognitive domains when compared to matched controls (Fioravanti et al., 2012; Heinrichs & Zakzanis, 1998). This global deficit in cognitive function is typically captured by g, a measure of general cognitive ability. In PSZ, g and across-task WIV in cognitive performance are negatively correlated (Cole et al., 2011) however, it has been proposed that WIV is a more sensitive marker of cognitive deficits. In a study of high-functioning PSZ, across-task WIV significantly differed between people with schizophrenia and healthy controls when premorbid IQ and mean performance measures on cognitive tasks did not (Rentrop et al., 2010). Despite the utility of WIV as an index of impaired cognition in schizophrenia, its clinical and functional significance in the disorder remains unclear. Studies have failed to show a consistent relationship between WIV and negative, positive, and disorganized symptoms in PSZ (Akiyama et al., 2016; Pellizzer & Stephane, 2007; Rentrop et al., 2010; Roalf, Ruparel, et al., 2013; Shin et al., 2013; Wexler, Nicholls, & Bell, 2004). Table 1 provides an overview of key findings from studies assessing the relationship between WIV and symptom severity in PSZ. The heterogeneity in results may be attributed to the different measures of WIV and symptom severity across studies. Alternatively, it is possible that the neural mechanisms responsible for increased WIV in PSZ differ from those underlying the other symptom domains. Findings regarding the association between WIV and functional ability in PSZ are more consistent. For example, multiple studies have shown a negative correlation between across-trial WIV in reaction time and measures of functional ability, including occupational and global functioning (Rentrop et al., 2010; Vinogradov et al., 1998; Wexler et al., 2004). To extend this knowledge, future research may consider the use of longitudinal studies to explore the utility of WIV as a predictor of functional outcomes.

Table 1.

Overview of studies assessing the relationship between cognitive variability and symptom severity in people with schizophrenia

Article	N	Study Design	Cognitive Assessment	Measure of Cognitive Variability	Measure of Symptom Severity	Main Findings
Akiyama et al. (2016)	PSZ (n = 288) HC (n = 308)	Cross-sectional	Brief Assessment of Cognition in Schizophrenia	Within-individual standard deviation of performance on cognitive tests	PANSS	Significant negative relationship between WIV and negative symptom score. No association between WIV and scores on positive or general psychopathology subscales.
Pellizzer and Stephane (2007)	PSZ (n = 21) HC (n = 18)	Cross-sectional	Choice RT Task	Within-individual interquartile range of reaction time across correct trials	BPRS SAPS SANS	No significant relationship between WIV and symptom score.
Rentrop et al. (2010)	PSZ (n = 28) HC (n = 28)	Cross-sectional	Go/Nogo task Continuous Performance Test	Ex-gaussian parameters for correct responses in the Frequent-Go condition in Go/Nogo task	PANSS	No significant relationship between ex-gaussian parameters and symptom scores.
Roalf, Ruparel, et al. (2013)	PSZ (n = 25) HC (n = 27)	Cross-sectional	University of Pennsylvania Computerized Neurocognitive Battery	Within-individual standard deviation of speed and accuracy performance measures across cognitive tests	BPRS SAPS SANS	Increased speed WIV was associated with higher SAPS scores. No association between speed WIV and SANS or BPRS. No association between accuracy WIV and symptom measures.
Shin et al. (2013)	PSZ (n = 37) HC (n = 38) ARMS (n = 27)	Cross Sectional	Stop-signal task from the Cambridge Computerized Neuropsychological Tests	Within-individual standard deviation of speed and performance measures across sub-blocks of the SST	PANSS BPRS	Trend level positive correlation between WIV in the stop process in PSZ and general psychopathology scores on the PANSS. No association between WIV and negative and positive symptom scores.
Wexler et al. (2004)	PSZ (n = 17)	Longitudinal	10 visual or auditory discrimination tasks from Psychological Software Services	Within-individual coefficient of variation in reaction time across trials	PANSS	No association between WIV and symptom scores.

Note. This table provides an an overview of key findings from studies assessing the relationship between WIV and symptom severity in PSZ. ARMS, at-risk mental state; BPRS, Brief Psychiatric Rating Scale; HC, healthy control; PANSS, Positive and Negative Syndrome Scale; PSZ, people with schizophrenia; RT, reaction time; SANS, Scale for Assessment of Negative Symptoms; SAPS, Scale for Assessment of Positive Symptoms; SST, stop-signal task; WIV, within-individual variability

5. The Neural Correlates of WIV in Schizophrenia

It has been suggested that disruptions in brain white matter underlie increased WIV in PSZ. White matter tracts that have been associated with elevated across-task WIV in the disorder include the cingulum bundle, inferior frontal fasciculus (Roalf, Ruparel, et al., 2013), and corpus callosum (Ahn et al., 2019). Roalf and colleagues (2013) found that fractional anisotropy was reduced in bilateral frontal, temporal, and occipital white matter in PSZ compared to healthy controls but that in these regions, only fractional anisotropy in the cingulum bundle and inferior frontal fasciculus was associated with WIV.

Abnormalities of cingulum bundle white matter have been associated with impaired performance in the cognitive domains of attention (Nestor et al., 2007), executive function (Tyburski et al., 2020) and visual memory (Nestor et al., 2008) whereas disruptions of the inferior frontal fasciculus are associated with impaired semantic processing (Duffau et al., 2005; Moritz-Gasser, Herbet, & Duffau, 2013) and emotion processing (Philippi, Mehta, Grabowski, Adolphs, & Rudrauf, 2009). While it is possible that disrupted white matter integrity in the cingulum bundle and inferior frontal fasciculus is responsible for increased WIV in PSZ, further studies are required to determine if this relationship is causal as the observed white matter changes may be a consequence of structural abnormalities in other regions of the brain. Although Roalf et al. (2013) observed reduced fractional anisotropy in the corpus callosum in PSZ compared to healthy controls, they did not find an association between the corpus callosum and WIV. A study by Ahn et al. (2019) focused on fractional anisotropy in the corpus callosum in PSZ and found that reduced fractional anisotropy in the genu of the corpus callosum was associated with elevated WIV. The genu of the corpus callosum is involved in the interhemispheric transfer and integration of information between the anterior cerebral hemispheres and prefrontal cortex (Anstey et al., 2007). Ahn et al (2017) hypothesized that abnormalities of this region may result in abnormal performance on neurocognitive tasks that require prefrontal input, thus increasing variability in performance across neurocognitive domains.

Electroencephalography and functional magnetic resonance imaging have been used to study the relationship between WIV in cognitive performance and neural activity. Decreased activation of the dorsolateral prefrontal cortex has been associated with periods of increased across-trial WIV in reaction time in PSZ (Fassbender, Scangos, Lesh, & Carter, 2014; Panagiotaropoulou et al., 2019). Hypoactivity of the dorsolateral prefrontal cortex has been linked to deficits in inhibitory control in PSZ (Rubia et al., 2001; Yoon et al., 2008), and it has been suggested that the associated reduction in cognitive stability is indexed by an increase in WIV. To better understand the abnormalities in cognitive control in PSZ, studies have explored the relationship between WIV in cognitive performance and event-related oscillations in the EEG spectra. Chidharom et al. (2021) observed that impairment of frontal midline theta phase coherence was associated with periods of increased across-trial WIV in reaction time in a Go/NoGo task in PSZ. These results are consistent with findings of lower theta inter-trial coherence during periods of increased WIV in reaction time in people with ADHD and healthy controls (Groom et al., 2010; Papenberg, Hämmerer, Müller, Lindenberger, & Li, 2013). Previous research suggests that theta band activity is a neural correlate of cognitive control processes mediated by the medial frontal cortex (Cohen, Ridderinkhof, Haupt, Elger, & Fell, 2008; Wang, Ulbert, Schomer, Marinkovic, & Halgren, 2005). Theta band phase inter-trial coherence is hypothesized to represent the engagement of task-relevant brain regions by the medial frontal cortex (Cavanagh, Cohen, & Allen, 2009). Thus, it is possible that one mechanism underlying increased WIV in PSZ is impaired engagement of the cognitive control network. Further, Kang et al. (2019) used event-related potentials to link across-trial WIV in reaction time to impaired cognitive control in context processing in PSZ. This study provided evidence that increased across-trial reaction time variability is related to deficits in pre-frontal motor coordination for reactive control in PSZ. Additionally, it was found that WIV was increased during trials with lower cognitive demands. Kang et al. (2019) hypothesize that WIV in PSZ is not only due to deficits in prefrontal motor control but that PSZ fail to suppress default mode network activity during easier trials, resulting in a further increase in variability.

Research relating WIV to changes in neurotransmitters and activity across functional networks has been conducted in healthy populations but, to our knowledge, no studies have examined these neural correlates of WIV in PSZ. Altered dopaminergic and cholinergic activity in the brain has been associated with increased behavioural WIV (MacDonald, Cervenka, Farde, Nyberg, & Bäckman, 2009; MacDonald, Li, et al., 2009). Although altered dopaminergic activity is present in schizophrenia, no research has directly explored the relationship between dopaminergic neurotransmission and WIV in cognitive performance in this population. Similarly, impaired regulation and coordination of the default mode and task-positive networks have been associated with an increase in behavioural WIV (Kelly, Uddin, Biswal, Castellanos, & Milham, 2008). Disrupted default mode network activity has been observed in PSZ, however further research is required to determine the extent to which this contributes to WIV in cognitive performance in this disorder.

6. Genetics of WIV in Schizophrenia

The genetic basis of WIV in cognitive performance remains unclear. There is evidence to suggest that WIV may vary in relation to genetic susceptibility for schizophrenia. Two large studies found that across-task WIV on a neurocognitive test battery significantly differed between PSZ, their unaffected family members, and healthy individuals (Cole et al., 2011; Roalf, Gur, et al., 2013). Both studies found that PSZ had higher WIV than their unaffected relatives, who showed increased WIV compared to healthy individuals. Contrastingly, studies comparing across-trial WIV in reaction times on a sustained attention task in PSZ, their family members and healthy individuals failed to find a difference between WIV in family members and healthy individuals (Birkett et al., 2007; Hilti et al., 2010). One possible explanation for this inconsistency is that genetic susceptibility to schizophrenia is unrelated to the deficits in attentional control indexed by reaction time variability. Future work should attempt to understand the neuropsychological domains that contribute to increased WIV in those with a genetic predisposition to schizophrenia.

To date, the genetic basis of WIV in cognitive performance has only been investigated using a candidate gene approach. A common polymorphism (Val158Met- rs4680) in the gene encoding for the enzyme, catechol-O-methyltransferase (COMT), has been associated with differences in measures of behavioural WIV; (Craddock, Owen, & O’Donovan, 2006). The Val allele has shown to be associated with a significant increase in enzymatic activity and more rapid degradation of dopamine in the prefrontal cortex (Craddock et al., 2006). It is hypothesized that the subsequent reduction in dopamine signaling results in impaired performance on frontally mediated cognitive tasks (Craddock et al., 2006). Results from studies investigating the relationship between the COMT genotype and WIV have been inconsistent. In healthy controls, increased Val loading has been associated with increased reaction time variability during a continuous performance test (Stefanis et al., 2005), and with higher reaction time variability for unfamiliar faces in a facial recognition task (Rostami et al., 2017). Contrastingly, Salunkhe et al. (2019) found that increased Met loading was associated with increased reaction time variability for a working memory task in healthy adults and Krabbendam et al. (2006) showed that the Met allele was associated with increased reaction time variability on a continuous performance test for PSZ, their first degree relatives, and healthy controls. Differences in sample characteristics as well cognitive measures may have contributed to the inconsistencies among study results. Additionally, it is likely that the effect of the COMT genotype on frontal lobe function is more nuanced than previously hypothesized (Craddock et al., 2006).

Given the limitations of the candidate gene approach and advances in molecular genetics, future research on the genetic basis of WIV in cognitive performance should consider a genome-wide approach. Research has proven that complex traits, such as cognitive ability, are generally polygenic in nature with many genes across the genome contributing to the phenotype (Davies et al., 2018). A well-powered genome-wide investigation of the genes associated with WIV in cognitive performance would allow for identification of novel genetic loci. Results from a genome-wide association study of WIV may inform our understanding of the biological pathways contributing to impaired cognitive performance in conditions such as schizophrenia.

7. Conclusions and Recommendations

In this narrative review, we have summarized the literature on the clinical, neural, and genetic correlates of WIV in cognitive performance in PSZ. WIV in PSZ may have multiple origins and has been associated with abnormal white matter integrity, functional connectivity, and dopaminergic activity in the disorder. Although WIV is non-specific to PSZ, elevations in WIV appear to be a sensitive indicator of psychopathology even when mean performance measures on cognitive tests are normal. WIV is relatively easy to measure and may be calculated from pre-existing datasets. This feature may further add to its utility as a predictor of psychosis risk in healthy populations and of functional outcomes in people with psychosis. Further, the relative ease of measurement of WIV makes it an attractive phenotype for large genome-wide association study.

Despite progress in our understanding of WIV in cognitive performance in PSZ, several key questions remain unanswered. First, the relationship between WIV and symptoms of schizophrenia is unclear. This may be due to heterogeneity of study designs and differences in measurements of WIV and symptom severity. The comparability of WIV and symptom severity measures across studies should be considered when planning future research in this area. An improved understanding of the relationship between WIV and symptom domains in PSZ may provide insight into the extent to which the neurobiological underpinnings of the symptoms of schizophrenia overlap or differ.

Second, although the neural basis of WIV has been explored using a number of brain imaging techniques, certain potential neural correlates of WIV are yet to be examined. Abnormalities in brain structural volumes are well described in PSZ (Kuo & Pogue-Geile, 2019) but the extent to which these relate to WIV in the disorder has not been studied. Similarly, disrupted functional network connectivity and abnormal dopaminergic activity are present in PSZ but their relationship to WIV in this disorder remains unexplored. Additionally, it is likely that multiple neural mechanisms contribute to increased WIV in schizophrenia and multi-modal neuro-imaging studies may be best suited to explore this interaction.

Third, evidence suggests that there may be a genetic basis for WIV in cognitive performance. Progress in molecular genetics and the formation of large genetic data consortia provide an opportunity to apply a genome-wide approach to discover genes associated with WIV. The discovery of genetic loci associated with WIV may provide further information about the biological processes contributing to impaired cognition in PSZ and ultimately, facilitate the discovery of novel therapeutic targets.

In closing, WIV may provide additional insight into cognitive function in PSZ over and above mean performance measures. Research on WIV in PSZ has the potential to heighten our understanding of the neurobiological disruptions that characterize the disorder. This knowledge may inform treatment strategies for schizophrenia as well as strategies to predict and mitigate risk of psychosis.