Sustained attention deficits in schizophrenia: Effect of memory load on the Identical Pairs Continuous Performance Test

Abstract

Background

Sustained attention and vigilance impairments are well documented in people with schizophrenia (PSZ). The processes implicated in this impairment remain unclear. Here we investigated whether vigilance performance varied as a function of working memory load, and also examined the role of attentional lapsing that might arise from a loss of task set resulting in mind wandering.

Method

We examined Continuous Performance Test Identical Pairs (CPT-IP) data from a cumulative sample of 247 (PSZ) and 238 healthy control (HC) participants collected over a series of studies.

Results

PSZ performed more poorly that HC across conditions with signal/noise discrimination (d′) decreasing with increasing working memory load across both groups However, there was a significant interaction of group and load suggesting that performance of PSZ was more negatively impacted by increasing load. We also found that PSZ has a significantly higher rate of attention lapsing than did HC.

Discussion

Our results suggest that difficulties maintaining task set and working memory limitations are implicated in the impairments observed on the Identical Pairs CPT. Difficulties with task set maintenance appear to explain the majority of between-group variance, with a more subtle impact of increasing working memory load.

1. Introduction

Impairments of the ability to sustain attention on an ongoing task and resist distraction have been considered central features of schizophrenia beginning with the first clinical descriptions of the disorder by Kraepelin (1919) who wrote that patients “lose both inclination and ability on their own initiative to keep their attention fixed for any length of time.” Variants of The Continuous Performance Test (CPT) have been used widely in both basic and clinical research to quantify the ability to sustain attention (Fortenbaugh et al., 2017). While different versions of the CPT stress additional cognitive processes, the foundational concept of the CPT is to ask a subject to identify infrequent target stimuli from a continuous series of nontarget stimuli (Rosvold et al., 1956). Task difficulty can be manipulated by increasing the rate of stimulus presentation, decreasing the perceptual discriminability of the stimuli, or by imposing a working memory load through the use of compound sequential targets (i.e. an X following an A) (Parasuraman and Davies, 1977). In the schizophrenia literature, the two most common versions of the task impose either a perceptual load the (degraded stimulus version of the CPT developed by Nuechterlein et al. (2015)) or a working memory (WM) load (Identical Pairs CPT developed by Cornblatt et al. (1988)).

In the Identical Pairs CPT (CPT-IP), subjects are asked to respond to targets that are defined as precise repetitions on consecutive trials. In the version of the task implemented in the Matrics Consensus Cognitive Battery (Nuechterlein and Green, 2002), these targets involve two, three, or four digit numbers which are presented in separate blocks, thereby imposing increasing WM load levels across blocks. Due to the nature of the identical pairs paradigm, targets are changing with each successive stimulus, thereby requiring the subject to continuously update target information. Thus, the task requires both the ability to sustain attention and the ability to update working memory at different levels of WM load. Both the rate of correct target detections and errors on distractor trials (i.e., false alarms) are quantified using signal detection theory to calculate a d′ score as an indicator of overall performance. As noted by Lenzenweger et al. (1991), the d prime measure does not assess if the ability to sustain attention diminishes with increasing time on task. Instead, it provides a measure of the overall target detection efficiency over the entire task.

The memory load manipulation is of particular interest in schizophrenia in light of the fact that working memory is considered to be a central deficit in schizophrenia (Park and Holzman, 1992; Gold and Luck, 2022). Consequently, it would be expected that patient performance would increasingly deviate from that of controls as memory load increases from two to three to four digits if this level of load strains or exceeds the level of available WM capacity. Somewhat surprisingly, this has not been carefully addressed in the literature. However, studies that have reported looking at the impact of load with repeated measures ANOVAS have reported robust main effects of load and diagnostic group, but failed to report group by load interactions effects that might reasonably be expected (Kahn et al., 2012; Rapisarda et al., 2014).The Kahn study may have been underpowered to detect such an interaction effect. However, the same cannot be said of the Rapisarda et al. that had an N of 654 PSZ and 1011 HCs.

This issue is important to understand because CPT IP deficits may arise for different reasons. For example, if PSZ have difficulty maintaining attention, they should be prone to momentary lapses of attention, defined as a brief shift of attention away from the task at hand. Such lapsing is often termed as mind wandering in the normal literature to describe a shift of attention to self-generated information such as internal thoughts or memories, shifts that lead to performance errors on ongoing external tasks (Mooneyham and Schooler, 2013). In principle, vulnerability to attentional lapses could be independent of the difficulty of trial type/increasing load. If that is the origin of the deficit in PSZ, we would expect to see similar levels of impairment relative to controls at each of the three WM load levels. In contrast, if performance is limited by WM capacity limitations or slowing in the rate of encoding items into WM, we would expect to see the performance of PSZ to increasingly deviate from that of controls as WM load increases. Note, these possibilities are not mutually exclusive: a general vulnerability to lapsing of attention might also be combined with WM impairment which would lead to impairment being observed at the lowest load as well as increased level of impairment at the highest load level. Our goal was to examine this question in a sample that was offered sufficient power to detect a group by load interaction effect.

2. Methods

2.1. Participants

Demographic information is provided in Table 1. Data from a total of 518 subjects (patients with schizophrenia, PSZ = 251, healthy control subjects, HCS = 267) was collected. These subjects were previously recruited as a part of various studies through the Outpatient Research Program at the Maryland Psychiatric Research Center over a period of 20 years (between 2001 and 2021). PSZ were evaluated during a period of clinical stability as evidenced by no changes in medication type or dosage for a period greater than or equal to four weeks. Consensus diagnosis was established via a best-estimate approach based upon detailed psychiatric history and multiple interviews. This diagnosis was subsequently confirmed using the Structured Clinical Interview for DSM-IV (SCID) (First, 1995). In PSZ, symptom assessments included the Brief Psychiatric Rating Scale (BPRS, Overall and Gorham, 1962). Age-, gender-, and education-matched healthy control subjects (HCS) with no history of psychiatric or substance abuse disorder and no first-degree relative with mental illness were recruited by internet advertisements and flyers posted in local libraries and businesses. HCs were screened using the Structured Clinical Interview for DSM IV diagnosis and Structured Clinical Interview for DSM IV Personality Disorders (First, 1995; Guze, 1995). All participants provided informed consent for a protocol approved by the University of Maryland Institutional Review Board.

Table 1.

Participant characteristics.

	HCS (N = 247)	PSZ (N = 238)	Statistic	p value
Age	37.10 (11.17)	37.39 (10.63)	t = −0.30	0.77
Gender (M | F)	152 | 95	158 | 80	φ = 1.24	0.27
Race (African American | Caucasian | Other)	95 | 138 | 8	95 | 124 | 19	φ = 5.21	0.07
Participant education	15.18 (2.11)	13.00 (2.24)	t = −11.06	<0.001
Antipsychotic medication
Total CPZ		587.67(498.65)
Clinical ratings
BPRS positive		2.25 (1.21)
BPRS negative		1.82 (0.73)
PRS disorganization		1.32 (0.39)
BPRS total		35.48 (8.54)

2.2. Stimuli, task and procedure

Sustained attention was measured using the version of the IP CPT that was included in the MATRICS battery (Nuechterlein and Green, 2002).Participants viewed the stimuli at a distance of 100 cm from a computer monitor. Each stimulus (either 2, 3 or 4 digits) (0 to 9) was presented for 50 ms, followed by an inter-stimulus interval of 950 ms, resulting in a total trial time of 1000 ms. Subjects were instructed to respond as quickly and as accurately as possible with a mouse click when they saw two consecutive identical numbers. 20 % target pairs and an equal percentage of “false alarm” pairs – i.e. catch trials – were presented. In these catch trials, one, two, or three digits were repeated on consecutive trials (in the two, three, and four digit conditions). On the other 60 % of trials (termed random trials), none of the digits repeated on consecutive trials. False alarm errors on these random trials would suggest a lapse of attention had occurred. Participants did not have any information regarding the proportions of trial stimuli. Each load condition (2-,3- or 4-digits) included 150 trials, with subjects completing 10 practice trials prior to task initiation, repeating practice trials if necessary. Responses were recorded as hits (responses to stimuli that matched the prior trial), false alarms (a response to a stimulus differing by only one digit from the immediately preceding stimulus), or random responses (response where there was no overlap between the stimuli across consecutive trials). The primary outcome measure, d′, assessed the ability of the participant to discriminate between target signals and noise distractors. Note, it is standard on the IP CPT that the calculation of d′ scores is done only using the rates of hits and false alarms, excluding the rate of random responses from the calculation.

Examples of trials are shown in Fig. 1.

Fig. 1.

Task Schematic. Trial layout for the Continuous Performance Test-Identical Pairs (4-digit condition shown).The CPT-IP requires responding on trials when the same number is presented consecutively (target trials); and response inhibition on all other trials (When subjects respond on these, they are considered to be Random responses and False alarms).

2.3. Analysis

2.3.1. Screening for valid CPT results

To exclude outliers, we excluded subjects whose average d′ across the three conditions was greater than two standard deviations below the group mean. Ultimately, 7 PSZ (2.87 %) and 10 (3.89 %) HCS participants were excluded from analysis, resulting in data from 244 PSZ and 257 HCS.

2.3.2. Statistical analysis

Statistical analyses were performed using MATLAB and JASP software (Version 0.8.1.1., JASP Stats). The alpha level was set at 0.05. The Hits, False alarms, Lapses and d′ measures for the individual participants were analyzed in a two-way ANOVA with factors of group (PSZ vs. HCS) and condition (3 levels, one each for 2-,3- and 4 digits), with the Greenhouse-Geisser epsilon correction for nonsphericity. To examine the robustness of our results, we created three independent subsamples of 79, 79, and 77 PSZ and 81, 81, and 82 HCs and did ANOVAs to see how often we were able to replicate the pattern of results seen in the total sample in the reduced samples of a size that is fairly typical in the experimental psychopathology literature.

3. Results

3.1. CPT deficits

Table 2 contains the statistical test results for each of the individual task variables across conditions, including p values and effect sizes. Consequently, these statistics will not be discussed in the text. In the 2,3, and 4 digit conditions HCs had significantly more hits (correct target responses) and had significantly fewer false alarms and random responses than did PSZ. In both groups, the hit rate significantly decreased while the rate of false alarms and random responses significantly increased across increasing levels of WM load.

Table 2.

Neurocognitive and CPT task measures.

	HCS (N = 247)	PSZ (N = 238)	Statistic	p value	Cohen's d
Neurocognitive test results
WTAR4	109.93 (12.13)	96.71 (17.00)	t = 9.82	<0.001	0.90
WASI-II	113.17 (11.67)	96.43 (14.51)	t = 13.96	<0.001	1.27
MD processing speed	53.26 (7.21)	40.35 (15.05)	t = 17.61	<0.001	1.61
MD attention vigilance	53.73 (8.14)	41.54 (12.49)	t = 14.12	<0.001	1.29
MD working memory	53.16(8.64)	4137 (10.24)	t = 14.71	<0.001	1.34
MD verbal learning	50.22 (12.37)	38.43 (8.04)	t = 13.72	<0.001	1.25
MD visual learning	45.22 (11.19)	37.15 (11.39)	t = 11.10	<0.001	1.01
MD reasoning	49.96 (8.71)	47.11 (11.97)	t = 10.10	<0.001	0.92
MD social cognition	54.48 (8.69)	42.00 (12.25)	t = 14.31	<0.001	1.31
MCCB composite	52.53 (7.47)	36.21 (12.12)	t = 20.36	<0.001	1.87
CPT variables
Hits 2 digit	29.02 (1.69)	26.31 (3.97)	t = 9.72	<0.001	0.89
False alarm 2 digit	0.51 (0.79)	1.10 (1.67)	t = −4.91	<0.001	−0.45
Randoms 2 digit	0.29 (0.78)	1.60 (12.49)	t = −7.03	<0.001	−0.64
Hits 3 digit	27.19(3.28)	22.79 (5.41)	t = 10.77	<0.001	0.98
False alarm 3 digit	1.24 (1.52)	2.52 (2.42)	t = −6.99	<0.001	−0.64
Randoms 3 digit	0.20 (0.49)	1.65 (2.52)	t = −8.70	<0.001	−0.80
Hits 4 digit	24.01 (4.50)	18.73 (5.89)	t = 11.07	<0.001	1.01
False alarm 4 digit	4.29 (3.34)	6.47 (4.12)	t = −6.37	<0.001	−0.58
Randoms 4 digit	0.78 (2.36)	2.90 (3.75)	t = −7.41	<0.001	0.68

Values are mean (SD).

HCS, healthy control subjects; PSZ, people with schizophrenia spectrum disorders; WTAR4, Wechsler Test of Adult Reading Fourth Edition; WASI-II, Wechsler Abbreviated Scale of Intelligence Second Edition; MD, MCCB (MATRICS Consensus Cognitive Battery) cognitive domain; MCCB, MATRICS Consensus Cognitive Battery composite total.

Our main measure of interest was the d-prime scores. As displayed in Fig. 2A, we observed, expectedly, that d′ scores decreased as CPT load increased. These data were submitted to a repeated measures ANOVA with one between-group factor (PSZ vs. HCs) and one repeated factor of load (2, 3, 4 digit conditions). Statistics for this analysis are reported in Table 3. The ANOVA yielded significant main effects of load (worse performance at higher loads than lower loads) and group (PSZ performed more poorly than HCs) as well as a significant group by load interaction effect. The difference scores between loads are displayed in Fig. 2B. Post-hoc tests revealed that PSZ had a larger decline in performance than did HC as load increased from 2 to 3 digits, and from 2 to 4 digits. Somewhat surprisingly, the difference scores between performances with 3 versus 4 digits were very similar across groups as seen in Fig. 2B.

Fig. 2.

A. D-Prime scores. Raincloud plot depicting average d′ scores by 2-, 3-, and 4-digit load, per participant (single dots), and within group. HCS are in blue, PSZ are in red. Each line indicates average d′ within group at each trial type.

B. d′ Difference Scores. Box and whisker plot depicting distribution in data of average d′ difference scores by load difference displayed by group, per participant (single dots). 3–4 load difference was nonsignificant. Median difference scores by group indicated by box. HCs are displayed in blue, PSZ displayed in red.

C. Lapses-d′ Association Correlations depicting average d′ scores across response types. Line of best fit is displayed for both groups. HCS are in blue, PSZ are in red. The greatest variability was within the FA condition.

Table 3.

Statistics.

	F	p	η2p
Hits
Load	542.73	<0.001	0.529
Group	160.99	<0.001	0.250
Group × Load	23.12	<0.001	0.046
False alarms
Load	601.72	<0.001	0.555
Group	62.80	<0.001	0.115
Group × Load	16.67	<0.001	0.033
Random responses (lapses)
Load	43.97	<0.001	0.083
Group	86.99	<0.001	0.015
Group × Load	7.574	0.151	0.004
d-Prime
Load	1386.12	<0.001	0.742
Group	202.96	<0.001	0.296
Group × Load	10.25	<0.001	0.021
d′ difference score between loads
Load	163.93	<0.001	0.254
Group	10.06	0.002	0.020
Group × Load	6.260	0.413	0.002

Posthoc t-tests
	t	pbonf	Cohen's d	Confidence interval for effect size
2–3	−3.583	<0.001	−0.426	Lower: −0.606 Upper: −0.246
3–4	0.128	1.000	0.045	Lower: −0.133 Upper: 0.223
2–4	−3.454	0.004	−0.304	Lower: −0.483 Upper: −0.125

We examined the correlation between d′ averaged across all conditions with the number of random responses which are thought to reflect lapses of attention. As displayed in Fig. 2C, the Spearman's rhos were significant in both groups, with a more robust relationship observed in the patient group. Thus, people who make more random responses have generally decreased discriminability.

As noted above, prior studies have not observed significant group by load interaction effects. To explore the robustness of our finding, we randomly divided our total sample into 3 groups of PSZ and 3 groups of HCs (78–82 PSZ and HCs for each of the 3 subsamples), and ran the repeated measures ANOVAs in each group separately. In all 3 subsamples, we observed robust main effects of group (HCs > PSZ) and load (worse performance at higher loads). However, the group by load interaction was only significant (p < .001) in one of the three subsamples (p's of 0.06 and 0.3 in the other two groups). These results are a striking demonstration of the importance of sample size in the detection of interaction effects.

3.2. Additional analyses

We also examined the Pearson correlation of average d prime and average number of random responses with BPRS total and factor scores (McMahon et al., 2002): none were significant (all r's > 0.13). We also examined the correlation of MCCB total scores with the average d prime score and the average number of random responses (r's = −0.51 and 0.70 respectively in PSZ; r's −0.55 and −0.27 respectively in controls), suggesting that the two IP CPT measures are similarly related to general cognitive ability.

4. Discussion

Leveraging the advantage of a large sample size, we were able to clarify the nature of impairment seen on the IP-CPT in PSZ. In essence, two different types of impairment can be distinguished. First, PSZ have difficulty maintaining task goal representations, leading to impairments in detecting targets as well as making more false alarm and random responses at each load level. Second, this impairment is amplified at the highest level of working memory load. The magnitude of this second impairment is quite modest in comparison to the first (the n²_p for the effect of group was 0.296 while that for the interaction was 0.021). The nature of the load-related impairment in PSZ is somewhat puzzling as we did not see a between group difference when comparing performance at load 3 versus load 4, but did see the effect when going from load 2 to 3 and from load 2 to 4. In essence, the effect is detectable only in comparison to performance at load 2, perhaps reflecting that higher levels of variability in performance at loads 3 and 4 relative to 2 which may increase noise in the calculation of the difference scores.

In the IP-CPT literature it has been a convention to exclude random responses from the calculation of the d′ scores based on the idea that these errors might reflect a different process than indexed by the hit rate and false alarm rate. The data shown in Fig. 2C can be seen as a challenge to that view given the robust correlations seen in both groups. That is, the tendency to make random responses appears to be quite related to overall signal detection performance and it is likely that this relationship is more robust in PSZ because they make many more of these responses than do HCS, making it easier to detect the bivariate relationship. It seems likely mind wandering may be a matter of degree from being fully focused on task-irrelevant thoughts to dividing attention between the task and other thoughts. Random responses may occur when attention has wandered further from the task than a false alarm that contains at least one target relevant feature. However, there is a robust correlation between the tendency to make both sorts of errors, and both implicate a failure in working memory and cognitive control.

It is also interesting to note that in both HCS and PSZ, the rate of making random responses is very similar at load 2 and load 3, with a marked increase at load 4. Thus, mind wandering may not be the result of some kind of random intrusive thought. Instead, it appears that increasing task difficulty may elicit higher rates of random responses either because some subjects simply get overwhelmed by the task demands while others might withdraw from the task because it demands a degree of effort that is difficult to sustain.

One major limitation of the study is the absence of a psychiatric control group. Thus, the impairments observed here may not be specific to schizophrenia. Further transdiagnostic research is needed to address this important question. The fact that IP CPT deficits are seen in populations at risk for SZ suggest that impairment is not tied to manifest illness, but do not establish specificity (Lenzenweger et al., 1991; Cornblatt and Malhotra, 2001).

In summary, the IP-CPT impairment in schizophrenia appears to reflect both working memory load independent and dependent processes. This impairment is independent of BPRS symptom severity. PSZ show robustly lower target detection performance across all load levels and an increased level of attentional lapses across all load levels. The target detection impairment is slightly amplified at the highest level of working memory load, suggesting a role for working memory capacity limitations in the most demanding task condition. While increased load might enhance impairment, it is important to emphasize that PSZ show reduced target detection and increased mind wandering at the lowest level of load that was tested. It appears that this deficit in the ability to maintain and use a task goal representation is the major driver of the IP CPT impairment seen in people with schizophrenia.

CRediT authorship contribution statement

JD: Data curation, formal analysis, original draft, review & editing; SB: Conceptualization, formal analysis, data visualization, review, and editing; BR, Data acquisition, data curation, review; JG Conceptualization, funding acquisition, review & editing.

Funding source

This work was supported by NIMH R01 MH065034. The study sponsors had no role in study design, analysis, interpretation, writing of the manuscript, or in the decision to submit the manuscript for publication.

Declaration of competing interest

None of the authors have any conflicts to report in relationship to the data presented in this paper.