Spatial and non-spatial feature binding impairments in visual working memory in schizophrenia

Abstract

Working memory (WM) impairments are well recognized in schizophrenia patients (PSZ) and contribute to poor psycho-social outcomes in this population. Distinct neural networks underlay the ability to encode and recall visual and spatial information raising the possibility that profile of visual working memory performance may help pinpoint dysfunctional neural correlates in schizophrenia. This study assessed the resolution and associative aspects of visual working memory deficits in schizophrenia and whether these deficits arise during encoding or maintenance processes. A total of 60 participants (30 PSZ and 30 healthy controls) matched in age, gender and education assessed on a modified object in place (OiPT), a delayed non-match-to-sample (DNMST) and a delayed spatial estimation (DSET) task. Patients demonstrated lower accuracy than controls in binding visual features of the same object and recognizing novel objects as well as lower precision recalling the location of a memorized target. Moreover, response choice set size affected recognition accuracy more in PSZ than controls. However, delay duration affected spatial recall precisions, binding, and recognition accuracy equally in the two groups. Our results suggest that visual working memory (vWM) impairments in schizophrenia predominantly reflect spatial and non-spatial binding deficits, with largely preserved discrete feature information. Moreover, these impairments likely arise more during encoding than during maintenance. These binding deficits may reflect impaired effective neural functional connectivity observed in schizophrenia.

1. Introduction

Schizophrenia is characterized by cognitive deficits in memory, attention, and executive functions (Heinrichs and Zakzanis, 2008). Impairments of working memory and episodic memory predict disability (Green, 1996) and have been long thought to represent a core feature of schizophrenia with crucial implications for understanding the neurobiology of the disorder (Goldman-Rakic, 1994). The last 30 years have seen major advances in the understanding of the functional and neurological organization of vWM, as psychophysical and animal lesion studies led to significant revisions of WM models. Visual WM was thought to be organized in terms of a limited number of slots, with each slot used to store information about a single object (Cowan, 2001; Luck and Vogel, 1997). Accordingly, memory capacity simply reflects the number of available slots. This view was challenged by the finding that precision recalling features decreases monotonically as memory load increases (Bays and Husain, 2008; Smyrnis et al., 2005) and people can erroneously report features belonging to objects other than the target (Smyrnis et al., 2005). These findings led to the suggestion that features rather than objects are stored in vWM using finite resolution representations and that additional associative processes are required to ensure that features belonging to the same object are bound together (Smyrnis et al., 2005). On the other hand, animal lesion work has been crucial in elucidating the functionality of various processes involved in visual memory. It brought about the idea that separate processes are used to store objects and their spatial-temporal context (Gaffan, 1994; Eichenbaum et al., 2007).

Although many studies have demonstrated vWM impairments in schizophrenia, it remains unclear whether associative processes in vWM are affected. For example, conjunctive binding, namely the binding of separate visual features belonging to one object was found to be unimpaired in PSZ (Gold et al., 2010). Others have reported instead impairments in binding an object's features to its location (Burglen et al., 2004; Rizzo et al., 1996). Furthermore, no study to date has assessed comprehensively the resolution of features stores, the accuracy of associative and recognition processes in schizophrenia. Investigations of recognition and associative memory impairments in schizophrenia have been extensive and largely based on an influential conceptualization of human declarative memory (Tulving, 2002), which emphasizes the experiential aspects of recall. This hypothesis assumes two processes, one that is familiarity based and allows the recognition of previously encountered items and an associative one, which enables the re-experience of the recalled event within its specific spatial temporal context. Investigations have suggested that in PSZ and their unaffected first-degree relatives, familiarity-based processes are spared, while associative ones are impaired, based on experimental procedures which sorted trials according to self-reported recall quality (Abhishek et al., 2020; Danion et al., 1999). Others found impaired recognition in schizophrenia, using paired-associates tasks which introduce associative processes into recognition (Luck et al., 2009). Indeed, PSZ show preserved recognition when probed with individual items (Achim and Lepage, 2003; Ragland et al., 2012).

To bridge this gap, we undertook a cross-sectional study in PSZ and age matched healthy controls. We used a DSET (Katshu and d'Avossa, 2014) to probe the resolution of spatial WM, a modified OiPT (Dundon et al., 2018) to probe and compare spatial and non-spatial associative processes and a DNMST to assess recognition. These tasks minimized crosstalk between recognition and associative memory processes and thus allowed us to identify domain specific WM impairments in patients. Finally, to examine separately the effects of encoding and maintenance factors, we varied the size of the choice set in the DNMST and the duration of the delay period in both the DSET and DNMST. We expected appreciable group differences for the effect of choice set size, but not delay duration, in keeping with previous evidence suggesting that WM impairments in schizophrenia reflect principally diminished encoding of sensory data, rather than a failure to maintain information across delays (Anticevic et al., 2013; Dias et al., 2011; Hahn et al., 2010; Tek et al., 2002).

2. Material and methods

2.1. Participants

We recruited 60 participants (30 patients) for this study. Two male patients were excluded from the analysis as they did not complete all the tasks. Patients were recruited from the psychosis unit of the Department of Psychiatry at Eginition Hospital in Athens, Greece. All patients had been assessed by a consultant psychiatrist and had received a diagnosis of schizophrenia using the criteria of the International Statistical Classification of Diseases and Related Health Problems, 10th revision (ICD-10). At the time of testing all patients were treated with atypical antipsychotics and were clinically stable. Participants gave written informed consent prior to taking part in any experimental procedure. The study protocol was approved by the local Ethics Committee of Eginition Hospital. Controls were recruited from the community and the Laboratory of Cognitive Neuroscience and Sensorimotor Control, University Mental Health, Neurosciences and Precision Medicine Research Institute “COSTAS STEFANIS”, Greece. Sociodemographic data (Table 1) was gathered from all participants. Raven's progressive matrices test was used to estimate the IQ in all participants (Raven and Raven, 2003). Severity of psychopathology in patients was assessed using positive and negative syndrome scale (PANSS; Kay et al., 1989).

Table 1.

Demographic and clinical characteristics of patients and controls.

	Patients (n = 30, 10 females)	Controls (n = 30, 10 females)	t-test p
Age (years)	32 ± 8.32	28.23 ± 7	0.069
Education level (years)	14.74 ± 1.53	15.77 ± 2.84	0.1
Estimated Raven score	35.74 ± 13.05	54.53 ± 2.86	0.001
Duration of illness (years)	4.48 ± 6.36	–
Total PANSS score	74.67 ± 12.76	–
Positive Syndrome PANSS score	19.63 ± 4.72	–
Negative Syndrome PANSS score	22.67 ± 10.04	–
General Psych/logy PANSS score	32.38 ± 10.17	–
Antipsychotic Medication (mg/day)	136.64 ± 28.52	–

Note: Mean ± SD, IQ: intelligence quotient.

2.2. Procedure

Participants performed three computer-based behavioural experiments. Testing took place in a dark room, where participants sat comfortably, approximately 80 cm from an LCD laptop screen (VAIO PCG-8141 M, 18.4 in.).

In the first experiment (OiPT), we examined vWM spatial and feature binding. On each trial (Fig. 1A) participants had to remember the colour, shape and location of two abstract shapes. After a short delay, participants were cued to recall the colour of one of the two shapes, identified either by the shape outline, or by its location on the screen. Prior to testing, detailed instructions were given to the participants and they practiced over 12 trials. Participants completed two blocks of 96 trials each, including both shape and location cued recalls. Trial order was randomized, minimizing participants' ability to predict whether a shape or a location cue would follow the sample display. To verify that participants had not forgotten the instructions of the task, they had to describe the whole procedure after each block. In each case, they correctly reported that they had recalled either the probed shape colour, or the probed location colour and avoided verbalization strategies.

Fig. 1.

OiPT. (A) Trial Structure. The sample display contained two abstract shapes appeared side-to-side in the lower half of the screen, at an eccentricity of 4.25° along the main diagonal, for 2.0 s. The shapes were colored red, blue, or green. A 200 ms pattern mask, and a 2.0 s blank screen, followed the sample display. The recall screen contained three colored rectangles, 1.0° wide and 3.0° high, whose lower edges were aligned 2.5° above the screen center and spaced 9.0° apart horizontally. A bright cross (location cue) or the outline of one of the two shapes (shape cue) identified the target. The location cue appeared at the location previously occupied by the one of the two shapes. The shape cue appeared 3.0° below the screen center. Participants had to report the target colour by placing a cursor over the corresponding-colored rectangle and clicking the mouse button. The mouse clicks prompted the beginning of a new trial following a 1.0 s delay, during which the screen remained blank. (Β) Recall performance. The graphs show the group average proportion of swap errors, when then colour of the non-target shape was reported, and generic errors, when the colour absent from the sample was reported, as a function of the probe type, i.e., space vs. shape. Patients made more swap errors than controls, especially following space probes. Error bars are standard error of the mean.

In the second experiment (DSET), we examined whether spatial binding impairments reflect diminished resolution of spatial information in WM, or spatial binding impairments. Therefore, we assessed the effects of duration of memory delay on the precision of spatial recall and the proportion of binding errors. On each trial (Fig. 2A) participants had to remember both the colour and location of three colored discs. After a brief pattern mask and delay of variable duration, participants had to recall the position of one of the discs. Each participant completed two blocks of 120 trials each. Prior to testing detailed instructions were given to the participants and they practiced over 12 trials.

Fig. 2.

DSET. (A) Trial Structure. Three colored discs, 0.8° in diameter. The discs were colored red, green, or blue, and were visible for 2.0 s. A 1.0 s long pattern mask followed the sample. A central colour cue (a 0.3° wide square), which identified the target of same colour, appeared either immediately after the pattern mask, or after an additional 4.0 s interval, during which only a white central fixation square was visible. Participants had to place the cursor at the recalled target location and click the mouse to record their response and initiate the next trial. The locations of the discs included the center of the screen and the vertices of a virtual square, at an eccentricity of 6.0°. Independent 2D Gaussian displacements (SD = 0.9°) were used to jitter the position of each disc trial by trial. (B) Recall precision. The two plots show the group average standard deviations of the variable errors reporting the memorized horizonal and vertical target position, respectively. Patients were less precise than controls. (C) Proportion of swap and guess errors. The two plots show the group average proportion of swap errors and guesses, respectively, following short and long delays. Patient made more swap errors than controls, however, the proportion of guesses was comparable.

The third experiment assessed participants ability to recognise complex shapes. On each trial (Fig. 3A), participants had to remember two colored fractals, used in previous studies of visual WM (Pertzov et al., 2013), at random locations. After a pattern mask and a blank delay of variable duration, two or three fractals (one novel) appeared in the test display. The participants had to indicate the novel fractal. Each sample and recall display contained a unique combination of fractals. Sixty shapes were drawn from a library of online fractal patterns. Each participant completed two blocks of 90 trials each. The order of presentation was randomized. Prior to testing, detailed instructions were given to the participants and they completed 12 practice trials.

Fig. 3.

DNMST. (A) Trial Structure. The sample display contained colored fractals, 130 pixels in diameter. The fractals' centers were placed at least 4.0° apart and away from the center and boundaries of the screen. The sample was visible for 3.0 s and was followed by a 1.0 s long pattern mask. The recall display followed the pattern mask either immediately or after a 3.0 s long blank delay. It contained one or two of the fractals that had appeared in the sample as well as a new fractal. The participants had to indicate the novel fractal. The recall display remained visible until the participant positioned a crosshair shaped cursor at the location of the chosen pattern and clicked the mouse. (B) Identification sensitivity. Accuracies were transformed into d'. The plots show group average performance following short and long delays, respectively, as a function of choice set size. Patients were less sensitive than controls and more susceptible to choice set size effects. Delay duration did not appreciably affect performance in either group.

2.3. Analysis

In the OiPT, participants could either report the colour of the target, that is indicate the correct choice, the colour of the non-target item (swap error), or the colour not presented in the sample (generic error). In trials in which participants forgot the sample and guessed, we assumed they were equally likely to be correct, make a swap error or a generic one (see supplementary methods). The group level analysis was conducted with a mixed ANOVA. In the DSET, the analysis was performed in two steps. First a model was fitted to each participants' data (see supplementary methods). Group level analysis was then conducted on the proportion of swap and guess errors and the standard deviation of the variable errors, using mixed ANOVAs. In the DNMST, to account for the effect of choice set size on report accuracies, these were transformed into sensitivity measures. Expected accuracies as a function of d', were calculated using a simple model of choice (see supplementary methods). Finally, the severity of symptoms of the studied sample assessed using PANSS. We performed an exploratory analysis, using Pearson's correlation coefficient to assess the existing correlations.

3. Results

3.1. Sample characteristics

Table 1 summarizes the characteristics of our sample. The two groups were well matched on all demographic variables. However, they differed in Raven scores.

3.2. OiPT performance

Fig. 1B shows that patients made more swap errors than controls. The group difference was significant [F(1,56) = 36.334, p < .001, ηp² = 0.4]. While the main effect of probe dimension was not significant [F(1,56) = 1.685, p = .2, ηp² = 0.029] its interaction with group was [F(1,56) = 11.108, p = .002, ηp² = 0.17]. Patients were more likely to make a swap error following a space than shape probe, while controls showed the opposite pattern. Patients also made more generic errors than controls [F(1,56) = 22.055, p < .001, ηp² = 0.28]. Neither probe dimension [F(1,56) = 0.858, p = .358, ηp² = 0.015] nor its interaction with group significantly affected the proportion of generic errors [F(1,56) = 0.472, p = .495, ηp² = 0.008].

3.3. DSET performance

Fig. 2B shows that patients made more swap errors than controls, [F(1,56) = 14.774, p < .001, η_p² = 0.21]. The duration of the memory delay did not affect binding accuracy [F(1,56) = 0.830, p = .37, η_p² = 0.015], nor its interaction with group [F(1,56) = 1.790, p = .186, η_p² = 0.032]. However, delay had a significant effect on recall precision [F(1,56) = 37.475, p < .001, η_p² = 0.41]. There was also a significant group difference [F(1,56) = 9.879, p = .003, η_p² = 0.15]. Fig. 2C shows that patients were less precise than controls and that precision decreased with longer delays. The interaction of delay by group was not significant [F(1,56) = 2.806, p = .1, η_p² = 0.049].

3.4. DNMST performance

There was a significant group effect [F(1,56) = 31.166, p < .001, η_p² = 0.2]. Fig. 3B shows that patients were less accurate than controls. Longer delays [F(1,56) = 4.015, p < .05, η_p² = 0.008] and larger choice set size [F(1,56) = 46.045, p < .001, η_p² = 0.097] worsened identification of the novel item. There was also a significant interaction of set size by group [F(1,56) = 5.406, p < .025, η_p² = 0.011] since the size of the choice set affected performance more in the schizophrenia than the control group. Neither the interaction of delay by choice set size [F(1,56) = 0.904, p = .346, η_p² = 0.016] nor the interaction of delay by group [F(1,56) = 0.899, p = .347, η_p² = 0.016] or the three-way interaction were significant [F(1,56) = 0.894, p = .349, η_p² = 0.016].

3.5. Correlation of cognitive tasks to schizophrenia symptoms (PANSS)

Table 2 summarizes the results of the correlation analysis between schizophrenia symptoms and our cognitive tasks. The number of binding errors, as well as patients' precision in DSET are positively correlated to PANSS positive subscale (r = 0.512, p = .012 and r = 0.457, p = .028, respectively), as well as PANSS total score (r = 0.424, p = .044 and r = 0.487, p = .018, respectively). Moreover, the number of binding errors following a shape probe in OiPT is negatively correlated to PANSS total score (r = −0.436, p = .038).

Table 2.

Correlation of cognitive tasks to schizophrenia symptoms (PANSS).

Tasks	PANSS positive	PANSS negative	PANSS total
OiPT (binding shape errors)	r = −0.023, p = .917	r = −0.187, p = .394	r = −0.436, p = .038
OiPT (binding location errors)	r = − 0.181, p = .409	r = 0.087, p = .694	r = − 0.358, p = .094
DSET (binding errors)	r = 0.512, p = .012	r = 0.228, p = .294	r = 0.424, p = .044
DSET (precision)	r = 0.457, p = .028	r = 0.301, p = .163	r = 0.487, p = .018
DNMST (accuracy)	r = 0.147, p = .503	r = −0.106, p = .630	r = 0.325, p = .130

Note: bold indicates significant values at p < 0.05.

4. Discussion

This study used three tasks designed to probe respectively WM spatial resolution, conjunctive binding and object recognition. Patients' performance demonstrated impairments that generalized across WM domains, suggesting that multiple neural processes are affected in this group. In the following discussion we overview the specific findings considering animal and human literature on the functional anatomy of vWM.

In the OiPT, patients demonstrated a profound impairment using spatial and to a lesser extent shape probes, to recall colour information. Deficits in binding location and colour were also evident in the DSET, suggesting that spatial binding processes are prominently impaired in PSZ, regardless of whether location was the probe or recall dimension (Dundon et al., 2018). Previous studies have provided contrasting findings regarding binding impairments in schizophrenia. Namely, one study compared change detection performance when participants had to decide whether the test display contained a change in either colour, orientation or both (Gold et al., 2003). Having found that group differences did not vary between these conditions, these authors concluded that binding of colour and orientation was not affected. However, the task did not evaluate binding, but the ability to store independently features belonging to different visual dimensions. Others reported that compared to controls patients were less accurate detecting swaps in the position of uniquely identifiable objects than changes in either object identity or position (Burglen et al., 2004). They concluded that binding processes are prominently impaired in keeping with our own findings.

Could the spatial binding impairments, documented in our study, simply reflect diminished precision recalling the target colour or location, rather than an impairment maintaining their conjunction? We think this is unlikely. In the OiPT memorizing the target location placed minimal demands on spatial WM, since the sample contained the same two positions in each trial. Similarly, in the DSET the same three distinctive colours were presented in each trial, thus minimizing the colour memory load. Finally, a previous study of WM impairments in a stroke patient, found that while recall precision was affected by delay duration, swap error rates were not (Dundon et al., 2018).

Recall precision bears a monotonic power law relation to memory load (Bays and Husain, 2008; Smyrnis et al., 2005) establishing precision as a proxy of a limited, continuous resource which determines the visual WM resolution of items sharing it. In this study patients showed decreased precision recalling the target position in the DSET, implying diminished spatial WM resolution in schizophrenia (Carter et al., 1996). Although individuals with schizophrenia were found to have undiminished precision when recalling basic visual dimensions, such as colour (Gold et al., 2010), more recent evidence has demonstrated diminished visual WM precision both in individuals with more schizotypal features and schizophrenia (Xie et al., 2018). This finding could be accounted by greater within-subject variability in the allocation of a limited resource for patients than controls (Zhao et al., 2021). Our study demonstrates that diminished precision and increased proportion of swap errors jointly affect recall performance of PSZ in the same task suggesting simultaneous, but distinct impairments of feature resolution and feature binding in WM.

We did not model within subject variability in precision, namely the possibility that precision varies independently both between items and trials. Hence, we cannot determine the extent to which group difference in precision may reflect differences in precision variability instead. Work in healthy participants indicates that variability in precision is introduced by unequal additive noise during maintenance of item level information (Fougnie et al., 2012) and unequal resource sharing between items competing for memory representation (Zhao et al., 2021). However, the observation that delay duration does not further modify group differences in recall precision is inconsistent with the proposal that patients with schizophrenia experience greater variability in temporal decay of item memory. Furthermore, the finding that recall accuracy and precision is lower even when a single item is memorized (Carter et al., 1996; Zhao et al., 2021) suggests that greater inhomogeneities in resource sharing among items cannot account for precision differences on its own.

Accuracy in the DNMST was appreciably worse in patients than controls. Furthermore, choice set size disproportionally affected performance in patients compared to controls. In animal studies, the choice set size effect has been interpreted as indicative of an encoding impairment, since the same mechanism that allows the encoding of visual data in memory, must be recruited to match perceptual representations of the choice set to the memorized sample (Eacott et al., 1994). Finally, group differences in precision were not further modulated by the duration of the delay period. Others found that PSZ exhibit lower precision than healthy controls recalling a target's location. Moreover, the standard deviation of spatial estimates increased linearly with delay duration in both controls and PSZ, but at a greater rate in the latter group (Gold et al., 2020; Starc et al., 2017). Our data also suggest that delays introduce noise in spatial estimates, but at a rate that is equal in controls and PSZ, when normalised by the shortest delay error magnitude, as indicated by parallel changes in error size when plotted in logarithmic coordinates (see Fig. 2B). This finding is most consistent with the idea that the two groups have different spatial WM resolutions, but comparable efficiencies across delays.

We also found impairments in object recognition in PSZ. A straightforward, if speculative, proposal is that diminished performance in the DNMS task and possibly shape to colour binding in the OiPT task reflects diminished functionality of an associative mechanism involved in object recognition, likely to rely on the functionality of perirhinal cortex (Bussey et al., 2002; Erez et al., 2016). If so, then PSZ may suffer from a comprehensive disruption of parallel memory circuits in the ventral occipito-temporal cortex affecting both spatial and object related processing.

Finally, the exploratory correlation analysis revealed a correlation between some domains (i.e., binding and precision) of our cognitive tasks and PANSS positive subscale. Severity of negative symptoms was uncorrelated with the measures examined. Treatment strategies, which overcome the severity of symptoms in schizophrenia, could improve cognitive impairment. We hope that future studies will find these strategies and ameliorate the quality of life of PSZ.

5. Conclusions

In conclusion, our findings are consistent with the idea that several WM systems are disturbed in schizophrenia, especially when the tasks require to bind objects, their features and locations. Moreover, these deficits likely arise during encoding rather than maintenance. Impaired function of fronto-temporal diencephalic structures may account for WM impairments in schizophrenia. Future studies should address how to disparate fronto-temporal circuits are all affected in schizophrenia.

CRediT authorship contribution statement

Antigoni Belekou: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Writing – original draft. Mohammad Zia Ul Haq Katshu: Conceptualization, Formal analysis, Methodology, Software, Writing – original draft, Writing – review & editing. Neil Michael Dundon: Conceptualization, Data curation, Formal analysis, Methodology, Software, Visualization, Writing – review & editing. Giovanni d'Avossa: Conceptualization, Data curation, Formal analysis, Methodology, Software, Supervision, Validation, Writing – original draft, Writing – review & editing. Nikolaos Smyrnis: Conceptualization, Funding acquisition, Project administration, Resources, Supervision, Validation, Writing – review & editing.

Declaration of competing interest

The authors declare no conflict of interest.

Supplementary Methods

Supplementary Methods