Retinotopic coding organizes the opponent dynamic between internally and externally oriented brain networks

Abstract

How the human brain integrates internally- (i.e., mnemonic) and externally-oriented (i.e., perceptual) information is a long-standing puzzle in neuroscience. In particular, the internally-oriented networks like the default network (DN) and externally-oriented dorsal attention networks (dATNs) are thought to be globally competitive, which implies DN disengagement during cognitive states that drive the dATNs and vice versa. If these networks are globally opposed, how is internal and external information integrated across these networks? Here, using precision neuroimaging methods, we show that these internal/external networks are not as dissociated as traditionally thought. Using densely sampled high-resolution fMRI data, we defined individualized whole-brain networks from participants at rest, and the retinotopic preferences of individual voxels within these networks during an independent visual mapping task. We show that while the overall network activity between the DN and dATN is opponent at rest, a latent retinotopic code structures this global opponency. Specifically, the anti-correlation (i.e., global opponency) between the DN and dATN at rest is structured at the voxel-level by each voxel’s retinotopic preferences, such that the spontaneous activity of voxels preferring similar visual field locations are more anti-correlated than those that prefer different visual field locations. Further, this retinotopic scaffold integrates with the domain-specific preferences of subregions within these networks, enabling efficient, parallel processing of retinotopic and domain-specific information. Thus, DN and dATN dynamics are opponent, but not competitive: voxel-scale anti-correlation between these networks preserves and encodes information in the negative BOLD responses, even in the absence of visual input or task demands. These findings suggest that retinotopic coding may serve as a fundamental organizing principle for brain-wide communication, providing a new framework for understanding how the brain balances and integrates internal cognition with external perception.

A fundamental goal of neuroscience is to understand how activity distributed across the brain’s functional networks gives rise to cognition^1–8. Central to this aim is understanding principles that govern interactions between different brain networks, particularly those involved in externally-oriented attention (e.g., processing sensory input) and internally-oriented attention (e.g., introspection and memory)^9–12. This knowledge gap confounds our understanding of human cognition: the interaction between internally- and externally-oriented neural systems is foundational to how we perceive, remember, and navigate our world, yet the ‘common language’ facilitating their communication remains a mystery.

One reason this knowledge gap exists is that the neural systems that subserve internally- and externally-oriented attention, such as the Default Network (DN) and dorsal attention network (dATN) respectively, are typically considered in competition^9–11. Seminal neuroimaging studies investigating externally-oriented attention (visual processing, working memory, etc.) showed that visual tasks reliably activate brain areas in lateral occipital temporal cortex, dorsal parietal cortex, and prefrontal cortex, now collectively referred to as the dATN^9,13,14. In contrast, visual tasks reliably deactivate regions in the internally-oriented DN, including lateral and medial parietal cortex, anterior temporal lobe, and medial prefrontal cortex^12,15. This pattern reverses during introspective tasks, e.g., scene construction or theory of mind tasks: the DN systematically activates and the dATN deactivates^6,16–20. Together, this is thought to reflect a network-level opponency: these networks are thought to globally inhibit each other, possibly to facilitate attending to external stimuli without competition from internal mnemonic representations, and vice versa^9,11,21,22. However, if internally- and externally-oriented networks compete, it is not clear how the brain accomplishes tasks that require integrating perceptual and mnemonic information (e.g., anticipatory saccades, memory-based attention tasks, and mental imagery).

Two recent findings have shed light on this question. First, while the internally-oriented DN is traditionally thought to use an abstract or semantic neural code^2,22–24, recent work suggests that a neural code that is typically associated with externally-oriented processing -- retinotopy -- also manifests in the DN^25–28. Crucially, compared to classic visually responsive areas, including the posterior portion of the dATN, the visual response of DN is an “inverted retinotopic code.” Specifically, while stimulation of the retina causes typical visual areas to increase neural activity in a position-dependent manner, the DN exhibits position-specific decreases in activity^25,27,28. Thus, the DN’s deactivation reflects specific properties of the attended external stimulus and is informative during visual attention tasks.

Second, an inverted retinotopic code has recently been shown to play a functional role in structuring mnemonic-perceptual interactions at the cortical apex²⁸. During familiar scene processing, activity in brain areas specialized for scene perception and memory differentially increases during perception and recall tasks, respectively. However, at the voxel level, these areas exhibit an interlocked retinotopically-specific opponent dynamic. In other words, stimuli in a specific visual field location activate perception voxels and suppress memory area voxels monitoring that location. This pattern reverses during memory tasks. This challenges the traditional view of internally- and externally-oriented brain networks as functionally-opposed via global opponent dynamics, suggesting instead that they are mutually engaged in a common information processing stream. It also emphasizes the importance of voxel-wise activity patterns in uncovering neural codes that underpin global opponent dynamics.

Based on these findings, we reasoned that opponent retinotopic coding could be a widespread mechanism that scaffolds global opponent interactions across large-scale internally- and externally-oriented brain networks²⁹. We leveraged a high-resolution 7T fMRI dataset³⁰ and voxel-wise modeling to test whether retinotopic coding structures the global opponent dynamic between the DN and dATN. We tested three hypotheses. First, the voxel-wise retinotopic push-pull dynamic observed in the highly-specialized subareas of the DN/dATN during visual tasks²⁸ will generalize to the overall networks’ spontaneous neural activity, even in the absence of experimenter-imposed task demands. Second, among subareas of the DN/dATN, retinotopic coding will be integrated (multiplexed) with an areas’ functional domain to allow fine-grained control of information exchange across cortex. Finally, retinotopy should be intrinsic throughout the brain, even in areas of the brain associated with internal attention. So, the retinotopic code will be evident in both top-down and bottom-up interactions between perceptual and mnemonic areas. Together, these results would suggest retinotopy is a unifying framework organizing brain-wide information processing and internal versus external attention dynamics.

Retinotopic coding in internally and externally oriented networks

To investigate the role of retinotopic coding in structuring activity between internally- and externally-oriented brain networks, we used voxel-wise visual population receptive field (pRF) modeling and resting-state fMRI data from the Natural Scenes Dataset³⁰ (Fig. 1A). We assessed the retinotopic responsiveness of all voxels in the brain by modeling BOLD activity in response to a sweeping bar stimulus³¹. Then, we used resting-state fMRI data to identify each participant’s individually parcellated cortical networks^8,32 and assess the retinotopic biases of voxels within these networks (Fig. 1B; Fig. S1). We considered any voxel with >8% variance explained by our pRF model to be exhibiting a retinotopic code, and hereafter we refer to these voxels as “pRFs”. PRF amplitude maps for all participants are shown in Fig. S2. Because we did not observe any hemispheric differences in any pRF features or subsequent analyses (ps > 0.55), data are presented collapsed across hemispheres.

Fig. 1.

Inversion of retinotopic coding between externally- and internally- oriented networks in the human brain. A. Population receptive field (pRF) modeling with fMRI. A visual pRF model was fit for all participants to establish visual responsiveness and visual field preferences for each voxel. Voxels with positive BOLD responses to the visual stimulus are referred to as positive pRFs (+pRFs), and those with negative BOLD responses to visual stimulation are referred to as negative pRFs (-pRFs). B. Individualized resting-state network parcellation. Resting-state fMRI was collected in all participants and used to derive individualized cortical network parcellations. Participants (N=7) had between 34–102 minutes of resting-state data. Networks were parcellated³³ using the multi-session hierarchical Bayesian modelling approach^8,32 with the Yeo 15 HCP atlas³³ as a prior. C. Task-negative and task-positive (internally/externally oriented) brain networks contain differential concentrations of +/−pRFs. Bars show the concentration of −pRFs within each individual’s cortical networks. The Default Networks A and B (DN-A/B), canonically internally-oriented, contained the highest proportion of -pRFs, while the Dorsal Attention Networks (dATN-A/B), canonical externally-oriented, contained the highest proportion of +pRFs.

Consistent with prior reports²⁵, we observed a strong distinction in the visual response of canonical internally- and externally-oriented brain networks, the Default Networks A-B and the Dorsal Attention Networks A-B, respectively (hereafter, DNs and dATNs) (Fig. 1C). On average, 55.3% of dATN voxels were retinotopic and 26.95% of DN voxels were retinotopic, consistent with their role as externally- and internally-oriented networks, respectively. However, the nature of retinotopy in these two networks differed mainly on amplitude (i.e. whether a stimulus in their preferred visual field location evoked a positive or a negative BOLD response) (Fig 1C). On average more than half of pRFs in the DNs were inverted (i.e., had a negative BOLD response to visual stimulation in their population receptive field, -pRFs). In contrast, less than 20% of the voxels in the dATN were −pRFs (i.e., the majority of the voxels had positive BOLD responses to visual stimulation in their receptive field, +pRFs)). This distinction is particularly remarkable given the proximity of the dATN and DN clusters in posterior cerebral cortex (Fig. 1B). The response amplitude of these pRF populations was reliable (Fig. S3). Together, these results establish the necessary foundation to assess whether the voxel-wise retinotopic code structures the global opponent dynamic between the internally- and externally-oriented networks.

To test whether the retinotopic code structured the opponent interaction between the internally- and externally-oriented brain networks at a voxel level, we examined whether the visual field preferences of individual voxels within these networks predicted their anti-correlation at rest. In other words, would the opponent dynamic between voxels in the DNs and dATNs be stronger for voxels with similar visual field preferences? During resting-state fMRI, participants simply fixate with not externally imposed stimulus (aside from a fixation cross)³⁴. This allowed us to examine whether the retinotopic code structures the intrinsic organization of neural interactions absent an experimentally imposed external or internal task^4,35. Resting-state data were preprocessed using ICA with manual noise component selection^36,37, and no global signal regression was performed³⁸.

For each pRF in the DNs and dATNs, we calculated the pairwise distance between the RF center position (x, y parameter estimates) of −pRFs in the DNs and +pRFs in dATNs (Fig.2A). For each DN -pRF, we found the 10 closest (Matched) and furthest (Anti-matched) dATN +pRF centers. Interestingly, matched pRFs were distributed across the dATN, spanning posterior areas in retinotopic cortex through prefrontal cortical regions not typically associated with visual analysis (Fig. S4). We then averaged these matched and antimatched pRF’s resting-state time series together, and we compared the correlation of these matched/antimatched timeseries with the time series from the DN −pRFs (see methods). This resulted in two correlation values per resting-state run, which represented the correlation of the average matched and anti-matched pRF’s time series between these areas. Importantly, the primary statistics relevant to the conclusions in the manuscript replicated when considering randomly sampled pRF pairs, and therefore our conclusions do not depend on matching only the bottom 10 voxels in a specific region or network (See Supplemental methods and results)

Fig. 2.

Retinotopic coding organizes spontaneous interaction between internally and externally oriented brain networks. A. Spatially-matched pRFs in dATNs and DNs. We assessed the influence of retinotopic coding on the interaction between internally- and externally-oriented brain areas’ spontaneous activity during resting-state fMRI, by comparing the correlation in activation between pRFs in these networks that represent similar regions of visual space. For each −DN pRF, we established the top 10 closest +dATN pRF voxels’ centers (“matched”) and the 10 furthest pRF centers (“antimatched”). In each resting-state fMRI scan, we extracted the average time series from −DN pRFs and correlated that time series with the average time series from the +dATN matched and antimatched pRFs. We repeated this procedure for all resting-state runs in all participants. Plot shows one example resting-state time series from a participant’s -DN, +dATN matched and +dATN antimatched pRFs and the associated correlation values. B. Spatially matched −DN/+dATN pRFs have a greater opponent interaction than anti-matched pRFs, showing that opponent dynamics depend on retinotopic preferences. Histogram shows the distribution in correlation values between matched (dark green) and antimatched (light green) pRF pairs for each resting-state run in all participants, which were significantly different (matched versus antimatched: D(392)=0.245, p<0.001). Bar plot shows the average correlation for each participant (matched versus anti-matched: t(6)=3.49, p=0.011). C. DN subnetworks A and B both evidenced a retinotopic opponent interaction (DN-A: t(6)=3.02, p=0.023, DN-B: t(6)=2.72, p=0.034; DN-A vs. DN-B: t(6)=1.64, p<0.15), although the opponency was stronger overall in DN-A compared to DN-B (difference in average correlation between −DN/+dATN pRFs: t(6)=5.41, p=0.002).

Retinotopic coding scaffolds DN and dATN interactions

If the −DN and +dATN opponent interaction is scaffolded by a retinotopic code, the correlation between spatially matched pRFs should be significantly more negative than the anti-matched pRFs. Our results were consistent with this hypothesis. We observed a negative correlation between both matched and anti-matched −DN and +dATN pRFs in the overwhelming majority of resting state runs (Fig. 2B, left). Critically, the distribution of matched +dATN and −DN pRFs was significantly shifted compared to unmatched pRFs (D(392)=0.22, p<0.001), confirming an overall stronger negative correlation, and thus a stronger opponent interaction, between matched compared to unmatched pRFs. The stronger opponent interaction for matched versus anti-matched pRFs was clear when resting-state runs were averaged within each participant (7/7 participants; t(6)=3.63, p=0.010, Fig.2B, right). Interestingly, retinotopy played a similarly strong role in structuring the opponent dynamic between the dATN and both subnetworks of the DN (DN-A: t(6)=3.02, p=0.023, DN-B: t(6)=2.72, p=0.034; DN-A vs. DN-B: t(6)=1.511, p<0.182), although the opponency was stronger overall in DN-A compared to DN-B (t(6)=5.532, p=0.002). Importantly, tSNR of the matched and anti-matched pRF’s resting-state time series did not differ (t(6)=1.007, p=0.353), suggesting that differences in signal quality did not underlie the difference in correlation. Together, these results suggest that the retinotopic code plays a role in structuring the opponent dynamic between the brain’s internally- and externally-oriented cortical networks.

Retinotopic coding organizes activity within functional domains

Our results so far clearly establish that retinotopic coding structures spontaneous interactions between internally- and externally-oriented neural systems in the absence of task demands. However, high-level cortical areas generally associate into networks based on their functional domain. For example, within the visual system, brain areas with differing retinotopic preferences (e.g., the scene-selective areas on the lateral and ventral surfaces of the brain)^39,40 nevertheless form functional networks based on their apparent preference for specific visual categories (e.g., faces, objects, or scenes)^41,42. This raises a crucial question: do retinotopic and domain-specific organizational principles interact to facilitate or constrain information flow across internally- and externally-oriented networks? Addressing this question would shed light on mechanisms that enable the brain to integrate information while maintaining functional specialization.

To address this question, we focused on the functional interplay between a set of areas in posterior cerebral cortex that are established models for mnemonic and visual processing in the domains of scene and face perception. Specifically, we considered the mnemonic lateral place memory area (LPMA^43,44), an area on the brain’s lateral surface that is implicated in processing mnemonic information relevant to visual scenes at the border between DN-A and the dATNs (Fig 2A). We examined how LPMA activation co-fluctuates with the adjacent, scene-perception area “occipital place area” on the brain’s lateral surface (OPA^45,46) compared to two face-perception regions on the lateral and ventral surfaces, the occipital and fusiform face areas (OFA and FFA^47,48). At a group level, these perceptual regions are situated within the dATNs and are at the same level of the visual hierarchy (Fig. 3A), but they are differentially associated with the domains of scene (OPA) and face (OFA, FFA) processing, making them ideal model systems to examine the impact of domain-specificity and retinotopic coding in organizing neural activity.

Fig. 3.

Retinotopic coding structures the spontaneous opponent interaction between functionally-coupled mnemonic and perceptual areas during resting-state fMRI. A. Isolating functionally coupled internally- and externally-oriented brain areas within the DNs and dATNs. We established brain areas relevant to two cognitive domains, visual analysis of 1) scenes and 2) faces. Specifically, we focused on a memory area in the domain of scene perception (the lateral place memory area (LPMA; from⁴³), white) at the posterior edge of the DN-A (purple; from³³). We examined LPMA’s relationship to a set of perceptual areas in the dATN (green; from³³), 1) the occipital place area (OPA; from⁴³), an area within the domain of scene perception, along with 2) the occipital face area (OFA) and 3) the fusiform face area (FFA), two areas involved in the domain of face perception (white; from⁵¹). B-C. We localized LPMA in all participants by contrasting the correlation in resting-state activity between anterior and posterior parahippocampal place area (PPA) (B). This yielded a region in lateral occipital-parietal cortex that overlapped with the LPMA defined in an independent group of participants (C). D-E. Consistent with prior work, the connectivity-defined LPMA had greater concentration of −pRFs compared to OPA (D), and exhibited a lower visual field bias to OPA (E), consistent with an opponent interaction between these areas during perception. F. We assessed the influence of retinotopic coding on the interaction between −pRFs in mnemonic and +pRFs in perceptual areas using the same pRF matching and correlation procedure described above. We compared pRFs within functional domain (scene memory × perception –LPMA to OPA) as well as across domains (scene memory × face perception –LPMA to the occipital face area (OFA) and fusiform face area (FFA)). G. Within functional domain opponent interaction reflects voxel-wise retinotopic coding. We observed a stronger negative correlation between matched compared to anti-matched −LPMA/+OPA pRFs (-LPMA × +OPA matched versus anti-matched pRFs: D(392)=0.22, p<0.001). H. Retinotopic coding did not impact the interaction between areas across functional domains. We found no significant difference between matched and anti-matched pRFs between the scene memory area LPMA and the face perception areas FFA and OFA (-LPMA × +OFA: D(392)=0.09, p=0.44; −LPMA × +FFA: D(392)=0.11, p=0.15). Histograms depict the distribution of correlation values between matched (dark) and antimatched (light) pRFs for all runs in all participants. I. Retinotopic coding organizes interactions within a domain, but not across domains. When the correlation values were averaged within each participant, we observed a significant difference between matched versus anti-matched pRFs within functional domain (-LPMA × +OPA: t(6)=4.45, p=0.004) but not across domains (-LPMA × +OFA: t(6)=1.04, p=0.34; −LPMA x +FFA: t(6)=1.48, p=0.188).

We defined the mnemonic area LPMA in the NSD participants using resting-state data by contrasting functional connectivity between the anterior and posterior halves of the parahippocampal place area^49,50 (Fig. 3B), which revealed a cluster in lateral parietal cortex with a similar topographic profile as LPMA based on a group analysis from our prior work⁴³ (Fig.3C). Replicating our previous findings²⁸, this connectivity-defined LPMA had a higher concentration of robust −pRFs compared to OPA (t(7)=5.26, p=0.002) (Fig.3D, Supplemental Fig. S3, S5) and exhibited a similar lower visual field bias as OPA (OPA: 8/8 t(7)=3.13, p=0.016; LPMA: 7/8 participants, t(7)=2.11, p=0.07; OPA v LPMA: t(7)=0.441, p=0.67) (Fig. 3E).

We first considered whether the retinotopic opponent dynamic we have previously shown in the domain of scenes (i.e., between −LPMA and +OPA pRFs) during perceptual and mnemonic tasks²⁸ was also present at rest (Fig. 3F). Like the overall DN and dATN networks, we found that −LPMA and +OPA pRFs are interlocked in a retinotopically-grounded opponent interaction. Resting-state activity of −LPMA and +OPA pRFs was reliably negatively correlated, and this negative correlation was stronger for matched compared with anti-matched pRFs (K-S test: D(392)=0.22, p<0.001; t(6)=4.45, p=0.004) (Fig.3–I; Fig. S6). This pattern was consistent when matched/anti-matched pRFs were equated for eccentricity and size (matched vs. anti-matched: D(392)=0.148, p=0.028; t(6)=2.03, p=0.087; 5/7 participants) (Fig. S7). Importantly, matched and anti-matched +OPA pRFs resting-state tSNR (t(6)=1.922, p=0.103) and variance explained by the pRF model (t(6)=0.47, p=0.65) did not differ, suggesting that idiosyncratic voxels did not drive these results. Additionally, the opponent interaction was specific to −pRFs in LPMA: the activity of the best matched +pRFs in LPMA and OPA was positively correlated, and significantly more so than anti-matched +pRFs in LPMA and OPA (t(6)=3.22, p=0.018; Supplementary Fig. S8). Taken together, these results show that the retinotopic code scaffolds the spontaneous interaction between perceptual and mnemonic brain areas within a functional domain, conceptually replicating our previous findings from task-fMRI²⁸.

Having established that the retinotopic opponent dynamic is present among functionally-paired brain areas within the domain of scenes, we next tested whether this opponent dynamic was modified by functional domain (i.e., the scene memory area LPMA paired with the face perception areas FFA and OFA). Remarkably, we observed no significant difference between the distribution of correlation values for matched and anti-matched pRFs across functional domains (Matched versus anti-matched pRFs --LPMA × +OFA: D(392)=0.09, p=0.44, t(6)=1.04, p=0.34; FFA: D(392)=0.11, p=0.15, t(6)=1.48, p=0.18; Fig. 2G–I), and retinotopic-opponency was greater within-compared to across-domain matching (scene-memory × scene-perception versus average scene-memory × face-perception: t(6)=2.46, p=0.049). Importantly, matched pRFs within domain had a significantly stronger opponent interaction than across domains (within vs across domains –LPMAx+OPA v −LPMAx+FFA: t(6)=5.94, p=0.001; −LPMAx+OPA v −LPMAx+OFA: t(6)=7.03, p<0.0005). These results indicate that retinotopic coding does not structure the interaction between pairs of regions associated with distinct functional domains. Instead, retinotopic scaffolding appears to be selective, operating only within a given functional domain. This coding scheme could allow for efficient, parallel processing of domain-specific information (e.g., faces, scenes) that can be flexibly adapted depending on task demands.

Retinotopic coding is inherent to internally-oriented areas

Our findings support the crucial role of retinotopic coding in scaffolding spontaneous interactions between functionally-coupled mnemonic and perceptual brain areas. However, a fundamental question remains: is retinotopy intrinsic to mnemonic cortical areas or merely adopted in response to perceptual input? Resolving this distinction is critical to understanding if the retinotopic scaffold is a general purpose mechanism for cross-network interaction that transcends specific task demands and cognitive domains (i.e,. memory vs. perception).

To address this question, we developed an analytical approach to disambiguate bottom-up perceptual signals from top-down mnemonic signals, in which we identified spontaneous neural “events” at individual −LPMA (top-down) or +OPA (bottom-up) pRFs and examined the co-activation of the target area at the event time. We defined an “event” as any time point where the z-scored BOLD signal of a given voxel exceeded the 99th percentile of activity in a given resting-state run. We then analyzed the peri-event activation in the target area’s matched and anti-matched pRFs (Fig. 3A). Importantly, because participants do not have any experimentally imposed task demands during resting-state fMRI, any structured interaction between areas will reflect these regions’ spontaneous dynamics.

Our analysis detected 20,958 top-down events and 7,193 bottom-up events. Top-down and bottom-up events occurred at the same rate per pRF (t(6)=0.71, p=0.50). Individual participants averaged 116±70.4 top-down and 32±11.76 bottom up (mean±sd) events per run. All pRFs had between 0 and 4 events per run (Fig. S8A–C). The wide distribution of events in time suggested that individual pRF-level correlation could be isolated from global fluctuations in regional activity (Fig. S8A), making this approach suitable for evaluating distinct interactions at the individual voxel level.

We found that both top-down and bottom-up events tended to reduce target area activity via a retinotopic code, supporting the hypothesis that retinotopic coding is intrinsic to mnemonic cortical areas. During top-down events, +OPA pRFs showed significant deactivation that was more pronounced in matched compared to anti-matched pRFs (Fig. 3B). On the other hand, −LPMA pRFs had elevated activity during bottom-up events, but, crucially, the activation of matched pRFs was significantly reduced compared to anti-matched pRFs. Thus, bottom-up events also showed evidence of an inhibitory, retinotopic influence: the elevated activity in −LPMA (e.g., due to mentation and mind wandering during rest) is reduced after input from retinotopically-matched +OPA pRFs. This is clear evidence that both top-down and bottom-up events evoke retinotopically-specific inhibitory responses in the target region, suggesting that retinotopic coding is intrinsic to memory areas, even in the absence of task demands or overt visual input.

Importantly, the influence of retinotopic coding was similar for both bottom-up and top-down events, indicating a symmetric opponent interaction. Target area activity was significantly lower for matched compared to anti-matched pRFs in both directions (top-down: t(6)=4.13, p=0.006; bottom-up: t(6)=3.17, p=0.02), with no difference between event types (t(6)=0.86, p=0.42). The balanced dynamic between −LPMA and +OPA pRFs mirrors other nervous system interactions^52–55, extending this well-established framework in these sensory/motor domains to higher-order cognitive functions.

Discussion

In summary, it is well-established that internally- and externally-oriented distributed brain networks, including DNs and dATNs, support higher cognition in humans^{1–8,10–12}. Yet we lack an understanding of what coding principles, if any, underpin interactions across these distributed brain networks^24,56–60. Here, we show that a retinotopic code scaffolds the voxel-scale opponent interaction between internally- and externally-oriented brain networks, even in the absence of overt visual demands. Moreover, by examining functionally-linked perceptual and mnemonic areas straddling the boundary between the DNs and dATNs, we found that this retinotopic information is multiplexed with domain-specific information, which may enable effective parallel processing of representations depending on retinotopic location, attentional-state, and task-demands. Finally, analysis of neural events in these functionally-linked regions showed that the retinotopic opponency is present in top-down as well as bottom-up events, suggesting that the retinotopic code is intrinsic to both perceptual and mnemonic cortical areas. Collectively, our results provide a unified framework for understanding the flow of neural activity between brain areas, whereby macro-scale neural dynamics are organized at the meso-scale by functional domain and at the voxel-scale by a low-level retinotopic code. This multi-scale view of information processing has broad implications for understanding how the brain’s distributed networks give rise to attention, perception, and memory.

Because of the DN’s importance in many cognitive processes, including spatial and episodic memory, social processing, and executive functioning^12,61–66, resolving the coding principles inherent to the DN is central to understanding human cognition^12,61–66. The traditional view of neural coding posits that sensory codes like retinotopy are shed in favor of abstract, amodal codes moving up the cortical hierarchy towards the DN^2,23,24. Our data contrast strikingly with this view. Instead of shedding the retinotopic code at the cortical apex, our prior work has shown that the low-level retinotopic code structures interactions between functionally paired perceptual and mnemonic areas in posterior cerebral cortex involved in visual scene analysis during visual and memory tasks²⁸. Here we significantly extend that earlier finding by showing that the retinotopic code organizes spontaneous interactions across the brain’s large-scale cortical networks, even in the absence of task demands. The importance of retinotopic coding is further underscored when assessing “top-down” events (i.e. events that originate in the DN) detected at rest, which had a retinotopically-specific suppressive influence on pRFs in their downstream target area. This result joins mounting evidence demonstrating retinotopic coding in the DN^25–28, as well as new evidence supporting the importance of visual coding in the DN during visual tasks^28,67. Together, these findings show that the retinotopic code is a “native language” of the DN, and they suggest that the DN plays an active role in perceptual experience and behavior by shaping visual responses via the retinotopic code.

Our findings of retinotopic, voxel-scale opponency between DN and dATN pRFs is at odds with theories that posit a global competition between these networks and a general disengagement of the DN from sensory processing^9–12. Instead of network-level opponency, where DN globally disengages during cognitive states that drive the dATNs and vice versa, the opponent interaction between the DN and dATN reflects their common engagement in retinotopic information processing at a voxel level. Without considering voxel-wise information this opponency appears to “average out” into a global task-negative/task-positive response¹¹, which has led to the conclusion that these networks are engaged in fundamentally different cognitive processes. In contrast, our results emphasize that the DN and dATNs are mutually engaged in processing common, retinotopic information. Taken together with prior results^25–28, our findings prompt a reevaluation of the role of the DN in perceptual processing, and the extent to which roles in “internally- and externally-oriented attention” adequately captures the DN and dATN’s role in cognition.

Our findings align with other recent studies suggesting an important role of the DN in shaping sensory responses beyond retinotopy. For example, recent studies have shown that ongoing prestimulus DN activity influences the sensitivity of near-threshold visual object recognition⁶⁷. Other work has shown that subareas of the DN represent the visuospatial context that is associated in memory with a perceived scene^28,43,44,68. Relatedly, DN activity has been shown to reflect semantic-level attentional priorities during visual search⁶⁹. Taken together, these findings show that, rather than being disengaged during visual tasks, the DN actively shapes responses in perceptually-oriented cortex. This raises a fundamental question: is the DN a “visual” network? One possibility is that the DN directly and obligatorily represents retinal position, like low-level visual areas, and is engaged in processing visual features in addition to more abstract features^31,70. Under this hypothesis, the retinotopic code indicates that the information represented in the DN is, in part, sensory. Alternatively, the apparent visual coding may simply represent an underlying connectivity structure⁷¹, which enables associative⁷² and semantic^24,73 information that the DN directly represents to be effectively transferred to sensory areas of the brain. Under this alternative hypothesis, the retinotopic code serves as a “highway” between networks that are engaged in fundamentally different functions: abstract (DN) and sensory (dATN) processing. Crucially, in either case the retinotopic scaffolding represents a latent structure linking high-level and low-level areas through consistent, spatially-organized interactions⁷¹. Further studies investigating this framework with diverse tasks and stimuli could reshape our understanding of how the brain processes and integrates information across different levels of cognitive complexity.

Finally, our data paint a clear picture of cortical dynamics spanning levels of analysis, from macro-scale brain networks, to meso-scale domain-specific subareas, to small-scale voxel-level interactions. Prior work has emphasized the importance of each of these organizational scales independently^{1,8,13,25,26,28,31,33,42,47,74,75}. Here, our data reveal a comprehensive framework for understanding the brain’s functional organization that spans these levels of description. Specifically, we demonstrate that retinotopic coding underpins voxel-scale interactions that are observable across the whole brain. However, these retinotopic interactions are constrained at the mesoscale by specific brain areas’ functional domains (e.g., processing visual scenes). Further, interplay among these domain-specific regions underpins the organization of large-scale brain networks, whose interactions give rise to specific complex behavior, like memory recall or visual attention. This nested hierarchy of neural interactions accounts for the efficiency of parallel information processing in the brain and the flexibility to adapt to ongoing task demands.

In summary, our results show that a retinotopic code organizes the spontaneous interactions of large-scale internally- and externally-oriented networks in the human brain. These findings challenge our classic understanding of internally-oriented networks like the DN, showing that the global opponent dynamic between the DN and dATN does not reflect disengagement from visual processing. Rather, the global opponent dynamic is structured by a voxel-wise retinotopic code that scaffolds interactions across these large-scale internally- and externally-oriented networks. Taken together, these results indicate that retinotopic coding, the human brain’s foundational visuo-spatial reference frame^29,71, structures large-scale neural dynamics and may be a “common currency” or subspace for information exchange across the brain’s functional networks.

Methods

The data analyzed here are part of the Natural Scenes Dataset (NSD), a large 7T dataset of precision MRI data from 8 participants, including retinotopic mapping, anatomical segmentations, functional localizers for visual areas, and task and resting-state fMRI. A full description of the dataset can be found in the original manuscript³⁰. Here we detail the data processing and analysis steps relevant to the present work.

Subjects

The NSD comprises data from 8 participants collected at the University of Minnesota (two male, six female, ages 19–32). One subject (subj03) was excluded from resting-state analyses for having insufficient resting-state runs that passed our quality metrics. All participants had normal or correct to normal vision and no known neurological impairments. Informed consent was collected from all participants, and the study was approved by the University of Minnesota institutional review board.

MRI acquisition and processing

For this study we made use of the following data from the NSD: anatomical (T1 and FreeSurfer segmentation/reconstruction^76,77), functional regions of interest ROIs, and minimally preprocessed retinotopy and resting-state time series. All analyses were conducted in original subject volume space, and data were projected on the surface for visualization purposes only.

Anatomical data

Anatomical data was collected using a 3T Siemens Prisma scanner and a 32-channel head coil. We used the anatomical data provided at 0.8mm resolution as well as the registered output from FreeSurfer recon-all, aligned to the 1.8mm functional data. For visualization purposes, we projected statistical and retinotopy data to the cortical surface using SUMA⁷⁸ from the afni software package⁷⁹.

Defining functional regions of interest (PPA, OPA, OFA, FFA)

We used the volumetric functional regions of interest provided with the NSD at 1.8mm resolution. Specifically, we used the parahippocampal place area (PPA), occipital place area (OPA), iog-faces (referred to here as occipital face area (OFA)), and pfus-faces (fusiform face area, FFA1) regions of interest. In brief, these regions were defined in the NSD using within-subject data collected from 6 runs of a multi-category visual localizer paradigm³⁰.

Defining functional region LPMA

Because the NSD did not include mnemonic localizers⁴³, we used the resting-state data to define the lateral place memory area (LPMA). Briefly, the LPMA is a region anterior to OPA and near caudal inferior parietal lobule^49,50 that selectively responds during recall of personally familiar places compared to other stimulus types. We have previous shown that OPA and LPMA are functionally-linked and work jointly to process knowledge of visuospatial context out of view during scene perception.

Prior work has suggested that a mnemonic area linked to scenes on the lateral surface can be localized by comparing resting-state co-fluctuations of anterior versus posterior PPA (aPPA and pPPA, respectively)^16,49,50, and we adopted that approach here. We preprocessed the resting-state fixation fMRI data, runs 1 and 14 from NSD sessions 22 and 23, in all participants (prior to data exclusion) and extracted the average time series of aPPA, pPPA, aFFA, and pFFA. We used these time series as regressors in a general linear model, and we compared the beta-values from the aPPA and pPPA. We considered any voxels with a t-statistic > 5 within posterior parietal-occipital cortex on the lateral surface as an LPMA ROI (Individual ROIs can be found in Supplemental Fig.4). Across subjects, there was considerable overlap between this connectivity-defined area and a group-level LPMA defined based on our prior work (Fig. 1).

Functional MRI data acquisition and processing

All analyses were conducted on 1.8mm isotropic resolution minimally processed runs of a sweeping-bar retinotopy task and resting-state fixation provided in the NSD.

Quality assessment

To ensure only high-quality resting-state data were included, we trimmed the first 25 TRs (40s) from each run⁶⁸, which left 4.25 min of resting-state data per run. We then used afni’s quality control assessment tool (APQC⁸⁰) on the raw trimmed resting-state time series to assess degree of motion in the resting-state scans. We excluded runs with greater than 1.8mm maximum displacement per run or 0.12mm framewise displacement from analysis and assessed runs with greater than 1mm displacement or 0.10mm framewise displacement on a case-by-case basis⁸. After exclusions, one participant (subj03) had only a single run that survived our criteria (4.25 minutes of data)), so we excluded them from resting-state analyses. The remaining 7 participants had at least 8 resting-state runs (> 34 minutes of data; mean number of runs: 14±6.13 (sd), range: 8–24 runs).

ICA denoising

To further denoise the retinotopy and remaining resting-state data, we used manual ICA classification of signal and noise on the minimally-preprocessed time series^36,37,43. We used manual classification because automated tools perform poorly on the high spatial and temporal resolution data of the NSD⁸¹. For each retinotopy and resting state run, we decomposed the data into independent spatial components and their associated temporal signals using ICA (FSL’s melodic^82,83). We then manually classified each component as signal or noise using the criteria established in³⁷. Noise signals were projected out of the data using fsl_regfilt³⁶.

No global signal regression was performed on these data³⁸. Data were normalized to percent signal change. A 2.5mm FWHM smooth was applied to resting-state fixation data used in functional connectivity network identification, otherwise no spatial smoothing was applied to the data.

Data analysis - retinotopy

pRF Modelling

The NSD retinotopy stimuli features a mosaic of faces, houses, and objects superimposed on pink noise that are revealed through a continuously drifting aperture. For our analysis, we considered only the bar stimulus time series, which is consistent with other studies investigating −pRFs in high-level cortical areas^25,27,28. We did not consider the wedge/ring stimulus for any analyses.

After denoising the retinotopy data, we averaged the three bar stimulus retinotopy runs together to form the final retinotopy time series. We performed population receptive field modeling using afni following the procedure described in³⁹. First, because the pRF stimulus in the NSD is continuous, we resampled the stimulus time series to the fMRI temporal resolution (TR = 1.333s). Next, we implemented afni’s pRF mapping procedure (3dNLfim). Given the position of the stimulus in the visual field at every time point, the model estimates the pRF parameters that yield the best fit to the data: pRF amplitude (positive, negative), pRF center location (x, y) and size (diameter of the pRF). Both Simplex and Powell optimization algorithms are used simultaneously to find the best time series/parameter sets (amplitude, x, y, size) by minimizing the least-squares error of the predicted time series with the acquired time series for each voxel. Relevant to the present work, the amplitude measure refers to the signed (positive or negative) degree of linear scaling applied to the pRF model, which reflects the sign of the neural response to visual stimulation of its receptive field.

Visual field coverage

Visual-field coverage (VFC) plots represent the sensitivity of an ROI across the visual field. We followed the procedure in Steel et al., 2024 to compute these²⁸, which we have reproduced here. Individual participant VFC plots were first derived. These plots combine the best Gaussian receptive field model for each suprathreshold voxel within each ROI. Here, a max operator is used, which stores, at each point in the visual field, the maximum value from all pRFs within the ROI. The resulting coverage plot thus represents the maximum envelope of sensitivity across the visual field. Individual participant VFC plots were averaged across participants to create group-level coverage plots.

To compute the elevation biases, we calculated the mean pRF value (defined as the mean value in a specific portion of the visual-field coverage plot) in the contralateral upper visual field (UVF) and contralateral lower visual field (LVF) and computed the difference (UVF–LVF) for each participant, ROI and amplitude (+/−) separately. A positive value thus represents an upper visual-field bias, whereas a negative value represents a lower visual-field bias. Analysis of the visual-field biases considers pRF center, as well as pRF size and R².

Reliability of pRF amplitude estimate

To assess the reliability of pRF amplitude (i.e., positive versus negative), we iteratively compared the amplitude of significant pRFs from individual runs of pRF data. Specifically, for each single run of pRF data, we fit our pRF model. We then binarized vertices according to significance and amplitude –voxels that surpassed our significance threshold (R2 > 0.08) in the full model were assigned a value of 0 or 1. When investigating +pRF reliability in OPA, significant vertices with a positive amplitude were assigned to 1, all other vertices 0. For -pRF reliability in LPMA, the opposite was done: significant negative amplitude vertices were assigned a value of 1, and all other vertices set to 0.

For each participant, after binarization, we calculated a dice-like coefficient within each ROI which considered all three retinotopy runs $\left(\mathit{\text{cofficient}}=\frac{3\left|\mathit{\text{run}}1\cap \mathit{\text{run}}2\cap \mathit{\text{run}}3\right|}{\left|\mathit{\text{run}}1\right|+\left|\mathit{\text{run}}2\right|\left|\mathit{\text{run}}3\right|}\right)$. We compared this value versus 5000 iterations of the same number of voxels randomly sampled from all voxels, both significant and non-significant, in the ROI. For each participant, this resulted in 1 “observed” dice coefficients, along with 5000 bootstrapped values representing the distribution of dice coefficients expected by chance which we used to evaluate the significance of each pRF amplitude’s run-to-run consistency.

Voxel-wise pRF matching

We matched −pRFs in source (i.e., the DN) with target (i.e., +pRFs in the dATN) using the following procedure. Within each participant, we computed the pairwise Euclidean distance between the center (x,y) of each source pRF with each target pRF. For each source pRF, we considered the top 10 closest target pRFs the “matched pRFs” and the 10 furthest target pRFs the “anti-matched pRFs.” So, each pRF within a memory area yielded 10 matched and 10 anti-matched pRFs.

To investigate the opponent interaction between areas for top-down interactions (below), we conducted this procedure using −LPMA pRFs as the source and +OPA pRFs as targets. To investigate bottom-up interactions, we considered +OPA pRFs as the source and −LPMA pRFs as targets. To investigate the importance of functional domain in retinotopic interactions, we considered −LPMA pRFs as the source and 1) +OFA and 2) +FFA pRFs as targets.

Resting-state analyses

Individual-specific cerebral network estimation

In each participant, we identified a set of 15 distributed networks on the cerebral cortical surface using a multi-session hierarchical Bayesian model (MS-HBM) functional connectivity approach^8,32. Briefly, the MS-HBM approach calculates a connectivity profile for every vertex on the cortical surface derived from that vertex’s correlation to all other vertices during each run of resting-state fixation. The MS-HBM then uses the resulting run- and participant-level profiles, along with a 15 network group-level prior created from a portion of the HCP S900 data³³, to create a unique 15-network parcellation for each individual.

The MS-HBM has two primary advantages compared to other approaches used to parcellate functional connectivity data, such as k-mean clustering. First, it accounts for differences in connectivity both within an individual (likely the result of confounding variables such as scanner variability, time of day, etc.), and between participants (reflecting potentially meaningful individual differences), allowing for more reliable estimates. Second, by incorporating a group prior which includes all the networks of interest, we ensure that all networks will be identified in all participants, while allowing for idiosyncratic topographic differences.

Opponent interaction at rest

To assess the influence of retinotopy on the correlation of areas at rest, we used the following procedure. First, we established matched source (i.e., −DN or -LPMA) and target (i.e., +dATN or +OPA) pRF pairs using the procedure described above. For this analysis, our primary focus was on the −pRFs in the DN or LPMA and their relationship with perceptual areas (dATN, OPA, OFA, FFA). Among these highly connected areas, large-scale fluctuations due to attention and motion will cause these voxels to be highly correlated. To control for this, consistent with prior work, we extracted the average time series of all +pRFs in the DN or LPMA and partialed out the variance associated with these pRFs from the −pRFs in the DN or LPMA and +pRFs in the perceptual areas of interest from each resting-state run^28,84.

To examine the opponent interaction for matched pRFs, we considered each resting state run separately. In each resting state run, we first calculated the average time series of all −pRFs in LPMA. We then calculated the average time series for the top 10 best matched +pRFs in OPA of all voxels. For example, if a participant had 211 −pRFs in LPMA, these voxels were averaged together to get a single −LPMA pRF time series, and the top 10 matched +pRFs in OPA (i.e., 2110 time series) would be averaged together to constitute the +OPA time series. Note that the matched +OPA pRFs were not unique for each voxel. We then correlated these time series (−LPMA and +OPA pRFs). A negative correlation was considered an opponent interaction.

To compare the importance of retinotopy in structuring the interaction between LPMA and OPA, we performed the same averaging and correlation procedure described above with the 10 worst matched pRFs (anti-matched). For each participant, all Fisher-transformed correlation values (z) for matched versus anti-matched pRFs were averaged together and we compared these matched and anti-matched correlation values using a paired t-test. To examine the specificity within each functional network, we repeated this procedure for −LPMA matched/anti-matched with OFA and FFA-1.

To confirm that retinotopic coding structured the opponent interaction, we repeated this anaylsis with two key differences: 1), to ensure that the choosing the bottom 10 voxels did not drive our results, we randomly sampled 10 pRFs from the furthest 33% of pRFs from each participant, and 2) we performed the analysis without averaging the pRF’s resting-state time series within each ROI. For each resting state run, we extracted each -pRF time series, and correlated this time series with the time series from the average of its top 10 best matched +pRF in OPA. We performed this matching and random sampling procedure 1000 times for each pRF in the source area. For example, if a participant had 211 −pRFs in LPMA, we would correlated each of these pRFs with the average time series from the their top 10 best matched +pRFs in OPA, resulting in 211 individual r-values for each resting-state run, and compared each of these 211 values with the correlation of 1000 randomly paired pRFs for that source pRF. We then Fisher-transformed these matched and randomly sampled values and averaged them to constitute a single value for each participant. The mean Fisher-transformed values were compared using a paired t-test.

Resting-state event detection

We reasoned that the influence of top-down versus bottom-up drive on the spontaneous interaction between regions could be isolated by examining periods of unusually high activity in voxels in these respective areas (“events”)^85,86. Specifically, we tested the hypothesis that top-down events in −LPMA pRFs would co-occur with periods of lower activity in +OPA pRFs, and that the suppressive influence of top-down events would be stronger compared with bottom-up events. To test these hypotheses, we isolated neural events in each source region’s (−LPMA (top-down) and +OPA (bottom-up)) pRF time series and examined the activity in the corresponding target region at these event times.

Event detection was performed for each +/− pRF independently. To detect events, we z-scored all pRF’s resting-state time series and identified TRs with unusually high activity in source region pRFs. Specifically, we considered each time point with a z-value greater than 2.4 (i.e., 99.18 percentile) as a neural event. Results were comparable with varying thresholds between 2.1 < z < 2.9. At each event, we extracted the 6 TRs before and after the event time (i.e., 13 TRs) surrounding the average time from that pRF’s top-10 matched pRFs in the target region. To make time series comparable across events, we normalized the event time series to the mean of the first 4 TRs. We repeated this procedure for all -LPMA/+OPA pRFs for top-down and bottom-up events.

For top-down and bottom-up events, we compared the activity of the target region (top-down: +OPA pRFs; bottom up: −LPMA pRFs) at event time using paired t-tests. We only considered matched pRFs for this analysis.

Statistical tests

Statistical analyses were implemented in Matlab (Mathworks, Inc). Given the small number of subjects in this dataset, we used two statistical analysis methods to ensure robustness of any detected effects. First, borrowing analytical methods from neuroscientific studies using animal models, we leveraged the large within participant data by pooling observations (e.g., resting-state runs or pRFs) from each participant. We then tested for differences in distributions using two-sample Kolmogorov-Smirnov goodness-of-fit hypothesis tests. Second, we adopted more classic statistical methods to test for effects within participants. We used paired-sample t-tests and corrected for multiple comparisons where appropriate.

Fig. 4.

Top-down vs. bottom-up neural events detected in spontaneous resting-state dynamics show evidence for retinotopically-specific suppression. A. Event detection and analysis procedure and example events from a single resting-state run. To detect events, we extracted the time series from each pRF in the source regions (top-down: -LPMA; bottom-up: +OPA) and isolated time points where the z-scored time series exceeded 2.4 s.d. (99th percentile). We then examined the activity of matched and anti-matched pRFs from the target region in this peri-event time frame (6 TRs (8 s) before and after the event). Overall, this event detection procedure yielded 20958 top-down and 7985 bottom-up events that were well distributed in time (Fig. S8). B. −LPMA pRF events co-occur with suppression of retinotopically-matched +OPA pRFs. Peri-event time series depicts the grand average activity of matched (dark) and anti-matched +OPA pRFs. Time series are baselined to the mean of the first three TRs (TRs −6 to −4 relative to event onset, dotted line). Red significance line shows time points with a significant difference between matched and anti-matched activation, corrected for multiple comparisons (alpha-level: p<0.05/13 = 0.0038). C. Suppression of ongoing activity in retinotopically matched −LPMA pRFs during +OPA events. Peri-event time series depicts the grand average activity of matched (dark) and anti-matched +OPA pRFs. As predicted, −LPMA activity is elevated during resting-state. This elevated ongoing activity is suppressed during events in retinotopically matched +OPA pRFs. Time series are baselined to the mean of the first three TRs (TRs −6 to −4 relative to event onset, dotted line). Blue significance line shows time points with a significant difference between matched and anti-matched activation, corrected for multiple comparisons. D. Target area shows retinotopically-specific suppression of activity for both top-down and bottom-up events. Bars show the average activation at event time of the target areas’ matched and anti-matched pRFs for each participant. Activity in matched pRFs was significantly lower than anti-matched pRFs for both top-down (t(6)=4.13, p=0.006) and bottom-up (t(6)=3.17, p=0.02), and there was no difference in the influence of retinotopic coding between these event types (t(6)=0.86, p=0.42).

Data availability:

All data is made publicly available via the Natural Scenes Dataset (https://naturalscenesdataset.org/)