Insights from the Evolving Model of Two Cortical Visual Pathways

Abstract

The two cortical visual pathways framework has had a profound influence on theories and empirical studies of the visual system for over 40 years. By grounding physiological responses and behavior in neuroanatomy, the framework provided a critical guide for understanding vision. Although the framework has evolved over time, as our understanding of the physiology and neuroanatomy expanded, cortical visual processing is still often conceptualized as two separate pathways emerging from the primary visual cortex that support distinct behaviors (“what” vs. “where/how”). Here, we take a historical perspective and review the continuing evolution of the framework, discussing key and often overlooked insights. Rather than a functional and neuroanatomical bifurcation into two independent serial, hierarchical pathways, the current evidence points to two highly recurrent heterarchies with heterogeneous connections to cortical regions and subcortical structures that flexibly support a wide variety of behaviors. Although many of the simplifying assumptions of the framework are belied by the evidence gathered since its initial proposal, the core insight of grounding function and behavior in neuroanatomy remains fundamental. Given this perspective, we highlight critical open questions and the need for a better understanding of neuroanatomy, particularly in the human.

INTRODUCTION

Among her many achievements, Leslie Ungerleider is perhaps best known for the two cortical pathways model of the primate visual system (Mishkin, Ungerleider, & Macko, 1983; Ungerleider & Mishkin, 1982), which has proven to be one of the most influential frameworks in the field (Kravitz, Saleem, Baker, Ungerleider, & Mishkin, 2013; Kravitz, Saleem, Baker, & Mishkin, 2011; Goodale & Milner, 1992). In its original instantiation, this framework identified two cortical pathways arising from primary visual cortex (V1): a ventral pathway projecting into the temporal cortex and a dorsal pathway projecting into the parietal cortex (Figure 1A). The framework has evolved over time and even been extended into other species, including mice (D'Souza et al., 2022; Wang, Sporns, & Burkhalter, 2012; Wang, Gao, & Burkhalter, 2011). Key to the framework's longevity and influence is its grounding of function in neuroanatomy. Each pathway was proposed to subserve different functions, with recognition or “what” processing localized to the ventral, and spatial or “where” processing to the dorsal pathway. Thus, visual function was reduced to two broad types, and the neuroanatomy was reduced to two “feed-forward” pathways that produced two distinct unitary representations to support those behaviors. This framework took two intractably complex and detailed domains, simplified and yoked them together, yielding a wealth of simple testable predictions around which the field could operationalize and experiment. However, the results of those experiments belie the complexity of both the neuroanatomy and function.

Figure 1. .

The evolving model of two visual cortical pathways. Different visualizations of the two cortical pathways showing how our understanding of them has grown over time. (A) Early description of two visual pathways emerging from V1. (B) Series of interconnected visual regions diverging into two pathways (Ungerleider & Desimone, 1986a). (C) Complex web of connections among visual areas (Felleman & Van Essen, 1991). (D) Quantitative analysis of visual connections (Young, 1992). (E) Ventral visual pathway (Kravitz et al., 2013). (F) Dorsal visual pathway (Kravitz et al., 2011).

Just over a decade ago, over the course of almost 3 years and many, many discussions, we were privileged to work with Mort Mishkin and Leslie Ungerleider, along with Saleem Kadharbatcha, to synthesize the literature published after the original proposal and update our understanding of the two cortical pathways (Kravitz et al., 2011, 2013). This was an extremely rewarding and humbling experience that gave us incredible insight into the thinking behind the two pathways framework. In this article, we want to briefly revisit the origins of the framework, how it has developed over time, and then discuss some of the key features that we think are often overlooked, focusing on the nature of the pathways and their putative functional roles.

ORIGINS OF THE TWO CORTICAL PATHWAYS FRAMEWORK

The two cortical pathways framework grew out of two prior bodies of research. First, studies in animals suggested there might be distinct visual systems subserving different behavioral goals. For example, on the basis of studies of motion perception in fish, Ingle (1967) proposed separate visual processes for evaluating object identity and helping to orient the fish in space. Similarly, Schneider (1967) reported a dissociation between visual processes subserving orienting and pattern recognition in hamsters with lesions to either superior colliculus or visual cortex, respectively. It was proposed that these distinctions arose from separate geniculostriate and retinotectal pathways (Schneider, 1969) that might also be present in primates (Trevarthen, 1968). Second, studies of human patients with focal damage revealed distinct visual impairments for lesions to the temporal or parietal lobes primarily affecting visual recognition (e.g., Benson, Segarra, & Albert, 1974; Milner, 1968) or visuospatial impairments (e.g., Butters, Soeldner, & Fedio, 1972; Ratcliff & Davies-Jones, 1972), respectively. Similarly, removal of inferior temporal cortex in monkeys produced a severe deficit in visual discrimination performance (Ungerleider & Pribram, 1977; Gross, 1973; Mishkin, 1972) whereas lesions of posterior parietal cortex produced deficits in spatial tasks (e.g., Ungerleider & Brody, 1977; Pohl, 1973). Collectively, this work suggested a general distinction between visual processes between spatial vision and visual discrimination that in primates seemed to culminate in the parietal and temporal cortex, respectively. However, whether this distinction reflected differential input from retinotectal and geniculostriate pathways was unclear.

In this context, the critical contribution of Ungerleider and Mishkin (1982) was to marshal a combination of behavioral, electrophysiological, and anatomical evidence to identify two diverging cortical pathways both originating in V1 that subserved object versus spatial perception. This formulation effectively relegated the retinotectal pathway to a subsidiary role in spatial perception, and a more primary role in orientation behavior. The critical experiments involved a comparison of performance on a “landmark” task, in which a monkey was rewarded for choosing one of two covered food wells that was closest to an object, with that on an object discrimination task, where reward depended on choosing one of two specific objects that differed in their visual features. Lesions of the parietal and temporal cortex selectively impaired performance on either the landmark or object discrimination task, respectively (Pohl, 1973). The origination of the pathways in V1 was demonstrated through a cross-lesion disconnection in which a lesion was made to either parietal or temporal cortex in one hemisphere combined with a lesion to striate cortex in the other hemisphere, limiting any communication between striate and parietal or temporal cortex to a single crossed pathway between hemispheres (Ungerleider & Mishkin, 1982; Mishkin, 1966). Impaired performance with either the V1 or high-level cortical lesion suggested these regions must be involved in the discrimination, but did not necessarily indicate that the pathway between them is critical. Such evidence came from the subsequent transection of the corpus callosum. This callosal transection effectively removed any connections between striate and extrastriate visual cortex, and the resulting behavioral deficits therefore reflected the existence of a pathway from striate cortex (Ungerleider & Mishkin, 1982).

Beyond demonstrating the existence of diverging pathways from V1, this body of work from Ungerleider, Mishkin, and colleagues also highlighted important differences between the dorsal and ventral pathways (Mishkin & Ungerleider, 1982; Ungerleider & Mishkin, 1982). First, the ventral pathway involves stronger connections between the hemispheres, that is, a greater contribution from the ipsilateral visual field. Second, lesions to medial and lateral striate cortex revealed that central vision is critical for the ventral pathway, whereas the dorsal pathway depends equally on both central and peripheral visions.

The functional importance of the two cortical visual pathways in macaques was subsequently confirmed by mapping metabolic activity elicited by direct retinal stimulation using [2–14C] Deoxyglucose (Macko et al., 1982), which also revealed projections from both pathways into frontal cortex (Figure 1A). Furthermore, the development of functional brain imaging in humans with PET provided evidence for two separate pathways in a human with distinct changes in rCBF in parietal and occipito-temporal cortex for spatial location and object (face) identity tasks, respectively (Ungerleider & Haxby, 1994; Haxby, Grady, Horwitz, et al., 1991; Haxby, Grady, Ungerleider, & Horwitz, 1991).

The two cortical visual pathways that emerged from this work are often depicted as arrows emerging from V1 and projecting into the parietal or temporal cortex. Such an illustration is common in textbooks and talks and can be found on Wikipedia (https://en.wikipedia.org/wiki/Two-streams_hypothesis). Yet, we think it is important to realize that this has been an evolving framework and that since the initial accounts, much work has been conducted to flesh out the details of the pathways, both anatomically and functionally. The picture of the two pathways that exists now is much richer and nuanced than the early accounts of a neuroanatomical bifurcation post-V1 and a functional dissociation between recognition and spatial perception.

EVOLUTION OF THE TWO CORTICAL PATHWAYS FRAMEWORK

Following the original proposal, the two pathways model was rapidly elaborated as the neuroanatomical connections were mapped out in greater detail and the associated functional literature expanded and matured. Here, we examine the consequences of this evolution in our understanding of the internal and external connectivity as well as connections between the pathways.

Internal Connectivity: From Serial Hierarchies to Recurrent Circuits

The connections of V1 (e.g., Rockland & Virga, 1990; Ungerleider & Desimone, 1986b; Van Essen, Newsome, Maunsell, & Bixby, 1986; Livingstone & Hubel, 1983; Rockland & Pandya, 1979; Ungerleider & Mishkin, 1979; Zeki, 1978), V2 (e.g., Gattas, Sousa, Mishkin, & Ungerleider, 1997), V4 (e.g., Ungerleider, Galkin, Desimone, & Gattass, 2008; Nakamura, Gattass, Desimone, & Ungerleider, 1993), and MT (Ungerleider & Desimone, 1986a) were all carefully mapped, as well as connections to and among areas within the parietal and temporal cortex (e.g., Galletti et al., 2001; Distler, Boussaoud, Desimone, & Ungerleider, 1993; Saleem, Tanaka, & Rockland, 1993; Baizer, Ungerleider, & Desimone, 1991; Colby, Gattass, Olson, & Gross, 1988). Slowly, the view of the two cortical pathways changed from simple diverging pathways emerging from V1 (Figure 1A) to distinct chains of connections projecting into the parietal and temporal cortex (Figure 1B) to the complex wiring diagram depicted by Felleman and Van Essen (1991; Figure 1C).

These different visualizations of the cortical visual pathways capture distinct aspects of the underlying anatomical data but are also somewhat qualitative, emphasizing particular interpretations. The simple depiction of the bifurcation of the projections from V1 (Figure 1A) emphasizes a separation of two cortical pathways, implying they are completely independent. This view led many to assume there are limited or no connections between the pathways, despite the fact that the emerging anatomy revealed a more integrated picture. The staged series of regions (Figure 1B) captures more of the underlying anatomy but also implies a discrete series of processing stages. Furthermore, the location of those regions across the cortical sheet suggests a larger scale organization and has typically been used to distinguish the dorsal and ventral pathways. In this context, it is interesting to consider the typical placement of the middle temporal area (MT), which is often shown closer to parietal regions (see the Interconnectivity between the Pathways section, below) and thus incorporated into the dorsal pathway (e.g., Figure 1 in Ungerleider, 1995). Felleman and Van Essen (1991) tried to capture the laminar patterns of the connectivity, using them to place individual regions at distinct levels of a hierarchy (Figure 1C). This depiction emphasizes hierarchical aspects of the pathways, but it is important to realize that this does not necessarily support the notion of serial processing: “The possibility that the visual cortex might operate by a strictly serial processing scheme can be ruled out just from knowing the multiplicity of connections per area and the near ubiquity of reciprocal connections” (Felleman & Van Essen, 1991).

Although many of the visualizations of the two pathways laid out the areas based on qualitative features and theoretical considerations, Young (1992; Figure 1C) conducted a quantitative analysis based on the connections reported by Felleman and Van Essen (1991). This analysis revealed the broad distinction between ventral and dorsal pathways, although the analysis itself included no information about spatial location of the regions.

We reviewed and summarized the known anatomical connections for both dorsal (Kravitz et al., 2011) and ventral pathways (Kravitz et al., 2013), depicting the pathways as complex webs of highly interconnected regions (Figure 1D and E). This synthesis indicated that not only are there reciprocal connections between areas; those connections and the “feed-forward” connections also skip intermediate levels of the putative hierarchy. Thus, although each of the cortical visual pathways are often described as serial hierarchies, it is clear from the connectivity that this is an oversimplification and they are better described as complex recurrent networks. For example, V1 projects not just to V2, but also to V3, V4, MT, and V6. Along the dorsal pathway, areas within the parietal cortex and caudal superior temporal areas are all highly interconnected (Figure 1E). Along the ventral pathway, there are also connections that skip intermediate regions such as direct connections between V4 and area TE (Ungerleider et al., 2008), or between area TEO and anterior TE (Distler et al., 1993). Furthermore, although extensive damage to V4 and area TEO disrupts some functional properties of area TE, the basic firing rate and selectivity of the neurons remain largely unaffected (Buffalo, Bertini, Ungerleider, & Desimone, 2005; Bertini, Buffalo, De Weerd, Desimone, & Ungerleider, 2004). More recently, it has been shown that lesions of TE but not TEO impair visual recognition (Eldridge et al., 2023). Furthermore, for visual categorization, bilateral removal of TEO produces a milder deficit (Setogawa, Eldridge, Fomani, Saunders, & Richmond, 2021) than removal of TE (Eldridge et al., 2018; Matsumoto, Eldridge, Saunders, Reoli, & Richmond, 2016) and combined TE + TEO removals produce a deficit in categorization roughly the sum of deficits observed following the removal of either TE or TEO alone. Collectively, these results suggest the presence of pathways to TE that do not pass through TEO with TEO and TE involved in parallel processing (Setogawa et al., 2021).

In summary, the reciprocal connectivity within pathways, connections that skip putative levels, and the resulting redundancy in representation all point to a highly recurrent circuit rather than a strict serial hierarchy.

External Connectivity: From Hierarchies to Heterarchies

In early descriptions, the outputs from the ventral and dorsal pathways were given as projections arising from the most anterior portions of each pathway to ventral and dorsal portions of the lateral pFC, respectively (Figure 1A). That naturally led to the view, particularly for the ventral pathway, of a series of “feed-forward” processing stages that culminated in a single complete representation that was then passed to the pFC. This view has proved extremely influential and has been fundamental to numerous theoretical and computational models of object vision (e.g., DiCarlo, Zoccolan, & Rust, 2012; Riesenhuber & Poggio, 1999).

However, our examination of the anatomical outputs of the two pathways directly challenges this view (Kravitz et al., 2011, 2013). From the ventral pathway, we identified six major connections, three to subcortical structures (neostriatum, ventral striatum, and amygdala; Figure 2A) and three to cortical regions (medial temporal lobe, orbitofrontal cortex, and ventrolateral pFC). From the dorsal pathway, we identified three major connections: to premotor and pFCs and to the medial temporal lobe via posterior cingulate and retrosplenial cortices (Figure 2B). This last connection is well conserved, being present even in rodents, and densely and directly connects the two pathways. These projections do not arise from a single region along each pathway but rather from heterogenous sets of regions. The projections from the ventral pathway to the striatum and from the amygdala to the ventral pathway essentially involve every region. In no sense does this pattern of connectivity suggest that the pathways come to a singular point. Moreover, the cortical areas and subcortical structures that receive these projections subserve heterogenous functions that contribute to a diversity of behaviors.

Figure 2. .

External connections of ventral and dorsal pathways. (A) Connections from the ventral pathway to striatum (top) and amygdala (bottom; Kravitz et al., 2013). (B) Three major sets of connections emerging from the dorsal pathway (Kravitz et al., 2011).

For example, the premotor areas (F2/F7, F4/F5), which are connected with dorsal pathway areas (ventral intraparietal area [VIP], V6A/MIP), are known to directly contribute to motor planning, implying that there are representations within the dorsal pathway areas that will be at least somewhat optimized to contribute to visually guided action. Goodale and Milner (1992) and Milner and Goodale (1995) in fact proposed that the dorsal pathway is best characterized as “vision for action” or the real-time guidance of the animal's movements, whereas the ventral pathway is “vision for perception” or recognition for identifying goals and threats (Goodale & Milner, 2018; Milner & Goodale, 2008). Strong support for this alternate view was provided by the study of patient D.F. who suffered brain damage primarily to the lateral occipital cortex with sparing of V1 (Bridge et al., 2013; James, Culham, Humphrey, Milner, & Goodale, 2003). D.F. had visual form agnosia and prosopagnosia with impaired ability to perceive shape and orientation information. Strikingly, D.F. was able to orient her hand correctly to post a card through a slot despite being unable to report the orientation of the slot, suggesting separate processing systems “not for different subsets of visual information, but for the different uses to which vision can be put.” (Goodale, Milner, Jakobson, & Carey, 1991). Subsequent fMRI data showed no evidence for object-selective activation in occipito-temporal cortex despite robust activation in parietal cortex during object grasping (James et al., 2003). This view crucially highlights the convergence between natural visually guided behavior, gross connectivity, and the localization of function.

It is important to consider, however, that other dorsal pathway regions are also densely connected to the lateral pFC and medial temporal lobe, which play a critical role in high-level cognition and navigation as well as the coordination of egocentric and allocentric representations of the environment, respectively (see Kravitz et al., 2011, for related evidence).

The external connections of the ventral pathway are even more complex, with heterogeneous sets of cortical regions connected with more specialized areas and structures supporting a range of complex behaviors (e.g., implicit/explicit memory, navigation, representing value, generating emotion). We would expect, and in fact observe, representations in these regions of the ventral pathway that reflect their contribution to these distinct behaviors (e.g., Steel, Billings, Silson, & Robertson, 2021; Kragel, Reddan, LaBar, & Wager, 2019; Epstein, 2008). Different regions within these pathways have different patterns of internal and external connectivity, leading to distinct, but equally complex and necessary, representations. The purpose of these circuits is not to come to a single representation, but to produce a diversity of representations that can contribute to processing in a similarly diverse set of external circuits and, through that contribution, help generate the wide variety of adaptive behaviors primates exhibit. Thus, the pathways are best conceived of as recurrent heterarchies rather than serial hierarchies, where representation within any particular area is constrained and informed by the representations in the others to which it is connected, but there is no necessary rank ordering among them.

In summary, the heterogeneous external connectivity of the dorsal and ventral pathways point to the generation of multiple representations that can support a wide variety of behaviors and not just recognition or visuomotor action.

Interconnectivity between the Pathways: From Isolation to Coordination

The original instantiation of the two pathways model focused on a bifurcation emerging directly from V1. This description presented the pathways as essentially distinct from one another, with dense internal connectivity, but little to no connections between them. That apparent modularity supported a concomitant division of the field and of the putative representations predicted to be found within them. For example, object representations were proposed to be independent of spatial information, despite the fact that objects consistently occur at certain retinotopic sizes and positions (e.g., Chan, Kravitz, Truong, Arizpe, & Baker, 2010; Kravitz, Kriegeskorte, & Baker, 2010; Kravitz, Vinson, & Baker, 2008). Parietal cortex was held as insensitive to shape, despite the fact that grasping clearly requires that information (e.g., Freud, Plaut, & Behrmann, 2016; Culham & Valyear, 2006; see also Vaziri-Pashkam & Xu, 2019; Konen & Kastner, 2008, for studies that show sensitivity to shape distinct from grasping). In essence, it provided an attractive but overly simple way to segment the field and a number of clear and testable predictions about the functional response to be found in certain regions and the impact of damage to them on behavior. Both neuroanatomical and functional evidence now highlight the extensive coordination of the dorsal and ventral pathways.

To begin, consider motion-selective area MT, which is often depicted as part of the dorsal pathway, spatially distinct from early visual areas (Ungerleider, 1995). Notably, Felleman and Van Essen (1991) placed MT in the middle of their connectivity diagram and it is clear that it has a connectivity profile across both dorsal and ventral regions (Figure 1C). A computational analysis of cortical connections (Young, 1992) found that it associated more closely with V1–V3 than with either parietal or temporal cortical regions. MT receives direct projections from V1 (Ungerleider & Mishkin, 1979), and although it is densely connected with many dorsal pathway regions such as the lateral intraparietal area (LIP) and VIP, it is also connected with V4 and with regions in and around the posterior STS, which project down the banks of the STS and into anterior TE (Boussaoud, Ungerleider, & Desimone, 1990). From a functional perspective, movement can directly define form (e.g., Robert, Ungerleider, & Vaziri-Pashkam, 2023; Sáry, Vogels, & Orban, 1993) and is a key aspect of many visual objects and can even be used to identify individuals (e.g., Troje, Westhoff, & Lavrov, 2005). These functions can also inform each other; recognizing a rigid form underlying the motion of apparently distinct objects both simplifies the processing and prediction of the movement and constrains the recognition of those objects. It was recently proposed that the projection from V1 to MT and then down the banks of the STS be considered a distinct third visual pathway that is specialized for social perception (Pitcher & Ungerleider, 2021).

Beyond the central role of MT, there is a wealth of connections between the pathways that can serve to usefully coordinate their responses. In particular, in humans, a major fiber bundle, the vertical occipital fasciculus connects parietal with occipito-temporal cortex (Takemura et al., 2016; Yeatman et al., 2014). In monkeys, TEO is connected with LIP and V3A (Webster, Bachevalier, & Ungerleider, 1994), and the pulvinar may provide a relay between LIP and V4 (Saalmann, Ly, Pinsk, & Kastner, 2018). Finally, one of the three major projections of the dorsal pathway courses to the medial temporal lobe via posterior cingulate and retrosplenial cortices and may help support navigation (Kravitz et al., 2011).

In summary, although the original description of the two pathways emphasized a neuroanatomical and functional bifurcation, it is now clear that there are multiple connections between the two pathways, with MT potentially acting as a major hub for relaying motion information to both.

RECONCEPTUALIZING THE TWO PATHWAYS FRAMEWORK: FROM PATHS TO MEDIATORS

Having rejected several of the simplifying assumptions that have accompanied its evolution, it is worthwhile to restate the key elements of the two cortical pathways framework and update our view of them. Far from a simple neuroanatomical and functional bifurcation of visual processing, the two pathways are recurrent heterarchies with heterogeneous connections to external cortical regions and subcortical structures. Grasping this complexity requires us to jettison two metaphors that arose from the original descriptions. First, the pathways are not highways, passively receiving and moving information between cortical areas. Each synapse is a learned nonlinear filter acting on the response of the presynaptic neuron, leading to a transformation of information rather than a relay. Second, the pathways are not assembly lines, iteratively and sequentially building a singular “high-level” representation. Beyond the clear recurrence, the response of each cortical region is unique, potentially useful in and of itself depending on the current goal, and many are directly connected to regions and structures external to the pathways. Both metaphors are also entirely too passive, viewing the pathways as only recipients of visual information, rather than being actively involved in directing how that information is sampled through, for example, attention and eye movements.

Each visual region and the pathways themselves are better conceptualized as mediating the integration of responses arising from the cortical regions and subcortical structures to which they are connected. The dorsal pathway connects the early visual, somatosensory, premotor, and lateral pFCs, as well as the medial temporal lobe. Its internal functional organization reflects the integration of responses from those areas, as do the behaviors it critically supports (e.g., reaching/grasping, eye movements, spatial reasoning, navigation). The ventral pathway connects the early visual, orbitofrontal, and lateral pFCs, as well as the striatum, amygdala, and medial temporal lobes. Its functional organization reflects those connections, as do the behaviors it critically supports (e.g., recognition, memory, establishing value). Responses arise both within the pathways and from primary visual cortex and other external regions and structures constantly and are integrated, resulting in distributed patterns of response that are reinforced to the degree they support adaptive behavior. The functional response of any region within the pathways is therefore the result of an optimization to efficiently produce an integrated response (representation) from the responses of the regions and structures to which it is connected.

This view places the unique connectivity pattern at the center of understanding functional response from single cells, to cortical regions, to broad cortical circuits like the pathways themselves. Connectivity patterns anticipate many aspects of functional organization within the pathways such as retinotopic biases in extrastriate areas (e.g., Silson, Chan, Reynolds, Kravitz, & Baker, 2015; Kravitz et al., 2010), the response properties of different category-selective regions (e.g., Harel, Kravitz, & Baker, 2013), responses in inferior parietal lobule (Rozzi, Ferrari, Bonini, Rizzolatti, & Fogassi, 2008), and the impact of retrosplenial cortex lesions (Hashimoto, Tanaka, & Nakano, 2010). Moreover, the external regions and structures to which the pathways are connected are very distinct and simultaneously converge with and lend converging evidence to their unique role in visual processing. The dorsal pathway targets in the medial temporal lobe, and pFC plays critical roles in immediate action and short-term representations (e.g., orienting oneself in an environment). Ventral pathway targets generally support longer-term memory and represent more stable, general applicable aspects of the environment (e.g., cues that anticipate rewards).

The idiosyncrasies of the representations within each region, combined with the interpathway connections, does beg the question of the utility of broad pathways as an organizing framework. Our view is that the pathways remain relevant because intrapathway connectivity is denser than interpathway or external connectivity. Regions within each pathway will have stronger impacts on each other and therefore more related responses and functions. When considering inferences and models that operate at that broad scale (e.g., lesion patients), the pathways are directly relevant and, even when considering the role of individual cortical regions, they are highly likely to serve related functions to other regions in the same pathway because of the denser connectivity. However, we must also recognize the limitations of the pathways, as any broad characterization will necessarily have exceptions that we must recognize and leverage, rather than downplay in service of simpler accounts. Ultimately, we are seeking theories that integrate the heterogeneous connectivity and functional responses across these pathways while simultaneously recognizing the role individual regions play within the larger circuits.

CHALLENGES, OPPORTUNITIES, AND OUTLOOK

The state of the two cortical pathways framework is still strong, but it will need to evolve further even as it informs where future research efforts should be directed. Here, we outline some critical gaps that need to be addressed and how we can advance the framework and our understanding of primate vision.

Capturing Complexity in Our Accounts and Models

Placing connectivity at the center of our understanding implies that it should be better integrated into our theories and formal models of visual processing. Yet, many current models center feed-forward processing toward a singular representation that supports a particular behavior. For example, the vast majority of deep neural networks are largely homogenous feed-forward hierarchies trained on a single task, often recognition. There is increasing recognition that some form of recurrence improves the fit of the models to neural data, but the implementation of such recurrence is still very limited (e.g., Sexton & Love, 2022; Kar & DiCarlo, 2021; Kar, Kubilius, Schmidt, Issa, & DiCarlo, 2019; Kietzmann et al., 2019). Although these models can offer important insights into the general utility of hierarchical processing and how neural networks can learn and be trained and have numerous proven pragmatic applications, they cannot capture the functional organization of cortical visual processing without better integrating the details of the neuroanatomy, the complexity and constraint inherent in recurrence, the need to support multiple qualitatively distinct forms of behavior, and the role of these areas in guiding active sampling of the environment. On the other hand, although challenging, the integration of this complexity can, and likely will, yield theories and models capable of explaining and producing more useful natural behaviors.

What Constitutes a Pathway: The Quest for the “Right” Description

Many have questioned whether the two cortical pathways framework is correct (e.g., de Haan & Cowey, 2011) and alternative frameworks have been proposed, often suggesting more than two pathways (Ritchie, Montesinos, & Carter, 2024). In some cases, these accounts have been grounded in the neuroanatomy, such as the suggestion of a third pathway passing down the banks of the STS that could be considered a motion processing pathway that integrates form and motion information (Boussaoud et al., 1990) and might be critical for social perception (Pitcher & Ungerleider, 2021). However, given our limited knowledge of the underlying human neuroanatomy and the limitations of techniques such as diffusion tractography (Maier-Hein et al., 2017; Thomas et al., 2014), these alternative models are often particularly underconstrained in humans. Going forward, we need more detailed and complete descriptions of the connectivity to establish thresholds for the relative levels of intrapathway and external connectivity necessary to define a pathway and place a region within it.

Other alternative pathway models have been based primarily on human neuroimaging studies where the underlying neuroanatomy is much less clear. For example, using a similar approach to Young (1992) but applied to resting-state MRI data, Haak and Beckmann (2018) reported evidence for a third pathway coursing from human V3 into LO1 and LO2 and then into MT, that is, separate from a dorsal and ventral pathway. Results from fMRI studies have also been used to argue for a lateral pathway that might play a role in action processing (Weiner & Grill-Spector, 2013) and could be considered a second what pathway (Wurm & Caramazza, 2022). Other analyses using MRI and magnetoencephalography data have argued for multiple distinct dorsal and ventral pathways (e.g., Rolls, Deco, Huang, & Feng, 2023; Rolls, Deco, Zhang, & Feng, 2023). However, casting the pathways as recurrent heterarchies whose internal processing adapts to the current goals of the organism implies that any functional measure will yield a characterization of the pathways somewhat specific to the context in which it is collected, including even rest. That specificity does not invalidate the observations or their potential importance, but it does suggest that the quest for a singular “right” description of these pathways, at least in purely functional data, is very challenging. The lack of converging evidence from human neuroanatomy is a serious limitation, providing further impetus for further research in that area.

Building on the Core Insight

Because the two pathways framework was originally proposed, the evidence collected forces the rejection of many of its details. The neuroanatomy, functional response, and adaptive behaviors supported by visual processing all defy any simple bifurcation. Pathways that were originally proposed as largely “feed-forward” processing streams have proven to be recurrent heterarchies whose functions result from and are best understood in terms of continuous online constraint from external regions and structures. However, even as the maturing science strips away the simplifying assumptions that undoubtedly contributed to the framework's uptake and popularity, we are left with a core insight—that functional response and the concomitant role a region plays in adaptive behavior is defined by its connectivity.

The space of adaptive visual behavior may well be unbounded, and functional response is difficult to measure, highly dynamic, and meaningfully operates on mutually constrained levels of spatiotemporal scale ranging from channels to lobes. In contrast, neuroanatomy, particularly the gross pattern of connectivity, as far as we currently understand it, is largely stable and can be measured with a relative lack of ambiguity, particularly in animal models. The convergence between connectivity and function gives us a place to begin, a basis for predicting the organization of the cortex, and thus a way to motivate and organize the field.

As we recognize the importance of connectivity, we must also appreciate that our knowledge of it remains woefully incomplete. Most of our current knowledge stems from laborious and highly localized tracer studies done in animal models. Although the data from these studies are largely unambiguous, it is far from complete even in nonhuman primates. Furthermore, gross connectivity observed in nonhuman primates is likely to be conserved in humans, but there are certainly additional connections. It is also necessary to understand and quantify variation in connectivity across individuals, which should predict, at least in part, concomitant variation in function and behavior, allowing for tighter relationships to be discovered. Thus, there is a great need for methods that can yield more efficient and complete quantifications of anatomical connections, particularly in the human.

In summary, although many of simplifying assumptions and some of the specific theories and accounts arising from Ungerleider and Mishkin's work have been challenged and may be misleading, the work would never have proceeded as productively had they never articulated the fundamentals of the neuroanatomy of cortical vision. Even as the two pathway framework necessarily evolves, it remains clear that the field must and will continue to expand on this work and push the boundaries of functional neuroanatomy. As we mark the passing of Leslie Ungerleider, it is clear that her career is what every cognitive neuroscientist should aspire to. Our theories are often limited by the evidence available at the time and ultimately, we can only hope to be partially and not fully wrong. However, if we approach the work in the right way, with an unwavering insistence on rigor and clarity, even our partial successes will be extremely informative.

Acknowledgments

We thank Dr. Shruti Japee for comments on earlier drafts of the article.

Corresponding authors: Chris Baker, Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892, or via e-mail: bakerchris@mail.nih.gov; or Dwight Kravitz, Department of Psychological and Brain Sciences, The George Washington University, Washington, DC 20052, or via e-mail: kravitzd@gwu.edu.

Author Contributions

Chris Baker: Conceptualization; Writing—Original draft; Writing—Review & editing. Dwight Kravitz: Conceptualization; Writing—Original draft; Writing—Review & editing.

Funding Information

Chris I. Baker is supported by the Intramural Research Program of National Institute of Mental Health, grant numbers: ZIA MH002909 and ZIA MH002893. Dwight J. Kravitz is supported by the National Science Foundation (https://dx.doi.org/10.13039/100000001), grant number: BCS-2022572/Division of Behavioral and Cognitive Sciences and the Army Research Office (https://dx.doi.org/10.13039/100000183), grant number: W911NF-16-1-0274.

Diversity in Citation Practices

Retrospective analysis of the citations in every article published in this journal from 2010 to 2021 reveals a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender identification of first author/last author) publishing in the Journal of Cognitive Neuroscience (JoCN) during this period were M(an)/M = .407, W(oman)/M = .32, M/W = .115, and W/W = .159, the comparable proportions for the articles that these authorship teams cited were M/M = .549, W/M = .257, M/W = .109, and W/W = .085 (Postle and Fulvio, JoCN, 34:1, pp. 1–3). Consequently, JoCN encourages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the opportunity to report their article's gender citation balance. The authors of this paper report its proportions of citations by gender category to be: M/M = .676; W/M = .099; M/W = .127; W/W = .099.