Spatial representation across species: Geometry, language, and maps

Summary

We review growing evidence that the reorientation system-- shared by both humans and non-human species-- privileges geometric representations of space and exhibits many of the characteristic features of modular systems. We also review evidence showing that humans can move beyond the limits of non-human species by using two cultural constructions, language and explicit maps. We argue that, although both of these constructions are uniquely human means of enriching the spatial system we share with other species, their representational formats, functions and developmental trajectories are quite different, yielding distinctly different tools for empowering human spatial cognition.

Introduction

Recent studies have shown that many species reorient using geometric representations of spatial layouts, often ignoring highly salient non-geometric information. These findings have led to the hypothesis that the reorientation mechanism is modular, in Fodor's [1] sense [2, 3, 4], i.e. that the mechanism underlying reorientation is designed to compute geometric layout, and is blind—or impenetrable—to salient non-geometric information about layout. The capacity to reorient using geometry is present in humans by the age of 18 months, but it undergoes significant change over development, with increasing ability to reorient using a combination of geometric representations of the layout together with non-geometric information. This developmental change raises intriguing questions about the possible role of language in building new, more powerful devices to represent space. Here, we review the special role of geometric representations in reorientation across a variety of species, including humans. We then evaluate several hypotheses about the role of language in human reorientation, and suggest that the formalism of language offers humans a unique tool to strengthen an existing capacity to integrate geometric and non-geometric information. Finally, we compare the role of language to another cultural construction: the map. We argue that, although both language and maps are uniquely human capacities that enrich the spatial system we share with other species, the representational formats, functions, and developmental trajectories of the two systems are quite different, yielding distinctly different tools for empowering human spatial cognition.

Geometry is special: A capacity for reorientation that is shared across species

The idea of a geometric module for reorientation was first proposed by Cheng [2] and Gallistel [3] on the basis of experiments in which rats were familiarized to a small rectangular environment whose corners had been baited with food, were then disoriented and finally returned to the environment, where they were allowed to search for the food. Search patterns showed an unusual signature: rats tended to split their searches between the correct corner and its geometric equivalent, ignoring the salient markings of the corners (Figure 1). This pattern indicates that the rat's reorientation mechanism engages a representation of the overall geometric structure of the environment—the lengths of the walls, their angles of intersection, and their sense relations-- but ignores salient surface markings. The signature geometric search pattern has since been replicated in many species, including chicks [5], fish [6], rhesus monkeys [7], pigeons [8], and human children and adults [4, 9, 10]. However, many of these studies also show that, under a variety of conditions, non-geometric information can be used along with the geometric representation, leading to search at the uniquely correct corner of the space. For some, this has challenged the idea of a geometric module, whose support has largely depended on the single criterion of impenetrability—i.e. blindness to non-geometric features [11]. More crucially for our purpose, the findings have raised new questions about the nature of the reorientation system that is shared by all species as well as the differences that may uniquely distinguish humans from other species.

Figure 1. Illustration of the experimental set-up and geometric pattern of search reported in [2].

In this testing environment, the location of the food is fully specified by the surfaces’ salient visual features and odors. However, after being familiarized to the testing environment, removed and disoriented, and then returned to the testing environment to search for the food, rats searched primarily the correct corner and the geometrically equivalent corner (as depicted by the ‘x’ in the illustration) (recreated from Cheng, 1986).

Considerable evidence now suggests that reorientation is a specialized system that privileges geometric representations of spatial layout. First, the geometric shape of layouts is primary, in the sense that it is used across a wide variety of species and does not depend on experience with layouts having particular lengths of surfaces or angles of intersection [12]. Whether chicks are raised in a rectangular or a circular environment, they spontaneously encode the geometric properties of both environments during later tests of reorientation. Second, geometric representations are engaged under functionally specific contexts, with more use of geometry in tasks requiring locomotion to a target than reaching to a target in tabletop arrays, which does not require locomotion to a goal [13, 14]. Geometry is used more often when people are inside a space than when they view the space as an observer from the outside, and performance is better when the viewer moves within the array than when the array moves [15, 16]. Third, the effective array must have extended surfaces: even a 30 cm high surrounding surface elicits geometric responses among children, but a geometrically equivalent array composed of large individual columns does not, nor does an equivalent structure formed by tape on the floor [17]. Fourth, although most tests of geometric reorientation have used spaces that are characterized by rectangular enclosures—which differ in relative lengths of the four walls—geometric responding may be computable with values on other relative scales, including different dot size, but not different colors [18]. This finding suggests that relative comparisons along a single spatial scale may be sufficient to induce geometric responding.

Finally, reorientation using geometric representations of layouts appears to have neural specificity, with the hippocampus as a potential site. Rats [19] and pigeons [20] with hippocampal lesions are impaired in using geometry for reorientation, with specific impairment in rats linked to encoding the relative lengths of walls [21]. The right and left hippocampal areas play different roles in an animal's use of geometric vs. non-geometric properties of space [22, 23, 24], consistent with evidence from humans [25]. Hippocampal place cells have long been implicated in the representation of spatial layout in the rat [26], and recent studies show that these place cells come to gradually and incrementally respond to the overall shape of an environment (e.g. round vs. square) during learning trials, with specific shape-related response patterns persisting over time [27]. Thus place cells appear to “learn” the shapes of layouts, raising the intriguing possibility that this is part of the neural substrate for the mechanism of reorientation, Another contributor may be recently discovered “grid cells” in the medial entorhinal cortex, which appear to encode distributions of regularly spaced locations across the environment [28]. Understanding the relationship between grid cells and place cells will be critical to developing theories of the specific neural mechanisms underlying the construction and use of cognitive maps [29, 30].

Although the evidence for neural specificity in humans is sparse at present, lesion studies suggest that topographical disorientation (the inability to orient in the environment) can occur with damage to a variety of regions of the brain, including the hippocampus, parahippocampal regions and parietal regions [31, 32]. Recent studies suggest that hippocampal functions play a primary role in the formation of cognitive maps, both accounting for variation in the speed of acquisition of new maps among normal individuals [33] and selective impairment in acquiring new maps in a case of developmental disorientation [34]. Studies of people with Williams syndrome, a genetic deficit giving rise to severe spatial impairments and damage to both parietal and hippocampal regions [35, 36, 37] show severely impaired use of geometry in the reorientation task (L Lakusta, B Dessalegn, B Landau, Cognitive Development Society, Santa Fe, October, 2007) (see also [38] for evidence of selective impairment in geometry-based but not feature- based reorientation in neglect patients). Because Williams syndrome is a genetic disorder, scientists are attempting to link specific genes with aspects of the spatial disorder. Although we are far from understanding this complex causal chain, intriguing studies show that Limk1 knockout mice are impaired in the classic Morris water maze, which tests place learning [39]. These mice also have abnormalities of hippocampal neurons, despite overall grossly normal development of the nervous system.

These results clearly implicate the role of the hippocampus in reorientation among humans, but considerably more research will be needed to distinguish effects of hippocampal damage per se from damage to other areas contributing to the reorientation mechanism, such as the parietal and parahippocampal regions. The use of animal models to probe the relative contributions of hippocampus and other structures in the formation and use of shape-based representations of the environment should be helpful in guiding specific hypotheses about humans.

It is worth noting that some of these properties fall within the criteria that Fodor [1] proposed for modular systems, in particular, domain-specificity, localization, ontogenetic invariance, and characteristic breakdown patterns. Although much debate over the modularity of reorientation has focused on the criterion of penetrability, Fodor's criteria were intended to be characteristic—not defining-- properties of modules. The properties described earlier suggest that the reorientation system clearly has some properties of modular systems (but see [11] for a different view). Understanding whether and how these properties change over development and across species could shed significant light on the nature of modular systems across species more generally.

Language and its role in reorientation

The privileged nature of geometry in reorientation contrasts to an interesting degree with the incorporation of non-geometric information. The conditions under which this occurs appear to be more variable, depending on settings such as the size of the space [6, 10, 11, 40, 41], the proximal vs. distal location of the feature within the space [11], and the location of the target relative to the feature [11]. Increasingly consistent integration of geometric and non-geometric properties over development has suggested to some that language could play a crucial role in this change. One hypothesis is that language provides a representational system which allows information from separate modular systems—in this case, the reorientation system and the object representation system-- to be combined in some common format [4, 42, 43, 44]. On this proposal, non-verbal species and human toddlers who search geometrically (and do not use non-geometric information) would be unable to combine properties that are the output of separate modules (geometric structure/ sense and object/color); whereas older children and adults would be able to do so by virtue of having the representational apparatus of language, which could produce a new representation of the target's location such as [[LEFT-OF [BLUE WALL]]. In its strong form, the hypothesis suggests that language is both necessary and sufficient to change the computations carried out during the reorientation task. If true, this would result in a vast restructuring of human spatial representation, conferring enormous additional power to a system that would otherwise only be capable of producing outputs from each system without combining them.

Considerable evidence casts doubt on this strong version of the language hypothesis. Empirical findings show that non-verbal species can integrate geometry and features [6, 7, 8, 11, 41, 45], so language is clearly not necessary for integration. Moreover, evidence supporting the role of language among children and adults has been equivocal: Verbal shadowing interferes with the use of features in some studies [42] but not others [46] while spatial shadowing interferes in other studies [46, 47]. Finally, evidence from 4−5 year olds showed that the production of “left” and “right” – but not comprehension of these terms, nor a range of other variables-- was correlated with the use of geometry and surface features [44]. It is unclear why production but not comprehension would be correlated if language were necessary to provide the appropriate (combinatorial) representational format. Being able to produce the expressions “X left of Y/ right of Y” requires a representation that should also underlie comprehension.

A different hypothesis about the role of language is that the reorientation system allows combination of geometry and non-geometrical properties, but that some combinations are either unstable or impoverished, leading to difficulty forming the representation or rapid forgetting, for example. The combinatorial properties of syntax and semantics could then be used on-line to enhance this representation or boost its natural life in working memory. This hypothesis of Momentary Interaction [48, 49] suggests flexible and powerful enhancement of spatial representations through language, but not radical restructuring.

To see how this could work requires two steps. First, the presence of the geometric error pattern among toddlers suggests that they already possess a non-linguistic representation of both relative wall length (or other properties along a scale [18]), and sense relations between pairs of wall lengths. This representation would include [X, LONG WALL], [Y, SHORT WALL], and [X LEFT-OF Y], [X RIGHT OF Y], where the latter relationships are defined by the relationship between the body's axes and those of the layout. Combining these, one can represent [[LONG WALL] [[LEFT-OF] [SHORT WALL]]]. This representation would be sufficient to locate the target in one of two geometrically equivalent corners, yielding the geometric error. Human toddlers can also represent objects and their non-geometric properties, e.g. [BLUE WALL] and indeed, can use these features to locate targets when they are not disoriented [4]. Language is therefore not necessary to encode the sense relationship between the two walls, nor is it necessary to encode object/color. If one assumes that non-linguistic representations are compositional, then language would not be necessary to combine the two either. However, language can express the two together in a single phrase, and this linguistic format might be more enduring and stable than the corresponding non-linguistic representation. The linguistic representation [[LEFT-OF [BLUE WALL]], could in this way facilitate search using both geometric and non-geometric properties.

Insights into how this would work can be drawn from studies examining the more general problem of how humans combine location and surface color information. Studies in perception and attention show that color- location conjunctions are represented in fragile form by the visual system, that is, binding of the conjoined features requires focused attention and is subject to disintegration under conditions of attentional impairment [50, 51, 52]. Recent studies further suggest that, over development, language plays an increasingly powerful role in promoting the strength, stability, and durability of representations for this kind of conjunction [49]. Four year-olds were shown a target square that was split in half by color (e.g. vertically split with red on the left and green on the right) and had to match it to its identity after a 1-second delay. Test items included the target, its reflection (e.g. red right, green left), and a square with a different geometric split (e.g. a diagonal split, half red and half green, see Figure 2). Children performed above chance, but they chose the target only about 60% of the time (with a chance level of 33%). Errors were predominantly the reflection items—consistent with robust representation and retention of the geometry (the vertical split) but quite fragile retention of the color-location combination. Follow-up experiments showed that this pattern of performance was robust, even when non-linguistic attention manipulations were used to draw attention to the location of the red part, e.g., flashing it on and off, growing and shrinking it, or having the child point to it before it disappeared. In contrast to these non-linguistic manipulations, introducing the square with the instruction “The red is to the left of the green” enhanced children's ability to match after delay, raising their accuracy to about 80%. Comparison with other linguistic manipulations revealed two critical variables: The syntax of the sentence, which provides an asymmetrical frame establishing which part of the display is figure and which part is reference object [53, 54], and the directional value of the predicate [X LEFT-OF Y] that relates the figure and reference object and establishes an asymmetrical direction (i.e. if X is left of Y, then Y cannot be left of X). Sentences such as “The red is touching the green”—while very similar in structure, did not enhance performance, showing that the asymmetrical or directional value of the predicate [LEFT-OF] was crucial. Importantly, post-tests showed that children knew that “left” and “right” referred to locations along the horizontal axis of a display, but did not know which term mapped onto which direction, ruling out the idea that they had retrieved a complete and correct meaning of [X LEFT-OF Y] from their long-term lexical memory. Rather, the instruction “The red is to the left of green” helped 4 year-olds represent the correct color-location pairings in the moment of the task, essentially telling them which part was figure and which was reference object, and which direction was called “left” (or “right”). The explicit representation of the color-location information through the instructional sentence was sufficient to improve 4 year-olds’ performance, suggesting the power of momentary, on-line interactions between language and visual attention, consistent with recent evidence from adults [55] (B Dessalegn, B. Landau, unpublished). This interaction has a developmental profile, with three year-olds unable to take advantage of these linguistic structures to improve performance in the task, and 6 year-olds automatically recoding the task into language, shown by ceiling performance even without explicit instruction (B Dessalegn, B. Landau, unpublished¹).

Figure 2. Illustration of the matching task used by [49].

Four-year-old children were presented with a target stimulus located in the top, center portion of a computer screen. The target then disappeared, there was a 1-second delay, and children then viewed three test items at the bottom of the computer screen (for example, from left to right: the target, a square with a different geometric split, and the target's mirror reflection). Children were asked to choose the item that was exactly the same as the one he/she first saw.

The results provide an intriguing parallel to the evidence on reorientation and suggest a specific mechanism whereby language could enhance humans’ ability to reorient by strengthening a unified representation of geometric and non-geometric properties of space. Specifically, the conjunction of color and geometric information does not require language for its encoding, but language can boost its strength—either during encoding or in immediate memory. The nature of the linguistic support is rich, involving both syntax and semantics, but is ideally suited to recoding relationships that may otherwise be represented in fragile form. In the case of reorientation, geometric layout and color could be combined in fragile form without language (as in the case of human toddlers and non-linguistic species), but receive a boost when recoded in linguistic format. If this is true, the use of linguistic encoding should speed and facilitate accurate search, as suggested by results of shadowing experiments [42], and consistent with reorientation performance among first generation deaf Nicaraguan signers who encode [LEFT] and [RIGHT] unsystematically (J Pyers, A Shusterman, A Senghas, K Emmorey, E. Spelke, Society for Research in Child Development, Boston, March). Related findings show more accurate searches among children who hear the location or reward value of the colored wall—but not among children who only hear the wall mentioned [56]. This too suggests that children may be recoding the layout in terms of the syntax and semantics that encode asymmetrical spatial relationships.

The momentary recoding hypothesis also fits well with recent evidence from the color domain that shows developmentally increasing lateralization for rapid between-category color judgments. Adults show an advantage of right visual field presentation of stimuli (projected to the left hemisphere), whereas infants show the opposite effect [57, 58]. The switch to the left hemisphere increases as children learn their color terms [58, 59], consistent with an increasingly powerful and automatic role for language in human cognitive function. However, as with reorientation, the role of language is not to create new kinds of representations, but rather, to provide a format which might speed, enhance, or facilitate performance.

Maps: The permanent record of our geometric knowledge

While language may provide a mechanism for momentarily strengthening the storage of geometric and non-geometric information, maps are the symbolic system par excellence for encoding and permanently retaining spatial information. Although language and maps are both created and used by humans alone to communicate information to others, the differences stop there. Unlike language, maps are symbolic devices whose explicit purpose is to encode spatial information about layout. Unlike language, the format of maps is roughly analog in nature, capturing the spatial layout of an environment by spatial transformations that preserve what is essential to finding things—most often, the geometric properties of layouts. Although language can encode spatial information, there are striking limits on how well it can represent complex spatial information. For example, surface textures, facial expressions, or the detailed contours of an overall spatial layout are difficult to represent in precise form by language; for these properties, a picture (map) is worth a thousand words. In contrast, maps are explicitly designed to preserve geometric information, providing an analogue representation that can articulate even the most complex and detailed aspects of spatial layout. And unlike language—which is largely mastered by the time of schooling-- the developmental timetable for achieving mastery in map use begins around age 3 but extends throughout the lifetime [60], with significant individual differences in mastery.

The capacity to use maps includes several core components. First is understanding of basic correspondences-- that the map is a representation having a purpose separate from its status as an object, that symbols on the map represent objects and places in space, and that the spatial layout on the map corresponds to a real physical layout [60]. The symbolic function of map-like objects emerges abruptly as early as 2−1/2 or 3 years [61] and children can use simple maps to locate objects as early as age 3 [62, 63, 64]. However, since maps can vary quite widely in the specific graphic devices used to represent landmarks and layout, as well as the choice of which properties are preserved, enhanced, and/or omitted, proficient map use has a long developmental trajectory tied to learning the specifics of map symbolism, perhaps by analogy with learning the specifics of different writing scripts. For example, children initially assume that properties such as color or exact line size correspond with fidelity to the objects they represent, leading to erroneous conclusions about the real dimensions of roads, bridges, etc. [60].

A second core component of map use is the mechanism whereby a representation of the map (its layout) is aligned with a representation of the layout in the real world. This can be accomplished by mentally aligning one with the other, or by physically aligning the map so that the geometric structures of the map and layout are viewed from the same perspective. Mental alignment is likely to build on the capacity to reorient oneself in an environment; formally, reorientation requires aligning a representation of layout that has been stored in memory with a representation of the currently observable layout [3].

Appreciation of core spatial properties of maps is species-specific, emerges quite early in development and emerges without special tutoring. Studies of chimps’ use of map-like objects have been limited, but evidence shows that they can use scale models to locate an object in the real space, but fail in even modest tests of generalization [65, 66]. By contrast, 4 year-old children can use maps without any prior specific experience. A congenitally blind 4 year-old who had never experienced explicit map-like representations of space used the angles among objects on a simple map to find targets in a room, and did so even when the entire map was displaced laterally or rotated vertically, destroying any direct alignment between symbols on the map and corresponding targets in the room [62]. By the age of four, and with no special exposure to maps, children can use both angular and distance relationships among symbols [62, 63, 64, 67] suggesting that they immediately understand these basic geometric correspondences between maps and real space. The use of distance and angle mappings requires no explicit tutoring, nor does it require extensive experience with symbolic devices, as the Munduruku, an isolated Amazonian group having no experience with graphic symbols, were found to immediately use distance and angle information represented on a simple map to locate objects in a nearby space [68]. In contrast to the ease of using angle and distance information, sense (right/left) relationships appear to be more challenging, with 4 year-olds showing considerable difficulty in using this property to disambiguate two identical targets [64]. This is consistent with the idea that left-right relationships are represented in unstable fashion by humans.

Although the findings suggest that children appreciate the correspondence between the map and the space it represents quite early and with little or no previous experience, other aspects of map use can be difficult and require both training and specific experience. For example, alignment of maps that are rotated in the horizontal plane (thus destroying viewpoint correspondence) present particular challenges to children [69] as well as adults. Learning to interpret particular graphic devices can also be a life-long process [60].

Concluding Remarks

The capacity to locate objects as we move through space is fundamental to the survival of many species. It is perhaps not surprising, therefore, that we share with many other species basic systems of navigation and reorientation. Where we differ from other species is in additional, powerful representational systems that allow us to share with others our representations of space. Both language and maps allow us to do this. However, there are crucial differences between the systems. Language may provide the tools to more stably represent some aspects of spatial organization that are otherwise fragile (such as sense relations); and through its combinatorial nature and infinite productivity, it provides us with the means to describe in infinite detail a given layout and the routes through it. But such descriptions take much time, can be hard for humans to process, and are severely limited in the extent to which they can result in a spatial representation of sufficient quality and resolve to guide navigation. In contrast, maps are portable representations of space; they relieve us of the burden of remembering detailed aspects of layout that might be discarded in the representations we use for reorientation, and provide a relatively immediate display of the spatial information we need to guide action. Using maps depends on having basic mechanisms of reorientation in place, but maps take us beyond the present, supporting our ability to know and understand spatial layouts without having experienced them firsthand.

The two systems of representation--both uniquely human, both emerging early in development with no formal tutoring-- have the capacity to extend our spatial capacities beyond those of other species. They provide the means for stably representing aspects of space that might be otherwise difficult to maintain in memory, and they allow us to explicitly communicate about space with each other. The capacity to communicate about space supports the development of additional tools of remarkable power. These tools are what ultimately transport human cognition far beyond the bounds of other species. With these tools, we can move backwards in time to reconstruct the layouts of bygone cities from ancient maps and forwards in time to explore lands not yet encountered. This freedom from the bounds of the present is the deepest signature of human understanding.