Drawing as a versatile cognitive tool

Abstract

Drawing is a cognitive tool that makes the invisible contents of mental life visible. Humans use this tool to produce a remarkable variety of pictures, from realistic portraits to schematic diagrams. Despite this variety and the prevalence of drawn images, the psychological mechanisms that enable drawings to be so versatile have yet to be fully explored. In this Review, we synthesize contemporary work in multiple areas of psychology, computer science and neuroscience that examines the cognitive processes involved in drawing production and comprehension. This body of findings suggests that the balance of contributions from perception, memory and social inference during drawing production varies depending on the situation, resulting in some drawings that are more realistic and other drawings that are more abstract. We also consider the use of drawings as a research tool for investigating various aspects of cognition, as well as the role that drawing has in facilitating learning and communication. Taken together, information about how drawings are used in different contexts illuminates the central role of visually grounded abstractions in human thought and behaviour.

Introduction

Tools for expressing ideas in visual form have been critically important throughout human history. These cognitive tools lie at the heart of some of humanity’s most important inventions, including art, writing, and mathematics^1–5 (Box 1). Perhaps the most enduring and versatile cognitive tool for externalizing ideas is drawing: the earliest known drawings date at least to 40,000–60,000 years ago^6,7. In the present day, drawings are used to capture what people perceive and know about the external world at many levels of abstraction, from realistic illustrations to simplified abstract diagrams.

Box 1. A generalized framework for cognitive tools.

Our Review focuses on how people use drawing as a cognitive tool to encode information in visual form. However, there are many other important kinds of cognitive tools that have emerged over human history. In modern times, people use writing to encode spoken language and numerals to encode exact quantities. There are also non-graphical cognitive tools, such as the knotted string-based devices, used in several South American Andean communities, known as quipu²⁰⁸. Despite this wide variation in surface form, these tools were all invented by people to support or offload cognitive functions, such as remembering, calculating, reasoning, imagining or communicating. As such, it might be possible to develop a more general psychological theory that extends beyond drawing and accounts for why various cognitive tools take the form that they do. These forms seem to reflect both individual cognitive constraints and cultural learning processes^1,2,4. For example, for a cognitive tool to be useful for communication, it must be expressive enough to represent a wide variety of meanings and simple enough to be learnable by novices¹⁸².

The next step towards a broader theory of cognitive tools might be to consider whether a theory of representational drawings produced by hand generalizes to other ways of externalizing knowledge in visual form. It might be especially promising to consider the perceptual, cognitive and motor processes that account for how people create effective maps and diagrams to communicate about spatial and conceptual relationships^209,210. In addition, future work could examine how people use computers to design visualizations to communicate about patterns in large amounts of data^211,212. Systematic study of how people produce and understand a wider variety of types of visualization — including illustrations, maps, diagrams and data representations — might also lead to a deeper understanding of why proficiency with some visualizations can be more readily acquired without specialized training (simple line drawings), whereas other techniques require substantial training to achieve proficiency (statistical graphs). To this end, a combination of approaches from cognitive neuroscience and computational modeling could be instrumental. For example, functional neuroimaging techniques could be used to compare the neural representations that are recruited when people interpret different types of visualization. These comparisons would be informative regarding the degree of specialization in different brain regions needed to support processing of each type of visualization. In addition, experiments with computational models could be used to identify the functional constraints (such as what kinds of task a system needs to perform) and structural constraints (such as how the system is internally organized) needed to emulate human behaviour in tasks involving visualizations. In the long run, these insights could be leveraged to develop new visualization techniques and to improve the way people learn how to use existing visualization techniques to think, communicate and solve problems.

Drawing can encompass several activities that leave marks on a surface, including sketching by hand and creating graphics aided by a computer. Regardless of the method of production, drawings are visible representations of thought that people intentionally create to be viewed. Drawings can be produced on a wide variety of surfaces, including stone, paper, concrete, glass and digital displays. In contemporary life, images can also be constructed using software programs that offer a menu of shapes that a person can arrange in various ways. However, these programs constrain the possible representations a person can create, whereas the open-ended nature of drawing using a handheld stylus (such as a pen or pencil) leads to an outcome that is primarily the product of the drawer’s intentions and experiences. Moving a handheld stylus across a physical surface is also one of the most basic and broadly accessible ways to produce a drawing, used by young children and seasoned artists alike^8,9. Thus, focusing on freehand drawings provides deeper and more detailed insight into how drawings are used as tools to capture what people perceive, remember and know about the external world. These types of drawing can communicate about concrete phenomena in a wide range of different settings.

The question of how drawings derive their meaning has posed a longstanding challenge for theories of depiction. One prominent perspective is that drawings are best understood as images that resemble the entities they depict, and therefore a theory of how people understand them should be grounded primarily in an understanding of optics and visual processing mechanisms^10,11. Another important perspective is that drawings are best understood as symbols composed from graphical primitives that need not resemble anything if there are cultural conventions that connect them to specific meanings^3,12,13. Each of these perspectives have generated valuable insights, but neither of them on their own can account for the sheer range of drawings, at various levels of abstraction, that people are able to produce and understand. Moreover, a drawing is an entity intended to convey some aspects of mental representations, but drawing is also an activity intended to record or to have an impact on ongoing mental processes.

Over the past several decades, the interpretation of drawings and the act of drawing have been investigated by researchers across many areas of psychology^14–20. Some of this work has studied the production and comprehension of drawings for their own sake, but other work has used drawings and the act of drawing to investigate other domains of cognition, such as vision and memory. Because drawing production tasks are open-ended, they can reveal more detailed information in a single trial than can typically be achieved using conventional paradigms. However, drawing production tasks have also been avoided, in part owing to this complexity compared with other behavioural measures, such as choice and reaction time. In the past several years, breakthroughs in machine learning, big data, and online research have ushered in a new era for using drawing tasks to gain insight into a wide array of psychological phenomena^21–25. Whereas classic studies employing drawing tasks often relied upon qualitative assessments of a small number of drawings based on bespoke criteria developed for each study or stimulus^26,27, new methods are now available to collect and analyse the content of drawings in a general manner at scale^{22–25,28,29}. These methods have accelerated the ability to leverage the high-dimensional nature of drawing data to gain insight into the content and structure of underlying mental representations that support a broad range of complex behaviours, including communication and collaboration^30–33, consistent with broader developments in psychology and neuroscience that have enabled these fields to embrace more complex, naturalistic behaviours^34,35. Moreover, these lines of work demonstrate that even though drawing abilities can vary strongly as a function of expertise^36–38, non-experts can be proficient in producing meaningful drawings that help to answer questions about psychological phenomena in new ways^23,25,30,33.

In this Review, we synthesize contemporary work that has examined links between drawing and cognition. We explore how drawing production and comprehension are constrained by interactions between perception, memory, and social inference. In addition, we consider the role of drawing production in mediating learning and communication. Each section focuses on a particular use case for drawing, and together these exemplify the variation in the balance of contributions from different cognitive processes, as well as the reciprocal influences between drawing and these cognitive processes. In the first section, we consider drawings that prioritize faithfulness to how the world looks and what drawing tasks reveal about visual recognition and visual experience. Next, we consider drawings that call forth previous experiences, and how such drawings reveal insights into how memory for specific details interacts with knowledge of general concepts. Then we consider how people produce drawings to facilitate their own learning and to communicate with other people, and how these activities illuminate the central role of visually grounded abstractions in human thought and behaviour. Our overarching goal is to bring together previously disparate perspectives on how drawings capture key aspects of the external world, moving towards a more unified understanding of the psychological mechanisms that explain how and why drawing is such a versatile cognitive tool. With this aim in mind, our analysis of the factors that influence human drawing behaviour is not comprehensive; for example, fine-motor-control mechanisms and how drawings evoke aesthetic responses are beyond the scope of this Review.

Drawing as a window into the visual system

Line drawings are a common type of drawing composed of lines and curves, generally without gradations in shade, that have been used throughout human history to faithfully record the visual appearances of specific objects and scenes^39,40. In this section, we first review the current understanding of how visual processing mechanisms support the production and comprehension of such drawings (Fig. 1). Second, we discuss how work with line drawings has yielded broader insights into how humans perceive the visual world.

Fig. 1 |. Drawing as a window into the visual system.

When an individual has the goal of drawing what they perceive, such as a bird on a branch, the resulting drawing preserves perceived contours, depth and texture, including biases introduced during perception that are not in the input. The effects of social cognition and memory are modest relative to the contribution of perception. Although motor processes are recruited during drawing production, they primarily serve to externalize the contents of perception, rather than to influence ongoing perceptual processing.

Visual recognition

Line drawings have been argued to contain the most critical information needed to identify visual objects⁴¹. Evidence in support of this claim comes from studies finding that visual categorization of objects in line drawings automatically recruits similar neural populations to those involved in visual categorization of photographs of objects^42,43. Such findings are used to justify the widespread use of line drawings as stimuli in studies of a wide variety of cognitive phenomena, including perception, learning, memory and language⁴⁴. Parallel work investigating scene perception has reached similar conclusions concerning line drawings of scenes^45–49. One way of making sense of these findings is to suppose that drawings approximate the edges that humans perceive in natural images^39,41, perhaps by taking advantage of the sensitivity of neurons in primary visual cortex to edge-like visual features^50–52. Although intuitively appealing, this edge-based account fails to explain the presence of contours in line drawings that convey information about three-dimensional shape, such as the depth and extent of apparent ridges^53,54, as well as surface texture and lighting information⁵⁵. It also does not explain why some edge-detection algorithms based on models of early visual cortex identify edges in photographs of objects that would not be included in human-made line drawings of the same objects⁵⁶. Last, this account does not explain the robust ability that many people have to recognize the real-world referent of drawings that lack details and contain distortions of the size and proportions of constituent object parts, such as drawings produced by non-experts^29,57.

Going beyond edge-based accounts, breakthroughs in computer vision^58,59 and computational neuroscience^60,61 have greatly expanded the set of hypotheses about how and why line drawings succeed in approximating the appearance of natural images of objects. These newer proposals often take the form of trainable neural networks inspired by the architecture of the primate ventral visual stream, including its hierarchical organization and specific local circuit properties^62–65. For instance, deep convolutional neural networks trained on large and heterogeneous image datasets have been applied successfully to a variety of visual tasks, and their internal components have been successfully mapped to the internal components of the primate ventral stream^66–70. Importantly, these deep convolutional neural network algorithms provide a strong basis for modelling the general-purpose visual computations that underlie recognition of both natural images and line drawings^{23,29,30,71,72}, outperforming earlier edge-based approaches⁵⁷. Even when deep convolutional neural networks are trained only to categorize objects in colour photographs, they can generalize to successfully categorize line drawings²³. Such generalization suggests that line drawings might be evocative of the external world in part because they take advantage of evolutionarily conserved computational mechanisms across the ventral stream to meet the challenge of real-world visual recognition. Nevertheless, deep convolutional neural networks trained to categorize objects in photographs have also been found to be somewhat biased towards categorizing on the basis of texture over shape information and sometimes make different errors from those typical of humans when interpreting line drawings. Thus, there remains an important gap between the abilities of this class of vision models and those of human observers for understanding a wide range of visual inputs^72–74. However, the pace of recent advances in computer vision suggests the continued promise of leveraging approaches from machine learning to develop more general mechanistic theories of human visual recognition.

Developmental, cross-cultural, and comparative studies investigating the ability to recognize objects in line drawings are also consistent with the basic idea that recognition of line drawings takes advantage of the specific functioning of the visual system. Prior experience with drawings does not seem to be a prerequisite for understanding line drawings. For example, human infants as young as five months old demonstrate sensitivity to the visual correspondence between line drawings and real-world objects^75,76. Moreover, adults living in communities without pictorial art traditions or substantial contact with Western visual media¹⁷ and some non-human primate species^77–79 can successfully recognize line drawings of familiar objects.

However, the notion that the ability to make sense of drawings and other pictorial representations is present from early infancy and universally shared across humans regardless of prior experience has not gone unchallenged^80,81. Although 15-month-olds understand that labels first applied to colourful illustrations of objects can be extended to their real-world referents, they are more likely to succeed on such transfer tasks when these illustrations are more realistic⁸². Moreover, although two-year-olds display a more sophisticated understanding of drawings than do younger children — both as representations of other objects and as objects in their own right^83,84 — children substantially improve in their ability to recognize the intended referent of a drawing between two and ten years of age²⁵. Thus, experience seems to have an important role in driving the development of robust drawing recognition abilities^85–87 (Box 2).

Box 2. Learning to draw.

The ability to draw varies with age and experience. Here we provide a brief overview of relevant work that has investigated the development of drawing ability in childhood, as well as the acquisition of drawing expertise in adulthood.

Basic drawing skills

Assuming that they have access to the appropriate tools (such as a pencil and paper), children in many cultures draw spontaneously and prolifically^16,213–216. One prominent view on the development of drawing behaviour is that it follows a consistent age-related progression, beginning with abstract expressions of movement and emotion, followed by a gradual transition from intellectual realism (drawing what you know) to visual realism (drawing what you see)^{127,128,217–220}. Although this descriptive account of drawing development remains popular, other perspectives and data bearing on the development of drawing behaviour have emerged. One account suggests that children display early competence in understanding the communicative function of drawings^{129,221–223} and that apparent production errors might be intentional and driven by preferences, rather than reflecting immature representational or motor abilities²²⁴. For instance, a child recreating an illustration of a house without a visible door might include a door in their own drawing, such that their depiction is more informative about the category they are drawing, even at the expense of visual fidelity to the original illustration¹²⁹. However, the notion that children’s tendency to draw what they know is eventually displaced by a tendency to draw what they see is inconsistent with newer large-scale studies showing that older children are clearly capable of producing drawings of visual concepts (‘what they know’) that need not look like any particular object (‘what they see’), and that their ability to produce and comprehend drawings of familiar concepts improves throughout middle childhood²⁵.

Together, these findings challenge the classic proposal that children’s drawing abilities proceed through developmental stages marked by fidelity either to what they know or what they see. Instead, the current evidence suggests that age-related changes in children’s drawing abilities might reflect the gradual development of greater representational flexibility²²⁵ and increased sensitivity to the implicit goals of drawing production^129,221,226. This learning might then permit children to use drawings to accomplish a wider variety of functions, including to facilitate learning and communication of content knowledge^131,134,227. In sum, although drawing abilities change in systematic ways throughout childhood as a function of age and experience, these developmental changes do not necessarily follow a sequence of ‘stages’ characterized by specific visual styles. As such, a promising direction for future research is to conduct detailed measurement of the actual experiences that drive the development of drawing abilities throughout childhood, as well as how these experiences vary within and between cultural contexts.

Technical expertise

Although many individuals acquire a basic competency in drawing, only a small number go on to develop an ability to reliably create visually accurate representations of entities in the external world that could be described as the results of genuine technical expertise. Those individuals who persist in developing their drawing abilities can show astounding representational skills, and such expertise has been shown to be associated with relevant cognitive and perceptual abilities¹³⁷. Studies investigating the acquisition of drawing expertise have probed multiple aspects of visuospatial ability, including shape detection^138,228, the allocation of visual attention^229,230, and visual working memory and visual imagery^231,232. These studies reveal associations between drawing expertise and enhanced visuospatial processing, albeit not differences in the basic phenomenology of vision²³³. However, much of this evidence is correlational and does not provide direct support for a causal link between drawing training and visuospatial ability.

Studies employing longitudinal designs and neuroimaging-based measures of learning have found that individuals engaged in a multi-week drawing course improved their ability to draw³⁷ and to perform certain visual tasks^36,38. Moreover, these changes were accompanied by distributed changes in neural activity, including in the prefrontal cortex and cerebellum^37,38. However, changes in drawing ability were directly related neither to changes in visual task performance³⁶ nor to changes in neural activity³⁷, suggesting that improvements in drawing were not the direct cause of these changes in visual processing. Taken together, these studies provide support for the notion that the rich set of activities associated with the acquisition of drawing expertise support a broad spectrum of improvements on related visual tasks, but that these improvements are mediated by complex and somewhat distinct mechanisms.

Taken together, this body of findings argues for a core capacity for visual abstraction that forms part of humans’ evolutionary endowment, insofar as it emerges early in human development and in artificial vision systems trained to handle realistic amounts of visual variability, without the need for large amounts of direct experience with drawn images. However, this ability appears to be neither monolithic nor static: performance on visual recognition tasks involving drawings depends on how detailed and realistic these images are, and performance changes as a function of age and experience. As such, although the use of line drawings of objects to stand in for photographs or actual objects in psychological research studies can be justified in many settings, it is important to verify that these drawings resemble their real-world targets to a sufficient degree for the specific population of interest.

Visual experience

Drawings intended to portray the external world provide a rich source of insight into the contents of human visual experience. Although it might be tempting to liken human vision to how a camera functions, many aspects of human visual perception do not follow from optical principles alone. For example, humans perceive visual forms differently and at lower resolution when they are in the periphery or are surrounded by (‘crowded by’) other similar forms, relative to when they appear in the centre of the visual field⁸⁸. When individuals produce drawings to recreate their perceptual experience of arrays in the periphery, their drawings manifest aspects of visual crowding that are consistent with psychophysical measurements⁸⁹. Moreover, these drawing-based responses provide insights that go beyond responses derived from traditional detection and discrimination-based paradigms, which might not necessarily provide response options that match what an observer perceives^90–92. For example, drawings have revealed other classes of crowding-related phenomena, including the suppression of awareness of certain shapes when flanked by other shapes with the same visual appearance⁹³.

When considering visual scenes, people perceive the sizes, shapes and locations of objects in three-dimensional layouts in ways that deviate from the simple application of optics^89,94,95. Indeed, people rate drawings produced following ‘natural’ perspective — which compresses the peripheral field while simultaneously enlarging the central visual field — as providing a better match to their perceptual experience than drawings produced using standard linear perspective^96,97. These findings are consistent with the notion that drawings meant to faithfully convey the appearance of a scene reflect underlying visual biases involved in viewing that scene.

Another way to use drawings to tap into the nature of visual experience is by studying individuals whose experience of the visual world is persistently altered, for example because of neurological disease or brain trauma. There is a rich tradition of using drawing tasks in clinical psychology and neuropsychology to gain insight into how visual experience differs between neurotypical and clinical populations²⁰. Indeed, one of the earliest attempts to model the process of drawing production was inspired by neuropsychological case studies that suggested the existence of a ‘drawing system’ that could be partitioned into different modules, each representing a different stage in the drawing process^98,99. Insofar as drawings that are faithful to perceptual experience primarily reflect differences in that experience, rather than other cognitive factors such as memory or motor control, individuals with altered visual perception would be expected to produce drawings that look different from those produced by healthy, neurotypical individuals. Among the most iconic examples of drawing behaviour altered by neurological damage is when individuals with spatial neglect following brain trauma are asked to draw a clock and spontaneously and selectively omit the left side of the clock, the same side in physical space in which they struggle to attend to objects^100–102. The clock-drawing task has also been used with individuals with Alzheimer’s disease to characterize the progression of constructional apraxia, a difficulty with assembling or drawing objects. Clock drawings produced by individuals with Alzheimer’s disease have been found to exhibit more spatial and semantic errors (such as incorrect numbers or incorrect positions) than those produced by healthy individuals¹⁰³.

Other drawing tasks have been used to confirm that changes to perceptual representations, rather than changes to memory systems, are responsible for changes to drawing behaviour. If only perceptual systems were affected, one would expect altered drawing of a currently present object, but intact drawing from memory. This pattern is found in studies of patients who neglected the left side of images when drawing from life but not from memory, suggesting that only perceptual and not memory or motor skills are affected^104–109. Analysis of drawings has also provided corroborating evidence for perceptual differences in individuals with autism spectrum disorder. For example, these individuals tend to focus on local visual details over global configurations, a bias associated with an enhanced ability to detect simple shapes embedded within larger, more complex figures^110–114. This focus on local details is also evident in the superior performance of individuals with autism spectrum disorder on non-drawing tasks like the block-design test, which is thought to rely on local visual processing^115,116.

Taken together, these lines of work provide converging evidence for a tight link between core visual processing mechanisms and drawings in which a person aims to produce a faithful representation of their perceptual experience. Moreover, this work exemplifies how an examination of complex behavioural outputs, such as drawings, can be informative about the contents of visual experience beyond what can be achieved using standard psychophysical measures^93,105.

Drawing from memory and knowledge

Drawings are also used for purposes other than to provide a faithful record of current visual experience. In this section, we review work investigating the content of drawings used to express what people remember about specific previous experiences, as well as work investigating what they generally know about the world. Drawings intended to encode such information provide an opportunity to investigate how visual perception interacts with memory systems (Fig. 2).

Fig. 2 |. Drawing from memory and knowledge.

When an individual has the goal of producing a drawing based on their prior experience, the resulting drawing is subject to perceptual distortions as well as memory decay and biases from semantic knowledge. Here, a beach scene drawn from memory is subject to boundary expansion and the addition of objects that were not present in the original scene. The clock symbol indicates the passage of time between initial viewing of the scene and subsequent recall during drawing production. The contribution of social cognition is modest relative to those of memory and perception. Although motor processes are recruited, they serve to externalize the product of interactions between perception and memory, rather than to influence ongoing cognitive processes.

Recall of visual information

The relationship between visual recall and visual perception can be probed by comparing drawings of complex real-world scene photographs conducted with the scene in view and from memory. One study found that people can produce detailed drawings of individual scenes that were previously viewed for only a few seconds and interleaved among dozens of other scenes²⁸. These memory drawings contained enough specific details to be matched to the original scene image by other people nearly as often as drawings produced with the original scene in view. Moreover, many of the objects in these memory drawings were drawn near their correct locations. However, this work found no relationship between an individual’s visual recall performance, as measured by the amount of detail contained in their drawings, and their recognition memory performance for the original images presented, suggesting that drawing-based measures tap into different aspects of the underlying memory representation than more common non-drawing recognition-based measures.

Although visual recall for scenes can be highly detailed, it is also subject to systematic distortions. For example, in drawings of scenes from memory, people often include visual details about the scene that extend beyond the boundaries of the original scene photograph. Such ‘boundary extension’ was first interpreted to reflect pervasive intrusions during memory retrieval of schematic knowledge about scene layouts¹¹⁷. Work examining a wider variety of scenes challenged this interpretation, finding instead that some scenes reliably induce boundary extension and others reliably induce boundary contraction, in which people omit details near the boundary of the original scene photograph¹¹⁸. Moreover, the degree to which a scene induced boundary transformation when drawn from memory was similar to the transformation induced when drawing the scene while concurrently viewing it. Thus, these spatial distortions might reflect perceptual biases that are present during initial encoding of the scene’s spatial layout, rather than introduced during memory maintenance or retrieval. These findings are consistent with other work investigating the organization of spatial details in drawings that are intended to capture the immediate visual experience of space, which show similar spatial transformations⁹⁶.

Although drawings that are intended to recall previous visual experiences preserve some information faithfully, reliable distortions can also manifest. For example, when people produce drawings of well known icons, they sometimes alter the image in systematic but incorrect ways, reflecting false memory for that icon. For example, the Monopoly mascot is typically drawn with a monocle, but does not actually wear one in the board game. This phenomenon, dubbed the Visual Mandela Effect, can arise when people produce drawings of iconic images even moments after viewing the correct version¹¹⁹. These findings suggest that drawings do not always take the form of what has been perceived before. Rather, drawings can reveal false memory for information that was never experienced. Another example of how drawings intended to depict a previous visual experience recruit disparate representations from those activated during ongoing visual perception comes from individuals with aphantasia. Aphantasia is a condition wherein a person reports being unable to engage in mental imagery despite having intact semantic memory and visual perception^120,121. When individuals with aphantasia draw scenes from memory, their drawings contain substantially fewer visual details than those drawn by individuals with typical mental imagery abilities³³. However, individuals with aphantasia and individuals with typical mental imagery abilities produce similar drawings of concurrently visible scenes, suggesting that the impairments associated with aphantasia manifest during visual recall, rather than during initial encoding.

Even among individuals without aphantasia, visual recall often engages mental representations that go beyond those formed during visual perception. General knowledge about the kinds of objects that are likely to appear in certain categories of scenes (such as beach scenes or a view of a laboratory) influence visual recall for details of individual scenes. Specifically, scenes containing a semantically incongruous object (such as a beach ball in a chemistry laboratory) were more likely to be recalled in a drawing-based free recall task than scenes without such incongruities, but these drawn recollections were often more impoverished¹²². These results suggest that even a drawing intended to represent a specific scene can be the product of complex interactions between episodic and semantic memory, wherein surprising or distinctive details might enhance the likelihood that a particular experience is recalled, even at the expense of visual detail in that recollection.

More broadly, these lines of work demonstrate that producing a drawing that conveys what a person has seen before relies on interactions between visual perception and multiple memory systems. Although visual recall can call forth richly detailed representations of prior visual experiences that exhibit many of the same biases that influence ongoing visual processing, visual recall is also subject to gaps and distortions that are the product of contributions from longer-term knowledge, including expectations about what an object looks like in general or which objects are likely to co-occur within the same visual scene.

Knowledge and concepts

The drawings we have discussed so far are intended to evoke a specific visual experience, but many drawings are intended to encode more abstract knowledge about the external world, such as object category⁴⁴, number² or causal mechanisms¹²³. Aiming to convey abstract concepts, rather than to preserve information about visual appearance, might influence the drawings people produce. Take, for example, a drawing that is meant to convey the general concept of a cat and another drawing that is meant to portray a specific cat at a particular moment in time (such as Garfield reclining on the kitchen counter in the morning). It is plausible that information about the specific cat is integrated with more general information about cats in the latter drawing, resulting in it containing features that are highly diagnostic at both the exemplar level (about Garfield) and at the category level (about cats). However, a study investigating this question found that drawings intended to preserve information about a specific object’s identity do not necessarily convey category information as effectively as drawings specifically intended to convey the object category^30,124.

This trade-off between conveying different types of information is also supported by studies investigating how people use drawings to convey abstract concepts, such as causal mechanisms¹²⁵ and number¹²⁶. In one study, participants were asked to create explanatory drawings to illustrate how machines worked, which were compared to depictive drawings they had created to capture the visual appearance of other machines¹²⁵. Analysing the resulting drawings produced under each goal, the explanatory drawings placed greater emphasis on parts of the machines that moved or interacted to produce an effect, whereas the depictive drawings emphasized parts that were visually salient, even if they were static. Moreover, these differences in visual emphasis affected what information naive viewers could extract from the drawings: explanatory drawings made it easier for viewers to infer how to operate the machine but more difficult to identify which machine was depicted. These findings suggest that people spontaneously prioritize functional information when generating visual explanations, but that doing so might facilitate inferences about physical mechanism at the expense of visual fidelity. More broadly, these findings support the notion that people are capable of internally representing the same object at different levels of abstraction, and these different construals are dissociable according to the visual properties of the resulting drawing of an object.

Dissociations between drawings produced under different task conditions also manifest even when they are not the product of voluntary pursuit of a particular representational goal. For example, it has been proposed that young children are more likely to produce drawings that reflect what they know about an object rather than what they can see^127,128 (Box 2). For example, even when drawing an object currently in view, children aged five to six years old often include features that are not visible from their vantage point but are nevertheless diagnostic of category membership (such as an occluded handle on a mug)^129,130. Conversely, individuals with semantic dementia are capable of accurately drawing an object currently in view but can produce highly impoverished drawings from memory when cued with a category label (such as ‘duck’)¹⁰⁹. These drawings from memory often omit key features that are diagnostic of the target category and sometimes erroneously contain features that are, if anything, diagnostic of concepts higher in the semantic hierarchy (for instance, drawing a duck with four legs, a feature diagnostic of the superordinate category ‘animal’). Taken together, these findings suggest that the ability to use drawings to express the contents of immediate visual experience and the contents of more abstract conceptual knowledge are dissociable.

More generally, the growing body of work investigating how people produce drawings based on visual memory and visually mediated abstract knowledge suggests that although these drawings preserve some aspects of visual experience (such as the objects situated in space), they do so selectively. More to the point, these drawings often go beyond the information available in any individual experience to instead highlight other relevant abstractions (such as general properties of object kinds). As such, drawings offer researchers a powerful tool for investigating the product of complex interactions between visual processing and multiple memory systems.

Drawing to learn

Drawings are the product of various cognitive processes, but the act of drawing can also influence cognition: what a person notices, remembers, and believes. In many cases, the target of this influence is the person who is drawing, a phenomenon described as ‘learning by drawing’ or ‘drawing to learn’^131–136. Psychologists working across many domains have investigated how and what people learn when they produce drawings, including the study of visual expertise^36,137,138, memory of specific items^136,139,140, and the acquisition of new knowledge in educational settings^131,135. In this section, we outline what unifies and distinguishes these different manifestations of learning by drawing, as well as how drawing production tasks have been used to investigate learning.

Regardless of the kind of information being conveyed, drawing is inherently a generative act¹⁴¹ and one that has been thought to share important similarities with other generative processes in cognition^142–145. For example, the usually unconscious act of perceptual inference about the structure of a visual environment has been theorized to rely upon an internal generative process (known as ‘inverse graphics’)^146–148. Drawings offer a valuable opportunity for understanding generative mental processes because the motor procedure for creating a drawing is externally visible, making the components of these processes potentially easier to discern^149–151. Drawing has also been likened to other elaborative forms of information processing, such as self-explanation, whereby a learner attempts to comprehend a new concept by explaining it to themselves^141,152,153. However, it remains an open question to what degree learning by drawing is governed by principles and mechanisms shared with other generative behaviours.

There are various ways in which learning by drawing can manifest, depending on the kind of information to be learned and how learning is measured. In one study, participants who repeatedly produced visually similar drawings of real-world objects (such as bedframes and chairs; Fig. 3) were better able to discriminate these objects in a subsequent categorization task relative to control objects that were not repeatedly drawn²³. These findings support the basic notion that accumulating more drawing experience engages visual processing mechanisms that can be accompanied by enhanced visual task performance. However, currently available evidence suggests that the benefits of practice drawing are also quite constrained: over short timescales (minutes to hours), these benefits appear to be specific to the items that were practised²³. And over longer timescales (weeks to months), individual gains in drawing skill do not appear to be strongly associated with individual gains in visual task performance³⁶ (Box 2).

Fig. 3 |. Drawing to learn.

Through the interplay among perception, memory and motor processes, drawing enhances memory for individual objects and contributes to the development of visual expertise. Here, drawing a previously seen chair contributes to a better ability to later distinguish the chair and bed images. The clock symbol indicates a brief delay between initial viewing of an object and drawing production. The contribution of social cognition is modest in this scenario.

Beyond an impact on immediate perceptual processing, drawing production seems to enhance subsequent recall of both verbal (such as word lists) and visual information (such as pictures)^154,155. Moreover, the mnemonic benefits of drawing remain even when compared to strong baselines that rule out the possibility that drawing production is simply a special case of deep semantic processing or internal visual imagery, which are both known to enhance memory¹³⁹. To understand what aspects of drawing were responsible for the learning benefits, one study decomposed drawing into component subtasks (including tracing, viewing, imagining, and drawing without seeing the output) and found graded decrements in memory performance for subtasks lacking the visual, motor, and generative components of natural drawing behaviour¹⁵⁶. These results suggest that drawing might enhance memory precisely because it concurrently engages multiple representational modalities, strengthening links between them and increasing the number of possible access routes to a memory^136,156.

In addition to improving memory for specific items when used in the laboratory, drawing can facilitate learning of complex concepts in real-world educational contexts^{131,134,135,157–159}. These benefits have been documented across a wide variety of scientific domains, including biology, chemistry, physics, geology and math^{153,160–168}. For example, one study found that middle-school (aged 13 to 14 years) students who were prompted to generate drawings to explain how chemical bonds work (such as how ionic and covalent bonds differ) performed better on a subsequent comprehension test than students who produced written explanations of the same chemical phenomena¹⁵³. However, not all studies have found facilitative effects of drawing on learning^{161,169–172}. One study found that secondary-school students (approximately 16 years of age) who were instructed to generate drawings to summarize a chemistry-related text performed less well than students only asked to imagine the phenomena being described, and that this disadvantage appeared to be mediated by self-reported measures of cognitive load and mental effort¹⁷². Other studies have found that providing more guidance to learners in drawing activities, achieved through instructor-led demonstrations and partially completed illustrations, is associated with better learning outcomes than drawing with minimal guidance, suggesting that combining sound pedagogical practices with drawing-based generative activities might help to mitigate learner cognitive load and thereby enhance learning¹³⁵.

Taken together, the work we have reviewed in this section suggests that the act of drawing is linked to improvement on a variety of perceptual and cognitive tasks, but that the nature and magnitude of these gains depends strongly on the conditions of drawing production and how improvement is measured. Nevertheless, the generative nature of drawing production offers valuable opportunities to understand how perceptual, memory and motor processes interact in the service of learning (Fig. 3). Overall, more research is needed to understand the conditions under which drawing activities support learning and generalization in real-world educational contexts, and thereby advance mechanistic theories of how drawing impacts ongoing cognitive processes^131,135.

Drawing to communicate

Whereas in the previous section we focused on drawings produced to facilitate one’s own learning, drawings are often also intended to be shared with others to communicate. In this section, we review work investigating the cognitive mechanisms that enable people to determine what information to include in their drawings to communicate effectively in different settings (Fig. 4). In addition, we discuss how studying the act of drawing can be used to advance understanding of natural communication.

Fig. 4 |. Drawing to communicate.

When an individual has the goal of producing a drawing to share information with other people, their success depends on the ability to adopt the perspective of the viewer and to know what information is widely shared, including knowledge of graphical conventions for representing certain concepts. Over time, drawings can become sparser without losing efficacy for communication. The clock symbol indicates the passage of time between initial viewing of an entity, such as a bird, and multiple subsequent attempts to convey its identity by drawing. Social cognition, memory, perception and action work in concert in this scenario.

The seemingly straightforward task of drawing an object currently in view requires decisions about the purpose of the drawing and therefore what information to include. As we reviewed above, the relevant information can depend on whether the person who is drawing is attempting to depict that specific object, evoke its general category¹²⁴ or explain how parts of the object interact with each other¹²⁵. However, these goals are not always supplied explicitly, so researchers seek to understand the factors that determine which goals people adopt in each scenario.

The ability to adopt the perspective of the viewer seems to be a critical factor for explaining how drawers spontaneously select which information to prioritize when drawing. In one study, participants were paired up to play a drawing-based communication game and assigned the roles of drawer and viewer³⁰. On each trial, both participants were shown four objects, but in different locations for each participant. The drawer’s goal was to draw one of the objects so that the viewer could select it based on its location. On some trials, the four objects belonged to the same basic-level category, whereas on other trials they belonged to different categories. Drawers exploited the information they shared with the viewer to efficiently communicate about the target object: they produced sparser drawings on different-category trials, but more detailed line drawings on same-category trials. Trial-to-trial drawing differences could be explained by a computational model combining two abilities: the ability to evaluate how well a drawing corresponds to a given object based on appearance alone and the ability to judge what information was most relevant for helping the viewer to infer the intended meaning in context. Critically, removing the latter pragmatic-inference component from this model substantially worsened its ability to emulate human drawing production behaviour, suggesting that the capacity to form expectations about how a viewer would behave is vital to communicating with drawings. More broadly, this work suggests important commonalities between the roles of social inference in how visual and linguistic communication behaviour is adapted to different contexts^173–177.

Indeed, one of the most striking aspects of human communication behaviour is how flexible and context-dependent it can be. This flexibility reflects pressures operating across a wide variety of timescales, ranging from factors influencing how communicators behave during real-time interactions to factors that shape the formation of communicative conventions across generations^178–182. A growing body of experimental work employs drawing-based communication games to investigate how people use drawings to communicate about a set of objects or concepts multiple times throughout an interaction with another person^32,183–185. This work has identified key roles for communicative need (how important it is for people to communicate about some concepts relative to other ones) and social feedback (such as how often and how quickly a viewer successfully decodes the meaning of a drawing). Concepts that an individual needs to repeatedly communicate about and that are communicated successfully tend to be depicted more simply over time³². Specifically, these studies suggest that successful visual communication using drawings can depend on how well people integrate perceptual information with previously learned associations to connect drawings to specific meanings^186–192. Previously learned associations do not always exert a strong influence, such as when a high degree of visual fidelity to the external world is paramount (such as in botanical illustrations or anatomical drawings) or when these associations do not (yet) exist. However, novel associations can emerge quickly during live communicative interactions and come to strongly determine pictorial meaning. For instance, as two communicators learn to associate a particular drawing more strongly with the object it is intended to depict, sparser versions of that drawing can still successfully evoke the original object, even if it directly resembles the object to a lesser extent. Instead, the meaning of these simpler drawings relies increasingly on memory for earlier communicative exchanges with the same individual. For example, two scientists starting a new collaboration might produce detailed whiteboard drawings in their initial meeting to ensure that they understand one another, but gradually simplify their sketches in subsequent meetings once they have established more common ground. However, perceptual considerations can still influence the kinds of novel association that form, such that visual information that is inherently more diagnostic about an object’s identity might be more likely to form the basis for ad hoc graphical conventions than other, equally salient visual attributes³².

Taken together, these lines of work suggest that the use of drawings to effectively communicate with others in different settings relies upon interactions between visual perception, memory and social cognition. Moreover, they highlight the value of using drawing-based tasks to investigate general principles governing human communication behaviour that are shared between verbal and visual modalities. In the long run, insights from these lines of work might contribute to explaining how consistency and variation in drawing styles across cultures initially emerges and endures across generations^189–192. For example, individuals living in countries where the most prevalent languages are more similar to each other also produce more similar drawings of everyday object concepts, consistent with the possibility that these shared graphical conventions emerge from a history of social interaction¹⁸⁹. In addition, these studies might ultimately shed light upon the perceptual and cognitive factors that shaped the emergence of modern symbolic systems, which rely on broadly shared associations between marks and their meanings^1,2.

Summary and future directions

Drawing is a technology that humans invented to create visible objects from the otherwise invisible contents of mental life. Creating and sharing drawings can in turn influence what people learn and remember. The goal of our Review was to synthesize different perspectives on how drawings capture key aspects of the external world, moving towards a more unified understanding of why drawing is such a versatile cognitive tool. Whereas classical theoretical perspectives on drawing focused on either the question of how drawings resemble entities in the world^10,11 or argued for their fundamentally conventional character^3,12,13, here we considered how the purpose of a drawing influences the balance of cognitive processes engaged during drawing production, how a drawing looks, and what information it contains. Moreover, whereas previous empirical work investigating drawing behaviour in cognitive psychology focused on different use cases for drawing separately, here we jointly considered how these different use cases relate to one another in terms of the cognitive processes they engage.

First, we considered drawings that prioritize visual fidelity to how the world looks and reviewed evidence in favour of their primary reliance on core visual processing mechanisms. Next, we considered drawings produced from memory for prior experiences and from general knowledge, exploring how such drawings differ from the first group, relying more heavily upon interactions between perception and the reconstructive nature of memory. Finally, we considered how drawings intended to support communication and learning reflect interactions between perception and other cognitive processes, including memory and social cognition, to generate external representations that highlight useful abstractions, even at the cost of visual fidelity to the external world.

There are major open questions concerning exactly how the visual processing mechanisms that form the basis for drawing comprehension and perception interact with other cognitive systems, including those supporting episodic and semantic memory^{122,124,193–195}, visuomotor planning and control^43,156,196, and social inference^30,175,176. Taken together, the behavioural evidence reviewed here suggests that these interactions are crucial for explaining how human drawing behaviour can vary so strongly across contexts. Thus, the next step is to develop more mechanistic cognitive theories that expose the specific computations responsible for this contextual variability. Progress might be accelerated by developing such theories in concert with detailed characterization of the neural representations recruited during drawing production in a broader array of settings, including variation in memory demands, motor output modality and social context. More generally, tight coordination between behavioural and neural measurement alongside computational model development could be an especially promising strategy for gaining mechanistic clarity into the cognitive basis for complex, naturalistic behaviours, including drawing production.

The findings reviewed here suggest that drawings are neither entirely like natural visual inputs^10,197 nor like language¹². Drawings can accomplish many of the communicative functions that people otherwise use words for, including to refer^30,32,183, to remember^28,139 and to explain^125,153. Thus, at least some aspects of how people communicate with drawings can be explained by generalizing theories originally developed to explain how people communicate using language^173–177. However, it is not clear how far these functional parallels between drawing and language go. There are some cases in which drawing and text-based representations, even when formally equivalent, diverge with respect to how easily they support logical reasoning^198–200 and learning about causal mechanisms^153,201. It is important to establish why people show differences in processing fluency across these two modalities and how they decide when to use language and when to draw a picture to communicate. As such, future research should work towards a deeper understanding of what aspects of human communication are general across or specific to information modalities.

Another key question for future work concerns the computational mechanisms that account for the various ways people can learn by producing drawings. For example, one possible route by which drawing might guide learning is by requiring individuals to actively highlight the features in their experience that are most relevant to their current goal, facilitating the discovery of useful and generalizable abstractions, such as visual attributes that are diagnostic of a visual concept^23,202–205. Another possibility is that drawing might drive learning because it is inherently multimodal and generative, requiring tight coordination between perceptual and motor representations of the same concept^156,196. These possibilities are not mutually exclusive. Drawings are characterized by a fundamental duality: a drawing can be represented by the generative process giving rise to it and the visual properties of the resulting image^149,151,206. In computational models, this first kind of representation is often modelled by a graphics program containing a sequence of ‘motor’ commands for generating an image, and the second is often modelled by a distributed pattern of visual feature activations in a neural network. When these two kinds of representation are united within the same system and provided with a mechanism for ‘bootstrapping’ new and useful concepts that can be expressed in both formats^206,207, activities engaging multiple modalities (such as drawing) might support the discovery of visually grounded abstractions that are especially durable and generalizable. A promising avenue for future studies is to directly evaluate these and other hypothesized mechanisms against behavioural and neural data, towards developing mechanistic theories that account for the broad array of drawing-induced learning phenomena in the cognitive science and educational psychology literatures.

In sum, uncovering the cognitive mechanisms that enable humans to produce and understand drawings should help to advance theories of visual perception and how perception interacts with other aspects of cognition. A thorough understanding of how people use drawings to express their ideas in different settings provides a strong foundation for developing psychological theories to explain how and why the full array of cognitive tools humans use today takes the form that it does (Box 1).