Neural and behavioral correlates of drawing in an early blind painter: A case study

Abstract

Humans rely heavily on vision to identify objects in the world and can create mental representations of the objects they encounter. Objects can also be identified and mentally represented through haptic exploration. However, it is unclear whether prior visual experience is necessary to generate these internal representations. Subject EA, an early blind artist, provides insight into this question. Like other blind individuals, EA captures the external world by touch. However, he is also able to reveal his internal representations through highly detailed drawings that are unequivocally understandable by a sighted person. We employed fMRI to investigate the neural correlates associated with EA's ability to transform tactilely explored three dimensional objects into drawings and contrasted these findings with a series of control conditions (e.g. nonsensical scribbling as a sensory-motor control). Activation during drawing (compared to scribbling) occurred in brain areas normally associated with vision, including the striate cortex along with frontal and parietal cortical regions. Some of these areas showed overlap when EA was asked to mentally imagine the pictures he had to draw (albeit to a lesser anatomical extent and signal magnitude). These results have important implications as regards our understanding of the ways in which tactile information can generate mental representations of shapes and scenes in the absence of normal visual development. Furthermore, these findings suggest the occipital cortex plays a key role in supporting mental representations even without prior visual experience.

Introduction

Objects in our environment can be identified through sight as well as though touch. Studies in crossmodal sensory processing have suggested that there is a convergence across these two modalities that ultimately can facilitate object recognition (Amedi et al., 2005; James et al., 2002; Sathian, 2005; Woods and Newell, 2004, 2004). For example, when sighted individuals explore an object by touch, they can produce a drawing of that same object so that it can be identified by another sighted observer. Certainly, the accuracy of rendering the drawing draws upon the individual’s ability to construct, manipulate and translate the contents of their own mental representations. Evidence suggests that haptic processing recruits visual areas in order to construct a representation of an object (Amedi et al., 2005; James et al., 2002; Sathian, 2005; Woods and Newell, 2004). However, does this ability reflect an underlying crossmodal representation based solely on prior visual experience? A unique way to address this question would be to study the drawing ability of an early blind artist with no prior conscious visual experiences or memories. Here, we present the results of such a study. With an early blind painter named EA, we employed functional neuroimaging (fMRI) to investigate the neurophysiological correlates associated with his drawing ability. Given his remarkable artistic skill, EA can reveal to us the content of his own internal representations. In this setting, we can begin to uncover the role of visual experience in crossmodal sensory transformations.

Results

Using functional magnetic resonance imaging (fMRI), six conditions were run using a block design paradigm: object recognition and perception of tactile features of an object, mental imagery of the explored object, drawing of the object and scribbling a "nonsense" figure using similar hand motions as for drawing. Two other control conditions - verbal memory and motor hand movements - were also tested (see methods section for complete details). Representative examples of his drawings and scribbles collected from a scanning run are shown in Figure 2a.

Figure 2. Neuroimaging data of subject EA for drawing and scribbling objects.

A) Example of behavioral data collected from a scanning run. Subject EA’s sketches (above) are shown compared to the object tactilely explored or in response to the control scribble condition (below). (B) Drawing versus scribbling contrast presented on a full inflated cortical reconstruction of EA's brain. Activation was found in a network of posterior occipital (including the calcarine sulcus), occipito-temporal, occipito-parietal and prefrontal areas.

The main contrast of interest was the drawing of objects versus scribbling (Figure 2B green scale clusters and Figure 3A). Activation was found in a network of areas including the posterior occipital, occipito-temporal, occipito-parietal and prefrontal cortical areas. Of particular interest was activation specific to the drawing task localized within the calcarine sulcus corresponding to the primary visual cortex (V1/V2). In V1/V2, activation was localized to the mid to anterior parts of the calcarine sulcus, corresponding to the mid to peripheral representations of the visual field respectively (Sereno et al., 1995).

Figure 3. Time courses from selected regions of interest (ROIs) for Drawing vs. Scribbling (A) and Drawing and Scribbling vs. Baseline (B).

For demonstrative purposes, we also present the time-course of all experimental conditions in different ROIs using the peak voxel in the smoothed volume of each ROI. (A) Several ROIs in the occipital cortex show robust and selective activation for drawing objects with a typical hemodynamic response. Each ROI (corresponding to the green scale clusters shown in Figure 2) can be identified by its header. A similar pattern was also found in clusters within the parietal cortex. Robust activation for drawing was also found in prefrontal cortex, but in these areas there was also relatively robust activation for imagery and naming/verbal memory (in spite of the fact that these conditions were ignored in the contrast used to select the ROIs in this test). (B) ROIs of peak activation for clusters significantly activated by drawing and scribbling versus baseline (similar to those shown in Figure 2 in red to yellow color scale). The time courses are depicted by the following color index: tactile objects (red), mental imagery (brown), drawing (blue), scribbling (cyan), motor control (orange) and naming/verbal memory (green).

Comparing the time courses for all experimental conditions from selected regions of interest (ROIs) revealed that several ROIs in the occipital cortex showed robust activation for drawing objects (including the calcarine sulcus; Figure 3A). It is worth noting that both drawing and scribbling require complex sensory and motor coordination. Thus it is perhaps not surprising that the most robust activation for these conditions included a large-scale fronto-parietal network (Figure 2 red-to-yellow scale clusters and Figure 3B).

Activation patterns associating object recognition by touch (purple), mental imagery (blue) and naming (yellow) are shown in Figure 4. Recognition of different features of an object by touch led to patterns of activation that included several similar areas including the prefrontal, parietal and occipital cortex (time courses from selected ROIs are presented in Figure 5). Imagery of the tactilely explored objects activated similar brain regions in EA (including the parietal, prefrontal and occipito-temporal areas), but to a lesser extent. Interestingly, several areas, most notably in the medial posterior occipital cortex, showed much greater selectivity for drawing compared to all other tested conditions (Figures 2–3). To further test this directly, we also performed a direct comparison between drawing objects and the tactile objects condition (Draw versus Touch contrast, Figure 6). We found greater and selective activation of the occipital cortex during the drawing condition. This differential activation was found mainly in the ventral visual stream stretching from the lateral-occipital sulcus to the calcarine sulcus. In addition, there was a general preference for right hemispheric activation for drawing (see also Supplementary Figure 1). Finally, in contrast to reports of robust occipital cortical activation seen in other early blind subjects (Amedi et al., 2003), EA did not show robust visual cortex activation during the verbal memory task (Figure 4).

Figure 4. Neuroimaging data of subject EA for object recognition by touch (purple), mental imagery (blue) and naming (yellow).

Each control condition was contrasted with all the other control conditions in order to exclude non-specific effect (e.g. the purple, touch cluster are defined by the contrast: touch > imagery, naming and sensory-motor control). Activation as a result of the recognition of different features of the object by touch and mental imagery of the object drawn included many similar areas in prefrontal, parietal and occipital areas.

Figure 5. Time courses from selected ROIs for naming (A) object recognition by touch (B) and mental imagery (C).

A) Several ROIs in the prefrontal and parietal cortex show robust activation for recalling the names of objects from memory. Activation was not very selective as these clusters also showed activation for mental imagery, drawing and tactile objects. (B) Time courses for tactile object exploration. All ROIs showed robust activation for both the drawing and scribbling tasks. (C) Time courses for mental imagery of the explored objects (which EA had to draw next). Most of these ROIs also showed activation for drawing and touching objects.

Figure 6.

Direct comparison of drawing versus tactilely exploring objects. (A) The contrast of drawing versus touching objects is presented on a full inflated cortical reconstruction of EA's brain. Activation was found mainly in several posterior ventral occipital areas (including the calcarine sulcus) and prefrontal cortex. Some activation was also found in parietal cortex and the Insula. (B) Time courses from selected regions of interest (ROIs) for drawing versus touching objects that were highlighted in (A).

Discussion

Subject EA is a unique example of an early blind individual able to communicate the internal representation of objects in his mind through his drawings guided by his tactile rather than his visual experiences. Contrasting EA’s drawing activity with nonsensical scribbling revealed strong activity implicating a specific network of cortical areas. This network included the frontal cortex, parietal regions, and most strikingly, widespread areas of the occipital cortex. Within the occipital cortex, activation specific to the drawing condition was found in occipito-temporal areas and most notably within the calcarine sulcus, corresponding to the primary and secondary visual cortical areas (areas corresponding to mid and peripheral visual field representations).

The process of producing an artistic image involves both technical and creative components. In this study, it is important to note that we attempted to assess the technical aspects of EA’s drawing ability as opposed to the source of his creativity. When asked to draw an object, EA was given the instruction to reproduce the object on paper as precisely as possible and was specifically asked not to attempt to create an “artistic rendition” (and in any case, the time limit of 15 seconds did not allow for such a rendition).

The technical aspects of EA’s work include the typical steps normally ascribed to a sighted artist: 1) perception of an object, 2) conversion of the three-dimensional perception into a two-dimensional form, and 3) reproduction of the two-dimensional concept so that it can be comprehended as being originally three-dimensional. What is unique to EA is the fact that he is able to convert a three-dimensional tactile perception into a two-dimensional mental representation without the mediation of prior conscious visual experience. It is likely that EA starts with the haptic perception of the object since he has no prior conscious visual experiences or a visualization of the result of his drawing. Given EA’s statements that he has had no visual experiences, and indisputable profound and early visual impairment, he must possess internal representations of objects and scenes acquired by non-visual (primarily tactile) means (Heller, 2002; Kennedy and Igor, 2003; Kennedy and Juricevic, 2006a; Kennedy and Juricevic, 2006b). Thus, the most striking finding of this study is indeed the fact that subject EA shows robust brain activation within visual cortical areas associated with his drawing abilities as compared to just scribbling (Figures 2 and 3) or tactile object recognition (Figure 6). This suggests massive crossmodal plasticity that might support EA’s extraordinary drawing capabilities (rather than low-level tactile or sensory-motor capabilities).

In sighted individuals, visual mental imagery draws on much of the same neural machinery as visual perception (Ishai and Sagi, 1995; Ishai et al., 2000; Kosslyn et al., 2001; Kreiman et al., 2000; Lambert et al., 2004; Mechelli et al., 2004; O'Craven and Kanwisher, 2000). Here we also show that a similar phenomenon (the convergence between perceiving an object, drawing an object and imagining it) can also be observed in an individual who has no prior conscious visual memory or visual recollections (especially not of objects, even if EA had some light perception in early years despite his other recollections, which we have no reason to question).

While the activation of visual cortical areas (including V1) during drawing suggests the involvement of these areas in non-visual functions, it is worth noting that the pattern of recruitment is different from other examples of crossmodal plasticity reported in early blind individuals. Specifically, EA does not exhibit activation of visual cortical areas during verbal memory tasks (Amedi et al., 2003). This study (Amedi et al., 2003) showed however that the level of recruitment of the visual cortex for verbal memory depends on performance (i.e. only subjects who exhibited superior verbal memory abilities had robust plasticity for verbal memory in visual cortex). Thus, the results we observed here extend this notion to a different domain. Because EA has consistently painted and drawn throughout his life, the resources of the occipital “visual” cortex may have been recruited for this purpose and not for other compensatory behaviors such as verbal memory skills. Evidence consistent with this hypothesis includes his Braille illiteracy and his self-report of poor verbal memory abilities. However, aside from the primary and secondary visual areas, and consistent with data from both early and late blind individuals (Amedi et al., 2003; Burton, 2003; Pascual-Leone et al., 2005; Pietrini et al., 2004; Sathian, 2005; Sathian and Lacey, 2007), EA’s ventral and lateral occipital cortex was recruited while he haptically explored objects.

In addition to occipital cortical areas, robust activation during the drawing task was also seen, involving a large network of frontal as well as parietal cortical areas. This parieto-frontal network has previously been shown to be implicated in tactile sensorimotor tasks (Binkofski et al., 1999; Stoeckel et al., 2003). Furthermore, studies in sighted subjects have indicated heightened activity in frontal networks during drawing tasks but not in the occipital lobe (Makuuchi et al., 2003). The process of creating a two-dimensional image from a three-dimensional perception requires a specific cognitive transformation in the brain. The activation observed in these areas may be part of this process but its role is most likely not specific to drawing and probably represents a higher level of cognitive processing than what usually occurs in the visual cortex. Evidence for this comes from reports of changes in artistic development and creativity in fronto-temporal dementia (FTD) (Mell et al., 2003) as well as lesion-specific changes in artistic ability and style following cortical damage (Annoni et al., 2005).

To our knowledge, this study represents the first to report the neural correlates associated with drawing and artistic abilities in an early blind painter. While EA shows recruitment of frontal-parietal areas during a drawing task as might be found in sighted artists, he also shows robust activation of the occipital cortical areas. Thus, in the case of both sighted and non-sighted artists, activity in frontal-parietal regions may correspond to transformations from perception to two-dimensional image production and to the complex sensory-motor coordination needed for drawing. However, the additional occipital activity present in our study of EA may represent a further adaptation allowing an early blind painter to construct, manipulate and express these images.

Art, on both a technical and conceptual level, externalizes the inner workings of the brain (Zeki, 2001). Through his technical and creative skill, EA has likewise externalized the workings of a uniquely adapted mind. Evidence from this case supports the hypothesis that internal mental representations of objects can be generated by haptic experiences that are readily translated into representations that can be unequivocally understood by sight. Such evidence enhances our understanding of the fundamental organization of the human brain as well as the perception of reality.

Experimental Procedure

EA Case Report

At the time of study, EA was a 51 year old, right-handed male. He was Braille illiterate and described himself as a self-taught artist. Growing up, he felt socially isolated because of his blindness and would often spend hours alone drawing in the sand and exploring the relief patterns of his figures. At the age of six, he engaged seriously in art and painting. Without any formal instruction or schooling, EA started to paint using the tips of his fingers, employing fast drying acrylics and water-colors as well as oils. He also learned to draw with a pencil and paper using a specially designed rubberized writing tablet (Sewell raised line drawing kit, Sewell M.P. Corp. Woodslide, NY) that allowed him to generate relief images that he could subsequently detect and explore tactilely (Figure 1a). The themes of his paintings vary, and include objects that can and cannot be tactilely examined (such as fruit and clouds respectively). His scenes utilize a vibrant palette of color; often containing shadows, depth cues, and perspective akin to that employed by sighted artists (see Figure 1b). His use of color is probably guided by instruction from sighted individuals who have informed him about typical color associations (such as water is blue, trees are green and roofs of houses are red). Other aspects of his painting are again probably similarly influenced by information received from sighted observers. However, his ability to explore a novel object and rapidly draw it in exquisite detail (see a model of a brain in Figure 1a, and a man sitting on a bench holding an apple and a book in Figure 1c and 1d) appears to be a skill he has developed independent of input or feedback from sighted observers as he never underwent any kind of formal instruction.

Figure 1. Examples of subject EA’s drawing abilities.

A) EA drawing a novel object (a model of the brain) using a pencil and paper and a specially designed rubberized writing tablet (Sewell raised line drawing kit). This technique allows him to generate relief images that he can subsequently detect and explore tactilely. (B) The themes of his drawings and paintings vary and include both tactile and non-tactile subjects. The drawing shows a landscape scene and illustrates how he applies colors to his paintings. His paintings often contain vibrant color, and he uses shading, depth cues, and perspective akin to that employed by sighted artists. (C) Example of a complex and novel object which EA had never encountered. Once EA explored the object by touch for a few minutes, he was able to render a very accurate drawing of the object (D).

A detailed neuro-ophthalmologic examination (including flash electroretinogram) confirmed his profound blindness. One of his eyes never developed and remains a rudimentary bud (phtisis bulbi). The other showed massive corneal scarring, a dense cataract and retinal pigment deposition consistent with early blindness. A resting nystagmus was also present. Examination of previous medical records revealed that at the age of five, he underwent a comprehensive ophthalmic evaluation documenting that he did not posses any functional vision at the time.

Prior to commencing the experiment, a separate functional scan was run using a standard black and white alternating checkerboard pattern. This stimulus was designed to elicit maximum visual field activation in visual cortex, and was used to test for any cortical response to light. Measured BOLD responses in visual cortex were not significant following presentation of this visual pattern versus baseline, which is consistent with his neuro-ophthalmologic evaluation. While at this time it is impossible to confirm EA’s self-report that he has been blind since birth, based on available medical records and accounts, it is clear that EA has never had normal vision and was profoundly blind by the age of five.

Experiments were carried out in accordance with NIH guidelines for human studies and the principles expressed in the Declaration of Helsinki. The study was approved by the Institutional Review Boards of Beth Israel Deaconess Medical Center and Boston University School of Medicine and informed consent was obtained prior to subject EA’s participation in the study. Consent form was read to subject EA orally in English. It was simultaneously translated from English to Turkish by a Turkish translator. After confirming that he understood all the details and verification of the fidelity of the translation, EA signed the consent form.

MRI acquisition

BOLD fMRI measurements were performed using a whole-body 3T Phillips scanner equipped with 22 mT/m field gradients with a slew rate of 120 T/m/s (Echospeed). The functional MRI protocols were based on a multi-slice gradient echo, echo-planar imaging (EPI) and using a standard head coil. Functional data were obtained under optimal timing parameters: TR=3 sec, TE=55 ms, flip angle=90°, imaging matrix=80x80, FOV=24cm. The 37 slices (slice thickness 3mm and 1mm gap) were oriented approximately to the axial plane and covered the whole brain.

Experimental design

The main fMRI experiment was composed of 6 conditions using a block design paradigm, (each block was separate by a rest baseline of 9 seconds): (1) object palpation: recognition and perception of tactile features and details of one object using both hands. Object ranged in size (up to twelve centimeters in length) and were selected so that they could be easily handled and palpable using both hands. (2) Mental imagery of the object’s shape without haptic exploration (3) Drawing of the palpated objects. EA's technique involves holding the pencil in his right hand to draw, while following the created indentations with his left hand. He cannot complete his drawings if he is not allowed to use his left hand to follow the indentations (see Figure 2A for a few examples of the objects used and drawing of objects and scribblings while inside the scanner); (4) Scribbling a "nonsense" figure using exactly the same method as for drawing. The main purpose of the study was to identify the neural correlates associated with the drawing of real objects. We designed the scribbling condition so that it would be similar (as much as possible) to the drawing condition, including the overall tactile stimulation of the fingers. We asked EA to draw nonsensical scribbles to imitate the way in which he normally draws (i.e., by holding the pencil in one hand and tactually following the created pattern with the other). The activation levels in the parietal cortex (and in the parieto-frontal network in general) were significant for the scribble condition and similar to the activation levels that were found in the drawing condition (see Figure 3B, bottom line); (5) Sensory-motor control condition: the subject was asked to move in the same way as during object palpation using both hands without receiving an object to palpate; (6) Retrieval from memory of the names of the same objects from a previously memorized list after a short cue. The purpose of the verbal memory condition was twofold. In the touching and drawing object conditions, EA named the object after recognizing it. This control condition was aimed at highlighting areas that are associated with the naming process but are devoid of any tactile or drawing component. Furthermore, we and others have previously shown that the occipital cortex of the blind is robustly activated by a similar task (e.g. (Amedi et al., 2003)). Thus, we wanted to characterize and compare the potential occipital activation associated with EA’s verbal memory with his drawing ability.

In the tactile objects, mental imagery and drawing conditions each run had 4 objects, one in each block. The other conditions also had four repetitions in each run. Task instructions were delivered (in Turkish) using MRI-compatible headphones. The time for the drawing and scribbling conditions was 15 seconds (to give EA enough time to finish the drawing or scribbling) while in all other conditions it was 12 seconds. The length of the rest baseline was 9 seconds to allow the signals to return to baseline after each condition. The drawing and scribbling tasks were performed using a Sewell kit laid on the subject's torso. Touching and sensory-motor controls required moving both hands. Verbal memory included listing the names of objects he was asked to recall prior to the scanning session (EA was tested before the scan begin to ensure he was able to perform this recall during a block time period). During the mental imagery condition, the subject was instructed to imagine the object he has just palpated and was about to draw. In total, five runs of this experiment were completed in order to increase signal to noise. All the results presented here cover this entire set of data without excluding any runs.

Data analysis

Data analysis was performed using Brain Voyager QX 1.9 software package (Brain Innovation, Maastricht, Netherlands). Preprocessing included head motion correction, slice scan time correction, linear-trend removal and high-pass temporal smoothing (minimal high pass filter of 3 cycles in time course) to remove drifts and to improve the signal to noise ratio. We also applied spatial smoothing (6mm Gaussian Kernel) to improve alignment and averaging across runs. To compute statistical parametric maps we applied a general linear model (GLM) using predictors convoluted with a typical hemodynamic response function (Boynton et al., 1996). We used a statistical threshold criterion of p < 0.05 corrected for multiple comparisons using a cluster-size threshold adjustment. This was done based on the Forman et al. Monte Carlo stimulation approach, extended to 3D data sets using the threshold size plug-in in BrainVoyager QX (For the original algorithm see (Forman et al., 1995)). The Cluster Threshold plug-in implements a randomization technique to estimate cluster-level confidence on the current overlaid 3D maps (VMR/VMP) given the current voxel-level confidence level. The intrinsic smoothness of the map was estimated (6mm Kernel) and a user-specified number of simulations (the recommended 1000 iterations in our case) were performed to assign an alpha-value to each active cluster. Based on the simulations, a minimum cluster size threshold was set for the current map to achieve a corrected p of 0.05. The resulted 3D volume (VMR/VMP) maps were then projected on EA's individual cortex reconstruction. In Figures 2 and 3 we contrasted the DRW and SCR conditions. In Figures 4 and 5 we used the following contrasts: 1. touch > Imagery, naming and sensory-motor control (purple clusters); 2. Imagery > touch, naming and sensory-motor control (blue clusters); and 3. Naming > Imagery, touch, and sensorymotor control (yellow clusters). In figure 6 we contrasted drawing objects with touching the same objects. Several other contrasts are provided in the Supplementary data.

3D recording and cortex reconstruction

Separate 3D recordings were used for surface reconstruction. High resolution 3D anatomical volumes and were collected using high-resolution T1-weighted images using a 3D-turbo field echo (TFE) T1-weighted sequence (equivalent to MP-RAGE). Typical parameters were: Field of View (FOV) 23cm (RL)×23cm (VD)×17cm (AP); Fold-over-axis: RL, data matrix: 160x160x144 zero-filled to 256 in all directions (approx 1mm isovoxel native data), TR/TE=9ms/6ms, flip angle = 8deg. Acquisition was segmented×3 in order to enhance gray/white matter contrast. The procedure for 3D reconstruction included the segmentation of the white matter using a grow-region function. The cortical surface was then inflated. The obtained activation maps were superimposed onto these inflated cortical representations.

Time course analysis

Activation was sampled from the calcarine sulcus for the drawing versus scribbling contrast and was averaged across runs (using the peak voxel in a smoothed volume, after convolution with a Gaussian kernel of 6 mm full width at half maximum). The averaged percent signal change and standard errors were then calculated for each condition selected based on the main areas activated in the drawing versus scribbling contrast. Time courses in Figures 3 and 5 are presented for purposes of depicting the hemodynamic response function of each condition and not for statistical analysis (which was carried out using the statistical parametric maps presented in Figures 2 and 4 respectively).

Hemispheric symmetry analysis

We conducted a hemispheric symmetry analysis using the following standard procedure: the volume time course was flipped along the X axis and we contrasted the regular with the mirror X symmetry brain (after spatial smoothing with a kernel of 6mm; (see also Amedi et al., 2003). This procedure thus subtracts out areas showing similar levels of activation in both hemispheres. Areas showing high levels of hemispheric specialization show strong activation in the preferred hemisphere.