Temporal evolution of brain reorganization under cross-modal training: Insights into the functional architecture of encoding and retrieval networks

Abstract

This study is based on the recent discovery of massive and well-structured cross-modal memory activation generated in the primary visual cortex (V1) of totally blind people as a result of novel training in drawing without any vision (Likova, 2012). This unexpected functional reorganization of primary visual cortex was obtained after undergoing only a week of training by the novel Cognitive-Kinesthetic Method, and was consistent across pilot groups of different categories of visual deprivation: congenitally blind, late-onset blind and blindfolded (Likova, 2014). These findings led us to implicate V1 as the implementation of the theoretical visuo-spatial ‘sketchpad’ for working memory in the human brain. Since neither the source nor the subsequent ‘recipient’ of this non-visual memory information in V1 is known, these results raise a number of important questions about the underlying functional organization of the respective encoding and retrieval networks in the brain.

To address these questions, an individual totally blind from birth was given a week of Cognitive-Kinesthetic training, accompanied by functional magnetic resonance imaging (fMRI) both before and just after training, and again after a two-month consolidation period. The results revealed a remarkable temporal sequence of training-based response reorganization in both the hippocampal complex and the temporal-lobe object processing hierarchy over the prolonged consolidation period. In particular, a pattern of profound learning-based transformations in the hippocampus was strongly reflected in V1, with the retrieval function showing massive growth as result of the Cognitive-Kinesthetic memory training and consolidation, while the initially strong hippocampal response during tactile exploration and encoding became non-existent. Furthermore, after training, an alternating patch structure in the form of a cascade of discrete ventral regions underwent radical transformations to reach complete functional specialization in terms of either encoding or retrieval as a function of the stage of learning. Moreover, several distinct patterns of learning-evolution within the patches as a function of their anatomical location, implying a complex reorganization of the object processing sub-networks through the learning period.

These first findings of complex patterns of training-based encoding/retrieval reorganization thus have broad implications for a newly emerging view of the perception/memory interactions and their reorganization through the learning process. Note that the temporal evolution of these forms of extended functional reorganization could not be uncovered with conventional assessment paradigms used in the traditional approaches to functional mapping, which may therefore have to be revisited. Moreover, as the present results are obtained in learning under life-long blindness, they imply amodal operations, transcending the usual tight association with visual processing. The present approach of memory drawing training in blindness, has the dual-advantage of being both non-visual and causal intervention, which makes it a promising ‘scalpel’ to disentangle interactions among diverse cognitive functions.

1. INTRODUCTION

1.1. Drawing as a diagnostic tool

Clinical neurology and neuropsychology have long recognized drawing as a powerful multi-component tool for the diagnosis of memory and spatiomotor dysfunctions, such as constructional apraxia, semantic dementia, Alzheimer’s disease, and other brain pathologies. Furthermore, studies in sighted individuals have reinforced the unique advantages of drawing, such as providing an explicit readout of a wide range of neural functions from a single, unified assay (e.g., Kleist, 1934; De Renzi, 1982; Grossi et al., 1999, 2002; Trojano & Conson, 2008; Schramm, 2002; Aprahamanian et al., 2010).

1.2. Drawing as a generic spatial assay

As developed in my recent studies (Likova, 2012a,b, 2013, 2014), drawing can be further conceptualized as a generic spatial assay (rather than an inherently visual one), which opens novel perspectives to the use of drawing, such as in cross-modal research and novel approaches to blindness rehabilitation. Loss of vision leads to the elimination of visuo-motor control, which dramatically affects both navigation and performance of the myriad manual tasks crucial for everyday and professional life. A core insight from my previous work is that, when blind, it becomes of key importance to develop the ability i) to generate robust memory representations and, furthermore, ii) to utilize them for replacement of the lost visual control, to replace the lost ‘eye-hand’ coordination by non-visual memory/hand coordination (Fig. 1). A potent paradigm to address this critical loss is training in drawing solely from memory of the structure to be drawn, as developed in Likova (2012a).

FIGURE 1.

Illustration of the need for targeted training to develop robust non-visual memory representations for spatiomotor coordination when the ‘eye-hand’ coordination is lost or impaired by vision loss. This conceptualization underlies the Cognitive-Kinesthetic rehabilitation training protocol used in this proposal. (from Likova, 2014)

1.3. The Cognitive-Kinesthetic training method

This new approach to training is based on a kind of drawing from tactile memory that engages the full ‘perception-cognition-action loop’ (see Discussion), which demands generation of high-resolution internal representations for accurate guidance of the drawing hand without any vision. The Cognitive-Kinesthetic training protocol leads to progressive enhancement of the individual’s ability both i) to generate and ii) to utilize such representations for the precise motor planning and motor control that is crucial for performing the memory drawing task. An illustration of the marked improvement in non-visual memory-guided motor control as demonstrated by the accurate post-training drawings can be seen in the right panel of Fig. 2 (compare to the pre-training scribbles for the same objects in the left panel).

FIGURE 2.

Left panel: Pre-training drawings of totally blind participants as recorded by our motion-capture stylus in the MRI scanner. Right panel: Post-training drawings by the same blind subjects. (After Likova, 2014.)

Complex raised-line shapes depicting objects and faces are tactilely explored and memorized, in order to be then drawn in detail guided solely by the tactile-memory representations (i.e., without access to the tactile stimuli during the drawing phase). The power of the novel training philosophy allows even congenitally blind individuals, who have never been exposed to any visual information, to learn in just a week to encode, retrieve and draw complex spatial structures such as objects and faces guided solely by tactile memory (Likova, 2012a,b, 2014).

Surprisingly, the remarkable drawing improvements were accompanied by similarly powerful changes in the BOLD response of the primary visual cortex VI. The right panel of Fig. 3 shows an example of the strong post-training increase in the fMRI activation (orange) in VI, compared to the negligible signal before training (left panel). Obtaining such results in congenital blindness implies a profound form of brainplasticityinrecruiting the ‘visual’cortex in the non-visual drawing of images from tactile memory. Furthermore, they suggest that V1 is acting as a modality-independent (or amodal) working-memory sketchpad for the readout of the memorized images for guiding the drawing hand.

FIGURE 3.

Voxel-wise activation (orange coloration) in the occipital lobe (and whole brain, inset). Left panel: Pre-training activation. Right panel: Post-training activation. Note the remarkable increase in activation after training specific to the calcarine sulcus (area V1; white outline) implying cross-modal brain re-organization driven by the non-visual Cognitive-Kinesthetic training. (After Likova, 2012a)

1.4. Cross-modal memory activation in V1 as a fulcrum of fundamental questions

The striking V1 plasticity shown in Fig. 3 raises the question of ‘who’ is activating V1, and to ‘whom’ V1 is sending its amodal-memory signal? How does the cross-modal function of V1 evolve over time through training? How are the established memory and object representation networks involved, and how do they interact? These are the key questions of brain reorganization that are addressed in the present study, framed by a set of experimental hypotheses.

2. EXPERIMENTAL HYPOTHESES

2.1. HYPOTHESIS 1: The hippocampal complex is involved in the V1 training-based reorganization

2.1.1. V1 neural connectivity

V1, although still treated in most textbooks as a unisensory bottom-up region, is now known to have extensive anatomical connectivity to cortical and subcortical regions far beyond the occipital hierarchy, revealing a wealth of interactions and complex feedback mechanisms (e.g., Ungerleider & Desimone, 1986a,b; Suzuki et al., 2000; Sur & Leamey, 2001; Falchier et al., 2002; Cappe & Barone, 2005). Also, a range of generic mechanisms hypothesized to underlie cross-modal reorganization, such as unmasking of pre-existing connections, changes in synaptic weights, long-distance feedback connections or a combination of such mechanisms (e.g., Florence and Kaas, 1995; Jones, 2000; Raineteau & Schwab, 2001; Pascual-Leone et al., 2005; Merabet et al., 2008; Van Brussel at al., 2011) may take place. These mechanisms are reviewed in more detail in Likova (2012, 2013).

2.1.2. V1 and the hippocampal complex

Beyond the established cortical and subcortical connectivity of V1, Clavagnier et al. (2004) found direct connections between V1 and the hippocampal complex. Consistent with this finding, hippocampal activation has been associated with both working memory (WM) maintenance and long-term (LTM) formation in studies on faces (Nichols et al., 2006), scenes (Schon et al., 2004), and complex 3D objects (Ranganath et al., 2005; Ranganath H, 2009). Thus, in the context of the finding of a memory-related role for V1 (Likova, 2012a, b), the hippocampal or/and perirhinal/entorhinal connections to V1 are of particular interest as these connections have been suggested to constitute possible pathway for the activation and modulation of V1 in memory retrieval, together with connections from the amygdala (Amaral et al., 2003; Clavagnier, 2004).

2.2. HYPOTHESIS 2: The ventral visual pathway is in interplay with V1 and the hippocampus

V1 and the hippocampus lie at the two extremes of an extended hierarchy (Felleman & van Essen, 1991), also known as the Posterior Representational Network (Fuster, 2008; O’Keefe & Nadel, 1978; Nadel & Peterson, 2013). V1 is thus the origin of the extended ventral pathway, which was initially described as coursing through the occipitotemporal cortex to the anterior part of the inferior temporal (IT) cortex (Ungerleider & Mishkin, 1982; Mishkin et al., 1983), with a likely extension into the ventrolateral prefrontal cortex.

However, a significantly reconceptualized neural framework of the ventral pathway has recently been proposed (Kravitz et al., 2013). This framework emphasizes the interactions between the lateral temporal regions of the ventral pathway (best known for their role in perceptual object processing) with the medial temporal lobe (MTL) regions thought to orchestrate memory functions, which include the hippocampal complex. Their evidence suggests that the ventral pathway is best understood as a recurrent occipito-temporal network containing neural object representations that are both utilized and constrained by at least six distinct cortical and subcortical systems.

Notably, all of the cortico-cortical output pathways in the Kravitz et al. scheme (the occipitotemporo-medial temporal, occipitotemporo-orbitofrontal, and occipitotemporo-ventrolateral prefrontal pathways) are critical in both long- and short-term visual memory). Moreover, all are bidirectional and originate from subregions of the infero-temporal cortex. In particular, the occipitotemporo/medial-temporal pathway, thought to support the formation of long-term (visual) memories, comprises both direct and indirect projections arising from IT and targets various structures within the MTL (specifically the perirhinal cortex, which projects in turn to both the entorhinal cortex and the CA1/prosubiculum regions of the hippocampus; Leonard et al., 1995). Thus, there is an established neural basis for an extended interplay of sub-networks of the IT pathway and the hippocampal complex during learning that we expect to be involved in cross-modal spatial learning, in particular.

2.3. HYPOTHESIS 3: Distinct patterns of learning evolution in the IT network

In view of the complex structure of at least six subcomponents of the ventral pathway architecture (Kravitz et al., 2013), it may further be hypothesised that they may serve differential functions in terms of the requisite processes of the encoding, storage, retrieval and consolidation of the cross-modal memory representations of objects to be drawn in the memory-drawing paradigm of the present study. These differential functions should be expected to be supported by different networks of, and within, cortical regions of the temporal lobes, and to be revealed by differential BOLD activation at various stages through the learning process.

3. METHODS AND PROCEDURES

3.1. Novel Memory Readout Paradigm and Rehabilitation Training

An innovative conceptual framework that we have developed previously (see Discussion) has motivated the multivalent use of memory-guided drawing, and underlies both the Cognitive-Kinesthetic Training Method and a novel Memory Readout Paradigm. These new methodologies have shown a high effectiveness in enhancing non-visual memory and brain plasticity (e.g., Likova, 2014). A major advantage of this drawing-based Memory Readout Paradigm that over more traditional methods is that it provides a direct readout of the memory contents at each stage of the learning evolution.

3.2. Experimental design

A battery of raised-line models of faces and objects was used as the drawing targets in a three-task block fMRI design, with interleaved baseline conditions. The three tasks were as follows: Tactile Exploration (E) involving perceptual exploration and encoding in memory of the model to be drawn; MemoryDraw (MD) — a memory-guided non-visual drawing task; and Scribble (S) — a motor-control and negative memory-control task. Each task duration was 20 s, with a 20 s baseline condition (NullInterval, NI) intervening between the tasks. Importantly, as opposed to the usual null periods, the subjects not only rested motionless but were instructed and practiced to clear any memory or image structure from awareness (‘blank-mind’). The start of each task or null interval was prompted by an auditory cue. The whole three-task sequence with interleaved null intervals (NI, E, NI, MD, NI, S, NI) was repeated 12 times in each 1-hour fMRI session using a new image for each repeat.

3.3. Participant and training

The congenitally blind subject CB4 was a 61-year-old right-handed female, totally blind with no light perception, who lost her vision as a result of German measles (rubella) in her expectant mother, severely and permanently damaging the fetal optic nerves. The subject gave informed consent for the experimental protocol approved by the local Institutional Review Board.

CB4 had not been previously studied by fMRI or behavioral methods of any kind. She is a sophisticated intellect and a fluent Braille reader, with a high education and lifetime employment, and was highly motivated to participate in the study. Nevertheless, she had no experience with writing or drawing, so her training to draw had to start with the basics of elementary geometric concepts such as a straight line vs a curve, right angles, etc., and was unable to reproduce any simple component through drawing. These issues were manifested at all levels of the experimental process – the tactile recognition and memorization phase, the memory recall in draw, the understanding of spatial relationships, and even the kinesthetic feedback and self-evaluation of her own performance. It nevertheless became clear that these ‘negatives’ could be turned into significant ‘positives’ that would for the first time allow us to track the full evolution of the neural development of an effective memory representation to guide the trajectory of the drawing hand.

The training was performed for 1–1.5 hours per day for 5 days during the week following the initial fMRI session. Our novel drawing method was able to inspire and to motivate CB4 to acquire the exciting drawing skill. Remarkably, after only a week of training, she advanced significantly relative to her starting level, although her capability was still not satisfactory to her. Two months later she came back for two ‘refresher’ training sessions, which she felt brought her up to an adequate skill level. To study the dynamics of the learning process, we ran fMRI before training, as well as after the prolonged period of consolidation and the ‘refresher’ session.

3.4. Innovative multimodal MRI-compatible drawing platform

As there are no previous neuroimaging studies in a similar paradigm, the needed MRI-compatible experimental instrumentation did not exist. Thus, we developed an experimental platform, integrating a set of technical innovations such as the first multisensory MRI-compatible drawing tablet (for either or both tactile and visual drawing), recording the drawing trajectory and speed in the scanner (Likova, 2010). The technological advances of this innovative multisensory platform, together with the training method and the memory paradigm are core innovations that make these memory-drawing studies in the blind possible.

3.5. MRI Data Collection, Analysis and Visualization

3.5.1. Auditory cue presentation

The auditory stimuli were presented through Resonance Technologies Serene Sound earphones (Resonance Technologies, Salem, Massachusetts). To reduce scanner noise, this equipment employs external ear protectors with perforated ear-plugs that conduct the auditory cues directly into the auditory passage while blocking much of the scanner noise.

3.5.2. FMRI acquisition

MR data were collected on a Siemens Trio 3T scanner equipped with 8-channel EXCITE capability, a visual stimulus presentation system, earphones for providing the auditory feedback, response buttons. A high-resolution anatomical (Tl-weighted) volume scan of the entire brain was obtained for each observer (voxel size = 0.8 × 0.8 × 0.8 mm). The fMRI blood-oxygenation-level-dependent (BOLD) responses were collected with EPI acquisition from the whole head coil. There were 34 axial slices at 2 s TR, with TE of 28 ms and flip angle of 80, providing 3.0 × 3.0 × 3.5 mm voxels throughout the whole brain. The functional activations were processed for slice-time correction and two-phase motion correction: within-scan correction and between-scan correction of each scan to the reference scan, both of which used mrVista (Stanford Vision and Imaging Science and Technology) to correct for six parameters of rigid-body motion. An anatomical segmentation algorithm (mrGray) was applied to the T1 scan, ensuring localization of the signal within the cortical gray matter close to the activated neurons and greatly reducing the blood drain artifacts that afflict studies in which cortical segmentation is not used. The activation was specified in terms of the statistical significance (p < 0.05) of the signal in each voxel (after Bonferroni correction for the number of gray matter voxels).

3.5.3. Pre-processing

The raw DICOM-format data from each fMRI scan were converted to a 4D NIFTI file. Using FSL tools, we run within-scan and between-scan motion corrections, bringing all functional data into alignment with the fMRI volume acquired closest in time to the Tl-weighted “inplane” anatomy. Then the functional data were averaged across scans, resulting in a single 4D NIFTI file for each test condition.

3.5.4. FMRI time course analyses

The data were analyzed to estimate the effective neural activation amplitudes (for each task across the l2 repeats of the 3-task sequence in a one-hour scan) by the following procedure. A General Linear Model (GLM) consisting of a (3+1)-parameter boxcar neural activation model convolved with an estimated hemodynamic response function (HRF) was fitted to the BOLD responses for each 3-task sequence, combined with a set of 1 sec boxcars corresponding to the auditory cue presentations and an additive 4^th-order polynomial to capture low-frequency drift in the BOLD signal. Examples of the drift-corrected BOLD signals and GLM model fits are shown in the upper panels of Fig. 4. The three parameters of the activation model were the amplitude parameters of the boxcar activation strengths for each of the three task periods, combined with one parameter for the amplitudes of the auditory signals. (The HRF parameters were determined once per session by optimizing this model to a subset of gray matter voxels identified as most responsive to the task/rest alternation frequency in this experiment.)

FIGURE 4.

Upper panels: Comparison of BOLD responses and model fits for pre/post fMRI activation to evaluate memory function: in V1 (A) and the hippocampus (B) during three tasks: Tactile Exploration (E), Memory Draw (MD) and control Scribbling (S). Lower panels: Estimated pre/post activation strength in the three tasks in V1 (A) and hippocampus (B). See text for details. Error bars show 1 SEM of the fits across 12 repeats.

3.5.5. Voxel-wise parametric maps

For each task, statistical parametric maps are based on the estimated activation amplitudes from the above GLM in each voxel, with all task-regressors optimized jointly to the detrended BOLD waveform. These maps can be viewed in the 3D brain volume or projected onto 3D views of the inflated cortex or on flatmaps of cortical regions of particular interest.

3.5.6. ROI activation analysis

The effective neural activation amplitudes (bar graphs in Fig. 4) for each condition in each region of interest (ROI) were estimated by the same GLM procedure but now applied to the average signal across all voxels within the ROI. This procedure also provided high-quality time courses for evaluation of the response dynamics and its comparison across tasks and stages of training.

The confidence intervals were defined by the amplitude variability across the 12 repeats of the 3-task sequence in each one-hour scan. The dashed lines and the error bars represent confidence intervals for two different forms of statistical comparison of the activation levels (the activation levels refer to the beta weights for the event types in the GLM): (i) The dashed lines represent the 99% ‘zero’ confidence interval (p < 0.01, uncorrected) within which the activation amplitudes are not significantly different from zero (i.e., relative to the noise variance for no stimulus-related activation defined as the residual variance after the GLM model fit of the FMRI time course analyses section described above); thus this statistical criterion is designed to indicate the significance of each individual activation (at p < 0.05, corrected for multiple applications within each figure); (ii) The error bars are ‘difference’ confidence intervals designed to illustrate the t-test to assess the significance of differences between activation levels in each figure (i.e., the differences are not significant unless they exceed the confidence intervals for both compared activations), again at p < 0.05 (corrected for multiple applications). In the text, all ROI-comparisons are specified as significant by the t-test using a statistical criterion threshold of p < 0.05 corrected for multiple comparisons.

3.5.7. Topographic maps in the blind

On the one hand, no informed analysis of the visual cortex could be done without knowledge of its retinotopic and functional organization; on the other hand, no retinotopic mapping or visual localizers is possible in the blind, so it is a challenge to localize any specific visual area. To resolve this issue and determine the borders of the primary visual cortex in blind participants, we combined three different approaches. First, the Freesurfer probability map atlas (see http://surfer.nmr.mgh.harvard.edu/fswiki/BrodmannAreaMaps ) was used to transform the primary visual area map back to the blind subject’s brain through the Freesurfer spherical surface registration procedure. To verify the process, this procedure was first run in sighted subjects, for which we already had individual retinotopic maps; the borders of the retinotopically defined V1 aligned fairly accurately with those from the Freesurfer. Second, the location of the V1 ROI was verified by intersecting with its anatomical markers (the calcarine sulcus). And third, an innovative 14-step procedure (Likova, 2010, 2012a) was used to warp the brains of sighted and blind subjects to the same MNI brain. This innovative three-way comparison enabled us to estimate the corresponding topographic regions locations in the blind brain. All methods converged very well to the definition of the V1 ROI in the bran of the congenitally blind participant.

4. RESULTS

4.1. Is the hippocampal complex involved in the V1 training-based reorganization?

The Cognitive-Kinesthetic training intervention allowed us to test the hypothesis that the hippocampal complex is involved in the V1 reorganization. The pattern of activation across the three tasks of tactile exploration (E), tactile-memory guided drawing (MD) and scribble (S) assessed before the week-long training regimen and after the prolonged post-training consolidation period are shown in Fig. 4.

The black lines in the upper panels of Fig. 4 show the average time-course of activation over three tasks (see caption). The colored lines show the fits of a GLM model of the activation waveform (see General Methods) amplitude-scaled to best fit the respective activation regions.

4.1.1. T raining effect in V1

Before training, V1 sho ws only a weak activation (PRE) with an anomalous waveform across all three tasks. However, following the training and a two-month consolidation period (POST), V1 develops strong activation during MD, and a weaker although significant activation during E, but not during the S task.

4.1.2. Training effect in the hippocampus

Before training, the hippocampus is activated in the encoding task, but is actually suppressed in the retrieval task. However, after training there is a major switch in the behavior of the hippocampus, which now shows massive activation during retrieval (MD), but loses any activation in the exploration (E) task. Note the similarity in activation patterns after training, which suggests a post-training relationship between V1 and the hippocampus during memory retrieval (but not during trained exploration).

4.2. Is there an interplay of the ventral visual pathway with V1 and the hippocampus?

Hypothesis 2 above, that our experimental paradigm would reveal a differential pattern of the evolution of learning across the three tasks even within single regions, was probed by comparing the patterns of activation for the encoding and retrieval tasks (E and MD) across the sequence of fMRI sessions. Fig. 5B, C shows an example of the complete response transformation observed in a single IT region over the three time points we measured.

FIGURE 5.

Learning evolution of the activation in an IT region during two tasks: Tactile exploration of raised-line objects (E) and tactile-memory retrieval (MD), and ti me-series correlation analysis of area connectivities.

A: fMRI design. Pre (blue): pre-training scan; Post-1 (green): immediate post-training scan; Post-2 (red): post-consolidation scan run two months later.

B: Response amplitudes for the tactile exploration E task in the IT region across these three periods. Before training (blue) this region is strongly activated during exploration and encoding (and not at all during retrieval in the memory drawing task, see C); immediately after a week of training (green), this region shows an increased response, but after the two-month consolidation (red) the amplitude drops close to zero.

C: The responses during the memory retrieval MD task are very different. The pre-training scan (blue) shows no significant activation, increasing to a strong response after the week of training (green) that is maintained at this high level following the two-month consolidation (red), consistent with the demanding retrieval function crucial for the high-quality memory drawing. Error bars are 1 SEM over 12 repeats of each condition.

D: Time-series correlation analysis of area connectivities after consolidation, showing strong correlation groupings between V1, hippocampus and two IT regions, vs other groups of uncorrelated and negatively correlated IT regions.

Before training (blue), the region responds as a classic ‘object-processing’ area, i.e., Strongly during the tactile image exploration task, but not at all during retrieval in memory drawing. However, after only a week of the training (green), this area seems to serve a dual function of both encoding and retrieval functions. Following the two-month consolidation period (red), a different story is revealed - the region no longer responds during exploration but remains strongly involved during retrieval.

Thus, the response of the region undergoes a complete transformation after training and further changes after a subsequent consolidation period. Note that, if studied outside the training context, i.e., in a typical “single-time-point” paradigm for the functional mapping (i.e., only before or just after completion of training, or only after prolonged consolidation), completely different inferences would be drawn for the function of this region.

4.3. Are there distinct patterns of learning evolution along the IT network? Alternating patchy structure of dissociated, non-overlapping encoding and retrieval regions.

The third hypothesis, that the learning-based functional evolution would be manifested differentially in the network of cortical subregions in IT, was tested by defining a sequence of ROIs along the posterior-anterior axis of IT.

4.3.1. Alternating patch structure: A cascade of functionally dissociated encoding and retrieval regions.

A remarkable, unexpected result of the post-consolidation fMRI was to find a cascade of ventral regions along the posterior-anterior axis that have reached a complete functional dissociation between the exploration (encoding) and retrieval (memory drawing) as a function of learning stage. This astonishing pattern of alternating patch organization is illustrated in Fig. 6.

FIGURE 6.

The remarkable pattern of post-training organization: a cascade of functionally dissociated encoding (E) and retrieval (MD) regions. As result of training, the cascade of ventral regions underwent radical transformations to reach distinct functional specialization for the encoding and retrieval processes (see Fig. 7 also).

This organization is highly reminiscent of that of the face-processing network (Moeller et al., 2008; Tsao et al., 2008a,b), although, in contrast to the present study, such functional compartmentalization have neither previously been differentiated in terms of memory processes such as encoding and retrieval, nor their roles been studied as a function of the stage of learning. Furthermore, the architecturally similar sequence found here emerged in the congenitally blind brain as result of the training and post-training consolidation (Fig. 6).

4.3.2. A cross-modal functional evolution within each region of the infero-temporal ‘cascade’ described in 4.3.1.

Figure 7 shows an analysis of the cross-modal functional evolution for a ‘cascade’ of 5 ROIs (IT1–5) along the extended object-processing/memory networks in left IT. Most importantly, these data reveal dramatic response changes not only as a function of area location along IT, but even within each area as function of both task and learning stage. Several of these IT areas have patterns of temporal evolution resembling those of the hippocampus (Hc, shown as the hatched bars at right in Fig. 7), suggesting that they form part of a hippocampal perception/memory network. Others show distinct evolution patterns, implying that these ROIs are participating in different sub-networks (see Kravitz et al., 2013 for details on known sub-networks related to the ventral pathway).

FIGURE 7.

Learning evolution of the activation in a sequence of ROIs (IT1–5) fro the posterior to anterior IT through three time-points before (blue bars), just after (green bars) and 2 months after (red bars) a week of the Cognitive-Kinesthetic training in tactile drawing for two tasks: tactile exploration & memorization (left panel), and tactile memory drawing requiring retrieval (right panel). The corresponding activations for the hippocampal (Hc) ROI from Fig. 4B is shown as the hatched bars at right. Error bars are 1 SEM over the 12 repeats of each condition.

5. DISCUSSION

5.1. Conceptual framework

This paper addresses the issue of the functional architecture of encoding and retrieval networks involved in brain reorganization during active spatial learning. Both the learning method and the interpretation of the results are based on a conceptual framework (Likova, 2012a, 2014) for the processes required to generate the level of brain organization required to support active spatial memory (rather than simple image recognition), as follows:

1. Space transcends any specific sensory modality. As emphasized by the phenomenon of drawing by the blind (e.g., Kennedy, 1993, 2000, 2003, 2006; Ponchilla, 2008; Likova, 2010; 2012a,b; 2013; 2014), space and spatial structure are not represented solely by vision. The visual system may be the modality best suited to process spatial information, but it is not the only one. When deprived of visual input, the brain is capable of employing the ‘free’ visual resources in the most relevant way for spatial tasks. (As there is an ambiguity in the use of the term ‘spatial’, particularly in the working memory and imagery literature, note that here ‘spatial’ refers to the perception of any spatial structure – 2D or 3D, static or dynamic – independently of the sensory modality exploring it.) It can thus be seen that drawing deals with spatial structures in this general sense, and consequently, drawing has the advantage that it can readily be ‘translated’ from its traditional visual into a tactile form.

2. Closing the perception-cognition-action loop is a powerful amplifier for learning. Drawing is just such an amplifier task, fully involving all three components of this ‘loop’ while precisely orchestrating multiple brain mechanisms to achieve an accurate result. This integrated capability leads to the hypothesis that drawing should be particularly effective in enhancing spatial learning and memory.

3. Training in highly engaging tasks with fun and inspiring outcomes is a fruitful paradigm for driving brain reorganization and assessing its temporal evolution. Drawing is ideally suited for this role, particularly when studied under the unusual circumstances of visual deprivation.

4. Tasks that demand detailed re-expression of memory force the development of a precise and robust memory representation. Drawing-from-memory requires just such explicit re-expression through the motor loop, and hence, it demands the development of ‘high-resolution’ internal representations to be communicated back through the drawing act.

5. Direct memory ‘readout’ would greatly benefit memory-studying paradigms. Drawing from memory provides such direct memory ‘readout’ in the form of the drawn image, as it ensures an explicit expression of the remembered information by externalization of the mental representation that guides the drawing hand.

5.2. How does V1 interact with other aspects of the learning and memory networks? Intrinsic involvement of primary visual cortex in the learning/memory networks

The typical approach to brain function is to treat primary visual cortex (V1) as a predominantly bottom-up input of the object processing stream), although with multiple feedback connections from higher processing regions. Recent studies have begun to delineate the role that some of these pathways may play in memory representations for visual information. However, the discovery of the activation of V1 in blind and blindfolded individuals during spatial learning (e.g., Likova, 2012a, 2013), indicates that V1 may play a special role in spatial learning under non-visual conditions. In anyone who has or had effective sight during their lives, it can be argued that V1 developed its spatial representation function as a consequence of its role in bottom-up processing of visual information, in conjunction with the manipulations of that information as the child interacts actively with the world. The same is not the case, however, with congenitally blind individuals, which are the particular focus of this and my previous related studies (Likova, 2012a,b, 2014). In such cases, there has never been any visual information to activate V1 or to target its role in tactile spatial processing, and indeed the lack of significant V1 activation prior to training supports the idea that it is lying fallow with little involvement in such tasks. It is only with intensive training on precise spatial-memory tasks, such as the present drawing paradigm in particular, that V1 becomes activated in congenital blindness, revealing its role as the amodal-spatial memory-sketchpad described in the Introduction (Likova, 2012)

What has not been previously addressed either in blind or sighted brain function is how V1 interacts with other aspects of the learning and memory networks. Is the learning function focused on V1 per se or does it operate in interplay with classic learning areas such as the hippocampus? The results of Fig. 4 (and Fig. 5D) indicate that the pattern of profound learning-based transformations in the hippocampus is strongly reflected in V1, with the retrieval function of the MemoryDraw task being the one that shows massive growth as result of the Cognitive-Kinesthetic training and consolidation. While this result does not establish the causal direction of the linkage, it strongly supports the idea that V1 is tightly coupled with the hippocampus in the use of memory images to guide the drawing details.

5.3. Alternating patch structure: A cascade of functionally dissociated encoding and retrieval regions.

The results depicted in Fig. 6 amplify the concept of a sequence of discrete processing regions extending along the ventral temporal lobe for the face-processing network of the macaque monkey and human (Moeller et al., 2008; Tsao et al., 2008a,b). This type of organization has been studied in various ways more recently, such as in terms of viewpoint generalization (Freiwald & Tsao, 2010). Forms of the face-processing network reorganization as a result of a range of forms of clinical disruption has been described in a number of studies (Bonelli et al., 2010; Powell et al., 2010; Tsapkini et al., 2011). However, the patch type of functional organization has not previously been differentiated in terms of encoding and retrieval functions, either in terms of its organization in congenital blindness or in terms of its within-patch reorganization as a function of training.

5.4. Distinct patterns of learning-based functional evolution along the ventral stream.

Hypotheses 2 and 3 represent two aspects of the evolution of the IT network during memory development. Hypothesis 2 is based on the concept that memory encoding is an early phase of the learning process, whereas memory consolidation takes place once the encoding has occurred. In the linguistic domain, it is analogous to the idea that when listening to a poem it is first simply heard (sensory input), then understood (encoding), then stored and committed to memory (consolidation). Although it may be adequately understood early in the process, that understanding is not sufficient for it to be retrieved and recited (except perhaps sporadically). Accurate recital requires a more extended process of learning and retrieval attempts until the full retrieval capability is achieved. A similar sequence of sensory exploration, encoding, storage and consolidation is hypothesized to take place with the blind drawing paradigm.

The results of Fig. 5 support the idea that some IT regions exhibit the hypothesized differential time-sequence. For example, the Lt4 region initially (before memory drawing training) responds as a classic ‘object-processing’ area, i.e., strongly during the tactile image exploration task, but not at all during retrieval in memory drawing. However, after only a week of the training, this area seems to serve a dual function of both encoding and retrieval functions. Following the two-month consolidation period, a radically different story is revealed - the region no longer responds during exploration but remains strongly involved during retrieval. This mean that if probed before training only, this area would be identified as an “object processing” or encoding area; if, however, probed only immediately after the training, it would be identified as an area equally involved in both encoding and retrieval; but if tested at a later time point, e.g., two months after the training is completed, it would be pronounced being a retrieval only area. Thus, the finding fo such dramatic reorganization of functional response, raises fundamental issues to functional brain mapping approaches.

As for V1, this behavior is analyzed in terms of the relation to the hippocampal region (Fig. 5D), but now with respect to the full dynamics across the untrained, trained and consolidation time-points. The hippocampal region shows early growth in activation for the encoding task, which has evaporated by the end of the consolidation period, whereas the boost in activation by the same region is maintained at full strength during the retrieval task. The correlation matrix of Fig. 5 indicates that such behavior is typical of one set of IT ROIs that have a high correlation with this pattern of hippocampal activation, but that there are two other groupings that are, respectively, anti-correlated with this pattern of temporal evolution (i.e., inhibited in a similar pattern to the activation) and un-correlated with it (i.e., undergoing a different pattern of evolution, which may or may not be similar across the uncorrelated group).

The results revealed, to our best knowledge for the first time, a learning-based organization in the form of a posterior-anterior cascade of patch structure of regions achieving a complete functional dissociation between encoding and retrieval as a function of the stage of learning (Fig. 6). Moreover, the present results support Hypothesis 3 in showing several different patterns of learning-evolution within the patches as a function of their anatomical location (Fig. 7), implying a complex reorganization of the object processing sub-networks through the learning period. This reorganization is consistent with the novel model of the ventral stream as incorporating a number of recursive subnetworks, instead of being a straightforward bottom-up pathway (Kravitz et al., 2013).

Furthermore, the dynamic interplay between a sequence of cortical regions lying along IT with the hippocampus and the ‘memory-sketchpad’ in V1 found in the present study further expands the view of IT involvement in object learning and memory. In particular, it reveals the multifold temporal evolution of cross-modal network reorganization involved in the differential processes of encoding and retrieval as the learning progresses.

5.5. Amodal nature of perception/working-memory/long-term memory commonalities

Interestingly in the clinical context, dependence not only on the antero-lateral temporal but also on the MTL (hippocampus and perirhinal cortex, in particular) has been found in semantic dementia, diagnosed by deficits on drawing tasks, reflecting the emerging view on processing commonalities between the lateral and medial temporal lobes (e.g., Graham et al., 2010; Barense et al., 2011). Taken together the results above support the hypothesis of close interactions and shared structures between cognitive functions frequently treated in isolation, notably perception, working-memory and long-term memory. As the present results were obtained in a case of life-long blindness, they imply that such commonalities in the object-processing hierarchy operate at an amodal level, transcending the usual association with visual processing. Thus, the dual advantage of the present experimental approach of memory drawing training in a congenitally blind individual being both non-visual and a causal intervention, makes it a powerful ‘scalpel’ to disentangle the interactions among these diverse cognitive functions.

In summary, the current results have particularly significant implications both for emerging theories of functional brain architecture and for the development of informed blindness rehabilitation through an enhanced understanding of the underlying neural mechanisms.

6. CONCLUSION

To explore cortical network reorganization during cross-modal memory drawing learning, fMRI was conducted in a congenitally blind individual undergoing a week of Cognitive-Kinesthetic Drawing training followed by a two-month consolidation period. The results revealed a remarkable temporal sequence of training-based response reorganization throughout the temporal-lobe object processing hierarchy. In particular, the pattern of responses to encoding and retrieval aspects of the learning process in both the hippocampal and ventral pathway responses underwent complete transformations that continued over the two-month consolidation period, despite there being no further training during that period.

Note that this form of extended temporal evolution of the reorganization process could not be uncovered with the single-time-point paradigms typical to functional specialization studies, or with conventional pre-post assessment paradigms. The present results, showing within-area transformations of the functional activation pattern through the temporal trajectory of learning, indicate that there may be fundamental issues with the traditional approaches to functional mapping, and these may have to be revisited. To our best knowledge, moreover, this is the first study to show differential patterns of learning-evolution of the sub-networks of the temporal-lobe object processing hierarchy, and their linkages to the key loci of the V1 ‘memory-sketchpad’ and the hippocampus that are the most specialized regions of this network. The findings in this study thus have significant implications for a newly emerging view of the interactive perception/memory organization and its network reorganization through the learning process.