Multisensory convergence of visual and haptic object preference across development

Abstract

Visuohaptic inputs offer redundant and complementary information regarding an object's geometrical structure. The integration of these inputs facilitates object recognition in adults. While the ability to recognize objects in the environment both visually and haptically develops early on, the development of the neural mechanisms for integrating visual and haptic object shape information remains unknown. In the present study, we used functional Magnetic Resonance Imaging (fMRI) in three groups of participants, 4 to 5.5 year olds, 7 to 8.5 year olds, and adults. Participants were tested in a block design involving visual exploration of two-dimensional images of common objects and real textures, and haptic exploration of their three-dimensional counterparts. As in previous studies, object preference was defined as a greater BOLD response for objects than textures. The analyses specifically target two sites of known visuohaptic convergence in adults: the lateral occipital tactile-visual region (LOtv) and intraparietal sulcus (IPS). Results indicated that the LOtv is involved in visuohaptic object recognition early on. More importantly, object preference in the LOtv became increasingly visually dominant with development. Despite previous reports that the lateral occipital complex (LOC) is adult-like by 8 years, these findings indicate that at least part of the LOC is not. Whole-brain maps showed overlap between adults and both groups of children in the LOC. However, the overlap did not build incrementally from the younger to the older group, suggesting that visuohaptic object preference does not develop in an additive manner. Taken together, the results show that the development of neural substrates for visuohaptic recognition is protracted compared to substrates that are primarily visual or haptic.

1. Introduction

Object recognition is ubiquitous and essential for interacting with the surrounding environment. Information from various sensory channels converges to guide perception and action. Percepts are driven by the interaction of multisensory inputs and past experiences, which are the direct result of our choice of actions. Vision and haptics are two modalities in particular that offer redundant and complementary information about the geometrical properties of objects (Amedi et al., 2005). The most synergistic interactions between the visual and haptic systems occur predominantly during shape and texture processing (James et al., 2005).

Within the adult ventral visual stream, a vigorous investigation surrounds the organization and function of the lateral occipital complex (LOC), which is located within the lateral occipital-temporal cortex and encompasses the middle occipital areas and fusiform gyrus (Grill-Spector, Golarai, & Gabrieli, 2008; Grill-Spector, Kourtzi, & Kanwisher, 2001; James et al., 2002a, 2002b; Kourtzi & Kanwisher, 2001; Malach et al., 1995; Tootell et al., 1996). The LOC is intimately involved in visual object recognition. It responds strongly to three-dimensional objects and two-dimensional images of objects, and responds only weakly to scrambled versions of 2-D object images or to textures (Amedi et al., 2001; Malach et al., 1995). Additionally, the LOC responds not only to visual presentations of objects, but also to haptic presentations of those objects (Amedi et al., 2001, 2002; James et al., 2002b; Stilla & Sathian, 2008).

Haptically, adults are highly efficient at extracting properties of objects, and exploiting that information for recognition (Klatzky, Lederman, & Reed, 1987; Lederman & Klatzky, 1993; Lederman & Klatzky, 1990). During haptic object recognition, the exploration of three-dimensional objects produces neural activation in the primary and secondary somatosensory cortices, as well as areas of the occipital and parietal cortices that are associated with visual object recognition (Amedi et al., 2005; James et al., 2002a, 2002b; Reed, Shoham, & Halgren, 2004; Sathian et al., 1997; Stilla & Sathian, 2008). In addition to the LOC, the intraparietal sulcus (IPS) is recruited more during overall shape perception as compared to the perception of basic shape features (e.g., the degree of curvature, edge length, T-junctions) (Bodegard et al., 2001). Moreover, activation in this latter region has been found in response to haptic stimuli comprised of everyday, common objects (Amedi et al., 2001, 2002, 2005; Deibert et al., 1999; Reed, Shoham, & Halgren, 2004), as well as simple, geometrical shapes (Bodegard et al., 2001; Roland et al., 1998).

Extending beyond unisensory object recognition, there has been a significant amount of research concerning visuohaptic convergence in the LOC and the IPS. Many studies have shown that the convergence of visual and haptic inputs for object preference in adults occurs at two particular sites: the lateral occipital tactile-visual region (LOtv, located within the LOC) for object recognition, and the anterior/middle aspects of the IPS for object-directed motor actions (Amedi et al., 2001, 2002; James et al., 2002a; James & Kim, 2010; Stilla & Sathian, 2008). Several more recent studies have shown that information from vision and haptics is also combined at these sites (Kim & James, 2010; Kim, Stevenson, & James, 2012; Tal & Amedi, 2009).

Yet, it remains unknown how and when, developmentally, multisensory information comes to converge on the LOC and the IPS. These questions can be addressed by examining the neural substrates of visual and haptic object recognition in children. This approach will also provide a window into the role of active perception and experience in this convergence. Historically, perceptual development has been conceptualized in terms of sensorimotor interactions based on the behaviors of infants and young children in tasks that require some degree of multisensory perception. For instance, as soon as infants can move their arms and hands, they are able to act on objects and receive multisensory information about those objects. As revealed by a large body of literature, these types of self-generated actions produce statistical regularities between the sensory and motor systems, and thus, play a critical role in perceptual learning (Held & Hein, 1963; James, 1890; Lungarella & Sporns, 2005, 2006; Smith, 2005). This form of active perception has been demonstrated in many behavioral studies examining the development of visual and haptic perception. For example, the development of visual perception has been shown to be highly influenced by the manner in which young children haptically explore objects (Bushnell & Boudreau, 1993; Ruff, 1984, 1986, 1989). Recent research has found increases in measures of visual object recognition in 24 month olds who demonstrate more adult-like manual exploration patterns (James et al., 2013). Additional studies examining the development of haptic perception in 4- to 5-year-old children have consistently indicated stereotypically adult-like patterns of haptic exploration by this age, as well as highly successful haptic object recognition abilities in the absence of vision (Bushnell & Baxt, 1999; Kalagher & Jones, 2011; Lederman & Klatzky, 1987). In spite of these achievements, however, a protracted development of visual processing of object shape in children, either behavioral or neural, may have cascading effects on the developmental trajectory of visuohaptic convergence overall.

Behaviorally, a bias for categorizing objects based on shape emerges only around 24 months of age (Jones & Smith, 2005; Pereira & Smith, 2009; Smith, 2009; Son, Smith, & Goldstone, 2008; and many others). Visual processing of object shape continues to follow this delayed trajectory, particularly for complex objects. That is, even much older children aged 6 to 8 years struggle with recognizing some complex classes of objects such as faces (Mondloch, Maurer, & Ahola, 2006; however, see Crookes and McKone, 2009), as well as objects from unusual views (Bova et al., 2007; Juttner, Muller, & Rentschler, 2006; Mondloch et al., 2003; Mondloch, Le Grand, & Maurer, 2002; for a review, see Nishimura, Scherf, & Behrmann, 2009). Further, there is evidence to suggest that tool recognition performance may also have a protracted development, becoming adult-like only during early adolescence (Bova et al., 2007; Mounoud et al., 2007). Taken together, these behavioral delays in children's visual recognition abilities, particularly of complex objects, suggest a protracted development of the occipital-temporal cortex, and specifically of the LOC.

While research investigating the neural correlates of object recognition in children is relatively scarce compared to adults, evidence suggests that the LOC is generally recruited for visual object perception in an adult-like manner in children by 5 to 8 years of age (Dekker et al., 2011; Golarai et al., 2007; Grill-Spector, Golarai, & Gabrieli, 2008; Scherf et al., 2007), and during word reading in literate children (Houdé et al., 2010; Schlaggar & McCandliss, 2007; Shaywitz et al., 2004). However, this is not to say that all object processing is adult-like at that age. Rather, the same studies also showed that neural processing of faces and scenes (Golari et al., 2007; Scherf et al. 2007), as well as animals and objects from unusual views (Dekker et al., 2011), continued to develop throughout childhood and into adolescence. This difference in the trajectory of neural development for different classes of objects could be interpreted as due to increased experience through active perception and exploration. In this case, it suggests that object recognition in the LOC is perhaps modulated by this type of experience. Indeed, studies suggest that recruitment of this region is experience-based; findings have shown that letter stimuli activate the LOC in an adult-like way in children only after active (printing) practice, but not in children that only have visual experience (James, 2010; James & Engelhardt, 2012). Moreover, expertise with a limited class of objects such as Pokémon characters results in enhanced recruitment in several areas of the ventral occipital-temporal cortex compared to nonexpert children (James & James, 2013). In sum, although the LOC is recruited during visual object recognition early on during development, its response profile appears to be continually shaped by experience. Given the limited number of studies that have investigated object processing in the LOC through development, and that they have only been done visually, it is difficult to know how and when vision converges with haptics for multisensory object recognition in this region, or even the neural substrates that underlie haptic recognition itself.

To address these gaps in knowledge, the present study aimed to: a) map out the neural systems that underlie visual and haptic processing of common objects in children sampled from two developmental age groups—specifically, 4 to 5.5 year olds and 7 to 8.5 year olds; and b) examine those neural systems for evidence of convergence at different stages of development and compare them to adults. According to previous findings, children between 8 to 10 years of age begin to show adult-like behavioral patterns for visuohaptic integration and form discrimination (e.g., size and orientation; Gori et al., 2008). As this integration of form information does not become statistically optimal until 8 to 10 years (Gori et al., 2008), this indicates a developmental shift in processing prior to 8 years. Moreover, several fMRI studies have suggested that the LOC becomes adult-like between 5 to 8 years in terms of object recognition (Grill-Spector, Golarai, & Gabrieli, 2008; Golarai et al., 2007; Scherf et al., 2007), which indicates a developmental transition prior to 5 years. Given these psychophysical and neuroimaging findings, we therefore selected age groups that would capture the transitional periods for integration and recognition prior to becoming adult-like for comparison with adults.

Our hypotheses were based on contrasts between common objects and real textures to obtain a measure of object preference for both visual and haptic modalities. We predicted that the LOC and perhaps the IPS would be key regions to show developmental trends for visuohaptic convergence. Additionally, we made three specific predictions regarding the division of labor between visual and haptic object shape preference in the LOC. First, we hypothesized that visual object preference would reach adult-like levels by 5 to 8 years of age (Grill-Spector, Golarai, & Gabrieli, 2008; Golarai et al., 2007; Scherf et al., 2007), particularly due to our use of common objects. Second, though the neural development of haptic object recognition is unknown, we hypothesized that it would follow a similar trajectory as vision. Third, we predicted that visuohaptic convergence would follow a protracted development compared to vision or haptics alone. Similar to the relatively delayed development of visual recognition of over-learned or more complex classes of objects, visuohaptic convergence of object preference may be a more complex form of processing than unisensory object preference. Just as there are subregions in the occipital-temporal cortex that process different visual object categories (e.g., faces, places), there is also a specific subregion that is involved in processing the combination of both visual and haptic object shape, namely the LOtv. It may be that this LOtv subregion has a more protracted development than the LOC proper.

2. Materials and methods

2.1 Participants

Participants were recruited from three age groups: 4 to 5.5 years (N = 15, 9 female; mean age = 4.9 years, SD = 0.5 years), 7 to 8.5 years (N = 13, 6 female; mean age = 8.1 years, SD = 0.5 years), and adults (N = 8, 3 female; mean age = 26.9 years, SD = 4.2 years). Participants had normal or corrected to normal vision, and had no known history of psychological disorders; all were healthy and met the criteria for MRI scanning. Written informed consent was obtained from the parents and adult participants, and written informed assent was obtained from the children aged 7 years or older. Parents and children were compensated with a gift certificate and a small toy; adult participants were compensated with $25. This research was approved by the Indiana University Protection of Human Participants Board.

2.2 Stimuli

The stimuli consisted of 8 objects and 8 textures that were explored visually and haptically. Four additional objects and textures were used during training. Stimuli were equally colorful and salient so as to maintain children's interest during the experiment, and included objects and textures commonly found in children's environments. The objects were three-dimensional, rigid, and solid bodies that were controlled for texture such that all objects were smooth, and for size to ensure that even the 4- to 5.5-year-old children could fit both of their hands around them during haptic exploration. The textures were real, and consisted of two-dimensional square sheets to control for shape (see Table 1 for specific stimuli and dimensions). All objects and textures were photographed at a typical viewing angle against a black background to facilitate recognition during visual exploration. Participants did not see or feel the test objects or textures prior to the scan.

Table 1.

Stimuli and dimensions of objects and textures.

Objects		Textures
	Dimensions (l × w × d in cm)		Dimensions (l × w in cm)
Eraser	6.0 × 1.5 × 2.0	Sponge	10.5 × 10.5
Ball	4.5 × 4.5 × 4.5	Feathers	10.5 × 10.5
Cup	7.5 × 4.5 × 4.5	Felt	10.5 × 10.5
Star	4.5 × 4.5 × 2.0	Scrubber	10.5 × 10.5
Whistle	13.5 × 1.5 × 1.5	Corkboard	10.5 × 10.5
Sunglasses	12.0 × 4.0 × 2.5	Plastic sheet	10.5 × 10.5
Plate	11.5 × 11.5 × 1.5	Fake fur	10.5 × 10.5
Crayon	10.0 × 1.0 × 1.0	Drawer liner	10.5 × 10.5
Ice cream cone*	11.0 × 4.0 × 4.0	Paper*	10.5 × 10.5
Toothbrush*	12.5 × 2.0 × 1.5	Aluminum foil*	10.5 × 10.5

^*Denotes training stimuli.

2.3 Neuroimaging procedure

After screening and obtaining informed consent, all participants were acclimated to an MRI environment. Children watched as a short cartoon was played on a screen in the MRI simulator, an artificial MRI environment with the same dimensions and sounds as the actual MRI environment. Participants were then trained in the experiment. They were instructed to lie still, and an MRI-safe lap desk was placed over their midsection. A cape was placed over their torso and arms, and was tucked under their chin. The cape covered the lap desk and allowed the participants to feel the stimuli with their hands without being able to see them. Participants were instructed to look at the stimulus presented on the screen when they saw the word “LOOK” and to feel the stimulus that was attached to the lap desk with a piece of Velcro when they saw the word “FEEL.” It was explained to them that this was the procedure they would follow in the actual testing environment. Once the participants were comfortable in this setting and could perform the task efficiently, they were then introduced to the actual MRI environment.

In the MRI, participants were again given the instructions, and the lap desk and cape were placed over their midsection. All visual stimuli were back-displayed via a Mitsubishi XL30 projector onto a screen located behind the participants in the bore of the MRI; this screen was viewed through a mirror that was placed on top of the head coil. Instructions and visual stimuli were presented using SuperLab Pro 2.0.4 software from an Apple MacBook laptop. A high-resolution anatomical scan was first acquired while participants watched a cartoon. Upon completion of this scan, the functional scans were acquired.

During the functional scans, participants were tested in a block design that involved unisensory visual exploration of 2D images of the objects and textures, and unisensory haptic exploration of the 3D stimuli. This yielded four conditions: a) visual objects (VO); b) visual textures (VT); c) haptic objects (HO); and d) haptic textures (HT). During the visual conditions, the participants viewed the stimuli presented on the screen using both eyes. During the haptic conditions, an experimenter who stood next to the participant in the MRI exchanged the objects and textures on the lap desk, and the participants explored the stimuli actively using both hands. All participants regardless of age manipulated the haptic stimuli within the scanner for the entire duration of time allotted to them. This was confirmed by the experimenter upon completion of every scan. Sixteen-second blocks of stimuli were interspersed with 10-second-long inter-block-intervals (IBIs), during which participants viewed a red fixation cross. Our experience with testing these age groups suggests that using a longer IBI increases the incidence of data loss due to excessive head motion, presumably because children lose interest during the longer interval. Furthermore, analyses of simulated data have shown that block design protocols with 10 second IBIs do not have appreciably greater statistical power than protocols with 12 second or 14 second IBIs with signal-to-noise ratios in the range that is typical on our scanner.

Stimuli were presented sequentially for 4 seconds each following 2 seconds of instructions (Fig. 1). A single block consisted of 4 out of the 8 stimuli from each condition, and was presented twice within a given run to comprise the entire set of stimuli per condition. This resulted in 8 blocks per run, and approximately 4-minute runs (118 volumes, 236 seconds). Trials were randomized, and blocks were counterbalanced. There were 4 functional runs administered for each participant. Due to the young age of the children, however, some runs were not completed due to fussiness or excessive motion; these runs were subsequently excluded from further analyses. Imaging sessions lasted a total of approximately 30 minutes. After the scanning was completed, participants were removed from the MRI environment and relocated to a controlled lab setting for behavioral testing.

Figure 1.

Graphical depiction of the fMRI block design. One sample run and stimuli used in the present study. The four conditions consist of visual objects (blue), visual textures (red), haptic objects (orange), and haptic textures (purple).

2.4 MRI data acquisition and preprocessing

Imaging was performed with a 3-Tesla Siemens Magnetom Trio whole body MRI system located at the Indiana University Psychological and Brain Sciences department within the Imaging Research Facility. With a phased array 12 channel head coil, whole-brain functional volumes were acquired using a gradient echo planar imaging (EPI) sequence (TE = 30 ms, TR = 2000 ms, flip angle = 70°) for BOLD-based imaging. The field of view was 192 cm with an in-plane resolution of 64 × 64 pixels and 33 slices per volume (3.8 mm thick with a 0 mm gap). This resulted in a voxel size of 3 × 3 × 3.8 mm. Using analysis tools in the BrainVoyager QX™ 2.2 software package (Brain Innovation, Maastricht, Netherlands), functional data underwent slice scan-time correction, 3D motion correction, linear trend removal, and Gaussian spatial blurring (FWHM 6mm). High-resolution T1-weighted anatomical volumes (resolution: 1.5 mm³, 120 sagittal slices) were acquired using a 3-D Turbo-flash inversion recovery sequence prior to the functional imaging. By applying an intensity-matching, rigid-body transformation algorithm, individual functional volumes were co-registered to the anatomical volumes. Both anatomical and functional volumes were normalized to a standard space using an affine transformation based on the 8 parameters of the Talairach reference (Talairach & Tournoux, 1988). During normalization, voxels of the functional volumes were resampled to 3 mm³.

2.5 Data analysis procedures

All of the functional data were entered into separate random-effects general linear models (GLM), one for each age group, in BrainVoyager QX™ 2.2. Predictors in the design matrix were based on the blocked stimulus presentation timing across runs and across participants, and were convolved with a two-gamma impulse response function. Motion parameters for each run for each participant were included in the design matrix as predictors of no interest. Functional runs with motion estimates exceeding 5 mm on any axis were excluded from the analyses. Although this is a more liberal threshold than is often used in studies with only adults, it was adopted here because a stricter criterion would have eliminated many of the child participants, thus reducing the practical utility of the procedure. While there may be concerns regarding this threshold and its impact on the data, a further analysis of motion artifacts revealed no significant correlations between head motion and BOLD signal change. These data are presented in the supplementary materials. This criterion resulted in a total of 36 usable runs (on average, 2.4 runs per participant) for the 4- to 5.5-year-old children, 45 runs (3.5 runs per participant) for the 7- to 8.5-year-old children, and 32 runs (4 runs per participant) for the adult group.

For the group-defined region-of-interest (ROI) analyses, multisensory object-selective regions, particularly the LOC and the IPS, were functionally localized by contrasting visual objects with visual textures, in conjunction with haptic objects over haptic textures (i.e., (VO > VT) ∩ (HO > HT)). These regions were defined across all participants in the three groups (N = 36) so as not to commit the egregious error of “double-dipping” (Kriegeskorte et al., 2009). BOLD signal change was measured in the same voxels across all participants to limit the bias of selecting different neural substrates, but the potential downside to this method is that individual participants may overlap with the group-defined ROIs to different degrees. Therefore, to assess the appropriateness of using group-defined ROIs, two additional analyses were performed: 1) the variability in location of the individual ROIs was measured across the three age groups; and 2) individually-defined ROIs were examined for comparison. These analyses are presented in more depth in section 3.4 Results of individual-based ROI analyses, as well as in the supplementary materials.

Estimates of the BOLD signal change as beta weights were extracted for each condition for each participant using the BrainVoyager ROI/VOI-ANCOVA table tool. Statistical hypothesis testing on these BOLD signal change values was performed using repeated measures analyses of variance (ANOVA) in SPSS.

In addition to the ROI analyses, supplementary whole-brain analyses were performed. Whole-brain statistical parametric maps (SPMs) were calculated for each group using GLMs. Specific contrasts were performed within each group to assess multisensory shape-selectivity (a conjunction: (VO > VT) ∩ (HO > HT)), and unisensory shape-selectivity (VO > VT; HO > HT). These contrasts were thresholded with a voxel-wise p-value of 0.01 per map and corrected for multiple tests using a permutation test. This resulted in a cluster threshold of at least 11 contiguous voxels for the conjunction contrast, and at least 21 voxels for the simple contrasts. Furthermore, conjunction contrasts were thresholded using an alpha-level of .05; simple contrasts were thresholded using the square root of the threshold used for the conjunction contrast, which resulted in an alpha-level of .22. This was done to facilitate comparisons of overall activation between the three contrasts such that activation from the conjunction threshold would overlap with the simple contrasts.

2.6 Post-scan crossmodal behavioral test procedure

To examine recognition ability, participants were behaviorally tested in two blocks of crossmodal haptic-to-visual 8-alternative-forced-choice match-to-sample tasks, including one block of objects and another of textures. Participants were first instructed to place their hands inside a box with two circular openings, one for each hand. A laptop screen displaying an array of the same 8 objects or textures from the fMRI portion was placed on top of the box. A black felt sheet covered the laptop keyboard and the box so that the participants could feel, but not see, the stimulus. Objects and textures were placed inside the box one at a time for 5 seconds of haptic exploration. Participants were then instructed to remove both of their hands from the box, and within 5 seconds, to point to the matching visual stimulus out of the 8-alternative array. This was ample time for all of the groups to perform the task. Each object and texture was presented inside the box only once, and the order of blocks was counterbalanced among participants. Upon completion of this task, participants were compensated for their time.

Responses for the behavioral 8-alternative-forced-choice match-to-sample task were coded during the experiment by a second experimenter, and were later analyzed for successful recognition. This was calculated as a proportion for objects and for textures by taking the number of correctly identified matches over the total number of trials per block. Chance performance was at 12.5%, or 1 out of 8, in selecting the correct match from the array of 8 stimuli. These proportions were then averaged within each group and compared using a repeated measures analysis of variance (ANOVA) in SPSS.

3. Results

3.1 Functional ROI results

To investigate the development of function in the brain regions previously implicated in visuohaptic object preference, regions of interest were defined using the data from all age groups and a contrast of visual objects versus visual textures in conjunction with haptic objects versus haptic textures (Fig. 2). Two bilateral visuohaptic object-preferring regions were localized in this way (Table 2). The ventral occipital region was located near the previously reported coordinates of the LOtv. The dorsal parietal region, rather than being near the coordinates of previously reported anterior or middle IPS (aIPS/mIPS), was instead closer to caudal IPS (cIPS). While this region has been reported in studies of adult visual object recognition (Faillenot et al., 1997; James et al., 2000, 2002b; Kraut et al., 1997), it is only rarely found during adult haptic object recognition (Peltier et al., 2007; Stilla & Sathian, 2008). Nevertheless, our results showed that this region was also active in children during haptic object recognition.

Figure 2.

Multisensory object preference (VO > VT ∩ HO > HT) in all participants (N = 36). Statistical parametric map (SPM) is presented at a threshold of p < 0.01 (uncorrected). This contrast was used to obtain the group-defined ROIs (white circles) in bilateral LOC, namely LOtv (top row), and in bilateral caudal IPS (bottom row). Functional data are presented on an averaged human brain. On this and other figures, lines on the sagittal plane correspond to axial slices along the z-axis and coronal slices along the y-axis. A = anterior; R = right.

Table 2.

Talairach coordinates (x, y, z), peak t-values, p-values, and number of voxels for visuohaptic object-preferring regions.

Region	x	y	z	t(35)	p-value	No. of Voxels
L LOtv	-47	-61	-12	3.31	.002	908
R LOtv	49	-65	-10	3.54	.001	1384
L cIPS	-21	-77	41	3.39	.002	217
R cIPS	25	-76	32	3.37	.002	886

The main hypotheses were tested with a 2 × 2 × 3 repeated measures ANOVA with BOLD signal change (beta weights) as the dependent variable, stimulus type (objects and textures) and sensory modality (vision and haptics) as the within-subjects factors, and age group (4 to 5.5 year olds, 7 to 8.5 year olds, and adults) as the between-subjects factor. The patterns of activation in the left and right hemispheres were similar for the LOtv and cIPS, which was to be expected as many studies have consistently found bilateral activity for visuohaptic object preference (Amedi et al., 2001, 2002; Saito et al., 2003; Zhang et al., 2004). Yet, it is important to note that previous studies have also reported laterality effects—some have shown what appear to be weaker signals in the right than in the left hemisphere (Kim & James, 2010; Kim, Stevenson, & James, 2012), while others have found task-dependent lateralization in the LOC (Large, Aldcroft, & Vilis, 2007). As such, the issue of laterality remains complex (for a brief, but pertinent discussion of the lateralization of visual and haptic processing, see Stilla & Sathian, 2008). Given our data, however, the effects seemed to be bilateral and were consequently collapsed across hemisphere for each region.

The results from the complete design for the two ROIs are shown in Fig. 3A-B. Results in the LOtv indicated a main effect of age group (F(2,33) = 10.41, p < .001, MSe = 2.11), and of stimulus type (F(1,33) = 54.41, p < .001, MSe = 3.80) with a greater response for objects than textures (t(35) = 5.69, p < .001). Modality, however, was not a significant main effect. Additionally, there was a significant stimulus type by age group interaction (F(2,33) = 6.15, p < .005, MSe = .43). This interaction is shown in Fig. 4A with the data collapsed across modality. Post-hoc t-tests revealed a developmental trajectory of increasing BOLD activation with objects across age; BOLD activation with objects was significantly greater in the 7- to 8.5-year-old children than in the 4- to 5.5-year-old children (t(26) = 2.29, p < .05), and significantly greater in adults than in 7 to 8.5 year olds (t(19) = 3.48, p < .01). Results in bilateral cIPS showed a main effect of stimulus type (F(1,33) = 31.63, p < .001, MSe = 2.01) in favor of objects over textures (t(35) = 3.34, p < .01). Similar to the findings in bilateral LOtv, there was a significant stimulus type by age group interaction (F(2,33) = 5.73, p < .01, MSe = .37; Fig. 4B). Post-hoc t-tests revealed increasing activation with objects between the 4 to 5.5 year olds and the adults (t(21) = 3.05, p < .01). Other key results were differences in activation for visual objects between adults and 4- to 5.5-year-old children (LOtv: (t(21) = 4.20, p < .001); cIPS: (t(21) = 3.48, p < .01)), and between adults and 7- to 8.5-year-old children (LOtv: (t(19) = 3.98, p < .001); cIPS: (t(19) = 2.63, p < .05); Fig. 3A-B).

Figure 3.

BOLD signal change of the complete design in bilateral LOC (A) and IPS (B). On this and subsequent figures, * denotes p < .05, ** denotes p < .01, *** denotes p < .001. Error bars represent SEM.

Figure 4.

Objects compared to textures by age. BOLD signal change is shown as a function of age group and stimulus type collapsed across modality (i.e., VH objects, VH textures) in bilateral LOC (A) and IPS (B).

To examine further the developmental increase in activation with objects, object preference was calculated as the difference between BOLD activation with object stimuli and texture stimuli for both vision and haptics (i.e., VO > VT; HO > HT) in the LOtv and cIPS (Fig. 5). Significant visual object preference was found in adults (LOtv: (t(7) = 9.50, p < .001); cIPS: (t(7) = 5.38, p < .001)), and in the 7- to 8.5-year-old children (LOtv: (t(12) = 4.47, p < .001); cIPS: (t(12) = 2.11, p < .05)). Haptic object preference was also significant in adults (LOtv: (t(7) = 4.29, p < .01); cIPS: (t(7) = 4.26, p < .01)), and in 7 to 8.5 year olds (LOtv: (t(12) = 2.61, p < .05); cIPS: (t(12) = 2.29, p < .05)). In the LOtv (Fig. 5A), adults showed significantly greater visual preference than the 4- to 5.5-year-old children (t(21) = 3.67, p < .01), and the 7- to 8.5-year-old children (t(19) = 4.24, p < .001). Haptic preference, however, was not reliably different among the three groups in any of the ROIs. As a result, adults demonstrated a significant difference between visual and haptic object preference (t(7) = 4.57, p < .01), but children did not. In bilateral cIPS (Fig. 5B), visual object preference followed the same pattern as in the LOtv. That is, adults showed greater activation for visual object preference than the 4- to 5-year-old children (t(21) = 3.42, p < .01), and the 7- to 8.5-year-old children (t(19) = 3.54, p < .001). Overall, these statistical effects did not appear to be driven solely by the adult activation pattern as the 7 to 8.5 year olds demonstrated significant BOLD activation with visual and haptic object stimuli above baseline, as well as significant levels of object preference.

Figure 5.

Visual object preference compared to haptic object preference by age. BOLD signal change is shown as a function of age group and object preference (i.e., objects > textures) by modality in bilateral LOC (A) and IPS (B).

3.2 Results from whole-brain contrasts

Whole-brain contrasts were performed to supplement the ROI analyses and obtain a more global perspective of the activation patterns that may be changing over the course of development. The results of a conjunction contrast assessing visuohaptic object preference (i.e., (VO > VT) ∩ (HO > HT)) in adults are shown in Fig. 6. As expected, adults showed bilateral bimodal visuohaptic object preference in regions previously indicated in the literature, including both the LOC and anterior/middle aspects of the IPS. The same conjunction contrast performed in the two groups of children yielded no significant clusters. However, a direct comparison of children and adults with the same contrast similarly produced no significant clusters. Combined with the results of the ROI analysis, these findings suggest that children activate a similar set of brain regions as adults, but with perhaps sub-threshold signal levels or higher variability.

Figure 6.

Multisensory object preference (VO > VT ∩ HO > HT) in adults. Group maps of averaged adult data are presented at a threshold of p < 0.05 (corrected). Activations are located bilaterally in the LOC (gray arrows) and in the IPS (green arrows). No clusters of activity in either group of children passed the significance threshold.

Results from the unisensory visual preference contrast (VO > VT) revealed both the LOC and the IPS as significantly activated bilaterally in adults (Fig. 7). The two groups of children showed activation in the LOC that overlapped with adults. However, in both groups, the cluster was only found in one hemisphere—the left hemisphere in 4- to 5.5-year-old children, and the right hemisphere in 7- to 8.5-year-old children. Neither group of children showed a significant cluster in the IPS.

Figure 7.

Visual object preference (VO > VT). Group maps are presented at a threshold of p < 0.05 (corrected). Overlapping activations between groups are located in the left LOC (green arrow) and the right LOC (gray arrow). Blue denotes the group of 4 to 5.5 year olds, purple denotes the group of 7 to 8.5 year olds, and orange/yellow denotes the adult group.

In the unisensory haptic object preference contrast (HO > HT), adults showed significant bilateral activity in the LOC and the IPS. Neither group of children showed significant clusters in the LOC. Instead, 4 to 5.5 year olds showed a significant cluster in the right postcentral gyrus, and 7 to 8.5 year olds showed a significant cluster in the right caudal IPS (cIPS) area, which also overlapped in part with the same cIPS area in adults (Fig. 8).

Figure 8.

Haptic object preference (HO > HT). Group maps are presented at a threshold of p < 0.05 (corrected). Overlapping activations are located in dorsal regions: the right caudal IPS (gray arrow), and the right postcentral gyrus (green arrow).

3.3 Behavioral results from the crossmodal matching task

Results from a 2 (stimulus type: objects and textures) × 3 (age group: 4 to 5.5 year olds, 7 to 8.5 year olds, and adults) repeated measures ANOVA with behavioral performance as the dependent variable showed significant main effects of stimulus type (F(1,33) = 28.23, p < .001, MSe = 1.12), and age group (F(2,33) = 13.73, p < .001, MSe = .63), as well a significant stimulus type by age group interaction (F(2,33) = 3.69, p < .05, MSe = .15). Post-hoc t-tests comparing proportional success within each group indicated greater success for objects than for textures (Adults: (t(7) = 3.21, p < .05); 7 to 8.5 years: (t(12) = 2.94, p < .05); 4 to 5.5 years: (t(14) = 4.77, p < .001)). Between groups, there were no significant differences with regard to objects; performance was either at or near ceiling in all groups. Comparisons of matching success on textures, however, indicated differential proportions of success (Fig. 9). The group of 4 to 5.5 year olds performed worse than 7 to 8.5 year olds (t(26) = 3.75, p < .001), and adults (t(21) = 4.20, p < .001), but the 7 to 8.5 year olds did not differ significantly from adults (t(19) = 0.95, p = n.s.).

Figure 9.

Proportional success of the behavioral match-to-sample task. Objects and textures are compared across age during crossmodal haptic-to-visual recognition. Red line indicates chance performance.

3.4 Results of individual-based ROI analyses

One important concern for using group-based functional regions-of-interest (ROIs), particularly across different developmental populations, is that the variability of overlap between individual ROIs and the group ROI may be different. Several studies have demonstrated that children show a more diffuse pattern of activation than adults (e.g., Casey, Galvan, & Hare, 2005; Durston et al., 2006; Stiles et al., 2003; however, see commentary by Brown, Petersen, & Schlagger, 2006). Thus, lower activation for children as compared to adults in a group-based ROI could be due to less overlap of individual ROIs with the group ROI rather than to a true decrease in BOLD signal. To allay this concern with respect to our findings, we performed two analyses. First, the variability in terms of the location of individual ROIs (i.e., ROI variability) was examined in the three age groups. This analysis—the results of which are reported below—served to explore further the development of neural substrates for visuohaptic processing by examining ROI variability as an additional dependent variable. Second, an analysis of individually-defined ROIs was performed for comparison with the group-defined ROI analysis. These results are reported in the supplementary materials as the individual-based BOLD ROIs indicated similar patterns and confirmed the primary findings from the group-based ROIs.

The analysis of individual ROI variability was conducted by implementing a multistep process. First, the four ROIs (i.e., left and right LOC and IPS) were identified in each participant using the conjunction contrast (i.e., (VO > VT) ∩ (HO > HT)). For participants who did not produce a significant cluster in the approximate location of the ROI at the standard FDR threshold, the statistical threshold was lowered until a cluster of at least 2 voxels appeared. This technique was used to ensure that variability was measured across the entire sample, and not just the participants that had the highest levels of BOLD contrast. The locations of the center of mass in Talairach coordinates for each cluster were then extracted (Talairach & Tournoux, 1988). Second, for each of the three age groups and each of the four ROIs, a prototypical ROI center was calculated as the mean center of mass across participants. Third, the Euclidean distance from the center of each participant's ROI to the prototypical ROI center was measured (in mm). Mean absolute distances were used to assess the variability around each prototypical ROI center. Finally, a 3 × 4 repeated measures ANOVA was performed with the Euclidean distance from the mean (mm) as the dependent measure, ROI (left LOC, right LOC, left IPS, and right IPS) as the within-subjects factor, and age group (4 to 5.5 year olds, 7 to 8.5 year olds, and adults) as the between-subjects factor.

The results of the variability analysis indicated a significant main effect of age group (F(2,33) = 14.87, p < .001, MSe = 4467.84). For all four ROIs, 4 to 5.5 year olds showed greater spread of ROI location (i.e., greater variability) than 7 to 8.5 year olds and adults (Fig. 10). No other effects were significant.

Figure 10.

Variability of individual ROI locations as a function of age group. Mean Euclidean distance from the prototypical ROI center is shown for each ROI for each age group. Error bars represent SEM.

The analyses of BOLD signal change suggested that the part of the LOC most involved in visuohaptic convergence (i.e., the LOtv) continues to develop beyond 8 years of age. Can these results be explained instead by differences in ROI location variability? Our analysis of variability did show significant differences across groups; however, this pattern of differences in variability did not match the pattern of differences in BOLD signal change. Specifically, the results of the variability analysis showed a separation between 4 to 5.5 year olds and the older 7 to 8.5 year olds as well as adults, whereas the findings with BOLD signal change showed a separation between adults and both groups of children. Thus, the results of BOLD signal change cannot be explained by differences in variability. One of our expectations was that BOLD signal change in the LOtv would show changes not only between children and adults, but also between younger and older children. The findings of BOLD signal change showed little evidence for this pattern. However, it is interesting to speculate that the decrease in variability of ROI locations from 4 to 5.5 year olds to 7 to 8.5 year olds may reflect the development (and perhaps consolidation) of visuohaptic convergence. As such, this pattern of variability must be taken into consideration for future studies because it could explain possible differences in BOLD signal change in children from 4 to 8.5 years.

4. Discussion

Object recognition is ubiquitous, complex, and crucial for interacting with the environment. It has been extensively documented that the LOC is critical for visual object recognition and that BOLD activation in this region is related to recognition performance (Grill-Spector et al., 2000; Grill-Spector, Kourtzi, & Kanwisher, 2001; James et al., 2002a, 2002b, 2006; see Grill-Spector, Golarai, & Gabrieli, 2008 for a review). Furthermore, a subregion of the LOC, the LOtv, is well-known to be involved in both visual and haptic object recognition in adults (Amedi et al., 2002; James et al., 2002b; James & Kim, 2010; James, Kim, & Fisher, 2007; Stilla & Sathian, 2008). However, one important factor in understanding the organization of the brain is to understand its development. To our knowledge, this is the first known study to track the neural development of the LOC in children as young as 4 years of age, and during a haptic task. The results of the current study showed a developmental increase in visual object preference—defined as a preference for objects over textures—in the LOtv that did not reach adult-like levels by 8.5 years of age. This finding is different from previous reports that object preference in the LOC as a whole reaches adult-like levels sometime between 5 to 8 years of age (Golarai et al., 2007; Scherf et al., 2007). The current results further demonstrated a dissociation between the developmental trajectories of visual and haptic object preference in the LOtv, with adult-like haptic preference reached earlier than visual preference. Taken together, these findings suggest that: 1) the development of the LOtv subregion is more protracted than the LOC proper; and 2) the division of labor between visual and haptic object shape processing in the LOtv is relatively equivalent early on in development, but becomes visually dominant sometime between 8.5 years and young adulthood.

Based on previous research, we initially hypothesized that visual object preference would be adult-like across all three groups of participants in the LOtv subregion. However, the results indicated a developmental increase in visual object preference in this area. As such, these results may be inconsistent with findings from previous studies indicating early maturation of the LOC for object-processing (Scherf et al., 2007; Golarai et al., 2007). In these studies, the processing of real, common, and colorful moving objects was reported to be comparable to adults by 5 to 8 years (Scherf et al., 2007), and the processing of novel, abstract, and gray-scale static objects was also found to be comparable to adults by 7 to 11 years (Golarai et al., 2007). Thus, the discrepancy between the current and previous results may be attributed to the use of different stimuli or to the differences in the binning of age groups. A more likely alternative is that the previous studies measured activation from the LOC “proper,” whereas the current study measured from the LOtv subregion. This difference in specificity may be similar to the difference in developmental timelines for the LOC proper versus the relatively protracted responses of particular regions within the occipital-temporal cortex for specific object categories (e.g., faces, places, animals, and objects from unusual views). Analogous to the protracted development of visual recognition of complex classes of objects, particularly those from unusual views, visuohaptic object recognition may also be delayed relative to visual or haptic object processing in terms of viewpoint independence. Crossmodal studies in adults have found visuohaptic object representations to be view-independent, arising from the integration of view-dependent, unisensory (visual and haptic) modalities; further, these more complex, multisensory representations appear to reflect spatial transformation abilities (Lacey et al., 2009; Lacey, Peters, & Sathian, 2007). Therefore, it seems likely that visuohaptic object processing requires more specialization or fine-tuning than either visual or haptic object processing alone, and that visual object processing in the LOtv may require more experience to develop fully. Our results indicate that full development occurs sometime between 8 years and young adulthood, which is consistent with previous psychophysical findings of visuohaptic integration suggesting that form perception does not become optimal until after 8 years (Gori et al., 2008).

Given that exploratory interactions in children may be restricted by their relatively limited experience, it is conceivable that this restriction results in neural processing that emphasizes different types of sensory inputs. As implicated in the developmental literature, infants and children interact with objects and gain multisensory experience as soon as they are able to move their hands. This rapid acquisition of coordination is crucial for producing statistical regularities between the sensory and motor systems and for the development of perceptual learning (Held & Hein, 1963; James, 1890; Lungarella & Sporns, 2005, 2006; Smith, 2005). Early integration of object properties from both vision and haptics aids in this development. However, as children age and gain experience, they may become more and more reliant on visual information, which results in the gradual separation between vision and haptics in the division of labor for object recognition. It is possible that the results shown here reflect this hypothetical developmental trend for young children to shift their emphasis from haptics to vision as they mature.

The increase in visual object preference through development in the LOC was paralleled by the results in the IPS. On the caudal aspect of this bilateral dorsal region (cIPS), adults demonstrated a visual dominance in object preference as compared to children, while haptic object preference did not change significantly between 4 years and adulthood. This region, considered to be homologous with the same area cIPS in the monkey (macaque) cerebral cortex, has been previously associated with the analysis of the structure of three-dimensional objects (Culham, Cavina-Pratesi, & Singhal, 2006; James et al., 2002b; Sakata et al., 1997; Shikata et al., 2003). Further, there is considerable neural evidence suggesting that this region is involved in visual object processing (Faillenot et al., 1997; James et al., 2000, 2002b; Kraut et al., 1997; Stilla & Sathian, 2008). However, the cIPS is rarely found in relation to haptic object processing (Peltier et al., 2007; Stilla & Sathian, 2008); instead, studies usually find visuohaptic preference in the anterior and/or middle IPS, which are well-known areas of neuronal convergence (Amedi et al., 2001, 2002; James & Kim, 2010; Stilla & Sathian, 2008). More recent evidence has indicated that the left cIPS is recruited during spatial processing, specifically for locating object parts (Sathian et al., 2011). The interplay between visuohaptic object shape and location perception suggests the possibility of a multisensory neuronal pool, which forms a link connecting the processing of spatial relations between object parts and the subsequent processing of global object shape in the cIPS (Sathian et al., 2011; Stilla and Sathian, 2008). In the current study, both children and adults showed visuohaptic preference in the cIPS; and while adults showed recruitment of the aIPS, the children did not. Thus, this is the first study to demonstrate reliable object preference in the cIPS in children, and the first to suggest that object preference in the aIPS may develop later than in the cIPS. In the adult primate (macaque) brain, the cIPS sends projections to the AIP, the homologue of the human aIPS (Culham, Cavina-Pratesi, & Singhal, 2006; Sakata et al., 1997). Perhaps these projections also form in humans during development, but possibly not by 8.5 years of age. Clearly, questions such as these could be further elucidated with studies of structural and functional connectivity across development.

At the whole-brain level, the network of regions shown by children closely approximates those demonstrated by adults. Generally, the two groups of children appeared to recruit similar sensory systems as adults for visual and haptic object processing with overlapping ventral regions in the LOC and dorsal regions in the IPS, although there were some differences in the location of maximal brain activation. Children at different stages of development may be recruiting different subregions within the LOC and IPS, thus resulting in variable activation patterns, although these patterns tended to overlap with adult activity. Even in adults, however, many studies have indicated different areas of the LOC for object recognition (Amedi et al., 2001, 2002; James et al., 2002b; Pietrini et al., 2004; Prather, Votaw, & Sathian, 2004; Reed, Shoham, & Halgren, 2004; Stoesz et al., 2003), as well as different aspects along the IPS (e.g., aIPS and mIPS) that activate during haptic exploration of objects (Binkofski et al., 1999; Bodegard et al., 2001; Culham & Kanwisher, 2001; Grefkes et al., 2002; James & Kim, 2010; Peltier et al., 2007; Roland et al., 1998; Zhang et al., 2004). While these distinctions are fairly well documented in adults, they are not so in children. In the current study, the differences in activation were predominantly reflected in the varying whole-brain response profiles shown during unisensory visual and haptic object preference as compared to the (lack of) response shown during visuohaptic convergence in children. The development of object preference in these systems did not follow an increasing trajectory of activation in the same location from younger to older children, but rather included shifts in location (e.g., between hemispheres). The resultant lack of overlap in activation in the two groups of children indicates that the development of visual and haptic convergence for object processing is not an age-dependent, additive amalgamation of individual unisensory modalities. Instead, the shifts in location likely signify a change in the strategy used for object processing. In children, slight differences in the recruitment of cognitive functions may in turn recruit different neural substrates. Due to the complexity of combining two sensory modalities, there may another factor, namely experience, that is contributing to these subtle shifts in activation for unisensory object preference. Nevertheless, the existent overlaps in activity between the adults and each of the groups of children reveal that neural processing does not move drastically in terms of location over the course of sensory development, but instead, provides evidence that maturity within the same system may stem from increased experience.

It could be argued that texture perception and recognition is more difficult for young children than adults. Despite efforts to select texture stimuli that would be familiar to all participants, it is clear from post-experiment debriefing that children had more difficulty naming the textures than adults. This could have contributed to their poorer performance as measured by behavioral accuracy during the crossmodal haptic-to-visual match-to-sample task. Children from 4 to 5.5 years of age performed significantly worse than 7 to 8.5 year olds and adults at crossmodal matching and recognition of textures. Yet, their recognition of objects was not significantly different from adult performance, and the fact that texture recognition across the visual and haptic modalities was more difficult did not have any differential effect on neural activity. As indicated by the ROI analyses in the LOC and IPS when collapsed across modalities, there were no significant group differences with regard to texture processing; this suggests that the use of real textures acts as an appropriate control condition. Presumably, if texture recognition is more difficult and object processing is easier, then neural activity for textures would increase over the course of development while activity for objects would not change. However, we have obtained the opposite effect: BOLD signal change increased for objects over time, but remained relatively stable for textures. One possible explanation for our effects may be due to the use of realistic textures instead of scrambled objects. While most of the developmental neuroimaging literature has made comparisons between intact objects and scrambled images of those objects (Golorai et al., 2007; Scherf et al., 2007), these comparisons may only be relevant in unisensory visual studies. Research examining haptic or visuohaptic object recognition necessitates the use of haptically recognizable textures to be contrasted with objects (Amedi et al., 2001). While both scrambled images of objects and realistic textures can be placed within a class of objects that lack characteristic shape, yet are perceptually differentiable, it is likely that only realistic textures can be haptically recognized and identified (although this remains to be tested empirically). As this is the first known study to examine haptic processing at a neural level in young children, the relation between the effects of different types of textures was difficult to predict. It is possible that the comparison between common objects and realistic textures would simply decrease object preference, and would result in less widespread activity. This, however, would only yield more conservative results relative to comparisons with scrambled objects. Nevertheless, the BOLD activity with textures seems to remain fairly stable across development and activate many of the same areas as the visual and haptic object conditions contrasted with rest. These findings, in concert with the lack of developmental change in haptic object preference, suggest that it is the development of object recognition abilities, and in particular visual object recognition rather than texture recognition, that drives the relative increase in object preference.

In summary, we have measured the neural systems of children at different ages, including those several years younger than reported in the literature, and have shown the developmental trajectories of the LOC and the IPS for visual and haptic object recognition. The present research provides evidence that the LOC, and the LOtv specifically, is indeed involved in visuohaptic object recognition early on, though it becomes increasingly visually dominant over the course of development. Additionally, this is the first known study to show caudal IPS recruitment during haptic object recognition in both adults and children. Disparate overlapping activity between children and adults in the LOC and the IPS suggests that object preference does not develop in an additive manner from early unisensory visual and haptic convergence of information. Taken together, we conclude that the development of multisensory visuohaptic neural substrates for object recognition is protracted compared to more unisensory substrates, possibly because adultlike connectivity among these substrates requires extensive experience through active perception and exploration.

Highlights.

• We measured the neural systems of children for multisensory object recognition

• We examined the development of visuohaptic object preference in LOC and IPS

• Object preference over textures increased during development

• Object preference became increasingly visually dominant with age

• Multisensory object processing was more protracted than unisensory processing