Developmentally stable representations of naturalistic image structure in macaque visual cortex

SUMMARY

To determine whether post-natal improvements in form vision result from changes in mid-level visual cortex, we studied neuronal and behavioral responses to texture stimuli that were matched in local spectral content but varied in “naturalistic” structure. We made longitudinal measurements of visual behavior from 16 to 95 weeks of age, and of neural responses from 20 to 56 weeks. We also measured behavioral and neural responses in near-adult animals more than 3 years old. Behavioral sensitivity reached half-maximum around 25 weeks of age, but neural sensitivities remained stable through all ages tested. Neural sensitivity to naturalistic structure was highest in V4, lower in V2 and inferotemporal cortex (IT), and barely discernible in V1. Our results show a dissociation between stable neural performance and improving behavioral performance, which may reflect improved processing capacity in circuits downstream of visual cortex.

In brief

Lee et al. use naturalistic texture images to test the visual sensitivity of macaque monkeys during development. In interleaved sessions, they record population neural responses using the same stimuli in V1, V2, V4, and IT. Behavioral sensitivity increases with age, but neural sensitivity is adult-like at all ages.

Graphical Abstract

INTRODUCTION

Behavioral performance improves during early life—animals learn new skills and refine their performance within the space of skills they already know.^1–5 The basis of these improvements is of interest; the acquisition of new skills often reflects a mixture of neural, muscular, and morphological changes,⁶ but improvement on psychophysical tasks can more simply be linked to changes in the brain. By understanding where these changes take place, we gain insight into how behavior emerges from the combined activity of sensory, association, and motor areas.

In the macaque visual system, behavioral capabilities on basic spatial vision tasks improve throughout the first year of life.⁴ Despite this, neural sensitivities in the lateral geniculate nucleus (LGN), primary visual cortex (V1), and cortical area V2 are mature by roughly 16 weeks of age.^7–9 Neurons in the inferotemporal cortex (IT) of infant macaques, like those in adults, can be selective for object identity but are immature in their temporal dynamics.¹⁰ Beyond this, the extent to which neural activity in developing cortical area V4 and IT limits behavioral development remains unknown. We wondered whether the gap between neural and behavioral development reflects immaturity in downstream visual areas like V4 or IT or from more remote areas that convert sensory information into decisions and actions.

To address these questions, we used synthetic visual textures¹¹ (Figure 1A) whose structural similarity to natural images can be titrated¹² and that preferentially drive activity in areas V2 and V4 (but not V1).^12–14 We designed a 4-choice task that allowed us to quickly and reliably estimate the behavioral texture sensitivities of developing macaque monkeys. To quickly obtain neural measurements during development, we then used multi-electrode recording arrays to measure texture sensitivities at the single site and population level from areas V1, V2, V4, and the posterior portion of IT. We recorded multiunit responses during passive fixation, using the same images as in our behavioral experiments. We also recorded responses to a larger library of similar texture images (Figure 1B). These measurements allowed us to compare neural encoding with behavioral performance, in largely overlapping sets of animals, from the youngest possible ages. They also allowed us to relate how early, middle, and late ventral visual areas encode naturalistic textures at all ages and to extend our understanding of how sensitivities to naturalistic texture in areas like V2 and V4 might support information representation in area IT.

Figure 1. Naturalistic texture stimuli.

(A) Texture images varying continuously in the strength of their naturalistic structure. We used these images for measurements of behavioral and neurometric sensitivities.

(B) Naturalistic (top row) and noise textures (bottom row) used to characterize discriminability. Here, we used a total of 35 families, each containing 15 samples.

RESULTS

The experiments detailed here used a total of 8 Macaca nemestrina monkeys (Table 1). We made behavioral measurements of naturalistic texture sensitivity from 7 of these macaques, including 5 tested longitudinally from as early as 16 weeks, through the first 1–2 years of life, as well as 2 older controls. We made neural measurements of naturalistic texture encoding from 6 animals in total—5 during early life and one adult. We have behavioral data for the 5 animals studied in early life but not for the adult.

Table 1.

Experimental details

Subject	Behavioral age range (weeks)	Areas recorded	Recording ages (weeks; number of sites)
M1 (f)	17–95	V1-V2 border, V4	29 (n = 56, 34, 90), 37a (n = 60, 33, 91), 56 (n = 61,32, 95)
M2 (f)	16–95	V2, V4	29 (n = 92, 21), 37a (n = 85, 45), 56 (n = 92, 57)
M3 (m)	16–65	FC, IT	20a (n = 87, 39), 31 (n = 81, 40), 34 (n = 55, 0 [FC only])
M4(f)	27–53	N/A	N/A
M5 (m)	17–64	N/A	N/A
M6(m)	184–194	V1, IT	30 (n = 0, 30) 34a (n = 0, 56), 36a (n = 13, 0)
M6	(see above)	IT	221a (n = 52)
M7(m)	184–194	IT	28 (n = 31), 34a (n = 48)
M8(m)	N/A	V4	409a (n = 96)

Animal numbers reflect the numbers used in behavioral experiments (f, female; m, male). M1–M7 are the same as M1–M7 from Rodríguez Deliz et al.¹⁵ Numbers given in the fourth column represent the number of visually responsive sites used at a given age, for the areas we recorded in that animal, using the texture set of 35 image families. For M3, we were not able to confidently delineate area boundaries for one array. Recordings from this array may therefore reflect activity in either V1, V2, or V4 and is denoted FC. M6 was implanted with arrays in both hemispheres at different ages and is listed twice, corresponding to the separate arrays.

^aFor the comparison in Figure 4D, we used data recorded at a single age.

Behavioral sensitivities double during the first year of life

To measure animals’ behavioral sensitivities, we designed a 4-choice oddity task (Figure 2A). We trained animals to fixate a red square at the center of the screen. Following a 200 ms delay, we showed 4 texture images (3 distractors, 1 target, detailed below), arranged in a square. Animals had 1200 ms to register a choice, which we marked as their first fixation of 400 ms or longer on one of the images.

Figure 2. Behavioral task and developmental time course.

(A) Task design. After fixating the center of the screen and a 200 ms delay, 4 texture stimuli would appear, each matched in “family” and differing in “sample.” The 3 distractors were noise textures. The target varied in the strength of its naturalistic structure. The display remained on for 1,200 ms, unless the animals correctly chose the target by looking at it for at least 400 ms.

(B) Psychometric functions for one animal at two ages. Filled circles representdata (± binomial variance); solid lines represent cumulative Weibull psychometric functions fit to the data. Vertical dashed lines mark thresholds; horizontal lines at the base represent 95% CIs. Numerical values above the abscissa reflect sensitivities (inverse thresholds).

(C) Sensitivities versus age. Each animal is represented with a distinct color.Points represent sensitivities (± 95% CI); solid lines represent Michaelis-Menten functions corresponding to each animal. The black point above the abscissa represents the half-maximum age extracted from the Michaelis-Menten fit (± 95% CI).

We generated all textures using the Portilla and Simoncelli model,^11,12 including the “noise textures” we used as distractors, which matched the local spectral content of a given natural image. We also generated “naturalistic textures” using the model—textures that additionally matched the local correlation structure of the original image (Figure 1A). By interpolating the model parameters between matched noise and naturalistic textures, we generated textures parametrically varying in the strength of their structure (see also Freeman et al.¹²). We used one of these naturalistic textures, varying in strength, as the target for each trial. Within a given experimental session, all textures belonged to the same “family”—they were derived from a single ancestral natural image. Within a trial, all 4 images (including distractors) came from different “samples”—no 2 images in a trial were identical. As a result, the only informative difference between the images was the presence of naturalistic structure in the target.

Animals learned the fundamentals of the task within a single 30- to 60-min session; they could reliably discriminate textures at the earliest ages we studied. Their sensitivities stabilized within 3–4 sessions. Figure 2B shows psychometric functions measured from one animal (M1) at 17 and 50 weeks. Performance varied lawfully with the strength of the naturalistic structure at both ages. While performance on highly naturalistic textures was perfect at both ages, performance close to threshold improved with age; sensitivities (vertical dashed lines, numbers listed above the abscissa) more than doubled in this example.

From our sample of 7 animals, we measured sensitivities to 5 different texture families from as early as 16 weeks to as late as 194 weeks (Figure 2C). We modeled the relationship between sensitivity and age with a modified Michaelis-Menten function, which had the same shape for all animals. Animals differed systematically in their sensitivity, so we fit a maximum separately for each animal; this model fit our data the most parsimoniously. We computed a half-maximum age of 23 weeks (95% confidence interval (CI): [20, 24]), suggesting that naturalistic texture sensitivity matured at a rate similar to other spatial vision tasks.^15–17 A goal of these experiments was to establish a benchmark for comparison with separate neural measurements (below), made across a different age span. We parametrically estimated the magnitude of behavioral change between 26 and 52 weeks. Across this span, we found that sensitivities increased by a factor of 1.39 (95% CI: [1.35, 1.44]).

To confirm that our results did not reflect the learning of image features particular to the tested families, we introduced a novel fifth texture family once animals approached 1 year of age (data are plotted separately for each family and animal in Figure S1). Performance measured with this held-out family was consistent with performance on the other 4, suggesting that sensitivity was primarily determined by age and not by stimulus-specific experience. In addition, we tested 2 animals cross-sectionally (M6 and M7) when they were nearly 4 years old (an age by which visual capabilities in the macaque are mature; see Kiorpes¹⁸). They had not been tested previously with these stimuli, but their sensitivities were consistent with those seen in the other animals at ages at or beyond 1 year, despite their lack of previous experience with this task.

Population neurometric sensitivities are stable across development

Having observed that behavioral texture sensitivities increased from 26 to 52 weeks of age, we asked whether neural correlates of this improvement were detectable in recordings made in the ventral visual areas where, in adults, neuronal responses are modulated by the same statistics, including areas V2, V4, and IT. We also recorded from V1, an area shown previously to be insensitive to naturalistic structure. In 3 of the animals studied longitudinally in our behavioral experiments, we implanted a pair of 96 electrode “Utah” recording arrays in the visual cortex and recorded neural responses to the same images used in our behavioral experiments during passive fixation. While we studied different combinations of areas in different animals, our measurements included longitudinal data recorded from areas V1, V2, V4, IT, and in the foveal confluence of the visual cortex (hereafter called FC, after Brewer et al.¹⁹), for which the specific area(s) studied could not be determined with certainty (see Table 1 for details).

To evaluate responses to a given texture family, we used multiple random samplings of 20 visually responsive sites (a number chosen to facilitate comparisons across all areas and ages; STAR Methods) to estimate neurometric texture sensitivity in a manner that would enable direct comparison to our measurements of behavioral sensitivity. To obtain these neurometric sensitivities, we first took the population response to all textures (Figure S2A), where each dimension represented the responses of a single site. We rotated these responses into the space of their first principal components (Figure S2B). We then used linear discriminant analysis (LDA) to find the axis perpendicular to the hyperplane that best separated fully naturalistic and noise textures. To simulate performance on a 4-alternative task, we projected 4 individual neural responses onto this axis: 3 noise texture responses (simulated distractors) and 1 response to a stimulus of varying naturalistic structure (simulated target). If the magnitude of the target projection was the largest, then we scored the trial as correct. Otherwise, we scored the trial as incorrect. We iterated across all responses in our held-out test set, extracted proportions correct for each level, and fit neurometric sensitivities (Figure S2C). Example neurometric functions are depicted in Figure 3A. In V1 and FC, we could not extract a meaningful sensitivity because performance never reached our criterion for determining threshold, though neural performance was higher for textures with more naturalistic structure. Measurable sensitivity emerged in V2, was largest in V4, and was lower in IT than in V4.

Figure 3. Stable neurometric sensitivities in the developing ventral stream.

(A) Example neurometric functions from each area. Each panel depicts a neurometric function and the population average performance for each condition for one example session. Points represent the average proportion correct for a given level; solid lines represent neurometric functions fit to the data. Vertical dashed lines represent thresholds, which were inverted to obtain sensitivities (black numeric values).

(B) Neurometric sensitivities versus age. Large points represent median sensitivities across 20 site subsampled populations, for all fitted data measured in a given area, at a given age. Small points represent the sensitivities measured for individual sessions (symbols indicate animal identity; symbols with a black outline correspond to the neurometric functions in A). Sessions for which a sensitivity could not be extracted are represented with points below the ordinate (“n.s.”— not significant); the vertical displacement of these points corresponds to the area under the neurometric function. Error bars on large points depict median absolute deviations across fit sensitivities. Black points (left) depict behavioral measurements taken from the animals used here; the curves reproduced on all panels depict a Michaelis-Menten fit to those data.

We extracted neural sensitivities from all sessions (Figure 3B). As in our examples, V1 and FC were functionally insensitive to naturalistic textures (with rare exceptions). Sensitivities increased from V2 to V4 and were comparable or reduced in IT. Critically, across all areas, sensitivities did not change as a function of age in any of the 3 animals measured here.

To facilitate comparisons between neural and behavioral sensitivities, we added the corresponding behavioral data collected from the same 3 animals to Figure 3B (black points and curves). We made two primary observations. First, we noted that neurometric sensitivities were lower than behavioral ones. We also noted that neurometric sensitivities varied with population size (Figure S3A), suggesting that this gap may reasonably be expected to shrink as population sizes approach the actual number of neurons in the brain. Second, whereas behavioral sensitivities in our animals increased with age, this change was not mirrored in any of the areas we measured. These results therefore suggest that the neural locus of behavioral development may either lie downstream of the visual cortex or involve factors beyond the simple encoding of stimulus information.

To determine whether the impact of correlated firing on performance changed with age, we repeated our measurements of sensitivity after shuffling the relationship between sites. We found that shuffling led to a modest decrease in sensitivity that did not vary with age. Finally, we measured discriminability by measuring the distance between fully naturalistic and noise textures, as a d’, when projected onto our discrimination axis (Figure S2D). By doing so, we could compare how well the response to fully naturalistic textures predicted sensitivity, a measure that depends on performance at threshold. We related these measurements to one another (Figure S4) and found that values tracked closely. Taken together, our measurements of neural sensitivities suggested that performance was at ceiling in all areas from the earliest ages we measured.

Single-site texture selectivity is stable across development

Behaviorally, we found that sensitivities increased during early life. In contrast, our parallel neural measurements of sensitivity revealed stable performance in each measured area: V1, V2, V4, and IT. In a larger sample of 6 animals (including the 3 from the previous experiment), we next wanted to more broadly survey how naturalistic textures were encoded in the developing ventral stream. In these experiments, we recorded responses to fully naturalistic and noise textures taken from 35 texture families (Figure 1B). Each family contained 15 samples. We positioned stimuli to cover the receptive fields of the sites on the arrays. Most of our recordings were longitudinal; we recorded in V1, V2, and V4 from 30 to 56 weeks and in FC and IT from 20 to 36 weeks. We also recorded single sessions at 409 weeks in V4 and at 221 weeks in IT. As before, we made these measurements during passive fixation.

Post-stimulus time histograms from all areas are shown in Figure 4A for sample sites recorded at the youngest (top row) and oldest (bottom row) ages measured in a given area. Texture selectivity—meaning a preference for naturalistic over noise textures—increased successively from V1 to V2 to V4 and declined from V4 to IT. We quantified selectivity in units of d’ and measured selectivities from populations at the youngest at oldest ages in our sample in response to all 35 texture families (Figure 4B). Selectivity again increased along the ventral stream; measurements in V2, V4, and IT were visibly more selective for texture than V1 or FC. As in Freeman et al.,¹² we observed similar tuning for texture families between sites in a given area; the horizontal bands visible in Figure 4B suggested similar tuning across sites. Qualitatively, we saw evidence of texture modulation from the earliest ages tested in all areas—as early as 20 weeks in IT. One notable feature of these data is that the response dynamics were similar across age for all areas except IT, where responses were quite sluggish at 20 weeks. We return to this issue below.

Figure 4. Single-site texture selectivities across ages and areas.

(A) Post-stimulus time histograms for example sites, recorded from each area (V1, FC, V2, V4, and IT) at early and late ages. Traces represent mean responses to all naturalistic textures (color), and all noise textures (gray), for stimuli presented in the first position of a stimulus block; numerical values represent selectivities for naturalistic versus noise textures, measured across all stimuli.

(B) Single-site selectivities across all stimuli and areas. Each element represents the selectivity (d’) for a given texture family for a given site. Rows are organized from bottom to top based on increasing perceptual sensitivities as reported in Freeman et al.¹² (their Figure 7). Each column represents 1 site from the total sample measured at a given age, sorted in order of increasing selectivity. The top row represent the youngest age measured in a given area, and the bottom row represents the oldest age. Colors were cut off at ±1.6, corresponding to 90% of a standard normal distribution.

(C) Single-site selectivities versus age. Points represent mean values, and error bars represent 95% CIs.

(D) Single-site selectivities between areas (one age used per array; Table 1). Conventions as in (C). Pairwise interactions were significant based on pairwise permutation tests (p < 0.05), unless marked with “n.s.”

We measured the mean selectivity across texture families for each visually responsive site from all ages and areas (Figure 4C; see also Figure S5). We estimated the relationship between texture selectivity and (log-transformed) age by linear regression in all areas except FC (where we lacked an older age point). We saw no evidence of age-related improvement. In V1, V4, and IT, slopes straddled zero (V1: −0.07, 95% CI [−0.16, 0.03]; V4: −0.04 [−0.1, 0.02]; IT: 0 [−0.07, 0.08])—further evidence that texture selective mechanisms were mature from the earliest ages we measured in those areas.

In V2, this slope was negative (−0.4 [−0.56, −0.23]). Rather than reflecting an effect of age, we suspected that this might instead reflect a change in the tuning in our arrays; among the areas selectively responsive to naturalistic textures, V2 was the only area for which we lacked an older, cross-sectional sample. Therefore, we wondered whether V2 might have been more vulnerable to changes reflecting the ages of the arrays rather than of the animals. In particular, we wondered whether tuning in our V2 arrays may have become more dispersed with time. To measure this, we computed the ratio of variance across texture families to the overall stimulus-driven variance (Figure S6). This measure reflects tuning for specific stimulus classes relative to the spread across all stimuli. A decline in this value would therefore reflect a shift toward signals that, while still visually evoked, may have become less reliably tuned. This ratio remained stable for recordings made in V1 and V4 but declined in V2, confirming that tuning, measured in this manner, may have declined with age in V2.

Single-site texture selectivity is lower in IT than in V4

We wanted to compare texture selectivities between areas, to compare our results with prior measurements in V1 and V2, to relate the selectivities of V2 and V4 using the same stimuli, and to furnish a first measurement of naturalistic texture selectivity in IT. To avoid double-counting, we chose one recording per array (Figure 4D; Table 1). We found a significant group-level difference between areas (F = 14.3, permutation ANOVA, p < 10⁻⁵) and significant pairwise differences between V1 and each of V2, V4, and IT (mean d’ difference: 0.31, 0.34, 0.27, respectively; permutation test p < 10⁻⁵). Downstream of V1, we found that selectivity was similar in V2 and V4 (mean difference: 0.03, p = 0.28). Selectivities were significantly lower in IT than in V4 (mean difference: −0.08, p < 0.002) but did not differ significantly from V2 (mean difference: −0.05, p = 0.09). These results suggest that selectivity to naturalistic texture may peak in V4, a result consistent with the measurements of sensitivity reported above.

Neural populations stably encode naturalistic textures across development

Our previous measurements suggested that naturalistic textures were encoded at ages as young as we could study, even as behavioral sensitivities continued to mature. We also found that texture selectivities peaked in mid-level visual areas and declined in IT. We wondered whether these observations also held at the population level. Thus, we asked whether population representations of texture, taken by combining responses across sites, might reveal a link to behavioral development.

We measured population discriminabilities using the same LDA method we used previously to compare neural sensitivities with discriminabilities; we learned the hyperplane best separating responses to naturalistic and noise textures for each texture family, projected held-out data onto the orthogonal axis, and measured the distance between naturalistic and noise texture distributions as a d’ (Figure S2D). As an analogous measure of detectability, we constructed identical decoders measuring the distance between responses to naturalistic (or noise) textures and blank stimuli.

Population naturalistic texture discriminabilities are plotted in Figure 5A for populations of 30 sites. In all areas, naturalistic texture encoding was stable from the earliest ages measured, including both longitudinal measurements made within the first year of life (as early as 20 weeks) and, in V4 and IT, recordings made at considerably older ages (roughly 8 and 4 years, respectively). We fit slopes relating population performance with (log-transformed) age and used those slopes to estimate the magnitude of change between 26 and 52 weeks (recall that behavioral sensitivities increased by a factor of 1.4 over this time). As in the case of single sites, changes in behavior were not accounted for by changes at the population level; in V4 and IT, performance at 52 weeks fell to 0.95 and 0.97 of the performance at 26 weeks (95% CI: [0.93, 0.97] and [0.96, 0.98], respectively). In V2, we saw a substantial decline, with performance falling to 0.69 of the 26-week estimate (CI: 0.64, 0.74), which can be attributed to the inferred change in recording quality discussed above. V1 measurements were more inconsistent, reflecting its overall lack of sensitivity to naturalistic texture (change between 26 and 52 weeks: 2.5 [CI: 2.1, 2.9]). We did not consider FC for this analysis, as we did not sample it at later ages. Finally, the 35 texture families used in this experiment were a superset of those used to measure neurometric sensitivities. As a result, we were able to compare performance between the 5 behaviorally tested families and the remaining 30 (Figure S7). We did not notice a developmental effect in either subset. Discriminability was higher for the 5 behavioral families than the average of the other 30 but covered the same range.

Figure 5. Population coding and representations in the developing ventral stream.

(A) Naturalistic texture discriminability between naturalistic and noise textures, versus age. Values are the mean across multiple population samplings of 30 sites.

(B) Stimulus detection performance versus age, for either naturalistic textures (white points), or noise textures (gray points, shifted horizontally when necessary for visibility). Note that the scale on the ordinate is halved in height relative to (A); conventions are otherwise the same.

(C) Triangular representation. As indicated in the legend on the left, the length of each side reflects the 3 population metrics from (A) and (B). The sides emanating from the bottom reflect detectability; the far side reflects discriminability. The leftmost measurement in V1 could not geometrically form a triangle; it has been left hollow to reflect this.

Our measurements of texture detectability provided additional context (Figure 5B); detectability remained stable for both texture types during development but was greater than texture discriminability (reflecting the ability of naturalistic textures to reliably evoke visual responses). Using the same 26- to 52-week comparison, we found modest increases in V1 and V4 (112% [CI: 107,117] and 113% [112, 115], respectively), a modest decrease in IT (97% [95, 99]), and no significant change in V2 (105% [99.8, 110]). We again did not consider FC in this analysis. Together with our single-site measurements, these results suggested that the increase in behavioral sensitivity to naturalistic textures that we observed did not stem from developmental changes in the ventral stream and must instead result from changes elsewhere in the brain.

Population representation of texture in different areas

We recorded population responses to naturalistic texture in V1, V2, V4, and IT. We wondered how well the activity in each area could support texture discrimination. We compared the performance of each area using our measurements of discriminability (Figure 5A; Figure S3B for other population sizes). Performance was poor in V1 and FC, higher in V2, and highest in V4. IT performance was lower than in V4 and was similar to V2.

Comparing texture discriminability in different areas revealed that these values did not always reflect the simple difference between the detectability values for naturalistic and noise textures. If the population representation of naturalistic structure (discriminability) is parallel to that corresponding to the presence of a stimulus (detectability), then the difference between naturalistic texture and noise should equal the difference between naturalistic texture detectability and noise texture detectability. To the extent that these values differ, they imply that the population code for naturalistic structure is encoded differently from the overall visual response.

For each measurement in Figures 5A and 5B, we constructed a triangle using each population measurement to determine the length of each side (Figure 5C). The length of the left, vertical arm represents an area’s ability to detect noise textures. The length of the right arm (emanating from the bottom) represents its ability to detect naturalistic textures. Finally, the length of the opposite arm corresponds to the discriminability between naturalistic texture and noise.

Our population measurements from V1 and FC suggested that neither area could discriminate naturalistic from noise texture. The corresponding triangles are consequently narrow. Stimulus detectability in V2 was similar to that observed in V1—the V2 triangles are therefore similar in height to those seen in V1. The difference in discriminability we observed between V2 and V1 is reflected in the longer length of the upper arm of the V2 triangles compared to those measured in V1. In V4, this upper arm not only gets longer than in V2 but, critically, the constructed triangles are wider than those seen in V2, suggesting that the representation of naturalistic structure is not only stronger in V4 than in V2 but also that it is encoded by activity patterns that differ more strongly from the patterns representing the presence of a visual stimulus.

In IT, the triangular representation of detectability and discriminability resembled a smaller version of that seen in V4. This suggests that the difference between the neural representations of detection and texture discrimination that emerges in V4 is preserved in IT, even as the response of IT to texture images is reduced. These separate representations may therefore represent a platform on which the tuning of IT neurons for object identity may be constructed.

IT dynamics become faster during development

Visual response latencies in areas V1 and V2 shorten during the first 8–16 weeks of life,^9,20 and latencies in the IT area are longer in infancy than in adulthood.¹⁰ Because latency shifts can reflect morphological changes like myelination, we wondered these measurements might reveal visual development. To study changes in dynamics, we measured population performance using the same texture discrimination and naturalistic texture detection schemes as before, training and testing performance separately for the spike count in each 10-ms time bin in our data, as opposed to the 150-ms window used in the above analyses (see Hung et al.²¹—their Figure 3B).

We used these curves (Figure 6A) to investigate whether neural dynamics changed developmentally. We measured the onset latency as the time when a performance curve first began to rise from baseline (in V1 and FC, we only measured latencies in the detection paradigm). Latencies remained largely stable for V1, V2, and V4 for both paradigms (Figure 6B). Comparing our simultaneous measurements from V2 and V4, we observed that naturalistic textures were discriminable in V2 at earlier times than in V4, despite the latter area’s overall heightened discriminability, evidence that this signal may first emerge in V2. In IT, we found that latencies shortened during the measured age range. While the stability we saw in areas V1 and V2 was unsurprising, given prior reports,^9,20 previous recordings from infant IT were made at the age of 4 weeks;¹⁰ our measurements extended that maturation out to 20 weeks. If behavioral development stems from changes downstream, then it is possible that the slow dynamics of immature IT may contribute in some way, even if the overall performance in IT itself is stable from the earliest ages we measured—a question for future experiments.

Figure 6. Age-related changes in the temporal dynamics of responses in IT.

(A) Population performance curves versus time, trained and tested separately for each 10-ms time bin. Lines represent means across population samples. Dashed lines represent naturalistic texture detection; solid lines represent texture discrimination.

(B) Half maximum latencies, measured as the first time at which performance exceeded half of the eventual maximum for both paradigms. Error bars represent 95% CIs across population resamples; they are often smaller than the symbols and thus invisible.

DISCUSSION

Visual development

We measured the rate at which macaque monkeys improved in their behavioral sensitivity to naturalistic textures. Sensitivities increased during early life at a rate similar to other spatial vision tasks.^15,16,22,23 Based on these results, and given prior evidence that neurons in areas V2 and V4 are sensitive to the same naturalistic structure, we recorded multiunit neural sensitivities in areas V1, V2, V4, and IT to naturalistic textures, using a superset of the images used for behavioral measurements. Other than changes in response dynamics in IT, which did not impact performance in IT itself, we saw no evidence of neural development of the representation of naturalistic texture in any of the ventral visual areas from which we recorded. In particular, we found population sensitivities, discriminabilities, and the impact of correlated neural firing on naturalistic texture encoding in the ventral stream to be unchanged across all ages tested, despite an increase in behavioral sensitivity over the same period.

Previous measurements from early visual areas established that both tuning and timing in areas like the LGN, V1, and V2 reach maturity by or before 16 weeks.^7,9,20 Previous measurements from IT suggest that neuronal responses are selective for object identity from as early as 4 weeks,¹⁰ and that response onsets are immature relative to adults. Through the use of population analysis methods and our use of a behavioral benchmark, we extended the disconnect between behavioral and neural development to the end of the ventral visual stream.

The stable neuronal representations we observed across development complement recent results obtained from infant humans and macaques using functional imaging. In both species, retinotopic organization has been observed across a number of ventral areas (including V1, V2, and V4), from early ages, including the first 1–2 weeks in macaques²⁴ and 5.5 months in humans.²⁵ In infant macaques, face-preferring patches in IT are visible from the first month of life.²⁶ In infant humans, face patches can be seen in IT-analogous areas from the age of 4 months.²⁷ While refinements of these selectivities have been observed, it is of note that refinement lags the onset of face-guided behavior in infants of both species.^28,29 The observed changes may instead reflect a process distinct from feedforward neuronal drive. Taken with our own findings, evidence suggests that the visual system is both anatomically and functionally mature from an early age; behavioral development may therefore reflect development downstream of the ventral visual cortex.

If behavioral development is not limited by sensory cortex, what might constrain it? One possibility relates to decision and action; the lateral intraparietal area (LIP), frontal eye fields (FEF), and superior colliculus represent information pertaining not just to the stimulus but also its relation to an eventual behavioral choice in saccade-based tasks.^30,31 Whereas neuronal motion sensitivity in macaque LIP correlates with behavioral improvements in a perceptual learning paradigm, the sensitivity of neurons in cortical area MT remains stable.³² We have previously shown a relationship between the visual stimulus used in a task and the corresponding rate of behavioral development.¹⁸ In particular, behavioral development tends to take longer for tasks that require the integration of stimulus information across relatively large spatial expanses. As a result, performance may be more dependent on development in downstream areas with large receptive fields, such as association areas related to visual recognition, which may develop more slowly. Protracted development in downstream areas is supported by anatomical measurements; occipital areas mature more quickly than parietal and frontal areas in macaques³³ and humans.³⁴ We designed our task to obtain psychophysical data as quickly as possible. A task with a more explicit delay period could allow for similar measurements in the LIP, FEF, or any other area spanning the space between the ventral stream and motor output, assuming that such a task could be learned by infants. A task whose timing could be precisely controlled would allow us to better document the role played by developmental changes in temporal dynamics, of the sort we observed in IT. Moreover, it would facilitate manipulations of attention, a factor we were unable to address here.

More broadly, these results, along with other demonstrations of adult-like tuning in the developing primate visual system, may help reorient our thinking about visual development (see also Makin and Krakauer³⁵ for a contemporary reevaluation of post-natal reorganization). Earlier studies of plasticity following monocular deprivation led to a focus on the malleability of the developing visual system,^36–39 as opposed to studies that demonstrated the relative maturity of the normal visual system.⁴⁰ Moving forward, it may be worth thinking of the visual cortex as a series of areas that are mostly adult-like in their tuning and function during typical development and that feed into downstream areas whose ability to effectively process those sensory inputs may develop more slowly.

Visual processing

While the primary focus of these experiments was to understand development, our measurements also enabled us to explore naturalistic texture encoding in the areas from which we recorded. Like others, we found selectivity for texture information in mid-level visual areas like V2 and V4.^{12,14,41–45} Our latency measurements suggest that neural signals supporting texture discrimination emerge first in V2 and are amplified in V4. Our population analyses found that naturalistic texture was more strongly represented in V4 than in V2, despite similar single-site metrics.

Our measurements from IT showed weaker responses to textures than we observed in earlier areas. In our data, naturalistic texture information was most strongly represented in V4. Yet, despite the diminished performance in IT, both areas appear to use distinct neural subspaces for naturalistic texture discrimination and for stimulus detection. An attenuation of texture responses in IT is consistent with its known sensitivity to visual objects.^21,46–49 In that light, our results suggest that a neural representation of naturalistic structure first emerges in V2. Those signals are amplified in V4 and are represented in a manner increasingly different from that reflecting the simple presence of the stimulus. In IT, the response to textures is attenuated, but the dimensionality is preserved. Combined with separate V4 representations of shape⁴¹ and color,⁵⁰ the neural subspaces we observed may therefore reflect a coding strategy in IT that can support scene segmentation and object-centric coding while still providing information about more elementary visual features.^51–53

Limitations of the study

We interpret our results to suggest that the neural locus of behavioral development during the measured age range may lie downstream of the visual areas we studied, but alternative possibilities remain. We chose to make parallel but separate measurements of behavioral and neural sensitivity, so we do not have recordings made concurrently with behavioral decisions, which could have provided a closer relationship between the measurements. This allowed us to gather considerably more data and to begin behavioral data collection at an earlier age. On the other hand, it left us unable to manipulate and measure the impact of cognitive factors, such as attention and learning, on our neural responses. The influence of attentional factors on performance in the developing visual cortex thus remains a question for future work.^54,55 And prolonged experience with our behavioral task might have resulted in tuning changes at a finer scale than our methods were able to detect (see the V4 results of Yang and Maunsell⁵⁶ and the V1/V2 results of Ghose et al.⁵⁷), resulting in an underestimate of age-related changes.

Another limitation arises from our choice to use chronically implanted multi-electrode arrays to measure neural sensitivities. This allowed us to record neural populations longitudinally and without head fixation. But it also meant that our results describe the average responses of populations whose constituent neurons probably shifted over time. In the future, new developments in either optical⁵⁸ or electrophysiological^59,60 recordings from primates might make it possible to longitudinally measure the responses of individual neurons during behavioral development.

STAR★METHODS

RESOURCE AVAILABILITY

Lead contact

Further information and requests for resources should be directed to and will be fulfilled by the lead contact Lynne Kiorpes (lk6@nyu.edu).

Materials availability

No new materials were created for this study.

Data and code availability

• All data used in this study have been deposited at Zenodo and are publicly available as of the date of publication. The DOI is listed in the key resources table.

• All original code has been deposited at Zenodo and is publicly available as of the date of publication. The DOI is listed in the key resources table.

• Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.

KEY RESOURCES TABLE.

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Experimental models: Organisms/strains
Macaque (Macaca nemestrina) monkeys	Washington National Primate Center	RRID:NCBITaxon_9545
Software and algorithms
MWorks	The MWorks Project	https://mworks.github.io/
MATLAB	MathWorks	http://www.mathworks.com
Custom MATLAB analysis code	This paper	https://doi.org/10.5281/zenodo.10888269

EXPERIMENTAL MODEL AND STUDY PARTICIPANT DETAILS

Animals

We performed all animal procedures in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals,⁶¹ and with the approval of the New York University Animal Welfare Committee.

Behavioral experiments

We trained 7 macaques on our behavioral task (Macaca nemestrina, 3 female – RRID:NCBITaxon_9545). We made longitudinal measurements from 5 animals. We made cross-sectional measurements from 2 animals. See Table 1 for the specific age range tested in each animal.

Neural experiments

We recorded multiunit neural responses from a total of 6 animals. We recorded longitudinal data from 5 of these animals, starting between 18 and 26 weeks, and continuing for as long as the recording arrays could be maintained (see Table 1). We implanted one of these animals a second time after it had reached 4 years of age. Finally, we recorded from area V4 of a separate adult. We combined data collected from similar ages (within 2 weeks).

METHOD DETAILS

Visual stimuli

We generated stimuli using the texture model of Portilla & Simoncelli,¹¹ using previously detailed methods.¹² Briefly, we used real world images of repeating patterns to generate textures. These images included photographs of both natural scenes (e.g., honeycomb) and artificial objects (e.g., a shower curtain). While we used images that were unrelated to our animals’ visual experience, they were reared in a rich visual environment, which included a variety of conspecific and human faces, objects, still images, and videos.

We used each photograph to generate a texture “family”, starting with “noise” textures, which matched the average spectral content of the original image, as measured with steerable pyramid filters.¹¹ We then synthesized “naturalistic” textures, which matched both the outputs of the oriented filters (as with the noise textures), as well as the correlations between filter types. For each texture family, we generated multiple “samples”, which were all matched within the model space, but differed in the precise spatial location of their elements.

For our measurements of behavioral and neural texture sensitivities, we used textures from 5 families (Figure 1A). To obtain textures varying in the strength of their naturalistic structure, we interpolated the model statistics between noise and naturalistic texture pairs of the same family and sample. We measured responses to one texture family per session. For behavioral measurements, we used 15 samples for each level of naturalistic structure from a given family; for neural measurements we used 5 samples. In both cases, the strength of structure varied from 0 (spectrally matched noise) to 1 (fully naturalistic).

For neural measurements of naturalistic texture discriminabilities, we used a larger texture image set of 1150 images (Figure 1B): 525 naturalistic and noise textures (1050 total, from a total of 35 texture families, and with 15 texture samples per family), and 100 blank images (used to estimate baseline firing rates).

We presented stimuli on a gamma-corrected CRT monitor with a mean luminance of 28 cd m⁻², a resolution of 1280 by 960 pixels, and a frame rate of 100 Hz. We seated animals in a custom primate chair 114 cm from the monitor, at which distance the monitor subtended 20 by 15 deg.

Behavioral experiments

We trained macaques using standard operant conditioning methods. Animals initiated trials by fixating a 1–2 deg red square at the center of the screen. Once they had done so, the square disappeared, and the screen remained blank for 200 ms. Four texture stimuli then appeared – 3 noise texture distractors, and 1 target texture. Each texture was 6.4 deg in diameter, centered ± 3.2 deg from the center of the screen in both the horizontal and vertical direction (thus 4.5 deg eccentric). Once the textures appeared, subjects had 1200 ms to register a choice, which was defined as fixation on 1 of the 4 stimuli for more than 400 ms. Trials ended immediately after correct responses, at which point subjects received a juice reward. There was no penalty for incorrect responses, except that the stimuli would remain on screen for the full 1200 ms. We did not analyze trials in which the animal failed to respond, which were rare.

We measured naturalistic texture sensitivity using a single texture family per session. On each trial, we varied the position of the target among the 4 locations as well as the strength of its naturalistic structure. The naturalistic structure varied along a series of fixed levels. We used different samples for all 4 images – including the noise texture distractors, to ensure that subjects could only distinguish targets from distractors on the basis of naturalistic texture statistics, instead of pixel-level cues.

We used the method of constant stimuli to determine the difficulty level for each trial. Roughly 5% of trials were catch trials in which the target was also a noise texture – we rewarded animals for choosing this target in the usual way. The levels we chose for a given session were adapted to span the psychometric function from chance to perfect performance, while maintaining the animals’ motivation. To maintain overall motivation levels, we showed fully naturalistic textures disproportionately often.

Sessions typically contained 600–1000 total trials, with roughly 100 trials per condition. We measured performance on each texture family multiple times at each age, with the exception of M3, due to constraints imposed by electrophysiological measurements. We measured performance on at least 4 texture families per animal. We introduced a fifth for some animals at older ages, as a way to both estimate animals’ ability to generalize on the task, and to probe whether their performance was learned or age-based. We excluded sessions containing fewer than 120 trials, or with a peak performance of less than 90%.

Physiological experiments

For physiological recording, we trained the animals to fixate the central 3 deg of the screen, which was marked with a red central square 0.1–0.2 deg across. Texture images appeared after 160 ms in a pseudorandom order. For our texture set with 35 texture families, we presented blocks of 8 images, and showed each image for 100 ms, followed by a 100 ms blank interval. For our measurements of neurometric thresholds, we showed blocks of 4 images for 200 ms (the same used to measure behavioral thresholds), and used a 200 ms blank interval.

All texture images measured 6.4 deg in diameter. For most data, receptive fields (detailed below) were centered within the central 1.5 deg – for these cases, we centered stimuli at the center of the monitor. Receptive fields in animal M8 were roughly 8 deg from the center of gaze. In this case, we centered stimuli over our estimate of the receptive field center. In all cases, the visual stimuli covered the aggregate receptive fields of the recording sites.

We rewarded animals for remaining fixated through a block with a juice reward. If an animal broke fixation during a stimulus, we interrupted presentation and presented the interrupted stimulus again later in the overall sequence. We recorded at least 4 repetitions of each stimulus from our multifamily texture image set. We recorded at least 36 repetitions of each stimulus in our (smaller) image set of textures varying in their naturalistic structure.

Neural recording

After training animals to perform the fixation task, we implanted 96 electrode (Utah) recording arrays under sterile conditions. We used gross anatomical features to inform decisions about array placement. Electrodes were arranged in a 10 × 10 square (four positions were not used for recording), had shanks 1 mm long and had an interelectrode spacing of 0.4 mm. Following implantation, we used anatomical landmarks (including gross anatomical landmarks such as sulci and vasculature, which we observed surgically and related to previously documented area boundaries^62,63), physiological response properties (e.g., response latencies), and receptive field characteristics to determine the cortical location of array sites. All sites on an array were typically within a single visual area, with 2 exceptions. In one animal, one array lay on the border between V1 and V2. In the other animal, we were unable to determine whether the arrays were located in area V1, V2, or V4. As receptive fields were close to the center of gaze, we refer to these data as stemming from the “foveal confluence” (see Brewer et al.¹⁹ – their Figure 8B), as opposed to one or another visual cortical area.

We recorded bandpass filtered (250 Hz–7.5 kHz) electrical activity, at a sampling rate of 30 kHz. To minimize the influence of common-mode signals, we subtracted the median sample-by-sample voltage across all sites, and then defined spike events (threshold crossings) as negative voltage deviations at least 3 times the root mean squared deviation of the baseline voltage.

QUANTIFICATION AND STATISTICAL ANALYSIS

Behavioral data analysis

For the 5 animals we tested longitudinally, we used between 22 and 33 total sessions. For the 2 animals we tested at older ages, we used 3 sessions in one, and 11 in the other. For each animal, we combined behavioral data from a given texture family collected within a 7 day span.

Two animals (M3 and M5) acquired a tendency to choose the same spatial position at older ages. Compared to the other animals in our sample, these two had run more 4 choice oddity tasks of various sorts. We retrained them to mitigate this spatial bias, and rejected sessions where they chose one target at least 40% of the time (all results and model predictions were similar with and without this inclusion criterion).

To measure behavioral sensitivities, we fit a shared cumulative Weibull psychometric function to all data.⁶⁴ We used a maximum likelihood fitting process to extract a slope parameter, common to all data, and separate thresholds and lapse rates for each individual measurement. We computed thresholds as the intercept of the Weibull function with 55% correct, corresponding to a d’ of 1.⁶⁵ We then estimated the individual variability for each threshold estimate using a nonparametric bootstrap,⁶⁶ for which we fixed the slope and lapse rate to the values extracted from the original fitting routine. We inverted thresholds to obtain sensitivities.

We modeled the relationship between sensitivity, $s$, and age, $a$, using a function of the form:

$$s=s_{max}\frac{a^{\alpha }}{a_{50}^{\alpha }+a^{\alpha }},$$

where $s_{max},a_{50}$, and $\alpha$ are free parameters representing the maximum sensitivity, the age corresponding to half-maximum performance, and a fit exponent capturing the rate of increase (larger values correspond to faster saturation), respectively. We fit this function by minimizing the squared error of the model predictions. Our psychometric function is fit in a logarithmic space (curves sharing a slope and lapse rate, but differing in threshold have the same shape in this space), thus we log-transformed our estimates prior to measuring model error. We tried a variety of models of this general form, and compared them using the corrected Akaike’s Information Criterion.⁶⁷ Across all such models, our data were best explained by one in which the $a_{50}$ and $\alpha$ parameters were shared across all data, and the $s_{max}$ was fit separately for each subject. To estimate the variability of model parameters, we nonparametrically resampled our data, and fit our model to the resampled data. We repeated this 1000 times, and extracted 95% confidence intervals around the parameter estimates.

Physiological data analysis

Initial analysis

For analysis, we binned multiunit threshold crossings into 10 ms windows. To determine single-site response latencies, we separately measured the discriminability between blank stimuli, and either naturalistic texture or noise texture stimuli as a d’ between response distributions:

$$\frac{{\mu }_{stim}-{\mu }_{blank}}{\sqrt{\frac{1}{2}\left({\sigma }_{stim}^2+{\sigma }_{blank}^2\right)}},$$

where $\mu$ reflects the mean of a distribution, and $\sigma$ its standard deviation. We defined the onset time for a given site as the first deviation above a d’ of ± 0.4, as long as the following 2 windows were also suprathreshold (and of the same sign). The onset time for a given population was the median onset time across visually responsive sites. For most analyses, we then summed responses across the 150 ms window following stimulus onset.

To determine whether a site was visually responsive in a given recording session, we measured the average response separately for even and odd repetitions of each stimulus. We measured the Pearson correlation between these vectors, and used sites with a correlation of 0.4 or greater (the results reported here remained stable across a variety of different metrics and thresholds).

We measured receptive field locations by recording neural responses to a small spot, which was tiled across the visual field. We used d’ values, summed for 200 ms following response onset, between spot stimuli and blanks to estimate receptive field centers and sizes. With one exception, that of M8, receptive fields were located within 2 deg of the center of gaze. For M8, receptive fields were roughly 8 deg eccentric from the center of gaze.

As a single site measure of naturalistic texture discriminability, we measured the d’ between naturalistic and noise texture evoked response distributions, using the above formula.

To compare tuning at the single site level, we measured the variance across all texture families (including naturalistic and noise textures), after averaging across stimulus repetitions, and computed its proportion of the overall stimulus-driven variance.

Population analysis

To measure population discriminability, we first performed singular value decomposition on a training set $US{V}^{\mathrm{\top }}=M_{train}$ of naturalistic and noise texture responses, for a given population ($M_{train}$ was a matrix organized in the form stimulus × site, see Figure S2A). Array recordings result in varying population sizes. We primarily addressed this by subsampling populations to a matched size, allowing us to directly compare performance between ages and area. We obtained these subsamples by randomly selecting from the pool of visually responsive sites. We repeated this random sampling to estimate variability across measurements. For our measurements of population neurometric sensitivities, we used populations of 20 sites. For our experiment using textures from many families, we used populations of 30 sites. In Figure S3, we measured the relationship between population size and performance, and found that our results generalized across population sizes.

To minimize overfitting, we reduced the dimensionality of the resultant basis set $V$ to rank 20 (except to rank 10 for neurometric sensitivities, see Figure S2B). We then fit a linear discriminant, $b$, to the rotated matrix $M_{train}V$, best separating naturalistic and noise responses.

For our measurements of population neurometric sensitivities, we used samples for 5-fold cross-validation, using 4 samples for training, and the 5th for testing. We measured proportion correct in analogy to a 4 choice task, by iterating through each response in $M_{test}$, with a simulated trial structure, where we projected each individual response in $M_{test}$ onto our learned axis (simulated target), along with 3 randomly chosen noise texture responses from $M_{test}$ (simulated distractors). If the projection of the simulated target onto the learned discriminant axis was larger than that of all 3 distractors, we scored the stimulated trial as correct (Figure S2C). Otherwise, we scored it as incorrect. After repeating this process for all entries in $M_{test}$, we computed an overall proportion correct for each level of naturalistic structure.

We then fit neurometric functions with cumulatives of the Weibull distribution, fit separately to each session where the maximum proportion correct reached at least 0.5. We estimated variability using a parametric bootstrap.⁶⁸

We measured neurometric sensitivities separately using a correlation-based classifier. In this approach, we replaced the projection of 4 trials (a target and 3 distractors) onto a discriminating hyperplane, with the correlation between the same 4 trials, and the average response to fully naturalistic textures, as measured from our training set. Here, we simulated choice as the trial with the highest correlation. Our results in this framework were qualitatively similar to those using the linear discriminant.

For our analysis of the influence of neural correlation structure on decoding, we recomputed the population discriminability as measured from our textures varying continuously in their naturalistic statistics, after randomly and separately shuffling the trial order for each site, to disrupt any trial-to-trial correlations in neural activity.

For our measurements of naturalistic texture discriminability, we measured performance across training-testing splits by projecting our held out data $M_{test}$ onto the same axis as $M_{test}Vb$. We measured the discriminability between naturalistic and noise texture response distributions as a d’ (Figure S2D). To measure stimulus detectability, we repeated this process, but using naturalistic textures and blank stimuli. We measured performance separately for each texture family. For cross-validation, we trained on 14 of the 15 samples, and tested on the held out sample. We reported performance as the average across all 15 cross-validations.

For our analysis of temporal dynamics, we measured discriminability using the same population scheme, but trained and tested separately for each 10 ms time bin. To measure population latencies, we fit a Heaviside function, modified to include a finite slope (which we fit as a free parameter), to the resultant average performance curves.

Statistical testing

Where possible, we used estimation statistics to state effect sizes.⁶⁹ For our behavioral data, we used the bootstrap process described above to estimate variability in our model parameters. We then used the original model to estimate the magnitude of behavioral change between 26 and 52 weeks, and the bootstrapped models to calculate a confidence interval around that magnitude. We used 100,000 resamples for all bootstraps.

For our neural data, we fit a line relating either the mean single-site selectivity, or the population discriminability, with the log-transformed age. We used the slope of the line to quantify the amount of age-related change. We then used a bootstrapping procedure to resample our single-sites or populations to calculate confidence intervals around those slopes. To compare single-site selectivities between areas, we performed a permutation ANOVA (after Anderson⁷⁰). We then used pairwise permutation tests to estimate the magnitude of mean differences in selectivity between individual areas.

Highlights.

• Behavioral sensitivity to naturalistic texture roughly doubles from 25 weeks to maturity

• Neural sensitivity in V1, V2, V4, and IT is stable over this age range

• Texture sensitivity is highest in V4, lowest in V1, and intermediate in V2 and IT